U.S. GOVERNMENT PUBLISHING OFFICE

36–307 PDF WASHINGTON : 2019

S. Hrg. 116–37 

FROM BEGINNING TO END: AN EXAMINATION 

OF AGENCIES EARLY PUBLIC ENGAGEMENT AND 

RETROSPECTIVE REVIEW 

HEARING 

BEFORE THE

SUBCOMMITTEE ON 

REGULATORY AFFAIRS AND FEDERAL 

MANAGEMENT 

OF THE

COMMITTEE ON 

HOMELAND SECURITY AND 

GOVERNMENTAL AFFAIRS 

UNITED STATES SENATE 

ONE HUNDRED SIXTEENTH CONGRESS 

FIRST SESSION 

MAY 7, 2019 

Available via http://www.govinfo.gov 

Printed for the use of the Committee on Homeland Security 

and Governmental Affairs 

( 
(II) 

COMMITTEE ON HOMELAND SECURITY AND GOVERNMENTAL AFFAIRS 

RON JOHNSON, Wisconsin, Chairman 

ROB PORTMAN, Ohio 

RAND PAUL, Kentucky 

JAMES LANKFORD, Oklahoma 

MITT ROMNEY, Utah 

RICK SCOTT, Florida 

MICHAEL B. ENZI, Wyoming 

JOSH HAWLEY, Missouri 

GARY C. PETERS, Michigan 

THOMAS R. CARPER, Delaware 

MAGGIE HASSAN, New Hampshire 

KAMALA D. HARRIS, California 

KYRSTEN SINEMA, Arizona 

JACKY ROSEN, Nevada 

GABRIELLE D’ADAMO SINGER, Staff Director 

DAVID M. WEINBERG, Minority Staff Director 

ZACHARY I. SCHRAM, Minority Chief Counsel 

LAURA W. KILBRIDE, Chief Clerk 

THOMAS J. SPINO, Hearing Clerk 

SUBCOMMITTEE ON REGULATORY AFFAIRS AND FEDERAL 

MANAGEMENT 

JAMES LANKFORD, Oklahoma, Chairman 

ROB PORTMAN, Ohio 

MITT ROMNEY, Utah 

RICK SCOTT, Florida 

MICHAEL B. ENZI, Wyoming 

KYRSTEN SINEMA, Arizona 

THOMAS R. CARPER, Delaware 

JACKY ROSEN, Nevada 

CHRIS WHITE, Staff Director 

JAMES MANN, Senior Counsel 

ERIC A. BURSCH, Minority Staff Director 

ANTHONY J. PAPIAN, Minority Senior Professional Staff Member 

MALLORY B. NERSESIAN, Subcommittee and Document Clerk 
(III) 

C O N T E N T S 

Opening statement: Page 

Senator Lankford .............................................................................................. 1 

Senator Sinema ................................................................................................ 3 

Senator Carper ................................................................................................. 11 

Prepared statement: 

Senator Lankford .............................................................................................. 25 

WITNESSES 

TUESDAY, MAY 7, 2019 

Hon. Susan E. Dudley, Former Administrator (2007–2009) of the Office of 

Information and Regulatory Affairs, Office of Management and Budget ........ 4 

Hon. Sally Katzen, Former Administrator (1993–1998) of the Office of Information and Regulatory Affairs, Office of Management and Budget ................ 6 

ALPHABETICAL LIST OF WITNESSES

Dudley, Hon. Susan E.: 

Testimony .......................................................................................................... 4 

Prepared statement .......................................................................................... 28 

Katzen, Hon. Sally: 

Testimony .......................................................................................................... 6 

Prepared statement .......................................................................................... 36 

APPENDIX 

Responses to post-hearing questions for the Record: 

Ms. Dudley ........................................................................................................ 43 

Ms. Katzen ........................................................................................................ 58 

(1) 

1The prepared statement of Senator Lankford appears in the Appendix on page 25. 

FROM BEGINNING TO END: AN EXAMINATION 

OF AGENCIES EARLY PUBLIC ENGAGEMENT 

AND RETROSPECTIVE REVIEW 

TUESDAY, MAY 7, 2019 

U.S. SENATE, SUBCOMMITTEE ON REGULATORY, AFFAIRS AND FEDERAL MANAGEMENT, OF THE COMMITTEE ON HOMELAND SECURITY

AND GOVERNMENTAL AFFAIRS, Washington, DC. 

The Committee met, pursuant to notice, at 10 a.m., in room 342, 

Dirksen Senate Office Building, Hon. James Lankford, Chairman of 

the Subcommittee, presiding. 

Present: Senators Lankford, Scott, Sinema, and Carper. 

OPENING STATEMENT OF SENATOR LANKFORD1

Senator LANKFORD. Good morning. Welcome to today’s hearing 

entitled From Beginning to End: An Examination of Agencies Early 

Public Engagement and Retrospective Review. I would like to first 

welcome Senator Sinema to the dais. I look forward to working 

with you in the Senate, and am grateful, specifically, for your work 

that you have done on these topics in the House, and coming over 

here and working on this in the Senate. I know the most coveted 

ranking position, is to be able to work on this Subcommittee, working on regulatory issues and Federal management. And what is 

wonderful is that you worked on these issues a lot in the House, 

and I am grateful that you have come over to be able to work on 

this here. 

Thank you, as well, to our witnesses who bring a lot of expertise 

into this, and we are very grateful for the time that you have spent 

on it. Former Office of Information and Regulatory Affairs (OIRA) 

administrators are a very valuable resource to us in this Committee because you have walked through this process and you bring 

some unique insight. 

I also jokingly say all the time I can ask former administrators 

questions that they can actually answer, that current administrators say, ‘‘I will have to get back to you with my legislative staff,’’ 

and note that. So we are grateful to be able to get the insight. 

Today we are focusing on how agencies conduct public outreach 

at the beginning of a rulemaking, and how real success or failures 

are measured years later. While these could be considered narrow 

issues they are by no means small. Advance Notice of Proposed 
2 

Rulemaking (ANPRM) are important tools that only a few agencies 

are statutorily required to actually utilize. 

Administrative Procedures Act (APA) sets out a process where 

agencies propose a rule, listen to comments from the public. They 

have the opportunity to make changes and then issue a final regulation. While the system looks good on paper, in practice agencies 

typically conduct a significant amount of work before formally engaging the public. They will consider various regulatory schemes 

and conduct economic analysis. Ideally, agencies would engage the 

public early, but there is no Administrative Procedures Act requirement to do so. 

Turning to the other end of the rulemaking process, retrospective 

review is a process to ensure rules achieve their intended goal in 

the least burdensome way. Over time, changing circumstances, improved technologies may render some regulations ineffectual or unnecessary. An agency’s job is not done after the final rule is published. As initial estimates of both costs and benefits prove inaccurate, agencies should revisit a rule to ensure the desired effects 

are actually achieved. 

Every President since Jimmy Carter has urged agencies to utilized retrospective review to examine existing regulations. While 

these directives were issued with good intentions, they gave agencies a significant amount of discretion in selecting which and how 

many rules to review. 

The focus of this hearing is two bills that Senator Sinema and 

I will introduce shortly, that codify the best practices for both procedures. The Early Participation in Rulemaking Act direct agencies 

to issue advance notices for rules costing more than $100 million 

annually. The agency must outline what problem the rule intends 

to solve and listen to the public’s input on the subject. The idea behind this bill is to require agencies to listen to the public before 

they craft the regulation. 

Washington does not have all the answers. Taking time to work 

with stakeholders, particularly our small businesses, is vital in 

crafting effective regulations. Less burdensome is not less effective. 

Business owners want to be good citizens, follow along, and have 

a safe and clean workspace. 

Setting Manageable Analysis Requirements in Text (SMART) 

Act, is a retrospective review bill that looks ahead. It requires 

agencies to set metrics for how a rule will be measured for success 

in the future. It is hard to imagine the measure of success of anything unless it is defined. This bill instructs regulators to define 

what success is for a given rule and then requires them to grade 

that rule within 10 years. 

This Subcommittee has been working on both of these issues for 

a while. Both bills have bipartisan support in the past two Congresses. I look forward to working with my colleagues to push both 

of them across the finish line. 

With that I would like to recognize Senator Sinema for her opening statement. 
3 

OPENING STATEMENT OF SENATOR SINEMA 

Senator SINEMA. Well, thank you, Chairman Lankford, and 

thank you to our witnesses for joining us today to help fix our regulatory process. 

I am happy to join Chairman Lankford as the Ranking Member 

of the Regulatory Affairs and Federal Management Subcommittee. 

As a Congresswoman I promoted policies that expanded business 

opportunities and fueled innovation in Arizona and beyond, and my 

goal for this Subcommittee is to continue that work and to find and 

push for targeted common-sense reforms to regulations that help 

hard-working Arizonans build better lives. 

With Chairman Lankford’s leadership we are off to a strong 

start. We have already succeeded in moving two common-sense regulatory transparency bills through committee markup, and that is 

only the beginning. 

The Providing Accountability Through Transparency Act and the 

GOOD Act will both help make government more accessible to Arizona businesses, and I am excited to accomplish even more. 

Today we continue the work of advocating for a modernized rulemaking process. Advance notice of proposed rulemaking allows 

agencies to engage with businesses, nonprofits, academics, and 

other everyday people, so an idea that may later become a rule is 

heading in the right direction. 

Retrospective review requires agencies to look back at a rule, to 

make sure it does what was intended and that the benefits outweigh the costs. Doing a retrospective review can be time-consuming and complex, but if an agency prepares in advance for the 

review it can be even more beneficial and efficient. 

Advance notice of proposed rulemaking and planning for retrospective review have been encouraged for decades but never enacted into law, and that should change. And soon I will be introducing the SMART Act to incorporate planning for retrospective review into major rules. 

Planning for the future is common sense, which is why planning 

for retrospective review has been promoted by the Government Accountability Office (GAO), the Administrative Conference of the 

United States (ACUS), and the American Bar Association (ABA). 

By requiring agencies to plan for review, the reviews will be more 

thorough and accurate, and less expensive and time-consuming. 

Our legislation will improve regulations, remove unnecessary burdens, and increase transparency and accessibility for Arizona businesses, communities, and others, and I look forward to hearing 

from our witnesses. 

Thank you, Mr. Chair. 

Senator LANKFORD. Let me proceed to testimony from our witnesses. The Hon. Susan Dudley is the Director of the Regulatory 

Studies Center and Distinguished Professor of Practice at the 

Trachtenberg School of Public Policy and Public Administration at 

George Washington (GW) University. Before joining the faculty at 

GW, she served as the Administrator of the Office of Information 

and Regulatory Affairs from 2007 to 2009. 

The Hon. Sally Katzen is a Professor of Practice and Distinguished Scholar in Residence at New York University School of 

Law. She served in multiple roles during the Clinton Administra-
4 

1The prepared statement of Ms. Dudley appears in the Appendix on page 28. 

tion, including Administrator of the Office of Information and Regulatory Affairs, Deputy Director for Management of the Office of 

Management and Budget (OMB). 

Thank you both for bringing your experience and your insight, 

for working with our staff leading up to this hearing, and continuing to contribute to your Nation. Thank you for that continued 

engagement. 

As you both know, because you have both been here before, it is 

the custom of this Subcommittee to swear in all witnesses before 

they testify. So if you would please stand and raise your right 

hand. 

Do you swear that your testimony given before this Subcommittee will be the truth, the whole truth, and nothing but the 

truth, so help you, God? 

Ms. DUDLEY. I do. 

Ms. KATZEN. I do. 

Senator LANKFORD. Thank you. You may be seated. Let the 

record reflect both answered in the affirmative. 

We are using a timing device today, and you will have 6 minutes 

for your opening statement. We do have a mercy rule here that if 

you go beyond that you are fine, because we have this hearing to 

get your testimony, and so if you go a little bit over we are going 

to be OK on both of those. 

But we will be glad to be able to receive your testimony now. Ms. 

Dudley, you are first. 

TESTIMONY OF THE HON. SUSAN E. DUDLEY,1 FORMER ADMINISTRATOR (2007–2009) OF THE OFFICE OF INFORMATION 

AND REGULATORY AFFAIRS AT THE OFFICE OF MANAGEMENT AND BUDGET. 

Ms. DUDLEY. Thank you. Do I get bonus points if I go under? 

Senator LANKFORD. Yes, clearly. 

Ms. DUDLEY. Thank you very much, Chairman Lankford, Ranking Member Sinema, Senator Carper, and Senator Scott for inviting me to talk about one of my favorite subjects. I appreciate your 

interest in improving how the U.S. Government develops and evaluates regulatory policy. You are continuing a long bipartisan tradition of efforts to make regulation well informed, transparent, and 

accountable to the American people. 

Agencies have long been required to seek public comment on proposed regulatory notices, yet these opportunities for public engagement often come after agencies have made key policy decisions. 

Proposed rules are legal documents, written to defend a selected 

approach against possible litigation. So this motivates agencies to 

circle the wagons, narrowing the menu of alternatives and the evidence they consider before the public has an opportunity to engage. 

The draft Early Participation in Regulations Act would require 

agencies to issue for public comment advanced notices of proposed 

rulemakings for major rules. This could free them to share their 

early thinking on whether a problem requires a regulatory solution, 

what objectives could be achieved, and what different options are 

available. 
5 

These ANPRMs and any differences between them and subsequent APA rulemaking steps are wisely exempt from judicial review in your draft bill. One virtue of the ANPRM is that it provides 

an opportunity for agencies to share their preliminary thinking 

about a problem and get input on potential solutions at a stage 

when they are truly open to feedback, analysis, and evidence. If 

agencies had reason to fear that this early notice could later be 

used against them in court that would discourage objective queries 

and underline those benefits. 

I do not think the ANPRM requirement would significantly slow 

agencies’ rulemaking. For one thing, 90 days is not a long time considering that agencies often take years studying a problem and 

evaluating options before they issue that first proposal. 

But probably more importantly, to the extent the ANPRM invites 

comments on preliminary deliberations that would otherwise have 

taken place behind closed doors, it may make the overall regulatory 

process more efficient. So rather than tacking 90 days on at the 

end of a rulemaking—or at the beginning, that makes it 90 days 

longer, it may provide valuable input that ends up streamlining the 

subsequent notice and comment process. In many cases, early engagement could lead to more efficient analysis at the proposal stage 

and fewer surprises during public comment. 

That said, there are cases where an ANPRM would not serve the 

public interest and bill provides for exceptions for those. 

Retrospective review is also very important. Program evaluation 

has a long tradition in the private sector and in activities financed 

through the fiscal budget, but it has received little attention in the 

regulatory arena, even though every President since Carter has 

asked agencies to evaluate existing rules. President Obama added 

new emphasis on evaluation, yet most regulations continued to be 

issued without a plan for review. 

As ex ante analysis, regulatory impact analyses (RIAs) are necessarily hypotheses of the effects that regulations will have if implemented. Better regulatory evaluation would allow us to test 

those hypotheses against actual outcomes. That feedback would not 

only help with decisions on current policies but it would improve 

future RIAs and future regulations. 

Your draft bill would require agencies to include in major regulations a framework for how they will measure effectiveness, benefits 

and costs, and a plan for gathering the information necessary for 

ex post review. It would also require agencies within 10 years, to 

assess a rule’s benefits and costs, evaluate how well it accomplishes 

its objectives, and determine whether it could be modified to 

achieve better outcomes. 

The draft bill focuses not just on reducing regulatory burdens but 

on improving outcomes by subjecting rules to rigorous evaluation 

and feedback. It could create an evaluation mindset where agencies 

learn from reviewing regulatory actions and apply those lessons to 

improve future rules. 

These two draft bills offer relatively modest yet potentially powerful changes to the rulemaking process. By engaging public input 

earlier in the process and providing for retrospective review of regulations to evaluate whether they are achieving their objectives, 

they can help ensure that regulations are based on the best avail-
6 

1The prepared statement of Ms. Katzen appears in the Appendix on page 36. 

able evidence and that they are working as intended. These bills 

could make regulatory decisions more transparent and accountable, 

leading to improved outcomes for the American people. 

Thank you. 

Senator LANKFORD. Thank you. Ms. Katzen. 

TESTIMONY OF THE HON. SALLY KATZEN,1 FORMER ADMINISTRATOR (1993–1998) OF THE OFFICE OF INFORMATION AND 

REGULATORY AFFAIRS AT THE OFFICE OF MANAGEMENT 

AND BUDGET 

Ms. KATZEN. Thank you, Chairman Lankford, Ranking Member 

Sinema, Senator Scott, and Senator Carper. Thank you for inviting 

me to testify today. 

I have been generally skeptical, if not highly critical, of the many 

attempts over the last several decades to rewrite the APA and update the process that produces the regulations that translate general statutory directives into concrete requirements that the public 

must comply with. Many of those attempts were highly partisan 

and would have converted the regulatory roadmap into an obstacle 

course, with the effect, if not the intent, of greatly delaying or shutting down the regulatory process rather than contributing to good 

decisionmaking. 

I understand that the effort of this Subcommittee is different. I 

understand that you are looking for surgical fixes to improve discrete problems with the goal of enlisting support by both Democrats and Republicans, and specifically you are looking at the beginning and the end of the process with suggestions that could be 

achieved by legislation, by Executive Order (EO), by OMB guidance, by agency practices. 

But regardless of the vehicle, it is important to be clear about 

what the problem is and how to best solve that problem without 

introducing unintended consequences. 

So at the beginning, the first official step in a rulemaking is the 

issuance of a notice of proposed rulemaking (NPRM), lawyers in 

practice and the Academy generally agree that by the time the 

agency issues the NPRM, the staff involved have invested so much 

time and energy in developing the proposal and supporting data as 

they are required to do—and analyzing the likely effects of the proposal—as they are required to do—and justifying the proposal—as 

they are often called upon to do by agency decisionmakers and during OIRA review, that they are virtually locked into their proposal 

and are less receptive to new ideas or even significant modifications of their proposal. 

While all of the up-front work is desirable, it often has the unintended consequence of restricting the options going forward. As you 

mentioned, both Republican and Democratic Presidents have tried 

to counter this tendency by encouraging agencies to consult with 

the public even before they have made the decisions reflected in the 

NPRM. Those admonitions have produced some additional outreach 

to the public, but the effect has been inconsistent and less productive than had been expected. 
7 

Now several agencies do use an advance notice of proposed rulemaking to solicit ideas at the outset, but it is not a universal panacea. It is especially useful when the agency is unsure what direction to take, what data to consider, how prescriptive to be. It is also 

useful when it is done before or at the outset of agency deliberations. 

It is less useful when the authorizing statute is itself prescriptive 

or there is genuine consensus about what is needed to respond to 

the identified problem. In short, it can be helpful at times, but at 

other times it may just add an unproductive but a time-consuming 

step to the already extended process. 

For this reason it is important that any requirement for an 

ANPRM be limited to economically significant—or you call them 

major regulations, that are required to use notice and comment, 

and second—that any such provisions not impose on the agency 

multiple requirements for explanations, analysis, data, etc. 

The purpose would be to engage those affected by the rule so 

they can contribute to its development and formulation before the 

agency settles on a particular course, not to lock the agency into 

a particular mindset before the process begins. The more the agency has to incorporate in an ANPRM, the more the agency will become invested in a particular outcome. This is the opposite of what 

it should do. 

With respect to retrospective review, for almost 40 years there 

have been concerns that there are too many rules, and so many of 

the rules in the books are obsolete, burdensome, unworkable. Notwithstanding the efforts of every President, from Reagan through 

Obama, we search and we search, and we do not find, and we do 

not eliminate many rules from the existing stock of regulations. 

Now one reason for this may be that since 1980, new regulations 

are not issued unless their benefits justify their costs. To eliminate 

such regulations would likely mean that the costs of rescinding the 

regulations would be greater than the benefits, which is counterintuitive. Other reasons for the limited success of look-back efforts 

are that agencies usually have not collected data along the way 

that would inform their retrospective reviews, and very importantly, any respective analysis requires resources, and for at least 

the last several decades regulatory agency budgets have generally 

been decreasing or straight-lined. Without the resources they cannot do the work. 

Nonetheless, there is growing support for the step that you are 

talking about here, namely encouraging agencies to plan for retrospective review when they are in the process of developing a final 

rule. This idea was endorsed by many of the organizations that you 

mentioned, and a report prepared by the Institute for Policy Integrity, which reflected the unanimous recommendation of almost all 

former OIRA administrators. 

Requiring agencies to provide a plan for later retrospective review of a newly issued rule would, in most instances, be salutary. 

If nothing else, it would require the agency personnel to focus on 

describing precisely what they want to accomplish and how to 

evaluate whether or not the rule is successful, at a time when the 

rule and its alternatives is foremost in their minds. 
8 

It is very important, however, to provide flexibility for the eventual implementation of the retrospective review. The agency can 

and should commit to a framework in the proposed and final rule, 

identifying the data and the metrics it anticipates using for that 

purpose, but this should not be cast in concrete. We learn a lot 

with time, including how to better analyze and measure what is 

going on. 

This is amply demonstrated by the increased sophistication of 

cost benefit analysis itself over the last decade or two, and it is also 

demonstrated by the general preference for performance standards, 

which specify the desired results, rather than design standards, 

which lock in a particular way of getting there. 

In addition, while periodic review is useful, there will likely be 

some, and maybe many situations where repeated retrospective reviews would yield greatly diminishing returns. After a decade or so, 

if rules survive a retrospective review intact, they are likely to 

have established their worth, and it would be wasteful to continue 

retrospective review after retrospective review. Some escape hatch 

should be provided. 

I thank the Committee for its efforts and for its courtesy in allowing me to exceed my time limit. Because you are not writing on 

a blank slate, even this limited surgical approach will likely face 

an uphill battle. It might well, however, have a better chance to 

succeed than many that have been tried before. And for that I congratulate you both. 

Senator LANKFORD. Thank you. We will take that congratulations when we are done. We are in the starting block at this point, 

though. 

Thank you both for your testimony. I am going to defer my questions to the very end and recognize Senator Sinema. 

Senator SINEMA. Thank you, Chairman. Thank you both, again, 

for being here today. 

Starting with Executive Order 12866, which directs agencies to 

make sure that regulations maximize benefits while limiting costs, 

Administrations have discussed the merits of retrospective review 

and have provided guidance to agencies related to the review of existing regulations, but, of course, these directives do not have the 

force of law. 

Over the years, as I mentioned in my opening statement, the 

GAO, the Administrative Conference of the United States, and the 

American Bar Association have all issued reports or made official 

comments regarding the need for post-enforcement analysis of the 

true costs and the benefits of regulations. 

So the question for both of you is considering the long-term support for retrospective review from the Clinton and all subsequent 

Administrations, why do we believe that this continues to be a 

challenge for agencies in our Federal Government? 

Ms. DUDLEY. I would say there are two reasons, lack of incentives, and then lack of data and analytical tools. I think your draft 

bill addresses both of those, and when I say lack of incentives for 

agencies—it is always more exciting to look at the next thing, the 

next problem to solve rather than to look back at what you have 

done. But it is not just the agencies because regulated parties also, 
9 

once they have complied, they are not so keen on revisiting it and 

maybe making their competitor upstart not have to comply. 

So by requiring this in your bill, I think you would address the 

incentives but perhaps more importantly you would address the 

lack of data and analytical tools. As you have said, this bill would 

require agencies up front to say here are the data we need, here 

is how we will collect it, and here is the outcome that we are going 

to look to measure it against. 

Ms. KATZEN. I agree with her response, as I often agree with 

things that she say. I would just simply add the lack of resources. 

It takes time. It takes money. And while you can frame it in terms 

of the new guys want to do their thing, it is also fair to say that 

relooking and relooking and relooking may not be as productive. 

But it certainly drains resources away from what might be the current need to address a pressing problem of health, safety, environment, open competition, whatever. 

And so you are dealing with a situation where you have asked 

an agency to do this and that and the next thing, and you may not 

have always given them enough resources to do that. 

Senator SINEMA. Based on current OMB guidance, agencies are 

expected to understand the impacts of proposed regulations while 

they are still under consideration, but OMB’s own documents related to the realized costs and benefits of regulation allow for a 

very wide range in estimates, creating ambiguity when agencies 

are estimating future impacts. 

So our legislation, the SMART Act, requires the solicitation of 

data from regulated entities to ensure that the regulation meets its 

objectives. How will the data that is collected to build retrospective 

review into rules help create efficiencies at agencies when it comes 

to creating new regulations or new major rules in the future? 

Ms. KATZEN. I think there are two aspects. One, as Susan mentioned in her opening statement, as you do retrospective analyses 

you learn whether the methodology you have been employing in 

doing your ex ante analysis is actually flawed in any way or could 

be improved in some way. So that helps the agency in thinking it 

through. 

It also alerts the regulated entities that they had a hand in this. 

They have a joint responsibility. They are going to have to come 

up with the data that show what the costs truly are. It is an urban 

myth that so many of the rules that we establish, at the time we 

say it is going to cost kazillions of dollars, and it turns out not to 

cost quite that amount of money. We will now be able to gauge that 

on a real-time basis with the kind of information that you are calling for. 

But one of the critical dimensions of whether a rule achieves its 

objectives is whether the rule is complied with, and that is up to 

the regulated entities, in large part. There is not a cop on every 

corner. Most businesses do want to comply. They want to know 

what the rules are and they will follow them. But it is not universal, and unless you have compliance data, you are not grappling 

with one of the key conditions. 

Ms. DUDLEY. I agree with Sally. I think it is not just the compliance costs that you will be gathering through that process, and 

that is one of the things that I really like about this draft bill, in 
10 

particular, is that it focuses on the cost, the benefits, and the outcomes. So it is a lot of information, and if agencies do not say what 

is expected, as Senator Lankford said in his opening statement, up 

front, how will we be able to measure against that? 

Oh, the other thing, I think, that is valuable in your rules is the 

definition of major. It includes more than just the $100 million 

threshold. It talks about effects on competition, health, safety, etc. 

Those are all things that agencies should be measuring in this 

planning for retrospective review, and I think that is all valuable. 

Senator SINEMA. Thank you. I often hear from Arizona businesses that they are not opposed to regulations but they feel that 

the opportunities to comment during the rulemaking process are 

often for show rather than for substance. This is partly due, of 

course, to the expectation that a draft regulation, as provided in a 

notice to propose rulemaking, already has a selected course of action, and it usually includes documentation and data to support 

that coming decision. 

So my question for you is how will the Early Participation in 

Regulations Act improve agency interactions with businesses, communities, and other regulated entities to help shape the process? 

Ms. KATZEN. Well, it sends a message that early communication 

is important, that information that comes at a very early stage will 

be received by the agency and understood by the agency. And it 

does that, actually, in a number of different ways, but probably the 

most important is if it is done early. If you wait until you have 

crafted the NPRM, and then send out an ANPRM, it is useless. It 

is sort of like doing a cost benefit analysis after you have already 

decided on the approach you want to use. You should do it at the 

outset, when you are considering all the alternatives. 

So here I would urge that you indicate—not in that statute, because it is just too much concrete—but generally suggest their 

agencies think about doing an ANPRM when they first get started, 

when they first send off notification to the Unified Agenda and say, 

we are about to have a rulemaking. We have not settled on a 

course yet. And if that message goes out, we hope it will be received and taken advantage of. 

The other thing, for all regulated entities and regulatory beneficiaries, is it is possible to talk to the agencies throughout the entire process. There are ex parte rules at some agencies, but not at 

most. And it is sometimes difficult to move someone off the mark, 

but at the same time you should not be shy about having your 

voice heard and your points made. That is an important part of the 

interaction that agencies are comfortable with and should be used. 

Ms. DUDLEY. I think that is true. I think it is important that it 

is not just businesses that will have an opportunity to get involved 

but others with information that may be relevant. And, in fact, I 

think that is what is really significant about the ANPRM idea. 

Stakeholders that have connections are involved at early stages in 

the rulemaking, working behind the scenes with agencies. What 

this does is it opens it up so that others that may have interesting, 

creative, new insights can get involved. 

I also, as I mentioned in my opening remarks, the fact that it is 

not judicially reviewable, agencies really will feel much more com-
11 

fortable getting a wide range of thoughts and really thinking 

broadly about how to solve the problem at hand. 

Senator SINEMA. Thank you. Thank you, Mr. Chair. 

Senator LANKFORD. Senator Carper. 

OPENING STATEMENT OF SENATOR CARPER 

Senator CARPER. Thanks, Mr. Chairman. I want to commend you 

and the Ranking Member for holding this hearing. This is an issue 

of real importance. You can tell by the number of people who have 

packed this hearing room and the number of television cameras 

that are just waiting outside to interview everyone who has been 

a part of this hearing. 

You have picked a couple of witnesses here. You could not have 

picked better witnesses. The balance of having Susan Dudley here 

and Sally Katzen shows a lot of wisdom, so thank you for that. 

I would like to say everything I do I know I can do better. I think 

the same is true of all of us, and I think it is probably true of most 

of our laws and rules, including the way we create rules and regulations. So thank you for coming and joining us here today. 

The last thing I am going to ask you is where do you think that 

you agree on some points that are most important, and where do 

you think maybe we do not agree on some things that are most important, in terms of what we are discussing today? 

But the last several years this Subcommittee has focused on how 

we can work to improve the regulatory process. I have served on 

this Committee for 18 years. I love this Committee, and that was 

before it was—Homeland Security, the department was created. So 

it is a great Committee and this is an important part of what we 

do. 

But the regulatory process is not perfect. No one is pretending 

that it is. And the Administrative Procedures Act and subsequent 

Executive Orders provide us with a robust roadmap for ensuring 

that agencies are promulgating rules fairly and we hope efficiently. 

The process also ensures that costs to industry are thoroughly considered and that the public has opportunities to be heard in the 

process. 

However, under the Trump administration we have seen an unprecedented number of regulatory rollbacks, and there has also 

been unprecedented effort to skip steps in the normal process for 

considering whether the costs of regulations outweigh the benefits 

that protect consumers, that protect workers, protect our health, 

and environment. 

I am especially concerned with the Office of Information and 

Regulatory Affairs’ lack of oversight of agency rulemaking during 

this Administration and with the office’s unwillingness to cooperate 

with oversight watchdogs who have legitimate requests for information. 

For example, just last week, the Environmental Protection Agency Inspection General (EPA IG) sent a rare notification to Congress 

saying that the White House Office of Information and Regulatory 

Affairs was refusing to cooperate with an audit that Senator Udall 

and I had requested about EPA’s proposal to exclude high-polluting 

glider trucks from emission rules. Glider trucks are not trucks that 

glide especially well. They are old diesel trucks that pollute a lot, 
12 

and they are cloaked in the shell of a bright, shiny, new truck. And 

the idea is that they are going to be able to continue to pollute. 

One of the greatest sources of pollution in our environment, for air 

pollution, is diesel engines, especially those that produce black carbon, which is far more dangerous to us and our environment than 

regular carbon dioxide. 

But, Ms. Katzen and Ms. Dudley, do either of you recall a time 

when OIRA refused to cooperate with an agency Inspector General 

or GAO? Do you recall a time? 

Ms. DUDLEY. Certainly during my tenure there was never a request from an agency IG of OIRA, and I think it may be unprecedented. 

Senator CARPER. Alright. 

Ms. DUDLEY. So that means I do not remember—— 

Senator CARPER. Yes. OK. 

Ms. DUDLEY [continuing]. Not refusing, but I also do not remember it ever having occurred, such a request. 

Senator CARPER. Thank you. Ms. Katzen? 

Ms. KATZEN. In my recollection it did not occur where OIRA did 

not comply with a legitimate request from GAO, from the Congress, 

or from an Inspector General. I think there may have been one but 

I am a tad fuzzy on that. 

Senator CARPER. OK. 

Ms. KATZEN. To say ‘‘no’’ to a legitimate oversight body I think 

is unprecedented, to use Susan’s word. 

Senator CARPER. OK. That is fine. You can hold it at that. Thank 

you. 

OIRA seems to be failing to ensure that the EPA is conducting 

the cost benefit and other analyses that the law requires. I am 

going to just briefly describe a couple of examples, if I could. 

First, during the interagency review of their proposal to exclude 

high-polluting glider trucks from emission rules, that I just mentioned, it was pointed out that EPA should have done more analysis because the rule was classified as, quote, ‘‘significantly, economically significant.’’ 

Instead of requiring the analysis to be done, on the day before 

the rule was signed, OIRA allowed the rule to be reclassified so 

that analysis would no longer be required. 

Second, when the EPA proposed its repeal of the clean water 

rule, it did not do a new cost benefit analysis to justify the rollback. 

Instead, EPA just deleted the benefits column of the Obama cost 

benefits table and OIRA allowed the proposed repeal rule to be 

published. 

Finally, in the EPA’s proposal to remove the legal underpinnings 

of rules to reduce emissions of mercury and other air toxics from 

power plants, OIRA allowed the agency to use the agency’s old projected cost of compliance, that were three times higher than what 

the industry actually spent to comply with the rules and ignored 

the full benefits of that rule. 

And, if I could, I would just like to ask both of you, and I do not 

expect you to be experts on any or all of the examples that I have 

just cited, but when each of you were privileged to run OIRA, 

would you have agreed to authorize the release of proposed rules 

that failed to perform a credible cost benefit analysis? 
13 

Ms. Katzen. 

Ms. KATZEN. I would have fought hard to live up to what I 

thought the office stood for, which is good analysis, good data, and 

the kind of approach to rulemaking that produces good decisions. 

Having said that, I want to just add two qualifications. One, I 

am not there now, and if you are not there, it is sometimes very 

difficult to know what pressures are being exerted, who is saying 

what, who is doing what. In terms of your specific examples, however, I could give you three or four more. 

I am an OIRA supporter. I am an OIRA booster. It was with 

great difficulty that I actually made an address earlier this year to 

the American Bar Association Ad Law Section criticizing OIRA, 

and saying that I thought it was not doing enough to ensure good 

analysis and being faithful to the cost benefit regime that it has 

guardianship of. 

There are many disturbing stories. By the same token, when I 

was in the government I read stories that were not quite right. And 

so it is not fair, necessarily, to say that something is smelly in Denmark. It may well be, and I am concerned, as well. 

Senator CARPER. OK. Thank you. Thank you. 

Ms. Dudley, this is not a trick question. I am not trying to put 

you on the spot. I think you know me. Both of you know me pretty 

well. But when you ran OIRA would you have agreed to authorize 

the release of proposed rules that failed to perform a credible cost 

benefit analysis? 

Ms. DUDLEY. I liked Sally’s comment that I would have fought 

hard, so I agree. I would have fought hard. It is important to understand that OIRA does wear several hats. One of them is interagency coordination. Another one is the analysis, the requirements 

of Executive Order 12866. And then the third is that they are part 

of the White House, and so sometimes those are in conflict. 

So in my experience there—I remember times when we held our 

nose and said, alright, but before you get to the final stage you 

must do an analysis and put teed up for questions the things that 

the agency thought they needed that would allow them to do the 

analysis. 

Senator CARPER. Good. Thank you for those responses. Mr. 

Chairman, thank you for being generous with the time and giving 

us a chance to actually have a conversation. 

Senator LANKFORD. That is great. Thank you. 

Thank you again. I am going to go ahead and say my questions 

now as well. I appreciate very much you coming through this process. Help us understand the inside of the machine. How long does 

it take to develop a rule and the economic impact, when you are 

talking about something that may be $100 million of economic impact? How much time is spent on the research side of that for the 

staff? And I know it is going to depend from rule to rule and agency to agency, but give me your ballpark guess. 

Ms. KATZEN. Well, you are right that it will vary, depending 

upon the seriousness of the issue, depending upon whether it is a 

new issue or something that is being revisited, whether there is already expertise, or whether it has to be newly learned. 

It can take a year or 2 years to do the background research. It 

can also be done in 4 or 5 months. You get a lot of help from the 
14 

outside sometimes, and a lot of obstruction from the outside sometimes. The whole process can take, as Susan said, years, occasionally. 

Senator LANKFORD. Susan. 

Ms. DUDLEY. It is hard to know exactly how long it takes, from 

the time an agency starts to think about the rule and the proposal. 

I can think of some that have been 10 years. Others can be shorter. 

There was a study that cite in my testimony that said, on average, 

5.2 years is how long it takes before the proposal is issued. 

Senator LANKFORD. So certainly 90 days is not too much time to 

be able to block up to get additional input. My thought on this is 

how to help agencies, OIRA, the future, to be able to think about, 

when is the moment to be able to put this out here? If there is 

going to be an advance notice of proposed rulemaking they are 

going to think about it for a while, maybe a year even. They are 

doing some of their own research and then they decide we may be 

headed toward a rulemaking here. This is a lot of information and 

it is all directed toward the same spot. 

Where is the appropriate moment for them to go drop out an advance notice of proposed rulemaking to make sure that the insight 

in the information is coming at the right time, rather than, again, 

they form everything they need to do over 2 years, or 5 years of 

time, and 90 days before they plan to do it anyway they just drop 

this out there because it is formality? 

Ms. KATZEN. Well, I mentioned in my testimony that I think it 

would be useful to issue an ANPRM as soon as the agency sends 

its first submission to the Unified Agenda. This is the document 

that is supposed to be available—well, it is available to the public. 

It is supposed to be a menu of everything that the agencies are 

thinking about doing. When they get to the point of saying, ‘‘We 

are thinking about doing a rulemaking on this,’’ that would be an 

appropriate time an ANPRM for for some rules. Again, I do not 

want to see too much of this cast in concrete—— 

Senator LANKFORD. Right. 

Ms. KATZEN [continuing]. And prescriptive. But I think once they 

know they are heading down that path, putting out an ANPRM 

would be helpful. 

Ms. DUDLEY. I think that is right. I think the first note is in the 

unified agenda, soon thereafter. That notice means we are thinking 

about this within the next 6 months. That would be a good time 

to do it. 

I also would hesitate to see the bill be too strict about that, because different agencies have different practices. But I think the 

way that you have put OIRA in charge of that, or OIRA as the coordinator for the ANPRM bill, that would be something that agencies could work with OIRA on. 

Senator LANKFORD. And again, the challenge that we have is we 

do not want to be prescriptive, because we cannot look in the future 10 years and we cannot see every rule and every entity. But 

the goal of this is to be able to get more dialogue early, not for the 

agencies to finish all their work and then go through the formalities. 

We are not trying to make it longer, or, as you mentioned, creating an obstacle course here at the end. We are trying to get dia-
15 

logue early on, and if the dialogue is too late in the process it does 

not matter. We are still in the same spot. But if it is too early, we 

do not even know if it is a $100 million rule at that point. We really do not see it. 

So there has to be some level of information even to note, hey, 

I think this may be headed toward a major rule. They at least have 

to have enough information there to know what questions to ask, 

the size and complexity of it. This is a health and safety issue that 

is coming. We see some of the things that are going to be around 

it so let’s get more information out there. Does that make sense? 

I guess I am asking a clarifying question. Are we hitting the 

right balance that we are not just creating an obstacle but we are 

getting this early enough? Should we be more prescriptive on when 

to be able to put this, or do we leave it at the discretion of OIRA, 

at a future OIRA, to say you did not do this ANPRM early enough 

and now we have a battle in the same process? 

Ms. KATZEN. I would not be prescriptive. When I was in the government, in the Administration, I took committee reports seriously, 

and I think to make clear your interest in having this done earlier 

rather than later in the committee report would be a very useful 

way of sending the message. But I would not cast it into legislative 

language. 

Senator LANKFORD. OK. Fair enough. 

Ms. DUDLEY. I agree. 

Senator LANKFORD. OK. So let me ask you an even sillier question. Advance notice of proposed rulemaking just gets all of us excited here, as a term. Some agencies have just said request for information, to make it more generic and blunt. There have been a 

lot of different terms they have used, to basically accomplish some 

of the same things. 

Is this the right term? The advance notice of proposed rulemaking has been used before. We have it in statutory language. Is 

that the right term to be able to even use on this? If we want to 

take off our D.C. hat and to say if someone outside of D.C. gets an 

advance notice of proposed rulemaking they have no idea what it 

is. If someone gets a request for information everyone knows what 

it is. Is one better than the other? 

Ms. DUDLEY. Even though it is a mouthful I think advance notice 

of proposed rulemaking is the best—is a good way to say it. Agencies can always change something that they might have called a 

notice of inquiry before and just change the name. 

One thing that I think is valuable about it, that you would not 

want to lose, you would not want to make it so loose that agencies 

could just reach out to specific stakeholders. 

Senator LANKFORD. Right. 

Ms. DUDLEY. You do want this to be something that—— 

Senator LANKFORD. It needs to be broad. 

Ms. DUDLEY [continuing]. Is public, yes. 

Senator LANKFORD. I agree. Ms. Katzen. 

Ms. KATZEN. And again I would refer to the committee report to 

clarify that an advance notice of proposed rulemaking, by any other 

name, would still accomplish the same objective. And you could call 

it a request for information. You could call it a request for pro-
16 

posals. And again, that would send a message that I think would 

be helpful. 

Senator LANKFORD. As long as it is public and as long as it is 

broad, because one of the challenges that we have is small business, large business, individuals, outside groups, environmental 

groups, think tanks, whatever it may be, they all need to have access to be able to submit ideas to it because they are all going to 

have different ideas and thoughts. 

OK. So let me ask this. Are we overcomplicating this by adding 

another step with the advance notice of proposed rulemaking? Is 

there a way to be able to fix the notice of proposed rulemaking and 

to try to get more input from the public earlier in that process 

without having to do a formal process? I am willing to try to say 

let us go back and do it, because right now that process is so late 

in the process, or that product is so late in the process that a lot 

of decisions have already been made, and it is just tough to be able 

to engage with people. 

As I think both of you mentioned in your testimony, once folks 

have done their own research, bounced their own ideas, have gone 

through the obligatory, working with all their bosses, explaining 

why this is the best idea, people get more and more entrenched. 

So what I am trying to figure out, is there a way to fix that process 

rather than add a new one? 

Ms. KATZEN. I think we had decades of court cases that have spoken to what needs to be in the NPRM itself, and how much the 

final rule can deviate from the NPRM. I am referring here to the 

‘‘logical outgrowth doctrine’’ that says that if the agency changes its 

mind between the NPRM and the final rule, in a dramatic fashion, 

that may well have deprived the commenting public of an opportunity to comment on the final product—— 

Senator LANKFORD. Right. 

Ms. KATZEN [continuing]. And, therefore, it has to go back to the 

agency. And with 20, 30, or 40 years of cases having been built up, 

I would reluctantly say add a new step at the beginning rather 

than try to change the NPRM at this point in the face of those 

cases. 

Senator LANKFORD. OK. 

Ms. DUDLEY. I agree. I mean, I think it is the judicial 

reviewability of the NPRM that really forces agencies to have chosen what they want and defended that in a way that they are discarding other options. 

Senator LANKFORD. OK. Let me switch to the end, if I can, here. 

OIRA has the unique opportunity to be able to see multiple agencies simultaneous, and when we talk about retrospective review 

some of the issue is other agencies have now formed regulatory 

processes, or there is something new in another entity that maybe 

this regulation is not as pertinent anymore because this entity has 

created something that is also doing something similar, or whatever it may be. 

How can we help OIRA continue to be able to see all agencies, 

and when a retrospective review happens it is not just isolated to 

a particular agency but it still has that broad review? Is there anything that we need to build in? 
17 

Ms. DUDLEY. More staff. I think that part of the reason that 

OIRA is so transactional is because their staff must triage always 

to be able to be reviewing things. So to be able to step back and 

look across, I would say more staff. 

Senator LANKFORD. OK. 

Ms. KATZEN. In her testimony, Susan mentioned the function of 

OIRA as convening an interagency process, and I think that is the 

beginning of what you would like to lead to. If you look at Executive Order 12866, one of the definitions of ‘‘significant’’ is where 

there is something that an agency is proposing to do that is inconsistent with what another agency is proposing, or has already done, 

so that we do not have the Department of Labor (DOL) working on 

transporting hazmats, and the Department of Transportation 

(DOT) working on transporting hazmats, and they are at across 

purposes. 

I thought that that would be a good vehicle for bringing together 

different agencies that were focused on a similar rule, and I see 

that in this context of retrospective review, where there might be 

similar rules at different agencies. That works sometimes, but by 

the time they have gone through the process of developing their 

rule, and how to implement it, how to enforce it, they are less willing to trade it for somebody else’s rule. 

I think additional staff would help, and I think, again, oversight 

by the committee and language in the committee report would be 

helpful to clarify that this is one Administration, one Executive 

Branch, and that the various fingers should be working together 

rather than at across purposes. 

Senator LANKFORD. Right. 

There is some confusion on repealing a rule and going through 

that process, only that some people outside of government think if 

you walk in and do a review, then you decide you do not like it, 

you just take it out and the next day it is gone. Can you walk us 

through a little bit of the process that happens to actually go 

through a rule repeal? 

Ms. KATZEN. OK. This is under the State Farm decision of the 

Supreme Court in the 1980s, having to do with airbags. And what 

it says is that if you are going to repeal or modify a regulation, you 

have to use the same processes that you used to create that regulation. Therefore, if you wanted to repeal or modify something that 

is on the books, you would need a notice of proposed rulemaking 

which sets forth the objective to modify an existing rule. It would 

need to be accompanied by data that documents the need for, the 

appropriateness of, making that modification. 

And then you would get comments. So you have a notice of proposed rulemaking, and possibly now an advance notice of proposed 

rulemaking, and then the notice of proposed rulemaking, and then 

the comment period, and then the final rule, which would repeal 

the earlier rule. 

Under a different Supreme Court case called Fox News, the Supreme Court has said that when you are changing your mind you 

have to (1) acknowledge that you are changing positions, and (2) 

if the previous rule was based on science, engineering, technical information, or economic information, you must show how that has 

changed, in what way is it different, why it is that you no longer 
18 

want to rely on the previous rule. Because there is new data? Because there is a new approach? Or whatever reason, and that explanation has to be part of the reason and basis, as it is called, of 

the new final rule. 

So that is a whole new rulemaking proceeding, which can take 

a year, 2 years, and depending upon the basis for it, the assemblage of a lot of information and analysis. I hope that was clear. 

Senator LANKFORD. That is exceptionally helpful, actually. 

Susan, do you want to add anything to that? 

Ms. DUDLEY. No. That was very clear. 

Senator LANKFORD. Is it the right process to go through, you 

think? 

Ms. DUDLEY. Yes. 

Senator LANKFORD. I would assume that you would say that, because there is a great need to be careful. When you change a regulatory scheme of things that affects a lot of people, and you want 

to have as much impact as you can. Taking a rule out is the same 

as putting a rule in, and that creates uncertainty in the environment, and you have to be able to make sure everyone has input 

through that process. 

Ms. KATZEN. I agree completely. 

Senator LANKFORD. OK. So the question about retrospective review really goes back to the question of how do you determine 

whether things are working, and the metrics. I jokingly used the 

term, if there is a high school basketball team that is really bad 

at free throws, and the coach decides every time you miss a free 

throw you are going to take a lap, and that is his way to fix it, but 

a year later they are still not any better at free throws. They may 

be better cardiovascular but their free throws are still bad—we did 

one action to try to fix a problem but it did not work. There has 

to be some metrics at some point that have to decide, are we better 

at free throws on this? 

I look at that in the same way with regulations, to say at some 

point we have a goal to say this is what we are trying to do, whether it clean the air or safer work environment, more consistency, 

better science. Whatever it may be there is a standard for what we 

are trying to accomplish. I think it is reasonable to say 10 years 

from now we should go back and look at it, and say did it actually 

accomplish that? The hard part is developing that metric. 

So my question for you is, the developing the metric portion of 

it, is there any other definition that we need to put into this to give 

to OIRA or to the agencies to say when you develop a metric make 

sure you are thinking about this. Do you think what we have put 

into place, between Senator Sinema and I, is broad enough but is 

also clear enough that it is going to lay the groundwork for them 

to be able to do metrics, when you are looking a decade in advance? 

Ms. DUDLEY. I do think it is so important to be able to define 

clearly what outcome you expect to get from the regulation, and a 

surprisingly limited number of regulations do that in a clear way. 

I thought that what you had in the draft made sense. It covers 

the things that people talk about—the outcomes, which is general 

enough to cover a lot of things; benefits costs. 

I would love to take the opportunity to look at it more and see 

if I do have some specific ideas I am in a public policy school where 
19 

a big focus is program evaluation. So some of the experts on that 

are at GW. I would love to talk with them, and say, ‘‘OK, does this 

get you what you would need 5 or 10 years down the road, to be 

able to do that evaluation?’’ 

Senator LANKFORD. That would be great. We would be glad to be 

able to have that input. 

Ms. DUDLEY. OK. 

Senator LANKFORD. Should I put out a formal request for information to be able to get to that? 

Ms. DUDLEY. An advance notice of a formal—yes. 

Senator LANKFORD. OK. Thank you. 

Ms. Katzen. 

Ms. KATZEN. I agree with Susan on that, and my only caveat 

would be not to be too prescriptive. Ten years is actually a long 

time, given how we learn and how we develop. And we learn so 

much that to try to be specific might be greatly counterproductive 

here. 

Senator LANKFORD. The hard part is trying to balance. You want 

to be specific enough that you actually know if you get there. I will 

use my free throw example. If the free throw percentage does not 

go up then that method did not work, so let’s go back and review 

it because we are trying to get to the end method. But general 

enough that the unknowns that are still out there can still be determined. 

I guess what I am trying to say is if you make the metrics so 

vague, anything can hit it. It is not really a metric then. It is improved energy in the country. OK, what does that mean? So there 

has to be something specific enough. 

Ms. KATZEN. I thought the proposal to include the objectives and 

the metrics at the NPRM stage was beneficial, because then those 

who are affected by the regulation, whether they be the regulated 

entities or the regulatory beneficiaries, will be able to comment on 

that during the comment period and lend their insight and their 

expertise to the agency, so that there would be a finite review of 

those two pieces when it is fresh in their minds, when they are still 

thinking about what they are trying to accomplish. I think that 

should be sufficient. 

Senator LANKFORD. OK. Great. 

Senator Sinema, you have a question as well? 

Senator SINEMA. Yes. Thank you, Mr. Chairman. 

For Ms. Katzen, you discussed escape hatches for repeated reviews of rules. In the SMART Act we have included a provision 

that allows the agency to create a list of circumstances in consultation with the OIRA administrator, which would require the performance of a subsequent review. 

Do you think that provision adequately guards against unreasonable review requirements? 

Ms. KATZEN. I believe so. I think the agency should be the responsible official who says we have looked at this, we are not going 

to make any changes, and further review would not be very helpful. 

I mean, I always use the example of airbags or seatbelts. Do we 

want to look at them in 10 years, then 10 years more, then 10 

years more? Well, assuming we do not have V2V, assuming we do 

not have autonomous vehicles—but even with autonomous vehicles 
20 

I want my seatbelt and airbag in the car. Why ask the agency to 

go through that? I think the agency, Secretary of the Department 

or the agency official, is the person to be able to say enough is 

enough. 

Senator SINEMA. Regulatory impact analysis documents, which 

accompany regulations, provide wide ranges of estimated costs and 

benefits. The uncertain nature of forecast-based analysis hides the 

true impacts and benefits for rules. So through the data collection 

requirements of the SMART Act, can we expect that, over time, 

agencies will become better positioned to accurately forecast both 

costs and benefits? 

Ms. DUDLEY. I would hope so. I think one of the most important 

potential outcomes of retrospective review is that it is going to 

make us better at predicting things in future regulations. So both 

thinking about the data and analysis and what models we use to 

make those predictions? They are so uncertain and we never go 

back and check. So I think that is a key benefit. 

Ms. KATZEN. Agreed. 

Senator SINEMA. A final couple of questions. In written testimony 

you discussed a number of rules that would be applicable under the 

Early Participation in Regulations Act, and I believe you said 70— 

is that correct, 70?—but that many rules would not benefit from 

the bill’s requirements. 

Could you discuss these rules that would be major but would not 

benefit from an ANPRM? 

Ms. DUDLEY. Yes, the 70 includes all the independent regulatory 

agencies. If you just look at the Executive Branch agencies it is 

closer to 40 or 50 that would fall under the rule. 

One set of rules are hunting bag limits, and there probably a half 

a dozen of those every year. OIRA has long let the Department of 

Interior (DOI) use the same regulatory impact analyses that they 

probably prepared 15 or 20 years ago, as a regulatory analysis. 

That would not be the kind of thing that would need any advance 

notice. 

I would expect, although I am not positive, that a lot of the Medicare and Medicaid rules—the regulations that are determining 

what fees different doctors or services should receive—again, that 

is not the kind of thing that advance notice might be valuable for. 

So I can imagine that early on OIRA and the agencies would 

streamline that and identify some exemptions. 

Ms. KATZEN. I agree with that, and in my testimony I mentioned 

that there are occasions when the congressional delegation, which 

is the basis for any rulemaking, it itself highly prescriptive. I am 

thinking of probably the unregulated example of positive train control, which did not give the agency any discretion to do anything 

other than what was specified in the statute. That being the case, 

an advance notice of proposed rulemaking would not make a whole 

lot of sense. The agency’s hands are tied by its authorizing statute. 

Senator SINEMA. Thank you. Thank you, Mr. Chair. 

Senator LANKFORD. Senator Carper. 

Senator CARPER. Thank you both again for your testimony today. 

As I am sure you know, under the Paperwork Reduction Act (PRA), 

the Office of Management and Budget must review and approve 

Federal collections of information before they are, I guess, con-
21 

ducted. After reviewing the agency request, the OMB may approve 

or disapprove that request, or can go ahead and define conditions 

that need to be met for approval. 

The OMB is required to ensure that any information collection 

maximizes practical utility and public benefits and protects integrity, objectivity, and impartiality of collected statistical information. 

Last May, I led a letter with 34 of our colleagues, both in the 

House and the Senate, requesting information from the Commerce 

Department and OIRA as to how they planned to ensure that the 

Paperwork Reduction Act requirements were met with respect to 

the addition of a question on citizenship on the 2020 Census. That 

was last May, a year ago. We have not received a response to that 

letter. 

Let me ask, first, if I could, Ms. Katzen, as you know, concerns 

have been raised about adding a question on citizenship, due to the 

potential negative consequences, including a lower self-response 

rate, which would lead to a less accurate and more costly Census. 

I would just ask would you please weigh in for us and explain 

OIRA’s role in reviewing information collections, and what OIRA 

should be reviewing with respect to the citizenship question and 

the 2020 Census? Could you take a shot at that? 

Ms. KATZEN. I will try. Thank you. 

Senator CARPER. Thank you. 

Ms. KATZEN. Under the Paperwork Reduction Act, any question 

being posed to the public, identical questions to 10 or more people, 

has to receive, in effect, a comment period by the agency, and then 

if the agency decides to go ahead with it, the agency will then send 

it to the OMB, which has a second comment period. And as you 

correctly stated, one of the conditions for approval by OMB is to 

maximize practical utility. 

There is also a requirement to minimize burden. It is a modified 

cost benefit analysis. Are we going to get something from this paperwork requirement that will be useful and well worth whatever 

offsetting cost there is? 

Now with the Decennial Census, for 2020, there has been a lot 

of publicity, and I have actually read the three district court cases, 

one of which is 267 pages. So I have some knowledge about the 

specifics of that particular paperwork requirement. 

I think the most significant aspect is that the primary purpose 

of the Decennial Census, which is embodied in the Constitution— 

it is the only paperwork requirement embodied in the Constitution—is for the enumeration every 10 years. That is the primary 

purpose. 

The addition of a question relating to citizenship is a question 

which, at least pretextually, has been justified by assisting the Department of Justice (DOJ) in better enforcing voting rights cases. 

That is a secondary purpose. If the secondary purpose is going to 

have an adverse effect on the primary purpose, one would have a 

very hard time justifying it under the Paperwork Reduction Act, 

because every past Census Bureau director, and vast numbers of 

statisticians, including the National Academy of Science, has said 

that this will decrease the response rate significantly for the enumeration purposes, and that the data are not needed in the Decen-
22 

nial Census because there are alternative places where this data 

reside, in the ACS study and other kinds of statistical compilations. 

With that, it is hard to see how the Decennial Census with the 

citizenship question would pass muster under the Paperwork Reduction Act. Now there are all sorts of politics, policies, whatever, 

but I was just trying to focus—— 

Senator CARPER. OK. Good. 

Ms. KATZEN [continuing]. On the PRA. 

Senator CARPER. Thank you. Mr. Chairman, you have been generous with your time. I would like to ask, if I could, for the witnesses to each give us like maybe one change, amendment that you 

would suggest to the legislation that Mr. Chairman has designed, 

just one change. And as I said earlier, I never introduced a perfect 

bill, or probably a perfect amendment, and as good as these two 

legislators are, these two Senators are, there is probably room for 

improvement. Can you think of one thing that you would suggest 

that we amend as we take it to markup? 

Ms. Dudley. 

Ms. DUDLEY. Perhaps that the 10-year window for review, it provides for one review at 10 years, and then nothing thereafter. And 

that may be appropriate for some types of regulations but not for 

others. 

So I think Sally and I disagree on this. I might like to see a continued requirement for that retrospective analysis that would continue to measure and continue to observe whether we are achieving 

the desired objective. 

Senator CARPER. All right. Thank you. Just one idea, Sally, if you 

would. 

Ms. KATZEN. Susan is correct that I disagree with her suggestion. 

I was looking at what was supposed to go in the ANPRM, and I 

came across something which I question its utility, and that was 

in the ANPRM to list the legal authority under which a major rule 

may be proposed. And then the following language: ‘‘including 

whether a rulemaking is required by statute, and, if so, whether 

by a specific date, whether the agency has discretion to commence 

a rulemaking.’’ 

I can envision circumstances where the discretion is reduced 

with greater incidence of death or harm or safety of some sort. I 

had not really focused on that particular language, but I am not 

sure that, if I were giving you one thing that think I might want 

to change from the latest draft—which looks very good, indeed—it 

might be to stop that little a iii, whatever it is, little iii sooner. 

Senator CARPER. Alright. Thank you. 

For some people, you remember the saying—this is about as exciting—talking about an experience in their life—they said it was 

about as exciting as watching wet paint dry. I suspect for some 

people a hearing on this subject is just like that. I think it is a terrifically exciting hearing, and I applaud our Chair and Ranking 

Member for bringing us together, and look forward to working with 

you to—what does it say in the Constitution?—‘‘We the People of 

the United States of America, in order to form a more perfect 

union,’’ to maybe work with you to see if we can form an even more 

perfect bill. Thank you. 
23 

Senator LANKFORD. Sure. By the way, my oldest daughter graduated and is out and has a real life, and my wife and I repainted 

her sunshine-yellow bedroom into a color where guests could actually sleep. I put the initial primer coat on, and put the regular coat 

on, and watched the yellow disappear, and as I watched the paint 

dry I thought, I am standing here watching paint dry, and it is exciting. [Laughter.] 

Because I feel like we are getting a room back. So watching paint 

dry is not always bad. 

Senator CARPER. Well, hopefully they will come back and sleep 

in these rooms sometime. That sounds like a lot of fun. 

Senator SINEMA. I think I would like the yellow. 

Senator LANKFORD. The sunshine yellow? 

Senator SINEMA. Yes. 

Senator LANKFORD. It was a happy color. 

Senator SINEMA. Yes, I would like that. 

Senator LANKFORD. She loved it, all the way through high school. 

She loved it. 

Senator LANKFORD. Let me say thank you to both of you. I would 

ask you both two quick questions, and one you can think about and 

send back your ideas, just as a message to say this Committee is 

always interested in your input. So as you have good ideas we are 

always listening. So consider this your advance notice of proposed 

legislating, that we are interested in the ideas. 

One is, Ms. Katzen, several times you have mentioned surgical 

changes to the APA. That is what we are trying to do, not a massive wholesale shift but surgical changes. If you have thoughts on 

other surgical changes where we need to legislate on—we focused 

today on beginning and end, but if there are other areas we are interested, and we are trying to be able to work through to try to find 

ways to be able to help fix the process long-term. So that is my 

homework assignment to you. If you think of anything away from 

here, contact us or our team, and we would be glad to be able to 

hear that. 

The other one is, based on where the text is right now—I got 

your comments from Senator Carper—do you support these bills 

where they are right now, and continue to move forward in the 

process? 

Ms. DUDLEY. Absolutely. 

Senator LANKFORD. Yes. 

Ms. KATZEN. I would indeed. 

Senator LANKFORD. Yes. Thank you. Thanks for all your input. 

You have both been exceptionally professional with our team in 

trying to be able to provide some additional input and thought. You 

both have very busy lives on your own. You do not work for us, but 

you do work for the American people still—that is pretty obvious— 

of your continuing engagement in policy areas. So thanks for continuing to be able to give your time, to be able to help the Nation 

in the future. 

Let me make a quick closing statement and we will shut us 

down. Before we adjourn I do want to announce that on May the 

22 this Subcommittee intends to hold a joint hearing with the Senate Small Business and Entrepreneurship Committee to examine 

the disproportionate impact regulations have on small businesses. 
24 

Both of you have dealt with that quite a bit, actually, in your time 

at OIRA. All members of the Homeland Security and Governmental 

Affairs Committee (HSGAC) are invited to attend this hearing. 

That concludes today’s hearing, though. I do want to thank again 

our witnesses for their testimony. The hearing record will remain 

open for 15 days until the close of business on May the 22, for the 

submission of statements and questions for the record. 

This hearing is adjourned. 

[Whereupon, at 11:20 a.m., the Subcommittee was adjourned.] 
(25) 

A P P E N D I X 
26 

• The Administrative Procedure Act sets out a process where agencies propose a 

rule, listen to comments from the public, have the opportunity to make changes, 

and then issue a final regulation. 

• While this system looks good on paper, in practice, agencies conduct a significant 

amount of work before formally engaging the public- they will consider various 

regulatory schemes and conduct economic analysis. Ideally, agencies would 

engage the public early, but there is no APA requirement to do so. 

• Turning to the other end of the rulemaking process, retrospective review is a 

process to ensure rules achieve their intended goal in the least burdensome way. 

Over time, changing circumstances and improved technology may render some 

regulations ineffectual or unnecessary. 

• An agency's job is not done after a final rule is published. As initial estimates of 

both costs and benefits can prove inaccurate, agencies should revisit a rule to 

ensure the desired effects are achieved. 

• Every President since Jimmy Carter has urged agencies to utilize retrospective 

review to examine existing regulations. While these directives were issued with 

good intentions, they gave agencies a significant amount of discretion in selecting 

which, and how many rules to review. 

• The focus of this hearing is two bills Senator Sinema and I will introduce shortly 

that codifY the best practices for both procedures. 

• The Early Participation in Rulemaking Act directs agencies to issue advanced 

notices for rules costing more than $100 million annually. 

• The agency must outline what problem the rule intends to solve and listen to the 

public's input on the subject. 
27 

• The idea behind this bill is to require agencies to listen to the public before they 

craft a regulation- Washington does not have all the answers, taking time to work 

with stakeholders, particularly our small businesses, is vital in crafting effective 

regulations. 

• Less burdensome does not mean less effective. Business owners want to be good 

citizens, follow the law, and have safe and clean workplaces. 

• The Setting Manageable Analysis Requirements in Text Act (or SMART Act) is a 

retrospective review bill that looks ahead- it requires agencies to set metrics for 

how a rule will be measured for success in the future. 

• It is hard to measure the success of anything unless it is defined. This bill instructs 

regulators to define what "success" is for a given rule and then requires them to 

grade the rule within 10 years. 

• This Subcommittee has been working on both of these issues for a while and both 

bills have had bi-partisan support for the past two Congresses. I look forward to 

working with my colleagues to push them both across the finish line. With that, I 

recognize Senator Sinema for her opening statement. 
28 

E E E 

Prepared Statement of Susan E. Dudley 

Director, GW Regulatory Studies Center 

Distinguished Professor of Practice, 

Trachtenberg School of Public Policy and Public Administration 

Hearing on 

From Beginning to End: An Examination of 

Agencies' Early Public Engagement and 

Retrospective Review 

Homeland Security and Governmental Affairs Committee 

Regulatory Affairs and Financial Management Subcommittee 

United States Senate 

May 7, 2019 

The George Washington University Regulatory Studies Center 

805 21" St, NW, MPA Suite 600 

Washington, DC 20052 

202-994-7543 

RegulatoryStudies@gwu.edu 

www.RegulatoryStudies.gwu.edu 
29 

From Beginning to End: An Examination of Agencies' Early Public 

Engagement and Retrospective Review 

Prepared Statement of Susan E. Dudley -'"'''"'''"'"~-···-······· 

Thank you, Chairman Lankford, Ranking Member Sinema, and Members of the Subcommittee 

for inviting me to share my thoughts on early public engagement and retrospective review of 

regulations. I am Director of the George Washington University Regulatory Studies Center, and 

Distinguished Professor of Practice in the Trachtenberg School of Public Policy and Public 

Administration.' From April 2007 to January 2009, I oversaw federal executive branch 

regulations as Administrator of the Office oflnformation and Regulatory Affairs (OIRA) in the 

Office of Management and Budget (OMB). I have studied regulations and their effects for over 

three decades, from perspectives in government (as both a career civil servant and political 

appointee), the academy, and private consulting. 

I appreciate the Subcommittee's interest in improving how the U.S. government develops and 

evaluates regulatory policy. Your efforts continue a long bipartisan tradition in the United States 

of efforts to make regulation well-informed, transparent, and accountable to the American 

people. By I) engaging public input earlier in the regulatory development process and 2) 

providing for retrospective review of regulations to evaluate whether they are achieving their 

objectives, the bills you have proposed can help ensure that regulations are based on the best 

evidence available and that they are working as intended for the American people. 

My testimony reviews the problems necessitating the practices required by your legislation and 

addresses and examines each bill's requirements and impacts. It concludes with some crosscutting comments and observations. 

Engaging Public Input Early in Rulemaking 

The Problem 

The Administrative Procedure Act of 1946 requires agencies to publish a "general notice of 

proposed rule making ... in the Federal Register," and "give interested persons an opportunity to 

participate in the rule making through submission of written data, views, or arguments with or 

1 The George Washington University Regulatory Studies Center works to improve regulatory policy through 

research, education, and outreach. This statement reflects my own views and does not represent an ofticial 

position of the GW Regulatory Studies Center or the George Washington University. 

The George Washington University Regulatory Studies Center 2 
30 

without opportunity for oral presentation."2 In addition, every president since Jimmy Carter3 has 

required agencies to examine expected regulatory impacts before issuing proposed and final 

regulations; Executive Order (E.O.) 12866 has guided this analysis for more than 25 years.4 

Despite these longstanding requirements, agencies often develop regulatory impact analyses 

(RIAs) after they have made key policy decisions; many analyses appear to be designed to justif'y 

regulatory actions, rather than inform them.5 Agencies view their RIAs and preambles to 

proposed rules as legal documents, prepared in anticipation oflitigation.6 The need to defend 

their selected regulatory approach motivates agencies to "circle the wagons," narrowing the 

menu of alternatives and the evidence they consider before the public has an opportunity to 

engage. As a result, changes in response to notice and comment "tend to be small and painful, 

and they are often subtractive rather than innovative or additive."7 

The Early Participation in Regulations Act of 2019 

The draft "Early Participation in Regulations Act of2019" would require agencies to issue for 

public comment advance notices of proposed rulemaking (ANPRMs) for major rules. This 

requirement could free the agency to share its early thinking on whether a problem requires a 

regulatory solution and what different options are available. As such, ANPRMs could be 

valuable for soliciting input from knowledgeable parties on a range of possible approaches, data, 

models, etc., before policy decisions are framed, or positions established.8 As the President's 

Jobs Council noted in 2011, resulting regulations would benefit from critiques, feedback, and 

other public input provided by ANPRMs.9 

2 5 U.S. Code§ 553(b) and (c). 3 Jimmy Carter, Exec. Order No. 12044, 43 Fed. Reg. 12661 (Mar. 24, 1978). 

4 See A Forum Cclebrating25 Years of Executive Order 12866, September 24, 2018. The George Washington 

University. 5 Christopher Carrigan and Stuart Shapiro. "What's Wrong with the Rack of the Envelope'' A Calll\11' Simpleiill:ll! 

l'imelyl Bene tit-Cost Analysis.·· Regulation and Governance. (2017) See also Susan Dudley, Richard Belzer, 

Glenn Blomquist, Timothy Brennan~ Christopher Carrigan, Joseph Cordes, Louis A. Cox, Arthur Fraas, John 

Graham, George Gray, James Hammitt, Kerry Krutilla, Peter Linquiti, Randall Lutter, Brian Mannix, Stuart 

Shapiro, Anne Smith, W. Kip Viscusi and Richard Zerbe. "Consumers Guide to Regulatory Impact Analvsis: 

Ten Tips for Being an Informed l'olicymaker." Journal of Benefit-Cost Analysis. (July 27, 2017). 6 Wendy E. Wagner. "The CAlR RIA: Advocacy Dressed Up as Policy Analysis." In Winston Harrington eta!. 

(Ed.), Reforming Regulatory Impact Analysis. Washington, DC: Resources for the Future. (2009). 7 William F. West. "Formal Procedures, Informal Process, Accountability, and Responsiveness in Bureaucratic 

Policy Making: An Institutional Policy Analysis." Public Administration Review 64( I): 66-80 (2004). 8 Steven J. Balla, and Susan E. Dudley. "Stakeholder l'articipation and Regula!Qry Policymaking in the United 

States." Report prepared for the Organisation for Economic Co-operation and Development. (2014). Susan E. 

Dudley and Kai Wegrich, "Reculatorv Policy and Practice in the United States and European Union'' (Mar. 10, 

2015) (Geo. Wash. Univ. Regulatory Studies Ctr. working paper). Susan Dudley and Marcus Peacock 

"Improving Regulatory Science: A Case Studv of the National Ambient Air Qualitv Standards," Supreme Court 

Economic Review. (August 2018). 9 President's Jobs Council. Road Map to Renewal: 2011 Year-End Report. 

The George Washington University Regulatory Studies Center 
31 

ANPRMs could encourage better, more informed regulatory analyses. Experts have suggested 

that "back of the envelope" analyses could encourage agencies to consider the effects of a wide 

range of alternatives before they narrow their focus to just a few options. 10 Empirical research 

found that "pre-proposal notice[s] requesting comment from the public ... are associated with 

higher quality regulatory impact analyses" supporting final regulations. 11 

The bill would require an ANPRM for a major rule to identifY the problem that may call for 

regulation, and data or information that supports that regulatory need. This is an essential first 

step for developing effective regulation. E.O. 12866 calls on each agency to "identifY the 

problem that it intends to address (including, where applicable, the failures of private markets or 

public institutions that warrant new agency action) as well as assess the significance of that 

problem." 12 Yet, in my experience, agencies too often proceed with developing a regulatory 

solution without first clearly articulating the nature and significance of the problem to be 

solved. 13 The bill's related requirement to identify "an achievable objective for the major rule" 

would also serve to focus public comment, subsequent analysis, and evaluation. 14 

ANPRMs subject to the bill would also present a general description of alternatives the agency 

has identified for consideration. Laying out a range of preliminary alternatives early in the 

regulatory development process could elicit invaluable input from the public, not only on the 

merits of those alternatives (including data, analysis, experience) but suggestions of other options 

for agency consideration. 

Together, these ANPRM elements should not be unduly demanding or burdensome; they would 

merely make the factors influencing the agency's thinking more transparent to the public at a 

stage when public input could be very valuable. 

Over the past few decades, agencies have issued an average of 13 significant ANPRMs per year, 

representing less than 5% of their significant regulatory actionsY As noted below, given the 

bill's definition of"major rule," as many as 70 regulatory actions a year 16 could begin with an 

10 Carrigan and Shapiro (2017). 11 Jerry E!lig and Rosemarie Fike. "'Regulatory Process, Regulatory Reform. and the Quality of Regulatory Impact 

Analysis." J. Benefit Cost Anal. 2016; 7(3):523-559. 12 E.O. 12866 Sec. 1(b)(l). 13 Dudley et al. (2017). 14 Marcus Peacock, Sofie E. Miller, and Daniel R. Perez, "A Proposed Framework for Evidence-Bused Regulation" 

(George Washington University Regulatory Studies Center working paper). p. 22. (Feb. 22, 2018). 

" Juliana Balla. ''Earlv but Not Ollen: A Look into the lise of ANPRMs in Rukmuking," Regulatory Policy 

Commentary. GW Regulatory Studies Center. May 3, 2019 16 Major rules include regulations issued by independent regulatory agencies, which are not captured in the above 

counts. Executive branch agencies issue an average of 40 regulations per year that would likely meet the bill's 

definition of major. See the data maintained by OIRA and the General Services Administration at 

www.Reglnfi.>.gov. 

The George Washington University Regulatory Studies Center 4 
32 

ANPRM. However, the number would likely be less than that since the bill provides for 

exceptions. 

The bill appropriately gives OIRA authority for determining whether a rule is major under the 

section, and whether an exemption should apply. OIRA is well-positioned to make those 

determinations given its authorities under the Congressional Review Act (CRA) and presidential 

executive orders. 

The bill also wisely precludes judicial review of any differences between an agency's ANPRM 

and subsequent NPRM. One virtue ofthe ANPRM is that it provides an opportunity for agencies 

to share their preliminary thinking about a problem and get input on potential solutions at a stage 

when they are truly open to feedback, analysis, and evidence. If agencies had reason to fear this 

early notice could later be used against them in court, that would discourage objective queries 

and undermine these benefits. 

The requirement to issue and accept comment on an ANPRM before proceeding to a proposal 

should not unduly slow agencies' regulations for several reasons. First, 90 days is not a long time 

considering that agencies often take years studying a problem and evaluating regulatory options 

before they issue a proposed rule. One study found that "the average interval between the formal 

initiation of research on a policy issue ... and the publication of a proposed rule ... was 5.3 

years." 17 

Perhaps more important, to the extent the ANPRM serves to open up for public engagement 

preliminary deliberations that would otherwise have taken place behind closed doors, it may 

make the overall rulemaking process more efficient. Rather than tacking 90 days onto the 

rulcmaking schedule, it may provide valuable input that ends up streamlining the subsequent 

notice-and-comment process. In many cases, early engagement could lead to more efficient 

analysis at the NPRM stage and fewer surprises during public comment. While there will 

certainly be cases where an ANPRM would not serve the public interest, the bill provides for 

those exceptions. 

Creating an Evaluation Mindset 

The Problem 

Ex posl evaluation has a long tradition in other areas (particularly in programs financed through 

the fiscal budget), 18 but it has received little attention in the regulatory arena, despite government 

17 West (2004 ). 

18 Susan Dudley and Brian Mannix, ''Improving Regulatory Bonolit-Cost Analysis:· The Journal of Law and 

Politics, Vol. XXXIV, No.I (2018). 

The George Washington University Regulatory Studies Center 5 
33 

guidelines requiring retrospective review. 19 RIAs are an important part of the regulatory process, 

but as ex ante analyses, they arc necessarily hypotheses of the effects regulatory actions will 

have if implemented. Better regulatory evaluation would allow agencies and others to test those 

hypotheses against actual outcomes.20 Retrospective review would not only inform decisions 

related to the benefits and costs of existing policy but would provide feedback that would 

improve future R!As and future policies.21 

Setting Manageable Analysis Requirements in Text (SMART) 

While no one questions the importance of evaluation, a lack of methods and data make 

retrospective review of regulations challenging." The draft SMART bill addresses that problem 

at the outset of rulemaking by requiring agencies to include in major regulations a framework for 

how they will measure effectiveness, benefits, and costs, and to incorporate plans for gathering 

the information necessary for ex post evaluation. It would also require agencies, within I 0 years 

of a rule's effective date, to assess its benefits and costs, evaluate how well it accomplishes its 

objectives, and determine whether it could be modified to achieve better outcomes. 

This would fill an important gap in current regulatory practice.23 The GW Regulatory Studies 

Center reviewed all major rules proposed in 2014 and found that, despite President Obama's 

requirements to do so, none of them included a plan for retrospective review, and not one was 

written and designed to facilitate review of its impacts.24 While we have not conducted a 

similarly comprehensive review since that year, case-by-case analysis suggests that most 

regulations continue to be issued without any plan for review. 

The bill would ensure not only that existing major regulations are being evaluated, but that new 

major rules are designed to facilitate such evaluation in the future. It focuses not just on reducing 

regulatory burdens, but on improving regulatory outcomes by subjecting regulatory programs to 

rigorous evaluation and feedback. Institutionalizing a requirement to evaluate whether the 

predicted effects of the regulation were realized would provide a powerful incentive to improve 

the ex-ante RIA tools used to predict the impacts of regulatory alternatives 25 The bill would 

19 Joseph Aldy, !.earning from Experience: An Assessment oft he Retrospective Reviews of Agencv Rules and the 

Evidence for Improving the Design and Impkmcntation of Regulatory Policv, ADMJN. CONF. U.S. (Nov. 18, 

20 14). Aldy writes that federal regulatory agencies have a mixed record on ex post review, despite their "long 

track record of prospective analysis of proposed regulations that can address these questions.'' 20 Susan E. Dudley. ·'Retrospective Evaluation of Chemical Regulations," Organisation tor Economic Co-operation 

and Development Environment Working Papers, No. 118. (2017). 

21 Michael Greenstone, "Toward a Culture of Persistent Regulatory Experimentation and Evaluation." in New 

Perspectives on Regulation. David Moss and John Cisternino ed. The Tobin Project. (2009). 22 Dudley (2017). 23 Peacock et al. (20 18). 24 Sofie E. Miller. "Evaluating Retrospective Review of Regulations in 20 14,'' the George Washington University 

Regulatory Studies Center. (Nov. 4, 2015). 25 Dudley and Peacock (20 18). 

The George Washington University Regulatory Studies Center 6 
34 

create an evaluation mindset and a feedback mechanism where agencies learn from evaluating 

regulatory outcomes and apply those lessons to improve future rules. 

Other Observations 

These two draft bills offer relatively modest, yet potentially powerful, changes to current 

rulemaking practices. If enacted, they could make regulatory decisions more transparent and 

accountable, leading to improved regulatory outcomes for the American people. My testimony 

concludes with observations on features common to both bills. 

The definition of"major rule" in both bills appropriately captures what are likely to be the most 

significant regulatory actions, while not unduly burdening agencies with additional procedures 

for all their rules. It maintains the annual $100 million annual impact trigger embodied in the 

CRA and E.O. 12866 (Sec. 4(f)(i)), but it is not purely a monetary test. It recognizes that rules 

that are likely to significantly affect consumer prices, competition, productivity, innovation. the 

environment, public health, or safety deserve greater public engagement and ex post evaluation. 

According to records kept by the Government Accountability Office (GAO), agencies issue 

approximately 3,000 regulations each year. Of those, GAO classifies around 900 as "substantive 

or significant."26 An average of70 of those meet the CRA definition of major, which closely 

resembles the bills' definition of major rule. This likely overstates the number of regulations that 

would be subject to these bills' requirements since many of them are routine (such as annual 

hunting and fishing limits) or affect annually recurring monetary transfers from taxpayers to 

program recipients (for example, Medicaid and Medicare payment rules). For these, the 

requirement to issue ANPRMs and develop a retrospective review framework could either be 

streamlined or they might qualify for an exemption. 

Accomplishing the important goals of these bills would require resources. As noted above with 

respect to timing, the ANPRM requirement need not impose additional resource costs to the 

extent it engages public participation in preliminary deliberations that have traditionally been 

closed. To support more rigorous retrospective review, Congress and OMB could reallocate 

resources from ex ante analysis to allow agencies to gather the information and evaluation tools 

necessary to validate ex ante predications. In the long run, shifting resources from ex ante 

analysis to ex post review would not only help with evaluation, but could improve agencies' ex 

ante hypotheses of regulatory effects.27 

" The OIRA/GSA Reglnlo.gov database classilics less than 300 rules as significant on average each year. Fewer 

than 50 of those would meet the bills' detinition of major. See GW Regulatory Studies Center ''Reg Stats" tor 

more detail. 

" Dudley (20 17). 

The George Washington University Regulatory Studies Center 7 
35 

Both bills would make OIRA responsible for overseeing compliance, which is appropriate. 

Executive branch oversight of regulatory actions has proven valuable, but it is not sufficient. 

Congress may also want to assign a congressional body, perhaps a new regulatory office in the 

Congressional Budget Office (CBO), responsibility for reviewing these assessments. Just as the 

CBO provides independent estimates of the on-budget costs of legislation and federal programs, 

a congressional regulatory otlice could provide Congress and the public feedback on legislation 

that enables regulation, as well as serve as an independent check on the analysis and decisions of 

regulatory agencies and OIRA.28 

28 Susan Dudley, "Improving Regulruorv Accountability: Lessons tram the Past and Prospects for the Future." 

Case Western Reserve Law Review, Vol. 65 Issue4. (2015). 

The George Wa<>hington University Regulatory Studies Center 8 
36 

Statement of Sally Katzen 

Professor, NYU School of Law 

before the 

Senate Homeland Security and Governmental Affairs Committee 

Subcommittee on Regulatory Affairs and Federal Management 

on 

"From Beginning to End: An Examination of Agencies Early Public Engagement and 

Retrospective Review" 

May 7, 2019 
37 

Chairman Lankford, Ranking Member Sinema, Members of the Subcommittee. 

I understand that this hearing is intended to explore specific areas of the process for 

federal rulemaking that would likely garner bipartisan support. To that end, I am 

pleased to participate and thank you for inviting me to testify today. 

I have worked on regulatory issues during most of my career in private practice, 

government service, and teaching and writing. I served as the Administrator of the 

Office of Information and Regulatory Affairs (OIRA) at the Office of Management and 

Budget (OMS) for the first five years of the Clinton Administration, then as the Deputy 

Assistant to the President for Economic Policy and Deputy Director of the National 

Economic Council, and then as the Deputy Director for Management of OMS. Before 

entering government service in 1993, I was a partner at the law firm Wilmer, Cutler & 

Pickering (now WilmerHale), specializing in regulatory and legislative issues, and among 

other professional activities, I served as the Chair of the American Bar Association 

Section on Administrative Law and Regulatory Practice (1988-89). During my 

government service, I was the Vice Chair (and Acting Chair) ofthe Administrative 

Conference of the United States (ACUS). After leaving the government in January 2001, 

I have been teaching courses in administrative law at various law schools, and since 

20111 have been at New York University School of Law, where I am currently a 

Professor of Practice and Distinguished Scholar in Resident. 

As this Committee well knows, the regulatory system -and the rules that it 

develops, promulgates and enforces- is an integral component of governance. 

Congress makes the law but it typically does not have the time, the expertise, or 

sometimes the ability to identify and resolve all the details. That responsibility is usually 

delegated to the agencies that are expected to issue regulations that translate general 

statutory directives into concrete requirements or prohibitions with which the public 

must comply. There are appreciably more regulations than statutes: some have 

depicted it as a pyramid, with the Constitution, the supreme law ofthe land on top, 

hundreds of statutes enacted by Congress on the next level, and then thousands of 

regulations issued by the agencies. 

Apart from the delegations from Congress- which provide the primary direction 

and constraint on an agency's substantive and procedural authority- the principal law 

that governs the development and promulgation of regulations is the Administrative 

Procedure Act ("APA"). It was enacted in 1946 and, with relatively few amendmentsmostly having to do with the Freedom of Information Act- and with a series of federal 

2 
38 

court cases fleshing out the general terms of the Act, it has generally withstood the test 

oftime. 

Most of the criticism and praise of the administrative state is focused on 

regulations produced by notice-and-comment rulemaking- the subject of Section 553 

ofthe APA. While it is commonly referred to as "informal rulemaking," there is nothing 

informal about the process. It is resource intensive and time consuming- some 

rulemakings take years rather than months to go from concept to a final rule, plus 

whatever additional time and effort goes into judicial review. 

The current process has its critics, from both conservatives and progressives. 

That suggests that perhaps the process is just about right. Conservatives are concerned 

that there are not sufficient checks along the way; progressives are concerned that the 

number of checkpoints has created ossification. In my view, process is good, but too 

much process can be counterproductive. The issuing agency should think, research, 

consult, analyze, question and continually refine. The public, both those who would 

bear the costs or burden of the regulation and those that would benefit from it, should 

be informed of what the agency is thinking and why, and be engaged in supplying and 

reviewing the information the agency is relying on and in critiquing the options the 

agency is considering. The public has a great deal to contribute, but the process should 

not be so extended as to unduly delay or disrupt the work of the agencies. And, most 

importantly, each step in the process (and any new steps imposed) should be evaluated 

in terms of its contribution to good decision-making, to what would help produce the 

most sensible, effective and efficient way forward. 

I understand that this Committee is focusing today on two specific aspects of the 

rulemaking process: the beginning and the end. I think that each of these pieces can be 

improved, whether by legislation, executive order, OMB guidance or simply agency 

practices. Regardless of the vehicle, however, it is important to be clear about what the 

problem is and how best to solve that problem without introducing unintended 

consequences. 

The Beginning: an Advanced Notice of Proposed Rulemaking. 

The first official step in a rulemaking proceeding is the issuance of a notice of 

proposed rulemaking ("NPRM"). Lawyers in practice and in the academy generally 

agree that by the time the agency issues the NPRM, the staff involved have invested so 

much time and energy in developing the proposal and supporting data (as they are 

required to do) and analyzing the likely effects of the proposal (as they are required to 

do) and justifying their proposal (as they are often called upon to do by their agency 

3 
39 

decision-makers and OIRA review) that they are virtually locked into their proposal and 

are less receptive to new ideas (or even significant modifications of their proposal). 

While all the up-front work (and documentation) is desirable, it often has the 

unintended result of restricting the options going forward. 

To counter this tendency, there have been various efforts to encourage the 

agencies to consult with the public even before they have essentially made their 

decisions reflected in the NPRM. This has been true of both Democratic and Republican 

Administrations. For example, Executive Order 12866 clearly states that "before issuing 

a notice of proposed rulemaking, each agency should ... (consistent with its own rules] 

seek the involvement of those who are intended to benefit from and those expected to 

be burdened by any regulation (including, specifically, State, local, and tribal officials)." 

Section 6(a) emphasis added. And Executive Order 13563 expanded on this concept 

with an entire section devoted to "Public Participation." See Section 2. Among other 

things, it states in subsection (a) that "regulations shall be based, to the extent feasible 

and consistent with law, on the open exchange of information and perspectives among 

State, local, and tribal officials, experts in relevant disciplines, affected stakeholders in 

the private sector, and the public as a whole." And it specifically provides in subsection 

(c) that "[b]efore issuing a notice of proposed rulemaking, each agency, where feasible 

and appropriate, shall seek the views of those who are likely to be affected, including 

those who are likely to benefit from and those who are potentially subject to such 

rulemaking." Section 2, emphasis added. 

Several agencies use an Advance Notice of Proposed Rulemaking (ANPRM) to 

solicit ideas at the outset of a rulemaking proceeding. It is especially useful when the 

agency is unsure what direction to take, what data to consider, how prescriptive to be, 

and the like. It is also more useful when it is done early in the process or even at the 

outset of agency deliberations (for example, when the agency first sends notice of its 

undertaking a rulemaking for inclusion in the Unified Agenda). It is less useful when the 

authorizing statute is itself prescriptive or if there is general agreement about the 

nature of the regulation necessary to respond to the identified problem. In short, it can 

be helpful at times; at other times, it may just add an unproductive, but time 

consuming, step to the already extended process. 

For this reason, it is important that any requirement for an ANPRM should be 

limited to economically significant (or "major") regulations that are required to use 

notice and comment under Section 553. Similarly, it is important that any such 

provision should not impose on the agency multiple requirements for explanations, 

analysis, data, etc. The purpose would be to alert those affected by the regulation, so 

4 
40 

they can contribute to its development and formulation in advance of the NPRM, not to 

lock the agency into a particular mind-set before the process even begins. The more the 

agency has to incorporate in an ANPRM, the more the agency will become invested in a 

particular outcome. This is the opposite of what an ANPRM should do- namely, obtain 

ideas and information from interested entities before the agency settles on a particular 

course. 

I understand that your and your staffs have been working on a draft bill that 

reflects these considerations and takes a sensible (but appropriately limited) step 

towards expanding the opportunity for public participation at the pre-NPRM stage. If 

the product of those efforts adheres to the principles (and specific provisions) we have 

been discussing, I would be supportive of the effort. 

The End: Retrospective Review 

For almost 40 years, there have been concerns that there are too many rules, 

and that so many of the rules on the books are obsolete, unnecessarily burdensome, 

unworkable, or just plain wrong. This was one of the themes President Reagan 

campaigned on, and, after his election, he set on a course to deregulate. President 

George H.W. Bush followed the same path with his Competitiveness Council, searching 

the existing stock of regulations for those that could be eliminated. President Clinton 

ordered agencies "to submit to OIRA a program ... under which [they] will periodically 

review existing significant regulations to determine whether any such regulations should 

be modified or eliminated .... " E.O. 12866, Section 5. President George W. Bush 

launched a similar effort. President Obama also emphasized the need for retrospective 

review of rules in his Executive Order 13563. And President Trump's "two-for-one" 

Executive Order is designed in part to accomplish the same objective- weed out 

unnecessary, out-of-date, ineffective rules. 

Despite these efforts, it doesn't happen. One reason may be that since 1980, 

new regulations are not issued unless their benefits justify their costs; to eliminate such 

a regulation would likely mean that the costs of rescinding the regulation would be 

greater than the benefits. [Note: This is so because removing a rule means that the 

foregone benefits ofthe existing rule become the costs ofthe new rule, and the 

foregone costs of the existing rule become the benefits of the new rule). Other reasons 

for the limited success of these efforts are that agencies have not undertaken to collect 

data along the way that would inform their retrospective reviews and, importantly, any 

retrospective analysis requires resources and, for at least the last few decades, 

5 
41 

regulatory agency budgets have generally been decreasing or straight-lined, with the 

situation compounded by continuing resolutions and sequestration. 

Nonetheless, there is growing support for one step that can be taken -namely, 

encouraging agencies to plan for retrospective review when they are in the process of 

developing a final rule. This idea came from a study by Joseph Aldy for the 

Administrative Conference of the United States (ACUS), Learning from Experience: An 

Assessment of Retrospective Reviews of Agency Rules & the Evidence for Improving the 

Design and Implementation of Regulatory Policy (Nov. 2014). This led to a series of 

ACUS recommendations designed to promote "a culture of retrospective review at 

agencies," which stressed the need to carefully select regulations for reevaluation and 

to coordinate with OIRA, other agencies and outside entities (including stakeholders) 

when designing and conducting retrospective reviews." For present purposes, it is 

instructive that, among other things, ACUS adopted a specific recommendation for 

"agencies to plan for retrospective review when drafting new regulations." A similar 

recommendation was part of a report to the 2016 transition teams developed by the 

Institute of Policy Integrity after consultation with almost all ofthe past OIRA 

Administrators. Strengthening Regulatory Review (2016). 

Requiring agencies to provide, along with the NPRM and the final rule, a plan for 

a later retrospective review of a newly issued regulation would in most circumstances 

be salutary. If nothing else it would force agency personnel to focus on describing 

precisely what they want to accomplish and how to evaluate whether or not the rule is 

successful in achieving that objective, at a time when the rule (and all its alternatives) is 

foremost in their minds. It also would enable those affected by the rule to participate in 

the framing of the subsequent retrospective review while they too are keenly focused 

on the provisions of the proposed rule. 

It is very important, however, to provide flexibility for the eventual 

implementation ofthe retrospective review. The agency can (and should) commit to a 

framework in the proposed and final rule, identifying the data and the metrics it 

anticipates using for that purpose. But this should not be cast in concrete. We learn a 

lot with time, including how to better analyze and measure what is going on around us. 

This is amply demonstrated by the increased sophistication of cost-benefit analysis itself 

over the last decade or two. It is also demonstrated by the general preference for 

performance standards, which specify the desired results, rather than design standards, 

which lock in a particular way to achieve those results. 

In addition, while periodic review is useful, there will likely be some (or many) 

situations where repeated retrospective reviews will yield greatly diminishing returns. 

6 
42 

After a decade or so, rules that survive a retrospective review intact are likely to have 

established their worth, and it would be wasteful to continue retrospective review after 

retrospective review. In this connection, Section 553(e) of the APA provides for 

petitions for rulemaking that can be used if, at some later point in the future, a 

consensus develops that a particular rule should be modified or rescinded. This would 

place some responsibility on the regulated entities, but that is appropriate because they 

are in the best position to identify (and document) rules that have outlived their 

usefulness. 

As above, I understand that you and your staffs have been working on a draft bill 

that would implement these recommendations in a straightforward, targeted way. 

Again, if the product of those efforts reflects the principles (and specific provisions) we 

have been discussing, I believe such a bill would be a constructive addition to the 

rulemaking process. 

Again, I appreciate this Committee's efforts to fine tune critical steps rather than 

redesign the whole rulemaking process. I emphasize this because, unfortunately, you 

are not writing on a blank slate. For the past three decades, there have been concerted 

efforts underway to revise and revamp the rulemaking process that have mostly 

proceeded on a highly partisan path. Not only have they not been successful in terms of 

being enacted into law, but they have also sown suspicion and distrust. As a result, 

bipartisan efforts on regulatory reform have been difficult to achieve. The limited, 

surgical approach that you are considering will likely face an uphill battle, but it might 

well have a better chance to succeed. 

Thank you again for giving me an opportunity to speak to these issues. I look 

forward to any comments or questions you may have. 

7 
43 

Susan Dudley Responses to 

Post-Hearing Questions for the Record 

"From Beginning to End: An Examination of Agencies' Early Public Engagement and 

Retrospective Review" 

From Senator Thomas R. Carper 

For the Honorable Susan Dudley and for the Honorable Sally Katzen 

OIRA's role in reviewing agency rulemaking and EPA's "Strengthening Transparency in 

Regulatory Science" 

On May 9, 2018 I joined Senator Hassan and several other Senators in a letter to then-OIRA 

Administrator Neomi Rao raising concerns and seeking answers as to how OIRA reviewed an 

EPA proposed rulemaking entitled: "Strengthening Transparency in Regulatory Science." 

According to Reginfo.gov, OIRA received the EPA's proposed rule on Thursday, April19, 2018. 

Then-EPA Administrator Scott Pruitt publicly announced this proposed rule and signed it three 

business days later on Tuesday, April24, 2018. Reginfo.gov initially stated that OIRA's review 

was completed on Wednesday, April 25, 2018 -the day after Administrator Pruitt announced 

and signed the rule. Later, following press inquiries, Reginfo.gov was changed to indicate that 

OIRA's review was completed on Monday, April23, 2018. 

Our letter sought detailed information on the scope and substance ofOIRA's review ofthis 

proposed ru1emaking. However, in a response to Senator Hassan on July 12, 2018, former 

Administrator Rao provided little in way of response and simply noted that the conclusion date 

for the OIRA review of this proposed rule of April 25, 2018 was due to a clerical error that was 

subsequently corrected. 

With this in mind, I ask that you review and respond to the following: 

1. Please comment on the general process by which OIRA would employ when reviewing a 

complex and far-ranging agency proposal such as this. 

OIRAfacilitates interagency review of agencies' draft proposed andfinal regulations. While this 

often occurs after the responsible agency has submitted the draft formally to OIRA, in many 

cases, that formal submission is preceded by substantial interagency discussion and planning. 

2. During your tenure as Administrator, how long, on average, would OIRA take to review 

complex and far-ranging agency proposals akin to the above mentioned EPA proposal? 

a. Similarly, do you ever recall during your time as OIRA Administrator where 

OIRA signed off on an agency rulemaking decision within 3 business days of 

receiving a rule? 

During my tenure, the average formal review time for proposed economically-significant EPA 

rules was 31 days (Reglnfo.gov). The shortest review was 0 days (lead NAAQS, subject to a 

Page 1 of2 
44 

judicial deadline), and the longest was 122 days (Revisions to the Spill Prevention, Control, and 

Countermeasure rule). 

3. If then-Administrator Pruitt did indeed sign off on the proposed rule prior to OIRA 

completing its review, would that violate sections 7 and 8 of Executive Order 12866? 

Without knowledge of the discussions between EPA and the Executive Office of the President, 

it's hard to know. In my experience, decisions on roll out and timing are context-specific, which 

makes commenting on a hypothetical difficult. 

a. What processes are generally in place to prevent agencies from proceeding with 

rules without OIRA approval? 

Communications and repeat interactions between agencies. OJRA, and other parts of the White 

House generally ensure agencies do not issue rules before review has concluded 

4. Do you believe agencies should use best available evidence in the rulemaking process? 

Yes. See Susan E. Dudley and Marcus Peacock, "Improving Regulatory Science: A Case Study 

of the National Ambient Air Quality Standards," Supreme Court Economic Review 24 (2016): 

49-99. 

5. Similarly, do you think it is appropriate to restrict the scientific evidence that agencies 

can consider during rulemaking? 

Agencies should rely on well-supported scientific evidence and they should be transparent 

regarding the models, data, studies, assumptions, and judgments they relied on to translate 

scientific inputs for use in regulatory analysis. 

My comments submitted on the record of EPA 's proposed "Strengthening Transparency in 

Regulatory Science" offer more detail. They are attached and available on the GW Regulatory 

Studies Center website. 

Page 2 of2 
45 

THE GEORGE WASHINGTON UNIVERSITY 

WASHINGTON, DC 

Public Interest Conunent1 on 

The Environmental Protection Agency's Proposed Rule 

Strengthening Transparency in Regulatory Science 

Docket ID No. EPA-HQ-OA-2018-0259; FRL-9977-40-0RD 

RIN: 2080-AA14 

May 18,2018 

Susan E. Dudley, Directoil 

The George Washington University Regulatory Studies Center 

The George Washington University Regulatory Studies Center improves regulatory policy 

through research, education, and outreach. As part of its mission, the Center conducts careful and 

independent analyses to assess rulemaking proposals from the perspective of the public interest. 

This conunent on the Environmental Protection Agency's proposed rule to strengthen the 

transparency of its regulatory science does not represent the views of any particular affected 

party or special interest, but is designed to evaluate the effect of the proposal on the reliability of 

the scientific information underlying EPA's regulatory decisions. 

Introduction 

In this proposal, EPA aims to strengthen the transparency of the science it considers "pivotal" to 

its significant regulatory actions by ensuring that "the data and models underlying the science is 

publicly available in a manner sufficient for validation and analysis." It cites existing authorities 

and policies, but acknowledges, "EPA has not previously implemented these policies and 

guidance in a robust and consistent manner." 

1 This comment reflects the views of the author and does not represent an official position of the GW Regulatory 

Studies Center or the George Washington University. The Center's policy on research integrity is 

http://regulatorystudies.columbian.gwu.edu/policy-research-integrity. 2 Director, the George Washington University Regulatory Studies Center and Distinguished Professor of Practice 

in the Trachtenberg School of Public Policy and Public Administration. 

The George Washington University Regulatory Studies Center • 1 
46 

The rule would not directly regulate non-governmental entities, but instead would require "EPA 

to ensure that the regulatory science underlying its actions is publicly available in a manner 

sufficient for independent validation.''3 The preamble says the policy is "designed to provide a 

mechanism to increase access to dose response data and models underlying pivotal regulatory 

science in a manner consistent with statutory requirements for protection of privacy and 

confidentiality of research participants, protection of proprietary data and confidential business 

information, and other compelling interests."4 In the long run, through this rule, EPA aims "to 

change agency culture and practices regarding data access so that the scientific justification for 

regulatory actions is truly available for validation and analysis. " 5 

The proposal defines "dose response data and models" as those "used to characterize the 

quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or 

substance and the magnitude of a predicted health or environmental impact. "6 "Pivotal 

regulatory science " refers to "specific scientific studies or analyses that drive the requirements 

and/or quantitative analysis of EPA final significant regulatory decisions."7 "Regulatory 

decisions" are limited to "significant regulatory actions" as defmed in Executive Order 12866.8 

"Regulatory science means scientific information, including assessments, models, criteria 

documents, and regulatory impact analyses, that provide the basis for EPA final significant 

regulatory decisions," and research data is used as "defined in Uniform Administrative 

Requirements, Cost Principles, and Audit Requirements for Federal Awards. "9 

The rule would require EPA to: 

1. Clearly identifY all regulatory science it relied on in selecting a regulatory action and 

make those studies available to the public "to the extent practicable."10 

2. Consistent with laws and sensitive to privacy, confidentiality, and national and homeland 

security, "ensure that dose response data and models underlying pivotal regulatory 

science arc publicly available in a marmer sufficient for independent validation." 11 

3. "Describe and document any assumptions and methods used," including the scientific 

basis for those assumptions, as well as analysis "showing the sensitivity of the modeled 

results to alternative assumptions." 12 

3 Proposed §30.1. 83 FR 18773 4 83 FR 18770 5 83 FR 18770 6 Proposed §30.2. 83 FR 18773 7 Proposed §30.2. 83 FR 18773 8 Proposed §30.2. 83 FR 18773 9 Proposed §30.2. 83 FR 18773 10 Proposed §30.4. 83 FR 18773 11 Proposed §30.5. 83 FR 18773 12 Proposed §30.6. 83 FR 18774 

The George Washington University Regulatory Studies Center + 2 
47 

4. "Give explicit consideration to high quality studies" that may challenge existing default 

assumptions including "parametric dose-response or concentration-response models; a 

robust set of potential confounding variables; nonparametric models that incorporate 

fewer assumptions; various threshold models across the dose or exposure range; and 

models that investigate factors that might account for spatial heterogeneity." 13 

5. "Conduct independent peer review on all pivotal regulatory science used to justify 

regulatory decisions, " consistent with existing peer review requirements and 

exemptions. 14 

It is unusual for an agency to provide guidelines such as these through a rulemaking rather than 

internal guidance; however, the transparency of this rulemaking approach is consistent with the 

goals of the proposal, and the robustness and legitimacy of any final po !icy will be enhanced if it 

is supported by the record the agency develops as it solicits public comment. 15 A 30-day 

comment period may not provide enough time for constructive input on key issues, however, and 

a May 12, 2018 memorandum from a working group of EPA's Science Advisory Board (SAB) 

argues that this "action merits further review by the SAB." 16 

EPA also seeks comment on its legal authority for the rulemaking, and the appropriateness of the 

proposed policies to regulatory decisions developed pursuant to its various statutory mandates. 

This is important, but this comment will leave those issues to others and focus on the intrinsic 

merits of the proposal. 

Merits of the Proposed Rule 

1) EPA will clearly identify and make publicly available the studies and 

science relied on for significant regulatory actions. 

When the Office of Information and Regulatory Affairs (OIRA) tallies up the total benefits and 

costs of all federal regulations each year, EPA's regulations always comprise the bulk of those 

figures. In OIRA's most recent draft report, for example, EPA's estimated annual benefits of the 

regulations it issued between fiscal years 2007 and 2017 ranged from $196 billion to $707 

13 Proposed §30.6. 83 FR 18774 14 Proposed §30.7. 83 FR 18774 15 Establishing agency administrative practices and procedures via rulemaking is not unprecedented. For example, 

without legislative mandate, the U.S. Food and Drug Administration used informal ru!emaking to codifY its Good 

Guidance Practices. 65 FR 56477, September 19, 2000. 16 Alison Cullen, Chair, SAB Work Group on EPA Planned Actions for SAB Consideration of the Underlying 

Science, Memorandum to Members of the Chartered SAB and SAB Liaisons "Preparations for Chartered Science 

Advisory Board (SAB) Discussions of Proposed Rule: Strengthening Transparency in Regulatory Science RIN 

(2080-AAI4)" May 12,2018. 

https:llyosemite.epa.govl sabl sabproduct.nsf!E2l FF AE956B548258525828C00808BB71$File/WkGrp _memo_ 20 

80-AAI4_final_ 05132018.pdf 

The George Washington University Regulatory Studies Center • 3 
48 

billion, constituting 68% to 78% of all the regulatory benefits agencies estimated during that 10-

year window. OIRA reported corresponding cost estimates for those EPA rules of$54 billion to 

$65 billion per year, which comprise between 57% and 69% of all the rules OIRA reviewed 

during that period. 17 

Given the significance of these estimates, documenting and making available for public review 

the underlying science supporting them is essential. EPA estimates that its National Ambient Air 

Quality Standards (NAAQS), for example, are among the most beneficial regulations issued. In 

setting NAAQS, EPA is statutorily proscribed from considering the costs of meeting standards, 

but it must evaluate available data on health and welfare impacts in presenting alternatives for 

the administrator to consider. EPA's formulation and presentation of the studies and data 

necessarily involve judgments about which studies to consider and which to exclude, but these 

decisions and their rationale are often not transparent. 18 Increasing transparency regarding which 

science it considered; how it weighted and combined individual studies; what competing theories 

were included, etc. would allow broader review and analysis, and improve the rigor of regulatory 

decisions. 19 

EPA's proposal to clearly identify available studies and make them available for public review is 

not only important for ensuring decisions are supported by the best information, but also 

consistent with policies on scientific integrity espoused by previous administrations. OMB's 

2002 Information Quality Guidelines directed agencies to make publicly available "peerreviewed studies known to the [agency] that support, are directly relevant to, or tail to support 

any estimate of [risk] effects and the methodology used to reconcile inconsistencies in the 

scientific data."20 In 2009, President Obama issued a memorandum to agencies that encouraged 

"transparency in the preparation, identification, and use of scientific and technological 

information in policymaking," and affirmed that "scientific and technological information 

should ordinarily be made available to the public.'m 

17 Office oflnformation and Regulatory Affilirs, "20 17 Draft Report to Congress on the Benefits and Costs of 

Federal Regulations and Agency Compliance with the Unfunded Mandates Reform Act." February 23, 2018. 

https://www. whitehouse.gov/wp-content/uploads/20 17!12/drafi_ 2017 _cost_ benefit _report. pdf 18 Susan Dudley and Marcus Peacock. "Improving Regulatory Science: A Case Study of the National Ambient Air 

Quality Standards," with Marcus Peacock. Supreme Court Economic Review. (forthcoming) Working paper 

h!tDs://regulatorystudies.columbian.gwu.edu/irnoroving-regulatorv-science-case-study-national-ambient-airgualitv-standards 2017. 19 Susan Dudley and George Gray. "Improving the Use of Science to Inform Environmental Regulation," in 

Institutions and Incentives in Regulatory Science, Lexington Books. Jason Johnston ed. (2012) 20 Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information 

Disseminated by Federal Agencies (2002), 67 Fed. Reg. 8452 (Feb. 22, 2002). 

https://www.whitehouse.gov/sites/whitehouse.gov/files/omblassets/OMB/inforeg/iqg__oct2002.pdf 21 Barrack Obama, "Memorandum for the Heads of Executive Department and Agencies on Scientific Integrity." 

Mar. 9, 2009. https://obamawhitehouse.archives.gov/the-press-office/memorandum-heads-executivedepartments-and-agencies-3-9-09 

The George Washington University Regulatory Studies Center • 4 
49 

2) EPA will make dose response data and models underlying pivotal 

regulatory science publicly available for independent validation. 

The selection of the model used to estimate responses to different exposures to contaminants can 

have significant impacts on estimated regulatory benefits. In 2007, OIRA and the Office of 

Science and Technology Policy (OSTP) observed in a memorandum to agency heads on risk 

assessment that a "high degree of transparency with respect to data, assumptions, and methods 

will increase the credibility of the risk analysis, and will allow interested individuals, internal and 

external to the agency, to understand better the technical basis of the analysis. "22 

In 2010, OSTP directed agencies to develop policies to "facilitate the free flow of scientific and 

technical information, consistent with privacy and classification standards." In a memorandum to 

department and agency heads, President Obama's science advisor John Holdren stated: 

Open communication among scientists and engineers, and between these experts 

and the public, accelerates scientific and technological advancement, strengthens 

the economy, educates the Nation, and enhances democracy. Consistent with the 

Administration's Open Government Initiative, agencies should expand and 

promote access to scientific and technological information by making it available 

online in open formats. Where appropriate, this should include data and models 

underlying regulatory proposals and policy decisions. 23 

In a 2013 memorandum, OSTP directed all Executive Departments with greater than $100 

million in yearly research and development expenditures to prepare plans for improving the 

public's access to the results of that research. EPA did not publish a plan to comply with the 

OSTP directive until November 29, 2016, years after many other agencies had begun to 

implement their plans.24.25 

EPA's proposal to make the data and models underlying its pivotal regulatory science pubic is 

also consistent with developments in scholarly journals. In recent years, scientific publishing has 

22 Susan Dudley and Sharon Hays, "Memorandum for the Heads of Executive Department and Agencies: Updated 

Principles for Risk Analysis." September 19,2007. 

https://www. whitehouse. gov/siteslwhitehouse. gov/fi1es/ omb/memoranda/2007/m07-24.pdf 23 John P. Holdren. "Scientific Integrity," OSTP Director Memorandum for the Heads of Executive Departments 

and Agencies. December 17,2010. 

https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/scientific-integrity-memo-121720 I O.pdf 24 Environmental Protection Agency, Plan to Increase Access to Results of EPA-Funded Scientific Research. 

November 29, 2016. https://www.epa.gov/sites/production/files/2016-

12/ documentslepascientificresearchtransperancyplan.pdf 25 Federal STI Managers Group (CENDI), "Implementation ofPub1ic Access Programs in Federal Agencies," 

https://cendi.gov/projects/Public _Access _Plans .us _Fed_ Agencies.htm1 

The George Washington University Regulatory Studies Center + 5 
50 

focused more on the sharing of data and experimental transparency26 Indeed, disclosure of 

underlying data and computer code has become standard among the more prestigious scientific 

and technical journals, which allow for data sharing agreements when personally identifiable 

information prevents public disclosure. These disclosure policies appear to improve the 

reproducibility of the results of published papers. 27 

In 2013, for example, the journal Nature took steps to ensure it reported "key methodological 

details and encourage[d] authors to be transparent by including the raw data used in their 

studies." While recognizing that experimental studies vary, the editors concluded that variation 

does not preclude "a full report of how a study was designed, conducted and analysed that will 

allow reviewers and readers to adequately interpret and build on the results. "28 

The journal Science has also undertaken "initiatives to increase transparency and promote 

reproducibility in the published research literature ... Connected to that progress, and an essential 

element to its success, an additional focus will be on making data more open, easier to access, 

more discoverable, and more thoroughly documented. "29 

EPA's proposal states that it would consider information to be "'publicly available in a manner 

sufficient for independent validation' when it includes the information necessary for the public to 

understand, assess, and replicate findings." This emphasis on replicability can encourage 

challenge and validation that is so important to the scientific method. 30 It is consistent with 

OMB's 2002 Information Quality Guidelines, which require that significant information 

disseminated to the public be "capable of being substantially reproduced, subject to an 

acceptable degree of imprecision. " 31 

As the Science editors observe, "When the greatest number of creative and insightful minds can 

find, access, and understand the essential features that led to the collection of a data set, the data 

reach their highest potential. "32 

26 Joel Achenbach, "The new scientific revolution: Reproducibility at last." Washington Post. January 27,2015. 27 Lutter, Randall and Zorn, David (2016). Reinforcing Reproducibility: What Role for the Federal Government? 

Regulation, Winter 2015-16, 15-16. 

https:/ /object.cato.org/sites/cato.org/files/serials/files/regulation/20 15/ 12/regulation-v38n4-8 _ 4.pdfllpage=' I 0. 28 Editors, "Reducing our Irreproducibility," Nature. Vol 496, p. 398. April25, 2013. 29 Science 2 January 2015: Vol. 347 no. 6217 p. 7 30 Dudley and Peacock, 2017. 31 OMB 2002, p. 8460 32 Science, January 2015. 

The George Washington University Regulatory Studies Center • 6 
51 

3) EPA will describe and document its assumptions and methods and show 

how sensitive modeled results are to those and alternative assumptions. 

This requirement comports with recommendations from various sources. For example, in a 

recent article, 19 regulatory analysis experts warned: 

Analyses that do not provide information on how sensitive the primary estimate is 

to assumptions, data, and models, and the range of outcomes possible under 

reasonable alternative analytic assumptions should raise questions. Sensitivity 

analysis examines different "what if' scenarios to see how changes in key 

assumptions (or combinations of assumptions) influence estimated outcomes. 

Because many uncertain factors determine the impact of any regulation, one 

should look for a convincing justification regarding which uncertain parameters 

have the most consequential effects on outcomes, and a sensitivity analysis that 

varies these factors over a reasonable range to gauge their effects on the rule's net 

benefits. 33 

In 2010, OSTP directed agencies to communicate scientific and technological findings to the 

public "by including a clear explication of underlying assumptions; accurate contextualization of 

uncertainties, and a description of the probabilities associated with both optimistic and 

pessimistic projections, including best-case and worst-case scenarios where appropriate. "34 

This is important, because, as Dudley and Peacock explain, "scientists will never have complete 

information to predict outcomes with certainty, so analysts rely on what the [National Research 

Council]35 calls 'risk assessment policy' -assumptions, judgments, and rules of thumb--to guide 

the use of scientific information in analyses that inform policy in the face ofuncertainty."36 The 

Institute of Medicine observed in 2013: 

Uncertainty is inherent in the scientific information upon which health risk 

estimates are based. Uncertainties enter the health risk assessment process at 

33 Dudley, S., Belzer, R., Blomquist, G., Brennan, T., Carrigan, C., Cordes, J., Cox, L.A., Fraas, A., Graham, J., 

Gray, G., Hammitt, J., Krutilla, K., Linquiti, P., Lutter, R., Mannix, B., Shapiro, S., Smith, A., Viscusi, W.K., 

Zerbe, R. (2017). Consumer's Guide to Regulatory Impact Analysis: Ten Tips for Being an Informed 

Policymaker. Journal of Benefit-Cost Analysis. 8(2), 187-204. doi:IO.l017/bca.20l7.ll. 

httos://www.cambridge.org/core/services/aop-cambridgecore/content/view/FAF984595B822A 70495621 AEA 7EF7DEB/S2l945888l7000 112a.pd£'consumers guide to 

regulatory impact analysis ten tips for being an informed policymaker.pdf 34 Holdren 20 I 0. 35 National Research Council and the Committee on the Institutional Means for Assessment of Risks to Public 

Health. Risk Assessment in the Federal Government: Managing the Process. 1983. Washington D.C.: National 

Academies Press, p. 3. 

36 Dudley and Peacock. 2017. 

The George Washington University Regulatory Studies Center + 7 
52 

every step and can be caused by the potential confounders in observational 

studies, by extrapolation from animal studies to human studies, by extrapolation 

from high to low dose exposures, by inter-individual variability, and by modeling 

the relationships between concentrations, human exposures, and human health 

responses and evaluating the effect of interventions or risk control options on 

public health risk. 37 

Assumptions and judgments made in each of these steps get embedded in predictions of health 

risk under different policy options. Intentionally or not, they can bias the ultimate advice 

provided to decision-makers and the public. Documenting those assumptions and estimating how 

sensitive predicted outcomes are to them and alternative assumptions and judgments could 

greatly improve the transparency and quality ofEPA's decisions. 

Gray and Cohen suggest: 

Fundamentally, the EPA should replace risk values that are built on sciencepolicy assumptions with risk estimates that acknowledge underlying uncertainties. 

For instance, the agency could follow the example of the Intergovernmental Panel 

on Climate Change and report a range of risks that correspond to different models. 

Users would then be able to see whether a value is sufficiently precise to support 

a particular course of action. 38 

They recognize that policymakers will face more difficult choices when faced with a range of 

reasonable estimates but argue "that is how it should be." 

The EPA's definitive values are illusions: they conceal uncertainty that cannot be 

resolved scientifically. Bringing conflicting value judgements into the open will 

enable honest debate and improve public health. 39 

4) EPA will explicitly consider high quality studies that offer new doseresponse information that may allow the agency to move away from default 

assumptions. 

In estimating adverse effects of exposure to many pollutants (especially potential carcinogens, 

but also fine particles), EPA's defimlt dose-response function is assumed to be linear within the 

range of exposures under consideration. Both theory and observation suggest that thresholds 

37 Board on Population Health and Public Health Practice; Institute of Medicine. Environmental Decisions in the 

Face of Uncertainty, Committee on Decision Making Under Uncertainty, 20!3. 

http://www.nap.edu/catalog.php?record id~J2568 

38 George Gray and Jason Cohen. ''Rethink Chemical Risk Assessment." Nature. September 2012; 489. p. 28. 39 Gray and Cohen, p. 28. 

The George Washington University Regulatory Studies Center • 8 
53 

exist below which further reductions in exposure do not yield changes in mortality response, 

however more accurate dose-response functions are elusive.40 The default linear, no threshold 

assumption is convenient in that it allows EPA to estimate incremental health improvements in 

proportion to estimated reductions in exposure; but, if it is inaccurate, it can lead to under or over 

estimates of risks at relevant exposure levels, and to a misallocation of resources. 

EPA's proposed commitment to consider research that can help clarify the effect of low-dose 

exposure to key pollutants would not only improve short term policy outcomes, but it would also 

provide incentives for researchers to devote attention and resources to exploring and reducing 

this key uncertainty. As the 19 regulatory experts observe, "if expected outcomes hinge on the 

value assumed for a particular uncertain variable, it might be appropriate to gather more 

information regarding that variable prior to making a decision,41 or to ask what policies would 

generate the information necessary to reduce that unccrtainty.42" 43 

5) EPA will conduct independent peer review on all pivotal regulatory science 

used to justify regulatory decisions. 

Peer review is a fundamental component of the scientific process. Concerns over the extent and 

rigor of review of important scientific analyses led OMB in 2004 to issue a memorandum 

establishing guidelines for the use of external peer review at all federal agencies and 

departments.44 EPA's proposed approach is consistent with those guidelines, and the exemptions 

therein. 

When engaging experts in peer review, EPA should consider the recommendations of recent 

interdisciplinary efforts regarding scientific advisory panels. Such advisors can provide a 

necessary and valuable source of information and peer review for agency science, but care 

should be taken in both the composition of the panels and the charges they are given. 

An important 2012 Keystone Center report offers a series of recommendations on "the 

composition of committees that are empaneled to review the science behind a regulatory 

40 See, for example Texas Commission on Environmental Quality, "PM,.s Standards may be set Lower than 

Scientifically Justifiable," noting that "extrapolations [to current exposure levels] can be contrary to the basic 

principles of toxicology where the biological threshold (a level below which no effect is apparent) is a key 

concept." http://www. tceq. texas.gov/assets/publ ic/comm exec/pubs/pd/020/20 13/0utlook-Mar-20 13-x. pdf 41 Office of Management and Budget (OMB) (2003). Circular A-4, Regulatory Analysis. September 17, 2003. Cited 

in Dudley et al, 2017. 42 Michael Greenstone (2009). Toward a Culture of Persistent Regulatory Experimentation and Evaluation. In 

David Moss and John Cisternino (Eds.), New Perspectives on Regulation (pp. 111-125). Cambridge: The Tobin 

Project. Cited in Dudley et a! 20 17. 43 Dudley et al, 2017. Journal of Benefit-Cost Analysis. p. 196. 44 U.S. Office of Management and Budget. 2004. Information Quality Bulletin for Peer Review. 

http://www.whitehouse.gov/sites/default/files/omb/memoranda/IY2005/m05-03.pdf 

The George Washington University Regulatory Studies Center • 9 
54 

decision."45 Acknowledging the importance of choosing panelists that "have the knowledge, 

training, and experience needed to address the charge to the panel,"46 it encouraged agencies "to 

recognize that all potential panelists will have conscious and unconscious biases." It 

recommended, "because biases exist, an agency should strive to engage a wide range of 

perspectives of qualified scientific experts. "47 

Both the Keystone group and a group convened by the Bipartisan Policy Center in 2009 

recommend that scientific peer reviewers restrict their advice to matters of science, and not be 

asked to recommend regulatory policies. EPA's Clean Air Scientific Advisory Committee 

(CASAC), for example, is tasked with advising on policy choices, which creates incentives to 

present policy views as scientific recommendations. 48•

49 When drafting charge questions for 

individual peer reviewers and scientific advisory committees, EPA should be careful to solicit 

their scientific expertise without encouraging them to blur the lines between scientific expertise 

and policy judgment. 5° As both the BPC and Keystone reports emphasized, the questions posed 

to experts "should be clearly articulated, and 'explicitly differentiate, to the extent possible, 

between questions that involve scientific judgments and questions that involve judgments about 

economics, ethics, and other matters of policy. "'51 

Applications of the Proposed Policies to Existing Regulatory Science 

When possible, EPA should apply the new guidelines to existing regulations and the regulatory 

science that supports them.SZ This may not always be feasible, especially for regulatory science 

45 The Keystone Center. Research Integrity Roundtable. Improving the Use of Science in Regulatory Decision 

Making: Dealing with Conflict of Interest and Bias in Scientific Advisory Panels. and Improving Systematic 

Scientific Reviews. Washington (DC): The Keystone Center; 2012: p. 4. 

https://www.keystone.org/imagesikeystone-center/spp 

documents/Health/Research%20Integritv%20Rountable%20Report.pdf 46 Keystone, 2012. p. 14 47 Keystone, 2012. p. 15 48 Dudley and Peacock, 2017. 49 See, for instance, the recommendation offormer CASAC member Morton Lippman regarding changing the 

Clean Air Act. Lippman noted "CASAC's role must be limited to highlighting the issues at the science-policy 

interface and the scientific knowledge that informs these issues." Dr. Morton Lippman. "Comments on the 

NAAQS Review Process." 2006, at A-22. 

http:/ /yosem ite.epa.gov/sab/sabproduct.nsf/WebCASA CN anessa%20Memo 03 -l6-06/$File/sabsocasac memo and comments.pdf 50 See Dudley and Peacock, 2017. Several former CASAC officials encouraged EPA to be clearer in its charge 

questions to distinguish between science and policy. Environmental Protection Agency Clean Air Scientific 

Advisory Committee (CASAC). CASAC Input on EPA's revised NAAQS Review Process; 2006 March. 

http://yosemite.epa.gov/sab/sabproduct.nsf/WebCASAC/NewNAAQSProcess?OpenDocument 51 The Keystone Center, 2012:8. (Internal citation to BPC at 5.) 52 83 FR 18772 

The George Washington University Regulatory Studies Center • 10 
55 

that was developed under conditions that would limit disclosure, and EPA should develop clear 

criteria for the types of research that would be eligible for exemptions to its transparency policy. 

Nevertheless, EPA has an opportunity, when conducting retrospective evaluation of regulatory 

impacts (as required by Executive Orders 13 563 and 13 771 ), to reexamine the studies, data, 

models and assumptions that generated ex-ante estimates of regulatory benefits and costs. Such 

retrospective review can provide data that either corroborates or raises questions about the 

assumptions on which previous estimates were based. As such, ensuring transparency and 

opportunities for independent analysis and evaluation could be particularly valuable in 

retrospective review, not only for decisions regarding continuation of existing policies, but also 

for improving the quality of the science used to design new policiesY 

For regulatory programs, like the NAAQS, that periodically review and update standards based 

on a record that has been built over decades, application of the proposed transparency and 

integrity procedures to that record, when feasible, could allow a broader group of experts access 

to pivotal regulatory science, including data, models, and assumptions. Such review would be 

consistent with EPA's statutory mandate for review under the Clean Air Act, and allow EPA to 

make future decisions on better data. 

Protecting Privacy and Confidentiality 

EPA acknowledges concerns that increased transparency and public access to data may risk 

exposing confidential or private information, but it points to practices at other federal agencies 

and in scientific publishing that can ensure confidential or personally-identifiable information is 

not disclosed. Lutter and Zorn review some of these practices and conclude that, depending on 

the situation and sensitivity of the information, a range of measures is available to share data in a 

way that allows access for replication and validation purposes while protecting personally 

identifiable information. 54 

The SAB working group memo raises concerns that "for studies already completed or underway, 

the participation of human subjects is undertaken according to terms approved by the cognizant 

IRB" (institutional review board) which may constrain data sharing. 55 Input from the SAB and 

public comments may elucidate specific concerns, as well as methods EPA can use to address 

them. 

53 Susan E. Dudley 2017. Retrospective Evaluation of Chemical Regulations. Environmental Working Paper No. 

118, Organisation for Economic Co-operation and Development. http://dx.doi.org/10.1787/368e41d7-en. 54 Randall Lutter and David Zorn, "On the Benefits and Costs of Public Access to Data Used to Support Federal 

Policy Making," Mercatus Working Paper, George Mason University. September 2016. 

https://www.mercatus.org/system/files/Mercatus-Lutter-Public-Access-Data-v3.pdf 55 Cullen, 2018. 

The George Washington University Regulatory Studies Center + II 
56 

Conclusion 

Regulations aimed at addressing public health and environmental risks depend heavily on 

scientific information. Regulatory impact analyses often hinge on assessments of risk that 

necessarily involve assumptions and judgments but often generate precise-sounding predictions 

that hide not only considerable uncertainty about the actual risk, but hidden policy judgements. 56 

A lack of transparency surrounding these judgments harms the credibility of scientific advice and 

results in poorer policy decisions. 

Former EPA scientist Robert T. Lackey cautions against what he calls "normative science": 

Science should be objective and based on the best information available. Too 

often, however, scientific information presented to the public and decision-makers 

is infused with hidden policy preferences. Such science is termed normative, and 

it is a corruption of the practice of good science. Normative science is defined as 

"information that is developed, presented or interpreted based on an assumed, 

usually unstated, preference for a particular policy choice. "57 

EPA's proposal to strengthen the transparency of its regulatory science includes reasonable steps 

that could improve the evidential basis for its regulatory policies, and thus improve regulatory 

outcomes by targeting resources to where they can achieve the largest benefits. As President 

Obama's science advisor observed, "open communication among scientists and engineers, and 

between these experts and the public, accelerates scientific and technological advancement, 

strengthens the economy, educates the Nation, and enhances democracy. "58 

The requirements proposed here are not a radical departure from existing guidelines. For 

example, since 2004, OMB has directed agencies to issue information quality guidelines to, 

among other things, ensure the objectivity of information, including "a high degree of 

transparency about data and methods to facilitate the reproducibility of such information by 

qualified third parties."59 President Obama in 2011 encouraged an "open exchange of 

information and perspectives among State, local, and tribal officials, experts in relevant 

disciplines, affected stakeholders in the private sector, and the public as a whole, ... including 

relevant scientific and technical findings. "60 

56 Dudley and Peacock, 2017. p. 36. 57 Robert T. Lackey, "Normative Science." Terra Magazine. Oregon State University. 2013;8(2). 58 Holdren memo, 2010. 59 U.S. Office of Management and Budget. 2002. "Guidelines for Ensuring and Maximizing the Quality, 

Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies." 67 FR 8452 60 Barrack Obama. Executive Order 13563. "Improving Regulation and Regulatory Review." 76 FR 3822 January 

18,2011. 

The George Washington University Regulatory Studies Center • 12 
57 

Greater transparency in the studies, models, assumptions, and risk assessment policy choices 

used in regulatory decisions could encourage more open, constructive debate on those choices. 61 

The scientific method depends on falsifiable hypotheses, data gathering, replication, dissent, and 

challenge, to ensure objective analysis and to minimize bias in the interpretation of results. 62 

Greater transparency is an essential step in improving scientific integrity. Clearer explanations 

regarding the policy rationales for choosing one set of assumptions or models over another 

would encourage more openness and constructive discussion about science and policy, 

improving the ultimate policy decision and engendering greater acceptance of that policy 

choice. 63 

61 Open Data Initiative https:llwww.whitehouse.gov/open 62 Dudley and Peacock, 2017. p. 37. 63 Dudley and Gray, 2012. 

The George Washington University Regulatory Studies Center • 13 
58 

Sally Katzen 

Post-Hearing Questions for the Record 

"From Beginning to End: An Examination of Agencies' Early Public Engagement 

and Retrospective Review" 

From Senator Thomas R. Carper 

For the Honorable Susan Dudley and for the Honorable Sally Katzen 

OIRA's role in reviewing agency rulemaking and EPA's "Strengthening 

Transparency in Regulatory Science" 

On May 9, 2018 I joined Senator Hassan and several other Senators in a letter to thenOIRA Administrator Neomi Rao raising concerns and seeking answers as to how OIRA 

reviewed an EPA proposed rulemaking entitled: "Strengthening Transparency in 

Regulatory Science." 

According to Reginfo.gov, OIRA received the EPA's proposed rule on Thursday, April 

19, 2018. Then-EPA Administrator Scott Pruitt publicly announced this proposed rule 

and signed it three business days later on Tuesday, April24, 2018. Reginfo.gov initially 

stated that OIRA's review was completed on Wednesday, April25, 2018- the day after 

Administrator Pruitt announced and signed the rule. Later, following press inquiries, 

Reginfo.gov was changed to indicate that OIRA's review was completed on Monday, 

April23, 2018. 

Our letter sought detailed information on the scope and substance ofOIRA's review of 

this proposed rulemaking. However, in a response to Senator Hassan on July 12, 2018, 

former Administrator Rao provided little in way of response and simply noted that the 

conclusion date for the OIRA review of this proposed rule of April25, 2018 was due to a 

clerical error that was subsequently corrected. 

With this in mind, I ask that you review and respond to the following: 

I. Please comment on the general process by which OIRA would employ when 

reviewing a complex and far-ranging agency proposal such as this. 

A: There is substantial variability in OIRA's review of draft proposed 

and final rules, depending on, among other things, the significance 

and complexity of the issues being addressed, the potential interests 

and equities of other agencies, the need for expedition (e.g., statutory 

or judicial deadlines or imminent health or safety concerns), and the 

extent to which OIRA staff had been informed or involved during the 

agency drafting process. Typically, when a draft is received, it is 

reviewed by a desk officer at OIRA and is also sent to other agencies 

for their comments. The process can move quickly if no concerns are 

raised and can be quite extended if any of the reviewers has serious 

issues that need to be addressed or resolved. The applicable 

Executive Order provides for a 90-day review time, which is 

generally adhered to; there are many examples of drafts being 

reviewed in substantially less and substantially more time. 
59 

2. During your tenure as Administrator, how long, on average, would OIRA take to 

review complex and far-ranging agency proposals akin to the above mentioned 

EPA proposal? 

A: During my tenure, the average time for review of complex and far-ranging 

proposals would typically be closer to (or even exceed) the 90-day limit , specially 

where there was significant interest in the proposal by other agencies with equities in 

the matter. Averages do not, however, reflect what might be the appropriate review 

time for a particular proposal. 

a. Similarly, do you ever recall tiuring your time as OIRA Administrator 

where OIRA signed off'on an agency rulemaking decision within 3 

business days of receiving a rule? 

A: While data on review times may be available from OIRA, I can recall a 

very limited number of instances where the review time was quite abbreviated 

-including for example, DOT's response to several deaths caused by airbags' expanding with infants or short-statured women in the passenger seat. 

As a rule, however, we took the time necessary to conduct a meaningful 

review, including the inter-agency process. 

3. If then-Administrator Pruitt did indeed sign off on the proposed rule prior to 

OIRA completing its review, would that violate sections 7 and 8 of Executive 

Order 12866? 

A: If the facts were as alleged, that would be inconsistent with the terms of Executive 

Order 12866 1 

••• 

a. What processes are generally in place to prevent agencies from proceeding 

with rules without OIRA approval? 

A: During my tenure as Administrator, we worked with the agencies to 

ensure that did not occur, and we had the support of the Director of OMB 

when necessary. On the one occasion that I recall where an NPRM for a 

significant regulation was released by the agency without having been 

approved by OIRA, the Secretary (or the General Counsel) of the Department 

immediately withdrew the poposal and submitted the draft to OIRA for 

review. 

4. Do you believe agencies should use best available evidence in the rulemaking 

process? 

A: Yes, unequivocally. 
60 

Æ 

5. Similarly, do you think it is appropriate to restrict the scientific evidence that 

agencies can consider during rulemaking? 

A: All available relevant and authoritative scientific evidence should be considered in 

a rulemaking; if certain material or studies are considered less trustworthy or 

probative the reasons for that judgment should be set forth in the preamble to the 

proposal.