U.S. GOVERNMENT PRINTING OFFICE

WASHINGTON : 

For sale by the Superintendent of Documents, U.S. Government Printing Office

Internet: bookstore.gpo.gov Phone: toll free (866) 512–1800; DC area (202) 512–1800

Fax: (202) 512–2104 Mail: Stop IDCC, Washington, DC 20402–0001

82–371 2013

S. HRG. 113–87 

REDUCING UNNECESSARY AND COSTLY RED TAPE 

THROUGH SMARTER REGULATION 

HEARING 

BEFORE THE

JOINT ECONOMIC COMMITTEE 

CONGRESS OF THE UNITED STATES 

ONE HUNDRED THIRTEENTH CONGRESS 

FIRST SESSION 

JUNE 26, 2013 

Printed for the use of the Joint Economic Committee 

( 
(II) 

JOINT ECONOMIC COMMITTEE 

[Created pursuant to Sec. 5(a) of Public Law 304, 79th Congress] 

HOUSE OF REPRESENTATIVES 

KEVIN BRADY, Texas, Chairman 

JOHN CAMPBELL, California 

SEAN P. DUFFY, Wisconsin 

JUSTIN AMASH, Michigan 

ERIK PAULSEN, Minnesota 

RICHARD L. HANNA, New York 

CAROLYN B. MALONEY, New York 

LORETTA SANCHEZ, California 

ELIJAH E. CUMMINGS, Maryland 

JOHN DELANEY, Maryland 

SENATE 

AMY KLOBUCHAR, Minnesota, Vice Chair 

ROBERT P. CASEY, JR., Pennsylvania 

MARK R. WARNER, Virginia 

BERNARD SANDERS, Vermont 

CHRISTOPHER MURPHY, Connecticut 

MARTIN HEINRICH, New Mexico 

DAN COATS, Indiana 

MIKE LEE, Utah 

ROGER F. WICKER, Mississippi 

PAT TOOMEY, Pennsylvania 

ROBERT P. O’QUINN, Executive Director 

NILES GODES, Democratic Staff Director 
(III) 

C O N T E N T S 

OPENING STATEMENTS OF MEMBERS

Hon. Kevin Brady, Chairman, a U.S. Representative from Texas ...................... 1 

Hon. Amy Klobuchar, Vice Chair, a U.S. Senator from Minnesota ..................... 3 

WITNESSES

Professor Susan Dudley, Director, Regulatory Studies, The George Washington University, Washington, DC .................................................................... 6 

Dr. Michael Greenstone, 3M Professor of Environmental Economics, Massachusetts Institute of Technology, Director of the Hamilton Project and a 

Senior Fellow at the Brookings Institution, Cambridge, MA, and Washington, DC ............................................................................................................. 8 

Dr. Jerry Ellig, Senior Research Fellow, Mercatus Center, George Mason 

University, Arlington, VA .................................................................................... 10 

Dr. Robert Kieval, Executive Vice President and Chief Technology Officer, 

CVRx, Inc., Minneapolis, MN .............................................................................. 12 

SUBMISSIONS FOR THE RECORD

Prepared statement of Chairman Brady ............................................................... 30 

Prepared statement of Professor Susan Dudley .................................................... 32 

Prepared statement of Dr. Michael Greenstone .................................................... 47 

Prepared statement of Dr. Jerry Ellig ................................................................... 51 

Prepared statement of Dr. Robert Kieval .............................................................. 84 

(1) 

REDUCING UNNECESSARY AND COSTLY RED 

TAPE THROUGH SMARTER REGULATION 

WEDNESDAY, JUNE 26, 2013 

CONGRESS OF THE UNITED STATES, 

JOINT ECONOMIC COMMITTEE, 

Washington, DC. 

The committee met, pursuant to call, at 9:55 a.m. in Room G– 

50 of the Dirksen Senate Office Building, the Honorable Kevin 

Brady, Chairman, presiding. 

Representatives present: Brady, Paulsen, Hanna, Maloney, 

and Delaney. 

Senators present: Klobuchar, Murphy, Coats, and Toomey. 

Staff present: Gabriel Adler, Ted Boll, Hank Butler, Colleen 

Healy, Conor Carroll, Gail Cohen, Connie Foster, Niles Godes, Patrick Miller, and Robert O’Quinn. 

OPENING STATEMENT OF HON. KEVIN BRADY, CHAIRMAN, A 

U.S. REPRESENTATIVE FROM TEXAS 

Chairman Brady. Good morning, everyone. We are going to do 

something unusual and start ahead of time. So we will see if that 

works in Washington. 

This month the current recovery celebrates its fourth anniversary. Now is a good time to assess how the U.S. economy is performing. 

Unfortunately, for American families the current recovery remains the weakest since World War II. There is a troubling Growth 

Gap in economic performance between this recovery and the average of post-war recoveries, leaving our economy 4 million private 

sector jobs short and $1.2 trillion missing from the economy. While 

Wall Street is booming, every man, woman, and child in America 

is missing nearly $3,000 in real disposable income due to the 

Growth Gap. 

During this Congress, the Joint Economic Committee has been 

examining the causes of the Growth Gap and the types of alternative policies to close that gap. The Joint Economic Committee 

has studied how current fiscal and monetary policies have held 

back this recovery. Today, the JEC will explore regulatory policy. 

From town hall meetings with my constituents in Texas, to conversations with business leaders and economists across America, 

there is one consistent message: Uncertainty over the costs of new 

regulations in healthcare, the environment, labor issues, and financial services is suppressing business investment and the creation of 

new jobs along Main Street. 
2 

The burden of federal regulations is large. At year-end 2012, the 

Code of Federal Regulations had 238 volumes and 174,000 pages. 

That burden is growing. In 2012, the Federal Register—which 

publishes proposed new rules and regulations, final rules, and 

changes to existing regulations—totalled 78,961 pages. Three of the 

four highest page counts since the Federal Register began publication have occurred during the current presidency. 

And that burden is costly. NERA Economic Consulting, in a 

study last year commissioned by Manufacturers Alliance for Productivity and Innovation, estimates the current direct cost of compliance with ‘‘major’’ regulations—that is, those with an estimated 

cost greater than $100 million a year—issued between 1993 and 

2011 to be between $265 billion and $726 billion every year. Clyde 

Wayne Crews of the Competitive Enterprise Institute estimates the 

total cost of regulation in America approaches $1.8 trillion annually, or nearly 12 percent of our Nation’s economy. 

Given this historically weak recovery, the rise of technology to 

help us meet regulatory goals more cheaply, and a shared belief 

that America should continue progress on a clean environment and 

safe workplace, when regulations are necessary doesn’t the public 

deserve the most effective regulation at the least cost? 

Smart regulations that improve the market process and its incentive structure to accelerate progress rather than dictate particular 

outcomes will prove superior to tens of thousands of pages of mandated rules and micro-managed instructions. 

Devising process-enhancing rules that engage the private sector’s 

versatility and creativity requires objective upfront analysis and 

thoughtful design. 

Yet federal agencies often do things the other way around: deciding first what they want to do, and then using whatever analysis 

is performed to justify their preconceived ‘‘solution.’’ This abuse 

must stop. 

In 1981, President Reagan issued an Executive Order requiring 

Executive Branch agencies to conduct Regulatory Impact Analysis, 

commonly known as cost-benefit analysis, before issuing major new 

regulations. This first step toward smarter regulation had its limitations. 

An executive order affects only Executive Branch regulatory 

agencies and therefore does not affect independent regulatory agencies such as the Consumer Product Safety Commission, the Federal 

Trade Commission, the Federal Reserve, and the Consumer Financial Protection Board. 

Over the years, Congress has exempted broad swaths of federal 

regulation from the scrutiny of cost-benefit analysis through provisions of the Clean Air Act, for example. While there are government-wide ‘‘best practice’’ standards on how agencies should conduct cost-benefit analysis, they are not uniformly applied and are 

not legally binding. So the quality of agency cost-benefit analyses 

varies greatly. 

Agency bureaucrats, as you would imagine, are naturally biased 

toward their proposed regulation and have learned how to manipulate cost-benefit analysis to justify whatever new regulations they 

wish to issue. 
3 

For example, former Administrator of the Office of Information 

and Regulatory Affairs, Professor John Graham, closely examined 

CAFE, the Corporate Average Fuel Economy standards for trucks 

in his testimony before the House Committee on Oversight and 

Government Reform in September 2011 and found that to inflate 

the benefits of their new rule regulators had cut the discount rate 

and the so-called ‘‘rebound effect’’ of increased driving with better 

mileage to half or less. He also found that they failed to carefully 

consider the rule’s effects on vehicle size, performance, and safety. 

In other words, today too few proposed rules are fully analyzed. 

There are too many loopholes, no uniform requirement across all 

agencies, a lack of standards with which to conduct the analysis, 

no check-and-balance against agency bias, no comparison of past 

analysis to real-life impacts, and little recognition of the total burdens on the economy of regulation. 

We must do better. The purpose of this hearing is to discover 

ways in which Congress can make the regulatory process ‘‘smarter,’’ more cost-effective, and better designed to accomplish the goals 

without damaging the economy. 

In particular, the Committee hopes to hear from today’s witnesses about the deficiencies in cost-benefit analysis as it is now 

practiced, and how agencies can do a better job of quantifying and 

measuring the costs and the benefits of both proposed and existing 

regulations. 

I look forward to the testimonies, and I recognize our Vice Chair, 

Senator Klobuchar, for her opening remarks. 

[The prepared statement of Chairman Brady appears in the Submissions for the Record on page 30.] 

OPENING STATEMENT OF HON. AMY KLOBUCHAR, VICE 

CHAIR, A U.S. SENATOR FROM MINNESOTA 

Vice Chair Klobuchar. Well thank you very much, Mr. Chairman, and thank you for holding this hearing, and thank you to all 

the witnesses. 

I particularly wanted to recognize two witnesses I invited, Dr. 

Greenstone and also Dr. Kieval, who heads up a company in Minnesota, CVRx, Inc., which is a medical device company. And I know 

that Representative Paulsen welcomes him, as well, as we both 

have done a lot of work—Representative Paulsen and myself—in 

the area of medical device regulation to make things a little easier, 

and also trying to get the medical device tax repealed. 

This hearing is especially important to me because, before coming to the U.S. Senate I spent 13 years representing companies in 

regulatory areas, doing everything from trying to get competitive 

telecom carriers into the telecommunications local market, to helping Dairy Queen get their 2-for-1 Sundae deal approved in every 

state in the Union. 

I know you’re curious about which the last state was that was 

willing to approve it. That would be Louisiana, not really a surprise. 

Americans expect and deserve a common-sense approach to regulation, one that protects consumers and the public interest, without 

stifling innovation and economic growth. And this means doing 

sensible things that advance technology and global competitive-
4 

ness, while also maintaining safety and security standards. And 

that is the way I look at this. 

At the same time, I would agree with the Chairman that we 

should be open to change. I believe we need to be a country that 

makes stuff again, invents things, exports to the world, and for 

that we need a competitive agenda. Which means building our 

work skills. Which means building on exports. Which means reducing our debt in a balanced way. Doing comprehensive tax reform. 

Bringing down that business tax rate, and paying for it by closing 

loopholes and other things. 

But this also means cutting down on red tape. One example that 

we will talk about today is medical device. My State has a long history of leadership in medical device manufacturing. The story of 

Earl Bakken tinkering around in his garage and launching 

Medtronic is the stuff of legends, but it is not just big companies 

like Medtronic that keep this industry running. It is the small- and 

medium-sized medical tech companies. 

I recently visited one that actually got its start in a chickencoop. 

The U.S. is the largest net exporter of medical devices in the world, 

enjoying a trade surplus of $5 billion a year. Yet we’ve seen a decline in venture capital funding largely due to delays in the approval process. 

According to one study, venture capital investment in the medical device and equipment industry fell 20 percent from the first 

quarter of 2012 to the first quarter of 2013. 

It is critical that we prevent regulatory burdens from interfering 

with the delivery of life-saving products. We recently passed 

MDFA/PDFA bills out of this Congress, signed into law, that contain some really good things, including bills that I introduced and 

Representative Paulsen and others worked on, the agency’s leastburdensome principles, which have been continuously ignored by 

the FDA reviewers, improving the conflict-of-interest provisions 

making it easier for the FDA to recruit top-line experts to take part 

in the review process, and requiring the FDA to use an independent contractor to assess the management processes. 

This is just one example. Another that I’ve worked on a lot is 

tourism. And I personally believe that one way we can do this, in 

addition to looking at some of the cost/benefit analysis, is looking 

industry by industry and figuring out what are things that we can 

do that we could actually get through Congress, or that the Administration could do on its own without Congress, to reduce some of 

these regulatory burdens and make things work for today. 

One of my best examples is tourism. Since 9/11, through two Administrations, we lost 16 percent of the international tourism market. Every point that we gained back is 161,000 jobs. A lot of that 

had to do with slowdown of the approval of the visa process. 

Senator Blunt and I worked on this together. We have seen dramatic changes in China now. We were unable to compete with Chinese wanting to go to England. We are now in the game, competing 

in terms of the wait times for those visas. 

Brazil is down to two days in several cities. That is simply because we move people around, realizing this was a profit center for 

our government, and did this without legislation. 
5 

Now we also can do more things legislatively with visa waivers. 

That is why I am so excited in part about the immigration bill. 

There are some really good things in there, like the JOLT Act that 

will help with that. And Senator Blunt and I, along with Representative Walz, passed last December the no-hassle flying act, 

which was about not doing double screening of luggage. As minor 

as it might sound, it makes a huge difference when you’re on the 

Canadian Border in Minnesota when you have airports that are already up to our standards for screening. 

Exports are another example. As we see the defense industry 

being hit by sequestration and other things, and some reduction in 

our work overseas, we have to try to keep those jobs in America. 

And part of this is going to be having to look at the export rules 

and make the defense export system more efficient by creating a 

unified list of restrictive items at one agency, rather than having 

lists at multiple agencies. 

This will help defense subcontractors and other businesses that 

make parts that are used in military equipment but are not exclusively military products. So we have to look at this export control 

list and make it work better. 

Agriculture is something we care a lot about in our State. We 

have seen a number of rules from the EPA that have come out, and 

then later been rejected, whether it’s milk spills being regulated as 

oil spills, or other things, where I think we can be smarter about 

how we do that. 

I actually had a bill that I had with Senator Lugar that I will 

be introducing again to try to get people with agriculture backgrounds on the EPA rulemaking group so that we can try to get 

that interest represented and stop things from happening that do 

not make sense. 

The Chairman mentioned the tax issues. I truly believe that we 

need comprehensive tax reform. After immigration, I would love it 

if that was the next issue that we went to, in addition to trying 

to work out a bipartisan agreement. Our Tax Code, as we all know, 

is too complicated. And while we have made some inroads like repealing the 1099 reporting requirement, I think there’s others we 

can do. 

That being said, we have to remember our economy is stable. We 

have seen improvements. In Minnesota we’re down to 5.3 percent 

unemployment. And so I want to keep on that road, not doing anything to interfere with safety or security but look at this red tape 

as part of the solution to improving even more. 

Thank you, Mr. Chairman. 

Chairman Brady. We have a terrific panel of witnesses today. 

I appreciate each of you being here today and look forward to your 

testimony. Let me introduce each of them. 

Ms. Dudley, Professor Dudley, directs the George Washington 

University’s Regulatory Studies Center, which she founded in 2009 

to focus on high-quality research in regulatory policy. She has previously served as Administrator of the Office of Information and 

Regulatory Affairs of the U.S. Office of Management and Budget. 

She has also directed and taught the Regulatory Studies Program 

at the Mercatus Center at George Mason. She holds a Master’s Degree from MIT’s Sloan School of Management, and a Bachelor’s De-
6 

gree in Resource Economics from the University of Massachusetts 

at Amherst. 

Mr. Greenstone, Dr. Greenstone, is the 3M Professor of Environmental Economics in the Department of Economics at MIT. He is 

on the MIT Energy Initiatives Council, and on their Environmental 

Research Council. He also serves as a Senior Fellow at the Brookings Institution, and a Research Associate at the National Bureau 

of Economic Research. Previously he served as the Chief Economist 

for President Obama’s Council of Economic Advisors. He holds a 

Ph.D. in Economics from Princeton, and a B.A. in Economics from 

Swarthmore College. Welcome, Doctor. 

Dr. Ellig is a Senior Research Fellow at the Mercatus Center at 

George Mason University, as well, focusing on the federal regulatory process, economic regulation, and telecommunications regulation. Previously he served as Deputy Director and Acting Director 

of the Office of Policy Planning at the Federal Trade Commission, 

and has also served as a senior economist for this Committee. Welcome. Dr. Ellig received his Doctorate and Masters in Economics 

from George Mason, and his B.A. in Economics from Xavier University. 

Dr. Kieval is founder and Chief Technology Officer of CVRx, a 

private medical device company located in Minnesota whose State 

has currently taken over the Joint Economic Committee. 

[Laughter.] 

He is over—much to my dismay—he has over 16 years of medical 

device industry experience, including being named Innovator of The 

Year by Twin Cities Finance and Commerce. He currently serves 

on the board of the Medical Device Manufacturers Association. He 

completed his undergraduate and graduate training at the University of Pennsylvania, where he received Doctorate Degrees in Veterinary Medicine and in Psychology. 

Welcome to each of the panelists today. I think this is a very important topic, and certainly our businesses large and small have 

raised regulatory concerns as their number one and number two 

concerns about their ability to hire. 

So just a nuts and bolts discussion on how we can achieve our 

regulatory goals in a smarter, cheaper, more cost-effective way by 

doing analysis up front is really critical. 

So, Professor Dudley, we have reserved five minutes for your oral 

remarks. Your statement will be made a part of the record. Please 

go ahead. Can you get that mic? 

STATEMENT OF PROFESSOR SUSAN DUDLEY, DIRECTOR, REGULATORY STUDIES CENTER, THE GEORGE WASHINGTON 

UNIVERSITY, WASHINGTON, DC 

Professor Dudley. Thank you, Chairman Brady, and Vice Chair 

Klobuchar. I really enjoyed your opening statements, and I am 

pleased that the Committee is holding this hearing. 

I would like to talk about the importance of Regulatory Impact 

Analysis, but also the need for institutional change to provide better incentives for ensuring better regulatory outcomes. 

While Regulatory Impact Analysis is widely accepted by practitioners and academics, and every modern President has endorsed 

its use, too many regulations are still being issued without mean-
7 

ingful analysis of the likely consequences of alternative actions. 

And you mentioned this in your opening remarks, Chairman 

Brady. 

First, many statutes preclude agencies from considering important tradeoffs when developing regulations. Now fixing that problem would likely require amending language in existing statutes, 

but also ensuring that new legislation requires agencies to weigh 

regulatory effects and provides them with adequate time to do the 

research, to deliberate, and to consult with the public before they 

issue new regulations. 

The other constraint, as you mentioned, is that independent regulatory agencies have not been subject to executive oversight, and 

they have been less than rigorous in their analysis. 

Now Senator Warner, who is a member of this Committee, has 

proposed legislation, co-sponsored with Senators Portman and Collins, that would subject independent agencies to regulatory analysis and oversight. And I think that is important. 

But even when it is conducted, regulatory analysis is not the silver bullet guaranteeing smarter regulation, as you both mentioned. 

Ex ante analysis necessarily rests on hypotheses of how regulatory 

action will alter outcomes and what it will cost. 

So even the most carefully analyzed regulations may result in 

unanticipated changes in behavior that undermine their desired effects. 

Compounding this problem is—and I feel like I’m saying everything that you said—is that agencies have strong incentives to 

demonstrate that their desired regulations will have benefits that 

exceed their costs. 

Regulatory impact analyses are often done after the decision is 

made to justify, rather than to inform, the decision. 

Agency staff are smart and motivated, but like everyone else 

they are susceptible to the confirmation bias. And their single-mission focus often leads them to discount data, research, values, and 

perspectives that do not support their preferred regulatory alternative. 

Institutional changes that provide more effective checks and balances and engage the wisdom of crowds are needed to counter 

these natural incentives. Judicial oversight provides an important 

Constitutional check, but courts defer to agency expertise when 

evaluating regulatory records and requirements, and Presidential 

Executive Orders are not enforceable by law. 

There are some promising legislative initiatives that would make 

Regulatory Impact Analysis judicially reviewable, and others that 

would alter the deference that courts give to agencies. 

Congress could also provide more checks and balances by voting 

on new regulations before they’re issued. It could also assign responsibility for evaluating regulatory bills and regulations to a 

Congressional office. 

Just as the CBO provides independent estimates of the on-budget 

cost of legislation and federal programs, a staff of Congressional 

regulatory experts could provide Congress and the public independent analysis regarding the likely off-budget effects of legislation and regulation. 
8 

And then the American public can provide checks and balances, 

too. Recognizing that not all problems require a regulatory solution 

is an important first step. Requiring earlier disclosure of key information could also engage broad public input, long before any policy 

decisions are formed and could bring greater transparency to the 

rationale behind regulatory decisions. 

Regulations also lack accountability because once they’re in place 

agencies seldom look back to evaluate whether they are having 

their intended effects. 

Initiatives to require ex poste evaluation of regulations have met 

with limited success largely because they did not change the underlying incentives. 

Two ideas that have the potential to impose needed discipline on 

regulatory agencies, and to generate a constructive debate on the 

real impacts of regulation, are a regulatory improvement commission that operates like the BRAC, and a policy similar to the UK’s 

one-in-one-out approach that requires agencies to make trade-offs 

when issuing new regulations. 

So in closing, Regulatory Impact Analysis is a longstanding and 

important element of U.S. regulatory policy, but a variety of institutional obstacles prevent it from being a silver bullet for producing smarter regulation. 

Greater Congressional oversight, judicial oversight, and opportunities for public involvement could provide better accountability 

and improve the reasoning underlying regulatory decisions, as well 

as the decisions themselves. 

[The prepared statement of Professor Susan Dudley appears in 

the Submissions for the Record on page 32.] 

Chairman Brady. Thank you, Professor. And don’t worry about 

agreeing with Senator Klobuchar and I. It rarely happens. We’re 

glad to hear that happened. 

[Laughter.] 

So, Dr. Greenstone. 

STATEMENT OF DR. MICHAEL GREENSTONE, DIRECTOR, HAMILTON PROJECT, 3M PROFESSOR OF ECONOMICS, MASSACHUSETTS INSTITUTE OF TECHNOLOGY, WASHINGTON, DC, 

AND CAMBRIDGE, MA 

Dr. Greenstone. Thank you, Chairman Brady, Vice Chair 

Klobuchar, and Members of the Joint Economic Committee, for inviting me to speak today about opportunities to improve the government’s regulatory system. 

American government, as all of you know, at every level regulates a broad array of social and economic life. Regulatory policy 

determines the air we breathe, the quality of the water we drink, 

the safety of our workplaces, the investments we make, and so 

much more. 

Government regulates these activities because, in cases of market failure, for example, our free market system does not create the 

necessary incentives for businesses and individuals to protect the 

public good. 

The challenge for regulators is to consistently set rules with benefits that exceed their costs, or otherwise achieve their statutory 

objectives. However, an important weakness in our regulatory sys-
9 

tem is that we generally don’t have the information to make these 

judgments over the long haul. 

This is because our evaluations are done before the regulations 

are enacted, and are almost entirely based on a regulation’s likely 

benefits and likely costs. Of course this is the point when we know 

the absolute least, precisely because the regulations are untested. 

Once a regulation passes this ex ante test, it goes on the books 

and generally stays there unexamined for years, and in many cases 

decades. In practice, some regulations work out exactly as intended, but others do not. 

For example, an air pollutant may prove to be more harmful 

than was originally understood; or innovation may lead to new and 

less expensive pollution abatement technology. 

President Obama’s Executive Orders 13563 and 13610 spell out 

what I think is a potentially revolutionary step forward in regulatory policy. Specifically, they require that agencies routinely revisit the measurement of costs and benefits of existing regulations, 

and identify the least costly ways to achieve a regulation’s goals. 

In the remainder of my testimony I am going to identify two further changes that I think would increase the chances that our regulatory system consistently produces rules with benefits that exceed costs. 

The first change is to make three reforms that build on the 

President’s recent Executive Orders. 

First, I recommend institutionalizing the retrospective review of 

economically significant rules so that these reviews are automatic. 

Depending on the particulars of the rules, the reviews should be 

completed within a prespecified period—say 5 to 10 years. 

In addition, the relevant agency would be required to prespecify 

the expected benefits—for example, reduced child mortality rates— 

and expected costs: say reduced business profits, so that the terms 

of the subsequent review would be known in advance and could not 

be changed later. 

Second, the relevant agency should commit to undertaking a new 

rulemaking when the results from the retrospective analysis differ 

from the benefits and costs that were expected prior to the regulation’s implementation. The new rulemaking should also operate 

under a time limit. 

Third, these efforts would be strengthened if they were accompanied by triggers to ensure that they are undertaken within a prescribed time period. One approach would be for agencies to post on 

their website the deadline for a rule’s review and reconsideration. 

A stronger approach would be to enable the Judiciary to compel reviews and new rulemakings in cases where an agency has failed to 

comply with the review timeline, or to act upon its results. 

There are some difficulties with this approach I have just outlined. Many agencies do not have the staff, expertise, or resources 

necessary to undertake these reviews. 

Further, as Dr. Dudley has pointed out, the process of self-evaluation is challenging for all organizations as it requires complete 

objectivity. 

My recommendation is to establish a new, independent body for 

regulatory review. This body could be housed within the Legislative 
10 

Branch and modeled after the Congressional Budget Office—or 

even become a division within the existing CBO. 

As you know, before the CBO was established only the President 

had a ready source of budgetary and economic data and analysis. 

The entire budget process has benefitted from CBO’s existence and 

its independence. 

Budgetary analyses and proposals throughout Washington are 

now created to a higher standard, knowing that they must ultimately face scrutiny by the nonpartisan CBO. 

My recommendation would be to place this new organization in 

the Legislative Branch and make it avowedly nonpartisan, just like 

the CBO. The organization would be charged with conducting independent regulatory impact evaluations. 

Of course the creation of such a body would require resources. 

My best estimate is that such an organization could be funded for 

less than $15 to $20 million annually. This is a modest amount of 

money when compared to the hundreds of billions of dollars of costs 

and benefits that regulations introduce in our economy. 

My judgment is that it is very likely that such an office would 

pay for itself many times over. 

To quickly summarize, I propose two key reforms: 

One, institutionalize a process by which agencies automatically 

undertake retrospective reviews of regulations, and initiate a new 

rulemaking when the results from the retrospective analysis differ 

from the expected benefits and costs. 

Two, create a new independent body for rigorous, objective regulatory review that is modeled on the Congressional Budget Office. 

We live in a rapidly changing economy and need a regulatory review structure that evolves to meet the new and different needs of 

our society. The reforms that I have outlined here would give policymakers tools for protecting those regulations with great benefits 

for our society, reforming those regulations that impose unnecessary costs, and culling those that no longer serve their purpose. 

That would be good for our well-being and good for the American 

economy. 

Thank you, once again, for inviting me to participate and I will 

gladly respond to any questions. 

[The prepared statement of Dr. Michael Greenstone appears in 

the Submissions for the Record on page 47.] 

Chairman Brady. Great. Thank you, Doctor. Dr. Ellig. 

STATEMENT OF DR. JERRY ELLIG, SENIOR RESEARCH FELLOW, MERCATUS CENTER, GEORGE MASON UNIVERSITY, ARLINGTON, VA 

Dr. Ellig. Well, Chairman Brady, Vice Chair Klobuchar, Members of the Committee, I would like to thank you for the opportunity to testify here today. 

If I could summarize my testimony in a nutshell, it’s this: Regulatory Impact Analysis is critically important for making sensible 

regulatory decisions. A lot of times it is not done very well, or it 

does not appear to have much of an effect on decisions, and improvement in that is probably going to require legislation. 

Let me elaborate on each of these three points. 
11 

First, why is Regulatory Impact Analysis important? Well one of 

my favorite old Calvin & Hobbs cartoons starts with the little boy, 

Calvin, holding a water balloon saying, 

‘‘I’m going to prove that there is no moral law governing the universe. I will throw this water balloon at Suzy Dirkens, unless the 

universe gives me a sign that that would be wrong in the next 10 

seconds.’’ 

So he counts down. He says, ‘‘Nope, no sign.’’ Throws the water 

balloon. 

She chases him. Beats him up. In the last panel of the cartoon, 

he’s lying on the ground saying: 

‘‘Why does the universe always give you the sign after you do it?’’ 

[Laughter.] 

That is why we need high-quality Regulatory Impact Analysis to 

inform regulatory decisions. 

We expect regulation to accomplish a lot of really important 

things. We expect regulation to protect us from financial fraud; to 

keep the air clean; keep the water clean; in the case of my old 

agency, the Federal Trade Commission, we expect regulation to 

prevent telemarketers from bothering us at home when we don’t 

want to be bothered. All of those kinds of good things. 

In order to get those good things, we usually have to give something up. Sometimes it’s money. Things cost more. Sometimes we 

have to give up privacy. We give up convenience. We give up dignity. We give up liberty. Regulation tells us what we may do, what 

we may not do, and sometimes what we must do. 

And if government is going to tell us what we may, may not, and 

must do, government has a moral responsibility to understand the 

likely consequences of regulation, to understand the likely consequences of alternatives, and to understand those things ahead of 

time before it makes decisions. 

And a good Regulatory Impact Analysis provides that kind of information. Regulatory Impact Analysis is a structured framework 

for comparing the potential results of different courses of action, 

and it is also a structured framework for assessing the nature of 

the problem we’re trying to solve so we can pick a solution that will 

actually work. 

Now Presidents have recognized this. For more than three decades, Presidents have directed Executive Branch agencies to conduct Regulatory Impact Analysis for important regulations. 

Unfortunately, we find that often Regulatory Impact Analysis is 

not done very well, or is not used very much in decisions. The most 

recent piece of research that looks at that is a project at the 

Mercatus Center that we call ‘‘The Regulatory Report Card.’’ 

We have a team of economics professors around the country who 

evaluate the economically significant regulations. Those are the 

ones that have an economic impact above $100 million. It is kind 

of a giant exercise in grading papers. We have done this for—from 

2008 to 2012, looking at the quality of the analysis and the extent 

to which it is used; evaluating them based on criteria that are in 

the Executive Order that governs regulatory analysis and review. 

And the result of this is that over the past five years on average 

regulations have earned about half of the available points that they 
12 

could earn on our grading scale. For me, 50 percent is about good 

enough for an ‘‘F.’’ 

The best we have ever seen earns about 80 percent of the possible points on our grading scale, so that might be a B-. These results are consistent with what other researchers find when they 

have looked at the quality of Regulatory Impact Analysis and when 

they have looked at the use of Regulatory Impact Analysis. 

We certainly do find some best practices, and we do find cases 

where the analysis seems to have affected decisions, but those are 

exceptions rather than the rule. 

So how could the U.S. Government improve the use of Regulatory 

Impact Analysis? Well the first step toward that is understanding 

that no single Administration, and no single political party, is to 

blame for this problem. 

We looked at regulations in the Bush Administration and the 

Obama Administration. On average, there was no difference in 

quality. On average, there was no difference in the extent to which 

the analysis was used. Other researchers have found the same 

thing when they compared the quality of Regulatory Impact Analysis across Administrations of different parties. 

It is not a partisan problem, or a problem with a particular Administration; it is an institutional problem that requires institutional solutions. 

The most logical obvious institutional solution is a legislative requirement that all agencies conduct Regulatory Impact Analysis for 

regulations of a certain level of importance combined with Judicial 

review to ensure that the analysis meets a certain standard of 

quality, and that the agency explained how the analysis affected its 

decision. 

In short, regulation is too important to be based just on good intentions. We need to actually know what we are doing before we 

do it. 

[The prepared statement of Dr. Jerry Ellig appears in the Submissions for the Record on page 51.] 

Chairman Brady. Great. Thank you, Dr. Ellig. 

Dr. Kieval. 

STATEMENT OF DR. ROBERT KIEVAL, EXECUTIVE VICE PRESIDENT AND CHIEF TECHNOLOGY OFFICER, CVRx, INC., MINNEAPOLIS, MN 

Dr. Kieval. Thank you, and good morning. 

Chairman Brady. Can you hit that (microphone). 

Dr. Kieval. Got it. Good morning, Chairman Brady, Vice Chair 

Klobuchar, Members of the Committee. It is an honor to have this 

opportunity to address you today. I will focus my remarks on the 

impact of regulation on medical technology and innovation. 

As you have heard, I am the founder and Chief Technology Officer of CVRx, a Minneapolis-based medical device company. I have 

worked in the medical technology industry throughout my entire 

career, with experience both at a large manufacturer, and also in 

the start-up environment. 

In addition to serving on the board of the Medical Device Manufacturers Association here in Washington, I also serve on the board 

of our local industry organization, LifeScience Alley in Minneapolis. 
13 

CVRx is now about 11 years old. We remain pre-revenue here in 

the United States, and are in early commercialization outside the 

U.S. 

The U.S. med tech industry supports at least 400,000 jobs, with 

nearly 2 million more in adjacent sectors. It is one of the few American industries that is a net exporter and we are the global leader. 

Small businesses like CVRx, often with fewer than 50 employees, 

are a vital source of innovation and comprise approximately 80 percent of the industry. Companies like ours, with a single product 

and no alternative revenue streams, depend on outside investment. 

Investors require assurance of a reasonable and predictable path 

to product approval. Ambiguous or overly burdensome approval 

thresholds can fatally inhibit investment in a company and prevent 

development of what could be a very meaningful new therapy. 

Since 2005, the time and capital required by a company to get 

a clear determination of its regulatory pathway, to negotiate clinical trial requirements, and to obtain a product approval decision 

have risen dramatically. 

The approval process itself has become increasingly inefficient, 

inconsistent, and unpredictable. This has led patients—excuse me, 

this has led to patients outside the U.S. frequently getting access 

to American innovations an average of two years before American 

patients do. 

In many cases, jobs and R&D have also moved overseas, weakening our industry’s competitiveness. 

This is also the case for CVRx. While we work through the approval process here, our product is treating patients in Germany, 

Italy, the Netherlands, Hungary, and Turkey. I just returned from 

Europe where I heard from doctors how patients there are benefitting from our technology. As a result, the jobs that we are adding 

are also largely overseas. 

The FDA has a crucial mission to protect the public health. 

Clearly this means providing reasonable assurance that products 

are safe before they are approved. However, it also means that patients should not be unduly deprived of innovations because of inefficient or overly burdensome approval processes. 

Finally, the medical device tax, a tax on revenues irrespective of 

a company’s earnings, further increases financial pressure and 

compounds these difficulties. For larger companies, these challenges often represent issues of profitability; for smaller companies 

like CVRx, they may be issues of survivability. 

In the first quarter of this year, first-time financings in the life 

sciences dropped dramatically to $98 million, the lowest quarterly 

amount since 1996. By comparison, in 2007 alone, start-up device 

companies raised over $700 million in initial financing. These early 

stage investments are a clear leading indicator of future innovations. So this does not bode well for patients. 

Federal regulators and policymakers are working to address 

these issues. Our industry appreciates the bipartisan support for 

the Food and Drug Administration’s Safety and Innovation Act of 

2012. This reauthorized the medical device user fee program, and 

includes reforms that, if implemented as intended, will really benefit patients, innovation, and our economy. 
14 

These include earlier substantive interactions between FDA and 

industry shared outcome goals that track performance in calendar 

days clarification of least-burdensome language provisions regarding conflict of interest and management review. 

On a personal note, I would like to thank yet again my home 

State Members and the Minnesota Delegation for their tireless 

work on these issues. 

The Medical Device Innovation Consortium, a public/private 

partnership that had roots in Minnesota but is now a national program, is a promising example of government and industry working 

collaboratively to identify and improve regulatory inefficiencies. 

Also encouraging are reports that FDA is concentrating on three 

highly practical priorities: Improving efficiency in clinical trials; 

balancing the pre- and post-market process; and identifying ways 

to shorten the lag between FDA product approval and reimbursement approval by CMS. 

In closing, capitalizing on many of these opportunities will require effective collaboration between patients, industry, and the 

FDA. However, Congress can play an important role as well by ensuring that all parties continue to work toward these goals in a 

highly constructive manner. 

Thank you very much for your attention. 

[The prepared statement of Dr. Robert Kieval appears in the 

Submissions for the Record on page 84.] 

Chairman Brady. Thank you, Dr. Kieval. And don’t keep giving 

the Minnesota Delegation a bigger head, please. 

[Laughter.] 

It’s tough enough to deal with these guys. 

I want to talk real quickly—ask, real quickly, about the loopholes 

at the agencies that are not part of the requirement, standards and 

review, looking back. But before I do that, you know our businesses 

really are concerned about the level of implementation of regulation. 

OIRA, looking at 2011, looked at 3,500 rules and regulations that 

year. About 58 of them were major rules. Only 13 underwent complete cost-benefit analysis. That seems like an awfully small 

amount of scrutiny and cost-benefit analysis ahead of an awful lot 

of regulation. 

I have adopted the Papa John’s motto ‘‘Better Ingredients, Better 

Pizza.’’ Better analysis means better regulation, when it’s done 

ahead of time. 

So I want to ask: Is it the belief of the panel that we ought to 

close the loopholes to ensure that independent regulatory agencies 

also conduct a net cost-benefit analysis up front? 

Dr. Ellig, you were at the FTC. They would be included in there. 

Do you agree? 

Dr. Ellig. I agree that it is important to have all agencies under 

the same set of requirements. I think there really are two loopholes. 

The one loophole is independent agencies that are not covered by 

the Executive Orders and do not have a separate Legislative requirement for cost/benefit analysis don’t have to do it. 

The other kind of loophole is, under the current set of institutions we have agencies analyzing their own regulations. We have 
15 

economists in the Executive Branch agencies who are expected to 

analyze basically their boss’ decisions—— 

Chairman Brady. Yes. 

Dr. Ellig [continuing]. And render an impartial verdict on that. 

And then we have OIRA (Office of Information and Regulatory 

Affairs), which is inside the Administration, you know, assessing 

what the agencies are doing. And I think we get better results with 

that than if we didn’t have those requirements and didn’t have 

OIRA, but we still have a lot of folks who are issuing regulations 

and essentially in charge of reviewing what they themselves are 

doing. 

So it would be nice to have, for the Executive Branch agencies, 

a more independent look at the quality of their analysis. And that 

is one of the things that some kind of Judicial review would accomplish. 

Chairman Brady. Great. To that point, standards, the transparency on those cost-benefit analyses ahead of time, where the 

public can comment. You can see what ingredients go into the cake 

ahead of time, the analysis. 

Professor Dudley, you were at OIRA. I know they have a best 

practices, best standards’ type approach, but statutorily should we 

set up some process where there are standards, where agencies can 

develop them publicly, you know, tailored to their regulatory challenges; they can be commented on and reviewed as part of setting 

appropriate standards up front? 

Professor Dudley. Yes, I do think it would be valuable to have 

a statutory standard that supercedes other authorities. I think a 

lot of the problem is, as you mentioned in your opening remarks, 

there are some statutes that direct agencies not to consider, or at 

least have been interpreted by the courts as telling agencies they 

can’t consider things that any sensible person would want to consider in making a decision. 

Chairman Brady. On the look-back, it seems common sense 

would be to review, as a number of you proposed, a regular process 

to look back and compare the actual real-life impact versus the 

original analysis, when it is done. 

In general—how best do we accomplish that? Is that a statutory 

change that requires it to be done? Is it including frequent Judicial 

review as part of the process? 

Dr. Kieval, sort of going backward—you know, you deal with the 

FDA in heading this way on the panel—your views? Because you 

see it in your industry, obviously, with regulations. Good goals, you 

know, shared goals, how those regulations are implemented create, 

as you say, huge impacts on patients, the economy, and industries 

like yours. 

Dr. Kieval. Yes. Absolutely. We have spent upwards of two 

years negotiating the latest MDUFA agreements with the FDA. 

And so I think there’s a shared sense that if we are able to meet 

those goals, then that will be to the tremendous benefit of patients. 

Certainly within industry we do look-backs and post-mortems on 

our programs and our processes with great regularity. One of the 

challenges that we have working with the agency is that sometimes 

the people who are present at the end of the project are not the 
16 

same people that were present at the beginning of the project. So 

better continuity of oversight of our projects would be very helpful. 

Chairman Brady. Dr. Ellig, your point was, you know, standards up ahead. Make sure you address the issue of agency bias, but 

look-back gives you a chance to really look at the quality of the impact? Is that your position? 

Dr. Ellig. Oh, yes, definitely. And I agree with Dr. Greenstone 

that the ideal look-back is done by someone other than the agency 

itself, to give kind of an independent view. 

Chairman Brady. And, Dr. Greenstone, what is the best way to 

create that process? Because if you do it independently and objectively, I would think it would help. This is not a partisan issue. 

This is really a smarter way to hit a goal. 

Dr. Greenstone. This is exactly an issue of how to best serve 

the American people. I think it would be good to embed in the process the agencies themselves, to engage in look-back. But I think it 

is very hard for people to be completely objective. 

I find it hard to be objective about myself sometimes, and I think 

having my wife around to tell me the truth sometimes is useful. 

Sometimes painful, but useful. And I think having an outside agency or institution, a CBO-like thing for regulation, would be really 

quite effective and would help even improve the analyses that 

agencies do themselves. 

Chairman Brady. My wife’s look-back on me is a continual improvement process. 

[Laughter.] 

And very timely, let me add. Thank you all, very much. Senator 

Klobuchar. 

Vice Chair Klobuchar. Thank you very much, all of you. 

Dr. Greenstone, just to follow up on the Chairman’s question 

there with your idea of having this automatic review, which I think 

is a good one, you would see this—you suggested it possibly could 

be each agency, but there are concerns that they would not probably be able to unmoor themselves from their closeness to having 

to enforce the regulation. So do you think it would be something 

that Congress would set up, then, to look back at it? 

And how do we do it efficiently? You know, my concern is we are 

going to have new regulations that come out as we confront new 

problems as a Nation, and you want to make sure that those get 

done in a speedy manner instead of people getting hung out to dry 

for years. So how would we be able to do this efficiently? 

Dr. Greenstone. With respect to creating an independent body 

to do this, I think it would really take a very small amount of 

money to set up a regulatory look-back organization say that could 

be housed in CBO, or it could be independent, but modeled after 

the CBO. 

My own view is that that should be combined with increased 

look-backs within government itself. And so one way to do that 

would be to build on the Executive Orders that the President put 

out. I assume that there’s also a Legislative way to mandate that. 

In forcing agencies to do it themselves, I think there is a little 

bit of the man ̃ ana problem, they will always want to do it tomorrow. And I think putting some trigger in there to force them to 

comply by a certain period I think would be effective. 
17 

Vice Chair Klobuchar. Very good. The World Bank produces 

their annual report about doing business. They actually look at 185 

of the world’s economies, and they look at their ease of doing business, their regulatory environments. 

In 2013, as well as 2012, the U.S. ranked 4th in ease of doing 

business, and 13th in ease of starting businesses. So we are doing 

some things right. 

However, when you look at our tax system, with the fact that we 

play red light/green light, that it is too complex, we actually rank 

69th out of 185 in that subcategory. 

And I continually get complaints—as I know everyone up here— 

not only about paying taxes, obviously, but also about the fact that 

no one knows what is going to happen year in and year out, and 

that it is too complex. 

Dr. Greenstone, just following up, do you think simplifying our 

Tax Code would benefit economic growth in the long run? 

Dr. Greenstone. I think there are excellent opportunities to do 

some simplification. And I think simplification might put some tax 

lawyers out of business, but I think it would probably be good for 

the overall economy and lead to greater growth. 

Vice Chair Klobuchar. Professor Dudley. 

Professor Dudley. Well, I am not a tax expert, so I would answer that question as a regular person, and concur with both of 

you. 

But if I could add something to your previous question? 

Vice Chair Klobuchar. Sure, the question the Chairman led 

with, yes. 

Professor Dudley. I agree with everything that my colleagues 

have said, because I really do think a Congressional office of regulatory oversight would be valuable for many, many different things. 

But I also think there needs to be a change in the default that 

triggers review. We already have statutes that say look back at 

your regulations. The Regulatory Flexibility Act does. And yet 

meaningful retrospective review does not happen for all the reasons 

that Michael mentioned. 

So the default somehow needs to change so that a regulation will 

expire unless you have shown it is having the effect it should have. 

Vice Chair Klobuchar. Okay. Thank you. 

Dr. Kieval, thanks again for coming and giving us some practical 

perspective on all of this. I also note, in addition to Representative 

Paulsen, Senator Coats is here. We co-chair the Medical Device 

Caucus in the U.S. Senate and he has done a lot of work in this 

area. 

Could you speak, Dr. Kieval, to the FDA’s culture and regulatory 

climate in recent years? You mentioned some proposals you had, 

but where have you seen improvement? And where do you—if you 

could just run it, where do you think we could see the most dramatic improvements? 

Dr. Kieval. Well I think the regulatory climate has become more 

challenging over the past several years. And I think that is reflected in the financing climate becoming more challenging, as well, 

because investors often cite regulatory concerns as some of the biggest inhibitors of their investment. 
18 

So, clearly the FDA has a focus on ensuring patients’ safety, and 

you can never be too sure that a product is safe, and that is an 

important part of the mission, but another important part of the 

mission is making sure that promising new products do reach patients in a timely manner. And we think that there are opportunities to continue to focus on that side of the equation. 

The agreements in MDUFA we think are going to increase greatly the efficiency, which is going to translate into tremendous cost 

savings for industry. And I think what Congress can do is just keep 

both parties accountable to the goals that have been set under 

MDUFA and FDASIA, and I think that will be to our mutual advantage. 

Vice Chair Klobuchar. In looking at the Medical Device Tax 

issue, and we are obviously, many of us up here would like to see 

it repealed and have been working—Senator Hatch and I have a 

working group working on how we could pay for that, and figuring 

out some practical ways it could actually get through Congress. 

Is it true that Excise Taxes are applied to the sales of medical 

device firms whether they are profitable or not? And doesn’t that 

mean that these companies may be forced to pay federal taxes even 

though they are not making a profit in many cases because in this 

area where the up-front costs are so high they have put a lot of 

money into research and development? 

Dr. Kieval. Yes, that is absolutely true. Our company has raised about $200 million of capital that we have invested and continue 

to invest in research and development and clinical trials. We probably will not reach profitability until we are gaining revenues of, 

you know, $70 to $100 million per year, as we recoup that investment and defray our ongoing costs. Yet we would be responsible to 

start paying the taxes as soon as the first U.S. dollar [of revenue] 

crosses the threshold. 

So as we think about continued investments in innovation, expanding manufacturing capabilities, and keeping high-paying jobs, 

we are already planning for the impact of the device tax that that 

is going to have on us. 

Vice Chair Klobuchar. Thank you, very much. 

Chairman Brady. Thank you. 

Representative Paulsen. 

Representative Paulsen. Thank you, Mr. Chairman. 

I will just follow up a little. I will start out by mentioning that, 

boy, when I’m in Minnesota I get a chance to tour a lot of Minnesota companies, and Senator Klobuchar and I have been quite 

frequently visiting our medical device companies in particular. 

And at a lot of these companies we get a chance to tour. They’re 

small business. They’re manufacturers. And so I have had a chance 

to talk with floor managers, the sea-level folks working on the 

floor, as well as the business owners. And the issue about uncertainty and the increasing regulatory burden comes up frequently as 

a part of those conversations. 

In fact, there was a recent study that was just done on the state 

of manufacturing in Minnesota on a survey that was done: 60 percent of businesses in manufacturing are very concerned about government policies and regulations. So it is not just even concerning 

the tax side of the equation, but it’s the regulatory environment. 
19 

So having that fear of onerous regulation does not create an environment that allows companies to invest with certainty in their 

people or their equipment, certainly. And that is definitely one factor that is a part of this Growth Gap which the Joint Economic 

Committee is focused on. 

Mr. Kieval, I just want to follow up. The medical device industry 

is absolutely an American success story. You know that. Senator 

Klobuchar and I both know it. Senator Coats knows that. And we 

represent heavy sectors in that area. 

But at the same time, I think we know that that leadership is 

threatened. And in particular, as Senator Klobuchar noted and you 

mentioned in your testimony, there is a recent study with the U.S. 

Venture Capital. Funding has declined and it continues to decline 

today. So it essentially is going to mean more movement towards 

Europe, or toward other countries for these new start-ups for 

break-through technologies. That harms patients, obviously. 

So knowing that the trend is going in that direction, and knowing that medical devices and technology get approved in Europe 

faster than it gets approved here, what are some ways as a technology officer, for instance, that maybe we should be looking down 

the road for, how can technology play a role in making sure we are 

doing regulation smarter from a technology perspective, and at the 

same time protecting from a safety perspective? 

Dr. Kieval. These are a part of the realities of what we are dealing with on a daily basis. And we, under the guidance of our boards 

of directors and our investors, have to make these investment decisions very carefully about whether to pursue products in the U.S. 

or outside the U.S. 

You know, we’ve been able to garner our leadership position by 

attracting top talent, and retaining top talent with high-paying 

jobs, and investing heavily in ongoing innovation and expansion of 

manufacturing capabilities. And, with the time and amount of capital required to get our products across the approval threshold in 

the U.S., there is just less money to devote to those value-add activities. 

On top of that, the Medical Device Tax is going to continue to 

divert precious capital away from ongoing innovation and U.S. jobs. 

So I think ensuring that the agreements under MDUFA are 

met—and I think we all expect clinical as well as economic benefits 

to come out of those—but as we have talked about, I think repealing the Medical Device Tax would also go a long way to freeing up 

the greatly needed capital to help us maintain our leadership position. 

Representative Paulsen. Dr. Ellig, I was going to ask a question. Because among the initiatives to assess the quality of federal 

regulation systematically is the Mercatus Regulatory Report Card 

as probably the most comprehensive. 

Your testimony lists a lot of different criteria as a part of that 

Report Card. What is the most important criteria, and why? 

Dr. Ellig. Oh, the most important criterion for a regulatory impact analysis is analysis of the systemic problem that the agency 

is trying to solve. Answering questions like: 

Is there a problem? 
20 

Is it a system-wide problem that you might be able to solve 

through a change in the rules of the game? 

Or is it just some bad behavior by particular bad actors where 

you might take more of a law enforcement approach rather than 

a regulatory approach? 

Is there a problem at all? 

And what is the nature of the problem so that we can then tailor 

a solution that would actually take care of the problem but also 

take care of the problem at minimum cost? 

Representative Paulsen. You have also used the term ‘‘Regulatory Impact Analysis.’’ Is that the same thing as cost/benefit 

analysis? Or does it mean something else? 

Dr. Ellig. Regulatory Impact Analysis is really broader. Regulatory Impact Analysis is an overall analysis that summarizes a lot 

of information about the nature of the problem, the alternative solutions to the problem, and the costs and benefits of the alternative 

solution. 

So really the cost/benefit analysis is one piece of a Regulatory 

Impact Analysis, and it would include economic analysis, but also 

any other science that goes into understanding the problem, understanding the harm, understanding what we are trying to do. 

Representative Paulsen. Well, Mr. Chairman, I know we are 

going to have a lot more conversation about regulatory policies, 

given the President’s proposed some new suggestions on the regulatory environment with energy, for instance, and this is something 

I think this Committee is going to continue to tackle. So I yield 

back. 

Chairman Brady. Thank you. 

Senator Coats. 

Senator Coats. Thank you, Mr. Chairman. 

And thanks to all the witnesses here for your presentations to us. 

I think it is an extremely important issue. I agree with my colleagues here. I think on a bipartisan basis if we are going to address our current economic malaise, or much lower and slower 

growth than we would like to see at this particular point in time 

in our economy, regulatory reform has to be an essential component of that, along with tax reform and budget reform, and particularly entitlement reform. 

And so I appreciate your contribution. I want to state that, the 

panel here is kind of weighted toward the medical device issue. 

And I enjoy co-chairing that Caucus with my colleagues from Minnesota on medical devices. So rather than be repetitive, let me just 

say Indiana says ditto to everything that Senator Klobuchar has 

said, and Congressman Paulsen has said, relative to medical devices. 

We have a big stake in that in Indiana, and it is one of the most, 

no pun intended, cutting-edge industries in our State, providing a 

glimpse into the future of the kind of technology innovation that 

we can accomplish here in the United States, and we don’t want 

to see that shipped overseas on the basis of an egregious tax, or 

failure to bring about some sensible regulations. 

Secondly, I also want to note for the record—and this is pretty 

startling; I don’t know that you need to comment on this—but the 

Small Business Administration a few years ago estimated that the 
21 

cost of complying with federal regulations exceeded $1.75 trillion 

every year—nearly 12 percent of the total GDP. 

Now if that is only half right, we are looking at a staggering cost 

here. And so the application of impact analysis, and sound examination of how we go forward before we regulate is extremely important. 

I want to just ask a couple of questions. Dr. Ellig, I will direct 

this to you: the President’s announcement yesterday relative to 

moving forward on climate control initiatives of course has a dramatic effect on regulations coming out of the EPA. 

What measurements and analysis have you done, if any, relative 

to EPA? And do you have any thoughts at this particular point as 

to where this agency would fall in terms of regulatory impact— 

sound regulatory impact analysis? 

Too often I think ideology tends to drive agency regulatory decisions, as opposed to sound analysis. I am not trying to make a political point here; I am just simply saying if we are going to address 

these problems we need to be more rational and more scientifically 

oriented in terms of making some of these decisions. 

So do you have any thoughts or comments on that? 

Dr. Ellig. Well, in the Regulatory Report Card the environmental regulations actually often tend to rank in about the top 

half. But what that is telling us, since the average is about—the 

average score is about 50 percent—what that is telling us is that 

the EPA, the Department of Energy, some of the other entities that 

issue environmental regulations, have made an investment in Regulatory Impact Analysis and are producing documents that are at 

least, you know, trying to appear to comply. 

In comparison with some other agencies where you read the Regulatory Impact Analysis and you say, ayah, they are not even trying. So sometimes they are some of the better ones, but even the 

better ones are not that great. So it is maybe a more complicated 

answer than, they are great or, they are lousy. 

Comparatively, there are some good examples but the best ones 

still are not that great. 

Senator Coats. Well this recent announcement by the President 

that he is going to use the regulatory process to achieve what a lot 

of us think needed to be legislated is going to have enormous impact on energy costs in this country. 

And so I think sound analysis, to the extent it can be provided 

for us as we examine how to go forward and address this, could be 

very helpful to us. I think the impact of this is going to transcend 

the impact of a lot of regulations that have come down, and it will 

particularly be relevant to different states depending on the source 

of energy that they supply to their people. 

I think my time is about expired, and thank you, Mr. Chairman. 

Chairman Brady. Thank you, Senator. 

Representative Hanna. 

Representative Hanna. In a recent transportation hearing, Administrator of the Federal Motor Carrier Safety Administration, 

Anne Ferro, someone who has worked well with my office, talked 

about hours of service for commercial trucks. This is sort of a reallife example of what we are talking about, and I want to ask you 

about the subjective nature, the culture of the bureaucrats and 
22 

workers behind these regulations, and the disincentives they may 

or may not have to not take risks but rather go through that regulation which offers the most safety for them and perhaps the most 

expensive for the public, not unlike Administrator Lisa Jackson’s, 

to paraphrase, cost is not our problem; in certain regulation, we 

simply do—we are in the business of protecting the environment, 

which I respect. 

The Federal Motor Carrier Safety Administration was working 

on a study. They didn’t finish it. They did not include a study on 

regulation that impacts specific industries—for example, concrete, 

cellphone, trucks, asphalt, local deliveries of aggregates. 

I simply asked the question, or it was asked at the committee 

hearing, how it made their new regulations credible when they had 

not even finished the study yet? And never really got an answer. 

But I wondered, for example, Dr. Greenstone, you have great 

ideas in terms of follow-up and analysis of what the real-world impact is of some of these things, but do you believe there is a cultural bias of, for lack of a better way for me to explain it, of personal protection, and a disincentive to create regulation which is 

more relevant in the real world, as opposed to kind of subjective 

and value-based, which is more inclusive? 

Dr. Greenstone. You know, I can’t speak to the exact issue that 

you mentioned about the Department of Transportation, but what 

I do think is: There’s not enough sunshine in the regulatory process. 

And through this kind of look-back mechanism that I think 

would be a fantastic idea, and through having outside approvals, 

that would produce the sunshine that would allow for a healthier 

examination of some of the tradeoffs that you are talking about. 

Representative Hanna. Dr. Dudley, do you believe that there 

is a bias towards more regulation inside of these organizations, as 

opposed to less, since the more regulation they create, the less risk 

comes to bear on their decision? 

Professor Dudley. I think that is true, and I think it has been 

documented that the incentives are to err on the side of over-regulating. Because there are two types of errors a regulator could 

make. 

If they make a mistake and allow action that later turns out not 

to have been safe, you all are going to haul them up before Congress and they will be berated. But if they take an action that 

maybe inhibits some innovation, that is less visible. That is certainly true in the medical area. 

Representative Hanna. How do you address, in the medical 

area particularly with the FDA and the approval of drugs that can 

be life-saving but long in the tooth, people are dying while they are 

waiting for them, how do you change that culture for anybody out 

there who—any of you four gentlemen and lady—who might have 

an idea? How do you incentivize people to be creative in the process, as opposed to restrictive? 

Professor Dudley. Let me concur with Professor Greenstone 

that the regulatory process is very insular, and there is not enough 

transparency and sunshine. 

We need to bring in the wisdom of crowds, and there are a variety of ways of doing it. It is one of the things that we at GW are 
23 

thinking about. Getting a lot of different perspectives on a regulation, both after it is is in effect, but also before, I think could improve the regulatory process. 

Dr. Ellig. I think one of the things Congress could do through 

oversight is continually asking agencies—and this is whether it is 

regulation, or spending programs, or whatever: 

What is the result of what you did? And what is the evidence 

that you are actually solving problems, and that this is working? 

I mean, a lot of times in this town we get caught up in the idea 

that, well, we’ve solved the problem because we passed a law. We 

solved the problem because we adopted a regulation. Sort of the 

government—the fact that the government engaged in activity is 

taken as proof that the problem is solved. 

Representative Hanna. Check that box. 

Dr. Ellig. Yes, check the box, and whatever. But, through vigorous oversight asking, all right, is the problem actually solved? Is 

the air cleaner? Is the water cleaner? Or whatever particular problem you are trying to deal with. 

Dr. Kieval. As an industry right now our focus is not as much 

on wholesale reform of regulation, but it is on making sure that the 

regulations are applied in a transparent and consistent and a reasonable and predictable manner, and we think that MDFUA is 

going to help that. 

You know, a simplistic comment that I could add is, the FDA is 

highly visible for all of the negative decisions that it makes, and 

they are highly criticized for all the decisions that people think are 

wrong. I think we could do a better job of celebrating the right decisions that they make, when they do make decisions to let new 

technologies out and to begin helping patients. 

And I think they deserve as much visibility for those decisions 

as they do for what we think are the wrong ones. 

Representative Hanna. Thank you. And my time has expired. 

Chairman Brady. Thank you. 

The former Chairman of JEC, Representative Maloney. 

Representative Maloney. Thank you, Mr. Chairman, and Vice 

Chairwoman. 

Mr. Greenstone, you testified that an equivalent of the Office of 

Information and Regulatory Affairs should be set up to provide 

Congress with regulatory impact evaluations. And why does a new 

office need to be created? And why can we not simply assign this 

task to an existing governmental body? 

Senator Coats in his questioning pointed out that we already 

spend $1.7 trillion a year in regulatory oversight, and regulatory 

impact studies. So why do we need a new office? Why can’t we just 

work through the $1.7 trillion that we are already spending? 

Dr. Greenstone. Thank you for that question, Congresswoman 

Maloney. 

I think actually currently we spend not very much on regulatory 

analysis. The regulations impose lots of costs and introduce lots of 

benefits on the overall economy, and I think that is what the $1.7 

trillion refers to. 

My view on why we need a new body—and I recognize that we 

are in tight budget times—is that currently the Regulatory Impact 

Analysis comes out of the very agencies that are implementing the 
24 

regulations, and it can be hard for people to be objective with 

themselves about what they are doing. 

And so having an outside body like the CBO serves Congress in 

terms of providing independent information. I think it would be an 

effective way both to improve the analyses that are done inside the 

Administration, and also just to provide sunshine that would benefit everyone. 

Representative Maloney. Also could you comment on the 

President’s speech yesterday on carbon pollution? In his speech, the 

President pointed out that Americans are paying for inaction on 

carbon pollution through higher food costs, and higher taxes to pay 

for disaster relief and rebuilding. He also mentioned that certain 

states will need to budget for larger wildfire seasons. And how will 

the EPA take these costs into account in preparing their own cost/ 

benefit analysis? 

Dr. Greenstone. So I think the President’s announcement yesterday is probably, to this Committee’s point, one of the most consequential regulatory actions in many years. And with respect to 

your question, we are already experiencing costs of climate change. 

Hurricane Sandy is an excellent example. And some of the crop 

failures in the last two summers have been linked to climate 

change. 

And I think what the President is saying is, we are already paying those costs. We are paying them in a very indirect way. Let’s 

start to pay them up front, and maybe we will save on net through 

reduced damages in the future. 

Representative Maloney. Well in creating this new body that 

you are advocating for, where does it end? There was a bill in one 

of my committees that I serve on to have more of a cost/benefit 

analysis on top of the cost/benefit analysis already in Dodd-Frank, 

and in the SEC. 

So it seems like if you have all these analyses, it might get to 

the point where you can never do anything. Not only do you have 

the analysis of the committee, you then have the cost/benefit analysis; you then have Judicial review; now you’re proposing another 

regulatory body to have yet another cost/benefit analysis. When 

does it end? And how much would it cost? 

Would it not be more appropriate if you have a problem to ask 

GAO to do a specific analysis of that particular cost? And we’re 

working within the regular framework that is already existing in 

government as opposed to going out and creating yet another layer? 

I fail to understand what is the benefit. If we have a problem, 

look at it. Have the analysis done. Several Presidents—I’ll cite the 

Grace Commission that President Reagan had—have come in and 

looked at particular areas, and particular areas and analyzed them. 

There is now a focus in energy where outside groups and governmental groups are doing more of an analysis of our cost/benefit in 

our energy strategies. 

But to go in and create another complete bureaucratic overlay, 

what is the benefit? Why not focus on the problem and solve it, as 

opposed to creating yet another huge governmental employment 

bureau? 

Dr. Greenstone. So I don’t come here before you today lightly 

to suggest that there be another bureaucracy. I think the things to 
25 

keep in mind are that—I’m just citing the figure other people have 

used—but that the overall impact of regulations on the economy is 

numbered in the trillions of dollars annually. 

And the problem is, although there are many people analyzing 

these regulations, as you point out, none of them, zero point zero, 

analyze them afterwards. And so all the analysis is done up front, 

and it is generally done with very good intentions, but the problem 

is people cannot predict the way things are going to turn out. 

And so the reality often ends up being different than what people 

projected. And so what I am proposing is that this new institution 

is just a small increase in CBO and would provide the kind of information that currently does not exist. 

Again, to put it in context, I believe that this could be funded for $15 to $20 million a year, probably less. And it is a small investment, in my view, relative to the trillions of dollars of impacts that 

the regulations have on the economy currently. 

Representative Maloney. My time is expired. 

Chairman Brady. Thank you. 

Representative Delaney. 

Representative Delaney. Thank you, Chairman Brady, and 

thank you for holding this important hearing. I want to thank all 

the the witnesses here today for their thoughtful comments. 

I think this question about a prospective versus a retrospective 

analysis is really important. And I think, Dr. Greenstone, you 

made the point quite well, which is our ability to do prospective 

analysis on these questions is inherently limited unless we were to 

somehow adopt and embrace and really understand an incredibly 

dynamic model for understanding all the implications of these various regulations which, much to my dismay, it has been very hard 

for us to do that in the CBO on a prospective basis. And I think 

it might be even more complex here, because behavior does change. 

I mean, the Clean Air Act I think—I don’t have a terrific historical perspective, but that was done really for a quality of life perspective. I don’t think people really understand that, depending 

upon the estimates, it has been an 8-to-1 to 25-to-1 savings from 

a pure economic standpoint. 

So I think this notion of an incredibly vigorous, rigorous datadriven retroactive analysis, if you will, done by an independent 

agency is really, really important. My question for you, Dr. 

Greenstone—and then I have a question for Professor Dudley—is: 

Do you anticipate that this entity would really be able to effectively analyze all of the various implications of regulations, including things that are not even anticipated based on the regulatory intent? Some of which may be quite positive under the category of 

unintended consequences. We could have really positive unintended 

consequences, or we could have really negative unintended consequences. Do you think that it can effectively do that? 

Dr. Greenstone. You know, that is a very important question. 

Thank you. No analysis is perfect. I feel very confident that doing 

the first bit of ex post analysis has got to be an improvement over 

doing none. 

I train graduate students. When Chairman Brady mentioned— 

and I wrote this down—that he was interested in doing a better job 

of quantifying the costs and benefits of regulations, you know, their 
26 

hearts were aflutter with the notion that people might be interested in that. 

[Laughter.] 

And I think having an agency, or a group of people devoted to 

that, who spend their lives on that—and they could be former graduate students of mine—would I think produce a lot of information 

that could help shape the regulatory process going forward in a 

way that better delivers benefits. 

Representative Delaney. And I assume it could be very transparent. Again, unlike some of the things we sometimes see in CBO, 

which it’s not quite clear how they came up with their answers, 

this, because it is a retrospective analysis, I would think we would 

be able to have detailed disclosure as to what inputs went in, and 

what analysis was done, which I think would also help inform new 

regulations even if we don’t do as much prospective analysis—because I generally think all regulation should be held to a higher 

standard. But we have to recognize that prospectively it is hard to 

figure some of this stuff out, whereas on a retrospective basis we 

ought to have absolute answers as to whether these things are actually working. 

And my question, maybe, for Professor Dudley in the same theme 

is: How do we think about overlapping regulations as it relates to 

this? Because my background is in the financial services industry, 

which prior to the crisis was subject to four banking regulators, 

and now has three banking regulators. And if you look at regulations, they put forth or promulgate individually, you’ll do a certain 

cost/benefit analysis. But if you actually weave our somewhat byzantine overlapping, in my judgment, banking regulatory approach 

to the market, you would conclude that the effects of various regulations are producing a different cost/benefit analysis than they 

would individually. 

How do you think about these decisions in that context? Because 

it is not just individual regulations; it is regulations working together towards the same goal. Do you think this capability could 

help us in some of that? Because that to me seems to be some of 

the low hanging fruit. Here we have multiple people trying to do 

the same thing. There’s turf wars, et cetera. Whereas, if we could 

kind of level-set all of these things and look at them individually, 

look at their effect together, we could actually come up with a really good analysis of what we should be doing. 

Professor Dudley. Well I agree. That is something that Presidents have assigned to the Office of Information and Regulatory Affairs. When they review agencies’ regulations, part of that review 

is: Are they doing the analysis they should? But part of it is interagency review, OIRA shares draft regulations with other agencies 

to look for conflict, or overlap. 

What is missing is your area, the financial regulations. The independent regulatory agencies don’t have the value of OIRA’s interagency review. 

Representative Delaney. Right. 

Professor Dudley. Which is another argument for the Congressional regulatory oversight. Because not only could that help you 

evaluate regulations, but as you develop legislation that office 
27 

would have that institutional knowledge to know what other agencies are doing similar things. 

Representative Delaney. So it could look at individual regulations, and then it could look at a holistic regulatory approach to 

problems we want to regulate. 

Like we clearly as a country want to regulate the banking industry. So we should look at not only individual banking regulations, 

but if the various banking—I use banking as an example; it applies 

to other areas, obviously, the economy, the quilt that we’ve weaved, 

if you will, for regulations, whether it’s actually achieved so it could 

do both. I think it is a great idea, and it is something we should 

be advancing hard. 

So thank you for your testimony. 

Chairman Brady. I want to thank all of you. Regulation reminds me, when my first son was 5 years old, putting the group 

of five-year-olds on a soccer field. It doesn’t actually mean you see 

a soccer game. 

[Laughter.] 

The way they run together in packs, and it doesn’t even resemble 

it. Adding regulation upon regulation does not always mean you 

achieve a goal. And so having independent, objective, high-quality 

analysis up front, removing the bias, having standards we can comment on, a look-back that is reliable to improve the next round 

seems to me to be areas of common ground as we go forward. 

I want to thank, on behalf of Vice Chair Klobuchar, each of the 

witnesses today. Very, very helpful, thoughtful, and insightful testimony and answers, as well. So thank you all very much for being 

here today. 

The hearing is adjourned. 

(Whereupon, at 11:15 a.m., Wednesday, June 26, 2013, the hearing was adjourned.) 

(29) 

SUBMISSIONS FOR THE RECORD 
30 

PREPARED STATEMENT OF HON. KEVIN BRADY, CHAIRMAN, JOINT ECONOMIC

COMMITTEE

This month the current recovery celebrates its fourth anniversary. Now is a good 

time to assess how the U.S. economy is performing. 

Unfortunately for American families, the current recovery remains the weakest 

since World War II. There is a troubling Growth Gap in economic performance between this recovery and the average of post-war recoveries, leaving our economy 

four million private sector jobs and $1.2 trillion short. While Wall Street is booming, 

every man, woman and child in America is missing nearly $3,000 in real disposable 

income due to the Growth Gap. 

During this Congress, the Joint Economic Committee has been examining the 

causes of the Growth Gap and the types of alternative policies to close that gap. 

The JEC has studied how current fiscal and monetary policies have held back this 

recovery. Today, the JEC will explore regulatory policy. 

From town hall meetings with my constituents in Texas to conversations with 

business leaders and economists across America, there is one consistent message: 

Uncertainty over the costs of new regulations in healthcare, the environment, labor 

issues and financial services is suppressing business investment and the creation of 

new jobs along Main Street. 

The burden of federal regulations is large. At year-end 2012, the Code of Federal 

Regulations had 238 volumes and 174,545 pages. 

That burden is growing. In 2012, the Federal Register—which publishes proposed 

new rules and regulations, final rules and changes to existing regulations—totaled 

78,961 pages. Three of four highest page counts since the Federal Register began 

publication have occurred during the Obama presidency. 

And that burden is costly. NERA Economic Consulting, in a study last year commissioned by Manufacturers Alliance for Productivity and Innovation (MAPI), estimates the current direct cost of compliance with ‘‘major’’ regulations—those with an 

estimated cost greater than $100 million per year—issued between 1993 and 2011 

to be between $265 billion and $726 billion per year. Clyde Wayne Crews of the 

Competitive Enterprise Institute estimates the total cost of regulation in America 

approaches $1.8 trillion annually—or nearly 12% of GDP. 

Given this historically weak recovery, the rise of technology to help us meet regulatory goals more cheaply and a shared belief that America should continue progress 

on a clean environment and safe workplace, when regulations are necessary doesn’t 

the public deserve the most effective regulation at the least cost? 

Smart regulations that improve the market process and its incentive structure to 

accelerate progress rather than dictate particular outcomes will prove superior to 

tens of thousands of pages of mandated rules and micro-managed instructions. 

Devising process-enhancing rules that engage the private sector’s versatility and 

creativity require objective upfront analysis and thoughtful design. Yet federal agencies often do things the other way around—deciding first what they want to do and 

then using whatever analysis is performed to justify their preconceived ‘‘solution.’’ 

This abuse must stop. 

In 1981, President Reagan issued an executive order requiring executive branch 

agencies to conduct Regulatory Impact Analysis, commonly known as cost-benefit 

analysis, before issuing major new regulations. This first step toward smarter regulation had its limitations. 

An executive order affects only executive branch regulatory agencies and therefore 

does not affect independent regulatory agencies such as the Consumer Product Safety Commission, the Federal Trade Commission, the Federal Reserve, and the Consumer Financial Protection Board. 

Over the years, Congress has exempted broad swaths of federal regulation from 

the scrutiny of cost-benefit analysis through provisions of the Clean Air Act, for example. While there are government-wide ‘‘best practice’’ standards on how agencies 

should conduct cost-benefit analysis, they are not uniformly applied and are not legally binding. The quality of agency cost-benefit analyses varies greatly. 

Agency bureaucrats are naturally biased toward their proposed regulation and 

have learned how to manipulate cost-benefit analysis to justify whatever new regulations they wish to issue. For example, former Administrator of the Office of Information and Regulatory Affairs, professor John Graham, closely examined Corporate 

Average Fuel Economy (CAFE) standards for trucks in his testimony before the 

House Committee on Oversight and Government Reform in September 2011 and 

found that to inflate the benefits of their new rule, regulators had cut the discount 

rate and the so-called ‘‘rebound effect’’ of increased driving with better mileage to 

half or less. He also found that they failed to carefully consider the rule’s effects 

on vehicle size, performance and safety. 
31 

In other words, today too few proposed rules are fully analyzed. There are too 

many loopholes, no uniform requirement across all agencies, a lack of standards 

with which to conduct the analysis, no check-and-balance against agency bias, no 

comparison of past analysis to real life impacts and little recognition of the total 

burdens on the economy of regulation. 

We must do better. The purpose of this hearing is to discover ways in which Congress can make the regulatory process ‘‘smarter,’’ more cost effective and better designed to accomplish the goals without damaging the economy. 

In particular, the Committee hopes to hear from today’s witnesses about the deficiencies in cost-benefit analysis as it is now practiced and how agencies can do a 

better job of quantifying and measuring the costs and benefits of both proposed and 

existing regulations. I look forward to the testimonies. 
32 

THE GEORGE WASHINGTON UNIVERSITY 

WASHINGTON, DC 

Prepared Statement of Susan E. Dudley 

Director, GW Regulatory Studies Center 

Research Professor, 

Trachtenberg School of Public Policy and Public Administration 

Hearing on 

Reducing Unnecessary and Costly Red Tape 

through Smarter Regulations 

Before the 

Joint Economic Committee 

June 26, 2013 

I]@ OW Regulatory Studies Center I www.RegulatoryStudies.gwu.edu I Follow us @RegStudies 
33 

Prepared Statement of Professor Susan E. Dudley, Joint Economic Committee, June 26, 2013 

Statement of Professor Susan E. Dudley 

Chairman Brady, Vice-Chairman Klobuchar, and distinguished members of the Committee, 

thank you for inviting me to testify today on smarter regulations. I am Director of the George 

Washington University Regulatory Studies Center, and Research Professor in the Trachtenberg 

School of Public Policy and Public Administration.' From April 2007 to January 2009, I 

oversaw executive branch regulations of the federal government as Administrator of the Office 

ofInformation and Regulatory Affairs (OIRA) in the Office of Management and Budget (OMB). 

I have studied regulations and their effects for over three decades, from perspectives in 

government (as both a career civil servant and political appointee), the academy, non-profit 

organizations, and consulting. 

In the 125 years since Congress created the first regulatory agency,2 the number of regulatory 

agencies and the scope and reach ofthe regulations they issue has increased significantly. In 

2013, there are over 70 federal agencies, employing over 300,000 people to write and implement 

regulation. 3 Every year, they issue thousands of new regulations, which now occupy over 

168,000 pages of regulatory code. For over a century, concerns over the accountability of what 

some have called the "fourth branch" of government have led all three branches of government 

to take steps to exercise checks and balances on the development and enforcement of 

regulations. 4 

The Legislative Branch 

Past congresses have passed several overarching laws governing regulatory practice, and 

establishing factors the executive branch must evaluate, information it must provide, and 

procedures for third-party review of regulations. Some of the most important regulatory review 

laws of the last few decades include: 

1 The George Washington University Regulatory Studies Center aims to improve regulatory policy through 

research, education, and outreach. This statement reflects my views, and does not represent an official position 

of the GW Regulatory Studies Center or the George Washington University. 

The Interstate Commerce Act established the Interstate Commerce Commission in 1887 to regulate railroad rates 

http://www.ourdocuments.gov/doc.php0flash=true&doc~49&page~pdf 

l Susan Dudley & Melinda Warren, FISCAL STALEMATE REFLECTED IN REGULATORS' BUDGET: AN ANALYSIS OF 

THE U,S. BUDGET fOR FISCAL YEARS 2012 AND 2013. The George Washington University Regulatory Studies 

Center and the Weidenbawn Center on the Economy, Government, and Public Policy. (2012) available at 

http://research.columbian.gwu.edu/regulatorvstudiesisites/defaultifilesi u41 iRegulators Budget 20 12,pdf Note 

that "agencies that primarily perform taxation, entitlement, procurement, subsidy, and credit /\mctions are 

excluded from this report," so these figures exclude staff developing and administering regulations in the Internal 

Revenue Service, the Centers for Medicaid and Medicare Services, etc, 

, Elena Kagan, Presidential Administration, 114 HARV, L. REv. 2245 (2001). 

The George Washington University Regulatory Studies Center www.RegulatoryStudies,gvr'u,edu 2 
34 

Prepared Statement of Professor Susan E. Dudley, Joint Economic Committee, June 26,2013 

• the Regulatory Flexibility Act (RF A) of 1980, which requires agencies to assess the 

impact of a regulation on small businesses and provides for review by the Small Business 

Office of Advocacy. 5 

• the Paperwork Reduction Act (PRA) of 1980 (amended in 1995), which established 

OIRA within the OMB to review the paperwork and information collection burdens 

imposed by the federal government. 6 

• the Unfunded Mandates Reform Act (UMRA) of 1995, which limits regulatory agencies' 

ability to place burdens on state, local, and tribal governments 7 

• the Small Business Regulatory Enforcement Fairness Act (SBREF A) of 1996, which 

enforces requirements for small business impact analyses under the RF A. 8 

• the Congressional Review Act (CRA) of 1996,9 contained in the SBREF A, which 

requires rule-issuing agencies to send all mandated documentation that is submitted to the 

OMB to both houses of Congress as well. It also allows Congress to overturn regulations 

within a specified time with a congressional resolution of disapproval. 

• the Omnibus Consolidated and Emergency Supplemental Appropriations Act of 1999 

(section 638(a», which requires the OMB to report to Congress yearly on the costs and 

benefits of regulations and to provide recommendations for reform. 10 

the Truth in Regulating Act of 2000, which gives Congress the authority to request that 

the GAO conduct an independent evaluation of economically significant rules at the 

proposed or final stages. II 

the Information Quality Act of 2000, which required the OMB to develop governmentwide standards for ensuring and maximizing the quality of information disseminated by 

federal agencies. Under the guidelines, agencies must follow procedures for ensuring the 

utility, integrity, and objectivity of information used in rulemaking and elsewhere. They 

also must offer an administrative mechanism for responding to public requests to correct 

poor-quality information that has been or is being disseminated. 12 

5 Available at: http://wv..W.sbaonline.sba.gov/advo/lawsllawJib.html 

6 Available at: http://wv.'W.archives.gov/federal-register/laws/paperwork-reductionl 

7 Available at: hltp:llwww.gsa.gov/portal/contcnt/245277 

8 Available at: http://www.sba.gov/advocacv/825i12186 

9 Available at: http;/l\V\vw.archives.uov/federal-rcgister/laws/congressional-revie\vl 

100MB's annual reports are available at: http://www.whitehouse.gov/omb/inforeg regpol reports cOl1&'fessl 

II Available at: http://www.gpo.gov/fdsys/pkg!PLAW-I06pubI3I2/html!PLAW-I06pubI3I2.htm 

12 Available at: http://v.ww.gpo.gov/fdsvs/pkg!PLAW-I06pubI554ipdf/PLA W-I06pubI554.pdf 

The George Washington University Regulatory Studies Center www.Re2:ulatorvStudies.Q:wu.edu 
35 

Prepared Statement of Professor Susan E. Dudley, Joint Economic Committee, June 26,2013 

These efforts have had mixed results. Agencies generally meet UMRA requirements with 

reference to regulatory impact analyses prepared pursuant to Executive Order 12866, but rarely 

do more.]3 While pursuant to the RF A and SBREF A, courts have overturned regulations that 

fail to consider impacts on small business, 14 agencies have successfully defended regulations that 

ignore the RFA requirements if the regulation's effects on small entities are considered to be 

"indirect." 15,16 Congress has used the CRA to enact a resolution of disapproval only once, 

overturning an OSHA regulation addressing ergonomics in the workplace. 17 

OMB reports annually to Congress on the costs and benefits of major regulations, but a 200 I 

Congressional Research Service report observed that OMB's reports, "have been incomplete, 

and its benefits estimates have been questioned.,·18 My own research corroborates those 

concerns, and shows that a large percentage of total reported benefit estimates are driven by a 

few questionable assumptions. 19 The General Accounting Office20 and others21 have noted that 

13 See testimony of Susan Dudley and other witnesses before the House Subcommittee on Technology, Information 

Policy, Intergovernmental Relations and Procurement Reform, Committee on Oversight and Government 

Reform, February 15,2011, available at 

http:// oversight.house.gov/index.php?option=com _ content&view=arti cl e&id~ I I 29:qunfunded-mandates-andregulatory-overreachq&catid= 14: subcommittee-on-technology 

14 Northwest Mining Association v. Babbitt 5 F.Supp. 2"d 9 (D.D.C. 1998), and Southern l'ishing Association vs. 

Daley, 995 F.Supp. 1411 (M.D. Fla. 1998). 

15 American Trucking Assns v. EPA 175 F.3d 1027, 1043 (D.c. Cir 1999) 

16 Jeffrey J. Polich, Judicial Review and the Small Business Regulatory Enforcement Fairness Act: An Early 

Examination afWhen and Where Judges Are Using Their Newly Granted Power over Federal Regulatmy 

Agencies, 41 Wm. & Mary L. Rev. 1425 (2000). 

17 While several resolutions of disapproval have passed one house of Congress, only one joint resolution of 

disapproval has passed both. It overturned an OSHA regulation addressing ergonomics in the workplace. Though 

resolutions of disapproval require only a simple majority in Congress, they filce the threat of presidential veto, 

which would require a two-thirds majority to override. The conditions surrounding the ergonomics regulation 

were likely key to its disapproval. It was a "midnight regulation" issued amid much controversy at the end of the 

Clinton administration. The resolution disapproving the rule came at the beginning of the Bush administration 

(which did not support the rule), eliminating the veto threat. Richard S. Beth, Disapproval of Regulations by 

Congress, Congressional Research Service (2011). Available at http://w\Vw.senate.gov/CRSReportsicrspublish.cfm'?pid=%)70EO/o2C*P\ %3D'%2?P~/o70%20%0A. Susan E. Dudley testimony 

http://judiciarv.house.govihearings!pdflDudley022820 I I . pdf 

18 Rogelio Garcia, Congo Research Serv., IB95035, Federal Regulatory Relorm: An Overview (200 I), available at 

http://www.thecre.com/pdJl2002-crs.pdt: 

19 Susan E. Dudley, "OMB's Reported Benefits of Regulation: Too Good to be TrueT Regulation. Vol. 36 No.2 

(2013) available at: http://www.cato.org/sites/cato.org/files/serials/files/regulation/2013/6/regulation-v36n2-

4.pdf 

20 U.S. Gen. Accounting Office, GAO/GGD-99-59, Analysis ofOMB's Reports on the Costs and Benefits of 

Federal Regulation (1999), available at http://www.gao.gov/arehivelI999/gg99059.pdf. 

21 Susan E. Dudley, Perpetuating Puffery: An Analysis of the Composition of OMB's Reported Benefits of 

Regulation, Business Economics (2012) 47, 165-176. doi:10.l057/be.2012.14. 

The George Washington University Regulatory Studies Center www.RegulatoryStudies.Qwu.edu 4 
36 

Prepared Statement of Professor Susan E. Dudley, Joint Economic Committee, June 26, 2013 

it is difficult for OMB to report objectively on estimates of regulatory benefits and costs. As 

discussed under Recommendations below, additional efforts are needed to ensure meaningful 

analysis of regulatory consequences. 

The Executive Branch 

The executive branch has also made efforts to improve regulatory analysis, accountability, and 

outcomes. President Carter built on initiatives of Presidents Nixon and Ford to create procedures 

for analyzing the impact of new regulations and minimizing their burdens,22 and every 

subsequent president has expanded executive oversight of regulatory agency activities. (See 

table below.) 

Executive Order 12866,23 issued by President Clinton in 1993, continues to guide the 

development and review of regulations today. E.O. 12866, like its predecessor E.O. 12291 

(issued by President Reagan), expresses the philosophy that regulations should (1) address a 

"compelling public need, such as material failures of private markets"; (2) be based on an 

assessment of "all costs and benefits of available regulatory alternatives, including the alternative 

of not regulating"; and (3) "maximize net benefits" to society unless otherwise constrained by 

law. 

E.O. 12866 requires, among other things, that a regulatory analysis be performed on all rules 

deemed to be of significant economic impact (i.e., that have an effect of$1 00 million or more in 

a year). The regulatory analysis must include a statement of need for the regulation, an 

assessment of alternative regulatory approaches, and a benefit- cost analysis. 

Like presidents before him, President Obama has reinforced and expanded the principles and 

practices of regulatory analysis and executive oversight. He retained OIRA, and its staff of 

under fifty career civil servants who operate within the Executive Office of the President, 

reviewing regulations to ensure they are consistent with the President's priorities, and 

coordinating interagency review to avoid redundancy and conflict. With its mission to ensure 

regulations' benefits justify their costs, OIRA plays an important role. It is institutionally more 

interested in impacts on society broadly and less susceptible to special interest pressures than line 

22 President Carter's E.O. 12044 required agency heads to determine the need for a regulation, evaluate the direct 

and indirect effects ofaltematives, and choose the least burdensome. Exec. Order No. 12044,43 fed. Reg. 12661 

(Mar. 24, 1978). 

23 Available at: http://www. whitehouse.govisitesidefault!fiIesiomb/inforeg!eo 1 2866/co 1 2866 I0041993.pdf 

The George Washington University Regulatory Studies Center www.RegulatoryStudics.gwu,cdu 
37 

Prepared Statement of Professor Susan E. Dudley, Joint Economic Committee, June 26,2013 

agencies,24 and provides what President Obama has called "a dispassionate and analytical 

'second opinion' on agency actions.,,25 

Executive Order 13563,26 issued in January 2011, reaffirmed the regulatory principles and 

practices that have been in effect since 1981. 27 It reinforced E.O. 12866 and stressed the 

importance of conducting sound analysis of likely regulatory impacts, of providing public 

opportunities to engage in the process of developing new regulations, and of designing lessburdensome, more flexible approaches to achieve regulatory goals. It also required agencies to 

develop plans for periodically reviewing regulations already on the books, with an eye toward 

streamlining, repealing, or expanding them to make them more effective and less burdensome. 

E.O. 13579, issued in July 2011, encouraged independent regulatory agencies to comply with 

E.O. 13563 requirements "concerning public participation, integration and innovation, flexible 

approaches, and science," to the extent permitted by law. E.O. 13579 also said that these 

agencies "should consider how best to promote retrospective analysis of rules that may be 

outmoded, ineffective, insufficient, or excessively burdensome, and to modify, streamline, 

expand, or repeal them in accordance with what has been learned," and make such information 

pUblic. 28 

While these executive branch efforts have done little to slow the growth in new regulation, they 

have focused attention on understanding the effects of regulations, and some argue they have 

resulted in "smarter regulation" that produces more benefits than costS.29 Ultimately, however, 

statements of principles from the President are not enforceable in court, and will accomplish 

little unless the President is willing and able to enforce them in practice. 

24 Susan E. Dudley, "Regulatory Reform: Lessons Learned, Challenges Ahead ... Regulation, Vol. 32, Number 2, 

Summer 2009. available at http://www.cato.orglpubs/regulation/regv32n2lv32n2-I.pdf 

2S Memorandum of January 30, 2009-Regulatory Review, 74 Fed. Reg. 5977 (Jan. 30, 2009), available at 

http://www .reginfo.gov/publ ic/jsp/EO/fedRegReview/POTUS _ Memo_on _Regulatory _ Review.pdf. 

26 Exec. Order No. 13563,76 Fed. Reg. 3821 (Jan. 21, 2011). 

27 Press Release, The White House, Fact Sheet: The President's Regulatory Strategy (Jan. 18,201 I), available at 

http://www.whitehouse.gov/the-press-office/201 1/01/18/fact-sheet-presidents-regulatory-strategy. 

28 http://\vww.whitehouse.gov/the-press-office/201 I 107/11/exeelitive-order-reglilation-and-independent-regulatoryagencies 

29 See, for example, John D. Graham, Paul R. Noe, and Elizabeth L. Branch, Managing the Regulatory State: The 

Experience of the Bush Administration, Fordham L. Rev 33(2005). and Cass Sunstein, Smarter Regulation: 

Remarks from Cass Sunstein, AdLawRev 63 (20 II) 

The George Washington University Regulatory Studies Center www.ReQulatorvStudies.Qwu.edu 6 
38 

Prepared Statement of Professor Susan E. Dudley, Joint Economic Committee, June 26,2013 

Executive Orders on Regulatory Analysis and OversightJO 

Executive Title President Date Signed 

Order 

EO 12044 "Improving Government Regulations" (revoked by Carter March 1978 

EO 12291) 

EO 12174 "Paperwork" (revoked b)' EO 12291) Carter November 1979 

EO 12291 "Federal Regulation" (revoked by EO 12866) Reagan y~bruary 1981 

EO 12498 "Regulatory Planning Process" (revoked by EO Reagan January 1985 

12866) 

EO 12866 "Regulatory Planning and Review" (amended by Clinton September 1993 

EO 13258) 

EO 13258 "Amending Executive Order 12866 on Regulatory .G. W. Bush February 2002 

Planning and Review" (revoked by EO 13497) 

EO 13422 "Further Amendment to Executive Order 12866 on G. W. Bush January 2007 

Regulatory Planning and Review" (revoked by EO 

13497) 

EO 13497 "Revocation of Certain Executive Orders Obama January 2009 

Concerning Regulatory Planning and Review" 

EO 13563 "Improving Regulation and Reglliatory Review" Obama Januar)'~2211 

EO 13579 i "Regulation and Independent Regulatory Agencies" Obama Jul)' 201 I 

EO 13609 '''Promoting International Regulatory Cooperation" Obama May 2012 

EO 13610 "Identifying and Reducing Regulatory Burdens" Obama May 2012 

Recommendations for Improving Regulatory Policy 

Recent Congresses have considered legislation to improve the quality of regulations, and make 

them more accountable to the American people. I have evaluated the possible consequences of 

different legislative initiatives elsewhere. 31 This section discusses several categories of reform 

that may prove useful. 

30 www.RegulatoryS(udies.gwu.edu 

31 Susan Dudley, "Prospects for Regulatory Reform," Engage Vol 12, Issue 1 (2011), available at: http://www.fedsoc.org/dodibi20 II 0603 DudlcyEngage 12.I.pdf, and prepared statement before the Senate Homeland Security 

and Government Affairs Committee, July 20 II, available at: 

http://www.tspppa.gwu.edu!docs/20110720~tes(imonLdudley.pdf 

The George Washington University Regulatory Studies Center www.RcgulatoryStudies,gwu.edu 7 
39 

Prepared Statement of Professor Susan E. Dudley, Joint Economic Committee, June 26, 2013 

Improving Regulatory Impact Analysis 

One focus of regulatory reform legislation has been on improving the quality of the analysis 

agencies conduct before issuing regulation. Given that presidents of both parties for over 30 

years have supported ex ante impact analysis of regulations, the creation ofa statutory obligation 

for doing so is probably not necessary to ensure continued analysis, however, codifying the 

requirements could have several advantages. 

• First, such legislation would lend Congressional support to these nonpartisan principles 

and the philosophy that before issuing regulations agencies should identify a compelling 

public need, evaluate the likely effects of alternative regulatory approaches, and select the 

alternative that provides the greatest net benefit to Americans. 32 

• Second, legislation could apply these requirements to independent agencies (which 

Administrations have been reluctant to do through executive order for fear of stirring up 

debate over the relationship between independent agencies and the President). Senators 

Portman, Warner and Collins have recently introduced the Independent Agency 

Regulatory Analysis Act of2013, which has received bipartisan support from former 

OIRA administrators, former heads of independent regulatory agencies, and legal 

academics. 33 

• Third, Congress could make compliance with them judicially reviewable. Judicial review 

could be valuable, not because the courts have a particular expertise in regulatory 

analysis, but because agencies tend to take more seriously aspects of their mission that 

are subject to litigation. Like executive and Congressional oversight, judicial oversight 

would likely make regulatory agencies more accountablc for better decisions based on 

better analysis. (Judicial review is discussed further below.) 

32 Section l(a) of Executive Order 12866 states the regulatory philosophy as follows: "Federal agencies should 

promulgate only such regulations as are required by law, are necessary to interpret the law, or are made 

necessary by compelling public need, such as material failures of private markets to protect or improve the health 

and safety of the public, the environment, or the well-being of the American people. to deciding whether and 

how to regulate, agencies should assess all costs and benefits of available regulatory alternatives, including the 

alternative of not regulating. Costs and benefits shall be understood to include both quantifiable measures (to the 

fullest extent that these can be usefully estimated) and qualitative measures of costs and benefits that are difficult 

to quantitY, but nevertheless essential to consider. Further, in choosing among alternative regulatory approaches, 

agencies should select those approaches that maximize net benefits (including potential economic, 

environmental, public health and safety, and other advantages; distributive impacts; and equity), unless a statute 

requires another regulatory approach." 

33 The legislation and letters of support are available at: http://www.portman.senate.Qovipublic!illdex.cfinipressreI eases"1 D~dd 88927 5-da5) -4 7 64-b2c9-fU2ab26 fe88 I 

The George Washington University Regulatory Studies Center www.RegulatoryStudies.gwu.edu 
40 

Prepared Statement of Professor Susan E. Dudley, Joint Economic Committee, June 26, 2013 

Requiring better regulatory impaet analysis before regulations are issued is important, but will 

not guarantee "smarter regulation" for several reasons. 

• First, unless the cross-cutting analytical requirements supersede the decision criteria 

expressed in individual authorizing statutes, such as Section 109 of the Clean Air Act,34 

many regulations will continue to be based on limited information. Statutes that ignore 

or explicitly prohibit analysis oftradeoffs lead to regulations with questionable benefits 

that divert scarce resources from morc pressing issues. 35 

• Second, ex ante regulatory impact analysis necessarily rests on hypotheses of how the 

regulatory action will alter outcomes and what they will cost. It is easy for regulators to 

fall prey to thc "planner's paradox,,36 without appreciating that efforts to address 

perceived problems often have unintended consequences. Planned solutions always look 

better on paper than unplanned solutions, because the planner sees only his "data, 

assumptions, biases, and understandings ofthc way the world works .... All of the unseen 

difficulties with the planned solution the data, assumptions, biases, and understandings 

of the world that turn out to be wrong are invisible to the analyst because the data he 

considers are his own." Even the most carefully analyzed regulations may result in 

unanticipated changes in behavior that undermine the desired effects of the regulation. 

• Third, agencies have strong incentives to demonstrate through analysis that their desired 

regulations will result in benefits that exceed costs. Regulatory impact analyses are often 

developed after decisions are made and used to justifY, rather than inform, them. In 

principle, a benefit-cost analysis should be "complete." It should include all the significant 

consequences of a policy decision: direct and indirect, intended and unintended, beneficial 

and harmfuL In practice, all such analyses must to some degree fall short of completeness. 

My review of agencies' analyses as reported through OMB's annual reports suggests that 

regulatory agencies are not approaching the problem objectively. On the benefit side of the 

equation, they quantify or list every conceivable good thing that they can attribute to a 

decision to issue new regulations, while on the cost side they only consider the most 

14 The Administrative Conference of the United States has conducted studies and provided recommendations on 

applications of these decision criteria that the Committee may find useful, including: 79-4 Cost-Benefit Analysis 

in Regulatory Decision-Making; 85-2 Regulatory Analysis of Agency Rules; 88-9 Presidential Review of 

Agency Rulemaking [60 Fed. Reg. 56312 (Nov 8,1995)]; and Paul Verkuil, A Critical Guide to the Regulatory 

Flexibility Act, Duke L.J. 213 (1982). 

15 See Susan Dudley and George Gray, "Improving the Use of Science to Inform Environmental Regulation," in 

Institutions and Incentives in Regulatory Science, Lexington Books, Jason Johnston ed. (20[2) 

36 Brian Mannix, "The Planners' Paradox," Regulation, Summer 2013, available at: 

http://www.cato.org/sites/cato.orglfiles/serialsifiles/regulation!2003l71mannix.pdf 

The George Washington University Regulatory Studies Center w-vvw.RcQulatorvStudies.Qwu.edu 9 
41 

Prepared Statement of Professor Susan E. Dudley, Joint Economic Committee. June 26, 2013 

obvious direct and intended costs of complying with the regulation. 37 (This is a problem of 

"confirmation bias," discussed further below.) 

Only Congress can address the first problem by amending language in existing legislation that 

precludes reliance on sound decision criteria or hinders APA procedures (such as requirements 

that agencies issue interim final regulations that limit public comment).38 New statutes that 

authorize executive agencies to issue regulations should require them to conduct careful analysis 

of likely effects, both intended and unintended, and provide them adequate time to research, 

deliberate, and consult with the public before issuing new rules. 

A minimum step toward addressing the second problem would be to require agencies to present 

evidence that the identified problem requires a federal regulatory solution, as well as an objective 

evaluation of alternative solutions. To this end, it is essential that analytical requirements not be 

limited to conducting benefit-cost analysis, but rather capture the broader philosophy and 

principles articulated in E.O. 12866. Legislation should require that regulatory decisions be 

based on the identification ofa compelling public need (a material failure of private markets), an 

objective review of alternatives (including the alternative of not regulating), and an 

understanding of the distributional impacts of different approaches. 

To truly address the latter two problems, however, institutional changes are needed to alter 

incentives for conducting analysis and making decisions. Despite requirements for public 

comment and practices for executive oversight, regulatory decision-making is often insulated 

from different perspectives. Regulatory agency staff are smart and motivated, but, like everyone 

else, they are susceptible to what behavioral psychologists call "confirmation bias,,·39 and their 

single-mission focus leads them to discount data, research, values and perspectives that do not 

corroborate their preferred regulatory action. As a result, as Justice Stephen Breyer observed in 

his 1993 book Breaking the Vicious Circle, "well-meaning, intelligent regulators, trying to carry 

out their regulatory tasks sensibly, can nonetheless bring about counterproductive results.,,40 

Breyer referred to this institutional phenomenon as "tunnel vision," where agencies single37 See Susan Dudley, "Perpetuating Puffery: An Analysis of the Composition ofOMB's Reported Benefits of 

Regulation," Business Economics Vol. 47, No.3, August 2012. 

38 See Susan Dudley, "GAO Report: Agencies Circumvent Public Comment on Major Rules," available at: 

http://research .colum bi an. Qwu.edu/reQulatorvstudi es! sites! defaultlfi I esiu41 IGA 0%20report% )0 Dudley. pd rand 

Sofie Miller, "What the Unified Agenda Tells Us About Notice and Comment Rulemaking," available at: 

http://research. columbian .gwu.edu/regulatorystudies/sitesl defaultlfi les/20 12agenda _ Mi lIer. pdf 

39 For a short description of confirmation bias, see http://skepdic.com/confirmbias.html. 

40 Stephen Breyer, Breaking the Vicious Circle: Toward Effective Risk Regulation, Harvard University Press, 1993. 

The George Washington University Regulatory Studies Center \vww.RegulatorvStudies.llwu.edu 10 
42 

Prepared Statement of Professor Susan E. Dudley, Joint Economic Committee, June 26, 2013 

mindedly pursue a particular goal to a point that "the regulatory action imposes high costs 

without achieving significant additional safety benefits.,,41 

Institutional changes that provide for more checks and balances, and harness the "wisdom of 

crowds,,42 are needed to counter these natural incentives. As discussed below, congressional 

oversight, judicial oversight, and opportunities for public involvement could provide greater 

accountability and improve the reasoning underlying regulatory decisions as well as the 

decisions themselves. 

Enhanced Congressional Oversight 

Executive branch oversight of regulatory actions has proven valuable, but it is not sufficient. 43 

Congress may also want to consider legislation that would strengthen its own ability to control 

regulation. One approach would require a Congressional vote before major new regulations can 

become effective, and another would establish a Congressional office to review and evaluate 

regulations. 

Congressional Approval of New Rules 

The Regulations from the Executive In Need of Scrutiny (REINS) Act44 would provide a tool for 

Congress to "increase accountability for and transparency in the federal regulatory process.,,45 It 

is patterned after the 1996 CRA, providing expedited procedures for evaluating and voting on 

major regulations, but it changes the default outcome. Rather than requiring Congress to enact a 

')oint resolution of disapproval" to prevent a rule from going into effect, no major rule could go 

into effect until Congress enacted an affirmative ')oint resolution of approval.,,46 

41 Susan E. Dudley, "Regulatory Reform: Lessons Learned, Challenges Ahead," Regulation, Vol. 32, Number 2, 

Summer 2009, available at http://www.cato.orglpubs/regulation/regv32n2lv32n2-I.pdf 

" James Surowiecki, The Wisdom of Crowds. Anchor Books, 2005. 

4J Susan Dudley, "Congress Needs its own Regulatory Oversight Office," Penn RegBlog, 

http://www.law.upenn.edulblogs/regblog/2011/0S/congress·needs·its·awn·regulatory.review.office.html(20 11) 

44 S. 15 available at: http://beta.congress.gov/billIl13th-congress/senate-biIll15 

45 Regulations from the Executive in Need of Scrutiny Act, H.R 10, 112th Congo § 2 (20 II). 

46 See my analysis of the advantages and disadvantages of REINS in a GW Regulatory Studies Center working 

paper available at: 

http://research.columbian.g\\u.eduirel!ulatorystudies/sites/defaultifilesiu38/regreform dudley workingpaper 20 

110405.pdf 

The George Washington University Regulatory Studies Center \vww.Regu!atoryStudies.Qwu.edu 11 
43 

Prepared Statement of Professor Susan E. Dudley, Joint Economic Committee, June 26, 2013 

Congressional Regulatory Oversight Office 

A Congressional office responsible for reviewing regulations would have several benefits. 47 

Most importantly, it would serve as an independent check on the analysis and decisions of 

regulatory agencies and DIRA. 48 While a Congressional office would not have the same 

authority DIRA exercises to affect agency draft regulations, it would be able to devote resources 

to areas OIRA does not, such as examining the effects of regulations issued by independent 

regulatory agencies. Just as the CBO provides independent estimates of the on-budget costs of 

legislation and federal programs, a Congressional regulatory office could provide Congress and 

the public independent analysis regarding thc likely off-budget effects oflegislation and 

regulation. 49 

Judicial Branch Oversight of Regulation 

Under the AP A, after a regulatory agency issues a final rule, an affected party may challenge it 

in court. Reviewing courts may reverse or remand the rule to the agency for reconsideration on 

constitutional grounds, on procedural grounds (whether the agency followed the procedures 

specified in the APA), or on the basis of the agency's interpretation of the authorizing statute. 

Recent courts have overturned several regulations of the Securities and Exchange Commission as 

being arbitrary and capricious and in violation of the APA, finding that compliance with the 

Commission's statutory criteria demanded a more rigorous analysis of benefits and costs to 

evaluate the rule's effects on efficiency, competition, and capital formation. 50 

Courts defer to agency expertise when evaluating regulatory records,sl however, and presidential 

executive orders governing regulatory impact analysis have stated that their requirements are not 

47 See Testimony of Robert W. Hahn and Robert E. Litan before the House Government Reform Committee, 

Subcommittee on Energy Policy, Natural Resources and Regulatory Affairs, March 2003, available at: 

http://www.brookings.edu/testimony/1999/04Jighttoknow_litan.aspx 

48 GAO noted "It is politically difficult for OMB to provide an independent assessment and analysis of the 

administration's own estimates in a public report to Congress. If Congress wants an independent assessment of 

executive agencies' regulatory costs and benefits, it may have to look outside of the executive branch or outside 

ofthe federal government." U.S. Gen. Accounting Office, GAO/GGD-99-59, Analysis ofOMB's Reports on the 

Costs and Benefits of Federal Regulation (1999), available at http://www.gao.goviarchiveIl999/gg99059.pdf. 

49 Susan Dudley, "Congress Needs its own Regulatory Oversight Office," Penn RegBlog, 

http://www.law.upenn.edu/blogs/rcgblogi201 1 i08icongress-needs-its-own-re2ulatory-rcview-officc.html (20 11) 

50 For a discussion of recent cases, see Jane Luxton, An Uncomfortable Wake-Up Call For Dodd-Frank Regulators, 

Futures & Derivatives Law Report (Vol. 32, Issue I) available at: 

http://www.pepperlaw.com/publications.update.aspx?ArticleKey-~2296 

51 In Chevron U.SA., Inc. v. Natural Resources Defense Council, Inc., the Supreme Court established a two-step test 

for determining whether to grant deference to a government agency's interpretation of a statute. Under the second 

The George Washington University Regulatory Studies Center www.RegulatoryStudies.g\vtl.edu 12 
44 

Prepared Statement of Professor Susan E. Dudley. Joint Economic Committee, June 26. 2013 

enforceable by law. 52 The Regulatory Accountability Act attempts to alter the deference to 

agencies by subjecting regulations issued under APA notice-and-comment rulemaking 

procedures to a "substantial evidence" standard of judicial review. which directs a reviewing 

court to set aside an agency action unless the record provides "such relevant evidence as a 

reasonable person would accept as adequate to support a conclusion.,·s3 This is arguably a more 

exacting standard than "arbitrary and capricious" standard of review. 54 

The small business community has been frustrated that courts have interpreted the Regulatory 

Flexibility Act's requirements to assess economic impact as applying only to direct compliance 

costs. They argue that agencies should consider reasonably foreseeable indirect economic 

impacts on small entities, such as increases in input prices (e.g., electricity, natural gas, or 

transportation) or state-level regulations issued pursuant to federal rules. This latter issue is 

particularly important for environmental regulations, where the "duty of regulating is passed on 

to the states without any corresponding analysis or requirements for states to consider less 

burdensome alternatives for small business.'·55 

Enhanced Public Input 

It is popular to talk about the possibility of using modern techno logy to improve regulatory 

policy by engaging the wisdom of crowds. While there are some promising ideas on this front, it 

is important to keep in mind that the most powerful technology for effectively using the 

decentralized wisdom of crowds is a very old one: the market. 56 There are countless 

opportunities to improve regulatory policy by giving greater deference to the wisdom of the 

market. Regulatory agencies continue to issue energy efficiency standards for appliances, 

step, ifCongressiona1 intent is not clear, "the issue for the court is whether the agency's answer is based on a 

permissible construction of the statute." Chevron U.S.A. v. NROC, 467 U.S. 837 (1984). 

" See EO 12866 Sec. 11. "Judicial Review. Nothing in this Executive ordcr shall affect any otherwise available 

judicial review of agency action. This Executive order is intended only to improve the internal management of 

the Federal Government and does not create any right or benefit, substantive or procedural, enforceable at law or 

equity by a party against the United States, its agencies or instrumc'l1talities, its officers or employees, or any 

other person." 

53 Mareno v. Apfel, 1999 U.S. Disl. LEXIS 8575 (S.D. Ala. Apr. 8, 1999) ("more than a scintilla but less than 

preponderance"). 

54 5 U.s.c. § 706(2)(A). 55 Hearing on Legislation to Improve the Regulatory Flexibility Act B~fore the H. Comm on Small Business, 11 Oth 

Congo (2007) (testimony of Thomas Sullivan, Chief Counsel for Advocacy, Small Business Administration), 

available at http://archive.sba.gov/advo/laws/test07 _1206.html. 

S6 See John O. i\.fcGinnis, Accelerating Democracy: Transforming Governance through Techn%gy. Princeton 

University Press, 2013. 

The George Washington University Regulatory Studies Center ww\v.RegulatoryStudics.gwu.cdu 13 
45 

Prepared Statement of Professor Susan E. Dudley, Joint Economic Committee, June 26, 2013 

vehicles, and businesses, all of which depend on a falsehood: the theory that regulators know 

more than consumers about consumers' own welfare. 57 

Apart from greater reliance on the market, engaging the wisdom of crowds to improve regulatory 

outcomes could take several forms. Requiring prc-rulemaking disclosure of key information 

related to problem formulation, risk assessment, and impact analysis, would engage broad public 

comment on the proper choice of studies, models, assumptions, etc. long before any policy 

decisions are framed, and positions established. 58 The bicameral Regulatory Accountability Act 

(RAA)59 would further these objectives by amending the Administrative Procedure Act to codifY 

and extend some of the analytical requirements in presidential executive orders and also provide 

for more extensive opportunities for public involvement, particularly for rules designated as 

"high impact." Depending on their expected impacts, rules and guidance documents would be 

subject to procedures beyond the notice and comment procedures currently embodied in the 

AP A, and could bring greater transparency to the basis for regulatory decisions, and engage 

broader public input earlier in the regulatory process. 

Incentives to Reexamine Existing Regulations 

Most legislative and executive branch reforms have focused on analyzing and improving new 

regulations, and agencies seldom look back to evaluate whether existing regulations are having 

their intended effects. Initiatives to require ex post evaluation of regulations that are in effect 

have met with limited success60 largely because they did not change the underlying incentives. 

Several initiatives would seek to alter those incentives. Senator Angus King is seriously 

considering a proposal by the Progressive Policy Institute for a Regulatory Improvement 

Commission, patterned after the Base Realignment and Closing Commission, that would review 

public recommendations for removing existing regulations and present a package 

recommendation to Congress for an up or down vote. 61 

57 Susan E. Dudley, Perpetuating Puffery: An Analysis of the Composition ofOMB's Reported Benefits of 

Regulation, Business Economics (2012) 47, 165-176. doi: I 0.1 057/be.20 12.14. 

58 See Dudley and Gray 2012 for more ideas related to engaging a wide range of resources to expand regulatory 

information. 

59 S. 1029 (available at: http://www.govtrack.us/congress/billslI 13is I 029/text) and H.R. 2122 (available at: 

http://mvw.govtrack.uslcongress/bills/I13/hr21?7) 

GO Susan E. Dudley prepared statement before the Senate Homeland Security and Government Affairs Committee, 

July 20 II, available at: http://www.tspppa.gwu.edu/docs/20110720 _testimony_dudley. pdf 

61 Based on discussion with Sen. King's staff. The Progressive Policy Institute's analysis ofthe problem of 

accumulating regulation and its proposal is available at: http://mvw.progressivepolicy.om!2013l0S/re,zulatorvimprovement-commission-a-politica!lv*viable-approach4o·u~s*regulatory-reform/ 

The George Washington University Regulatory Studies Center www.RegliiatoryStlidies.!S\vu.edu 14 
46 

Prepared Statement of Professor Susan E. Dudley. Joint Economic Committee, June 26,2013 

Congress is considering using budgeting concepts to alter regulatory agencies' incentives to issue 

new regulations and examine the effectiveness of existing regulations. 62 The United Kingdom's 

"one-in-one-out" approach to regulation forces agencies to make tradeoffs when issuing new 

regulation,63 and members of the U.S. Senate are considering similar legislation. 64 Under a strict 

"regulatory paygo" or "one-in-one-out" approach, regulatory agencies would be required to 

eliminate an outdated or duplicative regulation before issuing a new regulation of the same 

approximate economic impact. While subject to analytical challenges, this has the potential to 

impose some needed discipline on regulatory agencies, and to generate a constructive debate on 

the real impacts of regulations. 65 

The Regulatory Accountability Act would require all final rules to include a plan for review at 

least every 10 years, to "determine whether, based upon evidence, there remains a need for the 

rule, whether the rule is in fact achieving statutory objectives, whether the rule's benefits 

continue to justify its costs, and whether the rule can be modified or rescinded to reduce costs 

while continuing to achieve statutory objectives.,,66 

Conclusion 

Regulatory impact analysis is a longstanding and important element of US regulatory policy, but 

a variety of institutional obstacles prevent regulatory impact analysis from being a silver bullet 

for producing smarter regulation. As long as agencies themselves conduct the analysis, selected 

actions will suffer from confirmation bias. Analyses are often used to justify, rather than inform, 

decisions, and intentionally or unintentionally become tools for advocacy (by agencies and 

others). Current procedures do not provide other participants incentives to invest in careful, 

objective analysis, nor to conduct ex-post evaluation of regulatory outcomes (or empirical 

verification of ex ante estimates of impacts). 

Institutional changes that provide for more effective checks and balances, and engage the 

wisdom of crowds are needed to counter these incentives. Greater congressional oversight, 

judicial oversight, and opportunities for public involvement could provide greater accountability 

and improve the reasoning underlying regulatory decisions as well as the decisions themselves. 

62 See statement of Senator Portman before Homeland Security and Government Affairs Committee, June 23, 2011, 

available at http://www.hsgac.senate.gov/heGlingsljederal-regulation-a-review-qf-legislative-proposals-PGl1-i. 

63 http://vvw\v.bis.gov.uklpolicies/bre/one-in~one-out 

64 See statement of Senator Mark Warner before Homeland Security and Government Affairs Committee, June 23, 

20 II, available at http://www.hsgac.senate.gov/hearings/federal-regulation-a-review-of-Iegislative-proposalspart-i. 

65 Susan E. Dudley prepared statement before the Senate Homeland Security and Government Affairs Committee, 

July 2011, available at: http://www.tspppa.gwu.edu!docs/20110nO_testimony-dudley.pdf 

66 Regulatory Accountability Act Sec. 3«J)( 4)(G) 

The George Washington University Regulatory Studies Center www.RegulatoryStudies.gwu.cdu 15 
47 

1Kenneth Chay and Michael Greenstone, ‘‘The Impact of Air Pollution on Infant Mortality: 

Evidence from Geographic Variation in Pollution Shocks Induced by a Recession,’’ Quarterly 

Journal of Economics, 2003, 118(3); Olivier Deschenes, Michael Greenstone and Joseph Shapiro, 

‘‘Defending Against Environmental Insults: Drugs, Emergencies, Mortality and the NOx Budget 

Program Emissions Market,’’ Department of Economics, MIT (2011). 2Michael Greenstone, ‘‘The Impacts of Environmental Regulations on Industrial Activity: Evidence from the 1970 and 1977 Clean Air Act Amendments and the Census of Manufacturers.’’ 

Journal of Political Economy, 2002, 110(6); Michael Greenstone, John A. List and Chad 

Syverson, ‘‘The Effects of Environmental Regulation on the Competitiveness of U.S. Manufacturing,’’ Department of Economics, MIT (2011). 3 Reed Walker, ‘‘The Transitional Costs of Sectoral Reallocation: Evidence From the Clean Air 

Act and the Workforce,’’ Department of Economics, Columbia University (2011). 

PREPARED STATEMENT OF MICHAEL GREENSTONE, MASSACHUSETTS INSTITUTE OF

TECHNOLOGY, THE HAMILTON PROJECT, THE BROOKINGS INSTITUTION

Thank you Chairman Brady, Vice Chair Klobuchar, and members of the Joint 

Economic Committee for inviting me to speak today. 

My name is Michael Greenstone, and I am the 3M Professor of Environmental Economics at the Massachusetts Institute of Technology, the Director of the Hamilton 

Project, and a Senior Fellow at the Brookings Institution. My research focuses on 

estimating the costs and benefits of environmental quality, with a particular emphasis on the impacts of government regulations. 

I appreciate the opportunity to speak with you today about opportunities to improve the government’s regulatory system. Under all economic circumstances, regulatory efficiency and clarity are crucial objectives for the credibility and predictability of the government’s role in the marketplace. However, given the continuing 

weak economic environment, it is absolutely essential to design a regulatory structure that protects the wellbeing of our citizens without imposing unnecessary costs 

on American businesses and society as a whole. 

We can achieve these objectives without compromising our values in key areas 

ranging from the protection of public health to the supervision of financial markets 

by ensuring that the Executive and Legislative branches have the tools of analysis 

and measurement they need to review current and proposed regulations. The purpose of my testimony is to describe in concrete terms how this can be accomplished. 

INTRODUCTION

American government, at every level, regulates a broad array of social and economic life. Regulatory policy determines the air we breathe, the quality of the water 

we drink, the materials we use to construct our homes, the cars we buy, the safety 

of our workplaces, the investments we make, and much more. Government regulates 

these activities because in cases of market failures, for example, our free market 

system does not create the necessary incentives for businesses and individuals to 

protect the public good. 

But, in making decisions about regulations, public officials must choose which 

areas of our lives merit government rules, as well as how stringent those rules 

should be. 

The Clean Air Act is a classic example of a regulation with significant benefits 

and costs. Before its passage in 1970, there were few constraints on businesses that 

emitted pollution as a byproduct of their operations. The result was poor air quality. 

As one small example, white collar workers in Gary, Indiana, often brought an extra 

shirt to work because the first would be dirty from the air and unfit to wear by midday. Even more importantly, some of my research, as well as research by others, 

has found that the polluted air led to elevated mortality rates that reduced the lifespans of the American people.1 Obviously, no business sets out to cause these impacts; but, in trying to maximize their profits, it was not in their interests to install 

expensive pollution abatement equipment when their competitors did not. As a result, they did not act to adequately reduce emissions. 

At the same time, the Clean Air Act’s regulations require firms to alter their production processes in ways that raise their costs. Indeed, some of my recent research 

finds that an important set of Clean Air Act rules has raised polluting industries’ 

costs of production by roughly 2.6%. This has reduced firms’ profits and led to higher prices for consumers. Further, it has caused regulated firms to scale back their 

operations, which led to employment losses at those firms.2 Although the ultimate 

effect on the level of jobs in the economy is likely minimal in normal economic 

times, recent research indicates that workers who lose their jobs due to regulations 

often face prolonged periods of unemployment and become reemployed at lower 

wages.3
48 

The challenge then for regulators is to consistently set rules with benefits that 

exceed their costs. 

In a pair of Executive Orders, President Obama has created a framework that has 

the potential to be the most fundamental shift in regulatory policy in more than 

three decades. The Executive Orders require that federal agencies routinely review 

existing significant regulations in order to ‘‘determine whether any such regulations 

should be modified, streamlined, expanded, or repealed’’ with the purpose of making 

the ‘‘regulatory program more effective or less burdensome in achieving the regulatory objectives.’’ These reforms offer the promise of finding a better balance between our health and safety and our economic growth. 

To understand why the president’s efforts are so critical, imagine if the Food and 

Drug Administration approved new drugs without ever having tested them on people—that it approved drugs knowing only in theory how they were likely to affect 

the human body. Further imagine if such drugs remained on the market for years, 

or even decades, without their effects ever being subject to evaluation. This path is 

simply inconceivable, but until recently was how the vast majority of government 

regulations were treated. 

Make no mistake—inadequate regulatory policy can be, as with drug approvals, 

a life-or-death issue because of the significant role regulations play in every aspect 

of our daily lives. 

A bit of history: U.S. regulations used to be designed essentially in the dark. 

Then, in 1981, President Ronald Reagan issued an executive order institutionalizing 

the idea that regulatory action should be implemented only in cases when, among 

other provisions, ‘‘the potential benefits to society for the regulation outweigh the 

potential costs to society.’’ It sounds obvious. But this idea of applying cost-benefit 

analysis in the regulatory arena fundamentally altered the way in which regulations 

were considered. 

In 1993, President Bill Clinton outlined more specific guidelines for prospective 

analysis of cost-benefit trade-offs. And yet, the regulatory review process was still 

operating with one hand tied behind its back. As a general matter, a regulation’s 

likely benefits and costs were considered only before the proposal was enacted—the 

point when we know the least precisely because the regulations are untested. Consequently, prospective estimates of the costs and benefits must rest on many unverifiable and potentially controversial assumptions. 

And, once a regulation passed through a prospective analysis and went on the 

books, it could remain there for decades without any further evaluation. 

Some regulations work out exactly as intended. But some, of course, do not. For 

example, an air pollutant may prove to be more harmful than was originally understood. Or innovation may lead to new and less expensive pollution-abatement technology. In our rapidly changing world, regulations can and should adapt to change. 

President Obama’s Executive Orders take a critical step forward by looking backward. They require that agencies routinely reevaluate the costs and benefits of existing regulations and identify whether the goals of a regulation could be achieved 

through less expensive means. This revolutionary process of retrospective analysis 

offers the promise of finding a better balance between our health and safety and 

our economic growth 

In the remainder of my testimony, I will identify two further changes that would 

increase the chances that our regulatory system consistently produces rules with 

benefits that exceed costs. 

I. EXTENDING EXECUTIVE ORDERS 13563 AND 13610

The first change is to make three reforms that build on Executive Orders 13563 

and 13610. 

First, I recommend institutionalizing the retrospective review of economically significant rules in a public way so that these reviews are automatic in nature. In the 

case of rules that are currently in force, this would mean publicly committing to a 

retrospective analysis of each existing rule within a pre-specified period. This might 

be 5 or 10 years, with the length of time depending on the particulars of the rule 

and the results of any previous reviews. 

In the case of new rules, the implementing agency would be required to announce 

a timetable for review with a maximum allowable amount of time, perhaps 5 or 10 

years, with shorter time periods being preferable. In addition, the agency would be 

required to pre-specify the expected benefits (e.g., reduced child mortality rates) and 

costs (e.g., reduced business profits) so that the terms of the subsequent review 

would be known in advance. The agency would also be required to identify how 

these benefits and costs would be measured, such as the types of data and other 

information that it anticipates being necessary for review. 
49 

4 Congressional Budget Office, ‘‘CBO Testimony: Statement of Robert D. Reischauer, Director, 

Congressional Budget Office, before the Joint Committee on the Organization of Congress’’ 

(1993). http://www.cbo.gov/ftpdocs/105xx/doc10580/1993l06l10lmission.pdf 5Ted Gayer, ‘‘A Better Approach to Environmental Regulation: Getting the Costs and Benefits 

Right,’’ Discussion Paper 2011–06, The Hamilton Project, Brookings Institution (2011). 

Second, the relevant agency should commit to undertaking a new rulemaking 

when the results from the retrospective analysis differ from the benefits and costs 

that were expected prior to the rule’s implementation. As with the retrospective 

analysis, there should be a time limit for conducting the new rulemaking. In cases 

where the realized benefits exceed the costs by a wider margin than expected, there 

may be further opportunities to maximize net benefits. In cases where the rules are 

found to be ineffective or unjustified, agencies should identify ways to modify the 

rules or abandon them. Finally, if the retrospective analysis confirms the original 

expectation of benefits and costs, then there would not be a need for a new rulemaking. 

Third, these efforts would be strengthened if they were accompanied by a triggering mechanism to ensure that retrospective evaluations occur and, when appropriate, for new rulemakings to be undertaken within the prescribed time periods. 

One approach would be for agencies to announce publicly and post on their website 

the deadline for a rule’s review and reconsideration. A stronger approach would be 

for judicial action to compel reviews and rulemaking in the cases where an agency 

has failed to comply with a review timeline or to act upon its results. 

II. A CBO FOR REGULATIONS

The second change is to ensure that the quality of the reviews is commensurate 

with the stakes of getting regulatory policy right. In this spirit, there are some difficulties with the approach I just outlined. Many agencies do not have the staff, expertise, or resources necessary to undertake these reviews. Further, the process of 

self-evaluation is challenging for all organizations, as it requires complete objectivity. Indeed, history is unkind to organizations that fail to get outside reviews of 

their work. 

My recommendation is to establish a new, independent body for regulatory review. This body could be housed within the Legislative Branch, and it could be modeled after the Congressional Budget Office (CBO) or even become a division within 

the existing CBO. 

As you know, before the CBO was established, only the President had a ready 

source of budgetary and economic data and analysis. Congress was forced to largely 

rely on the Office of Management and Budget (OMB) for this sort of information. 

The CBO was invented to level the playing field. Its analyses allow Congress to consider the economic and budgetary implications of new policy ideas. Crucially, the 

CBO also helps Congress evaluate the information that it receives from the Executive Branch.4

The entire budget process has benefited from CBO’s existence. This is a direct result of its independence. The budgetary analyses and proposals of all legislators and 

Executive agencies are now created to a higher standard, knowing that they must 

ultimately stand up to scrutiny by the non-partisan CBO. 

This system of budgetary review and economic analysis could be a model for a 

reorganized regulatory review process. Like the CBO, this new organization would 

reside in the Legislative Branch, and it would be non-partisan. The organization 

would be charged with conducting independent regulatory impact evaluations. Some 

of the organization’s activities would be statutory in nature—for example, automatic 

reviews of economically significant regulations—while other evaluations could be 

performed at the request of Congressional committees and members. 

Such an organization would directly strengthen our regulatory system. Agency 

analyses would benefit from the scrutiny that they would ultimately receive from 

this new, independent organization. Further, the results of the retrospective reviews 

would become part of the agencies’ automatic assessments of their regulations that 

I described above. I believe that providing this type of rigorous, independent review 

would build confidence within the business community and a better sense of transparency. 

Finally, this new organization could help to increase the credibility of the regulatory evaluations by developing an explicit checklist to determine the rigor of regulatory analyses. The checklist should favor randomized control trials, the gold standard in terms of evidence, and natural experiments over models and observational 

studies. A 2011 Hamilton Project paper provides some other ideas for a check list.5
50 

6Office of Information and Regulatory Affairs, Office of Management and Budget, ‘‘2011 Report to Congress on the Benefits and Costs of Federal Regulations and Unfunded Mandates on 

State, Local, and Tribal Entities’’ (2011). 

Such a checklist could also be issued as guidance by the Administration to its agencies. 

Of course, the creation of such a body would require resources, which are difficult 

to come by in our current fiscal environment. However, I think it is extraordinarily 

likely that such an office would pay for itself many times over. To put this in context, the current CBO budget is less than $50 million annually. My best estimate 

is that the new budget for such an organization would be less, perhaps substantially 

so. 

This is a very small amount of money when compared to the potential costs and 

benefits that regulations impose on our economy. Although it is difficult to determine the total number of economically significant regulations that are on the books, 

the Office of Management and Budget reviewed 540 major regulations between 2001 

and 2010,6 which are defined as having an effect of more than $100 million on the 

economy annually—either in costs or benefits. Consequently, it seems safe to conclude that the total costs and benefits of regulations can be measured in the hundreds of billions of dollars annually. It is apparent that we have a lot at stake economically with regard to our regulatory system and the cost of finding out which 

parts are working is almost trivially quite small in comparison. 

By creating a body that can undertake rigorous analysis of the costs and benefits 

of regulation—both ex-ante and ex-post—policymakers will have better tools for protecting those regulations with great benefits for our society, reforming those regulations that impose unnecessary costs, and potentially culling those that no longer 

serve their purpose. 

IV. CONCLUSIONS

In conclusion, our regulatory system is a linchpin of our well-being. It allows us 

to live longer and healthier lives, among many other important impacts. However, 

these important benefits come with direct economic costs. The purpose of my testimony has been to identify some reforms that will help to ensure that our regulatory 

system does its job in the most cost-effective way possible—in which the benefits 

to society exceed the costs. 

To quickly summarize, I propose two key reforms: 

1. Institutionalize a process by which agencies automatically undertake retrospective reviews of regulations and initiate a new rulemaking when the 

results from the retrospective analysis differ from the expected benefits and 

costs. 

2. Create a new, independent body for rigorous, objective regulatory review 

that is modeled on the Congressional Budget Office. 

We live in a rapidly changing economy and need a regulatory review structure 

that evolves to meet the new and different needs of our society. The reforms that 

I have outlined here will allow our regulatory system to consistently produce rules 

with benefits that exceed costs. That would be good for our well-being, and good for 

the American economy. 

Thank you once again for inviting me to participate in this discussion. I will gladly respond to any questions. 
51 

.~. MERCATUS CENTER 

.~. George Mason University 

Bridging the gap between academic ideas and real-world problems 

IMPROVING REGULATORY IMPACT ANALYSIS 

THROUGH PROCESS REFORM 

BY JERRY ELLIG 

Joint Economic Committee 

United States Congress 

Hearing on "Reducing Unnecessary and Costly Red Tape through Smarter Regu!ations" 

June 26, 2013 

TESTIMONY 

Good morning Chairman Brady, Vice Chairman Klobuchar, and members of the committee. Thank you for inviting me to testify today. 

I am an economist and research fellow at the Mercatus Center, a 501(c)(3) research, educational, and outreach 

organization affiliated with George Mason University in Arlington, Virginia. I've previously served as a senior 

economist for this committee and as deputy director of the Office of Policy Planningatthe Federal Trade Commission. My principal research for the last 25 years has focused on the regulatory process, government performance, 

and the effects of government regulation. For these reasons, I'm delighted to testify on today's topic. 

For more than three decades, presidents of both political parties have instructed executive branch agencies to 

conduct Regulatory Impact Analysis when issuing significant regulations. Some independent agencies, such as 

the Securities and Exchange Commission, are required by law to assess the economic effects of their regulations. 

Executive orders and laws requiring economic analysis of regulations reflect a bipartisan consensus that economic 

analysis should inform, but not dictate, regulatory decisions. 

Unfortunately, agencies' Regulatory Impact Analyses are not nearly as informative as they ought to be, and there 

is often scant evidence that agencies utilized the analysis in decision making. These problems have persisted 

through multiple administrations of both political parties. The problem is institutional, not partisan or personal. 

Further improvement in the quality and use of Regulatory Impact Analysis will likely occur only as a result of 

legislative reform of the regulatory process. To achieve improvement, all agencies should be required to conduct 

thorough and objective Regulatory Impact Analysis for major regulations and to explain how the results of the 

analysis informed their decisions. 

Let me elaborate on each of these points. 
52 

WHY REGULATORY IMPACT ANALYSIS IS NECESSARY 

We expect federal regulation to accomplish a lot of important things, such as protecting us from financial fraudsters, preventing workplace injuries, preserving clean air. and deterring terrorist attacks. And regulation also 

requires sacrifices; there is no free lunch. Depending on the regulation, consumers may pay more, workers may 

receive less, our retirement savings may grow more slowly due to reduced corporate profits, and we may have 

less privacy or less personal freedom. Regulatory Impact Analysis is the key tool that makes these tradeoffs more 

transparent to decision makers. Understanding the effects of regulation has to start with sound Regulatory Impact 

Analysis. 

A thorough Regulatory Impact Analysis should do four things: 

1. assess the nature and significance of the problem the agency is trying to solve, so the agency knows 

whether there is a problem that could be solved through regulation and, if so, the agency can tailor a 

solution that will effectively solve the problem; 

2. identify a wide variety of alternative solutions; 

3. define the benefits the agency seeks to achieve in terms of ultimate outcomes that affect citizens' 

quality aftife and assess each alternative's ability to achieve those outcomes; 

4. identifY the good things that regulated entities, consumers, and other stakeholders must sacrifice in 

order to achieve the desired outcomes under each alternative. In economics jargon, these sacrifices arc 

known as "costs;' but just like benefits, costs may involve far more than monetary expenditures. 

Without all of this information, regulation becomes a faith-based initiative. That is, where regulators have discretion under the law, they would be making choices based merely on the hope that good intentions will produce good 

results. Given the enormous influence regulation has on our day-to-day lives, decision makers have a responsibility 

to act based on knowledge of regulation's likely effects, not just good intentions. 

SHORTCOMINGS IN THE QUALITY AND USE OF REGULATORY IMPACT ANALYSIS 

Scholarly research demonstrates that Regulatory Impact Analysis often falls short of the standards articulated in 

executive orders and Office of Management and Budget guidance. More often than not, agencies do not appear to 

use Regulatory Impact Analysis to inform major decisions. Regulatory Impact Analyses often seem to be advocacy 

documents written to justifY decisions that were already made. rather than information that helped regulators 

figure out what to do.' 

The Mercatus Center's Regulatory Report Card provides some of the most recent evidence on the quality and use 

of Regulatory Impact Analysis.' The Regulatory Report Card is a qualitative evaluation of both the quality and 

use of regulatory analysis in federal agencies. The scoring process uses 12 criteria grouped into three categories: 

1. Openness: how easily can a reasonably informed, interested citizen find the analysis. understand it, 

and verifY the underlying assumptions and data? 

2. Analysis: how well does the analysis define and measure the outcomes or benefits the regulation 

seeks to accomplish, define the systemic problem the regulation seeks to solve, identify and assess 

alternatives, and evaluate costs and benefits? 

1. Richard Williams. "The lnfluence of Regulatory Economists in Federal Health and Safety Agencies. ~ Mercatus Working Paper. Arlington, VA: 

Mercatus Center at George Mason University, July 2008, http://mercatus.org/sites/default/files/publication/WP0815_Regulatory%20 

Economists.pdf; Wendy E. Wagner, "The CAIR RIA; Advocacy Dressed up as Policy Analysis," in Reforming Regulatory Impact Analysis, ed. 

Winston Harrington et al. (Washington, DC: Resources for the Future, 2009), 57. 

2. The Report Card methodology and 2008 scoring results are in Jerry Etlig and Patrick Mclaughlin, "The Quality and Use of Regulatory Analysis 

in 2008," Risk Analysis 32, no. 5 (May 20l2): 855-80. Scores for all regulations evaluated in 2008 and subsequent years are available at 

www.mercatu5.org/reportcard. 

MERCATUS CENTER liT GEORGE MASON UNIVERSITY 
53 

3. Use: how much did the analysis affect decisions in the proposed rule. and what provisions did the 

agency make for tracking the rules effectiveness in the future? 

For each criterion. trained evaluators assigned a score ranging from 0 (no useful content) to 5 (comprehensive 

analysis with potential best practices).' Thus, each analysis has the oppnrtunity to earn between 0 and 60 points. 

The Report Card assesses how well a Notice of Proposed Rulemaking and the accompanying Regulatory Impact 

Analysis complies with key principles in Executive Order 12866, which governs regulatory analysis and review.' 

The scores do not assess whether the evaluators agree with the results of the analysis or believe the regulation is 

a good idea. 

The online Report Card database now includes evaluations of every economically significant, prescriptive regulation proposed between 2008 and 2012-a total of 108 regulations.' Figure 1 shows average scores for each of the 

three categories of criteria. In each category, the average falls far short of the maximum possible score of 20 points. 

20 

~ o 15 .... 

'" l! 

~ 10 

t:: 

o 

~ 5 II:: 

o 

Figure 1: Regulatory Report Card scores for 108 

economically significant regulations, 2008-2012 

Openness Analysis Use 

Maximum possible score on each category" 20 points 

Table 1 shows the average scores on each of the 12 criteria, plus the average total score. The average total score 

was just 31.2 out of 60 possible points-barely 50 percent. The highest total score ever achieved was 48 out of 60 

possible points (80 percent), equivalent to a B-. This was the joint Environmental Protection Agency/National 

3. For the first several years, the evaluators were senior Mercatus Center regulatory scholars and graduate students trained in Regulatory Impact 

Analysis. Since 2010, we have developed a nationwide team of economics professors who serve as evaluators in conjunction with senior Merca~ 

tU5 Center regulatory scholars. Biographical information on current evaluators is available at www.mercatus.orglreportcard. 

4. Exec. Order No. 12866, 50 Fed. Reg. 190 (Sept. 30, 1993), 51735-44, http://www.whitehouse.gov/sites/default/files/omb/inforeg 

!e012866!eo12866_10041993.pdf. President Obama reaffirmed Exec. Order No. 12866 in Exec. Order No. 13563, "Improving Regulation and 

Regulatory Review," 76 Fed. Reg. 14 (Jan. 21, 2011), 3821-23. http://www.whitehouse.gov/sites/default/file,/omb/inforegleo12866 

leo13563_01182011.pdf. 

5. "Prescriptive" regulations are what most people think of when they think of regulations: they mandate or prohibit certain activities. This is dis~ 

tinct from budget regulations, which implement federal spending programs or revenue collection measures. The Report Card evaluated budget 

regulations in 2008 and 2009, then discontinued evaluating budget regulations in subsequent years because it was dear the budget regulations 

have much lower-quality analysis. See Patrick A. Mclaughlin and Jerry Ellig, "Does OIRA Review Improve the Quality of Regulatory Impact Analysis? Evidence from the Bush II Administration," Administrative Law Review 63 (2011): 179-202; Jerry Ellig and John Morrall., "Assessing the 

Quality of Regulatory Analysis." (Working Paper No. 10-75, Mercatus Center at George Mason University, Arlington, VA, December 2010). 

MERCATUS CENTER AT GEORGE MASON UNIVERSITY 
54 

Highway Traffic Safety Administration regulation revising Corporate Average Fuel Economy standards proposed 

in 2009. 

Table 1- Regulatory Report Card scores on individual criteria for 108 economically significant regulations, 2008-2012 

Criterion 2008-2012 

Average Score 

Openness 

1. Accessibility 3.7 

2. Data documentation 2.9 

3. Model documentation 2.9 

4. Clarity 3.2 

Analysis 

5. Outcome dennitlon 3.2 

6. Systemic problem 2.2 

7, Alternatives 2.8 

8. Benent~cost analysis 2.6 

Use 

9. Any use of analYSIS 2.2 

10, Cognizance of ne: benefits 25 

11. Measures and goals 13 

12. Retrospective data 1.6 

Tota! 31.2 

Table 1 shows that most of the lowest scores arc for criteria measuring the use of analysis. The broadest Report 

Card criterion measuring use of analysis (Criterion 9) asks whetber the agency claimed or appeared to usc any 

part of the analysis to guide any decisions. As Figure 2 demonstrates, agencies often fail to provide any significant 

evidence tbat any part of the Regulatory Impact Analysis belped inform their decisions. Perhaps the analysis 

affects decisions more frequently tban these statistics suggest, but agencies fail to document this in tbe Notice of 

Proposed Rulemaking or the Regulatory Impact Analysis. If so, then at a minimum tbere is a significant transparency problem. 

MERCATUS CENTER AT GEORGE MASON UNIVERSITY 
55 

Figure 2: Use of RIAs in 108 economically 

significant regulations, 2008-12 

64 21 

Agency explained how 

RIA affected at least 1 

major decision 

ill Agency explained how 

RIA affected a minor 

decision 

No evidence of any use 

provided 

For each Report Card criterion, we have found a few examples of reasonably good quality or use of analysis, These 

are documented in past testimony and in a series of short Mercatus on Policy publications,6 But best practices are 

not widespread. 

Uufortunately, these less-than-stellar Report Card results are consistent with prior published research on Regulatory Impact Analysis. Case studies document instances in which Regulatory Impact Analysis helped improve 

regulatory decisions by providing additional options regulators could consider or unearthing new information 

about benefits or costs of particular modifications to the regulation.' But Government Accountability Office studies and scholarly research reveal that in many cases, Regulatory Impact Analyses are not sufficiently complete to 

serve as a guide to agency decisions. The quality of analysis varies widely, and even the most elaborate analyses 

still have problems: Surveying the scholarly evidence as of 2008, Robert Hahn and Paul Tetlock concluded that 

6. Jerry EUig. "Look Before You leap: Improving Pre-Proposal Regulatory Analysis," Congressional testimony. March 29, 2011, before the Committee on the Judiciary, Subcommittee on the Courts, Commercial, and Administrative Law, US House of Representatives; jerry EUig and James 

Broughel, "Regulation: What's the Problem?, H Mercatus on Policy no. 100 (Nov. 2011); Jerry Ellig and James Broughel, ~Regulatory Alternatives: 

Best and Worst Practices," (Mercatus on Policy, Mercatus Center at George Mason University, Arlington, VA, Feb. 2012); Jerry Ellig and James 

Broughel, "Baselines: A Fundamental Element of Regulatory Impact Analysis," (Mercatus on Policy, Mercatus Center at George Mason University, Arlington, VA, June 2012). 

7. Winston Harrington, Usa Heinzerling, and Richard D. Morgenstern, eds., Reforming Regulatory Impact Analysis (Washington, DC: Resources 

for the Future, 2009); Richard D. Morgenstern, Economic Analyses at EPA: Assessing Regulatory Impact (Washlngton, DC: Resources for the 

Future, 1997); Thomas O. McGarity, Reinventing Rationality: The Role of Regulatory Analysis in the Federal Bureaucracy (New York: Cambridge 

University Press, 1991). 

8. See Art Fraas and Randall Lutter, "The Challenges of Improving the Economic Analysis of Pending Regulations: The Experience of OMB 

Circular A-4," Annual Review of Resource Economics 3 no. 1 (2011): 71-85; Jamie Belcore and Jerry Ellig, "Homeland Security and Regulatory 

Analysis: Are We Safe Yet?," Rutgers Law Journal 40, no. 1 (20OS): 1-96; Robert W. Hahn, Jason Burnett, Yee-Ho L Chan, Elizabeth Mader, and 

Petrea Moyle, "Assessing Regulatory Impact Analyses: The Failure of Agencies to Comply with Executive Order 12,866." Harvard Journal of Law 

and Public Policy 23, no. 3 (2001); 859-71; Robert W. Hahn, and Patrick Dudley, "How Well Does the Government Do Cost-Benefit Analysisr 

Review of Environmental Economics and PoliCY 1, no. 2 (2007): 192-211; Robert W, Hahn, and Robert Utan, "Counting Regulatory Benefits and 

Costs: Lessons for the U.S. and Europe, tt Journal of International Economic Law 8, no. 2 (2005): 473-508; Robert W. Hahn, Randall W. Lutter, 

and W. Kip Viscusi. Do Federal Regulations Reduce Mortality? Washington, DC: AEI-Brookings Joint Center for Regulatory Studies (2000); 

MERCATUS CENTER AT GEORGE MASON UNIVERSITY 
56 

economic analysis has not had much impact, and the general quality of Regulatory Impact Analysis is low.' The 

Mercatus Center's Regulatory Report Card suggests that matters have not improved since then. 

IMPROVEMENT IN THE QUALITY AND USE OF REGULATORY IMPACT ANALYSIS REQUIRES REFORM 

OF THE REGULATORY PROCESS 

The problems identified by the Report Card occurred under both President Bush and President Obama. An econometric analysis that controls for other factors affecting the quality and use of analysis finds that there is no statistically significant difference in Report Card scores between the Bush and Obama administrations, although Bush 

administration regulations that cleared OIRA review after June I, 2008 tended to have lower Report Card scores. 'O 

Previous research by other scholars also finds little variation in the quality of Regulatory Impact Analysis across 

administrations of different parties." Another consistent-but disturbing-pattern is that administrations of both 

parties appear to require less thorough analysis from agencies that are more central to the administration's policy 

priorities. The Bush administration, for example, permitted the Department of Homeland Security to proceed 

with a number of regulations that were accompanied by very incomplete Regulatory Impact Analysis; the Obama 

administration did likewise with the first major regulations implementing the Patient Protection and Affordable 

Care Act.!2 This same pattern appears to occur with other agencies.!' 

The persistence of mediocre Regulatory Impact Analysis across administrations is an institutional problem, not 

a personal or partisan problem. Deficiencies in the quality and use of Regulatory Impact Analysis transcend individuals and administrations. Substantial impetus for improvement in the quality and use of Regulatory Impact 

Analysis will require changes in the regulatory process. 

Because current law rarely requires comprehensive economic analysis to inform regulatory decisions, agencies 

often treat Regulatory Impact Analysis as a paperwork exercise necessary to clear a regulation through OIRA, 

rather than a tool to aid decision making. Instead, they focus on other factors that are viewed as more central to 

ensuring that a regulation gets upheld in court. In the absence of a requirement for Regulatory Impact Analysis, 

tbese factors may actually hamper agencies from considering the pros and cons of a wide variety of alternatives. 

For example, agencies may tend to follow past precedent in designing new regulations because current regulatory 

approaches have already been defended and upheld in court. As a result, one agency economist noted, "We do 

what we always do, just trotting out the same old thing. That's why we don't come up with better regulations; we 

just come up with the same regulations in different areas!'14 

When evaluating regulations for the Regulatory Report Card, we have found that when Congress requires agencies to consider specific factors such as costs or efficiency, they usually do so. Agencies pay attention to what the 

law says they should do, because otherwise a court might vacate the regulation. To improve the quality and use 

Government Accountability Office, Regulatory Reform: Agencies Could Improve Development, Documentation, and Clarity of Regulatory 

Economic Analyses, Report GAO/RCED~98-142 (May 1998); Government Accountability Office, Air Polfution: Information Contained in EPA's 

Regulatory Impact Analyses Can Be Made Clearer, Report GAO/ReED 97-38 (ApriI1997). 

9. RobertW. Hahn and Paul C. Tetlock, "Has Economic Analysis Improved Regulatory Decisions?," Journal of Economic Perspectives 22, no. 1 

(200S): 67-S4. 

10, See Jerry EtHg, Patrick A. Mclaughlin, and John F. Morrall III. "Continuity, Change, and Priorities: The Quality and Use of Regulatory Analysis 

Across U.5. Administrations," Regulation & Governance 7 (2013): 153-73; Jerry EUig, "Midnight Regulation: Decisions in the Dark?, n (Mercatus 

on Policy, Mercatus Center at George Mason University, Arlington, VA, Aug. 2012). 

11. Hahn and Dudley, HHow Well Does the Government Do Cost~Benefit Analysis?" 

12. Be!core and Emg, "Homeland Security and Regulatory Analysis;" Christopher J. Conover and Jerry ElIig, "Beware the Rush to Presumption, 

Part C: A Public Choice Analysis ofthe Affordable Care Act's Interim Final Rules" (Workrng Paper No. 12-03, Mercatus Center at George Mason 

University, Arlington, VA, Jan. 2012); Christopher J. Conover and Jerry Ellig, "Rushed Regulation Reform," (Mercatus on Polley, Mercatus Center 

at George Mason University, Arlington, VA, Jan. 2012). 

13. ElIig, McLaughlin, and Morrall, "Continuity. Change, and Priorities." 

14. Williams, "The Influence of Regulatory Economists," 5. 

MERCATUS CENTER AT GEORGE MASON UNIVERS!TY 
57 

of Regulatory Impact Analysis, therefore, Congress could require federal agencies to conduct thorough Regulatory Impact Analysis before they write and propose significant regulations. The most obvious method would be 

a legislative requirement for Regulatory Impact Analysis coupled with judicial review. To enforce the law, judges 

need not engage in benefit-cost balancing or second-guess agency expertise. They would merely need to check 

that the agency's analysis covered the topics specified in the law (such as analysis of the systemic problem, development of alternatives, and assessment of benefits and costs of alternatives), ensure that the analysis included 

the quality of evidence required by the legislation, and ensure that the agency explained how the results of the 

analysis affected its decisions. 

Independent agencies are not currently subject to the executive orders on regulatory analysis and review. Some, 

such as the Securities and Exchange Commission, are required by law to conduct economic analysis when determining whether their regulations are in the public interest. Others have no such requirement. Scholarly research 

has found that many independent agencies conduct even less thorough economic analysis than executive branch 

agencies. 15 Requiring independent agencies to conduct Regulatory Impact Analysis and explain how they used 

it in decisions would likely improve their quality and use of analysis. Many of the independent agencies, such as 

the Consumer Product Safety Commission, Federal Trade Commission, Federal Communications Commission, 

and Consumer Financial Protection Board, deal with similar kinds of financial risk, physical risk, and consumer 

protection questions that executive branch agencies address in their assigned spheres of competence, so I see no 

reason a Regulatory Impact Analysis requirement could not apply to them as well. 

Thank you for your time, and I look forward to your questions. 

15. Arthur Fraas and Randall L lutter, "On the Economic Analysis of Regulations at Independent Regulatory Commissions," Administrative Law 

Review 63 (2011): 213-41. 

MERCATUS CENTER AT GEORGE MASON UNIVERSITY 
58 

Risk Analysis DOl: IO.lll lIj.1539-6924.2011.01715.x 

The Quality and Use of Regulatory Analysis in 2008 

Jerry ElIig1,. and Patrick A. McLaughlin!,3 

This article ass.esses the quality and apparent use of regulatory analysis for economically significant regulations proposed by federal agencies in 2008. A nine-member research team used 

a six-point (0--5) scale to evaluate regulatory analyses according to criteria drawn from Executive Order 12866 and Office of Management and Budget Circular A-4. Principal findings include: (1) the average quality of regulatory analysis. though not high, is somewhat better than 

previous regulatory scorecards have shown; (2) quality varies widely; (3) biggest strengths are 

accessibility and clarity; (4) biggest weaknesses are analysis of the systemic problem and retrospective analysis; (5) budget or "transfer" regulations usually receive low-quality analysis; 

(6) a minority of the regulations contain evidence that the agency used the analysis in significant decisions; (7) quality of analysis is positively correlated with the apparent use of the 

analysis in regulatory decisions; and (8) greater diffusion of best practices could significantly 

improve the overall quality of regulatory analysis, 

KEY WORDS: Benefit-cost; cost-benefit analysis; regulation; regulatory impact analysis; regulatory 

process; regulatory reform 

1. INTRODUCTION 

Since 1974, all presidents have issued executive 

orders requiring regulatory agencies to analyze the 

anticipated effects of proposed regulations. President Obama's Executive Order 13563 reaffirms the 

principles and review processes in Executive Order 

12866, which has guided regulatory analysis since 

1993.(1·2) 

Scholars, decisionmakers, interest groups, and 

advocates spill much ink debating whether and how 

agencies should do regulatory analysis. Some view 

regulatory analysis as an imperfect but necessary 

tool for understanding regulation's effects.(3) OthlMcrcatus Center at George Mason University, Arlington, VA, 

USA. 

2Federal Raiload Administration, U,S. Department of Transportation, Washington, DC, USA. 

3George Washington University's Regulatory Studies Center, 

Washington, DC, USA 

* Addres; correspondence to Jerry EJlig, Mercatus Center at 

George Mason University, 3351 N. Fairfax Dr., 4th Floor, 

Arlington, VA 22201, USA; tel: 703-993-4925; jcllig@gmu.edu. 

1 

ers see regulatory analysis as an attempt to "stack 

the deck" against new regulations, arguing that costs 

are easier to measure than benefits (pp. 35-36).14.

5) 

Yet others regard regulatory analysis as a tool for 

crafting "smart" regulations that do more good than 

harm.(6.7) Some view the whole enterprise as a fundamentally immoral attempt to put prices on public values that cannot be assigned monetary worth 

(pp.61-62).(4) 

Nonetheless, many scholars agree that regulatory analysis is here to stay.(J·6. 8-12) Executive Order 13563 provides further evidence of this. Yet as 

former Office of Information and Regulatory Affairs 

(OIRA) Administrator Sally Katzen noted, "we still 

do not know whether the agencies are implementing CBA [cost-benefit analysis] appropriately and 

whether the way the agencies usc CBA produces better regulatory decisions" (pp. 1314-1315).(11) 

This article takes up that challenge. We apply 

a 12-point qualitative framework to evaluate regulatory analyses of "economically significant" rules 

that were reviewed by OIRA in 2008 and proposed 

0272-4332/11/0100-0001$22.00111) 2011 Society for Risk Analysis 
59 

2 

in the Federal Register4 The evaluation criteria are 

drawn from Executive Order 12866 and Office of 

Management and Budget (OMB) Circular A-4, the 

2003 guidance document on best practices in regulatoryanalysis.(l3) 

Our evaluation yields numerous insights into the 

quality and use of regulatory analysis. Principal findings include: (1) the average quality of regulatory 

analysis is not high; (2) quality varies widely; (3) the 

biggest strengths in the analyses are accessibility 

and clarity; (4) the biggest weaknesses are analysis 

of the systemic problem and retrospective analysis; 

(5) budget or "transfer" regulations receive much 

lower quality analysis than other regulations; (6) the 

agency claimed to use the analysis in significant decisions for a minority of the regulations; (7) quality of 

analysis is positively correlated with the apparent use 

of the analysis in regulatory decisions; and (8) greater 

diffusion of best practices could significantly improve 

the overall quality of regulatory analysis. 

2. EXISTING LITERATURE AND 

OUR APPROACH 

Several strands of scholarly literature assess the 

quality of federal regulatory analysis. Some assessments are in-depth case studies, whereas others apply 

quantitative scoring methods to numerous regulatory 

analyses. 

2.1. Case Studies 

Case studies suggest that even extensive economic analyses of regulations can have significant 

flaws. They also find that regulatory impact analysis 

has had marginal effects on some regulations, but 

rarely if ever drives major decisions. 

McGarity presents five case studies from the 

Reagan administration.(l4) Some arc success stories, 

but they also reveal shortcomings in monetizing benefits, gathering reliable cost data, explaining large uncertainties in estimates, or identifying a wide range 

of regulatory options. Similarly, Fraas credits good 

4"Economically significant" regulations are defined as regulations 

that have an economic impact exceeding $100 million or that 

adversely affect in a material way the economy, a sector of the 

economy, productivity, competition, jobs, the environment, public health or safety, or state. local, or tribal governments or communities (Sec. 3[f]I1D. EconomicaHy significant regulations require an extensive Regulatory Impact Analysis that assesses the 

need, effectiveness, benefits, costs, and alternatives for the proposed regulation (Sec. 6fa][3][C)), 

EUig and McLaughlin 

Regulatory Impact Analyses (RIAs) with improvements in the Environmental Protection Agency's 

(EPA) rules removing lead from gasoline and banning asbestos, but he also finds analytical shortcomings and notes that they did not serve as "blueprints" 

for the entire EPA decision.(15) Case studies of 

12 EPA rules issued between 1985 and 1995 in 

Morgenstern (p. 458) reveal that economic analysis 

always helped reduce costs, and it increased the benefits of five rules.(16) Nevertheless, economics had 

little effect on decisions. The analyses exhibited a 

"considerable range" in quality (p. 456)J16) Posner 

performed the only published study we have seen 

that assesses the quality of regulatory analysis for 

budget or "transfer" regulations that define how the 

federal government will spend or collect money.(17) 

He concludes that agencies rarely perform analysis 

for these regulations and presents several cases of inadequate analysis. 

Several validation studies judge the quality of 

RIAs by comparing their predictions to the actual 

results of the regulation revealed by retrospective 

analysis, Validation studies find that agencies tend 

to overstate benefits and costs more frequently than 

they understate them. OMB concludes that agencies 

tend to overstate benefit-cost ratios, whereas independent scholars conclude there is no systematic bias 

in the ratios. All of these studies find that benefits, 

costs, and benefit-cost ratios are inaccurate (overor underestimated) more frequently than they are 

accurate.(18-20) 

Harrington et al. present the most recent collection of case studies.(10) They assembled multiple 

studies of three "relatively sophisticated" RIAs issued during the G. W. Bush administration. Despite their sophistication, the analyses had significant 

flaws. The authors' recommendations for improvement are: consider meaningful alternative policy options, llse baselines that reveal choices and tradeoffs, 

include a checklist of practices that should be in an 

analysis, and explain deviations from this list. Noting 

that some regulatory analyses are prepared after key 

decisions have been made, they also call on EPA to 

prepare a preliminary regulatory impact analysis six 

months before the agency's final review of proposed 

and final regulations. 

2.2. Quantitative Scoriug 

Quantitative approaches usually employ a 

"yes/no" checklist to assess whether RIAs include 

certain elements. Early Government Accountability 
60 

The Quality aud Use of Regulatory Analysis 

Office evaluations of health, safety, and environmental regulations found that RIAs frequently failed 

to include key elements recommended in OMB's 

guidance. (21 ,22) Robert Hahn co-authored a series 

of papers that evaluate the quality of analysis for 

health, safety, and environmental regulations across 

three administrations-Ronald Reagan, G. Ii. W, 

Bush, and Bill Clinton-using a "yes/no" checklist 

based on OMB guidanee.(23-26) The regulatory analyses of a large sample of environmental regulations 

covered an average of approximately 30 of 76 items 

on Hahn's scorecard, or 40% (p. 74).(27) 

Belcore and Ellig score the quality of analysis 

[or all economically significant regulations issued by 

the Department of Homeland Security between 2003 

and 2007,(28) They found that the quality of analysis is generally low but improved over time, They 

also found that quality tends to be lower for rules issued subject to tight legislative deadlines, or where 

Congress gave the department little discretion. 

Fraas and Lutter assess 13 of the most important rules issued by EPA between 2005 and 2009, 

after OMB issued new and more detailed guidance 

in the form of Circular A_4.(29) Scores ranged from 

three points to a maximum possible nine, with an av~ 

erage of 5.25 (58%), Fraas and Lutter maintain that 

the quality of analysis is generally higher [or rules issued under legislation that requires agencies to consider costs or net benefits. 

Shapiro and Morrall examine 100 economically 

significant regulations adopted between 2002 and 

2010.(30) They assign each analysis a score of between zero and six points based on six OMB criteria. Scores ranged from zero to six, with an average 

of 3.85 (64%). They find that the quality of analysis is unrelated to the size of the rule's net benefits, 

but rules with lower political salience have higher net 

benefits. 

Both the qualitative and quantitative literature 

reveal some general patterns. Some RIAs are relatively high quality, but many lack key infonnation, 

and even the best ones could be improved. Analysis has never dictated decisions but has sometimes 

influenced decisions on the margin, and occasionally 

these margins involve large benefits or costs. 

2.3. Onr Approach 

We develop a scoring system to evaluate the 

quality and use of regulatory analysis for a relatively 

large number of regulations, Our approach differs 

from most previous evaluations in several ways. 

3 

1. It is the first project that evaluates the regulatory analyses accompanying all economically significant regulations proposed in a 

given year. Most previous evaluations focus 

on health, safety, and environmental regulations. 

2. We focus on proposed regulations, rather 

than final regulations. We seek to gauge the 

quality of analysis at the earliest possible 

point, when it arguably has the best chance of 

affecting decisions, Of course, many decisions 

have already been made by the time a rule 

is proposed,t31) but we remain optimistic that 

better analysis may sometimes lead to better 

decisions (pp.18-19).(32) In any case, the anal, 

ysis accompanying the proposed rule is usually the first comprehensive regulatory analy, 

sis available to evaluate. 

3. We opt for a qualitative evaluation of how 

well the analysis was performed, rather than 

an objective "yes/no" checklist of analytical 

issues and approaches covered, 

4. Our approach assesses whether the agency 

actually claims to use regulatory analysis to 

guide decisions. We also evaluate whether the 

agency makes a commitment to conducting 

retrospective analysis to assess the actual outcomes of the rule in the future, 

Although we seek to evaluate the quality and use 

of regulatory analysis, one might also interpret this 

study as an evaluation of the quality of regulations 

themselves, for a couple of reasons, First, regulatory 

analysis and regulations are often jointly produced, 

with lawyers and economists working together on every step of a rule making, possibly making the quality of the analysis correlated with quality of the regulation. Second, the quality of regulatory analysis is 

likely correlated with the quality of regulations because the outputs of the regulatory analysis should 

serve as inputs to regulatory decisions. For example, one important component of regulatory analysis is the consideration of alternative approaches to 

achieve the desired outcomes. If a low,quality analysis fails to consider alternatives, how can an agency 

be confident that its regulatory approach represents 

the best one, however "best" may be defined? 

Although our results may provide a perspective 

on the quality of regulations, we caution against interpreting our study as a comprehensive evaluation 

of regulatory quality. We can rate the quality of regulatory analysis because a number of components 
61 

4 

are required by executive order or statute. These 

components are familiar to most economists and can 

easily be assessed by any economist with the right 

training. Conversely, we have not attempted to define a method of rating the quality of regulations 

themselves. Although such an endeavor may be 

worthwhile, it is beyond the scope of this article. 

3. EVALUATION PROTOCOL 

3.1. What Was Evaluated? 

Evaluations were performed for 45 economically 

significant proposed regulations whose OIRA reviews were completed in 200S5 The research team 

read the preamble to each proposed rule and the accompanying RIA. In some cases, agencies produced 

a risk assessment or additional analysis in technical 

support documents that we also considered. We included the Regulatory Flexibility Analysis. which assesses the effects on small entities, to the extent that 

this analysis had content relevant to our evaluation 

criteria. 

This approach is broader than just reading the 

RIA document or section of the Federal Register notice explicitly labeled "Regulatory Impact Analysis." 

It is necessary because agencies organize the content 

differently in different rules. Sometimes the RIA is a 

separate document only referenced or summarized in 

the Federal Register preamblep3) Alternatively, the 

entire RIA may constitute a separate section of the 

preamble.o4) In one case, the RIA for a proposal was 

in the preamble of the Notice of Proposed Rulemaking (NPRM) for a related regulation published the 

same day. (35) Some parts may be in a separate regulatory analysis section of the preamble, and other parts, 

such as environmental impact analysis or risk assessment, may be in other sections of the preamble that 

discuss justifications for the regulation. (36-38) Reading all of this material allowed us to give the agency 

credit when due, regardless of where the analysis 

appears. 

3.2. Scoring System 

We evaluate regulatory analysis on the basis of 

12 criteria, grouped into three categories: 

5Reginfo.gov lists 48 proposed, economically significant regula~ 

Hons whose OIRA reviews were concluded in 2008. Three RIAs 

could not be found at the time these evaluations were performed, 

leaving us with 45 regulations to evaluate. 

Ellig aud McLanghlin 

1. Openness: How easily can a reasonably informed, interested citizen locate the analysis, 

understand it, and verify the underlying assumptions and data? 

2. Analysis: How well does the analysis define 

and measure the outcomes or benefits the regulation seeks to accomplish, define the systemic problem the regulation seeks to solve, 

identify and assess alternatives, and evaluate 

costs and benefits? 

3. Use: How much did the analysis appear to affect decisions in the proposed rule, and what 

provisions did the agency make for tracking 

the rule's effects in the future? 

Fig. 1 lists the 12 criteria. Appendix A lists 

detailed questions considered under each criterion. 

Appendix B presents a cross-walk chart that maps 

OMB's November 2010 "Regulatory Impact Analysis Checklist" into our scoring criteria. The Openness 

and Analysis criteria, numbered 1-8, are straightforward interpretations of provisions in Executive 

Order 12866 and Circular A-4. 

The Use criteria deserve further explanation. 

The first Use criterion asks whether the analysis 

seemed to affect decisions. To score this criterion, 

we assess whether the agency claimed to use information about the regulation's expected outcomes. 

the systemic problem, or benefits or costs of alternatives to make decisions. The second Use criterion 

asks whether the agency made its decisions fully cognizant of the net benefits of alternatives. We do not 

expect the analysis to dictate the decision via a rigid 

rule, such as "regulate only when monetized benefits exceed monetized costs." Section 1 of Executive Order 12866 explicitly instructs agencies to regulate only when the benefits "justify" the costs, unless 

the law requires another approach. Thus, we look to 

see whether the agency either selected the alternative that maximized net benefits or clearly explained 

why some other alternative was preferable to the one 

that maximized net benefits. 

Searching the Federal Register notice for documentation of use will not identify any undocumented, 

"behind the scenes" influence of the analysis. We 

may also overestimate the effects of analysis in situations where the agency reached decisions, then 

crafted or cited the analysis to support those decisions.(31) The actual influence of economists and economic analysis in rulemaking (as opposed to the influence documented in the Federal Register notice) 

likely differs across agencies and even across rules 
62 

The Quality and Use of Regulatory Analysis 5 

Openness 

1> Accessibility: How easily were the RIA, the proposed rule, and any 

supplementary materials found online? 

2. Data Documentation: How verifiable are the data used in the analysis? 

3. Model Documentation: How verifiable are the models and assumptions used in 

the analysis? 

4. Clarity: Was the analysis comprehensible to an infonned layperson? 

Analysis 

5. Outcomes: How wen does the analysis identify the desired benefits or other 

outcomes and demonstrate that the regulation will achieve them? 

6. Systemic Problem: lIow well does the analysis identify and demonstrate the 

existence of a market failure or other systemic problem the regulation is supposed 

to solve? 

Fig. 1. Regulatory analysis assessment 

criteria. 

7. Alternatives: How well docs the analysis assess the effectiveness of alternative 

approaches? 

8, Benefit~Cost Analysis: How well does the analysis assess costs and benefits? 

Use 

9. Use of Analysis: Does the proposed rule or the RIA present evidence that the 

agency used the Regulatory Impact Analysis? 

10. Net Benefits: Did the agency maximize net benefits or explain why it chose 

another option? 

11. Measures and Goals: Does the proposed rule establish measures and goals that 

can be used to track the regulation's results in the future? 

12. Retrospective Data: Did the agency indicate what data it wi11 use to assess the 

regulation's perfonnance in the future and establish provisions for doing so? 

within agencies. At one extreme, economists can 

have a lot of influence when a regulation is drafted, 

although it may not be documented in the proposed 

regulation or preamble (pp. 6_7).<32) At the other extreme, economists and their analyses may be ignored 

entirely. Two points in between are: (1) the economic 

analysis has no effect, but the agency writes it to support the rule, or (2) the economic analysis has some 

effect that is documented in the notice. Our evaluation method identifies these latter kinds of cases. Because we cannot easily distinguish between the two 

on the basis of claims in the KPRM or RIA, our 

method only assesses whether the agency seemed to 

use the analysis. By examining the documentation of 

use, we at least identify where analysis is likely to 

have inHuenced rulemaking and offer a starting point 

for future research into the maller. 

Criteria 11 and 12 assess the extent to which the 

RIA or preamble to the regulation make provisions 

for retrospective analysis. The executive orders governing regulation offer scant guidance on this, but 

Executive Order 13563 reiterates the requirement in 

Executive Order 12866 that each agency have a plan 

for retrospective review of regulations. A recent edition of OMB's annual report on the benefits and costs 

of federal regulation declared, "we recommend that 

serious consideration be given to finding ways to employ retrospective analysis more regularly, in order 

to ensure that rules are appropriate, and to expand, 

reduce, or repeal them in accordance with what has 

been learned" (p. 43).(39) The Government Performance and Results Act (GPRA) Modernization Act 

of 201 0 requires tbe federal government and agencies 

to identify high-priority goals; specify the programs, 

activities, tax expenditures, and regulations that contribute to each goal; and rcgularly evaluate the contributions.(40) An agency can lay the groundwork for 

compliance with the law by establishing goals and 

measures, identifying data. and committing to retrospective analysis in the preamble to the regulation. 

Agencies have in fact done these things for some regulations.6(41-44) 

6Por readers who are still skeptical about the value of including the 

two retrospective analysis criteria. we calculated Spearman's rho 

and Kendall's tau-b to assess whether inclusion of these criteria 

substantially alters the ranking of the regulations, The rankings 

with and without the Use criteria are highly correlated-rho 

0,981 and tau-b = O,920-with p values of 0.000. 
63 

6 

Table I. What Do the Scores Mean? 

Complete analysis of all or almost all aspects, with one or 

more "be'St practices. ,-

Reasonably thorough analysis of most aspects andlor shows 

at least one "best practice.-' 

Reasonably thorough analysis of some aspects. 

Some relevant discussion with some documentation of 

analysis. 

Perfunctory statement with little explanation or 

documentation. 

Little or no relevant content. 

For each criterion, evaluators assigned a score 

ranging from 0 (no useful content) to 5 (comprehensive analysis with potential best practices), Thus, 

each analysis has the opportunity to earn between 

o and 60 points, In general, the research team used 

the guidelines in Table I for scoring, Because the 

Analysis criteria involve many discrete issues, we 

developed a series of subquestions for each of the 

four Analysis criteria (listed in Appendix A), and 

awarded a 0-5 score for each subquestion, These 

scores were then averaged to calculate the score for 

the individual criterion, 

Compared to an objective checklist, our qualitative approach provides a richer and potentially 

more accurate evaluation of the actual quality of 

the analysis, As OIRA notes: "Objective metrics can 

measure whether an agency performed a particular 

type of analysis, but may not indicate how well the 

agency performed this analysis" (p, 19),(45) For example, rather than just asking whether the analysis 

considered alternatives or counting the number of alternatives considered, we give an analysis a higher 

score if it considered a wider range of alternatives. Instead of just asking whether the agency named a market failure, we assess whether the agency provides a 

coherent theory and plausible evidence that the market failure exists, awarding a higher score on the basis of how convincing the evidence is. The qualitative 

approach also encourages agencies to find the best 

way to do analysis that can inform decisions, instead 

of treating regulatory analysis as a "check the box" 

compliance exercise. 

A qualitative evaluation can be more SUbjective, 

less transparent, and harder to replicate. Several aspects of our research design seek to minimize these 

drawbacks, We designed the evaluation process to 

achieve a common, intersubjective understanding of 

which practices deserve which kind of score, and 

Ellig and McLaughlin 

evaluators took notes justifying eaeh score7 The entire nine-member research team underwent extensive training in which we evaluated several of the 

same proposed regulations and accompanying RIAs, 

compared scores, and discussed major differences 

until we achieved a consensus on scoring standards, 

For questions that were particularly difficult to evaluate, we developed written guidelines describing practices that would justify various scores in most cases, 

Each analysis was scored by one of the authors of 

this article and another team member, with discussion to achieve consensus when scores differed significantly, Each author also reviewed the other's scores 

and notes, and then discussed and resolved differences to ensure that all documents were evaluated as 

consistently as possible on all questions, 

In addition, we subjected the scores to ex post 

statistical analyses to test whether our research design produced a high degree of interrater reliability, lnterrater reliability is the degree to which raters 

agree with each other about their SUbjective evaluations of a given object. The Cohen kappa index is the 

most commonly used statistical measure of interrater 

reliability in social sciences,(48.49) Other typical tests 

include Spearman's rho and Pearson's chi-squared, 

both of which test the independence of the ratings. 

The goal of our interrater reliability testing was 

to ascertain whether our evaluation system yields 

consistent agreement among raters trained in the system, First, we created agreement matrices using the 

prediscussion scores for all questions together and 

for each individual question, These scores reflected 

each rater's evaluation before any discussion and deliberation about differences in ratings. Appendix C 

reports these agreement matrices. The first matrix, in 

Table AI, uses score data for all criteria, Each subsequent table shows the agreement matrix for a specific 

scoring criterion, The first scorer's rating dictates vertical location whereas the second scorer's rating controls the horizontal location, Thus, each matrix that 

corresponds to a particular criterion-Tables AIl 

through AXIII-has 45 observations of score pairs. 

7The term "intersubjcctivc" refers to subjective interpretations 

that different individuals can share because they have commonly 

understood meanings. Social scientists most commonly usc the 

term to denote economic agents' ability to understand the interpretations and meanings of other economic agents, or the social 

scientist's ability to understand the interpretations and meanings 

of the economic agents who are the subject of study.(46A7) We 

think it applies equally well here, when colleagues share similar 

subjective understandings of what constitutes better and worse 

analyses. 
64 

The Quality and Use of Regulatory Analysis 

Table II. Analysis of Interrater Reliahility 

Criterion Spearman's rho p-Value 

All 0.621 0.000 

0.414 0.005 

().675 0.000 

0.713 0.000 

0.504 0.000 

0.447 OJlO2 

0.586 0.000 

0.646 0.000 

0.591 0.000 

0.639 0.000 

10 0504 0.000 

11 0.421 0.004 

12 0.465 0.001 

A well-designed system would show substantial 

agreement between scorers, regardless of whom the 

scorers were or which regulation was scored. Such 

agreement would produce density along the diagonal in the agreement matrices, and that is precisely 

what we observe. At the bottom of each agreement 

matrix, we list the count and percentage of score 

pairs that are in perfect agreement or disagreement 

by different numbers of points. Table A.I reveals 

considerable agreement between raters: 42.4% of all 

ratings (229 of 540) were in perfect agreement, and 

another 36.7% (198 of 540) exhibited a difference of 

only one point. About 15% (79 of 540) showed a difference of two points, and only 6.3% (34 of 540) had 

a difference of more than two points. Tables A.1IAXIlI show similar results for the agreement distributions for each individual question. No particular 

question stands out as an egregious generator of disagreement among the raters. 

Cohen's kappa for the entire sample is 0.4784, 

which, according to the rules of thumb put forth by 

Landis and Koch, indicates moderate agreement.(50) 

Of course, this kappa is calculated using prediscuss ion ratings. After discussions, there was 100% 

agreement, and Cohen's kappa equaled L Table II 

shows the results of Spearman's rho tests for the entire sample and for each individual question. The values of rho range from 0.414 to 0.713, and the null 

hypothesis that the two ratings are independent is 

strongly rejected for each question. We also tested 

independence by calculating Pearson's chi-squared 

(not reported), finding similar results. 

The tests indicate that our rating system would 

likely produce statistical agreement for any set of 

raters, assuming they underwent the same train7 

ing. Scores for each regulation on each criterion, as 

well as notes justifying the scores, are available at 

www.mercatus.orglreportcard. 

3.3. An Example 

To illustrate how the evaluation protocol works, 

Table III reproduces scores and notes for a regulation that received a middling score on criterion 5, outcomes. "Outcomes are not what the program itself 

did but the consequences of what the program did" 

(p.15)f51) We intentionally employ the broader term 

"outcomes" rather than "benefits" because some regulations seek to achieve goals that do not necessarily meet the economist's definition of a social benefit. 

We ask merely whether the regulatory analysis articulates, measures, and justifies an outcome that affects 

citizens' quality of life, regardless of whether the goal 

increases net social benefits. 

The regulation in Table III is an Occupational 

Safety and Health Administration (OSHA) regulation intended to improve safety around cranes and 

derricks at construction sites. The analysis identified workplace safety outcomes and explained how to 

measure them, earning a five on each of these questions. However, the analysis does not provide much 

documented theory or evidence that the regulations 

would reduce fatalities and accidents; the reader is 

simply assured that "OSHA analysis" proves this is 

so. An explicit theory, rather than just an assertion, 

and documentation of evidence supporting the theory would have earned this analysis a higber score 

on these two questions. Although the RIA acknowledges uncertainty about benefits, it provides little 

analysis showing how the uncertainties would affect 

estimates of injuries and fatalities. 

3,4, Caveats 

Three significant caveats accompany our findings. First, we evaluate the quality of regulatory analysis and its apparent use in decisions, but we do not 

evaluate whether the proposed rule is economically 

efficient, fair, or otherwise good public policy. This 

article is an assessment of how agencies conduct and 

claim to use regulatory analysis, not a policy analysis 

of the regulations themselves. 

Second, we evaluated whether the RIA and 

preamble to the proposed rule make a reasonable 

effort at covering the major elements of regulatory 

analysis. We did not seek to replicate the results, 

produce our own analysis, or verify the underlying 
65 

8 Ellig and McLaughlin 

Table III. Outcome Discussion in Labor Department's Cranes and Derricks Proposed Rule 

Criterion 

H,*wwen.doeJ! tlte analysis identify th~ desired outcOmes and 

demonStt:atec that the,'reguJation; Will achieve the:m'? 

Does the analysis dearly identify ultimate outcomes that affect 

citizens' quality of life? 

Does the analysis identify how these outcomes are to be 

measured? 

Does the analysis provide a coherent and testable theory 

showing how the regulation will produce the desired 

outcomes? 

Does the analysis present credible empirical support for the 

theory? 

Does the analysis adequately assess uncertainty about the 

outcomes? 

data and studies. Commenters on this article who 

have in-depth experience with particular regulations 

have usually told us that we have been too lenient. 

For example, an EPA air pollution regulation proposed in 2008 scores fairly high for its analysis of 

uncertainty regarding the size of benefits, but Fraas 

documents significant shortcomings in the EPA's uncertainty analysis of benefits of air quality regulations. (52) A high-scoring analysis may thus still have 

flaws and inaccuracies because of poor underlying 

data or theories that turn out to be wrong. Authors of 

previous regulatory scorecards have also noted this 

drawback (p. 196, p. 3),l24.29) Nevertheless, as Dudley 

(p. 8) notes: "Benefit-cost analysis isn't perfect, but 

it's the best we have."(3) The strength of a scorecard 

approach is its ability to rank numerous regulatory 

analyses according to consistent criteria. As we shall 

see, the approach identifies significant differences in 

the quality of analysis across different regulations. 

Third, we give each criterion the same weight. 

Of course, some criteria, such as whether the agency 

identified a systemic problem or whether it analyzed 

a broad array of alternatives, may have more policy 

impact than whether the RIA is clearly written for 

the average citizen. The results below often break out 

score data by our three categories of criteria or by individual criteria, so that readers who want to focus on 

particular criteria or groups of criteria can do so. For 

example, readers who are concerned solely about the 

quality of regulatory analysis can ignore our evaluations of Use and focus on criteria 1-8, which assess 

Score 

Workplace safety-reduced fatalities and accidents, 

Number of fatalities and accidents avoided. 

Asserts only that "OSHA analysis" shows that an indicated 

number of fatalities would be eliminated. The text of the rule 

does a better job explaining several published articles that 

identify major causes of crane accidents. 

Some examples of recent accidents are presented. and the 

preamble to the rule explains how the proposed rules would 

have prevented those accidents. It is not clear if these are 

typical or generalized examples. 

Uncertainty is acknowledged, and several benefit estimates are 

offered. However, the sensitivity discussion is cursory and 

does not provide much in~depth analysis on how injuries and 

fatalities would likely be affected. 

Openness and Analysis. We calculated Spearman's 

rho and Kendall's tau-b to assess whether inclusion 

of the Use criteria substantially alters the ranking of 

the regulations. The rankings with and without the 

Use criteria are highly correlated-p = 0.959 and 

fb = 0.861-with p-values of 0.000. Readers who believe some individual criteria should be omitted or 

weighted more heavily are welcome to download our 

Excel spreadsheet with a full set of score results for 

every regUlation, and conduct similar tests.s 

4. SCORES AND RANKINGS 

4.1. Summary Statistics 

Both the average and median score were 27 of 

60 possible points, or 45%. The best analysis received 

43 points (72%), and the worst received only seven 

points (12%). Fig. 2 shows the distribution of scores. 

In general, the documents score higher on Openness than on the other two categories. The average 

score on the Openness criteria was 11 of 20 possible points, compared to 8.5 points for Analysis, and 

7.7 points for Use. 

4.2. Best and Worst Analyses 

Table IV lists scores for all 45 regulations, 

along with their Regulation Identifier Numbers and 

8The spreadsheet is available at www.mercatus.org/reportcard. 
66 

The Quality and Use of Regulatory Analysis 

,S 

16 

.2 ,·1 

~ 12 

e 10 

~ I I I - Score (M,uliml.lm ptl'SSibJe ~ 60) 

the name of the issuing department. The best initial analysis in 2008 was for the Department of 

Transportation's (DOT) proposed Corporate Average Fuel Economy regulation, followed by the EPA's 

National Ambient Air Quality Standards (NAAQS) 

for Lead, and Housing and Urban Development's 

(HUD) proposed revisions to the Real Estate Settlement Procedures Act. The three worst analyses 

come from the Social Security Administration, Department of Veterans' Affairs, and Department of 

Defense. 

4,3. Average Scores by Regulation Type 

The 15 regulations in italics in Table IV are hudget or "lransfer" regulations. These regulations outline how the federal government will spend money, 

set fees, or administer spending programs. Most of 

these regulations score poorly. Calculating average 

scores by type of regulation reveals a big discontinuity, as Table V shows. Average scores for most 

types of regulations range between 30 and 35 points. 

Transfer regulations, however, average just 17 points. 

Transfer regulations score lower on all the three categories of criteria, but the biggest difference is in the 

Analysis category, where transfer regulations score 

only about one-third the points of other types of regulations. 

This finding is consistent with OMB's observation that agencies do not usually estimate the social 

benefits and costs of transfer regulations (p. 19)'<39) 

Posner documents the same phenomenon.(l7) It is 

not obvious why transfer regulations should receive 

different analytical treatment, for as OMB notes 

(p. 19), transfer regulations generate substantial social costs via mandates, prohibitions, and price distortions.(39) Our results on transfer regulations illustrate a more general point: the data from this project 

can provide a starting point for analyzing a variety of 

factors that might influence the quality of regulatory 

analysis, such as the nature of the regulation, politics, 

9 

Fig. 2. Distribution of analysis scores. 

legislative mandates, or deadlines. (See Shapiro and 

Morrall and Belcore and Ellig (pp. 38-41) for similar 

examples. )p0.2S) 

4.4. Agency Average Scores 

Table VI lists average scores for each agency. 

HUD's one regulation earned it the highest agency 

average. EPA placed second, and Homeland Security placed third. Scores decline relatively smoothly 

as one moves down the list, except for the 7.7-

point gap that separates Health and Human Services 

(HHS), ranked 13th, from State, ranked 14th. 

Most of the agencies in the top half of the list produced more than one economically significant regulation in 2008. All of the agencies in the bottom half 

produced just one, except for HHS (11 regulations) 

and Education (two regulations). Whether this pattern reflects economies of scale or mere coincidence 

remains to be seen. 

We caution the reader against drawing strong inferences about agencies' analytical abilities on the 

basis of these scores for one year. Most departments 

produced small numbers of regulations, and many 

consist of diverse agencies that may not all produce 

the same quality of analysis. Generalizations about 

different agencies' abilities would require either a 

larger data set spanning more years or in-depth case 

studies. 

4,5. Average Scores by Criterion 

Average scores on individual criteria reveal 

where regulatory analysis in practice is generally 

strongest and weakest. The criterion with the highest average score in Table VII is criterion 1, Accessibility. This is not surprising, as making documents 

accessible to the public via the Internet is relatively 

easy to do regardless of the quality of the analysis itself. The two lowest scoring criteria are both related 

to retrospective analysis: establishing measures and 
67 

16 Ellig and McLaughlin 

Table IV. Scores for 45 Economically Significant Regulations from 2008 

Proposed Rule RlN Department Total Openness Analysis Use 

Car and Light Truck Corporate Average Fuel Economy 2011-2015 2127-AK29 DOT 43 15 16 12 

National Ambient Air Quality Standards for Lead 2060-AN83 EPA 42 14 16 12 

Real Estate Settlement Procedures Act 2502-A161 HlJD 41 15 16 10 

Class Exemption for Provision of Investment Advice, Proposed Rule 121O-AB13 Labor 40 15 15 10 

Congestion Management Rule for LaGuardia Airport 2120-Al70 DOT 39 13 13 13 

Large Aircraft Security Program 1652-AA53 DHS 38 15 13 10 

US VISIT Biometric Exit System 1601-AA34 DHS 38 9 15 14 

Fiduciary Requirements fOf Disclosure in Participant-Directed Plans 121O-AB07 Labor 37 15 11 11 

Notice of Class Exemption for Provision of Investment Advice 1210-ZA14 Labor 37 12 14 11 

Effluent Limitations Guidelines and Standards for Construction 2040-AE91 EPA 37 14 14 9 

Electronic Prescriptions for Controlled Substances 1117-AA61 DOl 36 14 12 10 

Migratory Bird Hunting 1018-AV62 Interior 35 14 12 9 

Nondiscrimination in State/Local Government Services 1190-AA46 DOl 35 14 12 

Nondiscrimination by Public/Commercial Facilities 1190-AA44 DOl 34 14 11 

Railroad Tank Car Transportation of Hazardous Materials 2130-AB69 DOT 33 10 13 10 

HIPAA Code Sets 0958-AN25 HHS 33 15 10 

Family and Medical Leave Act of 1993 1215-AB35 Labor 33 18 10 

Congestion Mgt. for John F. Kennedy Airport and Newark Airport 2120-AJ2S DOT 30 10 12 

Cranes and Derricks in Construction 1218-AC01 Labor 30 14 7 

Refuge Alternatives for Underground Coal Mines 1219-AB58 Labor 28 12 

Integrity Management Program for Gas Distribution Pipelines 2137-AE15 DOT 28 7 11 10 

State~Specific Inventoried Roadless Area Management 0596-AC74 USDA 28 11 12 5 

Energy Conservation Standards for Fluorescent Lamps 1904-AA92 Energy 27 6 11 10 

Alternative Energy Production on the OCS lO10-AD30 Interior 27 8 10 9 

Standardized Risk~Based Capital Rules (Basel II) 1557-AD07 Treasury 27 9 

Changes fa the Outpatient Prmpective Payment System 0938-AP17 1I11S 27 13 

Hospitallnparienc Prospective Payment Systems 0938-API5 HlIS 27 14 6 

Oil Shale Management-General 1004-AD90 Interior 26 9 9 

HIPAA Electronic Transaction Standards 0938-AM50 HHS 25 12 8 

Employment Eligibility Verification 901JO-AK91 FAR 24 13 

Teacher Education Assistance Grant Program 1840-AC9J ED 23 10 

Abandoned Mine Land Program 1029-AC56 Interior 21 10 

Maximum Operating Pressure for Gas Transmission Pipelines 2137-AE25 DOT 21 11 

Federal Perkins Loan Program 1840-AC94 ED 21 10 

Revisions to Medicare Advantage and Prescription Drug Benefits 0938-AP24 fillS 19 8 

Prospective Payment System for Long~ Term Care Hospitals 0983-A094 HHS 17 9 

Medicare Program: Revisions to Physician Fee Schedules 0938-AP18 flflS 17 6 

Medicaid Program Premiums and Cost Sharing 0938-A047 fillS 17 JO 

State Flexibility for Medicaid Benefit Packages 0938-A048 lINS 16 9 4 

Proposed Hospice Wage Index for Fiscal Year 2009 0938-AP14 HlIS 16 9 3 

Prospective Payment System for Skilled Nursinf{ Facilities 0938-APII HflS 14 

Schedule of Fees for Consular Services I4fJO-AC41 State 13 4 

CifAMPUSITR1CARE 0720-AB22 Defense 12 4 

Post~9/11 GI Bill 2900-ANJO VA 10 2 

Time and Place for a Hearing Before an Administrative Law Judge 0960-AG6J SSA 7 4 0 3 

Average 27.3 11.04 8.5 7.7 

Note: Regulations in italics are budget or "transfer" regulations. 

goals to track the regulation's effects in the future 12866 leads off by stating that each regulation must 

(criterion 11) and gathering data for such assessment identify the problem it seeks to address and assess 

(criterion 12). the significance of that problem. The analyses that 

The other low-scoring criterion is identification score low on this criterion either simply assert a reaof thc market failure or other systemic problem son for the regulation, with no accompanying theory 

the regulation is supposed to solve. This low score or evidence, or mention no explicit rationale at all 

is puzzling because Section 1 of Executive Order beyond implementing a stalule. Such weaknesses are 
68 

The Quality and Use of Regulatory Analysis 

Table V. Average Scores by Regulation Type 

Number of Average 

Type Regulations Score Openness Analysis Usc 

Civil rights 34.5 14.0 9.0 11.5 

Economic 10 34.2 13.4 11.4 9.4 

Security 3 33.3 12.3 11.7 9.3 

Environment 31.8 11.2 11.6 9.0 

Safety 29.3 11.3 10.0 8.0 

Transfer 15 17.1 8.6 3.5 4.9 

Table VL Agency Average Scores 

Number of Average 

Agency Regulations Score Openness Analysis Use 

1.HUD 41.0 15.0 16.0 10.0 

2. EPA 395 14.0 15.0 10.5 

3.DHS 38.0 12.0 14.0 12.0 

4.DOJ 35.0 14.0 10.0 11.0 

5. Labor 34.2 14.3 11.2 8.7 

6. DOT 323 11.0 11.3 10.0 

7. USDA 28.0 11.0 12.0 5.0 

8. Interior 27.3 10.3 8.9 8.3 

9. Treasury 27.0 9.0 9.0 9.0 

10. Energy 27.0 6.0 11.0 10.0 

11. Federal 24.0 13.0 7.0 4.0 

acquisi~ 

tion 

12. Education 22.0 10.0 3.0 9.0 

13. HHS 1l 20.7 10.2 5.0 5.5 

14. State 1 13.0 7.0 4.0 2.0 

15. Defense 12.0 7.0 4.0 1.0 

16. Veterans lItO 6.0 2.0 2.0 

affairs 

17. Social 7.0 4.0 3.0 

security 

Table VII. Ranking of Scores on rndividual Criteria 

Criterion 

Accessibility 

Clarity 

Some use of analysis 

Outcome definition 

Model documentation 

Alternatives 

Data documentation 

Net benefits 

Benefit-cost analysis 

Systemic problem 

Retrospective data 

Measures and goals 

Overall average score 

Including 

Transfer 

Regulations 

3.53 

2.93 

2.44 

2.36 

2.33 

2.29 

2.24 

2.20 

2.09 

1.80 

1.73 

1.36 

27.31 

Excluding 

Transfer 

Regulations 

3.30 

3.50 

2.63 

3.10 

2.83 

2.93 

2.63 

2.93 

2.60 

2.40 

2.03 

1.53 

32.43 

11 

disturbing. It is hard to have confidence that a regulation really will solve a problem, or that the agency 

has selected the best option for solving a problem, if 

the agency cannot articulate the problem, cite con· 

vincing evidence that the problem exists, and explain 

its root cause. 

Given the lower average scores of transfer reg· 

ulations, it is no surprise that average scores on individual criteria are generally higher when transfer 

regulations are excluded. But even excluding transfer 

regulations, the average score on this criterion is only 

2.4 points. We can identify more than a few examples of prescriptive regulations that scored a 1 or 2 on 

identification of the systemic problem. These include 

Treasury's risk·based capital rules for banks, Interior's abandoned mine land program and oil shale 

management rules, DOT's maximum operating pres· 

sure for gas transmission pipelines, and Federal Acquisition's employment eligibility verification rules. 

5, USE OF REGULATORY ANALYSIS 

Different scholars offer different conclusions 

about whether economic analysis actually has much 

influence on regulatory decisions. Hahn and Tetlock 

conclude that few RIAs have much effect.(27) Offi· 

cials interviewed by West claim that decisionmak· 

ers often make up their minds before the analysis is 

done. (31) Williams, on the other hand, suggests that 

regulatory analysis can affect decisions behind the 

scenes, even if the agency does not explicitly explain 

in its Federal Register notice (pp. 6-7).02) Our scar· 

ing on the Use criteria offers another perspective on 

this question. 

5.1. Do Agencies Claim to Use 

Regulatory Analysis? 

Table VI shows that criterion 9, Use of Analysis, 

has the third highest average score. An agency can 

earn points on this criterion even if statutorily prohibited from considering some factors, such as costs 

or net benefits. For example, when setting NAAQS, 

"[a]ccording to the Clean Air Act, EPA must use 

health-based criteria in setting the NAAQS and 

cannot consider estimates of compliance COS1.,,(42) 

However because health is one of the key benefits 

of air quality standards, the EPA received two points 

on criterion 9 for using the health analysis to inform 

its decision. 
69 

U 

16 

14 

,~ 12 

I 1! 10 

~ '0 

I " 

• Score {Maximum pmslble ::; 5) 

18 

16 I i ~~ 110 

I e 8 

: G 

• Score • 

(M,utimlJm possible ~ 51 

Criterion 10, Net Benefits, receives a lower 

average score when transfer regulations are included (2.20 points) than when they are excluded 

(2.93 points). One might argne that net benefits 

are irrelevant when a regulation "merely" transfers 

money, but surely most federal expenditures are supposed to achieve some type of public benefit that 

could often be measurable. To achieve a good score 

on this criterion, the agency does not have to select 

the alternative that maximizes net benefits. Rather, 

the agency must demonstrate that it was cognizant 

of net benefits and weighed them against other factors when making its decision. If the RIA calculates 

net benefits of multiple alternatives but the preamble to the proposed rule clearly states the justification 

for choosing an alternative that did not maximize net 

benefits, the agency can still score well on this criterion. We score the Net Benefits criterion this way 

to avoid imposing the valne judgment that agencies 

"ought" to choose the alternative that maximizes net 

benefits. Instead, we evaluate whether decisionmakers considered net benefits and then determined what 

weight net benefits should have in the decision. 

Figs. 3 and 4 show that the scores on these 

two criteria have a somewhat bimodal distribution. 

About 10 regulations earned a score of 4 or 5. For 

more than 20 regulations, the agency seems to have 

used little or nothing of the analysis. The remaining 

Ellig and McLanghlin 

I • 

Fig. 3. Breakdown of criterion 9. 

I • 

Fig.4. Breakdown of criterion 10. 

regulations show some apparent use of the analysis, 

but not substantial use. We infer from this that agencies sometimes claim regulatory analysis had a significant effect on the regulation, but more often they 

claim a marginal effect or make no claim at all. 

The really low scores in the Use category are on 

the two retrospective analysis criteria. Only four regulations earned a 3 or better on criterion 11, Measures and Goals, and only 10 regulations earned a 

3 or better on criterion 12, Retrospective Data. Few 

economically significant regulations include any substantial plans for retrospective analysis of either costs 

or benefits. Seventeen years after passage of the 

GPRA required agencies to develop goals and measures for their major programs, this is disappointing 

news indeed. Because economically significant regulations are the ones with the largest impact, surely 

most of them are related to an agency's fundamental 

mission and strategic goals. 

5.2. Correlation of Qnality and Use 

Because we evaluated both the quality and the 

apparent use of regulatory analysis, we can test to 

see whether there is any correlation between the 

two. Table VIII shows regression results using all 45 

regulations; Table IX excludes transfer regulations. 

Both tables reveal that there is a tighter and more 
70 

The Quality and Use of Regulatory Analysis 13 

Table VIII. Use of Analysis Versus Quality, 45 Regulations (Tobit Regressions) 

Quality Score Analysis Score Pseudo 

Dependent Variable Constant (Criteria 1--8) (Criteria 5~8) Chi-Squared R-Squarcd 

Criteria 9~12 (All Four Use Criteria) 1.48 0.32 27.27" 0.12 

[1.36J [6.12J'" 

Criteria 9-12 (Al! Four Use Criteria) 3.20 0.54 34.77'" 0.15 

[4.49J·" [7.24J"· 

Criteria 9 and 10 (Some Use and Net Benefits) 1.69 0.35 22.37'" 11.11 

[2.69J'" [5.35J'" 

Criteria 11 and 12 (Measures and Goals and Retrospective 

Data) 

Criterion 9 (Some Use of Analysis) 

Criterion]O (Net Benefits) 

Criterion 11 (Measures and Goals) 

Criterion 12 (Retrospective Data) 

t-statistics in brackets. 

**"Significant at the 1 % leveL 

""Significant at the 5 % level. 

*Significant at the 10% leveL 

1.28 

[2.24J" 

1.43 

[-0.05J 

0.36 

[0.97J 

0.68 

[1.91J· 

0.21 10.99*** 0.06 

[3.51J'" 

0.12 6.88'" 0.04 

[2.71J"· 

0.26 29.46'" 0.18 

[6.26J'" 

0.1 I 7.70'" 0.06 

[2.85J"· 

0.12 9.37'" 0.07 

[3.22J'" 

Table IX. Use of Analysis Versus Quality, 30 Nontransfer Regs (Tobit Regressions) 

Dependent Variable 

Criteria 9-12 (All Four Use Criteria) 

Criteria 9-" 12 (All Pour Use Criteria) 

Criteria 9 and 10 (Some Use and Net Benefits) 

Criteria 11 and 12 (Measures and Goals and Retrospective 

Data) 

Criterion 9 (Some Use of Analysis) 

Criterion 10 (Net Benefits) 

Criterion 11 (Measures and Goals) 

Criterion 12 (Retrospective Data) 

t-statistics in brackets. 

n"Significant at the 1 % level. 

**Significant at the 5% level. 

* Significant at the 10% level. 

Constant 

4.35 

[2.09]" 

3.58 

[2.57J" 

2.31 

[1.63J 

0.90 

[O.72J 

0.67 

[O.7!J 

1.44 

[1.75J' 

0,15 

[O.72J 

0.63 

[O.77J 

significant correlation between Use (criteria 9-12) 

and the Analysis score (criteria 5-8) than between 

Use and the total Qualily score (criteria 1-8). In 

other words, agencies are likely to claim they used 

the analysis if it is more thorough, even if it is 

more difficult 10 find, less thoroughly documented, or 

harder to read. 

Quality Score 

(Criteria 1-8) 

0.209 

[2.37J'" 

Analysis Score 

(Criteria 5--8) 

0.51 

[4.17J"· 

0.30 

[2.44J" 

0.24 

[2.21J" 

0.18 

[2.19J" 

0.14 

[1.981' 

0.12 

[1.98}' 

0.12 

[1.85J· 

Chi-Squared Pseudo-R2 

5.14" 0.04 

13.73*** 0.10 

5.47" 0.04 

4.58" 0.04 

4.56" 0.04 

3.73' 0.04 

3.69* 0.04 

3.28' 0.03 

Table VIII shows that there is a positive and statistically significant correlation between the quality 

of the analysis and every subeomponenl of the Use 

score. When the sample is confined to nonlransfer 

regulations, however, the relationship is somewhat 

less extensive, as Table IX shows. Taken together, 

lhe Use criteria are still highly correlated wilh 
71 

14 

quality of the analysis. Quality of analysis is also 

correlated with the sum of criteria 9 and 10 (Some 

Use of Analysis and Net Benefits), and with the 

sum of criteria 11 and 12 (Measures and Goals and 

Retrospective Data). But when the regressions are 

run using individual criteria, the quality of analysis 

is only marginally significant for criteria 10-12. For 

nontransfer regulations, it seems that the principal 

source of correlation between quality and apparent 

use is criterion 9, which measures whether the agency 

claimed the RIA affected decisions in the regulation, 

For non transfer regulations, good analysis might not 

be correlated with consideration of net benefits, nor 

does it necessarily imply that the agency will provide 

for retrospective analysis. 

Nevertheless, there is some evidence that agencies claim they used the analysis in regulatory decisions when the analysis is better. Perhaps improving the quality of analysis improves the odds that 

decisionmakers will find it useful. Or perhaps when 

decisionmakers are willing to use regulatory analysis, better regulatory analysis gets produced, It is also 

possible that, by the time a proposed rule and the accompanying analysis emerge from several iterations 

of revision within the agency and the OIRA review 

process, quality and use are mutually interdependent. Finally, the correlation may be driven by other 

[actors, such as statutory requirements that agencies 

either must or must not consider various aspects of 

regulatory analysis when making decisions, 

Even if most agencies treat RIAs as a mere compliance exercise, it is interesting to note that agencies are more likely to claim that their analysis influenced their decisions when the analysis is better. 

Clearly, the relationship between quality of analysis 

and agencies' claimed use of analysis is an area ripe 

for further research. 

6, BEST PRACTICES 

Our qualitative evaluation method identifies 

which analyses have done a particularly good job according to the various criteria. Table X compares the 

average score on each criterion with the highest score 

any analysis achieved on that criterion. On most cri~ 

teria, only a handful of analyses earned the highest 

score of 5. No analysis earned a score of 5 for criterion 8, Benefit-Cost Analysis, but at least one earned 

a 5 on each subquestion under Benefit-Cost Analysis. 

Clearly, more widespread adoption of existing best 

practices could substantially improve the quality of 

most regulatory analyses. 

Ellig and McLaughlin 

Table X. Best Practices Not Widely Shared 

Highest No. Earning 

Average Score Highest 

Criterion Score Achieved Score 

1. Accessibility 3.53 12 

2. Data documentation 2.24 1 

3. Model documentation 2.33 

4. Clarity 2.93 

5. Outcome definition 2.36 

6. Systemic problem 1.80 

7. Alternatives 2.29 

8. Benefit-cost analysis 2.()9 

9. Some use of analysis 2.44 

10. Considered net benefits 2.2() 

11. Measures and goals 1.36 

12. Retrospective data 1.73 

7. CONCLUSIONS 

Regulatory analysis is supposed to inform regulatory decisions, not simply justify them after the 

fact or merely fulfill a requirement to clear a rule 

through OIRA, Because proposed regulations usually reflect a great deal of up-front work and are supposed to represent the agency's preferred approach 

to problem solving, we evaluated the quality of regulatory analyses accompanying proposed regulations. 

This allows us to assess whether the analysis publicly disclosed closest to the time when initial decisions are made is comprehensive and reliable enough 

to inform those decisions. In addition, we evaluated 

whether the agency claims to use regulatory analysis 

to inform its decisions, now and in the future, This 

allows us to assess whether the quality of regulatory 

analysis is correlated with its apparent use. 

Our findings on quality are generally consistent with prior literature. Previous regulatory scorecard literature finds that analyses earn an average of 

40-64 % of the total possible points, with higher 

scores for more recent years.(27,29,30) The average 

for all regulations we assessed was 27.3 of 60 possible points, or 46%. Excluding transfer regulations, 

the average was 32.4 points, or 54%. Along with 

Shapiro and Morrall and Fraas and Lutter, our figures may suggest that the quality of regulatory analysis has improved somewhat since Hahn's seminal 

scorecards. (30.29) 

Qualitative scoring allows us to distinguish between better and worse implementation of economic 

analysis. The scores clearly indicate that every aspect 

of regulatory analysis is done at least somewhat well 

by someone in some agency on some regulation, but 

no single analysis comes close to doing everything 
72 

The Quality and Use of Regulatory Analysis 

well. Substantial improvements in regulatory analysis could occur across the board if federal agencies 

had the incentives to mobilize and spread know-how 

that already exists. 

Our results also suggest that regulatory analysis 

is perhaps more widely used than previous research 

has shown. Agencies claimed that some aspect of the 

analysis affected some major aspect of the regulatory 

decision in about 10 rules, or 22 % of the total. Moreover, the apparent use of analysis is positively correlated with quality of analysis. Agencies are more 

likely to claim they used the RIA when the RIA 

is belter-though which way the causation runs remains to be seen. 

This article reports just the first steps in a multifaceted, ongoing research project. One extension 

would be to evaluate regulations issued in 2009, to assess whether there was much difference in the quality of regulatory analysis during the last year of the 

Bush administration and the first year of the Obama 

administration. Evaluations similar to ours could also 

be used to assess whether President Obama's Executive Order 13563 has any effect on the quality of regulatory analysis. 

Finally, the data can be used to analyze other 

factors that might affect the quality of regulatory 

analysis, such as deadlines, politics, statutory requirements, court decisions, or institutional factors unique 

to particular agencies. Some of the literature cited 

in Section 1 found that these types of factors affected the quality of regulatory analysis.(3o.,S) Other 

research is also suggestive. McLaughlin, for example, finds that "midnight" regulations issued late in 

an outgoing administration's term receive shorter review at OIRA.(53) McLaughlin and Ellig report that 

midnight regulations, transfer regulations, and regulations with statutory deadlines all have lower quality 

analysis, and the latter two types of regulations also 

receive shorter review times at OIRAJ54) This suggests that the quality of analysis varies systematically 

with institutional factors. The evaluations reported in 

this article are the first step in testing these types of 

hypotheses. 

ACKi'\ilOWLEDGMENTS 

This article reports on the first stage of an ongoing project initiated by the authors at the Mercatus 

Center at George Mason University. Although one 

of the authors (McLaughlin) is no longer affiliated 

with the Mereatus Center, his part in the evaluation 

of RIAs was performed when he was a research fel15 

low at the Mercatus Center and before his employment with the U.S. Department of Transportation. 

It is a revised version of a longer working article 

available online at http://mercatus.org/publication/ 

quality-and-use-regulatory-analysis-2008.(55) We acknowledge the substantial contributions of the other 

individuals who served on the research team that 

evaluated these regulatory analyses: Mark Adams, 

David Bieler, Katelyn Christ, Christina Forsberg, 

Stefanie Haeffele-Balch, Gabriel Okolski, and Kevin 

Rollins. We thank Mohamad Elbarasse for research 

assistance, and Rick Belzer, Susan Dudley, Art 

Fraas, Randy Lutter, John Morrall, Marcus Peacock, Richard Williams, and two anonymous peer reviewers for helpful comments. The views and opinions expressed by the authors do not necessarily 

state or reflect those of the U.S. government, the 

U.S. Department of Transportation, or the Federal 

Railroad Administration, and shall not be used for 

advertising or product endorsement purposes. 

APPENDIX A: MAJOR FACTORS 

CONSIDERED WHEN EVALUATING 

EACH CRITERION 

,,",ote: Regardless of how they are worded, all 

questions involve qualitative analysis of how well the 

RIA addresses the issue, rather than "yes/no" answers. 

Openness 

1. How easily were the RIA, the proposed rule, 

and any supplementary materials found online? 

How easily can the proposed rule and RIA be 

found on the agency's website? 

How easily can the proposed rule and RIA be 

found on Regulations.gov? 

Can the proposed rule and RIA be found 

without contacting the agency for assistance? 

2. How verifiable are the data used in the analysis? 

Is there evidence that the RIA used data? 

Does the RIA provide sufficient information 

for the reader to verify the data? 

How much of the data are sourced? 

Does the RIA provide direct access to the 

data via links, URLs, or provision of data in 

appendices? 

If data are confidential, how well does the 

RIA assure the reader that the data are valid? 
73 

16 

3. How verifiable are the models and assumptions used in the analysis? 

Are models and assumptions stated clearly? 

How well does the RIA justify any models or 

assumptions used? 

How easily can the reader verify the accuracy 

of models and assumptions? 

Does the RIA provide citations to sources 

that justify the models or assumptions? 

Does the RIA demonstrate that its models 

and assumptions are widely accepted by relevant experts? 

How reliable are the sources? Are the sources 

peer-reviewed? 

4. Was the Regulatory Impact Analysis comprehensible to an informed layperson? 

How well can a nonspecialist reader understand the results or conclusions? 

How well can a nonspecialist reader understand how the RIA reached the results? 

How well can a specialist reader understand 

how the RIA reached the results? 

Is the RIA written in "plain English"? (Light 

on technical jargon and acronyms, well organized, grammatically correct, and direct language used.) 

Analysis 

5. How well does the analysis identify the desired 

outcomes and demonstrate that the regulation 

will achieve them? 

How well does the RIA identify ultimate outcomes that affect citizens' quality of life? 

How well does the RIA identify how these 

outcomes are to be measured? 

Does the RIA provide a coherent and testable 

theory showing how the regulation will produce the desired outcomes? 

Does the analysis present credible empirical 

support for the theory? 

Does the analysis adequately assess uncertainty about the outcomes? 

6. How well does the analysis identify and 

demonstrate the existence of a market failure or 

other systemic problem the regulation is supposed to solve? 

Does the analysis identify a market failure or 

other systemic problem? 

Does the analysis outline a coherent and 

testable theory that explains why the problem 

Use 

EIlig and McLaughlin 

(associated with the outcome above) is systemic rather than anecdotal? 

Does the analysis present credible empirical 

support for the theory? 

Does the analysis adequately assess uncertainty about the existence and size of the 

problem? 

7. How well does the analysis assess the effectiveness of alternative approaches? 

Does the analysis enumerate other alternatives to address the problem? 

Is the range of alternatives considered narrow 

or broad? 

Does the analysis evaluate how alternative 

approaches would affect the amount of the 

outcome achieved? 

Does the analysis adequately address the 

baseline-What the state of the world is likely 

to be in the absence of further federal action? 

8. How well does the analysis assess costs and 

benefits? 

Does the analysis identify and quantify incremental costs of all alternatives considered? 

Does the analysis identify all expenditures 

likely to arise as a result of the regulation? 

Docs the analysis identify how the regulation 

would likely affect the prices of goods and services? 

Does the analysis examine costs that stem 

from changes in human behavior as consumers and producers respond to the regulation? 

Does the analysis adequately address uncertainty about costs? 

Does the analysis identify the approach that 

maximizes net benefits? 

Does the analysis identify the costeffectiveness of each alternative considered? 

Does the analysis identify all parties who 

would bear costs and assess the incidence of 

costs? 

Does the analysis identify all parties who 

would receive benefits and assess the incidence of benefits? 

9. Does the proposed rule or the RIA present evidence that the agency used the Regulatory Impact Analysis? 

Does the proposed rule or the RIA assert that 

the RIA's results affected any decisions? 
74 

The Quality aud Use of Regulatory Analysis 

How many aspects of the proposed rule did 

the RIA affect? 

How significant are the decisions the RIA affected? 

10. Did the agency maximize net benefits or explain why it chose another option? 

Did the RIA calculate net benefits of one or 

more options so that they could be compared? 

Did the RIA calculate net benefits of all options considered? 

Did the agency either choose the option that 

maximized net benefits or explain why it 

chose another option? 

How broad a range of alternatives did the 

agency consider? 

11. Does the proposed rule establish measures and 

goals that can be used to track the regulation's 

results in the future? 

Does the RIA contain analysis or results that 

could be used to establish goals and measures 

to assess the results of the regulation in the 

future? 

In the RIA or the proposed rule, docs the 

agency commit to performing some type of 

retrospective analysis of the regulation's effects? 

17 

Does the agency explicitly articulate goals for 

major outcomes the rule is supposed to affect? 

Does the agency establish measures for major 

outcomes the rule is supposed to affect? 

Does the agency set targets for measures 

of major outcomes the rule is supposed to 

affect? 

12. Did the agency indicate what data it will use to 

assess the regulation's performance in the future and establish provisions for doing so? 

Does the RIA or proposed rule demonstrate 

that the agency has access to data that could 

be used to assess some aspects of the regulation's performance in the future? 

Would comparing actual outcomes to outcomes predicted in the RIA generate a reasonably complete understanding of the regulation's effects? 

Does the agency suggest it will evaluate future 

effects o[ the regulation using data it has access to or commits to gathering? 

Does the agency explicitly enumerate data it 

will use to evaluate major outcomes the regulation is supposed to accomplish in the future? 

Does the RIA demonstrate that the agency 

understands how to control for other factors 

that may affect outcomes in the future? 

APPENDIX B: CROSS-WALK OF 2010 OMB REGULATORY IMPACT ANALYSIS CHECKLIST 

WITH OUR EVALUATION CRITERIA 

OMB Checklist 

Docs the RIA include a reasonably detailed 

description of the need for the regulatory action? 

Does the RIA include an explanation of how the 

regulatory action will meet that need? 

Does the RIA use an appropriate baseline (i.e., best 

assessment of how the world would look in the 

absence of the proposed action)? 

Is the information in the RIA based on the best 

reasonahly obtainable scientific. technical, and 

economic information and is it presented in an 

accurate, clear, complete. and unbiased manner? 

Arc the data. sources, and methods used in the RIA 

provided to the public on the Internet so that 

a qualified person can reproduce the analysiS? 

Our Evaluation Criteria 

Criterion 6: How well does the analYSis demonstrate the existence of a market 

failure or other systemic problem the regulation is supposed to solve? 

Criterion 5: How well does the analYSis identify the desired outcomes and 

demonstrate that the regulation will achieve them? 

Criterion 7, question D: Does the analysis adequately assess the baseline-what 

the state of the world is likely to be in the absence of further federal action? 

Criterion 2: How verifiahle are the data used in the analysis 

Criterion 3: How verifiable arc the models or assumptions used in the analysis? 

Criterion 4: Was the analysis comprehensible to an informed layperson? 

Criterion 3 includes an assessment of whether the models and assumptions are 

based on peer-reviewed or otherwise reliable publications. However, the 

evaluation does not assess the quality of the underlying science. 

Criterion 1 takes the first step by assessing how easily the RIA itself can be 

found on the Internet. 

Criteria 3 and 4 include nn assessment of how easily the reader could find the 

underlying data, sources, and methods from information or links provided in 

the RIA or the Federal Register notice. 
75 

18 

To the extent feasible, does the RIA quantify and 

monetize the anticipated benefits from the 

regulatory action? 

To the extent feasible, does the RIA quantify and 

monetize the anticipated costs'? 

Does the RIA explain and support a reasoned 

determination that the benefits of the intended 

regulation justify its costs (recognizing that some 

benefits and costs are difficult to quantify)? 

Docs the RIA assess the potentially effective and 

reasonably feasible alternatives? 

Does the preferred option have the highest net henefits 

(including potential economic, public health and 

safety, and other advantages; distributive impacts; 

and equity), unless a statute requires a different 

approach? 

Does the RIA include an explanation of why the 

planned regulatory action is preferable to the 

identified potential alternatives? 

Does the RIA use appropriate discount rates for the 

benefits and costs that are expected to occur in the 

future? 

Does the RIA include, if and where relevant an 

appropriate uncertainty analysis? 

Does the RIA include, if and where relevant, a 

separate description of the distributive impacts 

and equity (including transfer payments and effects 

on disadvantages or vulnerable popUlations)? 

Does the analysis include a clear, plain-language 

executive summary, induding an accounting 

statement that summarizes the benefit and cost 

estimates for the regulatory action under 

consideration, induding the qualitative and 

non monetized benefits and costs? 

Does the analysis include a clear and transparent table 

presenting (to the extent feasible) anticipated 

benefits and costs (qualitative and quantitative)? 

Goals and measures to assess results of the regulation in 

the fUlure-No content. 

Provisions for gathering data to assess results of the 

regulation in the future-No contenr. 

EUig and McLaughliu 

Criterion 5, question 2: How well does the analysis identify how the outcomes 

arc to be measured? 

Multiple questions under Criterion 8 (Benefits and Costs) assess how well the 

analysis identifies, quantifies, and monetizes costs. 

Criterion 8, question F: Does the analysis identify the approach that maximizes 

net benefits? 

Criterion 8, question G: Does the analysis identify the cost-effectiveness of each 

alternative considered? 

Criterion 7: How well does the analysis assess the effectiveness of alternative 

approaches? 

Criterion 10: Did the agency maximize net benefits or explain why it chose 

another option? 

Criterion 9: Does the proposed rule or RIA present evidence that the agency 

used the regulatory analysis? 

Criterion 10: Did the agency maximize net benefits or explain why it chose 

another option? 

Considered under Criterion 5, question 2: How well does the analysis identify 

how the outcomes are to be measured?, as well as several questions about 

measurement and comparison of benefits and costs under Criterion 8 

(Benefits and Costs). 

Criterion 5, question E: Does the analysis adequately assess uncertainty about 

the outcomes? 

Criterion 6, question D: Does the analysis adequately assess uncertainty about 

the existence and size of the problem? 

Criterion 8, question E: Does the analysis adequately address uncertainty about 

costs? 

Criterion 8, question H: Does the analysis identify all parties who would bear 

costs and assess the incidence of costs? 

Criterion 8, question I: Does the analysis identify all parties who would receive 

benefits and assess the incidence of benefits? 

Criterion 4: Was the analysis comprehensible to an infonned layperson? 

Criterion 4: Was the analysis comprehensible to an infonned layperson? 

Criterion 11: Does the proposed rule establish measures and goals that can be 

used to track the regulation's results in the future? 

Criterion 12: Did the agency indicate what data it will use to assess the 

regulation's performance in the future and establish provisions for doing so? 
76 

The Quality and Use of Regulatory Analysis 

from 

diagonal 

o 

1 

2 

>2 

diagonal 

0 

1 

2 

>2 

APPENDIX C: EVALUATION OF INTERRATER RELIABILITY Table A.I. All Questions 

Observations Percentage 

229 0.424 

198 0.367 

79 0.146 

34 0.063 

Table A.n. Question 1 

Observations Percentage 

22 0.489 

11 0.244 

7 0.156 

5 0.111 

19 
77 

20 ElJig and McLaughlin 

Table A.m. Question 2 

Distance 

from 

diagonal Observations Percentage 

0 26 0.578 

1 12 0.267 

2 7 0.156 

>2 0 0 

Table A.IV. Question 3 

Distance 

from 

diagonal Observations Percentage 

0 22 0.489 

1 18 0.400 

2 5 0.111 

>2 0 0 
78 

The Quality aud Use of Regulatory Analysis 21 

Table A. V. Question 4 

Distance 

from 

diagonal Observations Percentage 

0 21 0.467 

1 15 0.333 

2 6 0.133 

>2 3 0.067 

Table A.VI. Question 5 

Distance 

from 

diagonal Observations Percentage 

0 13 0.289 

1 14 0.311 

2 15 0.333 

>2 3 0.067 
79 

22 EUig and McLaughlin 

Table A. VII. Question 6 

diagonal Observations Percentage 

0 16 0.356 

1 18 0.400 

2 6 0.133 

>2 5 0.111 

Table A.VIR. Question 7 

Distance 

from 

diagonal Observations Percentage 

0 21 0.467 

1 20 0.444 

2 4 0.089 

>2 0 0.000 
80 

The Quality aud Use of Regulatory Aualysis 

Table A.IX. Question 8 

Distance 

from 

diagonal Observations Percentage 

o 

1 

2 

>2 

o 

1 

2 

>2 

19 

22 

4 

o 

0.422 

0.489 

0.089 

0.000 

Table A.x. Question 9 

15 

18 

8 

4 

0.333 

0.400 

0.178 

0.089 

23 
81 

24 

from 

diagonal 

o 

1 

2 

>2 

Distance 

from 

diagonal 

0 

1 

2 

>2 

Table A.XL Question 10 

Observations Percentage 

14 0.311 

19 0.422 

6 

6 

0.133 

0.133 

Table A.xIl. QUestion 11 

Observations Percentage 

19 0.422 

17 0.378 

4 0.089 

5 0.111 

EUig and McLaughlin 
82 

The Qualityaud Use of Regulatory Analysis 25 

Table A.xm. Question 12 

from 

diagonal 

o 

1 

2 

>2 

REFERENCES 

Observations 

21 

14 

7 

3 

1. Obama B. Improving regulation and regulatory reviewExecutive Order 13563. Presidential Documents. Federal 

Register. 2011; 76(14):3821-3823. 

2. Clinton WJ. Regulatory planning and review-Executive Or~ 

def 12866. Presidential Documents. Federal Register, 1993; 

58(190):51734-51744. 

3. Dudley SE. Lessons learned, chaUenges ahead. Regulation, 

2009; 32(2):6-11. 

4. Ackerman F, Heinzerling L. Priceless: On Knowing the Price 

of Everything and the Value of Nothing. New York: New 

Press, 2004. 

S, Parker RW. Grading the government. University of Chicago 

Law Review, ZOO3; 70(4):1345-1486. 

6. Graham J. Saving lives through administrative law and 

economics. University of Pennsylvania Law Review, 2008; 

157(1):395-540. 

7. Viscusi WK. Fatal Trade-Ofts: Public and Private Responsibilities for Risk. Oxford: Oxford University Press, 1992. 

8. Sunstein CR The Cost-Benefit State. Chicago: American Bar 

Association, 2002. 

9. Revesz RL, Livermore MA Retaking Rationality: How CostBenefit Analysis Can Better Protect the Environment and 

Our Health. New York: Oxford University Press, 200S. 

10. Harrington W, Heinzerling L, Morgenstern R Refonning 

Regulatory Impact Analysis. Washington, DC: Resources for 

the Future Press, 2009. 

11. Katzen S. Cost-benefit analysis: Where should we go from 

here? Fordham Urban Law Journal, 2006; 33(4):1313-1317. 

12. Hahn RW, Sunstein CR. A new executive order for improving 

federal regulation? Deeper and wider cost-benefit analysis. 

University of Pennsylvania Law Review, 2002; 150(5):1489-

1552. 

13. Office of Management and Budget Circular A-4,2003. Available at http://www.whitehouse.gov!omb/circulars_a004..a-4/, 

Accessed on April 27, 2010. 

Percentage 

0.467 

0.311 

0.156 

0.067 

14. McGarityT. Reinventing Rationality. Cambridge: Cambridge 

University Press, 1991. 

15. Fraas A The role of economic analysis in shaping regulatory 

policy. Law and Contemporary Problems, 1991; 54(4):113-

125. 

16. Morgenstern R. Economic Analysis at EPA. Washington, 

DC: Resources for the Future, 1997. 

17. Posner EA. Transfer regulations and cost-effectiveness analysis. Duke Law Journal, 2003; 53(3):1067-1110. 

18. Harrington W, Morgenstern R, Nelson P. On the accuracy 

of regulatory cost estimates. Journal of Policy Analysis and 

Management, 2000; 19(2):297-332. 

19. Validating Regulatory Analysis: Report to Congress on the 

Costs and Benefits of Federal Regulations and Unfunded 

Mandates on State, Local, and Tribal Entities. Washington, 

DC: Office of Information and Regulatory Affairs, 2005. 

20. Harrington W. Grading Estimates of the Benefits and Costs 

of Federal Regulation: A Review of Reviews. Discussion 

Paper 06--39. Washington, DC: Resources for the Future, 

2006. 

21. Air Pollution: Information Contained in EPA's Regulatory 

Impact Analyses Can be Made Clearer. Washington, DC: 

Government Accountability Office, 1997. 

22. Regulatory Reform: Agencies Could Improve Development, 

Documentation, and Clarity of Regulatory Economic Anal~ 

yses. Washington, DC: Government Accountability Office, 

1998. 

23. Hahn RW, Burnett J, Chan YI, Mader E, Moyle P. Assessing 

regulatory impact analyses: The failure of agencies to comply with Executive Order 12866. Harvard Journal of Law and 

Public Policy, 2000; 23(3):859-871. 

24. Hahn RW, Dudley P. How well does the government do costbenefit anaJysis? Review of Environmental Economics and 

Policy, 2007; 1(2):192-211. 

25. Hahn RW, Lutter RW, Viscusi WK. Do Federal Regulations 

Reduce Mortality? Washington, DC: AEI-Brookings Joint 

Center for Regulatory Studies, 2000. 
83 

26 

26. Hahn RW, Utan R. Counting regulatory benefits and costs: 

Lessons for the U.S. and Europe. Journal of International 

Economic Law, 2005; 8(2):473-508. 

27. Hahn RW, Tetlock Pc. Has economic analysis improved reg~ 

ulatory decisions? Journal of Economic Perspectives, 2008; 

22(1 ):67-84. 

28. Belcore J, EUig 1. Homeland security and regulatory analysis: 

Are we safe yet? Rutgers Law Journal, 2008; 40(1):1-96. 

29. Fraas A, Lutter R. The Challenges of Improving the Economic Analysis of Pending Regulations: The Experience of 

OMB Circular A-4. Washington, DC: Resources for the Future,2010. 

30. Shapiro S, Morrall 1. The Triumph of Regulatory Politics: 

BeA and Political Salience. Rutgers University, Working 

paper,201l. 

31. West W. Inside the black box: The development of proposed 

rules and the limitations of procedural controls. Administra~ 

tion & Society, 2009; 41(5):579-599. 

32. Williams R. The Influence of Regulatory Economists in 

Federal Health and Safety Agencies. Mercatus Center at 

George Mason University, Working paper. 200S. 

33. Department of Transportation. Passenger Car and Light 

Truck Corporate Average fuel Economy 2011-2015. Regu~ 

latory Impact Analysis, 200S. 

34. Department of Labor. Class exemption for provision of in~ 

vestment advice. Proposed Rules. Federal Register, 2008; 

73(164):49895-49920. 

35. Department of Labor. Class exemption for provision 

of investment advice. Notices. Federal Register, 2008; 

73(164):49924-49932. 

36. Environmental Protection Agency. Effluent limitations 

guidelines and standards for the construction and devclop~ 

ment point source category. Proposed rules. Federal Register. 

2OOS; 73(230);72561-72614. 

37. Department of Health and Human Services. Proposed hospice wage index for fiscal year 2009. Proposed rules. Federal 

Register. 2008; 73(85):24000-24035. 

38. Department of Health and Human Services. Proposed 

changes to the hospital inpatient prospective payment systems and fiscal year 2009 rales. Proposed Rules. Federal Register, 2008; 73(84):23528-23938. 

19. 2009 Report to Congress on the Benefits and Costs of Federal Regulations and Unfunded Mandates on State. Local, 

and Tribal Entities. Washington, DC: Office of Information 

and Regulatory Affairs, 2009. 

40. GPRA Modernization Act of 2010, Pub. L 111~352, 124 Stat. 

3866 (January 4. 2011). 

Ellig and McLaughlin 

41. Department of Homeland Security. Air/Sea Biometric Exit 

Project Regulatory Impact Analysis, 2008. 

42. Environmental Protection Agency. Proposed Lead NAAQS, 

Regulatory Impact Analysis, 2008. 

43. Department of Transportation. Pipeline Safety: Integrity 

Management Program for Gas Distribution Pipelines. Regulatory Impact Analysis, 2008. 

44. Department of Transportation. Congestion Management 

Rule for New York LaGuardia Airport. Regulatory Impact 

Analysis, 2008. 

45. 2008 Report to Congress on the Benefits and Costs of Federal 

Regulations and Unfunded Mandates on States, Local. and 

Tribal Entities. Washington, DC: Office of Infonnation and 

Regulatory Affairs, 2008. 

46. Schlitz A. Common-sense and scientific interpretation of human action. Philosophy and Phenomenological Research, 

1953; 14:(1):1-38. 

47. Lavoie D. Hermeneutics, subjectivity, and the LesterMachlup debate: Toward a more anthropological approach 

to empirical economics, Pp. 167-184 in Samuels WJ (ed). 

Economics As Discourse: An Analysis of the Language 

of Economists. Norwell. MA: Kluwcr Academic Publishers; 

1990. 

48. Siegel S. Castellan NJ. Nonparametric Statistics for the 

Behavioral Sciences, 2nd Ed. New York: McGraw-Hill, 

1988. 

49. Perreault WD, Laurence LE. Reliability of nominal data 

based on qualitative judgments. Journal of Marketing Research. 1989; 26(2):135-148. 

50. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics, 1977; 33(1):159-

174. 

51. Hatry HP. Performance Measurement. Washington, DC: 

Urban Institute Press, 1999. 

52. Fraas AG, Cncertainty in EPA's Analysis of Air Pollution 

Rules: A Status Report. Resources for the Future Discussion 

Paper 10-04, February, 2010. 

53. McLaughlin P A. The consequences of midnight regulations 

and other surges in regulatory activity. Public Choice, 2011; 

147(3):395-412. 

54. McLaughlin PA, Ellig J. Does OIRA review improve the 

quality of regulatory impact analysis? Evidence from the final year of the Bush II administration, Administrative Law 

Review. 2011; 63(SE fortheoming):1D2-123. 

55. Ellig J, McLaughlin P A. The Quality and Use of Regulatory 

Analysis in 2008. Mercatus Center at George Mason Univer~ 

sity, Working paper, 2010. 
84 

PREPARED STATEMENT OF ROBERT S. KIEVAL

Good morning. Congressman Brady, Senator Klobuchar, members of the Joint 

Economic Committee, it is an honor to have this opportunity to address you today 

and to endeavor to answer your questions. I have a deep respect for the work of 

this Committee, and for all of the policy makers striving to preserve and foster innovation in the United States. My name is Robert Kieval, and I am the Founder and 

Chief Technology Officer of CVRx, a Minneapolis-based medical device company. I 

have worked in the medical technology industry for over 20 years, with experience 

in both a large medical device company and in the entrepreneurial, start-up environment. In addition to my work at CVRx, I serve on the Board of Directors of two 

industry advocacy organizations, the Medical Device Manufacturers Association 

(MDMA) here in Washington, DC, and LifeScience Alley (LSA) in Minneapolis. 

In our 11 year history, CVRx has developed an implantable medical device that 

is intended to treat two prevalent cardiovascular diseases: hypertension, or high 

blood pressure, and chronic heart failure. Together, these diseases afflict over 80 

million Americans. They are a primary cause of more than 128,000 deaths each year 

in the United States, and represent an annual economic burden of over $100B to 

CMS and private insurers in health care costs and lost productivity. They are diseases for which effective new treatments are desperately needed. Our product was 

approved in Europe in 2011 for the treatment of hypertension, and it is under clinical evaluation here in the U.S. 

The medical technology industry accounts for at least 400,000 jobs in the U.S., 

supports nearly 2 million additional jobs in adjacent industries, and remains one of 

the few American industries that is a net exporter of goods and services. Small businesses like CVRx, often with fewer than 50 employees, are a vital source of innovation and comprise approximately 80% of the industry. 

Companies like ours, with a single product focus and no alternative revenue 

streams, depend on outside investment for our existence. Investors require assurance of a reasonable and predictable path to product approval. Ambiguous or overly 

burdensome approval thresholds can fatally inhibit investment and prevent development of a potentially important new therapy. This is especially critical for patients 

suffering from diseases that have few treatment options. 

Since 2005, the time and capital required for a company to get a clear definition 

of its required regulatory pathway, to negotiate product testing and clinical trial requirements, and to obtain an approval or clearance decision once a completed application has been submitted have risen dramatically. Small, venture capital-backed 

companies typically spend $500,000 to $2 million per month to operate. A six to 

twelve month delay, for example, in reaching agreement with the FDA about a clinical trial design issue, or in the time required to complete an overly burdensome 

clinical trial, could result in the loss of precious time to deliver a potentially lifesaving new treatment to patients, and require a company to raise millions of dollars 

of additional capital in order to get through the approval process. 

The regulatory approval process itself has become increasingly inefficient, inconsistent and unpredictable, and the level of clinical evidence required to obtain product approval has also continued to rise. This has led to a situation in which patients 

outside of the U.S. frequently gain access to American innovation and technology 

an average of two years before American patients do. In many cases it has also led 

to jobs and Research & Development moving overseas, weakening the competitiveness of our medical technology industry. Such are also the cases for CVRx. While 

we work through the regulatory approval process here at home, our product is being 

used to treat patients in Germany, Italy, the Netherlands, Hungary and Turkey. I 

just returned from a trip to Europe where I heard firsthand from doctors how patients there are benefiting from our technology. As a result, the jobs that we are 

adding are also largely overseas. Finally, the recently enacted Medical Device Tax, 

a 2.3% excise tax on revenues irrespective of a company’s earnings, has put additional financial pressure on companies and has compounded these difficulties. For 

large companies, these often represent issues of profitability. For small companies, 

they may be issues of survivability. 

A 2011 study by the National Venture Capital Association (NVCA) found that 

U.S. venture capital firms have and will continue to decrease their investment in 

biotechnology and medical device start-ups and shift focus away from the United 

States toward Europe and emerging markets. In that study, FDA regulatory challenges were identified as having the highest impact on these investment decisions. 

The first quarter 2013 MoneyTree report released by PriceWaterhouse Coopers and 

NVCA reflects a continued decline in medical technology investment. In fact, the 

Life Science sector experienced a dramatic drop to $98 million, the lowest quarterly 

amount since the third quarter of 1996. To put this in perspective, in 2007 alone, 
85 

116 early stage medical device companies raised approximately $720 million in initial venture capital. These early stage investments are the single largest indicator 

of future innovations and breakthroughs, and thus the current environment does 

not bode well for patients. 

To be sure, federal regulators and policy makers have acknowledged and have 

been working to address these issues. Our industry appreciates the overwhelming 

bipartisan support for The Food and Drug Administration Safety and Innovation Act 

of 2012 (FDASIA). This legislation reauthorized the medical device user fee program 

for five years and includes many reforms that, if implemented as intended, will be 

a real benefit for patients, innovation and our economy. 

These reforms include earlier substantive interactions between FDA and industry, 

better manager-to-reviewer ratios to deal with capacity issues and shared outcome 

goals that will track performance based on calendar days. 

While it is too soon to evaluate the ultimate impact of these measures, the industry is beginning to see early evidence of improvements, and CVRx has also had positive experience in this regard. In addition, the Medical Device Innovation Consortium, a public-private partnership that had its roots with LSA in Minnesota but has 

now become a national program, is a promising example of government and industry 

working collaboratively to identify and improve regulatory inefficiencies. While industry, including MDMA, AdvaMed and LSA, endeavor to work with all stakeholders to improve the regulatory environment, we will also be relying on the FDA 

to utilize its user fees and appropriations efficiently and effectively. 

Looking forward, opportunities remain for further improvements, and we need to 

continue to work together so that the United States doesn’t lose its leadership position in healthcare innovation. The FDA has a crucial mission to protect the public 

health. Clearly this means providing reasonable assurance that products are safe 

before they’re made available to patients. However, I believe it also means that patients in need of effective treatments should not be unduly deprived of new innovations because of an inefficient or overly burdensome approval process. Successfully 

implementing this aspect of its mission will depend on a cultural change at the FDA 

as much as it will rely on processes and procedures. 

As mentioned above, increasing numbers of medical technology companies are developing and evaluating their products in clinical trials outside the United States. 

Given the millions of dollars of investment that this entails, we look forward to 

working with FDA on ways to better leverage these data domestically in a meaningful manner. 

I am also encouraged by reports that the FDA is currently focusing on three highly practical priorities of 1) improving efficiency in clinical trials, 2) balancing the 

premarket and postmarket process, and 3) identifying ways to shorten the lag between product approval by the FDA and reimbursement approval by CMS and/or 

private payers. 

Capitalizing on many of these opportunities will require close collaboration between patients, industry and the FDA. However, Congress can play an important 

role as well, by ensuring that all parties continue to work in a highly constructive 

and productive manner. 

Thank you for your attention, and I look forward to your questions. 

Æ