U.S. GOVERNMENT PRINTING OFFICE

WASHINGTON : 

For sale by the Superintendent of Documents, U.S. Government Printing Office

Internet: bookstore.gpo.gov Phone: toll free (866) 512–1800; DC area (202) 512–1800

Fax: (202) 512–2250 Mail: Stop SSOP, Washington, DC 20402–0001

29–503 PDF 2007

S. Hrg. 109–923

AUTOPILOT BUDGETING: WILL CONGRESS

EVER RESPOND TO GOVERNMENT

PERFORMANCE DATA?

HEARING

BEFORE THE

FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT 

INFORMATION, AND INTERNATIONAL 

SECURITY SUBCOMMITTEE

OF THE

COMMITTEE ON 

HOMELAND SECURITY AND 

GOVERNMENTAL AFFAIRS 

UNITED STATES SENATE

ONE HUNDRED NINTH CONGRESS

SECOND SESSION

JUNE 13, 2006

Available via http://www.access.gpo.gov/congress/senate

Printed for the use of the Committee on Homeland Security 

and Governmental Affairs

(

(II)

COMMITTEE ON HOMELAND SECURITY AND GOVERNMENTAL AFFAIRS 

SUSAN M. COLLINS, Maine, Chairman 

TED STEVENS, Alaska 

GEORGE V. VOINOVICH, Ohio 

NORM COLEMAN, Minnesota 

TOM COBURN, Oklahoma 

LINCOLN D. CHAFEE, Rhode Island 

ROBERT F. BENNETT, Utah 

PETE V. DOMENICI, New Mexico 

JOHN W. WARNER, Virginia 

JOSEPH I. LIEBERMAN, Connecticut 

CARL LEVIN, Michigan 

DANIEL K. AKAKA, Hawaii 

THOMAS R. CARPER, Delaware 

MARK DAYTON, Minnesota 

FRANK LAUTENBERG, New Jersey 

MARK PRYOR, Arkansas

MICHAEL D. BOPP, Staff Director and Chief Counsel 

MICHAEL L. ALEXANDER, Minority Staff Director 

TRINA DRIESSNACK TYRER, Chief Clerk 

FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT INFORMATION, AND 

INTERNATIONAL SECURITY SUBCOMMITTEE 

TOM COBURN, Oklahoma, Chairman 

TED STEVENS, Alaska 

GEORGE V. VOINOVICH, Ohio 

LINCOLN D. CHAFEE, Rhode Island 

ROBERT F. BENNETT, Utah 

PETE V. DOMENICI, New Mexico 

JOHN W. WARNER, Virginia 

THOMAS CARPER, Delaware 

CARL LEVIN, Michigan 

DANIEL K. AKAKA, Hawaii 

MARK DAYTON, Minnesota 

FRANK LAUTENBERG, New Jersey 

MARK PRYOR, Arkansas

KATY FRENCH, Staff Director 

SHEILA MURPHY, Minority Staff Director 

JOHN KILVINGTON, Minority Deputy Staff Director 

LIZ SCRANTON, Chief Clerk 

(III)

C O N T E N T S 

Opening statements: Page 

Senator Coburn ................................................................................................. 1

Senator Carper ................................................................................................. 6

WITNESSES 

TUESDAY, JUNE 13, 2006

Hon. Clay Johnson III, Deputy Director for Management, U.S. Office of Management and Budget ............................................................................................ 8

Eileen Norcross, Government Accountability Project, Mercartus Center at 

George Mason University .................................................................................... 9

Adam Hughes, Director for Federal Fiscal Policy, OMB Watch .......................... 11

ALPHABETICAL LIST OF WITNESSES

Hughes, Adam: 

Testimony .......................................................................................................... 11

Prepared statement with attachments ........................................................... 89

Johnson, Hon. Clay III: 

Testimony .......................................................................................................... 8

Prepared statement with attachments ........................................................... 25

Norcross, Eileen: 

Testimony .......................................................................................................... 9

Prepared statement .......................................................................................... 49

APPENDIX 

‘‘A Working Paper in Government Accountabiity’’ by Eileen Norcross and 

Kyle McKenzie ...................................................................................................... 61

(1)

AUTOPILOT BUDGETING: WILL CONGRESS 

EVER RESPOND TO GOVERNMENT 

PERFORMANCE DATA? 

TUESDAY, JUNE 13, 2006

U.S. SENATE,

SUBCOMMITTEE ON FEDERAL FINANCIAL MANAGEMENT,

GOVERNMENT INFORMATION, AND INTERNATIONAL SECURITY

OF THE COMMITTEE ON HOMELAND SECURITY AND

GOVERNMENTAL AFFAIRS,

Washington, DC. 

The Subcommittee met, pursuant to notice, at 2:57 p.m., in room 

342, Dirksen Senate Office Building, Hon. Tom Coburn (Chairman 

of the Subcommittee) presiding. 

Present: Senators Coburn and Carper. 

OPENING STATEMENT OF SENATOR COBURN 

Senator COBURN. Good afternoon. The Federal Financial Management Subcommittee of the Homeland Security and Governmental Affairs Committee will come to order. Senator Carper will 

be here in a moment. We apologize for the delay. There was an official photo. We also have a conflict. There is a briefing ongoing now 

by the Secretary of Defense and the Secretary of State, which will 

limit Senator Carper’s time with us. So we are going to go on and 

go forward so we have it in the record. I apologize for the conflicting schedules. 

Americans have a crazy idea, that they should get something for 

their money, even when the money is spent by the government. It 

is a simple concept, and in policy-speak we call it performancebased budgeting. I know I am new in the Senate, but I am still surprised by the level of resistance in Washington to holding people 

accountable by measuring their performance. And it is a difficult 

thing to do. A multitrillion-dollar government imposing some sort 

of standardized outcome evaluation is difficult at best, and what it 

implies is that the tool will be very crude. But that does not say 

we should not attempt to make measurements, and I want to be 

one of many who should commend both Mr. Johnson and the Bush 

Administration, and the President himself, for being the first to attempt to do it. 

It is not novel. It is required in the competitive business environment that we find ourselves worldwide. It is being used effectively 

in many State governments, and it is something that is long overdue. The Performance Assessment Rating Tool (PART) was first introduced by the President 4 years ago as a tool to review the 

2

strengths and weaknesses of government programs to influence 

funding and programmatic decisions. The annual PART reports 

offer needed sunshine in government and provide good data for 

government managers to improve their programs. 

Today, the Office of Management and Budget has reviewed 793 programs, which account for $1.47 trillion in taxpayer money. Almost a third of these programs have proven to not meet up to 

standards based on the PART analysis. I have already admitted 

that it is a blunt tool. One-third of $1.5 trillion is $500 billion. 

Maybe this is why PART scores so far have created a stir not only 

among the agencies but among the Members of Congress who make 

budgeting decision. 

Some Members of Congress want to stick their heads in the sand 

and keep funding pet programs on autopilot year after year. To my 

amazement, just last week, the Appropriations Subcommittee that 

funds the Departments of Labor and Health and Human Services 

passed language prohibiting the use of PART assessments on those 

agencies. They may not like PART’s message, but they should not 

shoot the messenger. This sort of Orwellian immunization against 

any hint that our favorite programs may not be performing up to 

the idealized, utopian goals of their Congressional champions is one 

of the reasons why Americans are mad at Congress. 

The approval ratings for Congress are in the tank, and this prohibition of accountability for failing government is why the voters 

who fork over their hard-earned dollars every year may just have 

something new to say come this November. I am not sure why so 

many of my colleagues are afraid of assessment tools on performance. It may reflect their own performance. 

As part of our investigation for this hearing, we learned that low 

PART ratings do not always mean that OMB will recommend a 

budget cut or a cut in the program or a recommendation to go on 

the terminations list. In some cases, programs rated ineffective 

have had budget reductions recommended. But in other cases, the 

reason they were low was because they were not funded appropriately to begin with, and therefore, they could not accomplish 

what they were intended to because they did not have adequate 

funding. 

Each program is unique, and I do not know that a PART score 

should be the last word. But I do know that measurement of performance is something that every member of a Congressional authorizing or Sppropriations committee should be reading and using 

to inform their oversight work. Congress consistently neglects the 

duty to conduct oversight of Federal programs and spending. Instead, we spend most of the time passing spending bills that ignore 

PART ratings, the President’s termination list, or any other performance data as if the spending were on autopilot. Congress might 

as well write a blank check. 

By 2008, OMB will have applied PART to the entire government. 

In the last 4 years, OMB has scored 793 government programs. 

Here are the results: 15 percent were found to be effective; 29 percent were found to be moderately effective; 28 percent were rated 

adequate; 4 percent were found to be ineffective—that is one in 

every 25 programs—25 percent could not demonstrate results to 

get a rating and were labeled results not demonstrated. 

3

I do not believe the spin that results not demonstrated can mean 

that the program is either good or bad; we just do not have enough 

information to tell. On the contrary, the results not demonstrated 

designation is a red flag marking a program so poorly conceived by 

us or so directionalist that that unaccountability seems to have 

been built into it by design. Programs rated ineffective or results 

not demonstrated account for $152 billion in budget authority. 

Imagine what we could do with $152 billion right now. The ideas 

are endless. 

Outside of Washington, DC, any business or family with finite resources sets priorities and creates a budget based on the actual 

amount of bang they get from their buck. It is only inside the Beltway where that kind of information is not considered relevant, and 

in fact, some are even attempting to ban the collection of such information. But then, it is only Washington where you never have 

to declare bankruptcy, and debt is allowed to grow on the backs of 

future generations with impunity. 

Let me give you one case study, and my co-chairman on this will 

disagree, but my firm believing is the following: We held a hearing 

last year on the Advanced Technology Program that was created in 

1988 to subsidize high-risk research and development. This program has never demonstrated results. What it has demonstrated is 

corporate welfare. Its 2002 PART report, that the majority of ATP 

grants go to multibillion dollar corporations and that the GAO has 

found that ATP projects are very similar to private sector R&D undertaken without a government subsidy. An amendment to eliminate this funding that was offered last year lost by a vote of 68 to 

29. In the end, Congress wasted a portion of $79 million last year 

for that program. The 2007 Senate budget resolution promises to 

fund the program at almost twice that amount. 

It would be one thing if we were operating in a surplus. Then, 

we could have a legitimate debate about whether to keep failing 

programs, hoping that they would improve, or to give that surplus 

back to the taxpayers. But that is not where we are today. With 

a debt burden of $25,000 per man, woman, and child, we simply 

cannot afford to keep funding programs that cannot prove their 

worth. Non-defense discretionary spending has increased 45 percent since 2001. The President has requested a $2.8 trillion budget, 

and that does not include any of the so-called emergency, ‘‘supplemental bills in our future,’’ nor does it include the late night pork 

barrel frenzy each time Congress schedules an appropriations bill 

vote. 

Entitlement spending will tank our economy if we do not do 

something to get spending under control. The question remains: 

How do we get Congress to act? I would like to see OMB sell their 

PART terminations list more aggressively, forcibly sell the reforms 

and savings to Congress, fight for the cuts by taking the terminations list to the American people with the power of the bully pulpit. The President should veto spending bills that continue to issue 

blank checks for failing programs. 

There is a bit of hope on the horizon. I was encouraged to see 

that the House Appropriations Committee wrote in their 2006 

budget savings report that the only way to establish accountability 

in the budget process is to stop spending on programs that have 

4

outlived their usefulness or could be delivered more effectively at 

the State or local level. I will believe that when I see it, but I welcome any help that we can get. 

The best place to start is by immediately defunding all programs 

on the termination list and adopting other PART recommendation 

reductions. Granted, the list only cuts $20 billion from a $2.8 trillion budget, but we have got to start somewhere. What is more, we 

should suspend the creation of any new program until further notice or it is compared to the existing programs that it is meant to 

supplement. We need sunset legislation that would phase out government agencies on a timed basis, where we force ourselves to 

look at them and to reauthorize them. 

These are challenging times, and we can no longer afford to run 

on a budget that is on cruise control. I want to thank our witnesses 

for being here. 

[The prepared statement of Chairman Coburn follows:]

OPENING PREPARED STATEMENT OF CHAIRMAN COBURN 

Americans have a crazy idea: They should get something for their money, even 

when the money is spent by government. It’s a simple concept—in policy-speak, we 

call it ‘‘performance-based budgeting.’’ I know I’m new in the Senate, but I’m still 

surprised by how much resistance there is in Washington to performance-based 

budgeting. 

Now, to be fair, taking a multi-trillion dollar government and imposing some sort 

of standardized outcome evaluation on it is difficult at best. So I concede that any 

instrument we use will be a blunt instrument. But I want to commend President 

Bush for being the first to try. 

The Performance Assessment Rating Tool (PART) was first introduced by the 

President 4 years ago as a tool to review the strengths and weaknesses of government programs to influence funding and programmatic decisions. 

The annual PART reports offer needed sunshine in government and provide good 

data for government managers to improve their programs. To date, the Office of 

Management and Budget has reviewed 793 programs which account for $1.47 trillion in taxpayer money. Almost a third of these programs have proven either totally 

ineffective or are not demonstrating results. One-third of $1.5 trillion is $500 billion. 

Maybe this is why the PART scores have created a stir—not only among the agencies, but among the Members of Congress who make budgeting decisions. Some 

Members of Congress want to stick their head in the sand and keep funding their 

pet programs, as if on autopilot, year after year. 

Just last week the House Appropriations subcommittee that funds the Departments of Labor, Education and Health and Human Services passed language prohibiting the use of PART assessments on those agencies. They may not like PART’s 

message, but they shouldn’t shoot the messenger. This sort of Orwellian immunization against any hint that our favorite programs may not be performing up to the 

idealized utopian goals of their Congressional champions is why Americans are mad 

at Congress. The approval ratings for Congress are in the tank, and this prohibition 

of accountability for failing government is why the voters who fork over their hardearned dollars every year may just have something to say come November. 

I’m not sure why some of my colleagues are so afraid of PART. As part of our 

investigation for this hearing, we learned that low PART ratings don’t always mean 

that OMB will recommend a budget cut or put the program on the Terminations 

List. In some cases, programs rated ‘‘ineffective’’ had budget reductions, but in other 

cases their budgets increased. Each program is unique and I don’t know that a 

PART score should be the last word, but I do know that the PART is something 

every member of a Congressional authorizing or Appropriations committee should 

be reading and using to inform their oversight work. 

You see, Congress consistently neglects the duty to conduct oversight of Federal 

programs and spending. Instead, we spend most of the time passing spending bills 

that ignore PART ratings, the President’s terminations list and any other performance data. It is as if we’re spending on ‘‘auto pilot’’—Congress might as well just 

write a blank check. 

By 2008, OMB will have applied PART to the entire government. In the last 4 

years OMB has scored 793 government programs. Here are the results: Just 15 perVerDate 0ct 09 2002 08:58 Jun 28, 2007 Jkt 029503 PO 00000 Frm 00008 Fmt 6633 Sfmt 6621 C:\DOCS\29503.TXT SAFFAIRS PsN: PAT
5

cent were found to be ‘‘effective’’; 29 percent were rated ‘‘moderately effective’’; 28 

percent were rated ‘‘adequate’’; 4 percent were found to be ‘‘ineffective’’; and 24 percent cannot demonstrate results to even get a rating and were labeled ‘‘results not 

demonstrated’’! Don’t believe the spin that ‘‘results not demonstrated’’ could mean 

that the program is either good or bad, we just don’t have enough information to 

tell. On the contrary—the ‘‘results not demonstrated’’ designation is a red flag marking a program so poorly conceived or directionless that unaccountability seems to 

have been built into it by design. 

Programs rated ‘‘ineffective’’ or ‘‘results not demonstrated’’ account for $152 billion 

in budget authority. Imagine what we could do with $152 billion. 

Outside of Washington DC, any business or family with finite resources sets priorities and creates a budget based on the actual amount of bang they get for their 

hard-earned buck. It is only inside the beltway where that kind of information isn’t 

considered relevant and in fact, some are trying hard to ban the collection of such 

information. But then, it’s only in Washington where you never have to declare 

bankruptcy and debt is allowed to grow on the backs of future generations with impunity. 

Let me give you one case study. We held a hearing last year on the Advanced 

Technology Program. The program was created by Congress in 1988 to subsidize 

high-risk research and development. The program cannot demonstrate results. It is 

corporate welfare. The 2002 PART reported that the majority of ATP grants go to 

multimillion dollar corporations and that the GAO has found that ATP projects are 

very similar to private sector R&D undertaken without a government subsidy. An 

amendment to eliminate funding for ATP that I offered last year was voted down 

in the Senate 68–29. In the end, Congress wasted another $79 million last year for 

the program. The 2007 Senate budget resolution promises to fund the program at 

almost twice that amount. 

It would be one thing if we were operating in a surplus. Then we could have a 

legitimate debate about whether to keep funding failing programs hoping they will 

improve or to give that surplus back to the taxpayers. But that’s not where we are 

today, with a debt burden of $25,000 per man, woman and child in America. We 

simply cannot afford to keep funding programs that cannot prove their worth. 

Nondefense discretionary spending has increased over 45 percent since 2001. The 

President has requested a $2.8 trillion budget and that doesn’t include any so called 

‘‘emergency’’ supplemental spending bills in our future, nor does it include the latenight pork-barrel frenzy each time Congress schedules an Appropriations bill vote. 

Entitlement spending will tank our economy if we don’t do something to get spending under control. 

The question remains, how do we get Congress to act? I would like to see OMB 

sell their PART and Terminations List more aggressively:

• Forcefully sell these reforms and savings to Congress. 

• Fight for these cuts, by taking the terminations list to the American people 

with the power of the bully pulpit. 

• The President should veto spending bills that continue to issue blank checks 

to failing programs.

There’s a bit of hope on the horizon—I was encouraged to see that the House Appropriations Committee wrote in their 2006 Budget Savings report that ‘‘the only 

way to establish accountability in the budget process is to stop spending on programs that have outlived their usefulness or could be delivered more effectively at 

the State or local level.’’ I’ll believe it when I see it, but I welcome any help we can 

get. 

The best place to start is by immediately defunding all programs on the Terminations List and adopting the other PART reduction recommendations. Granted, the 

list only cuts $20.4 billion from a $2.8 trillion budget, but we’ve got to start somewhere. What’s more, we should suspend the creation of any new program until further notice. We need ‘‘sunset’’ legislation that would phase out every single government agency, department or program after a certain deadline if the Congress fails 

to act or if the program consistently performs poorly. These are challenging times 

and we can no longer budget on cruise control. 

I want to thank our witnesses for being here today and for the time they spent 

preparing testimony.

Again, I apologize for the lateness of our attendance, and Senator 

Carper, you are recognized. 

6

OPENING STATEMENT OF SENATOR CARPER 

Senator CARPER. Thank you. 

Senator COBURN. And I have already explained that you will 

probably have to attend the briefing that is ongoing. 

Senator CARPER. Thanks, Mr. Chairman. To our witnesses, welcome. It is good to see each of you. We appreciate you joining us 

and providing your testimony today. As the Chairman mentioned, 

Secretary of State Rice and Secretary Rumsfeld are briefing us as 

we speak over in the Capitol, and I want to slip out in a little bit 

and hear what they have to say and hopefully rejoin you before you 

leave. 

Mr. Chairman, thank you for holding this hearing. It is an important hearing, as we both know. And as we have discussed in 

any number of our similar hearings in the past over the last couple 

of years, our country is facing a large budget deficit for as far as 

the eye can see, and we are just about to embark on another appropriations season here in Congress, where we will be called on to 

make some difficult decisions about what to do with relatively 

scarce Federal resources. 

At the same time as GAO and other observers have pointed out 

again, and again, we are at a crossroads in our history, where we 

need to decide what we want our government to do in the 21st 

Century. Nearly 5 years after the attacks of September 11, 2001, 

we have a whole new set of needs, a whole new set of priorities 

that must be balanced against some of our older needs and priorities in scores of popular programs. And with the challenge of retiring baby boomers, guys like me, our generation on the horizon, we 

just cannot afford to do all of the things that we might want to do. 

That is why initiatives like OMB’s Program Assessment and Rating Tool (PART) are interesting and, I think, important. We should 

never be afraid of taking a hard look at Federal programs, my programs, Senator Coburn’s programs, whatever, to determine whether or not they are accomplishing what was intended for them to accomplish when we first created them. And in this day and age, we 

simply cannot afford to allow either poorly conceived or poorly 

managed programs to continue without reform or, frankly, for a 

program that has run its course and achieved its goals, to continue 

draining resources from other, newer priorities. 

That said, we need to be certain that PART or whatever mechanism we use to make these evaluations is in itself effective. I think 

to be effective, a program like PART must be totally separated 

from politics and ideology, at least to the extent we can make that 

happen. It must be closely coordinated with existing mechanisms 

agencies and Congress use to align the budget with program goals 

and outcomes such as the older government Performance and Results Act. And perhaps just as importantly, we also need to make 

sure that a program’s intended beneficiaries outside of Washington 

have a say before an evaluation is actually completed. 

Let me just add in closing, if I could, Mr. Chairman, that we are 

not going to close the budget deficit, we know, by reducing spending on a program here or eliminating a program there, although 

every little bit helps. But even if a program were to eliminate every 

single one of the programs receiving failing grades through PART, 

7

I still think the savings would cover just a fraction of our budget 

deficit, but they would cover a portion of our budget deficit. 

Non-defense discretionary spending, which is the target of many 

of the spending reductions and program eliminations in the President’s budget proposals, make up a relatively small percentage of 

the Federal budget. I am sure we can find ways to improve the 

management of some of the funding in that 16 percent or even to 

find and eliminate waste and inefficient use of resources within 

that 16 percent. 

If we truly want to tackle the fiscal problems facing us right now, 

however, we, and that is the Congress and I think the Administration needs to take a look at the entire budgetary picture. We need 

to look on both the spending and on the revenue side, and we need 

to make some tough choices. 

Thank you, Mr. Chairman. Again, we look forward to your testimony today. 

Senator COBURN. Thank you, Senator Carper. 

I am going to ask the witnesses to limit their verbal testimony 

to 5 minutes. Your complete written statements will be made a 

part of the official hearing record, and we will hold our questions 

until you have given your testimony. 

Let me first introduce Clay Johnson III, Deputy Director for 

Management at OMB, and in his capacity, he has provided the government-wide leadership to the Executive Branch agencies to improve agency and program performance. Formerly, he served as Assistant to the President for Presidential Personnel, responsible for 

the organization that identifies and recruits approximately 4,000 

senior officials, middle management personnel, and part-time board 

and commission members. At OMB, he oversees PART process. 

Eileen Norcross, Senior Research Fellow, Government Accountability Project, The Mercatus Center at George Mason University; 

she joined that center as a research fellow in January 2003. Her 

research areas include the U.S. budget, the use of performance 

budgeting in the Federal Government, tax and fiscal policy, and environmental regulation. She is one of the leading experts on performance-based budgeting, and her scholarship plays a vital role in 

the debate on PART and the importance of measuring outcomes. 

Adam Hughes is the Director for Federal Fiscal Policy at OMB 

Watch. He oversees Federal budget and tax policy, income and 

wealth trends, and government performance issues at OMB Watch. 

Senator Carper and myself very much appreciate the work that 

OMB Watch has done in their pursuit of transparent and accountable government and for the support of the Federal Funding Accountability and Transparency Act that we both authored. This bill 

would create an online public database that itemizes Federal funding so taxpayers can see how their money is being spent. 

I want to welcome you all. I will recognize Mr. Johnson first. 

8

1The prepared statement of Mr. Johnson with attachments appears in the Appendix on page 

25. 

TESTIMONY OF CLAY JOHNSON III,1 DEPUTY DIRECTOR FOR 

MANAGEMENT, U.S. OFFICE OF MANAGEMENT AND BUDGET 

Mr. JOHNSON. Mr. Chairman, Senator Carper, thank you very 

much. 

The title of this hearing is Will Congress Ever Respond to Program Performance Data? In preparing my response, I rephrased 

that to Does Congress Care Whether Programs Work or Not? My 

answer is ‘‘I am not sure, but I sure hope so.’’ I believe that taxpayers want Congress to ensure that they, the taxpayers, get what 

they pay for. I believe that we all, to widely varying degrees, however, want Federal programs to do what they are supposed to do 

and get better every year. 

I believe that money is tight, as you all have pointed out, and 

the biggest opportunity we have to add new services and expand 

existing services to more citizens is through causing our existing 

programs to work better, not spend more money. I believe that career employees want to be held accountable for how their programs 

perform. They tell me this in focus groups. And I also believe that 

career employees care about how their programs perform. 

Because of this, I believe it is important to have certain things. 

I believe it is important to have clear outcome goals for each Federal program. We do not have that now. I believe it is important 

to have Federal program performance information that is objective, 

as objective and reliable as possible. I believe that we need to have 

lots of transparency about how well programs are performing. If we 

do all of this in the dead of night, it cannot be used to hold people 

accountable. 

I believe that we need lots of debate about these performance assessments and how to make them better. As you said, Mr. Chairman, program assessment is going to be a blunt instrument, particularly in the early years. And it will only get better every year, 

but a blunt instrument is better than no instrument at all. I also 

believe it is important to have lots of discussion about how to help 

programs work better. We talk a lot about using the PART to make 

budget decisions. I believe the primary use of PART information is 

to help programs get better. If we cut programs, we might save $10 

billion here or $15 billion there per year. If we cause 1 percent improvement in program performance each year, that is $28 billion a 

year. Two percent is obviously twice that. 

After 5 years of effort, not 5 months, comprehensive program 

performance information is still time consuming and very hard to 

come by. We have program outcome goals, performance information, and lots of transparency, which other countries and several 

States are working to adopt, and most good government groups applaud. What we do not have from most Members of Congress is a 

lot of constructive debate about these assessments and how to improve and use them to improve program performance. We have 

asked for feedback. We have asked for engagement by Congress but 

have not gotten it. 

Currently, a majority of Appropriations subcommittees have no 

objection to the way agencies use performance information to jusVerDate 0ct 09 2002 08:58 Jun 28, 2007 Jkt 029503 PO 00000 Frm 00012 Fmt 6633 Sfmt 6601 C:\DOCS\29503.TXT SAFFAIRS PsN: PAT
9

1The prepared statement of Ms. Norcross appears in the Appendix on page 49. 

2The Working Paper in Government Accountability appears in the Appendix on page 61. 

tify their budgets. Some of these subcommittees actually use the 

PART to justify program funding in their bills. A few Members of 

Congress have advanced greater use of performance information in 

decisionmaking. Congressmen Platts and Tanner have proposed 

separate pieces of legislation, while Senators like you, Senator 

Coburn, and Senators Carper, Ensign, and Allard have spoken out 

on the subject, and Congressmen Cuellar, Conaway, and Diaz–

Balart have spoken out on it as well. 

But these expressions of interest in program performance are the 

exceptions. There is a big, a huge opportunity for Congress to challenge programs to clearly define success and their plan for achieving it, and then to hold agencies accountable for doing what they 

said they were going to do. 

That concludes my remarks, and I look forward to any questions. 

Senator COBURN. Ms. Norcross. 

TESTIMONY OF EILEEN NORCROSS,1 SENIOR RESEARCH FELLOW FOR THE GOVERNMENT ACCOUNTABILITY PROJECT, 

THE MERCATUS CENTER AT GEORGE MASON UNIVERSITY 

Ms. NORCROSS. Thank you, Chairman Coburn, Senator Carper, 

for inviting me to testify today on Autopilot Budgeting: Will Congress Ever Respond to Government Performance Data? Our work 

in the Government Accountability Project at the Mercatus Center 

at George Mason University focuses closely on performance information in government, and I note that the views expressed in my 

testimony are not an official position of the university. 

I would like to submit for the record our paper on the results of 

the fiscal year 2007 PART for your reference. 

Senator COBURN. Without objection, the document will be included in the record.2

A program is a tool to achieve a policy goal. Do economic development programs lead to prosperous communities? Are homeland security programs protecting the Nation? Congress needs to know the 

answers to these questions in order to make decisions about how 

to spend resources. Without performance information, Congress 

cannot reliably accomplish its policy aims. Not knowing its consequences, Congress has created anywhere from 180 to 342 programs dealing with economic development in over 24 agencies; 44 

job training programs in nine agencies. 

Program duplication on this scale tells us that Congress is not 

sure which programs are reaching their goals. It has no way of 

comparing programs around common outcomes. Not knowing if a 

job training program is employing people means not spending 

money on programs that are employing people. Not evaluating programs on a regular basis prevents the program from effectively 

reaching grantees or delivering results; performance information 

from its dialogue between agencies, the Executive Branch, and 

Congress around jointly defined objectives. 

Congress took the initiative in 1993, when it passed GPRA. 

GPRA has encouraged the development of performance measures 

and data, but it was not until OMB’s Program Assessment Rating 

10

Tool that real progress towards developing measures was made. 

That is because the Administration does not just require the information; it uses it. Congress has identified the need for performance 

information. It must now commit to using it. Otherwise, measuring 

and gathering data is a paper exercise. 

For the past 2 years, the President has issued a major savings 

and reforms report detailing his reasons for terminating or reducing funding for programs. Of the 154 recommended for termination 

or reduction in funds last year, 54 were PARTed. The document indicates where PART played a role. Other factors include lack of a 

Federal role, obsolescence, or completion of mission. 

The Administration uses PART along with other information and 

does not limit itself to the evaluations. It does not automatically reward satisfactory programs or cancel underperforming ones. By 

contrast, the House Committee on Appropriations report ‘‘On Time 

and Under Budget’’ lists 53 programs that were terminated. It only 

offers explanations for three of the terminations. We do not know 

if the remainder were terminated because they were underperformers or politically easy choices. 

The Administration’s report gives a rationale for each recommendation. The House report only provides a list. Ultimately, 

the goal is not to randomly kill programs. Making judgments about 

how to fund agency activity should be constructive, not destructive. 

Performance information helps make policy effective. We want to 

know what works, what does not, and why. 

The only way to give budgetary decisions credibility is to base 

them on a reliable evaluation of their performance. Is PART that 

system? PART’s methodology has been criticized. Improvements 

can and should be made. But what is important about PART is not 

the ratings; it is the Management 101 questions PART asks of 

agency activity. Is the program purpose clear? Is it effectively targeted? Has it demonstrated progress towards its goals? These questions are the substance of PART. These are the questions Congress 

should be asking before allocating resources. 

PART has a few virtues. It has identified and catalogued agency 

activity. It is transparent. It holds programs accountable to the 

same standards. It measures outcomes. Once strength often cited 

as a weakness: PART rates programs on statutory limitations. 

Though a source of frustration for agencies, here, PART provides 

a service by identifying those aspects of a program that are barriers to success. The hope is that Congress review the statute to 

see if it is preventing the program from meeting its objectives. 

Some limitations of PART: It rates programs against their own 

performance. We would like to see PART advanced to compare like 

activities. In some cases, scores may not fully reflect program performance, and there is a potential for different budget examiners 

to reach different conclusions. We do not believe Congress should 

adopt PART wholesale. We hope Congress would consider using the 

kinds of questions PART is asking as the basis of developing its 

own method of evaluating agency activity based on common outcomes. 

Indiscriminate cancellation of programs discredits the budgetary 

process. We leave program managers confused about why their programs failed. Programs need to deliver according to clear expectaVerDate 0ct 09 2002 08:58 Jun 28, 2007 Jkt 029503 PO 00000 Frm 00014 Fmt 6633 Sfmt 6601 C:\DOCS\29503.TXT SAFFAIRS PsN: PAT
11

1The prepared statement of Mr. Hughes with attachments appears in the Appendix on page 

89. 

tions and be given a chance to perform. When you do not meet the 

expectations, reduction in funding or termination should be the result. It should not be a surprise. 

We believe performance information is best used in conjunction 

with other criteria. All of these form the basis against which Congress should continually scrutinize agency activity. Efforts to advance what PART has set in motion can only aid Congress in its 

work and give the American people confidence that our Nation’s 

problems are being solved. Thank you. 

Senator COBURN. Mr. Hughes. 

TESTIMONY OF ADAM HUGHES,1 DIRECTOR OF FEDERAL 

FISCAL POLICY, OMB WATCH 

Mr. HUGHES. Chairman Coburn, thank you for having me here 

today and for holding this hearing. As you mentioned, I am the 

Federal Fiscal Policy Director at OMB Watch. OMB Watch was 

founded in the 1980s and has spent over 20 years advocating for 

government accountability, transparency, and access to government 

information, and citizen participation in governmental processes. 

OMB Watch believes citizens must take an active role in holding 

their government accountable and that the Federal Government, 

when supported by sensible fiscal policy, can develop effective programs and safeguards that meet the public’s needs. 

The issue of government performance, as you mentioned earlier, 

has taken on added importance during the Bush Administration, as 

a combination of factors, some avoidable and some not, have 

plunged the Federal Government into debt. Large and sustained 

deficits over the past 5 years have made efficient use of government resources all the more important. 

In light of the anticipated budget crunch due to the baby 

boomers’ retirement over the coming decades, the fiscal situation in 

this country will only deteriorate further. Performance measurement can therefore become a particularly attractive alternative for 

those who want to set Federal priorities based on the current fiscal 

prospects of a strained and shrinking revenue base. 

OMB Watch has been commenting on government performance 

issues for the better part of its existence. We have spent increased 

time and resources analyzing the Government Performance and Results Act and the Program Assessment Rating Tool over the last 10 

years, as government itself has spent more time focusing on performance and results. We are strongly supportive of improving the 

Federal Government’s capacity to meet the public’s needs. OMB 

Watch has worked to protect and improve that capacity, and we 

have been open to the possibility of using performance measurement as one means for achieving those ends. 

We bring a strong belief in the importance and potential of government itself to the work we do, and because of that belief, we, 

perhaps maybe more than anyone else, want government to be responsible to community needs, spend money wisely, and accomplish 

its goals. We are advocates for government and therefore have a 

strong motivation to see government programs succeed. 

12

PART, however, is a very poor mechanism for measuring program performance and results, introducing biases and skewed ideological perspective into a model claiming to present consistent and 

objective performance data and evaluations of government programs. Oftentimes, the PART actually decreases the efficiency and 

effectiveness of government through increased administrative burdens, distracted managers, and compliance costs. 

Ironically, we feel the PART mechanism itself does not produce 

the right type of results to support and improve government. We 

believe PART ratings should not be directly connected to the budgeting process of Congress because of significant deficiencies within 

the mechanisms, namely, the substantial biases and limitations 

embedded within the tool and the additional distortion and manipulation we have observed in OMB’s actual application of the PART. 

Based on our studies of the PART and our longstanding commitment to open, accountable government that is responsive to the 

public’s needs, I would like to make three points today. First, we 

feel the PART continues a troubling trend we have seen in other 

recent Executive Branch proposals and even some Congressional 

proposals, namely, a trend towards increasing the power of the 

White House and the Executive Branch even into some areas that 

have been constitutionally designed to be committed to Congress. 

Second, the PART is a limited and distorted tool that should not 

be used for either management of programs or for budget and appropriations decisions. In both the design of the tool and the process by which the tool is implemented, PART systematically ignores 

the reality and the complexity of Federal programs and judges 

them based on standards that are often deeply incompatible with 

the purposes those very programs are expected to serve. As one 

agency contact memorably explained to us, PART assessments are 

akin to a baseball coach walking to the mound to remove his star 

player and then chastising him for not kicking enough field goals. 

My third point is that there is a better way. Specifically, Congress already has the means to investigate and produce far more 

sophisticated analyses of the usefulness, effectiveness, and results 

of government programs in a deliberative way, including the opportunity for input from a wide array of stakeholder interests. The 

openness of the Legislative Branch allows the Congress to be informed and make better decisions, but it also serves to balance 

competing agendas and perspectives from both inside and outside 

Congress. 

The oversight and evaluation process is one of the primary if not 

the primary role for the Legislative Branch. While the oversight 

function of Congress may not be as robust as it once was because 

of significantly shorter legislative sessions and delays due to sharply divided political climates, the capacity to judge the results of 

government programs already exists within the existing structures, 

structures that we feel do not carry the significant limitations, 

biases, and negative consequences of the PART. 

In conclusion, we all agree that everyone in government, the 

President, agencies and departments, and their staffs, and especially Congress, needs to be focused on achieving results in a fair, 

effective, and balanced way. However, this job should most of all 

fall on Congress, which already has the necessary tools and reVerDate 0ct 09 2002 08:58 Jun 28, 2007 Jkt 029503 PO 00000 Frm 00016 Fmt 6633 Sfmt 6601 C:\DOCS\29503.TXT SAFFAIRS PsN: PAT
13

sources in place to do the most robust and equitable review of the 

entire Federal Government. 

Relying too heavily on PART ratings will not only gradually remove Congress from its funding and oversight responsibilities but 

will also continue to close the door on opportunities for outside 

stakeholder interests, the views of the public, to be infused into the 

Congressional budgeting and evaluation process. The limited perspective of the PART is one of its most glaring deficiencies. While 

subjectivity and bias will almost always creep into any rating system, the PART does not have a mechanism for balancing out the 

results of its one-size-fits-all, Executive Branch-focused perspective. 

While the expansion of the Executive Branch powers has been 

present in our government since the turn of the last century, the 

overreach of those powers into areas historically and constitutionally given to Congress, the structuring of programs, appropriating and authorizing of revenues, and oversight of government 

is a disturbing trend. Because of this, PART should not be taken 

with just a grain of salt or even a hefty dose of skepticism. Unless 

the tool design and implementation systems are significantly modified, the PART ratings should probably be largely ignored by Congress. 

Thank you. I look forward to your questions. 

Senator COBURN. Thank you. 

I wonder if either of you might want to comment on Mr. Hughes’ 

testimony. It is certainly different than what we heard from either 

Mr. Johnson or Ms. Norcross, and I have several questions for Mr. 

Hughes as well, but I thought—Mr. Johnson, would you like to 

comment? 

Mr. JOHNSON. Yes; several other countries around the world 

think the PART is great; other States in America think the PART 

is great. Most good government groups think it is great. It is an 

instrument. It has had blunt; will get better every year. 

Most people that observe Congress, that have been around Congress a long time, believe that the Executive Branch is more interested in how well programs work than Congress is. David Walker 

has said that in hearings; so have Dick Armey and others. I would 

bet you agree. It is very hard to produce performance information 

and program assessments. What the Administration has done with 

PART is a place to start. We have been working 5 years on this. 

I do not believe Congress is going to invest 5 years to put together 

the information that we have right now. The PART information is 

a starting point for building better mechanisms to holding agencies 

and programs accountable for what they do. 

So I believe, in spite of its flaws, that PART is an excellent tool. 

It is a wonderful beginning. It is the product of 5 years of effort. 

I do not see this as a power grab by the Executive Branch. The 

subject of this hearing is why won’t Congress pay attention to 

PART, so I don’t think Congress is actually reeling with this onslaught of performance information from the Executive Branch. 

Our challenge is to get them to pay attention to it. 

Senator COBURN. Thank you. Ms. Norcross. 

Ms. NORCROSS. What is the alternative to not using performance 

information? PART has given us—at least we have moved the discussion away from the policy preferences of an administration toVerDate 0ct 09 2002 08:58 Jun 28, 2007 Jkt 029503 PO 00000 Frm 00017 Fmt 6633 Sfmt 6601 C:\DOCS\29503.TXT SAFFAIRS PsN: PAT
14

wards evaluating programmatic activity. So I do not know what the 

option would be. Should we revert back to a system where we simply do not use performance information, expect it, gather it, or analyze it? And if there is discomfort with OMB performing these assessments, perhaps Congress should undertake that. 

I understand Congress only engages about 7 percent of its time 

in oversight. So the current legislative mechanisms that are supposed to be engaged in this activity are not working up to speed. 

So that would simply be my response is if Congress is supposed to 

be evaluating these programs, where is the evidence that it is, in 

fact, evaluating them and providing guidance to agencies along the 

lines of performance? 

Senator COBURN. All right; thank you. 

Mr. Hughes, you mentioned that there is significant bias and distortion and manipulation. Would you give me examples of bias, 

please? 

Mr. HUGHES. Sure. There are a number of different types of biases that can be involved in this. One is the perspective of the 

OMB officer. The budget officer at OMB is the person who has the 

final say on what the language will be for the answers to the questions, how that language that is written will translate into a yes 

or a no or a few of the modified answers that are possible now 

under the PART and also how those yeses and noes get translated 

not only into the numeric raw score but also into the actual rating. 

There are a lot of inconsistencies between the guidelines that 

have been laid down for what raw score equals what rating and 

what the programs that have been reviewed actually get. That is 

one type of bias, and that is from a kind of implementation perspective. There are other biases in the actual design of the tool. I 

think that the format under which it was designed, which was designed to be accessible to people who may not be policy experts or 

who want to just know, like you say, come and look and see whether the government is getting results and whether the program is 

working, that necessitates that certain things are left out. 

One of those things is whether the Congress has designed a program to have multiple goals. Many programs in the Federal Government are designed to have multiple goals. That sort of thing is 

not taken into consideration within the PART. Oftentimes, those 

goals can be conflicting. That does not necessarily mean that it is 

a bad design. That just means that it is a complex program. And 

that kind of complexity is lost in the way the tool was designed to 

apply to people who may not be policy experts. 

Senator COBURN. Are you saying that there could be another 

PART program that would take into account for that? What is 

wrong by demanding a clear program mission from agencies? 

Mr. HUGHES. Certainly nothing. 

Senator COBURN. And questioning how a program fulfills that 

goal; is there anything wrong with those two things? 

Mr. HUGHES. No. 

Senator COBURN. So you do not disagree that a PART program 

might be designed better to take out more bias, but you do not disagree with the fact that knowing what a program’s goal is and 

measuring performance against that goal, it should be an effective 

tool. You would not disagree with that? 

15

Mr. HUGHES. No, theoretically, I agree with you. 

Senator COBURN. The one problem I had with your testimony is 

the problem I have with the rest of Congress is we are lazy. And 

the fact is that this is the 37th oversight hearing of this Subcommittee. Go find another one that has done that. And the point 

is that ideally, Congress does have the responsibility, but they do 

not live up to it. And so, what we are working with is in a vacuum, 

is Congress ideally should be doing this. I do not disagree with you, 

but they are not. 

And to have a blunt tool that is getting better, even though it 

can be criticized, and I think Mr. Johnson would agree that it is 

subject to some criticism, as is any assessment tool when you first 

start using it. But to say we should not have them doing it because 

it is Congress’ role—I agree; that is why I am doing it; that is why 

we have done 37 of them, begs the question of how do we motivate 

Congress to do oversight? 

So if we are critical of this one, answer me the question how I 

motivate my peers to do the appropriate thing when it comes to authorizing a program, and in that authorizing, saying we are going 

to measure it and then having the incentive to have Congress do 

the oversight to see whether or not they have a goal, and they are 

meeting that goal. 

Mr. HUGHES. That is, of course, a very difficult question, one that 

I will probably be very insufficient in answering, giving a satisfactory answer for. I think that the oversight role of Congress, and 

you are correct, of course, in citing the fact that Congress does not 

really do oversight any more. That is indicative of larger things 

about our political system, about the way that the electoral process 

works, about the importance of fundraising. There are multiple 

things that are in there that actually have nothing to do with 

whether Congress should do oversight or not that are enormous 

problems that would be difficult to tackle. 

I think your leadership on this issue is important. I think we 

need to have more folks in Congress who are paying attention to 

these sorts of issues. I do not know if there is a magic bullet procedural change or a statute or something that we could do that would 

make it so that Congress would be forced to do oversight more. I 

do think that some of the suggestions that have been made in front 

of this Subcommittee in the past about taking the Program Assessment Rating Tool or a modified version of it outside of the Office 

of Management and Budget, perhaps maybe having the Government Accountability Office do it or establishing a committee within 

Congress that would provide oversight in that regard. I think those 

ideas are worth exploring. 

I do not think that you can just remove the PART the way it exists now and give it to GAO and have it work well. I think there 

are design flaws that need to be corrected, that need to be adjusted, and I am, of course, sympathetic to the point that if you 

change it too much, the previous reviews would not be as useful. 

But it is not necessarily just a problem with the way that the tool 

gets done at OMB. We think there are deficiencies within the way 

it was designed as well. 

16

Senator COBURN. Well, you would be agreeable, then, to submit 

to this Subcommittee the things that you think are deficient in the 

design so that we can look at that? 

Mr. HUGHES. Sure, and that was reflected in my written testimony. There is a section on that. 

Senator COBURN. One of the problems with oversight is that a lot 

of agencies do not respond to our questions. Let us say we had 

oversight, and they do not respond. The only way you can solve 

that is either have somebody who can squeeze them on their 

money, or we have to squeeze them until they respond. But that 

requires the sausage-making process to be able to accomplish that. 

The thing that is disconcerting is I have little faith that Congress 

is going to step up to the bar until they are absolutely forced to 

through a financial disaster to make the hard choices. Congress 

wants to avoid hard choices, and as long as they do not feel the 

pinch, they will not make the hard choices. And that is why 2016 

is going to be a very tough year for this country, because that is 

when the pinch starts, the big pinch. And so, having an assessment 

tool, blunt, maybe somewhat biased, maybe somewhat distorted is, 

in my mind, better than nothing at all. 

Mr. Johnson, and you may not care to comment on this, but you 

might comment on the motivation behind it: The House Subcommittee on Labor/HHS put a prohibition in their bill this past 

week that precludes any money from being spent on the PART assessment. Any comments on the motivation behind that or what 

you see? I am not trying to create a problem for you with the Subcommittee, but how did we get there? 

Mr. JOHNSON. Well, there is one unelected staff member who is 

opposed to the PART. He worked on the Treasury/Transportation 

bill last year and put a similar prohibition in there. He was at 

HUD before that, and he disagreed with HUD’s use of the PART, 

and he was at OMB before that. One unelected staff member is responsible for the provision. The chairman of the committee had no 

knowledge that it was in the bill. It is inexplicable to me that language like that is in the bill. That is my only comment. 

Senator COBURN. OK; one of the other things, Mr. Hughes, with 

your testimony which I find, well, less than congruent is the statement that the PART increases the White House’s power. And the 

problem with that is Congress ignores the PART assessment. We 

have been able to do nothing with the PART assessment. Even 

when I look at all of it, and I look at the agencies, and I have done 

the oversight, and I try to get somebody to do something about it, 

Congress ignores it. 

So there is not a power grab there, because Congress is not paying any attention to it. So explain to me your reasoning behind—

is it a potential? Because it is certainly not, in fact, acted out. 

There is no effect of the PART right now on the Congress, because 

they ignore it. 

Mr. HUGHES. I actually would agree with you, and I would say 

that would probably be a poor choice of words on my part. I do 

think it is a potential problem. Let us do a for instance. Let us suppose that Congress will appropriate funds according to whatever 

the rating on the PART is. Why do we even need Congress? Let’s 

just let OMB do it. So I think it is a slippery slope. I think that 

17

particularly with respect to budgeting, we have been working more 

in trying to explore the management side of it as well, of PART, 

and the usefulness within agencies. 

I think there is more potential for a productive use of the information there. I do not think that you can look at a PART score and 

say, OK, well, I know how to fund programs now because of these 

problems. So the way I chose my words is probably poor. I do not 

think it is a problem right now; as you say, you are correct, that 

Congress does not pay attention to them. 

Senator COBURN. Well, but let me create a scenario for you. Let 

us say that Congress is doing great oversight on everything. We 

are sunsetting things; we are reauthorizing them; we are bringing 

them back up; we really know what we are doing and that we are 

doing a good job of that. Let’s make that assumption. That is an 

absolute lie, but let’s make that assumption. 

Would you deny the fact that the Administration should have a 

performance tool themselves to measure what the goal is of the 

program and whether or not they are meeting that goal as a management tool to become more effective in carrying out the will of 

the Congress? 

Mr. HUGHES. No; I think the problem exists when the tool that 

the Administration designs, or it does not even have to be this one, 

the Executive Branch designs portrays itself as an unbiased, objective evaluation of how programs and management are going at 

agencies when, in fact, it is anything but that. So I do not think 

that—again, in theory, that this is necessarily a problem. But with 

this particular instance, it is kind of like a wolf in sheep’s clothing. 

You have a situation where they are saying we are doing this; it 

is systematic; it is transparent; it is on the Web; the public can 

view it; this is an innate good. 

But the kind of things that we worry about are the things that 

are not transparent within the PART, that you do not necessarily 

see up front when you look at the one-page review. That is where 

you get into a tricky situation, and it is perfectly fine for the Executive Branch to have their own systems and whatever they like, 

but the problem occurs when they try to sell that to Congress as 

the one objective evaluator. 

Senator COBURN. But they have not. They have just said since 

you are not doing one, we are going to do one, and here is what 

we have found, and here is what our recommendation is. We still 

control the purse strings, and it is obvious from the PART assessment that Congress has totally ignored the Administration when it 

comes to evaluating programs. So that is not seen as a risk to me 

whatsoever. 

Mr. HUGHES. Well, that is encouraging to hear. 

Senator COBURN. Well, they have not. 

Mr. HUGHES. Well, I would say that they have not succeeded. 

Senator COBURN. I think it is very discouraging to hear, because 

they are not looking at the other as well. 

Mr. HUGHES. Fair enough. 

Senator COBURN. They are paying attention to nothing and continue it. One of the battles I have, and I will share it in the Subcommittee, is there are a lot of bills that I block; they are authorizing bills. And I go to Members of the Senate, and I say these are 

18

the things that I have problems with. And they say, well, why do 

you have problems? And I say, well, you have not looked at the programs that are already there before you authorize another program, and you have not said we are going to eliminate this program and put this one in. You are authorizing another program to 

do the same thing that is already happening without deauthorizing 

another program. 

And what I get told: Well, we do not do things that way. Well, 

the American people do things that way. Business does things that 

way. States do things that way. Why should Congress not do it? So, 

really, we are shooting the messenger here. The messenger—there 

is a vacuum in terms of oversight, and we now have an Administration that has attempted, whether we think their tool is good or 

not. And you do not doubt that the tool is getting better as they 

have used it? They are using a tool that is improving, that does 

have maybe some bias and does have some risk for manipulation 

in it, but the fact is it is the only thing available right now, especially since this Subcommittee has time getting even agencies to 

come and testify before it or to give us information. 

Mr. HUGHES. I will respond with two points. One, your shooting 

the messenger analogy, I think that may be part of our criticism 

of it, but our problem with it is that when the messenger leaves 

with his message, and when he gets to his destination, he is carrying two different messages. There is a problem with the transmission along the way, and that is something that is important to 

realize, regardless of where the criticisms are being pointed at. 

I think the second thing is, and I sympathize with your frustrations about oversight in Congress, and that is certainly something 

that we would like to see a ton more of. I think you can kind of 

get around some of the rhetoric around what government—we have 

all these programs, and they do not do anything that is important. 

If we had more oversight, if we had more openness about what the 

government actually does, I think people will actually have a greater appreciation of things. 

Senator COBURN. Right. 

Mr. HUGHES. So I think our criticism—try to be focused on this 

particular instance of PART, the way that this PART assessment 

works. I do not think that it should be thrown in the garbage can. 

I think that it is very important that people in Congress and people 

in the agencies and the public know that this should be, despite the 

fact that there is not a lot going on elsewhere, this should be a 

really tiny part about evaluating how government works. That 

would be my caveat about—I am sympathetic to the fact that it is 

not going on elsewhere, but try not to latch on to it and say this 

is the tool, and this is what is going to get us there. 

Senator COBURN. Nobody has in Congress. Would all three of you 

agree that some type of assessment of goals and measurement 

against the goals changes expectations of program managers? 

Mr. JOHNSON. I agree totally. 

Senator COBURN. Ms. Norcross. 

Ms. NORCROSS. Totally correct. 

Senator COBURN. Mr. Hughes. 

Mr. HUGHES. In my limited experience, I would say that is right. 

19

Senator COBURN. Ms. Norcross, you have some experience with 

performance tools in New Zealand, and I also know that South 

Korea has adopted assessment programs. Could you comment on 

those two things? 

Ms. NORCROSS. Morris McTeague, with whom I work at the Government Accountability Program, has direct experience with the 

New Zealand experience in developing performance information 

systems and applying them to remedy some of New Zealand’s budget crises. And if I could answer that question later, I could get you 

more information in specific on some of the reforms that they have 

undertaken. We are right now doing an analysis of that. 

Senator COBURN. OK. 

Ms. NORCROSS. So I could provide that for you. 

Senator COBURN. Mr. Johnson, the question of bias in the instrument that you use, give us an example of three or four of the questions that PART asks about programs. 

Mr. JOHNSON. Well, it asks if the program has a clear definition 

of—this is not exact wording, but it asks about do you have a clear 

definition of success? Does it have a good way of measuring your 

performance relative to that? Is it meeting its performance goals? 

It asks about the quality of management the program has. Do the 

program have an efficiency goal? Is it Management 101, or it is Accountability 101? 

These assessments are put together by the agency and OMB, not 

by OMB alone. The agency and OMB are supposed to agree on the 

program performance goals. Just as agencies are afraid to disagree 

with Congress, agencies are sometimes afraid to disagree with 

OMB about its assessment, But if they really disagree with the assessment, agencies can submit their disagreements to an appeals 

board that I chair and that is made up of deputy secretaries from 

four or five agencies. We get a number of appeals every year, and 

we review them. Some of them, we approve, and some of them, we 

reject. 

And we also conduct what we call consistency checks, where we 

review if the PART follows the rules we have for answering the 

questions. We also review whether the answers in a PART are consistent with each other. 

As we look also at programs dealing with the same subject across 

agencies, we pull all the relevant program assessments together 

when we start doing a cross-cut analyses to make sure we are 

equally attentive to the issues, equally focused on the quality of the 

performance measures and so forth, because we are going to be 

using this information to compare one program to another. We use 

cross-cutting analyses to see if there is something an ineffective 

program can learn from an effective program dealing with the 

same topic. 

So there is a lot of effort to make the assessments consistent; to 

make the information reliable; and to remove bias. There is bias in 

anything a human being does. So I have no doubt that these are 

not perfect instruments, but they will get better over time, and the 

assessments that we have done in the last 2 years are better than 

the assessments we did in the first 2 years. 

Senator COBURN. So, in other words, the programs set their own 

outcome measurements. 

20

Mr. JOHNSON. The program does. OMB and the program staff 

have to agree that the performance measures are acceptable. 

Senator COBURN. And then, they measure themselves against it. 

Mr. JOHNSON. They then determine the metrics they will use to 

measure performance and how to collect the data, and how often 

to collect that data. 

Senator COBURN. And so, where is the bias in that? 

Mr. JOHNSON. I do not know; just because——

Senator COBURN. If they are participating in setting the goal, 

and they are participating in setting the metrics, and they are the 

ones doing the measurement of the metrics, where is the bias? 

Mr. JOHNSON. I do not know where there is unusual bias. I know 

that there is a bias in anything that human beings are involved in. 

So I do not know what specifically Mr. Hughes is talking about. 

Senator COBURN. In teaching to the test, a problem across agencies as they respond to PART questions, Mr. Hughes wrote in his 

testimony that agency officials told him they gamed the system to 

avoid negative scores and consequences. Do you think that is true? 

Is there something in the program to help alleviate? We know everybody games when they are being measured to an extent. Are 

there things in the PART assessment system that take that into 

account? 

Mr. JOHNSON. I know that agencies like to be green. They really 

like to be green, and they really like to have good PART scores. 

And so, they do a lot of things to please OMB and to get good 

scores and to look good on that scorecard. 

Senator COBURN. Does that carry out into changed programs and 

changed management to make the programs more effective to deliver better process and therefore better response by the government to the very people they are supposed to be helping? 

Mr. JOHNSON. Well, teaching to the test, gaming the PART to get 

a good score and providing only superficial analysis does not help 

the program work better. One approach that we have to improve 

the quality of program performance is shine a real big light on it 

all, which is one of the primary reasons we took all this assessment 

information, summarized it, and put it on the Website ExpectMore.Gov for all the world to see and for people to look up and say 

that is not the way I know the program works. An employee can 

look at it, or someone served by the program can look at it and say, 

well, that program does not work very well; it is ineffective as far 

as I am concerned, and they can complain to the agency or complain to OMB or complain to their Senator or Congressman. 

Shining a lot of light on how the program is assessed, on what 

performance measures are used, and on what the performance information says can drive improvements in the measures that are 

used, the data that are used, and the quality assessment. So that 

is why I believe it is so important to have No. 1 on your list posted 

on this sign, which is transparency. You can have all of this information, but unless we shine a really big light on it, it will not get 

better over time. That is why we took it with all of its warts, with 

all of its dimples, put it out there. Now, let us begin the process 

with agencies and with Congress, I would hope, to improve this 

program performance information, to make these assessments betVerDate 0ct 09 2002 08:58 Jun 28, 2007 Jkt 029503 PO 00000 Frm 00024 Fmt 6633 Sfmt 6601 C:\DOCS\29503.TXT SAFFAIRS PsN: PAT
21

ter and to make our plans to help program perform better more aggressive. 

Senator COBURN. Let me follow up on that for a minute. So Mr. 

Hughes can go to every PART assessment and via the government 

Website can look at the goals, the metrics, the measurement of the 

metrics, the response, and the rating. 

Mr. JOHNSON. Right. 

Senator COBURN. In other words, nothing is hidden. Everything 

that comes to develop that, that can be accessed by OMB Watch, 

so they can see all that. 

Mr. JOHNSON. Right. 

Senator COBURN. Right; OK. 

Mr. JOHNSON. There is a one-page summary of every PART on 

the ExpectMore.gov. There are links, at the bottom to the detailed 

PART, which is multiple pages. It is written in OMB-speak and has 

historical information and more detailed information. That is the 

meat of the assessment. The summary and all the details is available on ExpectMore.gov. 

Senator COBURN. But they can get access——

Mr. JOHNSON. Yes, sir. 

Senator COBURN [continuing]. If they need to as well. 

And so, given your emphasis on transparency, are we to assume 

that you are going to be very accepting of our OMB transparency 

bill that Senator Carper and I have. 

Mr. JOHNSON. That is on the contracting information. 

Senator COBURN. Online grants, contracting, everything. 

Mr. JOHNSON. We love transparency, and we are working very effectively with your staff to figure out——

Senator COBURN. I understand that. 

Mr. JOHNSON [continuing]. The best way to get there as soon as 

possible. We are big on transparency and shining the light of day 

on performance to a strengthen accountability. 

Senator COBURN. Any other comments from any of our panelists? 

Mr. HUGHES. If I could just respond to some of the stuff that we 

have been talking about, the bias in the data and how the data, 

which data is important and which data is not important, there is 

a tension between outcomes and outputs in any type of performance management initiative. The PART focuses on outcomes, which 

is certainly a good goal. We think it is more of a broad governmentwide goal, maybe something that should be included in something 

like the GPRA, the Government Performance and Results Act. 

A lot of times, you cannot judge the effectiveness of programs 

based purely on outcomes, and I will give you a couple of examples. 

One program that is run out of, I believe it is the National Park 

Service, is an office that works as a consultant with local communities to transform the neglected or unused areas into public space: 

Parks, playgrounds, those sorts of things. They have collected, long 

before PART came along. And another thing that you should know 

is that agencies have collected performance data before PART came 

along. It is not like they were not doing this to begin with, and 

then, all of a sudden, OMB says you have to do it. 

Senator COBURN. Some were not. 

Mr. HUGHES. Some were not; that is correct; not all of them. 

22

They had a couple of standards by which they judged whether 

they are doing a good job in this program that acts as a consultant. 

One is through surveys with the local communities that they consult with: Were you satisfied? Did people use the parks? Did you 

like the services we provided? Another way is they used to collect 

data about based on the amount of money that they were given, 

how many square acres of parks did they create? How many miles 

of jogging trails, those sorts of things. 

Those are outputs, the second part. The survey part could be 

both. OMB, in the PART process, wanted them to focus on outcomes. And one of the things that they said should be an outcome 

was, Are the people living in the community healthier? And I think 

that is a perfectly good goal. I think people should be healthier. 

But the program in the National Park Service has no way to force 

people to go and jog in the park. All they can do is say this is the 

money we got to create parks for communities. These are the parks 

we created. These are the people we worked with and what they 

thought of what we did. 

That is one instance, one example of multiple examples of the 

difference between outcomes and outputs and how certain programs are not necessarily structured to focus completely on outcomes, or maybe the outcomes are beyond their control. And one 

other example I will share about bias within the PART is the Appalachian Regional Commission. This is a program that Congress 

decided to be a patchwork, to cover the holes between other programs that were working in similar issue areas. 

Senator COBURN. I have been trying to get rid of it for 10 years. 

Mr. HUGHES. I am aware of that. [Laughter.] 

And I do not have a personal perspective on the Regional Commission myself, but Congress designed this program to fill in the 

blanks, in the holes between programs, and the PART assessment 

said that this is not a unique program, which of course, it is not, 

because Congress designed it not to be unique. It was designed to 

be duplicative, because the evidence that Congress had seen at the 

time said that there are things that are being missed. 

And we can talk more about, maybe we should have just pulled 

all of the programs together and redesigned them so that the holes 

are not missed, and that is certainly something that OMB is trying 

to do. 

Senator COBURN. The whole point is you raise the question about 

what Congress has not done so that they will do it better. And to 

say that it is a blunt—it is a blunt tool, but it raises it up to a level 

so that somebody has to now—let’s address this, and we have not 

addressed the Appalachian Regional Commission. What we have 

done is we have let it continue to do exactly what it does, and the 

danger with that is: One, we are not efficient; two, we could design 

a program that helps a whole lot more people with the same dollars; or three is we could help the same amount of people with a 

whole lot less dollars, which gives us dollars to help somebody else 

somewhere. 

So the point is that is a commission that I am very well informed 

on, and I believe even the blunt tool will show that we could be 

much wiser as Congress to make the goals of that program more 

effective. I believe outcomes is the measure. I believe the American 

23

people want outcomes as the measure. But part of it is laziness on 

our part. When we write a program, and this is something I am 

critical of Congress, we ought to be very specific about what our intentions are, and we are not. We ought to be very specific about 

what we expect, and we are not. We ought to be very specific on 

how we want to measure whether we got what we expected, and 

we are not. 

So a lot of the problems do not have anything to do with you all 

in front of us; they have to do with Congress not being good legislators so that we design a system that can be looked at later and say 

did we go after what we intended to go after? Did we accomplish 

what we intended? And did we do it in a way or within the cost 

parameters that we thought it would? 

And so, the real criticism is not at OMB. They are dealing with 

what we have dealt them. The real criticism is for us in not being 

specific enough in terms of—and you can ask staff: When I write 

a bill, I want it all the way down to the T. I want limited discretion, because if we are going to write a bill and do not know 

enough about it, we should not be writing the bill until we get the 

information to write a bill correctly. And I do not know many people who would disagree with that. It is just easier to write it loose 

and let somebody else worry with the details, and that is called 

lazy legislating. 

Mr. HUGHES. That has been our experience working with you on 

the transparency bill as well. I think, though, that it is not necessarily as easy as you might make it out to seem. There is another 

example: The Consumer Product Safety Commission was ruled 

down on the PART review for not using cost-benefit analysis in its 

regulatory rulemaking. It is actually prohibited by Supreme Court 

decision from using cost-benefit changes like that in their rulemaking. 

That is not something that the program can control. And I agree 

with you that it is good that even with the Appalachian Regional 

Commission example that these things are brought up to Congress. 

But as you have said many times, with the lack of the kind of investigatory role that Congress is playing now into how programs 

are made, we are concerned that the information that we all admit 

has some biases and those sorts of things will be taken as a snapshot, and the investigation will not be done to get underneath what 

the rating is. 

So the ineffective for the Consumer Product Safety Commission, 

it should not be said we should get rid of it. But it is the bias in 

the tool that gives you the ineffective. 

Senator COBURN. But experience tells us that is not happening, 

because Congress is not paying attention to PART, and they are 

not paying attention to their own. They are ignoring them both. So 

your fear is unfounded, because we are not using it. 

Mr. HUGHES. Well, we would like to be vigilant. 

Senator COBURN. We should do both, though. We should be using 

theirs plus our own, and that is the point. Outcomes, to me, is the 

measurement, not outputs. And outcomes, if we design something 

to have an outcome, then, we ought to know what that outcome 

measurement is, and then, we ought to hold agencies accountable 

to be to that outcome. And I will just give you a great example: 

24

How about the incidence of HIV reduction in this country, which 

has not happened, and then, we spend money on flirting classes? 

And there is no connection between the two. In other words, if 

somebody is going to measure outcome, we ought to be asking why, 

with all of the money we are spending on HIV that we are not seeing a reduction in the incidence of new HIV cases in this country. 

And yet, nobody is measuring the performance against that outcome, and that is an outcome that makes a difference in lives. It 

is not outputs; yes, we are spending a lot of money, but we are not 

measuring outcomes, and therefore, we are not getting the ability 

to make the programmatic changes that need to be made on the 

congressional side to accomplish that. 

If the court prohibits a program from operating well, OK, if it 

prohibits a program from operating well, that tells us we have a 

problem in the design of the program. 

Mr. HUGHES. Well, I would disagree with that classification. I do 

not think it is prohibiting it from operating well. I think it is saying that the program needs to take certain considerations into account when it does operate. It needs to say there are certain things, 

equity issues within programs in the Federal Government that are 

important to take a look at. It is not necessarily that the Supreme 

Court is putting up a roadblock in front of them getting the job 

done. The Supreme Court is making a value judgment about how 

the program should operate. 

Senator COBURN. Which is not the Supreme Court’s job. The Supreme Court’s job is to interpret the laws and the Constitution and 

the treaties, not to tell Congress how to run the budget of the country, and that is——

Mr. HUGHES. And I would also say to you, too, that it is not 

OMB’s job to tell you how to run the budget of the country either. 

Senator COBURN. No, we agree. 

Mr. HUGHES. Yes. 

Senator COBURN. You will not disagree with me on that at all. 

I believe we have abdicated our responsibility, and the reason OMB 

is having to do this is because we have not. But I have no heartburn with somebody doing it somewhere. At least we have some information with which to make a decision. 

I want to thank each of you for being here. I would like a little 

more formal response from your organization on specifics on how 

you would definitely change an assessment tool program for agencies and what we might be able to accomplish that would limit, and 

I want that as justifiable constructive criticism so that when we 

look at PART, we can have your thoughts in detail on how we can 

assess that and maybe make recommendations. 

Mr. HUGHES. We look forward to that opportunity. 

Senator COBURN. All right. 

Thank you all so much for being here. The hearing is adjourned. 

[Whereupon, at 3:43 p.m., the Subcommittee adjourned.] 

(25)

A P P E N D I X 


29503.001
26


29503.002
27


29503.003
28


29503.004
29


29503.005
30


29503.006
31


29503.007
32


29503.008
33


29503.009
34


29503.010
35


29503.011
36


29503.012
37


29503.013
38


29503.014
39


29503.015
40


29503.016
41


29503.017
42


29503.018
43


29503.019
44


29503.020
45


29503.021
46


29503.022
47


29503.023
48


29503.024
49


29503.025
50


29503.026
51


29503.027
52


29503.028
53


29503.029
54


29503.030
55


29503.031
56


29503.032
57


29503.033
58


29503.034
59


29503.035
60


29503.036
61


29503.037
62


29503.038
63


29503.039
64


29503.040
65


29503.041
66


29503.042
67


29503.043
68


29503.044
69


29503.045
70


29503.046
71


29503.047
72


29503.048
73


29503.049
74


29503.050
75


29503.051
76


29503.052
77


29503.053
78


29503.054
79


29503.055
80


29503.056
81


29503.057
82


29503.058
83


29503.059
84


29503.060
85


29503.061
86


29503.062
87


29503.063
88


29503.064
89


29503.065
90


29503.066
91


29503.067
92


29503.068
93


29503.069
94


29503.070
95


29503.071
96


29503.072
97


29503.073
98


29503.074
99


29503.075
100


29503.076
101


29503.077
102


29503.078
103


29503.079
104


29503.080
105


29503.081
106


29503.082
107


29503.083
108


29503.084
109


29503.085
110


29503.086
111


29503.087
112


29503.088
113


29503.089
114


29503.090
115


29503.091
116


29503.092
117


29503.093
118


29503.094
119


29503.095
120


29503.096
121


29503.097
122


29503.098
123


29503.099
124


29503.100
125


29503.101
126


29503.102
127


29503.103
128


29503.104
129


29503.105
130


29503.106
131


29503.107
132


29503.108
133


29503.109
134


29503.110
135


29503.111
136


29503.112
137


29503.113
138


29503.114
139


29503.115
140


29503.116
141


29503.117
142


29503.118
143


29503.119
144


29503.120
145


29503.121
146


29503.122
147


29503.123
148


29503.124
149


29503.125
150


29503.126
151


29503.127
152


29503.128
153


29503.129
154


29503.130
155


29503.131
156


29503.132
157


29503.133
158


29503.134
159


29503.135
160


29503.136
161


29503.137
162


29503.138
163


29503.139
164


29503.140
165


29503.141
166


29503.142
167


29503.143
168


29503.144
169


29503.145
170


29503.146
171


29503.147
172


29503.148
173


29503.149
174


29503.150
175


29503.151
176


29503.152
177


29503.153
178


29503.154
179


29503.155
180


29503.156
181


29503.157
182


29503.158
183


29503.159
184


29503.160
185


29503.161
186


29503.162
187


29503.163
188


29503.164
189


29503.165
190


29503.166
191

Æ


29503.167