U.S. GOVERNMENT PRINTING OFFICE

WASHINGTON : 

For sale by the Superintendent of Documents, U.S. Government Printing Office

Internet: bookstore.gpo.gov Phone: toll free (866) 512–1800; DC area (202) 512–1800

Fax: (202) 512–2104 Mail: Stop IDCC, Washington, DC 20402–0001

44–586 PDF 2009

S. Hrg. 110–818 

OFFLINE AND OFF-BUDGET: THE DISMAL STATE 

OF INFORMATION TECHNOLOGY PLANNING IN 

THE FEDERAL GOVERNMENT 

HEARING 

BEFORE THE

FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT 

INFORMATION, FEDERAL SERVICES, AND 

INTERNATIONAL SECURITY SUBCOMMITTEE 

OF THE

COMMITTEE ON 

HOMELAND SECURITY AND 

GOVERNMENTAL AFFAIRS 

UNITED STATES SENATE 

ONE HUNDRED TENTH CONGRESS 

SECOND SESSION 

JULY 31, 2008 

Available via http://www.gpoaccess.gov/congress/index.html 

Printed for the use of the Committee on Homeland Security 

and Governmental Affairs 

( 

(II) 

COMMITTEE ON HOMELAND SECURITY AND GOVERNMENTAL AFFAIRS 

JOSEPH I. LIEBERMAN, Connecticut, Chairman 

CARL LEVIN, Michigan 

DANIEL K. AKAKA, Hawaii 

THOMAS R. CARPER, Delaware 

MARK L. PRYOR, Arkansas 

MARY L. LANDRIEU, Louisiana 

BARACK OBAMA, Illinois 

CLAIRE MCCASKILL, Missouri 

JON TESTER, Montana 

SUSAN M. COLLINS, Maine 

TED STEVENS, Alaska 

GEORGE V. VOINOVICH, Ohio 

NORM COLEMAN, Minnesota 

TOM COBURN, Oklahoma 

PETE V. DOMENICI, New Mexico 

JOHN WARNER, Virginia 

JOHN E. SUNUNU, New Hampshire 

MICHAEL L. ALEXANDER, Staff Director 

BRANDON L. MILHORN, Minority Staff Director and Chief Counsel 

TRINA DRIESSNACK TYRER, Chief Clerk 

FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT INFORMATION, 

FEDERAL SERVICES, AND INTERNATIONAL SECURITY SUBCOMMITTEE 

THOMAS R. CARPER, Delaware, Chairman 

CARL LEVIN, Michigan 

DANIEL K. AKAKA, Hawaii 

BARACK OBAMA, Illinois 

CLAIRE MCCASKILL, Missouri 

JON TESTER, Montana 

TOM COBURN, Oklahoma 

TED STEVENS, Alaska 

GEORGE V. VOINOVICH, Ohio 

PETE V. DOMENICI, New Mexico 

JOHN E. SUNUNU, New Hampshire 

JOHN KILVINGTON, Staff Director 

KATY FRENCH, Minority Staff Director 

MONISHA SMITH, Chief Clerk 

(III) 

C O N T E N T S 

Opening statement: Page 

Senator Carper ................................................................................................. 1 

WITNESSES 

THURSDAY, JULY 31, 2008 

Hon. Paul A. Denett, Administrator, Office of Federal Procurement Policy, 

U.S. Office of Management and Budget ............................................................. 5 

Hon. Karen S. Evans, Administrator, Office of Electronic Government and 

Information Technology, U.S. Office of Management and Budget ................... 7 

David A. Powner, Director, Information Technology Management, U.S. Government Accountability Office ............................................................................. 8 

Alfred Grasso, President and Chief Executive Officer, MITRE Corporation ...... 23 

Norm V. Brown, Executive Director, Center for Program Transformation ........ 26 

Thomas M. Jarrett, Secretary, Department of Technology and Information, 

State of Delaware ................................................................................................. 28 

ALPHABETICAL LIST OF WITNESSES

Brown, Norm V.: 

Testimony .......................................................................................................... 26 

Prepared statement .......................................................................................... 111 

Denett, Hon. Paul A.: 

Testimony .......................................................................................................... 5 

Prepared statement .......................................................................................... 39 

Evans, Hon. Karen S.: 

Testimony .......................................................................................................... 7 

Prepared statement .......................................................................................... 45 

Grasso, Alfred: 

Testimony .......................................................................................................... 23 

Prepared statement .......................................................................................... 100 

Jarrett, Thomas M.: 

Testimony .......................................................................................................... 28 

Prepared statement .......................................................................................... 120 

Powner, David A.: 

Testimony .......................................................................................................... 8 

Prepared statement .......................................................................................... 50 

APPENDIX 

Questions and Responses for the Record from: 

Mr. Denett ......................................................................................................... 124 

Ms. Evans .......................................................................................................... 128 

Charts submitted by Senator Carper ..................................................................... 134 

(1) 

OFFLINE AND OFF-BUDGET: THE DISMAL 

STATE OF INFORMATION TECHNOLOGY 

PLANNING IN THE FEDERAL GOVERNMENT 

THURSDAY, JULY 31, 2008 

U.S. SENATE,

SUBCOMMITTEE ON FEDERAL FINANCIAL MANAGEMENT,

GOVERNMENT INFORMATION, FEDERAL SERVICE,

AND INTERNATIONAL SECURITY,

OF THE COMMITTEE ON HOMELAND SECURITY

AND GOVERNMENTAL AFFAIRS,

Washington, DC. 

The Subcommittee met, pursuant to notice, at 9:33 a.m., in room 

SD–342, Dirksen Senate Office Building, Hon. Thomas R. Carper, 

Chairman of the Subcommittee, presiding. 

Present: Senator Carper. 

OPENING STATEMENT OF SENATOR CARPER 

Senator CARPER. The Subcommittee will come to order. Our 

thanks to our guests and witnesses for being here with us today. 

This is the third hearing that our Subcommittee has held on the 

issue of poorly planned and poorly performing IT investments. This 

hearing will focus once again on the ability of the Office of Management and Budget to oversee and provide Congress with visibility 

into the $70-plus billion that agencies will spend on information 

technology this year. 

Up until March of this year, Congress had an extremely limited 

ability to understand why OMB considered an IT investment to be 

poorly planned, and I must commend OMB and in particular Ms. 

Evans for finally releasing this data. Although I believe this is a 

good start, more complete and accurate information needs to be 

shared. I firmly believe that in order to hold agencies accountable 

for their investments, Congress and OMB need to work together as 

partners. The American taxpayers demand it. 

Information technology investments, if planned and implemented 

properly, can increase productivity, improve efficiency, and reduce 

an agency’s cost, and also enable us to provide better service to our 

constituents. However, some of these projects can be extremely difficult to manage and mistakes may be made along the way. 

In fact, I think it was Richard Nixon who said the only people 

who don’t make mistakes are the people who don’t do anything. So 

we know if we try to do something in these complex areas, we are 

going to make mistakes. That is to be expected. The key is, I think, 

to make sure that we don’t continue to make the same mistake 

2 

over and over again. However, the mistakes that we do make, we 

learn from them. 

I experienced this firsthand when I was privileged to serve as 

Governor of Delaware. Sometimes we bit off more than we could 

chew and the IT project would eventually spiral out of control. 

When this happened, sometimes we came to the conclusion that the 

best course of action was just to finally pull the plug. It was a 

tough decision a lot of times, but it was often the right thing to do 

in certain cases. 

Unfortunately, many agencies in the Federal Government are allowed to spend billions of taxpayer dollars on investments that are 

duplicative, that lack clear goals, and that are managed by unqualified individuals. In fact, according to recently-released GAO 

data, some $25 billion in IT investments are poorly planned, poorly 

performing, or both. Even worse, some of these projects have been 

delayed up to a decade and are costing us billions more than was 

originally expected. This is simply unacceptable and it makes me 

wonder whether it is time for Congress to pull the plug on some 

of these failed investments. 

Regrettably, Congress still does not have the information necessary from OMB to hold agencies accountable and to choose where 

we want to invest scarce resources each year. With risky investments such as IT, it is important to increase collaboration and visibility, not hinder it. And so far, this has not always been the case. 

Since 1994, Congress through legislation called the Federal Acquisition Streamlining Act has required agencies to keep costs, delivery dates, and performance goals of major acquisition within 90 

percent of the originally proposed plan. OMB then was required to 

annually report to Congress on agency progress. 

Unfortunately, despite this requirement in law, OMB has, I am 

told, only issued three reports in the last 14 years. Although my 

staff tells me we may have received a report around midnight last 

night. Someone here was staying up late and I think may have received a fourth report in 14 years in the midnight hour. 

Moreover, agencies are required to create an investment baseline 

that takes into account potential risks that could lead to increased 

costs, delayed delivery dates, and reduced performance. Agencies 

then use this baseline to track whether an investment is progressing according to plan. 

There are obviously some legitimate reasons why an agency may 

change the original baseline on a given project, but I am disappointed to say that some agencies have used rebaselining to hide 

the cost overruns or schedule delays from Congress. 

We are about to hear an extremely troubling report from the 

Government Accountability Office today revealing that almost 50 

percent of all Federal agency IT investments are rebaselined. Even 

more disturbing, some agencies, such as the Department of Agriculture, the Department of Commerce, Veterans Affairs, have 

rebaselined, I understand, more than five times on a single investment. 

Although agencies are responsible for the excessive rebaselining, 

there is one thing in common between all of these investments. 

Every baseline and rebaseline was approved by OMB. Someone, 

somewhere, in my view, is not fulfilling their responsibility to enVerDate Nov 24 2008 09:09 Jun 09, 2009 Jkt 044586 PO 00000 Frm 00006 Fmt 6633 Sfmt 6633 P:\DOCS\44586.TXT SAFFAIRS PsN: PAT
3 

1The chart referred to appears in the Appendix on page 135. 

sure the taxpayer dollars are spent only on those investments that 

are well thought out and truly needed. 

Using the information that was provided to our Subcommittee in 

March, we have created a report card for agencies that take into 

account several criteria related to the planning and implementation of IT investments. In fact, we have two report cards side by 

side. I say this as a parent whose children have both now graduated from high school and who is used to getting report cards. 

Sometimes my children aren’t anxious for us to see these report 

cards. But on the report card on the left, we have a list of agencies 

who received a passing grade, and on the right, those who receive 

a failing grade. If anybody in the audience is able to read those, 

I take my hat off to you. We have copies of the report cards that 

are provided to our witnesses and I hope to the press. 

As we can clearly see the Federal Government is miserably failing. In fact, half the 28 agencies received an F. Let me go ahead 

and list some agency grades starting with the Department of 

Labor, whose score was about 64 percent. The Department of 

Treasury, 56 percent. Office of Personnel Management, 55 percent, 

all the way to the bottom of this chart on the right,1 down to the 

Department of Defense with 37 percent, followed by the Department of Agriculture with a 36 percent score. 

About half of the 28 agencies received an F. In total, these failing 

agencies are overseeing $57 billion in IT investments, and from 

what we can tell right now, we are getting too little in return for 

those enormous investments. 

In addition, you might notice on the left, we have a number of 

agencies who receive a passing grade. Several received scores of Aplus. They included HUD, the National Science Foundation, the 

Smithsonian Institute—they all got 100 percent—the Social Security Administration, 99 percent, Department of Energy, 93 percent, 

all the way down to the bottom of the chart, coming in at a 69 percent for a D-plus, and that is the Department of Justice. 

However, and while we are encouraged by the passing grades in 

the report card on the left, we want to temper our joy by a couple 

of things. HUD received their A-plus partly because they reported 

that every project was being delivered on cost, on schedule, and 

performing as planned. However, as GAO will testify today, HUD 

rebaselined at least one project seven times, possibly to mask spiraling costs, and GAO has consistently testified that data provided 

by agencies to OMB is oftentimes inaccurate or even incomplete. 

So we don’t really have the complete picture, even for those agencies that are reportedly doing a better job than others. That is why 

I plan on introducing legislation today, along with Senator Lieberman, Chairman of our full Committee, and Senator Collins, the 

Ranking Republican on the full Committee, which will give Congress and OMB the information we need to make better decisions 

about which IT investments should continue and which should be 

shut down. 

Our bill, called the Information Technology Investment Oversight 

Enhancement and Waste Prevention Act, would make agencies report regularly on significant deviations on cost, schedule, and perVerDate Nov 24 2008 09:09 Jun 09, 2009 Jkt 044586 PO 00000 Frm 00007 Fmt 6633 Sfmt 6633 P:\DOCS\44586.TXT SAFFAIRS PsN: PAT
4 

formance. But we don’t just want better information. Our bill also 

helps OMB take a crucial step aimed at preventing IT investments 

from drifting toward failure. Recognizing that agencies may not 

have the skills necessary to manage complex IT investments and 

may have trouble recruiting qualified managers, our bill would set 

up a team of experts from inside and outside of government that 

agencies may use as a resource. This team, which my staff and I 

regard as something of an IT strike force, would have the skills 

and background necessary to make sure that agencies are focused 

on the right things, making the right decisions, and spending the 

money wisely. 

I again want to thank our witnesses for joining us today. We look 

forward to hearing from you. We look forward to your testimony. 

We look forward to the discussion that will follow it and hopefully 

to the better IT performance, for the money that our taxpayers are 

investing in those projects today. Taxpayers expect us to be good 

stewards of their money. In fact, they demand it, and I know that 

everyone here in this room wants to see that become a reality. 

Before I introduce Mr. Denett, Ms. Evans, and Mr. Powner, none 

of whom are strangers to this Subcommittee or to this room, I realize these are tough issues. These are not easily done. If they were, 

somebody would have done them a long time ago. The problems 

were recognized not just in this Administration, but in the last Administration, as well. While we appreciate that the problems are 

being identified, what is really important is that we identify them 

more quickly and that we fix them. I believe that we need good information here in the Legislative Branch so that we can be better 

partners with the Executive Branch. 

We are going to see a new Administration coming in in 6 months. 

It is going to be President McCain or President Obama. But these 

problems and these challenges will still be before the next Administration and the next Congress. 

As Governor, I used to say the reason why we invested money 

in these IT projects was to enable us to provide better constituent 

service and to do it more cost effectively, and that is really the case 

here, too. In some areas, we are doing a pretty good job. In too 

many areas, though, we are not. In too many areas, we are masking mismanagement, misallocation of resources and cost overruns 

in ways that make it look like we are doing a better job on some 

projects than we truly are. We have to get beyond that. 

With that having been said, on that cheery note, we have a couple of panels before us. In the interest of time, I am just going to 

move right to the first panel for their testimony. Biographies of our 

witnesses are provided and will be submitted for the record. 

Our first witness is the Hon. Paul Denett. Mr. Denett serves as 

the Administrator of the Office of Federal Procurement Policy at 

the Office of Management and Budget. Previously, Mr. Denett 

served as counselor to Clay Johnson, who is now the Deputy Director for Management at OMB. Mr. Denett is a retired Senior Executive from the Federal service and has received many prestigious 

awards, including a Presidential Rank Award. When did you receive that? 

Mr. DENETT. I received that in the late 1990s. 

Senator CARPER. Did you really? Who was President then? 

5 

1The prepared statement of Mr. Denett appears in the Appendix on page 39. 

Mr. DENETT. President Clinton. 

Mr. DENETT. Yes, it was. 

Senator CARPER. All right. Some of us are lost in the 1960s. 

[Laughter.] 

Some days, I would like to be, but not today. [Laughter.] 

All right. Our next witness is the Hon. Karen Evans. Ms. Evans, 

welcome. She is Administrator of the Office of Electronic Government and Information Technology at OMB. Ms. Evans oversees the 

implementation of IT throughout the Federal Government, including capital planning and investment control. She is a 20-year veteran of government service and has testified before this Subcommittee on a number of occasions. It is a pleasure to have you 

join us again today. Thank you. 

Our final witness is Dave Powner, Director of Information Technology Management at the Government Accountability Office. I am 

glad that Mr. Powner doesn’t charge us for each visit that he 

makes and each time he testifies before this Subcommittee, because we would be owing him a lot. But in the private sector, he 

has held several executive level positions. In the telecommunications industry, Mr. Powner has been instrumental in helping this 

Subcommittee provide oversight of risky IT investments, along 

with other things. I know he has appeared before us on several occasions, including as we try to address our current challenges involving conducting a good Census for the year 2010. 

We are going to lead off and ask Mr. Denett to lead off. We will 

ask you to keep your statements to roughly 5 minutes. If you go 

a little bit beyond that, that is OK. If you go a lot beyond that, that 

is not OK. So we will turn to you now. As you present your testimony, just keep in mind that your full statements will be entered 

into the record. 

TESTIMONY OF HON. PAUL A. DENETT,1 ADMINISTRATOR, OFFICE OF FEDERAL PROCUREMENT POLICY, U.S. OFFICE OF 

MANAGEMENT AND BUDGET 

Mr. DENETT. Thank you. Chairman Carper, Ranking Member 

Coburn, and Members of the Subcommittee, I appreciate the opportunity to appear before you today. The Administration places a 

high priority on working with agencies to mitigate cost overruns, 

schedule delays, and performance shortfalls in their major acquisitions. 

This morning, my colleague, Karen Evans, will discuss the initiatives the Administration is pursuing to effectively manage the government’s IT portfolio. I would like to briefly summarize agencies’ 

progress in implementing performance-based management for their 

major non-IT acquisition programs. I have also prepared written 

remarks that I ask the Subcommittee to enter into the record. 

Similar to IT, major acquisitions of non-IT capital assets must be 

justified in terms of agencies’ strategic goals and reflect sound acquisition and capital planning decisions. The law requires agencies 

to apply performance-based management principles by establishing 

cost, schedule, and performance goals and achieving 90 percent of 

these goals, on average. 

6 

OMB guidance for meeting these requirements is set forth in Circular A–11, which provides guidance to agencies on preparing their 

budget submissions, and the Capital Programming Guide, which is 

a supplement to A–11. The guide was substantially revised in 2006 

to emphasize the importance of key steps in the acquisition planning process, such as needs assessment and alternative analysis. 

To assess the use of performance-based management, my office 

directed agencies to provide information on their new and ongoing 

non-IT major acquisition projects. We look for several indicators of 

progress, including the existence of capital planning and investment control policies, cost schedule and performance goals for new 

and ongoing projects, use of performance-based management systems to monitor progress and success in meeting goals. 

Results were mixed. While many agencies demonstrated progress, we found that capital planning policies for non-IT are often 

not as well established as they are for IT investments. In addition, 

performance-based management systems are not always being used 

to track cost schedule and performance. We can and must do better. 

A number of steps are being taken to strengthen the application 

of performance-based management to non-IT investments. First, 

the Chief Acquisition Officers’ Council created a Project Management Working Group to help OMB evaluate the appropriate application of performance-based management to different types of 

major non-IT investments, such as construction and aircraft. 

Second, OMB and GAO are partnering with DOD, DOE, and 

NASA, each of whom is on the GAO’s High-Risk List, in the development and implementation of corrective action plans to mitigate 

risk in major acquisitions and evaluate success against clear goals 

and metrics. 

Third, the Office of Federal Procurement Policy and the Federal 

Acquisition Institute launched a Federal Acquisition Certification 

Program to provide civilian program and project managers with 

standardized training on competencies critical to successful performance-based management. These include requirements analysis, 

cost estimating financial management, risk management, and quality assurance. Program and project managers that are assigned to 

major acquisitions will need to be certified under this program. 

Finally, OFPP created a standardized approach for agency selfassessments of the acquisition function based on a framework that 

was created by GAO. Self-assessments of major acquisition functions will address issues to help agency managers understand if 

planning is effective. If a material weakness is identified, it will be 

addressed during corrective action monitoring and reflecting on the 

agency’s statement of assurance prepared under OMB Circular A– 

123, which lays out management’s responsibilities for internal controls. 

In accordance with the Federal Acquisition Streamlining Act, 

OFPP issued a report—I think it was a little before midnight, but 

it was late—discussing civilian agency progress in implementing 

performance-based management—— 

Senator CARPER. Actually, I am told it was 12:04. [Laughter.] 

But who is keeping time. [Laughter.] 

7 

1The prepared statement of Ms. Evans appears in the Appendix on page 45. 

Mr. DENETT. OK. For both IT and non-IT investments. We will 

be happy to answer any follow-up questions the Subcommittee may 

have. Thank you. 

Senator CARPER. Right on the money. Thank you. Ms. Evans, 

welcome. 

TESTIMONY OF KAREN S. EVANS,1 ADMINISTRATOR, OFFICE 

OF ELECTRONIC GOVERNMENT AND INFORMATION TECHNOLOGY, U.S. OFFICE OF MANAGEMENT AND BUDGET 

Ms. EVANS. Good morning, Mr. Chairman. My remarks will focus 

on the Administration’s strategy and continued progress in performing oversight of agencies’ capital investments and information 

technology. Specifically, I will address the topic of OMB’s ability to 

effectively analyze, track, and evaluate agencies’ major capital IT 

investments. 

The President’s budget request for fiscal year 2009 includes approximately $71 billion for IT and associated support services government-wide. Of the $71 billion in the President’s budget request, 

$22 billion, or 31 percent, represents proposed funding for development, modernization, or enhancement of new or existing information systems, infrastructure, or services. Projected expenditures on 

existing systems, operations, and maintenance is $49 billion, or 69 

percent. 

As I have discussed in previous testimony on the topic of OMB’s 

oversight of IT investments, the Clinger-Cohen Act of 1996 established processes for executive agencies to analyze, track, and evaluate the risks and results of major capital investments for information systems. The operative means by which OMB evaluates agency 

capital asset plans and their associated budgetary requests is a 

business case, or Exhibit 300, for individual investments. 

Agencies develop and submit business cases with their annual 

budget requests to OMB and we, in turn, evaluate each business 

case in terms of its ability to support a given investment proposal, 

including factors such as alignment with IT architecture, plan performance, improvement goals, cost-benefit analysis, and eliminating costly, duplicative, and outdated systems. OMB also evaluates the capability to manage the investment as demonstrated in 

the business case and planning process, including factors such as 

having a qualified project manager, acquisition planning, systems 

security, risk management, and the use of earned value management to track and manage costs and schedule goals. 

Last year, I came before this Subcommittee to explain two specific tools OMB uses to track and review agency IT capital investments, the Management Watch List and the High-Risk List. I am 

pleased to report that we have since expanded upon our criteria for 

evaluating investments and improved upon the transparency of 

how investments and projects are placed on either or both lists. My 

written statement includes the details regarding these lists. 

GAO’s report on project rebaselining points out weaknesses in 

agencies’ policies in terms of specifying all of the elements of rebaselining according to best practices. We acknowledge more 

should be done in this area in the future. In particular, OMB needs 

8 

1The prepared statement of Mr. Powner appears in the Appendix on page 50. 

to clarify expectations for when an original baseline should be established and elaborate upon process steps for agencies to submit 

to and receive feedback back from OMB on rebaseline requests. 

OMB, Congress, and the agencies must work collaboratively to 

address weaknesses in IT program and project performance. We 

are all vested in a common interest, delivering results for the 

American people. 

How can we best do this going forward? I believe OMB has the 

foundational processes in place to perform program and project investment oversight at the macro level. These core processes—capital planning, architecture alignment, the Management Watch List, 

the High-Risk List, and the E–GOV scorecard—have been and can 

continue to be further enhanced and incrementally improved to 

provide better visibility into program performance before a project 

is in major trouble. 

One improvement for OMB would be to leverage the efforts of the 

budget formulation and execution line of business to ensure the use 

of analytical tools and the collaboration environment to improve on 

our own information management capabilities. The various data 

sets collected by OMB from the agencies can and should be better 

integrated into a more comprehensive knowledge base. 

I have brought a display of an example of how we could better 

integrate and expand upon information on the agency’s IT investments. This is one potential snapshot of project performance at a 

point in time. We would like to work with Congress to improve 

transparency and the ease of use of this information. With the 

transparency into departments’ and agencies’ performance will 

come improved accountability and results which the American people deserve. 

Thank you for this opportunity to discuss the Administration’s 

strategy and progress to date. With this foundation in place, we 

can continue to work together to achieve the outcome we both desire, which is successful implementation of information technology 

for program and mission results. We have accomplished a lot in the 

last 8 years and there is much more that we can continue to do 

in this area. 

I would be happy to take questions at the appropriate time. 

Senator CARPER. Thank you for that testimony. Mr. Powner. 

TESTIMONY OF DAVID A. POWNER,1 DIRECTOR, INFORMATION 

TECHNOLOGY MANAGEMENT, U.S. GOVERNMENT ACCOUNTABILITY OFFICE 

Mr. POWNER. Chairman Carper, we appreciate the opportunity to 

testify this morning on poorly planned and performing IT projects 

and the results of our rebaselining review completed at your request. 

Mr. Chairman, I would like to thank you and Dr. Coburn for your oversight of the Federal IT budget, which now exceeds $70 billion. This is the third annual hearing, and your oversight, along 

with the leadership of Karen Evans, has resulted in a clear picture 

of exactly what projects need work and the specific weaknesses 

that need to be addressed. That is the good news. The bad news 

9 

is that we have nearly $25 billion in IT investments that are currently at risk of being wasted and that this figure is likely much 

higher. 

This morning, I have three points to make. First, over 400 IT projects totaling nearly $25 billion are currently not appropriately 

planned for or managed. 

Second, the number of projects reporting issues with cost and 

schedule variances is understated because rebaselining is occurring 

excessively without adequate guidance or transparency. 

And third, additional oversight from both OMB and agency CIOs 

is needed to address project weaknesses and to bolster accountability. 

Expanding on each of these, first, currently, there are about 350 projects totaling $23 billion on the Management Watch List, and 

nearly 90 projects totaling $5 billion that are being reported as 

high-risk projects with shortfalls. Common to both lists are 26 

projects totaling about $3 billion, meaning that these projects are 

both poorly planned and performing. For example, DHS’s Secure 

Border Initiative project continues to be on both lists. 

In addition to the 26 projects on both lists, of particular concern 

are the repeat offenders, namely 32 projects that have been on the 

Management Watch List since September 2006 and 17 high-risk 

projects reporting shortfalls in each of the last four quarters. My 

written statement highlights each of the projects on both lists and 

those that have been on either list far too long. 

Mr. Chairman, having so many projects with longstanding planning and performance problems is unacceptable and requires more 

attention from both OMB and agency CIOs. Key reasons why 

projects remain on the Management Watch List are poor cost and 

schedule performance, poor security measures, poor privacy planning, and poor project management, while cost and schedule 

variances continues to be the primary performance shortfall associated with high-risk projects. 

Despite cost and schedule performance being a major reason why 

so many projects are highlighted here, our rebaselining report 

being released today raises significant questions about whether we 

are getting an accurate picture of project cost and schedule performance. 

First, I would like to mention that rebaselining is needed at 

times to reset realistic cost and schedule targets. However, this 

process should not be used to mask cost and schedule overruns and 

should be transparent and approved by both OMB and agency 

management. Our survey of nearly 200 projects showed that about 

half of all IT projects had been rebaselined, and of those that are 

rebaselined, about 50 percent are rebaselined at least twice, and 10 

percent are rebaselined four or more times. Clearly, these projects 

are operating nowhere near the desired 10 percent cost and schedule threshold. 

Because of the amount of rebaselining occurring across the Federal Government, we reviewed each of the major 24 departments’ 

rebaselining policies to ensure that it included basic items like 

specifying when a rebaselining is warranted, how a new rebaseline 

is validated, and who approves the new rebaseline. Agency policies 

were weak across the board and we recommended that OMB issue 

10 

rebaselining guidance and that each agency develop rebaselining 

policies that address the weaknesses we identified. 

Although we have more data than we have ever had historically, 

we still need better information so that all projects requiring attention are highlighted. For example, OMB could publicize high-risk 

projects with shortfalls as we have in our written statement. And 

we still need more transparency on rebaselining efforts so that we 

have a true picture of cost and schedule performance from all agencies. 

Put more simply, Mr. Chairman, do we think that of the 472 

high-risk projects only 70, or 15 percent, have cost and schedule 

issues? No. That number should clearly be higher. 

Now turning toward solutions, the Federal Government needs to 

focus on addressing root cause problems: Security, project management, risk management, and cost and schedule performance. We 

are encouraged by OMB’s efforts to address some of these weaknesses, like requiring techniques like earned value management to 

improve agencies’ cost and schedule performance, but there is 

much more work ahead. Agency CIOs and OMB from a government-wide view need to aggressively attack these problem projects 

by starting with those that have been on OMB’s radar far too long. 

Next, longer-term improvement efforts need to be pursued and 

these, at a minimum, need to focus on bolstering the IT workforce 

and addressing root causes more aggressively. I look forward to 

suggestions from our second panel of experts on other approaches. 

In summary, Mr. Chairman and Dr. Coburn, knowing what to fix 

is the first step, and we wouldn’t have this information without 

your leadership. Thank you. Keeping the current momentum and 

energy and focusing more on solutions will be essential as we transition to a new Administration. 

I would be pleased to respond to questions. 

Senator CARPER. Good. Thank you for that testimony. Thanks 

very much for the work that preceded it. On behalf of both Senator 

Coburn and myself and our Subcommittee and staff, thank you for 

the very good work that you have done. 

What I hope we focus on more today—yes, the sun is coming up. 

I don’t know if it seemed dark to the rest of you, but it seemed 

dark to me. I just asked my staff to brighten the lights. So if you 

are not awake, wake up now. [Laughter.] 

I was wondering, who are some good models on the public or private sector for the Federal Government? Who out there is managing IT projects well? Not just in planning and developing them, 

but implementing them on time and fairly close to budget without 

rebaselining six or seven times on a single project? Do we have any 

idea who is doing an especially good job and why? 

Mr. POWNER. I can tell you from our focus on some of the highrisk modernization efforts that we look at, even though some of 

these organizations received low department grades, if you look at 

pockets within the Federal Aviation Administration, if you look at 

IRS, recently there have been successes with delivering more 

projects on time and within budget. And if you look at what has 

resulted, there has been an extreme focus on improving their processes, executive governance over those projects, and one other technique that seems to have worked at some of those agencies is putVerDate Nov 24 2008 09:09 Jun 09, 2009 Jkt 044586 PO 00000 Frm 00014 Fmt 6633 Sfmt 6633 P:\DOCS\44586.TXT SAFFAIRS PsN: PAT
11 

ting in place Centers of Excellence where project offices can go for 

help, where there is best practices and things they can emulate 

across the board. They are not perfect by any means, but there are 

pockets of success in both of those agencies. 

Senator CARPER. Thank you for that response. In the National 

Governors Association, all 50 governors are members of that, we 

had a Center for Best Practices and if they had a particular State 

that was doing a good job in early childhood, the rest of us could 

learn from them. If we had a particular State that was doing a 

good job in reducing recidivism in the prison population, or if a 

State was doing an especially good job in providing better outcomes 

in our schools, we had the opportunity through the Center for Best 

Practices to see who was doing a good job and learn from them. Do 

we have that kind of ability in the Federal Government? 

Ms. EVANS. Yes, sir. Through the CIO Council, we have a Best 

Practices Committee where we do take on topics such as these specific areas, like who has implemented earned value management 

really well and can people learn from that. We need to do more in 

those Best Practices Committees, and I would also like to build off 

of what Mr. Powner talked about. 

What we have been doing from an OMB perspective is really 

zeroing in in particular areas. The two areas that Mr. Powner mentioned are also on the GAO High-Risk List. So there is a lot of 

work that we have done in conjunction with the agency, in conjunction with GAO, so that we could then cross-pollinate the expectations, the best practices in those areas, and then the intent is to 

take that and then share it back across the board with other agencies. 

So we have done that in the area of, with NARA and the Electronic Archives Initiative. As they go forward, what we have 

learned from FBI on their Sentinel project is they have improved. 

We have then partnered them up with other agencies as they are 

going forward on major investments so that they make sure that 

they don’t make the same mistakes. 

Senator CARPER. Mr. Denett. 

Mr. DENETT. We also have the Chief Acquisition Officers Council, 

and this past year to try to respond to the problems in the program 

project management area formed a new working group. It is 

chaired by Bill McNally, who is the Senior Procurement Executive 

of NASA. But all the major departments send representatives and 

they are sharing best practices and we are going to post them on 

the Federal Acquisition Institute website. 

Senator CARPER. All right. Mr. Powner. 

Mr. POWNER. I would just like to add that I agree with the comments that were made here, but one important thing that I think 

Ms. Evans made, an important comment, is the implementation of 

those best practices. Earned value management, a very good technique when used appropriately. Our preliminary work on earned 

value management implementation across the government is that 

it is rather immature, and I will give you an example. 

We just completed a review of FAA. It is likely one of the better 

agencies when it comes to implementing earned value. There has 

been a lot of focus and attention. But I can tell you there that a 

quarter of their projects don’t use earned value. Not everyone is 

12 

trained. And when you start looking at the data, the earned value 

data—and we did, we dove deep on a couple projects—we saw quite 

a few problems that we pointed out. 

Now, granted, that is probably one of the better Federal agencies, 

but when we look at the cost and schedule problem that we have 

in the Federal Government coupled with rebaselining, we have to 

get much better at using these techniques like earned value to 

make good management decisions to get early warning indications. 

Senator CARPER. All right. Thank you. The next question I want 

to ask, and I have touched on this already, but we started this 

hearing off with a little broader context than I think we had originally planned to discuss, but it is something that I am troubled by 

and I just want to come back to it again and discuss it. 

One of the Subcommittee’s purposes is, as you know, to prevent 

waste, fraud, and abuse. In order to do that, we need information 

on how agencies are managing our investments. Mr. Denett, in 

1994, Congress directed your office to provide a report on agency 

progress in delivering on their capital investments, including IT investments. However, as I stated in my opening statement, in 14 

years, OMB has only provided this report on three separate occasions, now maybe four. Let me just ask, why is OMB not fulfilling 

this critically important statutory requirement? 

Mr. DENETT. It was a mistake on our part, for sure. When I became aware of this, we have had the staff work diligently, go out 

to departments and pull it in, and I don’t have a good explanation 

as to why some years were not submitted. However, there is the 

annual budget process under A–11 where a lot of the data that is 

included in the report is submitted with the President’s budget 

each year, so that contains some of the information we sought. But 

it should have been wrapped up, put into a formal report, and provided to you each year. We have now put steps in to make sure 

that this doesn’t happen in future years. 

Senator CARPER. What can you do? Who is responsible for making sure this statutory requirement is met? 

Mr. DENETT. The Office of Management and Budget, so we are. 

Senator CARPER. All right. What assurance can you provide for 

us that it is going to be met in the future? 

Mr. DENETT. We have added it to a follow-up list that OMB has 

so that the new Administration will have it as a reminder before 

them. We have told the staff the importance of it, and it is not our 

prerogative to decide what years to submit it and what not, that 

we all need it as a tracking tool. This one will be stronger. We are 

going to have our own baselines on which ones are—just as you are 

having your report card, we are going to have that kind of method 

in there, and I believe in the future, you will get the reports each 

year. 

Senator CARPER. All right. I certainly hope so. 

Ms. Evans, this is the third hearing during which you have testified how billions of taxpayer dollars are wasted on bad IT investments. I want to commend you for all the progress that you and 

your team have made over the past several years. However, I believe that there is still more that needs to be done. I am sure you 

do, too. In fact, you said that. 

13 

Specifically, in 2006, GAO first recommended that OMB ensure 

agencies are accurately and reliably reporting on their investments. 

Also, GAO recommended that OMB provide Congress a single aggregate list for troubled IT investments that can allow us to track 

progress and performance government-wide, and it is now 2 years 

later. 

Just revisit with us, if you will, what specific progress since 2006 

have you and your colleagues made toward ensuring that agencies 

are reporting complete and accurate information to OMB. And second, why doesn’t OMB publish the list of high-risk projects with 

shortfalls as GAO highlights in their testimony? I am just asking, 

don’t you feel that Congress would want to know whether projects 

are over-budget and behind schedule? 

Ms. EVANS. Sir, I actually would like to answer the first question 

first—— 

Senator CARPER. Please. 

Ms. EVANS [continuing]. Which would be what we have done to 

increase the transparency in this. And I would say, first off, with 

the hearing and everything and the several hearings along this 

line, that I was hesitant at first, which I think you would admit, 

to releasing all of this information because we were concerned 

about some of the effects that it would have. 

Senator CARPER. What do you mean—— 

Ms. EVANS. I was going to go into this a little bit. And some of 

it, you are talking about and we have kind of gone around it a little 

bit, which is driving compliance versus actually achieving the results. And so things happen like the grades, not that this is not 

great because OMB has an A, so I am pleased that we have an A. 

But it is an A-minus. I would have preferred an A-plus, but I know 

why it is a minus. But what this will do is drive—— 

Senator CARPER. I asked my staff if we were grading on a curve 

here. They said, no, we are not. 

Ms. EVANS. No, but I don’t have as many investments as everyone else, so—— 

Senator CARPER. We are calling them as we see them. 

Ms. EVANS. But part of the issue is, one, that there is a part of 

us that love to see the grades, and so we strive for this. This is how 

our society is built. But the other part then drives compliance, 

where people will do just enough in order to get to the next level 

of the grade and they are focused more on the grade than actually 

achieving the result or getting the project implemented. 

And so this is a balance that my office has really been struggling 

with, about how much information do we release, how much shame 

and embarrassment do you bring upon an agency, because we are 

really supposed to be helping them. We are supposed to be there 

to help the agency achieve success. We are supposed to take areas 

in our knowledge that we see across the government and go in and 

help an agency achieve results. 

So I agree that all this information should be out there. I don’t 

necessarily disagree that we could provide better reporting, not 

necessarily along the lines of GAO, because I don’t view OMB as 

an auditor. I view OMB as a facilitator to help the Executive 

Branch achieve results. 

14 

So I would like us to be able to provide the information in a way 

that you would find it useful as an oversight entity the same way 

that we find it useful so that we could get to the solution for the 

agency and for the taxpayers. 

So we have done a lot to put all this out here. We have not put 

together the comprehensive list the way that GAO does or the way 

that it was written. I brought two examples, which are also in the 

written testimony, and if we want later on, I could go into a couple 

areas of how it could be done. We put a lot of effort into this so 

that we could match and cross-reference and make sure that all of 

this is reliable data, and then we could work with you in a way 

similar to the way that we have done with USA Spending and even 

release some of the raw data so that as you go forward, you guys 

could manipulate and do things with the data, as well. 

So I am always very cautious as we go forward on this because 

I am always really cautious about driving compliance versus trying 

to get to the result. 

Senator CARPER. All right. A related question. You may have spoken to this, but is there a reason why OMB hasn’t provided Congress with a single aggregate list that will allow us to provide proper oversight and track progress? Just try that. I think you have answered that question, but I want you to take another shot at it. 

Ms. EVANS. I was hoping you didn’t realize I didn’t answer that 

part—— [Laughter.] 

But I will tell you, the reason why, and this is our first—the tables that you have are our first attempt at aggregating the data together. The short answer is that based on what we had previously 

to provide the aggregated report, we believe would cause more confusion than actually bring clarity to the result. And then the other 

piece was we had to work a little bit more with the agencies to get 

better quality data in order to establish the relationship to do an 

aggregate report. I believe we are there now and what we would 

like to work with you on is how to actually provide that information in a way that would be useful for your use. 

Senator CARPER. Why not just provide the same information as 

GAO? 

Ms. EVANS. Because I would say that right now, we have several 

pieces of that, but we don’t go in and do full audits the same way 

that GAO does. So some of the information, for example, like the 

number of times that a project has been rebaselined, we would 

have to modify some of our A–11 guidance. Right now, we just capture that information as a yes and no, and I have not captured that 

information on an ongoing basis so that I could go down and map 

all the records and say that this investment has been rebaselined 

several times, X, Y, Z, and this year, those types of things like that. 

I have not kept the information at the micro level. The agencies 

are supposed to keep it at that level. And what we are trying to 

do is make sure that there is better quality as we start putting it 

forward. So I would be hesitant to publish the information because 

I do not feel that we have the quality still at that real detailed 

level because we haven’t conducted the audit as GAO has. 

Senator CARPER. I am going to ask Mr. Powner to comment on 

some of what Ms. Evans has just said. 

15 

Mr. POWNER. A couple thoughts here. One, first of all, I think 

Ms. Evans and OMB, they deserve a lot of credit for all the information we have on the Management Watch List, which are poorly 

planned projects. OK, you want to fix the planning up front. 

But the High-Risk List is not that important. What is important 

is Figure 4 on page 19 in my testimony, which shows those projects 

that have shortfalls and the reasons for the shortfalls. So the No. 

1 reason why we have shortfalls, and this is about performance 

now, it is not just about planning, but we are expending funds and 

we have shortfalls. We are not meeting cost and schedule goals. 

There are 70 projects. What I am hearing is I think the Administration—I am going to cut right to it—they are reluctant to highlight projects with shortfalls. 

Senator CARPER. Why do you suppose that is? 

Mr. POWNER. I think the discussion was along the lines of embarrassing agencies and that type of thing. You can’t fix problems—— 

Senator CARPER. There is a reluctance to embarrass them? 

Mr. POWNER. I believe so, but I don’t want—I will let OMB comment further on that. I don’t think you can fix shortfalls without 

fully disclosing all your problems, so you need to take those shortfalls—we have 10 percent—we have 70 projects that aren’t meeting 

the 10 percent threshold. That number is greatly understated, and 

I will tell you why. Earned value implementation is weak at many 

agencies, so some of that data is not based on reliable information 

coming from the agencies, and then the point about rebaselining, 

OK. 

If we had transparency on rebaselining—that number right there 

says that 85 percent of the high-risk projects are meeting the 10 

percent cost and schedule threshold. It is likely—that 85 percent, 

you are walking on water if you are a private, public organization, 

85 percent of your projects are meeting within 10 percent. You are 

doing very well, extremely well. 

So we need accurate data there, but we need to disclose all that 

information in order to fix those shortfalls, so that is why we, in 

our testimony, highlighted the important bit of information here 

are the high-risk projects with shortfalls so that we can fix performance. I actually think this is very consistent with the Administration’s President’s Management Agenda and their E–GOV scoring. One of their criteria to get to green is you operate within 10 

percent of costs and schedule using earned value techniques. So we 

are all talking the same thing here and we have the same goal. It 

is a matter of disclosing all the weaknesses and then fixing it. 

Senator CARPER. Ms. Evans. 

Ms. EVANS. I would like to respond to this. 

Senator CARPER. Please. 

Ms. EVANS. First and foremost, as the Administrator of this, I 

think it is my responsibility to make sure that I have done everything that I can for the agencies to make sure that the guidance, 

the policy, everything is clear, and what GAO has highlighted is 

that there is a systemic problem throughout the government. So we 

could be focused on the one-offs and let us zero in on this one particular project or let us zero in on this. But the tenets of the President’s Management Agenda is getting good management foundation processes, with these types of things in place, so that no matVerDate Nov 24 2008 09:09 Jun 09, 2009 Jkt 044586 PO 00000 Frm 00019 Fmt 6633 Sfmt 6633 P:\DOCS\44586.TXT SAFFAIRS PsN: PAT
16 

ter what the project is, no matter what the investment is, the agency is going to succeed. 

So when you start looking at the GAO report and you look at 

Table 5, which talks about who is consistent with the best practices, and going across and looking at that, what you want—I am 

looking at, what is the problem across the board, and right now, 

what I have gotten from all of this is that I need to do a better 

job of clarifying what is a baseline. How do you do integrated baselines? How do you set certain things up? What are the best practices, so that the agencies have the tools to succeed. 

If I am going to sit there and highlight one or two projects and 

say, Agency A is a real screw-up and they are not doing PDQ, we 

are going to drive behavior down underground and these issues of 

rebaselining are going to continue on and on and on because they 

are going to want to hit the mark of being at the 90 percent because grades are coming out. So I think it is my responsibility to 

do the things that I can in the first place to make sure that it is 

clear to the agencies. 

Now, once I believe that I have done everything that I can, that 

we have put everything in place, that it is totally clear in the agencies, the bulk of the agencies are producing, then I have no problems providing all the information, putting the transparency out 

there, having accountability, because that now becomes a different 

issue and that is a leadership issue within the agency itself. 

But right now, Mr. Powner has made it very clear that it is an 

immature process, at best, what we have in the agencies, and so 

they should be rewarded for moving forward and being forthcoming 

in the information that they have so that we can address and fix 

the problems, not that I feel passionately about this. 

Senator CARPER. Well, I would be disappointed if you didn’t. 

Mr. Powner, do you want to say anything else? Go ahead. 

Mr. POWNER. Just to piggyback off of what Ms. Evans was saying 

there, I think the point, too, about the accountability at the agency 

level, agency CIOs are accountable for these fixes. I mean, we need 

to highlight them, and it is not just all in OMB’s lap, but the agency CIOs are accountable here. 

Senator CARPER. Has OMB issued guidance on rebaselining? 

Ms. EVANS. The policy memos that we have in place are highlevel. They refer back to the Capital Planning Guide. They also 

refer back to A–11. But I, in preparation for this hearing, have 

gone back all through those, and maybe it is because I live and eat 

and breath this, it seems clear to me, but when I am looking at 

all the results across the board, I think there are other things that 

we can do like putting together a framework document that would 

show them how to do certain things the same way that the CIO 

Council put together a framework for project managers and made 

it very clear what a tier one project was, what a tier two project 

was, and those types of things. I think we need to go down now 

to another level and give them another set of tools in order to be 

able to improve their performance. 

Senator CARPER. Mr. Powner, should OMB issue guidance on rebaselining? 

Mr. POWNER. Yes. There clearly is some clarification that needs 

to occur so that agencies have clear direction going forward. Our 

17 

report highlighted very simplistic items that the policy should 

cover. For instance, when you establish a new baseline, how do you 

validate it? That is very consistent, Ms. Evans, with the integrated 

baseline reviews that you require and those types of things. So it 

is not inconsistent with things that OMB is already endorsing. 

But the other key part of their policy is it has to be approved by 

management. We found that a lot of policies didn’t even require 

when agencies rebaseline an approval from key managers. That is 

all about the transparency thing so that, in fact, we are getting accurate data on costs and schedule performance. 

Senator CARPER. Mr. Denett, does your office collect information 

on capital investments? 

Mr. DENETT. That is through the A–11 process, through the 

budget. So we get these 300s and the budget side, the Resource 

Management Office culls through those and looks at them to see 

which ones are using earned value management and which ones 

are behind or doing well on cost and schedule. 

Senator CARPER. OK. What do you do with that information? 

Mr. DENETT. The Resource Management Office looks through 

them and goes back, challenges them, sometimes sends them back, 

asks for more data, asks them what they are doing to fix it, and 

it all rolls up into the budget process. But, important also is the 

caliber of people we have working with this, and as I mentioned 

earlier, we are real pleased that we now have certification requirements for the managers and people assigned to this, contracting officers, program managers for the first time have to meet certain experience levels and they have to take mandatory training. All of 

that will also assist in doing a better job in this area. 

Senator CARPER. All right. Let me just ask you what Congress’ 

responsibilities are, Ms. Evans and Mr. Denett. I think we understand that GAO is our watchdog to help us better ensure that you 

are doing your jobs well. What do you see as the appropriate role 

for Congress in our oversight responsibility as the Legislative 

Branch? We don’t pretend to be experts on these hundreds of different IT projects or capital investments. At the same time, we 

have an obligation to authorize and appropriate monies to fund 

these projects. If we don’t have information that is timely and that 

is understandable, we are not able to do a very good job as legislators in our oversight capacity. What do you see as our appropriate 

role that would enable us to be constructive? 

I think part of it is to embarrass. I am a person who believes in 

rewarding behavior. I am a big positive reinforcer. But I think the 

time comes when people need a swift kick in the pants and there 

are times when folks just need to be embarrassed. Sometimes, I 

think that is an appropriate role for us. Sometimes, it is an appropriate role for OMB, and certainly for GAO. 

But let me just ask your thoughts about how the Congress, including this Subcommittee, can be more constructive. Mr. Powner, 

any thoughts? 

Mr. POWNER. Well, I would agree that you need to shine a spotlight on the problems. That is very helpful at times and perhaps 

it is public embarrassment, but it gets action. Hopefully, over time 

with the oversight, we can move down the road here and focus 

more on the solutions. I mean, we know all these projects, we know 

18 

the agencies, the projects, and the problems with specific projects. 

We know what the issues are. Now we need to put the fixes in 

place to look at how we implement things going forward. I think 

your second panel is a good start, where we start looking at that. 

But again, we are going to probably continue to need to step back 

to make sure that you are getting accurate information because I 

contend that there is still some information here that is understated. 

Senator CARPER. We have talked about some of it, but just be 

more specific. You say you contend that some of the information is 

understated. Just elaborate on that some more. I just want to stay 

on this point. 

Mr. POWNER. Out of 800 major IT projects, reporting 70 that 

have performance shortfalls on cost and schedule is an understatement. 

Senator CARPER. All right. Fair enough. Ms. Evans, my first 

question. 

Ms. EVANS. If I could point to—— 

Senator CARPER. No, I want you to come back to my question 

what—— 

Ms. EVANS. Well, I am. I am going to give you an example. 

Senator CARPER. We spent plenty of time telling you how you 

ought to do your job. 

You have a chance to give us some advice on how we can better 

do our job so you can do your job better. 

Don’t pass up this opportunity. 

Ms. EVANS. I know, and so I do believe that there would be indicators off of the information that we provide to you that by having 

hearings, and I don’t disagree that having public hearings ensures 

transparency and accountability, and you can use this set of hearings as an example because there is a particular area when we first 

came up here you had no information. You just had gross numbers 

of what we were saying and you kind of had to trust us that we 

were telling you the right thing. 

So I think that if the agencies come up here with the mindset 

along the lines that you are there to achieve the same goal as ourselves, that then it becomes a partnership. And I think that the 

hearings themselves and the way that you have approached the 

hearings has allowed for that partnership to happen, and so that 

is why we are at the point with all the information that is out 

there. 

But to your point about—and on one of the charts it has the Census project, which we can all talk about. By having those hearings, 

it does have the agency very focused on it. The way that the policies are set up, there are indicators on there that you could see it 

before they have to do a budget amendment. There are things that 

you can do based on pulling this information out that if you had 

a particular area that you were interested in, you can request the 

more detailed information. GAO can request the more detailed information from the agency themselves. 

We have set up everything that way so that you can get those 

quarterly reports, the monthly reports, everything that goes behind 

the high-level numbers, and by talking with the agencies or having 

your staff work directly with the agencies without a hearing—I 

19 

mean, a lot of the work that we have done under the GAO HighRisk List in conjunction with GAO and staff up here, those aren’t 

hearings, but the whole idea of having meetings with the staff and 

having to present their goals and their plans and then us reporting 

out on it on a quarterly basis and a 6-month basis really keeps 

agencies on focus, and that is not even a hearing. That is just saying what you are going to do in the next 6 months and then come 

back up here and present what you have done. 

Senator CARPER. All right. Let us talk about solutions. We have 

talked a bit about information technology projects that are making 

it or not making it, how we can get better information, more timely 

information on a regular basis, and information we can understand. In terms of solutions and solving these problems and moving 

forward, a long time ago, when I was a Naval flight officer, when 

I was on active duty on the West Coast, our job was to hunt for 

Red October. We flew airplanes, 13-man aircraft that tracked Soviet nuclear submarines in all the oceans of the world and we had 

a number of squads. We flew an airplane called the P–3. 

We would all go through a training command to be prepared to 

be assigned to our respective squadrons and then we would get to 

our squadrons and we were assembled into 13-man crews and fly 

out our missions. But our respective squadrons didn’t always do an 

especially good job and we had some squadrons that were better 

than others and we had some crews that were better than others. 

And the admiral that was in charge of our operations came up 

with an idea. I think he called it a Tactical Training Team, and established it in the basic training squadrons in which we all initially 

were trained. These Tactical Training Teams would be deployed to 

squadrons that weren’t doing a particularly good job throughout 

the Pacific and Indian Oceans. They would work with us for a 

while and then they would go back home or deployed to assist 

other squadrons on an as-needed basis. There was always plenty 

of work for them to do because we weren’t always uniform—just 

like these IT projects. You have pockets of excellence. We had pockets of excellence in the work that we did. 

My staff in working with others has come up with an idea. It reminds me of the Tactical Training Team idea. Our folks here call 

it an IT Strike Force. I would like to think that it is a constructive 

and realistic solution, or part of the solution to what we are trying 

to do here. I am not sure, but I hope that it is. 

But Ms. Evans, I believe you may have seen a pre-release draft 

of the bill that I am going to be introducing later today with Senators Lieberman and Collins that will allow your office to create, 

if you will, a Tactical Training Team, but really an IT Strike Force. 

It would be comprised of experts from both within government and 

maybe from outside of government, folks who can help agencies 

control some of our IT problems before they become even more unwieldy. 

And the hope here is that we won’t see a repeat performance like 

we have experienced with our Census hand-held devices. This 

Strike Team notion would help agencies control small problems before they become billion-dollar problems, as they are today. 

First of all, is this a realistic proposal in your idea? Is this an 

idea that is half-baked? Is this an idea that has promise but needs 

20 

to be further thought out? What do you like about it? What are 

your concerns about it? I would just start with asking you for your 

comments and then I will ask Mr. Powner and Mr. Denett, as well. 

Ms. EVANS. Sir, the initial reaction to it is that it is a solutionsoriented approach. It is trying to get to the goal and identifying the 

problems quick enough in a way that we can prevent a situation 

like the Census from happening again. So the initial concept, I believe, is a good concept and is a solutions-oriented concept. 

We do something on a very similar basis, but it is not as formal 

as that process would be, now as we identify the problems, but they 

are not as quick. We are not getting ahead of the problem. 

One of the efforts that we have underway now, which I think 

this particular feature would complement, is what we are calling 

the Policy Utilization Assessment effort that we started this year. 

So Mr. Powner has talked a lot about the quality of the data, and 

it is only as good as the agency reports to us. So what we have 

started is an initiative with GSA that actually looks at our policy 

in totality and then does an assessment. 

So, for example, in the cyber security area, we have a policy, 

total policy going from acquisition all the way through how you accept it. If I looked at the numbers reported by the agencies, I would 

be at 50 percent. But what we have worked on and what we have 

developed is a methodology that gives us statistical certainty. I am 

really only at 30 percent. 

So based on that, using that and then in complement with what 

is envisioned, I believe, in this bill, I could use this to go into an 

agency and say, OK, you said that you are green. You have all 

these things in place. You haven’t rebaselined, so we are going to 

do an assessment of the policy, not an audit like GAO, but an assessment, and it would tell me whether they were at 100 percent, 

90 percent, or 60 percent. If they were below a certain threshold, 

then this team could go in, analyze what the issues are, and then 

help the agency to move forward to really realize that 100 percent 

implementation. 

Senator CARPER. Mr. Powner, this notion, this IT Strike Team 

notion, is it half-baked? Is it more than half-baked? One of the 

things I think you mentioned in your testimony, we have these IT 

projects that we know are problematic. They appear on a High-Risk 

List or a Management Watch List, not just once, but again and 

again and again, for extended periods of time, for months and in 

some cases for years. And maybe that might be an appropriate 

place or instance in which to deploy these teams. 

Mr. POWNER. Yes. I think the idea is a good one from this point. 

Agencies need help, and if you look internally to some agencies, 

they set up similar—they refer to them as Centers of Excellence 

where they can go to for help in establishing contracts. How do we 

establish a baseline estimate up front? How do we define our requirements? How do we manage risk? And so you put the appropriate processes in place. And if you look at those things, a lot of 

those are the root causes why we have projects on this list, why 

we rebaseline. We got the requirements creep. Requirements creep 

is out of control so we have rebaselined. That was the No. 1 reason 

we found. 

Senator CARPER. Is that right? OK. I am not surprised. 

21 

Mr. POWNER. No, exactly. So if you put this group together, I can 

tell you right out of the gate what some of the areas you want to 

focus on: Getting good estimates up front, defining your requirements well, putting in place a sound risk management program, 

and overseeing contractors, and there are probably some other 

thoughts, but those are four areas where we have major weaknesses across the board. 

So I think it is a good idea because if you focus on solutions and 

areas where they can go for help—but what is important is that 

group from a central government point of view, there will be a lot 

of pressure on that group because there is a lot of help that is 

needed. What is important is to establish that group and then to 

have that somewhat replicated in agencies, too. And we already 

have some of that. I think DOD, FAA, IRS, some of the big organizations, you will see pockets of that occurring. 

So it is also important to do that centrally, but to make sure that 

it gets replicated and not to lose sight, too, that it is about the people, also, not just the processes but the people. We need to bolster 

our workforce. I think Ms. Evans comes out with an annual report 

showing that we need help in the PM area, with architects, engineers, those types of folks. 

One other item that I would like to highlight is if you look at IRS 

as an example, where they had some successes, they used what is 

called Critical Position Pay Authority, where you actually can pay 

above some of the Federal limits to attract some better folks. So 

in addition to these processes and strike forces, we need to continue to bolster our workforce through some of those existing processes. 

Senator CARPER. All right. Well, I think I have probably kept you 

here long enough. I wish that more of my colleagues were here 

with us today. I know Dr. Coburn has a lot on his plate these days, 

but is keenly interested in these issues. He has not gone away. He 

is going to be around for a while. He has self-imposed a 12-year 

time limit if the voters of his State concur with that, and they 

might. 

I am going to be around for at least another 4 years and my 

guess is that we will continue to work together on this Subcommittee for a while, so our interest in this issue, in these issues, 

as well as a broader range of issues that involve concerns about 

just how wisely we are spending our resources, we are going to stay 

on these, but we are going to stay on this one, as well. 

I think we are making some progress, but God knows we need 

to make more. There is plenty of work for us to do here. 

Mr. Denett. 

Mr. DENETT. I agree with what my colleagues have said. I would 

like to tack on, we do have an initiative called the SHINE initiative, because so often—— 

Senator CARPER. The SHINE? Is that an acronym or what is 

that? 

Mr. DENETT. It is just meaning shine, give people a chance to 

shine and look good. Too often, they are focused on, when they get 

behind schedule and over cost, and that is appropriate, but we 

don’t often enough praise those that are within budget and on 

schedule. So SHINE is an attempt to give recognition to programs 

22 

and employees that are doing well. So we gather them in, give 

them some praise—— 

Senator CARPER. That is good. 

Mr. DENETT. I think it would be helpful and would even like to 

approach you when we have our next round of those to see if you 

would be willing to participate in honoring some of those special 

projects. 

Senator CARPER. I would be pleased to do that. I would be 

pleased to do that. What we used to say in the Navy, praise in public, reprimand in private. We would be pleased to join in the praise 

for some of these folks. But every now and then we like to, around 

here, we like to reprimand in public, too. It has a salutary effect. 

[Laughter.] 

All right. I suspect you are going to have some follow-up questions for the record. I would just ask that you respond to them 

promptly, as you always do. 

Thank you for your stewardship here, and Mr. Powner, a real 

special thanks to you and your folks at GAO who are a big help 

to us in this area. Thank you. 

All right. Mr. Grasso, Dr. Brown, Dr. Jarrett, we are happy to 

see you, Tom Jarrett. Welcome. 

Our second panel today begins with Al Grasso. Mr. Grasso is 

President and Chief Executive Officer of the MITRE Corporation, 

a not-for-profit organization chartered to work in the public interest. I think you all have been involved, as I understand it, in doing 

some work on the Census project and we are very grateful for the 

work that you are doing there. At MITRE, Mr. Grasso is responsible for developing and leading the corporation’s overall strategic 

and business operations. MITRE has been involved in helping to 

oversee multiple complex and high-risk IT investments in the Federal Government. You have had plenty to do. 

Our next witness is Dr. Norm Brown, and I couldn’t help but notice Dr. Brown sitting in the first row behind the witnesses on our 

earlier panel. Dr. Brown did a lot of head nodding one way or the 

other, so now we will let you actually lend a voice to all the body 

language that you were sharing with our staff and me earlier. 

But he is Executive Director for the Center for Program Transformation. For over 30 years, I am told, Dr. Brown has served as 

a commercial program manager and program management troubleshooter, and you have also served as the Assistant Secretary of— 

did you serve as the Assistant Secretary or in the office of the Assistant Secretary of the Navy—in the Navy, good for you—where 

you worked across all military branches to bring troubled projects 

back on track. In addition, I am told that you represented the military services on the 2000 Defense Science Board with the mission 

of solving why risky IT investments spiral out of control. 

And our last witness, I am tempted to say saving the best for 

last—he is a home boy—but Tom Jarrett, whom I have been privileged to know for a long time, Secretary of our Department of Technology and Information for the State of Delaware. Mr. Jarrett is no 

stranger in testifying before this Subcommittee. Previously, he testified as President—President, that is a job a lot of my colleagues 

would like to have—President of the National Association of State 

Chief Information Officers on issues relating to cyber security when 

23 

1The prepared statement of Mr. Grasso appears in the Appendix on page 100. 

Senator Coburn was Chairman of this Subcommittee. Mr. Jarrett 

oversees an IT investment budget of over $200 million and has 

achieved 90 percent of cost, schedule, and performance goals for the 

past 7 years as the Chief Information Officer of Delaware. 

When I was governor, we made progress on a whole lot of different fronts overall in our education and welfare system. I was 

very pleased with much that we accomplished, such as job creation 

and job preservation. One of the areas that we made some 

progress, but maybe not enough, is the area of our government that 

Mr. Jarrett now leads. He has taken our State clearly to the next 

level. I think one of the very proud things that our current governor can take credit for is the work that Mr. Jarrett and his folks 

have done in his department. So, welcome. In terms of best practices, we think you are one. 

Mr. JARRETT. Thank you. 

Senator CARPER. Mr. Grasso, I am going to recognize you for 

your statement, and again, I would ask you to use about 5 minutes. 

If you go a little over, that is all right. And then once all of our 

witnesses have testified, we will come back and ask some questions. Your entire statement will be made part of the record, so 

please proceed. 

TESTIMONY OF ALFRED GRASSO,1 PRESIDENT AND CHIEF 

EXECUTIVE OFFICER, MITRE CORPORATION 

Mr. GRASSO. Thank you, Chairman Carper. Thank you for affording me the opportunity to appear before this Subcommittee. I fear 

that many of the remarks I had prepared have already been stated 

this morning, but I hope to reemphasize some of those key points. 

Senator CARPER. Around here, we talk about an echo effect. It is 

really hard to cut through the media and to actually deliver a message to constituents, to voters. If you are the President, you have 

a big megaphone and you have a lot of echo effect from your cabinet secretaries and others who work for the Administration, so you 

have a good echo effect. But things have to be said over and over 

and over again in order for them to get through, including to people 

like us up here. So a little repetition is not bad. 

Mr. GRASSO. Thank you. Our company’s 50 years of experience, 

contributions, and accomplishments has given us a perspective that 

I believe is highly relevant to the topic of information technology 

planning and management. From the early days of the SAGE air 

defense system to present-day deployment of advanced command 

and control and business modernization systems, MITRE has been 

witness to great successes and, similarly, to great disappointments. 

We are honored to be asked to share our lessons and insights with 

your Subcommittee. 

Federal IT programs operate in an environment of rapid technology evolution, where some system components become obsolete 

before the program completes. This pace of technology change challenges program teams to keep their technical skill base current. IT 

systems and business processes are increasingly interconnected 

within and across agencies, making it hard to achieve consensus on 

vision, operational concepts, and requirements. The Federal GovVerDate Nov 24 2008 09:09 Jun 09, 2009 Jkt 044586 PO 00000 Frm 00027 Fmt 6633 Sfmt 6633 P:\DOCS\44586.TXT SAFFAIRS PsN: PAT
24 

ernment’s stretched fiscal and human resources further complicate 

the situation. 

The net effect is the widespread failure of many programs to deliver on time and on budget, with only a few notable exceptions 

where programs are able to overcome these challenges and succeed. 

Our experience leads me to comment on several critical areas and 

to offer three steps for improvement. 

My first comment pertains to governance. Governance relates to 

decisions that define expectations, grant power, assign accountability, or verify performance. Effective governance comprises consistent management cohesive policies, processes, and decision 

rights for a given area of responsibility. Governance becomes increasingly complicated as programs and processes cross organizational boundaries and intersect multiple governing bodies. Authorities and responsibilities become ambiguous and program managers 

are disenfranchised. It is often said that the debate begins in government once the decision is made. 

Successful programs must have unambiguous governance. Decision making authority—— 

Senator CARPER. Say that again. Successful programs must have 

what? 

Mr. GRASSO. Unambiguous governance. 

Senator CARPER. OK. 

Mr. GRASSO. Decision making authority and accountability that 

address the implications of intersecting organizations must be 

clearly defined at the onset. These authorities must encompass the 

areas of budget and finance, investment portfolio management, 

business process, and program and project management. 

My second comment pertains to requirements, an equally important consideration. Requirements, reality, and flux are often recognized as the root cause of program rebaselining. Rebaselining is not 

necessarily a dirty word, but a necessary part of delivering capabilities that meet users’ needs. Requirements are too often determined in the absence of cost, schedule, and technology risk consideration, and once determined, they are very difficult to change. 

The biggest difference between successful commercial IT developments and troubled government IT acquisitions is how requirements are managed. Successful commercial IT developers handle 

requirements with great caution. If a certain requirement adversely drives cost, performance, or schedule, it is quickly modified 

or eliminated. This does not happen in a typical government IT acquisition. Time to market is a competitive driver in the commercial 

marketplace, and I would submit it is as important, if not more so, 

in a world where adversary capabilities change as quickly as the 

technology cycle. System requirements must be considered living, 

but managed with a controlled process to use regular trade-off 

analyses to determine the value of change. 

My final comment addressees program management practices. 

Successful programs are characterized by a strong government Program Management Office (PMO), capable of a peer relationship 

with the contractor on systems engineering and program management issues. With a strong and capable PMO, the government has 

the capability to make informed decisions and manage the risk in 

acquisition programs. 

25 

A key function of a strong PMO is best described by the metaphor of an architect’s relationship with the user and the builder of 

a building. The architect is the user’s agent as independent of the 

builder. The architect works to understand the user’s operational 

needs and translate them into technical requirements enabling 

builders to develop the needed capability. The architect evaluates 

development feasibility and performs an independent conceptual 

design and cost estimate. These architect functions enable the user 

to make informed cost and capability tradeoffs and prioritize requirements. The architect is accountable to the user to ensure that 

delivered capability meets the user’s highest priority needs within 

the constraints imposed by available technology, funding, and time. 

I offer the following recommendations based on our experience 

with these issues. First, change the tone and tenor of oversight to 

focus equally on programs that have gone from bad to good and 

good to great, to reveal best practices which then can be applied 

more broadly. No program is without risk. We should all be more 

interested in those programs that have managed the risks well and 

harvest those results for the betterment of the larger set of programs. 

Second, to navigate the dynamics of uncertainty of today’s environment, IT programs are best structured as a portfolio with internal planning and management flexibility. Oversight should focus 

on the long-term funding envelope and the overall capabilities to be 

delivered. This allows flexibility at the program level to make informed trade-off decisions and to concentrate on manageably-sized 

increments that deliver capabilities in shorter time frames. This 

approach makes it easier for programs to demonstrate success or 

to fail early, which is valuable if a program is put in place and 

funded contingencies. It also puts capabilities in the hands of the 

users more quickly. This incremental approach is the norm in commercial practice. 

Third, Congress should continue to support and refine programs 

such as the DOD’s Highly Qualified Experts program and, as Dave 

Powner mentioned earlier, the IRS’s Critical Pay Authority that 

helps attract and retain critical government professionals. Additionally, the IRS’s pay-for-performance program has helped motivate performance aligned to outcomes. These are valuable tools 

that address the capacity, capabilities, and incentives needed to 

manage effective programs. We encourage the Congress to look to 

these as models, streamline their execution, and broaden their application government-wide. 

I request that my prepared statement be included in the record 

and I would be pleased to answer questions. 

Senator CARPER. Your entire statement will be included in the 

record. That was an excellent statement, an excellent summary. 

Thank you very much. 

Mr. GRASSO. Thank you. 

Senator CARPER. Dr. Brown, welcome. 

26 

1The prepared statement of Mr. Brown appears in the Appendix on page 111. 

TESTIMONY OF NORM V. BROWN,1 EXECUTIVE DIRECTOR, 

CENTER FOR PROGRAM TRANSFORMATION 

Mr. BROWN. Good morning, Chairman Carper. First, let me congratulate you for holding this hearing since literally billions of taxpayer dollars are wasted every year in poorly managed IT. Clearly, 

you are onto something important. 

Vice Admiral Jerry O. Tuttle, Retired, the former Deputy Chief 

of Naval Operations for C4I and an icon for naval computing and 

net-centric warfare would counsel, ‘‘Lead, follow, or get out of the 

way.’’ Thank you, Senators. Thank you for leading the way. As we 

say, bravo zulu. 

Let us be clear about one thing. Although it is difficult to effectively manage a large-scale project, on the other hand, producing 

a large-scale IT train wreck is easy. The good news is that wrecks 

can be avoided by effectively using best practices. 

Today, I would like to briefly offer actions that government departments and agencies, OMB, and Congress can take to prevent 

wrecks. At its core, these actions address rapidly achievable improvements. 

I will begin with a structural observation. Much is expected of 

each agency CIO. Many have responsibility without real authority. 

Many Federal departments include numerous essentially independent fiefdoms because Congress has so arranged it, fiefdoms 

independently funded by Congress. The Pentagon rule is, he that 

has got the gold makes the rules. So, too, in Federal agencies. Although I don’t today have any solutions to offer, I would be happy 

to work with your staff. 

Next, an observation regarding those IT problems in project management and oversight. When OMB testified before you last September, they expressed a recent interest in IT program execution, 

and that is a very good thing, but IT programs simply don’t manage themselves. 

From my understanding and as we have heard this morning, 

very few agencies have much in the way of any real IT program 

management and oversight. Earned value is held up as a do-all silver bullet solution, yet little is done to prevent the easy gaming 

and corruption that earned value is vulnerable to, and associated 

rebaselining may lack the transparency needed to ensure effective 

oversight. Far too much is expected of earned value. Although 

earned value is a powerful visibility technique that supports program management, earned value cannot replace program management. 

Unfortunately, there seems little in the land of government IT 

program management that implements the needed essential techniques of managing risk, requirements, and change, or integrated 

baseline reviews. As a solution, I would recommend that each agency be required to actually have real program management and 

oversight, that they focus on implementing the important critical 

details with minimum overhead, and that they identify remaining 

weaknesses and vulnerabilities. 

Transparency is dandy, but it must be converted to visibility to 

be useful. A transparent contractor can deliver a 53-foot truck full 

27 

of boxes of data, but what you really need is only the bottom-line 

information. What is needed is true visibility of IT project health 

and progress in near real time. 

Since earned value seems to be the only principal visibility technique relied on by OMB and the agencies, I would propose a more 

comprehensive visibility product, let us say an Exhibit 350, to provide real project visibility indicators monthly, primarily for the program manager, with quarterly simplified versions for agency and 

component CIOs, OMB, and Congress. I will be happy to work with 

your staff and OMB on this. 

Tracking schedule progress is not easy. One reason for schedule 

surprises is that it is pressure to meet schedule increases. The 

hard-to-do things are kicked down the road, with difficult, 

uncompleted requirements now moved into the future, a future 

which was not planned to receive it. As it turns out, this unplanned 

future work will now require a successive series of miracles to be 

accomplished in order to complete the development on time and on 

budget. Don’t bet on the miracles happening. 

To motivate agencies to focus on IT project management and 

oversight, I would recommend adapting the Nunn-McCurdy notification process, not that it is a great visibility technique; it is not. 

But it, in fact, serves as a powerful motivator, as something really 

to be avoided. As little else can do, it gives a clear focus to the business of cost and schedule containment. 

We have a serious problem regarding people. It is difficult to reward good talent, to hire good talent, and to train good talent. We 

expect CIOs and our IT personnel to do more with less and then 

give them less as if to prove the point. OMB needs to address this 

people issue as a priority. 

Training project personnel to effectively implement fundamental 

processes is minimal. While DOD has a certificate program in IT 

program management, GSA long ago disbanded their excellent 

similar IT Trail Boss program for civilian agencies. The various 

communities charged with making IT development work have received essentially no education, training, or any certification to do 

what they are asked to do. No training, no education, no certification, and no experience—it is a wonder we do as well as we do, 

as bad as it is. 

Apropos of your remarks earlier this morning about pulling the 

plug, a program termination process should be considered. The 

former Assistant Secretary of the Army for Acquisition, the Hon. 

Claude Bolton, is a strong proponent of terminating projects if 

measurable outcomes cannot be achieved within the agreed-to program cost schedule and performance baselines. The DOD has no 

formal decisionmaking process or policies to terminate programs, 

and I am not aware of any for other agencies. I think it wise and 

prudent to consider including such a process among ways to improve the government’s IT acquisition process. 

Contract incentives and other considerations are important, and 

I discuss them further in my formal testimony. 

That concludes my summary and I will be happy to take questions at the appropriate time. 

28 

1The prepared statement of Mr. Jarrett appears in the Appendix on page 120. 

Senator CARPER. Again, another excellent testimony. Thank you 

for the thought that you put into it. Thank you for your years of 

service, too, and your counsel here today. 

Mr. Jarrett, you are recognized. Please proceed. Welcome. 

TESTIMONY OF THOMAS M. JARRETT,1 SECRETARY, DEPARTMENT OF TECHNOLOGY AND INFORMATION, STATE OF 

DELAWARE 

Mr. JARRETT. Chairman Carper, Ranking Member Coburn, and 

Members of the Subcommittee, thank you for the opportunity to be 

here today. As Secretary of Delaware’s Department of Technology 

and Information, I can well appreciate the complexity, the challenges, and the significant responsibilities associated with managing information technology projects in an investments portfolio 

that cuts across many agencies, and in Delaware’s case, all three 

branches of government. 

Albeit in a much smaller scale than the Federal Government, 

Delaware and other State Governments are faced with similar concerns regarding IT project management. In fact, Delaware’s Department of Technology and Information was established in part 

because of ongoing IT project delays and cost overruns that Delaware was experiencing. Delaware’s centralized IT structure charges 

our agency with direct oversight and approval on nearly $200 million in active IT projects. 

As a new agency, we were able to develop and employ new approaches to IT project management that incorporate many best 

practices from private industry and others in the government sector. While our methods are under constant review for improvement, 

we are enjoying some significant progresses. We have an excellent 

track record of delivering much-needed IT solutions that are on 

time and on budget. 

There are no silver bullets, no one or two changes that you can 

point to for perfect project management. Instead, there are many 

small improvements that we have made that, in the aggregate, are 

making the difference for Delaware. I would like to take a few minutes to give you a high-level view of a project life cycle using the 

Delaware model. In addition, I would like to point out what I believe are the significant processes and procedures that we have put 

in place to successfully manage our portfolio. 

Like the Federal model, we require agencies to submit a business 

case that addresses the major items we believe help ensure a 

project’s success. Our model includes the following major areas: 

Risk management, processes reengineering, architectural review, 

resource and funding availability, project management oversight, 

organizational change management, needs assessment, customization requirements, disaster recovery levels, and management 

and executive sponsorship. 

Recommended projects are forwarded for the concurrence of the 

State CIO before they move ahead in our process. This recommendation usually includes ongoing funding contingent on meeting project milestones. Once approved, DTI works with the agency 

customer to develop full and complete requirements so a request 

29 

for proposals can be released to secure vendor bids in order to meet 

the needs of the project. DTI stays involved to help the agency 

make the vendor selection and to structure a contract that ensures 

the project is delivered on time and on budget. 

I cannot emphasize enough how critical the requirements gathering process is to the project’s ultimate success or failure. The 

Delaware model does not allow for requirements gathering to be 

conducted solely by the vendor. There must be an active involvement by the DTI project management team. There is an old saying, 

‘‘The customer doesn’t know what they want until you give them 

what they ask for.’’ Requirement gathering is critical to the process 

and helps alleviate scope, time, and budget creep if it is done correctly. 

At the start of a new project, a nationally certified program or 

project manager is assigned, as well as a certified organizational 

change management team, to run parallel courses in managing our 

projects. It is not enough to be certified. We take extra care in selecting the people to fill these positions, as they are essential to the 

success of the project. We count on these folks to do what we call 

inflicting discipline and structure to these projects. 

It is important to note that DTI manages Delaware’s IT projects, 

not the vendors assigned to them. While we demand the vendor assign a certified and talented project manager, our State folks manage the project. 

Further, I would like to highlight our change management process. Change management starts with the review of existing business processes and is focused on preparing the organization for the 

cultural changes that are a part of any major project. According to 

a recent study conducted by the National Association of State Chief 

Information Officers, which represents State CIOs across the 50 

States, 80 percent of major IT project failures can be directly attributed to a lack of change management. Too often, the employees 

who will actually use the new application are left out of the project 

process, and when this takes place, it is almost certain that the 

project is doomed to fail or under-perform. 

Another major element of our success is how we manage the 

overall process. All of Delaware’s major IT projects have executive 

sponsors. Executive sponsors include high-level managers from the 

agencies involved as well as key executives from DTI and our Office of Management and Budget. The executive sponsors hold regular monitoring and update meetings and provide high-level oversight. When difficult project decisions need to be made, the executive sponsors are aware of the issues and have the authority to 

make critical decisions on whether or not to keep the project moving along. 

A real example is our present project involving all of Delaware’s 

courts. Due to the lack of employee training and readiness, the executive sponsors recommended that the project be paused so that 

the training could take place. In fact, Delaware’s Chief Justice himself invoked a 6-month pause in recognition of the need for court 

employees to be prepared if the project was to succeed. 

Besides ongoing project meetings, all projects in the State’s portfolio are reviewed weekly by our Project Management Office and bimonthly at a workload management meeting where project manVerDate Nov 24 2008 09:09 Jun 09, 2009 Jkt 044586 PO 00000 Frm 00033 Fmt 6633 Sfmt 6633 P:\DOCS\44586.TXT SAFFAIRS PsN: PAT
30 

agers present their projects’ status to the assembled DTI senior 

managers. Opportunities for improvement or needed adjustments 

are vetted in an open forum with the goal of keeping the project 

on track and transparent. 

Our agency is committed to project transparency all the way to 

our legislature and our governor. We believe in continuous information exchange and dialogue with our elected representatives so that 

they are educated and aware of the complexity of major IT projects. 

Providing information up front, even when it may be painful, is far 

better than saving unpleasant surprises for yearly budget sessions. 

Bad news does not age well. 

I wish I could tell you that everything is humming along perfectly and that the processes that we have put in place are the final 

answer, but I can’t, as we have many challenges similar to those 

being talked about today. However, we believe that by tightly managing these challenges, we can deliver projects on time and on 

budget. Although our project tracking system is complex, in our 

communication with stakeholders, we use a high-level red/green/ 

yellow scorecard to inform them how the projects are developing. 

All projects have or will have ongoing issues that require senior 

management attention. Under the Delaware model, we believe that 

providing the facts, good or bad, to all stakeholders, including the 

legislature, is the proper policy, and most importantly, lives up to 

one of my agency’s core values of integrity. Our vendors are held 

accountable to deliver what they said they would, as well, and we 

call the facts as we see them. 

I thank you for the opportunity to testify today and would be 

happy to answer any questions. 

Senator CARPER. Thank you. Thanks not just for your testimony, 

but for the terrific leadership that you provide for our State and 

have shared some of the fruits of that leadership with us today. 

I think I would like to ask, first of all, Mr. Grasso and Dr. Brown 

a question and ask you to reflect on what we have heard from Mr. 

Jarrett today. I have oftentimes described the 50 States as laboratories of democracy and the belief that somewhere in those 50 

States, somebody has come up with a solution that will help us 

solve a number of the problems we face at the Federal level, and 

the same is true of the private sector. Somebody has figured out 

how to solve most of the problems, not all, but most of the problems that we face as a Nation and we just have to figure that out 

and be able to grow them to size or to scale. 

What did you hear from Mr. Jarrett in talking about Delaware, 

the way we operate in our State, that might be applicable to us 

here at the Federal level? What are some good lessons learned that 

you think we could take from his testimony and apply them to the 

Federal Government? 

Mr. GRASSO. I think a strong element that I heard in the successes that we have seen in Delaware is the investment, again, 

that is made in the project management team, in the strength of 

that project management team. We heard that the project manager 

is certified, but it goes much further than just simple certification. 

There is a continuous development effort that is required. 

With a strong project management team, that project management team can be held accountable to basically continue to own the 

31 

technical baseline of the program and not just basically contract it 

away such that you just get what you get. The team is informed 

throughout the process, is able to establish that peer relationship 

which I mentioned in my testimony with the contractor, and that 

allows for the successful acquisition. 

Requirements are important. We heard that. The level at which 

requirements are stated are very important. All too often, we detail 

requirements down to a level that we believe perhaps causes some 

of the rebaselining that we see because we specify things in terms 

of technology as opposed to in terms of outcomes. Although he 

didn’t say it, the importance of requirements and the emphasis 

placed on requirements, I suspect they may have gotten that right 

in terms of the level by which they establish requirements. 

So I think a combination of the strength of the program management team, the investment made in those people to keep them 

highly qualified, and the ability to deal with requirements at the 

right level have achieved some of the successes. 

Senator CARPER. Thanks for that analysis. 

Dr. Brown, what did you hear from Mr. Jarrett’s testimony that 

you think might apply to us at the Federal level? 

Mr. BROWN. First, let me echo what Al Grasso has said. I think 

the emphasis on requirements development and control and change 

control is absolutely fundamental and determines much of our cost 

and schedule problems that we see. 

And I think there is one other thing that Mr. Jarrett hasn’t fully 

articulated to you that is very relevant, which is his process of paying their IT folks. Maybe you could talk more about that. 

Senator CARPER. You actually pay them? 

Mr. JARRETT. Yes, imagine, they want to get paid. [Laughter.] 

As you know, Mr. Chairman, a number of years ago, the legislature allowed Delaware, and I still believe the only State today that 

has ever done it, to take the entire IT organization and convert it 

from a civil service structure to a non-civil service structure, and 

along with that, they gave me the ability to write our own compensation structure. So I think we are one of the few, if only, State 

agencies in the country that actually pays its IT people to the market, and what that means is that we have very qualified people, 

and I am happy to say we also have the highest retention rate of 

any agency in the State. 

Senator CARPER. I used to be State Treasurer. I remember a time 

early on when we thought that we hired people, usually fairly 

young people out of school, and trained them so that they could be 

hired away by the private sector or some other employer. That 

sounds like that is not the case anymore, is it? 

Mr. JARRETT. That is not the case, not in my department. 

Senator CARPER. OK, good. Dr. Brown, do you want to add anything? 

Mr. BROWN. Sure. One thing that you had talked about that particularly resonated with me was the concept of the tactical training 

teams, and something that I had started at the Navy Department 

and later became DOD was the Software Program Managers Network. We had 10,000 members across the country. We supported 

over 200 programs in helping them. We had a stable of these people that we would call tactical trainers, experts in a wide range of 

32 

subject matter experts, over 100 of these people, and as programs 

need, they could just call in the tactical team. 

If they needed help on a task activity network to support earned 

value or if they needed to better understand how to identify risks 

in the program or had a plan for testing or had a better oversight 

of what the contractor is doing or incentives, anything like that, 

they could call in the team. That is very consistent with your IT 

support team, and I think that is a fundamentally important thing 

to do. 

Senator CARPER. All right. Mr. Grasso and Dr. Brown have been 

good enough to comment on your testimony. Let me just ask you 

if you would do the same with respect to some of what they said 

and just reinforce the relevance of their counsel to us. 

Mr. JARRETT. Well, I was telling Dr. Brown before we started, I 

was so pleased when I had a chance to read and hear what they 

both had said because I kind of threw up my arms and said, gee, 

we are doing that and have been doing it for some time and I think 

it proves out in a couple of areas. 

One is the requirements. People get sick about hearing about 

that, but in fact, what we have found and where we have spent all 

of our time is on the front end of the project process, not on the 

back end, which means that if you are going to get them right, 

then you have to determine what the requirements are. What we 

learned in the very beginning is that is not unlike—and you need 

actually qualified and trained people to be able to do that. That is 

not as simple as going out to an agency and saying, ‘‘OK, tell me 

what it is that you need,’’ because what we have found is that in 

a lot of cases, they can’t articulate what they need, at least not in 

a way that when you are looking to replace a system and do that. 

So you have to help them through that process. So we have spent 

a lot of time doing that. 

The other is in the change management area. I think a lot of 

States and a lot of folks are moving into the areas of PMOs, Project 

Management Offices. Something that we have spent an awful lot 

of time on is, again, in the organizational change management, and 

I will give you an example. We are doing a new financial system 

in the State. We are spending a lot of our time not on the technical 

aspects but in dealing with each of the agencies so that we can 

help the users kind of work through the processes and the changes 

that this new project or this new system is going to provide to 

them. Most times, we don’t spend time doing that. 

So I hear both of those areas in the testimony that was given, 

and from my perspective, I think it is well thought out and something that I think the Federal Government could do, as well. 

Senator CARPER. Mr. Grasso, did you have a point you wanted 

to make? 

Mr. GRASSO. I might just offer an additional remark. 

Senator CARPER. Before you do that, in your testimony, Mr. 

Jarrett, you said there is an old saying the customer doesn’t know 

what they want until you give them what they have asked for. 

There is a lot of wisdom in that. 

Mr. GRASSO. In my written statement, there are a couple of references that I would just point to which emphasize the requirements point, and one of those references points to the 32 programs 

33 

at NASA that were evaluated and the up-front investment and the 

program overrun result. What you see is when the definition phase 

as a percent of the total program is greater than 10 percent, what 

you find is that the program overrun is typically less than 20 percent. 

Senator CARPER. Say that one more time. I want to make sure 

I got it. 

Mr. GRASSO. The graph refers to 32 NASA programs that have 

been evaluated and it takes a look at the definition phase of those 

programs as a percent of the total program. And when the definition phase of the program is 10 percent or greater, what you find 

is that the program overrun is 20 percent or less. 

Senator CARPER. OK. Good. 

Mr. GRASSO. And that suggests that considerate thought is being 

given to the requirements up front and thus less change is required 

later on. 

Senator CARPER. All right. Thank you. My staff has been good 

enough to prepare a number of questions, and I am just going to 

take a moment and look at those and I suspect we will use these 

as a point of departure for some further discussion. In some ways, 

the kind of discussion that we are having here is actually more 

helpful than not. 

I have asked you to sort of reflect on your respective testimonies 

here today. Go back with me to the first panel and some of the 

things that you heard from the first panel, some of the things that 

we discussed. What were some of the things that struck you that 

were especially important or maybe something that you wanted to 

add something that wasn’t said, or maybe an answer that wasn’t 

given, or at least wasn’t given as well as it might have been? Does 

anything stand out for you, referring back to the first panel’s testimony? 

Mr. GRASSO. I would offer one remark because I can’t stop looking at the report card. Having several daughters in high school, I 

have encouraged my daughters to take Advanced Placement classes, whereas I know some of her friends are taking lesser challenging classes, if you will, and—— 

Senator CARPER. What grades are your daughters going to be in 

this fall? 

Mr. GRASSO. Actually, my oldest is going to the University of Virginia this year, so she is just graduating, and my middle daughter 

is a rising 11th grader. 

Senator CARPER. OK. We have two boys and our youngest son 

just graduated from the Charter School of Wilmington and he is 

going off to William and Mary and we know all about Advanced 

Placement tests. 

We just got our results about a week ago, high fives all around. 

[Laughter.] 

Mr. GRASSO. So you know the amount of work that goes into preparing—— 

Senator CARPER. Yes, we do. 

Mr. GRASSO [continuing]. And the level of effort that is required. 

The report cards are a great start, but at the same time, all IT programs aren’t borne of the same level of complexity, the same level 

of interdependency and challenge. So as we step back and I look 

34 

at the report cards, it begs the question of what are the correlations, if you will, to those grades. 

As I look at some of those organizations, some of those organizations are more insular than other organizations, if you will, and 

there are fewer interdependencies, and as a result, what you find 

is the complexity of the IT system is not necessarily as challenging 

as an enterprise system that touches a number of different organizations, crosses boundaries, and has unclear governance. 

So a comment that I will make is a report card as such, I don’t 

know if those are AP classes or if those are substandard classes. 

So an F on AP class may not necessarily give you all of the details 

that you need to respond. 

Senator CARPER. Well, you could not have picked a better analogy. 

Mr. BROWN. My view is that those don’t represent AP classes. 

[Laughter.] 

And with regard to what Ms. Evans and Dave Powner were saying about earned value management and particularly the problem 

of rebaselining, DOD deals with that in a very direct and forthright 

manner, which is just to track all the costs and the curve and what 

you see is a curve that starts going down and then the rebaseline 

goes up and goes up and it is just this downward spiral, a death 

spiral. And it is very easy to track what is happening when you see 

that picture, and that would be very easy for OMB to provide to 

you. I don’t know if they are willing to do that, but they should 

have that information when they say—I don’t know what agencies 

exactly report to them in their private conversations, but that 

should be certainly public information. 

Senator CARPER. OK. 

Mr. GRASSO. The 1987 Defense Science Board had an interesting 

conclusion and it said, technology is not our problem, management 

is, and they were talking about software problems, and hopefully—— 

Senator CARPER. Who said that? 

Mr. BROWN. The 1987 Defense Science Board study on software. 

Senator CARPER. OK. Mr. Grasso, the folks at MITRE have been 

good to help us in the Bureau of the Census. You tried to recover 

from a very bad situation. You may not be intimately familiar with 

this, but maybe you are, and to the extent that you have some familiarity with how the Census Bureau got into this mess as they 

prepared for and approached the 2010 Census, where do you think 

they went wrong and what lessons can we take from that experience to make sure that whether it is the 2020 Census or other IT 

projects, how we don’t let this kind of thing happen again? 

As it has turned out, it is an enormously expensive problem and 

we are going to not only end up spending more money, I don’t know 

that we will get a better product. We will probably not get as good 

of a product in the end, but we will have spent a lot more money, 

a very unhappy situation. But we appreciate what you all have 

done to try to minimize the loss and get us going in the right direction. But what are some lessons learned? 

Mr. GRASSO. I think the two lessons have already been described 

today. The first is requirements. In the case of Census, as you step 

back, there certainly has been a significant number of requireVerDate Nov 24 2008 09:09 Jun 09, 2009 Jkt 044586 PO 00000 Frm 00038 Fmt 6633 Sfmt 6633 P:\DOCS\44586.TXT SAFFAIRS PsN: PAT
35 

ments changes throughout the process which has led to changes 

with the contractor. In the time period in which this all has occurred, the technology has evolved, so the solution that was envisioned some time ago perhaps is not necessarily the best today in 

terms of the architectural basis for that solution. So the program 

was not able to evolve as quickly as the technology was able to 

evolve. 

The second part of that is as all of this is evolving and all of 

these changes are being made, the strength of the program office, 

quite frankly, I would say, could have used some building up in the 

early stages. That has occurred as a result of the response to the 

issues that they have been confronted with most recently, but quite 

frankly, I would say that many of the responsibilities that should 

have been within the program office in establishing the technical 

baseline were really given to the contractor as opposed to being 

held within the program office. 

Senator CARPER. This is sort of the inverse of the situation that 

I think Mr. Jarrett described. 

Mr. GRASSO. Exactly. And we find that to be the case quite often. 

When you don’t have a strong program office, a lot gets transferred 

to the contractor and you end up with a lot more dynamics in the 

program. 

Senator CARPER. All right. Let me just ask, do you all have any 

closing thoughts that you would like to give us? I like to talk about 

take-aways, and it is impossible for me to remember everything 

that is said here. Even our staff, as smart as they are, it is impossible to remember everything. But just some of the key take-aways 

for us from what you have heard from other witnesses and maybe 

some points that you have brought up that you would like to reemphasize as we prepare to conclude. 

Is it Secretary Jarrett? Do they call you Secretary? Mr. Secretary. 

Mr. JARRETT. You may call me anything you wish. 

Senator CARPER. No, but seriously, aren’t you a cabinet secretary. 

Mr. JARRETT. I am. 

Senator CARPER. Then I shouldn’t call you Mr. Jarrett. I should 

call you Mr. Secretary. Do you want to take the first shot at that? 

Mr. JARRETT. I guess the only thoughts I had, and it kind of goes 

back to your earlier question about comments from OMB, and that 

is that reports and all the things that they put out are good and 

I guess they are a requirement here, but in fact, I have always believed that reports are only as good as how you use them to actually make a difference and to change things. 

What I guess I don’t see in a lot of ways is how those reports 

are being utilized to actually begin to change the process. It just 

kind of has become embedded in the reports, and in fact, again, if 

you are not using them, then why even produce them. We spend 

less time on reports and more on the performance of the projects 

themselves. 

The second would be I see a lot of red. Well, we actually end up 

with red on some of our projects, too. The difference is, just because 

it is red doesn’t mean it is ready to go and tank. It just means, at 

least in our case, that it is something that needs a lot of extra lookVerDate Nov 24 2008 09:09 Jun 09, 2009 Jkt 044586 PO 00000 Frm 00039 Fmt 6633 Sfmt 6633 P:\DOCS\44586.TXT SAFFAIRS PsN: PAT
36 

see and effort on it to get it back to yellow or to green, where it 

ultimately needs to be. Projects go in and out of red, green, and 

yellow all of the time. It depends on the complexity. So I think 

maybe red is nice, but I think there is this perception that red is 

bad. Yes, it is not good, but it is not necessarily the end of all 

things. 

The final thing is that in the issue about information back to 

Congress, I was telling some folks earlier, kind of the mindset 

which they take, and having come from the private sector and then 

Delaware Government for the last 71⁄2 years, I am by no means an 

expert in the dynamics of Washington, but we took a very simple 

approach with our legislature which was we will notify them and 

show them this process on everything, painful as it may be on occasion. I have had some painful discussions with Senators and Representatives one-on-one and even in groups. 

Over time, what it has built, though, is the fact that they have 

come to understand the complexities with projects. They have become educated that it is not as easy as people think it is and that 

there are a lot of bumps in the road. So now what we find is that 

they have a far better understanding of that and we have a lot 

more productive dialogue back and forth as we work through those 

particular projects than we did when we started 7 years ago. 

So I would recommend to them to take the opposite approach, 

which is to provide that information almost as much as they can 

and not be concerned about the fact of whether it shames somebody 

or not because I think as you said earlier, I think sometimes that 

is not necessarily a bad thing. 

Senator CARPER. All right. Thank you. Dr. Brown. 

Mr. BROWN. I have three things that might be take-aways. One 

is to actually have Congress require visibility at agencies and at 

OMB. 

The second is to have Congress require the agencies to maintain 

and use effective critical practices. 

And the third is for Congress to maintain its involvement. 

Senator CARPER. All right. Expand just a little bit on the second 

one. 

Mr. BROWN. On maintaining critical practices, that is to have a 

real program management and oversight capability at each agency. 

What happens is there is so much focus on the Exhibit 300s and 

the planning of programs that their execution and management go 

almost unnoticed, and that is where the troubles begin. It has been 

said that the plan doesn’t survive the first battle, and so, too, with 

the plans for our programs. Things change and are very dynamic. 

Requirements change all the time and that is why, as Mr. Jarrett 

said, change management is very important. 

Agencies tend to ignore change management. They tend to ignore 

a very important thing called risk management to identify risks. 

They don’t pay a lot of attention to what is going on in programs 

in terms of incentivizing the contractor to do the kinds of things 

that are fundamental, to examine what the fundamental capabilities that are needed by government are and not be absolutely controlled by requirements, to ignore the kinds of things that allow 

you to have a real confidence in your way of understanding what 

is going on. 

37 

Instead of just requiring earned value as your sole indicator, you 

could look at real products being developed. You want to have products being developed all the time, weekly or monthly deliveries to 

the government, and to have assurance that those products are 

being properly integrated together. I have probably given you more 

than you want. 

Senator CARPER. All right. Just enough. Thank you. Mr. Grasso. 

Mr. GRASSO. I guess I would start by saying this is hard. I have 

seen good organizations fail at IT projects, and I would start by 

simply saying that we all have A teams and B teams. We need a 

lot more A teams, which means investment in people and the processes to do this job right. 

The teams need to feel ownership, if you will, in the programs 

and need to have the authorities to do the right things. I once used 

this analogy. As a homeowner, I may make some decisions to make 

some investments in my home. This year, I am going to put $3,000 

into a sprinkler system in my home. Well, as I proceed to do that, 

I spring a leak in my roof. If this were the government, I wouldn’t 

be able to fix that roof because I don’t have the right kind of money 

to fix that roof and I would continue to fix my sprinkler system. 

I don’t have the ownership of that portfolio, if you will, to treat my 

home as a whole complex suite of things. And the ability for a program manager to be able to manage a portfolio as opposed to an 

individual activity will allow him to manage some of those risks 

and make the necessary shifts as is appropriate. 

I mentioned earlier in my testimony about contingencies and 

funding contingencies. A risk management plan is absolutely critical to any program, but a risk management plan must have actionable alternatives. If the alternatives are not actionable, then we 

will just continue to mount on the risks. 

So I think the key points that I would make is we must continue 

to invest in the people and the organizations. As I mentioned earlier, the Highly Qualified Experts Program and the Critical Pay 

Program in IRS are good examples of investment. It sounds like 

Delaware has been able to do that. We must be able to manage 

these risks and empower our program managers with strong teams 

to make the right decisions and to have alternatives and choices as 

they see the technologies change and some of those risks surface. 

Senator CARPER. Good. Thank you. Well, in closing, I really want 

to commend our staffs for being smart enough to invite you to come 

and testify. The way it works here, you have Democrats and Republicans who submit ideas and come up with ideas and ultimately 

we end up with a witness list. But this has been an especially helpful and beneficial panel of witnesses for us. 

One of the things that you are especially good at is explaining 

things in terms that even I can almost understand in most cases, 

and these are not easily understood concepts. That is a great gift. 

Thanks a lot for making time in your schedules to be here with 

us today, for preparing for this hearing, and for giving us not just 

a lot of food for thought, but, I think, a pretty good road map to 

follow to better ensure that we are getting our money’s worth out 

of these IT projects that are going on throughout the government, 

actually around the world, and for all the taxpayers who are putVerDate Nov 24 2008 09:09 Jun 09, 2009 Jkt 044586 PO 00000 Frm 00041 Fmt 6633 Sfmt 6633 P:\DOCS\44586.TXT SAFFAIRS PsN: PAT
38 

ting their money into this. Thank you for them, too. Keep up the 

good work that you are doing and much obliged. 

With that, you will probably hear from some of us with some additional questions. We will try to get those out in the next week 

or so, but if you could respond promptly, we would be most grateful. 

Thank you so much, and with that, this hearing is adjourned. 

[Whereupon, at 1:41 p.m., the Subcommittee was adjourned.] 

(39) 

A P P E N D I X 


44586.001
40 


44586.002
41 


44586.003
42 


44586.004
43 


44586.005
44 


44586.006
45 


44586.007
46 


44586.008
47 


44586.009
48 


44586.010
49 


44586.011
50 


44586.012
51 


44586.013
52 


44586.014
53 


44586.015
54 


44586.016
55 


44586.017
56 


44586.018
57 


44586.019
58 


44586.020
59 


44586.021
60 


44586.022
61 


44586.023
62 


44586.024
63 


44586.025
64 


44586.026
65 


44586.027
66 


44586.028
67 


44586.029
68 


44586.030
69 


44586.031
70 


44586.032
71 


44586.033
72 


44586.034
73 


44586.035
74 


44586.036
75 


44586.037
76 


44586.038
77 


44586.039
78 


44586.040
79 


44586.041
80 


44586.042
81 


44586.043
82 


44586.044
83 


44586.045
84 


44586.046
85 


44586.047
86 


44586.048
87 


44586.049
88 


44586.050
89 


44586.051
90 


44586.052
91 


44586.053
92 


44586.054
93 


44586.055
94 


44586.056
95 


44586.057
96 


44586.058
97 


44586.059
98 


44586.060
99 


44586.061
100 


44586.062
101 


44586.063
102 


44586.064
103 


44586.065
104 


44586.066
105 


44586.067
106 


44586.068
107 


44586.069
108 


44586.070
109 


44586.071
110 


44586.072
111 


44586.073
112 


44586.074
113 


44586.075
114 


44586.076
115 


44586.077
116 


44586.078
117 


44586.079
118 


44586.080
119 


44586.081
120 


44586.082
121 


44586.083
122 


44586.084
123 


44586.085
124 


44586.086
125 


44586.087
126 


44586.088
127 


44586.089
128 


44586.090
129 


44586.091
130 


44586.092
131 


44586.093
132 


44586.094
133 


44586.095
134 


44586.096
135 


44586.097
136 

Æ 


44586.098