GAO

United States General Accounting Office

November 2000 Determining 

Performance and 

Accountability 

Challenges and High 

Risks

GAO-01-159SP
Page 1 GAO-01-159SP Performance and Accountability Challenges and High Risks

3

Introduction 4

Criteria for Identifying and Assessing Major Program and Mission

Areas 5

Criteria for Assessing Agency Management and Program Performance 6

Criteria for Determining if Individual Performance and Accountability

Challenges Merit Designation as High-Risk Areas 7

Criteria for Removing High-Risk Designations 14

Calling Attention to Performance and Accountability Challenges and

High-Risk Problems 15

Figure 1: Major Management Functions Key to High-Performing

Organizations 6

Figure 2: Information Sources for Assessing Major Program and

Mission Areas and Management Functions 7

Figure 3: Qualitative Factors Used in Determining High Risk 9

Figure 4: Factors to Consider in Assessing the Effectiveness of

Corrective Measures to Resolve Material Weaknesses 10

Figure 5: Determining High-Risk Programs and Functions 12

Figure 6: Criteria for Determining Governmentwide High Risks 13

Figure 7: Criteria Agencies Must Meet Before High-Risk Designations

Can Be Removed 15

Foreword

Contents

Determining

Performance and

Accountability

Challenges and High

Risks

Figures
Page 2 GAO-01-159SP Performance and Accountability Challenges and High Risks
Page 3 GAO-01-159SP Performance and Accountability Challenges and High Risks

Since 1990, we have periodically reported on government programs and

functions that we have identified as “high risk” because of their greater

vulnerabilities to waste, fraud, abuse, and mismanagement. In January

1999, we issued our first Performance and Accountability Series, which

discussed the major issues 20 agencies faced in addressing performance

and accountability challenges.

Over time, as high-risk areas have been corrected and other risks have

emerged, we have removed areas from the list and added new ones to

keep the Congress up to date on areas needing attention. In the January

1999 report series, we advised the Congress that because an increasing

amount of information is becoming available as a result of implementation

of various federal management reform legislation, we planned to reassess

the methodologies and criteria used to determine which operations and

functions should be included in the Performance and Accountability Series

and those which should be designated as high risk.

This document highlights the resulting criteria and process for

determining performance and accountability challenges and those that are

deemed to be high risks. It will be used by GAO auditors in making these

determinations and by the Congress and the executive branch agencies in

understanding our basis for reporting in these areas. As in the past, the

ultimate determinations will continue to involve the independent and

objective judgment of GAO professionals in applying the criteria.

We provided the opportunity to comment on an exposure draft of this

document to interested parties, including congressional staff from the

Senate Committee on Governmental Affairs and the House Committee on

Government Reform, both of which have been involved with the program

since its inception in 1990; heads of major agencies; the Chief Financial

Officers Council; the President’s Council on Integrity and Efficiency; and

others. The exposure draft was also posted to our Internet Web site. The

comments we received were generally positive, and we used this input to

make a number of clarifications in finalizing this guidance.

David M. Walker

Comptroller General

of the United States

Foreword
Page 4 GAO-01-159SP Performance and Accountability Challenges and High Risks

We have developed—and will continue to refine—criteria to use as a basis

in forming judgments on

• identifying and assessing the performance of the federal government’s

major program and mission areas,

• assessing agencies’ management functions to determine how they

contributed to program performance and affected the agencies’ ability

to ensure accountability and achieve results,

• determining if individual performance and accountability challenges

merit designation as high-risk areas as well as determining

governmentwide high risks, and

• removing high-risk designations.

The criteria presented in this guidance document will be used by GAO

auditors to assess performance and accountability for an agency’s major

program and mission areas and management functions. Based on these

assessments, we will include in our Performance and Accountability

Series a report for each major agency that presents the related challenges

we identify for that agency.

Before we issue these reports, we will meet with agency representatives to

discuss the preliminary observations from our assessments and will

consider their views in finalizing our determinations of any related

performance and accountability challenges and high-risk areas. As in the

past, the ultimate determinations in these matters will continue to be

based on the independent and objective judgment of GAO professionals in

applying the criteria.

In determining performance and accountability challenges and assessing

whether they rise to the level of high-risk exposure, we consider the

requirements of program-specific legislation. We also consider the

requirements of management reform legislation, such as the Chief

Financial Officers Act of 1990, the Government Performance and Results

Act of 1993, the Government Management Reform Act of 1994, the Federal

Financial Management Improvement Act of 1996, and the Clinger-Cohen

Act of 1996.

Determining Performance and Accountability

Challenges and High Risks

Introduction
Determining Performance and

Accountability Challenges and High Risks

Page 5 GAO-01-159SP Performance and Accountability Challenges and High Risks

Also, the Standards for Internal Control in the Federal Government1

provide a foundation for assessing the effects and underlying causes of

control weaknesses for major program and mission areas and key

management functions. We issued these standards pursuant to

requirements of the Federal Managers’ Financial Integrity Act. Internal

control is a major part of managing an organization. It comprises the

plans, methods, and procedures used to meet missions, goals, and

objectives and, in doing so, supports performance-based management.

Internal control also serves as the first line of defense in safeguarding

assets and preventing and detecting errors and fraud. In short, internal

control, which is synonymous with management control, helps

government program managers achieve desired results through effective

stewardship of public resources. The standards provide the overall

framework for establishing and maintaining internal control and for

identifying and addressing major performance and accountability

challenges and high-risk areas.

For each major agency, we will identify major program and mission areas

that will form the primary basis for reporting in the Performance and

Accountability Series and that

• are at the center of congressional and executive branch attention,

• have high public interest and/or large-dollar outlays,

• figure prominently in agencies’ strategic plans and annual performance

plans and reports, or

• have known performance and accountability or high-risk issues.

The major program and mission areas on which we will focus will be

selected in consultation with the Congress and coordinated with the Office

of Management and Budget, the top leadership of the major federal

agencies, and the inspectors general at these agencies. Thus, the

Performance and Accountability Series may not include all of an agency’s

program and mission areas, but would focus on those that are among the

most important for each agency. Program and mission areas that in the

past were designated as high risk or had performance and accountability

1

GAO/AIMD-00-21.3.1, November 1999.

Criteria for Identifying

and Assessing Major

Program and Mission

Areas
Determining Performance and

Accountability Challenges and High Risks

Page 6 GAO-01-159SP Performance and Accountability Challenges and High Risks

concerns will be carefully scrutinized to determine if the risks and

concerns have been resolved.

We will use agencies’ strategic plans, annual performance plans and

reports, accountability reports, and audited financial statements to identify

the key goals, strategies, performance measures, and reported

performance for each of the major program and mission areas. This

information, supplemented by relevant GAO products, inspectors general

reports, and other independent analyses, will enable us to develop a

profile of the actual performance for each of the selected program and

mission areas.

Based on our experience in examining a wide range of government

programs, we have found that effective performance of the management

functions shown in figure 1 are key to creating and sustaining highperforming organizations. Internal control is a critical aspect of each of

these management functions.

Figure 1: Major Management Functions Key to High-Performing

Organizations

We will assess the results that agencies’ major program and mission areas

achieve and how well the management functions key to high performance

are being carried out. When weaknesses are identified in one or more

management functions, it does not necessarily indicate that the entire

major program or mission area is not managed adequately.

For those program and mission areas where performance can be improved

substantially, we will identify the root causes of the current performance

Criteria for Assessing

Agency Management

and Program

Performance

• Strategic planning

• Budget formulation and execution

• Organizational alignment and control

• Performance measurement

• Human capital strategies

• Financial management

• Information technology

• Acquisition

Internal control

underpins each

management

function
Determining Performance and

Accountability Challenges and High Risks

Page 7 GAO-01-159SP Performance and Accountability Challenges and High Risks

levels and help pinpoint improvement opportunities. Also, our

assessments will help to identify best practices that show the specific

contributions better management makes to high performance.

In addition, in assessing major program and mission areas and

management functions and in making high-risk determinations, we will

draw from information available through a number of sources. We will

primarily rely on recent GAO audit reports and studies flowing from our

strategic plan, legislatively mandated work, or requested efforts. These

and other sources of information we will use are shown in figure 2.

Figure 2: Information Sources for Assessing Major Program and Mission

Areas and Management Functions

An individual performance and accountability challenge merits a high-risk

designation when it meets the criteria presented in this section. We will

first determine whether the performance and accountability challenge

involves

• a program or mission area having national significance or

• a management function that is key to performance and accountability.

We will then determine whether the risk stems from one of the following:

Criteria for

Determining if

Individual Performance

and Accountability

Challenges Merit

Designation as HighRisk Areas

• GAO audit reports, testimonies, studies, surveys, and ongoing work

• Annual audited financial statements for departments and agencies

• Inspectors general reports

• Agency (1) accountability reports, (2) strategic plans and annual

performance plans and reports, (3) program performance reviews,

(4) program evaluation and other internal reports, and (5) internal control

assessments, including those done under the Federal Managers’

Financial Integrity Act

• The President’s annual budget showing governmentwide performance

plans and management objectives

• Relevant reports by outside study panels and commissions

• Reports by congressional committees
Determining Performance and

Accountability Challenges and High Risks

Page 8 GAO-01-159SP Performance and Accountability Challenges and High Risks

• An inherent risk, which may arise when the nature of a program

creates susceptibility to fraud, waste, and abuse. A program involving

payments to claimants for services provided by third parties could

involve inherent risk, for example, due to the need for and difficulty of

verifying the accuracy of a large volume of claims.

• A systemic problem, which may arise when the programmatic,

management support, or financial systems, policies, and procedures

established by an agency to carry out a program are ineffective,

creating a material weakness.

Next, we will consider a number of qualitative and quantitative factors.

Additionally, before making a high-risk designation, we will consider the

corrective measures an agency may have planned or underway to resolve a

material weakness and the status and effectiveness of these actions.

These considerations, as well as criteria for determining governmentwide

high risks, are discussed further in the following sections. In all cases, the

ultimate determination of high risk will be made based on the independent

and objective judgment of GAO professionals.

We will consider the qualitative factors outlined in figure 3. These

qualitative factors are not meant to be all-inclusive. Other important

qualitative elements of risk may also be applicable to a given situation.

Considering Qualitative

and Quantitative Factors
Determining Performance and

Accountability Challenges and High Risks

Page 9 GAO-01-159SP Performance and Accountability Challenges and High Risks

Figure 3: Qualitative Factors Used in Determining High Risk

In addition to qualitative factors, we will also consider the exposure to

loss in monetary or other quantitative terms. At a minimum, $1 billion

must be at risk in areas such as

• the value of major assets (e.g., loans receivable) being impaired;

• revenue sources (e.g., taxes due) not being realized;

• major agency assets (e.g., inventory or property) being lost, stolen,

damaged, wasted, or underutilized;

• improper payments; and

Risk is seriously detrimental to

• Health or safety

• Service delivery

• National security

• National defense

• Economic growth

• Privacy or citizens’ rights

Risk could result in

• Significantly impaired service

• Program failure

• Significantly reduced effectiveness

• Significantly reduced efficiency

• Injury or loss of life

• Unreliable decision-making data

• Reduced confidence in government

• Unauthorized disclosure, manipulation, or misuse of sensitive

information, such as personal, financial management, or

programmatic data maintained in computerized systems
Determining Performance and

Accountability Challenges and High Risks

Page 10 GAO-01-159SP Performance and Accountability Challenges and High Risks

• contingencies or potential liabilities (e.g., environmental cleanup

costs).

The $1 billion threshold relates to that portion of a major program or

mission area that is at risk not to the financial aspects of the program or

mission area as a whole.

In making high-risk determinations, we will analyze the risks from

qualitative and quantitative standpoints. A program or function may be

highly vulnerable to risk arising from a qualitative factor, such as loss of

life, but may not necessarily meet the minimum quantitative dollar

threshold. Conversely, it is possible for an exposure to be significant

quantitatively, that is, placing $1 billion or more at risk, but not involve a

qualitative factor. In some instances, individual qualitative and

quantitative factors alone will not be high risk, but in combination, they

may call for a high-risk designation. Thus, we will consider the totality of

qualitative and quantitative factors in deciding whether a high-risk

designation is warranted.

Before assigning a high-risk designation, we will determine and assess the

effectiveness of an agency’s planned or ongoing corrective actions to

address a material weakness. In this regard, we will consider factors such

as those shown in figure 4.

Figure 4: Factors to Consider in Assessing the Effectiveness of Corrective

Measures to Resolve Material Weaknesses

Considering Corrective

Measures

• Whether the agency has demonstrated its commitment to resolving

the problem

• The extent of an agency’s progress to strengthen controls to

address the problem

• Whether the proposed remaining corrective action plans are

appropriate

• Whether effective solutions will be substantially completed near

term, as further discussed below

• Whether the solutions will resolve the root cause of the problem
Determining Performance and

Accountability Challenges and High Risks

Page 11 GAO-01-159SP Performance and Accountability Challenges and High Risks

For our high-risk initiative, near term is considered to be within the 2-year

period covered by the term of the Congress to which a high-risk update

report is addressed. In considering whether a corrective action will be

substantially completed in the near term, a high level of certainty must be

evident. Agencies will need to be able to demonstrate concrete results to

date, with a clear path toward addressing remaining problems. The final

determination will be based on GAO’s professional judgment.

The criteria we use to determine high-risk programs and functions,

including qualitative, quantitative, and corrective action factors, are

outlined in figure 5.

High-Risk Criteria

Summarized
Determining Performance and

Accountability Challenges and High Risks

Page 12 GAO-01-159SP Performance and Accountability Challenges and High Risks

Figure 5: Determining High-Risk Programs and Functions

a

See Standards for Internal Control in the Federal Government (GAO/AIMD-00-21.3.1,

November 1999).

Are corrective

measures

effective?

Yes

No

Yes

Quantitative factors Qualitative factors

• Significantly impaired

service?

• Program failure?

• Significantly reduced

effectiveness?

• Significantly reduced

efficiency?

• Injury or loss of life?

• Unreliable decision-making

data?

• Reduced confidence in

government?

• Sensitive information?

• Health and safety?

• Service delivery?

• National security?

• National defense?

• Economic growth?

• Privacy or citizens’ rights?

Is $1 billion or more at risk in

areas such as:

• The value of major

assets (e.g., loans

receivables) being

impaired?

• Revenue sources (e.g.,

taxes due) not being

realized?

• Major agency assets

(e.g., property) being

lost, stolen, damaged, or

wasted?

• Improper payments?

• Contingencies or

potential liabilities?

No

• Has the agency demonstrated a commitment to

resolve the material control weakness?

• Has substantial progress been made to

strengthen controls to address the risk?a

• Are corrective action plans appropriate?

• Will effective solutions be completed near term

and resolve the root causes of the problem?

Or

Yes

Yes

Is the exposure to

loss material in

quantitative terms?

Does the risk

stem from

Could the risk be

seriously

detrimental to

Could the risk

result in

Program or function

is high risk

Program or

function is not

high risk

• A program that has national significance?

• A management function that is key to

performance and accountability?

• An inherent vulnerability?

• A systemic problem?

Does the exposure

relate to

No

No

Based on professional judgment,

does the combination of

qualitative and quantitative factors

make the program or function

high risk?
Determining Performance and

Accountability Challenges and High Risks

Page 13 GAO-01-159SP Performance and Accountability Challenges and High Risks

In some instances, several agencies may share a common problem that

results in a high-risk situation. When this occurs, we will use the criteria in

figure 6 to determine whether a governmentwide high-risk designation is

warranted.

Figure 6: Criteria for Determining Governmentwide High Risks

If these criteria are met, we may designate the matter as a

governmentwide high-risk area.

For example, in 1997, GAO designated the Year 2000 (Y2K) computing

challenge as a governmentwide high risk, given its broad-based

implications and the potential serious disruption from a Y2K failure.

Subsequently, Y2K was successfully addressed by providing high-level

congressional and executive branch leadership, understanding the

importance of addressing the issues, providing standard guidance,

employing a constructive engagement approach, facilitating progress and

monitoring performance, and implementing fundamental improvements.

Another example is computer security, which is widely recognized around

the world by public and private organizations as a high-risk problem. In

September 1996, we reported that serious computer security weaknesses

were identified at 10 of the 15 largest federal agencies. In February 1997,

we designated information security as a governmentwide high-risk area

because of growing evidence indicating that controls over computerized

operations were not effective and compelling information that risks were

increasing.

Criteria for Determining

Governmentwide High

Risks

The material weakness must

• Be evident at multiple agencies

• Affect a significant portion of the government’s total budget or other

resources

• Stem from a deficiency that should be monitored and addressed

through individual agency actions as well as through Office of

Management and Budget initiatives, legislative action, and/or

congressional oversight
Determining Performance and

Accountability Challenges and High Risks

Page 14 GAO-01-159SP Performance and Accountability Challenges and High Risks

It may be possible for several agencies to have the same material

weakness and for it to rise to a high-risk level for some or all of those

agencies, but based on our criteria, it is not considered to be a

governmentwide high risk. In such cases, the weakness would be reported

as a performance and accountability challenge or a high-risk area for each

of the agencies it affects, as appropriate.

When legislative and agency actions, including those in response to our

recommendations, result in significant progress toward resolving a highrisk problem, we will remove the high-risk designation. In these cases, we

will continue to closely monitor the areas. If significant problems again

arise, we will consider reapplying the high-risk designation.

Since our high-risk initiative began in 1990, we have removed the high-risk

designation from several programs. For example, in 1991, we designated

the Bank Insurance Fund high risk because unprecedented numbers of

bank failures and insurance losses in the late 1980s and early 1990s had

depleted the fund’s reserves. In 1995, we removed this high-risk

designation because the fund’s risk had been reduced due to legislative

actions to rebuild the depleted deposit insurance fund and to require

significant governance, regulatory, and accounting reforms.

More recently, in 1999, we determined that the U.S. Customs Service had

made considerable progress in addressing its financial management

weaknesses. Given the significant improvement efforts, including those

related to assessing and collecting revenues, we removed our high-risk

designation from Customs’ financial management.

The criteria we use for determining whether to remove a high-risk

designation are shown in figure 7.

Criteria for Removing

High-Risk Designations
Determining Performance and

Accountability Challenges and High Risks

Page 15 GAO-01-159SP Performance and Accountability Challenges and High Risks

Figure 7: Criteria Agencies Must Meet Before High-Risk Designations Can

Be Removed

The criteria presented in this guidance document are intended to help us,

the Congress, and the executive branch give attention to identifying and

resolving performance and accountability challenges and high-risk

problems. Our Performance and Accountability Series and High-Risk

Updates are an important way to gauge progress in achieving this

objective.

Through these efforts, GAO is committed to helping the Congress improve

the economy, efficiency, and effectiveness of federal operations. Doing so

should also enhance the public’s respect for and confidence in the federal

government.

(924060)

Calling Attention to

Performance and

Accountability

Challenges and HighRisk Problems

• A demonstrated strong commitment and top leadership support to address

the risk(s)

• The capacity (that is, the people and other resources) to resolve the risk(s)

• A corrective action plan(s) that

• defines the root causes,

• identifies effective solutions, and

• provides for substantially completing corrective measures near term,

including but not limited to, steps necessary to implement solutions we

recommended

• A program instituted to monitor and independently validate the

effectiveness and sustainability of corrective measures

• The ability to demonstrate progress in having implemented corrective

measures
Ordering Information The first copy of each GAO report is free. Additional copies of 

reports are $2 each. A check or money order should be made out to 

the Superintendent of Documents. VISA and MasterCard credit 

cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are 

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

Orders by visiting:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders by phone:

(202) 512-6000

fax: (202) 512-6061

TDD (202) 512-2537

Each day, GAO issues a list of newly available reports and 

testimony. To receive facsimile copies of the daily list or any list 

from the past 30 days, please call (202) 512-6000 using a touchtone 

phone. A recorded menu will provide information on how to obtain 

these lists.

Orders by Internet:

For information on how to access GAO reports on the Internet, 

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web home page at: 

http://www.gao.gov

To Report Fraud, 

Waste, or Abuse in 

Federal Programs

Contact one:

• Web site: http://www.gao.gov/fraudnet/fraudnet.htm

• e-mail: fraudnet@gao.gov

• 1-800-424-5454 (automated answering system)

United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00