August 2000

Determining

Performance and

Accountability

Challenges and

High Risks

Exposure Draft

United States General Accounting Office

GAO

GAO/OCG-00-12

Page i GAO/OCG-00-12

1

2

Introduction 2

Criteria for Identifying and Assessing Major Program and Mission Areas 3

Criteria for Assessing Agency and Program Performance and Results by

Determining How Well They Are Managed 4

Criteria for Determining if Individual Performance and Accountability

Challenges Merit Designation as High-Risk Areas 5

Criteria for Removing High-Risk Designations 11

Figure 1: Major Management Functions Key to High-Performing

Organizations 4

Figure 2: Information Sources for Assessing Major Program and

Mission Areas and Management Functions 5

Figure 3: Qualitative Factors Used in Determining High Risk 6

Figure 4: Factors to Consider in Assessing the Effectiveness of

Corrective Measures to Resolve Material Control Weaknesses 8

Figure 5: Determining High-Risk Programs and Functions 9

Figure 6: Criteria for Determining Governmentwide High Risks 10

Figure 7: Criteria Agencies Must Meet Before High-Risk Designations

Can Be Removed 12

Contents

Preface

Determining

Performance and

Accountability

Challenges and

High Risks

Figures

Preface

Page 1 GAO/OCG-00-12

Since 1990, we have periodically reported on government programs and

functions that we have identified as “high risk” because of their greater

vulnerabilities to waste, fraud, abuse, and mismanagement. In January

1999, we issued our first Performance and Accountability Series, which

discussed the major issues 20 agencies faced in addressing performance

and accountability challenges. We plan to issue the next Performance and

Accountability and High-Risk Series in early 2001.

Over time, as high-risk areas have been corrected and other risks have

emerged, we have removed areas from the list and added new ones to

keep the Congress up to date on areas needing attention. In the January

1999 report series, we advised the Congress that because an increasing

amount of information is becoming available as a result of implementation

of various federal management reform initiatives, such as the Government

Performance and Results Act, the Chief Financial Officers Act and related

financial management legislation, and the Clinger-Cohen Act, we planned

to reassess the methodologies and criteria used to determine which

operations and functions should be included in the Performance and

Accountability Series and those which should be designated as high risk.

We have completed this comprehensive review and reassessment, and the

resulting criteria and process for determining performance and

accountability challenges and high risks are highlighted in this document.

It will be used by GAO auditors in making these determinations and by the

Congress and the executive branch agencies in understanding our basis

for reporting in these areas. As in the past, the ultimate determinations will

continue to involve the independent and objective judgment of GAO

professionals in applying the criteria.

This exposure draft was prepared under the direction of George H.

Stalcup, Associate Director, Accounting and Information Management

Division, and J. Christopher Mihm, Associate Director, General

Government Division. Questions or comments should be directed to

Mr. Stalcup before September 15, 2000, at (202) 512-9490 or

stalcupg.aimd@gao.gov.

David M. Walker

Comptroller General

of the United States

Preface
Determining Performance and Accountability

Challenges and High Risks

Page 2 GAO/OCG-00-12

We have developed—and will continue to refine—criteria to use as a basis

in forming judgments on

• identifying and assessing the performance of the federal government’s

major program and mission areas,

• assessing agencies’ management functions to determine how they

contributed to the agencies’ ability to ensure accountability and

achieve results,

• determining if individual performance and accountability challenges

merit designation as high-risk areas as well as determining

governmentwide high risks, and

• removing high-risk designations.

In determining performance and accountability challenges and assessing

whether they rise to the level of high-risk exposure, we consider the

requirements of program-specific legislation. We also consider the

requirements of management reform legislation, such as the Chief

Financial Officers Act of 1990, the Government Performance and Results

Act of 1993, the Government Management Reform Act of 1994, the Federal

Financial Management Improvement Act of 1996, and the Clinger-Cohen

Act of 1996.

Also, the Standards for Internal Control in the Federal Government1

provide a foundation for assessing the effects and underlying causes of

control weaknesses for major program and mission areas and key

management functions. We issued these standards pursuant to

requirements of the Federal Managers’ Financial Integrity Act. Internal

control is a major part of managing an organization. It comprises the plans,

methods, and procedures used to meet missions, goals, and objectives and,

in doing so, supports performance-based management. Internal control

also serves as the first line of defense in safeguarding assets and

preventing and detecting errors and fraud. In short, internal control, which

is synonymous with management control, helps government program

managers achieve desired results through effective stewardship of public

resources. The standards provide the overall framework for establishing

and maintaining internal control and for identifying and addressing major

performance and accountability challenges and high-risk areas.

1

GAO/AIMD-00-21.3.1, November 1999.

Determining Performance and Accountability

Challenges and High Risks

Introduction
Determining Performance and Accountability

Challenges and High Risks

Page 3 GAO/OCG-00-12

For each major agency, we will identify major program and mission areas

that will form the primary basis for reporting in the performance and

accountability series and that

• are at the center of congressional and executive branch attention,

• have high public interest and/or large-dollar outlays,

• figure prominently in agencies’ strategic plans and annual performance

plans and reports, or

• have known performance and accountability or high-risk issues.

These selections will be made in consultation with the Congress and

coordinated with the Office of Management and Budget and the top

leadership of the major federal agencies. Thus, the performance and

accountability series may not include all of an agency’s program and

mission areas. Rather, using the criteria above, the series will focus

principally on those program and mission areas that are among the most

important for each agency. Program and mission areas that in the past

were designated as high risk or had performance and accountability

concerns will be carefully scrutinized to determine if the risks and

concerns have been resolved.

We will use agencies’ strategic plans, annual performance plans and

reports, accountability reports, and audited financial statements to identify

the key goals, strategies, performance measures, and reported

performance for each of the major program and mission areas. This

information, supplemented by relevant GAO products, inspectors general

reports, and other independent analyses, will enable us to develop a

profile of the actual performance for each of the program and mission

areas.

Criteria for

Identifying and

Assessing Major

Program and Mission

Areas
Determining Performance and Accountability

Challenges and High Risks

Page 4 GAO/OCG-00-12

Based on our experience in examining a wide range of government

programs, we have found that effective performance of the management

functions shown in figure 1 are key to creating and sustaining highperforming organizations.

Figure 1: Major Management Functions Key to High-Performing Organizations

We will assess the results that agencies’ major program and mission areas

achieve and how well the management functions key to high performance

are being carried out. For those program and mission areas where

performance can be improved substantially, we will identify the root

causes of the current performance levels and help pinpoint improvement

opportunities. Also, our assessments will help to identify best practices

that show the specific contributions better management makes to high

performance.

In addition, in assessing major program and mission areas and

management functions and in making high-risk determinations, we will

draw from information available through a number of sources, the primary

source being recent GAO audit reports and studies flowing from our

strategic plan and legislatively mandated efforts. The sources of

information we will use are shown in figure 2.

Criteria for Assessing

Agency and Program

Performance and

Results by

Determining How

Well They Are

Managed

• Strategic planning

• Budget formulation and execution

• Organizational alignment and control

• Performance measurement

• Human capital strategies

• Financial management

• Information technology

• Acquisition
Determining Performance and Accountability

Challenges and High Risks

Page 5 GAO/OCG-00-12

Figure 2: Information Sources for Assessing Major Program and Mission Areas and

Management Functions

To determine whether an individual performance and accountability

challenge merits a high-risk designation, we will first determine whether it

involves

• a program or mission area having national significance or

• a management function that is key to performance and accountability.

We will then determine whether the risk stems from one of the following:

• An inherent risk, which may arise when the nature of a program

creates susceptibility to fraud, waste, and abuse. A program involving

payments to claimants for services provided to third parties could

involve inherent risk, for example, due to the need for and difficulty of

verifying the accuracy of a large volume of claims.

• A systemic problem, which may arise when the programmatic,

management support, or financial systems, policies, and procedures

established by an agency to carry out a program are ineffective,

creating a material weakness.

Criteria for

Determining if

Individual

Performance and

Accountability

Challenges Merit

Designation as HighRisk Areas

• GAO audit reports, testimonies, studies, surveys, and

ongoing work

• Annual audited financial statements for departments and

agencies

• Inspectors general audit reports

• Agency (1) accountability reports, (2) strategic plans and

annual performance plans and reports, (3) program

performance reviews, and (4) internal control assessments,

including those done under the Federal Managers’ Financial

Integrity Act

• Reports by outside study panels and commissions

• Reports by congressional committees
Determining Performance and Accountability

Challenges and High Risks

Page 6 GAO/OCG-00-12

Next, we will consider a number of qualitative and quantitative factors.

Additionally, before making a high-risk designation, we will consider the

corrective measures an agency may have planned or underway to resolve a

material control weakness and the status and effectiveness of these

actions. These considerations, as well as criteria for determining

governmentwide high risks, are discussed further in the following

sections. In all cases, the ultimate determination of high risk will be made

based on the independent and objective judgment of GAO professionals.

We will consider the qualitative factors outlined in figure 3. These

qualitative factors are not meant to be all inclusive. Other important

qualitative elements of risk may also be applicable to a given situation.

Figure 3: Qualitative Factors Used in Determining High Risk

Considering Qualitative

and Quantitative Factors

Risk is seriously detrimental to

• Public health or safety

• Service delivery

• National security

• National defense

• Economic growth

• Privacy or citizens’ rights

Risk could result in

• Significantly impaired service

• Program failure

• Significantly reduced effectiveness

• Significantly reduced efficiency

• Public injury or loss of life

• Unreliable decision-making data

• Reduced confidence in government

• Unauthorized disclosure, manipulation, or misuse of sensitive

information, such as personal, financial management, or

programmatic data maintained in computerized systems
Determining Performance and Accountability

Challenges and High Risks

Page 7 GAO/OCG-00-12

In addition to qualitative factors, we will also consider the exposure to

loss in monetary or other quantitative terms. At a minimum, $1 billion

must be at risk in areas such as

• the value of major assets (e.g., loans receivable) being impaired;

• revenue sources (e.g., taxes due) not being realized;

• major agency assets (e.g., inventory or property) being lost, stolen,

damaged, wasted, or underutilized; and

• improper payments.

In making high-risk determinations, we will analyze the risks from

qualitative and quantitative standpoints. A program or function may be

highly vulnerable to risk arising from a qualitative factor, such as loss of

life, but may not necessarily meet the minimum quantitative dollar

threshold. Conversely, it is possible for an exposure to be significant

quantitatively, that is, placing $1 billion or more at risk, but not involve a

qualitative factor. In some instances, individual qualitative and quantitative

factors alone will not be high risk, but in combination, they may call for a

high-risk designation. Thus, we will consider the totality of qualitative and

quantitative factors in deciding whether a high-risk designation is

warranted.

Before assigning a high-risk designation, we will determine and assess the

effectiveness of an agency’s planned or ongoing corrective actions to

address a material control weakness. In this regard, we will consider

factors such as those shown in figure 4.

Considering Corrective

Measures
Determining Performance and Accountability

Challenges and High Risks

Page 8 GAO/OCG-00-12

Figure 4: Factors to Consider in Assessing the Effectiveness of Corrective

Measures to Resolve Material Control Weaknesses

For our high-risk initiative, near term is considered to be within the 2-year

period covered by the term of the Congress to which a high-risk update

report is addressed. In considering whether a corrective action will be

substantially completed in the near term, a high level of certainty must be

evident. Agencies will need to be able to demonstrate concrete results to

date, with a clear path toward addressing remaining problems. The final

determination will be based on our professional judgment.

The criteria we use to determine high-risk programs and functions,

including qualitative, quantitative, and corrective action factors, are

outlined in figure 5.

High-Risk Criteria

Summarized

• Whether the agency has demonstrated its commitment to

resolving the problem

• The extent of an agency’s progress to strengthen controls to

address the problem

• Whether the proposed remaining corrective action plans are

appropriate

• Whether effective solutions will be substantially completed

near term, as further discussed below

• Whether the solutions will get to the root cause of the

problem
Determining Performance and Accountability

Challenges and High Risks

Page 9 GAO/OCG-00-12

Figure 5: Determining High-Risk Programs and Functions

Yes

No

Yes

Quantitative factors Qualitative factors

• Significantly impaired

service?

• Program failure?

• Significantly reduced

effectiveness?

• Significantly reduced

efficiency?

• Public injury or loss of life?

• Unreliable decision-making

data?

• Reduced confidence in

government?

• Public health and safety?

• Service delivery?

• National security?

• National defense?

• Economic growth?

• Privacy or citizens’ rights?

• Sensitive information?

No

• Has the agency demonstrated a commitment to

resolve the material control weakness?

• Has substantial progress been made to

strengthen controls to address the risk?1

• Are corrective action plans appropriate?

• Will effective solutions be completed near term

and get to the root causes of the problem?

Or

Yes

Yes

Is the exposure to

loss material in

quantitative terms?

Does the risk

stem from

Could the risk be

seriously

detrimental to

Could the risk

result in

Are corrective

measures

effective?

Program or function

is high risk

Program or

function is not

high risk

• A program that has national significance?

• A management function that is key to

performance and accountability?

• An inherent vulnerability?

• A systemic problem?

Does the exposure

relate to

No

No

Based on professional judgment,

does the combination of

qualitative and quantitative factors

make the program or function

high risk?

1

See Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1, November 1999

Is $1 billion or more at risk in

areas such as:

• The value of major assets

(e.g., loans receivables)

being impaired?

• Revenue sources (e.g.,

taxes due) not being

realized?

• Major agency assets (e.g.,

inventory, property) being

lost, stolen, damaged, or

wasted?
Determining Performance and Accountability

Challenges and High Risks

Page 10 GAO/OCG-00-12

In some instances, several agencies may share a common problem that

results in a high-risk situation. When this occurs, we will use the criteria in

figure 6 to determine whether a governmentwide high-risk designation is

warranted.

Figure 6: Criteria for Determining Governmentwide High Risks

If these criteria are met, we may designate the matter as a

governmentwide high-risk area.

For example, in 1997, GAO designated the Year 2000 (Y2K) computing

challenge as a governmentwide high risk, given its broad-based

implications and the potential serious disruption from a Y2K failure.

Subsequently, Y2K was successfully addressed by providing high-level

congressional and executive branch leadership, understanding the

importance of addressing the issues, providing standard guidance,

employing a constructive engagement approach, facilitating progress and

monitoring performance, and implementing fundamental improvements.

Another example is computer security, which is widely recognized around

the world by public and private organizations as a high-risk problem. In

September 1996, we reported that serious computer security weaknesses

were identified at 10 of the 15 largest federal agencies. In February 1997,

we designated information security as a governmentwide high-risk area

because of growing evidence indicating that controls over computerized

operations were not effective and compelling information that risks were

increasing.

Governmentwide High

Risks

The material weakness must

• Be evident at multiple agencies

• Affect a significant portion of the government’s total budget or

other resources

• Stem from a deficiency that should be monitored and

addressed through individual agency actions as well as

through Office of Management and Budget initiatives,

legislative action, and/or congressional oversight
Determining Performance and Accountability

Challenges and High Risks

Page 11 GAO/OCG-00-12

It may be possible for several agencies to have the same material

weakness and for it to rise to a high-risk level for some or all of those

agencies, but based on our criteria, it may not be considered a

governmentwide high risk. In such cases the weakness would be

designated and reported as a high-risk area for the agencies it affects, as

appropriate.

When legislative and agency actions, including those in response to our

recommendations, result in significant progress toward resolving a highrisk problem, we will remove the high-risk designation. In these cases, we

will continue to closely monitor the areas. If significant problems again

arise, we will consider reapplying the high-risk designation.

Since our high-risk initiative began in 1990, we have removed the high-risk

designation from several programs. For example, in 1991, we designated

the Bank Insurance Fund high risk because unprecedented numbers of

bank failures and insurance losses in the late 1980s and early 1990s had

depleted the fund’s reserves. In 1995, we removed this high-risk

designation because the fund’s risk had been reduced due to legislative

actions to rebuild the depleted deposit insurance fund and to require

significant governance, regulatory, and accounting reforms.

More recently, in 1999, we determined that the U.S. Customs Service had

made considerable progress in addressing its financial management

weaknesses. Given the significant improvement efforts, including those

related to assessing and collecting revenues, we removed our high-risk

designation from Customs’ financial management.

The criteria we use for determining whether to remove a high-risk

designation are shown in figure 7.

Criteria for Removing

High-Risk

Designations
Determining Performance and Accountability

Challenges and High Risks

Page 12 GAO/OCG-00-12

Figure 7: Criteria Agencies Must Meet Before High-Risk Designations Can Be

Removed

• A demonstrated strong commitment and top leadership

support to address the risk(s)

• The capacity (that is, the people and other resources) to

resolve the risk(s)

• A corrective action plan(s) that

• defines the root causes,

• identifies cost-effective solutions, and

• provides for substantially completing corrective measures

near term, including steps necessary to implement

solutions we recommended

• A program instituted to monitor and independently validate

the effectiveness of corrective measures

• The ability to demonstrate progress in implementing

corrective measures

(800449)