a

GAO

United States Government Accountability Office

GAO Workshop

July 2005 ECONOMIC 

PERFORMANCE

Highlights of a 

Workshop on 

Economic 

Performance 

Measures

GAO-05-796SP
What Participants Said

United States Government Accountability Office

Why GAO Convened This 

Workshop

Highlights

Accountability Integrity Reliability

www.gao.gov/cgi-bin/getrpt?GAO-05-796SP.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Nancy R. 

Kingsbury at (202) 512-2700 or 

kingsburyn@gao.gov. 

Highlights of GAO-05-796SP

July 2005

ECONOMIC PERFORMANCE 

Highlights of a Workshop on Economic 

Performance Measures 

Workshop participants identified a number of issues regarding the use of 

economic performance analysis—benefit-cost or cost-effectiveness 

analysis—in evaluating federal program performance. They generally said 

the following: 

• The quality of the economic performance assessment of federal 

programs has improved but is still highly variable and not sufficient to 

adequately inform decision makers. 

• The gaps in applying economic performance measures are that they are 

not widely used, mechanisms for revisiting a regulation or program are 

lacking, retrospective analyses are often not done, and homeland 

security regulations present additional challenges and typically do not 

include economic analysis. 

• Barriers include agencies’ lack of resources and only limited demand 

from decision makers for benefit-cost analysis. In addition, some 

participants stated that organizational barriers called stovepipes or silos 

hinder communication. 

• Some analytical issues that affect the application of economic 

performance measures are limited guidance on assessing unquantifiable 

benefits, equity, and distributional effects of federal actions; lack of 

agreement on some values for key assumptions; and lack of guidance on 

tools that do not monetize outcomes, such as multiobjective analysis. 

• Opportunities to expand the use of measures include evaluation of 

existing programs retrospectively and application to homeland security 

issues. 

• Ways to improve the general economic principles and guidance that 

economic performance analysis is based upon include developing a 

minimum set of principles and abbreviated guidelines for economic 

performance analysis, developing one-page summaries and scorecards of

analysis results, standardizing some key values for assumptions, and 

creating an independent and flexible organization to provide guidance 

and develop standards. 

Improving the economy and 

efficiency of federal programs has 

long been a key objective of the 

Government Accountability Office 

(GAO). To this end, GAO held a 

workshop on December 17, 2004, 

to discuss the use of economic 

analysis, such as benefit cost or 

cost effectiveness, for helping to 

measure the performance of 

federal programs. The workshop’s 

purpose was to 

• discuss the present state of 

economic performance 

measures and identify gaps in 

their application and the 

barriers and analytical issues 

that limit their use in helping 

assess the performance of 

federal programs and 

• identify opportunities for the 

federal government and 

professional and academic 

institutions to improve (1) the 

use of economic performance 

measures for evaluating 

federal programs and 

(2) the general economic 

principles and guidance on 

which economic performance 

analysis is based. 
Page i GAO-05-796SP Economic Performance Measures

Contents

Letter 1

The Workshop’s Objectives 2

Summary of Workshop Discussion 4

Participants’ Comments 6

Workshop Discussion 8

Background 8

The State of Economic Performance Evaluation, Including Gaps, 

Barriers, and Analytical Issues 9

The Extension of Economic Performance Measures for Evaluating 

Federal Programs 15

Improving General Economic Principles and Guidance 16

Appendixes

Appendix I: Economic Performance Workshop Participants: December 17, 

2004 20

Appendix II: Economic Performance Assessment: Uses, Principles, and 

Opportunities 21

Tables Table 1: The Use of Economic Performance Measures for 

Prospective Assessment of Federal Programs 28

Table 2: The Use of Economic Performance Measures for 

Retrospective Assessment of Federal Programs 28

Table 3: Summary of Three Programs’ Net Benefits 32

Table 4: Evaluating Economic Performance Assessments with a 

Scorecard 36

Table 5: The Hierarchy of Generally Accepted Accounting 

Principles 37

Table 6: Consistent Reporting Format: GAO’s WIC Assessment 39

Table 7: Consistent Reporting Format: GAO’s USDA Cotton 

Program Assessment 41

Table 8: Consistent Reporting Format: GAO’s OSHA Scaffold 

Assessment 44

Table 9: A Scorecard for Evaluating Economic Performance 

Assessments 47

Table 10: Prospective and Retrospective Assessments of OSHA’s 

Scaffold Rule Compared 50
Contents

Page ii GAO-05-796SP Economic Performance Measures

Abbreviations

AICPA American Institute of Certified Public Accountants

APB Accounting Principles Board

CDC Centers for Disease Control and Prevention

DOE Department of Energy

DOL Department of Labor

DOT Department of Transportation

EPA Environmental Protection Agency

FASAB Financial Accounting Standards Advisory Board

FASB Financial Accounting Standards Board

GASB Governmental Accounting Standards Board

OMB Office of Management and Budget

OSHA Occupational Safety and Health Administration

PART Program Assessment Rating Tool

USDA U.S. Department of Agriculture

WIC Special Supplemental Nutrition Program for Women, Infants, 

and Children

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately.
Page 1 GAO-05-796SP Economic Performance Measures

United States Government Accountability Office

Washington, D.C. 20548

A

July 2005 Letr

Improving the economy and efficiency of federal programs has long been a 

key objective of the U.S. Government Accountability Office (GAO). A focus 

on auditing the performance of government programs has complemented 

the agency’s focus on accounting for decades. In a recent report, GAO 

highlighted the importance of a fundamental review of federal programs 

and policies in addressing the nation’s long-term fiscal imbalance and in 

ensuring that the federal government’s programs and priorities meet 

current and future challenges.1

 In this regard, measuring the economic 

performance of federal programs, such as the extent to which program 

benefits exceed costs (net benefits) or are achieved at least cost (cost 

effectiveness), could be a useful way to assess, in conjunction with other 

measures, the extent to which federal programs are meeting the nation’s 

priorities. 

The economic performance of some federal actions is presently assessed 

prospectively, through an Office of Management and Budget (OMB) review 

of proposed capital investments and regulations. However, few federal 

actions are monitored for their economic performance retrospectively. In 

addition, reviews by GAO have found that economic assessments that 

analyze regulations prospectively are often incomplete and inconsistent 

with general economic principles.2

 Moreover, the assessments are often not 

useful for comparisons across the government, because they are often 

based on different assumptions for the same key economic variables. 

Furthermore, new areas of federal action, such as homeland security, 

present additional challenges because of the difficulty of assessing 

uncertainty and risk, such as those associated with terrorist activities.

1

GAO, 21st Century Challenges: Reexamining the Base of the Federal Government, 

GAO-05-325SP (Washington, D.C.: Feb. 2005).

2

See GAO, Regulatory Reform: Agencies Could Improve Development, Documentation, 

and Clarity of Regulatory Economic Analyses, GAO/RCED-98-142 (Washington, D.C.: May 

26, 1998), and Clean Air Act: Observations on EPA’s Cost-Benefit Analysis of Its Mercury 

Control Options, GAO-05-252 (Washington, D.C.: Feb. 28, 2005). 
Page 2 GAO-05-796SP Economic Performance Measures

The Government Performance and Results Act of 1993 (Results Act) 

requires, among other things, agencies to establish budgetary performance 

goals and to identify measures to determine whether their programs are 

meeting those goals. Although economic performance measures are 

consistent with the act, they are generally not used. For example, GAO 

found that few measures under the act clearly linked program costs to the 

achievement of program goals or objectives.3

 In addition, although federal 

agencies use OMB’s Program Assessment Rating Tool (PART) every year to 

assess the performance of their programs, almost 50 percent of the 234 

programs assessed for fiscal year 2004 received a rating of “results not 

demonstrated.” OMB had determined that program performance 

information, performance measures, or both were insufficient or 

inadequate.4

 In particular, OMB has indicated a preference for the use of 

more economic performance measures, including net benefits, in the PART 

process.

Accepted methods for estimating economic performance measures are 

based on general economic principles and guidelines derived from 

academic textbooks and research results presented in journal articles. 

Several federal agencies, such as the U.S. Department of Transportation 

and Environmental Protection Agency, have incorporated the principles 

and guidelines into guidance for agency economists to use in assessing 

economic performance. Unlike in some other professions, such as 

accounting, these principles and guidelines were not identified or created 

by a standard-setting authority representing the entire profession.

The Workshop’s 

Objectives

GAO convened a workshop on December 17, 2004, to discuss the use of 

economic analysis, such as cost benefit or cost effectiveness, for helping to 

measure the performance of federal programs. The workshop’s objectives 

were to

• discuss the present state of economic performance measures and 

identify the gaps in their application and the barriers and analytical 

3

GAO, Results-Oriented Government: GPRA Has Established a Solid Foundation for 

Achieving Greater Results, GAO-04-38, (Washington, D.C.: Mar. 10, 2004). 

4

GAO, Performance Budgeting: Observations on the Use of OMB’s Program Assessment 

Rating Tool for the Fiscal Year 2004 Budget, GAO-04-174 (Washington, D.C.: Jan. 30, 2004).
Page 3 GAO-05-796SP Economic Performance Measures

issues that limit their use in helping assess the performance of federal 

programs and

• identify opportunities for the federal government and professional and 

academic institutions to improve (1) the use of economic performance 

measures for evaluating federal programs and (2) the general economic 

principles and guidance on which economic performance analysis is 

based.

A summary of the workshop discussion is presented in the next section. 

The participants are listed in appendix I. A discussion paper prepared for 

the workshop by a number of GAO staff appears in appendix II.

We selected workshop participants from government and academia based 

on their professional publications about economic performance measures, 

their role in developing economic guidance, and the extent to which they 

have used economic performance measures in their agencies. In addition, 

four participants were asked to make presentations to the group on areas 

relating to the workshop objectives, including the use of economic 

performance measures for oversight in the executive branch, limitations of 

economic performance measures, the quality of agencies’ economic 

regulatory assessments, and the use of standard-setting authorities to 

develop principles and standards of guidance for the accounting 

profession. GAO provided the participants with a discussion paper for 

background information before the workshop began.

After the workshop was conducted, we used content analysis to 

systematically analyze a transcript of the workshop discussion and to 

identify participants’ views on key questions, as well as the key themes that 

developed from the discussion. As agreed by the participants, the purpose 

of the discussion was to engage in an open, not-for-attribution dialogue. As 

a result, this report is a synthesis of the key themes from the workshop, not 

a verbatim presentation of the participants’ statements. In addition, it does 

not necessarily represent the views of any individual participant. We did 

not verify the participants’ statements, and the views expressed do not 

necessarily represent the views of GAO. 

We would like to thank the workshop’s participants for taking the time to 

share their knowledge and providing their insight and perspective in an 

effort to improve government oversight accountability and performance.
Page 4 GAO-05-796SP Economic Performance Measures

Summary of Workshop 

Discussion

Although the workshop participants said that they recognized that the 

quality of federal agencies’ economic assessments of regulations and 

programs has generally improved over the years, they said that they 

believed that the assessments’ quality is still highly variable. Assessments 

vary in how they are performed and in the measures they use. The 

participants also said that many economic assessments conducted to 

support agency decisions are insufficient to inform decision makers 

whether proposed regulations and programs are achieving goals cost 

effectively or generating net benefits for the nation.

Participants identified gaps in the application of economic performance 

measures. First, economic performance measures are often not widely 

used for programs in the federal government. Second, while some agencies 

have done retrospective economic performance assessments, participants 

said that in general federal agencies often do not assess the performance of 

regulations or existing programs retrospectively, even though this 

information could be useful in managing programs. Third, once a program 

has been enacted, mechanisms often do not exist for determining whether 

actual performance is similar to predicted effectiveness. Fourth, 

regulations related to homeland security present additional challenges 

because of the difficulties associated with quantifying the probability of a 

terrorist attack and the benefits that might be generated as a result of 

proposals related to them. In addition, proposed regulations involving 

these issues generally do not measure their expected economic 

performance.

Some participants stated that economic performance measures are not 

widely used because of several barriers. They cited as an example a lack of 

demand from many decision makers to know the full costs of federal 

programs. In addition, participants pointed out that agencies often lack 

resources in terms of both funds and time for assessing the economic 

performance of programs already in place. Organizational stovepipes or 

silos that limit communication—between federal agencies and between the 

agencies and the economics profession—about how to conduct 

comprehensive and useful economic assessments were identified as 

another barrier.

The participants generally agreed that several analytical issues should be 

resolved to improve the consistency and credibility of economic 

performance measures. For example, they cited insufficient guidance for 

agencies to appropriately include benefits or costs of federal actions that 

Limited demand for benefit-cost analysis 

from decision makers.

Little provision of resources to agencies to 

assess existing programs.

Existence of organizational "stovepipes."

·

·

·

Economic performance measures are not 

widely used.

Performance of regulations or programs is 

often not assessed retrospectively.

Mechanisms for revisiting regulations or 

programs are lacking.

Homeland security regulations present 

challenges and typically do not include 

economic analysis.

·

·

·

·

· The quality of economic performance 

 assessments has improved but is still 

 generally not sufficient.
Page 5 GAO-05-796SP Economic Performance Measures

cannot be quantified or monetized or the effects of actions on different 

income, racial, or other population groups. In addition, lack of agreement 

and guidance regarding the most appropriate set of values to use for key 

economic assumptions, such as the benefit associated with a reduced risk 

of mortality, hinders the consistent application of economic performance 

measures across government agencies. Participants also cited lack of 

guidance for tools such as those used for multiobjective analysis of such 

things as the benefits of agency outcomes without putting the benefits into 

monetary terms.

There was general agreement that the use of economic performance 

measures should be expanded, especially for retrospective analysis of 

existing programs. Besides providing information on the performance of 

existing programs, retrospective analysis could provide lessons on how to 

improve prospective analysis of proposed programs. Along these lines, 

analyzing economic performance could be one way to evaluate agencies’ 

performance through budget processes. Some participants also indicated 

that economic performance measures could be used to evaluate the risk 

and uncertainty associated with homeland security programs and 

regulations.

The participants identified opportunities for the federal government and 

professional and academic institutions to improve economic principles and 

guidance that could ultimately enhance the use of economic performance 

measures for evaluating federal regulations and programs. For example, it 

was suggested that a minimum set of general economic principles and 

abbreviated guidelines might help agencies overcome barriers in assessing 

the economic performance of their regulations and programs. In addition, 

the analytical challenges of quantifying the risk and uncertainties 

associated with homeland security issues require more extensive guidance 

in order to deal with the development of regulations. Scorecards that rate 

the quality of economic assessments and one-page summaries of key 

results, as well as expert review of the agencies’ economic assessments, 

were cited, by some, as tools for improving quality and credibility. Some 

participants indicated that standardizing some key values for economic 

assumptions could help improve quality throughout the government.

The participants identified a number of existing organizations that might 

more formally develop and improve principles and guidance for economic 

performance analysis. For example, several participants expressed interest 

in the accounting profession’s use of standard-setting authorities to 

develop comprehensive principles, standards, and guidance to ensure the 

Limited guidance on assessing 

unquantifiable benefits, equity, and 

distributional effects.

Lack of agreement on some key values.

Lack of guidance on tools that do not 

monetize outcomes, such as 

multiobjective analysis.

·

·

·

Expand use of analysis, particularly for 

retrospective evaluation of existing 

programs.

Use economic performance measures to 

inform federal budgets and the risk and 

benefits of Homeland Security programs.

·

·

Develop a minimum set of principles and 

abbreviated guidelines.

Develop guidance for dealing with 

Homeland Security issues.

Develop one-page summaries and 

scorecards of economic performance 

analysis; use expert review to provide 

procedures and strategies.

Standardize some key values.

Develop an independent and flexible 

organization to provide guidance and 

develop standards.

·

·

·

·

·
Page 6 GAO-05-796SP Economic Performance Measures

quality, consistency, and credibility of accounting and financial reporting. 

Some participants indicated, however, that professional economics 

institutions are not designed to govern or monitor the application of 

economics.

The participants identified some other organizational formats that could be 

used, such as those that the National Academies and National Bureau of 

Economic Research use. For example, the National Academies convene 

expert panels, workshops, and roundtables to examine science and 

technology issues. These formats might help resolve analytical issues and 

improve principles and guidance. Alternatively, it was generally agreed that 

creating a new organization, if it were organizationally independent and 

flexible enough, might help address a variety of significant issues.

Participants’ 

Comments

We provided a draft of this report to the workshop participants for their 

review and comment. Seven of fourteen participants external to GAO 

chose to provide comments. They generally agreed with the summary of 

the workshop discussion and stated that it was fair and complete. In 

addition, they provided clarifying points and technical comments, which 

we incorporated as appropriate.
Page 7 GAO-05-796SP Economic Performance Measures

If you would like additional information on the workshop or this document, 

please call (202) 512-2700. The workshop was planned and this report was 

prepared under the direction of Scott Farrow, Chief Economist. Other 

major contributors were Carol Bray, Alice Feldesman, Tim Guinane, 

Luanne Moy, and Penny Pickett.

Nancy R. Kingsbury, Managing Director

Applied Research and Methods 

Scott Farrow, Chief Economist

Applied Research and Methods
Page 8 GAO-05-796SP Economic Performance Measures

Workshop Discussion

Background Economists typically use economic assessments to measure the 

performance of federal regulations and programs. The assessments 

estimate the net benefits or cost effectiveness of federal actions on a 

nationwide basis.1

 Economic performance assessment differs from a 

straightforward financial appraisal in that all gains (benefits) and losses 

(costs) that accrue to society in general (not just to the government) as a 

result of the program are to be counted.

Although other professions, such as accounting, rely on standard-setting 

authorities to develop principles and guidance, economic performance 

measures are based on principles and guidance that have been developed 

in the economic literature over more than 75 years. This literature includes 

academic textbooks and research presented in journal articles as well as 

federal agency guidance. The agency guidance includes, among other 

things, Office of Management and Budget (OMB) Circulars A-4 and A-11, 

Part 7, Section 300, and A-94.2

Circular A-4 is designed to assist analysts in the regulatory agencies in 

estimating the benefits and costs of proposed regulatory actions. Circular 

A-11, Part 7, Section 300, establishes policy for the planning, budgeting, 

acquisition, and management of federal capital assets. It provides guidance 

for budgetary analysis of alternatives for making federal capital 

investments. Circular A-94 provides additional guidelines and appropriate 

discount rates for benefit-cost and cost-effectiveness analysis. While OMB’s 

guidance for economic performance is useful both for producing economic 

assessments and auditing performance, it is distinctly less standardized 

than accounting guidance provided to accountants and auditors.

In addition, in some instances agencies use a multiobjective method of 

analysis to assess programs. In this type of analysis, program impacts are 

1

Cost-effectiveness analysis measures the least costly way of achieving an objective. It is 

typically used when the outcomes can be quantified but not monetized.

2

See OMB Circular No. A-4, 68 Fed. Reg. 58366 (Oct. 9, 2003); OMB Circular No. A-11, 

Preparation, Submission, and Execution of the Budget (May 17, 2005); OMB Circular No. 

A-94, 57 Fed. Reg. 53519 (Nov. 10, 1992). Under the Unfunded Mandates Reform Act of 1995, 

agencies are required to prepare a qualitative and quantitative assessment of the anticipated 

costs and benefits before issuing a regulation that may result in annual expenditures by 

state, local, and tribal governments or by the private sector of $100 million annually. Under 

Executive Order 12866 and OMB Circular A-11, certain federal agencies are required to 

consider the benefits and costs of proposed regulatory actions and capital expenditures.
Page 9 GAO-05-796SP Economic Performance Measures

not put into monetary terms. Instead, identified impacts are given a 

weighted ranking that allows decision makers to evaluate federal actions 

on the basis of their place in the ranked scale. The role of this analysis is 

somewhat uncertain in the context of economic performance 

measurement.

The State of Economic 

Performance 

Evaluation, Including 

Gaps, Barriers, and 

Analytical Issues

The workshop participants generally agreed that while economic 

performance analyses that assess government programs have improved 

somewhat, their quality is highly variable. In addition, the analyses often 

miss key information needed to inform decisions makers about whether 

the government actions that are proposed can be expected to be cost 

effective or generate positive net benefits.

One participant said that a comparison of present economic assessments, 

using estimates of the program’s cost per life saved, with assessments 

completed in the early 1980s had found a discernible improvement in 

analysis. Another participant agreed on the signs of increased 

sophistication in the types of measure used, such as the discount rate or 

approach to using discounting. In addition, there has been some diffusion 

of knowledge within the agencies from the economics literature about 

using a statistical value of life.

Despite these improvements, however, participants said that the quality of 

analysis is still highly variable—some analyses are quite good, others not. 

One participant said that there is incredible variability across agencies in 

how economic performance assessments are performed—whether an 

agency follows a fairly standard cost-benefit analytic framework or 

something else.

Another participant pointed out that many economic performance analyses 

are still not sufficient because they miss key information. For example, one 

participant said that the majority of economic performance evaluations 

reviewed did not discuss net benefits or analysis of alternatives to 

proposed regulatory options. In addition, only a fairly small number of 

analyses dealt well with the uncertainty associated with estimated benefits 

and costs; only a few provided both point estimates and a range of total 

costs or benefits.

These gaps limit the evaluations’ usefulness to decision makers. Without 

more information about the uncertainty associated with the estimates, the 

assessments may not be sufficient to inform decision makers about 
Page 10 GAO-05-796SP Economic Performance Measures

whether proposed regulations and programs would be likely to achieve 

their goals cost effectively or generate positive net benefits for the nation.

Gaps in the Application of 

Economic Performance 

Assessments

The participants identified some gaps in the application of economic 

performance analysis: (1) economic performance measures are generally 

not widely used for programs in the federal government, (2) retrospective 

analyses of programs are often not being done, (3) mechanisms for 

revisiting a program or regulation are often lacking, and (4) regulations 

involving homeland security issues present additional challenges and often 

do not include an economic assessment of the benefits and costs of 

proposed regulations and programs.3

The participants said that while some agencies have used economic 

performance measures, in general they were not widely used in the federal 

government. For example, one participant pointed out that while there has 

been progress on the quality of economic assessments being produced, 

there is still the issue of whether assessments are being done at all.

Participants observed that in some cases programs have been assessed 

retrospectively but that, generally, little retrospective analysis is being 

done. They believed that retrospective analysis is necessary to inform the 

Congress and other decision makers of the cost and effectiveness of 

legislative and regulatory decisions. One participant stated that about 

100,000 new federal regulations have been adopted since 1981, when OMB 

began to keep records of them. About a thousand of these were judged to 

be economically significant—that is, imposing costs greater than $100 

million per year. However, the participant said, few of the set of regulations 

has ever been looked at to determine whether they have achieved their 

objectives, what they actually cost, and what their real benefits are. In fact, 

the participant added, little is known about the impact of regulations once 

they are adopted. Another participant pointed out that there is no 

consistent mechanism for reviewing a regulation once it has been enacted.

Some participants observed that a retrospective analysis might reveal that 

the costs or benefits of a regulation or program after enactment might vary 

significantly from those estimated in the prospective analysis. Because the 

3

The sidebars appearing in the margins of this section (Workshop Discussion) are excerpts 

from the background paper distributed to the workshop participants. The full context of 

these excerpts can be seen in appendix II.

Few agencies appear to use measures of 

economic performance, even though they 

are consistent with the Government 

Performance and Results Act. For 

example, in a survey of federal managers 

across the government, the percentage 

who reported having measures related to 

economic performance for their programs 

to a "great" or "very great" extent was 12 

percentage points lower than any other 

type of measure GAO asked about. In 

addition, we found that of approximately 

730 performance measures six federal 

agencies used to assess their programs, 

none involved net benefits, and only 19 

linked some kind of cost to outcome. Of 

these 19, one agency used 16. 
Page 11 GAO-05-796SP Economic Performance Measures

prospective analyses of a regulation’s or program’s benefits and costs are 

based on projections of likely future impact, the estimates might vary 

significantly from actual effects. Variation can occur on either the cost or 

benefit side of analysis. For example, one participant pointed out that it has 

been shown that some prospective analyses have overstated costs and 

understated benefits, while others have done the reverse.

Some participants also mentioned the use of economic performance 

measures for new areas of federal action, such as homeland security. They 

indicated that regulations the Department of Homeland Security is 

developing present additional challenges for analyzing risk and 

uncertainties associated with terrorist activities. In addition, a number of 

regulations are being proposed for homeland security without an 

assessment of whether the proposals’ estimated benefits justify their 

estimated costs. One participant suggested that requiring an economic 

assessment for proposed homeland security regulations would be useful. 

Another responded that such a requirement raises the question of how to 

estimate the probability of future terrorist attacks or how to determine if 

any particular measure, such as airport screening or an extra border patrol, 

would reduce the probability of damages from a terrorist attack. Another 

participant said that the focus should be not only on reducing the 

consequences of an attack but also on the probability of an attack. 

Developing a process that reduces the impact of attack—or “public 

mitigation”—would reduce the expected value of disruption. Participants 

pointed to the little experience the United States has with quantifying 

terrorism issues and the time it will take to build a body of knowledge on 

how to quantify effects.

Major Barriers to the Use of 

Economic Performance 

Measures

The participants identified several major barriers that impede the use of 

economic performance measures. For example, they said that there is 

(1) frequently only limited demand from decision makers for assessments 

of program costs; (2) a lack of both time and funds for conducting 

economic assessments and, in some instances, a lack of incentive for 

agencies to use resources for implementing a program for conducting 

economic performance assessment, particularly when an agency has 

already decided to act; and (3) a number of organizational barriers, called 

stovepipes, that hinder communication within agencies, between agencies, 

and between economists and decision makers about how to conduct 

comprehensive and useful assessments.

A retrospective review of the Occupational 

Safety and Health Administration (OSHA) 

scaffold standards indicated that while still 

positive, the program's actual benefits 

were significantly less than the agency 

estimated when the rules were proposed. 

The program's annual net benefit was 

projected at $204 million before 

implementation; retrospectively, annual 

benefit was estimated at $63 million. 

Retrospective analysis can be useful to 

decision makers by providing information 

on whether a program has the potential to 

produce additional benefits or whether the 

benefits produced justify the costs.
Page 12 GAO-05-796SP Economic Performance Measures

One participant said that a main impediment to this kind of analysis is that 

some decision makers may not be interested in knowing how a favored 

program is performing. In addition, even though some agency decision 

makers may require an economic performance assessment of proposed 

regulations, they might not provide sufficient resources to the staff to 

conduct a thorough analysis. Another participant suggested that there 

seems to be little demand from decision makers to know the total 

economic cost of a federal action. Participants did agree, however, that 

decision makers should be aware that their actions have certain 

consequences and costs. For example, the effects of a regulation or 

program cannot be known if an analysis is not done.

Some participants mentioned the limited resources that agencies have for 

conducting economic performance analysis and the agencies’ apparent 

reluctance to spend their resources, particularly when a decision to act 

may have already been made. One participant pointed out that funds are 

generally not authorized or appropriated for economic assessment. 

Consequently, in order to do these studies, agencies must use funds that are 

authorized and appropriated for program purposes. Using funds that would 

reduce resources for the program itself works as a disincentive for 

economic assessment. One participant observed that agency departments 

often seem to believe that analysis is done only after the decision to 

regulate is made. Under these conditions, analysis may or may not provide 

input into the final decision-making process. For example, instead of 

considering all the relevant policy alternatives, some analyses focus on just 

the preferred alternative. Another participant observed that one difficulty 

is the revisions an agency makes to a regulation after it has been proposed 

but before the rule is final. For example, an analysis completed to support a 

proposed rule may not represent the alternatives and other economic 

factors that make up the final rule.

Another participant said that despite what appears to be a long lead time 

between developing a regulation and issuing it, the economic assessment is 

often done in a very compressed period, limiting the time in which the 

agency’s analysts can conduct the assessment. In addition, one participant 

said that regulations are often mandated by legislation and the legislation is 

generally not subject to economic performance assessment.

The participants also identified organizational “silos” as barriers to 

communication between agencies, within agencies, and between agencies 

and the economics profession on how to properly conduct assessments. 

One participant stated that it was surprising that interaction among 
Page 13 GAO-05-796SP Economic Performance Measures

analysts conducting the assessments was not more seamless. Another 

participant stated that the many silos within government departments limit 

interaction within the agencies. The participant noted having tried to 

obtain information from other departments but running into brick walls all 

the time—walls whose masonry is very firm.

Analytical Issues That 

Affect Consistency and 

Credibility

The participants generally agreed that the consistency and credibility of 

economic performance analysis of federal regulations and programs could 

be improved by resolving several analytical issues. These include, but are 

not limited to, how to appropriately consider the benefits that cannot be 

put into monetary terms and the effect of federal actions on different 

income, racial, or other population groups. The participants said that 

guidance is insufficient on how to appropriately include these issues in 

economic performance analyses.

In addition, some participants noted a general lack of agreement about the 

values to be used for key economic assumptions. One is the value of a 

statistical life, which is used to estimate the effect of safety, health, or 

environmental improvements in reducing the risk of mortality. The 

participants also indicated a lack of guidance on how to use alternative 

analytical tools, such as multiobjective analysis, which can be used to 

evaluate program benefits by ranking them with a weighted scale rather 

than in monetary terms.

Participants expressed concern, however, that leaving nonmonetized 

benefits or costs out might inappropriately bias estimates of the net 

benefits of the federal action being analyzed. For example, one participant 

pointed out that economic assessments typically conducted to support 

proposed health, safety, and environmental regulations quantify the 

benefits but do not express them in monetary terms. As a result, the net 

benefits estimates exclude the benefits that cannot be monetized. Another 

participant mentioned that the difficulty associated with quantifying and 

monetizing benefits is one of the underlying challenges related to homeland 

security. These benefits include such things as the gain from averting 

terrorist attacks, something that is very difficult to estimate.

In addition, one participant pointed out that a strict economic efficiency 

analysis—that is, one based on maximizing net benefits—might leave out 

important policy alternatives on fairness or equity that cannot be put in 

money terms. Equity issues include how a program might affect people in 

different income or racial groups. Another participant indicated the need 
Page 14 GAO-05-796SP Economic Performance Measures

for some rigorous analytical way to look at federal regulations that, by their 

very nature, cannot possibly be justified on economic grounds. For 

example, a program that might not provide the largest net benefits might be 

justified for other reasons, such as that it provides assistance to groups 

such as the nation’s disabled or poor.

One participant said that when economists talk about distribution, they 

usually mean the distribution of income. In a regulatory setting, 

distribution often refers to the regulatory costs across other types of 

groups. For example, one participant indicated that expenditures on health 

could be reallocated from healthier people to people who are sick.

The participants also generally agreed that the consistency and credibility 

of economic performance measures could be improved if there was 

agreement on the most appropriate values to use for key assumptions in an 

analysis. For example, federal agencies use different estimates of the value 

of a statistical life to estimate the benefits associated with a reduction in 

the risk of mortality.

One participant indicated that some agencies use cost-effectiveness 

measures such as cost per health outcome, or quality-adjusted life-years, 

instead of net benefits. In any case, variability in the values of key 

assumptions and measures makes for a lack of consistency and for 

difficulty in comparing measures across agencies.

Participants also pointed out that agencies are dealing with the difficulty 

associated with monetizing benefits and assessing equity issues by using 

multiobjective evaluation measures. Although this type of analysis does not 

put impacts into monetary terms, it derives an estimate of impacts from a 

weighted ranking of the objectives of the federal action. One participant 

explained that in simplistic terms, this is done by identifying the multiple 

objectives of the proposed federal action and eliciting a weight by which to 

rank each objective on a scale. The weights come from an assessment of 

the variation and importance of the action. Another participant pointed to a 

link between these kinds of methods and economic performance analysis. 

Nonetheless, while agencies are using this type of analysis more frequently 

to evaluate federal actions, it is generally not mentioned in federal 

guidance, such as OMB’s.

Different agencies often use significantly 

different values for these same measures. 

For example, the U.S. Army Corps of 

Engineers tends not to value "statistical 

lives saved," while, at the time of the 

reports reviewed, the method of the 

Centers for Disease Control and 

Prevention valued statistical lives saved 

(of a 35-year-old man, for example) at 

$0.94 million; the Department of 

Transportation, at $2.7 million; and the 

Environmental Protection Agency, at 

$6.1 million. Such differences make it 

difficult to compare economic 

performance measures across agencies.
Page 15 GAO-05-796SP Economic Performance Measures

The Extension of 

Economic 

Performance Measures 

for Evaluating Federal 

Programs

The workshop’s participants generally agreed that there were opportunities 

to expand the use of economic performance measures, especially in 

retrospective evaluations of existing programs. Along these lines, analyzing 

economic performance could be one way to evaluate agencies’ 

performance through budget processes. In addition, participants indicated 

that economic performance measures could be used to assess the risk and 

uncertainty associated with homeland security programs and regulations.

Several participants said that retrospective evaluations of existing 

programs or regulations would not only inform decision makers about their 

performance but could also help to identify ways to improve prospective 

analyses. For example, comparing the actual benefits and costs achieved 

by a regulation with the prospective estimates developed for the proposed 

rule might be useful in identifying errors in the methods and assumptions 

that the economists used to develop the estimates. One participant said 

that we could identify the mistakes made in these analyses and transfer 

that knowledge to the next prospective analysis.

Participants also pointed out that net benefits and cost effectiveness are 

important for assessing budgets as well as regulations. One participant 

indicated that OMB circulars A-11 and A-94 could be linked together to use 

economic performance analysis in examining the budgetary process. For 

example, Circular A-11 specifies that agencies provide at least three viable 

alternatives to proposed capital investments and that the economic 

performance criteria used to develop those alternatives be based on 

guidance from Circular A-94. The participant also said that while Circular 

A-94 guidance may not be as extensive as the more recent Circular A-4, it 

includes the same basic principles for assessing benefits and costs. 

Another participant asked whether we know how much the federal 

government spends on permanent laws, tax benefits, and entitlements. 

Economic performance measures could be used to evaluate them. 

The participants generally agreed that economic performance measures 

could be used to evaluate the performance of homeland security programs 

and regulations. One participant suggested that a substantial fraction of the 

federal budget involves homeland security issues. However, other 

participants indicated that federal agencies would have to build on the 

analytical foundation for assessing whether the benefits of these 

investments exceed their costs. For example, developing ways to assess 

the probability of a terrorist attack, and the extent to which a program or 

regulation might reduce that probability, could help.

Economic performance measures have 

several potential uses. For example, EPA 

has received a report advising that its 

approach to estimating fines for 

noncompliance that are based on profits 

should consider incorporating probabilistic 

external costs, an economic performance 

concept. Changing government budgeting 

practice toward performance budgeting 

may create opportunities for incorporating 

economic performance information 

including OMB's PART reviews. OMB has 

indicated a preference for using more 

economic performance measures in the 

PART process. 
Page 16 GAO-05-796SP Economic Performance Measures

Improving General 

Economic Principles 

and Guidance

The participants generally agreed that opportunities exist for improving the 

principles and guidance agencies use for conducting benefit-cost analyses 

and assessing economic performance. For example, it might be useful to 

have abbreviated guidance on the minimum key principles for conducting 

an economic analysis. One participant said that when an agency has to do 

an evaluation and it is confronted with OMB’s Circular A-4, it might throw 

up its hands, saying the resources are not available. 

Another participant pointed out that we have to be concerned about 

“ossification” of the process agencies might have to go through to assess 

economic performance. For example, too many analytical requirements in 

too many different guidance documents might lead agencies to move away 

from doing any analysis. One participant suggested that Circular A-4 could 

represent the comprehensive end of a continuum of guidance documents, 

while more abbreviated guidance would facilitate performance analysis 

when fewer resources are available.

Another participant said that some progress in getting the agencies to do 

more analysis could be made if the guidance at least stipulated a minimally 

accepted set of principles that they could use. Minimum standards could 

include such things as whether an analysis used a discount rate.

The participants also generally agreed that the uncertainty and risk 

associated with investments in homeland security present additional 

challenges. Additional techniques are needed to help evaluate the 

uncertainty of terrorist activities, for example. One participant said that we 

need a serious effort to build an analytical capability to look hard at 

proposals that come under the homeland security banner, such as a 

framework for looking at proposals on the risk of terrorist activities. 

One participant said that in time, if guidance such as Circular A-4 remains 

in place, agencies will develop technical expertise and will begin to 

conduct fuller and more complete economic analyses of homeland security 

issues. Another participant indicated that evaluating federal actions related 

to homeland security, particularly budgetary processes, requires clearly 

defining the objectives of an action. For example, the participants thought 

that it is probably not realistic to expect security in the United States to be 

restored to some level that existed before September 2001. It might be 

more realistic to engage in a mix of public and private sector activities 

designed to minimize the consequences of another attack. This may require 

developing additional principles and guidance.
Page 17 GAO-05-796SP Economic Performance Measures

One-Page Summaries Requiring that economic performance assessments include a one-page 

summary of the key results of the analysis could help improve consistency. 

The summary would present the analysis results concisely and 

understandably. The summary might include a statement of the program’s 

objectives, a description of the baseline, and some discussion of at least the 

quantities, if not the actual monetization, of the direct inputs and outputs 

for the program activity. One participant expressed the strong feeling that a 

standard summary in front of an economic performance analysis that 

presents the results by providing the point estimates as well as the range of 

the estimated benefits and costs, to account for the uncertainty of the 

estimates, would be extremely useful. A good summary would allow 

reviewers to compare the results from different analyses.

Scorecards Some participants stated that better consistency and coverage of economic 

performance measures could be achieved with tools like scorecards for 

rating the overall quality of assessments. For example, scorecards could be 

used, like checklists, to evaluate assessments for the extent to which they 

address a minimum set of economic criteria. The criteria might include 

whether the analysis estimated costs and benefits, used a discount rate to 

estimate present values, and considered a reasonable set of alternatives. 

One participant said that there should be a set of criteria for economic 

performance measures in the public domain that would allow us to monitor 

performance.

Expert Review The participants also suggested that external experts could review 

economic performance analyses and suggest procedures and strategies on 

how to develop and use such measures as the value of statistical life. For 

example, one participant recommended peer review of the procedures 

agencies use to conduct the analyses and particular decisions about 

assumptions and measures in the analysis. The strategies developed 

through this review by experts could be either general or very specific.

Auditors use generally accepted auditing 

standards in rendering their professional 

opinion; this opinion can be thought of as 

a scorecard summary of a financial 

statement's consistency with generally 

accepted accounting principles. No federal 

guidance is available to link principles and 

guidelines to a formal quality evaluation of 

economic performance assessments of 

federal programs: There is no generally 

accepted scorecard.

Economic performance measures are 

sometimes reported in a format similar to 

a statement of income. Published 

literature and government guidance are 

not clear about the format for such 

statements, and we did not find consistent 

reporting formats in the economics 

textbooks we reviewed.

OMB has asked agencies to report on a 

standard form in their annual regulatory 

accounting reports, but this form is not 

required for any other use.
Page 18 GAO-05-796SP Economic Performance Measures

Standardizing Key Values Several participants indicated that standardizing some values for key 

assumptions would improve the quality and consistency of federal 

agencies’ economic performance assessments. The use of common values 

for such things as the value of a statistical life would make it possible to 

compare the results of analyses across agencies. One participant said that 

instead of recommending that agencies develop their own best practices 

for assessments, they should be encouraged to collaborate on methods and 

key assumptions.

New Organizations and 

Processes

The participants identified a number of organizations that could serve as 

examples in developing and improving economic principles for measuring 

the economic performance of federal programs. 

In response to a presentation by GAO’s Chief Accountant, participants 

discussed accounting and auditing standards and how those standards are 

established. The Chief Accountant defined the difference between 

accounting and auditing standards. In general, boards of professionals and 

highly qualified subject matter experts develop the standards. Through 

deliberation, public exposure and comments, and other processes, the 

boards develop a hierarchy of standards broadly applicable to accounting 

and auditing. In both accounting and auditing, consistency and quality are 

important aspects of financial reporting.

While several of the workshop participants expressed interest in the 

accounting model for setting standards, they also expressed concern about 

adopting such a model for economic performance evaluation. One 

participant pointed out that the types of issues assessed in the federal 

government are more diverse than in accounting. Although there is 

certainly virtue in standardization, it is not clear what would constitute a 

set of standards for all benefit-cost analyses. Other participants, however, 

acknowledged that economics institutions such as the American Economic 

Association are not designed to govern or monitor the application of 

economics.

The participants identified other types of standard-setting organizations 

that could be turned to for improving economics principles and guidance. 

For example, the National Academies convene expert consensus 

committees, workshops, and roundtables. Because of the National 

Economics textbook authors and 

academics we consulted pointed out that 

the quality of economic performance 

analysis could be improved by better 

standardization of, among other things, 

value of days lost from work and values 

for cases of various diseases and 

mortality.

General principles and guidelines that 

economists use in assessing economic 

performance are based on textbook 

presentations, research in journal articles, 

and federal agency guidance but are not 

identified or created by standard-setting 

authorities. In contrast, accountants and 

auditors have several standard-setting 

authorities, as well as academic literature 

and agency guidance, to improve quality, 

consistency, and comparability.

Generally accepted accounting principles 

provide layers of guidance to those who 

produce financial statements and to 

auditors. At the top of a hierarchy are 

pronouncements by professional 

standards groups, the Financial 

Accounting Standards Board for the 

private sector and nonprofit organizations, 

Governmental Accounting Standards 

Board for state and local governments, 

Federal Accounting Standards Advisory 

Board for the federal government. Below 

these in acceptance are textbooks, 

published articles, and guidance from 

agencies.
Page 19 GAO-05-796SP Economic Performance Measures

Academies’ strict conflict of interest standards, their expert consensus 

panels include academicians but not members from sponsors. Workshops 

are often day gatherings that bring together experts who present and 

review papers. A roundtable is an ongoing series of meetings that bring 

together representatives from industry, government, and academia to 

discuss recent research.

Other types of organizations the participants mentioned included 

Brookings Institution type panels and working groups convened by the 

National Bureau of Economic Research. The panels and working groups 

typically consist of distinguished economists given a mandate to assess 

government programs. Research conferences were also suggested as a way 

to convene experts to discuss benefit-cost analysis issues and then produce 

a book of conference papers. The participants also mentioned, in general 

terms, the possibility of creating a new organization, such as a government 

management or performance advisory board, to assess government 

performance. One participant mentioned that funding a new organization 

could prove to be a major issue. Some participants agreed that if such an 

institution were established, it should be organizationally independent and 

flexible enough to address a variety of issues and settings.
Page 20 GAO-05-796SP Economic Performance Measures

Appendix I

Economic Performance Workshop Apendxies

Participants: December 17, 2004 ApendxiI

Source: GAO.

Name Title Organization

External to GAO

Neil R. Eisner Assistant General Counsel, Office of Regulation and 

Enforcement

U.S. Department of Transportation

John Graham Administrator Office of Management and Budget, Office of 

Information and Regulatory Affairs

Robert Hahn Executive Director AEI-Brookings Joint Center for Regulatory Studies

Robert Haveman Professor Emeritus University of Wisconsin, La Follette School of Public 

Affairs

Arlene Holen Associate Director for Research and Reports Congressional Budget Office

Sally Katzen Professor University of Michigan, Law School

Thomas McGarity Professor University of Texas, School of Law

Albert M. McGartland Director, National Center for Environmental 

Economics

U.S. Environmental Protection Agency

Wilhelmine Miller Senior Program Officer Institute of Medicine

John F. Morall III Branch Chief, Health, Transportation and General 

Government Branch

Office of Management and Budget, Office of 

Information and Regulatory Affairs

Daniel H. Newlon Economics Program Director National Science Foundation

Greg Parnell Professor, and President, Decision Analysis Society U.S. Military Academy, West Point

V. Kerry Smith University Distinguished Professor North Carolina State University 

Richard Zerbe Professor University of Washington

GAO staff

Robert F. Dacey Chief Accountant U.S. Government Accountability Office

Scott Farrow Chief Economist U.S. Government Accountability Office
Page 21 GAO-05-796SP Economic Performance Measures

Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities ApendxiI

Introduction The impact of federal programs and tax preferences on the U.S. economy, 

including their costs and benefits, is substantial.1

 The cost to implement all 

federal programs was about $2.2 trillion in 2003, or roughly 20 percent of 

the U.S. gross domestic product. Similarly, federal tax preferences were 

estimated to be approximately $700 billion in 2003. The overall economic 

benefits of these programs have not been estimated, but they are believed 

to be substantial.

Because federal agencies generally do not monitor the economic 

performance of their programs, the extent to which each program 

generates positive net benefits (benefits minus costs) or whether it 

achieves its goals cost effectively (for the lowest possible cost) is 

uncertain. We have reported that federal agencies are generally required to 

assess the potential economic performance of proposed major regulatory 

actions and some investments but that their assessments are often 

inconsistent with general economic principles and guidelines.2

 Without 

assessments that include elements of quality such as consistency and 

comparability, federal decision makers may be missing information that 

would aid in oversight and accountability.

Economic performance measures such as net benefits and cost 

effectiveness are based, to the extent feasible, on quantifying and valuing 

all material impacts on a nation’s citizens. Such measures create a structure 

in which to report costs and benefits, evaluate cost savings, and, with a 

number of assumptions, evaluate whether the nation’s well-being is 

improved. The appeal of the measures is demonstrated by the requirement 

in several statutes and executive orders that economic performance be 

assessed and factored into federal agency decision making.3

 Nonetheless, 

critics of economic performance measures question their usefulness 

because of imprecision in valuation and difficulties in determining the 

1

Scott Farrow, Tim Guinane, Carol Bray, Phillip Calder, Elizabeth Curda, Andrea Levine, 

Robert Martin, and Don Neff prepared this paper for discussion at the December 17, 2004, 

GAO Workshop on Economic Performance Measures, with assistance from Pat Dalton, Joe 

Kile, Nancy Kingsbury, Paul Posner, and Jeff Steinhoff. We are grateful to Jay Fountain, 

Edward Gramlich, Aidan Vining, and Richard Zerbe for their help in reviewing the paper. It 

has been edited for this report.

2

GAO/RCED-98-142.

3

See, for example, Unfunded Mandates Reform Act of 1995, 2 U.S.C. §§1501-56, and 

Executive Order 12866.
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 22 GAO-05-796SP Economic Performance Measures

effect of federal programs on the nation’s well-being. We assume in this 

study that economic performance measures are used in conjunction with 

other measures to evaluate federal programs and policies.

The objectives of our work were to assess the potential for improving the 

quality and expanding the application of economic measures. Specifically, 

we reviewed the extent to which

1. federal agencies are required or have chosen to measure the economic 

performance of their programs,

2. general economic principles and guidelines are available for creating 

and evaluating economic performance assessments of federal 

programs, and 

3. the federal government can improve its oversight and accountability of 

the economic performance of federal programs as part of its overall 

performance objectives.

To meet these objectives, we formed a GAO team with expertise in 

assessing the economic, accounting, budgetary, and performance effects of 

federal programs. We also solicited input from several external experts 

from the economics and accounting professions. 

For objective 1, we identified commonly known applications of economics 

measures and reviewed six federal agencies’ performance reports on the 

status of their programs under the Government Performance and Results 

Act of 1993 (Results Act) as of 2002.4

 We chose the agencies judgmentally, 

as agencies with programs for which economic performance assessments 

were more rather than less likely to be conducted. In addition, we reviewed 

GAO reports on the extent to which agencies have used economic 

assessments of the potential impact of major regulatory actions and 

infrastructure investments.

For objective 2, we reviewed OMB guidance on conducting economic 

assessments, and we reviewed elements of accounting standards and 

economic principles and guidelines for conducting economic assessments.

4

The six agencies were the Department of Agriculture (USDA), Department of Education 

(Education), Department of Energy (DOE), Department of Labor (DOL), Department of 

Transportation (DOT), and Environmental Protection Agency (EPA). 
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 23 GAO-05-796SP Economic Performance Measures

For objective 3, we used GAO economic evaluations of the Special 

Supplemental Nutrition Program for Women, Infants, and Children (WIC) 

and the USDA Cotton Program to demonstrate two of many ways in which 

consistency could be improved.5

 We also reviewed and supplemented the 

Department of Labor’s (DOL) Occupational Safety and Health 

Administration (OSHA) economic analysis of construction industry safety 

standards for scaffolds. To do this, we used information from the Federal 

Register notice and other published sources. The assessments generally 

reflect the programs and conditions as they were at the time of original 

publication. We chose the OSHA analysis, USDA Cotton Program, and WIC 

because economic assessments were readily available, the programs were 

relevant but not highly controversial, and they illustrated several measures 

of net benefit.

Summary Even though federal agencies are required to assess the prospective 

economic performance of proposed major regulatory actions, and some 

other activities, agencies are not required, and generally do not choose, to 

evaluate programs retrospectively. And when agencies are encouraged to 

use economic performance measures retrospectively, such as under the 

Results Act, they use few such measures. In a recent survey, for example, 

GAO found fewer federal managers reporting having measures that linked 

program costs to program results to a “great” or “very great” extent, 

compared to all other types of Results Act measures.6

 In addition, at the 

time of the analysis, GAO found that of approximately 730 performance 

measures six federal agencies used to assess their programs, none involved 

net benefits, and only about 19 linked some measure of cost to outcome. 

DOE used 16 of these 19.

General principles and guidelines are available for assessing the economic 

performance of federal programs, but certain aspects of them may be too 

general to ensure that the assessments address some elements of quality, 

such as consistency and comparability. In addition, while economists 

generally accept the principles and guidelines, some agencies and 

5

GAO, Cotton Program: Costly and Complex Government Program Needs to Be 

Reassessed, GAO/RCED-95-107 (Washington, D.C.: June 20, 1995), and Early Intervention: 

Federal Investments Like WIC Can Produce Savings, GAO/HRD-92-18 (Washington, D.C.: 

Apr. 7, 1992).

6

GAO-04-38.
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 24 GAO-05-796SP Economic Performance Measures

noneconomists are less accepting. For example, in conducting economic 

assessments, some agencies do not account for benefits like the value of 

reduced risk of mortality, because they disagree that these benefits can be 

appropriately valued. However, assessments that do not account for these 

benefits are inconsistent with general economic principles and guidelines. 

Moreover, when agencies do account for these benefits, different agencies 

often use significantly different values, generating results that are not 

comparable.

In general, economic principles and guidelines are based on the economics 

literature and federal guidance. In our opinion, these principles and 

guidelines are too general in certain areas, because no standard-setting 

authority in the economics profession identifies or creates more specific 

practices for assessing economic performance. The accounting profession, 

in contrast, has standard-setting authorities that identify or create generally 

accepted accounting principles for financial reporting and generally 

accepted auditing standards for auditing financial statements. This 

guidance helps ensure the quality of financial reporting by, among other 

things, improving consistency and comparability.

The federal government could improve its oversight and accountability of 

federal programs, regulations, and taxes by expanding its use of economic 

performance measures and improving the quality of its economic 

performance assessments, both prospective and retrospective. Specifically, 

oversight, accountability, and quality could be improved by

1. expanding the use of economic performance measures, especially for 

retrospective analysis of existing programs, and

2. using a consistent reporting format and developing a scorecard, based 

on existing economic principles and guidelines and best practices, for 

evaluating the quality of economic assessments.

In illustrating the use of economic performance measures for new 

applications, our retrospective review of OSHA’s construction industry 

safety standards for scaffolds demonstrated that the program’s benefits 

have been significantly less than the agency estimated when the standards 

were proposed, so that additional improvements may be possible. Our use 

of a consistent reporting format for the existing GAO economic 

assessments of the USDA Cotton Program and WIC demonstrated how 

such a format supports comparability in presentation, as does a scorecard 

in the evaluation of the quality of an assessment.
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 25 GAO-05-796SP Economic Performance Measures

Background The economic performance of government programs is typically assessed 

by economists using estimates of the nationwide net benefit or cost 

effectiveness of the programs.7

 Economics literature of more than 75 years 

supports these methods. Economic performance assessment differs from a 

straightforward financial appraisal in that all gains (benefits) and losses 

(costs) that accrue to society in general (not just to government) as a result 

of a program are to be counted. In general, if the discounted value of the 

benefits exceeds the costs, the net benefits are positive. If these positive 

net benefits exceed the net benefits of alternatives, the program is 

economically worthwhile, although decision makers may consider other 

performance criteria as well, such as geographic or socioeconomic impact. 

Cost effectiveness, or the cost to achieve a particular objective expressed 

in nonmonetary terms (for example, reductions in tons of pollutants), is a 

special case of net-benefits analysis, in which the benefits of a program are 

quantified but not valued in dollar terms.

While some programs may result in benefits that are greater or less than 

costs, other programs may have benefits that are just equal to costs 

because of an equal transfer from one party to another. These benefits 

merely redistribute income or transfer resources between social groups but 

do not affect production or productivity. Such programs are called transfer 

programs and are not counted as having net benefits. Transfer programs 

typically include Social Security, interest on federal debt held by the public, 

and some types of welfare programs. In some cases, however, it can be 

difficult to determine whether a program has an impact on an economic 

performance measure or just transfers resources between social groups.

By including monetary based measures (monetization), economic 

performance assessment allows the aggregation of program impacts. Costs 

are usually measured in terms of a program’s actual money costs. In 

general, benefits are more difficult to measure, because many benefits may 

have no observable market providing prices. In these cases, it is necessary 

to construct representational, or “surrogate,” markets—that is, models—in 

ways that are generally accepted by economists, in order to estimate the 

monetary value of the benefit.

7

Cost-effectiveness analysis is closely related to net-benefit analysis, but the two types of 

analyses ask different questions. Cost effectiveness asks, what is the least costly way of 

achieving a particular objective? Cost-effectiveness analysis is used when there are 

difficulties in assigning monetary values to the outcomes of projects but the outcomes can 

be quantified along one nonmonetary dimension. 
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 26 GAO-05-796SP Economic Performance Measures

Modeling can present substantial problems and areas of ambiguity that can 

lead to imprecise measurement and fundamental disagreements among 

economists and noneconomists. One such ambiguity is in determining a 

monetary amount to estimate the value of a reduction in the risk of 

mortality. This value generally represents a statistical assessment of the 

amount of money individuals would be willing to pay to reduce the risk of 

one death in a population. Other instances are benefits that cannot be 

expressed in monetary terms and noneconomic factors that are part of a 

program’s performance. For example, a welfare program may represent a 

transfer in economic terms but decision makers may consider the resulting 

income redistribution worthwhile. In this case, economic assessments 

expressing benefits in money terms would best be used in conjunction with 

other performance measures.

Agencies assess the economic performance of federal programs in several 

circumstances. Although with many exceptions, the Unfunded Mandates 

Reform Act of 1995 requires agencies to prepare a qualitative and 

quantitative assessment of anticipated costs and benefits before issuing a 

regulation that may result in annual expenditures by state, local, and tribal 

governments of $100 million annually, in the aggregate or by the private 

sector. In addition, under Executive Order 12866 and Circular A-11, Part 7, 

Section 300, certain federal agencies are required to consider the benefits 

and costs of proposed regulatory actions and infrastructure investments 

before selecting a regulatory alternative or a capital investment.8

 In this 

context, OMB and some other federal agencies have developed guidance 

that functions as general economic principles and best practices for 

assessing economic performance.

8

Under Executive Order 12866, agencies are required to assess the benefits and costs of 

proposed regulations that are expected to have an annual effect on the economy of 

$100 million or more. 
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 27 GAO-05-796SP Economic Performance Measures

In addition, under the Results Act, federal agencies are required to 

establish performance goals and to choose measures to determine whether 

their programs are meeting these goals. In response to congressional 

requests, GAO has sometimes used economic principles and best practices 

to assess the economic performance of government programs. Examples 

include reports on the progress of the USDA Cotton Program and WIC.9

Agency Economic 

Assessments 

Although federal agencies are generally required to assess the potential 

economic performance of proposed major regulatory actions, they 

generally do not monitor how these and other federal programs have 

actually performed.10 In addition, although measures of economic 

performance are consistent with the Results Act, few agencies appear to 

use them. For example, in our survey of federal managers at grades GS-13 

and above, the percentage of managers who reported having measures 

related to economic performance for their programs to a “great” or “very 

great” extent was 12 percentage points lower than any other type of Results 

Act measure we asked about.11 In addition, in our staff study, we found that 

of approximately 730 performance measures six federal agencies used to 

assess their programs, none involved net benefits and only 19 linked some 

kind of cost to outcome; one agency used 16 of these 19.12 Examples of a 

partial measure that linked cost to outcome are average cost per case 

receiving medical services and the administrative cost per grant 

application.

Table 1 gives a preliminary summary of examples for which prospective 

economic assessments are required. The broad-based uses are for 

regulatory and investment purposes.

9

GAO/RCED-95-107 and GAO/HRD-92-18.

10One example of retrospective analysis from GAO’s work is Environmental Protection: 

Assessing Impacts of EPA’s Regulations through Retrospective Studies, GAO/RCED-99-250 

(Washington, D.C.: Sept. 14, 1999).

11GAO-04-38. Only 31 percent of the federal managers we surveyed reported having 

performance measures that linked product or service costs with the results achieved to a 

“great” or “very great” extent. 

12These numbers depend on how the agencies enumerated their measures in 2002, the year 

of our review, and involved evaluating the text in the Results Act reports. The evaluation 

required a degree of professional judgment to determine the total number of indicators and 

measures linking cost to program outcome. Nonetheless, the general result did not depend 

on the specific result of the number used. 
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 28 GAO-05-796SP Economic Performance Measures

Table 1: The Use of Economic Performance Measures for Prospective Assessment of Federal Programs

Source: GAO analysis.

In addition, as table 2 shows, retrospective economic assessments—after 

program implementation—are generally not required.

Table 2: The Use of Economic Performance Measures for Retrospective Assessment of Federal Programs

Source: GAO analysis.

Under the Results Act, federal agencies are required to establish 

performance goals for the results of their programs and to track progress. 

These assessments are retrospective—occurring after a program’s 

implementation. In these studies, cost-effectiveness (cost efficiency) 

Economic 

performance 

measure

Authority or 

guidance Reporting form Required? Timing Note

Budget planning: 

investment (general)

OMB Circular A-11; 

OMB Circular A-94; 

congressional 

mandates

Benefit-cost 

statement; guidance; 

not much detail on 

form

Yes Before 

implementation

Regulatory 

evaluation

Executive order and 

regulatory 

accounting 

statement

Varies widely; 

guidance is for a 

benefit-cost analysis

Yes, for major 

regulations

Before 

implementation

Agency-specific 

statutes

Specific statutes Usually specifies a 

benefit-cost analysis

Yes, if exists Some before and 

some after 

implementation

U.S. Army Corps of 

Engineers offshore 

oil and gas leasing 

and pipeline safety, 

and some EPA 

programs

Economic performance 

measure Authority or guidance Reporting form Required? Timing 

Government Performance and 

Results Act

Cost effectiveness named in 

committee report; net benefits 

not so named 

Varies but generally cost per 

unit outcome

No After implementation

Program Assessment Rating 

Tool (PART) review

OMB’s suggestion to agencies to 

include such measures 

Cost effectiveness and net 

benefit

No After implementation

Program evaluation Used on an ad hoc basis Varies; cost effectiveness or 

net benefit

No After implementation

Economic analysis; for 

example, GAO self-initiated

or congressional request 

GAO statutory authority; 

congressional request

Varies No After implementation
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 29 GAO-05-796SP Economic Performance Measures

measures are encouraged, along with quantity impacts and other measures. 

Net-benefit measures are not specifically cited but are consistent with the 

act in that such measures provide objective information on the relative 

effectiveness of federal programs and spending.13 Although economic 

performance measures are encouraged, they are often not used, as we 

discussed above.

As table 2 shows, agencies may conduct economic assessments in 

instances other than to follow the Results Act. These include Program 

Assessment Rating Tool (PART) reviews and ad hoc assessments to 

monitor program progress. OMB has indicated a preference for using more 

economic performance measures in the PART process. In addition, GAO 

conducted several retrospective reviews in response to congressional 

requests to monitor the progress of the USDA Cotton Program and WIC.

Other potential uses for economic performance measures exist. EPA has 

received a report advising that its approach to estimating fines for 

noncompliance that are based on profits should also consider 

incorporating probabilistic external costs—an economic performance 

concept. Changes in government budgeting practice toward performance 

budgeting may also create opportunities for incorporating economic 

performance information in budget material.

Economic Principles 

and Guidelines 

Certain aspects of the general economic principles and guidelines available 

for assessing economic performance may be too general to ensure some 

aspects of their quality, such as their consistency and comparability. For 

example, in conducting economic assessments not associated with the 

Results Act, some agencies do not account for benefits like the value of 

reduced risk of mortality, because they disagree with economists that these 

benefits can be appropriately valued or expressed in a cost-effectiveness 

measure.14 Nonetheless, assessments that do not account for these benefits 

are inconsistent with general economic principles and guidelines. And 

when different agencies do account for these benefits, they often use 

significantly different values, generating results that are not comparable. 

13See S. Rep. No. 103-58, at 29-30 (1993).

14Typically, economists use an estimate of the value of a statistical life to estimate the value 

of reduced risk of mortality. This is the amount people are willing to pay to avoid the risk of 

one more death in a population. 
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 30 GAO-05-796SP Economic Performance Measures

The accounting profession has authorities that identify or create generally 

accepted accounting principles for financial reporting and generally 

accepted auditing standards for auditing financial statements. This 

guidance helps ensure the quality of financial reporting by, among other 

things, improving consistency and comparability. No standard-setting 

authority in the economics profession identifies or creates credible 

practices for agencies when they need to work through specific difficulties 

in assessing economic performance.

General principles and guidelines economists use for assessing economic 

performance are based on textbook presentations, research reported in 

journal articles, and federal agency guidance. In contrast, accountants and 

auditors have several standard-setting authorities to identify or create 

standards and principles, in addition to academic literature and agency 

guidance that provide specific guidance to improve consistency and 

comparability. Generally accepted accounting principles provide layers of 

guidance to those who produce financial statements.

Pronouncements by professional standards groups, such as the Financial 

Accounting Standards Board (FASB) for the private sector and nonprofit 

groups, Governmental Accounting Standards Board (GASB) for state and 

local governments, and Federal Accounting Standards Advisory Board 

(FASAB) for the federal government, are at the top of the hierarchy. (The 

hierarchy is described briefly in enclosure I.) Below these in acceptability 

are materials such as textbooks, published articles, and guidance from 

agencies. Guidance for economic performance measurement and reporting 

is at a comparably low level in terms of acceptable standards.

While OMB guidance is useful in producing economic assessments and 

auditing performance evaluations, it is distinctly less standardized than 

guidance for accountants and auditors. Existing general guidance on 

conducting a program’s economic assessment appears to leave many 

practical gaps for federal applications that reduce consistency and 

comparability. Issues for which guidance is general may include, but may 

not be limited to, the value of days lost from work, values for various 

diseases and mortality, efficiency losses from taxation, the incorporation of 

multiple sources of estimates, changes in risk, benefits from improvements 

in information, and estimates of the efficiency effects of incentives implicit 

in transfers.

Guidance is general in that it recommends assigning monetary values to 

benefits but it does not specify which value to use. For example, for 
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 31 GAO-05-796SP Economic Performance Measures

programs that might reduce the risk of fatalities, OMB’s guidance 

encourages agencies to include the value of the risk reduction (based on 

the value of a “statistical” life) as a benefit of a federal program. But OMB 

does not require this assessment or provide guidance on the generally 

accepted value of a statistical life to use in estimating the benefit. As a 

result, agencies’ economic assessments often do not include these benefits 

or, when they do, estimates of the benefit are based on different values. For 

example, the U.S. Army Corps of Engineers tends not to value statistical 

lives saved, while the Centers for Disease Control and Prevention (CDC) 

values statistical lives saved (based on the life of a 35-year-old man, for 

example) at $0.94 million, DOT at $2.7 million, and EPA at $6.1 million.15

Such differences create difficulty in comparing economic performance 

measures across agencies.

Improving Measures’ 

Use and Improving 

Their Quality 

The federal government could strengthen program oversight and 

accountability by expanding the retrospective analysis of existing 

programs and by adopting a consistent reporting format, and a scorecard, 

for evaluating their quality.

A retrospective review of program performance could provide benefits 

through the expanded application of economic performance measures. For 

example, our review of OSHA’s construction industry safety standards for 

scaffolds demonstrated that retrospective analysis can be informative. The 

actual benefits of the program are now estimated to be significantly less 

than the agency estimated when the standards were proposed. 

Our use of a trial reporting format for our economic assessments of the 

USDA Cotton Program and WIC demonstrated how a consistent format 

enhances the synthesis of information, for both individual assessments and 

several assessments compared across applications, as does using a 

scorecard to evaluate an assessment’s quality. (Enclosure II describes the 

programs; details of the scorecard are in enclosure III.)

Before OSHA’s program was implemented, the annual net benefit was 

projected at $204 million, taking into account costs to the private sector 

and government and benefits resulting from reduced injury and death in the 

private sector. Retrospectively, the annual benefit was estimated at $63 

15These are late 1990 values, which would have generally increased with inflation by 2005. 
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 32 GAO-05-796SP Economic Performance Measures

million (see table 3 and enc. IV).16 This kind of finding could assist 

congressional oversight by better informing federal decision makers about 

whether a program has the potential to produce additional benefits or 

whether the benefits produced justify the costs.

Table 3: Summary of Three Programs’ Net Benefits 

Source: GAO analysis.

a

Includes, for example, benefits that accrue from encouraging small businesses, helping minorities, or 

redistributing income to society’s less fortunate persons.

b

Nonmonetized benefits also include better maternal health, improved nutritional status, and improved 

health of children born subsequently.

Quality, including aspects of consistency and comparability, can also be 

improved by using a consistent format for reporting the results of 

economic assessments. As various economics textbook authors and 

academics we consulted pointed out, quality could be improved by better 

standardization of such things as value of days lost from work, values for 

cases of various diseases and mortality, efficiency losses from taxation, 

incorporating multiple sources of estimates, changes in risk, benefits of 

improvements in information, and estimating the efficiency effects of 

incentives implicit in transfers. 

Accounting has a set of standard financial statements, including balance 

sheets and statements of income. Economic performance measures are 

sometimes reported in a format similar to that of a statement of income, 

although the time covered may be long and value may be reported as 

16We did not retrospectively investigate assumptions of the prospective assessment, other 

than the evidence on the changes in fatalities. 

Dollars in millions

Prospective Retrospective Retrospective

Economic performance measure

OSHA

scaffold rule

OSHA

scaffold rule

USDA Cotton

Program WIC

Benefit: Total annual in money terms $217 $76 $770 $1,036

Cost: Total annual in money terms 13 13 1,509 296

Net benefits: Total annual in money 

terms

204 63 –739 740

Nonmoney or noneconomic benefitsa Not

identified

Not

identified

 Ensuring

producer income

Lower

anemia ratesb
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 33 GAO-05-796SP Economic Performance Measures

present value. Such statements can also summarize outcomes that cannot 

be put in monetary terms, such as distributional and qualitative outcomes 

and uncertainty or sensitivity results. Published literature and government 

guidance are not clear about the format for such statements. We did not 

find consistent reporting formats in the economics textbooks we reviewed. 

OMB has asked agencies to report their annual regulatory accounting 

reports on a standard form, but the form is not required for any other use.17

A consistent format for reporting the results of an economic assessment 

would make it easier to (1) integrate major program impacts, 

(2) understand the bottom line of the economic performance analysis, and 

(3) compare results between assessments. For example, in our review of 

the three economic assessments shown in table 3, we found that the results 

of each one were distributed throughout their reports. This is not unusual 

for such assessments. The lack of a common form comparable to a 

financial statement also hindered the synthesis of information. The results 

of the case studies are presented in table 3 in a consistent, but highly 

abbreviated, format. The more detailed example we provide in enclosure II 

would assist in identifying major impacts that cannot be valued and would 

account for uncertainty.

The type of consistency shown in table 3 (and in enclosure II) would enable 

a noneconomist to note key components of the benefits and their 

magnitude and whether they were positive or negative. Trained readers 

might be sensitive to complexities or assumptions of the analysis without 

further explanation. For example, in addition to clearly showing the 

benefits retrospectively attributable to the programs, the summary in table 

3 can facilitate synthesis of information.18 Two of the programs have 

positive benefits, one negative. These results are somewhat unexpected. 

For example, as a type of welfare program, WIC might be considered a 

transfer program with zero net benefits, since income is merely transferred 

from one social group to another.

17OMB, Office of Information and Regulatory Affairs, Informing Regulatory Decisions: 

2003 Report to Congress on the Costs and Benefits of Federal Regulations and Unfunded 

Mandates on State, Local, and Tribal Entities (Washington, D.C.: Sept. 2003).

18In a direct comparison of the net benefits, it is assumed that the methodologies used to 

measure those benefits have been standardized, making such comparisons feasible. All the 

studies, for example, would have had to include the same value of a statistical life, if 

applicable. 
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 34 GAO-05-796SP Economic Performance Measures

In fact, the economic assessment of the program illustrates that WIC is 

estimated to have an impact through increasing birth weights, as well as 

reducing neonatal mortality and the incidence of iron deficiencies. All 

these factors are linked to behavioral and development problems in 

children, which, if avoided, could reduce medical, education, and other 

costs. In addition, OMB has classified many farm programs, such as the 

USDA Cotton Program, as transfer programs with no economic effect. This 

assessment, however, shows that the program has significant effects on the 

economy that are negative. This demonstrates the type of confusion that 

often surrounds transfers. A common format for reporting would better 

inform decision makers about programs’ economic performance.

Developing a scorecard, based on existing principles and guidelines, for 

evaluating the quality of economic assessments would also improve 

comparability. For example, auditors use generally accepted auditing 

standards in rendering their professional opinion. This opinion can be 

thought of as a scorecard summary of the consistency of financial 

statements with generally accepted accounting principles. The opinion may 

be

1. “unqualified,” indicating that the audited financial statements are in 

conformity with generally accepted accounting principles;

2. “qualified,” indicating that except for the effects of the matter to which 

the qualification relates, the financial statements are in conformity with 

generally accepted accounting principles; 

3. “adverse,” indicating that the financial statements are not in conformity 

with generally accepted accounting principles; or

4. “disclaimer of opinion,” indicating that the auditor is unable to form an 

opinion as to the financial statements’ conformity with generally 

accepted accounting principles. 

In economics, no professional or federal guidance is available to link 

principles and guidelines to a formal, quality evaluation of economic 

performance assessments of federal programs. Therefore, there is no 

generally accepted scorecard for evaluating them.

A scorecard would clearly and concisely illustrate the extent to which an 

assessment complies with general principles and guidelines for assessing 

economic performance. For example, it could show whether a discount 
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 35 GAO-05-796SP Economic Performance Measures

rate was used in an assessment and whether it was used correctly. Table 4 

gives examples of general principles and illustrations of economic opinions 

from a scorecard applied to the OSHA, USDA Cotton Program, and WIC 

studies.19

19Since GAO developed the criteria and the initial reports on the Cotton and WIC programs, 

this evaluation was not independent. In addition, the standards for rendering an economics 

opinion have not been formally developed. Nonetheless, for illustration and discussion 

purposes only, we rendered an opinion of “unqualified” since the limitations (“N” and “P” in 

table 4) did not appear to be significant. 
Appendix II

Economic Performance Assessment: Uses, 

Principles, and Opportunities

Page 36 GAO-05-796SP Economic Performance Measures

Table 4: Evaluating Economic Performance Assessments with a Scorecard 

Source: GAO analysis.

a

Based on OMB guidelines and GAO analysis.

b

A = fully meets requirement; P = partially meets requirement; N = meets not at all; NA = not 

applicable.

c

No opinion, since the OSHA example was not a complete economic assessment.

Enclosure III details the complete scorecard that we developed from OMB 

guidance, supplementing it from comparisons with accounting standards. 

As can be seen in the partial scorecard in table 4, we evaluated the three 

assessments relative to the augmented OMB guidelines. By summarizing 

what the assessments did and how they ranked in quality, this type of 

scorecard could inform federal decision makers about the quality of 

economic performance measures. The USDA Cotton Program and WIC 

assessments in table 4 were rated with an economic “unqualified” opinion, 

because the principles were generally followed; the OSHA scaffold rule 

was not rated, because the assessment was incomplete.

How assessment meets principleb

General principlea Primary principle

OSHA scaffold 

rule

USDA Cotton 

program WIC

Accounting entity The responsible unit—the source initiating the impact 

(i.e., the federal program)

A AA

Measures nationwide impact A A A

Accounts for net impact and not transfers A A A

Discount rate Discount rate is based on OMB guidance or another 

rate developed by appropriate techniques

N NA A

Consistent format Presentation summarizes the key results, using a 

consistent format

N NN

Transparent Presentation explicitly identifies and evaluates data, 

models, inferences, and assumptions

N AA

Presentation and documentation are sufficient to permit 

readers to replicate and quantify the effects of key 

assumptions

NA A NA

Comprehensive 

monetization

P AP

Economic performance Net benefits or cost effectiveness reported A A A

Internal quality control NA A A

External quality control Peer review was done NA A A

Opinion of economic 

analysis

c Unqualified Unqualified
Page 37 GAO-05-796SP Economic Performance Measures

Enclosure I

The Hierarchy of Generally Accepted 

Accounting Principles EncolsureI

Generally accepted accounting principles are presented as a hierarchy for 

accountants to determine appropriate accounting principles for 

transactions and auditors forming opinions on financial statements. For 

nongovernment and federal government entities, principles in category “a” 

are ranked highest. Those in category “e”—guidance from regulatory 

agencies and sources such as textbooks, handbooks, and articles—have 

the lowest rank. Sources from category “e”—in the absence of literature 

comparable to “a” to “d”—provide guidance for economic assessments.

Table 5: The Hierarchy of Generally Accepted Accounting Principles

Source: D. M. Pallais, M. L. Reed, and C. A. Hartfield, PPC’s Guide to GAAS: Standards for Audits, Compilations, Reviews, Attestations, Consulting, Quality Control and Ethics: Audit Reports (Fort Worth, Texas: Practitioners Publications Co., Oct. 2002), ch. 18, exhibit 18-1.

Principles of accounting for

Category Nongovernment Federal government 

a • Financial Accounting Standards Board (FASB) Statements 

and Interpretations;

• Accounting Principles Board (APB) Opinions; and 

• American Institute of Certified Public Accountants (AICPA), 

Accounting Research Bulletins.

• FASAB Statements and Interpretations and

• AICPA and FASB pronouncements if made applicable to 

federal government entities by FASAB Statements and 

Interpretations.

b • FASB Technical Bulletins and 

• AICPA Industry Guides and Statements of Position if they have 

been cleared.

• FASAB Technical Bulletins and 

• Cleared AICPA Industry Guides and Statements of 

Position if specifically applicable to federal government 

entities.

c • Consensus positions of the FASB Emerging Issues Task Force 

and

• Cleared AICPA Practice Bulletins.

• AICPA Practice Bulletins if specifically applicable to 

federal government and cleared by FASAB and

• FASAB Accounting and Auditing Policy Committee 

technical releases.

d • AICPA accounting interpretations; 

• FASB “Q and As”; 

• Industry practices if widely recognized and prevalent; and

• FASB, AICPA audit guides, SOPs, and practice bulletins that 

have not been cleared.

• Implementation guides FASAB staff publishes and

• Practices widely recognized and prevalent in the federal 

government.

e • Other accounting literature, including FASB concept 

statements, AICPA Issues Papers, International Accounting 

Standards Committee statements, Governmental Accounting 

Standards Board (GASB) statements, interpretations, and 

Technical Bulletins;

• Pronouncements of other professional associations or AICPA 

technical practice aids and regulatory agencies; and

• Accounting textbooks, handbooks, and articles.

• Pronouncements in hierarchy categories “a” through “d” 

for nongovernment entities when not specifically 

applicable to federal government entities;

• Other accounting literature, including FASB concept 

statements, AICPA Issues Papers, International 

Accounting Standards Committee statements, GASB 

statements, interpretations, Technical Bulletins, and 

concept statements;

• Pronouncements of other professional associations or 

AICPA technical practice aids and regulatory agencies; 

and 

• Accounting textbooks, handbooks, and articles.
Page 38 GAO-05-796SP Economic Performance Measures

Enclosure II

A Consistent Reporting Format for Economic 

Assessments EncolsureI

To demonstrate how a consistent format could be used to synthesize 

information in a comparable way, we used GAO economic assessments of 

USDA’s Cotton Program and WIC and our retrospective review of OSHA’s 

scaffold regulation for the construction industry.1

 We selected these 

assessments because they were readily available, the programs were 

relevant but not highly controversial, and the programs illustrated income 

transfers and other measures of net benefit.

WIC USDA’s Food and Nutrition Service administers WIC. The program is 

designed for eligible pregnant, breastfeeding, and post partum women and 

for infants and children up to age 5. Participants must have family incomes 

at or below 185 percent of the federal poverty level and must be at 

nutritional risk, as judged by a competent professional. WIC provides 

supplementary food, nutrition, and health education and referral to health 

and social services. In particular, participants are given coupons for 

purchasing specified kinds of food.

GAO conducted an economic assessment of WIC to estimate the extent to 

which the program can reduce the cost of other federally funded programs, 

such as Medicaid. WIC might be viewed as a transfer program, merely 

transferring income from one group in society to another, with no 

economic impact. Our assessment, however, indicated that the program 

does have an impact through such benefits as increasing birth weight and 

reducing neonatal mortality and the incidence of iron deficiency. Low birth 

weight and iron deficiency are linked to children’s behavior and 

development.

Some of the program’s benefits cannot be monetized, and distributional 

considerations, such as equity, may be significant in determining benefits. 

Nevertheless, we concluded, at that time, that given what can be valued, 

the program’s benefits exceed the costs. The monetized benefits are in 

health care cost savings and special education, which are resource savings 

to the economy.

We summarize these in table 6 in a format we used for the two other 

programs. The results for all three are reported in a format similar to a 

statement of income. We designed this format to include information on 

1

GAO/RCED-95-107 and GAO/HRD-92-1.
Enclosure II

A Consistent Reporting Format for Economic 

Assessments

Page 39 GAO-05-796SP Economic Performance Measures

key quantitative measures, benefits, and costs. It allows the net benefits (or 

cost-effective results) to be seen and its major components understood.

Table 6: Consistent Reporting Format: GAO’s WIC Assessment

Source: GAO analysis.

Note: Values are average annual values. Averted cost per birth is in thousands of dollars; benefit of 

WIC for WIC dollars spent is in dollars; all other dollars are in millions.

a

Nonmonetized benefits include better maternal health, lower anemia rates, improved nutritional 

status, and improved health of children born subsequently.

b

Nonmonetized costs include medical costs for nondisabled low birth-weight children.

As shown in table 6, WIC services were estimated to save $1.036 billion 

annually, because an estimated 36.5 thousand births at low birth weights 

were estimated to have been averted and 30.8 thousand low birth weight 

babies survived the first year. Providing WIC services to pregnant women 

Expected value Range of dollar values

Category

Quantity: Number of

births (in thousands)

Unit value: Averted

cost per birth Medium Low High

Key quantitative measure

Total low birth weight births averted 36.5 $28.4

Total low birth weight births (first-year 

survivors)

30.8 $33.7

Benefit of WIC (for WIC dollars spent)

Federal savings $1.14 $1.12 $1.51

State and local government savings 1.04

Private sector savings 1.32

Total annual monetized benefit $3.50 $3.46 $3.50

Total benefit from averted 

expenditures

$1,036

Cost of WIC

Government cost $296

Total annual monetized cost $296

Performance measure

Net monetized benefits $740

Nonmonetizable impact

Benefits a

Costs b

Size of nonmonetized benefits needed 

to change sign

–$740
Enclosure II

A Consistent Reporting Format for Economic 

Assessments

Page 40 GAO-05-796SP Economic Performance Measures

who delivered their babies in 1990 cost the federal government $296 

million. The program resulted in a net benefit of $740 million ($1.036 billion 

minus $296 million).

The expected return on the investment in prenatal WIC services is large, 

because low birth weight is a socially expensive outcome. Low birth weight 

infants, especially those with very low birth weights (under 3.3 pounds), 

have higher initial hospitalization costs. In addition, a smaller portion of 

these infants survive their initial hospitalization. Finally, they typically 

require more care because of disability or special education, which is 

expensive. Additional information, such as the unit value of savings 

distributed to various segments of society (in this case, various government 

levels) is also provided in table 6.

USDA Cotton Program USDA’s Cotton Program is designed to support cotton farmers’ income and 

cotton exports. GAO conducted an evaluation of the program to estimate 

its costs. From 1986 through 1993 (the period GAO evaluated), about 

90 percent of all acreage devoted to cotton was enrolled in the program. 

The program has since been changed. 

As a program that shifts money from taxpayers to farmers, it initially 

appears to be a transfer program. In fact, OMB typically classifies 

agricultural programs like this one as transfer programs with no economic 

impact.2

 However, the reasonably predictable results of the program design 

affect cotton production and prices and, therefore, have an impact on the 

economy. This impact occurs because in addition to the program’s basic 

components—to support producers’ income—the program required 

producers to idle acreage. Through program benefits, the government pays 

producers not to produce on the idled acres. With land taken out of 

production, society is prevented from benefiting economically from 

potential crops or using the land for other purposes. We concluded that, 

based on what can be valued, the program benefits were less than costs, 

resulting in a negative net benefit.

The results of our economic assessment are summarized in table 7, the 

same type of table as table 6.

2

OMB, Informing Regulatory Decisions. 
Enclosure II

A Consistent Reporting Format for Economic 

Assessments

Page 41 GAO-05-796SP Economic Performance Measures

Table 7: Consistent Reporting Format: GAO’s USDA Cotton Program Assessment

Source: GAO analysis.

Note: Values are average annual values.

a

Nonmonetized benefits include ensuring producer income.

b

Not identified.

Dollars in millions

Expected value Range of dollar values

Category

Quantity:

million pounds

Unit value:

per pound Medium Low High

Key quantitative measure

Average in the absence of program 7,524 $0.66

Program average 6,865 $0.66

Benefit of USDA Cotton Program

Net gain to buyers $16 –$38 $63

Net gain to producers 754 659 866

Total annual monetized benefit 770 621 929

Total benefit from USDA Cotton 

Program

$770 $621 $929

Cost of USDA Cotton Program

Government cost $1,509

Total annual monetized cost $1,509 $1,509 $1,509

Performance measure

Net monetized benefits –$739 –$888 –$580

Nonmonetizable impact

Benefits a

Costs b

Size of nonmonetized benefits needed to 

change sign

$739 $888 $580
Enclosure II

A Consistent Reporting Format for Economic 

Assessments

Page 42 GAO-05-796SP Economic Performance Measures

As the table shows, the program cost taxpayers, through the federal 

government, an average of $1.5 billion annually in program payments. 

Because of provisions of the program that required farmers to idle acreage, 

however, benefits to farmers were estimated to be only $770 million. This is 

because, among other things, the idled acreage was economically 

inefficient. As a result, the program net benefits were negative—an annual 

loss of $739 million, on average, for crop years 1986–93.3

This assessment illustrates that while OMB typically identifies farm 

programs as transfers, standard economic analysis suggests that these 

programs have real net national impact.4

 In addition, this impact may be 

negative—farmers gaining and consumers incurring larger costs than they 

would have without the program.

OSHA’s Safety 

Standards for Scaffolds

OSHA administers the safety standards for scaffolds used in the 

construction industry; the standards are designed to protect employees in 

that industry from falls, structural instability, electrocution, and 

overloading. The standards can be viewed as an element of a regulatory 

program—that is, a rule on occupational safety. When the rule was written, 

OSHA determined that approximately 9 percent of all fatalities in the 

construction industry were attributable to accidents related to scaffolding. 

Although OSHA’s final rule on scaffolds did not require an economic 

analysis under Executive Order 12866, OSHA did a prospective economic 

analysis to help inform the federal decision making.

3

In general, consumers did not gain from the program—they paid higher prices than they 

would have paid in the absence of the program. The assessment shows a small gain for 

consumers for one year that affected the average. The gain occurred because the 

government released cotton, accumulated under the program in previous years, from 

government stock, lowering prices from what they would have been otherwise. 

4

There are no economic gains from a pure transfer payment because the benefits to those 

who receive such a transfer are matched by the costs borne by those who pay for it. 

Therefore, transfers should be excluded from the calculation of net present value. It should 

also be recognized that a transfer program might have benefits that are less than the 

program’s real economic costs because of inefficiencies that can arise in program delivery 

of benefits and in financing. 
Enclosure II

A Consistent Reporting Format for Economic 

Assessments

Page 43 GAO-05-796SP Economic Performance Measures

The rule’s key benefits were forecast as coming from reduced injuries and 

deaths.5

 OSHA did not originally value the risk of mortality, so from an 

economic performance perspective, the net benefits are undervalued.6

 The 

prospective rule, however, was cost beneficial, even without valuing 

fatalities avoided. In OSHA’s prospective analysis, the agency reported a 

positive annual net benefit for the rule, based only on monetizing the value 

of work days lost from injuries and the estimated cost of compliance and 

government costs.

We monetized the value of fatalities avoided by the scaffold rule, by 

applying EPA’s value of a statistical life ($6.1 million), DOT’s ($2.7 million), 

and CDC’s ($0.94 million) as estimated at the time of the rule.7

 When 

fatalities avoided are monetized, the estimated net benefits increase by 

tens to hundreds of millions of dollars per year. Table 8 summarizes the 

results of our assessment, in the same format as we applied to the USDA 

Cotton Program and WIC.

5

We did not investigate agency material cited as being publicly available in the regulation 

docket; we used only information from the Federal Register notice and other published 

sources. Consequently, the OSHA example is for illustration and might be materially 

different if the supporting information were investigated.

6

Since we completed this analysis, OSHA has used the EPA value of a statistical life for a 

proposed regulation. 

7

This uses CDC’s methodology for a 35-year-old man. 
Enclosure II

A Consistent Reporting Format for Economic 

Assessments

Page 44 GAO-05-796SP Economic Performance Measures

Table 8: Consistent Reporting Format: GAO’s OSHA Scaffold Assessment 

Source: GAO assessment using OSHA, Centers for Disease Control and Prevention, Environmental Protection Agency, and Department 

of Transportation data or methods.

Note: Values are average annual values. Dollars are in millions.

a

Based on Centers for Disease Control and Prevention’s methodology, yielding $0.94 million for the 

value of a statistical life of a 35-year-old man.

b

Based on Environmental Protection Agency’s value of $6.1 million per value of statistical life.

c

Here, a constant cost per injury is assumed, based on the total value provided and the number of 

injuries.

d

Based on Department of Transportation’s value of $2.7 million per value of statistical life.

e

OSHA omitted the value of life, with a net benefit of $77 million.

f

Not identified.

Expected value Range of dollar values

Category

Quantity:

Number

Unit value: 

Cost per

statistical life Medium Lowa Highb

Key quantitative measure

Injuries avoided 4,455 $20.2c

Fatalities avoided 47 $2.7d

Benefit of scaffold rule

Gain from injuries avoided $90 $90 $90

Gain from fatalities avoided 127 44 287

Total annual monetized benefit 217 134 377

Total benefit from scaffold rule $217 $134 $377

Cost of scaffold rule

Inspections $5 $5 $5

Training 2 2 $2

Protection against falls 6 66

Total annual monetized costs $13 $13 $13

Performance measuree

Net monetized benefits $204 $122 $364

Cost effectiveness (fatality avoided per 

cost)

0 cost per life

saved

0 cost per life

saved

0 cost per life

saved

Present value net benefit 7% $2,918 $1,737 $5,201

Nonmonetizable impact

Benefits f

Costs f

Size of nonmonetized benefits needed to 

change sign

–$204 –$122 –$364
Enclosure II

A Consistent Reporting Format for Economic 

Assessments

Page 45 GAO-05-796SP Economic Performance Measures

As shown in table 8, if fatalities avoided are included, the rule is estimated 

to generate $204 million in annual national net benefits. That value can be 

as low as $122 million and as high as $364 million, depending on the value 

of statistical life used. As the benefits of the rule exceed the costs, even if 

fatalities are omitted, the cost per life saved (a cost-effectiveness measure) 

is zero.
Page 46 GAO-05-796SP Economic Performance Measures

Enclosure III

A Scorecard for Evaluating Economic 

Program Assessments EnoclsureI

We developed a scorecard from OMB guidance, and other relevant criteria, 

in order to illustrate links between accounting and economics criteria. To 

demonstrate how the scorecard could be used, we applied it to the OSHA, 

USDA Cotton Program, and WIC programs previously discussed. The 

scorecard includes reference to an opinion, similar to opinions rendered in 

financial statement audits, that indicate the extent to which the economic 

assessments met the criteria.1

The scorecard’s categories are illustrative rather than comprehensive. 

Since our scope and methodology did not investigate OSHA’s data in detail, 

many items in OSHA’s assessment of the scaffold rule were identified as 

“not applicable.” Therefore, we were not able to render an opinion on that 

assessment. The categories in the scorecard present nonetheless a 

consistent format for evaluating the extent to which an economic 

assessment adhered to accepted principles and guidelines.

1

Since GAO developed both the standards and the reports, which were evaluated using the 

standards, the evaluation was clearly not independent. Recognizing this, we provide the 

scorecard for illustration and discussion. 
Enclosure III

A Scorecard for Evaluating Economic 

Program Assessments

Page 47 GAO-05-796SP Economic Performance Measures

Table 9: A Scorecard for Evaluating Economic Performance Assessments

How assessment meets principleb

General principlea Primary principle

OSHA scaffold

rule

USDA Cotton 

Program WIC

Accounting entity The responsible unit—the source 

causing the impact (i.e., the federal 

program)

A AA

Measures nationwide impact A A A

Accounts for net impacts and not 

transfers

A AA

Reliability Results of assessment are verifiable NA P P

Data and assumptions used are a 

faithful representation of what 

actually happened 

NA P P

Precision of results is made explicit N A A

Data, assumptions, and descriptions 

are unbiased

NA A A

Comparable Similar methods and assumptions 

are used when analyzing different 

entities 

NA A A

Consistent Similar methods and assumptions 

are used for analyzing similar events 

in different time periods 

NA A NA

Revenue and benefits 

recognition

Accounts for revenues and benefits 

when they are realized and earned

A AA

General measurement 

standard

Estimates dollar value of material 

impact resulting from, or affected by, 

program 

N AA

Estimates quantitative material 

impacts but does not monetize them

P PP

Alternative plans Evaluates most likely conditions 

expected, with and without the 

program 

A AA

Analyzes all reasonable alternative 

courses of action

N AA

Considers extent to which entities 

comply with related laws and 

regulations

A AP

Discount rate Discount rate is based on OMB 

guidance or other rate developed 

through appropriate techniques

N NA A

Uncertainty Considers the effect of uncertainty on 

results

N AA
Enclosure III

A Scorecard for Evaluating Economic 

Program Assessments

Page 48 GAO-05-796SP Economic Performance Measures

Source: GAO analysis.

a

Based on OMB guidelines and GAO analysis.

b

A = fully meets requirement; P = partially meets requirement; N = meets not at all; NA = not 

applicable.

c

No opinion, since the OSHA example was not a complete economic assessment.

Clear rationale Presents justification for program 

(e.g., market failure, legislative 

requirement)

NA NA A

Consistent format used Presentation summarizes key results 

consistently

N NN

Transparent Presentation explicitly identifies and 

evaluates data, models, inferences, 

and assumptions

N AA

Presentation and documentation are 

enough to permit readers to replicate 

and quantify the effects of key 

assumptions

NA A NA

Comprehensive 

monetization

P AP

Economic performance Net benefits or cost effectiveness are 

reported

A AA

Internal quality control NA A A

External quality control Peer review was done NA A A

Opinion of economic 

analysis

c Unqualified Unqualified

(Continued From Previous Page)

How assessment meets principleb

General principlea Primary principle

OSHA scaffold

rule

USDA Cotton 

Program WIC
Page 49 GAO-05-796SP Economic Performance Measures

Enclosure IV

Assessing OSHA’s Scaffold Rule by 

Retrospective Analysis EnoclsureIV

A retrospective analysis of OSHA’s scaffold rule, when compared to the 

results of the prospective analysis in enclosure II, provides information on 

the net benefits estimated before the rule was implemented, which can be 

compared with the net benefits after the rule had been in effect for some 

time. Most economic performance measures are estimated prospectively 

for regulatory or capital spending purposes. Feedback on what occurs after 

a program has been implemented can assist in a program’s oversight and 

modification, if appropriate, and can help improve the quality of other 

prospective studies.

Seong and Mendeloff recently reported a retrospective analysis of OSHA’s 

scaffold rule.1

 Their study focused on benefits; no retrospective 

information on costs is known to be available. In table 10, the prospective 

assessment is compared with the retrospective assessment.

1

Si Kyung Seong and John Mendeloff, “Assessing the Accuracy of OSHA’s Estimation of the 

Benefit of Safety Standards,” paper presented at the Research Conference of the 

Association for Public Policy Analysis and Management, Dallas, Texas, November 7–9, 2002; 

a revised version is available at www.aei-brookings.org (December 3, 2003). 
Enclosure IV

Assessing OSHA’s Scaffold Rule by 

Retrospective Analysis

Page 50 GAO-05-796SP Economic Performance Measures

Table 10: Prospective and Retrospective Assessments of OSHA’s Scaffold Rule 

Compared

Source: GAO analysis.

Note: Values are average annual values.

a

Value based on Department of Transportation value of a statistical life of $2.7 million.

b

Not identified.

Dollars in millions

Expected value

Prospective Retrospective

Category Quantity Valuea Quantity Value

Key quantitative measure

Injuries avoided 4,455 1,564

Fatalities avoided 47 17

Benefit of scaffold rule

Injuries avoided: Gain $90 $32

Fatalities avoided: Gain 127 45

Total annual monetized benefit $217 $76

Cost of scaffold rule

Inspections $5 $5

Training 2 2

Protection against falls 6 6

Total annual monetized cost $13 $13

Performance measure

Net monetized benefits (annual) $204 $63

Cost-effectiveness (fatality 

avoided per cost)

0 cost per life

saved

0 cost per life

saved

Present value net benefit 7% $2,918 $908

Nonmonetizable impact

Benefits b b

Costs b b

Size of unmonetized benefits 

needed to change sign

–$204 –$63
Enclosure IV

Assessing OSHA’s Scaffold Rule by 

Retrospective Analysis

Page 51 GAO-05-796SP Economic Performance Measures

As table 10 shows, in the prospective assessment, injuries avoided were 

estimated at 4,455, fatalities avoided at 47; in the retrospective assessment, 

injuries avoided were estimated at 1,564, fatalities 17. In addition, 

prospectively, the annual benefits of the program were projected to be 

$204 million; retrospectively, $63 million.2

These estimates, based on realizations of deaths in the construction 

industry, indicate that the expected benefits of the OSHA scaffold rule have 

not been fully realized, since the number of fatalities has not decreased as 

much as expected. Even with the lower realization of safety benefits in the 

retrospective assessment, however, it appears that the rule has a favorable 

economic performance evaluation. However, the retrospective analysis 

suggests that (1) additional safety benefits may be obtained from the rule 

and (2) OSHA may usefully investigate the difference between the expected 

number of fatalities avoided and the estimated number actually avoided. If 

the difference is found to be an understandable forecasting error, that 

result could potentially inform future estimates for this and other related 

programs.

2

We did not retrospectively verify other assumptions in the prospective analysis. 

(460571)