MANAGING FOR

RESULTS 

Data-Driven 

Performance Reviews 

Show Promise But 

Agencies Should 

Explore How to 

Involve Other 

Relevant Agencies 

Report to Congressional Addressees

February 2013 

GAO-13-228 

United States Government Accountability Office

GAO 
United States Government Accountability Office

Highlights of GAO-13-228, a report to 

congressional addressees.

February 2013

MANAGING FOR RESULTS

Data-Driven Performance Reviews Show Promise but 

Agencies Should Explore How to Involve Other 

Relevant Agencies

Why GAO Did This Study

Given the federal government’s central 

role in addressing many of the 

American public’s most pressing 

concerns, it is critical that government 

performance is managed effectively.

GAO’s previous work has shown that 

many federal agencies have struggled 

to adopt effective performance 

management practices. Congress took 

steps to improve federal performance 

management with the passage of the

Government Performance and Results 

Act Modernization Act of 2010

(GPRAMA), which included a provision 

for agency leaders to conduct 

quarterly, data-driven performance 

reviews. As part of GAO’s mandate to 

review GPRAMA implementation, this 

report (1) identifies practices that can 

promote successful data-driven 

performance reviews at the federal 

level and examines how they are being 

implemented at selected agencies and 

across the government, and (2) 

examines the impact of quarterly datadriven performance reviews on 

selected agencies’ progress toward 

high priority and other performance 

goals. To address these objectives, 

GAO reviewed academic and policy 

literature; information from practitioners 

at the local, state, and federal level; 

and Office of Management and Budget 

(OMB) guidance. GAO surveyed 

performance improvement officers at 

24 federal agencies and examined 

review implementation at three 

agencies—DOE, SBA, and Treasury. 

What GAO Recommends 

GAO recommends that OMB identify 

and share practices to use the reviews 

for interagency collaboration, when 

relevant, to achieve agency goals. 

OMB staff generally agreed with the 

recommendation.

What GAO Found

GAO identified nine leading practices to promote successful data-driven 

performance reviews—referred to as quarterly performance reviews—at the 

federal level. 

• Agency leaders use data-driven reviews as a leadership strategy to drive 

performance improvement. 

• Key players attend reviews to facilitate problem solving. 

• Reviews ensure alignment between agency goals, program activities, and

resources. 

• Agency leaders hold managers accountable for diagnosing performance 

problems and identifying strategies for improvement.

• Agency has capacity to collect accurate, useful, and timely performance data. 

• Agency staff have skills to analyze and clearly communicate complex data for 

decision making.

• Rigorous preparations enable meaningful performance discussions. 

• Reviews are conducted on a frequent and regularly scheduled basis. 

• Participants engage in rigorous and sustained follow-up on issues identified 

during reviews.

Treasury’s Deputy Secretary Leads the Department’s Quarterly Performance Reviews 

Most officials GAO interviewed at the Department of Energy (DOE), Small 

Business Administration (SBA), and Department of the Treasury (Treasury)

attributed improvements in performance and decision making to the reviews. 

DOE, SBA, and Treasury officials said their reviews allowed different functional 

management groups and program areas within their agencies to collaborate and 

identify strategies which led to performance improvements. GAO’s survey of 

performance improvement officers indicated that there was little to no 

involvement in the reviews from other agencies that could help achieve agency 

goals. This was also true at DOE, SBA, and Treasury, where officials expressed 

concerns about including outsiders in their reviews and described other means of 

coordinating with them. However, OMB guidance—along with a leading practice

GAO identified—indicates that including key players from other agencies can 

lead to more effective collaboration and goal achievement. View GAO-13-228. For more information, 

contact J. Christopher Mihm at (202) 512-6806

or mihmj@gao.gov.
Page i GAO-13-228 Managing for Results 

Letter 1 

Background 4

Practices that Promote Successful Data-Driven Performance 

Reviews at the Federal Level 8

DOE, SBA, and Treasury Officials Attributed Improvements in 

Performance and Decision Making to Quarterly Performance 

Reviews, although Some Reported Minimal Impact 32

Conclusions 37

Recommendation for Executive Action 38

Agency Comments and Our Evaluation 38

Appendix I Objectives, Scope, and Methodology 40 

Appendix II Responses to Selected Questions from GAO’s Survey on 

Agency Performance Management Positions and 

Responsibilities 46 

Appendix III Federal Agencies Subject to the Chief Financial 

Officers Act 51 

Appendix IV Comments from the Department of the Treasury 53 

Appendix V Comments from the Department of Energy 55 

Appendix VI Full Text for Interactive Figure 1 on Finding the Right Meeting 

Size and Composition for Effective Performance Reviews 56 

Appendix VII GAO Contact and Staff Acknowledgments 58 

Contents 
Page ii GAO-13-228 Managing for Results 

Selected Bibliography 59 

Related GAO Products 62

Tables 

Table 1: Question 22—To what extent does your performance 

improvement staff have each of the following 

competencies that OPM has identified? 46

Table 2: Question 27—Does your agency conduct GPRAMArequired quarterly performance reviews? 47

Table 3: Question 28—Did your agency conduct quarterly 

performance reviews (or similar reviews) before the 

GPRAMA requirement took effect in June 2011? 47

Table 4: Question 31—How often does your agency conduct 

performance reviews (although GPRAMA requires 

quarterly reviews, some agencies have established other 

review cycles to meet their management needs)? 48

Table 5: Question 32—How much involvement, if any, does each of 

the following officials have in your agency’s quarterly 

performance reviews? 48

Table 6: Question 33—For each of the following officials, has their 

involvement in agency performance management 

increased, remained about the same, or decreased as a 

result of your agency’s quarterly performance reviews? 48

Table 7: Question 34—How frequently do your agency’s quarterly 

performance reviews include the following 

characteristics? 49

Table 8: Question 35—How frequently do your agency’s quarterly 

performance reviews include the following characteristics 

or practices and how easy or challenging has it been to 

implement them? 49

Table 9: Question 35—How frequently do your agency’s quarterly 

performance reviews include the following characteristics 

or practices and how easy or challenging has it been to 

implement them? 50

Table 10: Information about How DOE, SBA, and Treasury Conduct 

Their Quarterly Performance Review Meetings 57
Page iii GAO-13-228 Managing for Results 

Figures 

Figure 1: Finding the Right Meeting Size and Composition for 

Effective Performance Reviews 13

Figure 2: Treasury Linked Shorter-Term Priority Projects to 

Achievement of Goals in its Quarterly Performance 

Reviews 18

Figure 3: Number of PIOs Reporting That Their Agency’s Reviews 

Included Selected GPRAMA Requirements More than Half 

the Time 20

Figure 4: Example of Materials SBA Used to Train Managers and 

Staff to Effectively Communicate Data Analysis to 

Decision Makers 25

Figure 5: Treasury Tracked its Own Performance on Completing 

Follow-up Items 31

Figure 6: Treasury Credits Stat Meetings with the Decision to Stop 

Minting $1 Coins for Circulation and Saving the U.S. 

Government Millions 35
Page iv GAO-13-228 Managing for Results 

Abbreviations 

APG Agency Priority Goal 

CFO Chief Financial Officer 

COO Chief Operating Officer 

DOE Department of Energy 

FPDS-NG Federal Procurement Database System-Next Generation 

FMS Financial Management Service 

GPRA Government Performance and Results Act of 1993 

GPRAMA Government Performance and Results Act Modernization 

 Act of 2010 

IRS Internal Revenue Service 

NYPD New York City Police Department 

OMB Office of Management and Budget 

PIC Performance Improvement Council 

PIO Performance Improvement Officer 

Recovery Act American Recovery and Reinvestment Act of 2009 

SBA Small Business Administration 

Treasury Department of the Treasury 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately. 
Page 1 GAO-13-228 Managing for Results 

United States Government Accountability Office

Washington, DC 20548 

February 27, 2013 

Congressional Addressees 

How federal agencies manage toward better performance has a 

significant effect on many of the American public’s most pressing 

concerns—ranging from unemployment to food safety to national security. 

However, our previous work has shown that many federal agencies have 

struggled to adopt effective performance management practices. 

Specifically, our work over the past 15 years has indicated that using 

performance data for decision making can lead to better results, but in 

several surveys we have done, less than half of federal managers 

reported using performance data for decision making to a great or very 

great extent.1

 In looking for solutions to this long-standing problem, 

Congress identified an effective management tool that has been widely 

adopted by local and state governments—data-driven performance 

review meetings, often referred to as “Stat” meetings.2

 Congress took 

steps to improve federal performance management with the passage of 

the Government Performance and Results Act Modernization Act of 2010 

(GPRAMA), which included a specific provision for quarterly performance 

reviews, modeled after those at the local and state level.3

 

 

1

See GAO, Government Performance: Strategies for Building a Results-Oriented and 

Collaborative Culture in the Federal Government, GAO-09-1011T (Washington, D.C.: 

Sept. 24, 2009); and Government Performance: Lessons Learned for the Next 

Administration on Using Performance Information to Improve Results, GAO-08-1026T 

(Washington, D.C.: Jul. 24, 2008). 

2

For the purposes of this report, we define data-driven performance review meetings as 

“regularly scheduled, structured, data-driven meetings to review performance indicators 

with department or program personnel.” See Harry Hatry and Elizabeth Davies, “A Guide 

to Data-Driven Performance Reviews,” IBM Center for the Business of Government 

Improving Performance Series, (Washington, D.C.: 2011). 

3

Pub. L. No.111-352, 124 Stat. 3866 (2011). In 1993, Congress passed the Government 

Performance and Results Act (GPRA), Pub. L. No. 103-62, 107 Stat. 285 (1993), as part 

of a statutory framework aimed at improving federal government management. GPRA 

required federal agencies to develop strategic plans with long-term goals, performance 

plans with annual goals and measures, and performance reports on prior year 

performance. GPRAMA amends GPRA to create a new government-wide performance 

management framework with several provisions—including the one for agency quarterly 

performance reviews—intended to increase federal agencies’ use of performance 

information to improve their performance and results. 

 
 

Page 2 GAO-13-228 Managing for Results 

As part of our mandate to review the implementation of GPRAMA, this 

report (1) identifies practices that can promote successful data-driven 

performance reviews at the federal level and examines how these 

reviews are being implemented at selected agencies and across the 

government, and (2) examines the impact of quarterly data-driven 

performance reviews on selected agencies’ progress toward high priority 

and other performance goals.4

 This report is the second in a series that 

examines how agencies are implementing various GPRAMA 

requirements.5

To address the first objective, we identified practices that can promote 

successful data-driven reviews at the federal level by conducting a review 

of relevant academic and policy literature, including our previous reports.6

We refined these practices with additional information obtained from 

practitioners at the local, state, and federal level who shared their 

experiences and lessons learned.7

 We also compared these practices 

with recent GPRAMA related guidance in the Office of Management and 

Budget’s (OMB) Circular No. A-11 and found them broadly consistent.8

We observed two data-driven review meetings at the Department of the 

Treasury (Treasury), which was one of the agencies selected to address 

our reporting objectives. We also examined how these reviews are being 

implemented at agencies across the government by conducting a survey 

 

4

31 U.S.C. § 1121(b). 

5

See GAO, Managing for Results: GAO’s Work Related to the Interim Crosscutting Priority 

Goals under the GPRA Modernization Act, GAO-12-620R (Washington, D.C.: May 2012). 

We will issue additional work on implementation of agency priority goals, agency 

performance management infrastructure, and other GPRAMA requirements at a later 

date. 

6

See bibliography. 

7

These practitioners included participants in the Performance Improvement Council 

working group on agency reviews and Stat managers at the municipal level. The 

Performance Improvement Council is an interagency committee composed of agency 

performance improvement officers charged with assisting the Office of Management and 

Budget with topics related to GPRAMA and facilitating the exchange of useful 

performance management practices among agencies. 

8

In August 2012 OMB revised OMB Circular No. A-11, Preparation, Submission, and 

Execution of the Budget (Aug. 3, 2012). The revisions included guidance for federal 

agencies related to the preparation and submission of strategic plans, annual performance 

plans, and annual program performance reports, in addition to information on 

implementing data-driven performance reviews. OMB Cir. No. A-11, at § 270. 
 

Page 3 GAO-13-228 Managing for Results 

of performance improvement officers (PIO) in the 24 agencies covered by 

the Chief Financial Officers Act of 1990 and subject to GPRAMA’s 

requirements.9

 We received responses from all 24 PIOs—a 100 percent 

response rate. 

To address both objectives, we selected three agencies to examine 

implementation of GPRAMA-mandated quarterly performance reviews in 

greater depth—Department of Energy (DOE), Small Business 

Administration (SBA), and Treasury. We selected these three agencies 

because they have been performing data-driven reviews for at least one 

year and, together, use a mix of government tools—such as direct 

service, regulations, grants, loans, and tax expenditures—to achieve their 

performance goals, among other reasons.10 At each selected agency, we 

focused on two agency priority goals (APG)11 to examine how quarterly 

performance reviews affected the agency components responsible for 

achieving performance outcomes.12 Because the scope of our review was 

to examine data-driven performance reviews as a leadership strategy, we 

did not evaluate whether these goals were appropriate indicators of 

agency performance, sufficiently ambitious, or met other dimensions of 

quality. 

We also reviewed memorandums, internal briefings, and other materials 

agencies used to prepare for the reviews, as well as documents used 

during the reviews and follow-up materials. We conducted interviews with 

officials at OMB, the Performance Improvement Council (PIC), and 

 

9

The survey also included questions on PIOs’ and other key officials’ characteristics, their 

involvement in performance management under GPRAMA, and other topics. These 

results will be included in our pending report on agency performance management 

infrastructure. See appendix III for a full list of agencies surveyed. 

10For additional details on our criteria for selecting case study agencies, see appendix I. 

11Every 2 years, GPRAMA requires agency heads from the Chief Financial Officers Act 

agencies, as well as any additional agencies designated by OMB, to identify agency 

priority goals from the performance goals of the agency. These goals should reflect the 

highest priorities of the agency as determined by the agency head. 31 U.S.C. § 

1120(b)(1). 

12See appendix I for a complete list of DOE’s, SBA’s, and Treasury’s 2012 to 2013 APGs. 
 

Page 4 GAO-13-228 Managing for Results 

officials involved in each agency’s performance review process.13 We 

asked to observe at least one review meeting at each agency. Treasury 

allowed us to observe two review meetings—one focused on the Bureau 

of the Fiscal Service and one on the Internal Revenue Service (IRS). 

DOE and SBA did not allow us to observe their meetings, citing concerns 

that our presence could inhibit open discussion. During the interviews, we 

asked officials to identify any challenges to effective implementation they 

faced as the process evolved or any lessons they learned. We also asked 

officials to identify examples of any impacts on performance that they 

attributed to the reviews. 

We conducted our work from April 2012 to February 2013 in accordance 

with generally accepted government auditing standards. Those standards 

require that we plan and perform the audit to obtain sufficient, appropriate 

evidence to provide a reasonable basis for our findings and conclusions 

based on our audit objectives. We believe that the evidence obtained 

provides a reasonable basis for our findings and conclusions based on 

our audit objectives. 

The model for data-driven performance reviews was established in the 

early 1990s by New York City Police Department (NYPD) leadership as a 

strategy to reduce crime. Dubbed “CompStat,” the NYPD’s weekly 

reviews grew from the premise that use of crime data could enable 

leadership to make better-informed, more effective decisions. CompStat 

followed four key tenets: 

1. Accurate and timely intelligence. 

2. Effective tactics. 

3. Rapid deployment. 

4. Relentless follow-up and assessment.14

 

13GPRAMA established in law the Performance Improvement Council to assist OMB with 

topics related to GPRAMA. Council membership includes OMB’s Deputy Director for 

Management as chairperson, performance improvement officers from the 24 Chief 

Financial Officers Act agencies, and others as determined by the Deputy Director of 

Management. 31 U.S.C. § 1124(b). 

14Former NYPD commissioner William Bratton and his deputy commissioner, Jack Maple, 

were the architects of CompStat. See bibliography for additional information on their work. 

Background 
 

Page 5 GAO-13-228 Managing for Results 

As the law enforcement community observed NYPD’s improved 

accountability and bottom-line results in crime reduction, many police 

departments nationwide began to replicate CompStat in their own 

organizations. The CompStat model was subsequently adopted by many 

cities, municipalities, and some states—most notably Washington and 

Maryland—as a general performance management tool. Many of these 

efforts were patterned on the same four tenets as CompStat. 

Nearly two decades later, the Obama administration began to encourage 

the use of data-driven review meetings as a performance management 

tool through several memorandums issued by OMB in 2010 and 2011, 

and a June 2011 executive order.15 During this timeframe, GPRAMA 

introduced the concept of data-driven performance reviews at the federal 

level with a provision that federal agencies conduct quarterly performance 

reviews on progress toward their APGs. Specifically, agencies are 

required to assess how relevant programs and activities contribute to 

achieving APGs; categorize goals by their risk of not being achieved; and 

for those at risk, identify strategies to improve performance. GPRAMA 

also specified that the reviews must occur on at least a quarterly basis 

and involve key leadership and other relevant parties both within and 

outside the agency.16 GPRAMA required agencies to begin conducting 

quarterly performance reviews by June 2011. Some agencies began 

conducting data-driven reviews earlier, in response to the executive 

order, OMB guidance, or other performance management efforts. 

Several efforts aid agencies in implementing the GPRAMA-required 

quarterly performance reviews. The PIC established a working group on 

internal agency performance reviews. The working group meets monthly 

to share leading practices and discuss strategies for improving 

performance. Participation in the working group is voluntary and 

according to OMB, PIOs or designees from 21 agencies across the 

federal government are currently represented. In addition, OMB 

 

15Exec. Order No. 13576, Delivering an Efficient, Effective, and Accountable Government, 

76 Fed. Reg. 35,297 (June 16, 2011). 

16Although the legislation refers to them as “quarterly performance reviews,” agencies 

may conduct these reviews on a regularly occurring basis more frequently than quarterly. 

GPRAMA requires these reviews to be conducted by an agency head and chief operating 

officer among other provisions. 
 

Page 6 GAO-13-228 Managing for Results 

supported performance review implementation by issuing new guidance 

in OMB Circular A-11 during August 2012.17

DOE, SBA, and Treasury each had experience with data-driven 

performance reviews before the June 2011 GPRAMA implementation 

deadline. Officials at both SBA and Treasury said that when new 

leadership came in, they brought interest in data-driven decision making, 

along with key staff that were experienced at data-driven review or data 

analysis. DOE officials said that their experience with the American 

Recovery and Reinvestment Act of 2009 (Recovery Act) work led them to 

begin data-driven performance reviews along with the GPRAMA 

requirements. 

 DOE built on its Recovery Act-related performance reviews with the 

establishment of its quarterly performance reviews, called business 

quarterly reviews, in 2011. According to DOE officials, the Deputy 

Secretary leads the review meetings with participation by under 

secretaries, and each review covers the department’s 8 priority goals, 

15 mission-related key goals, and 8 management and operations key 

goals.18 The Associate Deputy Secretary, who is responsible for 

agency-wide management and operations goals, also participates in 

the reviews. The review meeting follows a structured format, with the 

first half focusing on performance goals and the second half focusing 

on fiscal issues, such as budget execution. 

 SBA began conducting reviews, called quarterly performance reviews, 

during the third quarter of 2009. SBA started by holding separate 

meetings by program area but changed the review format in early 

2010 to include all program offices at each review meeting. According 

to SBA officials, the Deputy Administrator leads SBA’s quarterly 

performance reviews with support from the chief operating officer 

(COO) and the PIO.19 The PIO also assembles the data and 

 

17See OMB Circular No. A-11, at § 270. Our pending report on agency performance 

management infrastructure will examine the role of the PIC. 

18Under GPRAMA, agencies are required to identify a goal leader who is responsible for 

achieving each agency priority goal. 31 U.S.C. § 1120(b)(1)(C). At DOE, agency under 

secretaries typically serve as goal leaders in addition to relevant deputy assistant 

secretaries or program managers. In addition, DOE’s deputy secretary also serves as its 

chief operating officer. 

19SBA’s PIO also serves as its associate administrator for performance management, 

chief financial officer, and chief acquisition officer. 
 

Page 7 GAO-13-228 Managing for Results 

disseminates follow-up action items. Review meetings also include 

senior officials from functional management areas (e.g., information 

technology, human capital, procurement, etc.). In addition to 

discussing agency priority goals, SBA reviews cover the rest of the 

agency’s performance goals and objectives, as time permits. 

 Treasury started conducting department-level performance reviews, 

called Stats, in 2010. Treasury’s reviews, led by the Deputy Secretary, 

are performed on an agency component-by-component basis. 

Reviews also involve senior officials responsible for functional 

management areas such as information technology, financial 

management, and procurement, as well as policy officials.20 The Stat 

process is managed by Treasury’s Office of the Deputy Assistant 

Secretary for Management and Budget, which reports to the PIO.21

The Stats cover Treasury’s two priority goals along with a range of 

other department programmatic and operational goals and priority 

projects. 

 

20At Treasury, officials said that the PIO may lead the meeting in the absence of the 

Deputy Secretary if it is determined that issues to be discussed do not need be addressed 

at the deputy secretary level; otherwise, the review meeting will be rescheduled. 

Treasury’s PIO also serves as the department’s assistant secretary for management and 

the department’s chief financial officer. 

21Treasury’s Office of the Deputy Assistant Secretary for Management and Budget 

includes the Offices of Performance Budgeting and Strategic Planning and Performance 

Improvement. We use the term “performance budgeting staff” in this report to refer to 

managers and staff in these offices. 
 

Page 8 GAO-13-228 Managing for Results 

Our analysis confirmed that to be successful, data-driven reviews should 

be used as a leadership strategy to drive performance improvement.22

Agency leadership must be directly and visibly engaged in the review 

process and invest the time necessary to understand and interpret the 

data being discussed during the meetings. Moreover, GPRAMA requires 

an agency head and deputy head to conduct quarterly priority progress 

reviews which fosters ownership and helps ensure that participants take 

the reviews seriously and that decisions and commitments can be made. 

OMB Circular No. A-11 also emphasizes the importance of leadership 

involvement in quarterly performance reviews, allowing the COO, the 

agency head, or both to conduct the review.23

Our survey of PIOs indicated that agency leadership, with the exception 

of agency heads, actively participated in these reviews. This was 

consistent with what we found at DOE, SBA, and Treasury. The sidebar 

at left shows the survey results on the participation of key positions. 

At Treasury, where we observed two Stat meetings, the Deputy Secretary 

used the Stat meetings to challenge participants to stretch toward 

ambitious performance goals and to provide possible solutions for any 

issues that were discussed. By holding separate reviews for all of 

 

22For an explanation of data-driven performance reviews as a leadership strategy, see, 

Robert Behn, “Designing Performancestat: Or What Are the Key Strategic Choices That a 

Jurisdiction or Agency Must Make When Adapting the CompStat/CitiStat Class of 

Performance Strategies?,” Public Performance & Management Review, vol. 32, no. 2 

(2008): 206-235. 

23Under the section entitled “How should frequent data-driven performance reviews be 

conducted?” the guidance states: “The agency head and/or COO with the support of the 

PIO and his/her office should review with the appropriate goal leader the progress 

achieved during the most recent quarter, overall trend data, and the likelihood of meeting 

the planned level of performance.” OMB Circ. No. A-11, at § 270.5. 

Practices that 

Promote Successful 

Data-Driven 

Performance Reviews 

at the Federal Level 

Agency Leaders Use DataDriven Reviews as a 

Leadership Strategy to 

Drive Performance 

Improvement 
 

Page 9 GAO-13-228 Managing for Results 

Treasury’s bureaus and key offices, the Deputy Secretary committed a 

significant amount of time to these reviews. Other Treasury officials—at 

both the department headquarters and bureau levels—were appreciative 

of the amount of time the Deputy Secretary devoted to the reviews. The 

Deputy Secretary’s high-level position allowed him to speak with clear 

authority in the department and garner the attention of component agency 

leaders on performance issues. For example, during one review meeting 

that we observed, the Deputy Secretary challenged Treasury’s bureau 

leaders to develop new strategies for the department to collect delinquent 

debt payments owed to the federal government. 

We did not observe quarterly performance meetings at DOE or SBA. 

However, officials we interviewed said that top leadership was actively 

driving performance discussions. At DOE, the Deputy Secretary led the 

quarterly review sessions and guided performance discussions with the 

department’s under secretaries on progress made toward achieving their 

respective performance goals. DOE’s Deputy Secretary noted that it is 

important that under secretaries take ownership of the performance 

review process to ensure the reviews are useful. At SBA, the Deputy 

Administrator led the reviews with support from the COO and PIO. 

According to officials, SBA’s leadership asks probing questions of 

program heads concerning SBA’s performance goals. 

Officials from DOE, SBA, and Treasury expressed concern about 

maintaining the continuity of leadership engagement in performance 

review meetings and pointed to ways to help ensure that the review 

process continues with transitions to new agency leadership. For 

example, officials at DOE pointed out that they have a responsibility to 

remind new leadership of the legislative requirement for performance 

reviews as well as OMB’s guidance on these reviews. In addition, SBA 

officials noted that having a mix of career and political leadership involved 

in the reviews can facilitate continuity of the review practices since career 

officials generally span successive administrations. At Treasury, 

performance budgeting staff developed written standard operating 

procedures as a way to document the review process and pave the way 

for new leadership engagement at the department. 

To encourage new leadership to take ownership of the performance 

reviews, several agency officials noted the importance of designing the 

reviews to fit the leadership style and preferences of the incoming leader. 

For example, one official said that it is critical for the incoming leader of 

the review meetings to provide input into the presentation format and 

structure of the review based upon personal preferences. At Treasury, 
 

Page 10 GAO-13-228 Managing for Results 

headquarters staff responsible for managing the performance reviews 

worked closely with the Deputy Secretary to develop a review template 

and meeting format that met his needs. 

Our analysis indicated that performance review meeting participants 

should include high-level leaders and managers with an agency-wide 

perspective as well as those with programmatic knowledge and 

responsibility for the specific performance issues likely to be raised. In 

addition, participants should typically include those with agency-wide 

functional management responsibilities, such as information technology, 

budget, and human capital. This enables the reviews to facilitate problem 

solving by breaking down information silos and providing managers from 

across the agency and other contributing organizations with a forum to 

communicate with each other and identify improvement strategies and 

agree upon specific next steps. At the city and state level, data-driven 

performance reviews typically include senior management from multiple 

agencies. In addition to the benefit of in-person meeting attendance, there 

is also value in having key players participate in other parts of the review 

process, such as the data review and analysis leading up to the meetings 

and the follow-up actions that arise from the meetings. Consistent with 

this practice, OMB’s guidance directs agencies to include, as appropriate, 

relevant personnel from within and outside the agency in the review 

meetings.24

According to our survey, most PIOs—21 of 24—reported that their 

reviews included the participants needed to facilitate problem solving and 

identify performance improvements the majority of the time. Further, 19 

PIOs reported that goal leaders had large involvement in their agency’s 

performance reviews, and 15 PIOs reported that internal contributors to 

agency goals and functional management chiefs, such as the CFO, had 

large involvement in their agency’s reviews. 

At DOE, SBA, and Treasury, officials said they found that including senior 

management responsible for specific mission program areas—as well as 

those with functional management responsibilities in areas such as 

 

24OMB Circ. No. A-11, at § 270.5: “The agency head and/or COO with the support of the 

PIO and his/her office should include, as appropriate, relevant personnel within and 

outside the agency who contribute to the accomplishment of each Agency Priority Goal (or 

other priority).” 

Key Players Attend 

Reviews to Facilitate 

Problem Solving 
 

Page 11 GAO-13-228 Managing for Results 

budget, information technology, human capital, procurement, and legal 

counsel—enhanced performance improvement efforts at review 

meetings. For example, both DOE and Treasury officials said discussions 

with budget officials concerning performance goals provided an 

opportunity for performance issues to be discussed in the context of 

budget issues where relevant. 

However, differences in the scope of the reviews influenced which key 

players attended the review meetings. For instance, while Treasury’s 

reviews focused primarily on a specific component organization, such as 

a bureau or policy office, representatives from multiple components 

attended the review meetings when the achievement of a performance 

goal crossed component lines, or when the components had other 

commonalities. One Treasury official noted that the Bureau of Engraving 

and Printing and the U.S. Mint were asked to sit in on each others’ 

sessions, as they both work under the Treasurer and have similar 

operations. 

Treasury officials indicated that one of the reasons why they chose to 

focus their performance reviews on individual component agencies—such 

as IRS and the Bureau of the Fiscal Service—was that their performance 

goals are generally aligned by component rather than shared by 

components. In addition, according to a senior Treasury official, focusing 

reviews on components allows time for more in-depth reviews. Treasury’s 

reviews include officials with department-level responsibilities in functional 

management areas such as budget, procurement, and legal counsel who 

can contribute to problem solving or who may be called on to take followup actions. In addition, Treasury officials noted that component-bycomponent reviews enable Treasury not only to have attendance from the 

bureau head, but also from more members of the bureau’s leadership 

team, such as the bureau CFO and others. 

Officials said that if reviews were held with all of Treasury’s components 

at one time it would be impractical to gather all key members of bureau 

leadership because of the limited time which would be available for 

discussion on any particular topic. Treasury officials said that their 

practice of using the same information template for the Stat review with 

each component enables the reviews to cover similar issues across 

components and identify actions that should be addressed collaboratively, 

even though all components do not participate in the review meetings at 

the same time. 
 

Page 12 GAO-13-228 Managing for Results 

Conversely, at DOE and SBA, each review meeting included leadership 

participation from across all agency mission areas. However, DOE and 

SBA were similar to Treasury in that they included officials with agencywide functional management responsibilities in the reviews. Officials 

noted that DOE’s reviews provided an opportunity for senior leadership to 

discuss the context around their performance goals and improve results 

with help from other areas of the department. In addition, the presence of 

senior officials across DOE’s programs provided an opportunity to share 

leading practices in various areas across program lines. At SBA, officials 

noted that many senior career officials attending these reviews had 

experience managing multiple programs and their broader experiences 

helped them understand and identify relevant performance metrics across 

program areas. In addition, officials mentioned that, in contrast to a large 

department with many discrete mission areas, SBA’s programs are 

focused on achieving a relatively narrow mission of supporting small 

businesses, which creates more opportunities for cross-program 

collaboration. 

According to our survey results, PIOs did not see getting the right 

personnel included in the meetings as a challenge: Only 2 of 24 PIOs 

reported a challenge in including those managers or staff needed to 

facilitate problem solving and identify improvement opportunities. 

However, officials from DOE, SBA, and Treasury did note challenges in 

getting the right mix and number of participants to most effectively 

facilitate performance improvement efforts at the meetings. See figure 1. 
Interactive graphic Figure 1: Finding the Right Meeting Size and Composition for Effective Performance Reviews 

Page 13 GAO-13-228 Managing for Results

A print version of the text contained in this interactive graphic is available in appendix VI.

Sources: GAO (information);Treasury (photos).

Print instructions

In addition, balancing depth of knowledge and numbers of staff attending these reviews is

another consideration for finding the right meeting size. One official pointed out that having a 

relatively small group can help participants feel more comfortable in revealing performance 

problems but that there is a risk of leaving out key players that need to be part of a performance 

solution. 

Officials from each agency described different approaches to finding the right meeting size and 

composition for effective performance reviews, with several officials acknowledging that it takes 

time to get it right.

DOE

Main participants: 7

Additional attendees: 10-12

Treasury Deputy Secretary and COO Neal Wolin (top) leads the department’s quarterly performance reviews, with 

support from Assistant Secretary for Management Nani Coloretti, who also serves as Treasury’s PIO.

Several officials noted 

challenges in achieving the 

proper balance between 

having a small number of 

meeting participants, which 

can allow more air time for 

each participant, 

compared to a larger group 

which might not be a 

productive use of time for 

those attending.

Rollover each table diagram 

below to see additional 

details about how DOE, 

SBA, and Treasury conduct 

their performance evaluation 

meetings.

Directions:
 

Page 14 GAO-13-228 Managing for Results 

According to survey results, 16 of 24 PIOs indicated that there was little to 

no involvement in the reviews from external officials who contribute to the 

achievement of agency goals. This is consistent with what we found at 

DOE, SBA, and Treasury, as stakeholders from outside the agencies did 

not participate in their performance review meetings. 

At Treasury, officials said they addressed performance issues requiring 

external collaboration by conducting follow-up meetings with relevant 

external participants. These meetings were scheduled as part of 

Treasury’s practice of following up on issues raised during the review 

sessions. For instance, during a Treasury review meeting we observed, 

the Deputy Secretary directed staff to arrange a meeting with OMB to 

help address an obstacle to achieving a performance goal. 

Likewise, DOE and SBA officials said they undertook collaborative 

actions as a result of discussions in their quarterly performance reviews, 

but neither planned to invite external representatives to their meetings. 

For example, SBA’s reviews led to multiple discussions with other federal 

agencies on whether some of their government contracts that were going 

to large businesses could go to small businesses instead. 

Officials we interviewed cited several concerns that may explain why, at 

present, agencies are generally not including external participants—from 

other federal agencies or other relevant organizations—in their reviews. 

First, officials did not include external stakeholders because they wanted 

to keep discussions focused on internal problem-solving and were 

concerned that including external parties might inhibit open discussions 

on performance issues. Second, reviews at DOE, SBA, and Treasury 

mainly focused on goals achieved through internal contributors at each 

agency or office and officials noted that it would not currently be an 

efficient use of time to include external parties, even though external 

issues are discussed. One official said that the logistics of including highranking agency managers from other agencies could make it difficult to 

schedule review meetings on a timely basis. Another official who had 

experience managing data-driven performance reviews at different levels 

of government noted that city- and state-level reviews tend to be run by a 

mayor or governor with direct authority over the various agencies that 

participate in the reviews. This official pointed out that circumstances are 

different at the federal level, where an agency head could invite but not 

require outside participation and would not have control over the 

information shared or whether follow-up action was carried out. 
 

Page 15 GAO-13-228 Managing for Results 

However, we have previously reported that agencies can collaborate 

more effectively across organizational lines when presented with a clear 

and compelling rationale to do so and when agency leaders demonstrate 

their commitment to working collaboratively.25 Agency goals that require 

the efforts of more than one agency could serve as such a compelling 

rationale—even in the absence of direct authority requiring such 

collaboration. Moreover, our prior work has shown that agencies which 

participated in various planning and decision-making forums together—

such as interagency councils or planning bodies—reported that such 

interactions contributed to achieving their goals. Specifically, agencies 

reported that such participation opened lines of communication, fostered 

trust, and helped build relationships, which can in turn lead to more 

effective collaboration across agency lines.26 Despite the concerns that 

DOE, SBA, and Treasury raised about including external participants in 

their reviews, our survey results indicate that some agencies are doing 

so: 4 of 24 PIOs reported moderate to large involvement of external 

officials who contribute to the achievement of agency goals. In addition, 

OMB officials provided an example of two agencies which have been 

successfully making use of quarterly performance reviews to collaborate 

on their APGs. These officials told us that the Departments of Housing 

and Urban Development and Veterans Affairs—which both contribute to 

efforts to reduce veterans’ homelessness—had conducted several Stat 

meetings jointly. According to OMB officials, program staff members from 

both agencies regularly participate in HUD Stat meetings, where they 

jointly analyze performance data to understand trends, identify best 

practices, and prioritize the actions needed to achieve veteran 

homelessness goals. Officials reported that these collaborative meetings 

have contributed to better outcomes. Moreover, officials from both OMB 

and the PIC indicated that agencies have increasingly been observing 

others’ review meetings as a means of learning about different practices 

with no apparent harm to the effectiveness of the meetings. This suggests 

that the challenges, if any, to outside participation can be overcome. 

 

25GAO, Managing for Results: Key Considerations for Implementing Interagency 

Collaborative Mechanisms, GAO-12-1022 (Washington, D.C.: Sept. 27, 2012); and 

Results-Oriented Government: Practices That Can Help Enhance and Sustain 

Collaboration among Federal Agencies, GAO-06-15 (Washington, D.C.: Oct. 21, 2005). 

26GAO-12-1022. 
 

Page 16 GAO-13-228 Managing for Results 

While there are many approaches to managing performance to achieve 

goals that rely on multiple agencies, few are likely to provide the benefit of 

bringing together the leadership and all the key players to solve problems 

and motivate performance improvement. Moreover, when key players are 

excluded from quarterly performance reviews, agencies may be missing 

opportunities to have all the relevant parties participate in developing 

solutions to performance problems. Instead, agencies will need to rely on 

potentially duplicative parallel coordination mechanisms, which could 

result in less than optimal performance improvement strategies. 

Our analysis showed that quarterly performance reviews should be used 

to align an organization’s resources, programs, and activities to ensure 

they are contributing to the achievement of agency goals. To help ensure 

that reviews are focusing on the appropriate interim goals and measures, 

agencies can develop models that describe the logical relationship 

between an agency’s inputs, activities, outputs, and outcomes. These 

logical relationships, sometimes called logic models, should be 

periodically assessed to determine if outcomes are being achieved as 

expected, and should be revised if necessary. 

Our survey results indicated that PIOs do not find goal alignment to be a 

challenging aspect of implementing quarterly performance reviews, with 

13 of 24 PIOs reporting that ensuring alignment between performance 

reviews and strategic goals and performance objectives was easy. Only 

one PIO reported this as being a challenge. Moreover, 22 PIOs reported 

that the majority of their reviews are aligned with strategic goals and 

performance objectives. 

Consistent with our survey results, we found that DOE, SBA, and 

Treasury had selected performance metrics, initiatives, and other areas of 

focus in their reviews that were linked to the accomplishment of APGs 

and other goals, such as strategic plan objectives and key operational 

goals. Officials from both DOE and Treasury described the processes 

they undertook to choose useful performance information to frame the 

performance review discussion. For example, DOE narrowed its list of 

more than 190 performance measures to provide leadership with a 

focused view of the department’s key goals, while also providing sufficient 

depth of information to be meaningful. Treasury officials responsible for 

designing the department’s quarterly performance reviews described 

extensive interactions with the Deputy Secretary and the bureaus to 

identify performance measures that could be used to promote discussion 

of performance issues and opportunities for improvement. 

Reviews Ensure Alignment 

between Agency Goals, 

Program Activities, and 

Resources 
 

Page 17 GAO-13-228 Managing for Results 

In addition, the three agencies described efforts to use logic models, 

project milestones, and other approaches to identify early information on 

how they were progressing toward long-term outcome goals, which 

officials said could be challenging to monitor. For example, DOE’s Office 

of Energy Efficiency and Renewable Energy created a template for 

program managers to develop logic models linking program inputs and 

outputs to longer-term performance objectives, such as achieving clean 

generation of 80 percent of the nation’s electricity by 2035. DOE identified 

outputs or intermediate outcomes that contribute to the clean generation 

of energy, such as reducing the cost of solar energy, which the 

department measures to indicate progress toward its 2035 goal. 

According to officials, logic models helped program staff communicate a 

coherent story about how the program’s key activities contribute to its 

goals. 

At Treasury, performance budgeting staff said they included the status of 

priority projects—such as IRS plans to develop a streamlined, userfriendly website—in its quarterly performance review information (see 

figure 2). Officials explained that these priority projects were designated 

as such because Treasury sees them as the critical path to achieving 

agency priority goals and other key longer-term outcomes, which could 

not always be tracked on a quarterly basis. For example, the Deputy 

Secretary wanted to monitor the status of IRS’s website update because 

it is seen to be a lever toward the agency’s priority goal to improve the 

voluntary tax compliance rate, which can only be measured with data that 

lags by approximately five years. 
 

Page 18 GAO-13-228 Managing for Results 

Figure 2: Treasury Linked Shorter-Term Priority Projects to Achievement of Goals in its Quarterly Performance Reviews 
 

Page 19 GAO-13-228 Managing for Results 

Our analysis showed that because data-driven reviews are to foster 

improved performance, the focus of accountability should be on the 

responsible manager’s role in addressing problems and bringing about 

positive change. Agency leaders should hold goal leaders and other 

responsible managers accountable for knowing the progress being made 

in achieving goals and, if progress is insufficient, understanding why and 

having a plan for improvement. If data is insufficient for gauging progress, 

managers should be held accountable for improving the quality of the 

data so that it is sufficient for decision making. Managers should also be 

held accountable for identifying and replicating effective practices to 

improve performance. In addition, the goals addressed in the reviews 

should be aligned with managers’ and staff’s individual performance goals 

to create a line of sight that reinforces the connection between strategic 

goals and day-to-day activities of managers and staff.27

GPRAMA introduced specific roles and responsibilities for agency heads, 

COOs, PIOs, and goal leaders in conducting quarterly performance 

reviews.28 For each APG, agency heads and COOs, with support from the 

PIO, must: 

 review with the appropriate goal leader the progress achieved during 

the most recent quarter, overall trend data, and the likelihood of 

meeting the planned level of performance; 

 assess whether relevant organizations, program activities, 

regulations, policies, and other activities are contributing as planned 

to agency priority goals; 

 categorize agency priority goals by risk of not achieving the planned 

level of performance; and 

 identify prospects and strategies for performance improvement, 

including any needed changes to agency program activities, 

regulations, policies, or other activities for agency priority goals at 

greatest risk of not meeting the planned level of performance. 

 

27GAO, Results-Oriented Cultures: Creating a Clear Linkage between Individual 

Performance and Organizational Success, GAO-03-488 (Washington, D.C.: Mar. 14, 

2003). 

2831 U.S.C. § 1121(b). 

Agency Leaders Hold 

Managers Accountable for 

Diagnosing Performance 

Problems and Identifying 

Strategies for 

Improvement 
 

Page 20 GAO-13-228 Managing for Results 

Across the government, a majority of PIOs reported that their agency’s 

reviews met these GPRAMA requirements, as indicated in figure 3. 

Figure 3: Number of PIOs Reporting That Their Agency’s Reviews Included Selected GPRAMA Requirements More than Half 

the Time 

Our survey results indicated that most PIOs—21 of 24—reported using 

the reviews to identify actionable opportunities for performance 

improvement at least half the time. 

Consistent with the survey results, DOE, SBA, and Treasury reported that 

top agency leadership held officials accountable for identifying 

performance problems and opportunities for improvement. For example, 

Treasury officials said—and the sessions we observed confirmed—that 

their Stat meetings focus on performance and are a vehicle for the 

Deputy Secretary to challenge bureau heads to ensure their bureaus 

continually improve. For example, the Deputy Secretary reported that 

many of the bureaus had not reviewed their management metrics, such 

as internal controls, diversity issues and employee survey scores, for 

some time and the Stat reviews are an opportunity to engage the bureaus 

on these issues. At the Stat reviews we attended, we observed the 

Deputy Secretary discussing such management metrics. For example, he 

asked bureau leadership to explain why their scores on a governmentwide employee satisfaction survey had dipped during the past year, 

questioned a decline in survey response rates, and discussed specific 

next steps that he could take to support plans for improvement. 

At DOE, officials said that their quarterly performance reviews focused on 

areas where they were not on track to meet performance goals and ways 
 

Page 21 GAO-13-228 Managing for Results 

to address this. One official at DOE noted that the Deputy Secretary 

expected the under secretaries to be accountable for all of their goals at 

the meeting. At SBA, officials said the focus of the meetings is to fix 

problems and that it is important to be prepared because the 

Administrator asks probing questions and managers are called out when 

progress is not being made and asked to explain what is being done to 

resolve the issues. 

Each agency also reported taking steps to ensure that managers’ 

individual performance objectives are aligned with priority and other 

agency performance goals. For example, SBA’s performance agreements 

incorporate performance objectives which cascade from the agency’s 

priority goals and other performance goals and are aligned with its 

strategic plan. Samples of performance agreements we reviewed 

identified “performance elements linked to organizational goals.” For 

example, a district manager had a performance objective to hold outreach 

events to connect small businesses to contracting opportunities. This 

objective clearly links to SBA’s APG to increase small business 

participation in government contracting. 

Our analysis indicated that the capacity to collect and analyze accurate, 

useful, and timely data is critical to successful data-driven reviews. 

Agencies should track both outputs and outcomes. Agencies should also 

look for opportunities to leverage data produced by other agency 

components or outside entities. In addition, having the capacity to 

disaggregate data according to demographic, geographic, or other 

relevant characteristics can aid in highlighting significant variation, which 

can help meeting participants to pinpoint problems and identify solutions. 

Agencies also need to plan for the time and resources required to 

generate and communicate performance data in a timely manner. Easy 

access to relevant databases and systems-generated analysis, such as 

providing analysts with the ability to develop performance reports without 

relying on information technology staff, can streamline the data collection 

and analysis processes. 

While having accurate, timely, and useful data available is critical to 

successful performance reviews, 16 of 24 PIOs reported that this was a 

challenge—more than any other practice we asked about. However, all 

16 of those PIOs also responded that accurate, timely, and useful data is 

available for their agency’s reviews about half the time or more, which 

may indicate that some agencies have found ways to address this 

challenge. 

Agency Has Capacity to 

Collect Accurate, Useful, 

and Timely Performance 

Data 
 

Page 22 GAO-13-228 Managing for Results 

Our review of DOE, SBA, and Treasury illustrates how agencies can 

overcome some challenges to data availability. Our analysis of quarterly 

performance review documents indicated that each agency was 

producing data-rich analyses that identified trends and potential 

performance issues. However, agency officials described initial 

challenges in these areas and said that improving their capacities for data 

collection and analysis took time. For example, SBA’s Office of 

Government Contracting and Business Development collects data on the 

percentage of all federal agency contracts being awarded to small 

businesses.29 The office is dependent on a General Services 

Administration database, the Federal Procurement Database SystemNext Generation (FPDS-NG), for its information. SBA officials were 

concerned about the quality of the data since each federal agency enters 

its own information. To address this concern, SBA officials said they 

provided agencies with individualized reports of potential anomalies in 

their small business contracting data. This process allowed agencies to 

verify and correct if necessary the anomalies before SBA published the 

annual Small Business Procurement Scorecard report.30 For example, if 

an agency listed a contract in FPDS-NG as a small business set-aside at 

the same time that the agency listed the contract as an open procurement 

competition, this would be flagged. SBA would then notify the responsible 

agency and give it an opportunity to correct the anomaly in FPDS-NG. In 

addition, SBA noted instances where performance data lag behind the 

performance review cycle. For example, the Department of Defense holds 

its procurement data back from FPDS-NG for one quarter for national 

security purposes. SBA officials said that instead of waiting for the next 

quarter, they obtain preliminary information from the Department of 

Defense. 

 

29Specifically, the mission of SBA’s Office of Government Contracting and Business 

Development is to create an environment for maximum participation by small, 

disadvantaged, service-disabled veteran-owned, woman-owned, and HUBZone 

businesses in federal government contract awards and large prime subcontract awards. 

30The annual scorecard is an assessment tool to (1) measure how well federal agencies 

reach their small business and socioeconomic prime contracting and subcontracting 

goals, (2) provide accurate and transparent contracting data, and (3) report agencyspecific progress. 
 

Page 23 GAO-13-228 Managing for Results 

Our analysis indicated that agencies need staff with the skills to assess 

performance data for coverage and quality and to identify key trends, 

areas of strong or weak performance, and possible causal factors. In 

addition, those messages need to be effectively communicated to 

management and staff that will play a role in identifying and solving 

performance problems and making related decisions. Analysts and 

managers should carefully consider the type and amount of information 

that will be useful for performance reviews, as well as how to present the 

information to audiences with varying levels of technical or quantitative 

skills. Providing the right amount of easy-to-understand performance 

information can promote effective decision making during the quarterly 

performance reviews. For example, focusing the presentation on the 

message the data tell about performance, using well-designed graphics, 

and grounding the data in relevant context are effective communication 

techniques. 

Our PIO survey results included information on 15 specific competencies 

associated with performance improvement responsibilities.31 For each of 

these competencies, the majority of PIOs reported the competencies 

were present among performance improvement staff to a large extent. 

However, survey results were less positive about 2 competencies 

specifically related to analytic abilities. Of the 24 PIOs, 9 reported that 

performance measurement competencies and 10 reported that 

organizational performance analysis competencies were present among 

their performance improvement staff to a small or moderate extent. 

PIOs at DOE, SBA, and Treasury each described the teams they had 

assembled to support their performance improvement efforts. For 

example, SBA’s Deputy PIO had performance analysts to support the 

quarterly performance reviews and many other performance management 

activities, such as the production of a weekly dashboard of key 

performance metrics. However, SBA officials acknowledged that some 

staff were less comfortable working with data and they perceived this as a 

skills gap that needed to be addressed. These officials said they are 

 

31GPRAMA required the Office of Personnel Management to identify key skills and 

competencies needed by federal government personnel for developing goals, evaluating 

programs, and analyzing and using performance information for the purpose of improving 

government efficiency and effectiveness. The Office of Personnel Management identified 

the 15 competencies specifically related to performance improvement staff; for survey 

results on all of these, see appendix II. 

Agency Staff Have Skills to 

Analyze and Clearly 

Communicate Complex 

Data for Decision Making 
 

Page 24 GAO-13-228 Managing for Results 

addressing this through a combination of training and hiring. For example, 

as part of its leadership training, SBA began developing courses related 

to “decision support;” officials said the courses were designed to lead to 

competencies in spreadsheet development and analysis, presentation 

delivery, development of decision support datasets, and other analytic 

and presentation skills. Participants began training in late summer of 

2012 with courses titled Principles of Analytics and Analytic Boot Camp. 

Having staff with abilities to communicate analyses effectively is an 

important factor in successful performance reviews, and most PIOs—22 

of 24—reported that data and relevant analyses are presented effectively 

to participants in their agency’s reviews about half the time or more. 

However, 11 of those 22 PIOs also reported that effective presentation of 

data and relevant analysis was challenging—the second largest 

challenge cited by the PIOs among the practices we asked about. 

Consistent with our survey results, officials we interviewed at SBA and 

Treasury described the challenges they faced in developing skills sets 

that bridge the gap between data analysis and effective communication. 

At SBA, the Office of Performance Management developed internal 

training to help SBA managers improve their ability to communicate the 

message that the data suggest. One official recounted initial struggles to 

interpret data and then effectively communicate the key points relevant to 

performance improvement to those who were not analysts. He noted that 

the SBA Administrator told senior management that a “data dump” was 

not helpful, which helped them to realize what was needed. Training was 

developed to move managers and staff beyond basic analytic skills, with 

a focus on structuring presentations effectively, using data to drive 

management decisions, and in general, “telling your story so you’re 

drawing out insights, rather than just summarizing facts.” See figure 4. 
 

Page 25 GAO-13-228 Managing for Results 

Figure 4: Example of Materials SBA Used to Train Managers and Staff to Effectively Communicate Data Analysis to Decision 

Makers 
 

Page 26 GAO-13-228 Managing for Results 

At Treasury, performance budgeting staff developed a PowerPoint 

presentation template that was distributed to each bureau to complete in 

advance of the Stat meetings. The template provided a uniform data 

collection tool that incorporated data presentation design principles, to 

guide the bureaus in effectively communicating their message to the 

Deputy Secretary. For example, templates for line charts prompted 

bureaus to indicate whether the desired trend line direction was up or 

down, since this is not always immediately apparent to high-ranking 

reviewers who may not have the depth of background into the particular 

program or operation. According to a Treasury official, designing an 

effective presentation is as important as doing relevant, high-quality data 

analysis. One official pointed out, “If no one reads or understands the 

analysis, it doesn’t matter how good it was.” 

SBA and Treasury officials responsible for managing the reviews also 

described challenges in balancing presentation uniformity with the need 

to provide context that varies. These officials noted that consistency was 

key to making performance information quick and easy to absorb, 

especially for leadership that has limited time to review such information. 

However, SBA noted that the down side of consistency is people’s 

tendencies to tune out information that appears to be repetitive. SBA’s 

PIO and COO said they have to continually look for ways to keep the 

performance review meetings engaging to participants and that “meeting 

fatigue” can be a problem. Further, several bureau officials at Treasury 

we interviewed said that while they understood the need for uniformity, 

the templates did not always provide them with enough flexibility to 

provide sufficient context for their performance information. While bureau 

officials we interviewed said that the process and template had improved 

over time, some felt that in the early days of the Stat reviews, they were 

so limited in their ability to “tell their story” to the Deputy Secretary that 

they did not think he was getting an accurate understanding of the issues. 

However, our review of multiple Stat documents indicated that there was 

a specific page in the template left open for issues the bureaus wanted to 

raise, and further, that some bureaus appended information to the 

template to provide additional context. 

Our analysis found that sufficient preparation for the performance review 

meeting is critical for a successful review. Key participants must be 

prepared to discuss agenda items related to their performance measures 

and progress toward goals as well as any other issues to be addressed. 

The time allotted to prepare for reviews also provides a prompt for 

participants to continuously update their performance data, assess 

Rigorous Preparations 

Enable Meaningful 

Performance Discussions 
 

Page 27 GAO-13-228 Managing for Results 

progress toward their performance goals, and develop a response to any 

performance issues identified. Also, data to be presented during the 

reviews must be fully vetted prior to the meeting so that participants can 

focus discussions on data trends and analysis rather than on whether the 

data itself is correct. 

According to our survey results, 22 of 24 PIOs reported that review 

participants are adequately prepared for performance reviews more than 

half the time, and 20 reported that, overall, it is not challenging for 

participants to be prepared for the reviews. Nevertheless, several agency 

officials from DOE, SBA, and Treasury said that there was a significant 

time investment in preparing information and coordinating among 

managers and analysts across headquarters and components or offices. 

Officials at Treasury noted that the process of preparing for the reviews 

forced the department and its component agencies to closely examine 

performance data and make sure they could explain it to the Deputy 

Secretary, and said this process was a valuable part of the performance 

review. As one bureau-level official explained, nobody wants to go before 

the Deputy Secretary with data that indicates a performance problem, 

unless they are able to explain the issue and show that they have already 

thought of strategies for improvement. As a result, preparing for the 

reviews sometimes prompted participants to conduct additional analysis 

and have advance discussions on how to address performance problems. 

At DOE, SBA, and Treasury, we found several practices in place to 

prepare participants for review meetings. Officials at each agency 

stressed the importance of meeting preparation to ensure that the review 

sessions were productive. For example, Treasury employed a rigorous 

pre-meeting process which started with the performance budgeting staff 

developing a PowerPoint template, in consultation with the Deputy 

Secretary, specifying the performance information to be provided by each 

component agency for its Stat session. Treasury officials said that 

performance budgeting staff then met with component staff to discuss the 

new template and any changes. The templates we reviewed were 

organized into several categories, such as priority projects, management 

metrics, and other issues, and were distributed in advance of the Stat 

meeting and used as the meeting agenda. Treasury officials said there 

were typically several rounds of revisions to the PowerPoint template 

prior to the review session, with management and analysts at the 

component level coordinating with their counterparts at the department. 

Officials said that one of the goals of developing the template was to 

ensure that all participants are fully prepared and able to engage in 

meaningful discussions about performance. In particular, one official 
 

Page 28 GAO-13-228 Managing for Results 

explained that a guiding principle is that none of the participants should 

ever be surprised by any of the topics to be discussed. In advance of 

each review session, Treasury’s Deputy Secretary reviews the completed 

document along with an explanatory briefing memo prepared by 

performance budgeting staff, which provides relevant context for any 

issues, suggests lines of questioning, and highlights particular decisions 

to be made. Performance budgeting staff also review the completed Stat 

template with component staff to discuss the contents and ensure that the 

component is aware of issues likely to be brought to the attention of the 

Deputy Secretary. Treasury’s senior officials emphasized the importance 

of ensuring that data discussed at the meeting were sufficiently vetted 

during meeting preparation so as not to spend time during the sessions 

determining whether the information is correct. Several officials pointed to 

instances in early review meetings that used valuable meeting time on 

data issues because department-level and component staff disagreed on 

baseline data used during the reviews. 

DOE’s pre-meeting practices included briefings with the Deputy Secretary 

and under secretaries and development of a Business Quarterly Review 

binder that included descriptions of the program offices’ performance and 

budget information, a list of attendees, and background notes, as well as 

a list of actions from the previous performance review. 

Our analysis showed that in-person meetings which are both frequent and 

regularly scheduled are a defining characteristic of data-driven reviews. 

Regularly scheduled meetings foster a culture of performance 

management and continuous improvement. The frequency of the meeting 

schedule should depend on the urgency of the problems to be fixed, 

frequency with which the data are collected, and speed with which 

agency action can have an impact on these data. GPRAMA requires that 

starting no later than June 2011, agencies must conduct meetings at least 

quarterly, but agencies may meet more frequently if it meets their needs. 

According to our survey, all 24 of the PIOs reported that their agencies 

were conducting performance reviews at least quarterly, with 7 of those 

24 reporting conducting reviews more frequently. At DOE, SBA, and 

Treasury, we found each agency had frequent, regularly scheduled 

performance review meetings. Treasury’s Deputy Secretary met 

separately with each of the bureaus two or three times a year. Generally, 

one or two of the meetings with each Treasury bureau focused on 

performance, and the other meetings focused on budget. Treasury also 

holds Stat meetings on certain department-wide goals related to human 

Reviews Are Conducted on 

a Frequent and Regularly 

Scheduled Basis 
 

Page 29 GAO-13-228 Managing for Results 

capital, procurement, and strategic sourcing. As a result, Treasury 

conducted more than three dozen performance review meetings 

throughout the year, with more than 100 reviews conducted as of 

November 2012. DOE and SBA both met quarterly with representatives 

from across their entire departments. 

Our survey results indicated that while holding performance reviews as 

scheduled is generally occurring, it may present challenges at some 

agencies. Of 24 PIOs, 20 reported that their agency held the performance 

reviews as scheduled more than half the time, with 2 of these PIOs 

reporting scheduling as a challenge. In addition, the 4 PIOs who reported 

that this practice was occurring about half of the time or less also reported 

that it was a challenge. 

Experiences at Treasury illustrated how it can be challenging to schedule 

performance review meetings with high-ranking officials. At one bureau, 

officials said that last-minute cancellations due to the Deputy Secretary’s 

schedule caused inefficiencies since extensive meeting preparations, 

including performance data analyses, had to be redone each time. 

Officials from this bureau also commented that the performance review 

meetings sometimes coincided with times during which the bureau had 

heavy workloads. The officials noted that scheduling all of the meetings at 

the beginning of the year would be helpful so that they could plan around 

them. According to Treasury officials, currently sessions are scheduled 4 

to 8 weeks in advance. 

SBA found that it was best to schedule a standing date for the 

performance review meetings, which are held on the third Tuesday after 

the end of every quarter, rather than try to coordinate with numerous 

senior officials’ schedules every time. By selecting this meeting time, SBA 

was also able to leverage an existing weekly operations meeting, rather 

than scheduling a separate meeting for the quarterly performance review. 

Rigorous and sustained follow-up on issues identified during the meetings 

is also critical to ensure the success of the reviews as a performance 

improvement tool. Important follow-up activities include identifying the 

individual or office responsible for each action item as well as who will be 

monitoring the follow-up. Follow-up actions should be included as agenda 

items for subsequent reviews to hold responsible officials accountable for 

addressing the issues raised and communicating what was done. 

Participants Engage in 

Rigorous and Sustained 

Follow-up on Issues 

Identified During Reviews 
 

Page 30 GAO-13-228 Managing for Results 

According to our survey results, follow-up activities are generally 

occurring even though some PIOs reported that this practice was 

challenging. In particular, 21 of 24 PIOs reported that follow-up activities 

occurred at their agencies more than half the time. Although 7 PIOs 

reported that follow-up activities were challenging, 5 of those 7 PIOs also 

reported that this practice was occurring more than half the time, which 

may indicate that some agencies have found ways to overcome these 

challenges. 

DOE, SBA, and Treasury each had a different approach to ensure that 

follow-up items were carried out as agreed to at the review meetings and 

to ensure responsible parties were held accountable. At DOE, postmeeting activities included asking program offices questions that were 

generated by their review and assigning an analyst from headquarters 

performance staff to work with mission program staff to prepare answers 

to these follow-up questions and have them ready for the next 

performance review meeting. According to an SBA official, follow-up 

action items were typically discussed at weekly operations meetings, 

which helped officials to integrate their action plans into these other 

performance discussions. One example of an SBA follow-up item was to 

develop new strategies related to increasing federal contracts with small 

businesses. Officials said they ranked agencies by the total dollar value of 

contracts they issued, and they targeted procurement representatives at 

the top seven purchasing agencies for their outreach efforts. 

At Treasury, its performance budgeting staff generated follow-up 

memorandums immediately after the review meetings, naming action 

items, responsible parties, and due dates. The status of follow-up items 

from the previous review meetings was also incorporated into the 

materials for the next review meeting so the Deputy Secretary could see if 

there was any lagging action. In addition, performance budgeting staff 

tracked overall performance on post-review follow-up to ensure that this 

part of the process was being managed effectively (see figure 5). 
 

Page 31 GAO-13-228 Managing for Results 

Figure 5: Treasury Tracked its Own Performance on Completing Follow-up Items 

Several officials from Treasury and SBA noted the importance of 

receiving feedback from meeting participants to help improve the review 

process. For example, Treasury conducted a formal feedback meeting 

with bureau heads after the first round of review meetings and learned 

that components wanted more data from the department’s performance 

staff to help substantiate their program performance. More recently, 

Treasury developed a survey to obtain formal feedback from participants 

and other management and staff that contribute to the review process. 

The survey included questions on the amount of time invested in 

preparation and follow-up and asked respondents to rate their satisfaction 

with various aspects of the review process, including the template design 

and the guidance provided by performance budget staff. Respondents 

were also asked to rate the importance of the various aspects of the 

review to their bureau or office, among other questions. According to 

Treasury officials who manage the Stat reviews, they have made specific 

improvements based on the survey results, such as sending out 

templates earlier to give components more time to prepare. 

SBA also solicited feedback from meeting participants and made some 

changes as a result. For example, an SBA official said that the agency 

used to schedule its quarterly performance review meeting after all the 

data were available, which was 45 days after the end of each quarter. 

However, the official said that participants wanted the meeting to be held 

when the majority of the data was available—only one of its offices’ data 
 

Page 32 GAO-13-228 Managing for Results 

lagged—to allow them to take any needed action on a timely basis. As a 

result, the officials said SBA started scheduling the quarterly review 

meeting on the third Tuesday after the end of each quarter and 

conducting a separate meeting with the one SBA office that receives its 

data after that date.32

DOE, SBA, and Treasury officials said their quarterly performance 

reviews allowed different functional management groups and program 

areas within each agency to share information and ideas for performance 

improvement. Officials said these reviews helped them to solve problems 

that were impeding progress toward performance goals or to develop new 

performance improvement strategies. In some cases, officials were able 

to point to specific performance improvements that they attributed to the 

reviews. 

DOE officials said that quarterly reviews helped them take a more critical 

look at subordinate activities contributing to the achievement of their 

 

32According to SBA officials, small businesses can submit their data to the SBA Office of 

Entrepreneurial Development up to 30 days after the end of each quarter, and the office 

needs additional time to process the data before the quarterly performance review 

meeting. 

DOE, SBA, and 

Treasury Officials 

Attributed 

Improvements in 

Performance and 

Decision Making to 

Quarterly 

Performance 

Reviews, although 

Some Reported 

Minimal Impact 

Reviews Fostered Intraagency Collaboration and 

Problem Solving to 

Improve Performance 
 

Page 33 GAO-13-228 Managing for Results 

APGs. For instance, they examined barriers to achieving their 

weatherization goals and found they could produce better results by 

targeting not only individual homes but also multiuse buildings for 

retrofitting. Therefore, they created a program to leverage resources from 

nonprofit and private sector organizations focused on large scale retrofit 

projects for buildings. In another example, DOE identified barriers to 

achieving its solar energy cost reduction APG related to slow local 

permitting processes for solar installations along with other local-level 

activities that contributed toward the goal. To address these barriers, 

DOE funded a new program called the Rooftop Solar Challenge in which 

teams develop actions plans to standardize permit processes, update 

planning and zoning codes, improve standards for connecting solar power 

to the electric grid, and increase access to financing. Officials said they 

expected these new approaches to improve DOE’s performance in 

reducing solar energy costs. 

According to officials, performance reviews at DOE also facilitated 

information sharing across the agency that led to better results. For 

example, officials said discussions at DOE’s quarterly performance 

reviews led offices to share effective procurement practices. In this case, 

offices have been sharing best practices related to strategic sourcing to 

identify areas of cost reduction.33 For example, a recent reorganization 

between Environmental Management and National Nuclear Security 

Administration presented an opportunity for Environmental Management 

to leverage existing capabilities in strategic sourcing. The performance 

reviews have allowed for additional DOE-wide discussions on strategic 

sourcing lessons-learned and partnerships to potentially achieve greater 

efficiencies and cost savings. 

Another example provided by SBA officials illustrated how using the 

reviews to increase visibility of the small business contracting goal at 

higher management levels led to the adoption of new performance 

improvement strategies. After discussing contracting goal data at a 

quarterly review, officials said the Administrator and Deputy Administrator 

decided to call department secretaries at those federal agencies with the 

most potential for awarding small business contracts to emphasize the 

 

33For more information on strategic sourcing and other streamlining efforts at DOE, see 

GAO, Department of Energy: Additional Opportunities Exist to Streamline Support 

Functions at NNSA and Office of Science Sites, GAO-12-255 (Washington, D.C.: Jan. 

2012).
 

Page 34 GAO-13-228 Managing for Results 

importance of the goal. Officials said other strategies, such as providing 

more training for procurement center representatives, who, among other 

things, assist small businesses in obtaining federal contracts, came out of 

discussion at the reviews.34 Officials said they anticipated that the new 

strategies that came out of the reviews would lead to better performance 

toward the small business contracting goal. 

SBA officials also provided an example that illustrated how the reviews 

facilitated intra-agency collaboration that improved SBA’s bottom line. 

Officials said that during a quarterly performance review meeting, the 

head of the Office of Capital Access described anticipated staffing 

shortfalls and the head of SBA’s Office of Disaster Assistance noted that 

he expected to have staff members available during the slower season in 

disaster assistance work, which is cyclical in nature. SBA’s Administrator 

instructed the two offices to work together, and as a result, officials said 

they were able to reduce the Office of Capital Access’s labor costs by 20 

to 30 percent compared to the cost of paying employees overtime or 

hiring temporary contractor labor. 

DOE, SBA, and Treasury officials said the quarterly performance reviews 

provided a venue for top leadership to directly communicate their 

priorities and the priorities of the administration, which led to performance 

improvements in these areas. For example, at Treasury, nearly all of the 

bureau-level officials we interviewed said the Stat meetings were valuable 

because they allowed for a firsthand understanding of the Deputy 

Secretary’s priorities for their bureaus.35 Bureau officials said that the 

Deputy Secretary used these meetings to challenge their performance 

targets and approaches to addressing performance problems and to 

identify new opportunities for improvement. For example, the Bureau of 

the Public Debt Commissioner said that upon hearing about the Deputy 

 

34SBA established an online training website called the Government Contracting 

Classroom where small businesses can learn to participate in the federal contracting 

process and also where contracting officers can learn more about SBA’s programs and 

contracting policies. 

35We interviewed officials at IRS, the Bureau of the Public Debt, the Financial 

Management Service, and the Bureau of the Fiscal Service. During our review, Treasury 

was consolidating the Bureau of the Public Debt and the Financial Management Service 

within the new Bureau of the Fiscal Service to share a single administrative, management, 

and leadership structure. 

Leadership 

Communication of 

Priorities at Reviews 

Motivated Performance 

Improvements 
 

Page 35 GAO-13-228 Managing for Results 

Secretary’s interest in one of his ideas for improving how the agency 

communicates its bond pricing approach to customers, he moved ahead 

with developing an actionable strategy which he believes will ultimately 

lead to better customer service. Treasury officials said the Stat review 

process improved decision making by creating an environment where 

meaningful discussions on improving performance were held, citing 

performance improvements at the U.S. Mint as an example (see figure 6). 

Figure 6: Treasury Credits Stat Meetings with the Decision to Stop Minting $1 Coins 

for Circulation and Saving the U.S. Government Millionsa

a

GAO’s prior work indicated that while stopping production of $1 coins may save millions of dollars in 

production costs in the short term, eliminating $1 notes and replacing them with a $1 coin will have a 

larger net benefit over time. See GAO, 2012 Annual Report: Opportunities to Reduce Duplication, 

Overlap and Fragmentation, Achieve Savings, and Enhance Revenue, GAO-12-342SP (Washington 

D.C.: Feb. 28, 2012). 

b

This estimate was based on the assumption that demand would remain at 2012 levels. 
 

Page 36 GAO-13-228 Managing for Results 

Citing another example, Treasury officials said that the Deputy Secretary 

had made contracting with small business a department-wide priority. 

Although increasing small business participation in government 

contracting was a SBA priority goal, Treasury, like all federal agencies 

subject to the Small Business Act, had its own target to meet to contribute 

to the goal.36 Officials attributed increases in the department’s percentage 

of contracts with small businesses—Treasury was the only agency to 

achieve all of its fiscal year 2011 SBA small business prime contracting 

goals—to the Deputy Secretary’s “relentless attention” at the Stat 

meetings. At every Stat session with every bureau, the Deputy Secretary 

reviewed the individual bureau’s performance against the small business 

target goals.37 Officials said that Treasury’s chief procurement officer was 

present at every Stat meeting to facilitate goal achievement. 

The Financial Management Service (FMS) Commissioner cited another 

example of how the Stat meetings provided a venue for the Deputy 

Secretary to communicate his priorities, which led to better performance. 

FMS has been pursuing several priority projects to modernize its 

payments, collections, and central accounting systems that serve federal 

agencies across the government. For example, one project is to replace a 

paper process that many agencies use to accept vendor invoices with a 

central invoicing system. FMS’ analysts estimated that a central invoicing 

system could save the federal government $400 to $500 million annually, 

as well as provide vendors with online access to the status of their 

payments. FMS piloted the new system with the Bureau of Engraving and 

Printing. The new system enabled the Bureau of Engraving and Printing 

to pay their vendors more quickly than before, which resulted in the 

vendors having to provide “prompt pay” discounts. The decision was 

made to move forward government-wide, but to start with Treasury’s own 

bureaus. Following a Stat meeting in which the FMS Commissioner cited 

delays in adoption by other Treasury bureaus, the Deputy Secretary 

made it clear that implementing the new system was a priority. The FMS 

official said that, while he believed the adoption of the new system would 

 

3615 U.S.C. § 644(g). 

37The statutory goal for aggregate small business contracting government-wide is 23 

percent of the total value of all prime contract awards for each fiscal year. This is an SBA 

agency priority goal—not a formal Treasury agency priority goal—but Treasury leadership 

chose to make it an internal priority goal in order to meet its statutory requirement and 

negotiated target agreement with SBA. 
 

Page 37 GAO-13-228 Managing for Results 

have happened eventually even without the Stat meetings, he credited 

the Stat meetings with speeding up the process by three to four years. 

A small number of Treasury bureau officials we interviewed had mixed 

views on whether the Stat meetings had actually improved performance, 

with some pointing out that certain performance improvements may have 

occurred without the Stat meetings. For example, Treasury leadership 

said they used the Stat sessions to closely monitor the consolidation of 

Bureau of the Public Debt and FMS into a single bureau, the Bureau of 

the Fiscal Service. Some officials thought they would have achieved the 

various consolidation milestones and goals without the Stat reviews and 

one said that the reviews added another layer of reporting that was time 

consuming. However, Treasury department officials noted that 

component management may not be aware of how the Deputy Secretary 

uses the reviews to better understand performance issues, including, in 

some cases, to ensure that the department is providing necessary 

support to improve performance. 

Our review of DOE, SBA, and Treasury—as well as our survey of 24 

PIOs—indicated that data-driven quarterly performance reviews hold 

promise as an effective management tool at the federal level. However, 

unlike city- and state-level data-driven reviews, which typically include 

representatives from multiple agencies, officials at DOE, SBA, and 

Treasury viewed their quarterly performance review meetings as an 

internal management tool and therefore did not open the reviews to 

outside participation. Officials said they relied on other means of 

collaborating with outside agencies and other partners that contribute to 

achieving cross-cutting goals. Furthermore, the majority of PIOs we 

surveyed indicated that there was little to no involvement in the reviews 

from external officials who could contribute to achieving agency goals. 

Successful data-driven performance reviews, which require extensive 

preparation and significant leadership time, do not come without a cost, 

so it is critical that agencies implement their reviews in a way that 

maximizes their effectiveness. As the implementation of the various 

GPRAMA provisions continues, agencies may need to reevaluate the 

most effective way to engage outside contributors in the quarterly 

performance review process for APGs and other performance goals that 

depend on other organizations to achieve desired outcomes. While there 

are many approaches to managing performance toward such goals, 

agency quarterly performance reviews could provide opportunities to 

bring together the leadership and all the key players needed to improve 

cross-agency and internal agency performance. 

Conclusions 
 

Page 38 GAO-13-228 Managing for Results 

To better leverage agency quarterly performance reviews as a 

mechanism to manage performance toward agency priority and other 

agency-level performance goals, we are recommending that the Director 

of the Office of Management and Budget—working with the Performance 

Improvement Council and other relevant groups—identify and share 

promising practices to help agencies extend their quarterly performance 

reviews to include, as relevant, representatives from outside 

organizations that contribute to achieving their agency performance 

goals. 

We provided a draft of this report to DOE, OMB, SBA, and Treasury for 

review and comment. Each agency provided technical comments which 

we incorporated as appropriate. OMB staff generally concurred with the 

recommendation in our report. DOE and Treasury provided written 

comments, which are reproduced in appendixes IV and V, agreeing with 

our conclusions. 

We are sending copies of this report to the appropriate congressional 

committees and the Secretaries of Energy and Treasury, the 

Administrator of SBA, and the Director of OMB. In addition, the report is 

available at no charge on GAO’s Web site at http://www.gao.gov. 

If you or your staff have any questions about this report, please contact 

me at (202) 512-6806 or mihmj@gao.gov. Contact points for our Offices 

of Congressional Relations and Public Affairs may be found on the last 

page of this report. GAO staff who made major contributions to this report 

are listed in appendix VII. 

J. Christopher Mihm 

Managing Director, Strategic Issues 

 

Recommendation for 

Executive Action 

Agency Comments 

and Our Evaluation 
 

Page 39 GAO-13-228 Managing for Results 

List of Congressional Addressees 

The Honorable Thomas R. Carper 

Chairman 

The Honorable Tom Coburn 

Ranking Member 

Committee on Homeland Security and Governmental Affairs 

United States Senate 

The Honorable Mark R. Warner 

Chairman 

Task Force on Government Performance, Committee on the Budget 

United States Senate 

The Honorable Elijah Cummings 

Ranking Member 

Committee on Oversight and Government Reform 

House of Representatives 
Appendix I: Objectives, Scope, and 

Methodology 

Page 40 GAO-13-228 Managing for Results 

As part of our mandate to review the implementation of the Government 

Performance and Results Act Modernization Act of 2010 (GPRAMA), our 

reporting objectives were to (1) identify practices that can promote 

successful data-driven reviews at the federal level and examine how 

these reviews are being implemented at selected agencies and across 

the government, and (2) examine the impact of quarterly data-driven 

performance reviews on selected agencies’ progress toward high priority 

and other performance goals.1

 This report is the second in a series that 

examines how agencies are implementing various GPRAMA 

requirements.2

To identify practices that can promote successful data-driven reviews at 

the federal level, we conducted a review of relevant academic and policy 

literature, including our previous reports.3

 Based on our literature review, 

we developed a broad set of practices associated with the major 

contributors to success for data-driven reviews. We refined these 

practices with additional information obtained from practitioners at the 

local, state, and federal level who shared their experiences and lessons 

learned.4

 As part of this engagement, we also compared these practices 

with recent GPRAMA-related guidance in the Office of Management and 

Budget’s (OMB) Circular No. A-11 and found them broadly consistent.5

 

1

31 U.S.C. § 1121(b). 

2

See GAO, Managing for Results: GAO’s Work Related to the Interim Crosscutting Priority 

Goals under the GPRA Modernization Act, GAO-12-620R (Washington, D.C.: May 2012). 

We will issue additional work on implementation of agency priority goals, agency 

performance management infrastructure, and other GPRAMA requirements at a later 

date. 

3

We drafted our initial set of practices based on our analysis of a collection of 26 articles 

we identified as relevant to the successful implementation of data-driven reviews. See the 

bibliography for selected examples of articles we reviewed. 

4

These practitioners included participants in the Performance Improvement Council 

working group on agency reviews and Stat managers at the municipal level. The 

Performance Improvement Council is an interagency committee composed of agency 

performance improvement officers charged with assisting the Office of Management and 

Budget with topics related to GPRAMA and facilitating the exchange of useful 

performance management practices among agencies. 

5

In August 2012 OMB revised OMB Circular No. A-11, Preparation, Submission, and 

Execution of the Budget (Aug. 3, 2012). The revisions included guidance for federal 

agencies related to the preparation and submission of strategic plans, annual performance 

plans, and annual program performance reports, in addition to information on 

implementing data-driven performance reviews. OMB Cir. No. A-11, at § 270. 

Appendix I: Objectives, Scope, and 

Methodology 
Appendix I: Objectives, Scope, and 

Methodology 

Page 41 GAO-13-228 Managing for Results 

We observed two data-driven review meetings at the Department of the 

Treasury (Treasury), which was one of the agencies selected to address 

our reporting objectives. 

To examine how these reviews are being implemented at agencies 

across the government, we surveyed the performance improvement 

officer (PIO) at each of the 24 federal agencies subject to the Chief 

Financial Officers (CFO) Act.6

 We surveyed these agencies because 

GPRAMA directs us to focus on them in our reporting on the act. 

Additionally, several provisions of the act apply specifically to these 

agencies, including that the Performance Improvement Council (PIC) 

include them as members. We received responses from 24 PIOs—a 100 

percent response rate. 

The web-based survey was administered from October 18, 2012, to 

December 14, 2012. Respondents were sent an e-mail invitation to 

complete the survey on a GAO web server using a unique username and 

password. During the data collection period, we sent reminder e-mails 

and made phone calls to nonresponding agencies. Because this was not 

a sample survey, it has no sampling errors. The practical difficulties of 

conducting any survey may also introduce nonsampling errors, such as 

difficulties interpreting a particular question, which can introduce 

unwanted variability into the survey results. We took steps to minimize 

nonsampling errors by pretesting the questionnaire in person with PIOs 

and Deputy PIOs at three different agencies. We conducted pretests to 

make sure that the questions were clear and unbiased, the data and 

information were readily obtainable, and the questionnaire did not place 

an undue burden on respondents. We made appropriate revisions to the 

content and format of the questionnaire after the pretests and 

independent review. All data analysis programs used to generate survey 

results were independently verified for accuracy. Additionally, in reviewing 

the answers from agencies, we confirmed that PIOs had correctly 

bypassed inapplicable questions (skip patterns). Based on our findings, 

we determined that the survey data are sufficiently reliable for the 

purposes of this engagement. 

 

6

The survey also included questions on PIOs’ and other key officials’ characteristics, their 

involvement in performance management under GPRAMA, and other topics. These 

results will be included in our pending report on agency performance management 

infrastructure. See appendix III for a list of the 24 federal agencies subject to the CFO Act. 
Appendix I: Objectives, Scope, and 

Methodology 

Page 42 GAO-13-228 Managing for Results 

To evaluate how effectively selected agencies are applying the practices 

of data-driven reviews in their GPRAMA-mandated quarterly performance 

reviews, and the effectiveness of these reviews toward achieving agency 

priority and other performance goals, we chose three agencies—

Department of Energy (DOE), Small Business Administration (SBA), and 

Department of the Treasury (Treasury). 

Because one of our goals was to describe challenges and lessons 

learned that will be useful to as many federal agencies as possible—as 

well as to the administration and to Congress—we selected agencies for 

case study that could provide a new illustration of challenges or lessons 

learned, that is, agencies that were not the recent or current subject of 

GAO or other public administration or public policy case studies. We also 

looked for agencies that could provide broadly applicable case 

illustrations, based on the scope of their mission, organizational 

complexity, and the mix of government tools—such as direct service, 

regulations, grants, loans, and tax expenditures—used to achieve their 

performance goals.7

In addition, we excluded agencies that were undergoing significant 

management changes—such as leadership turnover or consolidation 

review—that could prevent us from gaining a clear picture of performance 

management or could interfere with our ability to make practicable 

recommendations. We also excluded agencies that had less than one 

year of experience in conducting data-driven reviews at the time that we 

started our field work. 

At each selected agency, we focused on two agency priority goals (APG)8

to examine how quarterly performance reviews affected the agency 

components responsible for achieving performance outcomes. Because 

the scope of our review was to examine data-driven performance reviews 

as a leadership strategy, we did not evaluate whether these goals were 

 

7

We reviewed agency priority goals and categorized each one using a widely-recognized 

public administration framework for our analysis based on Lester Salamon’s book. Lester 

M. Salamon, ed., The Tools of Government: A Guide to the New Governance, (New York, 

N.Y.: Oxford University Press: 2002). 

8

Every 2 years, GPRAMA requires agency heads from the CFO Act agencies, as well as 

any additional agencies designated by OMB, to identify agency priority goals from the 

performance goals of the agency. These goals should reflect the highest priorities of the 

agency as determined by the agency head. 31 U.S.C. § 1120(b)(1). 
Appendix I: Objectives, Scope, and 

Methodology 

Page 43 GAO-13-228 Managing for Results 

appropriate indicators of agency performance, sufficiently ambitious, or 

met other dimensions of quality. The following are the complete sets of 

the agencies’ 2013 APGs. An asterisk indicates the ones we focused on 

in our review: 

DOE: 

 Save low-income families money and energy through weatherization 

retrofits.* 

 Reduce the department’s Cold War legacy environmental footprint. 

 Reduce the cost of batteries for electric drive vehicles to help increase 

the market for plug-in-hybrids and all electric vehicles and thereby 

reduce petroleum use and greenhouse gas emissions. 

 Make solar energy as cheap as traditional sources of electricity.* 

 Reduce consumer energy use and costs for household appliances. 

 Prioritization of scientific facilities to ensure optimal benefit from 

federal investments. 

 Make significant progress toward securing the most vulnerable 

nuclear materials worldwide within four years. 

 Maintain the U.S. nuclear weapons stockpile and dismantle excess 

nuclear weapons to meet national nuclear security requirements as 

assigned by the President through the Nuclear Posture Review. 

SBA: 

 Process business loans as efficiently as possible.* 

 Increase small business participation in government contracting.* 

 Process disaster assistance applications efficiently. 

 Expand access to long term capital. 

Treasury: 

 Increase electronic transactions with the public to improve service, 

prevent fraud, and reduce costs.* 
Appendix I: Objectives, Scope, and 

Methodology 

Page 44 GAO-13-228 Managing for Results 

 Increase voluntary tax compliance.* 

To address both of our objectives, we reviewed memorandums, internal 

briefings, and other materials agencies used to prepare for the reviews, 

as well as documents used during the reviews and follow-up materials. 

We conducted interviews with officials at OMB, the Performance 

Improvement Council, and senior-level officials involved in each agency’s 

performance review process to gain various perspectives on these 

reviews.9

 DOE: We met with the Deputy Secretary, who also serves as DOE’s 

chief operating officer (COO), and other senior-level officials including 

but not limited to the PIO, the Deputy PIO, and budget officials.10 In 

addition, we met with two APG goal leaders: the Deputy Assistant 

Secretary for Energy Efficiency and the Deputy Assistant Secretary 

for Renewable Energy. 

 SBA: We met with senior-level officials, including but not limited to the 

SBA’s PIO, Deputy PIO, and COO. In addition, we met with SBA’s 

Associate Administrator/Office of Capital Access, who serves as the 

goal leader for one of the AGPs we reviewed. 

 Treasury: We met with the Deputy Secretary, who also serves as 

Treasury’s COO, and other senior-level officials at the department 

level, including the PIO, who also serves as the CFO, the Deputy PIO, 

and budget officials. We also met with senior-level officials at 

Treasury’s Bureau of the Fiscal Service, Bureau of the Public Debt, 

Financial Management Service, and the Internal Revenue Service.11 

 

9

GPRAMA established in law the Performance Improvement Council to assist OMB with 

topics related to GPRAMA. Council membership includes OMB’s Deputy Director for 

Management as chairperson, performance improvement officers from the 24 CFO Act 

agencies, and others as determined by the Deputy Director of Management. 31 U.S.C. § 

1124(b). 

10For example, these interviews included program officials responsible for the 

achievement of agency priority performance goals. 

11According to the President’s FY 2013 budget request, the Secretary of the Department 

of the Treasury is planning to consolidate two bureaus within Treasury’s Bureau of the 

Fiscal Service—the Bureau of the Public Debt and the Financial Management Service—to 

share a single administrative, management, and leadership structure. This consolidation 

process is scheduled to begin in 2013. 
Appendix I: Objectives, Scope, and 

Methodology 

Page 45 GAO-13-228 Managing for Results 

In addition, we met with the Assistant Fiscal Secretary who serves as 

the goal leader for one of the APGs we reviewed. 

We asked to observe at least one review meeting at each agency. 

Treasury allowed us to observe two of its quarterly performance review 

meetings—one focused on the Bureau of the Fiscal Service and one on 

the Internal Revenue Service. DOE and SBA did not allow us to observe 

their meetings, citing concerns that our presence could inhibit open 

discussion. During the interviews, we asked officials to identify any 

challenges to effective implementation they faced as the process evolved 

or any lessons they learned. We also asked officials to identify examples 

of any impacts on performance that they attributed to the reviews. 

We conducted our work from April 2012 to February 2013 in accordance 

with generally accepted government auditing standards. Those standards 

require that we plan and perform the audit to obtain sufficient, appropriate 

evidence to provide a reasonable basis for our findings and conclusions 

based on our audit objectives. We believe that the evidence obtained 

provides a reasonable basis for our findings and conclusions based on 

our audit objectives. 
Appendix II: Responses to Selected Questions 

from GAO’s Survey on Agency Performance 

Management Positions and Responsibilities 

Page 46 GAO-13-228 Managing for Results 

To address several research objectives related to implementation of the 

Government Performance and Results Act Modernization Act of 2010 

(GPRAMA)1

 we distributed a web-based survey to the performance 

improvement officer (PIO) at each of the 24 agencies subject to the Chief 

Financial Officers Act.2

 We received responses from all 24 PIOs. There 

were 45 survey questions; tables 1 through 9 below show questions and 

responses for the nine questions that were directly applicable to the 

research objectives in this report. We will publish the full survey results in 

a report in the spring of 2013 on the implementation of key management 

positions under GPRAMA. For more information about our methodology 

for designing and distributing the survey, see appendix I. 

Table 1: Question 22—To what extent does your performance improvement staff 

have each of the following competencies that OPM has identified? 

Question 

Large 

extent

Moderate 

extent

Small 

extent 

No 

extent

No 

opinion 

Competencies 

Accountability 18 5 1 0 0 

Attention to 

Detail 

20 4 0 0 0 

Customer 

Service 

19 3 2 0 0 

Influencing/ 

Negotiating 

17 5 2 0 0 

Information 

Management 

14 8 2 0 0 

Oral 

Communication 

21 3 0 0 0 

 

1

Pub. L. No. 111-352, 124 Stat. 3866 (2011). In 1993, Congress passed the Government 

Performance and Results Act (GPRA), Pub. L. No. 103-62, 107 Stat. 285 (1993), as part 

of a statutory framework aimed at improving federal government management. GPRA 

required federal agencies to develop strategic plans with long-term goals, performance 

plans with annual goals and measures, and performance reports on prior year 

performance. GPRAMA amends GPRA to create a new governmentwide performance 

management framework with several provisions—including the one for agency quarterly 

performance reviews—intended to increase federal agencies’ use of performance 

information to improve their performance and results. 

2

31 U.S.C. §901(b). See appendix III for a list of the 24 federal agencies subject to the 

Chief Financial Officers Act. 

Appendix II: Responses to Selected 

Questions from GAO’s Survey on Agency 

Performance Management Positions and 

Responsibilities 
Appendix II: Responses to Selected Questions 

from GAO’s Survey on Agency Performance 

Management Positions and Responsibilities 

Page 47 GAO-13-228 Managing for Results 

Question 

Large 

extent

Moderate 

extent

Small 

extent 

No 

extent

No 

opinion 

Organizational 

Awareness 

20 4 0 0 0 

Organizational 

Performance 

Analysis 

14 8 2 0 0 

Partnering 19 4 1 0 0 

Performance 

Measurement 

15 7 2 0 0 

Planning and 

Evaluating 

13 8 3 0 0 

Problem Solving 20 2 2 0 0 

Reasoning 21 3 0 0 0 

Technical 

Competence 

19 3 1 1 0 

Written 

Communication 

21 3 0 0 0 

Source: GAO survey results. 

Note: Unless otherwise indicated, the number of responses to each question is 24. 

Table 2: Question 27—Does your agency conduct GPRAMA-required quarterly 

performance reviews? 

Question Yes No 

Number of Responses 24 0 

Source: GAO survey results. 

Note: Unless otherwise indicated, the number of responses to each question is 24. 

Table 3: Question 28—Did your agency conduct quarterly performance reviews (or 

similar reviews) before the GPRAMA requirement took effect in June 2011? 

Question Yes No 

Number of Responses 20 4 

Source: GAO survey results. 

Note: Unless otherwise indicated, the number of responses to each question is 24. 
Appendix II: Responses to Selected Questions 

from GAO’s Survey on Agency Performance 

Management Positions and Responsibilities 

Page 48 GAO-13-228 Managing for Results 

Table 4: Question 31—How often does your agency conduct performance reviews 

(although GPRAMA requires quarterly reviews, some agencies have established 

other review cycles to meet their management needs)? 

Question 

Less often

than quarterly Quarterly 

More often

than quarterly 

Number of Responses 0 17 7 

Source: GAO survey results. 

Note: Unless otherwise indicated, the number of responses to each question is 24. 

Table 5: Question 32—How much involvement, if any, does each of the following officials have in your agency’s quarterly 

performance reviews? 

Question 

Large 

involvement

Moderate 

involvement

Small 

involvement

No 

involvement 

No 

opinion

Not 

applicable 

Officials 

Agency Head 4 5 9 6 0 0 

Chief Operating Officer 21 2 1 0 0 0 

Performance Improvement Officer 23 1 0 0 0 0 

Deputy Performance Improvement 

Officer 20 1 1 0 0 2 

Goal Leaders 19 2 3 0 0 0 

Management Chiefs 15 8 1 0 0 0 

Internal-to-agency contributors to goals 15 7 2 0 0 0 

Exteral-to-agency contributors to goals 1 3 9 7 0 4 

Source: GAO survey results. 

Note: Unless otherwise indicated, the number of responses to each question is 24. 

Table 6: Question 33—For each of the following officials, has their involvement in agency performance management 

increased, remained about the same, or decreased as a result of your agency’s quarterly performance reviews? 

Question 

Increased as a 

result of quarterly 

performance 

reviews 

Remained about the 

same as a result of 

quarterly performance 

reviews

Decreased as a result 

of quarterly 

performance reviews 

No

opinion

Not 

applicable 

Officials 

Agency Head 2 19 0 2 1 

Chief Operating Officera 15 6 0 2 0 

Performance 

Improvement Officer 16 7 0 1 0 

Deputy Performance 

Improvement Officer 11 10 0 1 2 

Goal Leaders 13 10 0 1 0 
Appendix II: Responses to Selected Questions 

from GAO’s Survey on Agency Performance 

Management Positions and Responsibilities 

Page 49 GAO-13-228 Managing for Results 

Question 

Increased as a 

result of quarterly 

performance 

reviews 

Remained about the 

same as a result of 

quarterly performance 

reviews

Decreased as a result 

of quarterly 

performance reviews 

No

opinion

Not 

applicable 

Management Chiefs 8 14 0 1 1 

Internal-to-agency 

contributors to goals 9 14 0 1 0 

External-to-agency 

contributors to goals 0 9 1 3 11 

Source: GAO survey results. 

Notes: Unless otherwise indicated, the number of responses to each question is 24. 

a

There were 23 responses to this question. 

Table 7: Question 34—How frequently do your agency’s quarterly performance reviews include the following characteristics? 

Question 

More than 

half the time

About 

half the time 

Less than 

half the time

No 

opinion 

Characteristics 

Reviews include information on progress and trends relevant 

to achievement of priority goals 23 1 0 0 

Reviews include coordination with personnel inside and 

outside agency, as relevant to achievement of priority goals 16 4 3 1 

Reviews include assessment of whether relevant 

organizations, programs, regulations, policies, etc. are 

contributing as planned toward goals 20 2 2 0 

Reviews include assessment of risk of not achieving priority 

goals and for the highest-risk goals 18 4 2 0 

Reviews include discussion of strategies for performance 

improvement 19 5 0 0 

Source: GAO survey results. 

Note: Unless otherwise indicated, the number of responses to each question is 24. 

Table 8: Question 35—How frequently do your agency’s quarterly performance reviews include the following characteristics 

or practices and how easy or challenging has it been to implement them? 

Question 

More than 

half the time

About 

half the time 

Less than 

half the time

No

 opinion 

Characteristics or Practices 

Accurate, timely, and useful data is available for the reviews 18 6 0 0 

Participants are adequately prepared for the reviews 22 1 1 0 

Reviews are held routinely as scheduled 20 3 1 0 

Reviews are aligned with strategic goals and performance 

objectives 22 1 1 0 

Leadership actively participates in the reviewsa 22 1 0 0 
Appendix II: Responses to Selected Questions 

from GAO’s Survey on Agency Performance 

Management Positions and Responsibilities 

Page 50 GAO-13-228 Managing for Results 

Question 

More than 

half the time

About 

half the time 

Less than 

half the time

No

 opinion 

Reviews include those managers/staff needed to facilitate 

problem solving and identify improvement opportunities 21 3 0 0 

Data and relevant analyses are presented effectively to 

participants of the reviews 16 6 1 1 

Reviews strike a balance between motivating improvement and 

holding participants accountable 17 5 0 2 

Actionable opportunities for performance improvement are 

identified through the reviews 13 8 3 0 

Follow up takes place after the reviews so that all action items 

arising from reviews are addressed 21 2 1 0 

Source: GAO survey results. 

Notes: Unless otherwise indicated, the number of responses to each question is 24. We are reporting the survey responses for how 

easy or challenging it has been to implement the characteristics or practices separately in table 9. 

a

There were 23 responses to this question. 

Table 9: Question 35—How frequently do your agency’s quarterly performance reviews include the following characteristics 

or practices and how easy or challenging has it been to implement them? 

Question Easy

Neither easy nor 

challenging Challenging No opinion 

Characteristics or Practices 

Accurate, timely, and useful data is available for the reviews 2 6 16 0 

Participants are adequately prepared for the reviews 5 15 4 0 

Reviews are held routinely as scheduled 10 8 6 0 

Reviews are aligned with strategic goals and performance 

objectives 13 9 1 1 

Leadership actively participates in the reviewsa 12 8 2 1 

Reviews include those managers/staff needed to facilitate 

problem solving and identify improvement opportunities 10 12 2 0 

Data and relevant analyses are presented effectively to 

participants of the reviews 2 10 11 1 

Reviews strike a balance between motivating improvement 

and holding participants accountable 4 12 6 2 

Actionable opportunities for performance improvement are 

identified through the reviews 6 11 5 2 

Follow up takes place after the reviews so that all action items 

arising from reviews are addressed 4 12 7 1 

Source: GAO survey results. 

Note: Unless otherwise indicated, the number of responses to each question is 24. 

a

The number of responses to this question is 23. 
Appendix III: Federal Agencies Subject to the 

Chief Financial Officers Act 

Page 51 GAO-13-228 Managing for Results 

 Department of Agriculture 

 Department of Commerce 

 Department of Defense 

 Department of Education 

 Department of Energy 

 Department of Health and Human Services 

 Department of Homeland Security 

 Department of Housing and Urban Development 

 Department of the Interior 

 Department of Justice 

 Department of Labor 

 Department of State 

 Department of Transportation 

 Department of the Treasury 

 Department of Veterans Affairs 

 Agency for International Development 

 Environmental Protection Agency 

 General Services Administration 

 National Aeronautics and Space Administration 

 National Science Foundation 

 Nuclear Regulatory Commission 

 Office of Personnel Management 

Appendix III: Federal Agencies Subject to the 

Chief Financial Officers Act 
Appendix III: Federal Agencies Subject to the 

Chief Financial Officers Act 

Page 52 GAO-13-228 Managing for Results 

 Small Business Administration 

 Social Security Administration 
Appendix IV: Comments from the Department 

of the Treasury 

Page 53 GAO-13-228 Managing for Results 

Appendix IV: Comments from the 

Department of the Treasury 
Appendix IV: Comments from the Department 

of the Treasury 

Page 54 GAO-13-228 Managing for Results 
Appendix V: Comments from the Department 

of Energy 

Page 55 GAO-13-228 Managing for Results 

Appendix V: Comments from the Department 

of Energy 
Appendix VI: Full Text for Interactive Figure 1 

on Finding the Right Meeting Size and 

Composition for Effective Performance 

Reviews 

Page 56 GAO-13-228 Managing for Results 

This appendix includes the print version of the text contained in 

interactive figure 1. 

Several officials noted challenges in achieving the proper balance 

between having a small number of meeting participants, which can allow 

more air time for each participant, compared to a larger group which 

might not be a productive use of time for those attending. 

Picture: Treasury Deputy Secretary and COO Neal Wolin (top) leads the 

department’s quarterly performance reviews, with support from Assistant 

Secretary for Management Nani Coloretti, who also serves as Treasury’s 

PIO. 

In addition, balancing depth of knowledge and numbers of staff attending 

these reviews is another consideration for finding the right meeting size. 

One official pointed out that having a relatively small group can help 

participants feel more comfortable in revealing performance problems but 

that there is a risk of leaving out key players that need to be part of a 

performance solution. Officials from each agency described different 

approaches to finding the right meeting size and composition for effective 

performance reviews, with several officials acknowledging that it takes 

time to get it right. 

Table 10 contains the rollover information from interactive figure 1 about 

how the Department of Energy (DOE), the Small Business Administration 

(SBA), and the Department of the Treasury (Treasury) conduct their 

quarterly performance review meetings. 

Appendix VI: Full Text for Interactive 

Figure 1 on Finding the Right Meeting Size 

and Composition for Effective Performance 

Reviews 
Appendix VI: Full Text for Interactive Figure 1 

on Finding the Right Meeting Size and 

Composition for Effective Performance 

Reviews 

Page 57 GAO-13-228 Managing for Results 

Table 10: Information about How DOE, SBA, and Treasury Conduct Their Quarterly Performance Review Meetings 

 DOE SBA Treasury 

Main participants 7 Included in average number 

of participants below 

Included in average number of 

participants below 

Average number of 

participants 

10-12 45-50 20-25 

Meeting lead Deputy Secretary with support from 

Budget Director and PIO 

Deputy Administrator with 

support of COO and PIO 

Deputy Secretary 

Performance Improvement PIO and Deputy PIO PIO, Deputy PIO, 

performance analysts 

PIO, Deputy PIO, performance 

budgeting analysts 

Management chiefs Director, Office of Management 

(Goal Leader) 

CHCO (Goal Leader) 

CIO (Goal Leader) 

CFO 

COO, CFO/PIO, CHCO, 

CIO 

Note: SBA’s CFO also 

serves as its PIO 

Budget Director, CHCO, CFO, 

CIO, Senior Procurement 

Executive, among other chiefs 

who may be invited on a reviewby-review basis 

Note: Treasury’s PIO also 

serves as its CFO. 

Component or Office 

Leadership 

Under Secretary of Energy (Goal 

Leader) 

Under Secretary, Office of Science 

(Goal Leader) 

Undersecretary for Nuclear Security 

& Administrator for National Nuclear 

Security Administration (Goal 

Leader) 

Associate Administrators 

(Goal Leader), Assistant 

Administrators 

Commissioner (Goal Leader), 

Deputy Commissioner (may be 

Goal Leader), Bureau CFO, and 

other Bureau management 

chiefs—such as the Bureau 

CIO—when appropriate. 

Goal Leaders Management Chiefs and 

Component/Office leadership serve 

as goal leaders (see above) 

Associate Administrators 

serve as goal leaders (see 

above) 

Component/Office leadership 

serve as goal leaders (see 

above) 

Additional Internal Participants Other Goal Leaders, Management 

Chiefs, and Performance and 

Budget staff who support the 

reviews 

Chief of Staff, Deputy Chief 

of Staff, General Council, 

Directors, Senior Advisors, 

Program Analysts 

Legislative Affairs, General 

Counsel, Policy Officials 

Outside participants None None None 

Source: GAO. 
Appendix VII: GAO Contact and Staff 

Acknowledgments 

Page 58 GAO-13-228 Managing for Results 

J. Christopher Mihm, (202) 512-6806 or mihmj@gao.gov

Elizabeth Curda and Laura Miller Craig managed this assignment. Darnita 

Akers, Virginia Chanley, Don Kiggins, and Bob Yetvin made contributions 

to all aspects of the report. Kim Gianopoulos, Kay Kuhlman, Judith 

Kordahl, Jill Lacey, Sarah M. McGrath, Tim Minelli, Kathleen Padulchick, 

William B. Shear, Albert Sim, and Meredith Trauner also provided 

assistance. In addition, Sabrina Streagle provided legal support and 

Donna Miller developed the report’s graphics. 

Appendix VII: GAO Contact and Staff 

Acknowledgments 

GAO Contact 

Staff 

Acknowledgments 
Selected Bibliography 

Page 59 GAO-13-228 Managing for Results 

Behn, Robert. “Collaborating for Performance: Or Can There Exist Such a 

Thing as CollaborationStat?” International Public Management Journal, 

vol. 13, no. 4 (2010): 429-470. 

Behn, Robert. “The Core Drivers Of CitiStat: It’s Not Just About the 

Meetings and the Maps,” International Public Management Journal, vol. 8, 

no. 3 (2005): 295-319. 

Behn, Robert. “The Varieties of CitiStat,” Public Administration Review, 

vol. 66, no. 3 (2006): 332-340. 

Behn, Robert. “What All Mayors Would Like to Know About Baltimore’s 

CitiStat Performance Strategy,” IBM Center for the Business of 

Government Managing for Performance and Results Series (Washington, 

D.C.: 2007). 

Campbell, Mary. “Bringing Performance Excellence to the Public Sector: 

Washington State’s Odyssey,” ASQ World Conference on Quality and 

Improvement Proceedings, vol. 59 (2005): 279-288. 

Esty, Daniel, and Reece Rushing. “The Promise of Data-Driven 

Policymaking,” Issues in Science and Technology, vol. 23, no. 4 (Summer 

2007): 67-72. 

Fillichio, Carl. “Getting Ahead of the Curve: Baltimore and CitiStat,” Public 

Manager, vol. 34, no. 2 (Summer 2005): 51-53. 

Godown, Jeff. “The CompStat Process: Four Principles for Managing 

Crime Reduction,” The Police Chief, vol. LXXVI, no. 8 (August 2009), 

accessed on September 5, 2012, 

http://www.policechiefmagazine.org/magazine/index.cfm?fuseaction=displ

ay&article_id=1859&issue_id=82009. 

Griffith, John, and Gadi Dechter. “Performance Reviews That Work: Four 

Case Studies of Successful Performance Review Systems in the Federal 

Government,” Center for American Progress (Washington, D.C.: February 

2011). 

Hatry, Harry and Elizabeth Davies, “A Guide to Data-Driven Performance 

Reviews,” IBM Center for the Business of Government Improving 

Performance Series (Washington, D.C.: 2011). 

Selected Bibliography 
Selected Bibliography 

Page 60 GAO-13-228 Managing for Results 

Henderson, Lenneal. “The Baltimore CitiStat Program: Performance and 

Accountability,” IBM Endowment for the Business of Government 

Managing For Results Series (Arlington, VA: May 2003). 

Kettl, Donald. “China Looks West for Performance Management,” 

Governing, August 2011, http://www.governing.com/columns/potomacchronicle/china-looks-west-performance-management.html. 

Kingsley, Christopher. “Smart Cities: PerformanceSTAT at 15,” University 

of Pennsylvania Fels Institute of Government (Philadelphia, PA; October 

2010). 

Malinowski, Chris, and Sasha Page. “Top 10 Performance Measurement 

Dos and Don’ts,” Government Finance Review, vol. 20, no. 5 (October 

2004): 28+. 

Moynihan, Donald, and Stéphane Lavertu. “Do Performance Reforms 

Change How Federal Mangers Manage?” The Brookings Institution 

Issues in Governance Studies Series, 52 (Washington, D.C.: October 

2012). 

O’Connell, Paul, and Daniel Forrester. “Think Differently: Update Your 

Stats to Unlock Outcomes,” The Public Manager, vol. 38, no. 4 (Winter 

2009-2010): 5-9. 

O’Connell, Paul. “Using Performance Data for Accountability: The New 

York City Police Department’s CompStat Model of Police Management,” 

The PricewaterhouseCoopers Endowment for the Business of 

Government Managing for Results Series (Arlington, VA: 2001). 

Partnership for Public Service. “From Data to Decisions: The Power of 

Analytics,” IBM Center for the Business of Government (Washington, 

D.C.: November 2011). 

Perez, Teresita and Reece Rushing. “The CitiStat Model: How DataDriven Government Can Increase Efficiency and Effectiveness,” Center 

for American Progress (Washington, D.C.: April 2007). 

Queensland Audit Office. Better Practice Guide Performance Reviews.

Brisbane, Australia: 2010. 
Selected Bibliography 

Page 61 GAO-13-228 Managing for Results 

Sanger, Mary Bryna. “From Measurement to Management: Breaking 

through the Barriers to State and Local Performance,” Public 

Administration Review, S1 (December 2008): S70-85. 

Scofield, Jennifer. “Asking Why Cuyahoga CountyStat,” Government 

Finance Review, vol. 27, no. 6 (December 2011): 40-43. 

Serpas, Ronal, and Matthew Morley. “The Next Step in AccountabilityDriven Leadership: “CompStating” the CompStat data,” The Police Chief, 

vol. LXXV, no. 5 (May 2008), accessed on September 5, 2012, 

http://www.policechiefmagazine.org/magazine/index.cfm?fuseaction=displ

ay_arch&article_id=1501&issue_id=52008. 

Shane, Jon. “Compstat Process,” FBI Law Enforcement Bulletin, vol. 73, 

no. 4 (April 2004): 12-21. 

Sharp, Cathy, Jocelyn Jones, and Alison Smith. “What Do We Measure 

and Why? An Evaluation of the Citistat Model of Performance 

Management and its Applicability to the Scottish Public Sector,” Scottish 

Executive Social Research (Edinburgh, UK: 2006). 

Solan, David. “The EPAStat Quarterly Report and Lessons Learned,” 

Public Performance & Management Review, vol. 33, no. 2 (December 

2009): 222-240. 

Steinberg, Harold. “Using Performance Information to Drive Performance 

Improvement,” Association of Government Accountants Corporate 

Partner Advisory GroupResearch Series: Report No. 29 (Alexandria, VA: 

2011). 

Useem, Greg. “Moving from Reporting Performance Information to Using 

It,” Government Finance Review, vol. 25, no. 2 (April 2009): 47-50. 

Weitzman, Beth, Diana Silver, and Caitlyn Brazill. “Efforts to Improve 

Public Policy and Programs through Data Practice: Experiences in 15 

Distressed American Cities,” Public Administration Review, vol. 66, no. 3 

(May/June 2006): 386-399. 
Related GAO Products 

Page 62 GAO-13-228 Managing for Results 

Managing for Results: Key Considerations for Implementing Interagency 

Collaborative Mechanisms. GAO-12-1022. Washington, D.C.: September 

27, 2012 

Strategic Sourcing: Improved and Expanded Use Could Save Billions in 

Annual Procurement Costs. GAO-12-919. Washington, D.C.: September 

20, 2012 

Solar Energy: Federal Initiatives Overlap but Take Measures to Avoid 

Duplication. GAO-12-843. Washington, D.C.: August 30, 2012. 

Entrepreneurial Assistance: Opportunities Exist to Improve Programs’ 

Collaboration, Data-Tracking, and Performance Management.

GAO-12-819. Washington, D.C.: August 23, 2012. 

Managing for Results: GAO’s Work Related to the Interim Crosscutting 

Priority Goals under the GPRA Modernization Act. GAO-12-620R. 

Washington, D.C.: May 31, 2012. 

2012 Annual Report: Opportunities to Reduce Duplication, Overlap and 

Fragmentation, Achieve Savings, and Enhance Revenue.

GAO-12-342SP. Washington D.C: February 28, 2012. 

Department of Energy: Additional Opportunities Exist to Streamline 

Support Functions at NNSA and Office of Science Sites. GAO-12-255. 

Washington, D.C.: January 31, 2012. 

Small Business Administration: Progress Continues in Addressing 

Reforms to the Disaster Loan Program. GAO-12-253T. Washington, D.C.: 

November 30, 2011. 

U.S. Coins: Replacing the $1 Note with a $1 Coin Would Provide a 

Financial Benefit to the Government. GAO-11-281. Washington, D.C.: 

March 4, 2011. 

Government Performance: Strategies for Building a Results-Oriented and 

Collaborative Culture in the Federal Government. GAO-09-1011T. 

Washington, D.C.: September 24, 2009. 

Government Performance: Lessons Learned for the Next Administration 

on Using Performance Information to Improve Results. GAO-08-1026T. 

Washington, D.C.: July 24, 2008. 

Related GAO Products 
Related GAO Products 

Page 63 GAO-13-228 Managing for Results 

Results Oriented Cultures: Creating a Clear Linkage between Individual 

Performance and Organizational Success. GAO-03-488. Washington, 

D.C.: March 14, 2003. 

(450966)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.” 

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information. 

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact: 

Website: http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548 

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548 

GAO’s Mission 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone 

Connect with GAO 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

Please Print on Recycled Paper.