GRANTS 

PERFORMANCE

Justice and FEMA 

Collect Performance 

Data for Selected 

Grants, but Action 

Needed to Validate 

FEMA Performance 

Data

Report to Congressional Requesters

June 2013

GAO-13-552

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-13-552, a report to 

congressional requesters

June 2013

GRANTS PERFORMANCE

Justice and FEMA Collect Performance Data 

for Selected Grants, but Action Needed to 

Validate FEMA Performance Data

Why GAO Did This Study

The Government Performance and 

Results Act (GPRA) of 1993 began 

moving the federal government toward 

a more results-oriented culture, and in 

2010, GPRAMA established a new 

framework for taking a more integrated 

approach to focusing on results and 

improving government performance by

establishing requirements for 

government-wide and agency-wide 

goals. Reporting progress on these 

and other goals may include grant

performance information. GAO was 

asked to review grant performance 

reporting. This report examines (1) the 

extent to which OMB provides 

guidance to agencies on collecting, 

validating, and reporting performance 

data that demonstrate progress toward 

agency-wide and federal goals; and (2) 

how the selected agencies collect and 

validate data on the performance of

selected grant programs and use those

data to report progress toward agencywide performance goals. For this 

review, GAO selected two grant 

programs each at Justice and FEMA. 

GAO reviewed OMB guidance on grant 

performance and relevant legislation, 

interviewed officials at OMB, Justice, 

and FEMA, and conducted a literature 

review to identify performance

measurement issues.

What GAO Recommends

GAO recommends that FEMA ensure, 

in accordance with OMB Circular No.

A-11, that there are consistent 

procedures in place at the program 

office and regional level to validate and

verify grant performance data that 

allow FEMA to attest to the reliability of 

EMPG and AFG grant data used to 

report progress toward goals. DHS, of 

which FEMA is a part, concurred with 

this recommendation.

What GAO Found

The Office of Management and Budget (OMB) provides detailed guidance in 

Circular No. A-11 for the agency goal development and the performance data

verification and validation processes. OMB Circular No. A-11 directs agencies to 

have a data validation plan for performance reporting and cost-effective 

validation and verification techniques in place to ensure the completeness and 

reliability of all performance measurement data used in annual performance 

plans. In 2012, OMB revised Circular No. A-11 to incorporate the requirements of 

the Government Performance and Results Act Modernization Act of 2010 

(GPRAMA), which requires agencies to establish 2-year agency priority goals for 

areas where they seek to achieve near-term performance results but provides 

agencies with discretion to select their goals and the performance data used to 

show progress towards achieving these goals. Agencies must ensure that the 

data they use to report progress toward performance goals is accurate and 

reliable. 

The Justice Assistance Grant (JAG) program at the Department of Justice 

(Justice) collects performance data quarterly and qualitative summary narrative 

information annually. Justice’s Juvenile Accountability Block Grant (JABG) 

program collects quantitative and qualitative performance data annually and 

financial performance data quarterly. For both JAG and JABG, performance data 

are passed up from the grant program offices to the Office of the Chief Financial 

Officer within the Office of Justice Programs and are ultimately made available to 

the Justice Management Division. JAG’s data collection tool has an edit-check 

function to improve the reliability of data entered into the system, and Justice has 

established a policy to implement a new data validation tool for future use with 

JABG grantees. Justice established four agency priority goals for 2012 to 2013, 

but it does not use JAG and JABG performance data as part of its evaluation of 

progress towards agency priority goals, as is allowed under GPRAMA and 

incorporated into Circular No. A-11. 

The Federal Emergency Management Administration’s (FEMA) Emergency 

Management Performance Grants (EMPG) and Assistance to Firefighters Grants 

(AFG) programs collect performance information through a variety of reporting 

mechanisms. These mechanisms collect performance data used by FEMA

regional offices and headquarters for different purposes. For example, 

headquarters focuses on the development of future program priorities and 

reporting progress toward the National Preparedness Goal, while regions use 

program information to monitor primary grant recipients. The Department of 

Homeland Security (DHS), of which FEMA is a part, developed agency priority 

goals that reflect agency-wide, near-term priorities. According to FEMA officials, 

the EMPG and AFG programs have an indirect link to a DHS agency priority 

goal, as well as the National Preparedness Goal, because they support states’ 

level of preparedness for disasters. According to FEMA officials, neither program 

has a standardized tool with which to validate the performance data that are selfreported by recipients; additionally, the regions are inconsistent in their 

approaches to verifying program performance data. The absence of a formal 

established validation and verification procedure, as directed by Circular No. A11, could lead to the collection of erroneous performance data. 

View GAO-13-552. For more information, 

contact Stanley J. Czerwinski at (202) 512-

6806 or czerwinskis@gao.gov. 
Page i GAO-13-552 Grants Performance

Letter 1 

Background 4

OMB Provides Agencies with Guidance on Reporting Progress 

toward Goals through Its Circular and Resource Management 

Officers 8

Selected Justice and FEMA Grant Programs Have Performance 

Data Collection Systems that Link Data to Goals, but FEMA 

Programs Lack Verification and Measurement Capabilities 9

Conclusions 24

Recommendation for Executive Action 24

Agency Comments and Our Evaluation 24

Appendix I Objectives, Scope, and Methodology 26

Appendix II The National Preparedness Goal Mission Areas and Core 

Capabilities 28

Core Capabilities 28

Appendix III Comments from the Department of Homeland Security 30

Appendix IV GAO Contact and Staff Acknowledgments 32

Tables

Table 1: Core Capabilities by Mission Area 29

Figures

Figure 1: Upward Flow of Programmatic and Financial 

Performance Data for the Justice Assistance Grants and 

the Juvenile Accountability Block Grants 11

Figure 2: Upward Flow of Programmatic and Financial 

Performance Data for the Emergency Management 

Performance Grants (EMPG) 17

Figure 3: Upward Flow of Programmatic and Financial 

Performance Data for the Assistance to Firefighters 

Grants (AFG) 19

Contents 
Page ii GAO-13-552 Grants Performance

Abbreviations

AFG Assistance to Firefighters Grants 

BJA Bureau of Justice Assistance

BSIR Biannual Strategy Implementation Report

DCTAT Data Collection and Technical Assistance Tool

DHS Department of Homeland Security

DOJ Department of Justice

EMPG Emergency Management Performance Grants

FEMA Federal Emergency Management Agency

FFR Federal Financial Report

GPD Grant Programs Directorate

GPRA Government Performance and Results Act

GPRAMA Government Performance and Results Act 

Modernization Act

JAG Justice Assistance Grant

JABG Juvenile Accountability Block Grants

JMD Justice Management Division

NAPA National Academy of Public Administration

NFIRS National Fire Incident Reporting System

NPAD National Preparedness Assessment Division

NPG National Preparedness Goal

OJJDP Office of Juvenile Justice and Delinquency Prevention

OMB Office of Management and Budget

PIO Performance Improvement Officer

PMT Performance Measurement Tool

PPR Semi-Annual Programmatic Performance Report

QPPR Quarterly Performance Progress Reports

SPR State Preparedness Report

THIRA Threat and Hazard Identification and Risk Assessment

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-13-552 Grants Performance

441 G St. N.W.

Washington, DC 20548

June 24, 2013

The Honorable Thomas R. Carper

Chairman

Committee on Homeland Security and Governmental Affairs

United States Senate

The Honorable Tom Coburn, M.D.

Ranking Member

Committee on Homeland Security and Governmental Affairs

United States Senate

The Honorable Claire McCaskill

Chairman

Subcommittee on Financial and Contracting Oversight

Committee on Homeland Security and Governmental Affairs

United States Senate

Grants are a tool agencies use at the direction of Congress to address 

national priorities, carry out their missions, and achieve program goals. 

While federal grants to state and local governments decreased in federal 

fiscal year 2012 to about $545 billion from a high of $608 billion in fiscal 

year 2010 as funds from the American Recovery and Reinvestment Act of 

2009 were spent, grant spending is and will continue to be of critical 

importance.1 As with any federal spending, tracking the performance of 

grant recipients could be helpful in ensuring transparency and 

accountability of federal agency operations, programs, and achievements 

to Congress as well as to taxpayers. Our past work has identified that 

opportunities exist to improve the design and implementation of federal 

grants, and that two issues with regard to grant performance reporting—

lack of appropriate performance measures, and data collection and 

validation—present challenges.2

 

1

Pub.L. No. 111-5, 123 Stat. 115 (Feb. 17, 2009). 

The level of federal spending on grant 

programs combined with the federal government’s continued long-term 

2

GAO, Grants to State and Local Governments: An Overview of Federal Funding Levels 

and Selected Challenges, GAO-12-1016 (Washington, D.C.: Sept. 25, 2012); and Grants 

Management: Enhancing Performance Accountability Provisions Could Lead to Better 

Results, GAO-06-1046 (Washington, D.C.: Sept. 29, 2006). 

 
 

Page 2 GAO-13-552 Grants Performance

fiscal challenges and constrained budgets only increases the importance 

of accurately measuring the performance of grant programs to ensure 

they contribute to federal and agency goals.

You asked us to review grant performance reporting. This report 

examines (1) the extent to which the Office of Management and Budget 

(OMB) provides guidance to agencies on how to collect, validate, and 

report performance information that demonstrates progress toward 

agency-wide and federal goals; and (2) how selected agencies collect 

and validate information on the performance of selected grant programs 

and use that information to report progress toward their agency-wide 

performance goals. For this review, we selected two grant programs at 

the Department of Justice (Justice), the Edward Byrne Memorial Justice 

Assistance Grant (JAG) program and the Juvenile Accountability Block 

Grants (JABG) program, and two at the Federal Emergency Management 

Agency (FEMA), which is part of the Department of Homeland Security 

(DHS), the Emergency Management Performance Grants (EMPG) 

program and the Assistance to Firefighters Grants (AFG). The two Justice 

programs allowed us to review formula grant programs with diverse 

obligation amounts directed to states.

3 The FEMA programs allowed us to 

review one formula grant program directed to states and one project grant 

program directed to local fire departments and emergency medical 

services organizations.4 These programs were selected to provide a 

variety of grants in terms of grant size (obligation amounts), grant type 

(project or formula), grantee type (state or local government), and links to 

agency goals. We selected agencies and programs that leveraged and 

built upon our prior knowledge of selected grant programs and related 

performance reporting.5

To determine the extent to which OMB provides guidance to agencies on 

collecting, validating, and reporting performance information, we reviewed 

OMB guidance on grant performance for the executive agency level. We 

also reviewed the Government Performance and Results Act (GPRA) of 

 

3

A formula grant is allocated to eligible recipients based on formula factors, such as state 

population, established in statutory and regulatory provisions. 

4

A project grant provides funding for fixed or known periods of specific projects. 

5

Findings included in this report are not generalizable beyond the four grant programs 

selected for this review. 
 

Page 3 GAO-13-552 Grants Performance

1993,6 as updated by the GPRA Modernization Act (GPRAMA) of 2010,7

To determine how the selected agencies collect and use data on the 

performance of their grant recipients, we reviewed key agency documents 

on the selected grant programs, such as grant solicitations and 

performance requirements, examples of related performance measures, 

and internal guidance. We interviewed officials at FEMA and Justice to 

determine the type and usefulness of guidance received for measuring 

grant performance and how these officials incorporated this guidance into 

measuring progress toward government-wide and agency goals. To 

identify performance management issues raised, we conducted a 

literature review. This included our previous reports on the selected grant 

programs at Justice and FEMA and reports by their Offices of Inspectors 

General, reports by the Congressional Research Service, and the 

Congressional Budget Office.

and hearings related to GPRAMA, along with agency goals established 

through GPRAMA to determine any relationship between selected grant 

programs and relevant goals. We interviewed officials at OMB 

responsible for developing and disseminating grant performance reporting 

guidance to executive agencies to determine the type and extent of 

guidance offered to agencies on grant performance data collection, 

validation, and reporting.

8

We conducted this performance audit from May 2012 to June 2013 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

For a more detailed version of our 

objectives, scope, and methodology, see appendix I.

 

6

Pub. L. No. 103-62, 107 Stat. 285 (Aug. 3, 1993). 

7

Pub. L. No. 111-352, 124 Stat. 3866 (Jan. 4, 2011). 

8

GAO reports include, National Preparedness: FEMA Has Made Progress in Improving 

Grant Management and Assessing Capabilities, but Challenges Remain, GAO-13-456T

(Washington, D.C.: Mar. 19, 2013); Managing Preparedness Grants and Assessing 

National Capabilities: Continuing Challenges Impede FEMA’s Progress, GAO-12-526T

(Washington, D.C.: Mar. 20, 2012); Homeland Security: DHS Needs Better Project 

Information and Coordination among Four Overlapping Grant Programs, GAO-12-303

(Washington, D.C.: Feb. 28, 2012); Measuring Disaster Preparedness: FEMA Has Made 

Limited Progress in Assessing National Capabilities, GAO-11-260T (Washington, D.C.: 

Mar. 17, 2011); and Recovery Act: Department of Justice Could Better Assess Justice 

Assistance Grant Program Impact, GAO-11-87 (Washington, D.C.: Oct. 15, 2010). 
 

Page 4 GAO-13-552 Grants Performance

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

Since the enactment of GPRA in 1993, the federal government has been 

moving toward a more results-oriented performance culture to improve its 

effectiveness and service delivery, as well as internal management. We 

have long reported on the challenges agencies face in creating a focus on 

results, including the challenge of developing meaningful performance 

measures and collecting reliable performance information. In 2010, 

GPRAMA established a new framework aimed at taking a more 

crosscutting and integrated approach to focusing on results and 

improving government performance. GPRAMA requires that executive 

branch agencies designate agency-level priority goals beginning with the 

fiscal year 2013 annual performance plans and report progress toward 

those goals, including goals measured using grant performance data. 

Accurate and reliable measurement of grant performance forms the basis 

for managing programs and decision making, evaluating the effectiveness 

of grant programs, gauging progress toward the achievement of broader 

agency-wide goals, and reporting on progress toward agency priority 

goals. We have also reported on the importance of measuring 

performance, communicating performance information frequently and 

effectively, and using performance information as a basis for decision 

making.9

GPRA required agencies to develop performance plans that establish 

performance goals and to compare actual performance achieved with 

those goals. GPRAMA emphasizes the need for credible performance 

information by requiring that agencies provide additional information 

about how they will ensure the accuracy and reliability of the data they 

use to report progress towards performance goals. Specifically, agencies 

are required to explain in their performance plans and reports the means 

used to verify and validate measured values; the sources of the data; the 

 

9

GAO, Government Performance: Lessons Learned for the Next Administration on Using 

Performance Information to Improve Results, GAO-08-1026T (Washington, D.C.: July 24, 

2008); Managing for Results: Enhancing Agency Use of Performance Information for 

Management Decision Making, GAO-05-927 (Washington, D.C.: Sept. 9, 2005); and 

Executive Guide: Effectively Implementing the Government Performance and Results Act, 

GAO/GGD-96-118 (Washington, D.C.: June 1996). 

Background 
 

Page 5 GAO-13-552 Grants Performance

level of accuracy required for the intended use of the data; any limitations 

to the data at the required level of accuracy; and how the agency has 

compensated for such limitations if needed to reach the required level of

accuracy. OMB’s Circular No. A-11 provides guidance to executive 

branch agencies on establishing standards for performance data 

validation and verification to ensure the completeness and reliability of all 

performance measurement data contained in reports.

10

Grant performance may be defined and measured in a variety of ways, 

which include financial performance, programmatic performance, and 

administrative performance.

• Financial performance pertains to disclosing grant expenditures 

related to federally-sponsored projects in accordance with reporting 

requirements.

• Programmatic performance includes both grant outputs (that is, direct 

results of the grant, such as individuals served or the number of 

programs implemented) and outcomes (that is, long-term results such 

as behavioral changes).

• Administrative performance includes measurement of how well 

program offices manage their operations and interactions with 

customers. Performance management information can inform key 

management decisions such as setting program priorities, allocating 

resources, identifying program problems, taking corrective action to 

solve those problems, and helping determine progress in meeting the 

goals of programs or operations.

The two Justice programs we reviewed were the Edward Byrne Memorial 

Justice Assistance Grant (JAG) program and the Juvenile Accountability 

Block Grants (JABG) program.11 Both are formula grant programs 

directed to states. In fiscal year 2012, according to Justice, the amount 

obligated for the JAG program was $288 million and for the JABG 

program it was $22 million.12

 

10OMB Circular No. A-11, Preparation, Submission, and Execution of the Budget

(Washington, D.C.: Aug. 3, 2012).

JAG is the primary provider of federal 

criminal justice funding to state and local jurisdictions. The JAG program 

11JAG and JABG awards are provided to all states, the District of Columbia, Guam, 

American Samoa, the Commonwealth of Puerto Rico, the Virgin Islands, and the Northern 

Mariana Islands.

12The federal fiscal year ends September 30 of each year. 
 

Page 6 GAO-13-552 Grants Performance

provides states and units of local government with funding necessary to 

support a range of program areas including law enforcement, prosecution 

and court programs, and prevention and education programs. JABG 

assists states and units of local government in their efforts to strengthen 

their juvenile justice systems. Both grant programs require states to select 

an agency to administer these grant funds and may pass funds from the 

state administering agency to units of local government.

The two FEMA grant programs we selected were the Emergency 

Management Performance Grants (EMPG) program and the Assistance 

to Firefighters Grant (AFG) program.13 The EMPG program makes grants 

to states to assist state, local, and tribal governments prepare for all 

hazards. The program plays an important role in supporting the 

development and sustainment of core capabilities to fulfill the five mission 

areas (prevention, protection, mitigation, response, and recovery) of the 

National Preparedness Goal (NPG).14 The AFG program provides federal 

grants directly to local fire departments and emergency medical services 

organizations that are unaffiliated with fire departments to help pay for 

equipment, training, and other firefighter needs.15

Some of our previous reports have raised concerns about grant 

performance reporting at Justice and FEMA. For example, we reported in 

2010 that Justice’s JAG program revised its performance measures and 

we recommended that Justice continue to revise JAG performance 

measures and validate the integrity of self-reported performance data.

In fiscal year 2012, 

according to FEMA, the amount obligated for EMPG was $340 million and 

for AFG it was $321 million.

16

 

13The AFG program comprises three distinct financial assistance programs: Assistance to 

Firefighters Grants (AFG); Staffing for Adequate and Emergency Response; Fire 

Prevention and Safety. The focus of this report is the AFG.

Justice concurred with this recommendation and since then, according to 

14The National Preparedness Report (NPR) released in March 2012 describes the 

National Preparedness Goal and its five mission areas. The mission areas include 31 core 

capabilities essential to preparedness. The NPR offers an assessment of the level of 

preparedness related to each of these 31 core capabilities. The NPG was developed as a 

part of the Presidential Policy Directive 8 (PPD-8). (See app. II.)

15GAO, Fire Grants: FEMA Has Met Most Requirements for Awarding Fire Grants, but 

Additional Actions Would Improve Its Grant Process, GAO-10-64 (Washington, D.C.: Oct. 

30, 2009).

16GAO-11-87 
 

Page 7 GAO-13-552 Grants Performance

Justice officials, it has revised the JAG performance measures in 

consultation with a group of grant recipients. With regard to FEMA, in 

March 2011, we reported that it was able to gather data on the progress, 

capabilities, and accomplishments of a state’s preparedness program 

through State Preparedness Reports (SPR). These reports include selfreported data that may be subject to interpretation by the reporting 

organizations in each state and not be readily comparable to other states’ 

data. We also reported that, in general, FEMA officials said that 

evaluation efforts they used to collect data on national preparedness 

capabilities were useful for their respective purposes, but that the data 

collected were limited by data reliability and measurement issues related 

to the lack of standardization in the collection of data.17 FEMA stated it 

had taken steps to address these reliability challenges, including creating 

a web-based survey tool to standardize how it collects state 

preparedness information. We also reported in February 2012 that FEMA 

was considering how to make better use of grant information from four 

grant programs—the State Homeland Security Program, Urban Area 

Security Initiative, Port Security Grant Program, and the Transit Security 

Grant Program—and we recommended that FEMA collect project 

information with the level of detail needed to better position the agency to 

identify any potential unnecessary duplication within and across the four 

grant programs, weighing any additional costs of collecting these data.18

DHS concurred with these recommendations and has actions under way 

to address them.

 

17GAO-12-526T and GAO-11-260T

18GAO-13-456T and GAO-12-303
 

Page 8 GAO-13-552 Grants Performance

The Office of Management and Budget plays a central role in guiding 

agencies on grant making and grant performance measurement by 

providing direction for the agency priority goal process.19 OMB revised 

Circular No. A-11 in 2012 to incorporate the requirements of GPRAMA to 

provide detailed guidance to executive branch agencies to establish 

government-wide performance reporting requirements and standards for 

performance data validation and verification to ensure the completeness 

and reliability of all performance measurement data contained in annual 

reports. OMB’s Circular No. A-11 states that agencies should have costeffective verification and validation techniques in place to ensure the 

completeness and reliability of all performance measurement data 

contained in their annual performance plans and reports as appropriate to 

the intended use of the data.20 OMB Circular No. A-11 directs agencies to 

have a data validation plan for performance reporting and to include an 

assessment by the agency head of the reliability and completeness of the 

performance data they selected to include in the plan and report. The 

description must include how the agency ensures the accuracy and 

reliability of the data used to measure progress toward its performance 

goals.21

OMB provides agencies additional guidance through its resource 

management officers (RMO). RMOs are charged with working with a 

portfolio of agencies organized by subject matter. As part of this, RMOs 

offer analysis and evaluation to support government-wide initiatives. 

According to OMB officials, RMOs are central to the conversation on 

 

19Every 2 years GPRAMA requires agency heads in those agencies defined under the 

Chief Financial Officers Act of 1990, or those otherwise determined by OMB, to designate 

a subset of priority goals from the performance goals in their agency performance plans.

These goals should reflect the highest priorities of the agency as determined by the 

agency head. 

20Agency Performance Plans establish performance goals to define the level of 

performance to be achieved during the year in which the plan is submitted and the next 

fiscal year. These plans are made available on a public website and cover each program 

activity set forth in the agency budget. 

21OMB Circular No. A-102, Grants and Cooperative Agreements with State and Local 

Governments (Washington, D.C.: Oct. 7, 1994, further amended Aug. 29, 1997); and OMB 

Circular No. A-110, Uniform Administrative Requirements for Grants and Agreements with 

Institutions of Higher Education, Hospitals, and Other Non-Profit Organizations

(Washington, D.C.: Nov. 19, 1993, further amended Sept. 30, 1999) both also set forth 

requirements for federal grant performance monitoring. 

OMB Provides 

Agencies with 

Guidance on 

Reporting Progress 

toward Goals 

through Its Circular 

and Resource 

Management Officers
 

Page 9 GAO-13-552 Grants Performance

performance with their assigned agencies and focus on targeting 

resources to improve program performance.

For 2012 to 2013, Justice has established four agency priority goals, 

which are aligned with its strategic goals and objectives as outlined in the 

Justice strategic plan. Justice’s four agency priority goals are to: (1) better 

inform the intelligence community; (2) reduce gang violence; (3) protect 

the American people from financial and health care fraud; and (4) protect 

those most in need of help, with a special emphasis on child exploitation 

and civil rights. The Justice Management Division (JMD) serves as the 

focal point for performance information across Justice, with respect to 

both receiving guidance from OMB and ultimately sharing Justice’s 

performance with OMB. Justice’s performance improvement officer (PIO) 

works within this component and represents the agency on the 

Selected Justice 

and FEMA Grant 

Programs Have 

Performance Data 

Collection Systems 

that Link Data to 

Goals, but FEMA 

Programs Lack 

Verification and 

Measurement 

Capabilities 

Justice Collects Grant 

Performance Information 

by Purpose Area for 

Selected Grant Programs, 

which Allows It to Assess 

Program Output
 

Page 10 GAO-13-552 Grants Performance

Performance Improvement Council.22

As shown in figure 1, for both the Justice Assistance Grant (JAG) and 

Juvenile Accountability Block Grants (JABG), programmatic and financial 

performance data are passed up from the grant program offices to the 

Office of the Chief Financial Officer within the Office of Justice Programs 

and ultimately made available to JMD.

According to Justice officials, JMD 

offers performance measure development feedback to all Justice 

components when requested, and the PIO schedules monthly agencywide conference calls related to performance management for all Justice 

component performance representatives.

 

22GPRAMA established in law the Performance Improvement Council (PIC), chaired by 

OMB’s Deputy Director for Management and composed of PIOs from various federal 

agencies. Among other responsibilities, the PIC is charged with facilitating the exchange 

of successful performance management practices among agencies and assisting OMB in 

implementing certain GPRAMA requirements. Our April 2013 report discusses the role of 

the PIC in facilitating the exchange of best practices and improving agency program 

management and performance - GAO, Managing for Results: Agencies Have Elevated 

Performance Management Leadership Roles but Additional Training Is Needed, 

GAO-13-356 (Washington, D.C.: Apr. 16, 2013).
 

Page 11 GAO-13-552 Grants Performance

Figure 1: Upward Flow of Programmatic and Financial Performance Data for the 

Justice Assistance Grants and the Juvenile Accountability Block Grants

Performance information from the JAG and the JABG programs is not 

used to report progress toward agency priority goals. Based on 

GPRAMA, agencies have discretion to select the data sources to use to 

report progress toward their goals. According to Justice officials, these 

two grant programs were excluded from reporting progress toward 

Justice’s agency priority goal of reducing gang violence because JAG 

does not collect data on specific targeted behaviors to reduce gang 

violence and only a small fraction of JABG subgrantees, if any, use grant 

funds for direct service programming related to gang prevention.
 

Page 12 GAO-13-552 Grants Performance

JAG performance measures are organized by seven overall program 

purpose areas.23 According to Justice officials, the JAG performance 

measures were redesigned beginning in 2011, and were developed in 

cooperation with grant recipients and shared with professional criminal 

justice organizations before being adopted in 2012. The JAG program 

collects quantitative performance data on a quarterly basis and qualitative 

narrative data on an annual basis. JAG grantees enter quantitative 

performance data directly into JAG’s electronic data collection system, 

the Performance Measurement Tool (PMT). For example, for the law 

enforcement purpose area, the PMT collects information on the number 

and type of programs implemented, expanded, or sustained during the 

reporting period. To help improve data reliability, JAG program staff 

review grantee reports on a quarterly basis to evaluate performance 

information and look for data anomalies. The PMT has edit checks built 

into the system which, along with manual review of quarterly reports, help 

prevent data entry errors.24

JAG collects qualitative performance data in reports from primary grant 

recipients about their grant-related activities during the past year. 

Grantees create a report from the PMT system and eventually upload it to 

the Grants Management System.

The edit check prompts grantees if they make 

obvious keystroke errors. For example, grantees cannot enter a grant 

amount for more than the grant award given. The system also notifies 

grantees if they enter a number that is inconsistent with a previous 

response.

25

 

23JAG purpose areas are: law enforcement programs; prosecution and court programs; 

prevention and education programs; corrections and community corrections programs; 

drug treatment and enforcement programs; planning, evaluation, and technology 

improvement programs; and crime victim and witness programs (other than

compensation).

For example, in the law enforcement 

purpose area, a qualitative measure asks grant recipients to describe any 

accomplishments during the reporting period along with any benefits or 

changes observed as a result of JAG-funded activities. The data allow the 

Bureau of Justice Assistance (BJA) to report progress toward its bureaulevel goals as identified in its strategic plan as well as for the annual JAG 

24According to Justice officials, they refer to the edit checks as data validation and 

verification rules. 

25The Grants Management System is a web-based application that provides support for 

the application, award, and management of awards at OJP. 

Justice Assistance Grant: 

Performance Data Collection, 

Validation, and Reporting
 

Page 13 GAO-13-552 Grants Performance

report.26

JABG primary grant recipients report data in support of core performance 

measures annually and financial performance quarterly. These core 

measures include the number and percent of programs and initiatives 

employing evidence–based programs and the number of youth served by 

JABG-funded programs. Grant recipients are also required to report at 

least one output or outcome measure for each of the 17 purpose areas 

under which they apply for funding.

For example, Justice officials pointed to the direct link between 

JAG performance measures on drug courts and BJA’s second major 

focus area of reduction of recidivism and prevention of unnecessary 

confinement.

27

 

26The Bureau of Justice Assistance Strategic Plan for fiscal years 2013 to 2016 has five 

major focus areas: reduction of violent crime, the improvement of community safety, and 

support for public safety officers; reduction of recidivism and prevention of unnecessary 

confinement; integration of evidence-based, research-driven strategies into the day-to-day 

operations of BJA and the programs BJA administers and supports; increasing program 

effectiveness with a renewed emphasis on data analysis, information sharing, and 

performance management; and ensuring organizational excellence. 

JABG grant recipients enter grant 

performance information into the Office of Juvenile Justice and 

Delinquency Prevention’s (OJJDP) Data Collection and Technical 

Assistance Tool (DCTAT), an electronic data collection system designed 

to help grantees compile performance data and make sure they are in the 

correct format for submission to OJJDP. Justice officials explained that 

there are some DCTAT system-level data validation and verification 

checks in place to prevent data entry errors. In addition, OJJDP reviews 

the aggregate data findings and grantee-level data reports for any 

obvious errors or inconsistencies. The contractor who manages the 

DCTAT system and the grant managers themselves verify and validate 

the data. Justice officials explained that a data validation tool has been 

piloted for OJJDP’s Title II Formula Grants Program, but is not yet 

27In fiscal year 2012, JABG purpose areas were: graduated sanctions, facilities, hiring 

court staff/pretrial services, hiring prosecutors, funding for prosecutors, training law 

enforcement/court personnel, gun courts, drug courts, juvenile records, information 

sharing, accountability-based programs, risk/needs assessments, school safety, 

restorative justice, court/probation programming, hiring detention/corrections staff, and 

reentry.

Juvenile Accountability 

Block Grants: Performance 

Data Collection, Validation, 

and Reporting
 

Page 14 GAO-13-552 Grants Performance

available OJJDP-wide.28 However, Justice officials noted that as of 

February 2013, OJJDP established a policy which states the 

department’s intent to implement the use of this new data validation and 

verification tool for future use with JABG grantees. Along with quantitative 

performance data entered into the DCTAT, grantees provide narrative 

descriptions of their projects. According to Justice officials, these 

descriptions help JABG grant managers compare descriptions to 

performance data and understand the nuances among different 

programs.

To support the National Preparedness Goal, FEMA uses the Emergency 

Management Performance Grants (EMPG) and the Assistance to 

Firefighters Grants (AFG) to enhance core capabilities of state 

governments (through EMPG) and fire departments and local emergency

medical services organizations (through AFG), its primary grant recipients 

(see app. II).29 Like Justice, the Department of Homeland Security 

developed agency priority goals that reflect agency-wide, near-term 

priorities.30

According to FEMA officials, the EMPG and AFG programs have an 

indirect link to an agency priority goal, as well as to the NPG, because 

both programs support states’ overall level of disaster preparedness. For 

example, EMPG-supported activities, such as building or sustaining 

capabilities through personnel training, plan development, and 

participation in exercises, may be specifically reported in the State 

 

28The Title II Formula Grants Program supports state and local efforts to plan, establish, 

operate, coordinate, and evaluate projects directly or through grants and contracts with 

public and private agencies for the development of more effective education, training, 

research, prevention, diversion, treatment, and rehabilitation programs in the area of 

juvenile delinquency and programs to improve the juvenile justice system.

29The AFG Fire Department funding priorities for 2012 are: fire operations and firefighter 

safety, and firefighting vehicles. The 2012 EMPG program objectives include: approved 

emergency plans; completion of Threat and Hazard Identification and Risk Assessment; 

development and maintenance of multi-year Training and Exercise Plans; and targeted 

training and verification of personnel capability.

30For 2012 to 2013, DHS’s three agency priority goals are: strengthen aviation security 

counterterrorism capabilities by using intelligence-driven information and risk-based 

decisions; improve the efficiency of the process to detain and remove criminal aliens from 

the United States; and ensure resilience to disasters by strengthening disaster 

preparedness and response capabilities.

FEMA Collects Grant 

Performance Data through 

Several Broad 

Preparedness Reporting 

Mechanisms, but Faces 

Challenges Identifying 

Outcomes at the Agency 

Level
 

Page 15 GAO-13-552 Grants Performance

Preparedness Reports (SPR). The SPR is one of FEMA’s evaluation tools 

used to gather data on the progress, capabilities, and accomplishments of 

states’ preparedness programs, and these state-reported assessments 

are linked to the agency priority goal for ensuring resilience to disasters. 

AFG performance data are self-reported by fire houses and local 

emergency medical services organizations but may have an impact on 

NPG core capabilities and therefore may be included in each state’s SPR. 

However, identifying the specific outputs or impacts of EMPG and AFG 

from within this broad reporting mechanism is not always possible 

because states do not always report on the performance of these grants 

individually in their SPRs.31

As shown in figure 2, FEMA collects EMPG grant performance data 

through multiple reporting mechanisms. All EMPG primary grant 

recipients are required to report on progress toward the EMPG program 

priorities. FEMA establishes its administrative and federal financial 

reporting requirements in its grant solicitation. The grant solicitation 

requires applicants to submit a work plan that includes a program and 

budget narrative, personnel data table, training data table, exercise data 

table, and grant activities outline. FEMA has developed the Threat and 

Hazard Identification and Risk Assessment (THIRA) and related guidance 

as a tool to help states identify and assess their risks or greatest 

hazards.

A FEMA official noted that FEMA has its own 

performance group and it has access to the DHS PIO and OMB 

performance guidance as needed.

32 States, territories, and urban areas submit THIRAs to FEMA. 

States and territories submit the SPR to FEMA. The SPR assesses a 

state’s current preparedness level against its THIRA targets.33

 

31Although EMPG and AFG performance measures are not always included in states’ 

SPRs, FEMA has developed management activity performance measures that assess 

management practices for the preparedness grant programs including EMPG and AFG. 

For example, these measures include the percent of preparedness grant awards closed 

within 90 days and the percent of Federal Financial Reports submitted on time.

The SPR 

and the THIRA data assist FEMA in annually summarizing progress 

across all 31 preparedness core capabilities. A range of reports—

32FEMA developed the Comprehensive Preparedness Guide 201: Threat Hazard 

Identification and Risk Assessment Guide to provide a comprehensive approach for 

identifying and assessing risks and associated impacts.

33Section 652(c) of the Post-Katrina Emergency Management Reform Act of 2006 (Pub. L. 

No. 109-295) requires any state that receives federal preparedness assistance to submit 

an SPR to FEMA.

Emergency Management 

Performance Grants: 

Performance Data Collection, 

Validation, and Reporting
 

Page 16 GAO-13-552 Grants Performance

Quarterly Performance Progress Reports, Biannual Strategy 

Implementation Reports, and quarterly Federal Financial Reports are 

submitted by the primary recipients to the FEMA regions; however, review 

of these reports varied from region to region.34,35 One FEMA regional 

official said that the FEMA analysts in their region review the quarterly 

financial and program reports for timeliness, accuracy, grantee match 

dollars, and level of funds drawn down.36

 

34The Quarterly Performance Progress Report is submitted to the region through the NonDisaster Grants Management System (ND Grants System).

Another FEMA regional official 

said that in their region, they review the quarterly financial and program 

reports to compare the recipient’s application work plan with actual 

progress reported. One FEMA regional official commented that even 

though it is labor intensive to read the programmatic narrative reports, 

these reports help regional officials manage grantees by identifying 

successes and challenges with the grant.

35The Biannual Strategy Implementation Report is a reporting requirement submitted by 

states to FEMA on the progress of certain grants. 

36The Federal Financial Report (also known as form SF 425) is submitted quarterly 

through the Payment and Reporting System.
 

Page 17 GAO-13-552 Grants Performance

Figure 2: Upward Flow of Programmatic and Financial Performance Data for the 

Emergency Management Performance Grants (EMPG)

FEMA officials said that not all states identify the impact of EMPG 

explicitly within their SPR. Officials explained that the SPR provides the 

opportunity to include grant results, and states may use data on 

programs, such as training and exercises, funded through EMPG grants 

in this report. The 2012 EMPG grant solicitation stated that the program 

supports all core capabilities and lists as one of the EMPG objectives the 

development and maintenance of multi-year Training and Exercise 
 

Page 18 GAO-13-552 Grants Performance

Plans.37

AFG establishes its performance reporting requirements in the grant 

solicitation. The FEMA regions collect grant performance data including 

obligation and expenditure data through the Semi-Annual Programmatic 

Performance Reports. Additionally, primary recipients must report 

obligations and expenditures semi-annually using the Federal Financial 

Reports. Grant recipients submit performance reports through the eGrants system, and FEMA may withhold future awards and fund 

drawdowns if these reports are delinquent. (See fig. 3.)

These trainings and exercises funded by EMPG may be reported 

on and identified in the SPR. For example, one state reported in its 2012 

SPR that exercises and trainings are planned and plans are in place to 

mobilize hazardous incident teams and hazmat mutual aid. The various 

EMPG reporting mechanisms, including the quarterly programmatic and 

financial reports, are used by FEMA units for different purposes such as 

checking on project cash-on-hand or for compliance with the work plan. 

The result is that performance data are used to report progress toward an 

agency priority goal as well as the NPG.

 

37The core capabilities contained in the NPG are highly interdependent and require FEMA 

to use existing preparedness networks and activities, improve training and exercise 

programs, promote innovation, and ensure that the administrative, finance, and logistics 

systems are in place to support these capabilities.

Assistance to Firefighters 

Grants: Performance Data 

Collection, Validation, and 

Reporting
 

Page 19 GAO-13-552 Grants Performance

Figure 3: Upward Flow of Programmatic and Financial Performance Data for the Assistance to Firefighters Grants (AFG)

According to FEMA officials, regional office staff periodically monitor AFG 

recipients to ensure that the project goals, objectives, performance 

requirements, timelines, milestones, budgets, and other related program 

criteria are being met. FEMA regional staff monitor the AFG grantees by 

desk-based review, on-site monitoring visits, or both. Monitoring involves 

the review and analysis of proposed work plans compared with actual 

progress toward financial drawdowns and compliance with administrative 

processes, policies, and activities. Monitoring results in identifying areas 
 

Page 20 GAO-13-552 Grants Performance

where technical assistance, corrective actions, and other support may be 

needed. According to FEMA officials, the only specific outcome measure 

for AFG is firefighter fatality and injury data. These data are from the 

National Fire Incident Reporting System (NFIRS).

38 While FEMA reviews 

program activity, aggregated analysis of AFG specific outputs is not 

readily available.

OMB’s Circular No. A-11 directs agencies to have a data validation plan 

for performance reporting that uses verification and validation techniques 

including an assessment by agency heads of reliability and completeness 

of reported performance data. Agencies should describe how they are 

ensuring the accuracy and reliability of the data used to measure 

progress towards performance goals. Circular No. A-11 makes reference 

to two of our reports when defining verification as the process of checking 

or testing performance data to assess for errors and validation as 

ensuring that data are free of systematic error or bias and that what is 

intended to be measured is actually measured.

39 According to the 

guidance in Circular No. A-11, data limitations can lead to bad decisions 

resulting in lower performance or inaccurate performance assessments, 

and data limitations can include imprecise measurement and recordings, 

incomplete data, and inconsistencies in data collection procedures. In 

addition, we have previously found that maintaining consistent data over 

time requires using consistent data collection procedures for ongoing data 

collections.40

 

38NFIRS is a data base of fire incident information initiated and supported by the U.S. Fire 

Administration. Fire departments report their fire incident information to states or directly to 

NFIRS. 

Consistent verification and validation of performance data 

support the general accuracy and reliability of performance information, 

reduce the risk of inaccurate performance data, and provide a sufficient 

level of confidence to the Congress and the public that the information 

presented is credible as appropriate to its intended use.

39GAO, Performance Plans: Selected Approaches for Verification and Validation of 

Agency Performance Information, GAO/GGD-99-139 (Washington, D.C.: July 1999); and 

The Results Act: An Evaluator’s Guide to Assessing Agency Annual Performance Plans, 

GAO/GGD-10.1.20 (Washington, D.C.: April 1998).

40GAO, Federal Telework: Program Measurement Continues to Confront Data Reliability 

Issues, GAO-12-519 (Washington, D.C.: Apr. 19, 2012).

FEMA’s Reliance on 

Self-Reported Data and 

Lack of a Data Verification 

Tool Present Reliability 

Concerns
 

Page 21 GAO-13-552 Grants Performance

According to FEMA officials, neither EMPG nor AFG currently has a 

standardized tool or process to validate program data. EMPG and AFG 

recipients provide self-reported performance data to FEMA headquarters 

and the 10 FEMA regions. Some performance reporting reviews, which 

FEMA officials explained may serve as performance information 

verification, are conducted at the regional level. However, the regions are 

inconsistent in their approaches to verifying EMPG and AFG performance 

information from the primary recipients. As previously mentioned, we 

reported in a March 2011 testimony that FEMA officials said the 

evaluation efforts that collected data on national preparedness 

capabilities were useful, but they said the data were limited by reliability 

and measurement issues related to the lack of standardization in the 

collection of the data.41

All EMPG primary grantees are required to report on progress toward the 

EMPG program priorities. However, all of the reporting documents 

contain self-reported data on states’ general level of preparedness with 

respect to core capabilities that support the NPG. Verification and 

validation of EMPG data take place on a case-by-case and region-byregion basis. FEMA regional officials told us they carry out desk and file 

reviews. On a quarterly basis, they may compare the application work 

plan with performance reports to track actual versus planned progress 

and they may look at the quarterly financial reports to check on grantee 

drawdown of the grant obligation. Additionally, depending on the region, 

desk reviews may be completed on a 1- or 2-year cycle. Other 

performance reporting mechanisms, such as the Biannual Strategy 

Implementation Reports, may be pulled for review; however, according to 

FEMA regional officials, this is not a consistent practice across all 

regions. Regional officials also said that as part of their monitoring 

responsibilities, they are sometimes present at emergency planning 

exercises and meetings conducted by primary recipients and they 

consider this a type of performance data verification and validation. A 

FEMA official said that, FEMA has conducted two nationwide reviews 

required by Congress that provide the status of catastrophic planning in 

all the states and the largest urban areas, as well as support data 

The current absence of a formal established data 

verification procedure, as directed by Circular No. A-11, could lead to the 

collection of erroneous performance data.

 

41GAO-11-260T
 

Page 22 GAO-13-552 Grants Performance

validation for these efforts. The first review took place in 2006, with a 

subsequent review in 2010; a date has not been set for the next review.42

FEMA officials said that the AFG grants management reporting system 

does not have a data validation tool and because of the high number of 

recipients, along with numerous data points to be verified on each AFG 

recipient’s Semi-Annual Programmatic Performance Report, data 

validation is a challenge.43

As discussed above, we found that the only outcome measure currently 

available for AFG is the firefighter fatality and injury data available 

through NFIRS, which provides one measure for the program’s 

performance. AFG and NFIRS staff are working to create a new system 

for performance assessment. We have previously reported on the general 

attributes of appropriate grant performance measures, noting that 

measures should be linked to agency goals and missions; be clearly 

stated; include measurable targets; and be objective, reliable, and 

balanced.

Officials also said that AFG performance data 

verification takes place on a case-by-case basis (individually) at the 

FEMA regional level. One FEMA regional official reported that in their 

region, reviews are all site visits—both scheduled and unscheduled.

According to a FEMA headquarters official, FEMA is beginning to apply a 

risk-based approach to AFG monitoring. Specifically, regional analysts 

will visit grantees who received large grant awards, as well as grantees 

who are implementing region-wide projects.

The Fire Grants Reauthorization Act (the act) of 2012, signed into law in 

January 2013, requires FEMA to establish a performance assessment

system that includes quantifiable performance metrics to evaluate the 

extent to which grants awarded are furthering the purposes of AFG, 

including protecting the health and safety of the public and firefighting 

personnel.44

 

42Department of Homeland Security, Nationwide Plan Review: Phase I Report

(Washington, D.C.: 2006); and Department of Homeland Security, Nationwide Plan 

Review: Fiscal Year 2010 Report to Congress (Washington, D.C.: 2010).

The act also requires that the Administrator of FEMA submit 

to the Committee on Homeland Security and Governmental Affairs, the 

43According to FEMA officials, AFG has more than 3,000 primary recipients a year. 

44Pub. L. No. 112-239, Div. A, Title XVIII, Subtitle A, § 1803, 126 Stat. 1632, 2100 (Jan. 2, 

2013) (15 U.S.C. § 2229(p)(2)). 

AFG Lacks a Performance 

Assessment System with 

Detailed 

Outcome Measures 
 

Page 23 GAO-13-552 Grants Performance

Committee on Science and Technology, and the Committee on 

Transportation and Infrastructure a report that provides information on the 

performance assessment system developed, using the performance 

metrics developed, and an evaluation of the effectiveness of the grants 

awarded by September 30, 2013 and each year thereafter through 2017.

If the system, which FEMA is currently developing, incorporates Circular 

No. A-11’s requirements for data validation and verification, it should 

enable FEMA to identify the specific impacts and outcomes related to 

AFG.

The act’s requirements are consistent with issues raised in a 2007 report 

published by a National Academy of Public Administration (NAPA) panel 

and that are still present today.45 The NAPA panel was formed and its 

report issued at the request of DHS and AFG. The report assesses AFG’s 

strategic position and offers suggestions on how FEMA might plan and 

manage based on performance. NAPA’s report was an effort to help 

identify potential new strategic directions for AFG and to provide it with 

advice on how it could most effectively plan, manage, and measure its 

accomplishments.46 The NAPA report concluded, among other things,

that AFG program metrics focused on inputs, financial management, and 

characteristics of the grant recipients rather than on accomplishing longterm goals or achieving intermediate improvements in public safety. 

NAPA provided an extensive analysis and made several 

recommendations, noting that there was no prescribed way for AFG 

grantees to analyze or report on whether they had realized the expected 

outcomes for the use of the grant funds. The report also included metrics 

that were proposed during stakeholder interviews and focus groups and 

recommended by the panel for consideration by AFG.

 

45NAPA is a nonprofit, independent coalition of public management and organizational 

leaders. 

46James Kunde, Paul D. Brooks, Glenn Corbett, Harry Hatry, Bruce D. McDowell, Darrell 

W. Stephens, Assistance to Firefighters Grant Program: Assessing Performance 

(Washington, D.C.: NAPA 2007).
 

Page 24 GAO-13-552 Grants Performance

Effectively tracking and reporting on the performance of grants is critical 

for agencies to determine whether their program goals are being 

achieved and to ensure transparency and accountability for federal grant 

spending. Accordingly, GPRAMA requires that federal agencies collect 

and validate performance data used to report progress toward 

performance goals. OMB Circular No. A-11 provides detailed guidance for 

the agency goal management process, as well as the related 

performance data verification and validation process which supports both 

reporting progress toward goals and data reliability efforts in general. We 

found that Justice and FEMA program offices collect performance data 

through various mechanisms for the selected grant programs. However, 

grant performance data for FEMA’s EMPG and AFG programs are selfreported by states, fire houses, or emergency medical services 

organizations as part of their overall preparedness assessment reports, 

and FEMA regional officials have varied and inconsistent approaches to 

data verification for these programs. The lack of a consistent approach to 

verifying data could lead to the collection of erroneous performance data. 

Additionally, while FEMA is collecting AFG performance data, it does not 

have a performance measurement system with quantifiable performance 

metrics linked to agency goals, clearly stated measurable targets, and 

reliable data. Such a system, which is being developed by FEMA, would 

help it to analyze or report on whether expected outcomes have been 

realized for the use of the grant funds.

To improve the collection and validation of performance data at FEMA, 

we recommend that the Administrator of FEMA take the following action:

• Ensure, in accordance with the guidance in OMB Circular No. A-11, 

that there are consistent procedures in place at the program office 

and regional level to promote verification and validation of grant 

performance data that allow the agency to attest to the reliability of 

EMPG and AFG grant data used for reporting progress toward goals.

We provided a draft of this report for comment to the Assistant Attorney 

General for Administration, Department of Justice, the Director of the 

Office of Management and Budget, and the Secretary of the Department 

of Homeland Security. Justice had no technical or agency comments on 

our report. OMB provided technical comments which we incorporated as 

appropriate. DHS provided comments in a letter signed by the Director of 

the Departmental GAO-OIG Liaison Office, which we have reprinted in 

Conclusions 

Recommendation for 

Executive Action

Agency Comments 

and Our Evaluation
 

Page 25 GAO-13-552 Grants Performance

Appendix III, and in technical comments, which were incorporated into the 

report as appropriate. DHS concurred with our recommendation and 

stated that FEMA would explore effective and affordable ways to verify 

and validate grant performance data for EMPG and AFG.

We are sending copies of this report to the interested congressional 

committees, the Assistant Attorney General for Administration at Justice, 

the Director of OMB, the Secretary of Homeland Security and other 

interested parties. In addition, the report is available at no charge on the 

GAO website at http://www.gao.gov. 

If you or your staff have any questions about this report, please contact 

me at (202) 512-6806 or czerwinskis@gao.gov. Contact points for our 

Offices of Congressional Relations and Public Affairs may be found on 

the last page of this report. Major contributors to this report are listed in 

appendix IV. 

Stanley J. Czerwinski 

Director, Strategic Issues
Appendix I: Objectives, Scope, and 

Methodology

Page 26 GAO-13-552 Grants Performance

This report examines (1) the extent to which the Office of Management 

and Budget (OMB) provides guidance to agencies on how to collect, 

validate, and report performance information that demonstrates progress 

toward agency-wide and federal goals; and (2) how selected agencies 

collect and validate information on the performance of selected grant 

programs and use that information to report progress toward their 

agency-wide performance goals. 

We interviewed officials at OMB’s Office of Performance and Personnel 

Management and the Office of Federal Financial Management, who were 

responsible for developing and disseminating guidance to executive 

agencies on grant performance information collection and validation to 

determine the type and extent of guidance offered by OMB to agencies 

when developing grant performance measurement systems. To 

determine which agencies and grant programs to review, we researched 

programs and agencies that provided a variety of grants in terms of grant 

size (obligation amounts), type (project/formula; categorical/block), 

grantee type (state or local government), and links to agency goals. We 

selected agencies and programs that leveraged and built upon our prior 

knowledge of selected grant programs and related performance reporting. 

We interviewed the officials at the Department of Homeland Security 

(DHS) and Federal Emergency Management Agency (FEMA) who were 

responsible for the Emergency Management Performance Grants and 

Assistance to Firefighters Grants program administration, as well as the 

DHS component that receives FEMA grant performance data to 

understand how grant performance data are collected, validated and used 

to report progress toward goals and manage programs. We also 

interviewed officials from the FEMA regions which allowed us to 

corroborate information and understand program management and data 

validation procedures for the related grant programs. We also interviewed 

the officials from the Department of Justice who manage the Justice 

Assistance Grant and Juvenile Accountability Block Grants programs, as 

well as Justice officials from the Office of Justice Programs and the 

Justice Management Division to better understand collection, validation, 

and use of grant performance data to report progress toward goals and 

manage programs.

We conducted a literature review to identify performance management 

issues raised, including those in our previous reports, on the selected 

grant programs at Justice and FEMA or by their Offices of Inspectors 

Appendix I: Objectives, Scope, and 

Methodology 
Appendix I: Objectives, Scope, and 

Methodology

Page 27 GAO-13-552 Grants Performance

General, the Congressional Research Service, or the Congressional 

Budget Office. We also reviewed the Government Performance and 

Results Act (GPRA) of 1993,1 as updated by the GPRA Modernization 

Act (GPRAMA) of 2010,2

We conducted this performance audit from May 2012 to June 2013 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

and hearings related to GPRAMA, along with 

relevant federal goals as well as agency goals established through

GPRAMA to determine any link to selected grant programs. Additionally, 

we reviewed government websites that report on government-wide 

performance, grant obligations, and grant types to identify relevant federal 

government-wide and agency goals established through GPRAMA and to 

select agencies and programs that leveraged and built on our prior 

knowledge of selected grant programs and related performance reporting. 

We reviewed relevant Justice and FEMA websites and grant solicitations

and related agency documents to understand grant performance 

measurement requirements for the selected grant programs.

 

1

Pub. L. No. 103-62, 107 Stat. 285 (Aug. 3, 1993). 

2 Pub. L. No. 111-352, 124 Stat. 3866 (Jan. 4, 2011). 
Appendix II: The National Preparedness Goal 

Mission Areas and Core Capabilities

Page 28 GAO-13-552 Grants Performance

Presidential Policy Directive 8: National Preparedness (PPD-8)

describes the nation’s approach to preparing for the threats and hazards 

that pose the greatest risk to the security of the United States. The 

nation’s security and resilience posture is described through the core 

capabilities comprised by the five mission areas.

Core capabilities are essential for the execution of each of the five 

mission areas (see below). To assess both the nation’s capacity and 

gaps, each core capability includes capability targets for which measures 

will be developed. The core capabilities and capability targets are not 

exclusive to any single level of government or organization, but rather 

require the combined efforts of the whole community.

Appendix II: The National Preparedness Goal 

Mission Areas and Core Capabilities

Core Capabilities
Appendix II: The National Preparedness Goal 

Mission Areas and Core Capabilities

Page 29 GAO-13-552 Grants Performance

Table 1: Core Capabilities by Mission Area

Prevention Protection Mitigation Response Recovery

• Planning

• Public Information and Warning

• Operational Coordination

• Forensics and 

Attribution

• Intelligence and 

Information Sharing

• Interdiction and 

Disruption

a

• Screening, Search, 

and Detection

a

• Access Control and 

Identity Verification

a

• Cybersecurity

• Intelligence and 

Information Sharing

• Interdiction and 

Disruption

a

• Screening, Search, 

and Detection

a

• Physical Protective 

Measures

a

• Risk Management 

for Protection 

Programs and 

Activities

• Supply Chain 

Integrity and 

Security 

• Community 

Resilience

• Long-term 

Vulnerability 

Reduction

• Risk and Disaster 

Resilience 

Assessment

• Threats and Hazard 

Identification

• Critical 

Transportation

• Environmental 

Response/ Health 

and Safety

• Fatality 

Management 

Services 

• Infrastructure 

Systems

• Mass Care Services

b

• Mass Search and 

Rescue Operations

• On-scene Security 

and Protection

• Operational 

Communications

• Public and Private 

Services and 

Resources

• Public Health and 

Medical Services

• Situational 

Assessment

• Economic Recovery

• Health and Social 

Services 

• Housing

• Infrastructure 

Systems

• Natural and Cultural 

Resources

b

Source: National Preparedness Report, March 30, 2012.

Note: Planning, Public Information and Warning, and Operational Coordination are core capabilities 

common to all mission areas.

a Intelligence and Information Sharing, Interdiction and Disruption, and Screening, Search, and 

Detection are core capabilities for both Prevention and Protection

b Infrastructure Systems is a core capability for both the Response and Recovery Mission Areas. 
Appendix III: Comments from the Department 

of Homeland Security

Page 30 GAO-13-552 Grants Performance

Appendix III: Comments from the 

Department of Homeland Security
Appendix III: Comments from the Department 

of Homeland Security

Page 31 GAO-13-552 Grants Performance
Appendix IV: GAO Contact and Staff 

Acknowledgments

Page 32 GAO-13-552 Grants Performance

Stanley J. Czerwinski, (202) 512-6806 or czerwinskis@gao.gov

In addition to the contact named above, Thomas M. James, Assistant 

Director, Keith O’Brien, analyst-in-charge, and Laurel Beedon made 

major contributions to this report. David Mauer, Joy Booth, Elizabeth 

Curda, Chris Keisling, and Sarah Veale provided key guidance 

throughout the engagement. Other key contributors included Cynthia 

Saunders, Amy Bowser, Robert Gebhart, Kim McGatlin, Beryl Davis, and 

Donna Miller.

Appendix IV: GAO Contact and Staff 

Acknowledgments 

GAO Contact:

Staff 

Acknowledgments: 

(450995)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations 

Public Affairs

Please Print on Recycled Paper.