Fire Grants: FEMA Could Enhance Program Administration and Performance Assessment
Released: 2016-09-15T00:00:00

Letter

Background

Fire Grant Roles and Responsibilities

FEMA’s GPD is responsible for managing DHS’s suite of preparedness grants, including three fire grants designed to provide funding for local fire departments, emergency medical service (EMS) organizations, and firefighter and fire prevention interest groups. GPD’s AFG Program Office administers the three fire grant programs:

AFG program grants fund personal protective equipment, vehicles, and other operational equipment for fire service and EMS organizations, as well as projects to modernize facilities, deliver training, and develop health and fitness programs.

SAFER grants fund the hiring of salaried firefighters and the costs of recruitment and retention of volunteer firefighters.

FP&S grants fund research and community initiatives, such as installing smoke detectors among high-risk populations or performing fire prevention research studies to support firefighter safety.

Figure 1 illustrates the general uses for each fire grant program, and the award funding available in fiscal year 2014.

Figure 2 shows the grant award levels for each fire grant program. For further information on annual funding levels, see appendix I table 3.

The U.S. Fire Administration (USFA) is another component in FEMA. Its mission is to provide national leadership for fire and emergency services stakeholders in prevention, preparedness, and response efforts. USFA initially administered the AFG program when it was established in December 2000, and DHS subsequently transferred program management responsibilities in fiscal year 2004. Since 2005, report language accompanying appropriations acts directed FEMA to fund fire grant applications according to local priorities and those established by USFA. USFA’s goals are to reduce fire and life safety risk through preparedness, prevention and mitigation, promote response, local planning and preparedness for all hazards, enhance the fire and emergency services’ capability for response to and recovery from all hazards, and advance the professional development of fire service personnel and of other people engaged in fire prevention and control. USFA staff also work with local responders to increase national capacity of firefighters by providing free training and education programs that support fire departments and emergency services organizations in preparing for, preventing and responding to fires and other hazards through the USFA National Fire Academy.

FEMA’s National Preparedness Assessment Division is responsible for assessing national “whole community” preparedness to, among other things, analyze and illustrate the impact of GPD’s preparedness grants. Officials conduct qualitative and quantitative assessments of 32 core capabilities that support the National Preparedness Goal by, among other things, analyzing State Preparedness Reports (SPR) and developing the National Preparedness Report. According to FEMA, the 32 national preparedness core capabilities provide a common standard to integrate federal, state, local, and tribal preparedness efforts. States use the core capabilities to conduct their Threat and Hazard Identification Risk Assessments (THIRA) and submit annual SPRs to FEMA on the level of that state’s preparedness, as a requirement for non-fire preparedness grants. In September 2015, FEMA updated the National Preparedness Goal to, among other things, include a new core capability for Fire Management and Suppression. Beginning in 2017, states will be required to include an assessment of their level of fire management and suppression capabilities to FEMA as a requirement to receive non-fire preparedness grants, according to FEMA officials.

Fire Grants Application and Awards Process

Each year, FEMA organizes a panel of subject matter experts to identify that year’s funding priorities and criteria for fire grants., Following review and discussion of the previous year’s information, the criteria development panel submits its recommendations for funding priorities and grant award criteria—including those on which participants do not reach consensus—in a report to GPD. After considering the panel’s recommendations and developing the grant guidance for the three grant programs, GPD submits the proposed grant guidance to other offices within FEMA for internal review. Subsequently, GPD sends the guidance to OMB for approval. Following approval, FEMA announces the availability of each grant program, providing Notices of Funding Opportunity online to potential applicants.

The grant application review process for all three competitive grant programs includes three evaluation stages: an automated scoring or manual prescreening process to determine eligibility and alignment with the grant programs’ funding priorities; a peer review panel, in which practitioners within the fire service community evaluate and score the applications; and a final technical review by subject matter specialists and GPD officials. The grant review and award processes are represented in Figure 3.

FEMA Changed Fire Grants Policies and Developed a Performance Assessment System in Response to FGRA Amendments

FEMA Changed Applications and Awards Policies Related to FGRA Amendments, and Grant Outcomes Generally Reflected These Changes

Our analysis of FGRA identified 26 amendments related to fire grant application and award policies. FEMA incorporated 16 of these 26 changes into its existing grant application and awards processes (see table 1). For the remaining 10 cases, FEMA used existing authorities to waive changes related to FGRA amendments or found that no change to existing policy was necessary. See Appendix II for a summary of these amendments and a detailed description of FEMA’s corresponding changes to the grant programs (see app. II, table 4).

The number of fire grants applications and awards from 2009 through 2014 are shown in Figure 4 and the outcomes associated with specific amendments and related policy changes are described below.

AFG Application Policy Changes—Four of the 26 FGRA amendments are related to application policies for the AFG program. FEMA addressed two of the amendments by expanding the criteria for eligible AFG applicants. For example, state fire training academies are now eligible to apply for fire grants, and two or more entities can now submit regional applications for a grant to fund a joint program or initiative. For the remaining two amendments, FEMA officials identified existing grant policies related to eligible grant uses or considerations required by FGRA when awarding AFG grants, and did not make changes since these policies were already in place. For example, AFG prioritizes firefighter personal protective equipment grants to increase the degree to which an award will reduce deaths and injuries among firefighters. Although FEMA expanded the criteria for eligible AFG applicants, AFG grant applications decreased from about 19,800 in fiscal year 2009 to about 10,000 in fiscal year 2014. FEMA officials and national fire service associations attributed the decrease, in part, to application fatigue—when departments give up after not receiving an award after applying over multiple years—and the increase in administrative requirements and the associated workload. According to FEMA officials and national fire service association representatives, these factors may have affected decreasing application rates for SAFER and FP&S grants as well.

AFG Award Policy Changes—Seven of the 26 FGRA amendments are related to FEMA’s award policies for the AFG program, which FEMA implemented by changing its AFG award policies. Specifically, FGRA included an amendment that established minimum award allocations for career, volunteer, and combination fire departments. Prior to FGRA, the distribution of AFG funds among career, combination, and volunteer departments fluctuated more across fiscal years. For example, volunteer departments received 43 percent of fiscal year 2009 funds, and 25 percent of fiscal year 2011 funds. After implementation of FGRA in the fiscal year 2013 grant cycle, career, combinations and volunteer fire departments received slightly more than 30 percent of AFG funds in fiscal years 2013 and 2014 (see app. III table 5). FGRA also requires that FEMA make not more than 3 percent of AFG funds available for grants to state fire training academies, with a $1 million grant maximum. Previous policy did not allow state academies to apply, so FEMA changed AFG policy accordingly and set a $500,000 grant maximum. In both fiscal years 2013 and 2014, state fire training academies received 3 percent of AFG funding and no academies were awarded grants larger than $500,000 (see app. III table 8). According to FEMA officials, they reached an agreement with the national fire service organizations to limit the grants to $500,000 to ensure that a larger number of academies could receive grants. FEMA implemented other FGRA amendments requiring that no individual AFG grant exceeds one percent of total AFG funding, revising grantee cost matching requirements for jurisdictions of various sizes, and allowing FEMA to waive the revised grantee cost matching requirements for economic hardship.

SAFER Application Policy Changes—One of the 26 FGRA amendments is related to application policies for the SAFER program. Specifically, FGRA makes national and tribal firefighter interest organizations eligible to apply for SAFER Recruitment and Retention grants. Under previous application policy, national organizations were ineligible to apply and tribal organizations were eligible but had to classify themselves as local entities in their applications. FEMA changed SAFER policies to allow national organizations to apply for recruitment and retention grants and for tribal organizations to be classified separately from local organizations beginning in the fiscal year 2013 SAFER grant cycle.

SAFER Award Policy Changes—Six of the 26 FGRA amendments are related to FEMA’s award policies for the SAFER program, establishing the length of the grant period and amounts for local cost sharing, among other changes. FEMA waived all six of these requirements for the SAFER program, citing provisions in the American Recovery and Reinvestment Act and annual Fire Grant appropriations as their authority to do so. These post-award requirements have been waived by FEMA since fiscal year 2009. FEMA officials said that they chose to waive the award policy amendments to provide greater flexibility to grantees since the award conditions for SAFER grants, both in FGRA and in previous law, are too restrictive and grantees frequently defaulted on their cost matching and performance responsibilities. As a result, SAFER grants have been 100 percent federally funded from fiscal year 2009 through fiscal year 2015. This waiver authority was not included in the Consolidated Appropriations Act, 2016. Appendix III, table 4 provides a detailed description of the FGRA amendments to SAFER award policies waived by FEMA.

FP&S Application Policy Changes—Two of the 26 FGRA amendments are related to application policies for the FP&S program, with one amendment allowing FEMA to fund the establishment of up to three fire safety research centers using FP&S funds, and another specifying eligible uses of FP&S funds. Using existing authorities, FEMA decided not to implement policies that would allow applicants to receive grants for the establishment of fire safety research centers. According to FEMA officials, there was not a common understanding of intent among stakeholders, and it was difficult to define how these research centers would work within the FP&S program. Additionally, FEMA officials stated that establishing research centers would require grants with multi- year periods of performance and would not be consistent with other shorter term FP&S grants, and their establishment would be difficult due to the small amount of funding for FP&S.

FP&S Award Policy Changes—Six of 26 FGRA amendments are related to FEMA’s award policies for the FP&S program. FEMA implemented all six of these changes by revising FP&S award policies such as changing the minimum funding for FP&S grants as a percentage of the total grant funding, increasing the cap for FP&S grants from $1 million to $1.5 million, and changing the matching requirement for FP&S grants to 5 percent. According to FEMA officials, the total number of FP&S grants awarded decreased post-FGRA as a result of the increased grant size. Specifically, since there was not a corresponding increase in available FP&S funds, the result was FEMA giving fewer FP&S awards since these larger awards each take a larger proportion of FP&S funding. Due to the decrease in the number of awards, FEMA has prioritized FP&S projects that will have national benefits. FEMA officials expect that FP&S award numbers will increase, as FEMA instituted a policy of dedicating six percent of FP&S funds to micro-grants beginning in fiscal year 2015, following advice from stakeholders. These micro-grants will be for up to $25,000 each, so FEMA expects to award over 80 FP&S micro- grants in fiscal year 2015. See figure 5 for FP&S grant awards pre- and post-FGRA.

FEMA Developed a Performance Assessment System in Response to FGRA

FGRA also required that FEMA establish a performance assessment system to track grant program progress and report to Congress on the effectiveness of fire grants. In response, FEMA developed a fire grant performance assessment system with 10 performance measures for the AFG program and 11 more for the SAFER program. According to FEMA, each measure is aligned with the achievement of program goals and objectives, and many are based on fire service standards. According to FEMA, increased compliance with industry standards reduces fire department response times and enhances operational capacity. In addition, these standards help fire departments minimize the possibility of fire-related injuries, property damage, and other effects, according to FEMA. A complete listing of FEMA’s 21 performance measures is included in appendix IV.

AFG performance measures reflect the extent to which grant recipients purchase equipment that brings them into compliance with national standards of the National Fire Protection Association (NFPA), or the Department of Labor’s Occupational Safety and Health Administration (OSHA), and with state and local regulations (or otherwise provide health or safety benefits). FEMA developed measures to take into account the extent to which equipment is used to protect critical infrastructure, replace aging fire vehicles, or supports automatic and mutual aid. For example, in 2015, FEMA reported that AFG grants awarded during fiscal years 2008 through 2011, increased the average percent of all on-duty firefighters equipped with industry-compliant protective gear from 90.7 percent to 98.4 percent; enabled 99 percent of fire departments receiving a vehicle grant to replace a sub-standard vehicle from service; and increased the percent of awardees whose equipment purchases primarily benefitted the health and safety of their members, from 90.1 percent to 94.6 percent.

Similarly, the SAFER performance measures reflect the extent to which grant recipients conduct structural fire responses in compliance with industry standards, have agreements to provide aid to neighboring or regional organizations (mutual and automatic), estimate a department’s staffing needs based on a department’s risk or needs analysis, protect critical infrastructure, and hire veterans. For example, FEMA reported in 2014, that SAFER awards increased or maintained full compliance with NFPA fire department standards for 77 percent of awardees across career, volunteer, and combined departments during fiscal years 2009 through 2012; identified challenges for “mostly volunteer” SAFER grantees to sustain compliance with NFPA standards; and demonstrated FEMA’s compliance with the statutory requirement that at least 10 percent of SAFER funding is set-aside for volunteer fire departments, by exceeding the 10 percent minimum each award year, including 16 percent in fiscal year 2013.

Opportunities Exist to Further Enhance FEMA’s Management of Fire Grant Processes and Performance Assessment Efforts

Establishing Defined and Measurable Targets

While FEMA developed performance measures as part of its new performance assessment system for the AFG and SAFER programs, the agency did not include measurable performance targets linked to AFG and SAFER program goals that could help better assess the impact of the program and report program achievements. According to FEMA officials, they consulted with fire service representatives in developing the performance system and its measures, as required by FGRA. According to FEMA officials, they considered measurable targets associated with program goals in developing the performance assessment reports for 2014 and 2015, but chose to present data for each performance measure that reflected trends. For example, FEMA reported in 2015 that 74 percent, 78 percent, and 77 percent of AFG awardees for equipment in fiscal years 2009, 2010, and 2011, respectively, used the grants to achieve compliance with state, local, NFPA, or OSHA standards. However, measures like this did not include a target goal that would provide a basis to better assess AFG program performance, such as the desired percentage of awardees who used grants to achieve compliance with equipment standards.

We have previously reported that performance measures should have quantifiable targets to help assess whether goals and objectives were achieved by comparing projected performance and actual results. The National Academy of Public Administration identified the need for quantified targets for measures, and recommended baselines for measuring and predicting improvement in outcome measures after reviewing the AFG program in 2007. Thus, establishing measurable performance targets linked to AFG and SAFER program goals could enhance the quality and utility of the new performance assessment system by providing a yardstick against which the national investments in the programs can be evaluated. FEMA officials agreed that measureable performance targets could help them better assess and report on the results and impact of fire grants and said they are examining the possibility of developing measureable targets, along with clearer performance measures, to include in their 2016 report to Congress.

Substantiating Grantee Information on Program Goals

FEMA officials said they were implementing a process to verify and validate grantee information on program goals that could further enhance the performance assessment system, in response to a recommendation in our previous report. Specifically, we reported in June 2013 on the AFG program’s performance data collection efforts, noting at the time that FEMA relied on self-reported data that was collected from awardees’ progress reports, and that FEMA lacked data validation tools to ensure data reliability. We found that FEMA did not have a standardized tool or process to verify and validate the performance data that grant recipients provide to FEMA headquarters and FEMA’s 10 regional offices. We recommended that FEMA ensure that there are consistent procedures in place at the program office and regional level to validate and verify grant performance data that allow FEMA to attest to the reliability of data used to report progress toward goals. DHS concurred with our recommendation.

According to FEMA officials, to address our past recommendation they have developed quality assurance tools to verify and validate the post- award grant monitoring information that awardees are required to provide as part of their progress and close-out reports. However, they said that the regional offices have not yet implemented these tools and GPD is developing an implementation work plan that includes additional verification steps. GPD officials said they are negotiating a Memorandum of Understanding to provide structure and guidance to the 10 regional offices, and plan to distribute quality assurance tools to the regional offices, once this agreement is in place. As these efforts are ongoing, our recommendation will remain open until FEMA demonstrates that they have fully implemented the additional verification and validation steps. Substantiating the post-award, grant monitoring information that awardees are required to provide will be important, particularly in light of recently completed assessments of AFG and SAFER grants’ post-award activities by the DHS Office of Inspector General. Specifically, the Inspector General reported in June 2016 that GPD did not sufficiently manage or oversee AFG or SAFER grants, grantees did not comply with grant guidance and requirements, and, as a result, FEMA cannot ensure grant funds were spent appropriately or used for their intended purpose. FEMA concurred and is planning to implement validation processes by December 31, 2016.

Aligning Performance Measures with National Standards

The data that FEMA collects and analyzes for two of its program measures do not fully align with national standards used for similar assessments related to national preparedness efforts. As a result, the information that FEMA reports will be of limited utility for broader national preparedness assessments. Specifically, in developing its 21 performance assessment system measures, FEMA identified a subset of six measures as program priority measures that were intended to identify the AFG and SAFER programs’ contributions to promoting fire safety and national preparedness. Two of these measures are designed to help describe the extent to which AFG and SAFER grantees are “responsible for protecting critical infrastructure.” According to FEMA, fire grant program policies define critical infrastructure as “any system or asset that if attacked would result in catastrophic loss of life or catastrophic economic loss” and include a list of examples, such as public water and power systems, chemical facilities, and facilities that support large public gatherings, such as sporting events or concerts. Using this definition, fire grant applicants indicate on their applications whether they have critical infrastructure within their jurisdiction, and FEMA counts the instances when applicants provide this information on their application. For example, for fiscal years 2013 and 2014, 91 percent of all fire grant applicants indicated that they had critical infrastructure within their jurisdictions.

In contrast, DHS’s National Protection and Programs Directorate (NPPD), as part of its efforts to support the National Infrastructure Protection Plan, is responsible for the National Critical Infrastructure Prioritization Program list, a single prioritized list of systems and assets that reflect the National Preparedness Goal’s definition of critical infrastructure: assets that would, if destroyed or disrupted, cause national or regional catastrophic effects. As we noted in a March 2013 report, the prioritization program uses the National Preparedness Goal’s definition to coordinate a voluntary effort with states and other partners to identify, prioritize, and categorize high- priority critical infrastructure in terms of four factors—fatalities, economic loss, mass evacuation length, and degradation of national security. According to the National Infrastructure Protection Plan, the list derived from the National Preparedness Goal’s definition of critical infrastructure identifies nationally significant critical infrastructure that DHS uses to enhance decision making, including implementing FEMA homeland security grant programs and federal incident management planning and response efforts. For example, FEMA uses the National Critical Infrastructure Prioritization Program list based on the National Preparedness Goal’s definition of critical infrastructure in prioritizing other homeland security preparedness grant programs, basing annual risk- based calculations on the differences in criticality of infrastructure in other preparedness grants such as the State Homeland Security Grant program and the Urban Area Security Initiative grants. In addition, committee report language states that there is an interest in maximizing the efforts of FEMA and NPPD through close coordination and information sharing and encourages the two components to work together on stakeholder outreach and sharing information internally on issues, including critical infrastructure protection.

However, because FEMA does not use this definition in its fire grants program performance assessments, it relies on applicants’ perceptions of the criticality of the infrastructure in their jurisdiction that may not be consistent with DHS priorities. GPD officials recognize that fire grant applicants provide information on critical infrastructure that may not be consistent with the NPPD definition, but said that they wanted to provide applicants with greater latitude to make their own local risk assessments using the fire grant programs’ definition because it is useful during the peer review process in evaluating an applicant’s need. However, FEMA officials agreed that using the national standardized definition would enhance their program performance reporting. Thus, using the National Preparedness Goal’s definition of critical infrastructure as the basis of collecting information from applicants and using the National Critical Infrastructure Prioritization Program list to measure fire grant programs’ performance could enhance FEMA’s efforts to assess and integrate the fire grant programs’ contributions to national preparedness.

Integrating New Data Systems

FEMA has awarded about $8.7 million in FP&S grants from 2011 to 2015, to national fire service associations to lead the development of two new information systems—the National Fire Operations Reporting System (NFORS) and the Fire Community Assessment Response Evaluation System (FireCARES)—that contain more extensive and detailed data on fire service operations than the data GPD collects from grant applicants and grantees. While GPD officials intend to use the new data from these systems to enhance program management, they have not reviewed the fire grants application and award processes or the new performance assessment system to determine what data elements could be used.

According to national fire service association officials, NFORS is designed to enable fire fighters to enter information on their fire response efforts to provide detailed descriptions of each fire department, and the community that each department serves and the manner in which it provides fire protection. The system is also designed to collect health and wellness data on firefighter or civilian exposure, injury, or death. The project team is also developing FireCARES, a complementary system to NFORS, which is designed to assess community risks and fire department performance, and evaluate how well fire department resources are deployed. According to the project team officials, they designed FireCARES to provide information on the risks fire fighters and the public face, and what they need in their fire departments to do the job safely and effectively. They said that a community will be less vulnerable to firefighter injury and death, civilian injury and death, and property loss when fire departments use FireCARES to deploy resources that match the community’s risk levels. At the time of our review, the project team was testing the data systems in 10 major metropolitan fire departments, and plans to expand the systems to 125 fire departments later in 2016 and then implement the new systems nationwide by 2019.

According to GPD officials, the information in NFORS and FireCARES could help improve the grant application and awards processes, as well as program performance measurement. For example, such data would provide more insight on firefighting operations and capability gaps and could be used by FEMA and its subject-matter experts during annual grant priorities and application criteria development sessions. GPD officials said these data systems could also be used to better inform the grant awards process, as NFORS could provide more detailed information on an applicant’s’ operational needs, and FireCARES data could provide qualitative information on an applicant’s community fire risk level. Finally, these systems could further inform the fire grant’s performance measures by providing data on vehicles, protective gear, and equipment purchased with fire grants, according to GPD officials. Although GPD intends to use the information in these new systems, officials have not developed a plan to identify relevant data elements in the systems and determine how they can be used to improve the fire grants application and award processes and the performance assessment system, or what GPD resources may be required to make associated changes in grant program processes and policies.

According to the Project Management Institute, project management plans help establish project objectives and outline the course of action required to attain those objectives, defining how projects are to be executed and controlled and providing a means to track, review, and report progress and performance of the project by defining project activities and developing cost and schedule estimates, among other things.

FEMA officials said that they have not developed a plan for how they intend to integrate NFORS and FireCARES information into their existing fire grant program processes because the systems are still being piloted, and they intend to wait until the systems have been more broadly implemented to conduct more detailed planning after the systems contain enough data to be operationalized. However, FEMA officials are familiar with the data elements and goals of NFORS as GPD has assisted in the development of the NFORS data system, and GPD receives updates on the status of implementation. Additionally, since NFORS was funded at least in part through FP&S grants, NFORS data will be available free of charge for use by FEMA. Therefore program officials could, while system testing was still ongoing, evaluate their application and award processes, as well as their performance assessment system to identify what data elements in the new NFORS and FireCARES systems could be beneficial to enhance program management and how these data can be incorporated. Thus, a plan to determine what, and how, new NFORS and FireCARES data elements can be used to improve the fire grants’ application and award processes and performance assessment system could help enhance program management efforts and subsequent changes to information systems and policies.

Enhancing Collaboration between GPD and USFA

The U.S. Fire Administration (USFA) plays an informal role as a consultant to GPD’s fire grant programs, but GPD could better leverage USFA’s expertise and resources to support programmatic management of the fire grant programs using collaborative mechanisms that our work across the federal government has identified. USFA supports the efforts of state and local fire and emergency medical services departments by providing training and education, applied research and technology, data collection and analysis, public education and awareness, and dissemination of information and deployment assistance. For example, according to USFA officials, USFA has regular and ongoing interactions with fire service organizations, including through USFA’s free training courses and programs delivered at the National Fire Academy in Emmitsburg, Maryland, online, and throughout the nation. This opportunity to interact with firefighters and emergency medical service technicians provides USFA with direct feedback and perspectives from fire grant programs constituencies. In addition, USFA also manages the National Fire Data Center, which operates the National Fire Incident Reporting System (NFIRS), conducts regular analyses of NFIRS data and publishes reports on national fire service trends. Finally, the USFA Administrator also serves as the fire protection and emergency response community expert to the FEMA Administrator and acts as a federal advocate to address challenges facing the nation’s fire service.

According to GPD officials, USFA plays a strategic role in fire grant program priorities, although there is no formal relationship between the two components. For example, USFA provides assistance to GPD such as providing facilities for GPD’s annual criteria development panels, and to review GPD’s draft Notice of Funding Opportunity.

USFA’s programs and GPD’s fire grant programs reflect the same broad area of national need—support of national fire service organizations—and both components have similar goals, target similar beneficiaries, and provide complementary services and products that align with these program goals. For example, USFA’s goals are to reduce fire and life safety risk through preparedness, prevention and mitigation; promote response, local planning, and preparedness for all hazards; enhance the fire and emergency services’ capability for response to and recovery from all hazards; and advance the professional development of fire service personnel and of other people engaged in fire prevention and control. GPD’s stated goals in managing the fire grants programs are to enhance the safety of the public and firefighters; provide a continuum of support for emergency responders regarding fire, medical, and all-hazard events; assist fire prevention programs; support firefighter health and safety research and development; and help build, sustain, and deliver core capabilities that are essential to achieving the National Preparedness Goal; assist local fire departments with staffing and deployment capabilities; and assure that communities have adequate protection from fire and fire-related hazards.

Despite the similarities in the goals and missions of the two organizations, key USFA and GPD policy documents do not describe how the two organizations’ programs could work together to achieve their common goals, and officials from both offices said there is no formalized relationship between them regarding program management. For example, USFA’s Strategic Plan for 2014-2018 does not include any reference to the fire grants programs or GPD when it describes its key initiatives in building fire service capabilities. Nor does GPD identify any program-wide role for USFA in supporting the fire grants programs in the AFG, SAFER, and FP&S grant policies presented in the annual Notice of Funding Opportunity, and according to GPD officials, have not historically used NFIRS data in reviewing the fire grants programs.

For programmatic management of other non-fire preparedness grants, GPD leverages the subject matter expertise of other DHS components. For example, the Transportation Security Administration was established after the terrorist attacks of 9/11 to ensure the security of the nation’s transportation system, and the agency works with GPD to assist with programmatic management of the Transit Security Grant Program. Similarly, the Coast Guard works with GPD to assist with programmatic management of the Port Security Grant Program. The relationships between GPD and these other DHS component subject-matter-expert agencies in coordinating their relative responsibilities are established in different ways. For example, GPD and the Coast Guard define their relative roles and responsibilities in the annual port security grant Notice of Funding Opportunity. Additionally, GPD and the Transportation Security Administration have an established Memorandum of Agreement to define their respective responsibilities with regard to surface transportation security grant programs.

These efforts reflect the types of mechanisms our work has shown that federal agencies have used to implement interagency collaborative efforts to address a range of purposes, including policy development; program implementation; oversight and monitoring; information sharing and communication; and building organizational capacity, such as staffing and training. Although collaborative mechanisms such as written agreements differ in complexity and scope, our prior work found that they were most likely to be successful if they addressed certain key features— such as clarity of roles and responsibility, and bridging organizational cultures. In addition, since 2005, report language accompanying appropriations acts has directed FEMA to fund fire grant applications according to local priorities and those established by USFA.

According to GPD and USFA officials, their collaboration in the fire grants program is informal. GPD officials said the current level of coordination between the staff and leaders of the two respective offices is effective and they have not sought to define or document roles and responsibilities because the current relationship meets program needs. They added that the coordination of programs and policies between USFA and GPD is effectively maintained through FEMA’s regular policy working group meetings of senior officials. According to USFA officials, however, their role and participation in the fire grants program is minimal, limited to providing facilities, observing the annual criteria development meetings, and reflects and depends on the level of interest and ability of staff from the two components; and as a result, USFA does not have a meaningful impact on fire grant application processes or in peer review panels, according to USFA officials. USFA officials said that they would prefer a more formal role in the fire grants program and could contribute more to the programmatic management of fire grants by providing access to subject matter experts in fire departments nationwide, and by providing expertise on long-term trends and current practices in the fire service industry, fire prevention, and research and development efforts. For example, USFA officials said that they could enhance GPD’s review and approval of FP&S grants because the purposes of FP&S grants are directly linked to USFA’s mission and role in prevention and research and development. They also said they could serve as a repository for the results of work conducted under the FP&S program.

USFA officials also said their National Fire Data Center office performs quantitative fire service trend analysis based on data in USFA’s system (NFIRS) that could also add value to fire grants program management. For example, the officials said that their subject matter expertise could have enhanced the utility of the new NFORS and FireCARES systems— and better coordinate the integration of NFIRS with these systems—if they had been involved earlier in the systems’ development. USFA officials provided us NFIRS data that indicated that the majority of fire service organizations response efforts are often not related to fire response. According to FEMA officials, greater consideration of USFA’s perspectives in developing the new systems could have helped GPD better align with the all-hazards approach used for national preparedness efforts, as documented in the National Response Framework. They added that it is difficult to strategically determine, plan, or assess the impact of their collaborative efforts on the fire grants programs when the efforts are based on personalities, rather than documented roles and responsibilities. Officials from both GPD and USFA agreed that formalizing USFA’s role would benefit fire grant program management.

Although a level of informal coordination exists between GPD and USFA, enhancing these efforts by using collaborative mechanisms that our work across the federal government has identified as key features and issues to consider during implementation —such as clearly defining and agreeing upon USFA’s role and responsibilities and documenting agreement regarding how they will be collaborating—could help GPD further leverage USFA expertise and resources in support of the fire grants programs, which could also help GPD manage the integration of fire grants into broader national preparedness efforts.

Incorporating Fire Grants into Broader National Preparedness Efforts

Moving forward, FEMA’s addition of a new fire service-related core capability to the National Preparedness Goal in September 2015 will require states to realign information on fire management capabilities from public and private services and resources and integrate information from other core capabilities into their assessments of the new fire management and suppression core capability, according to FEMA officials. To help transition the fire grants programs into these broader national preparedness assessment efforts, FEMA officials said that they are taking steps to educate states, territories, tribes, and locals on the new core capability and encourage them to include capability improvements resulting from the fire grants in their assessments.

In addition, FEMA officials said that they are working to increase the involvement of fire service stakeholders in annual state Threat and Hazard Identification Risk Assessments (THIRA) and State Preparedness Reports (SPR) to more effectively reflect the impact of fire grants. For example, FEMA released revisions to national planning frameworks and new guidance on developing THIRAs and SPRs in June 2016, along with a schedule of webinars to provide additional information and answer questions. States are expected to include an assessment of their level of fire management and suppression capabilities beginning in 2017. While these initial steps to increase involvement of fire service stakeholders into broader assessments of national preparedness are promising, the success of these efforts will not be measurable until after the new fire management-related capability assessments are made.

FEMA officials also said they planned to develop and implement a consolidated grant management system to integrate data used to manage fire grant programs with the data gathered and used for FEMA’s non-fire preparedness grants, and ultimately better measure the impact of fire grants on national preparedness efforts. Specifically, as we reported in May 2016, FEMA plans to develop and implement a new Grants Management Modernization system to provide agency-wide management for all of FEMA’s disaster and preparedness grants. According to the FEMA officials, the system is to be implemented in 2020. They said states will be able to use fire grant program information on AFG- purchased personal protective equipment, fire service vehicles, and training in developing SPRs because states use equipment and training as elements for assessing capabilities under existing national preparedness assessment efforts. Since GPD’s grant management systems consolidation will not occur for several years, it is too early to assess its potential for enhancing FEMA’s efforts to measure the impact of fire grants on national preparedness efforts. Because measuring national preparedness is an issue we have previously identified for FEMA’s management of its other preparedness grants, we will continue to monitor the planned consolidation as part of our ongoing review and annual report on efforts to address fragmentation, overlap, and duplication across federal programs.

Conclusions

FEMA has provided almost $5.8 billion since 2009 to fund fire grant programs that help firefighters and emergency medical service departments enhance their capabilities to respond to the needs of localities across the nation. In response to statutory amendments to the programs through FGRA in 2012, FEMA changed a number of its fire grants policies and the agency’s grant awards in 2013 and 2014 generally reflected these changes. FEMA also enhanced its assessment of program performance by establishing and reporting on measures of the effectiveness of these national investments. Yet, the performance measures do not include measurable performance targets linked to AFG and SAFER program goals that could enhance their quality and utility. In addition, aligning the fire grants programs’ use of data on, and definitions of, critical infrastructure to award fire grants and assess program performance with the more objective, quantitative approach used by DHS and GPD for other programs and non-fire preparedness grants could enhance GPD’s efforts to integrate the fire grants program into larger national preparedness efforts and more objectively assess the impact of fire grants.

Moving forward, FEMA is seeking to advance national fire service data collection by funding new systems—NFORS and FireCARES—that can be used to gather more extensive and comprehensive data on fire service organizations operation, but program officials have not taken steps to identify what and how data from the new systems could be used to enhance fire grants program management. A project management plan could help enhance the effectiveness of subsequent changes to FEMA’s information systems and program management policies that will be needed to operationalize the data. Finally, although a level of informal coordination exists between FEMA’s GPD and the USFA, enhancing these efforts by clearly defining responsibilities and documenting their agreement regarding how they will be collaborating could help GPD further leverage USFA’s expertise and resources in support of the fire grants programs, as well as help GPD manage the integration of fire grants into broader national preparedness efforts.

Recommendations for Executive Action

To better assess the impact of the fire grants program, the Secretary of Homeland Security should direct the FEMA Administrator to establish measurable performance targets linked to AFG and SAFER program goals, such as the desired percentage of awardees who used grants to achieve compliance with equipment standards.

To enhance FEMA’s efforts to assess and integrate the fire grant programs’ contributions to national preparedness, the Secretary of Homeland Security should direct the FEMA Administrator to use the National Preparedness Goal’s definition of critical infrastructure as the basis of collecting information from applicants and using the National Critical Infrastructure Prioritization Program list to measure fire grant programs’ performance in addressing national priorities.

To enhance FEMA’s efforts to incorporate new NFORS and FireCARES data elements into fire grants program management activities, the Secretary of Homeland Security should direct the FEMA Administrator to develop a project management plan for identifying relevant data elements in the new NFORS and FireCARES systems and determining how they can be used to improve fire grant applications and awards processes and the performance assessment system.

To further leverage the expertise and resources in support of FEMA’s fire grants programs, the Secretary of Homeland Security should direct the FEMA Administrator to define and document the roles and responsibilities of USFA in administering the fire grant programs.

Agency Comments and Our Evaluation

We provided a draft of this report to DHS for review and comment. In response, DHS provided written comments on August 25, 2016, which are summarized below and reproduced in appendix VI. DHS concurred with all four recommendations and described planned actions to address them. DHS had no technical comments on the draft report.

DHS concurred with our first recommendation that FEMA establish measurable performance targets linked to AFG and SAFER program goals, such as the desired percentage of awardees who used grants to achieve compliance with equipment standards. DHS stated that FEMA is reviewing all metrics to determine the feasibility for targets on the current set of metrics and will include approved targets in the annual report to Congress that will be issued by the end of fiscal year 2016. These actions, if implemented effectively, should address the intent of our recommendation.

DHS also concurred with our second recommendation that FEMA use the National Preparedness Goal’s definition of critical infrastructure as the basis of collecting information from applicants and using the National Critical Infrastructure Prioritization Program list to measure fire grant programs’ performance in addressing national priorities. DHS stated that FEMA will incorporate the National Preparedness Goal's definition of national critical infrastructure into the fire grant performance measures and will conduct an assessment of data from AFG program application and awards in order to verify recipients are reporting infrastructure that aligns with the National Preparedness Goal definition of critical infrastructure. According to DHS, the assessment will be completed by the end of calendar year 2016 and FEMA will report the measures in the fiscal year 2017 Annual Report to Congress, expected to be completed by December 30, 2017. These actions, if implemented effectively, should address the intent of our recommendation.

DHS concurred with our third recommendation that FEMA develop a project management plan for identifying relevant data elements in the new NFORS and FireCARES systems and determining how they can be used to improve fire grant applications and awards processes and the performance assessment system. DHS stated that FEMA will develop a plan to assess the data collected to determine feasibility for integration in to the fire grants programs. According to DHS, the plan will incorporate ways to improve the applications, awards processes, and tracking of performance and should be completed by March 31, 2017. These actions, if implemented effectively, should address the intent of our recommendation.

DHS concurred with our last recommendation that FEMA define and document the roles and responsibilities of USFA in administering the fire grant programs. DHS stated that FEMA will prepare a formal memorandum of understanding (MOU) between USFA and GPD to assign roles and responsibilities for coordination of their separately legislated responsibilities to (a) maximize FEMA's service to the Fire and Fire Based Emergency Management Services, and (b) prevent programmatic inconsistencies and conflicts. According to DHS, this effort should be completed by October 31, 2016. These actions, if implemented effectively, should address the intent of our recommendation.

We are sending copies of this report to the appropriate congressional committee, the Director of OMB, the Secretary of Homeland Security, and other interested parties. The report will also be available at no charge on the GAO web site at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-8777 or CurrieC@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VII.

Appendix I: Fire Grants Funding and Awards by Grant Program, Eligible Activity

Funding Distribution of Assistance to Firefighters Grants (AFG), Staffing for Adequate Fire and Emergency Response (SAFER), and Fire Prevention and Safety (FP&S) Awards Pre- and Post-Fire Grants Reauthorization Act of 2012, Fiscal Years 2002 through 2014

Table 2 shows the distribution of grants awards for the three grant programs fiscal years 2002 through 2014. The amount of funding increased from fiscal year 2002, peaking in fiscal year 2010 with a total amount awarded of $788.9 million.

Table 3 shows the outcomes for the AFG, SAFER, and FP&S grants awarded in the fiscal year 2014 grant cycle subdivided by the eligible activities under each grant program. AFG grants received the majority of applications and awarded the largest number of grants. In addition, this table corresponds to information in GAO-10-64, Table 2.

Appendix II: Fire Grants Applications and Awards Policies Related to Fire Grant Reauthorization Act (FGRA) Amendments

Our analysis of FGRA identified 26 amendments related to fire grant application and award policies. FEMA incorporated most of these changes into its existing grant application and awards processes (see table 4 below). In some cases, FEMA used existing authorities to waive changes related to FGRA amendments or found that no change to existing policy was necessary.

Appendix III: Fire Grants Funding, Applications, and Awards

Analyses of AFG, SAFER and FP&S Application and Awards Distributions, Fiscal Years 2009 through 2014

The following tables below provide additional information on fire grant applications and awards. In addition, some tables correspond to data analyses performed as part of our October 2009 review of the fire grants program to offer comparisons of data for fiscal years 2002 through 2008.

Table 5 shows the distribution of Assistance to Firefighters Grant (AFG) awards from fiscal years 2009 through 2014 by department types (i.e., career, combination, volunteer, paid on call/stipend, or a combination of these). In addition, this table corresponds to information in GAO-10-64, appendix VI.

Table 6 shows the distribution of Staffing for Adequate Fire and Emergency Response (SAFER) awards from fiscal years 2009 through 2014 by department types (i.e., career, combination, volunteer, interest organizations, or fire department/fire district). In addition, this table corresponds to information in GAO-10-64, appendix VI.

Table 7 shows the distribution of Fire Prevention and Safety (FP&S) awards from fiscal years 2009 through 2014 by department types (i.e., career, combination, volunteer, paid on call, interest organization, or state/local volunteer interest organizations). In addition, this table corresponds to information in GAO-10-64, appendix VI.

Table 8 shows the distribution of AFG awards from fiscal years 2009 through 2014 by community service areas (i.e., rural, suburban, urban, and state fire training academies). For fiscal years 2009 through 2014, rural departments received the most funding of any community service type each year. In addition, this table corresponds to information in GAO-10-64, appendix VII.

Table 9 shows the distribution of SAFER awards from fiscal years 2009 through 2014 by community service areas (i. e., rural, suburban, urban, and interest organizations). For fiscal years 2009 through 2014, urban departments received a majority of the grant funding. In addition, this table corresponds to information in GAO-10-64, appendix VII.

Table 10 shows the distribution of FP&S awards from fiscal years 2009 through 2014 by community service areas (i.e., rural, suburban, urban, and interest organizations). For fiscal years 2009 through 2014, interest organizations received a majority of the grant funding. In addition, this table corresponds to information in GAO-10-64, appendix VII.

Table 11 shows the distribution of AFG awards from fiscal years 2009 through 2014 by activity (i.e., operations and safety, vehicle acquisition, regional, and state fire training academies). The amount of funding provided for operations and safety activities is consistently higher than that awarded on each of regional activities, vehicle acquisition, and state fire training academies. Fiscal year 2014 awards for operations and safety grants amounted to $194.9 million out of a total of $304.7 million in grant awards. Grant funding for vehicle acquisition, regional applications, and state fire training academies was $62.4 million, $38 million, and $9.4 million, respectively. In addition, this table corresponds to information in GAO-10-64, appendix VIII.

Table 12 shows the distribution of SAFER awards from fiscal years 2009 through 2014 by activity (i.e. hiring and recruitment and retention). The amount of funding awarded for hiring is consistently higher than the amount awarded for recruitment and retention activities. Fiscal year 2014 funding for hiring amounted to $305.3 million out of a total of $340 million grant awards. Grant funding for recruitment and retention was $34.7 million.

Table 13 shows the distribution of FP&S awards from fiscal years 2009 through 2014 by activity (i.e. fire prevention, research and research and prevention). The amount of funding provided for fire prevention awards is consistently higher than that awarded on research and development and research and prevention. Fiscal year 2014 funding for fire prevention amounted to $25.2 million out of a total of $34 million grant awards. Grant funding for research activity and research and prevention was $7.5 million and $1.2 million, respectively.

Tables 14, 15, and 16 show the distribution of AFG, SAFER, and FP&S awards and funding, respectively, by department type for fiscal year 2014. In addition, these tables correspond to information in GAO-10-64, appendix IX.

Tables 17, 18, and 19 show the distribution of AFG, SAFER, and FP&S awards and funding, respectively, by community service area for fiscal year 2014. In addition, these tables correspond to information in GAO-10-64, appendix X.

Table 20 shows the spending on FP&S grants as a percentage of AFG and FP&S appropriations for fiscal years 2009 through 2014. FEMA funds AFG and FP&S from the same appropriation.

Appendix IV: Federal Emergency Management Agency’s (FEMA) Performance Assessment System Measures for Assistance to Firefighters Grant (AFG) and Staffing for Adequate Fire and Emergency Response (SAFER) programs

In 2014, FEMA created a performance assessment system for its fire grants programs in response to a Fire Grants Reauthorization Act of 2012 (FGRA) requirement that FEMA establish a system to track program progress and report to Congress on the effectiveness of these grants. FEMA has submitted annual reports to Congress on the AFG and SAFER programs in November 2014, along with a second annual report to Congress on the AFG program in October 2015. FEMA organized these congressional reports to (1) describe the methodology behind FEMA’s fire grants’ performance evaluation system and programmatic effectiveness measures, many based on applicable safety standards, and (2) report data by each measures, based on most recently available data to assess the effectiveness of the AFG and SAFER programs, and develop performance measures to continually track progress and implementation.

The data reported in these measures are based on most recent FEMA grants data available that includes both information included in awardees’ grant applications and their corresponding closeout reports, according to FEMA. FEMA also included case studies for related measures to further illustrate FEMA’s findings and describe examples of awardees’ best practices. For example, FEMA provided a case study in its 2015 report on how one nonprofit organization in Oklahoma used its Fire Prevention and Safety (FP&S) award to install visual and tactile alarms and alert devices in 125 homes with hearing-impaired occupants, 58 percent of which previously had no working smoke alarm.

However, FEMA did not develop FP&S program measures. According to FEMA officials, the relatively smaller available funding amount of the FP&S program, along with the numerous eligible categories of prevention projects available, posed a challenge to identifying a reportable performance measure with a statistically-valid sample size.

Specifically, FEMA’s performance assessment system identifies a total of 21 measures for the AFG and SAFER programs, as shown in tables 21 and 22, below. According to FEMA, each measure is aligned with the achievement of fire grants’ program goals and objectives, and many are based on fire service standards established by the National Fire Protection Association, Department of Labor’s Occupational Safety and Health Administration, as well as relevant state, and local governments and regulatory agencies. According to FEMA, such standards help fire departments minimize the possibility of fire-related injuries, property damage, and other effects.

Appendix V: Incorporating Fire Grants into Broader National Preparedness Efforts

FEMA officials said that they recognized that the quality of the fire management and suppression component of states’ Threat and Hazard Identification Risk Assessments (THIRA) and State Preparedness Reports (SPR) depends on the full integration and participation of different groups as part of the “whole community” effort. FEMA officials noted that they obtain information on participation of different groups of the whole community in the THIRA and SPR processes. FEMA provides states, territories, tribes, and urban areas to use FEMA’s Unified Reporting Tool (URT) to complete their THIRA and SPR assessments. As part of each assessment, jurisdictions are required to indicate the types and number of governmental departments and agencies participating in their process as well as the type and number of participating non- governmental organizations. See table 23 below for nationwide participation rates for whole community stakeholders in their jurisdiction’s THIRA and SPR processes as reported by FEMA.

Appendix VI: Comments from the Department of Homeland Security

Appendix VII: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Chris Keisling (Assistant Director), John Vocino (Analyst-in-Charge), David Beardwood, Dominick Dale, Lorraine Ettaro, Eric Hauswirth, Thomas James, Tracey King, John Mingus, Keith O’Brien, and Katherine Pfeiffer made key contributions to this report.