Fire Grants: FEMA Has Met Most Requirements for Awarding Fire Grants, but Additional Actions Would Improve Its Grant Process
Published: 2009-10-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Homeland Security, through the Federal Emergency Management Agency (FEMA), awards grants to fire departments and other organizations for equipment, staffing, and other needs. As of July 2009, FEMA had received about 25,000 and 22,000 applications for its fiscal years 2007 and 2008 fire grant programs, respectively, and had awarded more than 5,000 grants in both years. GAO was congressionally directed to review the application and award process for these grants. This report addresses the (1) extent to which FEMA has met statutory and program requirements for distributing the grant funds; (2) actions FEMA has taken to provide assistance to grant applicants and involve the fire service community in the grant process; and (3) extent to which FEMA has ensured that its grant process is accessible, clear, and consistent with requirements, including its grant guidance. GAO analyzed relevant laws and interviewed 36 randomly selected grant applicants to obtain their views, but the results are not generalizable.
section_title: What GAO Found
paragraphs: FEMA met seven of eight statutory requirements and two of three FEMA established program requirements for distributing fiscal years 2007 and 2008 grant funds. (GAO used fiscal year 2007 data for two requirements because not all fiscal year 2008 funds had been awarded by July 2009.) For example, FEMA met the statutory requirement that volunteer and combination fire departments (which have both paid and volunteer firefighters) collectively receive at least a minimum of 55 percent of fiscal year 2008 grant funds, and also met the program requirement that volunteer departments receive at least 22 percent. GAO was unable to determine whether FEMA met the statutory requirement that at least 3.5 percent of fiscal year 2008 grant funds be awarded for EMS. FEMA reported that its system is not designed to separately track grants awarded to fire departments for EMS purposes and, therefore, it could not determine if it met this requirement. FEMA reported that while it conducted research to determine that it met this requirement for 1 year, doing so was laborious. Establishing procedures to track awards for EMS purposes would allow FEMA to readily determine if it met statutory requirements. FEMA assists grant applicants by sponsoring workshops and involves representatives of the fire service community in establishing criteria and reviewing applications. Each year, FEMA convenes leaders of nine major fire service organizations to conduct a criteria development meeting to develop the program's criteria and funding priorities. FEMA's peer review process--in which members of the fire service organizations assess grant applications--also helps ensure that the fire service community is involved in the grant process. FEMA officials stated that they strive to provide an even chance for as many fire departments and other organizations as possible to serve on peer review panels. They also stated that they are considering conducting outreach efforts to expand peer review participation, such as announcing opportunities to serve on an upcoming peer review panel at workshops. FEMA has taken actions to ensure that its fire grants award process is accessible and clear to grant applicants--28 of 36 applicants GAO interviewed found the guidance to be clear--but GAO also identified inconsistencies between the stated grant application priorities and the application questions and scoring values. For example, the fiscal year 2008 guidance for the grant that funds the recruitment and retention of firefighters states that continuity--maintaining recruitment and retention efforts beyond the life of the grant--was a priority for grant awards. However, no grant application question addressed this priority and the scoring values did not include it. Thus, it is difficult for FEMA to ensure that grant funds are awarded in accordance with the agency's funding priorities. Further, four of the nine major fire service organizations voiced concerns about feedback FEMA provided to rejected applicants, and 22 of the 36 applicants stated that the feedback was helpful to little or no extent. FEMA officials stated that they could strengthen efforts to improve feedback. Providing specific feedback to rejected applicants could help FEMA strengthen future grant application processes.

section_title: Letter
paragraphs: In recent decades, the U.S. fire service community has experienced  changes in the responsibilities of firefighters and in local budget  allocations for fire departments. While firefighting has traditionally been  the responsibility of states and local communities, the Congressional  Research Service and others have reported that an increase in emergency  medical services (EMS) provided by firefighters combined with state and  local budget shortfalls in the 1990s led the fire service community to  request financial assistance from the federal government in areas such as  equipment, training, and hiring and recruiting and retaining firefighters.  Consequently, the Assistance to Firefighters Grant (AFG) program is in its  ninth year, with appropriations totaling over $4.8 billion since 2001.  According to the National Fire Protection Association (NFPA), in 2007 an  estimated 1.1 million firefighters from more than 30,000 fire departments  protected communities in the United States; of these firefighters, 72  percent were volunteers. In 2006, an NFPA assessment of the United  States fire service stated that it is likely that in communities with a  population of less than 2,500, 21 percent of fire departments would most  often fail to deliver the national standard of at least four firefighters to  respond to fire calls. In addition, the assessment recorded that an  estimated 60 percent of fire departments did not have enough self- contained breathing apparatuses to equip all firefighters on a shift, and 49  percent of all fire engines were at least 15 years old, which is indicative of  a potential need for replacements. Fire departments are also increasingly  responsible for responding to calls unrelated to fire, such as medical  emergencies. NFPA found that medical aid responses tripled from 1980 to  2007, with 6 percent of calls to fire departments in 2007 due to actual fires.
Within the Department of Homeland Security (DHS), the Federal  Emergency Management Agency (FEMA), Grant Programs Directorate’s  (GPD) AFG Program Office, in consultation with the U.S. Fire  Administration (USFA), administers the AFG program and two other  competitive fire grant programs designed to provide assistance to  firefighters—the Fire Prevention and Safety (FP&S) grant program and the  Staffing for Adequate Fire and Emergency Response (SAFER) grant  program. The purpose of the AFG program is to promote the health and  safety of the public and firefighting personnel against fire and fire-related  hazards. Funding is available for a variety of activities, such as certifying  fire inspectors, acquiring firefighting or personal protective equipment,  modifying firefighter facilities, acquiring firefighting vehicles, and funding  EMS. The FP&S program, which is funded through the appropriation for  the AFG program, is intended to support both fire prevention and safety  activities and firefighter safety research and development (R&D) activities.  The purpose of the SAFER program, which includes grants for hiring  career firefighters and recruiting and retaining volunteer firefighters, is to  increase the number of firefighters to help communities meet industry  minimum standards and attain 24-hour staffing.
This report responds to the accompanying explanatory statement to the  Consolidated Appropriations Act, 2008, which mandates that we review  the application and award process for AFG and SAFER grants. Although  FEMA implements the AFG and FP&S grant programs separately, the two  programs are authorized by the same legislation. Consequently, we also  examined the FP&S grant program in this report. We did not assess the  results achieved by those applicants receiving grants, as this was beyond  the scope of our review. Accordingly, this report addresses the following  questions:  (1) To what extent has FEMA met statutory and program requirements for  distributing the grant funds to a variety of applicants and activities?  (2) What actions has FEMA taken to provide assistance to grant applicants  and involve the fire service community in the grant process?  (3) To what extent has FEMA taken actions to help ensure the fire grant  process and related guidance are accessible, clear, and consistent with  applicable statutory and program requirements?
To address the first question, we identified relevant statutory requirements  from sections 33 and 34 of the Federal Fire Prevention and Control Act of  1974 (called the Fire Act and the SAFER Act, respectively, for purposes of  this report). We also identified relevant program requirements established  in FEMA’s grant guidance for the AFG, SAFER, and FP&S grant programs,  which related to distributing grant funds among different categories of  activities and applicants. We then compared these statutory and program  requirements to FEMA grant award data that stratified awards based on  the type of fire department—volunteer, career, or combination—and  based on the type of activity, such as awards for vehicle acquisitions. We  reviewed FEMA data from fiscal year 2002, which was when FEMA began  maintaining electronic fire grant award data. At the time of our review,  FEMA’s fire grant award data were current as of July 2009, at which time  the agency was in the process of awarding fiscal year 2007 FP&S grants  and fiscal year 2008 AFG and SAFER grants. These years were the latest  for which grants had been awarded at the time of our review and for  which we were able to determine FEMA’s compliance with statutory and  program funding distribution requirements. We analyzed FEMA’s annual  listings of applications and awards for the AFG and FP&S grant programs  from fiscal years 2002 through 2008 and SAFER grant program from fiscal  years 2005 through 2008 to provide descriptive information on a number  of other characteristics, such as the type of community served by the  applicant—urban, suburban, or rural. We also determined the number of  times that departments have applied for and been awarded grants. To  assess the reliability of data provided by FEMA, we reviewed and  discussed the sources of data with agency officials. We determined that  the data were sufficiently reliable for the purposes of this report.
To address the second question, we collected and reviewed pertinent  FEMA documents, such as program guidance, as well as observed FEMA’s  fiscal year 2010 criteria development panel process and the fiscal year  2008 FP&S peer review panel process. We analyzed the procedures that  FEMA uses to inform applicants about the fire grant programs, including  the various forms of outreach and the types of assistance that FEMA  provides to applicants. We obtained and reviewed information pertaining  to the selection procedures for peer reviewers, who are to independently  evaluate applications according to established criteria, and analyzed the  training that they receive. We conducted structured interviews with a  nonprobability sample of 36 randomly selected grant applicants that  applied for fiscal year 2008 funding from the AFG and SAFER grant  programs or fiscal year 2007 funding from the FP&S grant program.  Although the results of the interviews are not generalizable, they provided  insights on the perspective of grant applicants. We also conducted  interviews with officials of FEMA, USFA, and nine fire service  organizations.
To address the third question, we reviewed FEMA’s methods of making its  grant guidance and the application accessible to potential applicants. We  also identified statutory requirements pertaining to fire grant applications  and analyzed FEMA’s fiscal year 2008 grant guidance and application  forms to determine whether they consistently instructed applicants to  submit statutorily required information. In addition, we compared the  grant guidance to the application questions and scoring matrix to  determine the extent to which they were consistent. To determine whether  FEMA’s process for issuing grant guidance had adequate controls, we  obtained and analyzed information regarding approval and issuance of  fiscal years 2007 and 2008 program and application guidance. We  compared the agency’s process for documenting and monitoring the  program guidance approval process with criteria in Standards for Internal  Control in the Federal Government. We conducted interviews with  officials from the Office of Management and Budget (OMB), DHS, and  FEMA to obtain information about the approval and issuance of the  program and application guidance as well as how grant decisions are  announced. We analyzed the type of feedback provided to unsuccessful  grant applicants and determined the circumstances under which they may  appeal FEMA’s grant decisions. During our structured interviews with a  nonprobability sample of 36 randomly selected fire grant applicants, we  obtained their views regarding feedback that FEMA provided. To assess  the reliability of data provided by DHS and FEMA on the fire grant  applicants, review criteria, and award procedures, we reviewed and  discussed the sources of data with agency officials. We determined that  the data were sufficiently reliable for the purposes of this report.
We conducted this performance audit from January 2009 through October  2009 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives. Additional details on our  scope and methodology are contained in appendix I.
subsections: section_title: Background
subsections: section_title: Statutory and Program Requirements of the Fire Grant Programs
paragraphs: FEMA’s fire grant programs are available to a variety of fire departments— those composed of volunteer firefighters, career firefighters, or a  combination thereof. In the case of the AFG program, grants also extend  to nonaffiliated EMS organizations. In the case of the FP&S program,  grants also extend to local, state, national, or community organizations  that are not fire departments, such as research universities and fire service  organizations. The statutes authorizing FEMA’s fire grant programs specify  how funds are to be distributed among certain eligible applicants and  activities.
Authority for the AFG and FP&S programs derives from section 33 of the  Federal Fire Prevention and Control Act of 1974 (Fire Act). The Fire Act  requires FEMA to convene an annual meeting of individuals who are  members of national fire service organizations for the purpose of  recommending criteria for awarding grants for the next fiscal year. The  act also requires FEMA, in consultation with national fire service  organizations, to appoint fire service personnel to conduct a peer review  of the grant applications, the results of which FEMA is to consider in  awarding the grants. The Fire Act also contains specific grant application  requirements. In particular, AFG and FP&S grant applicants are statutorily  required to provide information demonstrating financial need; an analysis  of costs and benefits resulting from the assistance; a list of other sources  of federal funding received by the applicant to avoid duplicative funding;  and an agreement by the applicant to provide information during the grant  period to the National Fire Incident Reporting System, which represents  the world’s largest national, annual database of fire incident information.  AFG grant applicants are subject to an additional evaluation  requirement—the extent to which the grant would enhance the applicant’s  daily operations and the grant’s impact on the protection of lives and  property.
Section 34 of the Federal Fire Prevention and Control Act of 1974 (SAFER  Act) provides the authority for the third fire grant program administered  by FEMA. There are two types of SAFER grants: hiring grants, which are  open to career, volunteer, and combination fire departments, and  recruitment and retention grants, which are open to volunteer and  combination fire departments, or to state or local organizations that  represent the interests of volunteer firefighters. Hiring grants are subject  to specific cost-sharing requirements between the federal government and  the grantee, with the federal share decreasing over the 4-year grant  period. Furthermore, the grantee is required to commit to retaining any  firefighter hired through grant funds for at least 1 year after federal  funding ends, amounting to a 5-year service commitment. The statutory  cost-share and service commitment requirements applicable to SAFER  hiring grants do not apply to SAFER recruitment and retention grants.
SAFER grants, like AFG and FP&S grants, are awarded on a competitive  basis through a peer review process. The SAFER Act is also similar to the  Fire Act in requiring grant applications to include certain types of  information. In addition to any information FEMA may require applicants  to submit, the statute requires applicants to provide assurances regarding  diversity in hiring; to explain their inability to address the need without  federal assistance; and to specify long-term retention plans after federal  funding ends, including, for hiring grants, how the applicant plans to meet  the statute’s 5-year service commitment. A hiring grant applicant is also  required to discuss what it will do to ensure that its department does not  discriminate against firefighters who engage in volunteer activities in  another jurisdiction during off-duty hours. The SAFER Act has a  statutory sunset of 10 years from the date of enactment, such that the  agency’s authority to make SAFER grants will elapse on November 24,  2013. (See app. II for a table that sets forth the statutory requirements  applicable to the AFG, FP&S, and SAFER grant programs).
Each appropriations act enacted after January 2002 has made fire grant  appropriations available for 2 fiscal years, after which any unobligated  funds expire. The Consolidated Security, Disaster Assistance, and  Continuing Appropriations Act, 2009—which contains the fiscal year 2009  Department of Homeland Security Appropriations Act—provided   $775 million for firefighter assistance, including $565 million for fire grants  and $210 million for SAFER grants. Table 1 shows the AFG and SAFER  appropriations beginning with their first funded years, fiscal year 2001 and  fiscal year 2005, respectively. (The FP&S funds are included in the AFG  appropriation.)
section_title: Development of Grant Criteria and Funding Priorities
paragraphs: FEMA describes annual funding priorities for the grant programs in its  grant guidance document, which incorporates recommendations from the  programs’ criteria development panel. The criteria development panel is  composed of subject matter experts who meet annually for 3 days to  review and modify the previous year’s funding priorities and award criteria  for all three fire grant programs, and whose recommendations are  summarized in a report to FEMA. Detailed information about the mission  and purpose of each fire service organization is contained in appendix III.  The review process for all three grant programs includes three evaluation  stages: an automated scoring or prescreening process to determine  eligibility and alignment with the grant programs’ funding priorities; a peer  review panel, in which practitioners within the fire service community  evaluate and score the applications; and a final technical review by subject  matter specialists, FEMA officials in the AFG Program Office and the  Grants Management Division, as well as officials in state homeland  security offices, if applicable.
The criteria development panel also updates the previous year’s scoring  matrix, which is a confidential, weighted numerical scoring methodology  that reflects the identified funding priorities. The scoring matrix is used  to rate AFG and SAFER applications by scoring the answers to the  application questions that are weighted to reflect the funding priorities. If  applicants request funding for multiple activities, each activity is scored  independently of the others. The activities’ scores are then weighted based  on the dollar amount and combined to determine the application’s overall  score.
Following review and discussion of the previous year’s information, the  criteria development panel submits its recommendations on the grant  criteria and funding priorities—including those on which participants do  not reach consensus—to the AFG Program Office in the form of a report.  After incorporating the panel’s recommendations and developing the grant  guidance for the three grant programs, the AFG Program Office submits  the guidance to other offices within FEMA for internal review.  Subsequently, it sends the guidance to DHS and OMB for approval. The  grant review and award process is represented in figure 1. (See app. IV for  more detailed information about each of the steps involved in the  application and award process.)
Figure 2 shows the distribution of federal funds for the three fire grant  programs by state for fiscal year 2008.
As shown in table 2, grant applicants submitted more than 22,000  applications for the AFG and SAFER grant programs for fiscal year 2008,  and as of July 2009, FEMA had awarded a total of 5,060 grants. As of July  2009, applicants submitted about 2,500 applications for the fiscal year 2007  FP&S grants and FEMA had awarded a total of 216 grants.
FP&S application and award numbers are for fiscal year 2007 funding and reflect all fiscal year 2007  awards made through July 2009. No fiscal year 2008 FP&S grants had been awarded as of July  2009.
FEMA received about 25,000 applications for AFG, SAFER, and FP&S grants in fiscal  year 2007 and awarded more than 5,000 grants. See apps. V through X for more detailed  information about the distribution of AFG, SAFER, and FP&S program funds.
section_title: FEMA Met Most Statutory and Program Requirements for Distributing Grants, but Did Not Fully Track Compliance with One Statutory Requirement and Had Not Fully Met One Program Requirement for Fiscal Year 2007
paragraphs: In awarding fiscal year 2008 AFG and SAFER grants and fiscal year 2007  FP&S grants, FEMA met seven of eight statutory and two of three program  requirements. These requirements specified how FEMA was to distribute  appropriated grant funds between different applicants and activities. In  July 2009, when FEMA provided fire grant award data for our review, the  agency was in the process of awarding fiscal year 2007 FP&S grants and  fiscal year 2008 AFG and SAFER grants. These years were the latest for  which grants had been awarded and for which we were able to determine  FEMA’s compliance with statutory and program funding distribution  requirements.
Fire grant data for fiscal years 2007 and 2008 are current as of July 2009.
According to AFG program guidance, NFPA reported that combination departments protect 33  percent of the nation’s population and volunteer departments protect 22 percent. We did not verify  FEMA’s estimates.
subsections: section_title: As of July 2009, FEMA Had Met Four of Five AFG Statutory Requirements for Distributing Fiscal Year 2008 Grants, but Had Not Fully Met One of Three Fiscal Year 2008 Program Requirements
subsections: section_title: Although FEMA Met Most AFG Statutory Requirements, It Did Not Fully Track EMS Data to Ensure That It Met One Statutory Requirement for Fiscal Year 2008
paragraphs: FEMA met four of five statutory requirements related to AFG grants for  fiscal year 2008. First, as shown in figure 3, FEMA has consistently met the  population-based statutory requirement for awarding fire grants to  volunteer and combination fire departments from fiscal year 2002 through  July of fiscal year 2008.
Based on our review of fiscal year 2008 grant data, FEMA also met three  other statutory funding distribution requirements for the AFG program, as  identified in table 3.
Fire grant data for fiscal year 2008 are current as of July 2009. The fiscal year 2008 fire  grant period closed on September 30, 2009, which is the end of fiscal year 2009. maximum ceilings established by the Fire Act. In addition, FEMA a about $378,560 of its fiscal year 2008 AFG appropriation for foam  firefighting equipment. According to a program specialist responsible f administering the AFG and SAFER grant programs, FEMA granted all  requests that were submitted for foam firefighting equipment, since the  total amount requested was below the $3 million FEMA was required tomake available for this purpose. Thus, in meeting these requirements,  FEMA ensured that its grant award s were consistent with the funding  percentages mandated by statute.
section_title: FEMA Met Two AFG Program Requirements and Fell Slightly Short of Meeting Another
paragraphs: The AFG guidance states that no more than 45 percent of the AFG funds  may be awarded to career departments. As shown in figure 4, FEMA has   FEMA has  consistently met this requirement from fiscal years 2002 through 2008.  consistently met this requirement from fiscal years 2002 through 2008.
According to a program specialist responsible for administering the AFG  and SAFER grant programs, in fiscal year 2007, OMB instructed the AFG  Program Office to separate the percentage of funding given to volunteer  and combination departments in proportion with the population that each  type of department protects. Thus, FEMA incorporated this requirement in  its program guidance. Specifically, the AFG 2008 grant guidance specified  two requirements for FEMA to distribute grant awards: (1) volunteer  departments are to receive at least 22 percent and (2) combination  departments are to receive at least 33 percent of the total appropriation.  While FEMA met the requirement to award at least 22 percent to volunteer  fire departments in fiscal year 2008, it had not reached the minimum  requirement of awarding at least 33 percent to combination departments  as of July 2009. Specifically, FEMA awarded volunteer departments about  39 percent ($217 million) of its $560 million in fiscal year 2008  appropriated funds, but only awarded about 27 percent ($149 million) of  its appropriation to combination departments. We also analyzed fiscal year  2007 data to determine whether FEMA met the requirement related to  distributing funds to combination departments since the separation  between volunteer and combination departments occurred. FEMA fell  slightly short of meeting this program requirement because it awarded  combination fire departments about 32 percent ($173 million) of the total  appropriation of fiscal year 2007 grant funds—only 1 percent less than that  required by program guidance (see fig. 5). According to a program  specialist responsible for administering the AFG and SAFER grant  programs, FEMA attempts to comply with the guidance pertaining to  population-based proportional grant funding, and he indicated that the  shortfall in fiscal year 2007 awards to combination fire departments may  have been an oversight.
section_title: As of July 2009, FEMA Had Not Met the Statutory Requirement Related to Distributing Fiscal Year 2008 FP&S Grants, but Met This Requirement for Fiscal Year 2007 FP&S Grants
paragraphs: Because of delays in the approval of program guidance, no fiscal year 2008  FP&S grants had been awarded as of July 2009. Therefore, we reviewed  FEMA’s data related to its fiscal year 2007 FP&S grant funding  distributions. According to these data, FEMA met the 5 percent minimum  statutory requirement in fiscal year 2007 by awarding $33,887,071, or about  6.2 percent, of the total AFG appropriation for FP&S grants. Thus, FEMA  ensured that FP&S grant applicants received the percentage of funds  mandated by statute.
section_title: FEMA Met Statutory Requirements for Distributing Fiscal Year 2008 SAFER Grants
paragraphs: In fiscal year 2008, FEMA met statutory requirements related to  distributing funds for SAFER grants. FEMA is required to set aside 10  percent of the annual SAFER Act appropriation for all volunteer or  majority volunteer fire departments to compete for hiring grants, which  are otherwise open to all fire departments regardless of their career,  combination or volunteer status. Recruitment and retention grants, which  are open to volunteer and combination but not career departments, must  also account for at least 10 percent of appropriated funding, in addition to  the unused balance, if any, from the 10 percent hiring grant set-aside.  FEMA complied with these requirements by awarding $21 million, or  about 11 percent, of the $190 million in SAFER funding to volunteer and  majority volunteer fire departments for hiring efforts. In addition, FEMA  also awarded $20 million, or about 11 percent, of the funds to volunteer  and combination departments for the purpose of recruitment and  retention. Thus, FEMA ensured that SAFER grant applicants received the  percentage of funds mandated by statute (see fig. 6).
section_title: FEMA Has Taken Actions to Assist Grant Applicants and Involves the Fire Service Community in Establishing Grant Criteria and Reviewing Grant Applications
paragraphs: FEMA has developed various tools to assist grant applicants with the  application process and involves the nine major fire service organizations  in developing criteria for annual fire grant funding priorities and in the  peer review process. FEMA assists grant applicants by sponsoring  workshops, publishing an online tutorial, and providing a toll-free hotline,  among other actions. Each of the nine major fire service organizations  sends representatives to serve on the annual criteria development panel,  which recommends changes to the grant evaluation criteria and the  funding priorities for the next fiscal year. During the peer review process,  fire service practitioners independently rank the grant applications  according to the evaluation elements recommended by the criteria  development panel.
subsections: section_title: FEMA Has Assisted Grant Applicants through Workshops and Online Tutorials, among Other Actions
paragraphs: FEMA has developed various tools to assist fire grant applicants with the  application process. According to a program specialist responsible for  administering the AFG and SAFER grant programs, FEMA’s regional  offices sponsored approximately 400 AFG and SAFER workshops  throughout the country in 2008. While not all of the 36 randomly selected  fire grant applicants in our nonprobability sample had attended  workshops, 6 grant applicants that had done so explained that they  received basic information on the grant application and award process,  such as a review of the grant guidance and funding priorities, and how to  write the narrative section of the application. See appendix XI for a listing  of fire grant applicants included in our interviews. FEMA has also  contracted with the North American Fire Training Directors to provide a  grant-writing training class to fire departments in all 50 states throughout  the year. The training class explains the grant opportunities available  through the AFG program, describes the application process, and provides  detailed information to guide applicants in drafting narratives. The course  includes a slide presentation with an instructor’s guide and is designed to  be about 4 to 6 hours in length. The contract allots up to $5,000 per state,  totaling $250,000 for grant-writing technical assistance to be delivered  from January 2009 through January 2010.
In addition, FEMA has created an online tutorial to guide AFG, SAFER,  and FP&S applicants through the grant application process. Of the 36  applicants we interviewed, 20 applicants stated that they used online  tutorials and 13 of them stated that the tutorial was helpful to a great or  very great extent. Two of these applicants stated that the tutorials were  useful to a great extent because they provide needed information on  specific sections of the applications and help identify problematic areas.
Applicants seeking further assistance can call a toll-free hotline, which is  staffed by contract personnel who have firefighting experience, or they  can e-mail FEMA. FEMA provides technical assistance Monday through  Friday for each of the three grant programs and also provides such  assistance over the weekend for AFG applicants. In 2008, FEMA received  an estimated total of 12,000 calls to the hotline and responded to an  estimated 10,000 e-mails. FEMA has also established a mentoring program  designed for departments that have unsuccessfully applied for fire grants  for at least 5 years and offers this assistance to all departments that  qualify. About half of the departments accept FEMA’s offer to participate  in the mentoring program, in which each participating department is  paired with a former peer review panelist and given a tutorial to guide it  through the process. As of May 2009, about 400 departments were being  mentored and about 30 to 40 percent have been successful in receiving a  subsequent grant. While FEMA allows applicants to hire a grant writer to  assist them with the process, applicants are responsible for the accuracy  of information provided by the grant writer. The grant writer fees included  in the grant amount requested are reimbursable, providing that they are  declared in the application and do not depend on award.
section_title: FEMA Involves the Fire Service Community in Establishing Grant Criteria and Funding Priorities and in Assessing Grant Applications
paragraphs: FEMA has taken a number of steps to involve the fire service community  in the grant process. For example, each year, FEMA brings together a  panel of fire service professionals representing the leadership of the nine  major fire service organizations to conduct a criteria development meeting  to develop the program’s priorities for the coming year. According to a  program specialist responsible for administering the AFG and SAFER  grant programs, the panel convenes for 3 days in the summer before the  annual appropriations process and is composed of 3 representatives from  each of the organizations, totaling roughly 50 participants, including FEMA  staff. The panel is responsible for making recommendations to FEMA’s  AFG Program Office regarding the creation of program priorities,  modification of program priorities, or both for all three fire grant  programs—AFG, FP&S, and SAFER—as well as the development of  criteria upon which the evaluation of grant applications is based. The  panel’s recommendations are placed in a report that panelists submit to  FEMA, which then incorporates the suggestions into the next fiscal year’s  grant guidance.
subsections: section_title: Peer Review Panel Process Is Designed to Support Independent Assessment of Application Merits
paragraphs: FEMA’s peer review process—in which members of the nine major fire  service organizations participate in assessing grant applications—also  helps ensure that the fire service community is involved in making grant  awards. According to FEMA, the peer review process is a key component  for ensuring fairness in awarding fire grants. Peer review panelists are to  conduct an independent assessment of the merits of the applications  based on the extent to which the proposed projects align with the grant  year’s funding priorities and meet the program’s goals and objectives.
Before arriving at the peer review panel, participants are required to  complete an online tutorial and test, and then submit their certification of  a passing grade during the panel orientation. If a panelist has not  completed the tutorial, he or she is required to do so at the panel  orientation. When panelists arrive at orientation, they are required to sign  and submit a statement declaring that they have no known or apparent  conflicts of interest as well as a nondisclosure form agreeing to keep the  results of the review confidential. The panelists are then divided into  groups of four at different tables. FEMA instructs the panelists not to  review applications if they know the applicant or if the applicant is from  their state. In the event that a potential conflict of interest arises, FEMA  replaces the entire batch of applications provided to the table of panelists  with a new batch. FEMA also instructs the panelists not to share applicant  information with any panelists other than those seated at their table. All  panelists receive an evaluation sheet that lists the evaluation factors, along  with a rubric that provides guidelines for rating grant applications against  the evaluation factors. They also receive a copy of the grant guidance,  which contains the funding priorities.
New panelists receive a 2-½-hour orientation by FEMA program staff, who  provide instruction on distinguishing between average and good  applications, the logistics of individual scoring and table discussion, and  the possible need to reduce the requested grant amount, among other  things. Once the orientation is completed, the panelists individually read  and score the narrative section of the applications as well as responses to  other parts of the application, based on the applicable evaluation criteria.  For example, for fiscal year 2008 AFG applications, panelists provided  numerical scores on the basis of four evaluation factors stated in their  score sheets, which were (1) project description, (2) cost/benefit of the  proposed project, (3) financial need, and (4) effect of the proposed project  on daily operations.
After each panelist at the table scores an application, the panelists discuss  any differences of opinion and the merits or limitations of the application.  Orientation facilitators inform panelists that the aim of the table  discussion is not to arrive at a consensus, but rather to discuss each  application as it pertains to each of the evaluation elements. If panelists  are unable to reconcile any large scoring disparity (defined as 10 points or  greater), they bring the dispute to the attention of a panel chair member  who is responsible for ensuring that panelists document their discussion  and indicate the scoring disparity on their scoring sheets. Panelists may  amend their individual scores or choose to keep them unchanged on their  evaluation sheets. FEMA files all of the panelists’ evaluation sheets for  each application, including the panelists’ comments and recommendations  to reduce the funding amount, reject, or award, with the applications.  After evaluations are entered into FEMA’s database, an average score is  electronically generated that determines whether the application proceeds  to the technical review process, which occurs concurrently in a separate  room at the panel location for applications with the highest scores.
section_title: Fire Service Community Participates in Peer Review
paragraphs: FEMA AFG Program Office officials explained that the number of  panelists varies from year to year and the number of nominees that they  request from each of the nine major fire service organizations depends  upon the amount of appropriated funding as well as the number of  applications submitted. Typically, the organizations each nominate about  24 to 40 people for the AFG panel, 10 for the SAFER panel, and 24 to 28 for  the FP&S panel. FEMA also sends letters to subgroups within the  organizations that represent minorities to receive nominations to help  diversify the panel. According to a program specialist responsible for  administering the AFG and SAFER grant programs, while FEMA does not  verify whether the organizations’ nominees are qualified to attend the  panel, it asks for the résumés of self-nominees or of those nominated by  members of Congress. The official explained that in selecting the peer  review panelists, FEMA considers availability to attend, racial diversity,  and the ratio of new-to-repeat panelists. In fiscal year 2008, 285 people  served on the AFG panel, 47 served on the SAFER panel, and 160 served  on the FP&S panel. The official stated that FEMA considers panelists new  if they have not participated in that particular grant program panel review,  regardless of prior experience as a peer reviewer for another grant  program. Although FEMA selects both new and returning panelists to  review applications in any or all three grant categories, it tries to limit  returning panelists to no more than one-third of the total panel  composition. However, AFG Program Office officials may invite additional  returning panelists if there are not enough confirmed attendees. Panelists  are volunteers—although FEMA pays the entire cost of each panelist’s  transportation, food, and lodging, it does not compensate panelists for any  loss of income they may incur while serving on the panel. Lodging is  typically provided at a federal training facility in Emmitsburg, Maryland, at  no cost to the grant program.
In interviews with a nonprobability sample of 36 fire grant applicants, 22  applicants, or about 61 percent, stated that they had never been asked to  serve on a panel. When asked whether they thought that the peer review  process was fair and objective, 23 stated that it was, while 3 stated that it  was not, and 10 did not know. In addition, 32 of the applicants stated that  they believed that experience as a peer reviewer was beneficial to  completing a grant application.
Although FEMA officials attempt to ensure that new peer review panelists  make up two-thirds of the peer review panel each year, they stated that  they do not currently undertake additional outreach activities themselves  to encourage nominations of new panelists, such as notifying applicants of  opportunities to serve on peer review panels during FEMA’s workshops or  other assistance activities they sponsor for applicants. Rather, FEMA  relies on the nine major fire service organizations for nominations of new  panelists. AFG Program Office officials stated that while they strive to  provide an even chance to as many fire departments and other  organizations as possible to serve on peer review panels, representatives  of departments that are invited sometimes fail to appear to serve on the  panel without informing FEMA. Therefore, officials have invited some  departments multiple times because they have proven to be reliable and  good reviewers. They acknowledged that although FEMA does not  currently limit the number of times that a department can send  representatives to serve on the panels, establishing such a limit could  expand opportunities for other departments to participate in the peer  review process. In addition, they also stated that they are considering  conducting outreach efforts to expand peer review participation, such as  announcing opportunities to serve on an upcoming peer review panel at  workshops. In addition to expanding peer review participation, such  efforts could benefit peer review panelists by allowing them to incorporate  firsthand knowledge of the panel process into their future grant  applications.
section_title: While FEMA Has Taken Actions to Help Make Its Grant Process More Accessible, FEMA Could Benefit from Improved Clarity and Consistency of Grant Guidance and Better Controls over the Process for Reviewing and Approving Grant Guidance
paragraphs: FEMA has taken actions—such as publishing grant guidance and  applications online—to ensure that its grant process is more easily  accessible to grant applicants, but the agency could enhance the clarity  and consistency of its grant guidance and the controls over its review and  approval process. While grant guidance priorities are generally perceived  as clear by grant applicants we interviewed, we identified inconsistencies  between the grant guidance and the grant applications and grant scoring  matrix language. In addition, FEMA has experienced significant delays in  issuing grant guidance, and the agency does not have controls to monitor  the progress of the review process. Finally, the majority of fire grant  applicants that we interviewed felt they received inadequate feedback on  why their applications were rejected.
subsections: section_title: FEMA Publishes Annual Grant Guidance Online to Increase Accessibility
paragraphs: Before each annual grant application period, FEMA publishes updated  grant guidance on its Web site and has created an online grant application,  which is designed to be user-friendly. Publication of the annual grant  guidance on the FEMA Web site makes it more accessible to potential  applicants. The grant guidance provides applicants with an explanation of  the information that will be required in the application, as well as  informing them of any grant priorities for the fire grants for that year, such  as whether training will be given priority. FEMA encourages applicants to  apply for fire grants online because of delays and mistakes associated with  processing paper applications. The electronic application has built-in  “Help” screens and drop-down menus. Applicants for each of the three  grant programs are required to answer a series of questions about their  department and the particular grant they are applying for, as well as  provide a narrative that discusses the impact to result from the proposed  use of the grant funds, among other things. Both the answers to the  questions and the narrative are to be reviewed and scored by the peer  review panel.
section_title: FEMA’s Grant Guidance and Application Forms Generally Reflect Statutory Requirements for the Submission of Specific Information by Applicants, but May Not Obtain All Statutorily Required Information
paragraphs: The statutes authorizing the three fire grant programs contain specific  grant application requirements, which require FEMA to collect and  consider certain information from applicants in making grant awards;  however, not all of these requirements are included in FEMA’s grant  guidance and application forms. For the AFG and FP&S grant programs,  the Fire Act requires grant applicants to include (1) information  demonstrating financial need, (2) an analysis of costs and benefits  resulting from the assistance, (3) a list of other sources of federal funding  received by the applicant to avoid duplicative funding, and (4) an  agreement by the applicant to provide information to the National Fire  Incident Reporting System during the grant period. An additional  requirement applies to the AFG program, requiring FEMA to consider the  extent to which the grant would enhance the fire department’s daily  operations and the grant’s impact on the protection of lives and property.
Based on our review, the AFG and FP&S fiscal year 2008 grant guidance  and application forms instruct applicants to provide information  consistent with the above statutory requirements, with one exception  relating to the FP&S R&D activity, as indicated in table 4.
FP&S grants cover two activities: (1) fire prevention and safety and   (2) firefighter safety R&D. However, FEMA grant guidance only  instructed applicants to provide the statutorily required cost-benefit  analysis for projects proposed under the fire prevention and safety  activity, not the R&D activity. Apart from this exception, the grant  guidance and applications forms for both the AFG and FP&S programs  incorporate the Fire Act’s information requirements. In particular,  according to the grant guidance for both programs, fire departments that  are awarded grants are to provide information to the National Fire  Incident Reporting System during the grant period, as required by statute.  In addition, the application forms for both grant programs require  applicants to identify other sources of federal funding they are receiving  that may duplicate the purpose of their grant request. Although a program  specialist responsible for administering the AFG and SAFER grant  programs stated that few grant applicants receive grant awards from other  sources, FEMA queries its internal records of all grant applicants to  prevent making duplicate awards. The three remaining AFG statutory  requirements—financial need information, a cost-benefit analysis, and an  impact statement—appear as evaluation criteria in the AFG guidance that  applicants are to address in their project narratives. The two remaining  FP&S statutory requirements—financial need information and a cost- benefit analysis—appear as evaluation criteria for the fire prevention and  safety activity, but the evaluation criteria for the R&D activity include only  one of these two statutory requirements, financial need. By taking steps to  ensure that all statutorily required information is included in the grant  guidance and application forms, FEMA is better positioned to provide  reasonable assurance that grants are awarded in accordance with the  statute.
In addition, the SAFER Act also specifies certain information that  applicants are required to submit, in addition to any other information  required by FEMA. The statute requires each applicant to (1) provide  assurances regarding diversity in hiring, (2) explain its inability to address  the need without federal assistance, and (3) specify long-term retention  plans after federal funding ends. With respect to the latter requirement,  SAFER hiring grant applicants are to specifically discuss how they plan to  meet the statute’s 5-year service commitment (i.e., 1-year of service for  SAFER-funded firefighters after the 4-year funding period ends).  Furthermore, SAFER hiring grant applicants are to address another  statutory requirement, a commitment not to discriminate against  firefighters serving as volunteers in other jurisdictions during off-duty  hours.
15 U.S.C. § 2229a(b)(3)(B). and retention grants, the guidance and application form only partially  address one of the information requirements, as indicated in table 5.
FEMA’s guidance for SAFER hiring grants includes each of the statutory  information requirements within the evaluation factors that applicants are  to address in their project narratives. For example, each hiring grant  applicant is to include a statement regarding how the applicant plans to  meet the nonfederal match requirement for the 5-year service period,  including any long-term plans to retain the new firefighter positions, as  required by the statute. Although the SAFER hiring grant guidance  instructs applicants to submit all statutorily required information, the  statement of long-term retention plans is less specific in the SAFER  recruitment and retention grant guidance and application questions, which  ask applicants to include “specifics about the recruitment and/or retention  plan.” Because this language gives applicants the option of providing  specifics on recruitment or retention plans, FEMA may not receive  information on applicant’s long-term retention plans after federal funding  ends, especially from applicants seeking grants for recruitment purposes.  Clarifying the SAFER recruitment and retention grant guidance with  respect to applicants’ long-term retention plans could help FEMA ensure  that it collects the information necessary to determine whether awarded  grants used for recruitment or retention purposes will have a lasting  impact after federal funding ends. FEMA does explicitly instruct  applicants to address the other two statutory requirements—diversity in  hiring and inability to meet the need without federal assistance—which  apply to SAFER recruitment and retention grants.
section_title: Grant Priorities Are Generally Perceived as Clear by Grant Applicants, but Are Not Consistently Aligned with the Scoring Matrix and Grant Application Questions
paragraphs: Seventy-eight percent (28 of 36) of the grant applicants that we  interviewed described the grant guidance as being clear to a great or to a  very great extent, 7 said that it was clear to a moderate extent, and 1  applicant said that he did not know. For example, 1 applicant stated that  the guidance was consistently well written and another commented that it  was simple and user-friendly. However, 10 applicants provided  suggestions for how FEMA could further clarify its grant guidance— including its grant priorities. For example, 1 suggested that priorities be  more expressly stated so that he could make a more qualified decision on  whether to apply. The fiscal year 2008 AFG program guidance does not  summarize funding priorities for any activity other than for the vehicle  acquisition program. Likewise, the fiscal year 2008 FP&S and SAFER  program guidance also do not summarize funding priorities. AFG Program  Office officials acknowledged that the fire grants program guidance could  be made clearer, possibly by incorporating tables or charts highlighting  program priorities.
Moreover, while FEMA’s grant guidance and application questions for the  AFG, FP&S, and SAFER grant programs generally incorporate statutory  information requirements, priorities in the grant guidance are not always  reflected in the scoring matrix and application questions. For example, the  fiscal year 2008 SAFER guidance states that continuity—which refers to  whether an applicant’s recruitment and retention activities are designed to  continue beyond the grants’ period of performance—is a priority for  recruitment and retention grants. However, there is no application  question that addresses this priority, nor is there a scoring matrix value  that corresponds to continuity. Further, the fiscal year 2008 AFG guidance  for wellness and fitness grants prioritizes fitness and injury prevention  projects over rehabilitation, but in the scoring matrix all wellness and  fitness project categories are scored equally. Grant priorities/criteria in the  guidance are updated every year based on the recommendations made by  the criteria development panel. According to a program specialist  responsible for administering the AFG and SAFER grant programs, it  would be difficult to capture the concept of continuity in the form of a  question, and the misalignment regarding wellness and fitness priorities  may have occurred because of insufficient oversight by FEMA.
It is important that FEMA ensure that its grant guidance is not only clear  but also consistently aligned with the application questions and scoring  matrix. According to the National Procurement Fraud Task Force, grant  funds are awarded to carry out goals and objectives as they are identified  in the grant guidance. In order for there to be accurate and consistent  alignment between the grant awards and guidance, the application  questions and their weighted scoring values must also reflect the  intentions of the grant program as stated in the guidance. Developing grant  guidance and application questions that are consistent with funding  priorities could help FEMA ensure that grant funds are awarded in  accordance with the agency’s priorities.
section_title: Review and Approval Process of Draft Fire Grant Guidance Is Not Documented and Monitored
paragraphs: FEMA’s Section Chief for the FP&S program stated that although the  FP&S grant guidance is to be issued in August or September at the start of  each fiscal year, it has not been issued on time for the past 3 fiscal years— the review and approval process for the fiscal year 2008 grant guidance  took over 17 months, and guidance was not issued until February 2009.  See appendix XII for more detailed information about the time frames for  the fiscal year 2008 fire grants process. Because of this delay, the peer  review panel convened to assess grant applications in April 2009, and  FEMA began awarding fiscal year 2008 FP&S awards in August 2009.  FEMA program officials stated that because of the delays in the approval  of the grant guidance, FEMA was unable to reserve classroom and  dormitory space at the federal facility in Emmitsburg, Maryland, where  prior peer review panels had met, and the panel met at a private hotel in  Towson, Maryland, at a cost of about $90,000. As a result of this  expenditure, there were fewer funds to award to grant applicants.
According to FEMA’s Office of Policy and Program Analysis, there is no  systematic method for tracking the review and approval process for fire  grant guidance, no internal deadlines, and no documentation to help  determine the cause for delays in the issuance of grant guidance. AFG  Program Office officials said that they are not fully aware of the review  and approval process once the drafted guidance leaves the AFG Program  Office and is sent to other offices within FEMA and DHS. They said that  the delay in issuing the fiscal year 2008 FP&S guidance occurred when the  FEMA Policy Coordinating Group found that the AFG Program Office did  not possess a Paperwork Reduction Act clearance in order to collect  information from others outside of the federal government. In response,  the AFG Program Office submitted a request for an emergency clearance,  which OMB did not approve. According to OMB officials, the clearance  was denied because OMB believed that FEMA should go through proper  channels to obtain a routine clearance for its fire grant program because  FEMA had previously asked for other emergency clearances.
OMB officials stated that obtaining a routine clearance typically requires about 120 days.
section_title: Grant Applicants Desire More Specific Feedback on Why Their Grant Applications Were Turned Down
paragraphs: Once the grant review process is completed, unsuccessful applicants  receive letters notifying them of the reason(s) their applications were  turned down for grant awards, but some applicants have stated that they  would like more information on the reasons for their rejection. According  to AFG Program Office officials, applicants receive rejection letters at the  same time as grant awards are being announced. The officials further  stated that sending rejection letters to thousands of applicants is time and  resource intensive. The AFG Program Office has developed 16 templates  to use in sending AFG applicants letters explaining the reasons for their  rejection. Explanations that FEMA provides to unsuccessful applicants  include (1) discrepancies between the itemized request and the narrative  justification for those items, (2) an applicant or the specific activity for  which grant funding was requested is ineligible for funding, or   (3) incomplete fulfillment of the requirements of previous grant awards  received by the applicant. These letters inform the applicants that there  were an extremely high number of applications and a finite amount of  funding, which resulted in many worthy applicants not being funded.
In certain cases, the information contained in these letters is more positive  and does not provide detailed information on the reason for the rejection.  For example, FEMA may send an applicant a letter explaining that while  the peer review panelists’ scores indicated that its application was  generally good, the agency does not have enough funding to offer the  applicant an award after awarding grants to applicants with higher scores.  However, FEMA states that if it identifies any excess funding or if some of  the applicants that have been offered a grant decline the offers, the agency  might be able to fund the request. According to AFG Program Office  officials, applicants are not allowed to appeal panelists’ scores. Rather,  they can only request reconsiderations because of processing issues. For  example, they can argue that terminology in the grant guidance was  unclear.
Four of the nine major fire service organizations expressed concern about  the level of feedback provided to rejected applicants. One official stated  that FEMA’s denial letters lack specificity about why their applications  were denied, while another official stated that rejected departments were  frustrated with not knowing why their applications were rejected year  after year. Another official suggested that FEMA publish a list of the top 10  reasons why grants are turned down in order to provide greater clarity to  applicants.
Moreover, 61 percent of the 36 grant applicants that we interviewed (22 of  36) stated that the feedback they received from FEMA regarding why their  applications were turned down was helpful to little or no extent. One  applicant stated that he did not receive any feedback from FEMA and that  his fire department had called the agency to learn the status of its  application. In addition, 6 applicants stated that the feedback was helpful  to some or to a moderate extent and another 6 stated that the feedback  was helpful to a great or very great extent. However, 1 applicant reported  not knowing the extent to which the feedback was helpful. Seventy-five  percent (27 of 36) of grant applicants with whom we spoke suggested that  FEMA’s feedback should include specific reasons why the grant  application was denied. For example, 3 grant applicants suggested that it  would be helpful if FEMA provided information regarding the specific  stage in the application review process where an application was rejected.  One fire department suggested that FEMA cite whether an application  contained a poorly written narrative or was rejected for another reason,  such as a request for equipment that was not a funding priority. Another  applicant suggested that peer reviewers provide applicants the reasons  why their applications scored low, and another suggested that FEMA  include information on available assistance for future grant cycles, such as  the online tutorial or list of workshops.
Providing feedback to grant applicants is an important part of the fire  grant program. In its 2007 report on the AFG program, the National  Academy of Public Administration listed improving feedback to grant  applicants as a strategic objective for the grant management process. The  strategic objective is for FEMA to improve the feedback to unsuccessful  candidates so that applicants can understand why they did not receive  grants, thereby increasing participation and improving the quality of  requested grants. AFG Program Office officials acknowledged that they  could strengthen efforts to improve feedback to applicants who are turned  down for grants following the peer review process. According to the  Director of the AFG program, FEMA could modify the feedback provided  to unsuccessful applicants to better explain the reasons why applications  were rejected. We have previously reported the need to provide clear  feedback to unsuccessful applicants on the strengths and weaknesses of  their grant applications. Providing specific feedback to applicants  regarding the reasons that they are denied grants could help FEMA  strengthen future grant application processes and better position it to  achieve its intended benefits of assisting fire departments that are most in  need.
section_title: Conclusions
paragraphs: Through the years, the U.S. fire service community has experienced  changes in its responsibilities to the public as well as decreases in local  budget distribution, which underscore the need for fire departments  nationwide to have the resources necessary to protect their communities.  Through its fire grant programs, FEMA has an opportunity to assist fire  departments that are struggling to meet their responsibilities. While FEMA  distributed fire grants to a variety of applicants for a variety of activities,  developing and implementing a procedure for capturing the percentage of  appropriated funds awarded to fire departments related to EMS equipment  and training would better position FEMA to more readily determine if it  met the minimum amount established by statute. FEMA could improve the  clarity and consistency of the grant review and award process by  collecting all statutorily required information and eliminating  inconsistencies between the guidance, the scoring matrix, and the  application, which may confuse applicants. Additionally, by improving its  internal controls to document and track the grant guidance review and  approval against established milestones, FEMA could provide applicants  the opportunity to plan for matching funds by determining when guidance  will be issued each year. Finally, by providing more specific feedback and  information on assistance, FEMA could help ensure that applicants have  the opportunity to prepare better applications, and thus have a greater  chance of being awarded grants in the future.
section_title: Recommendations for Executive Action
paragraphs: To ensure compliance with all AFG statutory requirements, we  recommend that the Administrator of FEMA establish a procedure for  tracking the percentage of grant funds awarded to fire departments for  EMS purposes.
In addition, to improve the clarity, consistency, and controls of the grant  review and award process, we recommend that the Administrator of  FEMA take the following three actions:    Ensure that the priorities in the grant guidance are aligned with the  scoring matrix and the grant application questions, and that FEMA  requests applicants to submit all statutorily required information.
Coordinate with the Secretary of Homeland Security to document the  review and approval process for its grant guidance, develop a tracking  system to monitor the progress of the review within FEMA and DHS,  and set internal deadlines so that guidance can be issued in a timely  manner.
Inform unsuccessful applicants about the forms of assistance available  to them in future grant cycles and provide more specific feedback to  applicants that are turned down for grants following the peer review.
section_title: Agency Comments
paragraphs: We provided a draft of this report to DHS and FEMA for review and  comment. On October 22, 2009, DHS provided written comments on the  draft report, which are reprinted in appendix XIV. DHS concurred with our  recommendations and is taking actions to address them. DHS stated that  FEMA will examine the available options and adopt one for manually and  electronically monitoring percentages of grant funds awarded to fire  departments for emergency medical services purposes to ensure  compliance. DHS also stated that FEMA will explore options and identify  means for providing clear, concise, and consistent information to  applicants on the funding priorities and statutorily required information. In  addition, DHS stated that FEMA will work with applicable offices to  enable a timely review and tracking of program guidance material.  Further, DHS stated that additional training and outreach efforts are being  developed to enhance feedback to applicants.
We are sending copies of this report to the Secretary of Homeland  Security, Director of the Office of Management and Budget, interested  congressional committees, and other interested parties. The report also  will be available at no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, or wish to  discuss these matters further, please contact me at (202) 512-8777 or  jenkinswo@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.  Key contributors to this report are listed in appendix XV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The accompanying explanatory statement to the Consolidated  Appropriations Act, 2008, mandates that we review the application and  award process for Assistance to Firefighters Grant (AFG) and Staffing for  Adequate Fire and Emergency Response (SAFER) grants. Thus, we  addressed the following questions:    To what extent has the Federal Emergency Management Agency  (FEMA) met statutory and program requirements for distributing the  grant funds to a variety of applicants and activities?
What actions has FEMA taken to provide assistance to grant applicants  and involve the fire service community in the grant process?
To what extent has FEMA taken actions to help ensure the fire grant  process and related guidance are accessible, clear, and consistent with  applicable statutory and program requirements?
To review the extent to which FEMA met statutory and program  requirements for distributing fire grants to a variety of applicants and  activities, we reviewed relevant statutory requirements from sections 33  and 34 of the Federal Fire Prevention and Control Act of 1974 (called the  Fire Act and the SAFER Act, respectively, for purposes of this report).  Based upon these statutes, we identified a total of eight statutory  requirements that established specific percentages or dollar amounts  designating how FEMA was to distribute funds among different grant  applicants and activities. We also identified three relevant program  requirements established in FEMA’s grant guidance for the AFG, SAFER  and Fire Prevention and Safety (FP&S) grant programs, which related to  distributing grant funds among different categories of activities and  applicants. We then compared these statutory and program requirements  to FEMA grant award data that stratified awards based on the type of fire  department: volunteer, career, or combination—and based on the type of  activity, such as awards for vehicle acquisitions. FEMA began maintaining  electronic fire grant award data in fiscal year 2002. FEMA’s fire grant  award data are current as of July 2009, at which time FEMA was in the  process of awarding fiscal year 2007 FP&S grants and fiscal year 2008 AFG  and SAFER grants. These years were the latest for which grants had been  awarded at the time of our review and for which we were able to  determine FEMA’s compliance with statutory and program funding  distribution requirements. In addition to analyzing compliance issues, we  also analyzed FEMA’s annual listings of applications and awards for the  AFG and FP&S grant programs from fiscal years 2002 through 2008 and  SAFER grant program from fiscal years 2005 through 2008 to provide  descriptive information on a number of other characteristics, such as the  type of community served by the applicant—urban, suburban, or rural. We  provided descriptive information on the type of community served  because the Fire Act requires FEMA to distribute AFG grants to a variety  of different fire departments based on such characteristics as the type of  community served, although the statute did not provide a specific  percentage of funds against which we could evaluate compliance for the  community-based requirement. We also determined the number of times  that departments have applied for and been awarded grants. The  descriptive information regarding the distribution of grant awards appears  in appendixes V through X and appendix XIII of this report. To assess the  reliability of data provided by FEMA, we reviewed and discussed the  sources of data with agency officials. We determined that the data were  sufficiently reliable for the purposes of this report.
To determine the actions FEMA has taken to provide assistance to grant  applicants and involve the fire service community in the grant process, we  collected and reviewed pertinent FEMA documents, such as program  guidance, and observed FEMA’s fiscal year 2010 criteria development  panel process and the fiscal year 2008 FP&S peer review panel process.  We conducted interviews with officials from FEMA and the nine fire  service organizations to determine the type of information FEMA provides  to applicants on the grant application and review process. We analyzed  the methods FEMA uses to inform applicants about the fire grant  programs, including the various types of outreach and assistance that the  agency provides to applicants. Specifically, we reviewed information on  grant-writing workshops, online tutorials, technical support, and  mentoring, among other forms of applicant outreach, and reviewed the  contract between the North American Fire Training Directors and FEMA  to provide additional grant-writing assistance to applicants. We also  conducted interviews with officials from FEMA, the U.S. Fire  Administration, and the nine fire service organizations to understand how  FEMA establishes criteria for awarding grants to applicants. We analyzed  the information regarding FEMA’s procedures for selecting peer  reviewers; the training that panelists receive before reviewing  applications; and the measures FEMA takes to ensure that peer review  panelists maintain independence, safeguard against any conflict of  interest, and adhere to restrictions related to confidentiality. We analyzed  peer review guidance and other AFG Program Office documents, such as  the criteria development reports and panel application evaluation sheets,  to determine the process through which peer reviewers score grant  applications. We collected and analyzed information pertaining to the  technical review process and observed the fiscal year 2008 FP&S subject  matter specialists’ portion of the technical review to determine how FEMA  incorporates scores from the technical reviewers and makes final award  decisions.
To evaluate the extent to which FEMA has taken actions to help ensure  that the fire grant process and related guidance are accessible, clear, and  consistent with applicable statutory and program requirements, we  reviewed FEMA’s methods of publishing grant guidance online and the  online applications. We also identified statutory requirements pertaining to  information applicants are required to submit in their fire grant  applications and analyzed FEMA’s fiscal year 2008 grant guidance and  application forms to determine whether they consistently instructed  applicants to submit the statutorily required information. We obtained and  analyzed FEMA AFG Program Office documents, such as the scoring  matrix and documents describing the prescreening process. We analyzed  and compared the fire grant programs’ funding priorities contained in  fiscal year 2008 AFG and SAFER grant guidance and 2007 FP&S grant  guidance with the application questions and the scoring matrix to  determine the extent to which they were consistent based on criteria from  the National Procurement Fraud Task Force. Through  our analysis of grant program documents for fiscal years 2007 and 2008  and our interviews with officials from the Office of Management and  Budget, the Department of Homeland Security (DHS), and FEMA, we  obtained obtain information about the approval and issuance of the  program and application guidance as well as how grant decisions are  announced. We analyzed the procedures that FEMA uses to announce  grant decisions and the type of feedback it provides to unsuccessful  applicants and determined the circumstances under which applicants may  appeal FEMA’s grant decisions. We compared the views of a  nonprobability sample of 36 randomly selected fire grant applicants  regarding feedback to unsuccessful candidates, and reviewed the National  Academy of Public Administration 2007 assessment of the AFG program.
From June 22 through June 29, 2009, we conducted structured interviews  by phone with fire chiefs and other officials knowledgeable about the fire  grants program from a nonprobability sample of 36 randomly selected fire  grant applicants that did or did not receive fiscal year 2008 funding for the  AFG and SAFER grants and fiscal year 2007 funding for the FP&S grants.  We obtained their perspectives on the application and award process. The  sample included fire grant applicants across the continental United States  and Alaska. We obtained a list of the universe of applicants from FEMA for  the respective fiscal years, from which we randomly selected fire  departments within seven grant categories: (1) awarded AFG applicant,   (2) turned down AFG applicant following the peer review panel process,  (3) turned down AFG applicant following the initial electronic screening  process, (4) awarded FP&S applicant, (5) turned down FP&S applicant  following the peer review panel process, (6) awarded SAFER applicant,  and (7) turned down SAFER applicant following the peer review panel  process. We conducted two pretest interviews in person with  representatives of fire departments in South Carolina and Pennsylvania to  further refine our questions. An independent GAO methodologist reviewed  our questionnaire to identify and revise potentially biased questions.  Although we are not able to generalize the results of the nonprobability  sample to the general population of applicants, the questionnaire allowed  for a series of open-ended and close-ended responses on the grant  application and award process, including questions on the perceived  fairness and objectivity of the grant programs. Because of the scope of our  work, we reviewed the fire grant programs’ application and award  process, but did not assess the extent to which FEMA measures its  performance in implementing these fire grant programs.
We conducted this performance audit from January 2009 through October  2009 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. To assess the  reliability of data provided by DHS and FEMA on the fire grant applicants,  review criteria, and award procedures, we reviewed and discussed the  sources of data with agency officials. We determined that the data were  sufficiently reliable for the purposes of our review, and that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
section_title: Appendix II: Statutory Requirements for Fire Grants
paragraphs: The AFG, FP&S, and SAFER grant programs are authorized to award  funds for a range of purposes to various eligible organizations. Statutory  requirements pertaining to each grant program are shown in table 6.
section_title: Appendix III: Mission, Membership, and Description of Fire Service Organizations
paragraphs: Officials from the nine fire service organizations participate in the criteria  development panel to recommend changes to the upcoming year’s grant  priorities, as well as nominate members to serve on the peer review panel.  The organizations’ missions and memberships represent a range of  interests within the fire service community. (See table 7.)
section_title: Appendix IV: Application and Award Process for Fire Grants
paragraphs: The AFG, FP&S, and SAFER grant programs are evaluated in three phases  of review. The first phase is an automated scoring process to select  competitive AFG grant applications, or a prescreening process for FP&S  and SAFER grants. The second phase for all grants is a peer review panel  process to evaluate the extent to which an application aligns with the  grant year’s funding priorities and meets the programs’ goals and  objectives. The third phase is a technical review panel process to  determine technical feasibility, avoid duplication with state initiatives, and  make any modifications to the potential award.
subsections: section_title: Automated Scoring and Prescreening Process
paragraphs: After the application period is closed, all fire grant applications undergo  either an automated scoring process, where each application is ranked  relative to the funding priorities described in the guidance, or a  prescreening process, where each application is screened for eligibility.  For example, for the AFG and SAFER grant programs, the AFG Program  Office creates a scoring formula (following the criteria development  panel’s scoring matrix), which is then entered into a computerized system  by FEMA’s Information Technology Office. Through this formula, each  application is scored and ranked electronically. For the AFG program, if  an application for a project has a high-dollar item or activity that was  ranked as a low priority, that item or activity may have adversely affected  the scoring and thus may have taken the application out of the competitive  range for peer review. FEMA officials stated that because more than  21,000 AFG applications are submitted every year and the AFG Program  Office does not have the resources to review all of the applications, the  number of applications it submits for peer review is the number of  applications with the highest scores whose cumulative funding requests  total 200 percent of the appropriated funding. In other words, AFG grants  that go to peer review can total no more than twice the dollar value of the  available grant amount.
SAFER applications also undergo an automated scoring and ranking  process; however, FEMA submits all of them for peer review (1,314  applications for fiscal year 2008). Applications that best address the  funding priorities score higher than applications that do not. Unlike the  AFG automated scoring, which is used for the sole purpose of selecting  applications for peer review, the automated score for SAFER grants  accounts for one-half of the overall consideration provided each  application, with the peer review accounting for the balance of the  consideration.
Unlike AFG and SAFER grants, FP&S applications do not undergo an  automated scoring and ranking process. Because of the smaller size of the  FP&S program and the more technical and academic nature of some FP&S  requests, the AFG Program Office manually screens for eligibility all  applications submitted for the two FP&S activities, which are fire  prevention and safety grants and R&D grants. Both the applicants and the  projects are screened for eligibility based on statutory and programmatic  eligibility criteria, and those found ineligible are removed from further  consideration before the peer review process. For example, for-profit  applicants and projects requesting fire suppression equipment or fire  vehicles are considered ineligible. According to an AFG Program Office  section chief, out of 2,637 FP&S applications submitted for fiscal year 2008  funding, 170 applications were considered ineligible before the peer  review panel process and another 16 applications were found ineligible  during the peer review and therefore were not scored.
section_title: Peer Review Process
paragraphs: Peer review panel participants are fire service professionals who are  members of one or more of the nine major fire service organizations. The  panel’s goal is to evaluate the extent to which an application aligns with  the grant year’s funding priorities and meets the programs’ goals and  objectives. The AFG Program Office officials explained that FEMA  requests that each organization nominate members to serve as peer review  panelists. FEMA also sends letters to subgroups within the organizations  that represent minorities to receive nominations to help diversify the  panel. Although the officials select both new and returning panelists to  review applications in any or all three grant categories, they attempt to  limit returning panelists to no more than one-third of the total panel  composition. However, FEMA may invite more experienced panelists if  there are not enough confirmed attendees.
In the orientation for the peer review of fiscal year 2008 FP&S  applications, new panelists completed a review of two mock applications  and discussed them as a group to familiarize themselves with the review  process. These simulated exercises for new panelists occur for the AFG  and SAFER panels as well.
For fiscal year 2008 SAFER hiring grants, panelists also scored  applications numerically according to six evaluation factors stated in their  score sheets, which were the extent to which the application described   (1) a plan to use firefighters and the specific benefit these firefighters will  provide for the fire department and the community, (2) a risk to the  community and current firefighters that will be significantly reduced with  grant funding, (3) the need for financial assistance, (4) a plan to recruit  and hire minorities and women, (5) a long-range plan to make the  nonfederal match and retain firefighters, and (6) a policy to prevent  discrimination against firefighters who volunteer for other departments.  For SAFER recruitment and retention grants, only the first four evaluation  factors apply.
For fiscal year 2008 FP&S applications, panelists scored applications for  the fire prevention and safety activity based on six evaluation factors  stated in their score sheets using adjectives ranging from “Strongly Agree”  to “Strongly Disagree,” which the AFG Program Office subsequently  converts to numerical scores. The evaluation sheet contains six detailed  evaluation factors: (1) financial need, (2) vulnerability statement,   (3) implementation plan, (4) project evaluation plan, (5) sustainability, and  (6) cost-benefit analysis. For the FP&S research and development  activity, panelists scored applications reflecting the degree to which they  addressed the following evaluation factors: (1) study purpose(s), goals and  objectives, and specific aims; (2) scientific and technical merit of the  proposed research; (3) dissemination and implementation; (4) resources— people and time; (5) protection for human subjects; (6) financial need; and  (7) impact on firefighter safety.
section_title: Automated Scoring and Prescreening Process Results in Some Applications Not Undergoing Peer Review
paragraphs: In the House report accompanying DHS’s fiscal year 2008 appropriations  bill, the House Committee on Appropriations raised concerns about the  number of fire grant applications that did not reach the peer review stage  (9,268 out of 20,972 according to the House report). In the subsequent  explanatory statement accompanying DHS’s fiscal year 2008  appropriations act, both the House and Senate appropriations committees  directed FEMA to provide fire grant applicants whose applications were  not selected for peer review an official notification detailing the reasons  for their rejection. A program specialist responsible for administering the  AFG and SAFER grant programs stated that the agency sent all AFG  applicants that submitted applications that were not peer reviewed a letter  notifying them that their applications were not among those selected for  the second phase of the competitive review. The letter explained that  during the second phase, those applications that best addressed AFG’s  established funding priorities for each eligible activity were approved and  forwarded for peer review and that the objective of peer review is to  further ensure the best use of grant funds. According to AFG Program  Office officials, in fiscal year 2008, about 21,000 AFG applications were  submitted, of which 8,000 were subsequently not selected for peer review.  FEMA offered the applicants an opportunity to receive a more detailed  explanation. The officials stated that applicants that submitted about 7,000  of these applications requested additional information, and four FEMA  contractors spent about 4 months gathering the information needed to  send electronic responses to these applicants to clarify specifically why  the applications did not meet the criteria for peer review.
To determine the types of applicants whose applications were rejected  before the peer review process, we reviewed data provided by FEMA. For  fiscal year 2008, we found that 4,489 applicants (or about 29 percent) of  the 15,544 applicants were turned down for all applications they submitted  for AFG funds through the automated scoring process, and consequently  none of their applications were peer reviewed. Appendix XIII contains  more detailed information about unsuccessful fire grant applicants by  department type and community service area.
section_title: Technical Review Process
paragraphs: Fire grant applications that receive the highest scores by the peer review  panels are submitted for technical review. The technical review process  consists of reviews made by subject matter specialists, the AFG Program  Office or grants management specialists, or state homeland security  representatives. The AFG Program Office has a group of subject matter  specialists who review each potential award application to ensure that the  project is technically feasible and that the application does not contain  projects, activities, or items that are ineligible or otherwise not worthy of  funding. In addition, they identify potential modifications to projects that  would enhance the overall award, and identify applications that have  scored outside the fundable range but should receive the award. The  subject matter specialists review the entire application as well as the  panelists’ comments. Once they have completed their review, the AFG  Program Office staff reviews each potential award before making a  recommendation on whether to award a grant to the applicant. The AFG  Program Office staff assesses the findings from the previous reviews,  determines whether any duplicate applications exist, and validates the  eligibility of both the applicant and the items requested.
Following the review by the AFG Program Office staff, grant applications  that have been recommended for award are submitted to grants  management specialists in FEMA’s Grants Management Division. These  specialists are responsible for reviewing the financial information in  applications and for ensuring that the requested amounts are reasonable  and calculated correctly. They also ensure that applicants have provided  responses to a questionnaire that the Grants Management Division sends  to applicants. The questionnaire solicits information such as whether any  proposed reductions in the requested amount in the grant applications are  acceptable and if the applicant is a recipient of other federal grants. These  responses to the questionnaire, which are supplemental to the grant  application, are processed internally by the Grants Management Division.  The Grants Management Division might contact the grantee for additional  follow-up or send the application back to the AFG Program Office, as  appropriate. According to grants management specialists, their review  typically requires only a couple of hours, but the approval process might  take as long as a month or more, depending on how long applicants take to  respond to the questionnaire and the extent to which follow-up  information is necessary. Once the specialists approve the recommended  applications for award, the applications are sent to the assistance officers  in the Grants Management Division for approval, at which point the  assistance officers obligate the awards.
section_title: Grant Awards Process
paragraphs: Although FEMA generally makes funding decisions using rank-order  results from the peer review panel evaluation, it may deviate from the  panel’s scores and make funding decisions based on the type of  department (career, combination, or volunteer), the size and character of  the community the applicant serves (urban, suburban, or rural), or both to  satisfy statutory and programmatic funding goals. State homeland  security offices may also review applications to ensure that the relevant  proposed projects do not duplicate existing statewide programs. Since the  number of submitted application requests exceeds the appropriated  funding, applications reviewed within this final stage may not be awarded  grants, despite falling within the fundable range.
FEMA announces these awards over several months as decisions are  made, but does not make the awards in any specified order (i.e., by state,  program, or any other characteristic). Awards are made until the funding  is exhausted or the appropriation has expired.
section_title: Appendix V: AFG, SAFER, and FP&S Applicants and Award Distribution by State
paragraphs: Tables 8 and 9 show the AFG and SAFER applicants and award  breakdown, respectively, by state for fiscal year 2008. Table 10 shows the  FP&S applicants and award breakdown for fiscal year 2007.
section_title: Appendix VI: Distribution of AFG, SAFER, and FP&S Awards by Department Type, Fiscal Years 2002 through 2008
paragraphs: Tables 11, 12, and 13 show the distribution of AFG, SAFER, and FP&S  awards, respectively, from fiscal years 2002 to 2008 by department types  (e.g., career, combination, volunteer, paid on call/stipend, or a  combination of these).
section_title: Appendix VII: Distribution of AFG, SAFER, and FP&S Awards by Community Service Area, Fiscal Years 2002 through 2008
paragraphs: Tables 14, 15, and 16 show the distribution of AFG, SAFER, and FP&S  awards, respectively, from fiscal years 2002 to 2008 by community service  area (e.g., rural, suburban, and urban).
section_title: Appendix VIII: Distribution of AFG Awards by Activity, Fiscal Years 2002 through 2008
paragraphs: Table 17 shows the distribution of AFG awards by activity (e.g., operations  and safety, vehicle acquisition, and regional) for fiscal years 2002 through  2008. The amount of funding provided for operations and safety activities  is consistently higher than that spent on regional activities and vehicle  acquisition. Fiscal year 2008 funding for operations and safety grants  amounted to $273.1 million out of the total $453.9 million grant awards.  Grant funding for vehicle acquisition and regional applications was   $131.7 million and $49.2 million, respectively.
section_title: Appendix IX: Distribution of AFG, SAFER, and FP&S Awards and Funding by Department Type
paragraphs: Tables 18 and 19 show the distribution of AFG and SAFER awards and  funding, respectively, by department type for fiscal year 2008. Table 20  shows the distribution of FP&S awards and funding by department type  for fiscal year 2007.
section_title: Appendix X: Distribution of AFG, SAFER, and FP&S Awards and Funding by Community Service Area
paragraphs: Tables 21 and 22 show the distribution of AFG and SAFER awards,  respectively, by service area for fiscal year 2008. Table 23 shows the  distribution of FP&S awards by service area for fiscal year 2007.
section_title: Appendix XI: Fire Grant Applicants in Nonprobability Sample
paragraphs: We conducted structured interviews with randomly selected applicants  that applied for fiscal year 2008 funding for the AFG and SAFER grants  and fiscal year 2007 funding for the FP&S grants to discuss various aspects  of FEMA’s grant application and award process. (See table 24.)
section_title: Appendix XII: Time Frames for the Fiscal Year 2008 Fire Grants Process
paragraphs: We obtained information from FEMA officials and documents in order to  prepare a timeline depicting the time frames for the fiscal year 2008 fire  grants process. (See fig. 8.)
section_title: Appendix XIII: Unsuccessful AFG, SAFER, and FP&S Applicants by Department Type and Community Service Area
paragraphs: Tables 25 and 26 show the breakdown of unsuccessful AFG and SAFER  applicants, respectively, by department type and community service area  for fiscal year 2008. Table 27 shows the breakdown of unsuccessful FP&S  applicants by department type and community service area for fiscal year  2007.
section_title: Appendix XIV: Comments from the Department of Homeland Security
section_title: Appendix XV: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the contact named above, Leyla Kazaz, Assistant Director,  and Deborah Ortega, Analyst-in-Charge, managed this assignment. Sarah  Arnett, Marie Webb, and Su Jin Yon made significant contributions to the  work. Christine Davis provided legal support. Stanley Kostyla and Jerome  Sandau assisted with design, methodology, and data analysis, and Lara  Kaskie provided assistance in report preparation.