Fire Grants: FEMA Has Met Most Requirements for Awarding Fire Grants, but Additional Actions Would Improve Its Grant Process
Released: 2009-10-30T00:00:00

Letter

Background

Statutory and Program Requirements of the Fire Grant Programs

FEMA’s fire grant programs are available to a variety of fire departments— those composed of volunteer firefighters, career firefighters, or a combination thereof. In the case of the AFG program, grants also extend to nonaffiliated EMS organizations. In the case of the FP&S program, grants also extend to local, state, national, or community organizations that are not fire departments, such as research universities and fire service organizations. The statutes authorizing FEMA’s fire grant programs specify how funds are to be distributed among certain eligible applicants and activities.

Authority for the AFG and FP&S programs derives from section 33 of the Federal Fire Prevention and Control Act of 1974 (Fire Act). The Fire Act requires FEMA to convene an annual meeting of individuals who are members of national fire service organizations for the purpose of recommending criteria for awarding grants for the next fiscal year. The act also requires FEMA, in consultation with national fire service organizations, to appoint fire service personnel to conduct a peer review of the grant applications, the results of which FEMA is to consider in awarding the grants. The Fire Act also contains specific grant application requirements. In particular, AFG and FP&S grant applicants are statutorily required to provide information demonstrating financial need; an analysis of costs and benefits resulting from the assistance; a list of other sources of federal funding received by the applicant to avoid duplicative funding; and an agreement by the applicant to provide information during the grant period to the National Fire Incident Reporting System, which represents the world’s largest national, annual database of fire incident information. AFG grant applicants are subject to an additional evaluation requirement—the extent to which the grant would enhance the applicant’s daily operations and the grant’s impact on the protection of lives and property.

Section 34 of the Federal Fire Prevention and Control Act of 1974 (SAFER Act) provides the authority for the third fire grant program administered by FEMA. There are two types of SAFER grants: hiring grants, which are open to career, volunteer, and combination fire departments, and recruitment and retention grants, which are open to volunteer and combination fire departments, or to state or local organizations that represent the interests of volunteer firefighters. Hiring grants are subject to specific cost-sharing requirements between the federal government and the grantee, with the federal share decreasing over the 4-year grant period. Furthermore, the grantee is required to commit to retaining any firefighter hired through grant funds for at least 1 year after federal funding ends, amounting to a 5-year service commitment. The statutory cost-share and service commitment requirements applicable to SAFER hiring grants do not apply to SAFER recruitment and retention grants.

SAFER grants, like AFG and FP&S grants, are awarded on a competitive basis through a peer review process. The SAFER Act is also similar to the Fire Act in requiring grant applications to include certain types of information. In addition to any information FEMA may require applicants to submit, the statute requires applicants to provide assurances regarding diversity in hiring; to explain their inability to address the need without federal assistance; and to specify long-term retention plans after federal funding ends, including, for hiring grants, how the applicant plans to meet the statute’s 5-year service commitment. A hiring grant applicant is also required to discuss what it will do to ensure that its department does not discriminate against firefighters who engage in volunteer activities in another jurisdiction during off-duty hours. The SAFER Act has a statutory sunset of 10 years from the date of enactment, such that the agency’s authority to make SAFER grants will elapse on November 24, 2013. (See app. II for a table that sets forth the statutory requirements applicable to the AFG, FP&S, and SAFER grant programs).

Each appropriations act enacted after January 2002 has made fire grant appropriations available for 2 fiscal years, after which any unobligated funds expire. The Consolidated Security, Disaster Assistance, and Continuing Appropriations Act, 2009—which contains the fiscal year 2009 Department of Homeland Security Appropriations Act—provided $775 million for firefighter assistance, including $565 million for fire grants and $210 million for SAFER grants. Table 1 shows the AFG and SAFER appropriations beginning with their first funded years, fiscal year 2001 and fiscal year 2005, respectively. (The FP&S funds are included in the AFG appropriation.)

Development of Grant Criteria and Funding Priorities

FEMA describes annual funding priorities for the grant programs in its grant guidance document, which incorporates recommendations from the programs’ criteria development panel. The criteria development panel is composed of subject matter experts who meet annually for 3 days to review and modify the previous year’s funding priorities and award criteria for all three fire grant programs, and whose recommendations are summarized in a report to FEMA. Detailed information about the mission and purpose of each fire service organization is contained in appendix III. The review process for all three grant programs includes three evaluation stages: an automated scoring or prescreening process to determine eligibility and alignment with the grant programs’ funding priorities; a peer review panel, in which practitioners within the fire service community evaluate and score the applications; and a final technical review by subject matter specialists, FEMA officials in the AFG Program Office and the Grants Management Division, as well as officials in state homeland security offices, if applicable.

The criteria development panel also updates the previous year’s scoring matrix, which is a confidential, weighted numerical scoring methodology that reflects the identified funding priorities. The scoring matrix is used to rate AFG and SAFER applications by scoring the answers to the application questions that are weighted to reflect the funding priorities. If applicants request funding for multiple activities, each activity is scored independently of the others. The activities’ scores are then weighted based on the dollar amount and combined to determine the application’s overall score.

Following review and discussion of the previous year’s information, the criteria development panel submits its recommendations on the grant criteria and funding priorities—including those on which participants do not reach consensus—to the AFG Program Office in the form of a report. After incorporating the panel’s recommendations and developing the grant guidance for the three grant programs, the AFG Program Office submits the guidance to other offices within FEMA for internal review. Subsequently, it sends the guidance to DHS and OMB for approval. The grant review and award process is represented in figure 1. (See app. IV for more detailed information about each of the steps involved in the application and award process.)

Figure 2 shows the distribution of federal funds for the three fire grant programs by state for fiscal year 2008.

As shown in table 2, grant applicants submitted more than 22,000 applications for the AFG and SAFER grant programs for fiscal year 2008, and as of July 2009, FEMA had awarded a total of 5,060 grants. As of July 2009, applicants submitted about 2,500 applications for the fiscal year 2007 FP&S grants and FEMA had awarded a total of 216 grants.

FP&S application and award numbers are for fiscal year 2007 funding and reflect all fiscal year 2007 awards made through July 2009. No fiscal year 2008 FP&S grants had been awarded as of July 2009.

FEMA received about 25,000 applications for AFG, SAFER, and FP&S grants in fiscal year 2007 and awarded more than 5,000 grants. See apps. V through X for more detailed information about the distribution of AFG, SAFER, and FP&S program funds.

FEMA Met Most Statutory and Program Requirements for Distributing Grants, but Did Not Fully Track Compliance with One Statutory Requirement and Had Not Fully Met One Program Requirement for Fiscal Year 2007

In awarding fiscal year 2008 AFG and SAFER grants and fiscal year 2007 FP&S grants, FEMA met seven of eight statutory and two of three program requirements. These requirements specified how FEMA was to distribute appropriated grant funds between different applicants and activities. In July 2009, when FEMA provided fire grant award data for our review, the agency was in the process of awarding fiscal year 2007 FP&S grants and fiscal year 2008 AFG and SAFER grants. These years were the latest for which grants had been awarded and for which we were able to determine FEMA’s compliance with statutory and program funding distribution requirements.

Fire grant data for fiscal years 2007 and 2008 are current as of July 2009.

According to AFG program guidance, NFPA reported that combination departments protect 33 percent of the nation’s population and volunteer departments protect 22 percent. We did not verify FEMA’s estimates.

As of July 2009, FEMA Had Met Four of Five AFG Statutory Requirements for Distributing Fiscal Year 2008 Grants, but Had Not Fully Met One of Three Fiscal Year 2008 Program Requirements

Although FEMA Met Most AFG Statutory Requirements, It Did Not Fully Track EMS Data to Ensure That It Met One Statutory Requirement for Fiscal Year 2008

FEMA met four of five statutory requirements related to AFG grants for fiscal year 2008. First, as shown in figure 3, FEMA has consistently met the population-based statutory requirement for awarding fire grants to volunteer and combination fire departments from fiscal year 2002 through July of fiscal year 2008.

Based on our review of fiscal year 2008 grant data, FEMA also met three other statutory funding distribution requirements for the AFG program, as identified in table 3.

Fire grant data for fiscal year 2008 are current as of July 2009. The fiscal year 2008 fire grant period closed on September 30, 2009, which is the end of fiscal year 2009. maximum ceilings established by the Fire Act. In addition, FEMA a about $378,560 of its fiscal year 2008 AFG appropriation for foam firefighting equipment. According to a program specialist responsible f administering the AFG and SAFER grant programs, FEMA granted all requests that were submitted for foam firefighting equipment, since the total amount requested was below the $3 million FEMA was required tomake available for this purpose. Thus, in meeting these requirements, FEMA ensured that its grant award s were consistent with the funding percentages mandated by statute.

FEMA Met Two AFG Program Requirements and Fell Slightly Short of Meeting Another

The AFG guidance states that no more than 45 percent of the AFG funds may be awarded to career departments. As shown in figure 4, FEMA has FEMA has consistently met this requirement from fiscal years 2002 through 2008. consistently met this requirement from fiscal years 2002 through 2008.

According to a program specialist responsible for administering the AFG and SAFER grant programs, in fiscal year 2007, OMB instructed the AFG Program Office to separate the percentage of funding given to volunteer and combination departments in proportion with the population that each type of department protects. Thus, FEMA incorporated this requirement in its program guidance. Specifically, the AFG 2008 grant guidance specified two requirements for FEMA to distribute grant awards: (1) volunteer departments are to receive at least 22 percent and (2) combination departments are to receive at least 33 percent of the total appropriation. While FEMA met the requirement to award at least 22 percent to volunteer fire departments in fiscal year 2008, it had not reached the minimum requirement of awarding at least 33 percent to combination departments as of July 2009. Specifically, FEMA awarded volunteer departments about 39 percent ($217 million) of its $560 million in fiscal year 2008 appropriated funds, but only awarded about 27 percent ($149 million) of its appropriation to combination departments. We also analyzed fiscal year 2007 data to determine whether FEMA met the requirement related to distributing funds to combination departments since the separation between volunteer and combination departments occurred. FEMA fell slightly short of meeting this program requirement because it awarded combination fire departments about 32 percent ($173 million) of the total appropriation of fiscal year 2007 grant funds—only 1 percent less than that required by program guidance (see fig. 5). According to a program specialist responsible for administering the AFG and SAFER grant programs, FEMA attempts to comply with the guidance pertaining to population-based proportional grant funding, and he indicated that the shortfall in fiscal year 2007 awards to combination fire departments may have been an oversight.

As of July 2009, FEMA Had Not Met the Statutory Requirement Related to Distributing Fiscal Year 2008 FP&S Grants, but Met This Requirement for Fiscal Year 2007 FP&S Grants

Because of delays in the approval of program guidance, no fiscal year 2008 FP&S grants had been awarded as of July 2009. Therefore, we reviewed FEMA’s data related to its fiscal year 2007 FP&S grant funding distributions. According to these data, FEMA met the 5 percent minimum statutory requirement in fiscal year 2007 by awarding $33,887,071, or about 6.2 percent, of the total AFG appropriation for FP&S grants. Thus, FEMA ensured that FP&S grant applicants received the percentage of funds mandated by statute.

FEMA Met Statutory Requirements for Distributing Fiscal Year 2008 SAFER Grants

In fiscal year 2008, FEMA met statutory requirements related to distributing funds for SAFER grants. FEMA is required to set aside 10 percent of the annual SAFER Act appropriation for all volunteer or majority volunteer fire departments to compete for hiring grants, which are otherwise open to all fire departments regardless of their career, combination or volunteer status. Recruitment and retention grants, which are open to volunteer and combination but not career departments, must also account for at least 10 percent of appropriated funding, in addition to the unused balance, if any, from the 10 percent hiring grant set-aside. FEMA complied with these requirements by awarding $21 million, or about 11 percent, of the $190 million in SAFER funding to volunteer and majority volunteer fire departments for hiring efforts. In addition, FEMA also awarded $20 million, or about 11 percent, of the funds to volunteer and combination departments for the purpose of recruitment and retention. Thus, FEMA ensured that SAFER grant applicants received the percentage of funds mandated by statute (see fig. 6).

FEMA Has Taken Actions to Assist Grant Applicants and Involves the Fire Service Community in Establishing Grant Criteria and Reviewing Grant Applications

FEMA has developed various tools to assist grant applicants with the application process and involves the nine major fire service organizations in developing criteria for annual fire grant funding priorities and in the peer review process. FEMA assists grant applicants by sponsoring workshops, publishing an online tutorial, and providing a toll-free hotline, among other actions. Each of the nine major fire service organizations sends representatives to serve on the annual criteria development panel, which recommends changes to the grant evaluation criteria and the funding priorities for the next fiscal year. During the peer review process, fire service practitioners independently rank the grant applications according to the evaluation elements recommended by the criteria development panel.

FEMA Has Assisted Grant Applicants through Workshops and Online Tutorials, among Other Actions

FEMA has developed various tools to assist fire grant applicants with the application process. According to a program specialist responsible for administering the AFG and SAFER grant programs, FEMA’s regional offices sponsored approximately 400 AFG and SAFER workshops throughout the country in 2008. While not all of the 36 randomly selected fire grant applicants in our nonprobability sample had attended workshops, 6 grant applicants that had done so explained that they received basic information on the grant application and award process, such as a review of the grant guidance and funding priorities, and how to write the narrative section of the application. See appendix XI for a listing of fire grant applicants included in our interviews. FEMA has also contracted with the North American Fire Training Directors to provide a grant-writing training class to fire departments in all 50 states throughout the year. The training class explains the grant opportunities available through the AFG program, describes the application process, and provides detailed information to guide applicants in drafting narratives. The course includes a slide presentation with an instructor’s guide and is designed to be about 4 to 6 hours in length. The contract allots up to $5,000 per state, totaling $250,000 for grant-writing technical assistance to be delivered from January 2009 through January 2010.

In addition, FEMA has created an online tutorial to guide AFG, SAFER, and FP&S applicants through the grant application process. Of the 36 applicants we interviewed, 20 applicants stated that they used online tutorials and 13 of them stated that the tutorial was helpful to a great or very great extent. Two of these applicants stated that the tutorials were useful to a great extent because they provide needed information on specific sections of the applications and help identify problematic areas.

Applicants seeking further assistance can call a toll-free hotline, which is staffed by contract personnel who have firefighting experience, or they can e-mail FEMA. FEMA provides technical assistance Monday through Friday for each of the three grant programs and also provides such assistance over the weekend for AFG applicants. In 2008, FEMA received an estimated total of 12,000 calls to the hotline and responded to an estimated 10,000 e-mails. FEMA has also established a mentoring program designed for departments that have unsuccessfully applied for fire grants for at least 5 years and offers this assistance to all departments that qualify. About half of the departments accept FEMA’s offer to participate in the mentoring program, in which each participating department is paired with a former peer review panelist and given a tutorial to guide it through the process. As of May 2009, about 400 departments were being mentored and about 30 to 40 percent have been successful in receiving a subsequent grant. While FEMA allows applicants to hire a grant writer to assist them with the process, applicants are responsible for the accuracy of information provided by the grant writer. The grant writer fees included in the grant amount requested are reimbursable, providing that they are declared in the application and do not depend on award.

FEMA Involves the Fire Service Community in Establishing Grant Criteria and Funding Priorities and in Assessing Grant Applications

FEMA has taken a number of steps to involve the fire service community in the grant process. For example, each year, FEMA brings together a panel of fire service professionals representing the leadership of the nine major fire service organizations to conduct a criteria development meeting to develop the program’s priorities for the coming year. According to a program specialist responsible for administering the AFG and SAFER grant programs, the panel convenes for 3 days in the summer before the annual appropriations process and is composed of 3 representatives from each of the organizations, totaling roughly 50 participants, including FEMA staff. The panel is responsible for making recommendations to FEMA’s AFG Program Office regarding the creation of program priorities, modification of program priorities, or both for all three fire grant programs—AFG, FP&S, and SAFER—as well as the development of criteria upon which the evaluation of grant applications is based. The panel’s recommendations are placed in a report that panelists submit to FEMA, which then incorporates the suggestions into the next fiscal year’s grant guidance.

Peer Review Panel Process Is Designed to Support Independent Assessment of Application Merits

FEMA’s peer review process—in which members of the nine major fire service organizations participate in assessing grant applications—also helps ensure that the fire service community is involved in making grant awards. According to FEMA, the peer review process is a key component for ensuring fairness in awarding fire grants. Peer review panelists are to conduct an independent assessment of the merits of the applications based on the extent to which the proposed projects align with the grant year’s funding priorities and meet the program’s goals and objectives.

Before arriving at the peer review panel, participants are required to complete an online tutorial and test, and then submit their certification of a passing grade during the panel orientation. If a panelist has not completed the tutorial, he or she is required to do so at the panel orientation. When panelists arrive at orientation, they are required to sign and submit a statement declaring that they have no known or apparent conflicts of interest as well as a nondisclosure form agreeing to keep the results of the review confidential. The panelists are then divided into groups of four at different tables. FEMA instructs the panelists not to review applications if they know the applicant or if the applicant is from their state. In the event that a potential conflict of interest arises, FEMA replaces the entire batch of applications provided to the table of panelists with a new batch. FEMA also instructs the panelists not to share applicant information with any panelists other than those seated at their table. All panelists receive an evaluation sheet that lists the evaluation factors, along with a rubric that provides guidelines for rating grant applications against the evaluation factors. They also receive a copy of the grant guidance, which contains the funding priorities.

New panelists receive a 2-½-hour orientation by FEMA program staff, who provide instruction on distinguishing between average and good applications, the logistics of individual scoring and table discussion, and the possible need to reduce the requested grant amount, among other things. Once the orientation is completed, the panelists individually read and score the narrative section of the applications as well as responses to other parts of the application, based on the applicable evaluation criteria. For example, for fiscal year 2008 AFG applications, panelists provided numerical scores on the basis of four evaluation factors stated in their score sheets, which were (1) project description, (2) cost/benefit of the proposed project, (3) financial need, and (4) effect of the proposed project on daily operations.

After each panelist at the table scores an application, the panelists discuss any differences of opinion and the merits or limitations of the application. Orientation facilitators inform panelists that the aim of the table discussion is not to arrive at a consensus, but rather to discuss each application as it pertains to each of the evaluation elements. If panelists are unable to reconcile any large scoring disparity (defined as 10 points or greater), they bring the dispute to the attention of a panel chair member who is responsible for ensuring that panelists document their discussion and indicate the scoring disparity on their scoring sheets. Panelists may amend their individual scores or choose to keep them unchanged on their evaluation sheets. FEMA files all of the panelists’ evaluation sheets for each application, including the panelists’ comments and recommendations to reduce the funding amount, reject, or award, with the applications. After evaluations are entered into FEMA’s database, an average score is electronically generated that determines whether the application proceeds to the technical review process, which occurs concurrently in a separate room at the panel location for applications with the highest scores.

Fire Service Community Participates in Peer Review

FEMA AFG Program Office officials explained that the number of panelists varies from year to year and the number of nominees that they request from each of the nine major fire service organizations depends upon the amount of appropriated funding as well as the number of applications submitted. Typically, the organizations each nominate about 24 to 40 people for the AFG panel, 10 for the SAFER panel, and 24 to 28 for the FP&S panel. FEMA also sends letters to subgroups within the organizations that represent minorities to receive nominations to help diversify the panel. According to a program specialist responsible for administering the AFG and SAFER grant programs, while FEMA does not verify whether the organizations’ nominees are qualified to attend the panel, it asks for the résumés of self-nominees or of those nominated by members of Congress. The official explained that in selecting the peer review panelists, FEMA considers availability to attend, racial diversity, and the ratio of new-to-repeat panelists. In fiscal year 2008, 285 people served on the AFG panel, 47 served on the SAFER panel, and 160 served on the FP&S panel. The official stated that FEMA considers panelists new if they have not participated in that particular grant program panel review, regardless of prior experience as a peer reviewer for another grant program. Although FEMA selects both new and returning panelists to review applications in any or all three grant categories, it tries to limit returning panelists to no more than one-third of the total panel composition. However, AFG Program Office officials may invite additional returning panelists if there are not enough confirmed attendees. Panelists are volunteers—although FEMA pays the entire cost of each panelist’s transportation, food, and lodging, it does not compensate panelists for any loss of income they may incur while serving on the panel. Lodging is typically provided at a federal training facility in Emmitsburg, Maryland, at no cost to the grant program.

In interviews with a nonprobability sample of 36 fire grant applicants, 22 applicants, or about 61 percent, stated that they had never been asked to serve on a panel. When asked whether they thought that the peer review process was fair and objective, 23 stated that it was, while 3 stated that it was not, and 10 did not know. In addition, 32 of the applicants stated that they believed that experience as a peer reviewer was beneficial to completing a grant application.

Although FEMA officials attempt to ensure that new peer review panelists make up two-thirds of the peer review panel each year, they stated that they do not currently undertake additional outreach activities themselves to encourage nominations of new panelists, such as notifying applicants of opportunities to serve on peer review panels during FEMA’s workshops or other assistance activities they sponsor for applicants. Rather, FEMA relies on the nine major fire service organizations for nominations of new panelists. AFG Program Office officials stated that while they strive to provide an even chance to as many fire departments and other organizations as possible to serve on peer review panels, representatives of departments that are invited sometimes fail to appear to serve on the panel without informing FEMA. Therefore, officials have invited some departments multiple times because they have proven to be reliable and good reviewers. They acknowledged that although FEMA does not currently limit the number of times that a department can send representatives to serve on the panels, establishing such a limit could expand opportunities for other departments to participate in the peer review process. In addition, they also stated that they are considering conducting outreach efforts to expand peer review participation, such as announcing opportunities to serve on an upcoming peer review panel at workshops. In addition to expanding peer review participation, such efforts could benefit peer review panelists by allowing them to incorporate firsthand knowledge of the panel process into their future grant applications.

While FEMA Has Taken Actions to Help Make Its Grant Process More Accessible, FEMA Could Benefit from Improved Clarity and Consistency of Grant Guidance and Better Controls over the Process for Reviewing and Approving Grant Guidance

FEMA has taken actions—such as publishing grant guidance and applications online—to ensure that its grant process is more easily accessible to grant applicants, but the agency could enhance the clarity and consistency of its grant guidance and the controls over its review and approval process. While grant guidance priorities are generally perceived as clear by grant applicants we interviewed, we identified inconsistencies between the grant guidance and the grant applications and grant scoring matrix language. In addition, FEMA has experienced significant delays in issuing grant guidance, and the agency does not have controls to monitor the progress of the review process. Finally, the majority of fire grant applicants that we interviewed felt they received inadequate feedback on why their applications were rejected.

FEMA Publishes Annual Grant Guidance Online to Increase Accessibility

Before each annual grant application period, FEMA publishes updated grant guidance on its Web site and has created an online grant application, which is designed to be user-friendly. Publication of the annual grant guidance on the FEMA Web site makes it more accessible to potential applicants. The grant guidance provides applicants with an explanation of the information that will be required in the application, as well as informing them of any grant priorities for the fire grants for that year, such as whether training will be given priority. FEMA encourages applicants to apply for fire grants online because of delays and mistakes associated with processing paper applications. The electronic application has built-in “Help” screens and drop-down menus. Applicants for each of the three grant programs are required to answer a series of questions about their department and the particular grant they are applying for, as well as provide a narrative that discusses the impact to result from the proposed use of the grant funds, among other things. Both the answers to the questions and the narrative are to be reviewed and scored by the peer review panel.

FEMA’s Grant Guidance and Application Forms Generally Reflect Statutory Requirements for the Submission of Specific Information by Applicants, but May Not Obtain All Statutorily Required Information

The statutes authorizing the three fire grant programs contain specific grant application requirements, which require FEMA to collect and consider certain information from applicants in making grant awards; however, not all of these requirements are included in FEMA’s grant guidance and application forms. For the AFG and FP&S grant programs, the Fire Act requires grant applicants to include (1) information demonstrating financial need, (2) an analysis of costs and benefits resulting from the assistance, (3) a list of other sources of federal funding received by the applicant to avoid duplicative funding, and (4) an agreement by the applicant to provide information to the National Fire Incident Reporting System during the grant period. An additional requirement applies to the AFG program, requiring FEMA to consider the extent to which the grant would enhance the fire department’s daily operations and the grant’s impact on the protection of lives and property.

Based on our review, the AFG and FP&S fiscal year 2008 grant guidance and application forms instruct applicants to provide information consistent with the above statutory requirements, with one exception relating to the FP&S R&D activity, as indicated in table 4.

FP&S grants cover two activities: (1) fire prevention and safety and (2) firefighter safety R&D. However, FEMA grant guidance only instructed applicants to provide the statutorily required cost-benefit analysis for projects proposed under the fire prevention and safety activity, not the R&D activity. Apart from this exception, the grant guidance and applications forms for both the AFG and FP&S programs incorporate the Fire Act’s information requirements. In particular, according to the grant guidance for both programs, fire departments that are awarded grants are to provide information to the National Fire Incident Reporting System during the grant period, as required by statute. In addition, the application forms for both grant programs require applicants to identify other sources of federal funding they are receiving that may duplicate the purpose of their grant request. Although a program specialist responsible for administering the AFG and SAFER grant programs stated that few grant applicants receive grant awards from other sources, FEMA queries its internal records of all grant applicants to prevent making duplicate awards. The three remaining AFG statutory requirements—financial need information, a cost-benefit analysis, and an impact statement—appear as evaluation criteria in the AFG guidance that applicants are to address in their project narratives. The two remaining FP&S statutory requirements—financial need information and a cost- benefit analysis—appear as evaluation criteria for the fire prevention and safety activity, but the evaluation criteria for the R&D activity include only one of these two statutory requirements, financial need. By taking steps to ensure that all statutorily required information is included in the grant guidance and application forms, FEMA is better positioned to provide reasonable assurance that grants are awarded in accordance with the statute.

In addition, the SAFER Act also specifies certain information that applicants are required to submit, in addition to any other information required by FEMA. The statute requires each applicant to (1) provide assurances regarding diversity in hiring, (2) explain its inability to address the need without federal assistance, and (3) specify long-term retention plans after federal funding ends. With respect to the latter requirement, SAFER hiring grant applicants are to specifically discuss how they plan to meet the statute’s 5-year service commitment (i.e., 1-year of service for SAFER-funded firefighters after the 4-year funding period ends). Furthermore, SAFER hiring grant applicants are to address another statutory requirement, a commitment not to discriminate against firefighters serving as volunteers in other jurisdictions during off-duty hours.

15 U.S.C. § 2229a(b)(3)(B). and retention grants, the guidance and application form only partially address one of the information requirements, as indicated in table 5.

FEMA’s guidance for SAFER hiring grants includes each of the statutory information requirements within the evaluation factors that applicants are to address in their project narratives. For example, each hiring grant applicant is to include a statement regarding how the applicant plans to meet the nonfederal match requirement for the 5-year service period, including any long-term plans to retain the new firefighter positions, as required by the statute. Although the SAFER hiring grant guidance instructs applicants to submit all statutorily required information, the statement of long-term retention plans is less specific in the SAFER recruitment and retention grant guidance and application questions, which ask applicants to include “specifics about the recruitment and/or retention plan.” Because this language gives applicants the option of providing specifics on recruitment or retention plans, FEMA may not receive information on applicant’s long-term retention plans after federal funding ends, especially from applicants seeking grants for recruitment purposes. Clarifying the SAFER recruitment and retention grant guidance with respect to applicants’ long-term retention plans could help FEMA ensure that it collects the information necessary to determine whether awarded grants used for recruitment or retention purposes will have a lasting impact after federal funding ends. FEMA does explicitly instruct applicants to address the other two statutory requirements—diversity in hiring and inability to meet the need without federal assistance—which apply to SAFER recruitment and retention grants.

Grant Priorities Are Generally Perceived as Clear by Grant Applicants, but Are Not Consistently Aligned with the Scoring Matrix and Grant Application Questions

Seventy-eight percent (28 of 36) of the grant applicants that we interviewed described the grant guidance as being clear to a great or to a very great extent, 7 said that it was clear to a moderate extent, and 1 applicant said that he did not know. For example, 1 applicant stated that the guidance was consistently well written and another commented that it was simple and user-friendly. However, 10 applicants provided suggestions for how FEMA could further clarify its grant guidance— including its grant priorities. For example, 1 suggested that priorities be more expressly stated so that he could make a more qualified decision on whether to apply. The fiscal year 2008 AFG program guidance does not summarize funding priorities for any activity other than for the vehicle acquisition program. Likewise, the fiscal year 2008 FP&S and SAFER program guidance also do not summarize funding priorities. AFG Program Office officials acknowledged that the fire grants program guidance could be made clearer, possibly by incorporating tables or charts highlighting program priorities.

Moreover, while FEMA’s grant guidance and application questions for the AFG, FP&S, and SAFER grant programs generally incorporate statutory information requirements, priorities in the grant guidance are not always reflected in the scoring matrix and application questions. For example, the fiscal year 2008 SAFER guidance states that continuity—which refers to whether an applicant’s recruitment and retention activities are designed to continue beyond the grants’ period of performance—is a priority for recruitment and retention grants. However, there is no application question that addresses this priority, nor is there a scoring matrix value that corresponds to continuity. Further, the fiscal year 2008 AFG guidance for wellness and fitness grants prioritizes fitness and injury prevention projects over rehabilitation, but in the scoring matrix all wellness and fitness project categories are scored equally. Grant priorities/criteria in the guidance are updated every year based on the recommendations made by the criteria development panel. According to a program specialist responsible for administering the AFG and SAFER grant programs, it would be difficult to capture the concept of continuity in the form of a question, and the misalignment regarding wellness and fitness priorities may have occurred because of insufficient oversight by FEMA.

It is important that FEMA ensure that its grant guidance is not only clear but also consistently aligned with the application questions and scoring matrix. According to the National Procurement Fraud Task Force, grant funds are awarded to carry out goals and objectives as they are identified in the grant guidance. In order for there to be accurate and consistent alignment between the grant awards and guidance, the application questions and their weighted scoring values must also reflect the intentions of the grant program as stated in the guidance. Developing grant guidance and application questions that are consistent with funding priorities could help FEMA ensure that grant funds are awarded in accordance with the agency’s priorities.

Review and Approval Process of Draft Fire Grant Guidance Is Not Documented and Monitored

FEMA’s Section Chief for the FP&S program stated that although the FP&S grant guidance is to be issued in August or September at the start of each fiscal year, it has not been issued on time for the past 3 fiscal years— the review and approval process for the fiscal year 2008 grant guidance took over 17 months, and guidance was not issued until February 2009. See appendix XII for more detailed information about the time frames for the fiscal year 2008 fire grants process. Because of this delay, the peer review panel convened to assess grant applications in April 2009, and FEMA began awarding fiscal year 2008 FP&S awards in August 2009. FEMA program officials stated that because of the delays in the approval of the grant guidance, FEMA was unable to reserve classroom and dormitory space at the federal facility in Emmitsburg, Maryland, where prior peer review panels had met, and the panel met at a private hotel in Towson, Maryland, at a cost of about $90,000. As a result of this expenditure, there were fewer funds to award to grant applicants.

According to FEMA’s Office of Policy and Program Analysis, there is no systematic method for tracking the review and approval process for fire grant guidance, no internal deadlines, and no documentation to help determine the cause for delays in the issuance of grant guidance. AFG Program Office officials said that they are not fully aware of the review and approval process once the drafted guidance leaves the AFG Program Office and is sent to other offices within FEMA and DHS. They said that the delay in issuing the fiscal year 2008 FP&S guidance occurred when the FEMA Policy Coordinating Group found that the AFG Program Office did not possess a Paperwork Reduction Act clearance in order to collect information from others outside of the federal government. In response, the AFG Program Office submitted a request for an emergency clearance, which OMB did not approve. According to OMB officials, the clearance was denied because OMB believed that FEMA should go through proper channels to obtain a routine clearance for its fire grant program because FEMA had previously asked for other emergency clearances.

OMB officials stated that obtaining a routine clearance typically requires about 120 days.

Grant Applicants Desire More Specific Feedback on Why Their Grant Applications Were Turned Down

Once the grant review process is completed, unsuccessful applicants receive letters notifying them of the reason(s) their applications were turned down for grant awards, but some applicants have stated that they would like more information on the reasons for their rejection. According to AFG Program Office officials, applicants receive rejection letters at the same time as grant awards are being announced. The officials further stated that sending rejection letters to thousands of applicants is time and resource intensive. The AFG Program Office has developed 16 templates to use in sending AFG applicants letters explaining the reasons for their rejection. Explanations that FEMA provides to unsuccessful applicants include (1) discrepancies between the itemized request and the narrative justification for those items, (2) an applicant or the specific activity for which grant funding was requested is ineligible for funding, or (3) incomplete fulfillment of the requirements of previous grant awards received by the applicant. These letters inform the applicants that there were an extremely high number of applications and a finite amount of funding, which resulted in many worthy applicants not being funded.

In certain cases, the information contained in these letters is more positive and does not provide detailed information on the reason for the rejection. For example, FEMA may send an applicant a letter explaining that while the peer review panelists’ scores indicated that its application was generally good, the agency does not have enough funding to offer the applicant an award after awarding grants to applicants with higher scores. However, FEMA states that if it identifies any excess funding or if some of the applicants that have been offered a grant decline the offers, the agency might be able to fund the request. According to AFG Program Office officials, applicants are not allowed to appeal panelists’ scores. Rather, they can only request reconsiderations because of processing issues. For example, they can argue that terminology in the grant guidance was unclear.

Four of the nine major fire service organizations expressed concern about the level of feedback provided to rejected applicants. One official stated that FEMA’s denial letters lack specificity about why their applications were denied, while another official stated that rejected departments were frustrated with not knowing why their applications were rejected year after year. Another official suggested that FEMA publish a list of the top 10 reasons why grants are turned down in order to provide greater clarity to applicants.

Moreover, 61 percent of the 36 grant applicants that we interviewed (22 of 36) stated that the feedback they received from FEMA regarding why their applications were turned down was helpful to little or no extent. One applicant stated that he did not receive any feedback from FEMA and that his fire department had called the agency to learn the status of its application. In addition, 6 applicants stated that the feedback was helpful to some or to a moderate extent and another 6 stated that the feedback was helpful to a great or very great extent. However, 1 applicant reported not knowing the extent to which the feedback was helpful. Seventy-five percent (27 of 36) of grant applicants with whom we spoke suggested that FEMA’s feedback should include specific reasons why the grant application was denied. For example, 3 grant applicants suggested that it would be helpful if FEMA provided information regarding the specific stage in the application review process where an application was rejected. One fire department suggested that FEMA cite whether an application contained a poorly written narrative or was rejected for another reason, such as a request for equipment that was not a funding priority. Another applicant suggested that peer reviewers provide applicants the reasons why their applications scored low, and another suggested that FEMA include information on available assistance for future grant cycles, such as the online tutorial or list of workshops.

Providing feedback to grant applicants is an important part of the fire grant program. In its 2007 report on the AFG program, the National Academy of Public Administration listed improving feedback to grant applicants as a strategic objective for the grant management process. The strategic objective is for FEMA to improve the feedback to unsuccessful candidates so that applicants can understand why they did not receive grants, thereby increasing participation and improving the quality of requested grants. AFG Program Office officials acknowledged that they could strengthen efforts to improve feedback to applicants who are turned down for grants following the peer review process. According to the Director of the AFG program, FEMA could modify the feedback provided to unsuccessful applicants to better explain the reasons why applications were rejected. We have previously reported the need to provide clear feedback to unsuccessful applicants on the strengths and weaknesses of their grant applications. Providing specific feedback to applicants regarding the reasons that they are denied grants could help FEMA strengthen future grant application processes and better position it to achieve its intended benefits of assisting fire departments that are most in need.

Conclusions

Through the years, the U.S. fire service community has experienced changes in its responsibilities to the public as well as decreases in local budget distribution, which underscore the need for fire departments nationwide to have the resources necessary to protect their communities. Through its fire grant programs, FEMA has an opportunity to assist fire departments that are struggling to meet their responsibilities. While FEMA distributed fire grants to a variety of applicants for a variety of activities, developing and implementing a procedure for capturing the percentage of appropriated funds awarded to fire departments related to EMS equipment and training would better position FEMA to more readily determine if it met the minimum amount established by statute. FEMA could improve the clarity and consistency of the grant review and award process by collecting all statutorily required information and eliminating inconsistencies between the guidance, the scoring matrix, and the application, which may confuse applicants. Additionally, by improving its internal controls to document and track the grant guidance review and approval against established milestones, FEMA could provide applicants the opportunity to plan for matching funds by determining when guidance will be issued each year. Finally, by providing more specific feedback and information on assistance, FEMA could help ensure that applicants have the opportunity to prepare better applications, and thus have a greater chance of being awarded grants in the future.

Recommendations for Executive Action

To ensure compliance with all AFG statutory requirements, we recommend that the Administrator of FEMA establish a procedure for tracking the percentage of grant funds awarded to fire departments for EMS purposes.

In addition, to improve the clarity, consistency, and controls of the grant review and award process, we recommend that the Administrator of FEMA take the following three actions: Ensure that the priorities in the grant guidance are aligned with the scoring matrix and the grant application questions, and that FEMA requests applicants to submit all statutorily required information.

Coordinate with the Secretary of Homeland Security to document the review and approval process for its grant guidance, develop a tracking system to monitor the progress of the review within FEMA and DHS, and set internal deadlines so that guidance can be issued in a timely manner.

Inform unsuccessful applicants about the forms of assistance available to them in future grant cycles and provide more specific feedback to applicants that are turned down for grants following the peer review.

Agency Comments

We provided a draft of this report to DHS and FEMA for review and comment. On October 22, 2009, DHS provided written comments on the draft report, which are reprinted in appendix XIV. DHS concurred with our recommendations and is taking actions to address them. DHS stated that FEMA will examine the available options and adopt one for manually and electronically monitoring percentages of grant funds awarded to fire departments for emergency medical services purposes to ensure compliance. DHS also stated that FEMA will explore options and identify means for providing clear, concise, and consistent information to applicants on the funding priorities and statutorily required information. In addition, DHS stated that FEMA will work with applicable offices to enable a timely review and tracking of program guidance material. Further, DHS stated that additional training and outreach efforts are being developed to enhance feedback to applicants.

We are sending copies of this report to the Secretary of Homeland Security, Director of the Office of Management and Budget, interested congressional committees, and other interested parties. The report also will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions concerning this report, or wish to discuss these matters further, please contact me at (202) 512-8777 or jenkinswo@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix XV.

Appendix I: Objectives, Scope, and Methodology

The accompanying explanatory statement to the Consolidated Appropriations Act, 2008, mandates that we review the application and award process for Assistance to Firefighters Grant (AFG) and Staffing for Adequate Fire and Emergency Response (SAFER) grants. Thus, we addressed the following questions: To what extent has the Federal Emergency Management Agency (FEMA) met statutory and program requirements for distributing the grant funds to a variety of applicants and activities?

What actions has FEMA taken to provide assistance to grant applicants and involve the fire service community in the grant process?

To what extent has FEMA taken actions to help ensure the fire grant process and related guidance are accessible, clear, and consistent with applicable statutory and program requirements?

To review the extent to which FEMA met statutory and program requirements for distributing fire grants to a variety of applicants and activities, we reviewed relevant statutory requirements from sections 33 and 34 of the Federal Fire Prevention and Control Act of 1974 (called the Fire Act and the SAFER Act, respectively, for purposes of this report). Based upon these statutes, we identified a total of eight statutory requirements that established specific percentages or dollar amounts designating how FEMA was to distribute funds among different grant applicants and activities. We also identified three relevant program requirements established in FEMA’s grant guidance for the AFG, SAFER and Fire Prevention and Safety (FP&S) grant programs, which related to distributing grant funds among different categories of activities and applicants. We then compared these statutory and program requirements to FEMA grant award data that stratified awards based on the type of fire department: volunteer, career, or combination—and based on the type of activity, such as awards for vehicle acquisitions. FEMA began maintaining electronic fire grant award data in fiscal year 2002. FEMA’s fire grant award data are current as of July 2009, at which time FEMA was in the process of awarding fiscal year 2007 FP&S grants and fiscal year 2008 AFG and SAFER grants. These years were the latest for which grants had been awarded at the time of our review and for which we were able to determine FEMA’s compliance with statutory and program funding distribution requirements. In addition to analyzing compliance issues, we also analyzed FEMA’s annual listings of applications and awards for the AFG and FP&S grant programs from fiscal years 2002 through 2008 and SAFER grant program from fiscal years 2005 through 2008 to provide descriptive information on a number of other characteristics, such as the type of community served by the applicant—urban, suburban, or rural. We provided descriptive information on the type of community served because the Fire Act requires FEMA to distribute AFG grants to a variety of different fire departments based on such characteristics as the type of community served, although the statute did not provide a specific percentage of funds against which we could evaluate compliance for the community-based requirement. We also determined the number of times that departments have applied for and been awarded grants. The descriptive information regarding the distribution of grant awards appears in appendixes V through X and appendix XIII of this report. To assess the reliability of data provided by FEMA, we reviewed and discussed the sources of data with agency officials. We determined that the data were sufficiently reliable for the purposes of this report.

To determine the actions FEMA has taken to provide assistance to grant applicants and involve the fire service community in the grant process, we collected and reviewed pertinent FEMA documents, such as program guidance, and observed FEMA’s fiscal year 2010 criteria development panel process and the fiscal year 2008 FP&S peer review panel process. We conducted interviews with officials from FEMA and the nine fire service organizations to determine the type of information FEMA provides to applicants on the grant application and review process. We analyzed the methods FEMA uses to inform applicants about the fire grant programs, including the various types of outreach and assistance that the agency provides to applicants. Specifically, we reviewed information on grant-writing workshops, online tutorials, technical support, and mentoring, among other forms of applicant outreach, and reviewed the contract between the North American Fire Training Directors and FEMA to provide additional grant-writing assistance to applicants. We also conducted interviews with officials from FEMA, the U.S. Fire Administration, and the nine fire service organizations to understand how FEMA establishes criteria for awarding grants to applicants. We analyzed the information regarding FEMA’s procedures for selecting peer reviewers; the training that panelists receive before reviewing applications; and the measures FEMA takes to ensure that peer review panelists maintain independence, safeguard against any conflict of interest, and adhere to restrictions related to confidentiality. We analyzed peer review guidance and other AFG Program Office documents, such as the criteria development reports and panel application evaluation sheets, to determine the process through which peer reviewers score grant applications. We collected and analyzed information pertaining to the technical review process and observed the fiscal year 2008 FP&S subject matter specialists’ portion of the technical review to determine how FEMA incorporates scores from the technical reviewers and makes final award decisions.

To evaluate the extent to which FEMA has taken actions to help ensure that the fire grant process and related guidance are accessible, clear, and consistent with applicable statutory and program requirements, we reviewed FEMA’s methods of publishing grant guidance online and the online applications. We also identified statutory requirements pertaining to information applicants are required to submit in their fire grant applications and analyzed FEMA’s fiscal year 2008 grant guidance and application forms to determine whether they consistently instructed applicants to submit the statutorily required information. We obtained and analyzed FEMA AFG Program Office documents, such as the scoring matrix and documents describing the prescreening process. We analyzed and compared the fire grant programs’ funding priorities contained in fiscal year 2008 AFG and SAFER grant guidance and 2007 FP&S grant guidance with the application questions and the scoring matrix to determine the extent to which they were consistent based on criteria from the National Procurement Fraud Task Force. Through our analysis of grant program documents for fiscal years 2007 and 2008 and our interviews with officials from the Office of Management and Budget, the Department of Homeland Security (DHS), and FEMA, we obtained obtain information about the approval and issuance of the program and application guidance as well as how grant decisions are announced. We analyzed the procedures that FEMA uses to announce grant decisions and the type of feedback it provides to unsuccessful applicants and determined the circumstances under which applicants may appeal FEMA’s grant decisions. We compared the views of a nonprobability sample of 36 randomly selected fire grant applicants regarding feedback to unsuccessful candidates, and reviewed the National Academy of Public Administration 2007 assessment of the AFG program.

From June 22 through June 29, 2009, we conducted structured interviews by phone with fire chiefs and other officials knowledgeable about the fire grants program from a nonprobability sample of 36 randomly selected fire grant applicants that did or did not receive fiscal year 2008 funding for the AFG and SAFER grants and fiscal year 2007 funding for the FP&S grants. We obtained their perspectives on the application and award process. The sample included fire grant applicants across the continental United States and Alaska. We obtained a list of the universe of applicants from FEMA for the respective fiscal years, from which we randomly selected fire departments within seven grant categories: (1) awarded AFG applicant, (2) turned down AFG applicant following the peer review panel process, (3) turned down AFG applicant following the initial electronic screening process, (4) awarded FP&S applicant, (5) turned down FP&S applicant following the peer review panel process, (6) awarded SAFER applicant, and (7) turned down SAFER applicant following the peer review panel process. We conducted two pretest interviews in person with representatives of fire departments in South Carolina and Pennsylvania to further refine our questions. An independent GAO methodologist reviewed our questionnaire to identify and revise potentially biased questions. Although we are not able to generalize the results of the nonprobability sample to the general population of applicants, the questionnaire allowed for a series of open-ended and close-ended responses on the grant application and award process, including questions on the perceived fairness and objectivity of the grant programs. Because of the scope of our work, we reviewed the fire grant programs’ application and award process, but did not assess the extent to which FEMA measures its performance in implementing these fire grant programs.

We conducted this performance audit from January 2009 through October 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. To assess the reliability of data provided by DHS and FEMA on the fire grant applicants, review criteria, and award procedures, we reviewed and discussed the sources of data with agency officials. We determined that the data were sufficiently reliable for the purposes of our review, and that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Statutory Requirements for Fire Grants

The AFG, FP&S, and SAFER grant programs are authorized to award funds for a range of purposes to various eligible organizations. Statutory requirements pertaining to each grant program are shown in table 6.

Appendix III: Mission, Membership, and Description of Fire Service Organizations

Officials from the nine fire service organizations participate in the criteria development panel to recommend changes to the upcoming year’s grant priorities, as well as nominate members to serve on the peer review panel. The organizations’ missions and memberships represent a range of interests within the fire service community. (See table 7.)

Appendix IV: Application and Award Process for Fire Grants

The AFG, FP&S, and SAFER grant programs are evaluated in three phases of review. The first phase is an automated scoring process to select competitive AFG grant applications, or a prescreening process for FP&S and SAFER grants. The second phase for all grants is a peer review panel process to evaluate the extent to which an application aligns with the grant year’s funding priorities and meets the programs’ goals and objectives. The third phase is a technical review panel process to determine technical feasibility, avoid duplication with state initiatives, and make any modifications to the potential award.

Automated Scoring and Prescreening Process

After the application period is closed, all fire grant applications undergo either an automated scoring process, where each application is ranked relative to the funding priorities described in the guidance, or a prescreening process, where each application is screened for eligibility. For example, for the AFG and SAFER grant programs, the AFG Program Office creates a scoring formula (following the criteria development panel’s scoring matrix), which is then entered into a computerized system by FEMA’s Information Technology Office. Through this formula, each application is scored and ranked electronically. For the AFG program, if an application for a project has a high-dollar item or activity that was ranked as a low priority, that item or activity may have adversely affected the scoring and thus may have taken the application out of the competitive range for peer review. FEMA officials stated that because more than 21,000 AFG applications are submitted every year and the AFG Program Office does not have the resources to review all of the applications, the number of applications it submits for peer review is the number of applications with the highest scores whose cumulative funding requests total 200 percent of the appropriated funding. In other words, AFG grants that go to peer review can total no more than twice the dollar value of the available grant amount.

SAFER applications also undergo an automated scoring and ranking process; however, FEMA submits all of them for peer review (1,314 applications for fiscal year 2008). Applications that best address the funding priorities score higher than applications that do not. Unlike the AFG automated scoring, which is used for the sole purpose of selecting applications for peer review, the automated score for SAFER grants accounts for one-half of the overall consideration provided each application, with the peer review accounting for the balance of the consideration.

Unlike AFG and SAFER grants, FP&S applications do not undergo an automated scoring and ranking process. Because of the smaller size of the FP&S program and the more technical and academic nature of some FP&S requests, the AFG Program Office manually screens for eligibility all applications submitted for the two FP&S activities, which are fire prevention and safety grants and R&D grants. Both the applicants and the projects are screened for eligibility based on statutory and programmatic eligibility criteria, and those found ineligible are removed from further consideration before the peer review process. For example, for-profit applicants and projects requesting fire suppression equipment or fire vehicles are considered ineligible. According to an AFG Program Office section chief, out of 2,637 FP&S applications submitted for fiscal year 2008 funding, 170 applications were considered ineligible before the peer review panel process and another 16 applications were found ineligible during the peer review and therefore were not scored.

Peer Review Process

Peer review panel participants are fire service professionals who are members of one or more of the nine major fire service organizations. The panel’s goal is to evaluate the extent to which an application aligns with the grant year’s funding priorities and meets the programs’ goals and objectives. The AFG Program Office officials explained that FEMA requests that each organization nominate members to serve as peer review panelists. FEMA also sends letters to subgroups within the organizations that represent minorities to receive nominations to help diversify the panel. Although the officials select both new and returning panelists to review applications in any or all three grant categories, they attempt to limit returning panelists to no more than one-third of the total panel composition. However, FEMA may invite more experienced panelists if there are not enough confirmed attendees.

In the orientation for the peer review of fiscal year 2008 FP&S applications, new panelists completed a review of two mock applications and discussed them as a group to familiarize themselves with the review process. These simulated exercises for new panelists occur for the AFG and SAFER panels as well.

For fiscal year 2008 SAFER hiring grants, panelists also scored applications numerically according to six evaluation factors stated in their score sheets, which were the extent to which the application described (1) a plan to use firefighters and the specific benefit these firefighters will provide for the fire department and the community, (2) a risk to the community and current firefighters that will be significantly reduced with grant funding, (3) the need for financial assistance, (4) a plan to recruit and hire minorities and women, (5) a long-range plan to make the nonfederal match and retain firefighters, and (6) a policy to prevent discrimination against firefighters who volunteer for other departments. For SAFER recruitment and retention grants, only the first four evaluation factors apply.

For fiscal year 2008 FP&S applications, panelists scored applications for the fire prevention and safety activity based on six evaluation factors stated in their score sheets using adjectives ranging from “Strongly Agree” to “Strongly Disagree,” which the AFG Program Office subsequently converts to numerical scores. The evaluation sheet contains six detailed evaluation factors: (1) financial need, (2) vulnerability statement, (3) implementation plan, (4) project evaluation plan, (5) sustainability, and (6) cost-benefit analysis. For the FP&S research and development activity, panelists scored applications reflecting the degree to which they addressed the following evaluation factors: (1) study purpose(s), goals and objectives, and specific aims; (2) scientific and technical merit of the proposed research; (3) dissemination and implementation; (4) resources— people and time; (5) protection for human subjects; (6) financial need; and (7) impact on firefighter safety.

Automated Scoring and Prescreening Process Results in Some Applications Not Undergoing Peer Review

In the House report accompanying DHS’s fiscal year 2008 appropriations bill, the House Committee on Appropriations raised concerns about the number of fire grant applications that did not reach the peer review stage (9,268 out of 20,972 according to the House report). In the subsequent explanatory statement accompanying DHS’s fiscal year 2008 appropriations act, both the House and Senate appropriations committees directed FEMA to provide fire grant applicants whose applications were not selected for peer review an official notification detailing the reasons for their rejection. A program specialist responsible for administering the AFG and SAFER grant programs stated that the agency sent all AFG applicants that submitted applications that were not peer reviewed a letter notifying them that their applications were not among those selected for the second phase of the competitive review. The letter explained that during the second phase, those applications that best addressed AFG’s established funding priorities for each eligible activity were approved and forwarded for peer review and that the objective of peer review is to further ensure the best use of grant funds. According to AFG Program Office officials, in fiscal year 2008, about 21,000 AFG applications were submitted, of which 8,000 were subsequently not selected for peer review. FEMA offered the applicants an opportunity to receive a more detailed explanation. The officials stated that applicants that submitted about 7,000 of these applications requested additional information, and four FEMA contractors spent about 4 months gathering the information needed to send electronic responses to these applicants to clarify specifically why the applications did not meet the criteria for peer review.

To determine the types of applicants whose applications were rejected before the peer review process, we reviewed data provided by FEMA. For fiscal year 2008, we found that 4,489 applicants (or about 29 percent) of the 15,544 applicants were turned down for all applications they submitted for AFG funds through the automated scoring process, and consequently none of their applications were peer reviewed. Appendix XIII contains more detailed information about unsuccessful fire grant applicants by department type and community service area.

Technical Review Process

Fire grant applications that receive the highest scores by the peer review panels are submitted for technical review. The technical review process consists of reviews made by subject matter specialists, the AFG Program Office or grants management specialists, or state homeland security representatives. The AFG Program Office has a group of subject matter specialists who review each potential award application to ensure that the project is technically feasible and that the application does not contain projects, activities, or items that are ineligible or otherwise not worthy of funding. In addition, they identify potential modifications to projects that would enhance the overall award, and identify applications that have scored outside the fundable range but should receive the award. The subject matter specialists review the entire application as well as the panelists’ comments. Once they have completed their review, the AFG Program Office staff reviews each potential award before making a recommendation on whether to award a grant to the applicant. The AFG Program Office staff assesses the findings from the previous reviews, determines whether any duplicate applications exist, and validates the eligibility of both the applicant and the items requested.

Following the review by the AFG Program Office staff, grant applications that have been recommended for award are submitted to grants management specialists in FEMA’s Grants Management Division. These specialists are responsible for reviewing the financial information in applications and for ensuring that the requested amounts are reasonable and calculated correctly. They also ensure that applicants have provided responses to a questionnaire that the Grants Management Division sends to applicants. The questionnaire solicits information such as whether any proposed reductions in the requested amount in the grant applications are acceptable and if the applicant is a recipient of other federal grants. These responses to the questionnaire, which are supplemental to the grant application, are processed internally by the Grants Management Division. The Grants Management Division might contact the grantee for additional follow-up or send the application back to the AFG Program Office, as appropriate. According to grants management specialists, their review typically requires only a couple of hours, but the approval process might take as long as a month or more, depending on how long applicants take to respond to the questionnaire and the extent to which follow-up information is necessary. Once the specialists approve the recommended applications for award, the applications are sent to the assistance officers in the Grants Management Division for approval, at which point the assistance officers obligate the awards.

Grant Awards Process

Although FEMA generally makes funding decisions using rank-order results from the peer review panel evaluation, it may deviate from the panel’s scores and make funding decisions based on the type of department (career, combination, or volunteer), the size and character of the community the applicant serves (urban, suburban, or rural), or both to satisfy statutory and programmatic funding goals. State homeland security offices may also review applications to ensure that the relevant proposed projects do not duplicate existing statewide programs. Since the number of submitted application requests exceeds the appropriated funding, applications reviewed within this final stage may not be awarded grants, despite falling within the fundable range.

FEMA announces these awards over several months as decisions are made, but does not make the awards in any specified order (i.e., by state, program, or any other characteristic). Awards are made until the funding is exhausted or the appropriation has expired.

Appendix V: AFG, SAFER, and FP&S Applicants and Award Distribution by State

Tables 8 and 9 show the AFG and SAFER applicants and award breakdown, respectively, by state for fiscal year 2008. Table 10 shows the FP&S applicants and award breakdown for fiscal year 2007.

Appendix VI: Distribution of AFG, SAFER, and FP&S Awards by Department Type, Fiscal Years 2002 through 2008

Tables 11, 12, and 13 show the distribution of AFG, SAFER, and FP&S awards, respectively, from fiscal years 2002 to 2008 by department types (e.g., career, combination, volunteer, paid on call/stipend, or a combination of these).

Appendix VII: Distribution of AFG, SAFER, and FP&S Awards by Community Service Area, Fiscal Years 2002 through 2008

Tables 14, 15, and 16 show the distribution of AFG, SAFER, and FP&S awards, respectively, from fiscal years 2002 to 2008 by community service area (e.g., rural, suburban, and urban).

Appendix VIII: Distribution of AFG Awards by Activity, Fiscal Years 2002 through 2008

Table 17 shows the distribution of AFG awards by activity (e.g., operations and safety, vehicle acquisition, and regional) for fiscal years 2002 through 2008. The amount of funding provided for operations and safety activities is consistently higher than that spent on regional activities and vehicle acquisition. Fiscal year 2008 funding for operations and safety grants amounted to $273.1 million out of the total $453.9 million grant awards. Grant funding for vehicle acquisition and regional applications was $131.7 million and $49.2 million, respectively.

Appendix IX: Distribution of AFG, SAFER, and FP&S Awards and Funding by Department Type

Tables 18 and 19 show the distribution of AFG and SAFER awards and funding, respectively, by department type for fiscal year 2008. Table 20 shows the distribution of FP&S awards and funding by department type for fiscal year 2007.

Appendix X: Distribution of AFG, SAFER, and FP&S Awards and Funding by Community Service Area

Tables 21 and 22 show the distribution of AFG and SAFER awards, respectively, by service area for fiscal year 2008. Table 23 shows the distribution of FP&S awards by service area for fiscal year 2007.

Appendix XI: Fire Grant Applicants in Nonprobability Sample

We conducted structured interviews with randomly selected applicants that applied for fiscal year 2008 funding for the AFG and SAFER grants and fiscal year 2007 funding for the FP&S grants to discuss various aspects of FEMA’s grant application and award process. (See table 24.)

Appendix XII: Time Frames for the Fiscal Year 2008 Fire Grants Process

We obtained information from FEMA officials and documents in order to prepare a timeline depicting the time frames for the fiscal year 2008 fire grants process. (See fig. 8.)

Appendix XIII: Unsuccessful AFG, SAFER, and FP&S Applicants by Department Type and Community Service Area

Tables 25 and 26 show the breakdown of unsuccessful AFG and SAFER applicants, respectively, by department type and community service area for fiscal year 2008. Table 27 shows the breakdown of unsuccessful FP&S applicants by department type and community service area for fiscal year 2007.

Appendix XIV: Comments from the Department of Homeland Security

Appendix XV: GAO Contact and Staff Acknowledgments

Acknowledgments

In addition to the contact named above, Leyla Kazaz, Assistant Director, and Deborah Ortega, Analyst-in-Charge, managed this assignment. Sarah Arnett, Marie Webb, and Su Jin Yon made significant contributions to the work. Christine Davis provided legal support. Stanley Kostyla and Jerome Sandau assisted with design, methodology, and data analysis, and Lara Kaskie provided assistance in report preparation.