Federal Emergency Management Agency: Status of Achieving Key Outcomes and Addressing Major Management Challenges
Released: 2001-08-08T00:00:00

Letter

Background

GPRA is intended to shift the focus of government decisionmaking, management, and accountability from activities and processes to the results and outcomes achieved by federal programs. New and valuable information on the plans, goals, and strategies of federal agencies has been provided since federal agencies began implementing GPRA. Under GPRA, annual performance plans are to clearly inform the Congress and the public of (1) the annual performance goals for agencies’ major programs and activities, (2) the measures that will be used to gauge performance, (3) the strategies and resources required to achieve the performance goals, and (4) the procedures that will be used to verify and validate performance information. These annual plans, issued soon after transmittal of the president’s budget, provide a direct linkage between an agency’s longer term goals and mission and day-to-day activities. Annual performance reports are to subsequently report on the degree to which performance goals were met. The issuance of the agencies’ performance reports, due by March 31, represents a new and potentially more substantive phase in the implementation of GPRA—the opportunity to assess federal agencies’ actual performance for the prior fiscal year and to consider what steps are needed to improve performance and reduce costs in the future.

FEMA is an independent agency whose mission is to reduce the loss of life and property and protect our institutions from natural and technological hazards by leading and supporting the nation in a comprehensive, risk- based emergency management program of mitigation, preparedness, response, and recovery. Traditionally, the role of the federal government has been to supplement the emergency management efforts of state and local governments, voluntary organizations, and private citizens. According to FEMA’s strategic plan, the nation’s emergency management capability is built on a partnership of local, state, and federal governments; voluntary agencies; business and industry; and individual citizens. FEMA’s focus is on building partnerships and mitigating the effects of disaster by assisting state, tribal, and local governments to prepare for, respond to, and recover from natural, manmade, and technological disasters.

Assessment of FEMA’s Progress and Strategies in Achieving Selected Key Outcomes

This section discusses our analysis of FEMA’s performance in achieving its selected key outcomes and the strategies it has in place, particularly strategic human capital management and information technology, for accomplishing these outcomes. In discussing these outcomes, we have also provided information drawn from our prior work on the extent to which the agency provided assurance that the performance information it is reporting is credible.

Minimize Human Suffering and Property Losses After Natural Disasters

According to its fiscal year 2000 annual performance report, FEMA has made some progress in minimizing human suffering and property losses after natural disasters. FEMA has 10 performance goals related to this key outcome. It reported that it met three of this outcome’s goals, including efforts to process disaster declaration requests within an average of 8.3 days and achieved an 85-percent customer satisfaction rating for elements of its Public Assistance Program. We could not determine progress for two other goals because FEMA did not provide data for all measures associated with these goals. For example, FEMA did not provide data on reducing costs for disseminating FEMA documents and public announcements making it difficult to assess FEMA’s progress toward its goal of providing reliable data and communications services to disaster locations. FEMA also reported that it did not fully meet five goals related to this outcome but generally provided explanations for why goals were not achieved. For instance, FEMA reported that its growth rate for National Flood Insurance Program (NFIP) policies fell short of its 5 percent goal but discusses a number of factors that affected overall policy growth in fiscal year 2000.

FEMA’s fiscal year 2000 performance report was similar to its fiscal year 1999 performance report in that it did little to identify significant limitations that potentially affect the credibility of data used to measure its performance. For example, FEMA has a goal associated with this outcome to enhance recovery by expediting disaster operations through the National Emergency Management Information System. FEMA does not acknowledge reported limitations associated with this system. In a report to be issued in July 2001, we note that FEMA had identified weaknesses in the system, such as untimely data, as reported in FEMA after-action reports.

The fiscal year 2000 performance report presented more descriptive details for each of the annual performance goals than the fiscal year 1999 performance report, including more explanation on the goals’ background and why the goals are important. In some cases, FEMA included information on the benefits derived from achieving goals. For example, one of the Response and Recovery goals is to enhance a community’s disaster recovery process by improving administrative processes and training. FEMA stated in its fiscal year 2000 performance report that achieving this goal means that customers are satisfied with the overall Public Assistance Program and are being served in a timely manner by responsive, competent, and accountable staff. In some cases, FEMA also identified special conditions that affected the agency’s ability to meet the goals. For example, one of the measures used to determine progress toward the goal discussed in the previous example did not have complete data on fiscal year 2000. As a result, FEMA reported on data for the first half of the fiscal year and provided an explanation of why the data were not available for the entire year. FEMA also indicated that incomplete data would be a continuing problem because of the lag time between collecting and analyzing data on this goal.

FEMA’s fiscal year 2000 performance report also used more data to illustrate progress toward achieving goals. For example, FEMA has a goal to increase the effective delivery of response services by ensuring immediate response to a governor’s request for a presidential declaration. One measure is that FEMA will process a governor’s request within an average of 8.3 days. Figure 1 illustrates FEMA’s progress toward achieving its processing goal for fiscal years 1999 and 2000.

In its fiscal year 2002 performance plan, FEMA outlines several strategies to improve disaster response operations such as enhancing interagency coordination through a number of groups, including the Catastrophic Disaster Response Group, Emergency Support Function Leaders, and the Regional Interagency Committee and establishing improved disaster declaration criteria. FEMA’s plan also discusses specific actions for coordinating with other agencies to accomplish this goal. In the area of information technology, FEMA’s fiscal year 2002 performance plan states that it plans to reduce the resources needed and increase the speed for processing disaster assistance within the National Emergency Management Information System. However, FEMA mentions no specifics on how it plans to achieve this goal. In the area of strategic human capital management, FEMA presents a strategy to increase senior management effectiveness at disaster field offices for three goals, but provides no specifics on how it plans to pursue this strategy.

Provide Timely Responses to Disaster Aid Requests

Although FEMA reported accomplishing most performance measures under this outcome, it reported that it did not fully meet any goal under this outcome. This outcome contained three goals with numerous performance measures. For example, FEMA’s goal to increase the level of internal and external customer satisfaction with FEMA services contains nine performance measures. FEMA reported that four of nine measures were accomplished, two were not accomplished, and three were discontinued and not reported. Although FEMA provided a rationale for why some of these goals were not fully achieved, it did not provide a detailed strategy on how it plans to achieve these unmet goals in the future. For example, FEMA explained it did not fully achieve its goal of referring 100 percent of eligible delinquent debts to the Department of the Treasury. However, the only discussion provided on needed changes was that FEMA must develop a policy on the collection of delinquent debts, publish rules and regulations in the Federal Register before collecting such debts, and transfer remaining delinquent debts to the Treasury.

FEMA’s fiscal year 2002 performance plan has two performance goals under this outcome that represent goals for several FEMA offices. FEMA acknowledges that these goals are iterations of goals found in the other two outcomes—minimizing human suffering and mitigation efforts. Although the goals themselves appear to be clear and reasonable, they could result in confusion over who is responsible for achieving these goals. For example, the Response and Recovery Directorate has a performance goal in this outcome to determine cost drivers in the response and recovery processes and implement re-engineered processes to support improvements in FEMA cost efficiency. Under the outcome to minimize human suffering and property losses after natural disasters, FEMA has performance measures to improve disaster processing by 5 percent and reduce the total dollar value of assets at closed disaster field offices by 10 percent.

In the area of strategic human capital management, FEMA outlines the skills base it needs to achieve goals under this outcome and has one goal for the Human Resources Management Office to provide timely and effective human resource tools, services, information, and assistance to FEMA organizations. In the area of information technology, FEMA’s fiscal year 2002 performance plan has measures to expand public access to information through e-government services and deliver accessible and standardized information technology services at 98 percent availability with no undetected virus attacks. Expanding e-government services is a new administration initiative, which FEMA plans to use to provide information to the public in a more timely and efficient manner.

Prevent or Reduce Harm and Losses From Future Disasters Through Mitigation Efforts

FEMA’s fiscal year 2000 performance report showed that the agency was making progress in preventing or reducing harm and losses from future disasters through mitigation efforts. The agency reported meeting the majority of its goals for this key outcome, which included results such as entering into formal agreements with 11 agencies to support mitigation designating 63 communities as Project Impact communities,implementing building standards that increased the use and effectiveness of mitigation tools, and completing seven hurricane evacuation studies.

Most of the performance goals for this key outcome have at least one measurable or quantifiable performance measure that helps demonstrate progress toward reaching the goals. For example, FEMA reported that the Federal Insurance Administration (FIA) successfully refined and remeasured the savings achieved from flood loss reduction. FEMA estimated flood loss reduction amounted to just over $1 billion in fiscal year 2000. It also reported that tests of the National Warning System showed emergency alerts were disseminated within 2 minutes—exceeding FEMA’s goal of 3-minute dissemination.

In addition, as we pointed out in our May 2001 testimony, opportunities are emerging to better measure the success of the NFIP in protecting lives and preventing loss of property. FEMA’s FIA has a number of performance goals aimed at improving the results of the NFIP, including increasing the number of insurance policies sold. FEMA did not meet the 5 percent annual increase goal for policies sold in fiscal years 1999 and 2000. However, it decided not to change the goal, according to the acting FIA Administrator, because FIA felt it was an effective workforce driver. Although this goal may serve a workforce purpose and provide insight on program volume, it does not measure the degree of participation by the most vulnerable residents—those living in flood-prone areas. Capturing data on the number of both uninsured and insured structures in flood- prone areas could provide FEMA with another indication of how effectively the program is penetrating into those areas most at risk of flooding, whether the financial consequences of floods in these areas are increasing or decreasing, and where to better target marketing efforts.

Our May 2001 testimony also pointed out that opportunities are developing for FEMA to obtain valuable information about the program’s success through analysis of the rate of participation for those communities involved in the program. The participation rate is obtained by dividing the number of properties located in special flood hazard areas (SFHA) with flood insurance by the total number of properties in these SFHAs. FIA maintains a database on the number of flood insurance policies in force, including the number in SFHAs. However, the data FEMA has on the national and local community levels for the number of structures in SFHAs are of varying quality, according to FIA’s Acting Administrator. Even so, several current mapping technologies can be used to facilitate the collection of data on the number of structures in the SFHAs. Local communities, such as Dekalb County, GA and Charlotte, NC, are using mapping technologies to estimate the number of structures in SFHAs. Combining these technologies with the digital flood maps that FEMA is already producing would increase accuracy in the identification of structures within SFHAs and the calculation of participation rates. FIA officials agree that program participation rates are a useful measure that can provide insights for measuring the program’s success, including the effectiveness of marketing.

However, it is still difficult to determine the degree of progress FEMA has made in achieving some of it goals. This is because FEMA did not clearly state whether some goals were fully achieved. For example, FEMA cited that it had achieved “substantial advancement” in achieving its efforts to evaluate the effectiveness of the mitigation planning. It would be more helpful if FEMA clearly stated whether or not the goal had been met or not met as required under GPRA, then discuss the degree of achievement FEMA made under this goal.

In addition, FEMA did not report the results achieved for some of the performance measures. For example, FEMA’s goal to increase public awareness of fire hazards has two performance measures: (1) increase the usage of public education materials by 4 percent and (2) increase the number of hotels and motels with automatic sprinkler and smoke detection systems by 20 percent. FEMA provided performance data for the first measure but did not provide any performance data or information on the second measure. Also, FEMA still needs to provide better information on significant limitations to the data used to measure performance. Although the Director of FEMA acknowledges there are limitations with some of FEMA’s data in the fiscal year 2000 performance report, the report did not always discuss where data limitations exist and how they affect the accuracy, completeness, and availability of performance measurement data under this outcome.

FEMA’s fiscal year 2002 performance plan shows that the agency has made some improvements in its strategies by streamlining a number of its mitigation goals into one outcome-oriented goal that will support the development of disaster resistance in a number of communities. FEMA reports that this consolidation is in recognition that several fiscal year 2000 goals were better identified as means and strategies to the revised goal. FEMA’s strategies for achieving progress towards this key outcome include conducting post-disaster economic impact studies and coordinating with other federal departments and agencies via Memorandums of Understanding to identify ways existing programs and new initiatives can support national mitigation goals.

In the area of strategic human capital management, FEMA discusses the skills needed to achieve performance goals under this outcome in its fiscal year 2002 performance plan. In the area of information technology, FEMA plans to evaluate and apply emerging technologies that enable more cost- effective modeling and mapping.

Comparison of FEMA’s Fiscal Year 2000 Performance Report and Fiscal Year 2002 Performance Plan With the Prior Year Report and Plan for Selected Key Outcomes

For the selected key outcomes, this section describes major improvements or remaining weaknesses in FEMA’s (1) fiscal year 2000 performance report in comparison with its fiscal year 1999 report and (2) fiscal year 2002 performance plan in comparison with its fiscal year 2001 plan. It also discusses the degree to which the agency’s fiscal year 2000 report and fiscal year 2002 plan address concerns and recommendations made by the Congress, GAO, FEMA’s Inspector General, us and others.

Comparison of Performance Reports for Fiscal Years 1999 and 2000

FEMA made a number of improvements in its fiscal year 2000 performance report, such as using more trend data and providing explanations about variations in its performance goals but omitted a key piece of information that highlighted the degree of progress it made towards achieving its performance goals. However, unlike FEMA’s fiscal year 1999 performance report which clearly stated the extent to which goals were met, FEMA’s fiscal year 2000 performance report did not include a quantitative assessment of the progress the agency made in achieving each goal, which makes it difficult, in some cases, to determine whether the agency achieved its goal.

FEMA’s fiscal year 2000 performance report used more data to illustrate progress towards achieving goals. The report made frequent use of graphics to illustrate trends—which is helpful to the reader. For example, FEMA has an annual performance goal to provide a safe and secure environment for FEMA and its emergency management partners at disaster facilities. One measure is to provide trained safety and security staff for 80 percent of major declared disasters. FEMA illustrates its achievement with a graph of security staff deployments to major disasters for fiscal years 1998 through 2000.

FEMA’s fiscal year 2000 performance report also showed improvement in providing explanations for its inability to meet some of its performance goals and measures. For example, FEMA reported that it did not meet its goal to increase the number of policies in the NFIP but cited several factors, such as lack of flood activity and drought conditions across many parts of the United States, that affected overall policy growth. In addition, when applicable, FEMA identified whether its goal changed in the next fiscal year.

In some cases, it is difficult to determine the degree of progress FEMA made towards achieving goals under its three outcomes because FEMA did not provide a quantitative assessment of progress. For example, the goal on evaluating the effectiveness of mitigation planning processes and related initiatives does not explicitly state whether the goal was met. The narrative under “Achievement” states fiscal year 2000 efforts have resulted in substantial advancement in mitigation planning. The text goes on to discuss a variety of mitigation planning activities but does not clearly state whether the goal has been achieved. Both GPRA and the Office of Management and Budget (OMB) Circular A-11’s guidance for developing performance plans and reports require that agencies provide an indication of whether a goal is or is not met.

Comparison of Performance Plans for Fiscal Years 2001 and 2002

FEMA made some improvements to its fiscal year 2002 performance plan as compared to its fiscal year 2001 performance plan. FEMA continued to streamline its performance goals and make them more outcome-oriented. FEMA also provided more descriptive details on the background of the goals and means and strategies for achieving them; more graphics and trend data on annual outcomes; and added goals to reflect FEMA’s commitment to the administration’s reform goals. FEMA’s fiscal year 2002 performance plan also shows improvement in providing details about the procedures for verification and validation for most of the performance goals. However, the agency did not generally provide information on significant limitations that may potentially affect the credibility of the data used to measure performance.

As mentioned previously, FEMA continued its streamlining by consolidating a number of goals. As a result, FEMA has 19 performance goals in its fiscal year 2002 performance plan compared with 30 in its fiscal year 2001 performance plan. The fiscal year 2002 performance plan’s goals have some measures that were performance goals in the fiscal year 2001 plan. For example, FEMA consolidated a number of mitigation-related goals to form a more outcome-oriented goal to support the development of disaster resistance in communities and states.

FEMA’s fiscal year 2002 performance plan also has more descriptive details for each of the outcomes. For example, FEMA outlines a number of mitigation tools, education and outreach activities, and partnership agreements it will pursue to support the development of disaster resistance in communities and states. FEMA plans to coordinate with other federal departments and agencies to identify ways in which their existing programs and new initiatives can support national mitigation goals. FEMA also plans to discuss specific actions for coordinating with other agencies to accomplish these goals. FEMA makes use of trend data, for example, to illustrate the costs FEMA avoids when purchasing new equipment by recycling previously used equipment in support of its goals to operate a logistics program that provides timely and cost effective resources in support of emergency management missions.

FEMA added six goals to its fiscal year 2002 performance plan in support of its commitment to the administration’s reform goals issued in February 2001 through OMB. Five of the six reform goals are part of the outcome to provide timely responses to disaster aid requests. Of these five goals, one has elements of strategic human capital management—determine management levels for streamlining purposes. The remaining four goals include reducing erroneous payments, expanding on-line procurement, and making greater use of performance-based contracts. The sixth goal— related to minimizing human suffering and property losses after natural disasters—calls for the development of a disaster declaration process that better defines federal and state responsibilities for providing disaster assistance. Three of the six goals have quantitative measures to measure success, but FEMA does not provide any details on its strategies for achieving the goals.

FEMA has addressed weaknesses that we identified in the 1999 plan by providing additional descriptions of the procedures for verification and validation for most performance goals. For example, FEMA plans to use additional verification and validation strategies to measure progress in supporting the reduction of loss of life from fire-related incidents. FEMA’s fiscal year 2002 performance plan states it will use reports from several sources, including the National Fire Data Center, the National Center for Health Statistics, and the Consumer Product Safety Commission to compare data on fire-related deaths, injuries, and losses.

However, FEMA still needs to do a better job of identifying limitations that potentially affect the credibility of the data used to measure performance and identifying external factors that may affect performance data. For example, FEMA has a goal to improve response operations for which one measure requires FEMA to act on requests for water, food, and shelter within 12 hours after a presidential disaster declaration. FEMA reports the goal’s intent is to coordinate through partnerships with other federal agencies, states, and local governments and private and voluntary organizations for initial provision of these basic needs. The difficulties inherent in working with so many other agencies and organizations are not acknowledged, although they may have an effect on FEMA’s ability to meet this performance goal.

FEMA’s Efforts to Address Its Major Management Challenges Identified by GAO

We have identified two governmentwide high-risk areas: strategic human capital management and information security. We found that FEMA’s performance plan had goals and measures related to human capital, but the agency’s performance report did not explain its progress in resolving human capital challenges except that progress had been made in improving Disaster Field Office operations. We also found that FEMA’s fiscal year 2000 performance report discusses information security activities such as the interception of major viruses and strengthening its Internet firewall policies, but the fiscal year 2002 performance plan had no goals and measures directly related to information security.

While FEMA’s fiscal year 2002 performance plan has no goals and measures directly related to information security, it does have a performance measure indirectly related to information technology that is associated with its customer satisfaction goal. FEMA has a two-part performance measure that calls for (1) delivering accessible and standardized information technology services at 98-percent availability with no undetected virus attacks and (2) resolving 80 percent of help desk trouble issues on the first call. In its fiscal year 2000 performance report, FEMA reported it had established a Critical Infrastructure Assurance Officer position and an Information Assurance Branch and was taking actions to protect and strengthen its Intranet/Internet assets. FEMA also reported that it had intercepted major viruses, conducted network scan and security audits, and implemented Intranet security measures. However, as part of an audit of FEMA’s fiscal year 2000 financial statements, an independent auditor reported a material weakness in computer-based controls over FEMA’s automated financial information systems.

In addition, we identified three other major management challenges facing FEMA. In our January 2001 Presidential and Congressional Transition effort, we identified three management challenges—determining the cost effectiveness of mitigation efforts, reducing the cost of disaster assistance, and improving the financial condition of the NFIP. For these three major management challenges, FEMA’s performance plan had 19 goals or measures that were directly related to these challenges. Appendix I provides detailed information on how FEMA addressed these challenges, high-risk areas that we identified, and the challenges identified by FEMA’s Office of Inspector General.

Scope and Methodology

As agreed, our evaluation was generally based on the requirements of GPRA; the Reports Consolidation Act of 2000; guidance to agencies from OMB for developing performance plans and reports (OMB Circular A-11, part 2); previous reports and evaluations by us and others; our knowledge of FEMA’s operations and programs; our identification of best practices concerning performance planning and reporting; and our observations of FEMA’s other GPRA-related efforts. We also discussed our review with agency officials and with FEMA’s Office of Inspector General. The agency outcomes that were used as the basis for our review were identified by the Ranking Minority Member of the Senate Governmental Affairs Committee as important mission areas for the agency and generally reflect the outcomes for all of FEMA’s programs or activities. The major management challenges confronting FEMA, including the governmentwide high-risk areas of strategic human capital management and information security, were identified by us in our January 2001 Performance and Accountability Series and High-Risk Update or in our January 2001 Presidential and Congressional Transition effort. The management challenges identified by FEMA’s OIG were included in a December 2000 letter from the OIG to the Director. We did not independently verify the information contained in the performance report and plan, although we did draw from some of our other work in assessing the validity, reliability, and timeliness of FEMA’s performance data. We conducted our review from April 2001 through June 2001 in accordance with generally accepted government auditing standards.

Agency Comments

We provided a draft of this report to FEMA for its review and comment. FEMA chose to meet with us to provide oral comments on the draft report, and we met with FEMA’s GPRA Manager of the Administration and Resource Planning Directorate on June 28, 2001, to discuss these comments. While the FEMA official agreed with the report, she provided suggested changes that we have included in this final report. The suggested changes that we incorporated in this report include (1) adding clarifying language that FEMA did not “fully meet” five goals related to the outcome to minimize human suffering and property losses to recognize the progress FEMA had made, (2) adding five goals outlined in FEMA’s fiscal year 2002 performance plan related to the major management challenge to support terrorism preparedness coordination, and (3) adding additional information on progress made to resolve the major management challenge to improve the financial condition of the NFIP.

FEMA also asked us to identify its key outcomes with FEMA’s own numbering system in its performance plan in addition to the narrative description we provided to increase clarity in the discussion of agency outcomes. We did not include this change because it does not improve the clarity of the report.

As arranged with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after the date of this letter. At that time, we will send copies to appropriate congressional committees; the Director, Federal Emergency Management Agency; and the Director, Office of Management and Budget. Copies will also be made available to others upon request.

If you or your staff have any questions, please call me at (202) 512-8984. Key contributors to this report were Mark Abraham, Julia Duquette, David Gill, Signora May, Robert Procaccini, and Carrie Watkins.

Appendix I: Observations on FEMA’s Efforts to Address Its Major Management Challenges

The following table identifies the major management challenges confronting the Federal Emergency Management Agency (FEMA), including the governmentwide high-risk areas of strategic human capital management and information security. The first column lists the challenges identified by our office and/or FEMA’s Office of Inspector General (OIG). The second column discusses the progress that has been made in resolving the challenges as discussed in FEMA’s fiscal year 2000 performance report. The third column discusses the extent to which FEMA’s fiscal year 2002 performance plan includes performance goals and measures to address the challenges that we and/or FEMA’s OIG identified.

FEMA’s fiscal year 2000 performance report generally discussed the agency’s progress in resolving its challenges. Of the agency’s 9 major management challenges, its performance plan had (1) 38 goals or measures that were directly related to 7 challenges; (2) 3 goals or measures that were indirectly applicable to 2 challenges—one of these challenges also has goals or measures directly related to it; and (3) no goals or measures related to 1 challenge—financial management systems—but discussed strategies to address this challenge.