Emergency Management: Preliminary Observations on FEMA's Community Preparedness Programs Related to the National Preparedness System
Released: 2009-10-01T00:00:00

Letter

Background

The Post-Katrina Emergency Management Reform Act of 2006 (Post- Katrina Act) required that FEMA establish the national preparedness system to ensure that the nation has the ability to prepare for and respond to disasters of all types, whether natural or man-made, including terrorist attacks. The Community Preparedness Division is responsible for leading activities related to community preparedness, including management of the Citizen Corps program. According to fiscal year 2008 Homeland Security Grant Guidance, the program is to bring together community and government leaders, including first responders, nonprofit organizations, and other community stakeholders. Serving as a Citizen Corps council, government and nongovernment stakeholders are to collaborate in involving community members in emergency preparedness, planning, mitigation, response, and recovery. Councils and partner programs register online to be included in the national program registries. The Division also supports the efforts of non-DHS federal “partner programs,” such as the Medical Reserve Corps, that promote preparedness and the use of volunteers to support first responders. The CERT program’s mission is to educate and train people in basic disaster preparedness and response skills, such as fire safety, light search and rescue, and disaster medical operations, using a nationally developed, standardized training curriculum. Trained individuals can be recruited to participate on neighborhood, business, or government teams to assist first responders. The mission of the Fire Corps program is to increase the capacity of fire and emergency medical service departments through the use of volunteers in nonoperational roles and activities, including administrative, public outreach, fire safety, and emergency preparedness education.

FEMA also is responsible for a related program, the Ready Campaign, which works in partnership with the Ad Council, an organization that creates public service messages, with the goals of raising public awareness regarding the need for emergency preparedness, motivating individuals to take steps toward preparedness, and ultimately increasing the level of national preparedness. The program makes preparedness information available to the public through its English and Spanish Web sites (www.ready.gov and www.listo.gov), through printed material that can be ordered from the program or via toll-free phone lines, and through public service announcements (PSA). The Ready Campaign message calls for individuals, families, and businesses to (1) get emergency supply kits, (2) make emergency plans, and (3) stay informed about emergencies and appropriate responses to those emergencies.

FEMA Faces Challenges Measuring Performance of Citizen Corps Programs and the Ready Campaign

FEMA faces challenges in measuring the performance of local community preparedness efforts because it lacks accurate information on those efforts. FEMA is also confronted with challenges in measuring performance for the Ready Campaign because the Ready Campaign is not positioned to control the placement of its preparedness messages or measure whether its message is changing the behavior of individuals.

FEMA Faces Challenges Measuring Performance of Community Preparedness Efforts Because It Lacks Accurate Information on Local Programs

According to FEMA officials, FEMA promotes citizen preparedness and volunteerism by encouraging collaboration and the creation of community Citizen Corps, CERT, and Fire Corps programs. FEMA includes the number of Citizen Corps councils, CERTs, and Fire Corps established across the country as its principal performance measure. However, FEMA faces challenges ensuring that the information needed to measure the number of established, active units is accurate. In our past work we reported on the importance of ensuring that program data are of sufficient quality to document performance and support decision making. Although not a measure under the Government Performance Result Act, FEMA programs report the number of local units registered as a principal performance measure; however, our work showed that the number of active units reported may differ from the number that actually exist. For example, as of September 2009: Citizen Corps reported having 2,409 registered Citizen Corps councils nationwide that encompass jurisdictions where approximately 79 percent of the U.S. population resides. However, 12 of the 17 registered councils we contacted during our site visits were active and 5 were not.

The CERT program reported having 3,354 registered CERTs. Of the 12 registered CERTs we visited, 11 were actively engaged in CERT activities, such as drills, exercises, and emergency preparedness outreach, or had been deployed to assist in an emergency or disaster situation, although 1 had members that had not been trained. One registered CERT was no longer active.

State officials in two of the four states also said that the data on number of registered programs might not be accurate. One state official responsible for the Citizen Corps council and CERT programs in the state estimated that as little as 20 percent of the registered councils were active, and the state subsequently removed more than half of its 40 councils from the national Web site. Officials in the other state said that the national database is not accurate and they have begun to send e-mails to or call local councils to verify the accuracy of registrations in their state. These officials said that they plan to follow up with those councils that do not respond, but they were not yet certain what they planned to do if the councils were no longer active. These results raise questions about the accuracy of FEMA’s data on the number of councils across the nation, and the accuracy of FEMA’s measure that registered councils cover 79 percent of the population nationwide.

Some change in the number of active local programs can be expected, based on factors including changes in government leadership, voluntary participation by civic leaders, and financial support. FEMA officials told us that the Homeland Security Grant Program guidance designates state officials as responsible for approving initial council and CERT registrations and ensure that the data are updated as needed. According to FEMA officials, however, in practice this may not occur. Community Preparedness Division officials said that they do not monitor whether states are regularly updating local unit registration information.

FEMA officials said that FEMA plans to adopt a new online registration process for Citizen Corps councils and CERTs in 2010, which will likely result in some programs being removed from FEMA’s registries. They said that FEMA expects to use the new registration process to collect more comprehensive data on membership and council activities. According to FEMA officials, updating initial registration information will continue to be the responsibility of state officials. The Citizen Corps Director noted that the Citizen Corps program does not have the ability to require all local units to update information, particularly councils or CERTS that receive no federal funding. According to the Fire Corps program Acting Director, a state advocacy program initiated in 2007 may help identify inactive programs as well as promote the Fire Corps program. As of September 2009, there were 53 advocates in 31 states. We will continue to assess this issue as part of our ongoing work.

The Ready Campaign Faces Challenges Measuring Performance Because It Is Not Positioned to Control the Distribution of Its Preparedness Messages and Measure Whether Its Message Effects Individual Behavior

Currently, the Ready Campaign measures its performance based on measures such as materials distributed or PSAs shown. For example, according to a DHS official, in fiscal year 2008, the Ready Campaign had more than 99 million “hits” on its Web site, more than 12 million pieces of Ready Campaign literature requested or 43,660 calls to the toll-free call numbers.

The Ready Campaign relies on these measures because it faces two different challenges determining whether its efforts are influencing individuals to be more prepared. First, the Ready Campaign is not positioned to control the when or where its preparedness message is viewed. Second, the Ready Campaign is not positioned to measure whether its message is changing the behavior of individuals.

With regard to the Ready Campaign’s ability to control the distribution of its message, our prior work has shown that agencies whose programs rely on others to deliver services face challenges in targeting and measuring results in meeting ultimate goals, and when this occurs, agencies can use intermediate measures to gauge program activities. However, according to FEMA’s Acting Director for the Ready Campaign, funds are not available for the Ready Campaign to purchase radio and television time to air its PSAs; rather, the Ready Campaign relies on donations of various sources of media. As a result, the Ready Campaign does not control what, when, or where Ready Campaign materials are placed when the media is donated. For example, what PSA is shown and the slots (e.g., a specific channel at a specific time) that are donated by television, radio, and other media companies are not under the Ready Campaign’s control, and these are not always prime viewing or listening spots. Based on Ad Council data, the Ready Campaign’s PSAs in 2008 were aired about 5 percent or less of the time by English and Spanish television stations during prime time (8:00 pm to 10:59 p.m.), and about 25 percent of the PSAs were aired from 1:00 a.m. to 4:00 a.m. Similarly, about 47 percent of English radio and about 27 percent of Spanish radio spots were aired from midnight to 6:00 a.m. FEMA officials said that with the release of its September 2009 PSAs, they expect increased placement during hours where there are more viewers and listeners.

Just as the Ready Campaign has no control over the time PSAs are aired, it does not control the type of media (e.g., radio and television) donated. Based on Ad Council data on the dollar value of media donated to show Ready Campaign materials (the value of the donated media is generally based on what it would cost the Ready Campaign if the media space were purchased), much of the value from donated media is based on space donated in the yellow pages. Figure 1 shows the value of various types of media donated to the Ready Campaign to distribute its message during 2008.

The Ready Campaign also faces a challenge determining the extent to which it contributes to individuals taking action to become more prepared—the program’s goal. Measuring the Ready Campaign’s progress toward its goal is problematic because it can be difficult to isolate the specific effect of exposure to Ready Campaign materials on an individual’s level of emergency preparedness. Research indicates that there may be a number of factors that are involved in an individual taking action to become prepared, such as his or her beliefs as to vulnerability to disaster, geographic location, or income. A basic question in establishing whether the Ready Campaign is changing behavior is, first, determining the extent to which the Ready Campaign’s message has been received by the general population. The Ad Council conducts an annual survey to determine public awareness of the Ready Campaign, among other things. For example, in the Ad Council’s 2008 survey: When asked if they had heard of a Web site called Ready.gov that provides information about steps to take to prepare in the event of a natural disaster or terrorist attack, 21 percent of those surveyed said that they were aware of the Ready.gov Web site.

When asked a similar question about television, radio, and print PSAs, 37 percent of those surveyed said that they have seen or heard at least one Ready Campaign PSA.

Another factor is isolating the Ready Campaign’s message from other preparedness messages that individuals might have received. The Ad Council’s 2008 survey found that 30 percent of those surveyed identified the American Red Cross as the primary source of emergency preparedness information; 11 percent identified the Ad Council.

While the Ad Council survey may give a general indication as to the population’s familiarity with the Ready Campaign, it does not provide a measure of preparedness actions taken based on the Ready Campaign’s promotion, that is, a clear link from the program to achieving program goals. The Ad Council reported that those who were aware of Ready Campaign’s advertising were significantly more likely to say that they had taken steps to prepare for disaster, but acknowledged that the Ready Campaign could not claim full credit for the differences. Further, as the 2009 Citizen Corps survey showed, the degree to which individuals are prepared may be less than indicated because preparedness drops substantially when more detailed questions about supplies are asked. We will continue to assess FEMA’s efforts to measure the performance of the Ready Campaign as part of our ongoing work.

FEMA Has Not Developed a Strategy Encompassing How Citizen Corps, Its Partner Programs, and the Ready Campaign Are to Operate within the Context of the National Preparedness System

While DHS’s and FEMA’s strategic plans have incorporated efforts to promote community preparedness, FEMA has not developed a strategy encompassing how Citizen Corps, its partner programs, and the Ready Campaign are to operate within the context of the national preparedness system. An objective in DHS’s Strategic Plan for 2008-2013 to “Ensure Preparedness” envisions empowering Americans to take individual and community actions before and after disasters strike. Similarly, FEMA’s Strategic Plan for 2008-2013 envisions a strategy to “Lead the Nation’s efforts for greater personal and community responsibility for preparedness through public education and awareness, and community engagement and planning, including outreach to vulnerable populations.” FEMA’s Strategic Plan delegates to the agency’s components the responsibility for developing their own strategic plans, which are to include goals, objectives, and strategies. FEMA’s Strategic Plan states that the components’ strategic plans are to focus on identifying outcomes and measuring performance.

NPD has not clearly articulated goals for FEMA’s community preparedness programs or a strategy to show how Citizen Corps, its partner programs, and the Ready Campaign are to achieve those goals within the context of the national preparedness system. In our past work, we reported that desirable characteristics of an effective national strategy include articulating the strategy’s purpose and goals; followed by subordinate objectives and specific activities to achieve results; and defining organizational roles, responsibilities, and coordination, including a discussion of resources needed to reach strategy goals. In April 2009, we reported that NPD had not developed a strategic plan that defines program roles and responsibilities, integration and coordination processes, and goals and performance measures for its programs. We reported that instead of a strategic plan, NPD officials stated that they used a draft annual operating plan and Post-Katrina Act provisions to guide NPD’s efforts. The draft operating plan identifies NPD goals and NPD subcomponents responsible for carrying out segments of the operating plan, including eight objectives identified for the Division under NPD’s goal to “enhance the preparedness of individuals, families, and special needs populations through awareness planning and training.” NPD’s objectives for meeting this goal do not describe desired outcomes.

For example, one of NPD’s objectives for the Community Preparedness Division is to increase “the number of functions that CERTs will be able to perform effectively during emergency response,” but the plan does not describe how many and what type of functions CERTs currently perform, what additional functions they could perform, and what it means to be effective. NPD’s draft operating plan also does not include other key elements of an effective national strategy, such as how it will measure progress in meeting its goals and objectives; the roles and responsibilities of those who will be implementing specific programs within the Community Preparedness Division, such as Citizen Corps or Fire Corps; or potential costs and types of resources and investments needed to meet goals and objectives needed to implement civilian preparedness programs. As a result, NPD is unable to provide a picture of priorities or how adjustments might be made in view of resource constraints.

In our April 2009 report we recommended that NPD take a more strategic approach to implementing the national preparedness system to include the development of a strategic plan that contains such key elements as goals, objectives, and how progress in achieving them will be measured. DHS concurred with our recommendation and, in commenting on our report, stated that it reported making progress in this area and is continuing to work to fully implement the recommendation. NPD officials stated in September 2009 that DHS, FEMA, and NPD, in coordination with national security staff, were discussing Homeland Security Presidential Directive 8 (National Preparedness), including the development of a preparedness strategy and an implementation strategy. They said that community and individual preparedness were key elements of those discussions. However, NPD officials did not state when the strategy will be completed; thus, it is not clear to what extent it will integrate Citizen Corps, its partner programs, and the Ready Campaign. NPD officials stated that work is under way on revising the target capabilities, which are to include specific outcomes, measures, and resources. NPD officials said that the draft for public comment is expected to be issued in fiscal year 2010.

The Ready Campaign is also working to enhance its strategic direction. According to the FEMA Director of External Affairs, the Ready Campaign’s strategy is being revised to reflect the transition of the program from DHS’s Office of Public Affairs to FEMA’s Office of External Affairs, and the new FEMA Director’s approach to preparedness. Program officials said that the Ready Campaign will have increased access to staff and resources and is to be guided by a FEMA-wide strategic plan for external communications. As of September 2009 the plan was still being developed and no date has been set for completion. We will continue to monitor this issue as well FEMA’s effort to develop a strategy encompassing how Citizen Corps and its partner programs are to operate within the context of the national preparedness system.

Mr. Chairman, this completes my prepared statement. I would be happy to respond to any questions you or other members of the subommittee may have.

Contacts and Acknowledgments

For further information about this testimony, please contact William O. Jenkins, Jr., Director, Homeland Security and Justice Issues, at (202) 512- 8777 or JenkinsWO@gao.gov. Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page of this testimony. Major contributors to this testimony included John Mortin, Assistant Director, and Monica Kelly, Analyst-in-Charge. Carla Brown, Qahira El’Amin, Lara Kaskie, Amanda Miller, Cristina Ruggiero- Mendoza, and Janet Temko made significant contributions to the work.

Appendix I: Homeland Security Grant Funding for Community Preparedness, 2004 through 2008

Department of Homeland Security support for local community preparedness activities is provided through homeland security grants, specifically the Citizen Corps grant program, but community preparedness activities are also eligible for support under other homeland security grants. Citizen Corps grants are awarded to states based on a formula of 0.75 percent of the total amount available to each state (including the District of Columbia and the Commonwealth of Puerto Rico) and 0.25 percent of the total amount available for each U.S. territory, with the balance of funding being distributed on a population basis.

For other DHS homeland security grants, a state prepares a request for funding, which can include support for the state’s community preparedness efforts, as allowed under the guidance for a particular grant. For example, the 2009 Homeland Security Grant Guidance lists “Conducting public education and outreach campaigns, including promoting individual, family and business emergency preparedness” as an allowable cost for state homeland security grants. Grant funding can be used to support Citizen Corps, Citizen Corps partner programs, or other state community preparedness priorities. The Federal Emergency Management Agency’s (FEMA) grant reporting database does not categorize grants in a way that allows identification of the amount of funding going to a particular community preparedness program.

Table 1 summarizes the approximately $269 million in DHS grants that were identified by grantees as supporting community preparedness projects from fiscal years 2004 through 2008. The amount is an approximation because of limitations in identifying grants for such projects. Our selection of projects for inclusion relied on grantees identifying their projects under one of three predefined project types that FEMA officials said are relevant for community preparedness or were projects funded with a Citizen Corps program grant. Not all grantees may have used these descriptions. We worked with grant officials to identify the most appropriate grant selection criteria.