Department of Homeland Security
 

FEMA’s Management of Corrective Actions and 

Lessons Learned From National Level Exercises 

OIG-12-118 September 2012
 
OFFICE OF INSPECTOR GENERAL

Department of Homeland Security

Washington, DC 20528 / www.oig.dhs.gov

SEP 11 2012

MEMORANDUM FOR: Timothy W. Manning

Deputy Administrator

National Preparedness Directorate

Federal Emergency Management Agency

FROM: Ann~~

V-c.fvi~ Assistant Inspector General for Audits

SUBJECT: FEMA's Management of Corrective Actions and Lessons

Learned From National-Level Exercises

Attached for your action is our final report, FEMA's Management of Corrective Actions

and Lessons Learned From National-Level Exercises. We incorporated the formal

comments from the Office of Policy and Program Analysis, Federal Emergency

Management Agency in the final report.

The report contains three recommendations aimed at improving the National,

Preparedness Directorate. Your office concurred with all recommendations. Based on

information provided in your response to the draft report, we consider the

recommendations resolved. Once your office has fully implemented the

recommendations, please submit a formal closeout letter to us within 30 days so that

we may close the recommendations. The memorandum should be accompanied by

evidence of completion of agreed-upon corrective actions and of the disposition of any

monetary amounts.

Consistent with our responsibility under the Inspector General Act, we are providing

copies of our report to appropriate congressional committees with oversight and

appropriation responsibility over the Department of Homeland Security. We will post

the report on our website for public dissemination.

Please call me with any questions, or your staff may contact Mark Bell, Deputy Assistant

Inspector General for Audits, at (202) 254-4100.

Attachment
 

 

 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Table of Contents 

Executive Summary ............................................................................................................ 1 

Background ........................................................................................................................ 2 

Results of Review ................................................................................................................ 4 

Tracking, Managing, and Completing Corrective Actions ...................................... 5 

Validating Corrective Actions .................................................................................. 6 

Managing Lessons Learned ..................................................................................... 7 

Recommendations ................................................................................................. 9 

Management Comments and OIG Analysis ........................................................... 9 

Appendixes 

Appendix A: Objectives, Scope, and Methodology ............................................ 11 

Appendix B: Management Comments to the Draft Report ............................... 12 

Appendix C: Major Contributors to This Report ................................................ 16 

Appendix D: Report Distribution ........................................................................ 17 

Abbreviations 

CAP Corrective Action Program 

DHS Department of Homeland Security 

EEP Exercise and Evaluation Program 

FEMA Federal Emergency Management Agency 

LLIS Lessons Learned Information Sharing System 

NED National Exercise Division 

NLE National-Level Exercise 

NPD National Preparedness Directorate 

OIG Office of Inspector General 

RAMP Remedial Action Management Program 

TOPOFF Top Officials 

www.oig.dhs.gov OIG-12-118 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Executive Summary

Every 2 years, the Federal Emergency Management Agency (FEMA) carries out a 

National-Level Exercise aimed at assessing U.S. capacity to prevent, prepare for, respond 

to, and recover from a large-scale disaster. FEMA is responsible for managing, tracking, 

and validating corrective actions stemming from this exercise, as well as other exercises 

and actual events, and widely disseminating lessons learned. Our objective was to 

determine whether FEMA is managing and tracking corrective actions and lessons 

learned from National-Level Exercises, as well as from other exercises and disasters. 

FEMA did not consistently track and manage corrective actions assigned to it resulting 

from exercises carried out in 2007 and 2009, nor did it finalize and implement guidance 

for doing so. Furthermore, fewer than 40 percent of corrective actions resulting from 

these exercises, many of which FEMA was responsible for completing, were completed. 

FEMA also did not adequately validate corrective actions to improve planning and 

disaster response. Finally, FEMA did not sufficiently disseminate information on agencyspecific lessons learned. As a result, FEMA missed opportunities to validate its actions in 

future exercises and actual events, learn from and apply the experiences of its 

personnel, and improve its incident management operations. 

We are making three recommendations that, when implemented, should improve 

FEMA’s efforts to track, manage, and validate corrective actions, as well as better 

disseminate lessons learned. FEMA concurred with our recommendations and is taking 

steps to implement them. 

www.oig.dhs.gov 1 OIG-12-118 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Background 

National-Level Exercises

After the terrorist attacks of September 11, 2001, Congress passed the Homelandf

SecurityfAct off2002f(Public Law 107-296), as amended, which formalized what is now 

known as the National-Level Exercise (NLE) program. An amendment to that act, found 

in United States Code Title 6, then established the requirement for a continuous 

exercise cycle, which culminates in a biennial national full-scale exercise involving 

Federal, State, and local partners. An NLE aims to assess U.S. capacity to prevent, 

prepare for, respond to, and recover from a large-scale disaster. 

The NLE 2011 full-scale exercise, held from May 16 to 19, tested the response and 

recovery efforts for a simulated 7.7 magnitude earthquake in the New Madrid Seismic 

Zone and an additional 6.0 magnitude earthquake in the Wabash Valley Seismic Zone. 

The simulated catastrophic disaster affected four FEMA regions (IV, V, VI, and VII) and 

eight States (Alabama, Arkansas, Illinois, Indiana, Kentucky, Mississippi, Missouri, and 

Tennessee) (see figure 1). More than 10,000 Federal, State, local, international, 

nongovernmental, and private sector individuals participated in the exercise at about 

135 sites. The purpose of the exercise was to prepare and coordinate a multijurisdictional

integrated response to a catastrophic earthquake, with an emphasis on life-saving and 

life-sustaining missions in the initial 72 hours. 

Figure 1. Map of New Madrid Seismic Zone 

Source: FEMA. 

www.oig.dhs.gov 2 OIG-12-118 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

FEMA’s Roles and Responsibilities

FEMA coordinates the Federal Government’s role in all domestic disaster-related 

activities. In April 2007, FEMA established the National Preparedness Directorate (NPD) 

to oversee the coordination and development of strategies to prepare for both 

manmade and natural disasters. The National Preparedness Assessment Division of NPD 

manages the Remedial Action Management Program (RAMP), Corrective Action 

Program (CAP), and the Lessons Learned Information Sharing System (LLIS), and 

develops and tracks performance measures for preparedness programs. RAMP and CAP 

provide a standard method for handling corrective actions and Improvement Plans. LLIS 

is a Web-based national network of lessons learned and best practices for emergency 

response providers and officials involved in homeland security. The National Exercise 

Division (NED) of NPD provides exercise guidance and planning support to Federal-, 

State-, and local-level emergency preparedness personnel. 

NED established the National Exercise Program Base Plan to coordinate the efforts of 

other Federal, State, and local exercise programs. The base plan systematically

evaluates exercises with a focus on developing corrective actions, which are steps that 

need to be taken to close gaps in preparedness and correct shortcomings found during 

exercises and actual events. The base plan is also aimed at rapidly identifying, 

developing, and disseminating lessons learned. It requires all Federal departments and 

agencies to actively participate in the corrective action process and maintain corrective 

action programs. 

Under the base plan, NED ensures that, following exercises, After-Action Reports, 

Improvement Plans, and corrective actions are generated in a timely manner. AfterAction Reports summarize exercise events, analyze performance, and evaluate 

achievement of exercise objectives. Improvement Plans include corrective actions to 

enhance performance, timelines for implementing corrective actions, and identification 

of the agencies responsible for completing the corrective actions. The base plan also 

requires NED to manage the CAP system and report on the status of corrective actions 

to the National Security Staff in the White House. NED also manages the Homeland 

Security Exercise and Evaluation Program to provide common policy and program 

guidance for a national standard for exercises, as well as tools for managers to plan, 

conduct, and evaluate exercises to improve overall preparedness. 

Corrective Actions and Lessons Learned

Corrective actions are intended to close preparedness gaps and correct shortcomings 

experienced in exercises and actual disasters. A lesson learned is a positive finding or a 

validated corrective action implemented during an event or derived from analysis that 

improves performance or increases capabilities. A lesson learned can also be a negative 

www.oig.dhs.gov 3 OIG-12-118 
 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

finding that may need correction. Corrective actions and lessons learned can improve 

mission performance and future disaster management operations. 

FEMA first used RAMP to track internal issues, lessons learned, and best practices 

identified during actual disasters. CAP was developed to track, manage, and 

disseminate corrective actions and Improvement Plans. The agency and its Federal 

partners used CAP to address preparedness deficiencies they identified during exercises, 

policy discussions, and actual disasters. In February 2009, FEMA began integrating the 

two programs, and in July 2010, the integrated program, now known as CAP, became 

operational.1

 As part of its responsibility to oversee CAP, the National Preparedness 

Assessment Division provides administrative and technical support for corrective action 

processes at other Federal departments and agencies. 

In its “hotwash” process, FEMA identifies issues and lessons learned through facilitatorled discussion groups of key leaders and staff. The agency then uses LLIS to disseminate 

information and lessons learned from exercises and actual events internally and to the 

public. LLIS also serves as the national repository for After-Action Reports. 

According to FEMA Exercise and Evaluation Program (EEP) Management Directive 123-

15, issued in January 2009, all directorates, offices, and regional offices involved in 

exercises must provide senior-level leadership to the FEMA EEP Steering Committee.

The EEP Steering Committee coordinates FEMA activities related to exercises, including 

corrective action recommendations, and manages and tracks FEMA-specific corrective 

actions. 

Results of Review 

FEMA did not adequately manage and track corrective actions assigned to it resulting 

from disaster management exercises carried out in 2007 and 2009, and fewer than 

40 percent of all these corrective actions are completed. Furthermore, FEMA did not 

validate corrective actions to ensure that they could be used to improve future mission 

performance. Finally, FEMA did not effectively use its mechanism for disseminating 

agency-specific lessons learned to personnel. These weaknesses resulted in missed 

opportunities to validate its actions and apply lessons learned to improve future 

exercises, as well as improve planning and responses to actual disasters. 

1

 In January 2011, OIG released the report FEMA’sfImplementationfoffthefRemedialfActionfManagementf

Programf(OIG-11-32), outlining several issues with the program. 

www.oig.dhs.gov 4 OIG-12-118 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Tracking, Managing, and Completing Corrective Actions 

FEMA did not consistently track, manage, and complete the corrective actions 

assigned to it resulting from exercises carried out in 2007 and 2009. Insufficient 

oversight by the EEP Steering Committee made it more difficult to manage, 

track, and prioritize completion of corrective actions. The agency’s failure to 

complete corrective actions may also be attributed to the extended period 

allowed for completion. 

The National Exercise Program Base Plan requires the development of a 

corrective action process to ensure that lessons learned from exercises are 

either continued or improved upon as appropriate. The base plan also requires 

all departments and agencies to maintain a corrective action program that can 

generate data for, and track assignments from, the CAP system. At FEMA, 

immediately following exercises, NED and other exercise participants are to 

develop After-Action Reports that summarize exercise events, analyze 

performance, and evaluate achievement of objectives. Improvement Plans are 

also to be developed and should include corrective actions, timelines for 

implementing them, and the agencies responsible for their completion. 

Improvement Plans are entered into the CAP system, where FEMA’s National 

Preparedness Assessment Division is to ensure that they are tracked until 

completion. 

The EEP Steering Committee, which was established in January 2009 through 

Management Directive 123-15, did not meet to coordinate exercise activities and 

make corrective action recommendations. The Steering Committee is responsible

for managing and tracking FEMA’s corrective actions within CAP, RAMP, and LLIS. 

It is to include senior leadership from all FEMA directorates and offices involved 

in disaster exercises, and be chaired by the National Integration Center. When 

Management Directive 123-15 was issued, NED was part of the National 

Integration Center, which is no longer the case. However, the directive was not 

updated, and there was not a clear understanding of which office was responsible

for chairing the committee. According to our interviews, no one in FEMA called 

a meeting in several months. The lack of oversight by the EEP Steering 

Committee made managing and tracking corrective actions more difficult and 

completing them less of a priority. If the committee met regularly, it would be 

better able to recommend which offices should resolve issues related to 

corrective actions and prioritize completion of these actions. 

Fewer than 40 percent of the corrective actions identified from the Top Officials 

(TOPOFF) 4 exercise in 2007 and the NLE 2009 had been completed as of 

October 25, 2011. At the time of our review, 57 percent of corrective actions to 

www.oig.dhs.gov 5 OIG-12-118 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

be completed by DHS (including FEMA) from these two exercises had not met 

their targeted completion dates. FEMA was not primarily responsible for 

completing all corrective actions included in the Improvement Plans from these 

exercises, but many for which the agency was responsible were not completed. 

For example, of the 11 FEMA-specific corrective actions from the 2007 TOPOFF 4 

exercise, 8 were open and overdue. These corrective actions were to be 

completed by January 2011—more than 3 years after the exercise. One of the 

TOPOFF 4 actions yet to be completed by FEMA states that more governmentwide planning is needed to develop a process to determine where to send 

resources and supplies in case there are multiple disaster sites. FEMA is also 

responsible for two corrective actions from the NLE 2009, both of which were to 

be completed in May 2011 and are now overdue. One of these corrective 

actions states that pre-exercise seminars are needed to discuss the roles and 

responsibilities of executive branch organizations dealing with crisis situations to 

improve the decisionmaking process. According to FEMA, the time allotted to 

complete the FEMA-specific corrective actions is determined by each designated 

office and depends on the complexity of the improvement plan; there is no 

centralized office that determines the timeframe for completing corrective 

actions. 

Validating Corrective Actions

FEMA did not consistently and adequately validate all completed corrective 

actions for which it was responsible. Validation was hampered by the need for 

finalized guidance for the corrective action process. 

Guidance for the Homeland Security Exercise and Evaluation Program includes 

an improvement planning process to identify and resolve corrective actions from 

After-Action Reports. Furthermore, according to the guidance, once resolved 

and completed, corrective actions should be implemented, tested, and validated 

by incorporating them into subsequent exercises or responses to actual events. 

Once a corrective action has been completed and validated, it becomes a lesson 

learned, which should be disseminated throughout FEMA. (See figure 2.) 

www.oig.dhs.gov 6 OIG-12-118 
 

 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Figure 2. RAMP/CAP Work Flow 

Source: FEMA. 

The draft FEMAfRAMP/CAPfManual, dated September 2011, provides guidance 

to standardize and develop a process to evaluate operational performance in 

actual and simulated events, track agency-wide corrective actions, and, with 

FEMA’s partners, address lessons learned. According to the draft manual, a 

completed corrective action is considered to be validated when it improves 

mission performance. Validation confirms that improvements are being put into 

practice, which is critical to FEMA’s ability to learn and innovate. However, 

FEMA personnel who design internal training exercises said that they had not 

received any requests to validate corrective actions during future exercises.

Because the RAMP/CAPfManual has not been finalized, FEMA offices and 

divisions do not have updated, official guidance and are unaware of roles and

responsibilities in the corrective action process. This need for of guidance has 

hindered validation of corrective actions. 

Managing Lessons Learned

FEMA did not sufficiently disseminate information on agency-specific lessons 

learned. Without a final RAMP/CAPfManual, FEMA needs a standardized 

process for managing and sharing lessons learned. In January 2011, OIG 

reported in FEMA’sfImplementationfoffthefRemedialfActionfManagementf

Programf(OIG-11-32) that FEMA had discontinued its use of the RAMP, and, in 

the process, lost the ability to read historical data on lessons learned. Since 

then, FEMA has located some reports and posted them to LLIS, but all the posted 

reports were from December 2009 and earlier. There are no reports from the 

www.oig.dhs.gov 7 OIG-12-118 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

past 2 years in LLIS. For these reasons, lessons learned were not consistently 

disseminated throughout the agency. As a result, FEMA was missing 

opportunities to learn from the experiences of its personnel and improve its 

disaster management operations. 

Once corrective actions have been completed and validated, they become 

lessons learned, which should then be disseminated throughout FEMA. The 

National Preparedness Assessment Division of NPD is responsible for ensuring 

that FEMA-specific observations and lessons learned from all training, exercises, 

and actual events are promptly submitted to LLIS. Prompt submission to LLIS 

ensures that lessons learned are widely and easily accessible to FEMA personnel 

and eliminates the need for multiple searches to find reports and documents. 

However, 9 of the 10 FEMA regions posted a combined total of only 10 lessons 

learned to the FEMA channel within LLIS, while Region VII posted 47. Every 

exercise and actual disaster can potentially generate multiple lessons learned. 

For example, seven lessons learned were generated from the NLE 2011 alone, 

and once the corrective actions from the exercise have been completed and 

validated, they too will become lessons learned. Given this, we would expect to 

see more than 57 lessons learned posted in LLIS. 

Conclusion

FEMA did not provide proper oversight or clear guidance to track and manage its 

corrective actions or prioritize their completion, nor did it sufficiently disseminate

its agency-specific lessons learned from exercises and disasters. Convening 

regular meetings of the EEP Steering Committee would help to better track and 

manage corrective actions and ensure that their completion was made a priority 

and done in a timely manner. Moreover, if FEMA finalized and implemented the 

RAMP/CAPfManual, the agency could more effectively track and manage, and 

thus, complete and validate corrective actions as well as better disseminate 

lessons learned. By not completing and validating corrective actions and not 

disseminating all its agency-specific lessons learned, FEMA missed opportunities 

to gain knowledge from the experiences of its personnel and improve its disaster 

management operations. Unless FEMA takes steps to resolve issues that arise 

from exercises and disasters and improve its processes, its disaster management 

operations may be adversely affected. 

www.oig.dhs.gov 8 OIG-12-118 
 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Recommendations 

We recommend that the Deputy Administrator, National Preparedness 

Directorate:

Recommendation #1:

Revise Federal Emergency Management Agency Exercise and Evaluation Program 

Management Directive 123-15 to reflect the current organizational structure and 

ensure that the Exercise and Evaluation Program Steering Committee meets 

regularly.

Recommendation #2: 

Finalize, issue, and implement a RemedialfActionfManagementf

Program/CorrectivefActionfProgramfManual that standardizes the corrective 

actions and lessons learned processes and provides a clear delineation of all 

participants’ roles and responsibilities.

Recommendation #3:

Validate completed corrective actions through exercises or actual events and, 

once validated, submit them in a timely manner to the Lessons Learned 

Information Sharing website. 

Management Comments and OIG Analysis 

We obtained and reviewed written comments on the draft report from the 

Director of FEMA’s Office of Policy and Program Analysis. Where appropriate, 

we made changes to the report. FEMA concurred with all three 

recommendations. Appendix B includes a copy of the management comments in

their entirety. Our evaluation of FEMA’s response to the draft report follows. 

Management Response to Recommendation #1 

FEMA concurs with this recommendation. FEMA acknowledges that the 

Management Directive needs to be revised and will determine an appropriate 

steering committee organization to provide internal FEMA oversight of 

corrective actions and lessons learned and update all associated directives as 

appropriate. FEMA points out that the EEP Steering Committee is not expected 

to be the solution for corrective action tracking and that it addresses internal 

FEMA issues, but not interagency issues. 

www.oig.dhs.gov 9 OIG-12-118 
 

 

 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

OIG Analysis

We consider this recommendation resolved because steps are being taken to 

implement it. However, it will remain open until FEMA revises the Management 

Directive and ensures that the Exercise and Evaluation Program Steering 

Committee meets regularly. 

Management Response to Recommendation #2 

FEMA concurs with this recommendation and is developing new doctrine/policy 

that should be completed by August 2012. 

OIG Analysis

We consider this recommendation resolved because steps are being taken to 

implement it. However, it will remain open until the new doctrine/policy that 

standardizes the corrective actions and lessons learned processes is finalized. 

Management Response to Recommendation #3 

FEMA concurs with this recommendation. Validation of completed corrective 

actions and timely sharing of lessons learned will be addressed in the new 

doctrine/policy in development, which should be completed by August 2012. 

OIG Analysis 

We consider this recommendation resolved because steps are being taken to 

implement it. However, it will remain open until the new doctrine/policy is 

finalized. 

www.oig.dhs.gov 10 OIG-12-118 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix A 

Objectives, Scope, and Methodology 

The Department of Homeland Security (DHS) Office of Inspector General (OIG) was 

established by the HomelandfSecurityfActfoff2002 (Public Law 107-296) by amendment 

to the InspectorfGeneralfActfoff1978. This is one of a series of audit, inspection, and 

special reports prepared as part of our oversight responsibilities to promote economy, 

efficiency, and effectiveness within the Department. 

The objective of our review was to determine whether FEMA is managing and tracking 

corrective actions and lessons learned identified during the National-Level Exercises, as 

well as other exercises and disasters. The scope of our review included exercises from 

October 2007 (TOPOFF 4) to May 2011 (NLE 2011). 

To accomplish our objective, we reviewed and analyzed the HomelandfSecurityfActfoff

2002 (Public Law 107-296), as amended, and the Post-KatrinafEmergencyfManagementf

ReformfActfoff2006 (Public Law 109-295), as amended, prior DHS OIG and Government 

Accountability Office reports relevant to our review, and documents provided by FEMA, 

as well as other applicable documents. 

We interviewed officials from DHS and FEMA. We conducted these interviews by phone 

and in person at FEMA headquarters in Washington, DC. We observed the functional 

component of NLE 2011 in Washington, DC, and Herndon, VA. We attended both the 

After-Action Conference and the Lessons Learned Conference for NLE 2011. 

We conducted this review under the authority of the InspectorfGeneralfActfoff1978, as

amended, and according to the Quality Standards for Inspections issued by the Council 

of the Inspectors General on Integrity and Efficiency.

www.oig.dhs.gov 11 OIG-12-118 
 

OFFICE OF INSPECTOR GENERAL 

 Department of Homeland Security 

Appendix B 

Management Comments to the Draft Report 

U.S. Drl1:1l1ment c f IIomel:tntl Security 

\Va:;ltington. DC 20<112 

JUL 1 1 ZOlZ 

MEMORANDUM FOR: Anne L. Richards 

Assistant Inspector General for Audits 

Office of the Inspector Genera l 

FROM: Da""idJ. Kaufman ~ ~ 

Office 

Director 

of Policy and 

~ 

Program Analysis 

SUBJECT: FEMA's Management of Corrective Actions and Lessons Learned 

From National Level Exercises - For Official Use Only 

(OIG ProjecliVo. 11-139-EMO-FEMA) 

Thank you for the opportunity to review and comment on the draft report: FEM1 's Management of 

Corrective Acrions and Lessons Lcal'l1cdjrom National Lewd Exercises For Ufficial Usc Unly. 

This memo addresses our responses to specific findings and recommendations in the report. 

In general, the draft repolt accurate ly attributes SCVCIaJ deficiencies ill the FEMA and interagency 

corrective action processes to programs managed by FEMA. specificall y the National Exercise 

Program (NEP) and the Lessons Learned Program. Some of the discussion and findings require 

clarification to avoid possible misinterpretation of FEMA's authorities and responsibilities. FEMA 

does not have the authority to force the implementation or corrective actions identified by or 

assigned to other Federal departments and agencies. In addition, ne ither existing exercise guidance 

nor the anticipated Lcssons Learned and Continuous Improvcment Program (LL/CIP) assign 

validation responsibility to FEMA's National Preparedness Assessment Division (NPAD) or 

National Exercise Division (NED). Each issue identified during a National Level Exercise (NLE) 

that requires resolution is assigned to a depanment, agency. or directorate \,vith the appropriate 

expeltise to develop. complete. and validate the corrective action. 

Ove rall Comments 

FEMA notes that the 0 10 included discussion on three results of the review. FEMA has several 

comments, provided below: 

• Tracking, Managing, and Completing Corrective Actions 

o The report states "A lack of oversight by the EEP Steering Committee made it morc 

difficult to manage, track, and prioritize completion of corrective actions." The 

FEMA Exercise and Evaluation Pl'Ogralll Steering Committee was l:!'jtablished as an 

internal FEMA coordinating body with no responsibility or authority outside FEMA 

and cannot "coordinate recommendations for all Federal agencies corrective actions. " 

www.oig.dhs.gov 12 OIG-12-118 
 

OFFICE OF INSPECTOR GENERAL 

 Department of Homeland Security 

o The last sentence of this finding states "It is possible that the extensive amount of 

time FEMA allows to complete corrective actions resulted in a diminished sense of 

urgency to complete them." The time allotted is determined by the responsible 

department or agency and depends on the complexity of the improvement plan. 

o Update on corrective action completion rate: 97% ofDHS issues and 91% ofFEMA 

issues from TOPOFF 4 are complete. 100% ofDHS and FEMA issues from NLE 08 

are complete. 95% ofDHS and 100% ofFEMA issues from NLE 09 are complete. 

• Validating Corredive Attions 

o The report states "Guidance for the Homeland Security Exercise and Evaluation 

Program includes an improvement planning process to identifY and resolve corrective 

actions from After-Action Report.;;. Furthennore, according to the guidance, once 

resolved and completed, corrective actions should be implemented, tested. and 

validated by incorporating them into subsequent exercises or responses to actual 

events. Once a corrective action has been completed and validated. it becomes a 

lesson learned, which should be disseminated throughout FEMA." FEMA 

acknowledges that more sped fie direction is required for validating completed 

corrective actions and will provide this direction in its LUCIP documentation and 

revised HSEEP documentation. 

o Each issue identified during an NLE is assigned to a department, agency. or 

directorate with the appropriate expertise to develop the corrective action and submit 

to the CAP. It is each department's, agency's, or directorate's responsibility to 

validate the completed corrective action. Under the NEP, NPAD reports the 

completion and validation of the FEMA and interagency corrective actions that are 

submitted to, maintained, and updated in CAP. NLE participants can nominate 

appropriate exercise objectives to facilitate the validation of completed corrective 

actions. Hovvever, corrective actions also can be validated through real-world events. 

• Managing Lessons Learned 

o The following statement is misleading: nWithout a final RAMP/CAP Manual) FEMA 

lacks a standardized process for sharing lessons learned." The existing FEMA 

manual describes a process for sharing lessons learned within FEMA based on the 

defunct RAMP software ~- and is being updated. FEMA shares lessons learned with 

external partners via LLISgov. LLlS.gov has served as the national lessons learned 

management system since 2004. After-Action ReportslImprovement Plans (AARfIPs) 

and other lessons learned information are disseminated through LLISgov. 

Participation is voluntary. FEMA recognizes the current limitations of UIS.gov and 

the lessons learned program and is currently working to improve the LLIS,gov user 

interface and update the methodology to enhance participation and infonnation 

sharing to meet both FEMA internal and national requirements. 

o The report states "The National Preparedness Assessment Division ofNPD is 

responsible for ensuring that observations and lessons learned from all training, 

exercises, and actual events are promptly submitted to LLIS." NP AD manages the 

lessons learned program and LLlS.gov system, ensuring that LLISgov is available to 

those that have a need to access, that the configuration of LLlS.gol' supports the 

stakeholders, and that infonnation provided by stakeholders is displayed 

appropriately. The decision to submit any documents to LllS.gov remains with the 

individual, department, agency, or other organization that originated the document. 

2 

 

www.oig.dhs.gov 13 OIG-12-118 
 

OFFICE OF INSPECTOR GENERAL 

 Department of Homeland Security 

NP AD posts lessons learned, practice notes, WId best practices gathered through its 

own research and other open-source infonnation. 

o The data point that 9 out of 10 Regions submitted a total of 10 lessons learned, and 

that the 10th Region submitted 47 lessons learned, indicates the importance of culture 

on the development and sharing of lessons leamed, and how the cultural difference 

among FEMA components affects the number of respective lessons learned shared. 

o FE:MA lost the ability to read some, not all, historical data when RAMP was 

discontinued. FEMA was able to recover 86% of the data, including 99% of the 

lessons leamed data that was part of the defunct RAMP, and post related reports to 

LLlS.gov. 

Response to Recommendations 

DIG Recommendation #1: Revise Federal Emergency Management Agency Exen..;se and 

Evaluation Program Management Directive 123-15 to reflect the current organizational structure and 

ensure that the Exercise and Evaluation Program Steering Committee meets regularly .. 

FEMA Response to Reeommendation #1: FEMA concurs with this recommendation. 

FEMA acknowledges that the Management Directive needs to be revised. However, the 

Exercise and Evaluation Program Steering Committee is not expected to be the solution for 

corrective action tracking. Also, the Committee addresses internal FEMA issues, not 

interagency issues. Moreover, FEMA will determine appropriate steering committee 

organization to provide internal FEMA oversight of corrective actions and lessons learned 

and update all associated Directives as appropriate. 

DIG Recommendation #2: Finalize, issue. and implement a Remedial Action Management 

Program/Corrective Action Program (RAMP/CAP) Manual that standardizes the corrective actions 

and lessons learned processes and provides a clear delineation of all participants' roles and 

responsibilities. 

FEMA Response to Re(;ommendatioD #2: FEMA concurs with this recommendation. 

FEMA requests discretion to determine the optimal content and format of the document, e.g., 

broad lessons Jearned and corrective action doctrine versus a RAMP/CAP manual. This new 

doctrine/policy is in development, and should be completed by August 2012. 

OIG Recommendation #3: Ensure that completed corrective actions are validated tluough exercises 

or actual events and, once validated, are submitted in a timely manner to the Lessons Learned 

Infonnation Sharing website. 

FEMA Response to Recommendation #3: FEMA concurs with this recommendation. 

Validation of completed corrective actions and timely sharing of lessons learned will be 

addressed in the new doctrine/policy in development, and should be completed by August 

2012. 

Once again, thank you for the opportunity to comment on the findings and recommendations of the 

DIG Draft Report on FEMA 's Management ofCorreclive Actions and Lessons Learned From 

National Level Exercises prior to its publication. 

3 

 

www.oig.dhs.gov 14 OIG-12-118 
O

 

FFICE OF INSPECTOR GENERAL 

 Department of Homeland Security 

We look forward to working with you on future homeland security emergency management 

engagements to improve FEMA programs and initiatives. Please feel free to provide any 

comments/concerns to Ms. Gina Norton of our GAO/OIG Audit Liaison Office~ 202-646 M

4287. 

4 

www.oig.dhs.gov 15 OIG-12-118 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix C 

Major Contributors to This Report 

Amy Hall, Director 

Modupe Akinsika, Audit Manager 

Eric Hostelley, Program Analyst

S. Ryan Hartong, Program Analyst

Erica Stern, Program Analyst

Kelly E. Herberger, Communications Analyst

Ralleisha Dean, Referencer 

www.oig.dhs.gov 16 OIG-12-118 
 

 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix D 

Report Distribution 

Department of Homeland Security

Secretary

Deputy Secretary 

Chief of Staff 

Deputy Chief of Staff

General Counsel 

Executive Secretary 

Director, GAO/OIG Liaison Office 

Assistant Secretary for Office of Policy 

Assistant Secretary for Office of Public Affairs

Assistant Secretary for Office of Legislative Affairs 

Federal Emergency Management Agency

Administrator 

Assistant Administrator 

FEMA GAO/OIG Liaison

Office of Management and Budget

Chief, Homeland Security Branch 

DHS OIG Budget Examiner

Congress

Congressional Oversight and Appropriations Committees, as appropriate 

www.oig.dhs.gov 17 OIG-12-118 
 

ADDITIONAL INFORMATION AND COPIES 

To obtain additional copies of this document, please call us at (202) 254-4100, fax your 

request to (202) 254-4305, or e-mail your request to our Office of Inspector General 

(OIG) Office of Public Affairs at: DHS-OIG.OfficePublicAffairs@oig.dhs.gov. 

For additional information, visit our website at: www.oig.dhs.gov, or follow us on Twitter 

at: @dhsoig. 

OIG HOTLINE 

To expedite the reporting of alleged fraud, waste, abuse or mismanagement, or any 

other kinds of criminal or noncriminal misconduct relative to Department of Homeland 

Security (DHS) programs and operations, please visit our website at www.oig.dhs.gov 

and click on the red tab titled "Hotline" to report. You will be directed to complete and 

submit an automated DHS OIG Investigative Referral Submission Form. Submission 

through our website ensures that your complaint will be promptly received and 

reviewed by DHS OIG. 

Should you be unable to access our website, you may submit your complaint in writing 

to: DHS Office of Inspector General, Attention: Office of Investigations Hotline, 245 

Murray Drive, SW, Building 410/Mail Stop 2600, Washington, DC, 20528; or you may 

call 1 (800) 323-8603; or fax it directly to us at (202) 254-4297. 

The OIG seeks to protect the identity of each writer and caller.