Session G-1: Insights from the Congressional Watchdog on Equity in Evaluation Design

Assessing Disparities in FEMA Flood Map 

Investments using Multivariate Analysis 

Lijia Guo, Ph.D., ASA, MAAA

Government Accountability Office

Federal Committee on Statistical Methodology (FCSM) 

Research and Policy Conference

October 27, 2022
Presentation Overview

•Background/Issues/Challenges

•Data

•Geospatial Analysis

•Methodology/Analysis

•Findings

•Impact of Work
Flood

•Flood is the number one (frequently, costly, deadly) nature disaster in the US

•National Flood Insurance Program (NFIP) – Federal Program established in 1968

Estimated Loss* from 

Hurricane Ian

• NFIP: 

• $10-$13 billion 

• Private Market

• $7 billion (BL)

• $46-$67 BL

*Projected estimate made 

by CoreLogic
What are Flood Maps?

• FEMA Flood Maps are created and updated 

to identify the extent of flood risks and set

• Mandatory purchase requirement for NFIP

• Special community building standards. 

• Spending/Investment

• $2.3 billion on flood hazard mapping from 2014-

2020 ($414 million annually since 2019)

• 1.2 million miles of Flood Maps, over 22,500 

maps cover “NFIP communities”, with differing 

levels of “details”
FEMA Mapping Process

• Started at the FEMA community level

• FEMA’s 10 Regional offices collaborates with 

State and Local Governments to prioritize 

where to create or update new flood maps

• Only a third of the US territory currently 

falls under US flood maps

Has FEMA’s map investments been aligned with its strategic priorities: areas with higher risk for flooding and 

vulnerable/underserved populations over the years?
Definitions and Challenges

Risk: Function of 3 factors: Threats / Vulnerabilities / Consequences

Equity:

• GAO Yellow Book: GAO-21-368G: “...program provides equitable access to or distribution of public 

resources within the context of statutory parameters;” 

• Executive Order 13985: “... Advancing Racial Equity and Support for Underserved Communities Through the 

Federal Government

• FEMA Strategic Plan: “... Achieve Equitable Outcomes for Those We Serve”

GAO’s objectives: 

1. To what extent has FEMA developed flood maps (FIRMs) products that reflect current and future flood 

hazards? 

2. 2. To what extent has FEMA assessed its mapping program’s efforts to enhance flood resilience, ...? 

3. How has FEMA prioritized its mapping resources in determining where to create and update its maps and 

flood risk products over the years?
FEMA Data

FEMA’s Administrative data for Flood Map:

• Coordinated Needs Management Strategy (CNMS) - geospatially-enabled platform 

• New, Validated, or Updated Engineering (NVUE) - standard and performance metric

• Project Planning And Purchase Portal (P4) - multi-year cost and budget estimation tool 

• Map Information Platform (MIP) - +500 mapping standards, guidance, technical reference 

FEMA’s Open-Source, Risk Assessment Tools

• National Risk Index (NRI) - risk assessment for 18 natural disasters

• Resilience Analysis and Planning Tool (RAPT) – hazard and infrastructure information, as 

well as 20 community resilience indicators (including ACS data) 

• Community Engagement Prioritization Tool (CEPT) - floodplain management, monitor and 

promote program compliance

FEMA has not made use of a variety of additional quantitative data that could inform its decision-making process
Methodology/Analysis

•Research Question: have FEMA’s Flood Insurance Rate Maps investments for FY12-20 

aligned with its goals: 

prioritizing areas with the highest risks and areas with the most vulnerable populations

Measuring Flood Risks

•Flood zones (CNMS)

•Frequency, probability of 

flooding

•Exposure, expected annual 

loss 

•Historical loss ratio

•Flood risk scores (NRI)

Measuring community vulnerability and 

resilience

•Social vulnerability score and rating 

(NRI)

•CDC Social Vulnerability Index (SVI) 

(CEPT)

•FEMA community resilience ratings 

(RAPT)
Data Challenges

• FEMA makes mapping decisions 

about whether to target 

investments at the individual project 

level, by FEMA communities

• Over 22,000 communities

• FEMA measures of risk, 

vulnerability, and demographics are 

reported at the level of Census tract. 

FEMA Community 240030: Frederick City, MD
Geospatial Analysis

• We conducted geospatial analysis to 

identify the Census tracts that were 

located inside of each FEMA 

community, either fully or partially

• Overlaying the boundaries of 

Census tracts onto the 

boundaries of communities. 

• Community 240030 intersects 

with 24 censes tracts

Social Vulnerability Index for Census Tracts, Frederick, MD
Geospatial Analysis

• Created the Crosswalk: calculating 

the area of each Census tract that 

was contained within each 

community and weights:

• Estimated the measures of risk, 

vulnerability, and demographics at 

the FEMA community level

FEMA Community 240030: Frederick City, MD

wwiiii =

AAAAAAAA oooo GGGGGGGGGG ww ww hiiii CC ii

AAAAAAAA oooo CC ii
Evaluating FEMA Mapping Progress

•Measures of Program Outcomes

•Product delivered (miles mapped) 

•Quality of the product (tiers, NVUE 

complaint)

•How long it take to get the map updated 

•Cost/ ($) invested (FEMA P4)

•Multivariate analysis/modeling 

•Program outcomes linked to flood risks, 

vulnerability, resilience, and other 

characteristics 

•Variable selection using multivariate 

analysis

•Choose the most efficient models

•Sensitivity tests
Key Findings

•FEMA’s mapping investments from fiscal years 2012 to 2020 were 

•greater where flood risks were higher, while controlling all other factors

•lower for communities with higher levels of social vulnerability and underserved populations, other 

identified factors being equal

Communities with higher flood risks 

had

• more miles of digital maps

• greater increases in the percentage of 

mapped miles that met the NVUE standard

• shorter cycle times between the stages of 

FEMA’s mapping process

Communities with higher levels of social 

vulnerability had

• more unmapped miles or paper FIRMs in FY12

• smaller increase in the percentage of mapped 

miles that met the NVUE standard

• longer cycle times between the stages of 

FEMA’s mapping process
Modeling Summary/Takeaways

•Assessing program effectiveness and equitable services simultaneously 

•Identified opportunity for making data-informed decision

•Addressing equity Issues 

•Data linking considerations (Unit of analysis, data credibility)

•Using predictive analytics tools that fits your needs

•Be flexible & asking the right questions

•Statistical Considerations

•Sampling Error

•Collinearity of inputs

GAO Report(GAO-22-104079) 

FEMA Flood Maps: Better 

Planning and Analysis Needed to 

Address Current and Future 

Flood Hazards
Impact of work

GAO Recommendation: FEMA should consider ways to leverage and integrate available flood risk data, 

such as through statistical analyses, into its annual process for prioritizing flood mapping investments.

FEMA’s response/progress: 

• FEMA stated it has developed a new data tool that consolidates multiple datasets into its annual 

process for prioritizing flood mapping investments. 

• FEMA will assess the incorporation of additional criteria into project selection, as is their continuation 

of the evolution for a more comprehensive, data-driven approach to project selection.

Other Connections: 

• Current GAO works

• National Flood Conference 

• National medias (CNN, Business Insurance, ...)