NCST Technical Investigation of 

Hurricane Maria (Puerto Rico)

Characterization of Morbidity and Mortality

Project Leader: Judith Mitrani-Reiser, PhD, NIST

Project Team: Thomas D. Kirsch, MD, MPH

NCST Advisory Committee Meeting, June 10

-11, 2021

Objective: To complete a quantitative morbidity and mortality assessment of

Puerto Rico, to better understand how damaged buildings and supporting

infrastructure played a role in the injuries and deaths associated with

Hurricane Maria. The study results will provide guidance to improve codes,

standards and inform future approaches to accurately attribute and predict life

loss due to building failure(s) caused by windstorms.
Background

• Death certificates generally underestimate deaths caused by disasters because of

a lack of standards and consistent data collection/reporting.

*

• A standard for disaster-related mortality attribution was proposed in 1999** but it

has not been widely adopted nor applied to a disaster with a long recovery.

***

• Previous studies(^,^^,†,††) attempted to characterize the extent of mortality in Puerto

Rico after Hurricane Maria using available data and/or collecting primary data.

• The GW study†† concluded that the lack of appropriate death certification practice

after the hurricane and the local lack of communication about death certificate

reporting prior to the storm limited the count of deaths that were reported as

related to Hurricane Maria.

* Combs D.L., Quenemoen L.E., Parrish R.G., Davis J.H., 2009. “Assessing disaster-attributed mortality: Development and application of a definition and classification matrix.” International Journal 

of Epidemiology 28(6): pp. 1124–9. 

** Combs DL, Quenemoen LE, Parrish RG, Davis JH. Assessing disaster-attributed mortality: development and application of a definition and classification matrix. Int J Epidemiol 1999;28:1124–9.

*** National Center for Health Statistics. A Reference Guide for certification of deaths in the event of a natural, human-induced, or Chemical/Radiological disaster. (Reference guide No. 1). CDC, 

National Center for Health Statistics 2017. Retrieved from https://www.cdc.gov/nchs/data/nvss/vsrg/vsrg01.pdf

^ Robles, F., Davis, K., Fink, S, Almukhtar, S., 2017. “Official Toll in Puerto Rico: 64. Actual Deaths May Be 1,052.” The New York Times. December 9, 2017.

^^ Santos-Lozada AR, Howard JT. Use of Death Counts from Vital Statistics to Calculate Excess Deaths in Puerto Rico Following Hurricane Maria. JAMA; Aug. 2, 2018: 

doi:10.1001/jama.2018.10929.

† Kishore N, Marques D, et al. Mortality in Puerto Rico after Hurricane Maria. NEJM 2018; 379:162-170

†† George Washington University, in collaboration with the University of Puerto Rico Graduate School of Puerto Rico, 2018. “Ascertainment of the Estimated Excess Mortality from Hurricane Maria 

NCST Advisory Committee Meeting, June 10

in Puerto Rico,” a Project Report for the Governor of Puerto Rico, August 28, 2018. 

-11, 2021
Motivation

• To date, several studies(^,^^,††) have focused on assessing Hurricane Maria’s

death toll using excess mortality approaches, such as daily and monthly

comparisons with previous years.

• Additionally, a single population-based study† characterized the

predominant reasons associated with deaths in their sample as attributable

to a lack of healthcare access.

• The NIST mortality project will not produce another death count.

• Instead, NIST will use rigorous, scientific methods to understand the injury

mechanisms of those that perished due to the storm, within the greater

context of the hazards and the building and infrastructure system failures

experienced by those individuals.

NCST Advisory Committee Meeting, June 10

-11, 2021

^ Robles, F., Davis, K., Fink, S, Almukhtar, S., 2017. “Official Toll in Puerto Rico: 64. Actual Deaths May Be 1,052.” The New York Times. December 9, 2017.

^^ Santos-Lozada AR, Howard JT. Use of Death Counts from Vital Statistics to Calculate Excess Deaths in Puerto Rico Following Hurricane Maria. JAMA; Aug. 2, 2018: 

doi:10.1001/jama.2018.10929.

† Kishore N, Marques D, et al. Mortality in Puerto Rico after Hurricane Maria. NEJM 2018; 379:162-170

†† George Washington University , in collaboration with the University of Puerto Rico Graduate School of Puerto Rico, 2018. “Ascertainment of the Estimated Excess Mortality from Hurricane 

Maria in Puerto Rico,” a Project Report for the Governor of Puerto Rico, August 28, 2018. 
Updates on Mortality Contract

q Option Exercised: Option exercised on April 19, 2021 to the contract

awarded to the George Washington University Milken Institute School of

Public Health on July 27, 2020; subawards to University of Puerto RicoGraduate School of Public Health and independent external consultants;.

University of Puerto Rico-Medical Sciences:

Pablo Méndez-Lázaro, PhD

Local Project Coordinator

Laura Cabrera, Dr.PH (Cand)

Environmental Health

External consultants:

Bernardo Hernandez-Prado, ScD

Lead Verbal Autopsy, IHME, University of Washington

Abraham D. Flaxman, PhD

SmartVA, IHME, University of Washington

Aurelio Castro, MS; Glenda Roman, MS

Geographic Mapping Technologies, Corp., San Juan, PR:

GIS and Remote Sensing

George Washington University:

Carlos Santos-Burgoa, MD, MPH, PhD

Project Manager

Ann Goldman, MPH, PhD

Project Coordinator

Ramin Asgary, MD, MPH, MSc

Global Humanitarian Health 

Ximena Riesco-Cruzat, MD, MSc, PhD

Health systems

Leslie Zaragoza, MPH (cand)

Epidemiology

Maria Jose Talayero, MD, MSc, DrPH (cand)

Environmental Health 

Diane Uschner, PhD

Biostatistics Center

Sharon Edelstein, MS

Biostatistics Center

Jordan Kahn

Biostatistics Center

Michele Santacatterina, PhD

Biostatistics Center

Valerie Donohue

Biostatistics Center

Ralph Rivera-Gutierrez

Health Systems and Policy

Heriberto Marin

Health Services Analysis

NCST Advisory Committee Meeting, June 10

-11, 2021
Develop Integrated Database of Deaths in Puerto Rico

• Collect and merge geocoded data on the deaths occurring up to six months after Hurricane

Maria made landfall in Puerto Rico.

• Data sources include the Puerto Rico Vital Registration System with available datasets from the

Bureau of Forensic Sciences, including DMORT E-Cases records, pathology registry records,

case identification, case review, and cremation and institutional death databases.

• Additional data to consider are 911 emergency calls obtained from the Bureau of Police, and

funeral and burial assistance data from FEMA and the American Red Cross.

Integrated Database

Puerto Rico Vital 

Registration System 

e.g., death 

certificates

Bureau of Forensic 

Sciences

e.g., DMORT records, 

pathology records, 

cremation databases 

Bureau of Police

e.g., 911 

emergency call 

records

FEMA and/or

American Red 

Cross

e.g., funeral and 

burial assistance

Project Plans (1/4): Integrated Database

NCST Advisory Committee Meeting, June 10

-11, 2021
Project Plans (2/4): Spatial & Temporal Analysis

Spatial and Temporal Clustering of Deaths

• Analyze the integrated database to calculate cause-specific mortality rates adjusted for age and

gender and compare these to the prior years.

• Examine each broad cause of death (i.e. ICD-10 codes) and specific causes of death typically

attributed to hurricanes (e.g., drowning, death from a fallen object, etc.) and compare them to

previous years. Identify significant increases in death rates from particular causes between the

period after the storm and years prior; flag all deaths from those causes in the first two weeks

after the storm.

• Identify spatial and temporal clusters of deaths occurring up to six months after the storm

(entire island), with a separate analysis for the first 14 days after the event (4 study regions).

Source: Santos-Burgoa et al., 2018 (with permission)

NCST Advisory Committee Meeting, June 10

-11, 2021
NCST Advisory Committee Meeting, June 10

Source: Bernardo Hernández Prado (with permission)

-11, 2021

Project Plans (3/4): Verbal Autopsy (VA) + Social 

Environmental (S’E) Survey

Foundation for survey with Next of Kin 

and Key Informants

• SmartVA is an application developed by Institute for Health Metrics 

and Evaluation (IHME), University of Washington).

• SmartVA includes a VA questionnaire, and a module of analysis to 

ascertain of cause of death. 

• Spanish verbal autopsy instruments are also available.

• SmartVA-Analyze uses the Tariff method^^^ to ascertain causes of 

death from a list of 34 causes for adults, 21 for children and 6 for 

newborns, excluding stillbirths. 

• SmartVA-Analyze uses the Tariff method to provide most likely 

underlying cause of death, and the next two or three most likely 

ones. 

• Suggested mode was originally face-to-face; survey mode is being 

adapted to phone due to COVID-19.

• This VA+ S’E will mark the beginning of the development of a 

disaster-centric verbal autopsy. 

^^^James et al., 2011. Performance of the Tariff Method: validation of a simple additive algorithm for analysis of 

verbal autopsies. Population Health Metrics, 9:31.
Project Plans (4/4): Medical Records and 

Hospital Functions Review

Contract option exercised in April 2021

• Confirmation of individual information to help support VA+S’E survey effort.

• Develop structured interview survey to identify potential operational changes due to storm.

• Selection of expert panel with local and national experts.

• Retrieve additional data (e.g., medical records, hospital functions data) of the selected

hospital deaths to augment key elements required to inform causal path of death and for

final determination of the death disease category.

• Integrate secondary data to expand the explanatory variables used in analysis.

Gather data 

associated with the 

death from medical 

records.

Interview management 

and key staff at 

hospitals, while 

leveraging data 

collected by other 

projects.

Process 

information from 

cases and 

integrate any data 

from other sources.

Panel of 

experts makes 

final 

determination 

for all cases.

NCST Advisory Committee Meeting, June 10

-11, 2021
Recent Progress: Integrated Database

NCST Advisory Committee Meeting, June 10

-11, 2021

• Data Use Agreements (DUA) with the

Demographic Registry at the Puerto

Rico Department of Health and the

Institute for Forensic Sciences.

• The transfer of updated datasets

ensure that the project’s analyses

contain any additions or corrections

to the original 2017-2018 dataset.

• Data for the first 14 days after the 

storm’s landfall used to 

search/locate next of kin or key 

informants.

• GWUPR’s GIS team is standardizing 

and geocoding addresses for death 

place and residence of the deceased. 

(N=16,600)

Source: Carlos Santos-Burgoa (with permission)

• NIST HM and GWUPR’s GIS staff met to discuss leveraging data across the HM Program’s projects, 

and support analyses and visualization of hurricane impacts.

• Meetings held with FEMA to explore additional MOA(s) for data sharing.
Recent Progress: Morbidity and Mortality Library 

NCST Advisory Committee Meeting, June 10

-11, 2021

• The Hurricane Maria mortality and injury literature review consists of 240 resources, including

scientific publications, reports, instruments, and grey literature.

• Every document is available in a repository on a shared drive.

• As part of their literature review, GWUPR developed a compendium or compilation of resources

with key information (i.e., objective of study or highlights of contents) to assist in navigation.

• The taxonomy for the organization of the resources and the creation of a Thesaurus were

developed collaboratively by the NIST HM and GWUPR staff.

• Literature review resources into a citation management tool used across the NIST HM Program.

Superordinate Level (Hierarchical Classification)

Key Word (Framework Organization)

Broad Term (Thesaurus)

Subordinate Level (Hierarchical Classification)

Key Terms (Framework Organization)

Narrow Terms (Thesaurus)

Source: Ximena Riesco Cruzat, Lezlie Zaragoza, and 

Regina Avila (with permission)
Recent Progress: Verbal Autopsy Pilot Study

NCST Advisory Committee Meeting, June 10

-11, 2021

• Verbal Autopsy and Socio-Environmental (VA+S’E) instrument developed with the objectives to: (1)

ascertain the cause of deaths that took place within two weeks of the storm making landfall in Puerto

Rico, and (2) identify the needed socio-environmental contextual factors that could be related to the

occurrence of those deaths.

• PRA approval from OMB (March 2021) for the VA+S’E instrument’s use in a pilot study.

• Interviewer training took place over 5 half-days, totaling more than 20 hrs and including interactive

sessions with the following content:

1) background on the NCST 

investigation and 

mortality project,

2) survey procedures,

3) familiarization with the 

questionnaire and the 

tablets (see photo),

4) emotional containment 

strategies, and 

5) an exercise of empathy 

and self-care.

• Interviews (a total of 50 expected) for 

the pilot study began on April 26, 2021.

Photographed (left to right): Lyz Negrón, Miladys Perez, Ambar Meléndez, Tanialy Rivera, Katherine Torres, 

Jennifer Acevedo, Guillermo Gorbea, Janet Forestier, Adian Rodriguez, Rocío Rodríguez, Rocio Lamboy, 

Myrna Del Mar Gonzalez, Genesis Rodriguez, Julian Hernandez, Laura Cabrera, and Pablo A. Méndez Lázaro

Photo Credit: Laura Cabrera (with permission)
Next Steps

NCST Advisory Committee Meeting, June 10

-11, 2021

Integrated Database:

• Continue to work with other projects to inform the cause-specific mortality assessment.

• Finalize any required MOUs to retrieve secondary data, as needed.

o May need to develop contingency plans if some data cannot be acquired.

Spatial & Temporal Analysis:

• Finalize the standardizing and geocoding of location data.

• Continuously adding new layers to dataset and work across projects on data analysis.

• Complete analysis for spatial and temporal clustering of deaths.

Cause Specific Analysis:

• Analysis will use latest mortality data and is expected summer 2021.

• Analysis of excess mortality by groupings of death categories using the International Classification of Diseases†††.

• Analysis covers: (1) up to 14 days post landfall, and (2) up to 6 months post landfall (Sept 2017 – Feb 2018).

Verbal Autopsy:

• Finalize pilot study and update the survey instrument as required.

• Deployment survey in four study regions (expected by late summer).

o Concern with accuracy of contact information of next of kin and key informants.

o Concern with response rates, based on early feedback from pilot study.

o Develop a public engagement plan with public affairs officers at NIST, GW, and UPR to support interview recruitment.

• Finalize framework for attribution analysis, including risk factors associated with building failures.

Medical Records and Hospital Functions Review:

• Identify data required for attribution determination by clinical panel; finalize clinical panel members; develop 

survey instrument as needed to acquired additional hospital data.

†††World Health Organization. (2004). ICD-10 : international statistical classification of diseases and related health problems : tenth revision, 2nd ed. World Health Organization.
NCST Technical Investigation of 

Hurricane Maria (Puerto Rico)

Characterization of Morbidity and Mortality

Project Leader: Judith Mitrani-Reiser, PhD, NIST

Project Team: Thomas D. Kirsch, MD, MPH

*A special thanks to Captain Rebecca Noe (CDC) for sharing her time and 

expertise so graciously with the Hurricane Maria NCST investigation, and 

specifically, with this project!

NCST Advisory Committee Meeting, June 10

-11, 2021

Questions?

Please ‘raise your hand’ using the Blue Jeans 

Participant window and unmute your audio and video