NCST Technical Investigation of Hurricane 

María (Puerto Rico)

Characterization of Morbidity and Mortality

Project Leaders: Judith Mitrani-Reiser, PhD (National Institute of

 Standards and Technology, NIST)

 Thomas D. Kirsch, MD, MPH (National Center for Disaster

 Medicine and Public Health, NCDMPH)

NCST Advisory Committee Meeting, June 8-9, 2022
Background

• Death certificates often misclassify deaths caused by disasters because of a lack of 

attribution standards and consistent data collection/reporting.*

• Previous studies(^,^^,†,††) attempted to characterize the extent of mortality in Puerto 

Rico after Hurricane María using available data and/or collecting primary data.

• The GW study†† concluded that the lack of appropriate death certification practice 

after the hurricane and the local lack of communication about death certificate 

reporting prior to the storm limited the count of deaths that were reported as related 

to Hurricane María.

• The NIST mortality project will not produce another death count. Instead, NIST will use 

rigorous, scientific methods to understand the distribution of causes of death and the 

injury mechanisms of those that perished due to the storm, within the greater context 

of the hazards and the building and infrastructure system failures experienced by 

those individuals.

* Combs D.L., Quenemoen L.E., Parrish R.G., Davis J.H., 2009. “Assessing disaster-attributed mortality: Development and application of a definition and classification matrix.” International Journal 

of Epidemiology 28(6): pp. 1124–9. 

^ Robles, F., Davis, K., Fink, S, Almukhtar, S., 2017. “Official Toll in Puerto Rico: 64. Actual Deaths May Be 1,052.” The New York Times. December 9, 2017.

^^ Santos-Lozada AR, Howard JT. Use of Death Counts from Vital Statistics to Calculate Excess Deaths in Puerto Rico Following Hurricane Maria. JAMA; Aug. 2, 2018: 

doi:10.1001/jama.2018.10929.

† Kishore N, Marques D, et al. Mortality in Puerto Rico after Hurricane Maria. NEJM 2018; 379:162-170

†† George Washington University, in collaboration with the University of Puerto Rico Graduate School of Puerto Rico, 2018. “Ascertainment of the Estimated Excess Mortality from Hurricane Maria 

in Puerto Rico,” a Project Report for the Governor of Puerto Rico, August 28, 2018. 

2
Updates on Mortality Contract

❑ Contract Modified: Contract awarded to the George Washington University 

Milken Institute School of Public Health on July 27, 2020 was modified on 

May 23, 2022 to explore new methods in finding next-of-kin and key 

informants for VA’SE surveys.

3

George Washington University:

Carlos Santos-Burgoa, MD, 

MPH, PhD

Project Manager

Valerie Donohue

Biostatistics Center

Sharon Edelstein, MS

Biostatistics Center

Jordan Kahn

Biostatistics Center

Nolan Kuenster, MPH (cand)

Epidemiology

Ayush Dwivedi, MS

Computer Science

Ann S. Goldman, MPH, PhD 

Project Coordinator

Jone Renteria, MS

Biostatistics Center

Michelle Santacatterina, PhD

Biostatistics Center

Maria Jose Talayero, MD, MSc, 

DrPH (cand)

Environmental Health 

Diane Uschner, PhD

Biostatistics Center

Leslie Zaragoza, MPH (cand)

Epidemiology

University of Puerto Rico-Medical Sciences:

Pablo Méndez-Lázaro, PhD

Local Project Coordinator

Laura T. Cabrera-Rivera, MPH,

DrPH (s)

Environmental Health

Alejandro Rodríguez-Putnam, 

MPH

External consultants:

Bernardo Hernandez-Prado, ScD

Lead Verbal Autopsy, IHME, University of Washington

Abraham D. Flaxman, PhD

SmartVA, IHME, University of Washington

Aurelio Castro, MS; Glenda Roman, MS

Geographic Mapping Technologies, Corp., San Juan, PR:

GIS and Remote Sensing

Ralph Rivera-Gutiérrez, PhD

Health Systems and Policy

Heriberto Marín Centeno, 

PhD

Health Services Analysis

Andrea C. Zambrana 

Rosario, BA, MPH (cand)
Source: Bernardo Hernández Prado (with permission)

Project Plans (1/4): Verbal Autopsy (VA) + Social 

Environmental (S’E) Survey

Foundation for survey with next-of-kin and key informants

• VA+S’E survey marks the beginning of the development of a 

disaster-centric verbal autopsy, when including the social 

vulnerability and environmental conditions before, during, 

and after the event. 

• The VA+S’E survey builds on the Institute for Health Metrics 

and Evaluation (IHME) VA questionnaire 

• SmartVA-Analyze is an analysis program used to determine 

the cause of death and is an open application developed by 

IHME, University of Washington 

• SmartVA-Analyze uses the Tariff method^^^ to provide most 

likely underlying cause of death, and the next two or three 

most likely ones. 

• Suggested mode was originally face-to-face; survey mode was 

adapted to phone due to COVID-19.

^^^James et al., 2011. Performance of the Tariff Method: validation of a simple additive algorithm for analysis of 

 verbal autopsies. Population Health Metrics, 9:31.

4
Project Plans (2/4): Medical Records and Hospital 

Functions Review

MEDICAL RECORD ABSTRACTION

Q1

Q2

Q3

Q4

Q5

Q6

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Sociodemographic data

Pre-triage (reason for 

ambulance call, patient 

status, medications, vitals, 

prognosis, etc.)

Triage (problem list, level of 

consciousness, duration of 

illness, medications, vitals, 

prognosis, etc.)

Accident Information, ER 

Notes, Ward Admission Notes, 

Daily Progress, ICU Notes, 

Transfer, Death Certificate) 

5

Hospital Medical Records Abstraction

• Retrieve additional data (e.g., medical records, 

hospital functions data) of the selected hospital 

deaths to augment key elements required to 

inform causal path of death and for final 

determination of the death disease category

• Integrate secondary data to expand the 

explanatory variables used in analysis

• Confirmation of individual information to help 

support VA+S’E survey effort

Hospital Management and Clinical Staff Interviews

• Develop structured interview to identify potential 

operational changes due to storm

• Selection of expert panel with local and national 

experts
Develop Integrated Database of 

Deaths in Puerto Rico

• Collect and merge geocoded data on the 

deaths occurring in the first 15 days and 

up to six months after Hurricane María 

made landfall in Puerto Rico

• Data sources include the Puerto Rico Vital 

Registration System with available 

datasets from the Bureau of Forensic 

Sciences, including DMORT E-Cases 

records, pathology registry records, case 

identification, case review, and cremation 

and institutional death databases.

• Additional data from other projects in the 

investigation, NWIRP study, and other 

products from the project

Project Plans (3/4): Integrated Database

Integrated

Database

Puerto Rico 

Demographic 

Registry

e.g., death certificates

Puerto Rico Institute 

of Forensic Sciences

e.g., DMORT records, 

pathology records, etc.

Puerto Rico 

Bureau of Police

e.g., 911 emergency 

call records

FEMA and/or

American Red 

Cross

e.g., funeral and 

burial assistance

NCST + NWIRP

e.g., wind speeds, 

damage 

assessments, etc.

VA’SE + Hospital

e.g., SmartVA 

output, medical 

records, etc. 

6
Project Plans (4/4): Spatial & Temporal Analysis

Spatial and Temporal Clustering of Deaths

• Analyze the integrated database to calculate 

cause-specific mortality rates adjusted for age 

and gender and compare these to the prior seven 

years to obtain cause-specific excess mortality.

• Examine groups of causes of death (based on 

ICD-10 codes) and the causes usually attributed 

to hurricanes (e.g., drowning, death from a fallen 

object, etc.) and compare them to the 7 previous 

years. 

• Identify significant increases in death rates from 

specific causes in the aftermath of the storm; flag 

all deaths from those causes in the first two 

weeks after the storm. 

• Identify spatial and temporal clusters of deaths 

occurring up to six months after the storm (entire 

island), with a separate analysis for the first 14 

days after the event

Puerto Rico Hurricane Maria Mortality Study 

Population, by time-period, sex, age, and 

socioeconomic development index (SEI*).

*Junta de Planificación. Índice de desarrollo 

socioeconómico municipal. 2017.

7
• VA+S’E instrument developed with the objectives to: (1) ascertain the cause of deaths within two 

weeks of the storm making landfall in Puerto Rico, and (2) identify socio-environmental contextual 

factors that could be related to the occurrence of those deaths.

• Pilot study conducted April - June 2021; instrument was revised and submitted for IRB and PRA 

approvals July - August 2021; final VA’SE instrument approved by OMB on September 3, 2021. 

• Diverse outreach methods (e.g., > 950 letters mailed via USPS, study website, social media, etc.) were 

implemented to reach next-of-kin and key informants.

• UPR team completed 368 interviews as of May 31, 2022. Of the individuals contacted, 62% agreed to 

participate. More than 1500 informants remain unreachable.

Train interviewers IRB/OMB approvals Initiate full study

Analyze data

PHASE I: Pilot

Apr – Ju 2021

PHASE II: Study Prep

Jul – Sep 2021

PHASE III: Full Study

Oct 2021 – Mar 2022

PHASE IV: Next Steps

Mar 2022 – present 

Conduct pilot study

Report on prelim 

findings

Identify strategies to 

improve response rates

Outreach to informants

Call informants

Evaluate data quality

Assess biases

Identify strategies 

to locate additional 

 informants

Draft final report 8

Recent Progress: Verbal Autopsy and Socio-Environmental Survey
• Non-response analysis completed to assess any biases introduced based on respondents

• The team has been conducting quality control checks to make sure all informants had followed the 

3-call protocol, including one call on the weekend. 

• A modification was awarded on May 23, 2022 to GW-UPR to use a people finder service to contact 

more next of kin and key informants.

Recent Progress: Verbal Autopsy and Socio-Environmental Survey

9

Characteristics Interviewed Non-Interviewed P-Value*

Gender Female 171 (47.8%) 652 (45.1%) 0.36 Male 187 (52.2%) 794 (54.9%)

Age

0-39 years 13 (3.6%) 63 (4.4%)

0.20 40-64 years 54 (15.1%) 274 (18.9%)

65+ years 291 (81.3%) 1105 (76.4%)

Not Available 0 (0.0%) 4 (0.3%)

Education

Bachelor’s degree 41 (11.4%) 121 (8.4%)

0.01

Associate degree 33 (9.2%) 84 (5.8%)

High school diploma or GED 95 (26.5%) 328 (22.7%)

Graduate school 15 (4.2%) 41 (2.8%)

No high school diploma 167 (46.7%) 783 (54.1%)

Not Available 7 (1.9%) 89 (6.2%)

Marital status

Married 127 (35.5%) 444 (30.7%)

0.02

Never married 50 (13.9%) 287 (19.8%)

Separated 69 (19.3%) 217 (15.0%)

Widowed 108 (30.2%) 440 (30.4%)

Not Available 4 (1.2%) 58 (4.0%)

Residence zone

Rural 164 (45.8%) 610 (42.2%)

Urban 182 (50.8%) 746 (51.6%) 0.08

Not Available 12 (3.4%) 90 (6.2%)

*Pearson 

ChiSquare 
• Implemented a non-random, stratified, and paired matched sampling method to select 

the sample of hospitals (6 hospitals total)

• Data will be extracted from the medical records of each of the deaths that occurred in the 

sample hospitals within 14 days of the impact of Hurricane María. 

• Interviews with 5-6 members of the key administrative and medical-clinical staff who 

worked in the hospital at the time of Hurricane María's impact. The instrument will cover 

the following topics, among others:

o Position (in the hospital) of the person being interviewed 

o Information on pre-existing problems in the hospital prior to Hurricane María

o Hospital emergency plans and their implementation

o Hospital operational status

o Availability of utilities

o Operation of the hospital's clinical units

o Impact of these circumstances on hospital operations and patient care

Recent Progress: Medical Records and Hospital Functions 

Review

10
Recent Progress: Medical Records and Hospital Functions 

Review

• Hospital survey instruments developed with input 

across HM Program projects, and submitted for OMB 

approval on May 19, 2022

• Hospital recruitment efforts underway, in 

collaboration with relevant HM Program projects

• Recruitment efforts include letters of support from 

the Puerto Rico Department of Health (PRDOH).

• For the medical records abstraction, an electronic 

form will be utilized to gather the following data:

o Basic sociodemographic information

o "Pre-triage" and "triage" information, such as the 

patient's status, vital signs, main complaints, etc.

o Injuries information, if applicable

o Emergency Room Admission Notes

o Room Admission Notes

o Daily progress notes for length of hospital stay

11 o Patient transfers- last day; DNR order information
Recent Progress: Spatial and Temporal Analysis

• NIST HM and GWUPR’s GIS staff meet to discuss leveraging data across the HM Program’s projects, 

and support analyses and visualization of hurricane impacts; more than 10 layers of information 

for analysis. 

• Cause-Specific Excess Mortality report submitted to NIST in April 2022 for review and feedback.

12
Next Steps

Verbal Autopsy and Socio-Environmental Survey:

• Continue survey data collection with additional methods to identify next of kin and key 

informants 

• Employ finalized framework for attribution analysis, including risk factors associated 

with building failures

Medical Records and Hospital Functions Review:

• Collect medical records from deaths that occurred in the hospital sample

• Interview administrators and clinicians in sample hospitals

• Finalize clinical panel members

• Identify data required for attribution determination by clinical panel

Integrated Database:

• Continue to work with other projects to inform the cause-specific mortality assessment

• Finalize any required MOUs to retrieve secondary data, as needed

Spatial & Temporal Analysis:

• Continue to work with other projects to inform the cause-specific mortality assessment

• Continue adding new layers to dataset and work across projects on data analysis

• Complete analysis for spatial and temporal clustering of deaths

13
NCST Technical Investigation of 

Hurricane Maria (Puerto Rico)

Characterization of Morbidity and Mortality

Project Leader: Judith Mitrani-Reiser, PhD (NIST)

Project Team: Thomas D. Kirsch, MD, MPH (NCDPH), Emina Herovic, 

PhD (NIST), GW/UPR scientists

*A special thanks to Captain Rebecca Noe (CDC) for sharing her time and expertise so 

graciously with the Hurricane María NCST investigation, and specifically, with this 

project!

Questions?

Please ‘raise your hand’ using the Blue Jeans 

Participant window and unmute your audio and video

NCST Advisory Committee Meeting, June 8-9, 2022