National Center for Immunization & Respiratory Diseases

COVID-19 vaccine safety monitoring

Tom Shimabukuro, MD, MPH, MBA

CDC COVID-19 Vaccine Planning Unit (VPU) 

Vaccine Safety Team

August 26, 2020

On behalf of:
▪ Safety is a priority 

during all phases of 

vaccine 

development, 

approval, and use

▪ Post-licensure (postauthorization) safety 

monitoring is an 

established part of 

the vaccine life cycle

▪ Monitoring COVID19 vaccine safety will 

be a coordinated 

effort by multiple 

federal agencies

https://www.cdc.gov/vaccinesafety/ensuringsafety/history/index.html#anchor_1593624850886
Rationale for post-licensure/post-authorization 

vaccine safety monitoring

▪ Safety standards for vaccines are high

▪ Pre-licensure trials are not optimal for:

‒ Detecting rare adverse events (numbers enrolled too small)

‒ Monitoring vaccine safety in a real-world environment 

‒ Assessing safety in special populations (e.g., pregnant women and 

people with certain pre-existing medical conditions often excluded)

‒ Evaluating adverse events with delayed onset

3
Roles and responsibilities in vaccine safety 

monitoring (manufacturers vs. U.S. government)
Post-licensure (authorization) safety monitoring

▪ Manufacturers have Phase 4 responsibilities for their individual products

– Based on standard regulatory obligations as specified by FDA

– Can be guided by results from clinical trials

– Conducted or managed by manufacturers’ pharmacovigilance programs 

with regulatory oversight by FDA

– May include post-marketing commitments, post-marketing requirements, 

and pregnancy registries

– Also includes vaccine adverse event (AE) monitoring and reporting of AEs to 

the Vaccine Adverse Event Reporting System (VAERS)
Post-licensure (authorization) safety monitoring

▪ U.S. government has a responsibility for public safety

– 1986 National Childhood Vaccine Injury Act – authorized VAERS

– Monitoring is independent from manufacturers (i.e., no financial stake, less 

real and perceived conflict of interest, important for public confidence)

– Monitoring covers all vaccines from all manufacturers in a comprehensive and 

integrated fashion

– USG manages large data systems that are standing, long-term investments in 

public health surveillance (e.g., VAERS, VSD, CMS)

– Surveillance data from VAERS are made publicly available and surveillance 

findings are presented at federal advisory committee meetings
Post-licensure (authorization) safety monitoring

▪ Manufacturers play a critical role in post-authorization safety monitoring 

however;

▪ We can’t get all the answers from manufacturer monitoring

– USG maintains and has constant access to the largest, most robust, 

and most sophisticated electronic monitoring systems available

• The systems and the methods used by USG agencies are complementary

– USG agencies can freely cooperate and collaborate

• Share information, leverage expertise in other agencies, support each 

others’ surveillance efforts

• Can act in a coordinated and integrated way
COVID-19 vaccine safety monitoring: 

systems and populations
Co-managed by

CDC and FDA

Vaccine

Adverse

Event 

Reporting 

System

+

http://vaers.hhs.gov
Covered populations for COVID-19: Entire U.S. population

▪ VAERS has all 320 

million U.S. residents 

as a covered 

population for safety 

monitoring

▪ i.e., all ages, races, 

states, healthy 

people, those with 

co-morbidities, etc.

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

2015 2016 2017 2018 2019

US Reports Foreign Reports

VAERS total reports received by year
Covered populations for COVID-19: older adults

▪ Active surveillance 

– FDA’s Centers for Medicare & Medicaid Services (CMS) data monitoring 

(Medicare FFS and Medicare Advantage) 

• 55-60 million persons 65+ years old, 92% of the U.S. older adults 

– CDC’s Vaccine Safety Datalink (VSD) (8 integrated health systems) 

• 1.8 million persons 65+ years old 

– Veterans Affairs (VA) data warehouse and electronic health record

• ~1.56 million persons 65+ years old vaccinated annually for influenza in 

recent years
Covered populations for COVID-19: adults & children

▪ Active surveillance (cont.) 

– CDC’s VSD active surveillance (8 integrated health systems)

• 8 million persons 19-64 years old

• 2.3 million persons <18 years old
Covered populations for COVID-19: adults & children

▪ FDA’s Biologics Effectiveness and Safety (BEST) System

– >100 million persons - using Claims (billing) data

– >20 million persons - using Electronic Health Records (EHR)

– 6.2 million children – using PEDSnet Clinical Research Consortium

– 5 million persons - using Claims-EHR linked data

▪ FDA Sentinel - Post-Licensure Rapid Immunization Safety Monitoring (PRISM) Program

– Harvard Pilgrim Healthcare Institute – Operations Center

– 4 large insurers – HealthCore, Humana, Optum, Healthagen

– Claims data with access to medical charts

– Covers >100 million persons
Covered populations for COVID-19: adults & children

▪ DoD VAERS monitoring in collaboration with CDC through a VAERS data 

sharing agreement

– 1.4 million active duty and 860,000 reserves (majority <30 years old); 

dependents and beneficiaries if seen in DoD healthcare facilities

▪ DoD active surveillance

– DoD Immunization Healthcare Div plans to collaborate with the Armed Forces 

Health Surveillance Div to monitor vaccine safety in the DoD EHR systems

• Defense Medical Surveillance System (DMSS) and DoD Personnel and Readiness 

COVID-19 Registry
Covered populations for COVID-19: adults & children

▪ Indian Health Service (IHS) VAERS monitoring in collaboration with CDC 

through a VAERS data sharing agreement (MOU in progress)

– Mainly American Indian and Alaska Native patients seen in IHS and Tribal 

healthcare facilities

– Analysis will be conducted by National Pharmacy & Therapeutics Committee 

and IHS Division of Epidemiology
Case review and inquiry response
Case reviews and inquiry response

▪ CDC’s Clinical Immunization Safety Assessment (CISA) Project

– Assists U.S. healthcare providers with complex vaccine safety questions 

about their patients by conducting in-depth clinical case reviews

– Plan to establish a call service for clinician assistance

▪ CDC’s Immunization Safety Office inquiry response program

– Responds to vaccine safety inquiries and questions from the public, including 

patients, parents, healthcare providers, public health partners, and others

▪ DoD’s Regional Vaccine Safety Hubs (RVSHs) for case evaluation and 

Vaccine Adverse Event Clinical System (VAECS)

– Evaluates and track cases of adverse events following immunization in DoD 

and DoD-affiliated populations
Enhanced monitoring programs to meet the 

challenge of COVID-19
COVID-19 vaccine safety monitoring in early recipients

▪ Challenge

– During the early phase of a national COVID-19 vaccination program, initial doses 

may be distributed to specific groups such as healthcare personnel and other 

essential workers

– In this scenario, activities to enhance normal public health monitoring systems 

will be necessary

▪ Potential solutions

– Active surveillance in early recipients through smartphone- and email-based 

web surveys, with directed reporting to VAERS (enhanced passive surveillance)

– Vaccination capture and enhanced passive surveillance through other data 

sources from healthcare facilities
This Photo by Unknown Author is 

licensed under CC BY-SA

VAERS

Healthcare workers, 

essential workers, etc. Local and systemic 

reactogenicity and 

follow-up to 6 weeks via 

text/web messaging

Medically important 

adverse event 

reports to VAERS

Safety check-up text 

messages or email from CDC

(e.g., Since your vaccination, have you 

had a fever or felt feverish?...)
Additional potential data sources

▪ State Immunization Information Systems (i.e., vaccine registries) to 

capture ‘denominator’ data for adverse event rates

▪ Telehealth encounters in CDC’s Vaccine Safety Datalink (VSD)

▪ Healthcare provider and general public concerns

– Tracking clinical case review requests submitted to CDC’s CISA and public 

inquiries submitted to CDC’s Immunization Safety Office

▪ FDA plans to develop new electronic data sources through EHR partners
Signal detection and signal assessment
Vaccine safety signal*

▪ The Council for International Organizations of Medical Sciences 

(CIOMS) proposed a signal as:

‒ “Information...from one or multiple sources ..., which suggests a 

new potentially causal association, or a new aspect of a known 

association, between an intervention and an event or set of 

related events, either adverse or beneficial, that is judged to be of 

sufficient likelihood to justify verificatory action.”*

▪ In practice, efforts focus on detecting signals for “adverse” 

events

*Practical Aspects of Signal Detection in Pharmacovigilance: Report of CIOMS Working Group VIII. Geneva 2010. 
Approaches to analyzing VAERS data (spontaneous reporting)

▪ Traditional methods 

– Clinical review of individual reports 

• Verify diagnosis and onset interval, characterize clinical and laboratory 

features, identify other potential risk factors

– Aggregate report review (automated data), e.g., cases counts, frequencies of 

adverse event coding terms, reporting trends over time, reporting rates

▪ Statistical data mining methods

– Detect disproportional reporting of specific vaccine-adverse event 

combinations in VAERS database

• Empirical Bayesian and proportional reporting ratio analyses generate 

statistical signals when pre-specified thresholds are reached 
VAERS timeliness for signal detection and assessment

▪ CDC and FDA receive updated VAERS datasets daily

▪ Processing actions for VAERS reports include:

– MedDRA coding of symptoms

– Redaction of personally identifiable information

– Quality assurance

– Preparation for posting on the secure VPN (for investigator access)

▪ Processing times for COVID-19 vaccines

– Death reports: 1 day

– Reports classified as serious*: 3 days

– Reports classified as non-serious: 5 days

*Based on the Code of Federal Regulations if one of the following is reported: death, life-threatening illness, hospitalization or prolongation of hospitalization, 

permanent disability, congenital anomaly or birth defect
Approaches to monitoring in EHR, administrative, 

and claims data

▪ Near real-time sequential monitoring (e.g., Rapid Cycle Analysis [RCA] in VSD)

– Data are refreshed weekly in high volume situations

– Pre-specified outcomes are monitored (i.e., identified in advance)

– A surveillance activity, not the same as an epidemiologic study

– Designed to detect statistical signals (values above specified statistical thresholds)

– When a statistical signal occurs, assessment requires a series of evaluations using 

traditional epidemiologic methods

– Chart-confirmation of diagnoses to confirm or exclude cases as true incident cases 

is a key part of statistical signal assessment 
Closing
Importance of timeliness

▪ Real-time/near real-time safety monitoring will be critical during the 

early stages of the COVID-19 vaccination program

– To characterize the safety profile of COVID-19 vaccines in a real-world 

environment

– To rapidly assess COVID-19 vaccine safety in risk-based priority groups, such 

as older adults and individuals with certain pre-existing health conditions

▪ During a broad-based vaccination program, large amounts of COVID-19 

vaccine are anticipated to be administered during a short period of time

– Important to have established, high functioning systems and validated 

methods in place to rapidly detect and assess potential safety signals so 

public health action can be taken if necessary
ACIP 

COVID-19 

Vaccines 

WG

ACIP 

COVID-19 

Vaccine 

Safety Tech. 

SG

CDC: Kathleen Dooling, 

Sara Oliver

CDC: Tom Shimabukuro

▪ Advise on planning for the use of 

COVID-19 vaccines and advise on all 

components of program 

implementation during a large-scale 

COVID-19 immunization program

▪ Review post-authorization/approval 

vaccine safety surveillance data

▪ Advise on the safety of COVID-19 

vaccine candidates in development and 

safety monitoring of vaccines 

authorized/approved for use

▪ Review post-authorization/approval 

vaccine safety surveillance data

Process
Closing thoughts

▪ Multiple U.S. Government agencies will use complementary systems and 

methods to monitor COVID-19 vaccines

▪ Current monitoring systems have the capacity to effectively monitor 

COVID-19 vaccine safety both under EUA and post-licensure

▪ Analytic methods have been validated through years of development and 

refinement

▪ Data refresh and updates are timely, and analyses occur in near real-time

▪ New data sources will contribute to COVID-19 vaccine safety monitoring, 

especially early in the vaccination program
Topics for future presentations to ACIP

▪ COVID-19 vaccine safety monitoring plans and methods 

▪ Vaccine safety outcomes and adverse events of special interest

▪ Process for reviewing and presenting safety data as it becomes available
Questions?