CDC plans for COVID-19 vaccine effectiveness 

evaluation post-authorization and post-licensure

Katherine Fleming-Dutra, MD

Vaccine Effectiveness Team, Vaccine Evaluation Unit

Vaccine Task Force, COVID-19 Response

Centers for Disease Control and Prevention 

January 27, 2021
Need for post-authorization or post-licensure vaccine 

effectiveness (VE) estimates

▪ Real world protection may differ from efficacy under trial conditions

• Timing and coverage of 2-dose regimens

• Cold chain requirements may be difficult to implement

▪ Build on evidence from phase 3 clinical trials including VE for

• Key subpopulations

• Severe disease

• SARS-CoV-2 infection and transmission

• Duration of protection
VE policy priorities: Results of internal and external input

Immediate

First 2-4 months

• Does vaccine protect against symptomatic disease as expected?

Subsequent • VE against key outcomes

• Severe disease

• Non-severe disease 

• SARS-CoV-2 infection and transmission

• VE in key subpopulations

• Adults aged ≥65 years, including those in long-term care facilities (LTCF) 

• People with key underlying conditions (e.g., immunocompromised, obesity, diabetes)

• Disproportionately affected racial/ethnic populations (Black, Latinx, Native American/ 

Alaska Native)

• VE for regimen-related questions

• Single dose and prolonged dosing intervals; Mixed dose schedules (>1 product) 

Later stage • Viral evolution: Do genome changes threaten VE?

• Duration of protection

• Comparative VE: Is one product better than another?
Challenges for observational COVID-19 VE studies

▪ Decision to be vaccinated may correlate with risk of disease

▪ Prior infection may bias estimate

▪ Imperfect laboratory testing poses a risk of misclassification

▪ COVID-19 epidemiology is highly dynamic

▪ Multiple products are in use simultaneously 
VE priority Prospective data collection

Electronic health record (EHR) and claims 

analyses (coordination across US government)

Immediate priority

Does vaccine work as expected to 

prevent symptomatic disease?

Test-negative design case-control among healthcare 

personnel 

Subsequent priorities

Older adults, including residents of 

long-term care facilities (LTCF) 

Case-control among adults ≥65 years (COVID-NET linked 

to CMS); National Healthcare Safety Network

CMS cohort (FDA, CMS)

EHR datasets (CDC, VA, FDA)

Infection and transmission Prospective longitudinal cohorts, including among 

healthcare personnel & frontline workers; caseascertained household cohorts for transmission

Severe disease/hospitalization Test-negative design (for adults and children); 

conventional case-control using hospitalized controls; 

screening method

EHR datasets (CDC, VA, FDA): Retrospective 

cohort or test-negative design

Non-severe disease Test-negative design among outpatients Potentially using EHR datasets

Those with key underlying conditions 

(e.g., immunocompromised)

Captured in above studies CMS (FDA,CMS); EHR datasets (CDC, VA, FDA)

Disproportionately affected 

racial/ethnic groups 

Captured in above studies; test-negative design in 

American Indian/Alaska Native population 

CMS (FDA, CMS); EHR datasets (CDC, VA, FDA); 

Exploring IHS EHR (IHS)

Vaccine impact Ecologic analyses of disease incidence/seroprevalence and vaccine coverage; comparisons of expected 

vaccine impact from models with observed impact

Currently planned COVID-19 VE assessments
VE priority Prospective data collection

Electronic health record (EHR) and claims 

analyses (coordination across US government)

Immediate priority

Does vaccine work as expected to 

prevent symptomatic disease?

Test-negative design case-control among healthcare 

personnel 

Subsequent priorities

Older adults, including residents of 

long-term care facilities (LTCF) 

Case-control among adults ≥65 years (COVID-NET linked 

to CMS); National Healthcare Safety Network

CMS cohort (FDA, CMS)

EHR datasets (CDC, VA, FDA)

Infection and transmission Prospective longitudinal cohorts, including among 

healthcare personnel & frontline workers; caseascertained household cohorts for transmission

Severe disease/hospitalization Test-negative design (for adults and children); 

conventional case-control using hospitalized controls; 

screening method

EHR datasets (CDC, VA, FDA): Retrospective 

cohort or test-negative design

Non-severe disease Test-negative design among outpatients Potentially using EHR datasets

Those with key underlying conditions 

(e.g., immunocompromised)

Captured in above studies CMS (FDA,CMS); EHR datasets (CDC, VA, FDA)

Disproportionately affected 

racial/ethnic groups 

Captured in above studies; test-negative design in 

American Indian/Alaska Native population 

CMS (FDA, CMS); EHR datasets (CDC, VA, FDA); 

Exploring IHS EHR (IHS)

Vaccine impact Ecologic analyses of disease incidence/seroprevalence and vaccine coverage; comparisons of expected 

vaccine impact from models with observed impact

Currently planned COVID-19 VE assessments
Assessing VE among healthcare personnel (HCP)

▪ Prospective test-negative design among HCP

• Enroll HCP who are tested for COVID-19

• Cases are test positive & controls are test negative

▪ Objectives:

• Primary: evaluate VE of a complete schedule of COVID-19 vaccine against 

laboratory-confirmed symptomatic COVID-19 (by product if feasible)

• Secondary will include VE by number of doses received, if feasible

▪ Timeline: January launch and enroll until sites have reached vaccine 

coverage >80% (2 doses)
VE assessment is being conducted across 26 states in 

34 sites with >500,000 HCP.
VE priority Prospective data collection

Electronic health record (EHR) and claims 

analyses (coordination across US government)

Immediate priority

Does vaccine work as expected to 

prevent symptomatic disease?

Test-negative design case-control among healthcare 

personnel 

Subsequent priorities

Older adults, including residents of 

long-term care facilities (LTCF) 

Case-control among adults ≥65 years (COVID-NET linked 

to CMS); National Healthcare Safety Network

CMS cohort (FDA, CMS)

EHR datasets (CDC, VA, FDA)

Infection and transmission Prospective longitudinal cohorts, including among 

healthcare personnel & frontline workers; caseascertained household cohorts for transmission

Severe disease/hospitalization Test-negative design (for adults and children); 

conventional case-control using hospitalized controls; 

screening method

EHR datasets (CDC, VA, FDA): Retrospective 

cohort or test-negative design

Non-severe disease Test-negative design among outpatients Potentially using EHR datasets

Those with key underlying conditions 

(e.g., immunocompromised)

Captured in above studies CMS (FDA,CMS); EHR datasets (CDC, VA, FDA)

Disproportionately affected 

racial/ethnic groups 

Captured in above studies; test-negative design in 

American Indian/Alaska Native population 

CMS (FDA, CMS); EHR datasets (CDC, VA, FDA); 

Exploring IHS EHR (IHS)

Vaccine impact Ecologic analyses of disease incidence/seroprevalence and vaccine coverage; comparisons of expected 

vaccine impact from models with observed impact

Currently planned COVID-19 VE assessments
Assessing VE among adults aged ≥65 years

▪ CDC-led case-control assessment linking hospitalized cases from COVID-NET 

with CMS data

▪ FDA-led cohort analysis of CMS claims data

▪ Both will conduct separate analyses for adults ≥65 years who reside in

• Community

• Long-term care facilities (LTCF)
Assessing VE among residents of LTCF

▪ Data from the National Healthcare Safety Network (NHSN) LTCF surveillance 

and vaccine coverage modules 

• Weekly aggregate counts at the facility level of

• New laboratory-confirmed COVID-19 cases 

• Vaccination status among all residents and among cases

• Plan to calculate weekly attack rates among vaccinated and unvaccinated

▪ Data will be available starting early February

• Initial analysis will require at least 8 weekly transmissions from LCTF with 

at least 50% vaccine coverage

• Plans for ongoing analyses
Assessing vaccine impact in LTCF

▪ Ecologic analyses of vaccine coverage and COVID-19 disease rates among 

residents of LTCF

▪ Outbreak descriptive analyses before and after vaccine use
VE priority Prospective data collection

Electronic health record (EHR) and claims 

analyses (coordination across US government)

Immediate priority

Does vaccine work as expected to 

prevent symptomatic disease?

Test-negative design case-control among healthcare 

personnel 

Subsequent priorities

Older adults, including residents of 

long-term care facilities (LTCF) 

Case-control among adults ≥65 years (COVID-NET linked 

to CMS); National Healthcare Safety Network

CMS cohort (FDA, CMS)

EHR datasets (CDC, VA, FDA)

Infection and transmission Prospective longitudinal cohorts, including among 

healthcare personnel & frontline workers; caseascertained household cohorts for transmission

Severe disease/hospitalization Test-negative design (for adults and children); 

conventional case-control using hospitalized controls; 

screening method

EHR datasets (CDC, VA, FDA): Retrospective 

cohort or test-negative design

Non-severe disease Test-negative design among outpatients Potentially using EHR datasets

Those with key underlying conditions 

(e.g., immunocompromised)

Captured in above studies CMS (FDA,CMS); EHR datasets (CDC, VA, FDA)

Disproportionately affected 

racial/ethnic groups 

Captured in above studies; test-negative design in 

American Indian/Alaska Native population 

CMS (FDA, CMS); EHR datasets (CDC, VA, FDA); 

Exploring IHS EHR (IHS)

Vaccine impact Ecologic analyses of disease incidence/seroprevalence and vaccine coverage; comparisons of expected 

vaccine impact from models with observed impact

Currently planned COVID-19 VE assessments
Assessing VE against infection and transmission

▪ Leverage ongoing cohort of >5000 HCP and first-responders

• Weekly testing for SARS-CoV-2 infection

• Assessment of secondary transmission among household members

• Cohort began in July 2020 and will continue through March 2022

▪ Working to expand case-ascertained household transmission studies to 

general population during widespread adult vaccination
VE priority Prospective data collection

Electronic health record (EHR) and claims 

analyses (coordination across US government)

Immediate priority

Does vaccine work as expected to 

prevent symptomatic disease?

Test-negative design case-control among healthcare 

personnel 

Subsequent priorities

Older adults, including residents of 

long-term care facilities (LTCF) 

Case-control among adults ≥65 years (COVID-NET linked 

to CMS); National Healthcare Safety Network

CMS cohort (FDA, CMS)

EHR datasets (CDC, VA, FDA)

Infection and transmission Prospective longitudinal cohorts, including among 

healthcare personnel & frontline workers; caseascertained household cohorts for transmission

Severe disease/hospitalization Test-negative design (for adults and children); 

conventional case-control using hospitalized controls; 

screening method

EHR datasets (CDC, VA, FDA): Retrospective 

cohort or test-negative design

Non-severe disease Test-negative design among outpatients Potentially using EHR datasets

Those with key underlying conditions 

(e.g., immunocompromised)

Captured in above studies CMS (FDA,CMS); EHR datasets (CDC, VA, FDA)

Disproportionately affected 

racial/ethnic groups 

Captured in above studies; test-negative design in 

American Indian/Alaska Native population 

CMS (FDA, CMS); EHR datasets (CDC, VA, FDA); 

Exploring IHS EHR (IHS)

Vaccine impact Ecologic analyses of disease incidence/seroprevalence and vaccine coverage; comparisons of expected 

vaccine impact from models with observed impact

Currently planned COVID-19 VE assessments
Assessing VE for disease severity and key populations

▪ Test-negative design and conventional case-control with hospitalized 

controls 

• Designed to assess severe disease/hospitalization and non-severe disease

• Sites selected to include populations with underlying health conditions, 

racial/ethnic groups disproportionately affected by COVID-19, and 

American Indian/Alaska Native populations

▪ Screening method analyses

▪ EHR and claims-based assessments
VE priority Prospective data collection

Electronic health record (EHR) and claims 

analyses (coordination across US government)

Immediate priority

Does vaccine work as expected to 

prevent symptomatic disease?

Test-negative design case-control among healthcare 

personnel 

Subsequent priorities

Older adults, including residents of 

long-term care facilities (LTCF) 

Case-control among adults ≥65 years (COVID-NET linked 

to CMS); National Healthcare Safety Network

CMS cohort (FDA, CMS)

EHR datasets (CDC, VA, FDA)

Infection and transmission Prospective longitudinal cohorts, including among 

healthcare personnel & frontline workers; caseascertained household cohorts for transmission

Severe disease/hospitalization Test-negative design (for adults and children); 

conventional case-control using hospitalized controls; 

screening method

EHR datasets (CDC, VA, FDA): Retrospective 

cohort or test-negative design

Non-severe disease Test-negative design among outpatients Potentially using EHR datasets

Those with key underlying conditions 

(e.g., immunocompromised)

Captured in above studies CMS (FDA,CMS); EHR datasets (CDC, VA, FDA)

Disproportionately affected 

racial/ethnic groups 

Captured in above studies; test-negative design in 

American Indian/Alaska Native population 

CMS (FDA, CMS); EHR datasets (CDC, VA, FDA); 

Exploring IHS EHR (IHS)

Vaccine impact Ecologic analyses of disease incidence/seroprevalence and vaccine coverage; comparisons of expected 

vaccine impact from models with observed impact

Currently planned COVID-19 VE assessments
Assessing vaccine impact

▪ Ecologic analyses of the association of disease incidence and/or 

seroprevalence with vaccine coverage 

▪ Comparisons of expected vaccine impact from models with actual observed 

impact
Assessing VE for regimen-related questions

▪ Single dose 

▪ Prolonged dosing intervals (>3-4 weeks) 

▪ Mixed dose schedules (>1 product) 

▪ All platforms except NHSN will collect individual-level information on dose 

dates and type

▪ Best opportunity may come from large prospective, EHR, and claims 

assessments among the general adult population
Do viral genome changes threaten VE?

▪ Prospective platforms for general adult population will collect specimens 

from cases, where possible, for whole genome sequencing

• Will not be performed in real time

• May not be powered for variant-specific VE assessments

▪ A separate team in the vaccine evaluation unit is dedicated to assessing 

vaccine breakthrough cases

▪ Work is part of broader CDC efforts to monitor the impact of SARS-CoV-2 

variants 
Assessing VE among children and pregnant women

▪ Children

• Planning a prospective test-negative design assessment to evaluate VE 

against COVID-19 hospitalizations

• Leveraging an existing surveillance network of approximately 20-40 

sites for pediatric COVID-19 hospitalizations and multisystem 

inflammatory syndrome in children (MIS-C)

• EHR and claims database analyses will be used to estimate VE in children

▪ Pregnant women

• Exploring EHR cohort and prospective case-control VE assessments
Conclusions

▪ Urgent need for VE data to guide vaccine policy

▪ VE portfolio leverages multiple platforms, data sources, and methods

▪ Early VE assessments will focus on healthcare personnel and residents of 

LTCF 

▪ Portfolio will continue to evolve as more information from Phase 3 trials 

and real-world evidence become available
For more information, contact CDC

1-800-CDC-INFO (232-4636)

TTY: 1-888-232-6348 www.cdc.gov

The findings and conclusions in this report are those of the authors and do not necessarily represent the 

official position of the Centers for Disease Control and Prevention.

Questions?