For more information: www.cdc.gov/COVID19

EtR Framework: 

Public Health Problem, Resource Use 

and Equity Domains 

ACIP COVID-19 Vaccines

Sara Oliver MD, MSPH

ACIP Meeting

November 23, 2020
Evidence to Recommendations Framework
 Structure to describe information considered in moving from evidence

to ACIP vaccine recommendations

 Provide transparency around the impact of additional factors on 

deliberations when considering a recommendation 

Evidence to Recommendations (EtR) Framework

3
ACIP Pathway to Recommendation

FDA approval 

-Licensure

-Emergency use 

Authorization

-Expanded 

Access

Should COVID-19 

vaccine be 

recommended?

To whom should 

early allocation of 

COVID-19 vaccine 

be recommended?

Evidence to 

Recommendation 

Framework

GRADE 

Scientific Evidence 

Ethical Principles

Implementation 

ACIP 

RECOMMENDATION 

ACIP 

RECOMMENDATION 
ACIP Pathway to Recommendation

FDA approval 

-Licensure

-Emergency use

Authorization

-Expanded 

Access

Should COVID-19 

vaccine be 

recommended?

To whom should 

early allocation of 

COVID-19 vaccine 

be recommended?

Evidence to 

Recommendation 

Framework

GRADE 

Scientific Evidence 

Ethical Principles

Implementation 

ACIP 

RECOMMENDATION 

ACIP 

RECOMMENDATION 
Evidence to Recommendations (EtR) Framework

6

EtR DomainQuestion

Public Health Problem • Is the problem of public health importance?

Benefits and Harms

• How substantial are the desirable anticipated effects?

• How substantial are the undesirable anticipated effects?

• Do the desirable effects outweigh the undesirable effects?

Values

• Does the target population feel the desirable effects are large relative to the

_ undesirable effects?

• Is there important variability in how patients value the outcomes?

Acceptability • Is the intervention acceptable to key stakeholders?

Feasibility • Is the intervention feasible to implement?

Resource Use • Is the intervention a reasonable and efficient allocation of resources?

Equity • What would be the impact of the intervention on health equity?
Evidence to Recommendations (EtR) Framework

7

EtR DomainQuestion

Public Health Problem • Is the problem of public health importance?

Benefits and Harms

• How substantial are the desirable anticipated effects?

• How substantial are the undesirable anticipated effects?

• Do the desirable effects outweigh the undesirable effects?

Values

• Does the target population feel the desirable effects are large relative to the

_ undesirable effects?

• Is there important variability in how patients value the outcomes?

Acceptability • Is the intervention acceptable to key stakeholders?

Feasibility • Is the intervention feasible to implement?

Resource Use • Is the intervention a reasonable and efficient allocation of resources?

Equity • What would be the impact of the intervention on health equity?

“The vaccine” or “The intervention” = COVID-19 vaccine ‘X’

“The problem” = COVID-19 disease
Evidence to Recommendations (EtR) Framework

8

EtR Domain

Public Health Problem

Benefits and Harms Await Phase III clinical trial data

Values

Acceptability

Feasibility

Resource Use

Equity
Evidence to Recommendations (EtR) Framework

9

EtR Domain

Public Health Problem

Benefits and Harms

Values

Acceptability

Feasibility

Resource Use

Equity New addition to EtR Framework
Evidence to Recommendations (EtR) Framework

10

EtR Domain

Public Health ProblemNot impacted by individual vaccine characteristics

Benefits and Harms

Values

Acceptability

Feasibility

Resource UseMinimal impact currently by individual vaccine characteristics

Equity New addition to EtR Framework

Impacted by individual vaccine characteristics
Evidence to Recommendations (EtR) Framework

11

EtR Domain

Public Health Problem

Benefits and Harms

ValuesImpacted by individual vaccine characteristics

AcceptabilityImpacted by individual vaccine characteristics

FeasibilityImpacted by individual vaccine characteristics

Resource Use

Equity
 Presentations today will focus on current evidence and Work Group 

discussions around each EtR Domain for future COVID-19 vaccines

– Areas where EtR Domain judgment may vary by individual vaccine 

characteristics will be identified 

 No vote today: Once Phase III clinical trial data and an FDA decision are 

available, EtR framework for specific vaccine will be presented

 It is expected that information will continue to evolve: EtR Framework 

and Recommendations continually evaluated, updated as needed 

Evidence to Recommendations (EtR) Framework

12
EtR Domain: Public Health Problem
○ ○ ○ ○ ○ ○

Public Health Problem

Is COVID-19 disease of public health importance?

- Are the consequences of COVID-19 serious?

- Is COVID-19 urgent?

- Are a large number of people affected by COVID-19?

- Are there populations disproportionately affected by COVID-19? 

No Probably no Probably yes Yes Varies Don't know
0

20000

40000

60000

80000

100000

120000

140000

160000

180000

200000

Jan 22 2020 Mar 7 2020 Apr 21 2020 Jun 5 2020 Jul 20 2020 Sep 3 2020 Oct 18 2020

Number of COVID-19 Cases

Public Health Problem:

Review of the available evidence

January 21 – November 22

https://covid.cdc.gov/covid-data-tracker/#trends_dailytrendscases 15
 Hospitalization

–

–

–

–

Cumulative hospitalization rate between March 1 and November 14, 2020 was 

228.7 per 100,000 population

Among those hospitalized, 32% required care in an intensive care unit and 15% died

 Mortality

As of November 22, 2020, there were 255,076 COVID-19-associated deaths reported 

in the United States

Estimates of the SARS-CoV-2 infection fatality ratio range from 0.5% to 1.4%

16

Public Health Problem:

Summary of the available evidence

https://gis.cdc.gov/grasp/COVIDNet/COVID19_3.html . 

https://gis.cdc.gov/grasp/COVIDNet/COVID19_5.html . 

Hauser, A. et al. Estimation of SARS-CoV-2 mortality during the early stages of an epidemic: a modeling study in Hubei, China, and six regions in Europe. PLoS medicine, 17(7), p.e1003189

Yang, W. et al. Estimating the infection-fatality risk of SARS-CoV-2 in New York City during the spring 2020 pandemic wave: a model-based analysis. Lancet Infect Dis. 2020 

DOI:https://doi.org/10.1016/S1473-3099(20)30769-6 
Public Health Problem:

Work Group Interpretation

Is COVID-19 disease of public health importance? 

○ No ○ Probably no ○ Probably yes ○ Yes ○ Varies ○ Don't know
EtR Domain: Resource Use
Resource Use

Is COVID-19 vaccine 'X' a reasonable and efficient 

_allocation of resources? 

- What is the cost-effectiveness of COVID-19 vaccine ‘X’?

- How does the cost-effectiveness of COVID-19 vaccine ‘X’ change in response to 

_______changes in context, assumptions, etc? 

○ No ○ Probably no ○ Probably yes ○ Yes ○ Varies ○ Don't know
Resource Use:

Review of the available evidence

20

 Work Group reviewed estimates of economic costs related to COVID-19 

vaccinations, disease outcomes and disease mitigation activities
21

Resource Use:

Summary of the available evidence

 If 20% of the U.S. population is infected with COVID-19, the direct medical costs 

could be $163 billion1

 Health-related costs (including premature deaths, long-term health impairment 

and mental health impairment) have been estimated at $8.5 trillion2

Costs associated with COVID-19 disease

1. Bartsch et al. 2020. Health Affairs “The Potential Health Care Costs And Resource Use Associated With COVID-19 In The United States”.

2. Cutler and Summers. 2020. JAMA. “The COVID-19 pandemic and the $16 trillion virus.”
22

Resource Use:

Summary of the available evidence

 If 20% of the U.S. population is infected with COVID-19, the direct medical costs 

could be $163 billion 

 Health-related costs (including premature deaths, long-term health impairment 

and mental health impairment) have been estimated at $8.5 trillion

Costs associated with COVID-19 disease

Costs associated with COVID-19 vaccines

 U.S. Government has committed $10 billion to Operation Warp Speed for the 

provision of vaccines1

 Vaccine doses purchased with U.S. taxpayer dollars will be given to the American 

people at no cost2

1. https://www.hhs.gov/about/news/2020/05/15/trump-administration-announces-framework-and-leadership-for-operation-warp-speed.html

2. https://www.cdc.gov/coronavirus/2019-ncov/vaccines/faq.html
 No published cost-effectiveness analyses currently available

 Precise cost-effectiveness analysis and economic impact of vaccination 

depend on number of factors that are currently unknown:

–

–

–

Duration of vaccine protection

Vaccination coverage levels

Implementation costs associated with large vaccination program

 The Work Group concluded that cost-effectiveness may not be a primary 

driver for decision-making during a pandemic and for vaccine used under EUA 

– Will need to be reassessed for future recommendations

23

Resource Use:

Work Group Interpretation
Resource Use:

Work Group Interpretation

Is COVID-19 vaccine ‘X’ a reasonable and efficient 

_allocation of resources?

○ No ○ Probably no ○ Probably yes ○ Yes ○ Varies ○ Don't know
EtR Domain: Equity 
Equity 

What would be the impact of COVID-19 vaccine ‘X’ on 

health equity? 

- Are there groups or settings that might be disadvantaged in relation to COVID-19 disease 

_____burden or receipt of COVID-19 vaccine ‘X’? 

- Are there considerations that should be made when implementing the COVID-19 vaccine 

____‘X’ program to ensure that inequities are reduced whenever possible, and that they are 

____ not increased? 

○ Reduced ○ Probably reduced ○ Probably no impact

○ Probably increased ○ Increased ○ Varies ○ Don't know
Equity:

Review of the available evidence

27

 Identification of groups that might be disadvantaged in relation to COVID-19 

disease burden or receipt of COVID-19 vaccine ‘X’

–

–

–

–

PROGRESS-Plus1: Place of residence, race or ethnicity, occupation, gender or sex, religion, 

education, socioeconomic status, social capital, disability, other

 Review of the scientific and gray literature

 Review of CDC COVID-19 response data and resources

CDC COVID Data Tracker & COVID-19-Associated Hospitalization Surveillance Network (COVID-NET)

National Center for Health Statistics

COVID-19 Disproportionately Affected Populations Team critical populations review

1 PROGRESS-Plus is an acronym to identify factors associated with unfair differences in disease burden and the potential for interventions to reduce these differential effects. 

O'Neill J, et al. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J Clin Epi. 

2014;67: 56-64; Cochrane methods. Equity. https://methods.cochrane.org/equity/projects/evidence-equity/progress-plus
28

Equity:

Summary of the available evidence

 People from racial and ethnic minority populations

–

–

–

–

Represent 40% of U.S. population, but 50% of COVID-19 cases and 45% of COVID-19 deaths1

Age-adjusted COVID-19 hospitalization rates approximately 4 times higher among racial and 

ethnic minority groups, compared to non-Hispanic White persons2

Inequities in social determinants of health put racial and ethnic minority groups at increased 

risk of COVID-19 disease, including discrimination, lack of healthcare access, 

overrepresentation among essential workers, low-income, and crowded housing3

 People living in poverty or with high social vulnerability

COVID-19 cumulative case rate per 100,000 population is 1.5 times higher in high versus low 

poverty counties and 1.3 times higher in counties with high versus low social vulnerability4

1https://www.cdc.gov/covid-data-tracker/index.html#demographics , as of Nov 17, 2020 

2https://www.cdc.gov/coronavirus/2019-ncov/covid-data/covidview/index.html and COVID-NET Surveillance, as of Nov 13, 2020

3CDC Health Equity: https://www.cdc.gov/coronavirus/2019-ncov/community/health-equity/race-ethnicity.html

1 https://covid.cdc.gov/covid-data-tracker/ , as of Nov 17, 2020 
29

Equity: summary of the available evidence

 Essential workers

–

–

–

–

–

–

–

Large COVID-19 outbreaks have been reported in multiple essential industries (food and 

agriculture, manufacturing, construction, wholesale trade)1

Racial and ethnic minority populations disproportionately represented in subsets of essential 

industries2 and almost one quarter live in low-income families3

 Persons from other disadvantaged groups

Justice-involved: COVID-19 case rate for persons in federal/state correctional facilities 5.5 times 

higher than U.S. population4

Homelessness: Shelters associated with rapid spread/outbreaks of SARS-CoV-25

Disabilities: COVID-19 case and mortality rates ~4 and ~8 times higher, respectively, for NY group 

home residents with intellectual/developmental disabilities compared to NY State residents6

Substance use disorder (SUD): Analysis of e-health records found patients diagnosed with SUD in 

past year had 8-fold increased risk of COVID-19 diagnosis compared with non-SUD patients7

Sexual and gender minorities: Face social or structural inequities that can lead to health disparities8

1Bui DP, et al. MMWR 2020;69:1133–1138. DOI: http://dx.doi.org/10.15585/mmwr.mm6933e3external icon. 2Waltenburg MA, et al. Emerg Infect Dis. 2021 Jan. https://doi.org/10.3201/eid2701.203821https://wwwnc.cdc.gov/eid/article/27/1/20-3821_article 3 American Community Survey, 2011-2015: https://cepr.net/a-basic-demographic-profile-of-workers-in-frontline-industries 4Saloner B, et al. JAMA. 2020;324(6):602–603. doi:10.1001/jama.2020.12528 5Tobolowsky FA, et al. MMWR 2020;69:523–526. DOI: http://dx.doi.org/10.15585/mmwr.mm6917e2; 6Landes SD, et al. Disabil Health J 2020;13:1-5. https://doi.org/10.1016/j.dhjo.2020.100969 7Wang QQ, et al. Mol Psychiatry. 2020 Sep 14:1–10. doi: 10.1038/s41380-020-00880-7. 8 https://www.cdc.gov/lgbthealth/about.htm
 Although COVID-19 vaccines will be provided at no cost, personal investments 

in time and travel to obtain vaccine may be a barrier for some groups

 Characteristics of specific vaccines (e.g. storage and handling requirements) 

have potential to impact equitable distribution of COVID-19 vaccines

– The Work Group had different assessments for the impact on health equity for 

different vaccines

30

Equity:

Additional considerations 
 Are there considerations that should be made when implementing the COVID-19 

vaccine ‘X’ program to ensure inequities are reduced whenever possible, and that they 

are not increased?

–

–

–

–

Identify groups disproportionately affected by COVID-19 or who face health inequities 

Undertake focused outreach and education 

Identify and address barriers to vaccination

Conduct active follow-up of disadvantaged groups to ensure completion of a 2-dose series; 

consider one-dose COVID-19 vaccines for groups where follow-up may be difficult 

31

Equity:

Additional information questions 
 Successful implementation of the COVID-19 vaccination program and 

confidence in COVID-19 vaccines are pivotal to reducing health inequities

“...increasing the availability of an effective intervention within a country or 

region is not necessarily enough to reduce inequities. The intervention has to be 

accessible, acceptable, effective in, and used by the most disadvantaged groups 

within that population to be truly effective at reducing inequities in health”.1

1O'Neill J, Tabish H, Welch V, et al. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to 

illuminate inequities in health. J Clin Epidemiol. 2014; 67: 56-64.

32

Equity:

Summary
Equity: 

Work Group Interpretation 

What would be the impact of COVID-19 vaccine ‘X’ on 

health equity? 

○ Reduced ○ Probably reduced ○ Probably no impact ○ Probably increased

○ Increased ○ Varies ○ Don't know
Summary 
Summary:

35

EtR DomainQuestion Work Group Judgments

Public Health Problem Is COVID-19 disease of public health importance? Yes

Resource Use Is COVID-19 vaccine ‘X’ a reasonable and efficient 

allocation of resources?

Yes

Equity What would be the impact of COVID-19 vaccine ‘X’ 

on health equity?

Probably reduced/

Probably increased*

*Judgment differed by COVID-19 vaccine
Questions for ACIP:

36

 Public Health Problem:

–

–

Does ACIP agree that COVID-19 disease is a public health problem? 

Any additional information that ACIP needs to see before a vote?
Questions for ACIP:

37

 Resource Use:

–

–

Does ACIP agree that COVID-19 vaccines are a reasonable/efficient allocation of 

resources? 

Any additional information that ACIP needs to see before a vote?
Questions for ACIP:

38

 Equity:

–

–

Does ACIP agree with the WG conclusions for the Equity domain (acknowledging 

the conclusions depend on the individual vaccine characteristics)?

Any additional information that ACIP needs to see before a vote?
For more information, contact CDC

1-800-CDC-INFO (232-4636)

TTY: 1-888-232-6348 www.cdc.gov

The findings and conclusions in this report are those of the authors and do not necessarily represent the 

official position of the Centers for Disease Control and Prevention.

Thank you