For more information: www.cdc.gov/COVID19

EtR Framework: 

Values, Acceptability and Feasibility 

Domains 

ACIP COVID-19 Vaccines

Sara Oliver MD, MSPH

ACIP Meeting

November 23, 2020
Evidence to Recommendations (EtR) Framework

2

EtR Domain

Public Health Problem

Benefits and Harms

ValuesImpacted by individual vaccine characteristics

AcceptabilityImpacted by individual vaccine characteristics

FeasibilityImpacted by individual vaccine characteristics

Resource Use

Equity
EtR Domain: Values
○ No

Values

Criteria 1: 

Does the target population feel that the desirable 

effects are large relative to undesirable effects? 

-How does the target population view the balance of desirable versus undesirable effects?

-Would patients feel that the benefits outweigh the harms and burden?

-Does the population appreciate and value COVID-19 vaccine ‘X’? 

○ Probably no ○ Probably yes ○ Yes ○ Varies ○ Don't know
Values

Criteria 2: 

Is there important uncertainty about, or variability in, 

how much people value the main outcomes? 

-How much do individuals value each outcomes in relation to the other outcomes?

-Is there evidence to support those value judgments? 

-Is there evidence that the variability is large enough to lead to different decisions? 

○

○

○

○

○

Important uncertainty or variability

Probably important uncertainty or variability 

Probably not important uncertainty or variability 

No important uncertainty or variability 

No known undesirable outcomes 
Values:

Review of the available evidence

6

 Review of scientific literature

– Databases: Medline, Embase, Psycinfo, Global Health Ovid, CINAHL, ProQuest Coronavirus 

Research, Scopus, WHO COVID-19

– Search terms: SARS-CoV-2/COVID-19 string; vaccine string; intent, confidence, hesitancy, attitude, 

belief, accept, choice, decision, refusal

– Last search date: November 17, 2020

 Inclusion criteria

– Data collection in 2020 related to COVID-19 vaccine beliefs, attitudes, and intentions

 Review of scientific articles: 272 results, 14 papers included

 Review of news media and reports (Google): 10 sources included

 Preliminary findings from CDC vaccine intent survey and focus group discussions
 Overall acceptability of a COVID-19 vaccine was moderate1

– Proportion intending to receive vaccine ranged across surveys: 42-86%

– Attitudes towards Pfizer vaccine with news reports of 90% efficacy: 71% believed effective, 68% safe

 Many reported anticipated benefits of vaccination1

– Protect self, family, and community

– Prevent SARS-CoV-2 infection and severe illness

– Return to normalcy

 Vaccination intentions varied by time, population, and vaccine characteristics1

– Large national survey found decline from 72% in May to 51% in September2

– Acceptance lowest among Black respondents, highest among Asian respondents

– Acceptance greater with higher socioeconomic status 

– Acceptance greater with history of influenza vaccination and higher COVID-19 risk perception

– Acceptance greater with higher vaccine efficacy and healthcare provider recommendation 

Values:

Summary of the available evidence

2. Pew Research Center, 17 Sep 2020: https://www.pewresearch.org/science/wp-content/uploads/sites/16/2020/09/PS_2020.09.17_COVID-19-Vaccine_FINAL.pdf 7

1. APNORC; Harris; Fisher Ann Intern Med.; ICF; Kreps JAMA Netw Open.; Lazarus Nature Med.; Malik EClinicalMedicine.; Pogue Vaccines.; Reiter Vaccine.; Thunstrom SSRN. 
0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Mar Apr May Jun Jul Aug Sep Oct Nov

% Reporting Positive Vaccine Intentions*

COVID-19 Vaccination Intentions Varied by Survey Month

*Positive vaccine intentions includes persons reporting definitely, probably, or somewhat likely to get vaccinated. 

Ref Date N % Intent

Romer Mar 1,050 82%

Fisher Apr 991 58%

Earnshaw Apr 845 86%

Southwell Apr 2,279 75%

Roozenbeek Apr 700 75%

Hogan Apr 101 74%

Malik May 672 67%

Taylor May 1,772 75%

Reiter May 2,006 69%

APNORC May 1,056 49%

ICF May 1,000 63%

Pew May 10,957 72%

CUNY May 1,999 74%

Head May 3,159 66%

Lazarus Jun 773 75%

ICF Jun 1,000 63%

Perlis Jul 19,027 66%

Romer Jul 840 72%

Pogues Aug 316 69%

KFF Sep 1,199 42%

Pew Sep 10,093 51%

Harris Sep 1,971 54%

Gallup Oct 2,985 58%

IPSOS Oct 3,541 62%

USC Nov 2,703 63%

Harris Nov 1,963 60%
COVID-19 Vaccination Intentions Varied by Race/ethnicity 

72%

81%

72%

64%

68%

70%

56%

64%

52%

44%

68%

74%

37%

63%

56%

39%

40%

55%

25%

57%

32%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Fisher, APR (991)

Malik, MAY (672)

Reiter, MAY (2006)

APNORC, MAY (1056)

ICF, MAY (1000)

Pew, SEP (10093)

Black Hispanic White Asian

*Positive vaccine intentions includes persons reporting definitely, probably, or somewhat likely to get vaccinated.
 Common reasons for not intending to get vaccinated included1:

– Concern for vaccine side effects

– Uncertainty of vaccine efficacy

– Low risk perception of COVID-19 or severe disease

 Vaccine efficacy (90% or 70%) associated with preferred choice of hypothetical 

vaccine2

 Focus groups (49, n=239): most are open to vaccine, but many prefer not to be first3

 Many reported concerns that COVID-19 vaccine approval process was too fast1

 Limitations

– Surveys conducted prior to vaccine available

– Convenience samples may not be representative

Values:

Summary of the available evidence

10 1. Pew Research Center, 17 Sep 2020: https://www.pewresearch.org/science/wp-content/uploads/sites/16/2020/09/PS_2020.09.17_COVID-19-Vaccine_FINAL.pdf

2. Kreps et al. JAMA Netw Open. 20 Oct 2020. 3. Jorgenson C. CDC Presentation to ACIP Working Group. 3 Sep 2020.
11

 Across national surveys, many adults reported intentions to receive COVID-19 

vaccine. 

– Common desirable effects included protecting self, family, community from SARSCoV-2 infection and severe illness and return to normalcy. 

– Common concerns included vaccine side effects, uncertainty of vaccine efficacy, 

and speed of vaccine approval process.

 Vaccination intentions varied substantially by race or ethnicity and 

socioeconomic status of respondents.

Values:

Summary of the available evidence
12

 Strategies to consider for overcoming barriers to vaccine acceptance: 

– Engage trusted sources (e.g., social workers, faith leaders, community leaders, 

advocacy groups, facility administrators, union representatives) 

– Develop communication materials that are ADA-compliant and culturally, 

linguistically, and literacy appropriate

– Ensure providers have information on vaccine recommendations to counsel 

patients

– Educate throughout jurisdiction about vaccination recommendations and where 

to refer patients for free COVID-19 vaccination

– Educate non-clinical facility administrators

Values

CDC. Draft COVID-19 Vaccination Implementation Planning Reference Material for Critical Populations.
○ ○ ○ ○ ○ ○

Values: Work Group Interpretation

Criteria 1: 

Does the target population feel that the desirable 

effects are large relative to undesirable effects? 

No Probably no Probably yes Yes Varies Don't know
Values: Work Group Interpretation

Criteria 2: 

Is there important uncertainty about, or variability in, 

how much people value the main outcomes? 

○

○

○

○

○

Important uncertainty or variability

Probably important uncertainty or variability 

Probably not important uncertainty or variability 

No important uncertainty or variability 

No known undesirable outcomes 
EtR Domain: Acceptability
Acceptability

Is COVID-19 vaccine ‘X’ acceptable to key 

__stakeholders? 

- Are there key stakeholders that would not accept the distribution of benefits and harms? 

_ - Are there key stakeholders that would not accept the undesirable effects in the short term 

_ for the desirable effects (benefits) in the future? 

○ No ○ Probably no ○ Probably yes ○ Yes ○ Varies ○ Don't know
 Review of scientific literature

 Preliminary findings from CDC evaluations of COVID-19 vaccine attitudes

– Survey with State Health Officers (n=34)

– Focus group discussions with nurses (7 focus groups)

– National online survey: sub-group analysis for healthcare providers (n=216)

 Review of news media, professional society and workers’ unions websites

– AAFP, AFT, AFSCME, AGS, ANA, AMA, IDSA, SEIU

– American Nurses Foundation (ANF) survey (n=12,939)

 Consideration of programmatic, financial, and ethical aspects

– State/jurisdiction and partner planning for vaccine implementation

– Anticipated out-of-pocket costs

Acceptability:

Review of the available evidence

17
 No published provider knowledge, attitudes, and practices surveys

 CDC evaluations

– State health officers, Oct: concerns with rollout included vaccine hesitancy (53%), vaccine safety 

(32%), and communications (26%)1

– Focus groups with nurses (n=7), Jun-Aug: most supported prioritizing nurses, some reluctant to get 

vaccinated, and many do not want to get it right away2

– Vaccine intent survey, Sep-Oct: 63% healthcare providers would get COVID-19 vaccine3

 ANF nurses survey, Oct: moderate acceptability of COVID-19 vaccine4

– 63% somewhat or very confident vaccine will be safe and effective

– 34% would voluntarily receive COVID-19 vaccine

– 57% comfortable discussing COVID-19 vaccines with patients

Acceptability:

Summary of the available evidence

4. ANF, 16 Nov 2020. https://www.nursingworld.org/practice-policy/work-environment/health-safety/disaster-preparedness/coronavirus/what-you-need-to-know/covid-19-vaccine-survey/18

1. CDC COVID-19 Response Team. 2. Jorgenson. CDC Presentation to ACIP Working Group. 3 Sep 2020. 3. Lindley et al, CDC COVID-19 Response Team: Report in progress.
19

 All jurisdictions have submitted COVID-19 vaccine implementation plans

 Large and small pharmacy chains have committed to participate in COVID-19 

vaccination program

 In a CDC survey of 34 state health officers in October, common concerns about 

vaccine administration included vaccine hesitancy, vaccine safety, and 

communications

 A survey of nurses, most were confident the vaccine will be safe and effective, while 

less would voluntarily receive COVID-19 vaccine if not required

Acceptability:

Summary of the available evidence
Acceptability:

Work Group Interpretation 

Is COVID-19 vaccine ‘X’ acceptable to key 

stakeholders? 

○ No ○ Probably no ○ Probably yes ○ Yes ○ Varies ○ Don't know
EtR Domain: Feasibility 
Feasibility 

Is COVID-19 vaccine ‘X’ feasible to implement? - Is the COVID-19 vaccine ‘X’ program sustainable? 

- Are there barriers that are likely to limit the feasibility of implementing COVID-19 vaccine

_ ‘X’ or require consideration when implementing it? 

- Is access to COVID-19 vaccine ‘X’ an important concern? 

_

○ No ○ Probably no ○ Probably yes ○ Yes ○ Varies ○ Don't know
 Barriers to implementation may include: 

1) Financial barriers

2) Complexity of recommendations

3) Access to healthcare or vaccine providers

4) Vaccine storage and handling requirements 

Feasibility:

Summary of the available evidence

23
24

 All COVID-19 vaccines will be provided to U.S. population free of charge

 Health systems or health departments could incur costs for vaccine 

implementation, clinics 

Feasibility:

Summary of the available evidence

1) Financial barriers
25

 Multiple vaccines under an EUA could make overall COVID-19 vaccine 

recommendations more complex

 Individual vaccine recommendations may also contribute to complexity 

– Variations in number of doses, schedule 

Feasibility:

Summary of the available evidence

2) Complexity of recommendations
26

 Population access to healthcare could be limited in rural or other hardto-reach areas

 Range of providers providing vaccine could be impacted by:

– Cold storage requirements

– Population(s) with proven safety/efficacy

– Population(s) recommended to receive vaccine

Feasibility:

Summary of the available evidence

3) Access to healthcare or vaccine providers
27

 Vaccine with ultra-cold requirements unable to be integrated into 

providers’ practices

– Vaccines with refrigerator (2-8oC) temperature requirements easier to integrate

 Minimum size of orders

 Requirements for two-dose series for some vaccines

Feasibility:

Summary of the available evidence

4) Vaccine storage and handling requirements
 Innovative solutions to overcome barriers: 

– Expanded funding opportunities

– Pharmacy partnerships

– Technology, including second dose reminders

– Unique packing containers to maintain ultra-cold temperatures without freezer

– Detailed state micro-planning

Feasibility:

Summary of the available evidence

28
Feasibility: 

Work Group Interpretation

Is COVID-19 vaccine ‘X’ feasible to implement? 

_

○ No ○ Probably no ○ Probably yes ○ Yes ○ Varies ○ Don't know
Summary 
Summary:

31

EtR DomainQuestionWork Group Judgments

Values

Does the target population feel the desirable effects are _

_large relative to the undesirable effects?

Probably Yes; Varies

Is there important variability in how patients value the 

outcomes?

Important/

probably important 

uncertainty 

Acceptability

Is COVID-19 vaccine ‘X’ acceptable to key stakeholders?

Probably Yes; Varies

Feasibility

Is COVID-19 vaccine ‘X’ feasible to implement?

Probably Yes
Summary:

32

EtR DomainQuestion

Work Group 

Judgments

Public Health Problem

Is COVID-19 disease of public health importance?

Yes

Values

Does the target population feel the desirable effects are _

_large relative to the undesirable effects?

Probably Yes/ Varies

Is there important variability in how patients value the outcomes?

Important/

probably important 

uncertainty 

Acceptability

Is the intervention acceptable to key stakeholders?

Probably Yes/ Varies

Feasibility

Is the intervention feasible to implement?

Probably Yes/ Varies

Resource Use

Is COVID-19 vaccine X a reasonable and efficient allocation of 

resources?

Yes

Equity

Does COVID-19 vaccine X have the potential to increase health equity? Probably reduced/

Probably increased*

*Judgment differed by COVID-19 vaccine
Proposed Clinical Considerations:

33

 Pregnancy/Breastfeeding:

– Pregnancy/breastfeeding is not a contraindication to receiving a COVID-19 vaccine

– For those recommended to receive vaccine in an early allocation phase 

 Prior SARS-CoV-2 infection: 

– Vaccination is recommended regardless of prior infection

– Testing for SARS-CoV-2 antibodies is not recommended prior to vaccination

– While vaccine supplies are constrained, vaccination of persons with recent prior infection 

may be delayed. However, duration of protection after infection is unknown.

 Other topics for future presentations to ACIP: 

– Coadministration with other vaccines

– Vaccine dosing schedules, intervals

– Impact of vaccine reactogenicity for healthcare providers
Questions for ACIP:

34

 Values:

– What does ACIP think about the values of the target population? 

– Any additional information that ACIP needs to see before a vote?
Questions for ACIP:

35

 Acceptability:

– What does ACIP think about the acceptability of COVID-19 vaccines from stakeholders 

(providers, health departments, health systems)? 

– Can ACIP members, liaison organizations provide additional insight into acceptability 

of stakeholders?
Questions for ACIP:

36

 Feasibility:

– What does ACIP think about the feasibility of implementation? 

– Any additional information that ACIP needs to see before a vote?
For more information, contact CDC

1-800-CDC-INFO (232-4636)

TTY: 1-888-232-6348 www.cdc.gov

The findings and conclusions in this report are those of the authors and do not necessarily represent the 

official position of the Centers for Disease Control and Prevention.

Thank you