SENTINEL METHODS WHITE PAPER
EXPLORING THE FEASIBILITY OF CONDUCTING VACCINEEFFECTIVENESS STUDIES IN SENTINEL’S PRISM PROGRAM
Prepared by: Catherine A. Panozzo, MPH, PhD, 1 Maria Said, MD, MHS,2
Deepa Arya, MD, MPH, MBA 2 Roger Baxter, MD, 3 Elizabeth C. Dee, MPH, 1
Kevin Fahey,MA, 4 Sandra Feibelmann, MPH, 1 Bruce Fireman, MA, 3
Richard A. Forshee, PhD, 2 Hector Izurieta, MD,MPH, 2 Lisa Jackson, MD,
MPH, 5 Nicola Klein, MD, PhD, 3 Yun Lu, PhD, 2 David Menschik, MD,MPH, 2James Nordin, MD, MPH, 6 Douglas Pratt, MD, 2 Carla Rodriguez, MPH,
PhD, 7 Nandini Selvam, MPH, PhD, 8 Azadeh Shoaibi, MS, MHS, 2 Meghan
Baker, MD, ScD 1

Author Affiliations: 1.Department of Population Medicine, Harvard
Medical School and Harvard Pilgrim Health Care Institute , Boston, MA
2.Center for Biologics Evaluation and Research, US Food and Drug
Administration, Silver Spring, MD 3.Kaiser Permanente Northern
California Division of Research, Oakland, CA 4.America’s Health
Insurance Plans, Inc., Washington, D.C. 5.Kaiser Permanente
Washington, Seattle, WA 6.Health Partners Institute for Education andResearch, Minneapolis, MN 7.Mid-Atlantic Kaiser Permanente ResearchInstitute, Rockville, MD 8.HealthCore, Inc., Alexandria, VA
April 27, 2018
The Sentinel System is sponsored by the U.S. Food and Drug Administration (FDA) to proactively monitor
the safety of FDA-regulated medical products and complements other existing FDA safety surveillancecapabilities. The Sentinel System is one piece of FDA’s Sentinel Initiative, a long-term, multi-faceted
effort to develop a national electronic system. Sentinel Collaborators include Data and Academic
Partners that provide access to healthcare data and ongoing scientific, technical, methodological, andorganizational expertise. The Sentinel Coordinating Center is funded by the FDA through the
Department of Health and Human Services (HHS) Contract number HHSF223201400030I.

Sentinel White Paper
Exploring the Feasibility of Conducting Vaccine Effectiveness Studies in
Sentinel’s PRISM Program
Table of Contents
I. EXECUTIVE SUMMARY ........................................................................................................ 1
II. INTRODUCTION.................................................................................................................. 1
III. DATA ELEMENTS ................................................................................................................ 3
A. SENTINEL SYSTEM POPULATION ................................................................................................ 3
B. EXPOSURE DATA ................................................................................................................. 3
C. OUTCOME DATA ................................................................................................................. 4
D. COVARIATE DATA ................................................................................................................ 4
E. GENERAL LIMITATI ONS OF CLAI MS-BASED DATA ............................................................................ 5
IV. METHODS: STUDY DESIGNS AND STATISTICAL ADJUSTMENT METHODS ................................... 5
A. BETWEEN-PERSON C OMPARISONS ............................................................................................. 5
1.
2.
3.
4.
5.
B.

Cohort ........................................................................................................................ 5
Natural Experiment ...................................................................................................... 6
Case-control ................................................................................................................ 6
Test-negative Designs ................................................................................................... 6
Screening Methods....................................................................................................... 7
WITHIN-PERSON COMPARISONS ............................................................................................... 7

1.
C.

Case-only Designs (Self-controlled) ................................................................................. 7
DATA SHARI NG TECH NIQUE .................................................................................................... 91.

Case-centered Approach ............................................................................................... 9

V. ASSESSMENTS: ROUTINE QUERY TOOLS AND PROTOCOL-BASED ASSESSMENTS .......................10
VI. DATA AND DESCRIPTIVE TOOL EXPLORATION (USE CASE).......................................................11
A. ENROLLMENT CRITERIA ........................................................................................................12
B. OUTCOMES ......................................................................................................................12
C. COVARIATES .....................................................................................................................12
D. DESCRIPTIVE ANAL YSIS .........................................................................................................12
E. R ESULTS ..........................................................................................................................13
F. COMMENT .......................................................................................................................13
VII. STRENGTHS AND LIMITATIONS ...........................................................................................18
A. STRENGTHS ......................................................................................................................18
B. LIMITATIONS .....................................................................................................................18
VIII. CONCLUSION ....................................................................................................................18
IX. REFERENCES .....................................................................................................................19
X. ACKNOWLEDGMENTS ........................................................................................................22
CBER Sentinel White Paper-i-

Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM ProgramXI. APPENDICES .....................................................................................................................23
A. APPENDIX A .....................................................................................................................23
1. Appendix A: Summary of Prevalent Influenza Vaccine Use † in the Sentinel Distributed Databasebetween August 1, 2010 and July 31, 2013 and August 1, 2014 and July 31, 2015, High-dose versus
Standard-dose, by Influenza Season and Age Group ................................................................23
B.

APPENDIX B .....................................................................................................................24

1. Appendix B: Summary of Prevalent Influenza Vaccine Use † in the Sentinel Distributed Databasebetween August 1, 2013 and July 31, 2015, Trivalent versus Quadrivalent Vaccine, by Influenza
Season and Age Group........................................................................................................24
C.

APPENDICES C1-C4 ............................................................................................................25
1. Appendix C1: Summary of Prevalent Influenza Outcomes† in the Sentinel Distributed Databasebetween August 1, 2010 and July 31, 2011, by Outcome Definitions and Age Group.....................25
2. Appendix C2: Summary of Prevalent Influenza Outcomes† in the Sentinel Distributed Databasebetween August 1, 2011 and July 31, 2012, by Outcome Definitions and Age Group.....................27
3. Appendix C3: Summary of Prevalent Influenza Outcomes† in the Sentinel Distributed Databasebetween August 1, 2012 and July 31, 2013, by Outcome Definitions and Age Group.....................29
4. Appendix C4: Summary of Prevalent Influenza Outcomes† in the Sentinel Distributed Databasebetween August 1, 2014 and July 31, 2015, by Outcome Definitions and Age Group.....................31

D.

APPENDICES D1-D4 ............................................................................................................33

1. Appendix D1: Characteristics of Patients Receiving High-Dose and Standard-Dose Influenza
Vaccine during the 2010-2011 Influenza Season (183-day Covariate Window).............................33
2. Appendix D2: Characteristics of Patients Receiving High-Dose and Standard-Dose Influenza
Vaccine during the 2011-2012 Influenza Season (183-day Covariate Window).............................34
3. Appendix D3: Characteristics of Patients Receiving High-Dose and Standard-Dose Influenza
Vaccine during the 2012-2013 Influenza Season (183-day Covariate Window).............................35
4. Appendix D4: Characteristics of Patients Receiving High-Dose and Standard-Dose Influenza
Vaccine during the 2014-2015 Influenza Season (183-day Covariate Window).............................36
E.

APPENDICES E1-E4.............................................................................................................37
1. Appendix E1: Characteristics of Patients Receiving High-Dose and Standard-Dose Influenza
Vaccine during the 2010-2011 Influenza Season (548-day Covariate Window).............................37
2. Appendix E2: Characteristics of Patients Receiving High-Dose and Standard-Dose Influenza
Vaccine during the 2011-2012 Influenza Season (548-day Covariate Window).............................38
3. Appendix E3: Characteristics of Patients Receiving High-Dose and Standard-Dose Influenza
Vaccine during the 2012-2013 Influenza Season (548-day Covariate Window).............................39
4. Appendix E4: Characteristics of Patients Receiving High-Dose and Standard-Dose Influenza
Vaccine during the 2014-2015 Influenza Season (548-day Covariate Window).............................40

CBER Sentinel White Paper- ii -

Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program

I.

EXECUTIVE SUMMARY

The Post-licensure Rapid Immunization Safety Monitoring (PRISM) program, a component of the Foodand Drug Administration’s Sentinel System, was developed to monitor vaccine safety, but has neverbeen used to assess vaccine effectiveness. This work evaluates the feasibility of using available data
elements, existing programming tools, and major study designs to conduct vaccine effectivenessassessments in PRISM. While PRISM cannot address all questions of potential interest (e.g., questions
requiring lengthy follow-up time), it has several features, including the ability to conduct analyses in alarge, geographically diverse population and the possibility of chart validation, that could make theexamination of certain questions about vaccine effectiveness possible.

II.

INTRODUCTION

The Post-licensure Rapid Immunization Safety Monitoring (PRISM) program, initiated in 2009 by the
Department of Health and Human Services in response to the need to monitor the safety of the H1N1
influenza vaccine, has developed into the immunization safety monitoring component of the Food andDrug Administration’s (FDA) Sentinel System (“Sentinel”). The Sentinel System uses a large distributed
data infrastructure created to actively monitor the safety of medical products using electronic healthinformation 1,2 . The PRISM database, which is comprised of health information from a subset of Sentinel
Data Partners, has been used to evaluate vaccine safety using administrative claims data captured aspart of health system encounters. An evaluation of the risk of intussusception after rotavirus vaccine in
PRISM led to a change in the prescribing information and safety communications 3,4 . Two otherassessments in PRISM contributed to public safety risk communications: an evaluation of the risk offebrile seizures after the influenza vaccine 5,6 and an assessment of venous thromboembolism after thequadrivalent human papillomavirus vaccine 7,8 . The purpose of this report is to evaluate the suitability ofusing PRISM to estimate vaccine effectiveness.
Vaccine effectiveness may be defined as a measure of how well a vaccine reduces incidence of disease ina population under ordinary real-world conditions. Thus, vaccine effectiveness is usually evaluated after
a vaccine is licensed and marketed, using observational study designs. In contrast, vaccine efficacy
usually refers to a measure of how well a vaccine reduces disease incidence under ideal conditions (e.g.,
double-blind, randomized, well-controlled, well-monitored clinical trials) 9,10 . However, vaccine
effectiveness estimates in observational studies may be influenced by factors such as vaccination
coverage in the population of interest and exposure to the infectious disease of interest. Such factors
would be well-balanced across treatment arms in large randomized studies and unlikely to affectefficacy estimates.
Regulatory approval (licensure) of a vaccine in the U.S. requires demonstration of substantial evidenceof effectiveness 11 . There is an expectation that effectiveness data supporting licensure be obtained
from adequate and well-controlled clinical trials, as described below. Regulations provide for expedited
clinical development pathways for certain products intended to treat or prevent serious and lifethreatening conditions and that provide meaningful therapeutic benefit over existing treatments: 1)
Under accelerated approval, effectiveness is established in adequate and well-controlled clinical trials by
establishing the product’s effect on a surrogate endpoint or intermediate endpoint that is reasonably
likely to predict clinical benefit 12 . 2) Under the “animal rule”, evidence of effectiveness is based onadequate and well-controlled animal studies when the results of those animal studies establish that the
CBER Sentinel White Paper-1-

Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Programbiological product is reasonably likely to produce clinical benefit in humans 13 . The animal rule applies tocertain biologic products when human efficacy studies cannot be conducted because it would be
unethical or not feasible. Vaccines approved under either accelerated approval or animal rule
regulations are subject to a requirement to conduct additional confirmatory studies post-licensure toverify and describe the predicted clinical benefit. For products approved under the animal rule,confirmatory studies would be conducted when they become feasible and ethical.
The period from accelerated approval, or approval under the “animal rule”, to completion, submission,and FDA evaluation of the confirmatory clinical trials creates a potential role for observational studies
conducted in PRISM to provide an independent and complementary real-world assessment of a
vaccine’s effectiveness. Even for vaccines undergoing traditional approval, including a randomized Phase
III efficacy study, uncertainties may remain about their performance in select sub-populations for whichsample sizes were too small in clinical trials to fully characterize their effectiveness (e.g., the elderly
population), or that were excluded from clinical trials (e.g., pregnant women, immunocompromised
individuals), or for rare outcomes, including severe disease. There may be other situations, such as whena randomized controlled trial (RCT) is neither ethical nor feasible, or for rapid evaluation of an approved
vaccine during an epidemic or pandemic, that observational studies may play a role in providing
substantial evidence of effectiveness. For all vaccines, real-world effectiveness may vary from efficacy
estimates derived from clinical trials for many reasons, including differences in underlying characteristics
in the populations who receive them and the potential for waning immunity over extended observation
periods.
Although vaccine safety studies conducted within PRISM have contributed to FDA’s regulatory decision
making, it cannot be assumed that vaccine effectiveness studies would result in any regulatory action.
First, the Sentinel System was developed to obtain information specific to safety, as required by the2007 Food and Drug Administration Amendments Act (FDAAA). Second, regulations specify that clinicalinvestigations supporting claims of effectiveness be adequate and well-controlled, and thatcharacteristics of such investigations include a method of assigning patients to treatment and control
groups that minimize bias, such as randomization, and that adequate measures are taken to minimize
bias on the part of the subjects, observers, and analysts of the data, such as through blinding 11 . Because
PRISM can be used for observational studies, but not for RCTs that meet these regulatory expectations,
it cannot provide the same level of evidence needed to support vaccine approval. Nevertheless,evidence provided by PRISM vaccine effectiveness studies could supplement clinical trial data, and might
be of value in certain situations, where RCTs are neither ethical nor feasible, and when the observational
studies are well-designed to the extent possible to minimize bias.
Use of PRISM for vaccine effectiveness studies highlights a number of other important considerations.
Signal detection, a key focus in vaccine safety work and an activity conducted in PRISM, is not as
relevant to vaccine effectiveness since most effectiveness studies evaluate predefined health outcomes.
The availability of laboratory data and results from other diagnostic testing, may be more important invaccine effectiveness compared to vaccine safety studies. For example, a diagnostic test for influenza ispotentially important for assessment of influenza vaccine effectiveness 14 . Another consideration is thatvaccine effectiveness studies may require longer follow-up than vaccine safety evaluations 15,16 that
often focus on well-defined acute onset outcomes 3,6,8 . A rotavirus vaccine safety evaluation might focus
on intussusception within a week or three weeks of vaccination, whereas an effectiveness evaluation
might focus on gastroenteritis up to 12 or 24 months after vaccination 3, 14 .
The main objective of this white paper is to assess the potential of using PRISM for vaccine effectivenessstudies. The question of feasibility was approached by: (a) exploring available data elements, including
vaccine exposures, outcomes, and covariates; (b) developing an overview and comparison of designs
CBER Sentinel White Paper-2-

Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Programand methods that have been used to estimate vaccine effectiveness in administrative claims data 9,10,1719 ; and, (c) providing a description of relevant reusable and customizable analytic tools available in
Sentinel. We then use some of these tools to illustrate an example use case in which we identify highdose and standard-dose inactivated influenza vaccine exposures, influenza and pneumonia outcomes,
and potentially associated covariates. Finally, we address the most important strengths and limitationsof PRISM in assessing vaccine effectiveness.

III.

DATA ELEMENTS

A. SENTINEL SYSTEM POPULATION
Sentinel has one of the largest cohorts in the US general population for active medical product safetysurveillance 1,20,21 . Sentinel comprises primarily administrative health plan claims data held by 16 healthplan partner organizations and includes approximately 223 million individuals (178 million memberswith both medical and prescription coverage) and 425 million person-years of observation time 2 . The
subset PRISM database, which contains data from 2006, includes over 150 million individuals. Thirty-two
million individuals within PRISM are currently enrolled in participating health plans and accumulating
new data, and 33 million individuals, including both those who are currently or were previously enrolled,have at least one laboratory test result.
The population in the Sentinel distributed database is also notable for its geographic diversity, whichenables evaluations in subgroups, or of rare outcomes in a population that is likely representative of theprivately insured US population 1,2,22 . All age groups are included in the distributed database; however,the population ≥65 years, approximately 15% of the population in the Sentinel Distributed Database 2 , is
not as well represented in Sentinel compared to other data sources such as the Centers for Medicare
and Medicaid Services (CMS). The population is limited to an insured population, and althoughgeographic coverage is wide, data partners may be regionally focused.

B. EXPOSURE DATA
Vaccines given in the outpatient primary care setting are well-captured within the Sentinel System 23 ,and therefore, information about administration of early childhood vaccines tends to be complete.
Identification of individual vaccines may be possible for some, but not all vaccines; for example,
standard-dose trivalent inactivated influenza vaccines may be grouped together and cannot be analyzed
separately (i.e., by trade name). Vaccines from different manufacturers that share the same Current
Procedural Terminology (CPT) code and the same dosage schedule and administration route cannot beanalyzed separately. Information on the dose number within a vaccine series is not readily captured but
could be approximated using coding algorithms. PRISM has the ability to link with the immunization
registries, enabling identification of some vaccines given in locations other than the primary health caresetting where claims-based data may be missing; however, this approach may increase the timeline andcost of an assessment, and the data available in immunization registries vary by state 23-25 .
Identifying comparison groups for vaccine effectiveness assessments poses challenges. Determining agroup to be truly unvaccinated is difficult, as exposure data may be missing among individuals in theinpatient setting or among those who receive vaccines outside of the traditional medical home (and are
therefore not reimbursed via routine insurance company claims system), such as in employee influenza
vaccine clinics. Additionally, children who do not receive vaccines on a regular schedule may not becomparable to the majority of children who do, thus introducing selection bias if only certain groups ofchildren are selected to participate in studies. To address such bias, other approaches, such as
CBER Sentinel White Paper-3-

Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program

comparisons with individuals who received another unrelated vaccine which has no effect on theoutcome of interest, could be performed.

C. OUTCOME DATA
Outcomes can be identified using diagnosis codes (e.g., International Classification of Diseases, Ninth
Revision, Clinical Modification (ICD-9-CM) and International Classification of Diseases, Tenth Revision,
Clinical Modification (ICD-10-CM)), procedure codes (e.g., CPT), or dispensing codes (i.e., National Drug
Codes (NDCs)) 26 . Examples of vaccine effectiveness outcomes may include acute gastroenteritis or
rotavirus gastroenteritis for rotavirus vaccines, or influenza or pneumonia for influenza vaccines.
Certain important limitations related to defining outcomes in PRISM need to be considered. First, dataon mortality and cause of mortality are not systematically populated across data partners. Certain
health plans perform routine or ad hoc linkages with local or state death registries, but the process is
not standardized, which leads to variability in data quality and completeness. Second, only some of the
PRISM data partners contribute selected standardized laboratory test results. For these data partners,
the test results come from large laboratory vendors, but because not all vendors send results, capture isincomplete and this results in decreased specificity of certain outcomes. In the case of influenza
diagnosis codes, specificity can be improved by adding laboratory procedure or anti-viral dispensing
codes. Third, although many vaccine effectiveness outcome algorithms that use administrative claims
data have been validated in other settings 27,28 , most algorithms are not yet validated in the Sentinel
distributed database, potentially leading to misclassification of important study outcomes andpotentially biased vaccine effectiveness estimates. Finally, questions related to long-term vaccine
effectiveness or waning immunity may be difficult to explore using the Sentinel System because themedian follow-up time for health plan members is approximately 17 months, and Sentinel cannot track
individuals who switch insurance partners 29 . Although it may not be possible to assess long-term
effectiveness under most circumstances, questions about waning protection that involve a more limited
time frame, or that can be restricted to certain data partners or populations who tend to have longer
follow-up, could be explored in PRISM.

D. COVARIATE DATA
Potential confounders defined with a single or combination of diagnosis, procedure, or dispensing codes
can be incorporated into analyses. In addition, pre-existing programs developed within Sentinel todetermine the combined comorbidity index (CCI), a summary score that predicts short- and long-term
mortality 21 , and health care utilization metrics can be used to help define complex factors, such as
frailty within the elderly. Both the CCI and health care utilization measures have been used as
adjustment measures in propensity score-matched cohort studies 30 and can also be included in
descriptive analyses highlighting baseline characteristics of the study and comparator groups, as wasdone in our data and descriptive tool exploration (Section IV, Table 4 and Table 5). The PRISM programis currently evaluating the use of three-digit zip codes of members’ primary place of residence. These
data can be aggregated to geographic units of interest, including metropolitan statistical areas andregions using standard files available from the U.S. Census Bureau 31,32 . Such data could allow vaccine
effectiveness studies to focus on differences in vaccine impact among geographic areas (e.g., regionswith high versus low influenza circulation), or include geographic units as a potential confounder.

CBER Sentinel White Paper-4-

Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM ProgramE. GENERAL LIMITATIONS OF CLAIMS-BASED DATA
Aside from Sentinel and PRISM-specific limitations, there are general limitations of using administrative
claims-based data to examine vaccine effectiveness. First, claims data are generated by insurancecompanies for billing purposes, not public health surveillance. Diagnosis codes that appear in the claims
data may not be the main (e.g., due to reimbursement considerations) or final (e.g., due to latent key
signs/symptoms) diagnosis of the patient. In addition, exposure, outcome, and covariate definitions arelimited to how they are defined by Healthcare Common Procedure Coding System (HCPCS), NDC, or ICD 9-CM codes and may be affected by coding changes, such as the ICD-9-CM to ICD-10-CM transition that
occurred on October 1, 2015. However, use or development of validated algorithms and medical record
review have the potential to circumvent some of these well-known limitations in certain circumstances.
Second, although data can generally be accessed more rapidly from claims than from most prospective
data collection methods, Sentinel data are not available in real-time. The data are refreshed quarterly,
semi-annually, or annually, depending on the data partner, and then undergo a rigorous quality
assurance process by the Sentinel Operations Center, before the data are available for querying. Thus,by the time the data are available for analysis, they are typically at least 9 months old 33 .
Finally, vaccine effectiveness estimates based on claims data such as PRISM cannot account for the
probability of disease exposure or transmission; these must be assumed to be equal between vaccinatedand control groups 23 .

IV.

METHODS: STUDY DESIGNS AND STATISTICAL ADJUSTMENT METHODS

Study designs and statistical methods relevant to the study of vaccine effectiveness using administrative
claims data were identified and reviewed. Identification of methods primarily involved expert
discussion; however, to ensure that no approaches were overlooked, the work group also conducted aliterature scan in PubMed. The sections that follow briefly define and summarize the strengths andlimitations of six study designs and one data sharing technique in relation to the study of vaccine
effectiveness, highlighting a few examples from the literature, and describing their potential applicability
to PRISM.

A. BETWEEN-PERSON COMPARISONS
1. Cohort
In a cohort study, a group of people are identified and followed for a certain period to ascertain theoccurrence of health-related events. The goal is usually to determine whether an exposure is related tothe incidence of an event 34 . In vaccine effectiveness cohort studies that use administrative claims data,a cohort is often formed naturally via membership to a particular health plan (e.g., Aetna) from which an“exposed,” vaccinated group and “unexposed,” unvaccinated (or other comparator group) are drawn(Table 1). Strengths of the cohort study design for vaccine effectiveness using administrative claims
data include longitudinal data capture of most medical encounters, standardized medical coding
systems, low cost for data access, and a large sample size that enables subgroup analyses. An important
limitation is the difficulty in identifying an unvaccinated, comparator group either because the vaccine
has high coverage (e.g., measles, mumps, and rubella vaccine (MMR), diphtheria, tetanus, and acellular
pertussis vaccine (DTaP)), or because the vaccine may be received outside the medical setting (e.g.,
influenza vaccines received at the workplace will not generate an insurance claim). Thus, in someinstances, it may be best to focus on comparative vaccine effectiveness or investigate within-cohort
CBER Sentinel White Paper-5-

Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Programdifferences. The cohort design is applicable to the study of vaccine effectiveness in PRISM, and there are
currently analytic tools available to assist with such analyses.

2. Natural Experiment
One variation of a cohort study is the natural experiment, a situation in which a naturally occurring
circumstance subsets the population to different levels of exposure in a situation resembling arandomized study 35 . A natural experiment can potentially control for both measured and unmeasured
confounders, offering a major advantage over other observational study designs. Natural experiments
have been used to assess bias and ultimately produce more valid comparative effectiveness estimates of
the influenza vaccine in the elderly 15,36 . For example, Jackson et al. and McGrath et al. took advantage
of vaccine effectiveness demonstrated during periods with little to no seasonal influenza activity or witha known vaccine mismatch to demonstrate likely bias 15,36 . Estimates from these groups can ultimately
be used as the control in an influenza vaccine effectiveness study. The studies that are applicable tostudying vaccine effectiveness in PRISM are the ones where the natural conditions create the “right”
condition for a particular research question. For example, the situation in which a PRISM data partner
exclusively uses (via reimbursement policy) the pentavalent rotavirus vaccine and a second PRISM data
partner exclusively uses the monovalent rotavirus vaccine could lend itself to a comparative
effectiveness study of the two rotavirus vaccines. However, the low frequency of these types ofsituations limit the usefulness of this design in PRISM.

3. Case-control
A case-control study can be thought of as an efficient cohort study, whereby all of the cases are used,
but rather than including the entire source population that gave rise to the cases, only a sample ofcontrols is selected 37 . Any case-control study designed in PRISM would be considered a “nested” casecontrol study because the population available is already part of the health plan cohort. As such, likeother pharmacoepidemiologic studies that rely on secondary data sources, there would generally be nocost or time advantage to conducting a case-control as opposed to a cohort study in PRISM unless chart
review is planned or additional laboratory values need to be collected from the data partners. Similar tocohort studies, identifying a sufficient number of true, unexposed controls may be difficult for vaccineswith high coverage and limit study power. An additional challenge of case-control studies is theuncertainty of whether the selected controls accurately represent the source population.

4. Test-negative Designs
In a traditional, prospective test-negative design, people seeking health care for symptoms related to anon-specific infectious condition (e.g., acute respiratory illness) are recruited into a study and tested fora specific, vaccine-preventable disease (e.g., influenza) 18 . The vaccine effectiveness measurement isestimated as one minus the ratio of odds of vaccination in subjects testing positive to the odds ofvaccination in those testing negative to the specific, vaccine-preventable disease (e.g., influenza). The
test-negative design offers the advantage of mitigating confounding by health care-seeking behavior
which is often a major source of bias in influenza vaccine effectiveness studies, as persons who are more
likely to seek vaccination may also be more likely to seek health care for an acute respiratory illness 18 .
At this time, test-negative influenza vaccine effectiveness studies cannot be performed in PRISM. First,
while influenza laboratory outpatient test results (e.g., enzyme immunoassay, nucleic acid test) areavailable from 10 Sentinel data partners, over 80% of the influenza A and influenza B test results
contributed from 2006 through mid-2015 were from a single data partner and the results have not beencharacterized nor standardized within the Sentinel Common Data Model 38 . Even if additional resources
CBER Sentinel White Paper-6-

Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Programwere devoted to standardizing the test results across the data partners, influenza laboratory tests
performed in the outpatient setting are often performed at the provider’s discretion, which makes theiravailability prone to bias.
Second, to date, the test-negative design has been largely used in a prospective manner 39 , where acuterespiratory symptoms and other eligibility criteria (e.g., duration of illness, not immunocompromised)are verified and lead to study enrollment, and then standardized specimen collection and influenza
testing occurs. To obtain and verify information on symptoms and eligibility criteria using PRISM,
retrospective chart review would need to be conducted on patients who already have influenza
laboratory result information; however, not all of the necessary information is likely to be documented
in medical health records. Furthermore, a highly specific test is usually needed for test-negative studies
and the retrospective nature of the PRISM data would not be able to control for the sensitivity andspecificity of the laboratory test results. Thus, developing a case definition that can be consistently andfairly applied to all study subjects could be difficult.

5. Screening Methods
The screening method is useful for estimating vaccine effectiveness when denominator data areunavailable (e.g., fieldwork, other settings where resources are low) 40 . The validity of the method
depends on the accuracy of the external vaccine coverage estimates, and challenges concerning
confounding adjustment and confidence intervals estimates using the method persist 41,42 . Since PRISM
data are comprised primarily of administrative claims that allow longitudinal follow-up of health plan
members via enrollment data, a denominator is available. Thus, the main advantage of the screening
method is not relevant to PRISM, and the disadvantages outweigh the advantages. However, thismethod could be useful when non-comparative, vaccine effectiveness estimates are desired and vaccine
capture is suboptimal.

B. WITHIN-PERSON COMPARISONS
1. Case-only Designs (Self-controlled)
Case-only designs, such as the self-controlled risk interval and self-controlled case series, compare therisk of a health outcome of interest in a risk interval with that of a control interval within a single
individual and thereby eliminate the bias from both measured and unmeasured time-invariant
confounders between vaccinated and unvaccinated individuals that can plague cohort studies 43 . Whileself-controlled risk interval designs are widely used and often the preferred design in vaccine safety
studies that utilize large health care databases 3,6,8 , these designs are difficult to use in vaccine
effectiveness in PRISM because, in most cases, there is no clear period of transient, heightened risk ofcontracting an effectiveness outcome shortly following vaccination. This limitation may be mitigatedsomewhat during an epidemic or pandemic when information on disease circulation is available;
however, defining a precise and appropriate time interval of risk for a particular outcome may remain achallenge.

CBER Sentinel White Paper-7-

Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM ProgramTable 1. Summary of study designs
Brief description
Study
designBetween-person comparisons
CohortGroup of vaccinated andunvaccinated health plan
members (or othercomparator) identified
and followed up to
ascertain vaccinepreventable disease
eventsNaturalexperimentCase-control

Testnegative

Variation of a cohort
study where exposure isdetermined byconditions not
associated with theoutcome (thus, in a way,resembling a random
assignment)
Efficient study whereby
all available cases areused, but only aselection of members
from the source
population (controls) areused for the comparison
groupType of case-control
study where health plan
members seeking healthcare for symptoms
related to a non-specificinfectious condition are
tested for a vaccinepreventable disease
outcome; those testing
positive are compared tothose testing negativewith regards to vaccine
exposure

CBER Sentinel White Paper

Strengths and limitations

Recommended
for use in PRISM

Literature
examplesStrengths: Takes advantageof large, captive population
with longitudinal
information
Limitations: May be difficult
to accurately ascertain or
obtain sufficient numbers ofunvaccinated members socomparative effectivenessstudies may be more viable
Strengths: Can control formeasured and unmeasured
confounders
Limitations: Properconditions must exist on
their own so may not besuitable for every question

Yes

16,44,45

Yes

15,36,46Strengths: May be possible
to conduct chart review on
entire study population(cases and controls)
Limitations: May be difficult
to accurately ascertain or
obtain sufficient numbers ofunvaccinated members
among cases and controls;
controls may not accurately
represent source population
Strengths: Mitigatesconfounding by health careseeking behavior
Limitations: Lab test result
availability, bias in selectionof patients for testing,characterization of symptom
onset, specificity of casedefinition, accurately
identifying vaccine exposure

Yes

47,48

No

48-51

-8-

Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM ProgramStudydesignScreening
methodBrief description

Strengths and limitations

Recommended
for use in PRISM

Literature
examplesEstimates the odds ofvaccination in cases
compared with the odds
of vaccination estimatedfrom sources external to
the study (e.g. vaccine
coverage in the general
population)

Strengths: Requires data on
individuals for cases only.
Useful in low resource
settings wherecomparator/control groupdata are unavailable (usually
not the case in PRISM)
Limitations: Validity
depends on external vaccine
coverage estimates,
controlling for confoundinga challenge

No

40-42

Strengths: implicitly
controls for time-invariant
measured and unmeasured
confounders; time varying
covariates can usually be
included in the analysis
Limitations: Requires welldefined intervals ofheightened, transient risk ofcontracting outcomes after
vaccination

No

22,52

Within-person comparisons
Case only
Compares the risk of a(selfhealth outcome ofcontrolled)interest in a risk intervalwith that in a control
interval within a single
individual

C. DATA SHARING TECHNIQUE
1. Case-centered Approach
The case-centered approach is a data sharing technique that outputs only aggregate data and thus
maximizes patient privacy across data networks. It compares the observed odds of prior vaccination
among the cases with the expected odds (under the null hypothesis) within a stratum of similar
individuals based on the proportion in the entire stratum who were vaccinated. This approach can beespecially useful in examining differences-in-differences: differences between the vaccine’s effects
during different periods, on different sub-types of outcome events, or in different subgroups ofvaccinees. A complete description of the approach is detailed in Fireman et al and Baxter et al 17,53 .
The case-centered approach has several features that make it relevant for conducting vaccine
effectiveness studies in PRISM. First, since only risk set level data are required for analysis, data partners
can maximize patient privacy when returning data to the Sentinel Operations Center while allowing thestudy to produce the same results that would normally require patient-level data (e.g., time-to-event
analysis using Cox regression). Second, the approach is particularly useful when the exposure-outcome
association varies markedly over time, as in the case of influenza vaccination and influenza disease or
waning vaccine effectiveness. Third, computational time can be reduced compared with other methods.
Fourth, the approach can flexibly handle change over time in vaccine effectiveness. Although it has
mostly been used to examine vaccine safety during transient risk intervals, it can also be used to
CBER Sentinel White Paper-9-Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Programexamine the waning of vaccine effectiveness over long periods of time. This approach permits a unique
way of implementing a cohort design while preserving patient-privacy. Given the strengths of this data
sharing technique, vaccine effectiveness studies in PRISM should consider the approach in the study
design phase.

V.

ASSESSMENTS: ROUTINE QUERY TOOLS AND PROTOCOL-BASEDASSESSMENTS

Sentinel has a suite of tools that can be leveraged to quickly query data transformed into the Sentinel
Common Data Model. This querying system allows FDA to rapidly identify cohorts of interest andperform analyses. The routine query tools vary in complexity and can perform simple descriptive
analyses, as well as adjusted analyses, with confounding control. Analyses using these tools can be
performed much faster than traditional pharmacoepidemiology studies because the time required forboth programming and quality control is reduced significantly.
The simplest of Sentinel’s routine querying tools are referred to as Summary Table queries. With thistool, investigators can quickly identify counts, prevalence, and incidence of medical product dispensings,
diagnosis codes, and procedure codes. Output can be stratified by pre-determined age group, sex, year,
and care setting. The bulk of the routine querying tools include modular programs, defined as reusable
statistical software programs that are customized by adjusting various parameters, such as enrollment
requirements, inclusion and exclusion criteria, and exposure and outcome definitions. Modular
programs are grouped into three levels.
Level 1 modular programs can be used to identify cohorts of interest and perform unadjusted
descriptive analyses. Level 1 modular programs have the capability to answer many common questions
about the Sentinel population. These programs can describe: (a) background rates, (b) uptake, use, andpersistence of new medical products, (c) exposures and follow-up time, (d) health outcomes following
medical product exposure, (e) concomitant medical product use, (f) health outcomes duringconcomitant use, (g) most frequently observed diagnoses, procedures, or drug dispensing, (h) medical
product use during pregnancies that result in live births or (i) baseline distributions of potential
confounders. These analyses can be stratified by age group, sex, year, month, comorbidity score, or
healthcare utilization metrics.
Level 2 modular programs can be used to identify cohorts and perform adjusted analyses with
confounding control, generating effect estimates and confidence intervals. There are currently two
types of Level 2 modular programs. The first can be used to identify exposures, follow-up time,outcomes, and covariates, estimate propensity scores, and match or stratify patients in an exposurecohort to those in a comparison cohort based on propensity scores. The second type of Level 2 program
uses a self-controlled risk interval design, allowing investigators to identify exposures, define risk andcontrol windows relative to the exposure date, and evaluate the occurrence of outcomes during the risk
and control windows.
Level 3 modular programs can be used to identify cohorts and continually perform adjusted analyses
with confounding control as part of prospective sequential analysis. Currently, prospective sequential
analysis can be done with either propensity score matching or a self-controlled risk interval design,described above.
The major advantage to using the Sentinel routine querying tools is the speed at which results can begenerated. The time required to generate query results depends on several factors, including the
CBER Sentinel White Paper- 10 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program

complexity of the query specifications, the number of other queries undergoing processing, and thepriorities of the FDA. Once the Level 1 query parameters for a modular program have been finalized,
results are typically available within 3 to 8 weeks.
There are also limitations to these tools. Because tools are pre-defined, specific areas of interest may
not have been built into the capabilities. For example, the tools are currently unable to describe thelength of hospitalization for a cohort of interest. However, limitations are tracked and newenhancements or programs are prioritized and developed in consultation with FDA. In addition, as a
privacy protecting measure, results from data partners are returned in aggregate to the Sentinel
Operations Center so investigators cannot easily parse data after it is returned to troubleshoot or
examine alternative, non-pre-specified subgroups.
When existing tools do not meet the demands of a particular project, protocol-based assessments canbe used, as has been done in previous PRISM vaccine safety investigations 3,6,8.. This kind of study
generally has a longer timeline and costs more than a study that uses tools because it involves writing adetailed protocol and original analytic programming code, as well as enacting specific quality control andtesting measures. These types of studies may also involve medical health record review, significantly
increasing the timeline and cost of the study.
Beyond tools and protocol-based assessments, there are additional features available to the Sentinel
System, and thus PRISM, that may strengthen the study of vaccine effectiveness. First, medical record
review can assist in verifying exposures and outcomes and validating their timing 3,8 . Second, PRISM data
can be linked to vaccine registries to help verify vaccine exposure or obtain additional information about
the vaccination (e.g., lot number); however, these linkages are not actively maintained so they must be
performed on a study-by-study basis. To date, PRISM studies have linked once to vaccination registries
in Florida, Michigan, Minnesota, New York City, New York State, Pennsylvania, Wisconsin, and Virginia 23.
The main trade-offs of medical record review and vaccine registry linkage relate to the potentially highcosts and lengthy processes.
Current work essential to the ability of assessing the effectiveness of maternal vaccination (e.g., withTdap or influenza vaccine) in preventing disease in newborns includes: 1) estimating the percentage of
mothers that can be linked to their newborns and characterizing variation by data partner and 2) linking
claims data to birth certificates and understanding whether this effort results in improved estimates ofgestational age 54,55 . Future work may also include using PRISM to evaluate indirect effectiveness or
population impact, as other studies utilizing administrative claims data have done 16,56 . Such work may
be useful to more fully evaluate the benefits of vaccines that may have medium-to-low direct
effectiveness estimates but potentially large “herd immunity” effects.

VI.

DATA AND DESCRIPTIVE TOOL EXPLORATION (USE CASE)

We conducted a descriptive, exploratory exercise for the purpose of better understanding the strengths
and limitations of using PRISM to address questions about vaccine effectiveness. For our example, we
examined some of the elements necessary to conduct a vaccine effectiveness study comparing highdose, inactivated influenza vaccine (“high-dose”) to standard-dose, inactivated influenza vaccine(“standard dose”) using a Level 1 modular program designed to evaluate background rates. Since thegoal was not to determine an estimate for vaccine effectiveness, all exposures and outcomes were
evaluated separately. In addition, baseline covariate distribution was evaluated for members receiving
high-dose versus standard-dose influenza vaccines. The specifications and results are described in thesections below.
CBER Sentinel White Paper- 11 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM ProgramA. ENROLLMENT CRITERIA
To be considered in the exposure, outcome, and covariate cohorts, health plan members ≥65 years ofage were required to be continuously enrolled with both drug and medical coverage at one of six
Sentinel data partners for a minimum of 183 days, allowing gaps in coverage of up to 45 days. The
cohort was limited to members ≥65 years of age to match the FDA-approved indication and usage forthe high-dose, inactivated influenza vaccine. The covariate cohort included an additional cohort that
required members to have 548 days (1.5 years) of enrollment to better capture whether administrationof an influenza or pneumonia vaccine or hospitalization had occurred during the previous influenza
season. The query examination period spanned from August 1, 2010 – July 31, 2015. Exposures. All
vaccine exposures were defined using HCPCS codes. They were grouped according to whether they werethe 1) standard dose, inactivated influenza vaccine formulations (all manufacturers), or 2) the high-dose,
inactivated influenza vaccine formulation (Sanofi Pasteur). Within the standard dose group, information
was also gathered on the trivalent versus quadrivalent formulations for the 2013–2014 and 2014–2015
seasons. The vaccines could be provided in any care setting (e.g., outpatient, inpatient, emergency
department).

B. OUTCOMES
Five outcomes were considered: 1) influenza diagnosis; 2) pneumonia diagnosis; 3) influenza laboratory
test procedure performed in any care setting; 4) influenza rapid test performed followed by anoseltamivir dispensing within a two-day period in the outpatient setting; and 5) influenza laboratory test
performed followed by an influenza diagnosis within a three-day period in any care setting. Influenza
and pneumonia diagnoses were defined using ICD-9-CM codes and the care setting type varied between
inpatient or emergency department; inpatient only (primary position); emergency department only;
outpatient only; and any care setting. Laboratory tests were defined using CPT codes, and oseltamivir
dispensings were defined by NDCs.

C. COVARIATES
Covariates examined included patient characteristics (e.g., sex, age), pre-existing conditions or
medications (e.g., asthma, heart disease, statin use, combined comorbidity score), and health service
utilization intensity metrics (e.g., number of filled prescriptions, number of emergency room
encounters). For high-risk disorders, including asthma, blood disorder, chronic lung disease, diabetes,
heart disease, kidney disorders, liver disorders, neurological conditions, and weakened immune system,
we utilized the codes employed in Izurieta et al 44 .

D. DESCRIPTIVE ANALYSIS
Prevalent use of vaccines and the occurrence of outcomes were assessed separately and were reported
for each group by season (August 1– July 31, 2010–2015), age (65–74 years, 75–84 years, and ≥85
years), sex, and month. In the covariate analysis, we compared frequencies or means of the selectedvariables between those ≥65 years of age receiving the high-dose versus the standard dose, inactivatedinfluenza vaccine for each season. Baseline characteristics were assessed during 183 and 548 day
periods prior to the influenza vaccination.

CBER Sentinel White Paper- 12 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM ProgramE. RESULTS
Due to the volume of data generated, we present select tables and data elements from only the mostrecent and complete season (August 2013- July 2014) available at the time the analysis was distributedon April 6, 2016. Results for the other seasons, and for trivalent versus quadrivalent vaccine use can beviewed in Appendices A-E. Sample output from the Level 1 modular program highlighting the prevalenceof use of standard-dose and high-dose, inactivated influenza vaccines in the 65–74, 75–84, and ≥85 yearold age groups during the 2013–2014 season are shown in Table 2. The number of patients 65–74 years
old who received standard-dose versus high-dose, inactivated influenza vaccine was 581,423 and120,829 out of approximately three million eligible health plan members 65–74 years old in 2013–2014,
respectively. The proportion of patients 65-74 years of age receiving high-dose increased each season
while the proportion of patients receiving standard-dose remained steady across the seasons examined.
Similarly, sample output highlighting the prevalence of the five selected outcomes (irrespective of
influenza vaccine exposure) during the 2013–2014 season is shown in Table 3. The number of patients
65-74 years with an influenza diagnosis in the inpatient or emergency department settings was 3,058,
and the number of patients 65-74 years with a pneumonia diagnosis in either of these settings was
47,577among >3 million eligible members in 2013-2014; these figures were within the range of theestimates produced for the other seasons. Results for the 2013–2014 covariate analysis with the 183
day and 548 day lookback periods are shown in Table 4 and Table 5, respectively. In general, theprevalence of the pre-existing conditions increased as the lookback period increased from 183 to 548
days, and the prevalence of pre-existing conditions was higher in the standard-dose compared with thehigh-dose, inactivated influenza vaccine group for all seasons, including 2013- 2014. The most prevalentpre-existing conditions were heart disease and diabetes, and statin use during the baseline period wasapproximately 50%.

F. COMMENT
This use case demonstrated the ability of PRISM data and tools to identify a specific vaccine product(Sanofi Pasteur’s Fluzone High-Dose) and markers of outcomes (i.e., utilization of oseltamivir andinfluenza rapid test) that have been used to examine influenza vaccine effectiveness in other studies.
Although power calculations were beyond the scope of this feasibility exercise, the large number ofhealth plan members with documentation of influenza vaccination in the data and the large number ofoutcomes identified make a comparative vaccine effectiveness study of standard-dose, inactivatedinfluenza vaccine and high-dose influenza vaccine likely feasible, although the overall prevalence of
influenza vaccine use was lower than expected, perhaps as a result of missing exposures. Importantstrengths and some limitations relevant to this and other vaccine effectiveness questions are reviewed
in the section that follows.

CBER Sentinel White Paper- 13 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM ProgramTable 2. Summary of Prevalent Influenza Vaccine Use † in PRISM between August 1, 2013 and July 31,
2014, by Vaccine Type, and Age Group
VaccinatedPatients

Eligible
Members

High-dose
65-74 Years
120,82975-84 Years
70,96285+ Years
24,794Standard-dose, inactivated65-74 Years
581,42375-84 Years
354,466
85+ Years
129,237†Includes all vaccination episodes

Prevalence ofVaccination per1000 Members

3,037,491
1,465,343
550,100

39.78
48.43
45.07

3,037,491
1,465,343
550,100

191.42241.90234.93

Table 3. Summary of Prevalent Influenza Outcomes† in PRISM between August 1, 2013 and July 31,
2014, by Outcome Definition and Age Group
Patients
with anOutcomeInfluenza diagnosis
Inpatient or Emergency Department
65- 74 years
75- 84 years
85+ years
Inpatient/Principal Diagnosis
65- 74 years
75- 84 years
85+ years
Emergency Department
65- 74 years
75- 84 years
85+ years
Outpatient
65- 74 years
75- 84 years
85+ years
Any Care Setting
65- 74 years
75- 84 years
85+ years

CBER Sentinel White Paper

Eligible
Members

Prevalenceof Outcome
per 1000
Members

3,058
1,931
996

3,037,491
1,465,343
550,100

1.01
1.32
1.81

477
383
188

3,037,491
1,465,343
550,100

0.16
0.26
0.34

1,273
494
180

3,037,491
1,465,343
550,100

0.42
0.34
0.33

11,046
3,951
1,368

3,037,491
1,465,343
550,100

3.64
2.70
2.49

12,9225,274
2,078

3,037,491
1,465,343
550,100

4.25
3.60
3.78

- 14 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program

Pneumonia diagnosis
Inpatient or Emergency Department
65- 74 years
47,577
3,037,491
75- 84 years
48,647
1,465,343
85+ years
33,817550,100
Inpatient/Principal Diagnosis
65- 74 years
10,577
3,037,491
75- 84 years
11,4781,465,343
85+ years
8,414
550,100
Emergency Department
65- 74 years
7,391
3,037,491
75- 84 years
6,290 1,465,343
85+ years
4,172
550,100
Outpatient
65- 74 years
54,711
3,037,491
75- 84 years
46,944
1,465,343
85+ years
30,836550,100
Any Care Setting
65- 74 years
80,035
3,037,491
75- 84 years
72,754
1,465,343
85+ years
48,726550,100
Influenza laboratory test performed
65- 74 years
29,337
3,037,491
75- 84 years
11,9091,465,343
85+ years
3,863
550,100
Influenza rapid test performed and an oseltamivir dispensing
65- 74 years
3,221
3,037,491
75- 84 years
770 1,465,343
85+ years
153
550,100
Influenza laboratory test performed and an influenza diagnosis
65- 74 years
5,564
3,037,491
75- 84 years
1,639
1,465,343
85+ years
393
550,100†Includes all outcome episodes

CBER Sentinel White Paper

15.66
33.20
61.47
3.48
7.83
15.30
2.43
4.29
7.58
18.01
32.04
56.06
26.35
49.65
88.58
9.66
8.13
7.02
1.06
0.53
0.28
1.83
1.12
0.71

- 15 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM ProgramTable 4. Characteristics of Patients Receiving High-Dose and Standard-Dose Influenza Vaccine during
the 2013-2014 Influenza Season (183-day Covariate Window)†
Primary Analysis
Characteristic

High-DoseStandard-Dose
N%
1,065,046
100.0%

N
216,580%
100.0%

75.2
120,82970,95824,793
121,824
94,7479

7.1 (std)‡
55.8%
32.8%
11.4%
56.2%
43.7%
<0.1%

75.4
581,423
354,422
129,201
599,109465,90829

7.2 (std)
54.6%
33.3%
12.1%
56.3%
43.7%
<0.1%

9,533
32,46037,43961,04876,50939
360
22,256
4,879
25,771
5,421
2,676
104,042
24,123
0.7

4.4%
15.0%
17.3%
28.2%
35.3%
<0.1%
0.2%
10.3%
2.3%
11.9%
2.5%
1.2%
48.0%
11.1%
1.8 (std)

48,661
181,745
207,917324,593
410,686285
581
154,861
27,760135,702
23,066
14,881
519,869
122,677
0.9

4.6%
17.1%
19.5%
30.5%
38.6%
<0.1%
0.1%
14.5%
2.6%
12.7%
2.2%
1.4%
48.8%
11.5%
2.0 (std)

Mean number of generic drugs
5.9
Mean number of unique drug classes
5.6
Mean number of filled prescriptions
14.5
Mean number of inpatient hospital encounters
0.1
Mean number of non-acute institutional
encounters
0.1
Mean number of emergency room encounters
0.2
Mean number of ambulatory encounters
8.1
Mean number of other ambulatory encounters
1.8†Includes first vaccination episode; ‡std: standard deviation

4.5 (std)
4.2 (std)
13.6 (std)
0.4 (std)

6.1
5.8
15.2
0.1

4.6 (std)
4.2 (std)
14.1 (std)
0.4 (std)

0.8 (std)
0.6 (std)
8.2 (std)
4.5 (std)

0.1
0.2
8.7
2.1

0.8 (std)
0.6 (std)
9.6 (std)
4.8 (std)

Patients
Patient Characteristics
Mean age
Age: 65–74 years
Age: 75–84 years
Age: 85+ years
Gender (Female)
Gender (Male)
Gender (Unknown)
Recorded History of:
AsthmaBlood Disorders
Chronic Lung Disease
Diabetes
Heart Disease
Influenza Hospitalization
Influenza Vaccination
Kidney Disorders
Liver Disorders
Neurological Conditions
Pneumococcal Vaccine
Pneumonia Hospitalization
Statin Use
Weakened Immune System
Mean Combined Comorbidity Score
Health Service Utilization Intensity:

CBER Sentinel White Paper- 16 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM ProgramTable 5. Characteristics of Patients Receiving High-Dose and Standard-Dose Influenza Vaccine during
the 2013-2014 Influenza Season (548-day Covariate Window)†
Primary Analysis
Characteristic

High-DosePatients
Patient Characteristics
Mean age
Age: 65-74 years
Age: 75-84 years
Age: 85+ years
Gender (Female)
Gender (Male)
Gender (Unknown)
Recorded History of:
AsthmaBlood Disorders
Chronic Lung Disease
Diabetes
Heart Disease
Influenza Hospitalization
Influenza Vaccination
Kidney Disorders
Liver Disorders
Neurological Conditions
Pneumococcal Vaccine
Pneumonia Hospitalization
Statin Use
Weakened Immune System
Mean Combined Comorbidity Score
Health Service Utilization Intensity:
Mean number of generic drugs
Mean number of unique drug classes
Mean number of filled prescriptions
Mean number of inpatient hospital encounters
Mean number of non-acute institutional
encounters
Mean number of emergency room encounters
Mean number of ambulatory encounters
Mean number of other ambulatory encounters

CBER Sentinel White Paper

N
181,082%
100.0%

Standard-DoseN%
889,355
100.0%

75.7
96,516
62,211
22,355
101,696
79,3779

7.0 (std)‡
53.3%
34.4%
12.3%
56.2%
43.8%
<0.1%

75.9
462,230
310,842
116,283
499,798
389,52928

7.2 (std)
52.0%
35.0%
13.1%
56.2%
43.8%
<0.1%

14,713
48,67658,971
59,82389,728317
122,994
28,845
8,831
39,23020,997
6,241
95,91330,205
1.4

8.1%
26.9%
32.6%
33.0%
49.6%
0.2%
67.9%
15.9%
4.9%
21.7%
11.6%
3.4%
53.0%
16.7%
2.5 (std)

73,823
263,316
311,539
317,925
472,005
1,893
583,140
190,794
49,203204,959
92,33532,691
479,813
151,825
1.7

8.3%
29.6%
35.0%
35.7%
53.1%
0.2%
65.6%
21.5%
5.5%
23.0%
10.4%
3.7%
54.0%
17.1%
2.6

9.2
8.4
43.1
0.3

6.7 (std)
5.9 (std)
38.8 (std)0.8 (std)

9.6
8.7
45.3
0.3

6.8 (std)
5.9 (std)
40.1 (std)0.9 (std)

0.4
0.5
24.5
5.3

1.7 (std)
1.2 (std)
20.1 (std)10.7 (std)

0.4
0.5
26.1
6.1

1.8 (std)
1.2 (std)
24.5 (std)11.9 (std)- 17 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program

VII.

STRENGTHS AND LIMITATIONS

A. STRENGTHSThe PRISM distributed database includes a large, geographically diverse population. The large size
supports robust evaluations that may include subgroup analyses. Childhood vaccines and other vaccines
commonly administered in the primary care setting are well-captured in Sentinel and PRISM 23 . Finally,
chart validation of exposures, outcomes and covariates is possible, although it adds to the cost andtimeline of an evaluation.
Use of a quality checked common data model coupled with customizable analytic tools enables PRISM
to address standard queries efficiently. Immunization rates and background rates of health outcomesfor power calculations and feasibility evaluations may be obtained rapidly using existing tools. Additional
reusable and customizable analytic tools continue to be developed to support Sentinel analyses. Formal
assessment of a vaccine’s effectiveness, however, may require a protocol tailored to the specificquestion.

B. LIMITATIONS
Overall, as with other claims-based systems, PRISM data are captured primarily for administrative
purposes rather than research or public health surveillance. Misclassification of exposures andoutcomes can occur, possibly introducing bias. Additionally, the data are not available in real-time. With
the addition of quality checks, the data lag for most partners is at least 9 months. Identification of
specific vaccines by tradename is generally not possible if shared codes are used to identify
immunizations. Availability of laboratory test results and other diagnostic testing results are limited in
PRISM 57 . Many outcomes, as defined by codes, have not been validated using administrative claims
data. Waning vaccine effectiveness may not be feasibly assessed considering the relatively short medianfollow-up time in PRISM and because distinct vaccination histories are not available.

VIII.

CONCLUSION

In certain circumstances, PRISM may be a potentially valuable resource for vaccine effectiveness studies.
Existing Sentinel tools provide important groundwork for future effectiveness work. However, there areimportant factors specific to vaccine effectiveness evaluations that must be considered. First, the levelof evidence required to make regulatory decisions regarding vaccine effectiveness is high; studies
conducted within PRISM would have to be considered within the limitations of observational data andstudy designs, but could be considered, in some situations, as a complement to evidence provided in
randomized clinical trials. Second, limitations in the availability of certain data elements, particularly
laboratory and other diagnostic test results, as well as timeliness, affect the ability to fully and rapidly
assess vaccine effectiveness. Still, the PRISM system, for some specific effectiveness questions, may beable to provide important information that can supplement existing knowledge. Furthermore, theSentinel System, and thus PRISM, is dynamic, with ongoing work to improve the availability of data,
refine methods, and create more rapid and accessible querying tools. The work conducted by this group
into the feasibility of using PRISM to conduct vaccine effectiveness studies should be regarded as an
initial exploration that can be further evaluated and expanded as regulatory questions that might beanswered within PRISM arise.

CBER Sentinel White Paper- 18 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program

IX.
1.

2.
3.
4.

5.

6.
7.

8.
9.
10.
11.

12.

13.

14.
15.

REFERENCES
Nguyen M, Ball R, Midthun K, Lieu TA. The Food and Drug Administration's Post-Licensure Rapid
Immunization Safety Monitoring program: strengthening the federal vaccine safety enterprise.
Pharmacoepidemiol Drug Saf 2012;21 Suppl 1:291-7.
Sentinel System. Sentinel Coordinating Center, 2016. (Accessed November 21, 2016, at
www.sentinelsystem.org.)
Yih WK, Lieu TA, Kulldorff M, et al. Intussusception risk after rotavirus vaccination in U.S. infants.
N Engl J Med 2014;370:503-12.
FDA releases final study results of a Mini-Sentinel postlicensure observational study of rotavirus
vaccines and intussusception. U.S. Food and Drug Administration, 2013. (Accessed November
21, 2016, at http://www.fda.gov/BiologicsBloodVaccines/SafetyAvailability/ucm356758.htm.)
Update: FDA postlicensure rapid immunization safety monitoring program (PRISM) study
demonstrates no statistically significant association between trivalent inactivated influenza
vaccine and febrile seizures in children during the 2010-2011 influenza season. U.S. Food andDrug Administration, 2014. (Accessed November 21, 2016, at
http://www.fda.gov/BiologicsBloodVaccines/SafetyAvailability/ucm397611.htm.)
Kawai AT, Martin D, Kulldorff M, et al. Febrile Seizures After 2010-2011 Trivalent InactivatedInfluenza Vaccine. Pediatrics 2015;136:e848-55.
FDA Sentinel study finds no association between venous thromboembolism and Gardasil
vaccination. U.S. Food and Drug Administration, 2015. (Accessed November 21, 2016, at
http://www.fda.gov/biologicsbloodvaccines/guidancecomplianceregulatoryinformation/postmarketactivities/ucm442585.htm.)
Yih WK, Greene SK, Zichittella L, et al. Evaluation of the risk of venous thromboembolism after
quadrivalent human papillomavirus vaccination among US females. Vaccine 2016;34:172-8.
Clemens J, Brenner R, Rao M, Tafari N, Lowe C. Evaluating new vaccines for developing
countries. Efficacy or effectiveness? JAMA 1996;275:390-7.
Weinberg GA, Szilagyi PG. Vaccine epidemiology: efficacy, effectiveness, and the translational
research roadmap. J Infect Dis 2010;201:1607-10.
Electronic code of federal regulations: 21 CFR 314.126. 2017. (Accessed 30 June, 2017, athttps://www.ecfr.gov/cgi-bin/textidx?SID=3ee286332416f26a91d9e6d786a604ab&mc=true&tpl=/ecfrbrowse/Title21/21tab_02.t
pl.)Electronic code of federal regulations: 21 CFR 601.40-1. 2017. (Accessed 30 June, 2017, athttps://www.ecfr.gov/cgi-bin/textidx?SID=3ee286332416f26a91d9e6d786a604ab&mc=true&tpl=/ecfrbrowse/Title21/21tab_02.t
pl.)Electronic code of federal regulations: 21 CFR 601.90-1. 2017. (Accessed 30 June, 2017, athttps://www.ecfr.gov/cgi-bin/textidx?SID=3ee286332416f26a91d9e6d786a604ab&mc=true&tpl=/ecfrbrowse/Title21/21tab_02.t
pl.)Jackson ML, Rothman KJ. Effects of imperfect test sensitivity and specificity on observational
studies of influenza vaccine effectiveness. Vaccine 2015;33:1313-6.
McGrath LJ, Kshirsagar AV, Cole SR, et al. Influenza vaccine effectiveness in patients on
hemodialysis: an analysis of a natural experiment. Arch Intern Med 2012;172:548-54.

CBER Sentinel White Paper- 19 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program

16.

17.
18.
19.
20.
21.
22.
23.
24.
25.

26.
27.
28.

Panozzo CA, Becker-Dreps S, Pate V, et al. Direct, indirect, total, and overall effectiveness of therotavirus vaccines for the prevention of gastroenteritis hospitalizations in privately insured US
children, 2007-2010. Am J Epidemiol 2014;179:895-909.
Fireman B, Lee J, Lewis N, Bembom O, van der Laan M, Baxter R. Influenza vaccination andmortality: differentiating vaccine effects from bias. Am J Epidemiol 2009;170:650-6.
Jackson ML, Nelson JC. The test-negative design for estimating influenza vaccine effectiveness.
Vaccine 2013;31:2165-8.
Clemens JD, Shapiro ED. Resolving the pneumococcal vaccine controversy: are there alternatives
to randomized clinical trials? Rev Infect Dis 1984;6:589-600.
Curtis LH, Weiner MG, Boudreau DM, et al. Design considerations, architecture, and use of theMini-Sentinel distributed data system. Pharmacoepidemiol Drug Saf 2012;21 Suppl 1:23-31.
Psaty BM, Breckenridge AM. Mini-Sentinel and regulatory science--big data rendered fit andfunctional. N Engl J Med 2014;370:2165-7.
Baker MA, Lieu TA, Li L, et al. A vaccine study design selection framework for the postlicensure
rapid immunization safety monitoring program. Am J Epidemiol 2015;181:608-18.
Baker MA, Nguyen M, Cole DV, Lee GM, Lieu TA. Post-licensure rapid immunization safety
monitoring program (PRISM) data characterization. Vaccine 2013;31 Suppl 10:K98-112.
Placzek H, Madoff LC. The use of immunization registry-based data in vaccine effectivenessstudies. Vaccine 2011;29:399-411.
Yih WK, Lee GM, Lieu TA, et al. Surveillance for adverse events following receipt of pandemic
2009 H1N1 vaccine in the Post-Licensure Rapid Immunization Safety Monitoring (PRISM)
System, 2009-2010. Am J Epidemiol 2012;175:1120-8.
Lanes S, Brown JS, Haynes K, Pollack MF, Walker AM. Identifying health outcomes in healthcare
databases. Pharmacoepidemiol Drug Saf 2015;24:1009-16.
Cadieux G, Tamblyn R. Accuracy of physician billing claims for identifying acute respiratory
infections in primary care. Health Serv Res 2008;43:2223-38.
Hsu VP, Staat MA, Roberts N, et al. Use of active surveillance to validate internationalclassification of diseases code estimates of rotavirus hospitalizations in children. Pediatrics
2005;115:78-82.

29.

Core M-SD. Mini-Sentinel Distributed Database Summary Report Year Five, September 2013 –September 2014. October 2015.
https://www.sentinelinitiative.org/sites/default/files/data/DistributedDatabase/Mini-SentinelCoordinating-Center_Year-5-Report-Scientific-Operations-Center-Data-Group-Activities_0.pdf

30.

Leonard CE RM, Toh D, Kulldorff M, Nelson JC, Gagne JJ, Ouellet-Hellstrom RP, Moeny DG, Mott
KA, By K, Wang SV, Hennessy S. Prospective routine observational monitoring of mirabegronreport 1 of 42016.
Metropolitan and microplitan statistical areas main. (Accessed December 28, 2016, athttps://www.census.gov/population/metro/.)Regions. (Accessed December 28, 2016, athttps://www.census.gov/geo/reference/webatlas/regions.html.)
Mott K, Graham DJ, Toh S, et al. Uptake of new drugs in the early post-approval period in the
Mini-Sentinel distributed database. Pharmacoepidemiol Drug Saf 2016;25:1023-32.
Szklo JNM. Epidemiology: beyond the basics, 2nd edition: Jones and Bartlett Publishers; 2006.
Last JM. A dictionary of epidemiology. New York: Oxford University Press; 2001.
Jackson ML, Yu O, Nelson JC, et al. Further evidence for bias in observational studies of influenza
vaccine effectiveness: the 2009 influenza A(H1N1) pandemic. Am J Epidemiol 2013;178:1327-36.
Rothman KJ, Greenland, S., Lash, T.L. . Modern Epidemiology, 3rd Edition: LWW; 2008.

31.
32.
33.
34.
35.
36.
37.

CBER Sentinel White Paper- 20 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program

38.
39.

40.
41.

42.

43.
44.

45.
46.
47.

48.
49.

50.

51.
52.

53.
54.

55.
56.

Subgroup MCDEADA. Summary and key findings from influenza CIDA request. Modular Program
Report (unpublished data).
Emborg HD, Krause TG, Nielsen L, et al. Influenza vaccine effectiveness in adults 65 years andolder, Denmark, 2015/16 - a rapid epidemiological and virological assessment. Euro Surveill
2016;21.
Farrington CP. Estimation of vaccine effectiveness using the screening method. Int J Epidemiol
1993;22:742-6.
Cohen AL, Taylor T, Jr., Farley MM, et al. An assessment of the screening method to evaluate
vaccine effectiveness: the case of 7-valent pneumococcal conjugate vaccine in the United States.
PLoS One 2012;7:e41785.
Remschmidt C, Rieck T, Bodeker B, Wichmann O. Application of the screening method tomonitor influenza vaccine effectiveness among the elderly in Germany. BMC Infect Dis
2015;15:137.
Maclure M, Fireman B, Nelson JC, et al. When should case-only designs be used for safety
monitoring of medical products? Pharmacoepidemiol Drug Saf 2012;21 Suppl 1:50-61.
Izurieta HS, Thadani N, Shay DK, et al. Comparative effectiveness of high-dose versus standarddose influenza vaccines in US residents aged 65 years and older from 2012 to 2013 using
Medicare data: a retrospective cohort analysis. Lancet Infect Dis 2015;15:293-300.
Stoecker C, Hampton LM, Moore MR. 7-Valent pneumococcal conjugate vaccine and otitis
media: effectiveness of a 2-dose versus 3-dose primary series. Vaccine 2012;30:6256-62.
Jackson LA, Jackson ML, Nelson JC, Neuzil KM, Weiss NS. Evidence of bias in estimates ofinfluenza vaccine effectiveness in seniors. Int J Epidemiol 2006;35:337-44.
Jackson ML, Nelson JC, Weiss NS, Neuzil KM, Barlow W, Jackson LA. Influenza vaccination andrisk of community-acquired pneumonia in immunocompetent elderly people: a populationbased, nested case-control study. Lancet 2008;372:398-405.
Klein NP, Bartlett J, Rowhani-Rahbar A, Fireman B, Baxter R. Waning protection after fifth doseof acellular pertussis vaccine in children. N Engl J Med 2012;367:1012-9.
De Serres G, Skowronski DM, Wu XW, Ambrose CS. The test-negative design: validity, accuracy
and precision of vaccine efficacy estimates compared to the gold standard of randomised
placebo-controlled clinical trials. Euro Surveill 2013;18.
Donauer S, Payne DC, Edwards KM, et al. Determining the effectiveness of the pentavalentrotavirus vaccine against rotavirus hospitalizations and emergency department visits using two
study designs. Vaccine 2013;31:2692-7.
Sullivan SG, Tchetgen Tchetgen EJ, Cowling BJ. Theoretical Basis of the Test-Negative Study
Design for Assessment of Influenza Vaccine Effectiveness. Am J Epidemiol 2016;184:345-53.
Madigan D. FB, Maclure M. (Section I); Madigan D., Simpson S., Hua W., Paredes A., Fireman B.,
Maclure M. (Section II). Mini-Sentinel methods development: case-based methods workgroupreport2011.
Baxter R, Ray GT, Fireman BH. Effect of influenza vaccination on hospitalizations in persons aged
50 years and older. Vaccine 2010;28:7267-72.
Andrade SE HK, Rosofsky R, Liu C, Divan HA, Ekezue B, Selvam N, Kawai AT, Pointonn L, Dublin S.
Birth certificate data matching for the post-licensure rapid immunization safety monitoring(PRISM) program: development of standard file structures for birh and fetal death certificate
data and implementation of matching2016.
Kawai AT, Li L., Andrade S.E., Nguyen, M.D., Selvan M., Lin, N., McMahill-Walraven C.N., SelvamN., Lee, G. Mini-Sentinel/CBER protocol: influenza vaccines and birth outcomes, version 2. 2014.
Payne DC, Staat MA, Edwards KM, et al. Direct and indirect effects of rotavirus vaccination uponchildhood hospitalizations in 3 US Counties, 2006-2009. Clin Infect Dis 2011;53:245-53.

CBER Sentinel White Paper- 21 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program

57.

X.

Raebel MA, Haynes K, Woodworth TS, et al. Electronic clinical laboratory test results data tables:
lessons from Mini-Sentinel. Pharmacoepidemiol Drug Saf 2014;23:609-18.

ACKNOWLEDGMENTS

The authors would like to gratefully acknowledge the contributions of the following organizations andindividuals:
Affiliation
Harvard Pilgrim Health Care Institute

Name
Hana Lipowicz, April Duddy

Sentinel Data Partners

Aetna Informatics, Blue Bell, PA; Harvard
Pilgrim Health Care Institute, Boston, MA;
HealthCore, Inc. Government & Academic
Research, Alexandria, VA; Humana, Inc.,
Comprehensive Health Insights, Miramar, FL;
OptumInsight Life Sciences Inc., Boston, MA

CBER Sentinel White Paper- 22 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM ProgramXI.

APPENDICES

A. APPENDIX A
1. Appendix A: Summary of Prevalent Influenza Vaccine Use † in the Sentinel Distributed
Database between August 1, 2010 and July 31, 2013 and August 1, 2014 and July 31, 2015,
High-dose versus Standard-dose, by Influenza Season and Age Group
VaccinatedPatients
2010-2011 Influenza Season
High-dose
65-74 Years
75-84 Years
85+ Years
Standard-dose,
inactivated65-74 Years
75-84 Years
85+ Years
2011-2012 Influenza Season
High-dose
65-74 Years
75-84 Years
85+ Years
Standard-dose,
inactivated65-74 Years
75-84 Years
85+ Years
2012-2013 Influenza Season
High-dose
65-74 Years
75-84 Years
85+ Years
Standard-dose,
inactivated65-74 Years
75-84 Years
85+ Years
2014-2015 Influenza Season
High-dose
65-74 Years
75-84 Years
85+ Years
Standard-dose,
inactivated
CBER Sentinel White Paper

Eligible
Members

Prevalence ofVaccination per1000 Members

25,811
16,755
5,336

2,378,898
1,206,554
442,202

10.85
13.89
12.07

503,140
314,603104,286

2,378,898
1,206,554
442,202211.50260.75235.8356,247
36,12011,9542,640,734
1,318,291
491,609

21.30
27.40
24.32

527,905
327,720
111,828

2,640,734
1,318,291
491,609199.91248.59227.47

93,79058,16019,9892,806,684
1,379,848
521,812

33.42
42.15
38.31

598,670
364,591
128,864

2,806,684
1,379,848
521,812213.30264.23246.95

190,648
108,908
37,7393,159,277
1,526,993
567,331

60.35
71.32
66.52

- 23 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM ProgramVaccinatedPatients
65-74 Years
75-84 Years
85+ Years†

636,397
384,083
139,111

Eligible
Members
3,159,277
1,526,993
567,331Prevalence ofVaccination per1000 Members
201.44251.53245.20Includes all vaccination episodes

B. APPENDIX B
1. Appendix B: Summary of Prevalent Influenza Vaccine Use † in the Sentinel Distributed
Database between August 1, 2013 and July 31, 2015, Trivalent versus Quadrivalent
Vaccine, by Influenza Season and Age Group
VaccinatedPatients
2013-2014 Influenza Season
Standard dose, inactivated (trivalent)
65-74 Years
175,31375-84 Years
80,13385+ Years
27,896
Standard dose, inactivated (quadrivalent)
65-74 Years
23,64975-84 Years
11,73485+ Years
4,003
2014-2015 Influenza Season
Standard dose, inactivated (trivalent)
65-74 Years
129,72475-84 Years
57,43685+ Years
20,146
Standard dose, inactivated (quadrivalent)
65-74 Years
93,70475-84 Years
50,32085+ Years
17,522†

Eligible
Members

Prevalence ofVaccination per1000 Members

3,037,491
1,465,343
550,100

57.72
54.69
50.71

3,037,491
1,465,343
550,100

7.79
8.01
7.28

3,159,277
1,526,993
567,331

41.06
37.61
35.51

3,159,277
1,526,993
567,331

29.66
32.95
30.88

Includes all vaccination episodes

CBER Sentinel White Paper- 24 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM ProgramC. APPENDICES C1-C4
1. Appendix C1: Summary of Prevalent Influenza Outcomes† in the Sentinel Distributed
Database between August 1, 2010 and July 31, 2011, by Outcome Definitions and Age
Group
Patients with an
OutcomeInfluenza diagnosis
Inpatient or Emergency Department
65- 74 years
75- 84 years
85+ years
Inpatient/Principal Diagnosis
65- 74 years
75- 84 years
85+ years
Emergency Department
65- 74 years
75- 84 years
85+ years
Outpatient
65- 74 years
75- 84 years
85+ years
Any Care Setting
65- 74 years
75- 84 years
85+ years
Pneumonia diagnosis
Inpatient or Emergency Department
65- 74 years
75- 84 years
85+ years
Inpatient/Principal Diagnosis
65- 74 years
75- 84 years
85+ years
Emergency Department
65- 74 years
75- 84 years
85+ years
Outpatient
65- 74 years
75- 84 years
CBER Sentinel White Paper

Eligible
Members

Prevalence of
Outcome per1000 Members

1,570
1,315
884

2,378,898
1,206,554
442,202

0.66
1.09
2.00

228
245
167

2,378,898
1,206,554
442,202

0.10
0.20
0.38

565
299
122

2,378,898
1,206,554
442,202

0.24
0.25
0.28

7,288
3,085
1,209

2,378,898
1,206,554
442,202

3.06
2.56
2.73

8,324
4,061
1,901

2,378,898
1,206,554
442,202

3.50
3.37
4.30

39,06942,44829,5892,378,898
1,206,554
442,202

16.42
35.18
66.91

9,648
11,102
7,864

2,378,898
1,206,554
442,202

4.06
9.20
17.78

5,449
4,914
3,215

2,378,898
1,206,554
442,202

2.29
4.07
7.27

46,222
40,6892,378,898
1,206,554

19.43
33.72

- 25 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program

25,796442,202

Prevalence of
Outcome per1000 Members
58.34

67,38863,86041,9692,378,898
1,206,554
442,202

28.33
52.93
94.91

13,6175,391
1,655

2,378,898
1,206,554
442,202

5.72
4.47
3.74

1,408
369
98

2,378,898
1,206,554
442,202

0.59
0.31
0.22

2,505
792
238

2,378,898
1,206,554
442,202

1.05
0.66
0.54

Patients with an
Outcome85+ years
Any Care Setting
65- 74 years
75- 84 years
85+ years
Influenza laboratory test performed
65- 74 years
75- 84 years
85+ years
Influenza rapid test performed and an
oseltamivir dispensing
65- 74 years
75- 84 years
85+ years
Influenza laboratory test performed andan influenza diagnosis
65- 74 years
75- 84 years
85+ years

Eligible
Members†Includes all outcome episodes

CBER Sentinel White Paper- 26 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program2. Appendix C2: Summary of Prevalent Influenza Outcomes† in the Sentinel Distributed
Database between August 1, 2011 and July 31, 2012, by Outcome Definitions and Age
Group
Patients with an
OutcomeInfluenza diagnosis
Inpatient or Emergency Department
65- 74 years
75- 84 years
85+ years
Inpatient/Principal Diagnosis
65- 74 years
75- 84 years
85+ years
Emergency Department
65- 74 years
75- 84 years
85+ years
Outpatient
65- 74 years
75- 84 years
85+ years
Any Care Setting
65- 74 years
75- 84 years
85+ years
Pneumonia diagnosis
Inpatient or Emergency Department
65- 74 years
75- 84 years
85+ years
Inpatient/Principal Diagnosis
65- 74 years
75- 84 years
85+ years
Emergency Department
65- 74 years
75- 84 years
85+ years
Outpatient
65- 74 years
75- 84 years
85+ years
Any Care Setting
CBER Sentinel White Paper

Eligible
Members

Prevalence of
Outcome per1000 Members

888
899
604

2,640,734
1,318,291
491,609

0.34
0.68
1.23

103
112
86

2,640,734
1,318,291
491,609

0.04
0.08
0.17

293
159
70

2,640,734
1,318,291
491,609

0.11
0.12
0.14

4,388
2,308
946

2,640,734
1,318,291
491,609

1.66
1.75
1.92

5,037
3,024
1,439

2,640,734
1,318,291
491,609

1.91
2.29
2.93

41,41843,91031,0762,640,734
1,318,291
491,609

15.68
33.31
63.21

10,186
11,2348,142

2,640,734
1,318,291
491,609

3.86
8.52
16.56

5,812
5,020
3,354

2,640,734
1,318,291
491,609

2.20
3.81
6.82

48,47542,50727,3672,640,734
1,318,291
491,609

18.36
32.24
55.67

- 27 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program70,863
66,30844,1992,640,734
1,318,291
491,609

Prevalence of
Outcome per1000 Members
26.83
50.30
89.91

10,5194,428
1,445

2,640,734
1,318,291
491,609

3.98
3.36
2.94

534
198
57

2,640,734
1,318,291
491,609

0.20
0.15
0.12

1,107
482
151

2,640,734
1,318,291
491,609

0.42
0.37
0.31

Patients with an
Outcome65- 74 years
75- 84 years
85+ years
Influenza laboratory test performed
65- 74 years
75- 84 years
85+ years
Influenza rapid test performed and an
oseltamivir dispensing
65- 74 years
75- 84 years
85+ years
Influenza laboratory test performed andan influenza diagnosis
65- 74 years
75- 84 years
85+ years

Eligible
Members†Includes all outcome episodes

CBER Sentinel White Paper- 28 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program3. Appendix C3: Summary of Prevalent Influenza Outcomes† in the Sentinel Distributed
Database between August 1, 2012 and July 31, 2013, by Outcome Definitions and Age
Group

Influenza diagnosis
Inpatient or Emergency Department
65- 74 years
75- 84 years
85+ years
Inpatient/Principal Diagnosis
65- 74 years
75- 84 years
85+ years
Emergency Department
65- 74 years
75- 84 years
85+ years
Outpatient
65- 74 years
75- 84 years
85+ years
Any Care Setting
65- 74 years
75- 84 years
85+ years
Pneumonia diagnosis
Inpatient or Emergency Department
65- 74 years
75- 84 years
85+ years
Inpatient/Principal Diagnosis
65- 74 years
75- 84 years
85+ years
Emergency Department
65- 74 years
75- 84 years
85+ years
Outpatient
65- 74 years
75- 84 years
85 + years
Any Care Setting
CBER Sentinel White Paper

Patients with an
OutcomeEligible Members

Prevalence of
Outcome per1000 Members

3,946
3,752
2,508

2,806,684
1,379,848
521,812

1.41
2.72
4.81

619
877
719

2,806,684
1,379,848
521,812

0.22
0.64
1.38

1,863
1,302
542

2,806,684
1,379,848
521,812

0.66
0.94
1.04

16,031
7,605
3,096

2,806,684
1,379,848
521,812

5.71
5.51
5.93

18,386
9,918
4,652

2,806,684
1,379,848
521,812

6.55
7.19
8.92

47,95851,122
36,4112,806,684
1,379,848
521,812

17.09
37.05
69.78

11,42813,0329,581

2,806,684
1,379,848
521,812

4.07
9.44
18.36

7,294
6,657
4,404

2,806,684
1,379,848
521,812

2.60
4.82
8.44

56,16049,39332,9902,806,684
1,379,848
521,812

20.01
35.80
63.22

- 29 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program65- 74 years
75- 84 years
85+ years
Influenza laboratory test performed
65- 74 years
75- 84 years
85+ years
Influenza rapid test performed and an
oseltamivir dispensing
65- 74 years
75- 84 years
85+ years
Influenza laboratory test performed andan influenza diagnosis
65- 74 years
75- 84 years
85+ years

Patients with an
OutcomeEligible Members

81,732
76,172
51,9072,806,684
1,379,848
521,812Prevalence of
Outcome per1000 Members
29.12
55.20
99.47

33,454
15,2465,519

2,806,684
1,379,848
521,812

11.92
11.05
10.58

4,154
1,440
433

2,806,684
1,379,848
521,812

1.48
1.04
0.83

7,499
3,344
1,109

2,806,684
1,379,848
521,812

2.67
2.42
2.13

†Includes all outcome episodes

CBER Sentinel White Paper- 30 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program4. Appendix C4: Summary of Prevalent Influenza Outcomes† in the Sentinel Distributed
Database between August 1, 2014 and July 31, 2015, by Outcome Definitions and Age
Group
Patients with an
OutcomeInfluenza diagnosis
Inpatient or Emergency Department
65- 74 years
75- 84 years
85+ years
Inpatient/Principal Diagnosis
65- 74 years
75- 84 years
85+ years
Emergency Department
65- 74 years
75- 84 years
85+ years
Outpatient
65- 74 years
75- 84 years
85+ years
Any Care Setting
65- 74 years
75- 84 years
85+ years
Pneumonia diagnosis
Inpatient or Emergency Department
65- 74 years
75- 84 years
85+ years
Inpatient/Principal Diagnosis
65- 74 years
75- 84 years
85+ years
Emergency Department
65- 74 years
75- 84 years
85+ years
Outpatient
65- 74 years
75- 84 years
85+ years
Any Care Setting
CBER Sentinel White Paper

Eligible
Members

Prevalence of
Outcome per1000 Members

6,981
6,831
4,750

3,159,277
1,526,993
567,331

2.21
4.47
8.37

1,029
1,601
1,503

3,159,277
1,526,993
567,331

0.33
1.05
2.65

3,752
2,754
1,308

3,159,277
1,526,993
567,331

1.19
1.80
2.31

25,296
12,736
5,374

3,159,277
1,526,993
567,331

8.01
8.34
9.47

28,887
16,3838,061

3,159,277
1,526,993
567,331

9.14
10.73
14.21

50,39951,27835,724

3,159,277
1,526,993
567,331

15.95
33.58
62.97

10,78811,753
8,639

3,159,277
1,526,993
567,331

3.41
7.70
15.23

8,446
7,318
4,571

3,159,277
1,526,993
567,331

2.67
4.79
8.06

58,63050,32932,645

3,159,277
1,526,993
567,331

18.56
32.96
57.54

- 31 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program85,287
77,391
51,636

3,159,277
1,526,993
567,331Prevalence of
Outcome per1000 Members
27.00
50.68
91.02

59,23928,48910,5003,159,277
1,526,993
567,331

18.75
18.66
18.51

9,639
3,813
1,179

3,159,277
1,526,993
567,331

3.05
2.50
2.08

14,975
7,207
2,581

3,159,277
1,526,993
567,331

4.74
4.72
4.55

Patients with an
Outcome65- 74 years
75- 84 years
85+ years
Influenza laboratory test performed
65- 74 years
75- 84 years
85+ years
Influenza rapid test performed and an
oseltamivir dispensing
65- 74 years
75- 84 years
85+ years
Influenza laboratory test performed andan influenza diagnosis
65- 74 years
75- 84 years
85+ years

Eligible
Members*Includes all outcome episodes

CBER Sentinel White Paper- 32 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM ProgramD. APPENDICES D1-D4
1. Appendix D1: Characteristics of Patients Receiving High-Dose and Standard-Dose
Influenza Vaccine during the 2010-2011 Influenza Season (183-day Covariate Window)
Primary Analysis
High-DoseStandard-DoseN%
N%
47,901100.0%
921,966
100.0%

Characteristic
Patients
Patient Characteristics
Mean age
Age: 65–74 years
Age: 75–84 years
Age: 85+ years
Gender (Female)
Gender (Male)
Gender (Unknown)
Recorded History of:
AsthmaBlood Disorders
Chronic Lung Disease
Diabetes
Heart Disease
Influenza Hospitalization
Influenza Vaccination
Kidney Disorders
Liver Disorders
Neurological Conditions
Pneumococcal Vaccine
Pneumonia Hospitalization
Statin Use
Weakened Immune System
Mean Combined Comorbidity Score
Health Service Utilization Intensity:
Mean number of generic drugs
Mean number of unique drug classes
Mean number of filled prescriptions
Mean number of inpatient hospitalencounters
Mean number of non-acute institutional
encounters
Mean number of emergency room
encounters
Mean number of ambulatory encounters
Mean number of other ambulatory
encounters75.4
25,811
16,754
5,336
26,83721,0622

7.1 (std)‡
53.9%
35.0%
11.1%
56.0%
44.0%
<0.1%

75.3
503,140
314,560
104,266
515,504
406,43527

7.1 (std)
54.6%
34.1%
11.3%
55.9%
44.1%
<0.1%

1,983
7,029
8,487
13,01916,736
7
183
3,764
921
5,866
783
575
22,605
5,598
0.6

4.1%
14.7%
17.7%
27.2%
34.9%
0.0%
0.4%
7.9%
1.9%
12.2%
1.6%
1.2%
47.2%
11.7%
1.8 (std)

39,463
152,645
180,518
262,780
344,748
176
1,819
107,962
21,506
115,770
14,142
11,979438,000
108,101
0.8

4.3%
16.6%
19.6%
28.5%
37.4%
0.0%
0.2%
11.7%
2.3%
12.6%
1.5%
1.3%
47.5%
11.7%
1.9 (std)

5.9
5.5
15.6

4.4 (std)
4.1 (std)
14.3 (std)

6.0
5.7
16.2

4.4 (std)
4.1 (std)
14.6 (std)0.1

0.4 (std)

0.1

0.4 (std)

0.1

0.5 (std)

0.1

0.7 (std)

0.2
8.4

0.5 (std)
8.4 (std)

0.2
8.5

0.5 (std)
9.2 (std)

1.6

4.1 (std)

2.0

4.5 (std)†

Includes first vaccination episode; ‡std: standard deviation

CBER Sentinel White Paper- 33 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program2. Appendix D2: Characteristics of Patients Receiving High-Dose and Standard-Dose
Influenza Vaccine during the 2011-2012 Influenza Season (183-day Covariate Window)
Primary Analysis
High-DoseStandard-DoseN%
N%
104,320100.0%
967,388
100.0%

Characteristic
Patients
Patient Characteristics
Mean age
Age: 65–74 years
Age: 75–84 years
Age: 85+ years
Gender (Female)
Gender (Male)
Gender (Unknown)
Recorded History of:
AsthmaBlood Disorders
Chronic Lung Disease
Diabetes
Heart Disease
Influenza Hospitalization
Influenza Vaccination
Kidney Disorders
Liver Disorders
Neurological Conditions
Pneumococcal Vaccine
Pneumonia Hospitalization
Statin Use
Weakened Immune System
Mean Combined Comorbidity Score
Health Service Utilization Intensity:
Mean number of generic drugs
Mean number of unique drug classes
Mean number of filled prescriptions
Mean number of inpatient hospitalencounters
Mean number of non-acute institutional
encounters
Mean number of emergency room
encounters
Mean number of ambulatory encounters
Mean number of other ambulatory
encounters75.5
56,247
36,11911,95458,64545,6723

7.1 (std)‡
53.9%
34.6%
11.5%
56.2%
43.8%
<0.1%

75.4
527,905
327,679
111,804
541,373425,99124

7.1 (std)
54.6%
33.9%
11.6%
56.0%
44.0%
<0.1%

4,376
15,513
18,63429,32036,628
13
216
8,921
2,102
12,813
1,901
1,365
50,55911,713
0.7

4.2%
14.9%
17.9%
28.1%
35.1%
0.0%
0.2%
8.6%
2.0%
12.3%
1.8%
1.3%
48.5%
11.2%
1.8 (std)

41,854
160,773
189,643
283,053
363,587
156
806
121,632
23,461
121,559
17,31012,889473,275
111,901
0.8

4.3%
16.6%
19.6%
29.3%
37.6%
0.0%
0.1%
12.6%
2.4%
12.6%
1.8%
1.3%
48.9%
11.6%
2.0 (std)

6.0
5.7
15.4

4.4 (std)
4.1 (std)
13.9 (std)

6.1
5.8
16.0

4.4 (std)
4.1 (std)
14.3 (std)0.1

0.4 (std)

0.1

0.4 (std)

0.1

0.5 (std)

0.1

0.6 (std)

0.2
8.2

0.5 (std)
8.4 (std)

0.2
8.4

0.5 (std)
9.1 (std)

1.8

4.3 (std)

2.0

4.5 (std)†

Includes first vaccination episode; ‡std: standard deviation

CBER Sentinel White Paper- 34 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program3. Appendix D3: Characteristics of Patients Receiving High-Dose and Standard-Dose
Influenza Vaccine during the 2012-2013 Influenza Season (183-day Covariate Window)
Primary Analysis
High-DoseStandard-DoseN%
N%
171,939100.0%
1,092,037
100.0%

Characteristic
Patients
Patient Characteristics
Mean age
Age: 65–74 years
Age: 75–84 years
Age: 85+ years
Gender (Female)
Gender (Male)
Gender (Unknown)
Recorded History of:
AsthmaBlood Disorders
Chronic Lung Disease
Diabetes
Heart Disease
Influenza Hospitalization
Influenza Vaccination
Kidney Disorders
Liver Disorders
Neurological Conditions
Pneumococcal Vaccine
Pneumonia Hospitalization
Statin Use
Weakened Immune System
Mean Combined Comorbidity Score
Health Service Utilization Intensity:
Mean number of generic drugs
Mean number of unique drug classes
Mean number of filled prescriptions
Mean number of inpatient hospitalencounters
Mean number of non-acute institutional
encounters
Mean number of emergency room
encounters
Mean number of ambulatory encounters
Mean number of other ambulatory
encounters

75.4
93,79058,16019,98996,64675,2858

7.1 (std)‡
54.5%
33.8%
11.6%
56.2%
43.8%
<0.1%

75.3
598,670
364,544
128,823
611,006
481,00328

7.2 (std)
54.8%
33.4%
11.8%
56.0%
44.0%
<0.1%

7,201
25,14229,432
48,12759,85640
400
15,8293,652
20,422
3,619
2,192
81,047
19,065
0.7

4.2%
14.6%
17.1%
28.0%
34.8%
0.0%
0.2%
9.2%
2.1%
11.9%
2.1%
1.3%
47.1%
11.1%
1.8 (std)

47,842
180,100
207,747
323,485
410,155282
816
144,931
27,369135,606
22,581
14,516
519,782
125,291
0.9

4.4%
16.5%
19.0%
29.6%
37.6%
0.0%
0.1%
13.3%
2.5%
12.4%
2.1%
1.3%
47.6%
11.5%
2.0 (std)

5.7
5.4
14.5

4.4 (std)
4.1 (std)
13.7 (std)

5.9
5.6
15.0

4.5 (std)
4.1 (std)
14.0 (std)0.1

0.4 (std)

0.1

0.4 (std)

0.1

0.7 (std)

0.1

0.7 (std)

0.2
8.0

0.6 (std)
8.1 (std)

0.2
8.4

0.6 (std)
9.4 (std)

1.8

4.3 (std)

2.0

4.6 (std)†

Includes first vaccination episode; ‡std: standard deviation

CBER Sentinel White Paper- 35 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program4. Appendix D4: Characteristics of Patients Receiving High-Dose and Standard-Dose
Influenza Vaccine during the 2014-2015 Influenza Season (183-day Covariate Window)
Characteristic
Patients
Patient Characteristics
Mean age
Age: 65–74 years
Age: 75–84 years
Age: 85+ years
Gender (Female)
Gender (Male)
Gender (Unknown)
Recorded History of:
AsthmaBlood Disorders
Chronic Lung Disease
Diabetes
Heart Disease
Influenza Hospitalization
Influenza Vaccination
Kidney Disorders
Liver Disorders
Neurological Conditions
Pneumococcal Vaccine
Pneumonia Hospitalization
Statin Use
Weakened Immune System
Mean Combined Comorbidity Score
Health Service Utilization Intensity:
Mean number of generic drugs
Mean number of unique drug classes
Mean number of filled prescriptions
Mean number of inpatient hospitalencounters
Mean number of non-acute institutional
encounters
Mean number of emergency room
encounters
Mean number of ambulatory encounters
Mean number of other ambulatory
encounters†N
337,292

Primary Analysis
Standard-Dose%
N%
100.0%
1,159,541
100.0%

75.1
190,648
108,905
37,739189,222
148,05911

7.1 (std)‡
56.5%
32.3%
11.2%
56.1%
43.9%
<0.1%

75.4
636,397
384,049
139,095
655,636
503,88817

7.2 (std)
54.9%
33.1%
12.0%
56.5%
43.5%
<0.1%

15,33150,824
57,136
95,073
119,63269
569
37,723
8,082
40,064
8,487
3,856
168,579
37,849
0.7

4.5%
15.1%
16.9%
28.2%
35.5%
0.0%
0.2%
11.2%
2.4%
11.9%
2.5%
1.1%
50.0%
11.2%
1.9 (std)

53,537
202,673
223,004
357,641
449,545378
716
178,032
31,680148,594
25,772
15,218582,174
132,637
1.0

4.6%
17.5%
19.2%
30.8%
38.8%
0.0%
0.1%
15.4%
2.7%
12.8%
2.2%
1.3%
50.2%
11.4%
2.1 (std)

6.0
5.7
14.8

4.5 (std)
4.2 (std)
13.6 (std)

6.2
5.9
15.3

4.6 (std)
4.2 (std)
14.1 (std)0.1

0.4 (std)

0.1

0.5 (std)

0.1

0.7 (std)

0.1

0.8 (std)

0.2
8.2

0.6 (std)
8.3 (std)

0.2
8.7

0.6 (std)
9.8 (std)

1.8

4.4 (std)

2.1

4.8 (std)

High-Dose‡

Includes first vaccination episode; std: standard deviation

CBER Sentinel White Paper- 36 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM ProgramE. APPENDICES E1-E4
1. Appendix E1: Characteristics of Patients Receiving High-Dose and Standard-Dose Influenza
Vaccine during the 2010-2011 Influenza Season (548-day Covariate Window)
Characteristic
Patients
Patient Characteristics
Mean age
Age: 65-74 years
Age: 75-84 years
Age: 85+ years
Gender (Female)
Gender (Male)
Gender (Unknown)
Recorded History of:
AsthmaBlood Disorders
Chronic Lung Disease
Diabetes
Heart Disease
Influenza Hospitalization
Influenza Vaccination
Kidney Disorders
Liver Disorders
Neurological Conditions
Pneumococcal Vaccine
Pneumonia Hospitalization
Statin Use
Weakened Immune System
Mean Combined Comorbidity Score
Health Service Utilization Intensity:
Mean number of generic drugs
Mean number of unique drug classes
Mean number of filled prescriptions
Mean number of inpatient hospital encounters
Mean number of non-acute institutional
encounters
Mean number of emergency room encounters
Mean number of ambulatory encounters
Mean number of other ambulatory encounters

Primary Analysis
High-DoseStandard-DoseN%
N%
38,141100.0% 734,965
100.0%
75.8
19,797
13,785
4,559
21,407
16,7322

7.1 (std)‡
51.9%
36.1%
12.0%
56.1%
43.9%
<0.1%

75.7
382,703262,363
89,899411,748
323,19225

7.1 (std)
52.1%
35.7%
12.2%
56.0%
44.0%
<0.1%

2,894
9,901
12,30912,143
18,82330
25,180
4,735
1,680
8,454
3,279
1,279
20,197
6,632
1.3

7.6%
26.0%
32.3%
31.8%
49.4%
0.1%
66.0%
12.4%
4.4%
22.2%
8.6%
3.4%
53.0%
17.4%
2.4 (std)

58,167
210,978
254,962
247,623
384,327786
439,325
127,668
36,925
168,486
59,67524,922
393,457
128,716
1.6

7.9%
28.7%
34.7%
33.7%
52.3%
0.1%
59.8%
17.4%
5.0%
22.9%
8.1%
3.4%
53.5%
17.5%
2.6

9.4
8.4
47.1
0.4

6.6 (std)
5.6 (std)
41.0 (std)0.8 (std)

9.7
8.7
49.3
0.4

6.6 (std)
5.7 (std)
41.7 (std)0.9 (std)

0.3
0.5
25.4
4.6

1.1 (std)
1.0 (std)
20.5 (std)10.3 (std)

0.3
0.4
25.7
5.7

1.9 (std)
1.0 (std)
23.2 (std)11.1 (std)†Includes first vaccination episode; ‡std: standard deviation

CBER Sentinel White Paper- 37 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program2. Appendix E2: Characteristics of Patients Receiving High-Dose and Standard-Dose Influenza
Vaccine during the 2011-2012 Influenza Season (548-day Covariate Window)
Characteristic
Patients
Patient Characteristics
Mean age
Age: 65-74 years
Age: 75-84 years
Age: 85+ years
Gender (Female)
Gender (Male)
Gender (Unknown)
Recorded History of:
AsthmaBlood Disorders
Chronic Lung Disease
Diabetes
Heart Disease
Influenza Hospitalization
Influenza Vaccination
Kidney Disorders
Liver Disorders
Neurological Conditions
Pneumococcal Vaccine
Pneumonia Hospitalization
Statin Use
Weakened Immune System
Mean Combined Comorbidity Score
Health Service Utilization Intensity:
Mean number of generic drugs
Mean number of unique drug classes
Mean number of filled prescriptions
Mean number of inpatient hospital encounters
Mean number of non-acute institutional
encounters
Mean number of emergency room encounters
Mean number of ambulatory encounters
Mean number of other ambulatory encounters†

Primary Analysis
High-DoseStandard-DoseN%
N%
85,067100.0% 794,299
100.0%
75.8
44,19330,475
10,39947,99037,0743

7.0 (std)‡
52.0%
35.8%
12.2%
56.4%
43.6%
<0.1%

75.8
415,170
280,727
98,402445,513348,76323

7.1 (std)
52.3%
35.3%
12.4%
56.1%
43.9%
<0.1%

6,603
23,032
28,08927,94442,34273
56,631
11,464
3,829
19,1427,984
2,925
45,80014,291
1.3

7.8%
27.1%
33.0%
32.8%
49.8%
0.1%
66.6%
13.5%
4.5%
22.5%
9.4%
3.4%
53.8%
16.8%
2.4 (std)

62,946
231,443
277,714275,225
417,614807
511,739
148,959
40,972
182,187
71,40527,260434,646
137,133
1.6

7.9%
29.1%
35.0%
34.7%
52.6%
0.1%
64.4%
18.8%
5.2%
22.9%
9.0%
3.4%
54.7%
17.3%
2.6

9.6
8.6
46.7
0.3

6.7 (std)
5.7 (std)
40.3 (std)0.8 (std)

9.9
8.8
48.5
0.4

6.7 (std)
5.7 (std)
41.2 (std)0.9 (std)

0.3
0.5
25.1

1.2 (std)
1.1 (std)
20.7 (std)

0.3
0.5
25.6

1.5 (std)
1.1 (std)
23.2 (std)

5.0

10.9 (std)

5.8

11.3

(std)

‡

Includes first vaccination episode; std: standard deviation

CBER Sentinel White Paper- 38 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program3. Appendix E3: Characteristics of Patients Receiving High-Dose and Standard-Dose Influenza
Vaccine during the 2012-2013 Influenza Season (548-day Covariate Window)
Characteristic
Patients
Patient Characteristics
Mean age
Age: 65-74 years
Age: 75-84 years
Age: 85+ years
Gender (Female)
Gender (Male)
Gender (Unknown)
Recorded History of:
AsthmaBlood Disorders
Chronic Lung Disease
Diabetes
Heart Disease
Influenza Hospitalization
Influenza Vaccination
Kidney Disorders
Liver Disorders
Neurological Conditions
Pneumococcal Vaccine
Pneumonia Hospitalization
Statin Use
Weakened Immune System
Mean Combined Comorbidity Score
Health Service Utilization Intensity:
Mean number of generic drugs
Mean number of unique drug classes
Mean number of filled prescriptions
Mean number of inpatient hospital encounters
Mean number of non-acute institutional
encounters
Mean number of emergency room encounters
Mean number of ambulatory encounters
Mean number of other ambulatory encounters

Primary Analysis
High-DoseStandard-DoseN%
N%
138,692
100.0% 875,753
100.0%
75.8
72,44849,144
17,10078,04460,6408

7.0 (std)‡
52.2%
35.4%
12.3%
56.3%
43.7%
<0.1%

75.8
460,042
305,749
109,962
490,473385,25525

7.1 (std)
52.5%
34.9%
12.6%
56.0%
44.0%
<0.1%

10,80637,14544,69845,76568,552
91
91,081
20,025
6,461
30,72914,184
4,696
75,522
23,117
1.3

7.8%
26.8%
32.2%
33.0%
49.4%
0.1%
65.7%
14.4%
4.7%
22.2%
10.2%
3.4%
54.5%
16.7%
2.5 (std)

69,989255,825
300,834
306,787459,690604
559,621
173,795
47,144
199,595
86,19630,286
485,009
149,898
1.6

8.0%
29.2%
34.4%
35.0%
52.5%
0.1%
63.9%
19.8%
5.4%
22.8%
9.8%
3.5%
55.4%
17.1%
2.6

9.5
8.6
45.4
0.3

6.6 (std)
5.7 (std)
39.6 (std)0.8 (std)

9.8
8.9
47.4
0.3

6.7 (std)
5.7 (std)
40.4 (std)0.9 (std)

0.3
0.5
24.4
5.2

1.3 (std)
1.1 (std)
20.2 (std)10.4 (std)

0.3
0.5
25.5
6.0

1.5 (std)
1.1 (std)
23.2 (std)11.5 (std)†Includes first vaccination episode; ‡std: standard deviation

CBER Sentinel White Paper- 39 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program4. Appendix E4: Characteristics of Patients Receiving High-Dose and Standard-Dose Influenza
Vaccine during the 2014-2015 Influenza Season (548-day Covariate Window)
Characteristic
Patients
Patient Characteristics
Mean age
Age: 65-74 years
Age: 75-84 years
Age: 85+ years
Gender (Female)
Gender (Male)
Gender (Unknown)
Recorded History of:
AsthmaBlood Disorders
Chronic Lung Disease
Diabetes
Heart Disease
Influenza Hospitalization
Influenza Vaccination
Kidney Disorders
Liver Disorders
Neurological Conditions
Pneumococcal Vaccine
Pneumonia Hospitalization
Statin Use
Weakened Immune System
Mean Combined Comorbidity Score
Health Service Utilization Intensity:
Mean number of generic drugs
Mean number of unique drug classes
Mean number of filled prescriptions
Mean number of inpatient hospital encounters
Mean number of non-acute institutional
encounters
Mean number of emergency room encounters
Mean number of ambulatory encounters
Mean number of other ambulatory encounters

Primary Analysis
High-DoseStandard-DoseN%N%
261,552
100.0% 910,249
100.0%
75.6
140,344
88,87932,329145,981
115,560
11

7.0 (std)‡
53.7%
34.0%
12.4%
55.8%
44.2%
<0.1%

76.0
470,873
318,533
120,843
512,882
397,35017

7.2 (std)
51.7%
35.0%
13.3%
56.3%
43.7%
<0.1%

21,67670,295
81,53586,189130,419295
172,295
44,707
13,306
57,11429,516
8,257
142,138
43,638
1.5

8.3%
26.9%
31.2%
33.0%
49.9%
0.1%
65.9%
17.1%
5.1%
21.8%
11.3%
3.2%
54.3%
16.7%
2.5 (std)

75,626
274,057
308,797329,786
487,503
1,256
578,329
206,064
52,001
211,318
91,10931,942501,069
153,354
1.8

8.3%
30.1%
33.9%
36.2%
53.6%
0.1%
63.5%
22.6%
5.7%
23.2%
10.0%
3.5%
55.0%
16.8%
2.7

9.3
8.5
43.3
0.3

6.7 (std)
5.8 (std)
38.5 (std)0.8 (std)

9.7
8.8
45.2
0.3

6.8 (std)
5.9 (std)40.0 (std)0.9 (std)

0.4
0.5
24.7
5.1

1.7 (std)
1.2 (std)
20.3 (std)10.7 (std)

0.4
0.5
26.1
6.1

1.9 (std)
1.3 (std)
24.6 (std)11.9 (std)†Includes first vaccination episode; ‡std: standard deviation

CBER Sentinel White Paper- 40 - Exploring the Feasibility of Conducting Vaccine
Effectiveness Studies in Sentinel’s PRISM Program