Methodologies for Observational Studies Comparing Inpatient COVID-19 Treatments: IL-6 Receptor Inhibitors and JAK Inhibitors for Hospitalized COVID-19 Patients Receiving Corticosteroid Therapy
Conditions: COVID-19
Sponsor: Aetion, Inc.

Methodologies for observational studies comparing inpatient
COVID-19 treatments:
IL-6 receptor inhibitors and JAK inhibitors for hospitalized COVID-19
patients receiving corticosteroid therapy
[Research Objective 3]
Version: Draft v8
Version Date: [01/11/2023]
Nicolle Gatto (Principal Investigator)
Chief Science Officer, Aetion Inc., New York, NY
nicolle.gatto@aetion.com
Vera Frajzyngier (Co-Investigator)
Executive Principal Scientist, Aetion Inc., New York, NY
vera.frajzyngier@aetion.com
[FDA INVESTIGATOR] (Co-Investigator)
Aloka Chakravarty
Director, Data Analytics, FDA/OC/ODAR, Silver Spring
aloka.chakravarty@fda.hhs.gov
Full list of Aetion collaborators: Aidan M. Baglivo, BS; Andrew R. Weckstein, BA; David Lenis, MS, PhD;
Elisha Z. Beebe, BS; Elizabeth M. Garry, Ph.D., MPH; Nicolle M. Gatto, Ph.D., MPH, FISPE; Priya Govil,
BA; Sarah E. Vititoe, MPH; Vera Frajzyngier, Ph.D.
Full list of FDA collaborators: Aloka Chakravarty, Ph.D., MStat; Andrew Clerman, MD, Ph.D; Anil Rajpal,
MD, MPH; Donna R. Rivera, PharmD, MSc; Kenneth Quinto, MD, MPH; Marie C. Bradley, Ph.D., MSPH,
MPharm; Silvia Perez-Vilar, PharmD, Ph.D.; Tamar Lasky, Ph.D, FISPE.; MStat.
Affiliations: Aetion, Inc., New York, NY, USA [AB, ARW, DL, EB, EMG, NMG, PG, SEV, VF]; Division of
Pulmonology, Allergy, and Critical Care, Office of Immunology and Inflammation, Office of New Drugs,
Center for Drug Evaluation and Research, U.S. Food and Drug Administration, Silver Spring, MD, USA
[AC]; Division of Epidemiology, Office of Surveillance and Epidemiology, Center for Drug Evaluation and
Research, U.S. Food and Drug Administration, Silver Spring, MD, USA [MCB, SPV]; Division of
Rheumatology and Transplant Medicine, Office of Immunology and Inflammation, Office of New Drugs,
Center for Drug Evaluation and Research, U.S. Food and Drug Administration, Silver Spring, MD, USA
[AR]; Oncology Center of Excellence, U.S. Food and Drug Administration, Silver Spring, MD [DRR]; Office
of the Commissioner, U.S. Food and Drug Administration, Silver Spring, MD, USA [AC, TL]; Office of
Medical Policy, Center for Drug Evaluation and Research, U.S. Food and Drug Administration, Silver
Spring, MD, USA [KQ]
1

Disclaimer: This protocol reflects the views of the authors and should not be construed to represent FDA’s
views or policies.
Aetion disclosure: As a commercial supplier of real world data analytics (including both commercially
licensed software and commercially available analytic services), Aetion has been and is involved in
multiple studies and other projects for manufacturers of vaccines, therapeutics, preventative medications,
medical devices, and other medical products. This work includes involvement in studies that may be
submitted to FDA for regulatory purposes, including in support of product approval, clearance, licensure,
or authorization. Aetion’s involvement in studies and other projects spans across therapeutic areas and
includes work for multiple manufacturers of medical products that have been or may be authorized for the
treatment or prevention of COVID-19.
2

Contents
Abbreviations
5
Version History and Amendments
6
1. BACKGROUND AND RATIONALE
7
1.1 Collaboration between Aetion and the US Food and Drug Administration (FDA)
7
1.2 Rationale for Inpatient Comparative COVID-19 Treatment Example
7
1.3 Background on Immunomodulators for COVID-19: Corticosteroids, IL-6 Receptor
Inhibitors, and JAK Inhibitors
7
2. RESEARCH QUESTION AND OBJECTIVES
9
3. RESEARCH METHODS
12
3.1 Study Design
12
3.2 Study Design Diagram for the Illustrative Example
13
3.3 Setting
14
3.3.1 Data sources
14
3.3.2 Population
15
3.3.3 Cohort Entry, Treatment Index and Inclusion and Exclusion Criteria Assessment
Windows
16
3.3.4 Follow-up
17
3.3.5 Inclusion Criteria
17
3.3.6 Exclusion Criteria
18
3.4 Variables
19
3.4.1 Cohort Definition
20
3.4.2 Exposure Definition
20
3.4.3 Outcome Definition
20
3.4.4 Covariates
20
3.5 Study Size
22
3.6 Data Management
24
3.7 Analysis
25
3.7.1 Application of Single-phase Prespecification Approach (SPA)
25
3.7.2 Application of Intentional Multi-phase Approach (IMA)
26
3.7.3 Sensitivity Analyses
32
4. POTENTIAL LIMITATIONS
32
4.1 Addressed in Design and Analyses
32
4.2 To Be Considered in Interpretation Only
34
5. STRENGTHS
35
6. HUMAN SUBJECTS
36
7. REFERENCES
36
3

8. APPENDICES
46
Appendix A. SPACE Template and / or other documentation supporting development of a
hypothetical target clinical trial.
46
Appendix B: Description of Phased Study Approach
46
Appendix C: Detailed context and rationale for covariates, including DAG
46
Appendix D: Full START-RWE Tables
47
Appendix D1: START-RWE Table. Index Date (Time T) defining criterion
47
Appendix D2: START-RWE Table: Follow up
48
Appendix D3: START-RWE Table. Inclusion Criteria
50
Appendix D4: START-RWE Table. Exclusion Criteria
52
Appendix D5: START-RWE Table: Outcome
54
Appendix D6: START-RWE Table. Covariates
55
Appendix D7: START-RWE Table: Analyses
72
Appendix E: Data Management Process Details
74
Appendix F: Code Lists
75
Appendix G: Exploratory Analysis Results
75
Appendix H: Potential Addendum 1: Final List of Covariates Included in Model(s) and
Contingencies Pursued
75
Appendix I: References for Appendix
75
4

Abbreviations
Abbreviation
Description
AEP
Aetion Evidence Platform
BAR
Baricitinib
CDC
Centers for Disease Control and Prevention
CI
confidence interval
CMS
Centers for Medicare & Medicaid Services
COVID-19
coronavirus disease 2019
CPT
current procedural terminology
CS
corticosteroids
CSI
corticosteroids of interest
DAG
directed acyclic graph
ECMO
extracorporeal membrane oxygenation
ER
emergency room
EUA
Emergency Use Authorization
FDA
US Food and Drug Administration
HCPCS
Healthcare Common Procedure Coding System
HFO
high flow oxygen
HR
hazard ratio
ICD-10-CM
International Classification of Diseases, Tenth Revision, Clinical Modification
ICU
intensive care unit
IL6Ri
interleukin-6 receptor inhibitors
IMA
intentional multiphase approach
IMV
invasive mechanical ventilation
IPW
inverse probability weighting
IQR
interquartile range
JAKi
Janus kinase inhibitors
MCM
medical countermeasures
mWHO
modified WHO severity score
NDC
national drug code
NIH
National Institutes of Health
NIV
noninvasive ventilation
O2
supplemental oxygen
PS
propensity score
5

PSW
propensity score weighting
RDV
remdesivir
RWD
real-world data
SAR
sarilumab
SARS-COV-2
severe acute respiratory syndrome coronavirus 2
SPA
single-phase prespecification approach
SoC
standard of care
SPACE
Structured Preapproval and Postapproval Comparative Study Design Framework to Generate Valid
and Transparent Real-World Evidence for Regulatory Decisions
SPIFD
Structured Process to Identify Fit-For Purpose Data
STaRT-RWE
Structured template for planning and reporting on the implementation of real world evidence studies
TCZ
tocilizumab
TOF
tofacitinib
T
day of treatment initiation
US
United States
WHO
World Health Organization
Version History and Amendments
Version
date
Version
number
Section of
protocol
Amendment or update
Reason
11 Jan
2023
v8 (v7
posted on
clinicaltrials.
gov 2 Dec
2022))
●
Covariates
table
●
Footnote
●
STaRT-RWE
tables
(Appendix D)
●
Removed the facility level
prescribing practices variable from
covariates table
●
Removed the following footnote on
each protocol page: “This
document contains confidential
and proprietary business
information. Not disclosable
pursuant to FOIA Exemption b(4).
●
Updated STaRT-RWE table to
correspond with revisions to code
list
●
Variable highly
correlated with other
measured variables
●
Footer not relevant
●
STaRT-RWE table
revised to reflect
measures to be used
for implementation
All changes were made
prior to study
implementation.
6

1. BACKGROUND AND RATIONALE
1.1 Collaboration between Aetion and the US Food and Drug Administration (FDA)
Aetion is under contract with the US Food and Drug Administration (FDA) via a Broad Agency
Announcement to use the Aetion Evidence Platform® (AEP) to develop a system of studies and
a systematic process for the rapid assessment of COVID-19 inpatient medical
countermeasures. This includes identifying and analyzing fit-for-purpose data sources to
characterize inpatient COVID-19 patient populations, risk factors for COVID-19-related
complications, and to explore methods for scientific evaluation of potential interventions for
inpatient treatment of COVID-19.
1.2 Rationale for Inpatient Comparative COVID-19 Treatment Example
Understanding of coronavirus disease-2019 (COVID-19) pathogenesis and treatment evolved
rapidly after the emergence of the SARS-CoV-2 virus in late 2019. Clinical and in-vitro studies
have identified a dysregulated or overactive host immune response to SARS-CoV-2 infection as
a key feature of severe COVID-19 disease [Chen 2020, Zhou 2020, Blanco-Melo 2020, Ruan
2020, Del Valle 2020]. Immunomodulators such as corticosteroids (CS), interleukin-6 receptor
inhibitors (IL6Ri), and Janus kinase inhibitors (JAKi) have therefore been explored as potential
strategies for controlling overactive immune responses in COVID-19 [Burrage 2020,
​Ngamprasertchai 2022, Kim 2020, Mehta 2020].
Given the limited number of recommended treatments early in the pandemic, much of the initial
observational research evaluating immunomodulators for COVID-19 has relied on non-user
comparisons [Gatto 2020, Hu 2020, Nelson 2020, Li 2020]. With the subsequent emergence of
multiple immunomodulator options for management of inpatient COVID-19, it is now feasible to
apply the active-comparator, new-user design, which is generally recommended to mitigate bias
in observational (real-world data) studies (hereafter referred to as “observational” studies) [Lund
2015, Franklin 2017]. For the current collaboration, this active comparator approach will be
applied to an illustrative observational study of two regimens indicated for similar COVID-19
inpatient populations (and, thus, ideal for the active comparator design): IL6Ri versus JAKi as
add-on therapy to CS. Given limited comparative research for IL6Ri versus JAKi regimens, this
comparison is also a relevant and clinically meaningful illustrative example to improve upon
methods for conducting observational studies to evaluate the effectiveness of inpatient COVID
treatments.
1.3 Background on Immunomodulators for COVID-19: Corticosteroids, IL-6 Receptor
Inhibitors, and JAK Inhibitors
In June 2020, the systemic CS dexamethasone became the first immunomodulator in which a
clinical benefit was demonstrated in patients hospitalized for COVID-19 receiving oxygen or
mechanical ventilation [RECOVERY Collaborative Group 2020]. Since then, dexamethasone
has become the most commonly used inpatient COVID-19 treatment in the United States
7

[Weckstein 2021]. In 2021, several large randomized controlled trials (RCTs) demonstrated that
IL6Ri or JAKi, when added to dexamethasone, can yield further mortality benefits for certain
subgroups of patients hospitalized for COVID-19 [REMAP-CAP Investigators 2021, RECOVERY
Collaborative Group 2021, Marconi 2021]. As of February 28th, 2022 (the end of our study
period), NIH guidelines [NIH Treatment Guidelines, 2022 (a)] recommended the addition of
either IL6Ri or JAKi to dexamethasone for patients who are hospitalized and require
supplemental oxygen (O2), non-invasive ventilation (NIV), or high-flow oxygen (HFO) and who
have systemic inflammation and rapidly increasing oxygen needs (no recommended timing of
administration). IL6Ri were additionally recommended for patients requiring invasive mechanical
ventilation (IMV) or extracorporeal membrane oxygenation (ECMO) within 24 hours after
admission to the intensive care unit (ICU) [NIH Treatment Guidelines, 2022 (a)]; JAKi were not
recommended for this patient population as of February 20221.
By the end of February 2022, two IL6Ri medications (tocilizumab, TCZ; sarilumab, SAR) were
included in NIH treatment guidelines for use in addition to systemic CS for therapeutic
management of adults hospitalized with COVID-192. SAR is only recommended when TCZ is
not available and is not currently authorized for emergency use or approved by FDA for
COVID-19 related indications. Intravenous TCZ and subcutaneous SAR are monoclonal
antibodies that inhibit binding of interleukin-6, a pro-inflammatory cytokine thought to play a
critical role in COVID-19-induced systemic inflammation [Zhong, 2020]. NIH recommendations
for use of TCZ in patients requiring supplemental O2 or NIV/HFO were largely based on results
of the RECOVERY trial, which found a reduction in 28-day mortality for TCZ among those with
elevated C-reactive protein requiring oxygen supplementation (RR 0.85; 95% CI 0.76–0.94)
[RECOVERY Collaborative Group 2021]. Recommendations for IL6Ri among patients requiring
IMV/ECMO recently admitted to the ICU were informed by REMAP-CAP trial findings that
receipt of TCZ or SAR within 24 hours of ICU admission among critically-ill patients was
associated with improvements in inpatient survival (HR 1.61; 95% CI 1.25-2.08) [REMAP-CAP
Investigators 2021]. The totality of evidence from smaller RCTs [Stone 2020, Rosas 2021] and
meta-analyses [Juul 2021, Kow 2021, Albuquerque 2022, REACT Working Group 2021, Rezaei
2021, Zhang 2022, Tleyjeh 2021] has varied for the effectiveness of IL6Ri in COVID-19,
particularly with regard to the populations most likely to benefit. A meta-analysis of 21 RCTs
conducted by the WHO [REACT Working Group 2021] found that TCZ was associated with a
meaningful mortality reduction among those receiving supplemental O2 or NIV/HFO, with
uncertain benefit for those receiving IMV/ECMO.
Two JAKi medications (baricitinib, BAR; tofacitinib, TOF) were included in NIH treatment
guidelines by the end of the study period3. Like SAR, TOF is only recommended when BAR is
not available and does not currently have EUA or approval for COVID-19 related indications.
3 JAKi were first included in NIH guidelines on July 9th, 2021.
2 IL6Ri were first included in NIH guidelines on April 23rd, 2021.
1 Of note, updated NIH guidance released in August 9th, 2022 [NIH Treatment Guidelines, 2022 (b)] added BAR as
an option for patients receiving IMV/ECMO, and expanded the recommended TCZ and BAR populations by removing
requirements for systemic inflammation and rapidly increasing oxygen needs (among NIV/HFO patients) and for
admission to ICU within 24 hours (among IMV/ECMO patients). However, given the study period for our illustrative
example ends on February 28th, 2022, we focus here on NIH guidance available in February 2022 [NIH Treatment
Guidelines, 2022 (a)].
8

BAR and TOF are oral JAKi that block release of pro-inflammatory cytokines implicated in
COVID-19 immuno-pathophysiology [Bronte, 2020, Petrone 2021, Stebbing 2020, Marconi
2020]. In addition to known immunosuppressive effects, BAR also has potential antiviral activity
through inhibition of proteins involved in viral propagation of SARS-CoV-2 [Richardson 2020,
Stebbing 2020]. NIH recommendations for BAR use among patients on O2 or NIV/HFO were
based on results from the ACTT-2 [Kalil 2020] and COV-BARRIER [Marconi 2021] trials, which
found significant improvements in recovery time (RR 1.16; 95% CI 1.01-1.32) and 28-day
mortality (HR 0.57; 95% CI 0.41–0.78), respectively, for those randomized to receive BAR for up
to 14 days compared to trial-specific comparators. Positive results from the STOP-COVID trial
[Guimarães 2021] led the NIH to suggest that TOF can be considered for patients receiving O2
and NIV/HFO when BAR is unavailable. A recent meta-analysis [Zhang 2022] of two BAR RCTs
found that similar to TCZ, there is a mortality benefit in patients receiving O2 (pooled RR = 0.62,
95% CI 0.41–0.95) and NIV/HFO (pooled RR 0.59, 95% CI 0.42–0.85) that is attenuated among
those requiring IMV/ECMO (pooled RR 0.77, 95% CI 0.51–1.15).
To our knowledge, only two studies (both single center observational RWD studies) comparing
IL6Ri (TCZ) to JAKi (BAR) for COVID-19 have been published in peer-reviewed journals.
Neither study found significant differences in outcomes for TCZ versus BAR treatment, however,
both studies were limited by small sample size (n=20 vs. 11 [Rosas 2022] and n=63 vs. 33
[Kojima 2022]). A non peer-reviewed pre-print study [Karampitsakos 2022] for an RCT found
BAR to be non-inferior to TCZ for the composite outcome of IMV or death and for time to
discharge among a severe COVID-19 population.  However, this study was not designed to
determine relative benefits across severe and critical disease (IMV/ECMO) subgroups.  The
non-inferiority design does not permit conclusions regarding one drugs’ relative superiority to
the other. In the absence of direct head-to-head evidence, some researchers have attempted to
compare results of separate IL6Ri and JAKi studies. Two separate indirect meta-analyses found
seemingly contradictory results: BAR was better than TCZ for preventing 28-day mortality, but
worse at preventing disease progression to IMV/ECMO [Shah 2022, Ngamprasertchai 2022].
Results of these indirect meta-analyses must be interpreted with caution due to the
heterogeneity in inclusion criteria and differing background standards of care (SoCs)4 within
separate IL6Ri and JAKi RCTs.
2. RESEARCH QUESTION AND OBJECTIVES
In this study we seek to evaluate current methodologies for observational comparative studies
of inpatient COVID-19 treatments [Overall Study Objective]. To support this overall study
objective, we have defined additional supporting objectives related to the research process
[Process Objectives] as applied to an illustrative example of an observational study to evaluate
the comparative effectiveness of inpatient COVID-19 treatments [Illustrative Example].
4 Many studies cited in NIH guidance for use of IL6Ri and JAKi in COVID patients did not include systemic CS
therapy as a consistent background SoC [Rosas 2021, Stone 2020, Lescure 2021, Kalil 2020], limiting generalizability
to the current COVID-19 treatment landscape. Inclusion of RDV as either an explicit combination therapy or optional
background SoC also varied across clinical trials.
9

Process Objectives:
Characterize differences in study results when an intentional multiphase approach (IMA) to
diagnostics and contingencies is applied to RWD analysis compared to an approach that
pre-specifies all covariates and statistical approaches without consideration of whether key
statistical assumptions hold (Single-phase Prespecification Approach, SPA). The SPA and two
separate IMA approaches will be further described in section 3.1 Study Design and section 3.7
Analysis.
Illustrative Example:
Our illustrative example will use a large population-based US claims data source to emulate a
hypothetical target trial to assess the comparative effectiveness of IL6Ri (TCZ or SAR) versus
JAKi (BAR or TOF) added to systemic corticosteroids of interest (CSI)5. The underlying
hypothetical target trial of interest consists of patients hospitalized and requiring respiratory
support for COVID-19 and receiving a CSI who are assigned to receive either an IL6Ri or JAKi
in addition to CSI within 4 days6 after hospital/ICU admission. The patients are followed for up to
28 days for the outcomes of inpatient mortality and progression to IMV/ECMO. The Illustrative
Example is separated into two sub-objectives based on the different COVID-19 severity
populations of interest.
●
Illustrative Example - Objective I aims to characterize the risk of inpatient mortality
[Primary Outcome] and progression to IMV or ECMO [Secondary Outcome] up to 28
days after IL6Ri or JAKi initiation among patients hospitalized with COVID-19 who
initiate a CSI and require supplemental O2/NIV/HFO (but not IMV/ECMO).
●
Illustrative Example - Objective II aims to characterize the risk of inpatient mortality
[Primary Outcome] up to 28 days after IL6Ri or JAKi initiation among patients admitted
to the ICU at hospital admission7 with COVID-19 who initiate a CSI and require
IMV/ECMO.
Hazard ratios (HR) and corresponding 95% confidence intervals (CI) will be estimated and
reported for all outcome risks in Illustrative Example objectives. Comparisons and populations
within Illustrative Example objectives are summarized in Table 1 below.
7 Patients in the Illustrative Example - Objective II cohort will be required to have an admission to the ICU on the
same day as hospital admission. The same-day requirement was selected after an initial query confirmed that >70%
of ICU admissions within this study population occur on the same day as hospital admission (see Exploration 4, in
section 3.1). See section 3.3.2 for further rationale.
6 Threshold of 4 days was determined based on the distribution of duration between hospitalization and initiation of
either IL6Ri or JAKi in the dataset (~90% of IL6Ri or JAKi initiators begin treatment within 4 days of hospital
admission). See exploration 2 in section 3.1 for more detail.
5 CSIs include the following systemic glucocorticoids, administered in oral or injectable formulations: dexamethasone,
methylprednisolone, prednisone, hydrocortisone, prednisolone, triamcinolone, cortisone, or betamethasone. Although
dexamethasone is the preferred CS for COVID, NIH guidelines indicate that other CS types in equivalent doses can
be used when dexamethasone is unavailable [NIH Treatment Guidelines].
10

Table 1. Summary of comparisons of interest within the Illustrative Example
Objective
Comparison
Population
Effectiveness
outcomes
Main measure of
effect
Illustrative Example -
Objective I
CSI + IL6Ri
(TCZ or SAR) within 4 days after
hospital admission
versus
CSI + JAKi
(BAR or TOF) within 4 days after
hospital admission
O2 or NIV/HFO
-
Primary Outcome:
Inpatient mortality
-
Secondary
Outcome:
Progression to
IMV or ECMO
HR
(95% CI)
Illustrative Example -
Objective II
CSI + IL6Ri
(TCZ or SAR) within 4 days after
hospital/ICU admission
versus
CSI + JAKi
(BAR or TOF) within 4 days after
hospital/ICU admission
IMV or ECMO
-
Primary Outcome:
Inpatient mortality
HR
(95% CI)
11

3. RESEARCH METHODS
3.1 Study Design
The Illustrative Example is an observational cohort study using the HealthVerity Chargemaster
and Linked Medical and Pharmacy Claims secondary healthcare data source.
The research process for this study will be separated into three distinct sequential phases: (1)
Exploratory (data explorations done in advance to facilitate key design decisions; linkage of
treatment and outcome data was not done in this phase), (2) Diagnostic (requirements that must
be satisfied prior to viewing treatment-specific outcomes for IMA analyses only, e.g., covariate
balance), and (3) Inferential (final comparative analyses to address study objectives).  See
Table 2 for more details.
The following explorations were completed prior to protocol finalization to inform overall study
design, inclusion and exclusion criteria, and identification of potential confounding and effect
modifying variables within the study dataset:
●
Exploration 1: Description of characteristics of initiators of IL6Ri, JAKi, CSI,
remdesivir (RDV), and other COVID-19 treatments.
●
Exploration 2: Description of trends in drug initiation by class and individual drugs,
including monotherapies and combination therapies, by calendar week, over the
study period.
●
Exploration 3: Description of inpatient treatment pathways (including time to
treatment initiation/switch) among hospitalized COVID-19 patients stratified by
calendar time, within severity subgroups based on oxygen requirements (at any time
during hospitalization), and by COVID-19 severity at admission.
●
Exploration 4: Description of the distribution of time to ICU admission among
patients hospitalized and progressing to ICU.
For the Diagnostic Phase (applicable to IMA only), we developed a checklist of diagnostic
criteria (see section 3.7.2, Table 6) that must be satisfactorily met prior to beginning the
implementation of the IMA Inferential Phase. The Diagnostic Phase is split into two distinct
phases to separate diagnostic checks that do not require post-index information (Diagnostic
Phase I) from checks that do require post-index data (Diagnostic Phase II), with guardrails to
maintain blinding of exposure-outcome relationships.  This distinction between pre-index and
post-index diagnostics will separate traditional baseline diagnostic checks that maintain
complete blinding of follow-up information from diagnostic checks that require analysis of
follow-up data and therefore may to some degree require linkage of exposure and outcome
information (e.g., assessment of the proportional hazards assumption). Accordingly, we will
conduct two separate sets of IMA analyses to descriptively compare the impact of applying
diagnostics and contingencies considering only baseline (pre-index, inclusive) data (Diagnostic
Phase I) to IMA analyses that consider both baseline and post-index data (Diagnostic Phases I
and II). The relationship between the treatment and outcome of interest will not be described or
evaluated in the analytic dataset until consensus is reached that the relevant diagnostic criteria
12

are satisfied. Diagnostic criteria will also be evaluated for SPA, however analyses will proceed
per a priori specifications regardless of whether or not each diagnostic check is satisfied.
Application of these research phases within separate SPA and IMA research Process
Objectives are described in Table 2.
Table 2. Overview of SPA and IMA research process objectives
Single-phase
Prespecification Approach
(SPA)
Intentional Multi-phase Approach (IMA)
IMA-1
IMA-2
Description of
research
process
approach
Pre-specifies covariates and
statistical approaches without
consideration of whether key
statistical assumptions hold.
Apply diagnostic checks
using baseline (pre-index)
data (Dx phase I only) and
pursue contingencies as
indicated, prior to Inferential
Phase.
Apply diagnostic checks using
both baseline (pre-index) data
(Dx phase I) and post-index
data (Dx phase II), and pursue
contingencies as indicated,
prior to Inferential Phase.
Exploratory
Phase
Completed in advance to inform
study/protocol development
Completed in advance to inform
study/protocol development
Completed in advance to inform
study/protocol development
Diagnostics
Phase I
Diagnostics assessed for context
only; no contingencies or
changes to a priori specified
analyses will be considered.
Diagnostics assessed and
contingencies implemented as
indicated. Proceed to Inferential
Phase after satisfying all criteria.
Diagnostics assessed and
contingencies implemented as
indicated. Proceed to Diagnostics
Phase II after satisfying all criteria.
Diagnostics
Phase II
Diagnostics assessed for
context only; no Diagnostics
Phase II contingencies will be
considered.
Diagnostics assessed and
contingencies implemented as
indicated. Proceed to Inferential
Phase after satisfying all criteria.
Inferential
Phase
Proceed to Inferential Phase
using a priori specified methods,
regardless of whether or not all
diagnostic criteria are satisfied.
Proceed to Inferential Phase
after satisfying all Diagnostics
Phase I criteria and pursuing
indicated contingencies.
Proceed to Inferential Phase after
satisfying all Diagnostics Phase I
and Diagnostics Phase II criteria
and pursuing indicated
contingencies.
3.2 Study Design Diagram for the Illustrative Example
The study diagram in Figure 1 depicts the assessment windows for entry criteria, covariates,
and outcomes relative to hospital/ICU admission date and treatment initiation date.
13

Figure 1: Study design diagram
3.3 Setting
3.3.1 Data sources
Details on the metadata of the database are shown in Table 3.
Table 3. Metadata on database
Data source
HealthVerity Chargemaster + Claims Data
Study period
(including look-back period)
June 16 2020 to February 28 2022*
Database Date Range
March 1 2019 to February 28 2022 [Available June 2022]*
Total Patients in Database
~1.2 million
Eligible cohort entry period
June 16 2020 to February 1, 2022
Data version (date received)
June 2022*
14

Data sampling / extraction
criteria
Patients with at least one day of medical benefit enrollment during the
study period
AND
Patients with a hospitalization recorded in the chargemaster data
Type of data
Closed medical and pharmacy claims (Private Source 17, Private Source
20) and inpatient chargemaster (private Source 88)
Data linkage
HealthVerity chargemaster and medical and pharmacy claims data (linked
at patient level prior to receipt of data)
Conversion to common data
model
N/A
HealthVerity data has several strengths for answering this Illustrative Example, including
near-real time delivery of data, information on both patient history (e.g., baseline medical and
pharmacy claims) and granular inpatient hospitalization data not typically available in claims
data sources (e.g., day-level inpatient drug utilization, mortality outcomes), and a large patient
population. HealthVerity data include patients from all US regions, with healthcare insurance
coverage from all major payer types (Medicare, Medicaid and commercial).
This data source also has limitations. As of yet, there are no validation studies of this data
source for our endpoints of mortality or progression to IMV/ECMO. However, use of discharge
status to determine mortality endpoints among hospitalized COVID-19 patients has been used
in both a similar chargemaster data source [Rosenthal 2020] and for national surveillance
reporting [CDC-NCHS 2021].
In a previous study [Gatto 2021], we compared use of discharge status of death from the
hospital chargemaster files to the separate raw data files from the vendor providing month of
death (date of death was redacted for privacy concerns) and found that all patients with a
recorded death month also met our endpoint definition. Further, when we compared weekly
deaths over time in our overall cohort of patients hospitalized with COVID-19, we identified
similar trends to those of two external national benchmarks, the Centers for Disease Control
[CDC-NCHS 2021] and data sourced from State and local health agencies [NYT 2021],
mitigating potential concern of mortality misclassification. validity
Despite lack of validation studies for the endpoint of progression to IMV or ECMO, we anticipate
this outcome to be well captured in the data source, given that mechanical ventilation is an
objective and serious outcome that is typically  recorded for billing purposes via standardized
procedure codes and/or chargemaster charge codes. IMV/ECMO endpoints have also been
used in other studies of COVID-19 within similar inpatient data sources [Giabicani 2022,
Rosenthal 2020, Nguyen 2021].
3.3.2 Population
The overall cohort for our Illustrative Example will consist of insured adults hospitalized with
COVID-19 between June 16, 2020 to February 01, 2022 who require supplemental oxygen or
ventilation support and initiate either IL6Ri (TCZ or SAR) or JAKi (BAR or TOF) in addition to
systemic CSIs within 4 days after hospital admission. All patients will be required to have at
least one procedure for oxygen supplementation and at least one recorded CSI administration
15

from admission to treatment initiation, since during the study period IL6Ri and JAKi were only
indicated for patients receiving CSIs and some level of oxygen or ventilation support.
From this population, the subcohort used to meet Illustrative Example - Objective I will consist
of adults hospitalized with COVID-19 who require supplemental O2/NIV/HFO (but not
IMV/ECMO) from admission to the day of IL6Ri or JAKi initiation. The subcohort used to meet
Illustrative Example - Objective II will include adults who are admitted to the ICU at hospital
admission and require IMV/ECMO prior to the day of IL6Ri or JAKi initiation. Given the lack of
day-level diagnosis codes (i.e., occurring after hospital admission) in the database, requiring the
ICU admission date be the same as the hospital admission date (Day 0) for Illustrative
Example - Objective II cohorts decreases the potential for misclassification of variables
assessed during periods anchored to the hospital admission date (e.g., baseline clinical
covariates). In other words, if ICU admission was not anchored to the hospital admission date,
any clinical events (defined by diagnostic codes) that occurred between hospital admission date
and the date of ICU admission would not be captured in the data, leading to differential capture
of such variables for patients admitted to the ICU later in their hospital stay.
Algorithms for identification of respiratory support requirements for O2/NIV/HFO (Illustrative
Example- Objective I) and IMV/ECMO (Illustrative Example - Objective II) subcohorts will be
based on a previously published WHO (mWHO) disease severity algorithm [Garry 2022], with
minor modifications to fit our study specifications [see Appendix F, available upon request]. This
algorithm classifies disease severity using procedure and free text charge codes for various
oxygen and ventilation-related procedures, as well as admitting diagnosis codes indicating
clinical need for different levels of respiratory support (e.g., hypoxia for O2, ARDS for IMV).
Operationalization of these measures is detailed in Appendix D3, Appendix D4, Appendix D5,
and section 3.3.3 below; inclusion and exclusion criteria are described in more detail in section
3.3.5 and section 3.3.6.
3.3.3 Cohort Entry, Treatment Index and Inclusion and Exclusion Criteria Assessment
Windows
See study design diagram in Figure 1 for a detailed overview of study time anchors and
assessment windows for the Illustrative Example. Appendix A further describes key study
components with respect to the hypothetical target trial that we seek to emulate.
Patients will enter the cohort upon the admission date of their first qualifying hospital admission
observed in the hospital chargemaster data (cohort entry date, CED; or Day 0). The first eligible
cohort entry date is June 16, 2020. This date was chosen to align with availability of
dexamethasone results from the RECOVERY trial, after which systemic CS became
increasingly used among adults hospitalized with severe or critical COVID-19. The last possible
cohort entry date will be 28 days prior to the last date of available data (i.e., February 1, 2022 at
the latest), to ensure complete follow-up data for all patients.
Patients are eligible for IL6Ri or JAKi treatment initiation (Day T, treatment index) any time from
admission date (Day 0) to 4 days after admission (Day 4). CSI use, oxygen support
16

requirements (O2/NIV/HFO for Illustrative Example - Objective I, IMV/ECMO for Illustrative
Example - Objective II) will be assessed from hospital admission (Day 0) until treatment index
(Day T). Timing of treatment initiation will be assessed relative to hospital/ICU admission (Day
0). Inclusion criteria, exclusion criteria, and covariates of interest will be assessed during the
time periods specified in Figure 1. Assessment windows are defined relative to time anchors for
hospital/ICU admission (Day 0) and IL6Ri/JAKi treatment initiation (Day T). Operationalization
details are shown in Appendix D1.
3.3.4 Follow-up
Follow-up for outcomes will begin 1 day after treatment index (Day T+1) to distinguish between
events occurring before or at treatment initiation from outcomes beginning after treatment
initiation. This is necessary given that in this dataset we cannot identify the specific time of
treatment and outcome events, only the specific day, therefore outcome events recorded on Day
T may have begun before or after treatment initiation. For primary analyses, follow-up for
outcomes will begin on Day T+1 and continue until the earliest occurrence of death, outcome (if
different from death), discharge from the hospital, or 28 days of follow-up reached (Day T+29).
This utilizes an initial treatment design, whereby all patients initiating either IL6Ri or JAKi within
4 days after admission (Days 0 to 4) are assumed to complete their treatment course per
guidelines, regardless of early discontinuation (for JAKi users only) or potential crossover
between groups. Changes in inpatient treatments after initial treatment (Days T+1 to T+29) are
assumed to be part of routine care and therefore post-index treatment changes are not
considered in SPA.
This initial treatment approach may be modified to an as-treated follow-up design in IMA-2
contingency analyses depending on diagnostic checks assessing the extent of crossover and
discontinuation of JAKi during follow-up [see Table 7]. See operationalization details in
Appendix D2.
Potential bias resulting from censoring on competing risks will also be addressed in IMA-2 via
contingent analyses if indicated [See Table 7]. A competing risk is an event that precludes the
chance of occurrence of the primary event of interest and can occur when a person is at risk of
more than one type of event [Dutz 2019]. IMA-2 contingency analyses will include assessment
of potential bias from the competing risk of hospital discharge (for primary outcome in both
Illustrative Example - Objectives I and II) and for the competing risk of death (for secondary
outcome of IMV/ECMO in Illustrative Example - Objective I).
3.3.5 Inclusion Criteria
Patients who meet the following criteria will be included in the overall study cohort for the
Illustrative Example:
●
Hospitalized from June 16, 2020 to February 01, 2022 with an ICD-10 diagnosis code of
U07.1 in any admitting diagnosis position (Day 0)
●
Initiate either IL6Ri or JAKi within 4 days after hospital admission (Days 0 to 4)
●
Receiving systemic CSI on day of IL6Ri/JAKi initiation (Day T)
17

●
Receipt of at least one respiratory support procedure (oxygen supplementation at a
minimum) from admission to IL6Ri/JAKi initiation (Days 0 to T)
○
For Illustrative Example - Objective I only: maximum mWHO disease severity
of O2/NIV/HFO from admission to IL6Ri/JAKi initiation (Days 0 to T)
○
For Illustrative Example - Objective II only: Admission to ICU at hospital
admission (Day 0) and mWHO disease severity of IMV/ECMO from hospital/ICU
admission to IL6Ri/JAKi initiation (Days 0 to T)
●
Continuous medical claims enrollment (60-day gaps permitted) during the 183 day
baseline period prior to and including hospital or ICU admission (Days -183 to 0) to
minimize the potential for misclassification of baseline covariates
Hospitalization due to COVID-19 will be defined as an inpatient chargemaster record with an
ICD-10 diagnosis code of U07.1 in any admitting diagnosis position, as per CDC Coding
Guidance for COVID-19 effective on April 1, 2020 [CDC Coding Guidelines 2020]. Although this
definition does not explicitly require COVID-19 as the primary admitting cause, requirements for
CSI use, oxygen supplementation, and IL6Ri or JAKi initiation within the first 4 days after
hospital admission should exclude most patients with incidental SARS-CoV-2 infections. See
operationalization details in Appendix D3.
3.3.6 Exclusion Criteria
Patients not meeting the above inclusion criteria or who meet any of the following criteria will be
excluded from the study cohort:
●
Exclude patients without continuous medical claims enrollment (60-day gaps permitted)
during the 183-day baseline period prior to and including hospital/ICU admission (Days
-183 to 0) to minimize the potential for misclassification of baseline covariates.
●
Exclude patients if COVID-19 hospitalization (Day 0) begins >14 days after initial
COVID-19 diagnosis. Patients will be excluded if any COVID-19 diagnosis is recorded
from 90 days to 15 days before admission (Days -90 to -15) to exclude patients with
possible long-term COVID or post-acute sequelae while still permitting prior infections
recorded more than 90 days pre-admission.
●
No age, sex, or geographic region recorded on hospital admission (Day 0)
●
Age less than 18 years at hospital admission (Day 0)
●
Evidence of a prior COVID-related inpatient hospitalization in the previous 14 days (Days
-14 to -3), with a two-day buffer to permit brief inpatient utilization directly proceeding
transfer to a chargemaster hospital (i.e., inpatient utilization permitted on Days -2 to -1)
●
Systemic CSI use (dispensing or remaining supply) in the previous 14 days (Days -14 to
-3), with a two-day buffer to permit CSI use beginning in the emergency department or
other healthcare setting immediately prior to chargemaster hospital admission (CSI use
permitted Days -2 to -1)
●
Any recorded use of IL6Ri or JAKi (dispensing or remaining supply) in the 90-day
washout period before IL6Ri or JAKi initiation (Days -90 to T-1), to satisfy new use
definition
●
Baseline receipt of oxygen or ventilation support for non-COVID conditions (e.g.,
supplemental oxygen for COPD), defined as two or more respiratory support procedures
18

recorded on different days from Days -90 to -15 via procedure codes from medical
claims, chargemaster, and where available, oxygen supplies recorded in pharmacy
settings.
●
Death or discharge occurs before or on the day of treatment initiation (Days 0 to T)
●
Recorded use of both IL6Ri and JAKi on Day T
The 14-day window for COVID-related exclusion criteria was chosen based on published
research describing time from illness onset to hospital admission in COVID-19 cohorts from
Europe [Faes 2020] and the United States [Bhatraju 2020]. Given unavailability of laboratory
results and potential for diagnosis of COVID in healthcare settings where insurance claims are
not submitted, this window assumes that illness onset may occur as early as 14 days before
hospital admission for all patients, regardless of the presence or absence of pre-admission
COVID-19 confirmation in the dataset. Similarly, systemic CSI use, new oxygen
supplementation requirements, and prior inpatient hospitalizations beginning during this 14-day
pre-admission period are assumed to be plausibly COVID-related events. We exclude patients
with these events beginning within 14 days up until 3 days prior to chargemaster admission
(Days -14 to -3) to remove patients with longer duration of severe disease (to reduce
heterogeneity of the study population), with a two-day buffer to permit utilization occurring in the
emergency department or other healthcare setting immediately prior to chargemaster
admission.
Other than known hypersensitivity to these medications, there are no absolute contraindications
for IL6Ri/JAKi use in COVID-19 [FDA 2020, FDA 2021, Lilly USA 2022]. However, there are
populations for which IL6Ri and/or JAKi use is cautioned, including patients with concurrent
active non-COVID infections, liver disease, demyelinating disorders, patients who are severely
immunosuppressed (including those who have been recently treated with other biologic
immunomodulators), patients with history of tobacco use or other cardiovascular risk factors,
patients with known malignancies, and patients with renal impairment, including end stage renal
disease or acute kidney injury or those on dialysis. SPA analyses will a priori adjust for these
conditions without excluding patients with potential contraindications. To account for uncertainty
with regard to adherence to such precautions in real-world clinical practice, IMA analyses will
permit inclusion of patients with these conditions if balance for these factors can be achieved for
IL6Ri and JAKi groups during the study’s diagnostic stage.
See operationalization details of exclusion criteria in Appendix D4.
3.4 Variables
STaRT-RWE templates [Wang, 2021] were used to document the operational definitions of the
variables in this study (see Appendix D). Key study variables are described in sections 3.4.1-
3.4.4 below.
19

3.4.1 Cohort Definition
Cohort definition details, including the operationalization of cohort entry and inclusion and
exclusion defining variables can be found in Appendix D1, Appendix D3, and Appendix D4.
3.4.2 Exposure Definition
The primary exposures of interest are new initiation of either IL6Ri or JAKi medications within 4
days after hospital admission (Days 0 to 4). IL6Ri and JAKi exposures will be identified in
inpatient chargemaster records with standardized procedure codes (from current procedural
terminology/healthcare common procedure coding system, CPT/HCPCS [CMS 2021]; and
ICD-10 procedure code system, ICD-10 PCS) and hospital charge codes containing text strings
indicating administration of IL6Ri or JAKi medications (e.g., ‘tocilizumab’, ‘Actemra’, ‘baricitinib’,
‘Olumiant’). Only procedure and hospital charge codes for routes of administration
recommended by NIH for COVID-19 will be included in exposure definitions (e.g., intravenous
for TCZ, oral for BAR).
Inpatient IL6Ri and JAKi use is considered “new” if patients had no record of IL6Ri nor JAKi
dispensing during the 90-day treatment washout period (Days -90 to T-1). Treatment washouts
for prior IL6Ri or JAKi use will include all possible routes of administration captured in all
available data types with treatment-specific information, including inpatient chargemaster (Days
-90 to T-1), as well as outpatient hospital chargemaster, medical claims, and pharmacy claims
(Days -90 to T-1). See operationalization details in Appendix D1.
All IL6Ri and JAKi initiators will be required to have systemic CSI use on the day of treatment
initiation (Day T), per NIH recommendations that IL6Ri/JAKi should only be given in combination
with course of dexamethasone or other CSI [NIH treatment guidelines]. CSI use can begin prior
to or concurrent with IL6Ri/JAKi initiation (Day T). Both IL6Ri and JAKi exposed patients may be
treated with other therapeutic agents. Complete exposure and comparator definition details can
be found in Appendix D1; code lists can be found in Appendix F.
As detailed in diagnostic check #1 of Table 7, IMA analyses may instead conduct comparisons
of individual drugs (e.g., TCZ versus BAR) rather than evaluation of class-level effects (as done
in SPA) if in both drug classes an individual IL6Ri or JAKi drug constitutes greater than 90% of
use within its respective class.
3.4.3 Outcome Definition
The outcomes of interest will be inpatient mortality (primary outcome) and progression to
IMV/ECMO (secondary outcome for Illustrative Example - Objective I) during the follow-up
period beginning 1 day after IL6Ri or JAKi treatment initiation and continuing for up to 28 days.
Death and date of death will be sourced from the chargemaster discharge status field.
IMV/ECMO will be defined using standardized procedure codes and hospital charge codes.
Outcome definition details can be found in Appendix D5; code lists can be found in Appendix F.
3.4.4 Covariates
Propensity score-based weighting methods will be used to control for confounding in the
analysis. Table 4 provides a list of the covariates considered for inclusion in the propensity
20

score (PS) model and their corresponding assessment windows (see further operationalization
details in Appendix D6). These variables were selected a priori due to a determination that they
are prognostically important (i.e., related to outcomes) or are potential confounders of the
relationship between treatment and outcomes. Covariates were informed by prior published
studies of inpatient COVID-19 [Nigo 2021, Gupta 2021, Crothers 2021], including other studies
using the HealthVerity chargemaster data source [Stewart 2021, Gordon 2020], clinical expert
input, and CDC’s list of conditions placing individuals at high risk of severe COVID-19 [CDC
2022]. SPA analyses will include all a priori specified covariates in Table 4 in final PS models,
regardless of whether or not assumptions of positivity and other diagnostic criteria are met. For
analyses following the IMA approach, PS model assumptions for Illustrative Example -
Objective I and Illustrative Example - Objective II populations will be checked via a diagnostic
checklist (see section 3.7.2) prior to model finalization. IMA analyses will consider all covariates
in Table 4, but some covariates may not be included in final models as described in section
3.7.1. Individual covariates and assessment windows may be combined if warranted based on
variable distributions, and final PS model inputs may vary across different subcohorts within this
study. Missingness among covariates of interest within our study cohort will be enumerated and
reported in study findings for both SPA and IMA approaches.
Table 4: Description of patient characteristics and covariate assessment windows
Baseline
Covariates
(Days -183 to
-1)
Baseline
(pre-admission)
comorbidities,
co-medications,
and health
resource utilization.
- Lifestyle factors: history of smoking/tobacco use*
- Comorbidities and comedications: asthma, any cancer (active
hematological cancer, active solid tumor), chronic lung disease,
cardiovascular disease (any), diabetes, immunosuppressive
conditions or medications (including blood/organ transplant,
rheumatoid arthritis and other autoimmune conditions), kidney
disease (end stage renal disease, acute kidney injury, dialysis),
overweight or obese (via diagnosis codes), liver dysfunction
(hepatic impairment, active liver disease), other conditions with
potential IL6Ri or JAKi cautions (bowel obstruction, necrosis,
ulcerative disease, Iatrogenic GI injury, demyelinating disorders,
serious infections, cystic fibrosis, pregnancy, sickle cell /
thalassemia), frailty score, neurological/cognitive impairment,
mental health/psychosis, hypertension, statin use, systemic and
non-systemic corticosteroid use, anticoagulant and/or
antiplatelet use as well as the combined comorbidity score
[Gagne 2011]
- Health resource utilization: number of days hospitalized,
number of outpatient visits, number of pharmacy claims, number
of distinct medications dispensed, any encounter indicating
skilled nursing facility (SNF) or long-term care (LTC), 7+ days in
SNF or LTC, COVID-19 vaccination status (ever vs. never),
record of prior COVID-19 (from start of data to Day -15)
Pre- admission
COVID-related
covariates
(Days -14 to
-2)
Pre-admission
COVID-19 severity
- COVID-19 severity before admission: time since first
COVID-19 diagnosis to admission, pre-admission (Days -14 to
-1) use of potential COVID-19 medications (systemic or inhaled
corticosteroids, monoclonal antibodies, antivirals, RDV,
COVID-19 convalescent plasma), COVID-19-related utilization
prior to admission (urgent care, emergency department),
respiratory support required
21

(Days -1 to 0)
Admitting
healthcare
resource utilization
Admitting healthcare resource utilization characteristics:
admission source (admitted from emergency department,
immediate transfer from other inpatient facility, etc), ambulance,
admitted directly to ICU at hospital admission, admitting type,
mWHO disease severity at inpatient admission
Demographics,
clinical and
admission
covariates
(Day 0)
Demographics,
hospital facility
characteristics, and
COVID-19 severity
on the admission
date.
- Demographics: age, sex, insurance type, US region, month and
year of hospital admission
- Hospital characteristics: hospital setting type (urban vs rural),
hospital teaching status, hospital number of beds, facility
COVID-19 admitting volume, facility level of immunomodulator
prescribing
- Admitting diagnoses: dyspnea or hypoxia, pneumonia, ARDS
or acute respiratory failure, shock, non-respiratory organ failure,
delirium, sepsis, acute cardiovascular events, thromboembolic
events, do not resuscitate order, COVID vs non-COVID
diagnoses in primary position, presence of possible
contraindication (concurrent non-COVID infection, end stage
renal disease or acute kidney injury, liver dysfunction)
Inpatient
Covariates
(Days 0 to T)
COVID-19 status
and treatments
- Concomitant medications: Anticoagulant and/or antiplatelet
agents, antibiotic (any and individual classes), vasopressors or
inotropes, neuromuscular blockades, number of unique
medication classes administered**, inhaled CS, RDV, non-RDV
antivirals (including lopinavir/ritonavir, other HIV protease
inhibitors, ivermectin, etc), other immunomodulators (non-JAK
kinase inhibitors, non-IL6 interleukin inhibitors, anti-TNF
biologics, B or T-cell inhibitors, etc)
- Pre-treatment COVID-19 healthcare resource utilization
characteristics: Intensive care utilization (regular ICU,
intermediate ICU, critical care procedures), major surgical
procedures, specialty care services (radiology, cardiovascular,
respiratory therapy, surgical services), renal replacement therapy
or dialysis, IMV/ECMO, number of O2/ventilation procedures
administered, COVID-19-related lab orders (e.g., PCT, ferritin,
D-dimers, troponin, PaO2, CRPs, AST/ALTs, blood viscosity, and
others), number of unique procedure codes**, number of unique
department codes, maximum intensity respiratory support
procedure recorded, most frequently recorded mWHO status,
number of days from admission to treatment initiation, number of
days from CSI initiation to treatment initiation, number of days
from max respiratory support status to treatment initiation,
number of days with O2/NIV/HFO/IMV/ECMO support prior to
treatment initiation, mWHO disease severity at treatment
initiation
*Smoking/tobacco use may be under-reported [see section 4. Potential Limitations for more detail]
**IL6Ri and JAKi will be excluded from variable definitions to avoid adjustment for exposure-related
characteristics
The baseline covariate period will be extended to 12 months (Days -365 to -1) as part of a
sensitivity analysis (see Table 8).
3.5 Study Size
22

Minimal sample size estimates were calculated for context only, given that the analyses are an
illustrative example used to compare different methodological approaches. Analyses will
proceed regardless of sample size considerations for this reason.
The sample size that would be required to estimate differences between outcome rates in
CSI+IL6Ri versus CSI+JAKi populations is dependent on the expected outcomes rates in our
study population. Rates of 28-day mortality and progression to IMV/ECMO in hospitalized adults
with COVID-19 vary across published RCT and RWD studies due to differences in underlying
study populations, study periods, background and concomitant therapies, disease severity, and
other factors.
Among RCTs of IL6Ri,  during the study period, mortality rates for IL6Ri-treated patients ranged
from <10% in studies of moderately-ill cohorts (e.g., EMPACTA [Salama 2020], BACC Bay
[Stone 2020, Lescure 2021]) to over 25% in studies including severe-critical patients
(RECOVERY [RECOVERY Collaborative Group 2021] and REMAP-CAP [REMAP-CAP
Investigators 2021]). 28-day mortality rates from JAKi RCTs also varied, with mortality under
10% in the ACTT-2 [Kalil 2020] and COV-BARRIER trials [Marconi 2021] and over 35% in the
critically-ill addendum population to COV-BARRIER [Ely 2022]. For studies of both IL6Ri and
JAKi medications, mortality rates increased with higher levels of disease severity, ranging from
<2% mortality for JAKi-treated patients with moderate disease (O2 only) in the ACTT-2 trial [Kalil
2020] to 49% mortality for critically-ill (IMV) IL6Ri-treated patients in the RECOVERY trial
[RECOVERY Collaborative Group 2021].
Rates of progression to IMV or ECMO among those not on IMV/ECMO at baseline also varied
across published studies. Only 7% of IL6Ri-treated patients in the BACC Bay trial [Stone 2020],
progressed to IMV, compared to 35% among critically-ill patients in REMAP-CAP [REMAP-CAP
Investigators 2021]. For BAR, the only RCT evaluating progression to IMV or ECMO reported an
outcome rate of 10% for BAR-treated patients [Kalil 2020].
Given uncertainties around outcome rates in our specific study population, we calculated the
sample size that would be required to detect differences in outcome rates for IL6Ri or
JAKi-treated populations across a range of hypothetical scenarios. To determine the sample
size required to estimate differences between outcome rates in CSI+IL6Ri versus CSI+JAKi
populations, we used the EpiR package (R version 4.1.2) with the following assumptions:
●
80% power
●
5% test size (alpha)
●
2-sided confidence interval
Table 5 summarizes the sample size requirements assuming a 1:1 ratio of IL6Ri to JAKi-treated
patients and outcome rate estimates ranging from 0.1 to 0.6. A 1:1 ratio is provided for ease of
display, however, differing ratios may decrease total sample size requirements. Finer granularity
of risk estimates is arbitrarily shown for the JAKi+CSI group only in Table 5 to demonstrate
sample size scenarios where outcome risks are fairly similar (+/-10%) between groups.
23

Table 5: Sample size requirements for number of JAKi+CSI-treated patients assuming 1:1
matching of IL6Ri+CSI to JAKi+CSI patients
Outcome Risk in IL6Ri + CSI group
0.1
0.2
0.3
0.4
0.5
0.6
Outcome
Risk in JAKi
+ CSI group
0.1
NA
123
38
19
11
7
0.15
419
581
76
30
16
10
0.225
84
2718
342
67
27
14
0.25
61
705
790
92
33
16
0.275
47
323
3225
134
41
19
0.325
31
122
3317
377
68
26
0.35
26
86
837
849
92
31
0.375
22
65
374
3389
131
37
0.425
17
40
135
3351
354
59
0.45
14
32
94
830
784
79
0.475
13
27
68
364
3081
111
0.5
11
23
52
202
NA
169
0.6
7
12
22
46
169
NA
3.6 Data Management
Measure and cohort creation and analysis will be done on the Aetion Evidence Platform (AEP).
At Aetion, raw data review is conducted to understand the contents of the database and
scientific integrity checks are performed to ensure the contents of the data are consistent with
the expected data as laid out in the applicable data usage agreement.
Following receipt and review of the raw data, each data cut is connected to the AEP. A data
connector specification is drafted by a data scientist, which provides a map for transformation of
raw data to the Aetion longitudinal patient timeline. Validation of the database connection, via
double programming, is completed to ensure that the implementation of database connection
logic leads to transformed data output that connects to and behaves within AEP exactly as
intended. Following validation, the specification files are used to create an Aetion data dictionary
for the dataset. Prior to connection of the data to the AEP, a manual test to ensure certain
platform features and dataset values are visible and testable on the front-end is implemented.
See Appendix E for additional details on the process of raw data review and connection to the
AEP.
24

Software used for data analyses is detailed in Table 6 below.
Table 6. Software used for cohort creation and data analyses
Software
Version
Analyses
Validation Studies
Aetion Evidence
Platform®
2022
Measure and cohort creation, time
to outcome analyses. All SPA
analyses and IMA-1 analyses will be
conducted with this software.
[Wang, 2016]
R
4.1.2
Some analyses (e.g., IMA-2
analysis components) may be
conducted with this software. Use
(including version and libraries
used) will be documented as
required
Various; contingent
on IMA analyses
required
3.7 Analysis
We describe below two analytic approaches: SPA and IMA. SPA will be implemented based on
a priori specified covariates and statistical approaches without consideration of whether key
statistical assumptions hold. Conversely, IMA will conduct diagnostic checks and implement
contingencies as indicated to evaluate whether such an approach can be used to strengthen
current methodologies for observational comparative studies of COVID-19 treatments.
3.7.1 Application of Single-phase Prespecification Approach (SPA)
Time to death (for primary outcome in Illustrative Example - Objectives I and II) and
progression to IMV or ECMO (for secondary outcome in Illustrative Example - Objective I)
among patients treated with IL6Ri+CSIs will be compared with time to death or progression to
IMV or ECMO among those treated with JAKi+CSIs. Censoring will occur at the earliest of
death, receipt of IMV or ECMO (for secondary outcome of progression to IMV or ECMO only),
hospital discharge or end of the 28-day follow-up period. See Appendix D2 and Figure 2 for
additional details on censoring and follow-up designs.
To compare differences in time to outcome events, we will generate Kaplan-Meier plots and
estimate hazard ratios and 95% confidence intervals using a Cox proportional hazards analysis.
Inverse probability weighting (IPW) methods will be used to control confounding.
A propensity score model will be estimated with IL6Ri receipt as the outcome of the logistic
regression model. Variables to be included in the SPA IPW model will comprise all variables in
Table 4. Patient demographic and clinical characteristics at baseline will be compared for
IL6Ri-treated and JAKi-treated patients pre- and post-IPW. Balance will be assessed by
calculating absolute standardized differences (ASD); covariates with ASD greater than or equal
to 0.1 will be considered imbalanced. The a priori specified approach of SPA will proceed with
inferential analyses regardless of covariate imbalance.
25

The weights for each patient will be calculated as the inverse of the probability of receiving the
treatment the patient actually received conditional on observed covariates. Each individual’s
contribution to the survival curves and to the Cox regression model will be weighted by the
inverse probability of receiving the treatment the patient actually received conditional on
observed covariates. Use of robust standard errors will account for the weighted design and
potential clustering, including clustering by facility.
SPA analyses will be applied separately for comparisons outlined in Illustrative Example -
Objective I and Illustrative Example - Objective II.
3.7.2 Application of Intentional Multi-phase Approach (IMA)
IMA analyses will conduct a series of diagnostic checks to assure analytic assumptions are met
(Table 7) prior to implementation of the Inferential Phase. Each diagnostic check includes a
description of the requirements that must be satisfied prior to beginning the implementation of
the Inferential Phase, as well as objective thresholds used to determine if a contingency should
be triggered.
Diagnostic Phase I includes diagnostic checks conducted using baseline treatment index data,
without use of post-index information; Diagnostic Phase II includes certain checks of post-index
data, including outcome information, with guardrails to maintain blinding of exposure-outcome
relationships. Upon completion of the Diagnostic Phase, the analytic dataset is locked and
propensity score models are finalized. All treatment effects will be examined during the
Inferential Phase, which cannot begin for IMA until each diagnostic step has been satisfactorily
checked to determine what contingencies are appropriate.
As described in Table 2, we will conduct two separate IMA analyses to characterize the impact
of applying diagnostics and contingencies considering only baseline treatment index data
(Diagnostic Phase I only for IMA-1) to analyses that consider both baseline and post-index data
(Diagnostic Phases I and II for IMA-2).
Both IMA-1 and IMA-2 will follow SPA with exceptions and possible contingencies as described
here. Follow-up for outcomes and censoring criteria will similarly follow SPA, with possible
inclusion of  additional censoring criteria in the IMA-2 analysis as described in Appendix D2 and
Figure 2.
26

Figure 2. Censoring reasons for Illustrative Objectives, by Process Objective
Balance in patient demographics and clinical characteristics will be assessed by calculating
absolute standardized differences (ASD); covariates with ASD greater than or equal to 0.1 will
be considered imbalanced and addressed according to diagnostic contingencies described in
Table 7.
IPW will be used to adjust for confounding as described in section 3.7.1. Covariates listed in
Table 4 will be considered as potential independent variables, and included pending diagnostics
and contingencies described in Table 7. Weight stabilization or trimming / truncation will also be
considered pending diagnostic checks described in Table 7. Variables not balanced after
weighting will either be included in the outcome model or considered for restriction (see Table
7).
For IMA-2, in which both Diagnostic Phases I and II are completed prior to the Inferential Phase,
we will conduct additional diagnostic checks for competing risks, treatment crossover and
discontinuation, and inspection of survival curves to assess that the proportional hazards
assumption holds over the follow-up period. Contingent analyses for IMA-2 are described in the
Diagnostic Phase II section of Table 7.
27

Table 7: Diagnostic checklist (with contingency plans*) to complete prior to progression to the inferential phase of IMA
Diagnostic Phase I
#
Diagnostic
Description
Threshold
Contingency triggered*
1
Evaluate
distribution
of individual
drugs within
each class
to determine
whether
class-level
comparisons
are
warranted
Evaluate proportion of patients in
each class-level treatment arm
(IL6Ri, JAKi) who initiate individual
drugs within each class (TCZ and
SAR for IL6Ri; BAR and TOF for
JAKi).
90% of patients within both classes
initiate individual drugs
Proceed with comparison of individual drugs
(e.g., TCZ versus BAR) rather than
evaluation of class-level effects.
In this scenario, patients initiating individual
drugs constituting <10% of class-level use
will be excluded from study cohorts. All
protocol elements will otherwise remain
unchanged.
If only within one class 90% of patients
initiate a single drug, class level analyses
will be pursued.
2
Assess
baseline
confounder
balance
before IPW
weighting
We will evaluate balance in the
distributions of all potential
confounders between IL6Ri+CSI
versus JAKi+CSI groups in the
unweighted sample. Covariate
balance will be defined as absolute
standardized differences (ASDs) <
0.10 [Austin 2009].
Covariate imbalance, considered ASD
≥0.1
Variables with balance prior to IPW may be
excluded from PS model if needed to satisfy
checks #4-7 below.
28

3
Assess
baseline
confounder
balance
following
IPW
We will confirm that the
distributions of all potential
confounders are balanced between
IL6Ri+CSI versus JAKi+CSI groups
in the weighted sample. Covariate
balance will be defined as absolute
standardized differences (ASDs) <
0.10 [Austin 2009]. Although
variables with balance prior to IPW
may be removed from the PS
model per diagnostic check #2,
balance of these variables will still
be confirmed after IPW.
Covariate imbalance, considered ASD
≥0.1
For variables with residual imbalance after
weighting one of two courses of action may
be taken:
1)
Variables not balanced may be
included in the subsequent outcome
model.
2)
Study cohorts may be restricted,
stratified, or direct-matched on
certain variables as deemed
appropriate if balance cannot be
achieved via IPW.
4
Assess the
positivity
assumption
Positivity violations will be
assessed using tabular approaches
[Westreich 2010] and / or by
evaluating the distribution of
weights [Peterson 2012] using
graphical aids (e.g., histograms of
weights) and data driven tools.
Exposure probabilities larger than 6
standard deviations away from the
average probability value will be
evaluated as extreme. However, a more
stringent criteria (i.e., probabilities that
are within less than 6 standard
deviations away from the average
probability value) may be considered,
depending on the variability of the
computed probabilities.
In the event that the positivity assumption is
violated, sample restriction or combination
or recategorization of covariates and
assessment windows will be considered if
warranted.
5
Assess
model
coefficients
for extreme
values
Evaluate PS model coefficients to
identify extreme coefficients which
may indicate collinearity or positivity
violations.
Any coefficient with extreme values that
strongly predicts presence or absence
of the exposure of interest (i.e.,
potential instrumental variables) or
another covariate (i.e., multicollinearity).
In the case of extreme coefficients, variables
may be dropped, combined, or
recategorized within PS models and
coefficients re-evaluated for extreme values.
6
Confirm
models are
not overfit
We will confirm that all models
contain a sufficient number of
exposed patients per covariate
level (non-continuous variables
only).
At least 12 exposed patients or more
per covariate level
Variables with balance prior to IPW may be
removed from the PS model, and if
necessary, covariates without at least 12
exposed patients per covariate level may be
removed from the model. Individual
covariates and assessment windows may
also be combined to address overfitting if
warranted based on variable distributions.
7
Explore
The distribution of estimated
Weights larger than 6 standard
Weight stabilization or trimming / truncation
29

weight
distributions
to evaluate
whether
outlier
observations
influence
data
weights will be evaluated [Peterson
2012] via graphical approaches
(e.g., histograms or boxplots);
these approaches will be used to
evaluate the presence of extreme
weights and to define cut-off values
for stabilization or
truncation/trimming, if necessary.
deviations away from the average
weight value will be evaluated as
extreme. However, a more stringent
criteria (i.e., weights that are within less
than 6 standard deviations away from
the average weights value) may be
considered, depending on the variability
of the computed weights.
will be considered.
Diagnostic Phase II
#
Diagnostic
Description
Threshold
Contingency triggered*
8
Evaluate
probability of
censoring
due to a
competing
risk**
Calculate proportions of individuals
in each treatment group
experiencing a competing risk over
time
20% (or more) of patients in either IL6Ri
or JAKi arms experience a competing
risk during follow-up.
Analyses accounting for discharge as a
competing risk will be considered.
Thresholds and possible contingencies will
be considered separately for each outcome
(death for both Illustrative Example
objectives; progression to IMV/ECMO for
illustrative Example - Objective I).
9
Evaluate
treatment
crossover
after
treatment
index (Day
T)
Evaluate proportion of patients in
each treatment arm who initiate the
other medication after day T
(IL6Ri-treated patients on Day T
initiating JAKi after Day T, and vice
versa).
20% (or more) of the patients in either
IL6Ri or JAKi arms initiate the other
medication during follow-up from days
T+1 to T+29.
Implications with regard to modifying the
research question or for result interpretation
will be evaluated.
If threshold is met, IMA-2 will implement an
as-treated follow-up design with censoring
on treatment crossover between groups
(initiation of JAKi medications among those
initially treated with IL6Ri, and vice-versa). If
crossover is differential by treatment group,
this analysis may incorporate models to
adjust for informative censoring.
10
Evaluate
early
treatment
discontinuati
on among
JAKi-treated
Evaluate proportion of patients in
JAKi+CSI arm who discontinue
JAKi treatment prior to discharge
and before completion of
recommended course of 14 days.
20% (or more) of the patients in the
JAKi+CSI arm discontinue their JAKi
treatment before discharge and before
14 days of treatment.
Implications with regard to modifying the
research question or for result interpretation
will be evaluated.
If threshold is met, IMA-2 will implement an
as-treated follow-up design with censoring
30

patients
for early discontinuation of JAKi (before 14
days of treatment or discharge). Given that
a single infusion of IL6Ri (at initiation) is
considered a complete treatment dose,
whereas a full course of JAKi requires
once-daily oral treatment administered for
up to 14 days or until discharge, only JAKi
treated patients can be subject to censoring
for discontinuation. Thus, this censoring
design may also consider informative
censoring adjustments to address potential
selection bias.
11
Inspect
survival
curves to
assess the
proportional
hazards
assumption
over the
follow-up
period
Prior to calculation of overall hazard
ratios, we will visually inspect
survival curves to assess that the
proportional hazards assumption
holds over the follow-up period.
Proportional hazards assumption is not
satisfied, as demonstrated in visual aids
(e.g. Schoenfeld residual plots or
log-log plots).
If the proportional hazards assumption is not
satisfied then shorter follow-up times, during
which this assumption is likely to hold, will
be analyzed. Alternatively other methods,
such as restricted mean survival times, will
be considered as an alternative to
calculating hazard ratios via Cox models.
Restricted mean survival times are a
measure of average survival from time 0 to
a specified time point, and may be
estimated as the area under the survival
curve up to that point. This analysis would
compare the computed difference between
the areas under the survival curve for each
treatment arm. This method does not rely on
an assumption of proportional hazards of
the treatment effect.
*Contingent analyses are changes to the primary (SPA) analyses unless otherwise specified. IMA-1 and IMA-2 will consider contingencies in
Diagnostic Phase I; IMA-2 will additionally consider contingencies in Diagnostic Phase II.
**Models incorporating competing risks require measurement of time-varying covariates; these covariates after day T will include measures of
disease severity (operationalized via procedure codes, charge codes, and department codes) and medications outlined in Table 4.
31

3.7.3 Sensitivity Analyses
A priori-defined sensitivity analyses are described in Table 8 below.
Table 8: Sensitivity Analyses*
#
Sensitivity
Analysis
Parameter
being
varied?
Expected learning
Weaknesses and strengths of the analysis
compared to the primary
Analysis
type
1
Extend baseline
period from 183
days to 365 days
Baseline
period
This analysis will help
elucidate the extent to which
extension of the baseline
period influences study
results, including treatment
effects but also prevalence
of comorbidities and
medications recorded during
baseline.
Limitations: Requiring 365 days of continuous
enrollment may decrease sample size.
Strengths: Health-care seeking behavior during the
pandemic (particularly early in the pandemic) is not
representative of routine behavior. This analysis may
improve capture of pre-COVID conditions and
potential confounders by providing a wider span of
time for baseline assessment to account for
potentially sparse medical visits during the pandemic.
Design
variation
2
Restrict cohorts to
patients initiating
IL6Ri or JAKi
medications
within 1 day after
hospital/ICU
admission (Days
0 to 1)
Inclusion
criteria
This exploratory analysis
will help elucidate the
extent to which truncation
of the IL6Ri/JAKi exposure
assessment window
influences relative
effectiveness of IL6Ri
versus JAKi medications.
Limitations: These analyses may be underpowered
and may have limited generalizability relative to
primary analyses.
Strengths: ​By restricting to patients treated at or
shortly after hospital admission, this analysis may
improve specificity of disease severity classifications
given that diagnoses recorded at admission (Day 0)
are likely more reflective of disease severity at
treatment initiation (Day T) than in primary analyses.
Design
variation
*Table template adapted from STaRT-RWE.
4. POTENTIAL LIMITATIONS
4.1 Addressed in Design and Analyses
The following limitations have been addressed via study design or analyses, but will still be
taken into consideration when interpreting and disseminating the findings from the
implementation of this protocol:
●
Unstructured inpatient medication and procedure data: In addition to standardized
procedure codes (HCPCS/CPT or ICD-10-PCS), inpatient medication and oxygen
supplementation information can be recorded in non-standardized free-text charge code
descriptions. Therefore, text search strings will be used to query the charge code data
when procedure codes are not available. Text strings used to query the data are not
based on validated algorithms and may result in misclassification. Comprehensive text
string searches will be used to minimize the potential for misclassification and additional
manual review will be incorporated for key measures.
●
Limited capture of indication and heterogeneity in COVID-19 severity: Although
best practices will be applied to control for confounding, the potential for residual
confounding still exists in observational cohort analyses. The indication for use of both
32

IL6Ri or JAKi in patients on O2 and NIV is “rapidly increasing oxygen needs and
systemic inflammation”, which are not possible to measure using HealthVerity data.
Further, since JAKi are not recommended for patients receiving IMV/ECMO, those
patients receiving JAKi for these indications may be different (e.g., have more severe
illness) than patients receiving IL6Ri for these indications. We are unable to fully
measure the indication for O2 and NIV patients receiving these drug classes, nor assess
whether one class of drugs is preferentially prescribed for more severe patients within
any population defined by respiratory support requirements, potentially resulting in
residual confounding by indication. Residual confounding is also a possibility if certain
unmeasured characteristics associated with disease severity are also associated with
preference for a given route of administration (e.g., if intravenous infusions are preferred
to oral tablets for heavily-sedated patients).
While we cannot fully eliminate the potential for residual confounding, we will adjust for
many measurable characteristics relating to disease severity, and assess covariate
balance after application of IPW. Further, given that both drug classes are indicated for
the same O2/NIV/HFO populations, we anticipate that any residual confounding will be
minimal. Although not empirically verifiable within our dataset, if warranted, we will
consider sensitivity analyses that evaluate the potential impact of unmeasured
confounding [Schneeweiss 2006].
In this study (as in previous studies), we will compare the risk of endpoints in those
treated or not treated with either drug class within 4 days after admission under routine
care. Treatments on days 5-28 after admission are thus part of routine care and should
not be controlled. Likewise, changes in severity may occur during follow-up and should
not be controlled. Changes in treatment, care, and severity during follow-up may be
causal mediators and thus contribute to the effect of IL6Ri or JAKi on the endpoints of
interest. Any association of these post-index covariates with the treatments of interest
could arise from the treatments (i.e. would be mediators), or from other baseline
covariates that predicted treatment (i.e. would be mediators of a baseline covariate and
controlled via control of the baseline covariate). Thus, the only post-index control
considered in this study will be for differences in censoring.
●
Baseline (pre-admission) clinical characteristics (comorbidities, COVID-19 health
care utilization) are limited to information available in claims: Claims data are
primarily collected for reimbursement purposes rather than research or clinical
record-keeping, which can lead to misclassification and under-reporting of certain
conditions or variables. In particular, behaviors and events such as smoking/tobacco
use, COVID-19 diagnostic test results and outpatient or home oxygen use are likely to
be under-recorded. Our data does not include nursing notes, which have been shown in
a previous study conducted by the FDA Sentinel System to increase capture of O2 use
[Cocoros 2021]. Where possible, claims algorithms have been defined broadly to
account for potential under-reporting of such variables. For example, smoking/tobacco
use is defined with diagnoses indicating use or abuse of tobacco, procedure codes for
cessation consultations, and pharmacy claim records for cessation-related medications;
similarly, although we do not have access to pre-admission laboratory results we define
33

pre-admission COVID-19 with diagnosis codes for confirmed COVID-19. We assume
misclassification in baseline clinical characteristics to be non-differential between
treatment groups.
●
Possibility of uncontrolled confounding by facility-level differences: Facility-level
factors (e.g., drug availability, facility- or provider-level treatment practices) may be
associated with both use of IL6Ri/JAKi and patient outcomes, and cannot be directly
accounted for within this analysis. However, our analyses will adjust for hospital
characteristics such as number of beds, teaching status, level of immunomodulator
prescribing, and demonstrated preference for either IL6Ri or JAKi as a proxy to help
account for facility-level differences.
●
Possibility of exposure mis-classification and uncontrolled confounding due to
differences in mode of administration and treatment courses for IL6Ri and JAKi:
We will attempt to restrict exposures evaluated in this study to IL6Ri or JAKi events
delivered via their recommended routes of administration for COVID-19 (e.g.,
intravenous for TCZ, oral for BAR). However, there is a potential for misclassification due
to the availability of other modes of administration for certain products (e.g.,
subcutaneous injection for TCZ and SAR), potential for under-capture of route of
administration in this database, and uncertainty with regard to how subcutaneous
formulations of IL6Ri reconstituted for intravenous use are recorded in chargemaster
data. Finally, IL6Ri and JAKi treatments differ in terms of their recommended treatment
courses. A single infusion of IL6Ri (at initiation) is considered a complete treatment
dose, whereas a full course of JAKi requires daily oral treatment administered for up to
14 days or until discharge. We account for these differences by considering a separate
as-treated contingency analysis for and possible adjustment for informative censoring in
diagnostic phase II of the IMA-2 analysis.
4.2 To Be Considered in Interpretation Only
The following limitations will be taken into consideration when interpreting and disseminating the
findings from the implementation of this protocol:
●
Lack of indicators of socio-economic status (SES): The HealthVerity data leveraged
for this study does not include SES measures. We will include insurance type in the PS
model for covariate adjustment as it may be associated with SES (e.g., patients with a
PPO generally have higher SES than those with HMO), but no other proxies are
available to adjust for SES.
●
Lack of race/ethnicity information: The HealthVerity data leveraged for this study does
not include race or ethnicity demographic data. Therefore, we are unable to evaluate
whether the effect of either IL6Ri or JAKi on 28-day mortality varies by race or ethnicity,
explore potential racial and ethnic disparities related to COVID-19 severity and
treatment, or control for confounding by race or ethnicity.
●
Lack of post-discharge outcome information: Follow-up for death and IMV/ECMO
outcomes must be censored at discharge from hospital, given lack of reliable
34

post-discharge outcome information. Therefore, there may be some patients who
experience outcomes after discharge from their initial hospitalization, either during
subsequent re-admissions or after transfer to a different healthcare facility. We will not
capture these outcomes nor consider post-discharge follow-up time in our analyses.
●
Lack of generalizability to certain populations: Study design decisions that improve
internal validity of Illustrative Example Objectives may in some cases limit
generalizability to populations of interest. For example, patients with baseline O2 use (to
the extent it was captured in claims data) were excluded from study cohorts in order to
ensure that any O2 use during their hospitalization was for COVID-19 and not a
pre-existing comorbidity. However, this may result in exclusion of patients most at-risk for
severe COVID-19 (e.g. patients with COPD) from study populations. Similarly,
requirements for IL6Ri and JAKi initiation to occur no later than 4 days after admission,
the exclusion for pre-admission CS use, and the exclusion for prior COVID-19
hospitalizations improve internal validity while potentially reducing generalizability to
populations of interest. While HealthVerity data include patients from all major insurance
types (Medicaid, Medicare and commercial insurance), results may not be generalizable
to non-insured patients.
5. STRENGTHS
The generation of methodologically robust evidence from observational studies can be
challenging; this is particularly true when evaluating potential treatments for COVID-19 when
there is still more to learn about its natural history, and when the standard of care is still
evolving. By implementing a head-to-head comparison of treatments with the same indication,
along with a new user design, our study minimizes the potential of bias including confounding by
indication and immortal time bias, and ensures the correct temporality between covariate and
exposure assessment [Lund 2015; Franklin 2017]. In order to evaluate optimal methodologies
for observational comparative studies of COVID-19 treatments, we evaluate and compare two
different approaches: 1) an approach based on pre-specified covariates and statistical
approaches without consideration of whether key statistical assumptions hold, and 2) an
approach whereby two-phases (pre- and post-index) of diagnostic checks are performed and
corresponding contingencies implemented to ensure that analytic assumptions are fully satisfied
prior to finalizing the inferential analysis plan and estimating and interpreting treatment effects.
Diagnostic checks evaluate potential sources of bias (e.g. competing risk bias) omitted in many
other studies evaluating drug effectiveness in hospitalized patients with COVID-19 [Martinuka
2021].
For the Illustrative Example we followed structured processes [Gatto 2019] to articulate the
research question, conceptualize the underlying hypothetical pragmatic target trial [Hernan,
Robins 2016; Hernan, Sauer 2016], design the real-world emulation study, and to identify a
fit-for-purpose data source [Gatto 2022, FDA 2018]. Based on this systematic assessment we
selected a RWD source with near-real-time information, a reasonable population size, and
sufficient capture of key study parameters and longitudinal patient experience to evaluate our
35

research question. Next, we explored the data to confirm data feasibility and inform key study
design decisions. Sensitivity analyses have been developed a priori with documented
justifications to evaluate the robustness of findings.
6. HUMAN SUBJECTS
The use of this de-identified data source was approved for exemption by the New England
Independent Review Board.
7. REFERENCES
Albuquerque, A. M., Tramujas, L., Sewanan, L. R., Williams, D. R., & Brophy, J. M. (2022).
Mortality Rates Among Hospitalized Patients With COVID-19 Infection Treated With Tocilizumab
and Corticosteroids: A Bayesian Reanalysis of a Previous Meta-analysis. JAMA Network Open,
5(2), e220548. https://doi.org/10.1001/jamanetworkopen.2022.0548
Bhatraju, P. K., Ghassemieh, B. J., Nichols, M., Kim, R., Jerome, K. R., Nalla, A. K., Greninger,
A. L., Pipavath, S., Wurfel, M. M., Evans, L., Kritek, P. A., West, T. E., Luks, A., Gerbino, A.,
Dale, C. R., Goldman, J. D., O’Mahony, S., & Mikacenic, C. (2020). Covid-19 in Critically Ill
Patients in the Seattle Region—Case Series. New England Journal of Medicine, 382(21),
2012–2022. https://doi.org/10.1056/NEJMoa2004500
Blanco-Melo, D., Nilsson-Payant, B. E., Liu, W.-C., Uhl, S., Hoagland, D., Møller, R., Jordan, T.
X., Oishi, K., Panis, M., Sachs, D., Wang, T. T., Schwartz, R. E., Lim, J. K., Albrecht, R. A., &
tenOever, B. R. (2020). Imbalanced Host Response to SARS-CoV-2 Drives Development of
COVID-19. Cell, 181(5), 1036-1045.e9. https://doi.org/10.1016/j.cell.2020.04.026
Bronte, V., Ugel, S., Tinazzi, E., Vella, A., Sanctis, F. D., Canè, S., Batani, V., Trovato, R., Fiore,
A., Petrova, V., Hofer, F., Barouni, R. M., Musiu, C., Caligola, S., Pinton, L., Torroni, L., Polati,
E., Donadello, K., Friso, S., … Olivieri, O. (2020). Baricitinib restrains the immune dysregulation
in patients with severe COVID-19. The Journal of Clinical Investigation, 130(12), 6409–6416.
https://doi.org/10.1172/JCI141772
Burrage, D. R., Koushesh, S., & Sofat, N. (2020). Immunomodulatory Drugs in the Management
of SARS-CoV-2. Frontiers in Immunology, 11.
https://www.frontiersin.org/article/10.3389/fimmu.2020.01844
Centers for Disease Control. (2022). COVID19 People with Certain Medical Conditions
Accessed May 25, 2022.
https://www.cdc.gov/coronavirus/2019-ncov/need-extra-precautions/people-with-medical-conditi
ons.html
36

Centers for Disease Control. (2020). ICD-10-CM Official Coding and Reporting Guidelines April
1, 2020 through September 30, 2020. Accessed May 25, 2022
https://www.cdc.gov/nchs/data/icd/COVID-19-guidelines-final.pdf
Centers for Disease Control National Center for Health Statistics (CDC-NCHS). (2021).
In-hospital mortality among hospital confirmed COVID-19 encounters by week from selected
hospitals. Accessed May 25, 2022.
https://www.cdc.gov/nchs/covid19/nhcs/hospital-mortality-by-week.htm
Centers for Medicare and Medicaid Services (CMS) (2021). HCPCS Coding Questions. Page
Last Modified: February 11, 2020. Accessed March 23, 2021.
https://www.cms.gov/Medicare/Coding/MedHCPCSGenInfo/HCPCS_Coding_Questions
Chen, G., Wu, D., Guo, W., Cao, Y., Huang, D., Wang, H., Wang, T., Zhang, X., Chen, H., Yu,
H., Zhang, X., Zhang, M., Wu, S., Song, J., Chen, T., Han, M., Li, S., Luo, X., Zhao, J., & Ning,
Q. (2020). Clinical and immunological features of severe and moderate coronavirus disease
2019. The Journal of Clinical Investigation, 130(5), 2620–2629.
https://doi.org/10.1172/JCI137244
Cocoros, N.M., Fuller, C.C., Adimadhyam, S., Ball, R., Brown, J.S., Dal Pan, G.J., Kluberg, S.A.,
Lo Re V 3rd, Maro, J.C., Nguyen, M., Orr, R., Paraoan, D., Perlin, J., Poland, R.E., Driscoll,
M.R, Sands, K., Toh, S., Yih, W.K., Platt, R.; And the FDA-Sentinel COVID-19 Working Group. A
COVID-19-ready public health surveillance system: The Food and Drug Administration's
Sentinel System. Pharmacoepidemiol Drug Saf. 2021 Jul;30(7):827-837. doi: 10.1002/pds.5240.
Epub 2021 Apr 18. PMID: 33797815; PMCID: PMC8250843.
Crothers, K., DeFaccio, R., Tate, J., Alba, P. R., Goetz, M. B., Jones, B., King, J. T., Marconi, V.,
Ohl, M. E., Rentsch, C. T., Rodriguez-Barradas, M. C., Shahrir, S., Justice, A. C., Akgün, K. M.,
& Group, V. A. C. S. C. C.-19 W. (2021). Dexamethasone in hospitalised coronavirus-19
patients not on intensive respiratory support. European Respiratory Journal.
https://doi.org/10.1183/13993003.02532-2021
Del Valle, D. M., Kim-Schulze, S., Huang, H.-H., Beckmann, N. D., Nirenberg, S., Wang, B.,
Lavin, Y., Swartz, T. H., Madduri, D., Stock, A., Marron, T. U., Xie, H., Patel, M., Tuballes, K.,
Van Oekelen, O., Rahman, A., Kovatch, P., Aberg, J. A., Schadt, E., … Gnjatic, S. (2020). An
inflammatory cytokine signature predicts COVID-19 severity and survival. Nature Medicine,
26(10), 1636–1643. https://doi.org/10.1038/s41591-020-1051-9
Dutz, A., & Löck, S. (2019). Competing risks in survival data analysis. Radiotherapy and
Oncology, 130, 185–189. https://doi.org/10.1016/j.radonc.2018.09.007
Ely, E. W., Ramanan, A. V., Kartman, C. E., de Bono, S., Liao, R., Piruzeli, M. L. B., Goldman, J.
D., Saraiva, J. F. K., Chakladar, S., & Marconi, V. C. (2022). Efficacy and safety of baricitinib
plus standard of care for the treatment of critically ill hospitalised adults with COVID-19 on
37

invasive mechanical ventilation or extracorporeal membrane oxygenation: An exploratory,
randomised, placebo-controlled trial. The Lancet. Respiratory Medicine, 10(4), 327–336.
https://doi.org/10.1016/S2213-2600(22)00006-6
Faes, C., Abrams, S., Van Beckhoven, D., Meyfroidt, G., Vlieghe, E., & Hens, N. (2020). Time
between Symptom Onset, Hospitalisation and Recovery or Death: Statistical Analysis of Belgian
COVID-19 Patients. International Journal of Environmental Research and Public Health, 17(20),
7560. https://doi.org/10.3390/ijerph17207560
FDA. (2020). Fact Sheet for Healthcare Providers Emergency Use Authorization (EUA) of
Baricitinib. Accessed April 18, 2022.
https://www.fda.gov/media/143823/download#:~:text=The%20U.S.%20Food%20and%20Drug,n
on%2Dinvasive%20or%20invasive%20mechanical
FDA. (2021). Fact Sheet for Healthcare Providers Emergency Use Authorization (EUA) for
Actemra (tocilizumab). Accessed April 18, 2022. https://www.fda.gov/media/150321/download
FDA. (2018). Framework for FDA’s Real-World Evidence Program. Page last reviewed: April 18,
2022. Accessed April 18, 2022. https://www.fda.gov/media/120060/download
Feuillet, V., Canard, B., & Trautmann, A. (2021). Combining Antivirals and Immunomodulators to
Fight COVID-19. Trends in Immunology, 42(1), 31–44. https://doi.org/10.1016/j.it.2020.11.003
Franklin, J. M., Patorno, E., Desai, R. J., Glynn, R. J., Martin, D., Quinto, K., Pawar, A.,
Bessette, L. G., Lee, H., Garry, E. M., Gautam, N., & Schneeweiss, S. (2017). Emulating
Randomized Clinical Trials With Nonrandomized Real-World Evidence Studies. Circulation,
143(10), 1002–1013. https://doi.org/10.1161/CIRCULATIONAHA.120.051718
Garibaldi, B. T., Wang, K., Robinson, M. L., Zeger, S. L., Bandeen-Roche, K., Wang, M.-C.,
Alexander, G. C., Gupta, A., Bollinger, R., & Xu, Y. (2021). Comparison of Time to Clinical
Improvement With vs Without Remdesivir Treatment in Hospitalized Patients With COVID-19.
JAMA Network Open, 4(3), e213071. https://doi.org/10.1001/jamanetworkopen.2021.3071
Gatto, N. M., Reynolds, R. F., & Campbell, U. B. (2019). A Structured Preapproval and
Postapproval Comparative Study Design Framework to Generate Valid and Transparent
Real-World Evidence for Regulatory Decisions. Clinical Pharmacology & Therapeutics, 106(1),
103–115. https://doi.org/10.1002/cpt.1480
Gatto, N. M., Campbell, U. B., Rubinstein, E., Jaksa, A., Mattox, P., Mo, J., & Reynolds, R. F.
(2022). The Structured Process to Identify Fit-For-Purpose Data: A Data Feasibility Assessment
Framework. Clinical Pharmacology & Therapeutics, 111(1), 122–134.
https://doi.org/10.1002/cpt.2466
38

Gatto N, Garry EM, Chakravarty A. Effect of dexamethasone on inpatient mortality among
hospitalized COVID-19 patients - Full Text View - ClinicalTrials.gov (2021). Accessed February
11, 2022. https://clinicaltrials.gov/ct2/show/NCT04926571
Ghosn, L., Chaimani, A., Evrenoglou, T., Davidson, M., Graña, C., Schmucker, C., Bollig, C.,
Henschke, N., Sguassero, Y., Nejstgaard, C. H., Menon, S., Nguyen, T. V., Ferrand, G., Kapp,
P., Riveros, C., Ávila, C., Devane, D., Meerpohl, J. J., Rada, G., … Boutron, I. (2021).
Interleukin‐6 blocking agents for treating COVID19: A living systematic review. Cochrane
Database of Systematic Reviews, 3. https://doi.org/10.1002/14651858.CD013881
Giabicani, M., Menestrel, A. L., Roux, O., Rautou, P.-E., & Weiss, E. (2022). Focus on the
decisions to forego life-sustaining therapies during ICU stay of patients with cirrhosis and
COVID-19: A case control study from the prospective COVID-ICU database. Journal of
Hepatology, 76(3), 742–744. https://doi.org/10.1016/j.jhep.2021.09.018
Gordon D., Hiatt J., Bouhaddou, M., Rezelj, V., Ulferts, S., Braberg, H.,
Jureka, A., Obernier, K., Guo, J., Batra, J., Kaake, R., Weckstein, A., Owens, T., Gupta, M.,
Pourmal, S., Titus, E., Cakir, M., … Krogan, N. (2020). Comparative host-coronavirus protein
interaction networks reveal pan-viral disease mechanisms. Science, 370(6521), eabe9403.
https://doi.org/10.1126/science.abe9403
Guimarães, P. O., Quirk, D., Furtado, R. H., Maia, L. N., Saraiva, J. F., Antunes, M. O., Kalil
Filho, R., Junior, V. M., Soeiro, A. M., Tognon, A. P., Veiga, V. C., Martins, P. A., Moia, D. D. F.,
Sampaio, B. S., Assis, S. R. L., Soares, R. V. P., Piano, L. P. A., Castilho, K., Momesso, R. G. R.
A. P., … Berwanger, O. (2021). Tofacitinib in Patients Hospitalized with Covid-19 Pneumonia.
New England Journal of Medicine, 385(5), 406–415. https://doi.org/10.1056/NEJMoa2101643
Gagne, J. J., Glynn, R. J., Avorn, J., Levin, R., & Schneeweiss, S. (2011). A combined
comorbidity score predicted mortality in elderly patients better than existing scores. Journal of
Clinical Epidemiology, 64(7), 749–759. https://doi.org/10.1016/j.jclinepi.2010.10.004
Garry, E. M., Weckstein, A. R., Quinto, K., Bradley, M. C., Lasky, T., Chakravarty, A., Leonard,
S., Vititoe, S. E., Easthausen, I. J., Rassen, J. A., & Gatto, N. M. (2022). Categorization of
COVID-19 severity to determine mortality risk. Pharmacoepidemiology and Drug Safety, 31(7),
721–728. https://doi.org/10.1002/pds.5436
Gupta S, Wang W, Hayek SS., … Leaf, D. (2021). Association Between Early Treatment With
Tocilizumab and Mortality Among Critically Ill Patients With COVID-19. JAMA Intern Med.
2021;181(1):41–51. doi:10.1001/jamainternmed.2020.6252
Hernán MA, Robins JM. (2016). Using Big Data to Emulate a Target Trial When a Randomized
Trial Is Not Available. Am J Epidemiol. 2016 Apr 15;183(8):758-64. doi: 10.1093/aje/kwv254.
Epub 2016 Mar 18. PMID: 26994063; PMCID: PMC4832051.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4832051/
39

Hernán MA, Sauer BC, Hernández-Díaz S, Platt R, Shrier I (2016). Specifying a target trial
prevents immortal time bias and other self-inflicted injuries in observational analyses. J Clin
Epidemiol. 2016 Nov;79:70-75. doi: 10.1016/j.jclinepi.2016.04.014. Epub 2016 May 27. PMID:
27237061; PMCID:PMC5124536.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5124536/
Hu, Y., Wang, T., Hu, Z., Wang, X., Zhang, Z., Li, L., & Peng, P. (2020). Clinical efficacy of
glucocorticoid on the treatment of patients with COVID-19 pneumonia: A single-center
experience. Biomedicine & pharmacotherapy, 130, 110529.
https://doi.org/10.1016/j.biopha.2020.110529
Juul, S., Nielsen, E. E., Feinberg, J., Siddiqui, F., Jørgensen, C. K., Barot, E., Holgersson, J.,
Nielsen, N., Bentzer, P., Veroniki, A. A., Thabane, L., Bu, F., Klingenberg, S., Gluud, C., &
Jakobsen, J. C. (2021). Interventions for treatment of COVID-19: Second edition of a living
systematic review with meta-analyses and trial sequential analyses (The LIVING Project). PLoS
ONE, 16(3), e0248132. https://doi.org/10.1371/journal.pone.0248132
Kalil, A. C., Patterson, T. F., Mehta, A. K., Tomashek, K. M., Wolfe, C. R., Ghazaryan, V.,
Marconi, V. C., Ruiz-Palacios, G. M., Hsieh, L., Kline, S., Tapson, V., Iovine, N. M., Jain, M. K.,
Sweeney, D. A., El Sahly, H. M., Branche, A. R., Regalado Pineda, J., Lye, D. C., Sandkovsky,
U., … Beigel, J. H. (2020). Baricitinib plus Remdesivir for Hospitalized Adults with Covid-19. The
New England Journal of Medicine, NEJMoa2031994. https://doi.org/10.1056/NEJMoa2031994
Karampitsakos, T., Papaioannou, O., Tsiri, P., Katsaras, M., Katsimpris, A., Kalogeropoulos, AP.,
Malakounidou, E., Zarkadi, E., Tsirikos, G., Georgiopoulou, V., Sotiropoulou, V., Koulousousa,
E., Chourpiliadi, C., Matsioulas, A., Lagadinou, M., Sampsonas, F., Akinosoglou, K., Marangos,
M., & Tzouvelekis, A. (2022). Tocilizumab versus baricitinib in hospitalized patients with severe
COVID-19: an open label, randomized controlled trial. medRxiv (pre-print).
https://doi.org/10.1101/2022.06.13.22276211
Kim, M. S., An, M. H., Kim, W. J., & Hwang, T.-H. (2020). Comparative efficacy and safety of
pharmacological interventions for the treatment of COVID-19: A systematic review and network
meta-analysis. PLoS Medicine, 17(12), e1003501.
https://doi.org/10.1371/journal.pmed.1003501
Kojima, Y., Nakakubo, S., Takei, N., Kamada, K., Yamashita, Y., Nakamura, J., Matsumoto, M.,
Horii, H., Sato, K., Shima, H., Suzuki, M., & Konno, S. (2022). Comparative Efficacy of
Tocilizumab and Baricitinib Administration in COVID-19 Treatment: A Retrospective Cohort
Study. Medicina, 58(4), 513. https://doi.org/10.3390/medicina58040513
Kow, C. S., & Hasan, S. S. (2021). The effect of tocilizumab on mortality in hospitalized patients
with COVID-19: A meta-analysis of randomized controlled trials. European Journal of Clinical
Pharmacology, 1–6. https://doi.org/10.1007/s00228-021-03087-z
40

Lescure, F-X., Honda, H., Fowler, R. A., Lazar, J. S., Shi, G., Wung, P., Patel, N., & Hagino, O.
(2021). Sarilumab in patients admitted to hospital with severe or critical COVID-19: A
randomised, double-blind, placebo-controlled, phase 3 trial. The Lancet. Respiratory Medicine,
9(5), 522–532. https://doi.org/10.1016/S2213-2600(21)00099-0
Lilly USA. (2022). Highlights of Prescribing Information OLUMIANT (baricitinib). Page last
reviewed: April 18, 2022. Accessed April 18, 2022. https://www.fda.gov/media/120060/download
Li, Q., Li, W., Jin, Y., Xu, W., Huang, C., Li, L., Huang, Y., Fu, Q., & Chen, L. (2020). Efficacy
Evaluation of Early, Low-Dose, Short-Term Corticosteroids in Adults Hospitalized with
Non-Severe COVID-19 Pneumonia: A Retrospective Cohort Study. Infectious diseases and
therapy, 9(4), 823–836. https://doi.org/10.1007/s40121-020-00332-3
Lund, J. L., Richardson, D. B., & Stürmer, T. (2015). The Active Comparator, New User Study
Design in Pharmacoepidemiology: Historical Foundations and Contemporary Application.
Current Epidemiology Reports, 2(4), 221–228. https://doi.org/10.1007/s40471-015-0053-5
Marconi, V. C., Ramanan, A. V., de Bono, S., Kartman, C. E., Krishnan, V., Liao, R., Piruzeli, M.
L. B., Goldman, J. D., Alatorre-Alexander, J., de Cassia Pellegrini, R., Estrada, V., Som, M.,
Cardoso, A., Chakladar, S., Crowe, B., Reis, P., Zhang, X., Adams, D. H., & Ely, E. W. (2021).
Efficacy and safety of baricitinib for the treatment of hospitalised adults with COVID-19
(COV-BARRIER): A randomised, double-blind, parallel-group, placebo-controlled phase 3 trial.
The Lancet. Respiratory Medicine, 9(12), 1407–1418.
https://doi.org/10.1016/S2213-2600(21)00331-3
Martinuka, O., Cube, M. von, & Wolkewitz, M. (2021). Methodological evaluation of bias in
observational coronavirus disease 2019 studies on drug effectiveness. Clinical Microbiology and
Infection, 27(7), 949–957. https://doi.org/10.1016/j.cmi.2021.03.003
Mehta, P., McAuley, D. F., Brown, M., Sanchez, E., Tattersall, R. S., & Manson, J. J. (2020).
COVID-19: Consider cytokine storm syndromes and immunosuppression. The Lancet,
395(10229), 1033–1034. https://doi.org/10.1016/S0140-6736(20)30628-0
Nelson, B. C., Laracy, J., Shoucri, S., Dietz, D., Zucker, J., Patel, N., Sobieszczyk, M. E., Kubin,
C. J., & Gomez-Simmonds, A. (2021). Clinical Outcomes Associated With Methylprednisolone in
Mechanically Ventilated Patients With COVID-19. Clinical infectious diseases : an official
publication of the Infectious Diseases Society of America, 72(9), e367–e372.
https://doi.org/10.1093/cid/ciaa1163
New York Times (NYT). Coronavirus in the U.S.: Latest Map and Case Count. Death data
sourced from State and local health agencies. Updated May 24, 2021. Accessed May 20, 2021.
https://www.nytimes.com/interactive/2021/us/covid-cases.html
41

Nigo M, Rasmy L, May SB, Rao A, Karimaghaei S, Kannadath BS, De la Hoz A, Arias CA, Li L,
Zhi D. (2021). Real World Long-term Assessment of The Efficacy of Tocilizumab in Patients with
COVID-19: Results From A Large De-identified Multicenter Electronic Health Record Dataset in
the United States. International Journal of Infectious Diseases. 2021;113:148-54.
Ngamprasertchai, T., Kajeekul, R., Sivakorn, C., Ruenroegnboon, N., Luvira, V., Siripoon, T., &
Luangasanatip, N. (2022). Efficacy and Safety of Immunomodulators in Patients with COVID-19:
A Systematic Review and Network Meta-Analysis of Randomized Controlled Trials. Infectious
Diseases and Therapy, 11(1), 231–248. https://doi.org/10.1007/s40121-021-00545-0
Nguyen, J. L., Benigno, M., Malhotra, D., Reimbaeva, M., Sam, Z., Chambers, R., Hammond,
J., & Emir, B. (2021). Hospitalization and mortality trends among patients with confirmed
COVID-19 in the United States, April through August 2020. Journal of Public Health Research,
11(1), 2244. https://doi.org/10.4081/jphr.2021.2244
NIH Treatment Guidelines (a): NIH Treatment Guidelines for Hospitalized Adults: Therapeutic
Management. COVID-19 Treatment Guidelines. Last Updated February 24, 2022; Retrieved
April 25, 2022, from
https://files.covid19treatmentguidelines.nih.gov/guidelines/archive/covid19treatmentguidelines-0
2-24-2022.pdf
NIH Treatment Guidelines (b): NIH Treatment Guidelines for Hospitalized Adults: Therapeutic
Management. COVID-19 Treatment Guidelines. Last Updated August 8, 2022; Retrieved
September 6, 2022, from
https://www.covid19treatmentguidelines.nih.gov/management/clinical-management/hospitalized-
adults--therapeutic-management/
Petersen, M. L., Porter, K. E., Gruber, S., Wang, Y., van der Laan, M. J. (2012). Diagnosing and
responding to violations in the positivity assumption. Statistical methods in medical research,
21(1):31-54. https://doi.org/10.1177/0962280210386207
Petrone, L., Petruccioli, E., Alonzi, T., Vanini, V., Cuzzi, G., Fard, S. N., Castilletti, C., Palmieri,
F., Gualano, G., Vittozzi, P., Nicastri, E., Lepore, L., Grifoni, A., Antinori, A., Vergori, A., Ippolito,
G., Cantini, F., & Goletti, D. (2021). In-vitro evaluation of the immunomodulatory effects of
Baricitinib: Implication for COVID-19 therapy. Journal of Infection, 82(4), 58–66.
https://doi.org/10.1016/j.jinf.2021.02.023
RECOVERY Collaborative Group. (2020). Dexamethasone in Hospitalized Patients with
Covid-19—Preliminary Report. The New England Journal of Medicine, NEJMoa2021436.
https://doi.org/10.1056/NEJMoa2021436
RECOVERY Collaborative Group. (2021). Tocilizumab in patients admitted to hospital with
COVID-19 (RECOVERY): A randomised, controlled, open-label, platform trial. The Lancet,
397(10285), 1637–1645. https://doi.org/10.1016/S0140-6736(21)00676-0
42

REMAP-CAP Investigators. (2021). Interleukin-6 Receptor Antagonists in Critically Ill Patients
with Covid-19. New England Journal of Medicine, 384(16), 1491–1502.
https://doi.org/10.1056/NEJMoa2100433
Rezaei, S., Fatemi, B., Karimi Majd, Z., Minaei, H., Peikanpour, M., Anjidani, N., Taheri, A.,
Dastan, F., & Mosaed, R. (n.d.). Efficacy and safety of Tocilizumab in severe and critical
COVID-19: A Systematic Review and Meta-Analysis. Expert Review of Clinical Immunology,
1–13. https://doi.org/10.1080/1744666X.2021.1908128
Richardson, P., Griffin, I., Tucker, C., Smith, D., Oechsle, O., Phelan, A., Rawling, M., Savory,
E., & Stebbing, J. (2020). Baricitinib as potential treatment for 2019-nCoV acute respiratory
disease. Lancet (London, England), 395(10223), e30–e31.
https://doi.org/10.1016/S0140-6736(20)30304-4
Rosenthal, N., Zhun, C., Gundrum, J., Sianis, J., Safo, S. (2020). Risk Factors Associated With
In-Hospital Mortality in a US National Sample of Patients With COVID-19 | Cardiology | JAMA
Network Open | JAMA Network. https://doi.org10.1001/jamanetworkopen.2020.29058
Rosas, I. O., Bräu, N., Waters, M., Go, R. C., Hunter, B. D., Bhagani, S., Skiest, D., Aziz, M. S.,
Cooper, N., Douglas, I. S., Savic, S., Youngstein, T., Del Sorbo, L., Cubillo Gracian, A., De La
Zerda, D. J., Ustianowski, A., Bao, M., Dimonaco, S., Graham, E., … Malhotra, A. (2021).
Tocilizumab in Hospitalized Patients with Severe Covid-19 Pneumonia. New England Journal of
Medicine, 384(16), 1503–1516. https://doi.org/10.1056/NEJMoa2028700
Rosas, I. O., Diaz, G., Gottlieb, R. L., Lobo, S. M., Robinson, P., Hunter, B. D., Cavalcante, A.
W., Overcash, J. S., Hanania, N. A., Skarbnik, A., Garcia-Diaz, J., Gordeev, I., Carratalà, J.,
Gordon, O., Graham, E., Lewin-Koh, N., Tsai, L., Tuckwell, K., Cao, H., … Olsson, J. K. (2021).
Tocilizumab and remdesivir in hospitalized patients with severe COVID-19 pneumonia: A
randomized clinical trial. Intensive Care Medicine, 47(11), 1258–1270.
https://doi.org/10.1007/s00134-021-06507-x
Rosas, J., Liaño, F. P., Cantó, M. L., Barea, J. M. C., Beser, A. R., Rabasa, J. T. A., Adsuar, F.
M., Auli, B. V., López, I. F., Sainz, A. M. G., Ramis, P. E., Pérez, L. R., Rebollo, M. L. N., Lorido,
R. H., & Escolar, L. G. (2022). Experience With the Use of Baricitinib and Tocilizumab
Monotherapy or Combined, in Patients With Interstitial Pneumonia Secondary to Coronavirus
COVID19: A Real-World Study. Reumatologia Clinica, 18(3), 150–156.
https://doi.org/10.1016/j.reuma.2020.10.009
Ruan, Q., Yang, K., Wang, W., Jiang, L., & Song, J. (2020). Clinical predictors of mortality due to
COVID-19 based on an analysis of data of 150 patients from Wuhan, China. Intensive Care
Medicine, 46(5), 846–848. https://doi.org/10.1007/s00134-020-05991-x
43

Salama, C., Han, J., Yau, L., Reiss, W. G., Kramer, B., Neidhart, J. D., Criner, G. J.,
Kaplan-Lewis, E., Baden, R., Pandit, L., Cameron, M. L., Garcia-Diaz, J., Chávez, V.,
Mekebeb-Reuter, M., Lima de Menezes, F., Shah, R., González-Lara, M. F., Assman, B.,
Freedman, J., & Mohan, S. V. (2021). Tocilizumab in Patients Hospitalized with Covid-19
Pneumonia. New England Journal of Medicine, 384(1), 20–30.
https://doi.org/10.1056/NEJMoa2030340
Schneeweiss S. (2006). Sensitivity analysis and external adjustment for unmeasured
confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiology and
drug safety, 15(5), 291–303. https://doi.org/10.1002/pds.1200
Shah, S., Bhagavan, M., Naik, A., Khanam, R., Kumar, J., Alcaraz, R., Majdizadeh, S., Gao, L.,
& White, K. (2022). JAK Antagonists and IL-6 Inhibitors for Management of SARS-CoV-2: A
Meta-Analysis of RCTs. 1.
Stebbing, J., Phelan, A., Griffin, I., Tucker, C., Oechsle, O., Smith, D., & Richardson, P. (2020).
COVID-19: Combining antiviral and anti-inflammatory treatments. The Lancet Infectious
Diseases, 20(4), 400–402. https://doi.org/10.1016/S1473-3099(20)30132-8
Stewart, M., Rodriguez-Watson, C., Albayrak, A., Asubonteng, J., Belli, A., Brown, T., Cho, K.,
Das, R., Eldridge, E., Gatto, N., Gelman, A., Gerlovin, H., Goldberg, S. L., Hansen, E., Hirsch,
J., Ho, Y.-L., Ip, A., Izano, M., Jones, J., … Allen, J. (2021). COVID-19 Evidence Accelerator: A
parallel analysis to describe the use of Hydroxychloroquine with or without Azithromycin among
hospitalized COVID-19 patients. PLOS ONE, 16(3), e0248128.
https://doi.org/10.1371/journal.pone.0248128
Stone, J. H., Frigault, M. J., Serling-Boyd, N. J., Fernandes, A. D., Harvey, L., Foulkes, A. S.,
Horick, N. K., Healy, B. C., Shah, R., Bensaci, A. M., Woolley, A. E., Nikiforow, S., Lin, N.,
Sagar, M., Schrager, H., Huckins, D. S., Axelrod, M., Pincus, M. D., Fleisher, J., … Mansour, M.
K. (2020). Efficacy of Tocilizumab in Patients Hospitalized with Covid-19. New England Journal
of Medicine, 383(24), 2333–2344. https://doi.org/10.1056/NEJMoa2028836
The WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group. (2021).
Association Between Administration of IL-6 Antagonists and Mortality Among Patients
Hospitalized for COVID-19: A Meta-analysis. JAMA, 326(6), 499–518.
https://doi.org/10.1001/jama.2021.11330
Tleyjeh, I. M., Kashour, Z., Damlaj, M., Riaz, M., Tlayjeh, H., Altannir, M., Altannir, Y., Al-Tannir,
M., Tleyjeh, R., Hassett, L., & Kashour, T. (2021). Efficacy and safety of tocilizumab in
COVID-19 patients: A living systematic review and meta-analysis. Clinical Microbiology and
Infection, 27(2), 215–227. https://doi.org/10.1016/j.cmi.2020.10.036
44

Wang S V, Pinheiro S, Hua W, Arlett P, Uyama Y, Berlin J A et al. STaRT-RWE: structured
template for planning and reporting on the implementation of real world evidence studies BMJ
(2021); 372 :m4856 doi:10.1136/bmj.m4856
Weckstein, A., Vititoe, S., Rivera, D., … Rassen, J. (2021). International Society for
Pharmacoepidemiology—Temporal Trends in Medication Utilization Among Hospitalized
COVID-19 Patients. https://icpemeeting.secure-platform.com/a/gallery/rounds/3/details/1607
Westreich D., Cole, Constructing inverse probability weights for marginal structural models S.R.
(2010). Invited commentary: positivity in practice. American journal of epidemiology,
171(6):674-681. doi:10.1093/aje/kwp436
Zhang, J., Chen, C., Yang, Y., & Yang, J. (2022). Effectiveness of tocilizumab in the treatment of
hospitalized adults COVID-19. Medicine, 101(9), e28967.
https://doi.org/10.1097/MD.0000000000028967
Zhang, X., Shang, L., Fan, G., Gu, X., Xu, J., Wang, Y., Huang, L., & Cao, B. (2022). The
Efficacy and Safety of Janus Kinase Inhibitors for Patients With COVID-19: A Living Systematic
Review and Meta-Analysis. Frontiers in Medicine, 8.
https://www.frontiersin.org/article/10.3389/fmed.2021.800492
Zhong, J., Tang, J., Ye, C., & Dong, L. (2020). The immunology of COVID-19: Is immune
modulation an option for treatment? The Lancet Rheumatology, 2(7), e428–e436.
https://doi.org/10.1016/S2665-9913(20)30120-X
Zhou, Y., Fu, B., Zheng, X., Wang, D., Zhao, C., Qi, Y., Sun, R., Tian, Z., Xu, X., & Wei, H.
(2020). Pathogenic T-cells and inflammatory monocytes incite inflammatory storms in severe
COVID-19 patients. National Science Review, 7(6), 998–1002.
https://doi.org/10.1093/nsr/nwaa041
45

8. APPENDICES
Appendix A. SPACE Template and / or other documentation supporting development of a
hypothetical target clinical trial.
Available upon request.
Appendix B: Description of Phased Study Approach
Not applicable; included in protocol
Appendix C: Detailed context and rationale for covariates, including DAG
Directed acyclic graph (DAG) was generated using Daggity: http://www.dagitty.net/
For purposes of display, variables in the DAG have been condensed where similar in terms of
relationships with exposure, outcome, and antecedents. The following high-level DAG applies to
both death and IMV/ECMO outcomes.
46

Appendix D: Full START-RWE Tables
For SPA analyses unless otherwise specified. Complete code lists and definitions for all study variables are available in Appendix F.
Code Lists.
Appendix D1: START-RWE Table. Index Date (Time T) defining criterion
Study
population
name(s)
Day 0
Description
Type of
entry
Washout
window
Care
Setting¹
Code Type
Incident
with
respect to…
Pre-
specified
Varied for
sensitivity
Source of
algorithm
Exposure
Groups
IL6Ri
initiation
within 4 days
after Day 0
Incident
[Days -90
to T-1]
IP CDM
ICD-10-PCS,
HCPCS/CPT, charge
code (see code list
appendix)
IL6Ri
dispensing
or remaining
supply
during
washout
Yes
Yes
De Novo
Code
Search
JAKi initiation
within 4 days
after Day 0
Incident
[Days -90
to T-1]
IP CDM
ICD-10-PCS,
HCPCS/CPT, charge
code (see code list
appendix)
JAKi
dispensing
or remaining
supply
during
washout
Yes
Yes
De Novo
Code
Search
All temporal windows anchored on study population entry date (Day 0) unless otherwise specified.
( ) represent open intervals that do not include the endpoints
[ ] represent closed intervals that do include the endpoints
¹ IP CDM = inpatient (chargemaster only), OP CDM = outpatient (chargemaster only), Med Claims = medical claims, Rx Claims = pharmacy claims
47

Appendix D2: START-RWE Table: Follow up
Pre-specified
Varied for
sensitivity
Begins
Day T+1
Yes
Yes
Ends
Select all that
apply
Specify
Date of outcome:
Yes (all analyses)
-
Inpatient mortality
(Illustrative Example Objectives I & II)
-
Progression to IMV/ECMO
(Illustrative Example Objective I only,
for secondary outcome of IMV/ECMO
only)
Yes
Yes
Day 29 following T+1
Yes (all analyses)
T+29
Yes
No
Date of discharge
Yes (all analyses)
Date of discharge from inpatient hospital
Yes
No
End of exposure
(JAKi discontinuation)
Possible
contingency for
IMA-2
Early discontinuation of JAKi
(Only individuals who initiate JAKi at treatment
index are eligible for this criterion (i.e., IL6Ri
initiators at treatment index cannot be censored
for this reason). For this study, JAKi treatment
is considered complete when a person has
been treated for 14 days OR if a person with
ongoing JAKi treatment is discharged from the
hospital (before 14 days), after applying a 2 day
event extension and 2 day grace period.
Please see code list for further detail on
operationalization of this censoring criterion.)
Yes
Yes
Date of crossover (add
to/switch from one exposure
to the other)
Possible
contingency for
IMA-2
Treatment crossover between IL6Ri or JAKi
drug groups (any add-on or switch)
Yes
Yes
48

Supplemental Figure for Appendix D2. Censoring Reasons During Follow-up
49

Appendix D3: START-RWE Table. Inclusion Criteria
Criterion
Details
Order of
application
Assessment
window
Care
Settings1
Code Type
Diagnosis
Position2
Applied to
study
populat-
ions:
Pre -
specified
Varied
for
sensitivity
Source
for
algorithm
Hospitalized Patients
Date Range:
June 16 2020 to February
01 2022
Before
selection of
index date
Day 0 -
Hospital
Admission
IP CDM
-
-
All3
Yes
No
-
Patients with
admitting diagnosis of
COVID-19
-
Before
selection of
index date
Day 0
IP CDM
ICD-10-CM
Any
admitting
All3
Yes
No
CDC
Coding
Guideline
s, 2020
Initiate either IL6Ri or
JAKi
-
Defines index
event
[Days 0, 4]
IP CDM
ICD-10-PCS,
HCPCS/CPT,
charge code
(see code list
appendix)
-
All3
Yes
Yes
De Novo
Code
Search
Receipt of at least
one respiratory
support
procedure
Includes codes for oxygen
supplementation or
ventilation (Does not
include intubation codes
alone)
After selection
of index date
[Days 0, T]
IP CDM,
OP CDM
ICD-10-PCS,
HCPCS/CPT,
charge code
(see code list
appendix)
-
All3
Yes
No
Garry,
2022; De
Novo
Code
Search
50

Criterion
Details
Order of
application
Assessment
window
Care
Settings1
Code Type
Diagnosis
Position2
Applied to
study
populat-
ions:
Pre -
specified
Varied
for
sensitivity
Source
for
algorithm
Maximum mWHO
disease severity
Objective I: max = O2 or
NIV/HFO
Objective II: max =
IMV/ECMO
After selection
of index date
[Days 0, T]
IP CDM,
OP CDM
Various (see
code list
Appendix.
This inclusion
variable uses
mWHO
version with
both
procedure
and admitting
diagnosis
codes)
-
-
Yes
No
Garry,
2022; De
Novo
Code
Search
Receipt of systemic
CSI
-
After selection
of index date
Day T
IP CDM,
OP CDM
Various (see
code list
Appendix)
-
All3
Yes
No
De Novo
Code
Search
ICU admission
Objective II population
only; any ICU/CCU
Before
selection of
index date
Day 0 = same
day as ICU
admission
IP CDM,
OP CDM
Dept codes
HCPCS/
CPT, charge
codes
-
Objective
II
population
only
Yes
No
-
All temporal windows anchored on study population entry date (Day 0) unless otherwise specified.
( ) represent open intervals that do not include the endpoints
[ ] represent closed intervals that do include the endpoints
¹ IP CDM = inpatient (chargemaster only), OP CDM = outpatient (chargemaster only), Med Claims = medical claims, Rx Claims = pharmacy claims.
² Specify whether a diagnosis code is required to be in the primary position (main reason for encounter)
3 All study populations, including Illustrative Example Objective I population (O2/NIV/HFO) and Illustrative Example Objective II population (IMV/ECMO +ICU).
51

Appendix D4: START-RWE Table. Exclusion Criteria
Criterion
Details
Order of
application
Assessment
window
Care
Settings¹
Code Type
Diagnosis
Position2
Applied to
study
populations:
Pre-
specified
Varied for
sensitivity
Source
for
algorithm
Missing continuous
medical claims
enrollment
60 day gaps permitted
Before
selection of
index date
[Days -183,
0]
Med
Claims
-
-
All3
Yes
No
-
COVID-19 related
hospitalization occurs
>14 days after initial
diagnosis (or if any
diagnosis is recorded
90 days prior to
admission)
Excludes patients with
possible long-term COVID
or post-acute sequelae
while still permitting prior
infections recorded more
than 90 days pre-
admission
Before
selection of
index date
[Days -90,
-15]
IP CDM,
OP CDM,
Med
Claims
ICD-10-CM
Any
All3
Yes
No
-
Missing age,  sex
and geographic
region
-
Before
selection of
index date
Day 0
-
-
-
All3
Yes
No
-
Age less than 18
years
-
Before
selection of
index date
Day 0
-
-
-
All3
No
No
Evidence of prior
COVID-related
hospitalization
2 day buffer permits brief
inpatient utilization directly
proceeding transfer to
charge-
master hospital
Before
selection of
index date
[Days -14, -3]
IP CDM,
Med
Claims
ICD-10-CM
in inpatient
settings
-
All3
Yes
No
-
Evidence of prior
systemic CSI use
2 day buffer permits CSI
use in other healthcare
settings directly before
charge-
master hospital admission
Before
selection of
index date
[Days -14, -3]
IP CDM,
OP CDM,
Med
Claims, Rx
Claims
Various
(see code
list
Appendix)
-
All3
Yes
No
De Novo
Code
Search
52

Criterion
Details
Order of
application
Assessment
window
Care
Settings¹
Code Type
Diagnosis
Position2
Applied to
study
populations:
Pre-
specified
Varied for
sensitivity
Source
for
algorithm
Evidence of prior
IL6Ri use
Dispensing or remaining
supply
Before
selection of
index date
[Days -90, -1]
for claims
only; [Days
-90 to T-1] for
IP/OP CDM
only
IP CDM,
OP CDM,
Med
Claims,
Rx Claims
Various
(see code
list
Appendix)
-
All3
Yes
No
De Novo
Code
Search
Evidence of prior
JAKi use
Dispensing or remaining
supply
Before
selection of
index date
[Days -90, -1]
for claims
only; [Days
-90 to T-1] for
IP/OP CDM
only
IP CDM,
OP CDM,
Med
Claims,
Rx Claims
Various
(see code
list
Appendix)
-
All3
Yes
No
De Novo
Code
Search
Baseline respiratory
support for
non-COVID
conditions
2 or more respiratory
support procedures
recorded on different days
Before
selection of
index date
[Days -90,
-15]
IP CDM,
OP CDM,
Med
Claims
Various
(See code
list
appendix)
-
All3
Yes
No
Garry,
2022
Death or discharge
occurs
Before or on the day of
treatment initiation
Before
selection of
index date
[Days 0, T]
IP CDM
-
-
All3
Yes
No
-
Recorded use of both
IL6Ri AND JAKi on
index date
-
On index
date
Day T
IP CDM
Various
(see code
list
Appendix)
-
All3
Yes
No
De Novo
Code
Search
All temporal windows anchored on study population entry date (Day 0) unless otherwise specified.
( ) represent open intervals that do not include the endpoints
[ ] represent closed intervals that do include the endpoints
¹ IP CDM = inpatient (chargemaster only), OP CDM = outpatient (chargemaster only), Med Claims = medical claims, Rx Claims = pharmacy claims.
² Specify whether a diagnosis code is required to be in the primary position (main reason for encounter)
3 All study populations, including Illustrative Example Objective I population (O2/NIV/HFO) and Illustrative Example Objective II population (IMV/ECMO+ICU).
53

Appendix D5: START-RWE Table: Outcome
Outcome
name
Outcome measurement
characteristics
Primary
outcome?
Type of
outcome
Assessment
window
Care
Settings¹
Code
Category
Applied to
study
populations:
Pre -
specified
Varied for
sensitivity
Source of
algorithm
Inpatient
mortality
Hazard ratios and 95%
CIs will be estimated
using a marginal
structural model Cox
proportional hazards
analysis
Yes
Time-to-
event
[Days T+1,
end of FU3]
IP CDM, OP
CDM
Defined using
discharge
status field
(see code list
Appendix)
All3
Yes
No
-
Progression
to IMV or
ECMO
Hazard ratios and 95%
CIs will be estimated
using a marginal
structural model Cox
proportional hazards
analysis
No
Time-to-
event
[Days T+1,
end of FU3]
IP CDM, OP
CDM
ICD-10-
PCS,
HCPCS/
CPT, charge
codes (see
code list
Appendix)
Illustrative
Example
Objective I
population
(O2/NIV/HFO)
only
Yes
No
Garry,
2022; De
Novo
Code
Search
All temporal windows anchored on study population entry date (Day 0) unless otherwise specified.( ) represent open intervals that do not include the endpoints
[ ] represent closed intervals that do include the endpoints
¹ IP CDM = inpatient (chargemaster only), OP CDM = outpatient (chargemaster only), Med Claims = medical claims, Rx Claims = pharmacy claims, ED =
emergency department, any, other, n/a = not applicable.
² All study populations, including Illustrative Example Objective I population (O2/NIV/HFO) and Illustrative Example Objective II population (IMV/ECMO +ICU).
3 For primary analyses, follow-up for outcomes will begin on Day T+1 and continue until the earliest occurrence of death, outcome (if different from death),
discharge from the hospital, or 28 days of follow-up reached (Day T+29).
54

Appendix D6: START-RWE Table. Covariates
Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
History of
smoking/tobacco use
(baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims, Rx
Claims
ICD-10-
CM, HCPCS/
CPT, NDC
generic name
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Asthma (baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
ICD-10-
CM
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Any cancer (active
hematological cancer,
active solid tumor)
(baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
ICD-10-
CM
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Chronic lung disease
(baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
ICD-10-
CM
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Any cardiovascular
disease (baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
ICD-10-
CM
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Type 1 or 2 Diabetes
(baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
ICD-10-
CM
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Immuno-
suppressive conditions
or medications
(including blood/organ
transplant, rheumatoid
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims, Rx
Claims
ICD-10-
CM, ICD-10-
PCS,
HCPCS/CPT,
charge
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
55

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
arthritis and other
autoimmune conditions)
(baseline)
codes, NDC
generic name,
NDC code, WHO
ATC code
Kidney disease (end
stage renal disease,
acute kidney injury,
dialysis) (baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
ICD-10-CM,
HCPCS/CPT,
ICD-10-PCS,
charge codes,
revenue codes,
dept codes
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Overweight/
obese (baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
ICD-10-
CM
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Liver dysfunction
(hepatic impairment,
active liver disease)
(baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
ICD-10-
CM
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Other conditions with
potential IL6Ri or JAKi
cautions (bowel
obstruction, necrosis,
ulcerative disease,
Iatrogenic GI injury,
demyelinating
disorders, serious
infections, cystic
fibrosis, pregnancy,
sickle cell /
thalassemia) (baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
ICD-10-
CM
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
56

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
Pregnancy (baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
ICD-10-
CM, HCPCS/CPT
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Frailty score (baseline)
Numeric
Baseline†
(assessed at
Day -1 with
183 day
lookback)
IP CDM, OP
CDM, Med
Claims
ICD-10-
CM, HCPCS/CPT
-
All3
Yes
Yes (SA-1
extended
baseline period)
Kim, 2017
Neurological/
cognitive impairment
(baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
ICD-10-
CM
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Mental health/
psychosis (baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
ICD-10-
CM
-
All3
Yes
Yes (SA-1
extended
baseline period)
Component of
Gagne, 2011
Hypertension (baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
ICD-10-
CM
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Combined Comorbidity
score
Continuous
Baseline†
(assessed at
Day -1 with
183 day
lookback)
IP CDM, OP
CDM, Med
Claims
ICD-10-
CM
-
All3
Yes
Yes (SA-1
extended
baseline period)
Gagne, 2011
Statin use (baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims, Rx
Claims
HCPCS/CPT
NDC generic
name,
charge codes
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
57

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
Systemic and inhaled
corticosteroid use
(baseline)
Binary (Yes or No)
(Record of new or
existing corticosteroid
use during baseline
period)
Baseline†
IP CDM, OP
CDM, Med
Claims, Rx
Claims
Various (see code
list appendix)
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Anticoagulant and/or
antiplatelet use
(baseline)
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims, Rx
Claims
HCPCS/CPT
NDC generic
name,
charge codes
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Days hospitalized
(baseline)
Numeric
Baseline†
IP CDM, Med
Claims
-
-
All3
Yes
Yes (SA-1
extended
baseline period)
-
Number of outpatient
visits (baseline)
Numeric
Baseline†
OP CDM,
Med Claims
-
-
All3
Yes
Yes (SA-1
extended
baseline period)
-
Number of pharmacy
claims (baseline)
Numeric
Baseline†
Rx Claims
-
-
All3
Yes
Yes (SA-1
extended
baseline period)
-
Number of distinct
medications dispensed
(baseline)
Numeric
Baseline†
Rx Claims
NDC generic
name
-
All3
Yes
Yes (SA-1
extended
baseline period)
-
Any baseline encounter
indicating skilled
nursing facility (SNF) or
long-term care (LTC)
Numeric
Baseline†
IP CDM, OP
CDM, Med
Claims
Various (see code
list Appendix)
-
All3
Yes
Yes (SA-1
extended
baseline period)
Twiddy 2016;
De Novo Code
Search
58

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
7+ days in SNF or LTC
during baseline
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
Various (see code
list Appendix)
-
All3
Yes
Yes (SA-1
extended
baseline period)
-
COVID-19 vaccination
status
Binary
(Yes or No)
Baseline†
IP CDM, OP
CDM, Med
Claims
Various (see code
list Appendix)
-
All3
Yes
Yes (SA-1
extended
baseline period)
De Novo Code
Search
Record of prior
COVID-19
Binary
(Yes or No)
[Start of
data, -15]
IP CDM, OP
CDM, Med
Claims
ICD-10 CM
-
All3
Yes
No
CDC Coding
Guidelines,
2020; De Novo
Code Search
Days since first
COVID-19 diagnosis to
admission
Numeric
[Days -14,
-1]
IP CDM, OP
CDM, Med
Claims
-
-
All3
Yes
No
-
Pre-admission use of
potential COVID-19
medications (systemic
or inhaled
corticosteroids,
monoclonal antibodies,
antivirals, RDV, etc)
Binary
(Yes or No)
[Days -14,
-1]
IP CDM, OP
CDM, Med
Claims, Rx
Claims
ICD-10 PCS,
HCPCS/CPT,
NDC
generic name,
charge codes
-
All3
Yes
No
De Novo Code
Search
COVID-19-
related utilization prior
to admission (urgent
care, emergency
department)
Binary
(Yes or No)
[Days -14,
-2]
IP CDM, OP
CDM, Med
Claims
Place of service,
revenue, dept
codes,
HCPCS/CPT
-
All3
Yes
No
De Novo Code
Search
59

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
Receipt of respiratory
support prior to
inpatient admission
Binary
(Yes or No)
[Days -14,
-1]
IP CDM, OP
CDM, Med
Claims
ICD-10 PCS,
HCPCS/CPT
charge codes
-
All3
Yes
No
De Novo Code
Search
Age
Continuous Numeric
Day 0
-
-
-
All3
Yes
No
-
Sex
Categorical (male or
female)
Day 0
-
-
-
All3
Yes
No
-
Insurance type
Categorical
(Commercial only,
Medicare w/ or w/o
Commercial, Medicaid
w/ or w/o
Medicare/Commercial)
Day 0
-
-
-
All3
Yes
No
-
US region
Categorical (Midwest,
Northeast, South, or
West)
Day 0
-
-
-
All3
Yes
No
-
Month and year of
hospital admission
Ordinal
Day 0
-
-
-
All3
Yes
No
-
Quarter-Year of
admission
Ordinal
Day 0
-
-
-
All3
Yes
No
-
Hospital setting type =
Urban
Binary (Yes for urban
hospital, no for rural
hospital)
Day 0
IP CDM
-
-
All3
Yes
No
-
Hospital teaching status
= teaching
Binary (Yes for major or
minor teaching hospital;
no for non-
Day 0
IP CDM
-
-
All3
Yes
No
-
60

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
teaching)
Hospital number of
beds
Categorical (<200 beds,
200-299 beds, 300-399
beds, 400+ beds)
Day 0
IP CDM
-
-
All3
Yes
No
-
Hospital facility
COVID-19 admitting
volume4
Categorical (low:
admitted to facility with
<10 COVID-19
admissions per
year-quarter; medium:
admitted to facility with
10-29 COVID-19
admissions per
year-quarter; high:
admitted to facility with
≥ 30 COVID-19
admissions per
year-quarter)
Individuals were
assigned to one of
these mutually
exclusive categories
based on observed
COVID-19 admitting
volume at their
admitting facility during
their year-quarter of
admission.
Day 0
IP CDM
-
-
All3
Yes
No
COVID-19
admission
volume was
calculated by
facility and
year-quarter
among the
overall
hospitalized
cohort prior to
identification of
IL6Ri/JAKi
initiators.
61

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
Hospital facility level of
immunomodulator
(IL6Ri/JAKi)
prescribing4
Categorical (low: ≤25th
percentile; medium:
26th-74th percentiles;
high: ≥75th percentile).
Percentiles represent
the facility-level
percentage of
COVID-19 admissions
with recorded
immunomodulator (IM)
treatment during a
given year-quarter,
relative to all other
facilities in that same
year-quarter.
Individuals were
assigned to one of
these mutually
exclusive categories
based on their facility’s
level of IM prescribing
percentile within that
year-quarter of
admission.
Day 0
IP CDM
-
-
All3
Yes
No
Level
(percentage) of
IM treatment
was calculated
by facility and
year-quarter
among the
overall
hospitalized
COVID-19
cohort prior to
identification of
IL6Ri/JAKi
initiators.
Admitting diagnoses:
Dyspnea/ hypoxia
Binary
(Yes or No)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Any
Admitti
ng
All3
Yes
No
De Novo Code
Search
Admitting diagnoses:
Pneumonia
Binary
(Yes or No)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Any
Admitti
ng
All3
Yes
No
De Novo Code
Search
62

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
Admitting diagnoses:
ARDS or acute
respiratory failure
Binary
(Yes or No)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Any
Admitti
ng
All3
Yes
No
De Novo Code
Search
Admitting diagnoses:
ARDS (for objective II
only)
Binary
(Yes or No)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Any
Admitti
ng
Objective
II only
Yes
No
De Novo Code
Search
Admitting diagnoses:
Shock
Binary
(Yes or No)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Any
Admitti
ng
All3
Yes
No
De Novo Code
Search
Admitting diagnoses:
Non-respiratory organ
failure
Binary
(Yes or No)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Any
Admitti
ng
All3
Yes
No
De Novo Code
Search
Admitting diagnoses:
Delirium
Binary
(Yes or No)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Any
Admitti
ng
All3
Yes
No
De Novo Code
Search
Admitting diagnoses:
Sepsis
Binary
(Yes or No)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Any
Admitti
ng
All3
Yes
No
De Novo Code
Search
Admitting diagnoses:
Do not resuscitate order
Binary
(Yes or No)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Any
Admitti
ng
All3
Yes
No
De Novo Code
Search
Admitting diagnoses:
Acute cardiovascular
event
Binary
(Yes or No)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Any
Admitti
ng
All3
Yes
No
De Novo Code
Search
63

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
Admitting diagnoses:
Thromboembolic events
Binary
(Yes or No)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Any
Admitti
ng
All3
Yes
No
De Novo Code
Search
Admitting diagnoses:
U07.1 diagnosis in
primary position
Binary
(Yes for U07.1 in
primary position; No for
non-U07.1 in primary
position)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Primary
,
Priority
1, or
Primary
Admitti
ng
All3
Yes
No
De Novo Code
Search
Admitting diagnoses:
COVID
complication-related
diagnoses in primary
position
Binary
(Yes for COVID-related
diagnosis in p; No for
non-COVID diagnosis
in primary position)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Primary
,
Priority
1, or
Primary
Admitti
ng
All3
Yes
No
De Novo Code
Search
Admitting diagnoses:
Presence of possible
contraindication
(concurrent non-COVID
infection, end stage
renal disease or acute
kidney injury, liver
dysfunction)
Binary
(Yes or No)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Any
Admitti
ng
All3
Yes
No
De Novo Code
Search
Admitting diagnoses:
non-COVID
opportunistic or serious
infections
Binary
(Yes or No)
Day 0
IP CDM, OP
CDM
ICD-10-
CM
Any
Admitti
ng
All3
Yes
No
De Novo Code
Search
64

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
Admitted from
Emergency Department
Binary
(Yes or No)
[Days -1, 0]
IP CDM, OP
CDM, Med
Claims
Place of service,
revenue, dept
codes,
HCPCS/CPT
-
All3
Yes
No
-
Ambulance
Binary
(Yes or No)
[Days -1, 0]
IP CDM, OP
CDM, Med
Claims
Place of service,
revenue, dept
codes
-
All3
Yes
No
-
Admitted directly to
ICU/CCU at admission
Binary
(Yes or No)
Day 0
IP CDM,  OP
CDM
Revenue Dept
codes, IP
indicator
-
Objective
I only3
Yes
No
-
Admitting type
Binary (Yes for
emergency/traumal, no
for elective/urgent)
[Days -1, 0]
IP CDM,  OP
CDM
Admit type
-
All3
Yes
No
-
Admitting status is
transfer from other
facility
Binary
(Yes or No)
[Days -1, 0]
IP CDM, OP
CDM
Admission Source
-
All3
Yes
No
De Novo Code
Search
Pre-admission SNF,
Nursing Home, or LTC
utilization occurs
Binary
(Yes or No)
[Days -2, -1]
IP CDM, OP
CDM, Med
Claims
Revenue/Dept
codes, IP
indicator, ICD-10
PCS,
HCPCS/CPT,
Place of Service,
Bill Type
-
All3
Yes
No
De Novo Code
Search
Pre-admission inpatient
utilization occurs
Binary
(Yes or No)
[Days -2, -2]
IP CDM, Med
Claims
Suspected IP
Indicator (claims),
any IP CDM event
-
All3
Yes
No
De Novo Code
Search
65

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
Admitting mWHO
severity
(max level of respiratory
support received,
diagnosis and
procedure-based
mWHO)
Categorical (NONE,
O2, NIV/HFO,
IMV/ECMO)
[Days -1, 0]
IP CDM,  OP
CDM
Various (see code
list Appendix. This
inclusion variable
uses procedure
and
diagnosis-based
mWHO)
-
All3
Yes
No
De Novo Code
Search
Anticoagulant/
antiplatelet agents
Binary
(Yes or No)
[Days 0, T]
IP CDM,  OP
CDM
HCPCS/CPT,
generic name,
charge codes
-
All3
Yes
No
De Novo Code
Search
Antibiotics
Binary
(Yes or No)
[Days 0, T]
IP CDM,  OP
CDM
Generic name,
charge codes
-
All3
Yes
No
De Novo Code
Search
Vasopressors or
inotropes
Binary
(Yes or No)
[Days 0, T]
IP CDM,  OP
CDM
HCPCS/CPT,
charge codes
-
All3
Yes
No
De Novo Code
Search
Neuromuscular
blockades
Binary
(Yes or No)
[Days 0, T]
IP CDM, OP
CDM
HCPCS/CPT,
generic name,
brand name,
charge codes
-
All3
Yes
No
De Novo Code
Search
Number of unique
medication classes
administered5
Numeric (see code lists
for details)
[Days T-4, T]
IP CDM, OP
CDM
Various (see code
list Appendix)
-
All3
Yes
No
-
Number of unique
injectable medication
procedure codes
recorded
Numeric
[Days 0, T]
IP CDM, OP
CDM
HCPCS/CPT
-
All3
Yes
No
-
66

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
Number of unique oral,
injectable, or inhaled
medication procedure
codes recorded
Numeric
[Days 0, T]
IP CDM, OP
CDM
HCPCS/CPT
-
All3
Yes
No
-
Inhaled CS
Binary
(Yes or No)
[Days 0, T]
IP CDM, OP
CDM
HCPCS/CPT,
generic name,
charge codes
-
All3
Yes
No
De Novo Code
Search
Remdesivir
Binary
(Yes or No)
[Days 0, T]
IP CDM, OP
CDM
ICD-10-
PCS,
HCPCS/CPT,
generic name,
brand name,
charge code
-
All3
Yes
Yes
De Novo Code
Search
Non-RDV antivirals
(including
lopinavir/ritonavir, other
HIV protease inhibitors,
ivermectin, etc)
Binary
(Yes or No)
[Days 0, T]
IP CDM, OP
CDM
Various (see code
list appendix)
-
All3
Yes
No
De Novo Code
Search
Other biologic
immunomodulators
(Non-JAK kinase
inhibitors, non-IL6
interleukin inhibitors,
anti-TNF biologics, B or
T- cell inhibitors, etc)
Binary
(Yes or No)
[Days 0, T]
IP CDM, OP
CDM
Various (see code
list appendix)
-
All3
Yes
No
De Novo Code
Search
Any ICU/CCU
Binary
(Yes or No)
[Days 0, T]
for Objective
IP CDM, OP
CDM
Dept Codes
(Rev), HCPCS/
-
All3
Yes
No
De Novo Code
Search
67

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
I; [Day T] for
Objective II
CPT, charge
codes
Specific ICU/CCU (not
intermediate)
Binary
(Yes or No)
[Days 0, T]
IP CDM, OP
CDM
Dept Codes
HCPCS/
CPT, charge
codes
-
All3
Yes
No
De Novo Code
Search
Intermediate or
step-down ICU
Binary
(Yes or No)
[Days 0, T]
IP CDM, OP
CDM
Revenue/Dept
Codes HCPCS/
CPT, charge
codes
-
All3
Yes
No
De Novo Code
Search
Critical care procedures
Binary
(Yes or No)
[Days 0, T]
IP CDM, OP
CDM
HCPCS/
CPT
-
All3
Yes
No
De Novo Code
Search
Major surgery
Binary
(Yes or No)
[Days 0, T]
IP CDM, OP
CDM
HCPCS/CPT,
ICD-10-PCS (see
code lists)
-
All3
Yes
No
Component of
Kim, 2017 and
Clinical
Classifications
Software
Refined
(CCSR) 2021
Specialty care services
(radiology,
cardiovascular,
respiratory therapy,
surgery, dialysis)
Numeric, count of
unique specialty care
services recorded
[Days T-4, T]
IP CDM, OP
CDM
Dept codes,
HCPCS/CPT,
ICD-10-PCS,
charge codes
-
All3
Yes
No
De Novo Code
Search
specialty care
component: respiratory
therapy/services
Binary
(Yes or No)
[Days 0, T]
IP CDM, OP
CDM
Dept codes,
HCPCS/CPT,
ICD-10-PCS
-
All3
Yes
No
De Novo Code
Search
68

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
specialty care
component:
cardiovascular
therapy/services
Binary
(Yes or No)
[Days 0, T]
IP CDM, OP
CDM
Dept codes,
HCPCS/CPT,
ICD-10-PCS
-
All3
Yes
No
De Novo Code
Search
specialty care
component:
radiology/imaging
services
Binary
(Yes or No)
[Days 0, T]
IP CDM, OP
CDM
Dept codes,
HCPCS/CPT,
ICD-10-PCS
-
All3
Yes
No
De Novo Code
Search
specialty care
component: surgical
utilization
Binary
(Yes or No)
[Days 0, T]
IP CDM, OP
CDM
Dept codes,
HCPCS/CPT
-
All3
Yes
No
De Novo Code
Search
Renal replacement
therapy or dialysis
Binary
(Yes or No)
[Days 0, T]
IP CDM, OP
CDM
HCPCS/CPT,
ICD-10-PCS,
charge codes,
dept codes
-
All3
Yes
No
De Novo Code
Search
Number of
O2/ventilation
procedures
administered
Numeric
[Days 0, T]
IP CDM, OP
CDM
ICD-10-PCS
HCPCS/CPT
charge
codes
-
All3
Yes
No
Garry, 2022
(Count of unique)
COVID-19 related lab
orders (e.g. PCT,
ferritin, D-Dimers,
troponin, PaO2, CRPs,
AST/ALTs, blood
Numeric, count of
unique
COVID-19-related lab
orders
[Days T-4, T]
IP CDM, OP
CDM
HCPCS/CPT,
charge codes
-
All3
Yes
No
De Novo Code
Search
69

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
viscosity, lactate
dehydrogenase, etc.)
Number of unique
procedure codes5
Numeric
[Days 0, T]
IP CDM, OP
CDM
HCPCS/CPT,
ICD-10-PCS
-
All3
Yes
No
-
Number of unique
department codes
Numeric
[Days 0, T]
IP CDM, OP
CDM
Dept codes
-
All3
Yes
No
-
Max respiratory support
procedure recorded
(procedure-based
mWHO)
Categorical (NONE,
O2, NIV/HFO,
IMV/ECMO)
[Days 0, T]
IP CDM, OP
CDM
Various (See code
list appendix. This
covariate uses the
procedure only
mWHO algorithm)
-
All3
Yes
No
-
Most frequently
recorded mWHO status
(procedure-based
mWHO)
Categorical (NONE,
O2, NIV/HFO,
IMV/ECMO)
[Days 0, T]
IP CDM, OP
CDM
Various (See code
list appendix. This
covariate uses the
procedure only
mWHO algorithm)
-
All3
Yes
No
-
Number of days from
admission to treatment
initiation
Numeric or categorical
(0, 1, 2, 3, 4, 5)
[Days 0, T]
IP CDM
-
-
All3
Yes
No
-
Number of days with
CSI use prior to
treatment initiation
Numeric
[Days -2, T]
(assessed
using a
6-day
extension)
IP CDM, OP
CDM
-
-
All3
Yes
No
-
Number of days from
max respiratory support
Numeric
[Days 0, T]
IP CDM, OP
CDM
Various (See code
list appendix. This
-
All3
Yes
No
-
70

Covariate /
Characteristic
Type of variable
Assessment
window
Care
Settings¹
Code Type
Diag-
nosis
Pos-
ition2
Applied
to study
popul-
ations:
Pre -
specified
Varied for
sensitivity
Source for
algorithm
status to treatment
initiation
(procedure-based
mWHO)
covariate uses the
procedure only
mWHO algorithm)
Number of days with
O2/NIV/HFO/IMV/ECM
O support prior to
treatment initiation
Numeric
[Days 0, T]
IP CDM, OP
CDM
Various (see code
list appendix)
-
All3
Yes
No
-
mWHO disease severity
at treatment initiation
(procedure-based
mWHO)
Categorical (NONE,
O2, NIV/HFO,
IMV/ECMO
Day T
IP CDM, OP
CDM
Various (See code
list appendix. This
covariate uses the
procedure only
mWHO algorithm)
-
All3
Yes
No
-
All temporal windows anchored on study population entry date (Day 0) unless otherwise specified.
† Baseline period is defined as [Days -183, -1]
( ) represent open intervals that do not include the endpoints
[ ] represent closed intervals that do include the endpoints
¹ IP CDM = inpatient (chargemaster only), OP CDM = outpatient (chargemaster only), Med Claims = medical claims, Rx Claims = pharmacy claims.
² Specify whether a diagnosis code is required to be in the primary position (main reason for encounter)
3 All study populations, including Illustrative Example Objective I population (O2/NIV/HFO) and Illustrative Example Objective II population (IMV/ECMO+ICU).
4 For some patient covariates pertaining to the facility at which they are admitted, it is necessary to first conduct a facility-level analysis of the overall hospitalized
COVID-19 cohort. This analysis will be executed on a separate analytic cohort consisting of all hospitalized COVID-19 patients with CSI use and oxygen/ventilation
supplementation from Day 0 to Day 4, prior to identification of IL6Ri and JAKi initiators. Within this population, COVID-19 admission volume and level (proportion)
of IM (IL6Ri or JAKi) use will be calculated separately by facility and by calendar year-quarter, to account for temporal changes in COVID-19 incidence and IM
treatment. Finally, within Objective I and II study cohorts, each individual will be assigned a categorical value for “Hospital facility COVID-19 admitting volume “ and
“Hospital facility level of immunomodulator (IL6Ri/JAKi) prescribing” based on their admitting facility and year-quarter of admission.
5 IL6Ri and JAKi will be excluded from variable definitions to avoid adjustment for exposure-related characteristics
71

Appendix D7: START-RWE Table: Analyses
Hypothesis:
Hypothesis is that IL6Ri+CSI exposure, as compared to JAKi+CSI exposure, is associated with different risks of
death and IMV/ECMO (for Illustrative Example - Objective I only), with either the lower bound of the 95% hazard ratio above 1.0
or the upper bound of the 95% hazard ratio below 1.0.
Study population(s):
IL6Ri/JAKi initiators within 4 days of hospital admission
Outcome:
Inpatient 28-day mortality (primary outcome for both Illustrative Example Objectives), progression to IMV/ECMO (secondary
outcome for Illustrative Example Objective I)
Software:
Aetion Evidence Platform, version 2022 for SPA and IMA-1 analyses. R version 4 for select IMA-2 analyses (contingent on
diagnostic checks)
Model(s):
Outcome model: Cox proportional hazards; Propensity score model: logistic regression
See potential covariates for model inclusion in Table 4
Confounding adjustment method
Bivariate
☐
Multivariable
☐
Propensity score
matching (specify
matching algorithm,
ratio and caliper)
☐
Propensity score
weighting (specify
weight formula,
trimming, truncation
decisions)
☒Inverse probability weighting. The weights for each patient will be calculated as the inverse of the probability of receiving the
treatment the patient actually received conditional on observed covariates. Each individual’s contribution to the survival curves
and to the Cox regression model will be weighted by the inverse probability of receiving the treatment the patient actually received
conditional on observed covariates. A robust variance estimator will account for the weighted design. Any variables not balanced
after weighting will be included in the outcome model.
Trimming and / or truncation will be considered pending assessment of weight distribution in diagnostic check #9. If appropriate,
stabilized weights will be used and a select number of patients (e.g. 10) with the most extreme high and low weights will be
trimmed.
72

Propensity score
stratification (specify
strata definition)
☐
Other (specify details)
☐
Missing data methods
(check all that apply, provide relevant details)
Missing indicators
☒Patients with missing age, gender, or geographic region were excluded from study cohorts.
Complete case
☒Patients with missing or unknown gender, age and region are excluded
Last value carried
forward
☐
Multiple imputation
(specify variables)
☐
Other (please
specify)…
☒Assumed that if there were no codes indicative of a binary (yes/no) condition or characteristic within the assessment window,
that it was not present for the patient.
Subgroup Analysis
NA
73

Appendix E: Data Management Process Details
Raw data review
At Aetion, raw data review is conducted to understand contents of the data table(s), establish
relationships, and help inform the database connection specification. Scientific integrity checks
are performed to understand if the contents of the data shipment is consistent with the expected
data as laid out in the applicable data usage agreement. Some of the key characteristics
explored in this process include:
●
Table structure (number of rows, columns, column names etc.)
●
Summary counts per table (i.e., non-missing counts, unique counts)
●
Variable distribution (e.g., min, mean, median, max for numeric variables; top
frequencies for categorical variables)
●
Date range (min, max and distribution over a time period)
●
Missingness percentage of attributes
Database connection process
Following receipt and review of the raw data, a data connector specification is drafted by a data
scientist.  The specification provides a map to Engineering for transformation of raw data to the
Aetion longitudinal patient timeline. It includes information such as:
●
Overall schema including tables (event types), rows (events), and columns (attributes);
derivation of attributes to improve data flexibility on Platform and rationale for any
attributes or events that are dropped
●
Event dates that define how data will be reflected on the longitudinal patient timeline,
and any minimal processing rules (e.g., drop an event that does not include a start or
end date)
●
Skeleton structure that represents the logical view of the entire database, defining how
the data is organized and how the relations among them are associated
●
Information for UI and labeling
●
Codes and definitions; typically used to substitute users’ having to look-up multiple
resources to understand/process data
Validation of the database connection (DBC) is completed to ensure that the implementation of
DBC logic leads to transformed data output that connects to and behaves within AEP exactly as
intended. Raw data are never loaded as-is directly into the Aetion Evidence Platform; rather,
data are transformed (via the DBC) into an Aetion-compatible format. DBC validation is required
to confirm that this transformation was performed correctly. This helps to ensure
validity/accuracy of the connected data and its importance cannot be ignored.  Validation is
performed via double programming, where two different people work independently from the
same DBC specification and then compare their output. The DBC is considered validated if the
outputs are identical. If the outputs are not identical, then the source of the discrepancy must be
investigated and resolved.
Following validation, the specification files are used to create an Aetion data dictionary for the
dataset. In addition, throughout the data connector spec / creation process, any issues or
decisions that have to be made that are not otherwise specified in the Specification files (e.g.,
how missing dates are handled), are noted in the data dictionary.
Prior to deployment on the AEP, a pre-flight checklist (PFC) is implemented.  The PFC is a
manual test of certain platform features and dataset values as visible and testable on the
74

front-end. A PFC is run following any deployment activity (such as a version update and/or
data/shard update). What’s checked for includes:
●
Baseline values for database information (dataset name, patient counts, earliest and
latest event dates)
●
Database configuration (specified dataset values)
●
Measure, Cohort, and Analysis Generations to confirm this functionality using the
dataset
●
Output from generated analysis output
●
Coding Systems, if applicable
Appendix F: Code Lists
Available upon request
Appendix G: Exploratory Analysis Results
Not applicable; included in protocol
Appendix H: Potential Addendum 1: Final List of Covariates Included in Model(s) and
Contingencies Pursued
Will be included in the final study report.
Appendix I: References for Appendix
Contains references within Appendices A-H that are not cited in main protocol text.
Clinical Classifications Software Refined (CCSR). Healthcare Cost and Utilization Project
(HCUP). October 2021. Agency for Healthcare Research and Quality, Rockville, MD.
www.hcup-us.ahrq.gov/toolssoftware/ccsr/ccs_refined.jsp.
Kim, D. H., Schneeweiss, S., Glynn, R. J., Lipsitz, L. A., Rockwood, K., & Avorn, J. (2017).
Measuring Frailty in Medicare Data: Development and Validation of a Claims-Based Frailty
Index. The Journals of Gerontology: Series A, 73(7), 980–987.
https://doi.org/10.1093/gerona/glx229
Twiddy, D. (2016). Medicare: Physicians are mixing up E/M codes for skilled nursing facilities.
American Academy of Family Physicians. Retrieved May 25, 2022, from
https://www.aafp.org/pubs/fpm/blogs/gettingpaid/entry/medicare_physicians_are_mixing_up.htm
75