# Multinational characterization of neurological phenotypes in patients hospitalized with COVID-19

Trang T. Le, Alba Gutierrez-Sacristan, Jiyeon Son, Chuan Hong, Andrew M. South, Brett K. Beaulieu-Jones, Ne Hooi Will Loh, Yuan Luo, Michele Morris, Kee Yuan Ngiam, et al.

<figure><img src="image_1.png" /></figure>

► To cite this version:

Trang T. Le, Alba Gutierrez-Sacristan, Jiyeon Son, Chuan Hong, Andrew M. South, et al. Multinational characterization of neurological phenotypes in patients hospitalized with COVID-19. Scientific Reports, 2021, 11 (1), pp.20238. 10.1038/s41598-021-99481-9. hal-03471113

HAL Id: hal-03471113

https://hal.science/hal-03471113

Submitted on 8 Dec 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

OPEN

# Multinational characterization of neurological phenotypes in patients hospitalized with COVID-19

Trang T. Le<fnref n="1" />,⁹², Alba Gutiérrez-Sacristán²,⁹², Jiyeon Son³,⁹², Chuan Hong², Andrew M. South⁴, Brett K. Beaulieu-Jones², Ne Hooi Will Loh⁵, Yuan Luo⁶, Michele Morris⁷, Kee Yuan Ngiam⁸, Lav P. Patel⁹, Malarkodi J. Samayamuthu⁷, Emily Schriver¹⁰, Amelia L. M. Tan², Jason Moore<fnref n="1" />, Tianxi Cai², Gilbert S. Omenn¹¹, Paul Avillach², Isaac S. Kohane², The Consortium for Clinical Characterization of COVID-19 by EHR (4CE)*, Shyam Visweswaran⁷,⁹³, Danielle L. Mowery<fnref n="1" />,⁹³ & Zongqi Xia³,⁹³✉

Neurological complications worsen outcomes in COVID-19. To define the prevalence of neurological conditions among hospitalized patients with a positive SARS-CoV-2 reverse transcription polymerase chain reaction test in geographically diverse multinational populations during early pandemic, we used electronic health records (EHR) from 338 participating hospitals across 6 countries and 3 continents (January–September 2020) for a cross-sectional analysis. We assessed the frequency of International Classification of Disease code of neurological conditions by countries, healthcare systems, time before and after admission for COVID-19 and COVID-19 severity. Among 35,177 hospitalized patients with SARS-CoV-2 infection, there was an increase in the proportion with disorders of consciousness (5.8%, 95% confidence interval [CI] 3.7–7.8%, p<0.001) and unspecified disorders of the brain (8.1%, 5.7–10.5%, p<0.001) when compared to the pre-admission proportion. During hospitalization, the relative risk of disorders of consciousness (22%, 19–25%), cerebrovascular diseases (24%, 13–35%), nontraumatic intracranial hemorrhage (34%, 20–50%), encephalitis and/or myelitis (37%, 17–60%) and myopathy (72%, 67–77%) were higher for patients with severe COVID-19 when compared to those who never experienced severe COVID-19. Leveraging a multinational network to capture standardized EHR data, we highlighted the increased prevalence of central and peripheral neurological phenotypes in patients hospitalized with COVID-19, particularly among those with severe disease.

The World Health Organization declared coronavirus disease 2019 (COVID-19) due to the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection as a global pandemic on March 11, 2020¹. Growing evidence points to the multi-organ involvement of COVID-19, particularly the nervous system, which increases morbidity and mortality²⁻⁴. Given the health consequences of neurological complications, recognizing the

<footnote n="1">¹Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA. ²Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA. ³Department of Neurology, University of Pittsburgh, Biomedical Science Tower 3, Suite 7014, 3501 5th Avenue, Pittsburgh, PA 15260, USA. ⁴Department of Pediatrics, Wake Forest School of Medicine, Winston Salem, NC, USA. ⁵Department of Critical Care, National University Health Systems, Singapore, Singapore. ⁶Department of Preventive Medicine, Northwestern University, Chicago, IL, USA. ⁷Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA. ⁸Department of Surgery, National University Health Systems, Singapore, Singapore. ⁹Department of Internal Medicine, University of Kansas Medical Center, Kansas City, KS, USA. ¹⁰Data Analytics Center, University of Pennsylvania Health System, Philadelphia, PA, USA. ¹¹Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA. ⁹²These authors contributed equally: Trang T. Le, Alba Gutiérrez-Sacristán and Jiyeon Son. ⁹³These authors jointly supervised this work: Shyam Visweswaran, Danielle L. Mowery and Zongqi Xia. *A list of authors and their affiliations appears at the end of the paper. ✉email: zxia1@post.harvard.edu</footnote>
<figure><img src="image_4.png" /><figcaption>**Figure 1.** Schematic diagram of the cohort and data generation workflow for each healthcare system. The figure was created with Biorender (Biorender.com).</figcaption></figure>

neurological phenotypes associated with COVID-19 would inform prevention, diagnosis and treatment that could potentially mitigate disability and death.

Early reports highlighted central and peripheral neurological phenotypes in adults with COVID-19, including cerebrovascular disease⁵, meningoencephalitis and encephalomyelitis⁶,⁷, encephalopathy⁵, cranial neuropathies⁸, Guillain-Barré syndrome⁹⁻¹¹, plexopathy¹², anosmia and ageusia¹³,¹⁴ and cognitive and neuropsychiatric issues¹⁵. Children with COVID-19 have similar presentations, including ischemic stroke, encephalopathy, headache, and muscle weakness¹⁶⁻¹⁸. Prior prevalence studies of neurological conditions in COVID-19 largely examined data from single countries (e.g., China¹⁹, the United Kingdom²⁰,²¹ and Italy²²) or single healthcare systems⁴,¹²,²³⁻²⁵. Few large-scale studies have used a standardized data capture approach to examine the multinational prevalence of the spectrum of neurologic conditions in COVID-19 patients and with careful local data quality control, particularly those who experienced severe respiratory and/or critical illness status²⁶,²⁷.

Electronic health records (EHRs) data can facilitate clinical discovery efforts. Our team created the International Consortium for Clinical Characterization of COVID-19 by EHR (4CE; http://www.covidclinical.net) to standardize and aggregate multinational EHR data (from 34 healthcare systems and 338 affiliated hospitals across six countries at the time of data freeze for this study) to address critical clinical and epidemiological questions relevant to COVID-19²⁸⁻³¹. Central to the 4CE effort is the ability of local clinician experts and data scientists at each contributing healthcare system to ensure the quality of common EHR data elements. Leveraging the highly scalable, federated, multinational networks of the 4CE consortium, we computed the prevalence of a wide range of central and peripheral neurological conditions in hospitalized patients with reverse transcription polymerase chain reaction (PCR)-confirmed SARS-CoV-2 infection by healthcare system and by country. We also compared the differences in the prevalence of neurological conditions between patients with and those without severe COVID-19 based on the internationally validated 4CE severity criteria³⁰.

## Results

Following a consortium-wide standardized procedure (Fig. 1), we collected the EHR data from 35,177 hospitalized patients with PCR-confirmed SARS-CoV-2 infection from 338 hospitals affiliated with 34 healthcare systems in six countries (eTable 1). Aggregate demographic data were available for 34,647 patients (98.5%). The cohort had a greater proportion of men (20,814, 60.1%) than women (13,546, 39.1%), while 287 (0.8%) patients had unknown gender (Fig. 2A). The study captured a broad range of proportions of severe COVID-19 cases (based on the internationally validated 4CE COVID-19 severity criteria³⁰) across this multinational network of healthcare systems. There was no clear relationship between COVID-19 severity and median age (Fig. 2B). Most healthcare systems in Europe did not report race. Among the US healthcare systems, there was a disproportionately high proportion of self-reported Black individuals (Fig. 2C). The study population included a high proportion of individuals above age 50 years and a low proportion of children (age < 18 years) (Fig. 2D).

We first assessed the prevalence of a wide spectrum of neurological conditions during the first hospital admission for COVID-19, importantly, using the estimates in the 1-year pre-admission period at each healthcare system as the comparator. Towards this end, we used EHR data from the early phase of the pandemic (January to September 2020) and queried all potential neurological conditions based on a comprehensive literature review at the start of the analysis. Most of the contributing healthcare systems (77%) reported an increase in the proportion of hospitalized COVID-19 patients with disorders of consciousness (ICD-10 R41: "Other symptoms and signs involving cognitive functions and awareness") with a mean increase of 5.8% (95% CI 3.7-7.8%, $p_{FDR} < 0.001$)
<figure><img src="image_5.png" /><figcaption>B</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>C</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>D</figcaption></figure>

<figure><img src="image_8.png" /></figure>

**Figure 2.** Characteristics of the study population across healthcare systems and countries. (A) Total number of male (left) and female (right) patients grouped by country shown in square-root scale. (B) Proportion of ever-severe cases by median age estimate at each healthcare system, grouped by country. Node size corresponds to the total number of patients per system. (C) Distribution of self-identified race among patients at healthcare systems in Singapore and the United States. The Other/Unknown category includes patients who did not identify with any of the predefined race categories and/or whose data were not reported. Most European healthcare systems did not report race. (D) Average proportion of patients in each age group within each country. FR, France; DE, Germany; ES, Spain; IT, Italy; SG, Singapore; US(A), United States of America.

after admission (Fig. 3, eFig. 1). Similarly, 84% of healthcare systems reported an increase in the proportion of patients with “Other disorders of the brain” (ICD-10 G93, including “encephalopathy”, “cerebral edema”, “brain death”) with a mean increase of 8.1% (5.7–10.5%, $p_{FDR} < 0.001$) after admission (see online interactive data repository: https://covidclinical.github.io/Phase1.1NeuroRCode/01-analysis-icd10.html#prevalence-change-table). The proportion of patients with “epilepsy and recurrent seizures” (ICD-10 G40), “encephalitis, myelitis, and encephalomyelitis” (ICD-10 G04) and “other and unspecified myopathies” (ICD-10 G72) increased after admission, but these findings were not significant after adjusting for multiple testing. Likewise, none of the other neurological conditions showed a statistically significant difference in prevalence after admission.

To assess the association with COVID-19 severity, we next used Fisher's exact test to examine the enrichment or depletion of each neurological condition among patients hospitalized for COVID-19 who ever experienced severe disease based on the published 4CE COVID-19 severity criteria³⁰, using those who never experienced severe disease as the comparator (Fig. 4). A positive log₂ value of enrichment (LOE) value denoted a higher proportion of severe cases for a given neurological condition than the never-severe cases, while a negative LOE value indicated the opposite. For instance, a LOE value of 0.283 for ICD code R41 meant that the observed number of severe cases with R41 (disorder of consciousness) was $2^{0.283}$ or ~1.22 times higher than the expected number of severe cases for R41, which was equivalent to a 22% increase in relative risk (i.e., relative risk difference RRD = 22%). Table 1 listed the neurological phenotypes that exhibited *statistically significant* associations with severe COVID-19 status ($p_{FDR} < 0.05$). The interactive data table (https://covidclinical.github.io/Phase1.1NeuroRCode/01-analysis-icd10.html#enrichment_tab) and the results directory of the project online data repository (https://github.com/covidclinical/Phase1.1NeuroRCode/tree/master/results) listed the LOE, 95% confidence intervals and *p* values for all neurological conditions examined.

In the period after hospital admission for COVID-19, using the patients who never experienced severe COVID-19 as reference, a significantly higher proportion of patients with severe disease had “other symptoms and signs involving cognitive functions and awareness” (ICD-10 R41: RRD$_{after}$ = 22%), “other cerebrovascular
<figure><img src="image_9.png" /><figcaption>B</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>**Figure 3.** Prevalence of neurological phenotypes among all patients. (A) Difference in prevalence of each neurological ICD-10 code by healthcare system and country, calculated as after admission—before admission date (eEq. 2). Pink color on the heat map indicates increased prevalence, while green color indicates decreased prevalence. Please see eFig. 1 for the absolute values of prevalence. (B) Total counts of patients with a given neurological ICD-10 code (left) and the mean proportion of patients (right) before and after admission date across all healthcare systems. The mean proportion estimates are shown as circles and the 95% confidence intervals are shown as bars.</figcaption></figure>

disease” (ICD-10 I67: RRD<sub>after</sub> = 24%), “nontraumatic subarachnoid hemorrhage” (ICD-10 I60: RRD<sub>after</sub> = 28%), “other and unspecified nontraumatic intracranial hemorrhage” (ICD-10 I62: RRD<sub>after</sub> = 34%), “nontraumatic intracerebral hemorrhage” (ICD-10 I61: RRD<sub>after</sub> = 36%), “other disorders of the brain” (ICD-10 G93, including “encephalopathy”: RRD<sub>after</sub> = 36%), “encephalitis, myelitis and encephalomyelitis” (ICD-10 G04: RRD<sub>after</sub> = 37%), and “other and unspecified myopathies” (ICD-10 G72, including “inflammatory and immune myopathies” and “critical illness myopathies”: RRD<sub>after</sub> = 72%) (Table 1, Fig. 4). In contrast, a significantly lower proportion of patients with severe disease had “blindness and low vision” (ICD-10 H54: RRD<sub>after</sub> = −23%), “dizziness and giddiness” (ICD-10 R42: RRD<sub>after</sub> = −28%), “other headache syndromes” (ICD-10 G44: RRD<sub>after</sub> = −47%), “transient cerebral ischemic attacks and related syndromes” (ICD-10 G45: RRD<sub>after</sub> = −45%), and “unspecified psychosis not due to a substance or known physiological condition” (ICD-10 F29: RRD<sub>after</sub> = −44%) during hospitalization. Subgroup analysis comparing the US and non-US healthcare systems yielded consistent results (eFig. 2).

As 12 healthcare systems in Italy and the US contributed EHR data entirely or partially comprising ICD-9 codes, we performed separate subgroup analyses using only the ICD-9 data (eTable 2), given that one-to-one mapping of some ICD-9 codes to ICD-10 codes was not feasible. To summarize, the prevalence of “disorders of consciousness” and “other neurological conditions” increased after COVID-19 admission date when compared to the 1-year pre-admission period, as in the ICD-10 data analysis (eFigs. 3, 4). There was no statistically significant difference when examining the change in prevalence of other neurological ICD-9 codes after admission date. In severity analysis, there were similarities with the ICD-10 data (e.g., “disorders of consciousness”) but also differences involving seizure and cerebrovascular events that would require caution in interpretation due to differences in sample size (see Supplementary Material, eFig. 5, eTable 3).
<figure><img src="image_11.png" /><figcaption>**Figure 4.** Analysis of enrichment or depletion of neurological conditions after admission in patients with severe disease. For each neurological ICD-10 code, we show the log₂ enrichment (LOE) and its 95% confidence interval (left), and the absolute difference between the observed (filled triangle) and expected (·) number of patients experiencing severe COVID-19 in square-root scale (right). A purple positive LOE value for an ICD-10 code indicates a statistically significantly higher proportion of severe cases having a given neurological ICD-10 code when compared to the never-severe cases. Conversely, a green negative LOE value indicates a statistically significantly lower proportion of severe cases having a given neurological ICD-10 code when compared to the never-severe cases. Neurological ICD-10 codes are ordered by the expected number of severe cases after admission.</figcaption></figure>

<table><caption>**Table 1.** Statistically significant associations of neurological conditions and severe disease status after admission ($p_{FDR} < 0.05$). <sup>a</sup>The interactive data table (https://covidclinical.github.io/Phase1.1NeuroRCode/01-analysis-icd10.html#enrichment_tab) and the results directory of the project online data repository (https://github.com/covidclinical/Phase1.1NeuroRCode/tree/master/results) show the log₂ value of enrichment (LOE), 95% confidence intervals, and *p* values for all neurological ICD-codes adjusted for multiple hypothesis testing.
<sup>b</sup>RRD: Relative risk difference = Observed relative risk – 1.</caption><thead><tr><th>Neurological condition (ICD-10 code)</th><th>LOE<sup>a</sup></th><th>RRD<sup>b</sup> (%)</th><th>RRD 95% CI (%)</th><th>p<sub>FDR</sub></th></tr></thead><tbody><tr><td>Blindness and low vision (H54)</td><td>-0.38</td><td>-23</td><td>(-33, -11)</td><td>2.0 × 10<sup>-4</sup></td></tr><tr><td>Dizziness and giddiness (R42)</td><td>-0.47</td><td>-28</td><td>(-33, -22)</td><td>6.3 × 10<sup>-19</sup></td></tr><tr><td>Encephalitis, myelitis and encephalomyelitis (G04)</td><td>0.45</td><td>37</td><td>(17, 60)</td><td>0.0081</td></tr><tr><td>Nontraumatic intracerebral hemorrhage (I61)</td><td>0.45</td><td>36</td><td>(23, 51)</td><td>3.6 × 10<sup>-5</sup></td></tr><tr><td>Nontraumatic subarachnoid hemorrhage (I60)</td><td>0.35</td><td>28</td><td>(10, 48)</td><td>0.019</td></tr><tr><td>Other and unspecified myopathies (G72)</td><td>0.78</td><td>72</td><td>(67, 77)</td><td>8.8 × 10<sup>-45</sup></td></tr><tr><td>Other and unspecified nontraumatic intracranial hemorrhage (I62)</td><td>0.43</td><td>34</td><td>(20, 50)</td><td>3.0 × 10<sup>-4</sup></td></tr><tr><td>Other cerebrovascular diseases (I67)</td><td>0.31</td><td>24</td><td>(13, 35)</td><td>2.0 × 10<sup>-4</sup></td></tr><tr><td>Other disorders of the brain (G93)</td><td>0.44</td><td>36</td><td>(32, 40)</td><td>5.6 × 10<sup>-73</sup></td></tr><tr><td>Other headache syndromes (G44)</td><td>-0.91</td><td>-47</td><td>(-59, -31)</td><td>9.4 × 10<sup>-9</sup></td></tr><tr><td>Other symptoms and signs involving cognitive functions and awareness (R41)</td><td>0.28</td><td>22</td><td>(19, 25)</td><td>2.1 × 10<sup>-39</sup></td></tr><tr><td>Transient cerebral ischemic attacks and related syndromes (G45)</td><td>-0.85</td><td>-45</td><td>(-56, -31)</td><td>5.0 × 10<sup>-10</sup></td></tr><tr><td>Unspecified psychosis not due to a substance or known physiological condition (F29)</td><td>-0.84</td><td>-44</td><td>(-57, -28)</td><td>7.7 × 10<sup>-8</sup></td></tr></tbody></table>

Discussion

Following a standardized approach for aggregating EHR-derived clinical facts from a federated network of multinational healthcare systems while preserving patient privacy, we report the change in prevalence of a wide spectrum of central and peripheral neurological conditions among a large cohort of patients hospitalized with PCR-confirmed SARS-CoV-2 infection across geographically diverse healthcare systems from six countries, using the 1-year period before COVID-19 hospitalization as reference. We further report the difference in the prevalence of neurological conditions among hospitalized patients with severe COVID-19 when compared to the non-severe patients. Given the challenges with evaluating patients and obtaining data during the pandemic, EHR-based studies complement the more labor-intensive physician-reported registry studies.

Disorders of consciousness and other disorders of the brain are the most prevalent neurological phenotypes among all patients hospitalized with COVID-19 during the early pandemic. While the specific underlying causes could be broad (e.g., metabolic disturbance, hypoxia, medication effect, seizures, stroke), these findings are
consistent with prior reports and associated with worse health outcomes¹⁹,²⁰,²⁴,²⁵. Given that hospitalized COVID-19 patients can manifest a wide range of symptoms and signs, certain neurological conditions do not cleanly fall within parent diagnostic codes with clearly specific descriptions (e.g., ICD-10 G04: encephalitis, myelitis and encephalomyelitis; ICD-10 G61: inflammatory polyneuropathy). In the case of “other disorders of brain” (including “unspecified disorders of the brain”), the parent diagnostic code of ICD-10 G93 covers a wide range of conditions, some of which are potentially relevant to hospitalized COVID-19 patients though they do not belong to a single unifying category (e.g., G93.1: anoxic brain damage; G93.2: benign intracranial hypertension; G93.3: post-viral fatigue syndrome; G93.4: other and unspecified encephalopathy; G93.5: compression of brain; G93.6: cerebral edema; G93.82: brain death). Other diagnosis codes under G93 (e.g., G93.9: unspecified disorders of the brain) likely capture symptoms and signs that may *not* be easily categorized, commonly reported (i.e., symptoms and signs that are *not* headache, visual changes, dizziness, confusion, weakness, or sensory changes), and/or revealing any objective pathology based on common clinical tools such as physical exam, laboratory, neuroimaging or electrodiagnostic tests. Here are two potential clinical scenarios that could lead to the diagnosis code of “unspecified disorder of a brain”: (1) non-specific findings on the brain magnetic resonance imaging that were not consistent with ischemic stroke, intracranial hemorrhage, meningitis or encephalitis, neoplasm, or demyelination; (2) subjective sensory complaint without objective findings on exam or diagnostic tests that was not coded with specific diagnosis. In contrast, the diagnosis code ICD-10 R41 (“disorders of consciousness”) specifically refers to symptoms and signs involving awareness and cognitive function, which include disorientation, amnesia, neglect, age-related cognitive decline, altered mental status, specific cognitive deficits involving attention, concentration, communication, visuospatial and psychomotor domains. This code will cover common clinical scenarios in hospitalized patients such as delirium (fluctuating arousal state) and decreased responsiveness.

On the other hand, we find no statistically significant increase in the prevalence of other previously reported neurological conditions such as dizziness, headache, seizure, vascular, and vision disorders after the first admission to the hospital for COVID-19. The lower prevalence of early COVID-19 symptoms such as alterations in smell and taste¹³ in this study is likely attributable to the incomplete documentation of these symptoms in the EHR for the hospitalized patient population, particularly those with severe COVID-19. Crucially, these discrepancies may also be due to methodological differences, as our analysis accounted for baseline pre-admission prevalence by reporting the change in prevalence after admission.

Our other major finding indicates that a significantly higher proportion of hospitalized patients with severe COVID-19 (based on a computational phenotyping algorithm of COVID-19 severity that our group previously published³⁰) had disorders of consciousness and other disorders of the brain, encephalitis and/or myelitis, cerebrovascular events and myopathy when compared to patients who never had severe disease. Beyond corroborating prior reports¹⁹,²³⁻²⁵, our findings highlighted similar patterns across geographically diverse multinational healthcare systems using a standardized approach. First among these findings, disorders of consciousness and other disorders of the brain include altered mental status, disorientation, cognitive deficits, and encephalopathy. Hypoxemia from respiratory failure, metabolic disturbance, sedation for advanced respiratory support, acute delirium, and other more specific neurological involvement in the setting of severe COVID-19 could all be contributory³².

Second, encephalitis and myelitis have variable manifestations¹² and were likely under-reported due to difficulties of performing diagnostic studies (e.g., magnetic resonance imaging, lumbar puncture) especially during the early phase of pandemic. Neuropathological examinations have not uncovered evidence of direct viral infection of the central nervous system (CNS)³³, though more studies are needed to confirm whether the mechanisms underlying encephalitis (with or without myelitis) are direct CNS invasion by SARS-CoV-2, acute systemic inflammation with secondary CNS involvement, and/or post-infectious immune-mediated effect on the CNS²,³⁴,³⁵.

Third, our finding of cerebrovascular diseases associated with severe COVID-19 include both ischemic strokes and intracranial hemorrhages (nontraumatic intracerebral and nontraumatic subarachnoid), highlighting the difficult balance when managing severe COVID-19 with respect to antiplatelet and anticoagulation therapy. Strokes that occurred in the setting of COVID-19 were associated with high mortality and morbidity³⁶. COVID-19 might increase the risk of ischemic stroke through mechanisms such as activation of innate immune system, cardioembolic events, hypoxia-induced ischemia secondary to severe pulmonary disease, coagulation activation, thrombotic angiopathy and endothelial damage². Proposed mechanisms underlying intracranial hemorrhage in COVID-19 include coagulation abnormalities, endothelial dysfunction, dysregulation of the renin-angiotensin system, and disruption of cerebral blood flow autoregulation²,³⁷. Patients with severe COVID-19 may have additional risk factors such as hypertension or cardiovascular disease that could further drive cerebrovascular diseases³⁸.

Finally, myopathy is common among severe COVID-19 patients. The cause is likely multifactorial: prolonged or severe critical illness, complications due to multi-organ involvement, or medication-induced myotoxicity (e.g., Hydroxychloroquine, steroids)²,³⁹. The current data set does not permit subgroup analysis (e.g., ventilated versus non-ventilated patients). Further, most clinical studies to date, including our own, cannot establish whether the neurological phenotypes associated with severe COVID-19 are the direct consequence of SARS-CoV-9 neurotoxicity or due to secondary causes.

Interestingly, some neurological phenotypes are less prevalent in patients with severe COVID-19, including psychosis, dizziness, vision impairment, transient ischemic attack, and headache. The likely explanation is that critically ill patients with or without respiratory failure would either not have an objective evaluation or proper documentation and coding for these conditions given that patients with severe disease are likely to be sedated and/or having altered mental status.

Several analytical elements strengthen the study. Chief of among them, the federated approach of adopting common EHR data elements and standardized processes for representing clinical events with *local* quality control
differentiates this study from other EHR-based efforts, complements physician/neurologist-reported registry efforts<sup>12,24,25,40</sup> and is well suited for multinational and multi-institutional clinical discovery. Critically, the local clinician experts and data scientists at each 4CE contributing healthcare system ensure the control of EHR data quality according to the consortium standard and improve the study rigor. The overall concordance of the main study findings with other registry-based studies is reassuring. Second, our approach of aggregating EHR data, specifically the ICD codes at categorical level, reduced the potential concern for variations in coding practice across diverse healthcare systems that include general medical, neurology, critical care, and rehabilitation settings. Third, the 4CE contributing healthcare systems shared aggregate clinical data following a pre-defined analysis plan while adhering to multi-national patient privacy laws such as the United States Health Insurance Portability and Accountability Act (HIPAA) and the European Union General Data Protection Regulation (GDPR). Fourth, we rapidly implemented the analysis plan at scale by leveraging existing informatics infrastructures and frameworks at each 4CE contributing healthcare system. This federated approach of using EHR data for clinical discovery presents a complementary and alternative approach to the more labor-intensive registry-based approach. Finally, our prevalence study used the 1-year pre-admission period for comparison, which generated more realistic prevalence estimation of neurological conditions among hospitalized patients with COVID-19 than approaches that do not account for pre-admission prevalence. Given the protracted and changing nature of the COVID-19 pandemic, we intentionally included only data during the first hospitalization for COVID-19, specifically during the early phase of the pandemic, to appropriately account for pre-admission prevalence.

Our study has limitations as the result of trade-offs to standardize common data collection from multinational healthcare systems while strictly preserving patient privacy and adhering to privacy laws governing all contributing healthcare systems. First, this study relied on ICD codes that may not capture fully or accurately the disease phenotypes, particularly for conditions better documented in clinical notes. To standardize the collection of ICD codes across diverse contributing healthcare systems and to mitigate coding discrepancies, we used ICD codes at the categorical level (e.g., the first 3 alphanumeric characters before the decimal point for ICD-10). As such, further characterization of certain conditions such as “other disorders of the brain” was not feasible at this stage, though we are working towards a standardized approach of capturing full ICD codes for the next stage of analyses. Second, because we aggregated data across healthcare systems, we were unable to consolidate all related ICD codes (e.g., organizing into PheCode<sup>41</sup>) at the individual patient level. Similarly, the 4CE consortium is preparing a standardized approach to enable aggregate patient-level analyses from each contributing healthcare system. Third, we might not have captured all pre-admission EHR data if patients did not receive their entire care in the same hospital system as the COVID-19 admission. This is a limitation common to all research using EHR data from countries without universal health systems such as the USA. Reassuringly, subgroup analysis showed consistent results between US and non-US contributing healthcare systems. Finally, contributing healthcare systems with small patient counts used obfuscation to reduce the risk of re-identifying the health systems, but the effect of obfuscation is negligible because few healthcare systems had neurological conditions of interest in cases fewer than the obfuscation level. Despite the limitations of deploying this rapid, scalable, patient privacy-preserving research strategy, our key findings were consistent with prior reports from well-characterized but often smaller, single-country and/or single-center cohort studies.

In conclusion, this multinational prevalence study highlighted a range of central and peripheral neurological phenotypes in hospitalized patients with PCR-confirmed SARS-CoV-2 infection, particularly among patients with severe disease. Our multinational and multi-institutional EHR-based efforts using a standardized procedure and common data elements with careful local data quality control complement registry-based research design. In future studies, we will conduct individual-level analysis using additional EHR data such as complete ICD codes, identify risk factors for worse health outcomes (e.g., hospitalization duration, death, re-admission) and examine long-term sequelae in COVID-19 patients with neurological phenotypes.

## Methods

**Patients and data.** 4CE contributing healthcare systems began in March 2020 to collect EHR data from hospitalized patients with positive SARS-CoV-2 RT-PCR tests. The analyzed data captured the early phase of the pandemic, spanning from January 2020 through early September 2020. We defined COVID-19-related hospitalization as the first hospital admission that occurred between 7 days before and up to 14 days after the first positive SARS-CoV-2 PCR test. The first admission date within this 7-day window is the index admission date.

According to the 4CE consortium agreement, we de-identified contributing healthcare systems to protect their confidentiality. The institutional review board of each participating health system (eTable 1) approved the sharing of anonymous, aggregate data in compliance with multi-national patient privacy laws exempting the requirement for individual patient consent as there was no direct patient recruitment or contact. Some healthcare systems applied a small level of obfuscation (i.e., masking of low counts, eTable 1) to preserve system-specific privacy and to reduce the risk of patient re-identification, though it had no significant impact on the total patient counts.

Using the 4CE standard of common EHR data elements<sup>28–31</sup>, we collected demographics (age, gender, self-identified race/ethnicity) and the International Classification of Disease (ICD) codes (versions 9 or 10) pertaining to neurological conditions (Fig. 1) as well as COVID-19 severity, based on the internationally validated 4CE COVID-19 severity criteria<sup>30</sup>. Among the contributing healthcare systems, only Italian healthcare systems provided exclusively ICD-9 codes while the rest of the healthcare systems provided predominantly ICD-10 codes. As such, we used ICD-10 data for the main conditions and ICD-9 data for supplementary analyses. To standardize EHR data elements, we used the first three alphanumeric characters of a given ICD code, which designates the category of the disease or injury (e.g., ICD-10 G44 denotes the category of “Other headache syndromes”).
<table><caption>**Table 2.** The 4CE criteria of severe COVID-19. Comprising the occurrence of diagnoses, procedures, laboratory results and medications, this computational phenotyping algorithm of severity has been internationally validated (through manual chart review by local clinician experts at participating healthcare systems) to be a clinically reasonable proxy for hospitalized patients who experienced severe status of COVID-19. This approach is applicable when the aggregate electronic health records data from each contributing healthcare systems are available but not the patient-level data. Please see Methods and further detail in a separate 4CE publication³⁰.</caption><thead><tr><td>Severe illness category</td><td>Clinical events</td></tr></thead><tbody><tr><td>Diagnoses</td><td>Acute respiratory distress syndrome, ventilator-associated pneumonia</td></tr><tr><td>Procedures</td><td>Insertion of endotracheal tube; invasive mechanical ventilation</td></tr><tr><td>Laboratory results</td><td>PaCO<sub>2</sub>, PaO<sub>2</sub></td></tr><tr><td>Medications</td><td>General anesthetics; benzodiazepine derivatives; muscle relaxants; other hypnotics and sedatives; adrenergic and dopaminergic agents; other cardiac stimulants; other respiratory system products; phosphodiesterase inhibitors; platelet aggregation inhibitors excluding heparin; vasopressin and analogues</td></tr></tbody></table>

For all patients hospitalized with COVID-19, we collected ICD codes at two time periods, *before* and *after* the date of admission of the first hospitalization for COVID-19. The period before COVID-19 hospitalization ranged from – 365 days to – 15 days preceding the admission date. Inclusion of EHR data up to 1 year before admission is a pragmatic decision to balance available data from all healthcare systems and minimize past medical conditions that might not be relevant. The period after admission date ranged from the date of admission to the end of the hospitalization. According to the pre-planned consortium-wide strategy, we excluded all codes in the 2 weeks preceding the index hospital admission date to ensure that diagnoses before admission were independent of COVID-19. It could take a few days from SARS-CoV-2 infection to symptom onset and additional days before a positive PCR test and/or hospital admission. Similarly, we analyzed ICD codes before and after the admission date according to whether patients ever met the 4CE criteria for severe COVID-19³⁰ (Fig. 1).

**Exposures.** We first examined all hospitalized patients with PCR-confirmed positive SARS-CoV-2. We then examined hospitalized patients with COVID-19 who met the 4CE criteria for severe COVID-19, including advanced respiratory care management at any point during their hospitalization. Including diagnoses, procedures, laboratory results and medications (Table 2). Because we used aggregate EHR data in this study, patient-level indicators of severity (e.g., patient-level laboratory value or medication) were unavailable. In response, we applied a computational algorithm of COVID-19 severity that the 4CE consortium previously developed and internationally validated (through chart review by local clinician experts at participating healthcare systems) to be a clinically reasonable proxy for hospitalized patients who experienced severe COVID-19 status³⁰.

**Neurological outcomes.** We queried 21 ICD-10 codes in 12 categories pertaining to neurological phenotypes following a comprehensive literature search in August 2020. The neurological disease categories included consciousness, coordination, dizziness, headache, inflammatory, muscle, neuropathy, psychiatric, seizure, vascular, vision and other unspecified neurological conditions (Table 3).

**Statistical analyses.** We first compared the prevalence of each neurological ICD code and disease category among all hospitalized patients with COVID-19. For each ICD code, we reported the total count and the proportion of patients hospitalized with COVID-19 at each healthcare system (and each country), both before and after admission date (Supplementary Material, eEq. 1). We used the proportion data before admission as reference control. We calculated the difference in the proportion of cases with a given ICD code before and after admission date (eEq. 2) and used paired two-sided t-tests to examine whether there was a statistically significant difference in the proportion after admission date when compared to that before admission date.

We next compared the prevalence of each neurological ICD code and disease category after admission date between patients who ever met the criteria of severe COVID-19 and those who did not, by healthcare system and by country. For each ICD code, we computed the expected number of severe cases (eEq. 3) and compared with the observed number of severe cases for the given neurological code. To examine the difference in proportion of severe cases for the neurological ICD codes, we calculated the enrichment of each neurological ICD code by dividing the observed number of severe cases by the expected number of severe cases and reported a value of log₂ enrichment (LOE) and its 95% confidence interval (CI) (eEq. 4). We estimated the LOE 95% CI using the Delta method⁴². We chose LOE as a statistic measure for the difference between the proportion of severe cases and never-severe cases for a given neurological condition because it allows symmetric visualization of enrichment or depletion (equivalent confidence intervals for enrichment and depletion. Finally, we computed the *p*-values using Fisher's exact test⁴³ and corrected for multiple hypothesis testing with Benjamini-Hochberg's false discovery rate (FDR) procedure⁴⁴. A result was statistically significant if *p*<sub>FDR</sub> < 0.05. All analyses were performed in the R environment.
<table><caption>**Table 3.** Mapping of neurological disease categories to ICD-10 category codes and their descriptions. <sup>a</sup>To standardize the collection of ICD codes across diverse contributing healthcare systems and to mitigate coding discrepancies, we used ICD codes at the categorical level (e.g., the first 3 alphanumeric characters before the decimal point for ICD-10).</caption><thead><tr><td>Disease category</td><td>ICD-10 code<sup>a</sup></td><td>ICD-10 code description</td></tr></thead><tbody><tr><td>Consciousness</td><td>R41</td><td>Other symptoms and signs involving cognitive functions and awareness</td></tr><tr><td>Coordination</td><td>R27</td><td>Other lack of coordination</td></tr><tr><td>Dizziness</td><td>R42</td><td>Dizziness and giddiness</td></tr><tr><td>Headache</td><td>G44</td><td>Other headache syndromes</td></tr><tr><td>Inflammatory</td><td>G03</td><td>Meningitis due to other and unspecified causes</td></tr><tr><td>Inflammatory</td><td>G04</td><td>Encephalitis, myelitis and encephalomyelitis</td></tr><tr><td>Muscle</td><td>G72</td><td>Other and unspecified myopathies</td></tr><tr><td>Muscle</td><td>M60</td><td>Myositis</td></tr><tr><td>Neuropathy</td><td>G61</td><td>Inflammatory polyneuropathy</td></tr><tr><td>Neuropathy</td><td>G65</td><td>Sequelae of inflammatory and toxic polyneuropathies</td></tr><tr><td>Neuropathy</td><td>R43</td><td>Disturbances of smell and taste</td></tr><tr><td>Other</td><td>G93</td><td>Other disorders of the brain</td></tr><tr><td>Psychiatric</td><td>F29</td><td>Unspecified psychosis not due to a substance or known physiological condition</td></tr><tr><td>Seizure</td><td>G40</td><td>Epilepsy and recurrent seizures</td></tr><tr><td>Vascular</td><td>G45</td><td>Transient cerebral ischemic attacks and related syndromes</td></tr><tr><td>Vascular</td><td>G46</td><td>Vascular syndromes of brain in cerebrovascular disease</td></tr><tr><td>Vascular</td><td>I60</td><td>Nontraumatic subarachnoid hemorrhage</td></tr><tr><td>Vascular</td><td>I61</td><td>Nontraumatic intracerebral hemorrhage</td></tr><tr><td>Vascular</td><td>I62</td><td>Other and unspecified nontraumatic intracranial hemorrhage</td></tr><tr><td>Vascular</td><td>I67</td><td>Other cerebrovascular diseases</td></tr><tr><td>Vision</td><td>H54</td><td>Blindness and low vision</td></tr></tbody></table>

## Data availability

The 4CE consortium does not have permission from each individual contributing healthcare system to release electronic health records data for public access. Only aggregate data were shared by healthcare systems for this study. All aggregate data in a de-identified fashion are available for download (http://www.covidclinical.net).

## Code availability

Our analysis results are available in browsable R notebooks (https://github.com/covidclinical/Phase1.1NeuroRCode/) under CC BY 4.0, with source code distributed under a BSD 3-Clause License. Figure 1 was created with Biorender (Biorender.com). Figures 2, 3, 4 as well as all supplementary figures were created using R v.4.0.3 (http://www.R-project.org)<sup>45</sup> and the package ggplot2<sup>46</sup>.

Received: 22 April 2021; Accepted: 23 September 2021

Published online: 12 October 2021

## References

1. Mahase, E. Covid-19: WHO declares pandemic because of “alarming levels” of spread, severity, and inaction. *BMJ* **368**, m1036 (2020).

2. Koralnik, I. J. & Tyler, K. L. COVID-19: A global threat to the nervous system. *Ann. Neurol.* **88**, 1–11 (2020).

3. Pezzini, A. & Padovani, A. Lifting the mask on neurological manifestations of COVID-19. *Nat. Rev. Neurol.* **16**, 636–644 (2020).

4. Eskandar, E. N. et al. Neurologic syndromes predict higher in-hospital mortality in COVID-19. *Neurology* **96**, e1527–e1538 (2021).

5. Ellul, M. A. et al. Neurological associations of COVID-19. *Lancet Neurol.* **19**, 767–783 (2020).

6. Moriguchi, T. et al. A first case of meningitis/encephalitis associated with SARS-Coronavirus-2. *Int. J. Infect. Dis.* **94**, 55–58 (2020).

7. Novi, G. et al. Acute disseminated encephalomyelitis after SARS-CoV-2 infection. *Neurol. Neuroimmunol. Neuroinflamm.* **7**, e797 (2020).

8. Dinkin, M. et al. COVID-19 presenting with ophthalmoparesis from cranial nerve palsy. *Neurology* **95**, 221–223 (2020).

9. Toscano, G. et al. Guillain-Barré syndrome associated with SARS-CoV-2. *N. Engl. J. Med.* **382**, 2574–2576 (2020).

10. Dalakas, M. C. Guillain-Barré syndrome: The first documented COVID-19-triggered autoimmune neurologic disease: More to come with myositis in the offing. *Neurol. Neuroimmunol. Neuroinflamm.* **7**, e781 (2020).

11. Fragiel, M. et al. Incidence, clinical, risk factors and outcomes of Guillain-Barré in Covid-19. *Ann. Neurol.* **89**, 598–603 (2021).

12. Paterson, R. W. et al. The emerging spectrum of COVID-19 neurology: Clinical, radiological and laboratory findings. *Brain* **143**, awaa240 (2020).

13. Spinato, G. et al. Alterations in smell or taste in mildly symptomatic outpatients with SARS-CoV-2 infection. *JAMA* **323**, 2089 (2020).

14. Xydakis, M. S. et al. Post-viral effects of COVID-19 in the olfactory system and their implications. *Lancet Neurol.* https://doi.org/10.1016/s1474-4422(21)00182-4 (2021).

15. Rogers, J. P. et al. Psychiatric and neuropsychiatric presentations associated with severe coronavirus infections: A systematic review and meta-analysis with comparison to the COVID-19 pandemic. *Lancet Psychiatry* **7**, 611–627 (2020).

16. Oxley, T. J. et al. Large-vessel stroke as a presenting feature of Covid-19 in the young. *N. Engl. J. Med.* **382**, e60 (2020).
17. Abdel-Mannan, O. et al. Neurologic and radiographic findings associated with COVID-19 infection in children. *JAMA Neurol.* https://doi.org/10.1001/jamaneurol.2020.2687 (2020).

18. Ray, S. T. J. et al. Neurological manifestations of SARS-CoV-2 infection in hospitalised children and adolescents in the UK: A prospective national cohort study. *Lancet Child Adolesc. Health* https://doi.org/10.1016/s2352-4642(21)00193-0 (2021).

19. Mao, L. et al. Neurologic manifestations of hospitalized patients with coronavirus disease 2019 in Wuhan, China. *JAMA Neurol.* 77, 683 (2020).

20. Varatharaj, A. et al. Neurological and neuropsychiatric complications of COVID-19 in 153 patients: A UK-wide surveillance study. *Lancet Psychiatry* 7, 875–882 (2020).

21. Drake, T. M. et al. Characterisation of in-hospital complications associated with COVID-19 using the ISARIC WHO Clinical Characterisation Protocol UK: A prospective, multicentre cohort study. *Lancet* 398, 223–237 (2021).

22. Benussi, A. et al. Clinical characteristics and outcomes of inpatients with neurologic disease and COVID-19 in Brescia, Lombardy, Italy. *Neurology* 95, e910–e920 (2020).

23. Helms, J. et al. Neurologic features in severe SARS-CoV-2 infection. *N. Engl. J. Med.* 382, 2268–2270 (2020).

24. Liotta, E. M. et al. Frequent neurologic manifestations and encephalopathy-associated morbidity in Covid-19 patients. *Ann. Clin. Transl. Neurol.* 324, 603 (2020).

25. Frontera, J. A. et al. A prospective study of neurologic disorders in hospitalized COVID-19 patients in New York City. *Neurology* https://doi.org/10.1212/WNL.0000000000010979 (2020).

26. Taquet, M., Geddes, J. R., Husain, M., Luciano, S. & Harrison, P. J. 6-month neurological and psychiatric outcomes in 236 379 survivors of COVID-19: A retrospective cohort study using electronic health records. *Lancet Psychiatry* 8, 416–427 (2021).

27. Chou, S.H.-Y. et al. Global incidence of neurological manifestations among patients hospitalized with COVID-19—A report for the GCS-NeuroCOVID consortium and the ENERGY consortium. *JAMA Netw. Open* 4, e2112131 (2021).

28. Brat, G. A. et al. International electronic health record-derived COVID-19 clinical course profiles: The 4CE consortium. *NPJ Digit. Med.* 3, 109 (2020).

29. Weber, G. M. et al. International comparisons of harmonized laboratory value trajectories to predict severe COVID-19: Leveraging the 4CE collaborative across 342 hospitals and 6 countries: A retrospective cohort study. *medRxiv* https://doi.org/10.1101/2020.12.16.20247684 (2020).

30. Klann, J. G. et al. Validation of an internationally derived patient severity phenotype to support COVID-19 analytics from electronic health record data. *J. Am. Med. Inform. Assoc.* 28, 1411–1420. https://doi.org/10.1093/jamia/ocab018 (2021).

31. Bourgeois, F. T. et al. International analysis of electronic health records of children and youth hospitalized with COVID-19 infection in 6 countries. *Jama Netw. Open* 4, e2112596 (2021).

32. Pun, B. T. et al. Prevalence and risk factors for delirium in critically ill patients with COVID-19 (COVID-D): A multicentre cohort study. *Lancet Respir. Med.* 9, 239–250 (2021).

33. Cosentino, G. et al. Neuropathological findings from COVID-19 patients with neurological symptoms argue against a direct brain invasion of SARS-CoV-2: A critical systematic review. *Eur. J. Neurol.* https://doi.org/10.1111/ene.15045 (2021).

34. Zubair, A. S. et al. Neuropathogenesis and neurologic manifestations of the coronaviruses in the age of coronavirus disease 2019: A review. *JAMA Neurol.* 77, 1018–1027 (2020).

35. Thakur, K. T. et al. COVID-19 neuropathology at Columbia University Irving Medical Center/New York Presbyterian Hospital. *Brain* https://doi.org/10.1093/brain/awab148 (2021).

36. Siegler, J. E. et al. Cerebrovascular events and outcomes in hospitalized patients with COVID-19: The SVIN COVID-19 Multinational Registry. *Int. J. Stroke* 52, 1747493020959216 (2020).

37. Levi, M., Thachil, J., Iba, T. & Levy, J. H. Coagulation abnormalities and thrombosis in patients with COVID-19. *Lancet. Haematol.* 7, e438–e440 (2020).

38. Guan, W.-J. et al. Clinical characteristics of coronavirus disease 2019 in China. *N. Engl. J. Med.* 382, 1708–1720 (2020).

39. Bagnato, S. et al. Critical illness myopathy after COVID-19. *Int. J. Infect. Dis.* 99, 276–278 (2020).

40. McNett, M. et al. The Global Consortium Study of Neurological Dysfunction in COVID-19 (GCS-NeuroCOVID): Development of case report forms for global use. *Neurocrit. Care* 33, 793–828 (2020).

41. Zhang, Y. et al. High-throughput phenotyping with electronic medical record data using a common semi-supervised approach (PheCAP). *Nat. Protoc.* 14, 3426–3444 (2019).

42. Wasserman, L. *All of Statistics: A Concise Course in Statistical Inference* (Springer, 2013).

43. Fisher, R. A. On the interpretation of $\chi^2$ from contingency tables, and the calculation of P. J. R. Stat. Soc. 85, 87–94 (1922).

44. Benjamini, Y., Drai, D., Elmer, G., Kafkafi, N. & Golani, I. Controlling the false discovery rate in behavior genetics research. *Behav. Brain Res.* 125, 279–284 (2001).

45. Team, R. C. R: *A Language and Environment for Statistical Computing* (R Foundation for Statistical Computing, 2021).

46. Wickham, H. *ggplot2: Elegant Graphics for Data Analysis* (Springer, 2009).

## Acknowledgements

We greatly appreciate Margaret Vella in her coordination of the 4CE consortium effort and assistance in the manuscript submission.

## Author contributions

T.L., A.G.S., J.S., G.O., P.A., S.V., D.M., and Z.X. contributed to concept and design of the study. B.B., N.L., Y.L., M.M., K.N., L.P., M.S., E.S., A.T., I.K., S.V., D.M., and Z.X. contributed to data collection. T.L., A.G.S., J.S., C.H., K.N., T.C., P.A., S.V., D.M., and Z.X. contributed to data analysis. T.L., A.G.S., J.S., C.H., A.S., B.B., N.L., Y.L., J.M., L.P., T.C., G.O., P.A., S.V., D.M., and Z.X. contributed to drafting and editing the manuscript. All named authors have approved the final manuscript.

## Funding

AS is funded by National Institutes of Health (NIH) National Heart Lung, and Blood Institute (NHLBI) K23HL148394 and L40HL148910, and NIH-National Center for Advancing Translational Sciences (NCATS) UL1TR001420. JM is funded by NIH-National Institute of Allergy and Infectious Disease (NIAD) AI11679. LP is funded by NCATS Clinical and Translational Science Award (CTSA) Number UL1TR002366. GO is funded by NIH National Institute of Environmental Health Sciences (NIEHS) P30ES017885 and National Cancer Institute (NCI) U24CA210967. SV is funded by NIH-National Library of Medicine (NLM) R01LM012095 and NCATS UL1TR001857. DM is funded by NCATS CTSA Number UL1-TR001878. ZX is funded by NIH National Institute of Neurological Disorders and Stroke (NINDS) R01NS098023.
Competing interests

The authors declare no competing interests.

Additional information

**Supplementary Information** The online version contains supplementary material available at https://doi.org/
10.1038/s41598-021-99481-9.

**Correspondence** and requests for materials should be addressed to Z.X.

Reprints and permissions information is available at www.nature.com/reprints.

**Publisher's note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_12.png" /></figure>

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International
License, which permits use, sharing, adaptation, distribution and reproduction in any medium or
format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the
Creative Commons licence, and indicate if changes were made. The images or other third party material in this
article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the
material. If material is not included in the article's Creative Commons licence and your intended use is not
permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from
the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

© The Author(s) 2021

The Consortium for Clinical Characterization of COVID-19 by EHR (4CE)

James R. Aaron¹², Giuseppe Agapito¹³, Adem Albayrak¹⁴, Mario Alessiani¹⁵,
Danilo F. Amendola¹⁶, François Angoulvant¹⁷, Li L. L. J. Anthony¹⁸, Bruce J. Aronow¹⁹,
Andrew Atz²⁰, James Balshi²¹, Douglas S. Bell²², Antonio Bellasi²³, Riccardo Bellazzi²⁴,
Vincent Benoit²⁵, Michele Beraghi²⁶, José Luis Bernal Sobrino²⁷, Mélodie Bernaux²⁸,
Romain Bey²⁵, Alvar Blanco Martínez²⁷, Martin Boeker²⁹, Clara-Lea Bonzel², John Booth³⁰,
Silvano Bosari³¹, Florence T. Bourgeois², Robert L. Bradford³², Gabriel A. Brat²,
Stéphane Bréant²⁵, Nicholas W. Brown², William A. Bryant³⁰, Mauro Bucalo³³,
Anita Burgun³⁴, Mario Cannataro³⁵, Aldo Carmona³⁶, Charlotte Caucheteux³⁷,
Julien Champ³⁸, Krista Chen³⁹, Jin Chen⁴⁰, Luca Chiovato⁴¹,⁴², Lorenzo Chiudinelli²³,
James J. Cimino⁴³, Tiago K. Colicchio⁴³, Sylvie Cormont²⁵, Sébastien Cossin⁴⁴, Jean B. Craig⁴⁵,
Juan Luis Cruz Bermúdez²⁷, Jaime Cruz Rojo²⁷, Arianna Dagliati²⁴, Mohamad Daniar⁴⁶,
Christel Daniel²⁵, Anahita Davoudi¹, Batsal Devkota⁴⁷, Julien Dubiel²⁵, Loic Esteve⁴⁸,
Shirley Fan⁴⁹, Robert W. Follett²², Paula S. A. Gaiolla⁵⁰, Thomas Ganslandt⁵¹,
Noelia García Barrio²⁷, Lana X. Garmire¹¹, Nils Gehlenborg², Alon Geva³⁹,⁵²,
Tobias Gradinger⁵¹, Alexandre Gramfort³⁷, Romain Griffier⁴⁴, Nicolas Griffon²⁵,
Olivier Grisel³⁷, David A. Hanauer⁵³, Christian Haverkamp²⁹, Bing He¹¹,
Darren W. Henderson¹², Martin Hilka²⁵, John H. Holmes¹, Petar Horki²⁹,
Kenneth M. Huling², Meghan R. Hutch⁶, Richard W. Issitt³⁰, Anne Sophie Jannot³⁴,
Vianney Jouhet⁴⁴, Ramakanth Kavuluru¹², Mark S. Keller², Katie Kirchoff⁵⁴,
Jeffrey G. Klann⁵⁵, Ian D. Krantz¹,⁵⁶, Detlef Kraska⁵⁷, Ashok K. Krishnamurthy⁵⁸, Sehi L'Yi²,
Judith Leblanc⁵⁹, Andressa R. R. Leite¹⁶, Guillaume Lemaitre³⁷, Leslie Lenert⁴⁵,
Damien Leprovost⁶⁰, Molei Liu⁶¹, Sarah Lozano-Zahonero⁹, Kristine E. Lynch⁶²,
Sadiqa Mahmood¹⁴, Sarah Maidlow⁶³, Adeline C. Makoudjou Tchendjou²⁹,
Alberto Malovini⁶⁴, Kenneth D. Mandl³⁹, Chengsheng Mao⁶, Anupama Maram⁶⁵,
Patricia Martel⁶⁶, Aaron J. Masino⁶⁷, Michael E. Matheny⁶⁸, Thomas Maulhardt²⁹,
Maria Mazzitelli³⁵, Michael T. McDuffie², Arthur Mensch⁶⁹, Fatima Ashraf⁷⁰,
Marianna Milano³⁵, Marcos F. Minicucci⁵⁰, Bertrand Moal⁴⁴, Cinta Moraleda⁷¹,
Jeffrey S. Morris¹, Karyn L. Moshal⁷², Sajad Mousavi², Douglas A. Murad²²,
Shawn N. Murphy⁷³, Thomas P. Naughton⁶⁵, Antoine Neuraz³⁴, James B. Norman²,
Jihad Obeid⁴⁵, Marina P. Okoshi⁷⁴, Karen L. Olson³⁹,⁷⁵, Nina Orlova²⁵, Brian D. Ostasiewski⁷⁶,
Nathan P. Palmer², Nicolas Paris²⁵, Miguel Pedrera Jimenez²⁷, Emily R. Pfaff³²,
Danielle Pillion², Hans U. Prokosch⁷⁷, Robson A. Prudente¹⁶, Víctor Quirós González²⁷,
Rachel B. Ramoni⁷⁸, Maryna Raskin¹⁴, Siegbert Rieg²⁹, Gustavo Roig Domínguez²⁷,
Pablo Rojo²⁷, Carlos Sáez⁷⁹, Elisa Salamanca²⁵, Arnaud Sandrin²⁵, Janaina C. C. Santos¹⁶,⁸⁰,
Maria Savino<sup>81</sup>, Juergen Schuettler<sup>82</sup>, Lucia Scudeller<sup>31</sup>, Neil J. Sebre<sup>30</sup>,
Pablo Serrano Balazote<sup>27</sup>, Patricia Serre<sup>25</sup>, Arnaud Serret-Larmande<sup>2</sup>, Zahra Shakeri<sup>2</sup>,
Domenick Silvio<sup>63</sup>, Piotr Sliz<sup>39</sup>, Charles Sonday<sup>83</sup>, Anastasia Spiridou<sup>30</sup>, Bryce W. Q. Tan<sup>84</sup>,
Byorn W. L. Tan<sup>84</sup>, Suzana E. Tanni<sup>74</sup>, Deanne M. Taylor<sup>1,47</sup>, Ana I. Terriza-Torres<sup>27</sup>,
Valentina Tibollo<sup>64</sup>, Patric Tippmann<sup>29</sup>, Carlo Torti<sup>35</sup>, Enrico M. Trecarichi<sup>35</sup>, Yi-Ju Tseng<sup>85</sup>,
Andrew K. Vallejos<sup>86</sup>, Gael Varoquaux<sup>37,87</sup>, Margaret Vella<sup>2</sup>, Jill-Jenn Vie<sup>88</sup>, Michele Vitacca<sup>89</sup>,
Kavishwar B. Wagholikar<sup>55</sup>, Lemuel R. Waitman<sup>90</sup>, Demian Wassermann<sup>37</sup>, Griffin M. Weber<sup>2</sup>,
Yuan William<sup>2</sup>, Nadir Yehya<sup>67</sup>, Alberto Zambelli<sup>91</sup>, Harrison G. Zhang<sup>2</sup>, Daniela Zoeller<sup>29</sup> &
Chiara Zucco<sup>35</sup>

<sup>12</sup>Department of Biomedical Informatics, University of Kentucky, Lexington, KY, USA. <sup>13</sup>Department of Legal, Economic and Social Sciences, University Magna Graecia of Catanzaro, Catanzaro, Italy. <sup>14</sup>Health Catalyst, INC., Cambridge, MA, USA. <sup>15</sup>Department of Surgery, ASST Pavia, Lombardia Region Health System, Pavia, Italy. <sup>16</sup>Clinical Research Unit of Botucatu Medical School, São Paulo State University, Botucatu, Brazil. <sup>17</sup>Pediatric Emergency Department, Hôpital Necker-Enfants Malades, Assistance Publique-Hôpitaux de Paris, Paris, France. <sup>18</sup>National Center for Infectious Diseases, Tan Tock Seng Hospital, Singapore, Singapore. <sup>19</sup>Departments of Biomedical Informatics, Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, OH, USA. <sup>20</sup>Department of Pediatrics, Medical University of South Carolina, Charleston, SC, USA. <sup>21</sup>Department of Surgery, St. Luke's University Health Network, Bethlehem, PA, USA. <sup>22</sup>Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, CA, USA. <sup>23</sup>UOC Ricerca, Innovazione e Brand Reputation, ASST Papa Giovanni XXIII, Bergamo, Italy. <sup>24</sup>Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy. <sup>25</sup>IT Department, Innovation & Data, APHP Greater Paris University Hospital, Paris, France. <sup>26</sup>I.T. Department, ASST Pavia, Pavia, Italy. <sup>27</sup>Health Informatics, Hospital Universitario, 12 de Octubre, Madrid, Spain. <sup>28</sup>Strategy and Transformation Department, APHP Greater Paris University Hospital, Paris, France. <sup>29</sup>Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany. <sup>30</sup>Digital Research, Informatics and Virtual Environments (DRIVE), Great Ormond Street Hospital for Children, London, UK. <sup>31</sup>Scientific Direction, IRCCS Ca' Granda Ospedale Maggiore Policlinico di Milano, Milan, Italy. <sup>32</sup>North Carolina Translational and Clinical Sciences (NC TraCS) Institute, University of North Carolina, Chapel Hill, NC, USA. <sup>33</sup>BIOMERIS (BIOMedical Research Informatics Solutions), Pavia, Italy. <sup>34</sup>Department of Biomedical Informatics, HEGP, APHP Greater Paris University Hospital, Paris, France. <sup>35</sup>Department of Medical and Surgical Sciences, Data Analytics Research Center, University Magna Graecia of Catanzaro, Catanzaro, Italy. <sup>36</sup>Department of Anesthesia, St. Luke's University Health Network, Bethlehem, PA, USA. <sup>37</sup>Université Paris-Saclay, Inria, CEA, Palaiseau, France. <sup>38</sup>INRIA Sophia-Antipolis-ZENITH Team, LIRMM, Montpellier, France. <sup>39</sup>Computational Health Informatics Program, Boston Children's Hospital, Boston, MA, USA. <sup>40</sup>Department of Internal Medicine, University of Kentucky, Lexington, KY, USA. <sup>41</sup>Unit of Internal Medicine and Endocrinology, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy. <sup>42</sup>Department of Internal Medicine and Therapeutics, University of Pavia, Pavia, Italy. <sup>43</sup>Informatics Institute, University of Alabama at Birmingham, Birmingham, AL, USA. <sup>44</sup>IAM Unit, Bordeaux University Hospital/ERIAS-Inserm U1219 BPH, Bordeaux, France. <sup>45</sup>Biomedical Informatics Center, Medical University of South Carolina, Charleston, SC, USA. <sup>46</sup>Clinical Research Informatics, Boston Children's Hospital, Boston, MA, USA. <sup>47</sup>Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, PA, USA. <sup>48</sup>SED/SIERRA, Inria Centre de Paris, Paris, France. <sup>49</sup>Health Information Technology & Services, University of Michigan, Ann Arbor, MI, USA. <sup>50</sup>Internal Medicine Department, Botucatu Medical School, São Paulo State University, Botucatu, Brazil. <sup>51</sup>Heinrich-Lanz-Center for Digital Health, University Medicine Mannheim, Heidelberg University, Mannheim, Germany. <sup>52</sup>Department of Anesthesiology, Critical Care, and Pain Medicine Boston Children's Hospital, Boston, MA, USA. <sup>53</sup>Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, MI, USA. <sup>54</sup>MSHI Medical University of South Carolina, Charleston, SC, USA. <sup>55</sup>Department of Medicine, Massachusetts General Hospital, Boston, MA, USA. <sup>56</sup>Division of Human Genetics, Department of Pediatrics, The Children's Hospital of Philadelphia, Philadelphia, PA, USA. <sup>57</sup>Center for Medical Information and Communication Technology, University Hospital, Erlangen, Germany. <sup>58</sup>Renaissance Computing Institute/Department of Computer Science, University of North Carolina, Chapel Hill, NC, USA. <sup>59</sup>Clinical Research Unit, Saint Antoine Hospital, APHP Greater Paris University Hospital, Paris, France. <sup>60</sup>Clevy.io, Paris, France. <sup>61</sup>Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA. <sup>62</sup>VA Informatics and Computing Infrastructure, VA Salt Lake City Health Care System, Salt Lake City, USA. <sup>63</sup>MICHR Informatics, University of Michigan, Ann Arbor, MI, USA. <sup>64</sup>Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy. <sup>65</sup>Harvard Catalyst, Harvard Medical School, Boston, MA, USA. <sup>66</sup>Clinical Research Unit, Paris Saclay, APHP Greater Paris University Hospital, Paris, France. <sup>67</sup>Department of Anesthesiology and Critical Care, Children's Hospital of Philadelphia, Philadelphia, PA, USA. <sup>68</sup>VA Informatics and Computing Infrastructure, Tennessee Valley Healthcare System, Veterans Affairs Medical Center, Washington, D.C., USA. <sup>69</sup>École Normale Supérieure, PSL Université Paris, Paris, France. <sup>70</sup>BIG-ARC, The University of Texas Health Science Center at Houston, School of Biomedical Informatics, Houston, TX, USA. <sup>71</sup>Pediatric Infectious Disease Department, Hospital Universitario, 12 de Octubre, Madrid, Spain. <sup>72</sup>Department of Infectious Diseases, Great Ormond Street Hospital for Children, London, UK. <sup>73</sup>Department of Neurology, Massachusetts General Hospital, Boston, MA, USA. <sup>74</sup>Internal Medicine Department of Botucatu Medical School, São Paulo State University, Botucatu, Brazil. <sup>75</sup>Department of Pediatrics, Boston Children's Hospital, Boston, MA, USA. <sup>76</sup>Center for Biomedical Informatics, Wake Forest School of Medicine, Winston-Salem, NC, USA. <sup>77</sup>Department of Medical Informatics, University of Erlangen-Nürnberg, Erlangen, Germany. <sup>78</sup>Department of Veterans Affairs, Office of Research and Development, Baltimore, MD, USA. <sup>79</sup>Biomedical Data Science Lab, ITACA Institute, Universitat
Politècnica de València, València, Spain. ⁸⁰Nurse Department of FMB-Medicine School of Botucatu, Botucatu, Brazil. ⁸¹Management Engineering, ASST Pavia, Lombardia Region Health System, Pavia, Italy. ⁸²Department of Anesthesiology, University Hospital Erlangen, FAU Erlangen-Nürnberg, Erlangen, Germany. ⁸³Critical Care Medicine, Department of Medicine, St. Luke's University Health Network, Bethlehem, PA, USA. ⁸⁴Department of Medicine, National University Hospital, Singapore, Singapore. ⁸⁵Department of Information Management, Chang Gung University, Taoyuan, Taiwan. ⁸⁶Clinical & Translational Science Institute, Medical College of Wisconsin, Milwaukee, WI, USA. ⁸⁷Montréal Neurological Institute, McGill University, Montreal, Canada. ⁸⁸SequeL, Inria Lille, Villeneuve-d'Ascq, France. ⁸⁹Respiratory Department, ICS S Maugeri IRCCS, Pavia, Italy. ⁹⁰Department of Health Management and Informatics, University of Missouri, Columbia, MO, USA. ⁹¹Department of Oncology, ASST Papa Giovanni XXIII, Bergamo, Italy.