dataesr/scientific-paragraphs-categorization huggingface dataset:
NER model comparison, showing F1score on the test sets after 50 epochs averaged over 5 runs. This result is for all 4 tags in the dataset: PER, ORG, LOC, DATE. Bold marks the top score (tied if within the range of SE). mBERT and XLMR are trained in two ways (1) MeanE: mean output embeddings of the 12 LM layers are used to initialize BiLSTM + Linear classifier, and (2) FTune: LM finetuned end toend with a linear classifier. Lang. BERT & Lang XLMR (base) are models finetuned after language adaptive finetuning.
Transfer Learning Result (i. e. F1score). 3 Tags: PER, ORG & LOC. WikiAnn, engCoNLL, and the annotated datasets are trained for 50 epochs. Finetuning is only for 10 epochs. Results are averaged over 5 runs and the total average (avg) is computed over ibo, kin, lug, luo, wol, and yor languages. The overall highest F1score is in bold, and the best F1score in zeroshot settings is indicated with an asterisk (*).
Acknowledgements We would like to thank  Heng Ji  and  Ying Lin  for providing the ELISA NER tool used for annota tion. We also thank the  Spoken Language Systems Chair, Dietrich Klakow at Saarland University  for providing GPU resources to train the models. We thank  Adhi Kuncoro  and the anonymous review ers for their useful feedback on a draft of this pa per. David Adelani  acknowledges the support of the EUfunded  H2020  project  COMPRISE  under grant agreement No. 3081705. Finally, we thank  Mohamed Ahmed  for proofreading the draft. based  Framework for Multitask Crosslingual Transfer. In Proceedings of EMNLP 2020. Jonas Pfeiffer ,  Ivan Vulić ,  Iryna Gurevych , and  Sebastian Ruder. 2020b. Unks everywhere: Adapting multilingual language models to new scripts. arXiv preprint arXiv: 2012.
B-cell receptor (BCR) signaling is crucial for the pathophysiology of most mature B-cell lymphomas/leukemias and has emerged as a therapeutic target whose effectiveness remains limited by the occurrence of mutations. Therefore, deciphering the cellular program activated downstream this pathway has become of paramount importance for the development of innovative therapies. Using an original ex vivo model of BCR-induced proliferation of chronic lymphocytic leukemia cells, we generated 108 temporal transcriptional and proteomic profiles from 1 h up to 4 days after BCR activation. This dataset revealed a structured temporal response composed of 13,065 transcripts and 4027 proteins, comprising a leukemic proliferative signature consisting of 430 genes and 374 proteins. Mathematical modeling of this complex cellular response further highlighted a transcriptional network driven by 14 early genes linked to proteins involved in cell proliferation.
Moreover, it is worth noting that the experimental conditions using isolated BCR activation in these previous studies led to CLL lymphocyte apoptosis instead of proliferation  [11] , which therefore missed the malignant proliferative output of BCR engagement observed in patients. Recently, we designed a novel ex vivo experimental setting in which BCR engagement coupled to minimal mandatory costimulating agents (CD40L, IL-4 and IL-21) recapitulate the proliferation of primary CLL cells  [12]. In the present study, we used this improved ex vivo culture model to generate a unique set of 108 combined transcriptional and proteomic profiles over time after activation of human primary CLL cells. As existing analysis methods of such high dimensional datasets are limited in terms of precision to select the relevant actors of the genetic program, we have developed a mathematical approach, validated on synthetic datasets and supported by extensive simulations, allowing the selection of critical actors of the CLL's cell response  [13].
Furthermore, in order to characterize the underlying structure of the concerted temporal interactions between these actors, we refined our previously developed regression method based on linear equations  [14]  which has proved its capacity to handle high dimensional dataset while taking into account the inherent sparsity of biological processes  [15, 16]. By using this mathematical modeling approach applied to the 108 omics points of measurement, we characterized the temporal cellular response of CLL cells to BCR activation and we identified within this response a nested and structured core proliferative program that could sustain CLL cell leukemogenesis.
Transcriptomic analysis Before BCR engagement (T0) and at eight time points after activation (1 h, 1 h 30 min, 3 h 30 min, 6 h 30 min, 12 h, 24 h, 48 h, 96 h), 4. 10 6 cells were resuspended in 1 mL TRIzol (Sigma-Aldrich, Saint-Louis, MO, USA). Total RNA was purified using the RNeasy Mini kit (Qiagen, Hilden, Germany). After ribosomal RNA depletion, the sequencing library was prepared with the Ion Total RNA-seq kit v2 (Thermo Scientific) and the sequencing was performed on an Ion Proton sequencer with the Ion PI Hi-Q Sequencing 200 Kit (Thermo Scientific). Reads was estimated with the package HTSeq  [17]  and the edgeR package  [18]  was used to derive the reads per kilobase per million values. The transcriptomic dataset is available in GEO (GSE130385).
Proteomic analysis Before BCR stimulation (T0) and at eight time points after stimulation (1 h, 2 h, 4 h, 7 h, 12 h, 24 h, 48 h, 96 h), 8. 10 6 cells were resuspended in lysis buffer. Proteins were acetone precipitated and 10 µg of each sample were concentrated in a stacking gel band, in-gel reduced, alkylated, and trypsin digested. NanoLC-MS/MS analyses of extracted trypsic peptides were performed on a nanoAcquity UPLC device (Waters Corporation) coupled to a Q-Exactive Plus mass spectrometer (Thermo Scientific) operated in data dependent acquisition mode. Label-free extracted ion chromatogram-based quantification was performed using MaxQuant software (version 1. 5. 5. 1)  [19]. The proteomics dataset was deposited to the ProteomeXchange Consortium via the PRIDE partner repository (PXD013573).
Gene expression and protein abundance analysis Quality-based filtering of low expressed genes was performed with the HTSFilter package  [20]. The selection of differentially expressed (DE) genes was made with the glmLRT and the glmTreat functions of the edgeR package  [18]. Identification of temporal clusters of gene expression was performed with the HTSCluster package  [21]. After quantile normalization, differential analysis of protein abundancies was made using the peptide-level robust ridge regression implemented in the MSqRob package  [22].
Experimental design and multiomic dataset To investigate the proliferative response of primary human CLL cells after ex vivo BCR engagement, BCR stimulation was performed on six untreated CLL samples of the most aggressive subgroup (IGHV unmutated), including three samples that proliferate and three samples that do not proliferate ex vivo within our culture conditions, the latter being used as controls (Table  1 ). Transcriptional (RNA-seq) and proteomic (mass spectrometry) responses of these CLL samples were determined at T0 (before stimulation) and at eight time points between 1 and 96 h after BCR-mediated cell activation, generating a total of 108 points of measurement (Fig. S1 ). A total of 23,348 transcripts and 4664 unique proteins were identified and quantified in the whole dataset.
BCR response program inference in proliferative CLL cells To model the cellular response displayed in the multiomic dataset, we used a mathematical unsupervised reverse engineering approach based on regression and system of  equations that already proved its efficiency in our previous transcriptional study  [15]. Temporal matrix of interactions between genes and proteins (Fig. S4A-C ) was estimated by mean of penalized regression using a weighted variant of stability selection algorithm  [25]  in order to retain the best potential regulators of each gene or protein in the network and to determine the timing of these interactions after cell stimulation. To enhance the reverse engineering relevance  [26] , we imposed biological constraints by favoring links based on known transcriptional or protein-protein relationships implemented in the RegNetwork database  [27].
G _ T 1 G _ T 2 G _ T 3 G _ T 4 G _ T 5 G _ T 6 G _ T 7 G _ T 8 The robustness of this inference has been ascertained by cross validation  [28]  and the best result was retained by linear regression estimation. Performances of the resulting model, including sensitivity, precision, predictive positive value, and F-score, were validated with simulated data and compared to the performance of other algorithms (Fig. S4D-F ). Inferring the formalized model with the temporal dataset of proliferating CLL cells identified a regulatory network of 2167 genes and 1074 proteins representing 2846 unique symbols (Table  S2 ), among which 395 gene-protein pairs, connected by 53,131 oriented links (Fig. S5 ). This network exhibited a scale-free topology, where a limited number of hub genes and proteins with multiple links (12 genes and 52 proteins exhibit ≥ 10 statistical links) drove the structure of this cellular program.
Deciphering a CLL proliferative program within the BCR response Considering the ability of CLL cells to generate a proliferative response after BCR activation, we investigated whether a subnetwork sustaining cell proliferation can be identified within the above response network. To address this, we identified within the proteomic dataset of proliferating CLL cells the 267 proteins associated with the BP terms "cell cycle regulation" and "proliferation," designed as "seeding proteins," and their 243 connected neighbors (gene or protein) in the network (Fig. S6A ). Analyzing the level of interconnection within this subgroup of proteins and genes revealed a nested subnetwork comprising 388 elements including 173 of the seeding proteins linked to 215 neighbors (Fig. S6B , Table  S2 ). Of note, among these 388 genes and proteins elements, 31% belonged to the "proliferative signature" defined above in the supervised analysis (see also Fig.
Important actors of the proliferative subnetwork are missing in nonproliferative cells We next investigated if this proliferative subnetwork composed of TFs and hub proteins is specific to the CLL proliferative program. For this purpose, we inferred with the same approach the formalized model with the temporal datasets of nonproliferative samples. This led to the identification of a regulatory network of 1399 genes and 750 proteins representing 1933 unique symbols, which were connected by 17,332 oriented links (Table  S2 ). Functional analysis (BP terms) revealed 193 proteins associated with "cell cycle regulation" or "proliferation" process. Among them, 89% were shared with the seeding proteins displayed in the proliferative samples and corresponds to the core response program. The search for neighbors of these 193 proteins revealed 131 elements with a nested subnetwork comprising 114 actors (37 genes and 77 proteins) (Fig.
However, comparison of the 388 actors (genes or proteins) of the proliferative subnetwork (Fig. 6 ) and the 114 actors of the nonproliferating subnetwork (Fig. S7 ) shows 60 common actors (representing 52% of the total nonproliferating actors and 15% of the proliferating subnetwork actors), constituting the core of the common response of this group of lymphocytes of the aggressive form of CLL. In conclusion, using a large dataset of temporal transcriptional and proteomic measurements coupled with mathematical modeling, this study unveils the genetic program downstream the signaling cascade activated by the BCR engagement and triggering primary CLL cell proliferation ex vivo. Noteworthy, this program organizes around a limited number of genes and proteins whose sequential commitment drives the cellular response leading to proliferation days after cell activation.
Acknowledgements The authors are grateful to  Antoine Hanauer  ( Inserm UMR S-1109 ) and  Nathalie Perrusson  ( Inserm UMR S-1113 ) for technical help. The authors thank  Dr. Anne-Cécile Galoisy  and  Dr. Caroline Mayeur-Rousse  ( Laboratoire d'Hématologie, Hôpitaux Universitaires de Strasbourg, France ) for CLL cell characterization,  Beatrice Uring-Lambert  ( Laboratoire d'Immunologie, Hôpitaux Universitaires de Strasbourg, France ) for help with flow cytometry,  Elise Toussaint  and  Blandine Guffroy  ( Service d'Hématologie Adulte, Hôpitaux Universitaires de Strasbourg, France ) for providing CLL samples, and  Jozo Delic  ( Commissariat à l'Energie Atomique, Fontenay-aux-roses, France ) for helpful suggestions.
Data availability RNA-seq data: GEO accession GSE130385. Proteomic data: ProteomeXchange Consortium accession PXD013696. Packages availability: The Cascade package is available at https://CRAN. R-project. org/package=Cascade and https://fbertran. github. io/Cascade/. The SelectBoost package is available at https://fbertran. github. io/ SelectBoost/. The Patterns package is available at https:// fbertran. github. io/Patterns/.
Catalonia was heavily hit by the first phase (March-May 2020) of the COVID-19 pandemic (10). This region has a universal taxpayer-funded primary care-based health system in which general practitioners have been the first point of contact for care throughout the pandemic. Electronic health records from primary care encompassing demographic, historical lifestyle information, and disease diagnoses linked to SARS-CoV-2 reverse transcription polymerase chain reaction (RT-PCR) test results, hospital records, and regional mortality data offer a unique opportunity to study the role of BMI in the course of COVID-19. We aimed to investigate the associations between BMI and risks of COVID-19 diagnosis, hospitalization with COVID-19, and death after a COVID-19 diagnosis or hospitalization (ie, subsequent death), accounting for potential effect modification by age and sex, using electronic health record data from Catalonia.
Study Design, Setting, and Data Sources We conducted a cohort study from March 1 to May 6, 2020. We used prospectively collected primary care records from the Information System for Research in Primary Care (SIDIAP; www. sidiap. org) in Catalonia, Spain. SIDIAP includes data from the Institut Català de la Salut (Catalan Health Institute), the largest public primary healthcare provider of Catalonia (covering 5. 8 million people, 80% of the population of Catalonia) since 2006 and is representative of the Catalan population in terms of age, sex, and geographic distribution  (11). SIDIAP includes high-quality data on anthropometric measurements, disease diagnoses, laboratory tests, and demographic and lifestyle information. SIDIAP has been linked to COVID-19 RT-PCR test results, hospital records, and regional mortality data and mapped to the Observational Medical Outcomes Partnership Common Data Model  (12).
We selected these conditions based on their relevance to the obesity and COVID-19 research fields and their availability in the Observational Medical Outcomes Partnership Common Data Model mapped version of the SIDIAP database and were defined as in previous studies conducted using SIDIAP data  (20) (21) (22). The definitions are available in a Web application [https://livedataoxford. shinyapps. io/ MultiStateCovidCohorts/ (index event breakdown tab)]. The outcomes of interest were an outpatient (primary care) clinical diagnosis of COVID-19, a hospitalization with COVID-19, and death. We defined outpatient COVID-19 diagnoses based on a recorded clinical code for COVID-19 disease (International Classification of Diseases, Tenth Revision, Clinical Modification codes B34. 2 "Coronavirus infection, unspecified" and B97.
We replicated the main analyses, first, including only BMI values recorded in the previous 5 years (March 1, 2015-February 29, 2020) and, second, including only BMI values recorded in the previous 2 years (March 1, 2018-February 29, 2020). We used R version 3. 6 for data analysis and visualization. The R packages used for the analyses included numerous tidyverse packages, mstate, survival and rms  (29) (30) (31) (32). The analytic code we used is available at https://github. com/SIDIAP/MultiStateBmiCovid-19. This study was approved by the Clinical Research Ethics Committee of the Fundació Institut Universitari per a la recerca a l'Atenció Primària de Salut Jordi Gol i Gurina (project code: 20/070-PCV).
Results There were 4 765 757 adults from the SIDIAP population registered in the database on March 1, 2020 who were eligible to enter the study. We excluded 104 022 individuals with less than a year of prior clinical history; 306 with a prior COVID-19 clinical diagnosis or positive test; 41 588 who were hospitalized or living in a nursing home on March 1; 1 357 553 with unavailable BMI measurement; and 737 362 who had missing data on smoking status and/ or MEDEA deprivation index [Supplementary Figure  1   (17) ]. We included 2 524 926 participants, of which 45 382 were living with underweight (2%), 905 898 with normal weight (36%), 952 479 (38%) with overweight, and 621 167 (24%) with obesity (Table  1 ). The participants' median BMI [interquartile (IQR) range] was 26  (24) (25) (26) (27) (28) (29) (30)  kg/m 2 , and age was 52 (39-67) years.
Large observational studies from the United States, the United Kingdom, and Spain using multiple categories of BMI only found an association between morbid obesity (BMIs > 35 or 40 kg/m 2 ) and COVID-19 mortality  (39, (43) (44) (45) (46). Our results for high BMIs are consistent with the latter studies and revealed BMI was associated in a J-shaped fashion with the risk of subsequent death. Only BMIs above 37 kg/ m 2 and 40 kg/m 2 were linked with a higher risk of death after a COVID-19 hospitalization and after a COVID-19 outpatient diagnosis, respectively. The J-shaped association between BMI and risk of COVID-19 related death has also been reported in a study conducted in a New York hospital and England using a large primary care database  (7, 47). Interestingly, we observed a lower risk of death for individuals living with overweight among people diagnosed with COVID-19 in outpatient settings.
Acknowledgments We would like to acknowledge the patients who suffered from or died of this devastating disease and their families and carers. We would also like to thank the healthcare professionals involved in the management of COVID-19 during these challenging times, from primary care to intensive care units in the Catalan healthcare system. Authors' Contributions: S. F. B. , M. A. , and T. D. S. mapped source data to the Observational Medical Outcomes Partnership Common Data Model. M. R. , A. P. , E. B. , and T. D. S. led the data analysis. M. R. and H. F. performed a literature review. All authors were involved in the study conception and design, interpretation of the results, and the preparation of the manuscript. Funding: This project is funded by the  Health Department  from the  Generalitat de Catalunya  with a grant for research projects on SARS-CoV-2 and COVID-19 disease organized by the  Direcció General de Recerca i Innovació en Salut.
This project has received support from the  European Health Data and Evidence Network (EHDEN)  project. EHDEN received funding from the  Innovative Medicines Initiative 2 Joint Undertaking (JU)  under grant agreement no. 806968. The JU receives support from the  European Union 's  Horizon 2020 research and innovation programme and EFPIA. The  University of Oxford  received a grant related to this work from the  Bill & Melinda Gates Foundation  ( Investment ID INV-016201 ) and partial support from the  UK National Institute for Health Research (NIHR) Oxford Biomedical Research Centre. D. P-A. is funded through a  National Institute for Health Research (NIHR)   Senior Research Fellowship  (grant number  SRF-2018-11-ST2-004 ). MR is funded by  Wereld Kanker Onderzoek Fonds  (WKOF), as part of the  World Cancer Research Fund   International grant program  (grant number:  2017/1630 ).
Canarin Nano The hardware of the Canarin II runs on Linux, and therefore, provides a good amount of processing power and software development possibilities. The main board consumes approximately 3 watts, requiring a battery able to sustain a normal day of use. The size and weight of the device (around 1. 2 kg) is not ideal for individuals who have to carry the device all day long, especially if they are affected by respiratory diseases, as was the case in the POLLUSCOPE project. Such drawbacks had been previously outlined during the performed benchmarking  [21] , and volunteers of the POLLUSCOPE project commented on these aspects. Technological advancements in microcontrollers, PM and gas sensing technologies have made size reduction possible without compromising the quality of the acquired data.
This secondary PCB header board opens the possibility to upgrade to newer electrochemical or metal-oxide gas sensors, without needing to remanufacture the entire device. Flash memory has been used to prevent data loss if the on-board 3G modem loses connectivity with cell towers. A third party IoT (Internet of things) or M2M (Machine to machine) SIM (subscriber identity module) card provider has been selected instead of the default Particle cloud services  [47]. Table  1  provides some of the components used in the Canarin Nano. The cloud infrastructure server is hosted by Amazon Web Services (AWS) through an EC2 T2 instance  [48]. One vCPU (virtual central processing unit) and 1 GB of RAM (random access memory) are enough to provide service for approximately 100 Canarin Nanos. Multiple docker containers  [49]  run Python scripts to ingest data.
The database is also hosted by AWS in Frankfurt, following the GDPR (General Data Protection Regulation). The front end  [50]  allows users to download or visualize data in real time. Once the first prototypes provided satisfying performance in a real-world scenario, a redesign of the device occurred, to switch from locally soldered and printed materials, to a custom PCB and a fused deposition modeling (FDM) 3D printed enclosure, produced externally in larger quantities. Complexity to source the bill of materials, logistics and delays increased the overall cost per unit. Any possibility to streamline the production chain was abandoned due to lack of manpower. Redesigning the entire device enclosure and PCB successfully eased the assembly and maintenance of the sensor, while greatly improving reliability.
For the particulate matter, a Thermo Scientific TM 1405-F tapered element oscillating microbalance (TEOM), for PM 10 , and a ThermoFisher TEOM 1400 were used  [21]. If sensors displayed unexpected behaviors, they were set aside for servicing. The second goal of the project was to assess individual exposure to air pollution as a function of mobility. Three population groups were considered: volunteers from Versailles Grand Parc (VGP), participants of the population-based survey RECORD [51] and volunteers affected by respiratory diseases. VGP volunteers had to fill an online survey, and provide written consent for their data to be used. Volunteers were required to wear POLLUSCOPE's sensors, including a Canarin II (Figure  2 ), for one week. The volunteers were asked to keep the sensors turned on at all times, and to let them charge when possible.
POLLAR Study The EIT-HEALTH POLLAR clinical study  [45]  began in March 2019, with the objective of estimating the impact of air pollution and pollen exposure on rhinitis, asthma and sleep. Pollen data were obtained through the Réseau National de Surveillance Aérobiologique (R. N. S. A. , French aerobiology network [53]). The impact on sleep was studied at the Grenoble Alpes University Hospital where sleep apnea patients were recruited to perform, at two different periods, a polysomnography (PSG), followed by three days of air pollution exposure using a Canarin Nano or a Canarin II. A PSG is a comprehensive sleep study that requires the subject to stay at the hospital for one night equipped with monitoring devices (Figure  3 ). The main PSG outcome considered in the present study is the severity of the sleep apnea, given by the apnea-hypopnea index (AHI).