dataesr/scientific-paragraphs-categorization huggingface dataset:
This new strategy leverages the final prediction of the multi-source framework to guide the learning of the per source CNN encoders supporting the network to learn from itself. Experiments are carried out on two real world benchmarks, namely the Reunion island (a french overseas department) and the Dordogne study site (a southwest department in France) where the annotated reference data were collected under operational constraints (sparsely annotated ground truth data). Obtained results, providing an overall classification accuracy of about 94% (resp. 88%) on the Reunion island (resp. the Dordogne) study site highlight the effectiveness of our framework based on CNNs and self-distillation to combine heterogeneous multi-sensor remote sensing data and confirm the benefit of multimodal analysis for downstream tasks such as land cover mapping.
To assess the effectiveness of the proposed framework, we consider two real-world benchmarks, namely the Reunion island (a French overseas department located in Indian Ocean) and the Dordogne study site (a southwest department in France) both involving highly sparse ground truth data obtained by means of field campaigns and institutional surveys (see Fig. s 1 and 2). Our framework adopts CNNs as per-source encoders since they are consolidated strategies to deal with VHSR image and, recent studies (e. g. [8] ,  [18] ,  [9] ,  [19] ) have highlighted that such models are even competitive for multi-temporal information such as SITS data. When dealing with real-world LULC mapping in an operational setting, the collected GT is generally sparse due to human-effort and cost constraints  [20] ,  [21] ,  [22].
This means that a limited number of polygons (in terms of surface with respect to the study site) is annotated by field experts with the aim to have samples covering the whole study area without taking care to highlight possible spatial correlations among classes (class polygons are far away from each other). For instance, Fig. 1  depicts a study area characterized by sparse GT data. In the extract to the right of the figure we clearly observe that only a small portion of the area is labelled and polygons are spatially sparse. Matter of fact, the most common GT data collection protocol in operational settings prevents the use of standard semantic segmentation approaches  [23] ,  [24] ,  [25]  widely adopted in the computer vision community, since semantic segmentation strategies require densely annotated patches on which the model is trained on (each pixel should be associated to a label information).
The GT data for the Reunion island was built from various sources: the Registre Parcellaire Graphique (RPG) reference data for 2016 (the French land parcel identification system), Global Positioning System records from June 2017 and the visual interpretation of a SPOT image completed by a field expert with knowledge of territory. The Reunion island dataset is publicly available 4   [32]. Similarly for the Dordogne site 5 , the GT was built from RPG reference data for 2016 and the visual interpretation of a SPOT image as well. For both study sites, the GT was assembled in Geographic Information System (GIS) vector file, containing a collection of polygons, each attributed with a land cover category (See Tables  I  and II ). Finally, the polygons have been rasterized at the Sentinel spatial resolution (10-m), obtaining 880 723 labeled pixels for the Reunion island (respectively 814 436 labeled pixels for the Dordogne site).
Such model explicitly manages the sequential information of the SITS since it performs multi-dimensional convolutions on the temporal dimension. Here, only pixel time series information is considered. For the VHSR SPOT image, we still consider a 2D-CNN model with the aim to exploit as much as possible the available fine scale spatial information. In addition, the SPOT image has Panchromatic (PAN) and Multi-Spectral (MS) bands with a resolution of 1. 5 and 6 meters, respectively. With the aim to manage such data at their native resolution avoiding as much as possible intermediate resampling steps (e. g. pan-sharpening), the 2D-CNN model for the SPOT image starts processing the PAN information and, once feature maps at the same resolution of the MS information are produced, the MS bands are integrated in the analysis by concatenation.
To fit the input requirements of the M 3 F usion competitor, the VHSR SPOT images were pansharpened on both study sites and multi-spectral image patches of size 32×32 at the highest spatial resolution i. e 1. 5-m were extracted. For the MMCNN 10 SD baseline, the pansharpened images were resampled to 10-m spatial resolution using the nearest neighbor method and finally multi-spectral image patches of size 5×5 (covering approximately the same spatial extent as the native resolution image patches) were extracted. Note that we have considered the 2D-CNN designed for the Sentinel data in order to process the SPOT patches at 10-m spatial resolution. Nonetheless, for compatibility purposes, a zero padding was set up for the first convolutional layer. The values of the dataset were normalized per band in the interval [0, 1], considering the time series and the VHSR, pansharpened and resampled images.
The datasets were split into training, validation and test set with a proportion of 50%, 20% and 30% of samples respectively. We imposed that pixels belonging to the same ground truth polygon were assigned exclusively to one of the data partition (training, validation or test) with the aim to avoid possible spatial bias in the evaluation procedure. The evaluated models were optimized via training/validation procedure  [36]. Their hyper-parameter settings are reported in Table  IV. For the settings of the M 3 F usion model, we adopted the same hyper-parameter values as reported in  [10]. The assessment of the model performances was done considering test set and the following metrics: Accuracy (global precision), F1 score (harmonic mean of precision and recall) and Cohen's Kappa (level of agreement between two raters relative to chance).
Since the model performances may vary depending on the split of the data due to simpler or more complex samples involved in the different partitions, all metrics were averaged over five random splits of the dataset following the strategy mentioned above. Experiments were carried out on a workstation with an AMD Ryzen 7 3700X CPU, 64 GB of RAM and RTX 2080 NVIDIA GPU. The number of trainable parameters of the evaluated models and the associated time costs are reported in Table  V. The different architectures were implemented using the Python Tensorflow library. The code implementation of MMCNN SD is available at https://github. com/eudesyawog/S1S2VHSR.
B. Per Sensor encoder assessment The performances of the per sensor encoders at the two study sites are reported in Table VI and Table VII, respectively. As regards average results, we note first that leveraging temporal or spatial dependencies for S1 and S2 exhibits different behaviours. Employing 2D convolutions in the CNN instead of 1D convolutions is clearly more effective for S1 while obtained results are comparable for S2. This specific behaviour comes from the fact that 2D convolutions reduces in turn the spatial speckle noise  [34]  in the S1 data exploiting the spatial context information available when input patches are used. About the 3D-CNN, it achieves overall slightly lower (e. g. for S1) or similar results (e. g. for S2) than the 2D-CNN encoder. Only average results for S2 in the case of Reunion island are slightly better than those of the 2D-CNN.
Then, the benefit here of leveraging simultaneously convolutions in both spatial and temporal domains via the 3D-CNN is minimal, especially regarding trainable parameters and training time costs (See Table  V ). For the rest, SAR data (S1) is less effective than optical ones (S2 or SPOT) for the land cover mapping tasks. However, note the significance of the fine scale spatial information provided by the VHSR SPOT data on the Reunion island, which gives competitive performances than those of S2 data, with respect to the Dordogne site. Overall, the validation of per source CNN encoders suggests that the 2D-CNN model is the most effective to deal with S1 SITS while the 1D-CNN seems more appropriate to manage S2 SITS owing to a cheaper cost in terms of computational training time. Hereafter, S1 and S2 refer to the single-modality models with 2D and 1D-CNN, respectively.
Part I: Disease Quantification We report on a deep learning-based segmentation tool to quantify the COVID-19 disease and lung volume, using an ensemble network approach combining a 2D (AtlasNet framework  20  ) and a 3D (3D-UNet  21  ) architecture. We investigated a combination of 2D slice-based  20, 22  and 3D patch-based ensemble architectures  21. The development of the deep learning-based segmentation solution was done on the basis of a multi-centric cohort of unenhanced chest CT scans performed at initial evaluation of COVID-19 patients with positive Reverse transcription polymerase chain reaction (RT-PCR). The multicentric dataset was acquired at 6 hospitals, equipped with 4 different CT models from 3 different manufacturers, with different acquisition protocols and radiation doses (Table  1 ).
Fifty CT exams from three centers were used for training and 130 CT exams from three different centers were used for test (Table  2 ). Disease and lung were delineated on all 23, 423 slices used of the training dataset, and on only 20 slices -equispaced, covering the entire lung region per exam -but by 2 independent annotators in the test dataset (2, 600 images). The overall annotation effort took approximately 800 hours and involved 15 radiologists with 1 to 7 years of experience in chest imaging. The consensus between manual (2 annotators) and AI segmentation was measured using the Dice similarity score (DSC)  23  and the Hausdorff distance (HD). The CovidENet performed equally well to trained radiologists in terms of DSCs and better in terms HD (Figure  2, 5 ). The mean/median DSCs between the two expert annotations on the test dataset were 0.
Our study reviewed outcomes in patient charts within the 4 (short-term) and 31+ days (long-term) at admission, dividing the patients in 2 groups: patients who died, or required mechanical ventilation either at initial or a subsequent admission as severe cases (S), and patients who were non-severe cases (NS). Two disjoint independent sets were built for training (536 patients from 5 centers) and testing (157 patients from the remaining 3 centers) (Table  3 ). COVID-19 multidisciplinary, multi-omics patient profiling was achieved through an evidence-driven mining approach. Information relevant to the disease volume and nature (CT-imaging), the non-COVID 19 affected lungs (CT-imaging), the heart condition / volume of the heart (CT-imaging), obesity (CT-imaging), biological data (lymphocytes, C-reactive protein (CRP) levels) and known demographic (age & sex)/confounding (diabetes, hypertension) factors were aggregated.
6/26 Part V: Conclusions & Future Work AI-enhanced imaging/clinical/biological/ information proved capable to identify patients with severe short/long-term outcomes, bolstering healthcare resources under the extreme pressure of the current COVID-19 pandemic. The information obtained from our AI staging and prognosis could be used as an additional element at admission to assist decision making. To conclude, our work contributes to addressing the pandemic in terms of (i) patient stratification with respect to the different therapeutic strategies, (ii) accelerated drug development through rapid, reproducible and quantified assessment of treatment response through the different mid/end-points of the trial, and (iii) continuous monitoring of patient response to treatment. In terms of future work, the continuous enrichment of the database with new examples is a necessary step on top of updating the outcome of patients included in the study.
Study Design and Participants This retrospective multi-center study was approved by our Institutional Review Board (AAA-2020-08007) which waived the need for patients' consent. Patients diagnosed with COVID-19 from March 4th to April 5th from eight large University Hospitals were eligible if they had positive reverse transcription polymerase chain reaction (PCR-RT) and signs of COVID-19 pneumonia on unenhanced chest CT. A total of 693 patients formed the full dataset (321, 360 CT slices). Only the CT examination performed at initial evaluation was included. Exclusion criteria were 1/ contrast medium injection and 2/ important motion artifacts. No patient was intubated at the time of the CT acquisition. For the COVID-19 radiological pattern segmentation part, 50 patients from 3 centers (A: 20 patients; B: 15 patients, C: 15 patients) were included to compose a training and validation dataset, 130 patients from the remaining 3 centers (D: 50 patients; E: 50 patients, F: 30 patients) were included to compose the test dataset (Table  2 ).
The patients from the training cohort were annotated slice-by-slice, while the patients from the testing cohort were partially annotated on the basis of 20 slices per exam covering in an equidistance manner the lung regions. The proportion between the CT manufacturers in the datasets was pre-determined in order to maximize the model generalizability while taking into account the data distribution. For the staging (severe versus non-severe case) and prognosis (short and long-term) study, 513 additional patients from centers A (121 patients), B (157 patients), D (138 patients), G (77 patients) and H (20 patients) were included. Data of 536 patients from 5 centers (A, B, C, D and H) were used for training and those of 157 patients from 3 other centers (E, F and G) composed an independent test set (Table  3 ).
Data annotation Fifteen radiologists (GC, TNHT, SD, EG, NH, SEH, FB, SN, CH, IS, HK, SB, AC, GF and MB) with 1 to 7 years of experience in chest imaging participated in the data annotation which was conducted over a 2-week period. For the training and validation 7/26 set for the COVID-19 radiological pattern segmentation, the whole CT examinations were manually annotated slice by slice using the open source software ITKsnap 1. On each of the 23, 423 axial slices composing this dataset, all the COVID-19 related CT abnormalities (ground glass opacities, band consolidations, and reticulations) were segmented as a single class. Additionally, the whole lung was segmented to create another class (lung). To facilitate the collection of the ground truth for the lung anatomy, a preliminary lung segmentation was performed with Myrian XP-Lung software (version 1.
19. 1, Intrasense, Montpellier, France) and then manually corrected. As far as test cohort for the segmentation is concerned, 20 CT slices equally spaced from the superior border of aortic arch to the lowest diaphragmatic dome were selected to compose a 2, 600 images dataset. Each of these images were systematically annotated by 2 out of the 15 participating radiologists who independently performed the annotation. Annotation consisted of manual delineation of the disease and manual segmentation of the lung without using any preliminary lung segmentation. Additionally, 3 radiologists, an internationally recognized expert with 20+ years of experience in thoracic imaging (Reader A ), a thoracic radiologist with 7+ years of experience (Reader B ) and a resident with 6-month experience in thoracic imaging (Reader C ) were asked to perform a triage (severe versus non-severe cases) and for the severe cases (short-term deceased versus short-term intubated) prognosis process to predict the short-term outcome.
Disease quantification COVID-19 segmentation tool was built using an ensemble method combining a 2D & 3D neural network method. AtlasNet framework  20  , a 2D fully convolutional network (CovidE2D) with a SegNet-based architecture  22  was coupled with a 3D fully convolutional network based on the 3D-UNet 21 (CovidE3D). The AtlasNet framework combines a registration stage of the CT scans to a number of anatomical templates and consequently utilizes multiple deep learning-based classifiers trained for each template. At the end, the prediction of each model is back-projected to the original anatomy and a majority voting scheme is used to produce the final projection, combining the results of the different networks. A major advantage of the AtlasNet framework is that it incorporates a natural data augmentation by registering each CT scan to several templates.
Lung/Breast/Heart & Body Contours Annotations Lung, breast, heart and body contours segmentation masks of all patients were extracted by using ART-Plan software (TheraPanacea, Paris, France). ART-Plan is a CE-marked solution for automatic annotation of organs, harnessing a combination of anatomically preserving and deep learning concepts. This software has been trained using a combination of a transformation and an image loss. The transformation loss penalizes the normalized error between the prediction of the network and the affine registration parameters depicting the registration between the source volume and the whole body scanned. These parameters are determined automatically using a downhill simplex optimization approach. The second loss function of the network involved an image similarity function -the zero-normalized cross correlation loss -that seeks to create an optimal visual correspondence between the observed CT values of the source volume and the corresponding ones at the full body CT reference volume.
This information was applied for every organ of interest presented in the input CT Scan. In average, 6, 600 samples were used for training per organ after data augmentation. These networks were trained using a conventional dice loss. The final organ segmentation was achieved through a winner takes all approach over an ensemble networks approach. For each organ, and for each full body reference CT a specific network was built, and the segmentation masks generated for each network were mapped back to the original space. The consensus of the recommendations of the different subnetworks was used to determine the optimal label at the voxel level.
Staging Mechanism The staging/prognosis component was addressed using an ensemble learning approach. Similarly to the biomarker extraction, the training data set was subdivided into training and validation sets on the principle of 80%-20%. This subdivision was performed such that the distribution of classes between the two subsets was identical to the observed one. We have created 10 subdivisions on this basis and evaluated the average performance of the following supervised classification methods: Nearest Neighbor, {Linear, Sigmoid, Radial Basis Function (RBF), Polynomial Kernel} Support Vector Machines (SVM), Gaussian Process, Decision Trees, Random Forests, AdaBoost, XGBoosting, Gaussian Naive Bayes, Bernoulli Naive Bayes, Multi-Layer Perceptron & Quadratic Discriminant Analysis. These classifiers have been trained using the identified holistic signature.
Concerning the implementation details, to overcome the unbalance dataset for the different classes, each class received a weight inversely proportional to its size. For the NS versus S majority voting classifier the top 5 classifiers consists in RBF SVM, Linear SVM, AdaBoost, Random forest, Decision Tree (Table  5 ). The SVM methods were granted a polynomial kernel function of degree 3, the Linear kernel one had a penalty parameter of 0. 3 while the RBF SVM had a penalty parameter of 0. 15. In addition, the RBF SVM was granted a kernel coefficient of 1. The Decision Tree classifier was limited to a depth of 2 to avoid overfitting. The Random Forest classifier was composed of 25 of such Decision Trees. AdaBoost classifier was based on a decision tree of maximal depth of 1 boosted 4 times. For the SI versus SD majority voting classifier the top 5 classifiers consists in polynomial SVM, Linear SVM, Decision Tree, Random Forest and AdaBoost (Table  7 ).
Statistical Analysis The statistical analysis for the deep learning-based segmentation framework and the radiomics study was performed using Python 3. 7, Scipy 32 , Scikit-learn 33 , TensorFlow 34 and Pyradiomics 35 libraries. The dice similarity score (DSC)  23  was calculated to assess the similarity between the 2 manual segmentations of each CT exam of the test dataset and between manual and automated segmentations. The DSC between manual segmentations served as reference to evaluate the similarity between the automated and the two manual segmentations. Moreover, the Hausdorff distance was also calculated to evaluate the quality of the automated segmentations in a similar manner. Disease extent was calculated by dividing the volume of diseased lung by the lung volume and expressed in percentage of the total lung volume.
Disease extent measurement between manual segmentations and between automated and manual segmentations were compared using paired Student's t-tests. For the stratification of the dataset into the different categories, classic machine learning metrics, namely balanced accuracy, weighted precision, and weighted specificity and sensitivity were used. The weighted metrics are expressed as follows. If we denote N the total number of samples, N l the number of samples with class of label l and S l the non-weighted score in one-vs-rest classification for class of label l, the corresponding weighted score WS is then: W S = 1 N ∑ l N l S l Moreover, the correlations between each feature and the outcome was computing using a Pearson correlation over the entire dataset. Patient characteristics between training/validation and test datasets were compared using chi-square and Student's t-tests.
Patient characteristics of the training and testing datasets used for the creation of the Holistic Multi-Omics COVID19 signature and the development of short, long-term prognosis tools. Note. -For quantitative variables, data are mean ± standard deviation, and numbers in brackets are the range. For qualitative variables, data are numbers of patients, and numbers in parentheses are percentages. CT = computed tomography, CTDIvol = volume Computed Tomography Dose Index; DLP = Dose Length Product * Available clinical data: n = 692 for diabetes and high blood pressure(leading to 0. 19% of missing data on the training set), n = 674 for lymphocyte count (leading to 2. 05% and 5. 10% of missing data on the training and test sets respectively), n = 654 for CRP (leading to 4. 66% and 8. 92% of missing data on the training and test sets respectively), n = 362 for Body Mass Index, and n = 339 for D-dimers.
We present a novel algorithm to identify potential dispersed signals of new physics in the slew of published LHC results. It employs a random walk algorithm to introduce sets of new particles, dubbed "proto-models", which are tested against simplifiedmodel results from ATLAS and CMS (exploiting the SModelS software framework). A combinatorial algorithm identifies the set of analyses and/or signal regions that maximally violates the SM hypothesis, while remaining compatible with the entirety of LHC constraints in our database. Demonstrating our method by running over the experimental results in the SModelS database, we find as currently best-performing proto-model a top partner, a light-flavor quark partner, and a lightest neutral new particle with masses of the order of 1. 2 TeV, 700 GeV and 160 GeV, respectively.