dataesr/scientific-paragraphs-categorization huggingface dataset:
Introduction The combination of different MRI sequences, signal weighting techniques and contrast agents that are currently used for MRI gives rise to diverse modalities and qualities of the output image. Although each technique yields exploitable results, the variation in the image contrast can be detrimental for the development of automatic analysis tools in medical imaging. To answer to the input diversity problem in machine learning segmentation, Seeböck et al. used an unpaired modality transfer generator network to reduce the variability between multi-centric datasets  [12]. On the other hand, Isensee et al. proposed the nnU-Net (no-new U-Net), which automatically generates a CNN pipeline that is optimised for each specific dataset  [5]. However, these methods require a sufficient mono-modality dataset, as they were built to be used for mono-modality segmentation.
In this study, we propose an alternative approach to this problem. We design a data augmentation method to train a single Deep Learning model to be robust to multi-modality input, including the modality that was not used for optimisation, thus the trained model can be used as a generalised segmentation tool. The style data augmentation introduces diversity of image contrast into the training dataset, with the goal to prevent the model from over-fitting toward the training image modality and to focus the network attention to the fundamental geometry features of the target. We base this method on the idea that despite having different contrasts, the organ geometry features are consistent between MRI modalities. In this study, we use a 3D convolutional neural network for the segmentation  [1].
Style Data Augmentation The style data augmentation strategy focuses on introducing contrast diversity in the training dataset, via different image processing algorithms. The aim is to prevent the model from over-fitting to any specific contrast and to focus the optimisation toward the fundamental geometry features of target. The image transformation algorithms were selected arbitrarily, as the goal is to simply increase the variety of the training images. For this study, we selected 5 transformation functions, including adaptive histogram equalisation  [3] , Laplacian transformation, Sobel edge detection, intensity inversion and histogram matching  [10] , as shown in figure  2. The histogram matching method can be used to convert the histogram of the original training images (C0 and T2) toward the histogram of the validation images (LGE) without the ground truth mask.
More details on the dataset used for this study is described in subsection 3. 1. These functions were applied to the normalized original image using the functions provided by SimpleITK's python package  [8, 14]. Our goal is to pre-train a segmentation model that is robust to any unknownto-the-model modality. As the method focuses on the geometry features, it is only suited to be used for the image modalities where the target shape is consistent.
At each epoch, 100 images will be chosen randomly from the training dataset to be used to train the network. The network is updated using Adam optimisation and Dice loss, calculated using equation 1, where Ŷ is the prediction mask, and Y is the manual labelled mask. During the training, we also measure the original Dice coefficient  [2]  between the prediction and the manual mask, by applying the "half to even" round function to binarise the output segmentation, equation 2. The round function breaks the gradient chain, which prevents Dice coefficient from being used for backpropagation. Dice Loss = 1 -2 * ( Ŷ * Y ) Ŷ + Y (1) Dice Coef f = 2 * (round( Ŷ ) * Y ) round( Ŷ ) + Y (2) 3 Evaluation on Clinical Data  While the SDA method was not designed to be used with mono-modal datasets, we still wish to study the effectiveness of the training input's contrast diversity under this context.
We used our local dataset, which consists of 119 mono-centric LGE-CMR images provided by IHU Lyric. Since the dataset is mono-modal, we remove histogram matching from the SDA algorithms. The original dataset was first split in 9:1 ratio for training and validation, before the augmentation method was applied to the training images. We then compare the mean of best validation scores from 5 different sets of validation images of the model trained with and without SDA method.
Results Multi-Modality. The table 1 shows the validation results of TCL-Net and Isensee on the multi-modal dataset. Using the TCL-Net with dual network and SDA method, the validation scores reach 0. 967 and 0. 904 for the epicardium and endocardium segmentations. This is a considerable improvement from 0. 833 and 0. 692 without these two improvements. There is a slight decrease in performance when histogram matching is removed from the SDA algorithms. Nonetheless, the network still performs better compared to training with only original images. As shown in figure  4 , TCL-Net performance is enhanced considerably with multi-modality training set (SDA and C0&T2) compared with mono-modality. We can observe in figure  4b  that the model would quickly overfit to the training modality, as the gap between training and validation scores get higher each epoch.
On the contrary, the over-fitting becomes less severe when there is diversity in the training input, as shown in figure  4c. The validation also appears more stable at the end of the training with SDA compared with the training with only original C0&T2. The figure  5  shows the validation output of epicardium and endocardium segmentations using the dual TCL-Net models trained with SDA method. Both models perform well and produce accurate segmentation in the region where there is no myocardial scar. Yet, the models struggle at the scar regions, as pointed by the arrows in the figure  5b  and 5c. Mono-Modality. When testing on the mono-centric and mono-modal dataset the SDA method does show improvement in validation Dice coefficients from 0. 874 to 0. 905 for the epicardium segmentation in the first TCL-Net, table  2.
Discussion SDA-Epicardium and Endocardium. The image processing functions implemented in SDA create images of different contrasts with defined border and geometric features, thus making the method applicable to the target regular structure such as the epicardium and endocardium. The results in section 3. 2 show the increase in performance in both mono-and multi-modal datasets for the segmentation of the epicardium. As shown in multi-modal experiments, the SDA improves segmentation validation score of the LGE images, without any optimisation with the actual LGE data. The slight decrease in performance when histogram matching was not included in SDA further proves that the strategy does not overly depend on this particular transformation. It also validates that the increase in contrast diversity in augmentation algorithms leads to the increase in performance, rather than over-fitting.
Nevertheless, the trained model still reaches a limit, as observed in dual network segmentation in table  1. SDA-Myocardial Scar. Because the model can no longer depend on image contrast for the segmentation when training with SDA method, it has to rely on the patterns of the target, such as the traces of the myocardium wall and the homogeneity of the intensity of each structure. Therefore, the method might not be suitable for the targets without uniform structure, such as the myocardial scar. For instance, when training on C0 and T2 images of MS-CMRSeg challenge, the model is only familiar with homogeneous myocardium. Thus, it does not perform well when scar is present on the myocardium of LGE images, figure  5. The inconsistency in contrast of the myocardial scar may explain why the network achieves better result without SDA for the scar segmentation on the mono-modal dataset.
As shown in figure  6a  by the red arrow, the scar region does not include the entire area of the same intensity, since the upper area belongs to the cavity of the ventricle. Because the scar does not have specific geometric shape like the epicardium or endocardium, the model trained with only the original image would perform better, since it can depend more on the specific contrast of the LGE modality during optimisation than the model trained with SDA. TCL vs. Isensee. In their original paper  [4] , Isensee et al. integrate a more elaborate preprocessing technique on the input image than what is done in this experiment. Therefore, our experiment might not present the optimal condition for the baseline network. Our goal is to simply compare the performance of the larger network with our new architecture on the minimal processing datasets.
Conclusion We proposed a data augmentation strategy that increases the accuracy of the segmentation and is invariant to the modality of the validation image. The SDA strategy forces the network to be independent from the input image modality and prevents it from over-fitting to any specific contrast. This validates our theory that the diversity in training input increases the neural network performance. The image transformation algorithms in SDA can also be seen as placeholders and be easily replaced by the real world MR modalities. Our current experiment uses the validation result of LGE images to terminate the training, thus making the trained coefficients bias toward the LGE modality. A more diverse realworld multi-modality dataset is needed to improve the universality of the trained network.
For the multi-modal dataset, we use the dataset provided in MS-CMRSeg 2019 segmentation challenge [15, 16]. The challenge dataset consists of 135 3D CMR images of 45 patients taken under three modalities: T2-Weighted (T2), balanced-Steady State Free Precession (C0) and late Gadolinium enhanced (LGE), figure3. The manually labelled masks were provided for the first 35 images of T2 and C0 but only for the first 5 images of LGE modality. The provided labels include epicardium and endocardium of the left ventricle and endocardium of the right ventricle. By applying our data augmentation method, we trained the network with 420 images of different variations of the original C0 and T2 images and validated the network with the 5 LGE images.
3. 1 Materials 1 For the multi-modal dataset, we use the dataset provided in MS-CMRSeg 2019 segmentation challenge [15, 16]. The challenge dataset consists of 135 3D CMR images of 45 patients taken under three modalities: T2-Weighted (T2), balanced-Steady State Free Precession (C0) and late Gadolinium enhanced (LGE), figure3. The manually labelled masks were provided for the first 35 images of T2 and C0 but only for the first 5 images of LGE modality. The provided labels include epicardium and endocardium of the left ventricle and endocardium of the right ventricle. By applying our data augmentation method, we trained the network with 420 images of different variations of the original C0 and T2 images and validated the network with the 5 LGE images.
used in many studies to estimate the observation errors and inter-channel error correlations  (Garand et al. , 2007; Weston et al. , 2013). In the present work, we estimate observation errors and their inter-channel correlations for IASI using Desroziers statistics. We evaluate, then, their impact on ozone assimilation in a chemical transport model (MOCAGE). The paper is organized as follows. The chemical transport model, the radiative transfer model, the assimilation algorithm, the data, and the experimental framework are described in section 2. The estimation of R is discussed in section 3. Then, the impact on data assimilation is reported in section 4. Finally, the summary and conclusions are given in the last section. It is a three-dimensional CTM providing the space and time evolution of the chemical composition of the troposphere and the stratosphere.
Assimilation algorithm The variational data assimilation system of MOCAGE was developed jointly by CERFACS and Météo France in the framework of the European project ASSET (ASSimilation for Envisat data)  (Lahoz et al. , 2007). It has been used in several studies such as chemical data assimilation research  (Emili et al. , 2014; Massart et al. , 2009) , aerosols data assimilation  (Sič et al. , 2015)  and tropospheric-stratospheric exchange using data assimilation  (El Amraoui et al. , 2010). The MOCAGE data assimilation system is flexible and allows multiple assimilation options, for example, the choice of the variational method (3D-Var, 4D-Var), the representation of the background error covariance, and the type of observation assimilated. It is also used to produce operational air quality analyses for the European Project CAMS  (Marécal et al.
, 2015). The background error covariance matrix is divided into two distinct parts, the diagonal matrix of the standard deviations and the correlation matrix. The latter, allowing to spatially smooth the assimilation increments, is modeled through a diffusion operator  (Weaver and Courtier, 2001). In the data assimilation system of MOCAGE, the observation error covariance matrix can be read from the data file previously defined. In the case of diagonal matrix, the variances can be calculated as a percentage of the observation values. The 3D-Var implementation has been used with hourly assimilation windows. The variational cost function is minimized using the BFGS (Broyden-Fletcher-Goldfarb-Shanno ) algorithm  (Liu and Nocedal, 1989). The control vector includes the Skin Surface Temperature (SST) and the ozone.
, 2009). The swath width on the ground is 2200 km which provides global Earth coverage twice a day. The measurements provide information on atmospheric chemistry compounds such as O3, surface properties (Skin Surface Temperature SST), and meteorological profiles (humidity and temperature). For this study, L1c data have been downloaded from the EUMETSAT Earth Observation data portal (https://eoportal. eumetsat. int, last access: 16 July 2019) in NETCDF format. Data files also contain the co-located land mask and cloud fraction values, obtained from the Advanced Very High-Resolution Radiometer (AVHRR) measurements, also on board Metop.
Ozonesondes Ozonesondes are in situ instruments carried by a radiosonde transmitting continuously the measurements as the ascent in the atmosphere. The profiles of O3 are provided up to an altitude that often exceeds 30 km  (Jiang et al. , 2007)  with a vertical resolution of 150-200 m. They have been used for several applications such as validating satellite products  (Jiang et al. , 2007). In our study, vertical profiles of ozone, collected and distributed by the Word Ozone Ultraviolet Radiation Data Centre (http://www. woudc. org, last access), are used to validate the model simulations.
In order to avoid any contamination from clouds, data were filtered using a cloud mask and only pixels with cloud fraction less than or equal to 1 % were kept. The cloud fraction values are obtained from the AVHRR measurements onboard Metop. Since the spatial resolution of MOCAGE is coarser than the pixel size, the number of ground pixels was reduced by thinning the data using a grid of 1°x 1°of resolution and only keeping the first pixel that falls in every two grid boxes. A dynamical rejection of observations -with a threshold of 12 % -based on the relative differences between simulated and measured values with respect to simulated values was considered. Some channels affected by H 2 O absorption (1008-1019,1028-1030, 1064-1067,1072-1076,1089-1092 cm -1 ) were removed. Pixels affected by aerosols are detected and then removed using the index based on V-shaped sand signature as discussed in  Emili et al.
Acknowledgements. We acknowledge  EUMETSAT  for providing IASI L1C data, WOUDC for providing ozonesondes data and the  NASA Jet Propulsion Laboratory  for the availability of Aura MLS Level 2 O3. We also thanks the  MOCAGE team at Météo-France  for providing the chemical transport model, the RTTOV team for the radiative transfer model, and  Gabriel Jonville  for the help on technical developments of the assimilation code. This work has been possible thanks to the financial support from the  Région Occitanie.
Enrichment of TNF-alpha, IL6, and hypoxia-related pathways in PBMC of coronavirus patients Based on the expression levels of key genes, including TNF, we next set out to explore the functional implications of the CoV-Up-gene signature. To do so, we performed enrichment analysis using the genes in CoV-Upgene signature and MSigDB's hallmark gene set database  20  (Fig. 3C ). This revealed multiple highly ranked pathways involved in cytokine storm and acute infection including TNF signaling, IL6 (IL6-JAK-STAT3) and IL2 (IL2-STAT5) signaling, inflammatory response, and KRAS/MTOR and late responses to estrogen pathways (Fig. 3C  and Supplementary Table  2A ). This is consistent with clinical manifestations including observations of high IL6 levels in COVID-19 patients  25. Interestingly, outside of the inflammatory response, multiple pathways related to hypoxia, angiogenesis, and oxygen transport (heme/iron) were also implicated, consistent with the oxygen limitation  26  experienced during coronavirus infection (Fig.
3C ). Of particular interest was the enrichment of complement and coagulation pathways which may explain the high hypercoagulability observed in COVID-19 patients  27   patients and healthy individuals (n = 3; datasets from Xiong et al. 17  ). For C and D statistical significance was not considered due to low sample size. E, F. CoV-Up-gene signature scores (E) and CoV-Down-gene signature scores (F) and their association with acute and recovering SARS-CoV-infected patients (n = 25; 44 samples), bacteria-infected patients (n = 16), and healthy individuals (n = 9; datasets from Lee et al. 19  ). Kruskal-Wallis statistical test with p < 0. 001 for E and F. G Acute vs. recovering gene signature from Lee et al. 19  predicted acute-like and recovering-like SARS-CoV-infected patients (n = 10) from Raghunathan et al.
Potential role for a network of related pathways representing cytokine storm and innate immune changes in PBMC of coronavirus-infected patients Given that different pathways were enriched in infected patients, next, we interrogated how these pathways are linked together to convey a network of processes or changes at the cellular level. Hence, we used the REAC-TOME pathway database  28  to connect different but related pathways that were enriched in infected patients using network analysis. Two evident and distinct networks were significantly enriched in CoV-Up-gene signature: (a) interleukins and cytokine signaling (potentially representing cytokine storm) and (b) neutrophils and innate immunity (Fig. 4A  and Supplementary Table  2B ). Nevertheless, we observed an increased enrichment of a unique network linking granulopoiesis, megakaryocyte differentiation, and platelet activation (Fig.
This suggests that coronavirus infection and SLE may be related to each other with respect to disease symptoms. The prevalence and risk factors of severe COVID-19 in SLE patients remain unclear  30, 31. The understanding at the systemic level and additional clinical reports are needed to know whether certain COVID-19 disease conditions may be associated with SLE. A more relevant and well-known individual pathway associated with this disease is the renin-angiotensin system associated with ACE2 function (Fig. 4B ). There is also an enrichment of RIG-I-like receptor signaling pathway representing potential anti-viral event through pathogen-associated molecular patterns  32. Multiple chemokine/cytokine and metabolism pathways were enriched as a part of the CoV-Up-gene signature in the KEGG database (Fig.
2) increase in natural killer (NK) cells, cytolytic activity, and plasmacytoid dendritic cells (pDCs) in recovering-like patients compared to acute-like patients was observed (Fig. 5A ). It is noteworthy that the absolute levels of CD8 + T cells and co-stimulating helper T cells are not different between recovering-like and acute-like patients (Fig. 5A ). This result suggests that the CD8 + T cells are potentially activated (cytolytic) in the recovering patients. Certain results from SARS patients were assessed using PBMC from limited (see figure on previous page) Fig. 4  Highly represented related networks of molecular, cellular, and development pathways show cytokine (storm) network and innate immunity in coronavirus-infected patients. A REACTOME  28  database-based connection of different but related pathways that were enriched (hypergeometric test of FDR < 0.
2 and overlap a similar index of 0. 5 using hypeR  57  ) in infected patients using network plots. B KEGG  60  pathwaysbased network showing enrichment (hypergeometric test of FDR < 0. 2 and overlap a similar index of 0. 25 using hypeR  57  ) of different diseases and infection related pathways. For A and B, nodes and edges are of same size and length, respectively. C Enrichment statistical analysis (hypergeometric test of FDR < 0. 2 using hypeR  57  ) using CoV-Up-gene signature and pathways/processes based on gene sets from COMPARTMENTS database  61. number of COVID-19 samples (n = 3; FDR < 0. 2; Fig. 5B  and Supplementary Table  2F ). No difference in B cells between both types of patients and healthy individuals was found. Intriguingly, higher level of expression of interferon (IFN)-γ type-II gene was found in the recovering-like patients and lower levels in both acute-like patients and healthy volunteers (Fig.
Similarly, MHC class-I HLA gene, HLA-F showed a trend akin to HLA-DQA1 (Fig. 5C  and p < 0. 05). On the other hand, there was no change in the antigen processing machinery (data not shown). However, all these speculations from limited data warrant further validation, and their functional immunological significance is currently unclear. To confirm these analyses of immunologic composition, we performed two additional independent analyses. Hypergeometric test-based enrichment analysis of MSigDB's C7 immune signature  20  showed similar conclusions that innate immunity and myeloid (neutrophils and macrophages) cells are upregulated in coronavirus patients (in general), whereas adaptive immunity is downregulated in these patients (Fig. 5D  and Supplementary Table  2G ). Similarly, analysis using the BioGPS database-gene sets  36  demonstrated an increased enrichment of CD33 + myeloid and CD14 + monocytes associated with upregulated genes in our CoV expression signature (Fig.
Our findings may suggest that individuals with activated, appropriate immune responses, especially with increased INF-γ type-II responses and cytolytic activity, which may also serve as biomarkers with further validation, maybe on their way to recovery from symptoms. Moreover, those patients' incapable of such progression may have multiorgan failures that may be represented in our data. Although our study may be timely for the current pandemic, there are limitations. We have performed analysis using publicly available small number of coronavirus-based training (n = 10) and test (n = 47) samples from less annotated datasets with limited clinical data, which may be appreciated provided the current global lock-down scenario. Also, the acute-like and recovering-like patients may overlap with the symptomatic and asymptomatic patients, respectively, described in the original publication from where the training dataset was derived  16.
Data sources PBMC transcriptomes from SARS, COVID-19, and other patients were obtained from published studies  [16] [17] [18] [19]  with GEO Omnibus identifiers-GSE1739 and GSE6269, and normalized data directly from the original publications. Raw. CEL files from Affymetrix Human HG-Focus Target Array for GSE1739 were obtained from the authors of the original publication  16  and robust microarray analysis-based normalization was performed using R-based Bioconductor package-affy  49. GEOquery R package  50  was used to obtained gene expression and/or phenotypic data from GEO Omnibus for GSE6269.