dataesr/scientific-paragraphs-categorization huggingface dataset:
The EM algorithm used to estimate the model parameters is known to be sensitive to the initialisation point. Coseq uses a small-EM strategy, and in DiCoExpress, we go further to get robust results. First, mixture models with 5, 10, 15, 20, 25, and 30 subpopulations are estimated 5 times each to identify an interval for the final number of co-expressed gene clusters. A second collection of models on this interval of a subpopulation is then estimated 40 times each (per default). The best mixture model is the one minimising the Integrated Completed Likelihood (ICL). The ICL curve is expected to be a convex function of the number of subpopulations, and we use this criterion to assess that the chosen model fits well the data. When a different behaviour of the ICL curve is observed, we advise the user to modify the dataset removing some genes that show a too flat normalised profile.
Results and discussion We illustrate the use of DiCoExpress by analysing a dataset associated with the publication of Haddad et al. [42]. This RNAseq dataset describes gene expression in roots and mature leaves of Brassica napus with or without silicon (Si) treatment. Three biological replicates are available. The experimental design can be described by two biological factors Tissue and Treatment, and a technical factor Replicate with three modalities (Table  1 ). To illustrate the outputs of enrichment tests, we used the GO annotation of B. napus v. 5 from the Brassica genome database  [43, 44]  to perform enrichment analyses. We tested DiCoExpress on the full dataset available in contrast to Haddad et al, who only focused on the root samples. The procedure is described in Additional file 4 as a tutorial of DiCoExpress.
In this interaction analysis, we found 106 genes differentially expressed, and an example of a gene in this list is shown in Additional file 3: Fig. B. The hierarchical clustering on the top 50 DEGs ranking on their p-values for this contrast is also proposed by DiCoExpress (Additional file 3: Fig. C ). On the bottom of this plot, we observe groups of genes with a clear opposite behaviour between the two tissues. For the others, the behaviour is more variable, but all these genes are declared to be the most impacted genes by the treatment and in different ways in the two tissues. As users often need to compare DEG lists, in DiCoExpress, we propose the Venn_Intersection_Union function to generate these lists quickly. In the Brassica napus dataset, we unite three contrasts: [MatureLeaf_NoSi-MatureLeaf_Si], [Root_NoSi-Root_Si] and [Mature-Leaf_NoSi-MatureLeaf_Si]-[Root_NoSi-Root_Si] to study genes impacted in their transcription by the treatment (Additional file 5: Fig.
In cluster 2 (85 genes) and cluster 5 (130 genes) are grouped genes showing a small difference in expression levels between the two tissues. For both clusters, genes show over-expression following the treatment in leaves (more apparent in cluster 5), but no significant change in roots. The cluster composition based on probabilistic modelling of the normalised gene profiles is, as it could be expected, different from the groups of genes found with the DEG list comparisons (Additional file 6). Following the co-expression analysis, that finishes the statistical analysis of this dataset used to illustrate the use of DiCoExpress, we performed enrichment analyses on these 7 clusters, and also of the 106 DEGs for the interaction contrast, and they are available in the tutorial (Additional file 4).
Conclusions DiCoExpress is an R script-based tool for analysing efficiently multifactorial RNAseq transcriptome experiments from quality controls to co-expression analysis through differential expression analysis. We based the development of DiCoExpress on neutral comparison studies combining the most performant statistical approaches for each step of a standard RNAseq analysis. The choice of a statistics defined framework, limiting the free selection of models and methods, was deliberate to propose to the users the possibility to perform RNAseq analysis in a standardised manner. In DiCoExpress, we used generalized linear models implemented in the R-package edgeR for differential gene expression analysis and Gaussian mixture models implemented in the R-package coseq to perform the co-expression analysis.
DiCoExpress simplifies the GLM analysis proposing automated writing of all possible contrasts and optimises the co-expression analysis with the re-estimation of the collection of Gaussian models. DiCoExpress produces a collection of files to visualise the results and multiple summary files of the data for further data exploration. The integrated enrichment analysis with the hypergeometric test gives the user the first glimpse at potential biological functions underlying the different gene lists. In conclusion, DiCoExpress allows the user to perform RNAseq data analysis with validated statistical methods with a set of R-scripts in a pre-existing and organised working environment.
Availability and requirements • Project name: DiCoExpress • Project home page: https ://forge mia. inra. fr/GNet/ dicoe xpres s • Operating system(s): Windows, Mac OS, Linux • Programming language: R • Other requirements: R version 3. 5. 0 or higher with FactoMineR_2. 0 • License: GPL-2 | GPL-3 • Any restrictions to use by non-academics: None Availability of data and materials https ://forge mia. inra. fr/GNet/dicoe xpres s.
Infections have become the major cause of morbidity and mortality among patients with chronic lymphocytic leukemia (CLL) due to immune dysfunction and cytotoxic CLL treatment. Yet, predictive models for infection are missing. In this work, we develop the CLL Treatment-Infection Model (CLL-TIM) that identifies patients at risk of infection or CLL treatment within 2 years of diagnosis as validated on both internal and external cohorts. CLL-TIM is an ensemble algorithm composed of 28 machine learning algorithms based on data from 4,149 patients with CLL. The model is capable of dealing with heterogeneous data, including the high rates of missing data to be expected in the real-world setting, with a precision of 72% and a recall of 75%. To address concerns regarding the use of complex machine learning algorithms in the clinic, for each patient with CLL, CLL-TIM provides explainable predictions through uncertainty estimates and personalized risk factors.
This causes hypogammaglobinemia, pseudo-exhausted immune function, and cytokine changes that may be partially reversed by targeted therapies  12, [17] [18] [19]. If patients at high risk of infection or CLL treatment could be identified at time of diagnosis, targeted therapies modulating the immune dysfunction among these high-risk patients could be prospectively tested in a clinical trial. Thus, we developed the CLL Treatment-Infection Model (CLL-TIM), to select patients for a randomized clinical trial (PreVent-ACaLL, NCT03868722), investigating whether three months of venetoclax and acalabrutinib combination therapy can improve the natural history of immune dysfunction due to CLL. To our knowledge, this is the first time a machine learning model will be used for patient selection in a randomized clinical trial.
05 with one-tailed Wilcoxon signed rank test, Supplementary Fig. 7 ). We assembled five datasets on 4149 CLL patients from the Nationwide Danish CLL registry, the Danish Microbiology Database, the Persimune data warehouse and health registries. c Using the Bag-Of-Words (BOW) approach, we modeled the frequency of occurrence of 216 diagnoses, 153 pathologies, and 46 microbiology findings (including 9 blood culture findings). We modeled the distribution of past infections and laboratory test results and designed latent features that model urgency of patient's condition and patient symptoms as interpreted by the treating physician. d Generation of a single base-learner (single outlook) required the random selection of: a machine learning algorithm; hyper-parameters; a target outcome and feature selection.
e. those found in <1% of the CLL training cohort) and inflammation events (pathology diagnoses). ECOG -Eastern cooperative oncology group, IGHV-The immunoglobulin heavy chain gene. Β-2M -Beta-2 microglobulin, TIBC -Total iron-binding capacity, MCV -Mean cell volume, ECOG -Eastern Cooperative Oncology Group. better than CLL-IPI. CLL-TIM met this robustness condition under the unavailability of all laboratory test variables, pathology data, blood culture data or diagnosis data; even when set to missing all togetherhence, even on predictions for patients with only baseline data, CLL-TIM was still superior to CLL-IPI (Supplementary Fig. 8 ). Robustness to missing data on external CLL7 test cohort. Next, we gathered data from the German phase 3 CLL7 watch and wait cohort (NCT00275054; n = 365 patients) as a real-world validation of CLL-TIM's robustness to missing data.
We also expect that PERSIMUNE data lake will provide the basis for automating and integrating CLL-TIM directly into the medical record, further integrating medical AI. The next step is using CLL-TIM for patient selection in the investigator-initiated, randomized phase 2-3 trial within the Nordic and the Hovon CLL study groups, the PreVent-ACaLL trial (clinicaltrials. gov: NCT03868722). This trial investigates whether three months of acalabrutinib and venetoclax combination targeted treatment can improve the grade ≥3 infection-free, treatment-free survival compared to the standard-of-care observation arm of the study. Addressing the lack of prospective validation for machine learning in the clinic  57  , through the PreVent-ACaLL trial we will also be able to further assess the predictive performance of CLL-TIM, as the observation arm and the patients predicted with low risk or low confidence predictions will be followed for infections and CLL treatment.
Methods Data sources. To model multiple aspects of each patient, we assembled data from several sources (Fig. 1a ) into five datasets. (1) Baseline variables taken at time of diagnosis (±3 months). We extracted these variables from the Danish National CLL registry and include age, gender, Binet stage, family history of CLL, Eastern Cooperative Oncology Group (ECOG) performance status, β2-microglobulin levels, CD38 positivity, 70 kDa ζ-associated protein (ZAP70) positivity, IGHV status, FISH for del(13q), tri (12), del(11q) and del(17p) (Supplementary Table  25 ). (2) Routine lab; Routine laboratory tests including all available biochemical blood work-up such as hemoglobin levels, complete blood counts, and C-reactive protein levels. (3) Microbiology including blood culture findings from the Persimune Microbiology Database 55.
Given that we take the average probabilistic output over all base-learners, this means that we are implicitly down-weighting base-learner predictions with missing data. Benchmark models. CLL-TIM was benchmarked against the current gold standard in CLL prognostication, CLL-IPI  11, 65. The protocol used to generate CLL-TIM (i. e. data-driven strategy that models patient data prior to CLL diagnosis and predicts the composite outcome) was compared to five other protocols of generating ensembles. Details and motivations of all benchmark models and protocols are provided in Supplementary Table  5. The protocols include: ensembles trained to predict CLL treatment as an outcome and those that predict infection as an outcome. This enabled us to evaluate the effect of modeling the composite outcome of CLL treatment and infection, both separately and jointly.
In addition, other protocols included: ensembles comparing the data-driven approach of CLL-TIM to that of two experienced physicians pre-selecting variables; ensembles comparing the effect of modeling data prior to CLL diagnosis; and ensembles using only CLL-IPI variables that are trained to predict the composite outcome. Evaluation metrics. We assessed the 2-year composite outcome of risk and/or infection using Kaplan-Meier and Aalen-Johansen plots. As 26% of the patients in the training cohort had the composite outcome (Supplementary Fig. 1 ), the classimbalance renders common evaluation criteria such as accuracy and the Area-Under-Curve (AUC) of the receiver-operating-characteristics (ROC) to have overoptimistic results  30. We thus opted for metrics that are insensitive to classimbalance, namely the Matthews Correlation Coefficient (MCC): MCC ¼ TP TN À FP FN ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðTP þ FPÞðTP þ FNÞðTN þ FPÞðTN þ FNÞ p ð1Þ In detail, we calculated MCC using correct high-risk and low-risk predictions as true positives (TP) and true negatives (TN), respectively, while incorrect high-risk and low-risk predictions were used as false positives (FP) and false negatives (FN), respectively.
The PR-AUC calculates the precision and recall of the algorithm at all possible thresholds from the continuous values generated by CLL-TIM. When used for the CLL-IPI  11  , the precision and recall were calculated for each possible CLL-IPI score of 0-10 points. This renders comparison of two algorithms without any bias that is introduced when committing to a single threshold. For the above metrics, we generated confidence intervals (CIs) using 5,000 bootstrap sampled datasets of the internal test cohort and the CLL7 external cohort  41, 66. When algorithm variations generated predictions on different subsets from these cohorts, a new set of 5,000 bootstraps were generated. Sampling was performed with replacement and stratified to preserve original high-risk to low-risk patient ratios within each bootstrapped dataset.
We then generated CIs of 95% for all metrics over the bootstrapped datasets. We performed comparison of CLL-TIM to CLL-IPI score models using one-tailed Wilcoxon signed-rank test on the difference in means of MCC and PR-AUC over the bootstrapped datasets. For comparison of models, which predicted on different subsets of the cohort like CLL-TIM High-Confidence (HC) and CLL-IPI NIH_7+/ CLL-IPI NI_4+, we performed a one-tailed Mann-Whitney U-Test on the difference in means of MCC and PR-AUC over the subsets of the bootstrapped datasets. As mentioned, we also generated ensemble models with different protocols than CLL-TIM (ENS-COMP DC , ENS-COMP 3m , ENS-COMP CLL-IPI ). For comparison of the protocol that generated CLL-TIM (ENS-COMP) to the different protocols used as benchmarks (ENS-COMP DC , ENS-COMP 3m , ENS-COMP CLL-IPI ) we used a onetailed Wilcoxon signed rank test on the difference in 29 mean MCCs.
We generated 29 ensembles (Size 2-30 base-learners) for each protocol, and pairing for the Wilcoxon signed rank test was performed with ensemble size. Clinical trial requirements. In parallel with development of CLL-TIM, we aimed at implementing CLL-TIM for the selection of patients for a clinical phase II trial of preemptive BTK and BCL2 inhibition for newly diagnosed patients with CLL at high-risk of infection or treatment (PreVent-ACaLL, NCT03868722). To ensure a clinically meaningful hypothesis to be tested in the trial, the following requirements for the CLL-TIM algorithm were predefined: (i) High-risk patients should have at least a 65% 2-year risk of infection or CLL treatment, and only patients predicted with high-confidence to be high-risk should be eligible for enrollment. At least 20% of patients should fulfilling these criteria.
Acknowledgements This work is in part supported by funding from the  Novo Nordisk Foundation , grant  NNF16OC0019302  and the  Danish National Research Foundation  grant  126. This work was initiated during a Modeling Immune System and  Pathogen Camp  (MispCamp). The authors thank the Danish hematology centers that participated with data submission to the  Danish National CLL Registry. The following physicians contributed to data collection and represent the Danish Hematology centers participating in the Danish National CLL Registry:  Christian Hartmann Geisler ,  Lisbeth Enggaard ,  Christian Bjørn Poulsen ,  Peter de Nully Brown ,  Henrik Frederiksen ,  Olav Jonas Bergmann ,  Elisa Jacobsen Pulczynski ,  Robert Schou Pedersen ,  Ilse Christiansen , and  Linda Højberg Nielsen.
Spectral Data Processing The initial processing stage of Method3 consists in the MRR spectral data processing and follows the flow chart described in Figure  1. The first step is to transform the original signal backscattered by hydrometeors to spectral reflectivity ( ) following the equation proposed by the manufacturer (Metek  [37] ): 𝜂(𝑛, 𝑖) = 𝑓(𝑛, 𝑖) • ( ) • •∆ , ( 1 ) where i is the range gate number (i = 0, …, 31), n is Doppler bin number (n = 0, …, 63), f(n,i) is the original MRR signal saved in the so-called raw data files, TF(i) is a transfer function specific for each height, C is the radar calibration constant and Δh is the range resolution in m. The spectral reflectivity (n,i) has units of m -1 and TF(i) and C are stored in the original raw data files. The innovations introduced in Method3 are the modification of the integration time, the average of spectra and the determination of maxima (peaks) in the signal.
Parameters Calculation Once Method3 spectral processing is completed, data are ready to proceed with the calculation of subsequent parameters, which is divided in two parts. The first part computes the basic Doppler radar parameters: the Doppler velocity, which is assumed here to be the hydrometeor fall speed, and the radar equivalent reflectivity factor. The second part provides an estimation of hydrometeor type and based on this, the calculation of derived parameters, such as precipitation rates for different precipitation types. Figure  4  provides an overview of parameters calculations.
Case Study The performance of the new methodology proposed (Method3) is assessed during a precipitation event, mostly stratiform, that took place on 27 March 2017. The event produced rainfall at Das AWS (4. 9 mm) and 6. 3 mm of equivalent rainfall amount, which fell as snow, at Malniu AWS located 1100 m aloft. These precipitation amounts are relatively modest in terms of daily amounts for the season and region (Gonzalez and Bech  [55] ). As the freezing level was about 750 m above ground, a substantial part of the profile observed was snow, which allows us to illustrate different features of Method3. The results are compared with Method1, Method2 and with data from other instruments-see Supplementary Materials Table  S1. Comparisons may include different data subsets; for example, Method2 vs.
Both data sets compare reasonably well for speeds up to 8 m/s, and particularly well for 3 to 6 m/s fall speeds (note the higher density of data for that range, as shown in Figure  7a ). All Method3 speeds are below 8 m/s, whereas disdrometer data exceed 12 m/s, so major discrepancies occur for disdrometer speeds above 8 m/s. This discrepancy could be partly due to raindrop coalescence in the lowest 100 m above ground level. principle of each instrument and also the different heights compared. Both data sets compare reasonably well for speeds up to 8 m/s, and particularly well for 3 to 6 m/s fall speeds (note the higher density of data for that range, as shown in Figure  7a ). All Method3 speeds are below 8 m/s, whereas disdrometer data exceed 12 m/s, so major discrepancies occur for disdrometer speeds above 8 m/s.
This discrepancy could be partly due to raindrop coalescence in the lowest 100 m above ground level. An analysis of the first three height bins obtained with Method3 is performed with a comparison with Method1 (Figure  7b ). Note that Method2 cannot be included in this comparison as it discards the two lowest height bins. All negative fall speeds detected by Method1 are discarded according to the manufacturer approach, which assumes only positive values. The agreement found is generally good, with a few cases where Method3 overestimates Method1. A comparison of profiles except for the lowest two bins not processed by Method2 is performed between Method2 and Method3 (Figure  7c ). As displayed by the data density the fall speed agreement is generally good (R 2 of 0. 995); however, a few cases present discrepancies, typical of the order of 1 m/s.
It displays the first (lowest) and third height bin provided by Method3 (which is the first bin available for Method2) and the AWS and disdrometer rain rates. It shows a substantial agreement between the disdrometer and the AWS data and some variability when compared to Method3 rain rates. Another comparison is performed considering 30 min averages, which is the AWS time resolution, shown in Figure  12. It displays the first (lowest) and third height bin provided by Method3 (which is the first bin available for Method2) and the AWS and disdrometer rain rates. It shows a substantial agreement between the disdrometer and the AWS data and some variability when compared to Method3 rain rates. Another comparison is performed considering 30 min averages, which is the AWS time resolution, shown in Figure  12.
Verification Data and Methodology The quality of Method3 hydrometeor classification is assessed by comparing the lowest height bin (from 100 to 200 m a. g. l. ) with Parsivel present weather precipitation type at 1 min intervals. This high temporal resolution may easily introduce double penalty effects in the case of rapidly changing precipitation types, so a fuzzy verification approach is required considering neighborhoods either on the observations or on the forecast-see, for example, Ebert  [57]  or Trapero et al. [58]  for two dimensional fuzzy verification procedures. Here, this is evaluated as a one-dimensional data set so the neighborhood is simply a time interval around the observation time. some periods of convective rain. Figure  13b  shows a similar scatterplot comparing the lowest MRR height bin (100 m a.
Verification Data and Methodology The quality of Method3 hydrometeor classification is assessed by comparing the lowest height bin (from 100 to 200 m a. g. l. ) with Parsivel present weather precipitation type at 1 min intervals. This high temporal resolution may easily introduce double penalty effects in the case of rapidly changing precipitation types, so a fuzzy verification approach is required considering neighborhoods either on the observations or on the forecast-see, for example, Ebert  [57]  or Trapero et al. [58]  for two dimensional fuzzy verification procedures. Here, this is evaluated as a one-dimensional data set so the neighborhood is simply a time interval around the observation time. To choose the interval length, two aspects are considered. First, the time required for hydrometeor particles to reach the ground due to their fall speed.
A data set of 32 different days from January 2017 to October 2019 was selected (45,384 min), representing a wide variety of precipitation types and coverage of every season according to local climatology. The seasonal day distribution was: 5 winter days, 19 in spring, 4 in summer and 4 in autumn. Two examples of verification days are shown in Figure  14 , a ground level transition from rain to snow, and a warm season convective event with some minutes of hail. Verification scores based on a contingency table were calculated for each individual hydrometeor type, in particular the Probability of Detection (POD), the False Alarm Rate (FAR) and the Odds Ratio Skill Score (ORSS), which assess how good a forecast is compared to random chance (see Appendix A).