dataesr/scientific-paragraphs-categorization huggingface dataset:
False positives Sensitivity Specificity F-Score   Furthermore, the same observations can be drawn from datasets filtered with the criterion of a minimum of 2 out of 3 observed values in each group for the Arabidopsis thaliana + UPS1 experiment (Tables  S8. 19 and S8. 21) as well as for the Saccharomyces cerevisiae + UPS1 experiment (Table  S8. 24). These observations translate a loss of global information in the dataset, as filtering criteria lead to fewer peptides considered with fewer missing values per peptide. The mi4p methodology also provides better results at the protein-level (after aggregation) in terms of specificity, precision, F -score and Matthews correlation coefficient, with a minor loss in sensitivity (Table  S8. 25). In particular, a decrease of 63. 2% to 80% in the number of false positives is observed with a lower loss on the number of true positives and on sensitivity (up to 2.
Conclusion In this work, we presented as a key step of a workflow a rigorous multiple imputation method by estimating both the parameters of interest and their variability. We considered this variability downstream of the statistical analysis by including it in the moderated t-test statistic. The methodology was implemented in the R statistical language through a package called mi4p. Its performance was compared on both simulated and real datasets to the state-of-the-art methodologies, such as the package DAPAR, using confusion matrix-based indicators. The results showed a trade-off between those indicators. In real datasets, the methodology reduces the number of false positives in exchange for a minor reduction of the number of true positives. The results are similar among all imputation methods considered, especially when the proportion of missing values is small.
S2 Aggregation of peptides' intensities The methodology implemented in the mi4p R package can be applied to peptidelevel quantification data as well as protein-level quantification data. However, we were interested in evaluating our method on a peptide-level dataset and inferring results at a protein level, as it is common practice in proteomics. Therefore, for intensity aggregation, we chose to sum all unique peptides' intensities for each protein. We then adjusted our pipeline as follows: 1. Out-filtration of non-unique peptides from the peptide-level quantification dataset. 2. Normalisation of the log2-transformed peptide intensities. 3. Multiple imputation of log2-transformed peptide intensities. 4. Aggregation by summing all peptides intensities (non-log2-transformed) from a given protein in each imputed dataset.
S4. 1 Simulation design We considered an experimental design where the distributions of the two groups to be compared scarcely overlap. This design led to a fixed effect one-way ANOVA model, which can be written as: y ij = µ + δ ij + ij (6) with µ = 100, δ ij = 100 if 1 ≤ i ≤ 10 and j = 2 and δ ij = 0 otherwise and ijk ∼ N (0, 1). Here, y ij represents the log-transformed abundance of peptide i in the j-th sample. Thus, we generated 100 datasets by considering 200 individuals and 10 variables, divided into 2 groups of 5 variables, using the following steps: 1. For the first 10 rows of the data frame, set as differentially expressed, draw the first 5 observations (first group) from a Gaussian distribution with a mean of 100 and a standard deviation of 1. Then draw the remaining 5 observations (second group) from a Gaussian distribution with a mean of 200 and a standard deviation of 1.
S4. 2 Performance evaluation This subsection provides the evaluation of the mi4p workflow compared to the DAPAR workflow on the first set of simulations. The performance is described using the indicators detailed in Section S3. Figure  S4. 1: Distribution of the difference of performance between mi4p and DAPAR workflows on the first set of simulations imputed using maximum likelihood estimation. The following tables provide results expressed as the mean of the given indicator over the 100 simulated datasets ± the mean of the standard deviations of the given indicator over the 100 simulated datasets. Results are based on adjusted p-values using the Benjamini-Hochberg procedure  [Benjamini and Hochberg, 1995]  and a false discovery rate of 1%. 186. 8 ± 2 0 ± 0 100 ± 0 98. 6 ± 0. 9 80. 5 ± 10.
S5. 1 Simulation design Secondly, we considered an experimental design, where the distributions of the two groups to be compared might highly overlap. Hence, we based it on the random hierarchical ANOVA model by  Lazar et al. [2016] , derived from  Karpievitch et al. [2012]. The simulation design follows the following model: y ij = P i + G ik + ijk (7) where y ij is the log-transformed abundance of peptide i in the j-th sample, P i is the mean value of peptide i, G ik is the mean difference between the condition groups, and ij is the random error term, which stands for the peptide-wise variance. We generated 100 datasets by considering 1000 individuals and 20 variables, divided into 2 groups of 10 variables, using the following steps: 1. Generate the peptide-wise effect P i by drawing 1000 observations from a Gaussian distribution with a mean of 1.
5 and a standard deviation of 0. 5. 2. Generate the group effect G ik by drawing 200 observations (for the 200 individuals set as differentially expressed) from a Gaussian distribution with a mean of 1. 5 and a standard deviation of 0. 5 and 800 observations fixed to 0. 3. Build the first group dataset by replicating 10 times the sum of P i and the random error term, drawn from a Gaussian distribution of mean 0 and standard deviation 0. 5. 4. Build the second group dataset by replicating 10 times the sum of P i , G ik and the random error term drawn from a Gaussian distribution of mean 0 and standard deviation 0. 5. 5. Bind both datasets to get the complete dataset.
S5. 2 Performance evaluation This subsection provides the evaluation of the mi4p workflow compared to the DAPAR workflow on the second set of simulations. The performance is described using the indicators detailed in Section S3. Figure  S5. 2: Distribution of the difference of performance between mi4p and DAPAR workflows on the second set of simulations imputed using maximum likelihood estimation. The following tables provide results expressed as the mean of the given indicator over the 100 simulated datasets ± the mean of the standard deviations of the given indicator over the 100 simulated datasets. Results are based on adjusted p-values using the Benjamini-Hochberg procedure  [Benjamini and Hochberg, 1995]  and a false discovery rate of 1%. 170. 3 ± 4. 9 5. 9 ± 2. 9 794. 1 ± 2. 9 29. 7 ± 4.
S6. 1 Simulation design Finally, we considered an experimental design similar to the second one, but with random effects P i and G ik. The 100 datasets were generated as follows. 1. For the first group, replicate 10 times (for the 10 variables in this group) a draw from a mixture of 2 Gaussian distributions. The first one has the following parameters: a mean of 1. 5 and a standard deviation of 0. 5 (corresponds to P i ). The second one has the following parameters: a mean of 0 and a standard deviation of 0. 5 (corresponds to ij ). 2. For the second group replicate 10 times (for the 10 variables in this group) a draw from a mixture of the following 3 distributions. (a) The first one is a Gaussian distribution with the following parameters: a mean of 1. 5 and a standard deviation of 0. 5 (corresponds to P i ).
S6. 2 Performance evaluation This subsection provides the evaluation of the mi4p workflow compared to the DAPAR workflow on the first set of simulations. The performance is described using the indicators detailed in Section S3. Figure  S6. 3: Distribution of the difference of performance between mi4p and DAPAR workflows on the third set of simulations imputed using maximum likelihood estimation. The following tables provide results expressed as the mean of the given indicator over the 100 simulated datasets ± the mean of the standard deviations of the given indicator over the 100 simulated datasets. Results are based on adjusted p-values using the Benjamini-Hochberg procedure  [Benjamini and Hochberg, 1995]  and a false discovery rate of 1%. Table  S6. 16: Performance evaluation on the third set of simulation imputed using random forests.
S7 Real datasets generation S7. 1 Complex total cell lysates (Saccharomyces cerevisiae and Arabidopsis thaliana) spiked UPS1 standard protein mixtures We consider a first real dataset from  Muller et al. [2016]. The experiment involved six peptide mixtures, composed of a constant yeast (Saccharomyces cerevisiae) background, into which increasing amounts of UPS1 standard proteins mixtures (Sigma) were spiked at 0. 5, 1, 2. 5, 5, 10 and 25 fmol, respectively. In a second well-calibrated dataset, yeast was replaced by a more complex total lysate of Arabidopsis thaliana in which UPS1 was spiked in 7 different amounts, namely 0. 05, 0. 25, 0. 5, 1. 25, 2. 5, 5 and 10 fmol. For each mixture, technical triplicates were constituted. The Saccharomyces cerevisiae dataset was acquired on a nanoLC-MS/MS coupling composed of nanoAcquity UPLC device (Waters) coupled to a Q-Exactive Plus mass spectrometer (Thermo Scientific, Bremen, Germany) as extensively described in  Muller et al.
S7. 2 Data preprocessing For the Saccharomyces cerevisiae and Arabidopsis thaliana datasets, Maxquant software was used to identify peptides and derive extracted ion chromatograms. Peaks were assigned with the Andromeda search engine with full trypsin specificity. The database used for the searches was concatenated in house with the Saccharomyces cerevisiae entries extracted from the UniProtKB-SwissProt database (16 April 2015, 7806 entries) or the Arabidopsis thaliana entries (09 April 2019, 15 818 entries) and those of the UPS1 proteins (48 entries). The minimum peptide length required was seven amino acids and a maximum of one missed cleavage was allowed. Default mass tolerances parameters were used. The maximum false discovery rate was 1% at peptide and protein levels with the use of a decoy strategy.
For the Arabidopsis thaliana + UPS1 experiment, data were extracted both with and without Match Between Runs and 2 prefiltering criteria were applied prior to statistical analysis: only peptides with at least 1 out of 3 quantified values in each condition on one hand and 2 out of 3 on the other hand were kept. Thus, 4 datasets derived from the Arabidopsis thaliana + UPS1 were considered. For the Saccharomyces cerevisiae + UPS1 experiment, the same filtering criteria were applied, but only on data extracted with Match Between Runs, leading to 2 datasets considered.
S7. 3 Supplemental methods for Arabidopsis thaliana dataset Peptide separation was performed on an ACQUITY UPLC BEH130 C18 column (250 mm × 75 µm with 1. 7 µm diameter particles) and a Symmetry C18 precolumn (20 mm ×180 µm with 5 µm diameter particles; Waters). The solvent system consisted of 0. 1% FA in water (solvent A) and 0. 1% FA in ACN (solvent B). The samples were loaded into the enrichment column over 3 min at 5 µL/min with 99% of solvent A and 1% of solvent B. The peptides were eluted at 400 nL/min with the following gradient of solvent B: from 3 to 20% over 63 min, 20 to 40% over 19 min, and 40 to 90% over 1 min. The MS capillary voltage was set to 2 kV at 250 °C. The system was operated in a data-dependent acquisition mode with automatic switching between MS (mass range 375-1500 m/z with R = 120 000, automatic gain control fixed at 3 × 106 ions, and a maximum injection time set at 60 ms) and MS/MS (mass range 200-2000 m/z with R = 15 000, automatic gain control fixed at 1× 105, and the maximal injection time set to 60 ms) modes.
Complex total cell lysates (Saccharomyces cerevisiae and Arabidopsis thaliana) spiked UPS1 standard protein mixtures. S7. 2 Data preprocessing. S7. 3 Supplemental methods for Arabidopsis thaliana dataset. Saccharomyces cerevisiae + UPS1 experiment. Distribution of the difference of performance between mi4p and DAPAR workflows on the first set of simulations imputed using maximum likelihood estimation. S5. 2 Distribution of the difference of performance between mi4p and DAPAR workflows on the second set of simulations imputed using maximum likelihood estimation. S6. 3 Distribution of the difference of performance between mi4p and DAPAR workflows on the third set of simulations imputed using maximum likelihood estimation. State of the art on imputation methods used in quantitative proteomics and type of data used.
S5. 10 Performance evaluation on the second set of simulations imputed using principal component analysis. S5. 11 Performance evaluation on the second set of simulations imputed using random forests. S6. 12 Performance evaluation on the third set of simulation imputed using maximum likelihood estimation. S6. 13 Performance evaluation on the third set of simulations imputed using k-nearest neighbours method. S6. 14 Performance evaluation on the third set of simulation imputed using Bayesian linear regression. S6. 15 Performance evaluation on the third set of simulation imputed using principal component analysis. S6. 16 Performance evaluation on the third set of simulation imputed using random forests. S8. 17 Performance evaluation on the Arabidopsis thaliana + UPS1 dataset, filtered with at least 1 quantified value in each condition.
S8. 18 Performance evaluation on the Arabidopsis thaliana + UPS1 dataset, filtered with at least 1 quantified value in each condition and focusing only on the comparison 5fmol vs. 10fmol. S8. 19 Performance evaluation on the Arabidopsis thaliana + UPS1 dataset, filtered with at least 2 quantified values in each condition. S8. 20 Performance evaluation on the Arabidopsis thaliana + UPS1 dataset, extracted without Match Between Runs and filtered with at least 1 quantified value in each condition. S8. 21 Performance evaluation on the Arabidopsis thaliana + UPS1 dataset, extracted without Match Between Runs and filtered with at least 2 quantified values in each condition. S8. 22 Performance evaluation on the Arabidopsis thaliana + UPS1 dataset at the protein-level, filtered with at least 1 quantified values in each condition.
S8. 23 Performance evaluation on the Saccharomyces cerevisiae + UPS1 dataset, filtered with at least 1 quantified value in each condition. S8. 24 Performance evaluation on the Saccharomyces cerevisiae + UPS1 dataset, filtered with at least 2 quantified values in each condition. S8. 25 Performance evaluation on the Saccharomyces cerevisiae + UPS1 dataset, at the protein-level and filtered with at least 1 quantified values in each condition.
Acknowledgements The authors wish to thank  Leslie Muller  and  Nicolas Pythoud  for providing the real proteomics datasets used in this work, as well as  Thomas Burger  and  Quentin Giai-Gianetto  for their help on the DAPAR and imp4p R packages. The real datasets were deposited with the ProteomeXchange Consortium via the PRIDE partner repository with the dataset identifiers  PXD003841  and  PXD027800  (Deutsch et al. , 2017).
S8. 23 Performance evaluation on the Saccharomyces cerevisiae + UPS1 dataset, filtered with at least 1 quantified value in each condition. S8. 24 Performance evaluation on the Saccharomyces cerevisiae + UPS1 dataset, filtered with at least 2 quantified values in each condition. S8. 25 Performance evaluation on the Saccharomyces cerevisiae + UPS1 dataset, at the protein-level and filtered with at least 1 quantified values in each condition. Tyanova, S. et al. (2016) The Perseus computational platform for comprehensive analysis of (prote)omics data. Nature Methods, 13(9), 731-740. Van Buuren, S. and Groothuis-Oudshoorn, K. (2011). mice: Multivariate Im-putation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. Wang, J. et al. (2017) In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values.
Scientific Reports, 7. Wang, M. et al. (2021). RobNorm: model-based robust normalization method for labeled quantitative mass spectrometry proteomics data. Bioinformatics, 37(6), 815-821. Supplemental Information for Accounting for multiple imputation-induced variability for differential analysis in mass spectrometry-based label-free quantitative proteomics Marie Chion, Christine Carapito & Frédéric Bertrand Contents S7 Real datasets generation S7. 1 S8 Results on real datasets S8. 1 Arabidopsis thaliana + UPS1 experiment. S4. 1 List of Tables S8. 2 List of Figures S1. 1 White, I. R. ,Royston, P. and Wood, A. M. (2011)Multiple imputation using chained equations: Issues and guidance for practice. Statistics in Medicine, 30, 377-399. Wieczorek, S. et al. (2017) DAPAR & ProStaR: software to perform statistical analyses in quantitative discovery proteomics.
To address the limits of IForest, we performed different experiments on commonly used real datasets and also on synthetic datasets with non trivial distributions. We designed multidimensional datasets where anomalies are carried by several dimensions simultaneously. Moreover, we used a varying density and distance between anomalies and normal data, for a variable similarity between these two data classes. We compared the performance of IForest against its improved version called Extended IForest. Finally, we designed and validated a new extension of IForest, based on the different individual trees decisions instead of a global forest decision that we call Majority Voting IForest (MVIForest). The experiments show that MVIForest has a shorter execution time than IForest, with almost the same accuracy.
Isolation Forest (IForest) is an anomaly detection method based on a different concept compared to the approaches presented above (statistical, clustering, nearest neighbors, etc. ). It does not compute any distance or density and therefore significantly reduces execution time and memory requirement. IForest was designed by  Liu et al. in 2008 [22] ,  [24]. Based on binary trees, it successively splits data and identifies anomalies as data that get rapidly isolated in the trees (compared to the majority of data). Isolation Forest is an unsupervised method that gives excellent results in terms of accuracy with a linear temporal complexity and a low constant memory requirement. In  [17] , authors proposed a new version of IForest called Extended Isolation Forest (EIF) that enhances the way to split data for a more accurate detection.
In order to design and develop later a distributed version of IForest adapted to streaming context, we first propose in this paper a complete and deep study of this method. It is a necessary step to be able to propose adaptations of IForest to the context of streams for a distributed version. The objective of this paper is to study, in depth, Isolation Forest and to evaluate the impact of its different parameters on its performances and the efficiency of the detection. We particularly focus on non-trivial anomalies, in multidimensional datasets to explore the limits of Isolation Forest. We compared Isolation Forest to its extended version (EIF) that enhances false positives but with a longer execution time. We also proposed a new version that we called Majority Voting Isolation Forest (MVIForest) providing the same accuracy as Isolation Forest with a shorter execution time.
To compare these three algorithms, we performed several experiments on both synthetic and real datasets, commonly used in the anomaly detection area, and having different characteristics. This paper is organized as follows: Section II presents principles and advantages of Isolation Forest. In Section III, we provide a survey on anomaly detection based on Isolation. In Section IV, we focus on EIF method. We present different datasets and metrics considered to evaluate and compare the different approaches in Section V. Section VI highlights a study of performances of IForest method. In Section VII, we propose and evaluate an improvement of IForest called MVIForest. We discuss the experimental evaluations of presented methods in Section VIII. Finally, Section IX concludes the work by providing future research directions.
II. ISOLATION FOREST: PRINCIPLE AND ADVANTAGES Anomalies are considered as patterns in data that do not conform to a well defined model of normal patterns. Isolation is the process of separating such unusual patterns from the entire dataset. Datasets are often unbalanced, containing much more normal data than anomalies, the anomaly being the exception. The isolation technique is based on two following characteristics of anomalous data: • Anomalies represent a very small proportion of the dataset. • Anomalies are distinct. Anomalies have a different behaviour or characteristics compared to normal data. Moreover, they are easy to identify when they are distinct from each other, which avoids the masking effect. A direct consequence of these two characteristics is that anomalous data are easier to isolate than normal data.
The majority of existing anomaly detection approaches construct a model from existing data, either based on knowledge acquired from unlabeled data for unsupervised methods, or using labels provided by an administrator (prior identification of a class) for semi-supervised and supervised methods. Modeling the behavior of normal (majority) data enables the identification of the abnormal data as elements that do not respect the standard behavior. Some approaches calculate the distance or density of the different elements in the dataset and identify anomalies as elements with a distance exceeding a given threshold. These methods provide good detection results but often suffer from a lack of scalability depending on the dimension or size of the dataset. Indeed, in case of a massive dataset, the execution time and memory requirement of such a method can quickly reach its limits.
In general, nearest neighbor anomaly detection methods like LOF  [3] , k-NN  [35] , etc. have a quadratic complexity O(n 2 )  [15] , as they are based on distance computation between each pair of data. The response time of these methods is also not adapted to the real-time processing of data streams produced today by various systems and sensors with an always increasing arrival rate. The calculation of all the distances needed to design the model and detect the anomalies can be difficult to achieve in such conditions. Isolation Forest is an anomaly detection method based on an approach different from the others (statistics, clustering, nearest neighbors, etc. ). It does not calculate either distance or density and therefore significantly reduces execution time and memory requirement. IForest has a low and small memory requirement  [22]  and a linear execution time, proportional to the size of dataset (section VI-C).
It has an excellent scalability that makes this method suitable for large datasets, as well as for real-time processing. Anomalies are rare and have a different behavior compared to normal data. Figure  1  shows an example of anomalous data (X 0 ) and normal data (X i ). Isolation is performed by successive splits of the dataset. One can notice that anomalous data are easier to isolate than normal data. X 0 is isolated in 3 steps whereas it took 11 steps to isolate the normal data (X i ). IForest (  [22] ,  [24] ) is the first method that has been proposed in the category of isolation based anomaly detection. This method uses a set of random and independent trees (itree) called a random forest. IForest generates a score for each data using all the trees in the forest. IForest uses two input parameters to calculate the data score.
The parameters are ψ which is the size of the randomly chosen sample from the entire dataset and t which is the number of trees in the forest. Each tree is built independently by sampling the dataset, thus, the number of trees corresponds to the number of samples. IForest has two stages: the training phase which essentially corresponds to the construction of the forest and the so-called scoring phase where the score is generated for each item in the dataset. Let X (n, m) ⊂ R m be a dataset containing anomalies, n denotes the number of data in X and m the dimension of the data.