dataesr/scientific-paragraphs-categorization huggingface dataset:
This inherently limits the data cardinality and consequently reduces the discriminative power of subcellular niches with similar profiles, as illustrated in Fig. 3. These protein profiles represent well-characterized mitochondrial and endoplasmic reticulum residents from two different experiments, namely, a TMT 6-plex (red) and a labelfree experimental (black) design, that have been fully quantified using 6 and 16 gradient fractions, respectively. The quantitation of more fractions using additional independent mass spectrometry acquisitions must be mitigated by the prevalence of incompletely characterized profiles (described in the section "Data Processing" and Fig. 4 ) due to missing values, which are rife in label-free datasets because of the semistochastic nature of data-dependent acquisition mass spectrometry approaches.
In the following sections, we use published datasets from Dunkley et al. (7)  and Tan et al. (18) , using augmented marker sets described by Breckels et al. (19) , in order to illustrate our findings. These datasets are also distributed as well-documented and computer-friendly data structures  (13)  in the pRolocdata  (11)  package and can be used as input to the software packages used to run the computational experiments described below. Both datasets mentioned above were derived through use of the LOPIT technique to localize integral and associated membrane proteins. Dunkley et al. (7)  prepared two independent Arabidopsis thaliana callus membranes and fractionated them by using self-generating iodixanol density gradients. For each preparation, two iTRAQ 4-plex tags (16) were used to quantify seven different fractions, with one fraction that was the same in both acquisitions.
Firstly, fraction 1 (least dense) was labeled with reagent 114, fraction 4 with 115, and fraction 7 with 116, and fractions 11 and 12 were pooled and labeled with 117. Then, fraction 2 was labeled with iTRAQ reagent 114, fraction 5 with 115, fraction 8 with 116, and fractions 11 and 12 with 117. The total experiment yielded replicated measurements for eight quantitation values along the gradient. Labeled peptides were separated using strong cation ex-FIG. 2. Left, representation of a fully described spatial proteomics dataset containing quantitative data for n proteins along m fractions. Each protein is described by additional metadata, in particular the definition of the known subcellular localization for well-known residents. Fractions are also decorated with specific metadata. Right, summarization of the quantitative data and annotation of the "markers" protein metadata using a principal component analysis figure.
FIG. 3. Effect of the number of fractions on the resolution along the separation gradient for the endoplasmic reticulum (ER) and mitochondria from a TMT experiment using 6 fractions (red) and the equivalent label-free experiment quantified along 16 fractions (black). The profiles are those of carefully selected Drosophila markers that were quantified in all the fractions. 15 and 11 fully characterized mitochondrial and ER markers were identified in the TMT dataset, whereas the label-free approach identified only 9 and 5 complete profiles because of missing values. change chromatography and analyzed via LC-MS/MS on a QSTAR XL quadrupole-time-of-flight mass spectrometer (Applied Biosystems, Foster City, CA). Tan et al. (18)  collected Drosophila melanogaster embryos at 0 to 16 h. The material was homogenized and centrifuged to collect the supernatant, thereby removing cell debris and nuclei.
Membrane fractionation was performed on an iodixanol gradient, and fractions were quantified using iTRAQ 4-plex isobaric tags (16) as follows: fractions 4/5, 114; fractions 12/13, 115; fraction 19, 116; and fraction 21, 117. Labeled peptides were then separated using strong cation exchange chromatography and analyzed via LC-MS/MS on a QSTAR XL quadrupole-time-of-flight mass spectrometer (Applied Biosystems). Data Visualization-One can directly visualize the experimental data by plotting the protein intensities, or generally their relative intensities, along the separation dimension. Although very useful, this representation is limited by the number of proteins or clusters of proteins that can be represented in one figure. It is often essential to be able to visualize the full dataset in one figure, possibly in summarized form, in order to assess the underlying structure of the data and develop an intuition of what can reasonably be achieved in light of the data's properties.
A popular technique for describing multivariate data is dimensionality reduction. This allows one to represent the data in a reduced set of dimensions, generally two or three, instead of the original higher number corresponding to the individual fractions (the columns of our data matrix represented in Fig. 2 ), while maintaining as much of the initial information as possible. Principal component analysis (PCA) or nonlinear versions thereof are procedures that transform the original data into a set of orthogonal components that are ordered according to the amount of variability that they describe (Fig. 2 , right). For a well-structured dataset, representation as a projection along the two first components is often an effective means to obtain a simple yet representative visualization of the data.
Other studies have either explicitly  (22)  or implicitly  (10)  applied data imputation without necessarily assessing the biases of the procedure. To illustrate the issues of imputation of missing data, we assessed the impact of imputation using the specific goals of spatial proteomics (i. e. the identification of subcellular protein clusters and the assignment of proteins). In Fig. 4 , we present data from Ref. 7, which provides complete profiles for 16 fractions (some being replicates) for nearly 700 proteins. After random assignment of missing values and data imputation using nearest neighbor imputation  (23) , we estimated the effect of the imputation method by tracking the shift of the imputed value with respect to the original data points. As expected, we noted an increasing effect of imputation on the data with an increasing number of missing values (Fig.
From a computational point of view, markers allow the mapping of regions in the multidimensional data space to subcellular localizations (Fig. 2 ). The validity of markers, and thus the reliable mapping of biological information to the multivariate data, is generally ensured by expert curation of the proteins in the dataset. Gene Ontology (GO)  (25) , and in particular the cellular compartment (CC) namespace, is an essential starting point for protein annotation and marker definition. Nevertheless, automatic extraction of GO CC is only a first step, and additional curation is required to counter unreliable annotation based on data that are inaccurate or out of context for the biological question under investigation. Although proteins with genuine multiple localizations are of particular interest (see below), one must be careful when assessing multiple GO CC terms and distinguish proteins present in more than one subcellular niche (multilocalization) from changes in localization under different conditions and incorrect annotation.
Although marker definition is an important step, it is only the initial step of the analysis workflow and is often time consuming. To facilitate the identification of markers, we curated proteins and mined publicly available datasets  (7, 8, 18 -20, 22, 26, 27)  to provide marker sets for Arabidopsis thaliana (543 markers, 13 organelles), Drosophila melanogaster (220 markers, 11 organelles), Saccharomyces cerevisiae (128 markers, 12 organelles), Gallus gallus (102 markers, 5 organelles), mouse (648 markers, 21 organelles), and human (507 markers, 16 organelles) as part of our software infrastructure  (11)  (see details in the supplemental material). These markers are now available in the pRoloc software and can be directly added to quantitative datasets. They are provided as a starting point for the generation of reliable sets of organelle markers but still need to be verified against any new data in light of the quantitative data and the study conditions.
The application of such methods, however, is often hindered by failure to extract organelle markers that cover the whole subcellular diversity in the data; this leads to prediction errors, as protein profiles of unknown localization can only be associated with organelles that appear in the labeled training data. The extraction of all organelle and organelle-related clusters is a difficult task owing to the limited number of marker proteins that exist in databases and elsewhere and the time-consuming nature of obtaining reliable markers. To address these issues, Breckels et al. (19)  developed a novelty detection algorithm that is able to identify subcellular groupings such as organelles and protein complexes in spatial proteomics experiments. The algorithm, phenoDisco, uses a semi-supervised machine learning schema that employs iterative cluster merging combined with Gaussian mixture modeling and outlier detection to identify putative subcellular compartments.
In a semi-supervised scenario, a classifier is learned in the presence of both labeled (i. e. organelle markers) and unlabeled (i. e. proteins of unknown localization) data. In order to apply Breckels's algorithm, one requires an initial set of high-quality input organelle markers that cover a minimum of two classes, each containing six or more marker protein profiles, and of course some unlabeled data that can be mined for new phenotype organelle clusters. The choice of labeled data (i. e. organelle markers) from which the applied machine learning system will learn is extremely important, as it can have a significant effect on the success or failure of the learner. To illustrate this paradigm, we investigated the effect that marker choice has upon the application of the phenoDisco novelty detection algorithm in mining a Drosophila melanogaster dataset that had been produced using the LOPIT technology  (18).
We considered three sources of markers for use as input for the phenoDisco algorithm: (i) a highly and manually curated set from experts in the field (20 endoplasmic reticulum (ER), 6 Golgi, 14 mitochondrial, and 15 plasma membrane markers from our curated marker sets, originally obtained from Ref. 18); (ii) unique GO CC annotations assigned a localization based on experimental evidence plus those assigned a unique localization as inferred from structural sequence or similarity in the GO database; and (iii) only unique GO CC annotations assigned a localization based on experimental evidence in the GO database. Reassuringly, it was found that in case i, where a small set of manually curated markers were used as input, six out of the seven previously unlabeled phenotype clusters that were found in the phenoDisco experiments published in Ref.
19  were identified: a cluster of proteins that represented two ribosomal subunits (40S and 60S), nucleus, proteasome, lysosome, and the cytoskeleton. An additional cluster of cytoplasmic proteins was also identified (phenotype 7, Fig. 6A , right). Remarkably, we found that the use of organelle marker set ii had a detrimental effect on the ability of the algorithm to identify any new organelle clusters, and we were able to identify only one new phenotype (Fig. 6B , right). Examination of the organelle markers in set ii showed a lack of cluster resolution and overlap of the Golgi apparatus, plasma membrane, and ER (Fig. 6B , left). We also found a mitochondrial outlier in the dataset that was completely separated from the other mitochondrion markers and located toward the ER cluster. It was found that the inclusion of this one clear outlier forced a negative constraint on the phenotype modeling, which resulted in a lack of new phenotypes detected.
An important step in the application of any novelty detection algorithm is the careful examination of the protein content of any new clusters identified. Curation and examination by experts in the field is an essential step in the discovery analysis pipeline. Using such approaches, a researcher is able to mine MS datasets at a deeper level and bring to light interesting subcellular compartments for more comprehensive validation for use in a supervised machine learning analysis for robust protein localization assignment. Classification-In machine learning, the task of classification falls under the broad area of supervised learning. In supervised learning, the aim is to train a classifier to learn a mapping between a set of observed instances and a set of associated external attributes that are being predicted (usually known as the class label or predictor).
Estimation of the algorithmic performance can be assessed in many ways, such as via cross-validation. In the pRoloc package, algorithmic performance is estimated using stratified 20/80 partitioning in conjunction with 5-fold cross-validation in order to optimize the free parameters via a grid search. This procedure is usually repeated 100 times, and then the best parameters are selected upon investigation of associated macro F1 scores. A high macro F1 score indicates that the marker proteins in the test dataset are consistently correctly assigned by the algorithm. Often more than one parameter or set of parameters gives rise to the best generalization accuracy. Thus it is always important to investigate the model parameters and critically assess the best choice. The best choice might not be as simple as the parameter set that gives rise to the highest macro F1 score, and one must be careful to avoid overfitting and to choose parameters wisely.
Once the best parameters have been selected, they can be used to build a classifier from the training data of organelle markers. The classifier will return a classification result for all unlabeled instances in the dataset corresponding to their most likely subcellular compartment. In addition, it is possible to extract classification accuracy scores that can inform on the reliability of the assignment. Many supervised machine learning algorithms have been developed, some of the most popular being the support vector machine (SVM), k-nearest neighbor, random forest, neural networks, and naive Bayes, among others. These methods, along with newer state-of-theart algorithms such as the Perturbo (29) classification algorithm, are available in the pRoloc package. With the vast number of classification methods available, it is often a daunting task to choose the method that is best suited to the classification task; however, it is not often the choice of algorithm that underpins robust results.
In fact, it is widely accepted that it is not algorithm choice that matters but the way in which the algorithm is applied and the availability of good training data. As an example of an application of protein localization prediction using supervised machine learning, we took the first replicate from Tan et al. (18)  and applied a weighted SVM classifier for protein classification. The labeled training data (Fig. 7 , left) were constructed from manually curated markers from Ref. 18, which were further refined using Breckels et al. 's  (19)  phenotype discovery algorithm. Here, using the pRoloc package, we employed a weighted SVM with a Gaussian kernel to learn a nonlinear decision function on the training data to map proteins of unknown localization to one of the known organelle classes. Class specific weights were used when creating the SVM model and were set to be inversely proportional to the class frequencies to account for class imbalance.
On the training data, the two free SVM parameters, cost and sigma, were optimized over 100 rounds of stratified 5-fold cross-validation via a grid search, and the FIG. 6. The effect of different organelle marker sets (left) on the application of the novelty discovery algorithm phenoDisco (right) in mining a Drosophila melanogaster dataset produced using the LOPIT technology  (18). A, a set manually curated by experts in the field. B, unique GO CC annotations from experimental evidence or computational predictions. C, unique GO CC annotations from experimental evidence. best pair of parameters for the classifier were chosen from evaluation of the macro F1 scores (Fig. 7 , middle). The optimized SVM classifier was then used to predict protein localization on the unlabeled data (Fig. 7 , right).
We modeled changes in localizations and moved proteins from one organelle to another by updating their observed quantitative data along the gradient with new meaningful values inferred from the same dataset. Multi-localization-The protein databases provide multiple localization annotations per protein in about 60% of human UniProt entries (see the supplemental material). Although a certain number of these annotations are likely to be erroneous or represent specialization of identical compartments and do not necessarily imply that proteins multi-localize under iden- tical conditions, dual-localization is an important aspect that needs to be addressed. We used the complete data from Ref. 7 to create a variable mixture of protein relative abundances along the gradient to simulate dual-localization.
The original relative quantitation values were replaced by the mean fraction values of all destination marker proteins. The trans-localizations are highlighted by arrows in Fig. 9. Below, we demonstrate important aspects influencing the analysis of dynamic spatial proteomics data, namely, data normalization, the identification of trans-localization, and concerted trans-localizations. Data Normalization-The two replicates displayed both biological and substantial technical variability, as illustrated in Fig. 9  (left). The first and second replicates are represented by circles and diamonds, respectively, and the corresponding pairs of proteins are linked by dotted segments. The colors represent all marker proteins for the nine subcellular niches identified for this dataset. Although the mass spectrometry processing is becoming more reliable and reproducible, the density separation gradient is a sensitive operation that is executed manually.
This is a useful exercise to undertake, especially in such an emerging field as spatial proteomics data analysis. The first applications of large-scale organelle proteomics data analysis were protein correlation profiling efforts (8, 21) that calculated the 2 metric using in-house tools and LOPIT  (7, 18, 20)  that applied partial least squares discriminant analysis using the commercial SIMCA software (Umetrics, Umea, Sweden). Trotter et al. (9)  implemented custom R code  (12)  and used the SVM algorithm from the kernlab package  (33) , but no code for others to repeat this state-of-the-art procedure is provided. Others have applied other contemporary machine learning algorithms, including random forests  (10) , naive Bayes  (22) , and neutral networks  (34) , but did not provide means to apply their analyses to new data.
Although proteomics data are commonly being disseminated through appropriate repositories, it is not commonplace to provide reproducibility in terms of software and data analysis, despite their recognized importance  (35). Here we have attempted to redress access to code and the ability to reproduce data analysis by performing the analysis and creating illustrations using the R language and a set of well-documented Bioconductor (14) software add-ons specifically developed for quantitative proteomics data. The MSnbase package (13) allows the consistent management and processing of quantitative data and associated metadata, and the pRoloc package  (11)  provides a visualization and statistical machine learning (including all the algorithms mentioned above, as well as novel ones (  29 )) framework to analyze and interpret spatial proteomics data.
The software allows the implementation of a robust and reproducible analysis pipeline and is flexible enough to accommodate various designs and foster the development of innovative analysis strategies. The software provides extensive documentation and tutorials for a fully reproducible organelle proteomics framework. Finally, pRoloc benefits from the Bioconductor infrastructure and its full integration with various online resources, including, among many others, the Gene Ontology (the GO. db package  (36) ), the UniProt database (the biomaRt package (37)), and the Human Protein Atlas  (38, 39)  (the hpar package  (40) ). In this study we also sought to develop analysis pipelines that will be useful to dual-/multi-and trans-localization study designs. Such approaches build on robust single condition classification accuracy that relies on good resolution of the subcellular space to reduce inter-organelle variability (welldefined clusters) and enable reliable organelle assignments.
CONCLUSION The path to reliable data analysis results is never written in stone, in particular for complex experimental designs and multivariate data. There are, however, certain requirements that are always applicable. Visualization of the complete dataset is essential in order to describe its major features; in the case of a spatial proteomics experiment, we have highlighted multiple applications of dimensionality reduction techniques such as PCA. This is, of course, a simplification of the complete data, but it can provide a first inkling of the extent of separation and success of classification. It is also important to set basic assumptions about the data, assess the organelle markers in light of the data structure, describe how it is processed, and assess the effects of the treatments it undergoes.
L. G. , C. M. M. , and M. F. were supported by the European Union 7th Framework Program (PRIME-XS Project, Grant No. 262067). L. M. B. was supported by a BBSRC Tools and Resources Development Fund (Award No. BB/K00137X/1). T. B. was supported by the Proteomics French Infrastructure (ProFI, ANR-10-INBS-08). A. C. was supported by BBSRC Grant No. BB/D526088/1. A. J. G. was supported by BBSRC Grant No. BB/E024777/ and a generous gift from King Abdullah University for Science and Technology, Saudi Arabia. D. J. N. H. was supported by a BBSRC CASE studentship (BB/I016147/1).
* L. G. , C. M. M. , and M. F. were supported by the European Union 7th Framework Program (PRIME-XS Project, Grant No. 262067). L. M. B. was supported by a BBSRC Tools and Resources Development Fund (Award No. BB/K00137X/1). T. B. was supported by the Proteomics French Infrastructure (ProFI, ANR-10-INBS-08). A. C. was supported by BBSRC Grant No. BB/D526088/1. A. J. G. was supported by BBSRC Grant No. BB/E024777/ and a generous gift from King Abdullah University for Science and Technology, Saudi Arabia. D. J. N. H. was supported by a BBSRC CASE studentship (BB/I016147/1).
, 2013). This is of particular interest in the field of HIV-1 'reservoir' research, where accurate quantitation of proviral DNA in CD4T cells is critical to HIV-1 cure strategies  (Eriksson et al. , 2013; Finzi et al. , 1997; Wong et al. , 1997). Quantitative real-time PCR (qPCR) is used widely to quantify HIV-1 DNA levels. Since qPCR uses an indirect measurement (cycle threshold) and any assay noise is amplified exponentially, there is the potential for limitations in data accuracy and reproducibility  (Miotke et al. , 2014). Also, the reliance on logarithmic standard curves may result in a trade-off between accuracy and reproducibility and high dynamic ranges, especially where standards are not optimised between laboratories. In quantitation of HIV-1 copy number in clinical samples, ddPCR has been reported to be both as sensitive  (Kiselinova et al.