dataesr/scientific-paragraphs-categorization huggingface dataset:
Marker selection A protocol for deep sequencing and data analysis was developed for two molecular markers, namely the P. falciparum csp gene (PF3D7_0304600) and gene PF3D7_0104100, annotated in the malaria sequence database PlasmoDB as "conserved Plasmodium membrane protein" (cpmp). Results from these two markers were compared with classical length polymorphic genotyping using the highly diverse marker msp2. Sizes of msp2 fragments amplified for genotyping range from 180 to 515 bp in PNG using published primers (Additional file 1: Table  S1 ). Marker csp has been used for deep sequencing in the past  [9]  and the previously published primers (Additional file 1: Table  S1 ) were used. The csp amplicon spans the T-cell epitope of the circumsporozoite protein from nucleotide position 858 to 1186 of the 3D7 reference sequence.
The newly validated marker cpmp was identified by calculating heterozygosity in 200 bp windows of 3′411 genomic P. falciparum sequences from 23 countries available from the MalariaGEN dataset  [17]. Genes from multi-gene families or regions of poor sequence alignments, often caused by length polymorphism of intragenic tandem repeats, were excluded from the list of potential markers. A 430 bp fragment of cpmp spanning nucleotide positions 1895 to 2324 scored highest in expected heterozygosity (H e ) and was prioritized as candidate for a highly diverse amplicon sequencing marker. H e in the worldwide dataset was 0. 93 for cpmp compared to 0. 86 for csp (Table  1 , Additional file 1: Figure  S1  and S2 ). Genomes originating from Papua New Guinea (PNG) revealed 9 haplotypes in 22 genomes for cpmp and 3 haplotypes in 30 genomes for csp.
Resampling was repeated 1000 times and included only sequence data from mixtures of strains that were sequenced at a coverage of >3000 reads. At a coverage of 10′000 sampled reads the minority clone was robustly detected at ratios 1:1 to 1:1000 for cpmp and up to 1:500 for csp (Fig. 2 , Additional file 1: Figure  S7  and S8 ). The cut-off set for haplotype positivity required that a haplotype was detected ≥3 times and represented ≥0. 1% of all reads from the respective blood sample. More stringent criteria to call a haplotype (i. e. a higher minimum number of reads) would require a higher coverage for the detection of minority clones. Thus, more stringency in haplotype definition on the one hand reduces sensitivity, but increases specificity by eliminating false haplotypes attributable to background noise (Additional file 1: Figure  S7  and S8 ).
Specification of default cut-off settings in software HaplotypR Cut-off values for the analysis of sequencing data were defined to support removal of background noise caused by sequencing and amplification errors. The following values represent minimal stringency and can be adjusted to higher stringency to increase specificity in the HaplotypR pipeline: (i) Cut-off settings for SNP calling were defined by a population-based approach. A SNP was required to be dominant (>50% of all reads) in ≥2 samples. A single dominant occurrence of a SNP is likely caused by amplification or sequencing error. (ii)Cut-off settings for haplotype calling required a haplotype to be supported by ≥3 reads in ≥2 samples (including independent replicates of the same sample). Per haplotype a minimum of 3 reads are needed to distinguish SNPs from sequencing errors, because a consensus sequence cannot be determined from 2 disparate reads alone.
This more stringent criterion was introduced to prevent erroneous over-estimation of multiplicity due to false haplotypes. All samples were genotyped for length polymorphic marker msp2 using capillary electrophoresis (CE) for fragment sizing. Msp2 genotyping was reproducible and consistent between different laboratories (Fig. 3 , Additional file 1: Figure  S9 : left column). A mean multiplicity of infection (MOI) of 2. 2 was observed in 37 field samples analysed by msp2 genotyping and 25/37 (67. 5%) of samples harboured multiple clones (Fig. 4 , Additional file 1: Table  S6 ). Mean MOI and H e were compared between the Fig. 3  Comparison of genotyping by length-polymorphic marker msp2 and amplicon sequencing of cpmp and csp. Raw data from lengthpolymorphism-and SNP-based genotyping for one P.
The diversity of cpmp was high also in the global MalariaGEN  dataset (H e = 0. 93); its resolution as genotyping marker in other geographic regions remains to be shown. In contrast, marker csp, analysed in parallel to cpmp and also used in earlier studies, showed a limited diversity with only 4 haplotypes detected in 37 field samples. Earlier studies reported similar low diversity for csp in regions of Asia Pacific  [21]. Thus, csp is not suited to serve as a single genotyping marker in PNG. However, the global diversity of csp according to the MalariaGEN dataset seems to be high (H e = 0. 86), and high diversity has also been observed in African isolates  [21]. Implementing amplicon sequencing required parallel development of a bioinformatics pipeline. A known problem in sequence analysis is the robust detection of minority clones from a background of experimentally induced artefacts.
We addressed this problem with the design of HaplotypR, a software package dedicated to stepwise analyse of sequence reads for samples containing multiple clones. The HaplotypR pipeline can be divided into three steps: In the first step, this pipeline de-multiplexes and clusters raw sequence reads to clusters of related sequences, so called "representative haplotypes". This step employs Swarm2 software, which expands pools of amplicons (identical sequence reads) by iteratively joining other pools of amplicons that are separated by a defined number of mismatches (e. g. one substitution, insertion or deletion)  [22, 23]. This strategy permits unbiased clustering of sequence reads without the need to define a list of SNPs. This enables capturing of previously unknown SNPs without any adjustments to the pipeline.
Genotyping using length polymorphic marker msp2 For determination of mean MOI field samples were genotyped using the classical P. falciparum genotyping marker msp2. Fluorescently labelled nested PCR products were sized by CE on an automated sequencer and analysed using GeneMapper software according to previously published protocols  [5]. Each DNA sample was genotyped twice in independent laboratories to assess reproducibility of clone multiplicity (Fig. 3  and Additional file 1: Figure  S9 ).
A SNP was defined as a nucleotide position with a > 50% mismatch rate in the sequence reads from at least two independent samples. For prediction of haplotypes, fused reads were clustered individually per sample with Swarm2 software (parameters: boundary = 3 and fastidious mode)  [22, 23]. The centre of each cluster represents the most abundant sequence of the cluster and thus constitutes a predicted haplotype. The cluster size represents the within-sample clone frequency in the tested sample. Haplotypes with a cluster size of 1 were classified as singletons and considered background noise. Haplotypes were checked for PCR artefacts such as indels and chimeric reads. Indels are caused by polymerase slippage which occurred primarily at stretches of homopolymers. Chimeric reads, caused by incomplete primer extension and inhomologous re-annealing, were identified with vsearch software (parameters: uchime_denovo mode, mindiffs = 3, minh = 0.
2)  [36]. To distinguish chimera haplotypes resulting from PCR artefacts from true recombined haplotypes, a population-wide approach (combining all samples of the entire study) is implemented in Haplo-typR. A chimera was classified as such if a haplotype was identified as chimera by vsearch at all instances it occurred. On the other hand, if a chimera was detected in only some of the samples, it was not classified as chimera, but as a true haplotype. However, in such instances this haplotype was always flagged and the outcome "true chimera" or "true haplotype" was resolved by using replicates. This approach is justified, as it is expected that a true recombinant haplotype would be transmitted without its parent haplotypes. The full analysis pipeline, named HaplotypR, was implemented as R package and is illustrated in Figure  5  (https://github.
Acknowledgements We are grateful to the study participants and their guardians and to the field and laboratory team of the  PNG Institute of Medical Research , in particular  Anna Rosanas-Urgell  and  Alice Ura. This publication uses data from the  MalariaGEN Plasmodium falciparum Community  Project as described in Genomic epidemiology of artemisinin resistant malaria, eLife, 2016 (DOI: https://doi. org/10. 7554/eLife. 08714).
INTRODUCTION The majority of the studies exploring gene expression data result in one or more gene signatures, i. e. , list of genes sharing a common pattern of expression that can be employed to classify groups of samples in any independent dataset. Together with such data-derived signatures, a priori knowledge-based signatures can be produced from the available gene ontologies or pathway databases. In recent years, data-derived and a priori knowledgebased signatures have been widely employed to interpret the results of gene expression data analyses (e. g. , differential expression, clustering). The number of available signatures is getting larger allowing users to benefit from a more exhaustive coverage of the existing biological processes. However, not all the signatures contained in these compendia are equally informative and the number of gene sets representing the same biological process is not equilibrated.
In this paper, a new approach to prioritize and classify gene signatures is proposed. Our method is based on the concept of "an informative signature", which is a gene set capable of defining a natural ranking of samples in unsupervised way. Considering the simplest case of a gene signature composed of only two genes X and Y, their co-variance can define three possible scenarios of sample distribution (ranking), as reported in Fig. 1b. Whereas the ranking defined by informative signatures presents a distinguishable axis (corresponding to the direction of the first principal component), no naturally distinguished sample ranking can be observed in the non-informative ones. As a consequence, when the dysregulation of an informative signature is tested on a transcriptomic dataset whose samples are divided according to two conditions (e.
g. , tumor vs. normal), in most cases, a significant enrichment score will be observed whenever two requirements are met: (i) the direction of the largest variance sufficiently separates the samples belonging to the two conditions and (ii) this variance is significantly greater than randomly expected (the gene set is "overdispersed"). In all other cases, a very specific distribution of the sample labels is needed to obtain a significant enrichment score. Therefore, informative signatures, defining in many datasets robust and "objective" sample ranking along the principal variance direction, are more valuable for data analysis. Starting from a vast collection of signature compendia, composed of 12096 a priori knowledge and data-derived signatures, we defined a restricted set of 962 informative signatures, which is made available to the users for further applications.
The collection was defined by exploiting various pan-cancer The Cancer Genome Atlas (TCGA) transcriptomic profiles (32 cancer types with totally 8991 samples). Among the databases under investigation, the Signaling Pathway Enrichment using Experimental Data sets (SPEED)  8  signatures, including 11 gene sets, proved to be the most informative with 6 of them being part of our prioritized collection. The reliability of our signature collection was then validated by comparing its performances with those of the complete collection in some typical data analysis scenarios. In all the examples, the informative gene sets were found much more frequently significant than the others, confirming the rationale behind the selection procedure proposed here. As a result of our analysis, in each dataset, a set of weights is also assigned to the genes composing the signatures.
We found that many signatures are functionally redundant and this does not seem to be always linked to the intersection between two gene sets. Therefore the H and the other previous works, only measuring the intersection in terms contained genes, are underestimating the scale of signature redundancy. In order to visually and interactively represent the structure of functional redundancies between informative gene signatures, we developed InfoSig Map (http://navicell. curie. fr/pages/maps_avcorrmodulenet. html), a user-friendly interactive Google Maps-based tool, where nodes correspond to our informative signatures and the edges represent the two types of redundancies (compositional and functional). InfoSigMap can be used for data visualization to provide a quick navigation into any set of scores associated with the informative signatures (e.
As shown in Table  1 , the most informative compendium resulted to be SPEED (55% of informative signatures). The reliability of this database is thus corroborated by our results, suggesting that this collection particularly fits for cancer transcriptomic data analysis. Overall, good performances were also obtained by CIT (28%), the MSigDB C4 (31%), and the H (30%). The best performing knowledge-based database was Atlas of Cancer Signaling Network (ACSN), with 13% of informative signatures. Studying the distribution of the number of cancer types in which the signatures were found to be informative (Fig. 2a  and Supplementary Table  S2 ), we found that 30% of the informative signatures is cancer-specific (significant in only 2 cancer types), 14% are associated with >15 different cancer types and only 8 signatures are informative in >25 cancer types.
As shown in Fig. 2b , in all four cases, the informative signatures in the output of the Gene Set Enrichment Analysis (GSEA)  17  were strongly enriched (average p-value 10 -41 ). Note that, while the selection of the informative signatures was performed using an unsupervised approach, the validations presented in this section are realized using a supervised one (GSEA). Nevertheless, the amount of informative signatures obtained in the output of the GSEA analysis is significantly higher than what could be expected at random. Finally, the results of points (ii) and (iv) prove that this analysis weakly depends on the platform used for transcriptomics profiling and it is well reproduced across independent datasets for the same biological case of study, respectively. Informative signatures perform better than the MSigDB Hallmarks in typical cancer data analysis Given that the only other attempt to prioritize the most reliable non-redundant signatures is done by the MSigDB H, its performances were compared with those of our compendium in the four test cases previously mentioned.
Indeed, the KS p-values are always <0. 05, except for 2 out of the 9 cases. Therefore, not only the informative signatures are more frequently significant in a GSEA analysis but also in the GSEA output they tend to be among those with the highest NES score. This result indicates that our compendium is capturing the strongest sources of expression variation in all three transcriptomic datasets. As a further check, given that 15 out of the 50 H signatures are also contained in our informative collection, the fraction of H signatures present in both the output of the GSEA analysis and our informative compendium is evaluated: (i) colorectal cancer KRAS mutated vs. wild type 67%; (ii) colon cancer metastatic vs. primary 67 and 80% in RNA-seq and microarray, respectively; (iii) normal tissue vs.
A first consequence of this result is that the H collection, based on the JI as a measure of redundancy, is not able to completely capture analogous signatures. The intrinsic limitation of such approach is that it requires the wide use of expression data and thus its output is at least partially data corpus-dependent. InfoSigMap a user-friendly interactive representation of the functional redundancy between informative signatures for insightful gene set score visualization GSEA or alternative approaches can be used to score the informative signatures based on a transcriptomic dataset whose samples are divided according to multiple conditions. The output of such analyses usually consists of a table of gene sets with the corresponding enrichment values. This tabular organization of the output (which represents a one-dimensional ordering) is not easy to interpret and it frequently does not help the formulation of consistent biological hypothesis.
To improve the interpretation, we developed InfoSigMap (http://navicell. curie. fr/pages/ maps_avcorrmodulenet. html), following the procedure described in the Methods section. The obtained network (Fig. 4 ), whose nodes are the informative signatures and links denote their redundancy, contains eight connected components. The largest connected component is composed of two main clusters: one associated with core cellular functions (i. e. , all those basic functions that are fundamental for the life of the cell) and the other to the tumor microenvironment. These two areas of the network can be then further subclustered. The core cellular functions can be split into homeostasis and proliferation Fig. 4  InfoSigMap: user-friendly interactive representation of the informative signatures and their redundancies.
The network map of the 962 informative signatures plus SPEED and Hallmarks is here reported as available on the website (http://navicell. curie. fr/pages/ maps_avcorrmodulenet. html). The signatures are organized as nodes of the network. Node colors correspond to the different signature categories, while the shape is a diamond for informative signatures and circular for those that are also conserved. The links correspond to redundancy between couples of gene sets (functional redundancy in light gray, dark gray if also the Jaccard-index intersection is significant). The names annotated on the top of the map denote areas of the network containing signatures associated with the same biological function. The interactive online version of this map can be browsed as an instance of Google Maps, with the possibility of zooming in and out, getting description of gene signatures, and visualizing data (various gene set scores, such as GSEA or ROMA scores) on top of the map (composed of cell cycle, mRNA translation, splicing, MYC targets, protein degradation, and oxidative phosphorylation).
Indeed, the informative signatures derived from EI are split into two main areas (see Fig. 4 ): one belongs to the tumor microenvironment component and it is strongly linked to the immune system/inflammation signatures. The second is part of the proliferation component, strongly linked to the cell cycle area. We considered that such unexpected configuration could be caused by the presence of gene sets belonging to the EI category but obtained from the comparison of two conditions characterized by a strong difference in proliferation To test this hypothesis, the dataset used to define the MSigDB signature "LEE_NAIVE_-T_LYMPHOCYTE", obtained from the expression profiling of human CD4+ T cell during differentiation induction, was reanalyzed  21  (see Supplementary Figure  S2 ). The informative signatures differentially activated during the experiment (differentiated vs.
undifferentiated states) were detected as described in the Methods section. The two areas of EI signatures show an opposite behavior coherent with that of the signatures around them. Indeed, the EI area near cell cycle shows a donwregulation in the differentiated cells, while the areas belonging to the immune cluster are upregulated. This confirms our starting hypothesis that, when EI signatures are used to analyze an expression dataset, their deregulation can be due to a variation in proliferation rather than in the immune system functioning. To avoid this uncertainty, a reorganization of the EI signatures (part of the MSigDB C7 collection) in several categories would be recommended for their future use in data analysis. Another non-intuitive observation is that signatures coming from the same collection tend to co-localize in the map and dataderived signatures tend to be clearly separated from knowledgebased ones.
The strongest effect reported in Fig. 5a  (bright red area) is the upregulation of a subset of the metastatic signatures. This result fits with previous evidences that KRAS mutation is associated with metastasis in patients with CRC. 24, 25 Moreover an alteration of the metabolism is detectable from an upregulation of the mitochondria and oxidative phosphorylation areas. This result fits with previous experimental evidences. 7] [28]  In CRC, KRAS mutation also causes an alteration of the transcriptional response and amino acid metabolism machineries, two processes involved in cancer cell proliferation and maintenance. 29, 30 This effect is captured in our analysis by the upregulation of the mRNA translation/protein metabolism areas of InfoSigMap. (ii) Colon cancer metastatic vs. primary: The differential module activity between metastatic and primary colon cancer (CC) is investigated (Fig.
DISCUSSION Data-driven and a priori knowledge-based gene signatures are largely used in cancer studies to score clinical samples according to distinct tumor subtypes, identify important cellular responses to stimuli, predict clinical outcomes, and quantify the activation of signaling pathways. Nowadays, signature collections are getting larger, providing the benefit of a more complete coverage of the existing biological processes. However, the growth of these compendia is posing two main challenges related to the reliability and the redundancy of the collected gene sets. Here we developed a new methodology for assessing the value of a gene set, which is based on the notion of informative signature, i. e. , a gene set able to systematically and robustly rank tumor samples in many independent datasets.
A restricted collection of 962 informative gene sets is suggested for transcriptomic data analysis in cancer biology. The methodology used for the selection of the informative signatures currently relies on Principal Component Analysis (PCA), a linear matrix factorization technique optimal for close-to-normal data distributions. In particular, the gene set overdispersion estimation is based on a linear model of gene regulation. 42 For complex datasets, recapitulating many different and dynamically changing transcriptional programs, non-linear and strongly non-Gaussian features might play an important role. 4] [45]  This might be particularly important in the case of exploiting the data coming from the application of single-cell sequencing technology. 46 However, the concepts of informative gene signature and functional redundancy introduced in this study should be useful and instrumental independently on the concrete methodology used for quantifying them.
We developed InfoSigMap, a user-friendly interface designed for insightful data visualization. The advantage of using our map with respect to the classical tabular outputs of GSEA or analogous tools is proved by the wide use of pre-existing tools with a similar aim (e. g. , Enrichment Map, GOIorize, and ClueGO). At the same time, InfoSigMap has multiple novelties with respect to those tools. It is a map of informative signatures and not only of Gene Ontology (GO) categories. Moreover, a link between two signatures in InfoSigMap reflects the existence of a functional redundancy between the two gene sets, never considered before. The NaviCell-based implementation of InfoSigMap allows easy integration of our tool in any pipeline for omics data analysis (from Java, R, or Python programming languages) using simple REST API.