dataesr/scientific-paragraphs-categorization huggingface dataset:
Parallel Strategy Implemented To take advantage of multi-core architectures, TIP_finder is parallelized using the MPI standard and executes the same process several times with different data  [51]. First, the read files (in FASTQ format) are split into n files (where n is the number of processes); then, each working process maps reads against the TE consensus databases. The mapping output is used to extract the "unmapped reads", which are aligned against the reference genome. Each process generates a BLAST output file and the master process finally joins all of these into one file. The next step includes the detection of reads with a unique hit in the genome. For this, the number of lines in the BLAST output is divided by the processes number, and a start and end line number is assigned to each work process; the process analyzes its corresponding region and creates a Python dictionary, which is unified by the master process.
TIP_finder used the following bioinformatic software: Bowtie2 (v. 2. 3. 4. 1) to map the paired reads of genomic data to the indexed consensus sequence of each TE/HERV family  [30, 60] , Samtools (v. 1. 9) to process the bowtie2 output and keep only the unmapped reads  [61] , bedtools (v. 2. 26. 0) to split the reference genome into 10 Kb windows and count reads in these windows  [62] , and NCBI-BLASTn (v. 2. 10. 0) [63]  or Magic-BLAST  [64]  to align the unmapped reads. Although Magic-BLAST is significantly faster than NCBI-BLASTn, it requires more disk space. TIP_finder was developed using Python3 (3. 8) and the following libraries: sys, time, os, subprocess, argparse, and MPI4py  [65]. Furthermore, TIP_finder_utils. py requires the additional libraries: math, Pandas  [66] , matplotlib  [67] , Seaborn  [68] , and SciPy [69].
We used publicly available datasets from the NCBI Whole Genome Sequencing (WGS) database, corresponding to people with breast cancer (i. e. , case patients) and people without the disease (i. e. , control patients). Case patients were obtained from the SRA (Sequence Read Archive) database through search Equation  (8). We retrieved 512 datasets and discarded those obtained by sequencing technologies other than Illumina and that lacked public permission for download. As a result, ~300 datasets of paired reads were pre-selected and only 10% were used (30 patients) due to the huge disk space required for storage. Control patients were consulted from Chen 2016  [78] , obtaining a list of 30 patients from the BioProject PRJNA551447 on a study of schizophrenia in a Chinese population (Supplementary Material S1):  (((((((((( For each patient (i.
e. , case and control), a maximum of 30 million reads was downloaded using the fastq-dump tool from the SRA toolkit  [79]. Overall, the datasets comprised approximately 1. 1 Terabytes of disk space (642 Gb for controls and 458 Gb for cases). HERV-K consensus sequences were obtained from Repbase (free version 2017)  [80]. Finally, the human reference genome (3. 3 Gb) in FASTA format was downloaded from the NCBI (assembly GRCh38. p13).
6 Mb. Several factors explain this result, such as bottlenecks in the execution time of the BLAST program, the writing time and format of the results, and the lack of overall parallelization of the tools. Based on this observation, we decided to develop a new tool that could overcome these different bottlenecks in data analysis time to obtain a faster program that could be more easily deployed on supercomputers. Thus, TIP_finder was developed for use in current supercomputers to analyze TIPs in huge genomic datasets. We first tested TIP_finder on coffee data and we obtained a computational time of 594 seconds and a BLAST result file size of 448 Mb, showing a decrease in the execution time of 15. 9X and a decrease in file size of 16. 9X. To test TIP_finder on very large genomes, we decided to analyze the insertional polymorphisms caused by HERVs of type K in the human genome since several studies have demonstrated that these elements correlate with breast cancer  [26] , a disease that causes high mortality rates in women  [77].
TIP_finder implements the MPI standard to reduce execution times; thus, allowing researchers to use more datasets. We tested TIP_finder (using NCBI-BLAST and MagicBLAST as aligners) and TRACKPOSON using a maximum of 30 million of reads (~1X of the human genome) from one randomly selected case dataset (Figure  3A, B ) and one control dataset (Figure  3C ,D) using 2, 4, 8, 16, 32, 44, and 56 cores, executed 10 times, to determine the behaviors of runtime (Figure  3A, C ) and speedup (Figure  3B, D ). To calculate the speedup, we used the execution with two cores as serial time since our MPI implementation used one processor as master and the others as workers. Biology 2020, 9, x 7 of 17 of the BLAST program, the writing time and format of the results, and the lack of overall parallelization of the tools.
Based on this observation, we decided to develop a new tool that could overcome these different bottlenecks in data analysis time to obtain a faster program that could be more easily deployed on supercomputers. Thus, TIP_finder was developed for use in current supercomputers to analyze TIPs in huge genomic datasets. We first tested TIP_finder on coffee data and we obtained a computational time of 594 seconds and a BLAST result file size of 448 Mb, showing a decrease in the execution time of 15. 9X and a decrease in file size of 16. 9X. To test TIP_finder on very large genomes, we decided to analyze the insertional polymorphisms caused by HERVs of type K in the human genome since several studies have demonstrated that these elements correlate with breast cancer  [26] , a disease that causes high mortality rates in women  [77].
TIP_finder implements the MPI standard to reduce execution times; thus, allowing researchers to use more datasets. We tested TIP_finder (using NCBI-BLAST and MagicBLAST as aligners) and TRACKPOSON using a maximum of 30 million of reads (~1X of the human genome) from one randomly selected case dataset (Figure  3A, B ) and one control dataset (Figure  3C ,D) using 2, 4, 8, 16, 32, 44, and 56 cores, executed 10 times, to determine the behaviors of runtime (Figure  3A, C ) and speedup (Figure  3B, D ). To calculate the speedup, we used the execution with two cores as serial time since our MPI implementation used one processor as master and the others as workers. We analyzed the behavior of TIP_finder for each of the four steps: Step 1: mapping and alignment, Step 2: creation of the dictionary with reads, Step 3: filter reads with one hit, and Step 4: the post-processing of TIPs.
Here, we were particularly interested in monitoring the execution time and the speedup for each step with two different numbers of cores: two and 32. We used the same case and control datasets used for the speedup analyses. Figure  4  shows that Step 1 in both data sets  We analyzed the behavior of TIP_finder for each of the four steps: Step 1: mapping and alignment, Step 2: creation of the dictionary with reads, Step 3: filter reads with one hit, and Step 4: the post-processing of TIPs. Here, we were particularly interested in monitoring the execution time and the speedup for each step with two different numbers of cores: two and 32. We used the same case and control datasets used for the speedup analyses. Figure  4  shows that Step 1 in both data sets benefited the most (up to 11. 5X) from the parallel strategy used, because this Step is the most computationally time consuming (Figure  S1 ).
Correlation of HERV-K TIPs with Breast Cancer as a Proof of Concept After implementing TIP_finder and testing it in several configurations (e. g. , different numbers of cores and datasets), we were interested in applying the software to a specific problem, such as the association of TIPs (in this case HERV-K) with breast cancer. We used publicly available Illumina read datasets found in the SRA database (NCBI)  [82]. For this proof of concept, we selected 30 datasets for cases and 30 for controls. After obtaining the datasets, we ran TIP_finder to generate a tabularseparated file (in bed format) per individual, which contained the information of TIPs located throughout the chromosomes of each genome. Then, we ran TIP_finder_utils. py to automatically generate the analyses based on these data.
Finally, we compared the execution times between TIP_finder and TRACKPOSON by randomly selecting five case and control datasets, in addition to the two patients used in the speedup analyses. This analysis aimed to measure the runtimes based on different numbers of TIPs in the datasets (Figure  5 ). All executions of TRACKPOSON and TIP_finder were performed using 32 cores and a maximum of 30 million reads. Finally, we compared the execution times between TIP_finder and TRACKPOSON by randomly selecting five case and control datasets, in addition to the two patients used in the speedup analyses. This analysis aimed to measure the runtimes based on different numbers of TIPs in the datasets (Figure  5 ). All executions of TRACKPOSON and TIP_finder were performed using 32 cores and a maximum of 30 million reads.
Correlation of HERV-K TIPs with Breast Cancer as a Proof of Concept After implementing TIP_finder and testing it in several configurations (e. g. , different numbers of cores and datasets), we were interested in applying the software to a specific problem, such as the association of TIPs (in this case HERV-K) with breast cancer. We used publicly available Illumina read Biology 2020, 9, 281 9 of 17 datasets found in the SRA database (NCBI)  [82]. For this proof of concept, we selected 30 datasets for cases and 30 for controls. After obtaining the datasets, we ran TIP_finder to generate a tabular-separated file (in bed format) per individual, which contained the information of TIPs located throughout the chromosomes of each genome. Then, we ran TIP_finder_utils. py to automatically generate the analyses based on these data.
TRACKPOSON displays a speedup of ~1X (Figure  3 ) on 56 processors compared to two processors, due to the suboptimal execution of NCBI-BLAST and BLAST output processing using a single-processor Perl script. This demonstrates that a parallel strategy is required for better execution and this limitation is a disadvantage for massive sequencing projects that generate huge genomic datasets. To address this deficiency, the HPC and MPI techniques used by TIP_finder allow researchers to use many processors simultaneously in an efficient manner, taking advantage of all the benefits of supercomputers. TIP_finder can accelerate up to 55× the runtime compared to TRACKPOSON (Figure  3B, D ). This improvement is important considering that TRACKPOSON is the fastest TIP detector in a recent benchmarking analysis  [36].
Furthermore, TIP_finder can process larger datasets, improving its speedup using a high number of processors. Particularly, Step 1, which involves mapping (Bowtie2) and alignment (NCBI-BLAST or Magic-BLAST), requires the highest execution time and computational resources (Figure  S1 ); thus, greatly benefiting from a parallel programming approach to speed up the analysis compared to TRACKPOSON (Figure  4 ). Furthermore, TIP_finder is a scalable software since it can run single-node applications (i. e. , software than can run over multiple processors but are limited to one server), such as Bowtie2, NCBI-BLAST, and MagicBLAST, on multiple servers by splitting data into different inputs (i. e. , data is split to run on different servers, see Figure  2 ). This feature allows taking advantage of current clusters that follow a distributed memory structure and makes the TIP_finder an especially useful tool for the massive sequencing projects required in the post-genomic era.
A larger sample of patients is necessary to further study this association and particularly, identify the coding regions involved. Moreover, TIP_finder should be tested on other human genetic diseases involving the mobility of endogenous retroviruses. Overall, TIP_finder is a software that, besides its computational versatility (scalability and high performance), can run analyses on any group of organisms (plants, bacteria, animals) and uses information from any type of mobile element. Finally, this software will be used in future work on applications of the deep analysis of retrovirus interference or association with diseases in humans. These applications will help unravel potential applications in the industry of personalized medicine and pharmacology, as well as innovate with biological pathways for the treatment and prevention of the condition analyzed.
Conclusions The development of HPC-based applications in bioinformatics is required on a daily basis. Thanks to the reduced cost of sequencing technologies, the challenge is not in obtaining genomic datasets, but rather in the analysis of this information in record time. Studies on TIPs have been performed in large datasets, demonstrating promising results for solving biological questions. Nevertheless, TIPs analyses show limitations in scalability since the available tools do not implement parallel strategies and follow a single-node approach. TIP_finder proves to be a very useful tool to be implemented on an HPC computing cluster architecture with many servers and with genomic datasets of considerable size. Moreover, the analysis of polymorphic activities of HERV-K could be used to perform a genomic-scale study of certain cancer types, such as breast cancer.
Acknowledgments: The authors acknowledge the IFB Core Cluster that is part of the  National Network of Compute Resources (NNCR)  of the  Institut Français de Bioinformatique  (https://www. france-bioinformatique. fr), the  Genotoul Bioinformatics platform  (http://bioinfo. genotoul. fr/),and the  IRD  itrop (https://bioinfo. ird. fr/) at  IRD Montpellier  for providing HPC resources that have contributed to the research results reported in this paper.
Introduction Many elds have complex data that need labeled graphs, i. e. graphs where vertices and edges have labels, for an accurate representation. For instance, in chemistry and biology, molecules are represented as atoms and bonds; in linguistics, sentences are represented as words and dependency links; in the semantic web, knowledge graphs are represented as entities and relationships. Depending on the domain, graph datasets can be made of large graphs or large collections of graphs. Graphs are complex to analyze in order to extract knowledge, for instance to identify frequent structures in order to make them more intelligible. In the eld of pattern mining, there has been a number of proposals, namely graph mining approaches, to extract frequent subgraphs. Classical approaches to graph mining, e.
Ports represent how adjacent occurrences of patterns are connected. We evaluate our approach experimentally (Section 4) on two datasets with different kinds of graphs: one on AIDS-related molecules (few labels, many cycles), and the other one on dependency trees (many labels, no cycles). Experiments validate our approach by showing that the data can be signicantly compressed, and that the number of selected patterns is drastically reduced compared to the number of candidate patterns. More so, we observe that the patterns can have complex and varied shapes, and are representative of the data. 2 Background Knowledge.
This encoding is optimal. An integer n ∈ N without a known upper bound can be encoded with a universal integer encoding, whose size in bits is noted L N (n) 1. Description lengths of elements that are common to all models are usually ignored, since they do not aect their comparison. Krimp  [10]  is a pattern mining algorithm using the MDL principle to select a characteristic set of itemset patterns from a transactional database. Because of its good performances, Krimp has been adapted to other types of data, such as sequences  [9]  and relational databases  [4]. In our approach we redene Krimp's key concepts on graphs, in order to apply a Krimp-like approach to graph mining.
Encoding the Data with a Code Table The intuition behind GraphMDL is that we can represent the original graph G o (i. e. the data) as a set of pattern occurrences, connected via ports. Encoding the data with a CT consists in creating a structure that explicits which occurrences are used and how they interconnect to form the original graph. We call this structure the rewritten graph G r. Denition 3. A rewritten graph G r = (V r , E r , l r V , l r E ) is a graph where the set of vertices is V r = V r emb ∪ V r port : V r emb is the set of pattern embedding vertices and V r port is the set of port vertices. E r ⊆ V r emb × V r port is the set of edges from embeddings to ports, l r V ∈ V r emb → P and l r E ∈ E r → Π are the labelings. 2 MDL approaches deal with theoretical code lengths, which may not be integers.
Port vertices do not have any associated information, so we just need to encode their number. The description length L(D|M ) = L(G r ) of the rewritten graph is the length needed for encoding the number of vertex ports plus the sum of the description lengths L emb (v, P, G r ) of the pattern embedding vertices v. Every pattern embedding vertex has a label l r V (v) specifying its pattern P , encoded with the code c P of the pattern. We then encode the number of edges of the vertex i. e. the number of ports of this embedding in particular (between 0 and |Π P |). Then for each edge we encode the port vertex to which it is connected and to which port it corresponds (using the port code c π ). the number of elementary graphs in the dataset, the total amount of vertices, the total amount of edges, the number of dierent vertex labels, and the number of dierent edge labels.
Quantitative Evaluation Table  2  presents the results of the rst experiment. For instance the rst line tells that we ran GraphMDL on the AIDS-CA dataset, with as candidates the 2194 patterns generated by gSpan for a support threshold of 20%. It took 19 minutes for our approach to select a CT composed of 115 patterns, yielding a description length that is 24% of the description length obtained by the singleton-only CT 0. Selected patterns have a median of 9 labels and 3 ports. We observe that the number of patterns of a CT is often signicantly smaller than the number of candidates. This is particularly remarkable for experiments ran with small support thresholds, where GraphMDL reduces the number of patterns up to 300 times: patterns generated for these support thresholds probably contain a lot of redundancy, that GraphMDL avoids.
We also note that the description lengths of the CTs found by GraphMDL are between 20% and 40% of the lengths of the baseline code tables CT 0 , which shows that our algorithm succeeds in nding regularities in the data. Description lengths are smaller when the number of candidates is higher: this may be because with more candidates, there are more chances of nding good candidates that allow to better reduce description lengths. 6 https://sites. cs. ucsb. edu/~xyan/software/gSpan. htm We observe that GraphMDL can nd patterns of non-trivial size, as shown by the label count in Table  2. Also, most patterns have few ports, which shows that GraphMDL manages to nd models in which the original graph is described as a set of components without many connections between them. We think that a human will interpret such a model with more ease, as opposed to a model composed of entangled components.
Qualitative Evaluations Interpretation of rewritten graphs. Fig. 7  shows how GraphMDL uses patterns selected on the AIDS-CM dataset to encode one of the graphs of the dataset (more results are available in our git repository). It illustrates the key idea behind our approach: nd a set of patterns so that each one describes part of the data, and connect their occurrences via ports to describe the whole data. We observe that GraphMDL selects bigger patterns (such as P2), describing big chunks of data, as well as smaller patterns (such as P3, edge singleton), that can form bridges between pattern occurrences. Big patterns increase the description length of the CT, but describe more of the data in a single occurrence, whereas small patterns do the opposite. Following the MDL principle, GraphMDL nds a good balance between the two types of patterns.
We now ask ourselves if the extracted patterns are characteristic of the data. To evaluate this aspect, we adopt the classication approach used by Krimp  [10]. We apply GraphMDL independently on each class of a multi-class dataset, and then use the resulting CTs to classify each graph: we encode it with each of the CTs, and classify it in the class whose CT yields the smallest description length L(D|M ). Since GraphMDL is not designed with the goal of classication in mind, we would expect existing classiers to outperform GraphMDL. In particular, note that patterns are selected on each class independently of other classes. Indeed, GraphMDL follows a descriptive approach whereas classiers generally follow a discriminative approach. Table  3  presents the results of this new experiment. We compare GraphMDL with graph classication algorithms found in the literature  [8] , and a baseline that classies all graphs as belonging to the largest class.
The AIDS-CA/CI dataset is composed of the CA class of the AIDS dataset and a same-size same-labels random sample from the CI class (corresponding to negative examples). The other datasets 7 are from  [8]. We performed a 10-fold validation repeated 10 times and report average accuracies and standard deviations. GraphMDL clearly outperforms the baseline on two datasets, AIDS and Mutag, but is only comparable to the baseline for the PTC datasets. On Mutag, GraphMDL is less accurate than other classiers but closer to them than to the baseline. On the PTC datasets, we hypothesize that the learned descriptions are not discriminative w. r. t. the chosen classes, although they are characteristic enough to reduce description length. Nonetheless results are still better than random guessing (accuracy would be 50%).
Here, we address this question using a new multivariate analysis method and a large functional magnetic resonance imaging (fMRI) dataset (N=284). Although the complexity of pain is widely acknowledged, the underlying brain processes are often conceptualized as a unitary system that is activated by nociceptive input. Brain regions traditionally associated with pain include primary (S1) and secondary (S2) somatosensory, anterior midcingulate cortices (aMCC), medial and lateral thalamus, and posterior and midinsular cortices  (Apkarian et al. , 2005; Schweinhardt and Bushnell, 2010; Jensen et al. , 2016). But regions not directly targeted by afferent pathways are also activated by acute pain stimuli  (Apkarian et al. , 2005; Wager et al. , 2013; Jensen et al. , 2016; Seminowicz and Moayedi, 2017).
Each PDM represents a pattern of whole brain activity chosen because it maximizes the indirect (mediating) effect between stimulus intensity and pain report. The voxel weights of each PDM inform us about the contribution of individual brain regions to the generation of a painful experience following noxious stimulation. This approach decomposes activity across the brain into multiple networks that independently mediate stimulation effects on outcomes (i. e. , pain report). Furthermore, these independent PDMs can be combined into a single, joint PDM that can be prospectively applied to new datasets as a predictive model. Using data from eight different heat pain studies (N=284), we comprehensively investigate the role of brain mediators in the generation of pain experiences. Seven of the eight studies, were used as training data for the mediation analyses (N=209), and the largest individual study (N=75) was used as a test data set, using the model parameters estimated in the training data to validate model predictions in new individuals.