dataesr/scientific-paragraphs-categorization huggingface dataset:
Datasets and preprocessing The benchmarking was realized on four datasets: Chen, Liu, Duren and Semrau (see Supp Table  1 ). The Chen and Liu datasets consisted of paired single-cell RNA sequencing (scRNA-seq) and single-cell chromatin accessibility profiling (scATACseq) data from human embryonic stem cells (hESCs). Duren and Semrau consisted of unpaired scRNA-seq data from mouse embryonic stem cells (mESCs). The Semrau dataset contained only scRNA-seq data, we thus used it together with the Duren's scATAC-seq data. Description of the data and download links can be found in Supp Table  1. Regarding data preprocessing, for both scRNA-seq and scATACseq data, we filtered out the features expressed in less than 1% of the cells. Gene counts were then log2-transformed and peak accessibilities were binarized by replacing the non-null values by 1.
Running the state-of-the-art methods Pando  11  First, unpaired datasets were computationally paired with SCOTv2 78 , running SCOTv2. align with default parameters (k=50, e=1e-3, balanced=True, rho=5e-2, normalize=True). Following the default Pando pipeline, pseudocells were then aggregated as described in https://github. com/quadbiolab/ organoid_regulomes/blob/main/pando/pseudocells. R to reduce data sparsity. 11  Motifs were obtained from JASPAR2020 and cisBP, and matched to ATAC peaks with find_motifs(). The GRN network was finally inferred with infer_grn() using the parameters suggested in the Pando vignette, plus upstream = 100k, downstream = 100k and only_tss = TRUE to consider regulatory regions both downstream and upstream than the TSS, as done by the other tools here considered.
CellOracle 10 We applied CellOracle as described in https://github. com/morris-lab/CellOracle. ScATAC-seq datasets were analyzed with Cicero to find co-accessible regions (coaccessibility score >0. 8) in a genomic window of 500kb. Peaks co-accessible with promoters were associated with genes through CellOracle integrate_tss_peak_with_cicero function. Peaks were also scanned with the CellOracle TFinfo function and its default parameters and default motifs to identify TF binding sites, to produce TF-gene edges. Finally, the TF-gene edges were inferred by get_links function with alpha = 10.
2 and 0. 8, with 0. 8 being the default value. The obtained predictions were then compared with a ground-truth based on a combination of six enhancer databases. We first defined a list of potential enhancer-genes interactions from the union of PEGASUS  24, 27  , ENdb  23  and EnhancerAtlas2. 0  25. We then filtered this list, keeping only the links whose enhancers were present in the union of Fantom5 28 , VISTA  29  , SCREEN. ENCODE  26  databases. Finally, we only kept in the ground-truth enhancers overlapping with the peaks of the scATAC data. The quality of the overlap between predicted regulatory regions and the databases was finally assessed using F1 scores.
To find the optimal clustering resolution for each of the methods, we tested 21 values between 0 to 2 with a step size of 0. 1 (see Supp Table  4 ). Only resolutions providing at least 10 communities out of thousands of nodes (see Supp Table  3  for details on the number of nodes per method and dataset) were considered for the following part of the analysis. We considered five different databases to evaluate the quality of the clustering : GO Cellular Component, GO Biological Process, GO Molecular Function, KEGG 2021 (human) / 2019 (mouse) and Reactome 2016  [33] [34] [35] [36] [37]. For each method and resolution, we then used the enrichR package  79  to find enriched pathways in each of their communities. We then counted the number and the proportion of communities significantly enriched (p-value < 0.
HuMMuS application from HMLN reconstruction to GRN extraction To illustrate the potential of HuMMuS we used a single-cell dataset of cortical neurons composed of snmC, snATAC-seq and scRNA-seq. The data were downloaded from  [38] [39] [40]   We then used HuMMuS to contract a HMLN consisting of four layers: a TF layer, a snmC layer, a scATAC layer, and a scRNA layer. To follow transcriptional regulation structure we placed the scmC layer in the middle between the scATAC layer and the scRNA layer. We didn't link the snmC layer to the TF layer because TF binding motifs are specific to small regions, making gene bodies too large for precise binding motifs. As in the benchmark, we didn't put links in the TF layer. For the scATAC layer we used Cicero setting a co-accessibility score threshold at 0.
2, as almost all correlations were above 0. The scRNA layer was computed with the python version of GRNBoost2, GENIE3 did not manage to get results on such a big dataset. Then the 50k links with the highest weights were kept. For the scmC layer, since we did not find methods designed to infer networks on methylation data, we used partial correlation from the pingouin0. 5. 3 python package, accessible at https://github. com/raphaelvallat/pingouin/tree/master. All the links with an absolute corrected correlation above 0. 3 were kept. The inter-layer connections not involving the snmC layer were structured as in the benchmark. The connections between the snmC layer and the scATAC layer were set based on the distance of the scATAC peaks from the transcription start site (TSS) of the genes, nodes of the snmC layer (500 bp before and after the TSS).
A first goal may be to identify sequences of interest such as those associated with specific functions. This task will be referred to as "functional annotation". Some methods involve processing each read individually to search for specific sequences associated with pathogens or other global functions, such as antibiotic resistance genes (ARGs) (  [15] ), phages (  [16] ) or viral sequences (  [17] ). However, a key issue in metagenomics is also to identify which microorganisms are  August 3, 2023  3/54 actually present in the sample. This can be achieved by performing either de-novo metagenomic assembly of metagenomic reads or assembly-free approaches where metagenomic reads are used directly for taxonomic and functional profiling based on reference databases. To identify and reconstruct the genomes of the species that make up the microbiome, raw reads are first assembled into contigs, which carry more information.
When using MAGs, it is also possible to calculate the relative abundance of each MAG in the metagenome by considering the number of reads mapped to a MAG. In both cases, this results in an abundance table representing the metagenome by the abundance of each species. Another approach is to start by building representative, non-redundant gene catalogs (  [26]   [27] ), which are themselves binned to Metagenomic Species (MGS)(  [22] ) and (  [28] )). At the end of this step, the output is an abundance table linking each taxon to its Metagenomic Gene Abundance (MGA). Other methods, called "assembly-free methods" start by grouping together the reads that belong to a particular taxonomic unit, such as species. They exploit sequence similarity (  [18] ,  [29] ,  [30] ) or kmer-content similarity (  [19] ,  [31] ) against reference databases.
For example, reads are aligned against gene markers for taxonomic profiling (  [32] ) or comprehensive gene catalogs that maximize the genomic knowledge of microbial ecosystems, such as Genome Taxonomy Database (GTDB), the Global Microbial Gene Catalog (GMGC) or the Kyoto Encyclopedia of Genes and Genomes (KEGG). This provides a representation of the composition of a metagenome as well as its functional potential. The number of reads in each bin provides an estimate of their relative abundance within the metagenome, when normalized by the respective size of their respective genomes. Traditional bioinformatics methods, while very useful and widely used, have several drawbacks: they can be computationally expensive, are affected by sequencing errors and are often dependent on reference databases.
The task of predicting patient phenotype can be addressed using various Machine Learning (ML) models. With an increasing number of public example datasets, these algorithms can learn and extract important patterns from the data in order to classify samples based on their various characteristics. Deep Learning (DL) is a specific branch of ML that focuses on algorithms based on layers of artificial neurons that receive and process information from previous layers of neurons  ( [34] ). The creation of diverse network types hinges on the choice of layers, neurons, and their respective organization. These range from traditional multi-layer perceptrons to recurrent neural networks, which excel at managing data that evolves over time. ( [35] ). Data is channeled through the network to generate an output, facilitating the learning process as the network adjusts the neuron weights via backpropagation of errors.
The most notable strides empowered by deep learning are discernible in domains like image recognition and Natural Language Processing. (NLP). Deep Learning stands out for its superior performance on large datasets, outdoing many other machine learning algorithms that reach a performance plateau with a given quantity of data. Furthermore, deep learning techniques possess a robust capacity to unearth intricate features, often imperceptible to human observation. This concept, known as "representation learning," involves automatic discovery of necessary representations for feature detection or classification directly from unprocessed data. DL can also perform various learning paradigms (unsupervised (  [36] ), semi-supervised (  [37] ), Multiple Instance Learning (  [38] )), etc. These paradigms allow different types of learning : exploring the data in a certain direction with supervised learning, letting the network the task to draw conclusions with unsupervised learning, etc.
Automatic enrichment with the Connected Papers tool With such a strict search equation, it seemed appropriate to expand the set of articles identified to ensure that no critical articles were left out. To do so, we used the August 3, 2023 8/54 Connected Papers software (https://www. connectedpapers. com/), which takes an article as a query and searches a database to select the most closely related articles using a similarity measure based on co-citation and bibliography. This database is enriched with the Semantic Scholar database, which is composed of more than 240 million papers. This corresponds to step (B) of our pipeline). An example of a graph generated by Connected Papers can be seen in Fig  S1. We created a co-citation article directed graph for each article. 24 of them were not available on Connected Papers, or did not have enough co-citation neighbors to build a graph.
This process allowed us to fetch up to 2443 new articles that were not captured by the restrictive search described in step 1 of the pipeline. For each connected-papers graph the raw list of articles was obtained. We then developed a small Python program to parse and analyze the graphs by computing various stats, including the number of times an article appeared in all graphs, along with the distribution of the number of citations (https://github. com/CorvusVaine/analyzing_connected_papers_articles. git). Finally, we computed an integrated graph, including all articles present in the Connected Papers database. The connectivity of an article in this graph varied from 1 to 34. We decided to add to our dataset the articles with a co-citation connectivity > 4. We chose this threshold because it allowed us to reject as few articles as possible while not adding more articles than the original database size.
Many non-DL based methods have been developed to perform such tasks and show impressive results. Many of them allow to bin contigs into genomes and thus provide a list of species representing the microbiome. We can cite MetaBAT (  [72] ) and  MetaBAT 2 ( [20] ), which use probabilistic distances and tetranucleotide frequencies, as MaxBin (  [73] ) and MaxBin 2 ( [74]) do. Finally, a method like GraphBin  [75, 76]  use assembly graphs and De Bruijne graphs to cluster contigs. On the other hand, the method described by  [77]  uses ML to compute August 3, 2023 12/54 taxonomic classification of metagenomic sequences. All of these methods provide good results when binning natural and synthetic datasets, such as CAMI datasets (  [78] ). But DL methods bring numerous novelties notably in terms of discovering new relevant features and embedded representations.
Composition-based methods The one-hot encoding of a sequence is a limited method with respect to the goal of grouping it with others. It should be noted that various methods perform binning using autoencoders but relying on one-hot encoding (  [79]  and  [53] ) or reference database annotations only (  [80] ). However, these methods are now outperformed. Indeed, other features can be extracted from a sequence that better represent it. For instance, several methods work with what can be called computed features. This means that they process a sequence by modifying its representation with features inferred from the reads. In particular, tetranucleotide frequencies (  [81] ) and, more generally, k-mer frequency distributions are well known for their utility in characterizing sequences, acting like "signatures".
VAMB works with contigs and takes as input both the k-mer frequency and the abundance of reads mapped to the contig. These inputs are treated by a Variational Autoencoder, creating a new feature vector then mapped to a latent space. This space is then clustered using an iterative medoïd algorithm. On the basis of the VAMB architecture, various methods have been developed for its extension or the use of other sources of information. First, the authors of CLMB (  [90] ) took into account the noise, a phenomenon too rarely considered in metagenomic Taking into account this graph allows this method to outperform VAMB, and paves the way to graph embedding methods in metagenomic binning. Another method outperforming VAMB is SemiBin (  [85] ), which follows the concept of Semi-supervised learning, by adding information from reference databases while still being able to discover new bins outside of reference datasets.
SemiBin relies on the notion of constraints by creating must-link and cannot-link constraints between contigs. The must-link constraints are created by breaking contigs up, while the cannot-link constraints use reference contig annotations. These constraints are combined with the same inputs as VAMB (abundance by mapping and k-mer frequencies). Deep Siamese networks embed these features in a distance between two contigs, generating a sparse graph clustered with k-means algorithm. SemiBin outperforms existing binners, in particular VAMB and SolidBin (  [94] ), on both real and simulated datasets. More specifically, it recovers with great completeness a high number of complex bins. It is precise enough to differentiate B. vulgatus from human and dog gut microbiomes. Noteworthy, the three methods (VAMB, CLMB, and SemiBin) work with contigs rather than raw reads.
Consequently, Metagenome2Vec serves as an efficient and potent solution for metagenomic data analysis, as it curtails the necessity for domain-specific knowledge and enables a more seamless workflow. Meta1D-CNN tries to enhance the precision in sequence classification with NLP methods by introducing 1D-CNN. They train a word2vec algorithm with different k-mer lengths from 1 to 8 (8 giving the best results). The embedding of a sequence is obtained by calculating the mean of all k-mer embeddings. Two layers of convolution are then used to extract features that will ultimately be used to classify sequences. On its dataset, Meta1D-CNN achieves 83. 89% F1 Score at the genus level and 65. 65% at the species level, while NLP-MeTaxa achieves respectively 83. 89% and 60. 95%. While these methods are proof of concepts they have not outperformed alignment-based methods outlined earlier.
The idea behind a GAN is to use two competing networks: one to generate data coherent with the input dataset, and the other to try to detect whether that dataset is real or generated. The two models are trained in an adversarial way. CGANs offer the possibility to parameterize this generation: the network can then decide to generate for example healthy or disease-related data. By training a CGAN with enough different types of data it becomes possible to generate new data for each class. This study from  [118]  shows that the integration of generated data into an original dataset when training ML methods leads to better results. But the issue with GAN is that finding an optimal model is often challenging, and therefore there is a risk of generating unrealistic data. Furthermore, their training requires a large amount of data, which is currently lacking in experiments with them.
Although the proof of concept is promising, it is still a problem to get sufficient quality data to train GANs and subsequently classification Autoencoders (VAE) were all tested and gave good results, none of them outperforming the others, as the best result depended on the datasets which contained data corresponding to 6 different diseases. The conclusion drawn by the article was that the best type of autoencoder depends strongly on the structure of the data (CAE when working on Abundance profile for example, while none outperformed the others on Marker profile) and that there is no absolute good answer. However, due to the black box nature of the models, the difference mostly lies in the performance in predicting several diseases (DAE for Obesity and Colorectal, and CAE for C-T2D and Cirrhosis) and does not offer much insight on the reasons why some autoencoders perform better than others.
Functional and genetic information Some authors have chosen to use the functions of genes or specific communities contained in a metagenome. However, metagenomic diversity remaining largely unexplored, using reference databases might be challenging or incomplete. Still, some tools try to extract relevant information from these databases. Most of these tools rely on classical Machine Learning and not Deep Learning. [146] uses functional profiles extracted from orthologous genes given a reference database to add these features to abundance, while DeepMicro ( [127]) uses strain-level marker profiles to contextualize and deepen abundance data by the presence or not of a certain strain. As for abundance data, strain-level markers is very sparse information. Therefore, it can lead to the same difficulties as those encountered while using abundance data.
Discussion For this metagenomic review, we wanted to focus exclusively on the intersection between the two fields of DL and metagenomics. In need of a reproducible method, we designed a specific search equation. The objective of this equation was to select articles from all other the fields while remaining stringent in order to focus on our theme, as both of the themes composing it present a large litterature. This is why our equation is very specific and searches for words in the title, which can be considered as too stringent. We are aware of this limit, and this is why we decided to enirch our database with connected papers. We are aware that such a choice relies on external tools and leads to choices that can be considered as arbitrary, such as choosing a threshold for the connectivity of articles found via connected papers.
All classic models found their use, be it classic MLP, CNN, LSTM or, more recently transformers. If some architectures seem well-suited for some types of tasks, for example CNN when studying abundance table and phylogeny or LSTM for longitudinal data, sometimes combined with CNN, many different have been tested. To go beyond the state of the art, methods inspired by recent DL technologies like those found in NLP work, such as those mentioned above, are emerging. However, they still need to be further explored, especially using large amounts of data and more powerful models. New powerful Transformer-based models like BERT ( [103]) are tested on metagenomic datasets, although adaptations of these approaches to metagenomics are still very recent and the field remains young and very active. Experimental approaches such as BERTax ( [105]) and ViBE ( [169]) aim to use these powerful models.
BERTax, for instance, uses the power of BERT-like models to "learn a representation of the DNA language" and design several taxonomic classification models, at different taxonomic levels (superkingdom, phylum and genus). The ViBE method uses a pre-trained BERT architecture with a reference viral database to identify viruses in the metagenome. Of course, these methods remain challenging because they require very large and representative databases while microbiomes are still composed of many unknown microorganisms. The human gut microbiome is however one of the most studied and described at the genetic and genomic level, with important effort deployed in generating ). In a context of data explosion and models designed to aggregate an ever-growing amount of this data, this issue remains of high importance.
Furthermore, it is difficult to compare the performance of the different methods in the literature in a rigorous manner. Indeed, the variety of metagenomic data used, the limited number of samples per study and the recent explosion of many DL methods, do not facilitate their comparison. Most developed methods compare themselves with alignment-based methods or classical machine learning methods like MetaML  ( [112] ), but there is a lack of comparison between DL methods, especially between methods with similar goals but different approaches. Simulated datasets from the CAMI project (  [78] ) are often used, but even the metrics are difficult to set : good specie classification, quality of bins or differentiation of closely related species. In the case of disease predictions, although datasets are very diverse, the mostly studied diseases form a small group.
Due to their huge number of parameters (345 million for classic BERT, 175 billion for GPT3), they require an even larger amount of data to obtain good results. If this data is still difficult to obtain, its recent and continuous expansion could overcome classic ML algorithms in prediction tasks and paves the way to new models and results. Finally, future work includes improving end-to-end analysis of metagenomic data, paving the way for point-of-care applications. Supporting information• Fig S1. Example of a graph generated using Connected Papers. It has been generated from article "A deep siamese neural network improves metagenome-assembled genomes in microbiome datasets across different environments" ( [85]). The darker the color is, the more recent the article, and the bigger the circle is, the most citations it has.
• Fig S2. Number of articles according to the number of links from original articles pointing to them. "Sequence classification" values mean the number of articles pointed by one of the articles of this category, same for "Phenotype prediction". "All" represents all articles pointed by those two categories + 4 miscellaneous articles. (a) represents the distribution of articles through the number of times they are pointed at, (b) is a zoom of the precedent figure, starting with a threshold of minimum 4 citations. (c) and (d) represent the same thing but considering only newly discovered articles and not the ones already present in the database. • Fig S3. Examples of generated graphs of articles. A vertex is an article (labelled by DOI). Red dots represent articles that were already selected by research equation, blue dots represent articles discovered through Connected Papers.