dataesr/scientific-paragraphs-categorization huggingface dataset:
jacionline. org). Most importantly, we invite other stakeholders in the field to join our endeavor so that it remains truly a collaborative resource made for and by the IEI community. There are many ways to participate; they include but are not limited to (1) curating a gene of interest, (2) collaborating to reduce redundant curation when writing articles, (3) informing us about available functional studies and/or assay data for resolution of VUSes (variants of uncertain significance), (4) sharing published data before an article becomes publicly available, (5) alerting us to missing genotype-phenotype relationships, (6) offering critical expertise and insight on what has already been curated, or (7) even just replying to our e-mail inquiries seeking collateral information on publications.
geniadb. net/app/disease/info. php?id5141 Clinical signs d Assigned preferred and alternative clinical terms with associated definitions d Hierarchically classified using ''child'' and ''parent'' terms d Novel terms introduced if not found in HPO or any of the ontologies mentioned in the Methods section d All patients in the database reported to present with the indicated clinical finding Example: www. geniadb. net/app/clinterm/info. php?id525 Articles d Basic information from the publication (journal, year, abstract, PMID, DOI, etc) d List of all reported relevant variants d List of all subjects (probands and affected or unaffected family members) d List of first, last, and corresponding authors Example: www. geniadb. net/app/ref/info. php?id5607 Cell types d Preferred and alternative terms, cross-referenced to Cell Ontology d General description of the cell type d Hierarchic classification using ''child'' and ''parent'' terms d Normal reference ranges used for cell type of interest in individual research studies, annotated with age, group, sex, d Basic demographic and familial information d Option to display the subject's family pedigree d All publication codes used to reference the patient if the patient was reported in the literature more than once d Molecular diagnosis, age at diagnosis or study, age at the first manifestation d Reported genetic variants with associated classification, zygosity, and clinical relevance d Present and absent clinical manifestations, with age at presentation or reporting d Reported cell counts and immunoglobulin or other plasma protein levels d Functional assays performed, with associated experimental parameters and outcomes Example: www.
C, Number of times the same genetically diagnosed patient is reported in the litera-article. Conditions are colored according to discovery year. The same gene may appear more than once if ture (using only patients reported in the inspected articles shown in Fig 3, D). s Description of the methodology used s Parameters measured s Link to further details on website of laboratory performing test s Other available tests for the same gene or condition (Currently populating laboratory studies available in Europe and North America, but with plans to expand to information for labs on other continents) Example: www. geniadb. net/app/assay/info. php?id58 ACMG, American College of Medical Genetics; CLO, Cell Line Ontology; dbSNP, Single-Nucleotide Polymorphisms database; DOI, digital object identifier; PMID, PubMed identifier.
Partial least squares regression (PLSR) is a reference method in chemometrics. In agronomy, it is used for instance to predict components of chemical composition (response variables y) of vegetal materials from spectral near infrared (NIR) data X collected from spectrometers. The principle of PLSR is to reduce the dimension of the spectral data X by computing vectors that are then used as latent variables (LVs) in a multiple linear model. A difficulty is to determine the relevant dimensionality (number of LVs) of the model for the given available data. This step can also become time consuming when many different datasets have to be processed and/or the datasets are frequently updated. An alternative to determinate the relevant PLSR dimensionality is the ensemble learning method "PLSR-averaging".
In the past, this method has been demonstrated to be efficient for complex biological materials such as mixed forages, and facilitates to automatize predictions (e. g. in user-friendly web interface platforms). This article presents the extension of the PLSR-averaging to a k-nearest neighbors locally weighted PLSR pipeline (kNN-LWPLSR). The kNN-LWPLSR pipeline has the advantage to account for non-linearity between X and y existing for instance in heterogeneous data (e. g. mixing of vegetal species, collection from different geographical areas, etc. ). In the article, kNN-LWPLSR-averaging is applied to an extensive NIR database built to predict the chemical composition of European and tropical forages and feed. The main finding of the study was the overall superiority of the averaging compared to the usual kNN-LWPLSR.
fitted and used to predict the response variables from new spectral observations. Spectral data are known to be highly collinear and, in general, matrix X is ill-conditioned. Specific regression methods have to be implemented, in particular partial least squares regression (PLSR)  [1] [2] [3]. The general principle of PLSR is to reduce the dimension of X to a limited number a << p of orthogonal vectors n  1 maximizing the squared covariance with Y and referred to as scores. The scores are then used as regressor latent variables (LVs) in a multiple linear regression (MLR). PLSR is very efficient when the relationship between X and Y is linear  4. From several years, a general trend in the agronomic databases (e. g. in feed, food or soils researches) is to aggregate large numbers of samples of different natures or origins.
One of the most common is the cross-validation (CV) that searches the value a that minimizes the CV-error curve. These strategies face to several difficulties. Firstly, despite attempts of automated procedures, they often require case-by-case decisions based on expertise. Secondly, to find a unique optimal dimensionality is in general difficult when the data contain heterogeneity, which is often the case with complex biological materials such as forages (mixing of stems, leaves, different stages of development and geographical areas, etc. ). Finally, it can also become time consuming when many datasets have to be processed (e. g. different models for a set of chemical compositions) and/or when the datasets are periodically updated with new training observations. As an alternative, the ensemble learning method "PLSR-averaging"  [16] [17] [18] [19]  is a tentative to bypass the determination of a.
The method consists in averaging the predictions of PLSR models with dimensionalities r = 0, 1, 2 … A LVs, where A is given a priori, voluntary larger than the expected values of "optimal" a. In a recent study on six datasets of forage  19  , PLSR-averaging was shown more efficient than the usual procedure where the PLSR dimensionality a was determined by CV. In the past, PLSR-averaging was proposed on a simple local-PLSR pipeline  18. The present article proposes the extension to the pipeline kNN-LWPLSR  8. The article is organized as follows. Theoretical points on kNN-LWPLSR and PLSR-averaging are firstly presented. Then, the kNN-LWPLSR pipeline with averaging, say kNN-LWPLSR-AVG, is applied to an extensive NIR database (Cirad, Selmet research unit) on chemical composition of European and tropical forages and feed.
Datasets The method is assessed on a NIR database built by Cirad (Selmet research unit) on European and tropical forages and feed. The absorbance spectra were collected on dried and grounded materials using Foss Instruments 5000 or 6500 models in the spectral range 1100 to 2498 nm (2 nm steps). The database objective is to predict twelve variables of chemical composition (Table  1 ). The total size of the dataset was N = 18813 observations but, depending on the response variables, available data ranged from N = 2020 observations to N = 18055 observations (Table  1 ). After preliminary exploration, a preprocessing was applied to the spectra consisting to a standard normal variate (SNV) transformation followed by a Savitzky-Golay 2nd derivation (polynomial of order 3, and window of 21 spectral points).
kNN-LWPLSR-AVG The same values as above were considered for h and k. For each xnew, the prediction was the average of the predictions of the models (LWPLSRs on the neighborhood) with nlv = 0, …, 20 LVs (A = 20), i. e. the average of 21 predictions. kNN-LWPLSR-AVG with omnibus strategy Some applications require to process many training datasets and response variables. This is for instance the case of platforms of automated predictions when there is frequent updating of the training data. In such a situation, PLSR-averaging predictions can be facilitated in practice by using an "omnibus" strategy  19. This strategy consists in defining one unique model, i. e. a set of a priori parameters values, that is applied blindly to all the training sets and response variables, instead of trying to optimize the model for each dataset and variable.
The defined omnibus model is expected (ideally) to have an acceptable efficiency for most of the datasets/variables to predict, even if not specifically optimal. In this article, an omnibus strategy was applied to kNN-LWPLSR-AVG and compared the two above methods (kNN-LWPLSR and kNN-LWPLSR-AVG) optimized by gridsearch. From preliminary studies on forages (sorghum and other data not considered in this article  8, 19  ), the omnibus parameters were set to h = 1 and k = 1000 neighbors.
Although more elaborated weightings can be considered, they are not always more efficient  19  and, in the context of local-PLSR, can become too time consuming to compute. For instance, additional computations (not detailed in this article) on the study dataset showed that a weighting based on the CV-errors  19  did not improve the predictions. Moreover, a weighting based on AIC  19  required several days of computation to run the nrep = 30 replications, which is not realistic in practice. The uniform weighting seems to be a good compromise for kNN-LWPLSR-AVG. To reinforce the finding of this article, kNN-LWPLSR-AVG should be evaluated on other agronomic materials than mixed forages and feed, with the same level of heterogeneity or, conversely, with less complexity. In this last case, the method may be less advantageous.
3% of the dataset) of 22 groups contained at least one T6SS cluster (Figure  5 ). Based on the phylogenetic analysis of TssC (Figure  6 ), T6SS clusters were classified into three families i1 (n=5), i3 (n=10), and i4 (n=57). Just four species among the 28 groups of S. maltophilia species complex contain one T6SS except group 65 which contains two T6SS belonging to two different taxonomic groups (i3 and i4). All S. rhizophila strains (n=5) have a T6SS related to group i4.
Genomic analysis of S. rhizophila CFBP13503 T6SS and effector prediction The genomic sequence of S. rhizophila CFBP13503 (SAMN09062466) was initially obtained through paired-end Illumina sequencing  (Torres-Cortés et al. , 2019). To circularize the genomic sequence of CFBP13503, PacBio sequencing was performed on an RS2 machine (Genotoul, Castanet-Tolosan, France). PacBio reads were filtered and demultiplexed using the ccs v6. 3. 0 and lima v2. 5. 1 tools of the PacBio SMRT Tools v11. 0. 0. 146107 toolkit and then assembled and circularized using Flye v2. 9  (Kolmogorov et al. , 2019). The sequence start was fixed using the fixstart option of Circlator v1. 5. 1  (Hunt et al. , 2015). Polishing with PacBio reads was performed using Flye v2. 9  (Kolmogorov et al. , 2019). Polishing with Illumina HiSeq3000 short reads was done using Pilon v1.
24  (Walker et al. , 2014)  with the setting --mindepth 0. Genome annotation was performed with Prokka v1. 14. 6  (Seemann, 2014). T6SS components were identified by conducting NCBI BLASTP analysis on protein sequences. Effector-immunity encoding gene pairs and chaperones were identified by analyzing genes downstream of vgrG and PAAR motif-containing genes. The conserved domain database of the NCBI was used to identify T6SS-related conserved domains. Structural homology-based searches were made using Alphafold2  (Jumper et al. , 2021)  structure prediction of putative effectors followed by a DALI  (Holm et al. , 2023)  search analysis or using the Foldseek search server (van  Kempen et al. , 2023).
Phylogenetic analysis of T6SS A total of 991 genome sequences of Stenotrophomonas were downloaded from the NCBI. Genomes with fewer than 10 markers (Marker_lineage = f__Xanthomonadaceae) absent or in multicopy  (CheckM v1. 1. 6; Parks et al. , 2015)  were conserved for further analysis. A multilocus species tree was created with Automlst  (Alanjary et al. , 2019). Non-Stenotrophomonasaffiliated genomes were excluded from further analysis, resulting in a final dataset comprising 835 Stenotrophomonas genomes. Sequence relatedness between the selected 835 genomes sequences was assessed with KI-S  (Briand et al. , 2021)  using 50% of shared 15-mers, a proxy for delineating bacterial species. Genome sequences were annotated with Prokka 1. 14. 6  (Seemann, 2014). The presence of T6SS genomic clusters was predicted with MacSyFinder2  (Néron et al.
ACKNOWLEDGEMENTS This work was undertaken with support from the fish facility of  Zefix-LPHI (University of Montpellier ),  Catherine Gonzalez  ( LPHI, CNRS ) and the  qPCR Haut  Debit platform of the  University of Montpellier. We acknowledge the imaging facility  BioCampus Montpellier Ressources Imagerie (MRI) , member of the national infrastructure France-BioImaging supported by the  French National Research Agency  ( ANR-10-INSB-04 , " Investments for the future "). We thank  Dr. Ferric C. Fang  ( University of Washington School of Medicine ) for Salmonella strain SM022. Dr Laurent Manchon  for his advices for the RNAseq analysis. ICFO authors also acknowledge financial support from the  Spanish Ministry of Economy and Competitiveness  through the " Severo Ochoa" program for Centres of Excellence in R&D  ( CEX2019-000910-S ), from  Fundacio Privada Cellex ,  Fundacio Mir-Puig ,  Generalitat de Catalunya  through the  CERCA program , and  Laserlab-Europe   EU-H2020  ( 871124 ).
FUNDING This work has received funding from the  European Union 's  Horizon 2020 research and innovation programme  by the grant  Marie-Curie Innovative Training Network   ImageInLife: Grant  Agreement n°  721537  and under the  Marie Skłodowska-Curie Innovative Training Network grant Inflanet : Grant Agreement  n°955576. This work was also supported by the  French Agence Nationale de la Recherche  [ ANR-19-CE15-0005-01 , MacrophageDynamics], a grant from  Region Occitanie  ( REPERE « INFLANET   » ). MB, EJG and PL-A acknowledge financial support from the  Spanish Ministerio de Economía y Competitividad (MINECO)  through the " Severo Ochoa" programme for Centres of Excellence in R&D  ( CEX2019-000910-S ),  MINECO/FEDER   Ramon y Cajal programme  ( RYC-2015-17935 );  Laserlab-Europe   EU-H2020 GA  no.
A colored de Bruijn graph (also called a set of k-mer sets), is a set of k-mers with every k-mer assigned a set of colors. Colored de Bruijn graphs are used in a variety of applications, including variant calling, genome assembly, and database search. However, their size has posed a scalability challenge to algorithm developers and users. There have been numerous indexing data structures proposed that allow to store the graph compactly while supporting fast query operations. However, disk compression algorithms, which do not need to support queries on the compressed data and can thus be more space-efficient, have received little attention. The dearth of specialized compression tools has been a detriment to tool developers, tool users, and reproducibility efforts. In this paper, we develop a new tool that compresses colored de Bruijn graphs to disk, building on previous ideas for compression of k-mer sets and indexing colored de Bruijn graphs.
Introduction Modern methods for analyzing biological sequences often reduce the input dataset to a set of short, fixed length strings called k-mers. When working with a collection of such datasets E = (E 0 ,. , E |E|-1 ), it is fruitful to represent them as one union set of k-mers and, for each k-mer, the indices of the datasets to which it belongs. The set of indices of each k-mer is referred to as its color class, and E is referred to as a colored de Bruijn graph  [13]. A colored de Bruijn graph (cdBG) is commonly used to represent a sequence database, such as a collection of sequencing experiments or a collection of assembled genomes. For example, it is used by tools for inferring phylogenies  [29] , quantification of RNA expression  [23] , and studying the evolution of antimicrobial resistance  [4].
As sequence database sizes grow to petabytes  [22] , the cost of storing or transferring the data (e. g. on Amazon Web Services or in-house compute infrastructure) has underscored the need for efficient disk compression algorithms. Such costs are often prohibitive for smaller labs and, even for larger labs, limit the scale of data that can be analyzed. Large file sizes also hamper tool development, which relies on iterative loading/copying/modifying data, and reproducibility efforts, which require downloading and storing the data. For example, storing the 31-mers from 450,000 microbial genomes in compressed form takes about 12 Tb  [4]. Unfortunately, there has not been a lot of work to develop methods for disk compression of colored de Bruijn graphs. In contrast to disk compression, indexing cdBGs have received much attention  [19].
A slew of data structures have been developed, optimizing various metrics such as index size, construction time, or query time (see the survey  [19]  and its follow up  [18] ). Indexing data structures exploit the structure of cdBGs and use clever tricks to compress the color classes of similar k-mers. But they also carry a space overhead to efficiently support queries; since this is not needed for disk compression, indexing data structures are usually not competitive with custom made cdBG compression methods. The simplest option for compressing a cdBG is to compress each color (i. e. dataset) independently, using a compression tool designed for a single set of k-mers (e. g. [25] ). This approach can work well when k-mers tend to not be shared among colors. However, most cdBGs have a large overlap between the k-mers of various colors.
We exploit the fact that consecutive k-mers in an ESS have similar color classes in order to efficiently compress the color vectors of each k-mer. We evaluate ESS-color on a variety of datasets, including bacteria, fungi, human, and including whole genome sequencing data, metagenome sequencing data, and whole assembled genomes. ESS-color achieves better compression than all evaluated tools and on all datasets, with all other tools using ≥ 44% more space on at least one of our datasets. On some datasets the improvement over all other tools is quite large, e. g. for a gut metagenome, all the other tools use at ≥ 27% more space than ESS-color. Compressing each color independently uses between 1. 2x and 6. 9x more space than ESS-color. The absolute compression ratio is more than 26x on datasets of assembled genomes and between 1.
Evaluated tools and datasets As far as we are aware, there are two other tools designed for compressing colored de Bruijn graphs: KS  [16]  and GGCAT  [7]. We refer to the first tool as KS after the authors' last names  [16]. KS is limited to support only three k values  (15, 19, and 23) , so we compare against it for k = 23 but also evaluate ESS-color on a more practical k value of k = 31. For GGCAT, we additionally compressed its Fasta output file with MFC and its binary color table with gzip to maximize its compression ratio. We also compare ESS-color against the naive approach of compressing each color independently using the algorithm of  [25] , which we refer to as ESS-basic. Table  1  shows the datasets we use for evaluation and their properties. We chose five datasets so as to cover a broad range of input types.
Three of the datasets are from assembled genomes, one is from RNA-seq reads, and one is from metagenome reads. We used all k-mers from the three assemblies datasets and all k-mers that appear at least twice from the two read datasets. The datasets cover various species, from Bacteria to Fungi to Human. Concretely, we have 1) 100 arbitrarily selected E. coli strains from GenBank, 2) an arbitrary subset of 10 of those, 3) 20 arbitrarily selected fungi sequences from RefSeq, 4) gut microbiome read sets from nine individuals sequenced in  [21] , and 5) 19 paired-end, human, bulk RNA-seq short-read experiments previously used in  [2]. All accession numbers are listed in https://github. com/medvedevgroup/ESSColor/wiki/Experiments. Finally, all experiments were run on a server with an Intel(R) Xeon(R) CPU E5-2683 v4 @ 2.
Comparison against other disk compression tools Table  2  shows the bits per k-mer achieved by ESS-color compared with KS, GGCAT ESS-basic, and a p7zip compression of the original fasta file. ESS-color achieves better compression than all evaluated tools and on all datasets. No other tool was able to consistently achieve less than 44% space overhead compared to ESS-color. On some datasets the improvement over all other tools is quite large, e. g. for Gut (k = 23), all the other tools use at least 27% more space than ESS-color. Table  2  Compression results, in bits per k-mer. ESS-color is our new tool. ESS-basic is the non-integrative approach of compressing every color separately. KS is tool from  [16] , and a hyphen indicates that it does not support k > 23. fa. p7zip is the p7zip compression of the original data.
We show bits per k-mer, which is the total compressed size divided by the number of distinct k-mer in the input (i. e. |E|). Compression ratios can be inferred by comparing to the fa. p7zip column. The compression ratio of ESS-color relative to the original Fa. p7zip files varies (Table  2 ). For the read datasets, it is more than 15x, since high-coverage FASTA files are by their nature very redundant. For the assemblies datasets, the ratio is between 1x and 7x. We found that a good predictor of compressibility of assemblies is the percentage of k-mers that have exactly one color (Table  1 ). At one extreme, 93% of Fungi20 k-mers have exactly one color, and ESS-color achieves little improvement over fa. p7zip. At the other extreme, only 30% of Ecoli100 k-mers have exactly one color, and the compression ratio is relatively high at 7.
1x (for k = 23). This trend makes intuitive sense since single-color k-mers do not benefit from ESS-color's multi-color compression algorithm. KS is not as effective as ESS-color on our datasets, using between 1. 4x and 3. 3x more space than ESS-color (Table  2 ). We note that even though KS is also designed to exploit the fact that k-mers are shared across colors, it makes a different design trade-off compared to ESS-color. Specifically, it does not allow simplitigs to extend beyond a single color class (resulting in more space needed to store k-mers), but, in exchange, it is more efficient in storing color information. GGCAT is generally the closest competitor against ESS-color, using between 14 and 44% more space on the non-Fungi datasets (note that for Fungi all tools did well). Like ESS-color, it builds a kind of global class table, constructs an SPSS of the k-mers, and annotates each run of single-class k-mers with their class ID.