dataesr/scientific-paragraphs-categorization huggingface dataset:
Only the last is a deep learning algorithm. The package scikit-learn  [73]  is used for implementation. Self-training is when labeled data is used to build a first classifier, which predicts 'pseudo-labels' for unlabeled samples. Pseudo-labels are then used along with labels for rounds of training. Label propagation (LP)  [27]  is also implemented with scikit-learn. This is a transductive method that builds a fully-connected similarity graph over the training samples, and label information is propagated to unlabeled samples in the graph. 4. 1. 2. 2 SSL approaches. Few semi-supervised classification approaches can be applied on tabular data. We include two recent methods for which code was made available: VIME  [24]  and TabNet  [21]. VIME is designed to extend the use of self-and semi-supervised learning to tabular data.
It uses an encoder to learn a new representation of the data focusing on important features. It is intrinsically semi-supervised, combining a supervised and an unsupervised term in the loss. TabNet  [21]  is an attention-based model that learns which features are important for prediction. Through unsupervised preprocessing and supervised fine-tuning, TabNet can be used as a semi-supervised method. 4. 1. 2. 3 SOM approaches. For semi-supervised methods based on SOM, we include the only recent methods which made their code available, SS-SOM  [51]  and SuSi  [53]. SS-SOM is based on a growing SOM which switches between supervised and unsupervised learning depending on the nature of each sample. If the sample is unlabeled, the standard unsupervised SOM algorithm is performed. Otherwise, the label will influence the decision when the closest neuron is found: the label associated with the neuron might change, or a new neuron might be inserted into the map.
The method can be seen as intrinsically semi-supervised. Specifically, we use the variant Batch SS-SOM (BSSSOM)  [51]  as it is the only one with an explained python package  [74]  that can be included in our experimental protocol. SuSi  [53]  first trains an unsupervised SOM on all data, then uses a second SOM to associate labels to the neurons of the first map. It is a method using unsupervised preprocessing. The accompanying package  [75]  is used. Note that this method's computations on MNIST were extremely long, thus they are not presented here. A comparison of our method with implemented supervised classifiers can be found in supplementary file A, Section 2, as well as details on the hyperparameters used for the methods presented above. 4. 1. 3 Benchmark results. A3SOM was compared to the methods presented in the previous section.
We defined different levels of labeled data seen during the training phase as percentages of the total training data available. The labels of the remaining samples were set to -1. Thus, all semi-supervised methods were trained on the entire dataset, where only some of the samples were labeled. All methods were evaluated on the same validation sets, in which all labels were present. For each method, different sets of hyperparameters were implemented. We present the variant with the highest score for each percentage of labeled data. The methods were scored on their mean validation accuracy when performing five-fold cross-validation. Fig  6  gives the results obtained by A3SOM and the other classifiers on the datasets described in Section 4. The numerical values of the results, along with the standard deviation, are given in supplementary file A, Section 3.
As we can see on the figure, a general trend is that the accuracy of the different methods improves as more labeled data is included during training, which is predictable. 4. 1. 3. 1 Implemented baselines. LP, ST-RF and ST-SVM are not deep-learning methods; we expect them to perform well on smaller datasets, and less so on bigger datasets. LP behaves as anticipated. ST-RF actually performs well on most datasets except for Artificial and WDBC, for which scores are slightly lower. ST-SVM also reaches a good accuracy for the most part, with one notable exception on MNIST, where results are low; moreover, it was extremely slow to run. ST-MLP generally makes good predictions, although its performance is relatively low on Ionosphere. 4. 1. 3. 2. SSL approaches. VIME and TabNet both obtain good accuracies for MNIST and Iris.
VIME performs well on datasets Cardiotocography and WDBC; and TabNet on the Artificial dataset. On the other datasets, however, both methods are slower to increase and do not always catch up, even with 100% of labeled data. This is the case with Artificial for VIME, and with WDBC for TabNet. 4. 1. 3. 3 SOM approaches. SuSi seems to struggle on bigger datasets, and although BSSSOM obtains high scores on Iris and Cardiotocography, its performance is average on the other datasets. 4. 1. 3. 4 A3SOM. We observe that A3SOM is among the most accurate classifiers in our benchmark, and gives the best results for all datasets except for WDBC. On the latter dataset, A3SOM obtains slightly lower accuracies than ST-SVM or VIME for some labeled cases. When considering the accuracies averaged on all percentages of labeled data, A3SOM gives the best results for all datasets, as shown in Table  2.
Interpretation of abstained classification. One advantage of this work is its ability to identify and visualize potential new classes and ambiguous classification areas in the input data using local abstention options. To show these properties, we split the artificial dataset into a training and a test set. The training set is curated to include data from all six classes that compose the dataset, but the labels for class 1 are removed and considered unknown. In this experiment, class 1 represents the new class that our algorithm must discover with the distance abstention rule. We also randomly removed 90% of labels across all the remaining classes, so that the hidden class is not the only one with masked labels. For the distance rule, we found the highest predicted probability for each sample and associated it with the corresponding class.
4. 3. 2 Dataset construction. We obtained publicly-available data from The Cancer Genome Atlas (TCGA, https://www. cancer. gov/tcga), using the R package TCGAbiolinks  [77]. Specifically, we downloaded gene expression data of patients with breast cancer. This dataset presents the counts of 19,947 genes for 1,215 patients. Gene counts are normalized, which means that the counts were scaled to the total number of reads for each sample. We also downloaded clinical data, including subtype information, which was given for 1,083 patients. Patients are associated with a total of five subtypes: the ones described above and the 'Normal-like' subtype. Due to a low number of examples in our dataset and the fact that this subtype is not always described in articles on breast cancer, we decided to exclude it from our dataset.
As the Her2 subtype was also underrepresented, but important, we decided to randomly remove some samples from the other classes to make the dataset more balanced. This makes visualizations easier to interpret in the rest of this section. Gene expression datasets consist of many genes, but relatively few of them play a role in diseases. To reduce noise in our dataset, we decided to limit the number of features to only keep genes that have been shown to be relevant in breast cancer subtyping. The 50-gene signature described in  [78]  is widely used to classify breast tumors. Finally, the dataset was normalized before training.
A3SOM application. As in Section 4. 2, we processed the training dataset by removing the labels of an entire class, to manufacture a class that is represented in the data but for which the label is unknown. We chose to remove the labels for the Basal-like subtype as it is the one with the worst prognosis. We also removed at random 40% of labels in the dataset, no matter the class, so that Basal-like was not the only subtype with removed labels. This percentage is chosen based on the complexity of the genomic data and the low number of samples belonging to certain classes (enriched with Her2 in particular): we could remove more labels but some classes can become completely unlabeled or contain very few samples. We then trained the abstained mode of A3SOM to predict the three classes Luminal A, Luminal B and Her2-enriched (denoted LumA, LumB, Her2).
Discussion We present in this paper an original method that is, to our knowledge, the only method that can simultaneously handle semi-labeled data, perform abstained classification, provide an explanation for the prediction and produce visualization that helps analyze the model. We showed that our method is competitive to semi-supervised black-box approaches (VIME, TABNET,. ) in addition to providing explanation. We detail in the following the comparison between our approach and SOM-based methods. SuSi  [53]  was primarily designed to handle hyperspectral data, while A3SOM is generic and can be applied on different applications (A3SOM showed good results on various datasets). Another difference is that although the model does propose classification, so far the authors mainly focused on developing and optimizing their proposal for regression, which can explain why SuSi's results were relatively low in our classification benchmark (Section 4.
(Re-)Designing algorithms for specific pseudoknot classes Our pipeline for automated generation of DP folding equations given a fatgraph has been implemented using Python and Snakemake  [28]. The implementation is freely available at: https://gitlab. inria. fr/bmarchan/auto-dp Since the algorithms in  [33]  have been described in terms of a finite number of fatgraphs (called irreducible shadows in the paper), one can directly apply our method to obtain an efficient algorithm that covers the same class as gfold, namely 1-structures that are recursive expansions of the four fatgraphs of genus 1 corresponding to simple PK 'H' ([)], kissing hairpin 'K' ([)(]), three-knot 'L' ({[)}] and 'M' ([{)(]}) (here, represented in dot-bracket notation, i. e. corresponding opening and closing brackets correspond to arcs).
In all cases except the one denoted by (*), the complexity of folding is equal to O(n tw+1 ). For the kissing hairpins case, we are in the specific case where the most complex routine is the alignment of a "clique case" helix, which is done in O(n 4 ) despite a treewidth of 4. These examples are detailed in the Appendix, Figure  9. The DP equations for each of these examples have been automatically generated by a Python implementation of our pipeline, freely available at https://gitlab. inria. fr/bmarchan/auto-dp. O(n 8 )  [33] ). The complexity of 5-chain is remarkably low, when considering that previously described algorithms covering this configuration take O(n 8 ) (e. g. gfold's generalization to 2structures and a hypothetical blow-up of the Rivas and Eddy algorithm  [39]  to 6-dimensional instead of 4-dimensional DP matrix elements-both of which have never been implemented).
C Detailed examples Figure  8  shows a canonical tree decomposition for the minimal length-5 expansion, shown in the upper half of the figure, for the fatgraph showed in Figure  1. This tree decomposition is optimal, and was computed with  [43] , a solver that empirically works quite fast on RNA   1. The equations have been automatically generated, and the pipeline code is freely available at https://gitlab. inria. fr/bmarchan/auto-dp. In particular, the optimal tree decompositions were computed by  [43].
fr/bmarchan/auto-dp Funding This project has received funding from the  European Union 's  Horizon 2020 research and innovation programme  under the  Marie Skłodowska-Curie  grant agreement No  101029676 , and from the  French-Austrian PaRNAssus  project ( ANR-19-CE45-0023 ;  I 4520-N ) supported by the  ANR/FWF agencies.
In contrast, previous eQTL studies in chickens have been limited in sample size, the number of studied genomic features, and tissue/cell types  (36) (37) (38) (39) (40). For instance, in an intercross population of 125 chickens,  Johnsson et al. (2015)  identified 6,318 cis-eQTL that influence female femoral gene expression, as measured by microarrays  (36). To fully unlock the genetic code of the chicken genome, the Chicken Genotype-Tissue Expression (ChickenGTEx) project, as part of the international Farm animal GTEx (FarmGTEx) initiative  (41) , has been lunched to build a comprehensive reference panel of regulatory variants based on the chicken transcriptome in various biological contexts (e. g. , development, sex and environmental exposure). In this pilot study, through analyzing 7,015 bulk RNA-Seq datasets from 52 tissues/cell types (hereafter referred to as "tissues") and 2,869 whole genome sequences (WGS) from over 100 breeds/lines (hereafter referred to as "breeds") worldwide, we systematically associated approximately1.
Harmonizing large transcriptome and genome datasets in chickens We analyzed 8,668 bulk RNA-Seq samples using a uniform pipeline, yielding 304. 4 billion clean reads. After filtering out low-quality data, 7,015 samples remained for subsequent analyses, representing 28 tissues (fig. S1 , Tables  S1  and 2 ). Based on gene expressions, samples were clustered well regarding their tissue types (fig. 1a , fig. S2 ). Across all the tissues, an average of 23,056 (94. 7% of all annotated genes) genes were expressed (Transcripts per Million, TPM > 0. 1) (Table  S3 ), showing patterns of ubiquitous or tissue-specific expression (fig. S3 ). An average of 1,938 tissue-specific genes were then detected across tissues (fig. S3d ), and their functions recapitulated the known tissue biology (fig. 1b , fig.
In addition, out of 45 genes associated with Mendelian traits in chickens  (43) , 41 showed tissue-specific expression (fig. S5a ). For instance, SLCO1B3 is the causal gene of blue eggshell in chickens  (44) , which was specifically and highly expressed in the liver and had liver-specific promoters and enhancers (fig. S5b ). To further annotate the function of chicken genes with this extensive transcriptome data, we conducted gene co-expression network and transcript assembly analyses. Based on co-expression analysis, we identified 3,583 co-expression modules containing 25,023 genes, 41. 3% (10,332) of which were not functionally annotated in the current Gene Ontology (GO) database (figs. S6). In the set of 2,940 unannotated protein-coding genes, 56. 3%  (1, 654)  were able to be assigned to coexpression modules.
S7d  and g ). For instance, we observed a new transcript on chromosome 2 was highly and specifically expressed in the testis (fig. S7h ). To obtain genotypes of these RNA-Seq samples, we called ~9 million single nucleotide polymorphisms (SNPs) from bulk RNA-Seq data using the GATK best practice pipeline(45) (fig. S8 ). To impute missing genotypes, we built a chicken multi-breed genotype imputation reference panel consisting of 2,869 global WGS data sets, which had a similar population composition as the RNA-Seq data (fig. 1c , Table  S6 ). Adopting a missing rate of 0. 6, the imputation accuracy of 1. 5 million SNPs reached 97% (figs. S8b-h ). The independent datasets from three different chicken breeds confirmed a high concordance rate (> 90%) between imputed genotypes and those directly called from WGS data of the same individuals (fig.
72) (fig. 3a ). To elucidate molecular mechanisms of action behind these molQTL, we examined sequence ontology and multi-omics data in chickens, including 15 chromatin states predicted from 377 epigenetic data sets in 23 tissues  (30) , and 9,898 topologically associating domains (TADs) detected from high-throughput chromosome conformation capture (Hi-C) in three tissues (i. e. , muscle, liver and testis)  (52). As expected, conditionally independent molQTL were significantly enriched with various regulatory DNA sequences, including synonymous variants (1. 67-fold in lncQTL to 3. 03-fold in exQTL), 5'UTR variants (1. 82-fold in eQTL to 3. 64-fold in sQTL), 3'UTR variants (2. 29-fold in sQTL to 3. 77-fold in 3a'QTL), and non-coding (NC) transcripts (1. 48-fold in 3a'QTL to 2. 36-fold in exQTL).
Besides, we offer highly-useful visualization tool, Integrative Genomics Viewer (IGV)  (73)  for exploring molecular phenotypes, enhancer-gene interactions, chromatin states, epigenetic modifications, and publicly available GWAS results. The web portal also includes single-cell RNA-Seq data that were collected and analyzed from six chicken tissues, enabling users to query the expression of their desired genes at both the cellular and bulk tissue level. Additionally, we provide batch data download and advanced search options for data resource generated in this study, and will continue updating the database to ensure its future accuracy and relevance. Overall, this first GTEx resource in avian species serves as a valuable resource for a global atlas of regulatory variants in chickens and informs vertebrate genome evolution at the functional level, benefiting future research in animal, plant, and human genetic and biomedicine research.
Limitations and outlooks: The current ChickenGTEx provides the most expansive source of regulatory variants in the chicken genome. Some limitations and challenges remain in the genotype and molecular phenotype assessments. New chicken assemblies with more complete representation are becoming available with fewer computational limitations that we experienced using the GRCg6a reference genome (Ensembl version 102)  (12, (92) (93) (94). Future studies will consider long-read sequences to better resolve splice-variants  (95) (96) (97) , and pangenome references to annotate complex structural variants  (98) , mobile element variation  (99) , and short tandem repeats  (92, 100). In addition, it would be of great interest to investigate the functional impacts of rare and somatic variants on molecular phenotypes, where multi-tissue samples are collected from the same individuals with deep WGS data available.
The fully developed ChickenGTEx will contribute substantially to research in complextrait genetics, animal breeding, functional biology, and vertebrate genome evolution at the functional level. All raw data analyzed in this study are publicly available for download without restrictions from SRA (https://www. ncbi. nlm. nih. gov/sra/) and NGDC BioProject (https://bigd. big. ac. cn/bioproject/) databases. Details of RNA-Seq, WGS, ChIP-Seq peaks and single-cell RNA-Seq can be found in Table  S1 , S6, S7 and S15, respectively. All processed data and the full summary statistics of molQTL mapping and genotype imputation reference panel are available at http://chicken. farmgtex. org. All the computational scripts and codes for RNA-Seq, WGS, single-cell RNA-Seq and Hi-C datasets analyses, as well as the respective quality control, molecular phenotype normalization, genotype imputation, molQTL mapping, functional enrichment, colocalization, SMR and TWAS are available at the FarmGTEx GitHub website (https://github.
RNA-Seq data analyses and molecular phenotype definition We downloaded 8,338 RNA-Seq data sets from the Sequence Read Archive (SRA, https://www. ncbi. nlm. nih. gov/sra)and 140 public data sets from the Genome Sequence Archive (GSA, https://ngdc. cncb. ac. cn/gsa/). We also included 155 newly-generated RNA-Seq data sets. The metadata relating to all the RNA-Seq samples is summarized in Table  S1. For quality control, we removed adaptors and trimmed low-quality reads using Trim Galore (v0. 6. 6, https://github. com/FelixKrueger/TrimGalore) with options of "--gzip --trim-n --length 30 --clip_R1 3 --clip_R2 3 --three_prime_clip_R1 3 --three_prime_clip_R2 3". We aligned the clean reads to the GRCg6a reference genome (Ensembl version 102) using STAR (v2. 7. 7a)(111) with parameters of "--quantMode GeneCounts --chimSegmentMin 10 --chimOutType Junctions --chimOutJunctionFormat 1 --outFilterMismatchNmax 3".
For downstream analyses, only 7,015 samples with uniquely mapping rates ≥ 60 % and a number of clean reads > 500,000 after removing potentially mislabeled samples were kept. For each of these samples, we then obtained raw read counts and normalized expression (i. e. , Transcripts Per Million, TPM) of 16,779 PCGs annotated in the Ensembl v102 and 22,792 lncRNA genes annotated by FR-AgENCODE (http://www. fragencode. org/)(112), using featureCounts (v2. 0. 1)(113) and StringTie (v2. 1. 5)(114), respectively. Using the same software(113), we counted the total number of reads as a function of annotated exons, which were further transformed into TPM using TBtools(115). We performed the tree clustering of all the RNA-Seq samples using the GGTREE package(116). The distance between samples was measured by 1-r, where r was Pearson's correlation coefficient based on the log2(TPM+0.
The generated matrix of per individual counts was normalized and used for clustering samples based on 1-r, where r is the Pearson's correlation coefficient between samples. To link intron clusters to genes, we mapped their coordinates to the gene model provided by the FR-AgENCODE database (112) using the script "map_clusters_to_genes. R" (https://github. com/broadinstitute/gtex-pipeline). Afterward, we filtered out introns if no reads were detected in >50% of samples or the number of counts was less than max(10, 0. 1n) where n is the sample size. In addition, we discarded introns with low variability across samples: ∑i(|zi| < 0. 25) ≥ n-3 and ∑i(|zi| > 6) ≤ 3, where zi is the z-score of the i th cluster read fraction across individuals. The filtered counts were further normalized between samples using the script "prepare_phenotype_table.
Single-cell RNA-Seq data analyses We retrieved single-cell RNA-Seq data from the chicken heart (n = 7) (121) and muscle (n = 2) (122) from the public database. Raw sequencing data was processed by using the "count" function after preparing the genome annotation. gtf file (Ensembl v102) with the mkgtf tool of the Cell Ranger pipeline(123). The Seurat R package (v4. 0. 5)(124) was used for subsequent celltype identification. We first created the Seurat object based on the raw read count of each sample in a tissue using the CreateSeuratObject function. In this step, we filtered out cells with unique gene counts < 200 and with mitochondrial counts > 20% of the total counts. We then normalized raw counts of gene expression using the LogNormalize algorithm and further identified highly variable genes (HVG) using the FindVariableFeatures algorithm with default parameters.
The HVG count matrices of all samples for a given tissue were integrated and combined to form a single Seurat object using the FindIntegrationAnchors and IntegrateData functions. We scaled the integrated dataset using the ScaleData function, which was further used to run principal components analysis (PCA) with the RunPCA function. The top 15 PCs, where the percentage of variance explained tended to be constant based on the elbow plot by the JackStraw function, were selected for running Uniform Manifold Approximation and Projection (UMAP) analysis for cell clustering using the RunUMAP function. The nearest neighbors between cells were constructed using the FindNeighbors function and cell clusters were thus determined using the FindClusters function at a resolution of 0. 05. We manually assigned cell names based on original publications (121, 122) and the PanglaoDB database(125).
Finally, cell clusters were visualized using the UMAP algorithm with the DimPlot function. To further deconvolute bulk RNA-Seq data using single-cell RNA-Seq data, we first created a signature matrix using the CIBERSORTx tool (126) with default parameters. Using the "Impute Cell Fractions" from the same tool, we imputed cell fractions with the custom mode and 1000 permutations after uploading the gene expression matrix for the bulk RNA-Seq data.