dataesr/scientific-paragraphs-categorization huggingface dataset:
To ensure independence from the training set, testing peptides were manually excluded from training data (STAR Methods). Both methods performed comparably (AUC = 0. 97 for RBM vs 0. 95 for NetMHC, 1% PPV = 0. 52 for RBM vs 0. 53 for NetMHC). When there is limited or no data for a given HLA allele, NetMHCpan extrapolates from the most similar HLA allele for which data are available. Fig. 1D  shows that NetMHCpan performance degrades with the distance between the queried HLA and that nearest neighbor in the training dataset  [22]. By contrast, the RBM performs equally well on all HLAs, which is expected since in that case the distance to the training dataset is always zero. This highlights the importance of being able to flexibly and rapidly train models on new data, especially for HLA alleles that are poorly covered in previously available datasets.
SARS-CoV-2 epitope discovery Predicting which antigens are presented by virus-infected cells is key to the rational design of vaccines targeting these antigens. We tested RBM's ability to perform this task on the example of SARS-CoV-2, using recent in vitro measurements of binding stability  [24]  of SARS-CoV-2 candidate epitopes (selected using NetMHC4. 0  [7]  and NetMHCpan4. 0  [8] , see STAR Methods). We first trained an allele-specific RBM for each HLA-I involved in the experiment. Since the experiment measures binding, we chose our training sets as binding assay (BA) datasets, as well as MS datasets that were reweighted to make them comparable to BA (correcting for amino-acid frequency biases, see STAR Methods). We then attributed an RBM score to each peptide, and compared it to its experimental binding stability (Supplementary Fig.
RBM offers useful low-dimension representations In addition to providing a presentation score, RBM hidden units allow for mapping peptides onto a lower dimensional "representation space" given by the inputs to the RBM hidden units  [16]  (see STAR Methods). The full potential of this representation is best illustrated on an RBM trained on multi-allelic data, where peptides may be bound to up to 6 different HLA-I proteins. Fig. 2A  shows an example of such a representation projected in 2 dimensions, on a synthetic dataset obtained by pooling peptides from 6 HLA restrictions in IEDB. The low-dimensional projection organizes peptides into 6 well-defined clusters, reflecting the HLA-binding specificities present in the sample. Each HLA recognizes specific "sequence motifs" (patterns of preferred residues at each position) in the presented peptides.
HLA classification with RBM-MHC This low-dimensional representation of peptides suggests an efficient way to classify them by HLA-I specificity, using the annotation of a small number of sequences by their HLA preference. In practice these annotated sequences may come from other experiments or public databases as IEDB  [13]. Using these annotations, we train a linear classifier to predict the HLA restriction of individual peptides from their RBM representation (see STAR Methods, Fig. 2A , Supplementary Figs. 2A, 3C-E ). We refer to this architecture, which combines the RBM (trained on unannotated peptides) and the HLA-I classifier (trained on a few annotated ones), as RBM-MHC. To test performance in a case where the ground truth is known, we trained RBMs on 10 synthetic "single-individual" samples pooling together IEDB 9-mer antigens presented by 6 HLA-I proteins per individual, covering 43 different alleles in total (12 HLA-A, 17 HLA-B, 14 HLA-C, see STAR Methods and Supplementary Table  2 ).
2C-E ). We next extended our HLA type predictions to peptides of variable length (8-11 residues). Multiallelic datasets pose the additional challenge that sequences must be aligned together across different HLA motifs. To optimize this multiple sequence alignment, we resorted to an iterative procedure, where the sequence alignment is refined using the the HLA assignments from a first iteration of RBM-MHC, after which a second RBM-MHC is trained to obtain the final model. This procedure allows the steps of HLA classification and alignment to inform each other (STAR Methods and Supplementary Fig. 1E ). We benchmarked our alignment routine against other well established alignment strategies (based on the MAFFT software  [29] ) and we verified that our routine ensures higher HLA classification performance, even without the refinement step (STAR Methods and Supplementary Fig.
RBM-MHC outperforms MixMHCp2. 1  [10] [11] [12]  in terms of both overall accuracy and stability across the 10 synthetic datasets (Fig. 2C , Supplementary Fig. 7 ). This is in part thanks to the 10% labeled data it exploits. The RBM also captures global sequence correlations that MixMHCp's independent-site models miss, which are important for correctly classifying antigens across alleles with similar binding motifs. The major drop in MixMHCp2. 1 performance occurs precisely in datasets that mix samesupertype alleles  [30]  (HLA-B*27:05 and HLA-B*39:01 belonging to supertype B27, HLA-B*40:01 and HLA-B*44:03 belonging to B44, see Supplementary Table  2 ). RBM-MHC also performs better in terms of AUC than NetMHCpan4. 1  [9] , which can be applied here to predict binding to each of the 6 alleles in our datasets, and stays comparable to NetMHCpan4.
Application to single-patient cancer immunopeptidome To assess the relevance of our approach in a clinical setting, we considered single-patient, melanomaassociated immunopeptidomic datasets  [26, 27, 31] , complemented with patient HLA typing and Whole Exome Sequencing (WES) of tumorous cells, where in total 11 neoantigens were identified. We tested the RBM-MHC approach for motif reconstruction. Since in this case true peptide-HLA associations are unknown, model performance is evaluated through the correlation between the predicted motifs and motifs reconstructed from IEDB mono-allelic data (Fig. 2D-E , Supplementary Fig. 8 ). RBM-MHC, MixMHCp2. 1 and NetMHCpan4. 1 perform comparably, with the only exception of sample Mel8, where MixMHCp2. 1 merges antigens specific to the 2 HLA-C into the same cluster, causing a drop in the average ρ (Fig.
2C-E ). These generated peptides could be experimentally tested in future, providing additional evidence for the validity of the model  [32]. RBM-MHC's use will also depend on the choice of training dataset. Single-allele datasets allow for training allele-specific models to score antigen presentation by a specific HLA allele. In this way, we have proposed new putative SARS-CoV-2 epitopes for HLA-C alleles that could be tested experimentally (Supplementary Table  1 ). Multi-allelic datasets (from e. g. peptidome mass spectrometry of clinical samples), allow for training models to score antigen presentation in a given donor (with all its HLA-I alleles), and to predict the HLA-binding specificity of given peptides. We benchmarked our approach on several examples of datasets. In single-allele datasets, we found that RBM-MHC can perform similarly to established methods, like NetMHCpan, for frequent, well-characterized alleles (Figs.
1D-G , Supplementary Figs. 5  and 6 ). This finding is remarkable, since NetMHC training is fully supervised, incorporates more information such as binding affinities, and uses positive (binders) as well as negative (non-binders) examples. The number of binders per allele in the version NetMHCpan4. 0 ranges between ∼ 50 and ∼ 9500  [8] , while RBM-MHC is trained on datasets of presented peptides only, making our approach less data-demanding. The amount of 8-11mer peptides per allele considered in this work from IEDBderived MS records varies from ∼ 500 to ∼ 10 4 , with an average (2350) comparable to NetMHCpan4. 0 positive examples. RBM-MHC is a more flexible machine-learning scheme that can be easily trained on newly-produced datasets, tracking the fast growing number of available datasets to improve its predictive power, especially for previously under-represented HLA-I alleles (Figs.
These results support our main conclusion that our approach is especially useful in scenarios where new data become available but current methods are not updated yet to cover the corresponding alleles with good accuracy, as was the case when data from Refs. [6, 24]  came out while NetMHCpan4. 0 was the latest version. RBM-MHC can also be applied to unannotated, moderate-size multi-allelic datasets available from clinical studies, for which there may exist only a limited number of HLA annotations in existing databases (< 100) for the patient's rarer HLA alleles. We have shown that RBM-MHC efficiently exploits the statistical information in these samples and combines it with limited annotation information to deliver accurate and stable predictions of HLA-I binding (Figs. 2C-E , Supplementary Figs.
7  and 8 , Supplementary Tables  2  and 3 ). The flexibility of choice of the training dataset and corresponding applications is even broader. The method is not limited to MS datasets but can be trained on datasets from binding affinity assays to customize presentation score predictions towards the identification of high-affinity ligands (see Fig. 1F ). If only MS data are available to build a predictor for this task, as is the case for HLA-C alleles in Fig. 1F , a commonly acknowledged issue is that biases of MS techniques can potentially affect the amino acid frequency distribution in training datasets and hence the predictor's results. To address this issue we developed, similarly to  [11] , a sequence re-weighting scheme to efficiently compensate for detection biases in MS and better score ligands tested in vitro (STAR Methods, Supplementary Fig.
4 ). Overall, our results show that the approach is useful for systematic applications with newly produced large-scale datasets covering an increasing range of HLA types. Future work will be devoted to also develop an extension to HLA class II presentation. Finally, since our method is designed to be trained on custom samples, it could be of relevance to produce sample-specific insights about the complexity of endogenous antigen presentation. As future direction, we will investigate how the probabilistic framework provided by RBM-MHC can be exploited to develop these insights in a quantitative manner. RBM projects sequences onto a representation in terms of "hidden units" (we selected hidden unit 8 and 10 for illustration) through the set of learnt weights. In this representation space, each cluster groups together antigens with the same HLA-binding specificity (given by the color code).
Linear classification, guided by the knowledge on the HLA-I restriction of a few antigens in each cluster ("labels"), is performed through the HLA-I classifier to predict the HLA-I type of all antigens. Data: IEDB-derived dataset for Haplotype 1, Supplementary Table  2. (B) RBM distinguishes 2 subclasses in HLA-B*51:01-binding antigens. The HLA-B*51:01-peptide bond can be established  [12] : (i) via the interaction of the HLA-B*51:01 residue 62 with peptide position 2, requiring a hydrophilic residue there, typically Alanine (A), and a polar or negatively charged residue, typically Aspartic Acid (D), at position 1; (ii) with R62 sidechain facing the solvent, requiring co-occurrence at positions 1-2 of hydrophobic residues, typically Proline (P), at position 2). Inspection of the inputs to the 10th hidden unit (h 10 ), found by projecting peptides predicted by RBM-MHC as HLA-B*51:01-specific onto the corresponding weights (STAR Methods, Eq.
4), reveals a bimodal distribution, enabling the discrimination of the "hydrophilic/charged" pattern (i) from the "hydrophobic" pattern (ii), as recapitulated by the sequence logos. (C) Performance (AUC) of 6 methods for HLA-I prediction on 9-mers in 10 synthetic-individual samples, each carrying 6 HLA-I covering A,B,C alleles (see Supplementary Table  2 ). Bars are standard deviations over the 10 datasets. (D) Sequence logos of clusters found with RBM-MHC trained on melanoma-associated sample 12T  [26, 27]  and IEDB mono-allelic data; ρ gives the Pearson correlation between respective amino acid frequencies. (E) Performances (average Pearson correlation ρ over clusters) for 3 samples from  [26, 27, 31]. In C,E the MixMHCp and NetMHC versions used are respectively MixMHCp2. 1 and NetMHCpan4.
Schematic outline of the RBM-MHC pipeline The essential steps of the RBM-MHC application pipeline are: 1. Collection of training datasets. RBM-MHC can be trained on patient-derived clinical samples or on datasets collected from public repositories -here we constructed training datasets from mass spectrometry and binding affinity assays available in IEDB. See Supplementary Fig. 1A  and STAR Methods section "Dataset collection from IEDB and preparation" for a detailed description. 2. Alignment. RBM-MHC works with fixed-length sequences, hence peptide sequences are first reduced to the same length through an alignment routine, see Supplementary Fig. 1E  and STAR Methods section "Antigen sequence alignment" for a detailed description. 3. RBM-MHC algorithm. The full RBM-MHC architecture (Fig. 2A ) combines a Restricted Boltzmann Machine (RBM) and a HLA-I classifier, trained in successive steps.
Mass spectrometry datasets The search in IEDB for immunopeptidomic data from mass spectrometry (MS) was performed as follows. The full set of curated HLA-I ligands was downloaded from IEDB (file mhc_ligand_full. csv from http://www. iedb. org/database_export_v3. php, accessed in July 2020). In this IEDB file, we looked for linear, human peptides eluted from cells and detected by MS techniques (field Assay Group = "ligand presentation", field Method/Technique = "cellular MHC/mass spectrometry", "mass spectrometry", "secreted MHC/mass spectrometry"). Among these data, we prioritized HLA-specific peptides from mono-allelic sources, in such a way that the assignment of HLA-binding specificity is unambiguous and does not rely on additional in silico predictions. To do this in practice, we set the field Allele Evidence Code = "Single allele present", indicating that antigen presenting cells are known to only express a single HLA-I molecule, as is the case for mono-allelic cell lines  [4, 6].
If, for a given allele, we found less than a minimal number (set to 300) of sequences among mono-allelic-source data, we extended the search to peptides with Allele Evidence Code = "Allele Specific Purification", since this procedure attaches greater confidence to the HLA assignment than its inference by in silico methods. Only for one allele (HLA-B*39:01) did we extend the search to all other MS data, including the evidence code "Inferred by motif or alleles present". MS datasets filtered through these steps were used to train allele-specific RBM presentation models (as in Figs. 1C-F ) and multi-allelic models (as in Fig. 2C-E ). In a multi-allelic setting, we first tested motif reconstruction by considering 10 "synthetic-individual" datasets of antigens with known HLA-I specificity to assess the RBM-MHC classification performance by comparing RBM-MHC predictions against the known HLA-I specificity.
These datasets were built by collecting all IEDB antigens (searched as above) associated to 2 haplotypes, i. e. , combinations of an A, a B, a C HLA allele observed to co-occur in the human population to preserve linkage (see Supplementary Table  2 ). Information on haplotypes co-occurrence was found at allelefrequencies. net  [34]. To apply the RBM-MHC method to multi-allelic, patient-derived immunopeptidomic samples, we sought to have a small amount of labeled peptides, here set to 10%. To this end, we either retrieved a HLA annotation in IEDB for portions of the samples or, when this was not possible, we added to each of them labeled peptides from IEDB (searched as above) for the 6 HLA-I given by the patient's HLA typing. The RBM-MHC training set was then defined as this minimally extended dataset, to guide the learning of the HLA-I classifier by the labeled peptides, whose predictions are used to reconstruct HLA-I motifs in the original dataset.
Binding assay datasets Records of binding affinity (BA) assays from IEDB (as of July 2020) were filtered following  [6] , i. e. selecting peptides annotated with a quantitative measure of binding dissociation constants < 500 nM and excluding assay types that led to documented discrepancies between predicted and effective affinity ("purified MHC/direct/radioactivity/dissociation constant KD", "purified MHC/direct/fluorescence/half maximal effective concentration (EC50)", "cellular MHC/direct/fluorescence/ half maximal effective concentration (EC50)"). BA datasets were used to train allele-specific models of Figs. 1F-G.
Antigen sequence alignment The typical length of HLA-I peptides, structurally constrained by the MHC-I binding cleft, is generally recognized to be 8 to 11 amino acids  [6, 7, 17, 18] , with 9-mers being the most abundant except for very few exceptions  [6]. Hence we focus on datasets containing 8-11-mers (Supplementary Figs. 1B-D ) and an alignment procedure is implemented to reduce peptide sequences to the typical length of 9 residues. This choice of a 9-mer-centered alignment is also consistent with the treatment of variable-length class I peptides by other algorithms as MixMHCp, which scores their positions based on 9-mer motifs  [12] , and NetMHC, which applies to sequences with length different from 9 the insertions and deletions that give optimal predicted scores for presentation by a given allele  [7].
1E ). We estimate Position Weight Matrix (PWM) profiles of subsets of sequences with the same length and we align these profiles, using respectively the functions seqprofile and profalign (with default options) of the Matlab Bioinformatics Toolbox (release R2018b). The alignment is made progressively from the minimal length considered (here 8 residues) to the maximal one (here 11 residues) by inserting gaps in shorter profiles, resulting in an alignment of the maximal length considered  (11). This alignment is used as seed to learn a Hidden Markov Model (HMM) profile of the reference length = 9 by appeal to the routines hmmprofstruct and hmmprofestimate (with default parameters) of the Matlab Bioinformatics Toolbox. • Align sequences to the HMM profile, Step 2 in Supplementary Fig. 1E. Sequences with length different from 9 are re-aligned to the HMM profile relying on the position-specific insertion/deletion probabilities of the HMM (by the hmmprofalign function).
• Re-align peptides based on the best HMM alignment scores (Step 4 in Supplementary Fig. 1E ). We take every unlabeled peptide and we consider the alignment to each of the classes' seeds and the corresponding HMM alignment score (both given by the hmmprofalign function). We retain, for each peptide, the alignment with the highest score. Such best scoring alignment can be to a class that is different from the classifier-predicted one, allowing us to re-classify peptides more accurately in the subsequent iteration. This step helps therefore correct classification errors arising from the initial, suboptimal alignment by means of allele-specific HMM alignment scores. • Repeat the RBM-MHC training after the re-alignment. In the 10 "synthetic-individual" datasets considered for testing motif reconstruction (Supplementary Table  2 ), high classification performance was reached already at the first iteration and we observed a further, systematic improvement after 1 re-alignment step, see Supplementary Fig.
1F. For 2 or more iterations, there are cases in which the classification performance is degraded, therefore in our motif reconstruction applications (Figs. 2C-E ) we re-align and re-train once. The alignment routine is designed in such a way as to be easily tuned to a different reference length and to the inclusion of longer peptides, depending on the particular length preferences of the alleles of interest and on data availability. For instance, in Supplementary Fig. 1H  we show that HLA classification performance is rather stable with respect to the choice of the reference length in the range 8-11 residues and could even slightly increase for reference lengths > 9. We recall however that the choice of reference length = 9 is made to reflect the typical length of class I peptides: peptide length distributions of the datasets considered in this work confirm that 9 residues is by far the most abundant length (Supplementary Figs.
1B-D ). To test the quality our alignment routine, we compared its performance at HLA classification to that of other common alignment software, MAFFT (latest version 7. 471  [29] ) and HMMER (http: //hmmer. org/). Default options of these methods tend to produce very gapped alignments, that might cause a rather drastic loss of information on the original peptides and hence affect the RBM-MHC classification performance. In contrast, the progressive alignment of profiles obtained from subsets of sequences of the same length (Step 1 in Supplementary Fig. 1E ) allows for well controled gap insertions. We found that large gap penalties are more suitable for applying MAFFT to the type of alignment of interest here, and yield better performance. After a grid search we set in MAFFT the penalty for opening a gap to 8, the one for extending a gap to 12.
RBM-MHC algorithm Restricted Boltzmann Machine (RBM) The RBM learns an underlying probability model for the antigen sequences in the training datasets, in our case the HLA-I presented antigens. A RBM  [14, 15]  consists of one layer of N v observed units v = {v i } N v i=1 and one layer of N h hidden units h = {h µ } N h µ=1 connected by weights W = {w iµ } (see Supplementary Fig. 2A ). The observed units v stand here for the antigen sequences to model, hence N v = 9 and the number of "symbols", q, for each observed unit is 21 (20 amino acids and the gap). Mathematically, the model is defined by a joint probability over presented antigen sequences and hidden units: P (v, h) ∼ exp   N v i=1 g i (v i ) - N h µ=1 U µ (h µ ) + i,µ h µ w iµ (v i )   (1) where g i (v i ) is a matrix of N v × q local "potentials" (biases) acting on observed units, U µ (h µ ) are N h local potentials on hidden units and the weights w iµ (v i ) (arranged in a tensor of size N v × N h × q) couple hidden and observed units.
This leads to inferring the RBM probability distribution that optimally reproduces the statistics of the training dataset (see Supplementary Fig. 3H  for comparisons of data and model single-site frequency and pairwise connected correlations). During training, the contribution of sequences to L RBM can be re-weighted following a re-weigthing scheme we have conceived (see STAR Methods section "Sequence re-weighting scheme") to correct for amino-acid frequency biases in the training dataset, as the ones introduced by mass spectrometry (MS) detection. A regularization, i. e. a penalty term over the weight parameters is introduced to prevent overfitting. The function maximized during training then becomes: L RBM - λ 2 1 2q N v µ i,v |w iµ (v)| 2 (5) This regularization, which was first introduced in  [16] , plays effectively the role of a L 1 regularization, imposing sparsity of weights, with a strength that is adapted to increasing magnitude of weights, hence favouring homogeneity among hidden units (see  [16]  for more detailed explanations).
Examples of inferred sets of weights at different regularization strength λ 2 1 are provided in Supplementary Figs. 2B  and 3G. The package used for training, evaluating and visualising RBMs is an updated version of the one described in  [16]. The package was ported to Python3 and the optimization algorithm was changed from SGD to RMSprop (i. e. , to ADAM without momentum) with learning rate 5 • 10 -3 , β 1 = 0, β 2 = 0. 99, = 10 -3 ) (see  [36]  for definition of the parameters). The parameter values were chosen for their robust learning performance across a variety of datasets studied in previous works such as MNIST, Ising models, and MSAs of protein domains of various sizes. Overall, the adaptive learning rate of RMSprop/ADAM result in larger updates for the fields and weights attached to rare amino acids, and hence speeds up convergence.