dataesr/scientific-paragraphs-categorization huggingface dataset:
Comparison to RATE Recently, Paul et al  [10]  have proposed a novel algorithm "RATE" to determine immunogenic peptide:HLA complexes from ELISpot data. We applied the RATE tool to both the Burkholderia and the Pseudomonas dataset. RATE returns two reports: a concise one, which contains the pHLA combinations who have a positive ELISpot relative frequency (RF) greater than two; and a complete report which ranks all peptide:HLA combinations in the population. We calculated the AUC for each dataset based on the RF of all the pHLA complexes whose posterior distributions passed the Beta(1,2) cut-off criterion described above. Validation on an HIV-1 dataset (HLA class I restricted CD8+ T cells) While BIITE was designed to address the issue of elucidating peptide:HLA immunogenicity for class II HLA, we hypothesized that it could solve a larger class of problems (see Discussion).
To explore the performance of BIITE on different datasets, we downloaded patient data and CD8 + T cell ELISpot outcomes for a range of HIV-1 peptides from  [16]. We used 2-digit level typing for all HLAs, except for HLA-B Ã 15 (where we used B Ã 15:03 and B Ã 15:10 if this subtyping was provided, and B Ã 15 else), HLA-B Ã 58 (B Ã 58:01 and B Ã 58:02) and HLA-A Ã 30 (A Ã 30:01 and A Ã 30:02). For other HLA alleles, we either lacked sufficient HLA-typing on the 4-digit level, or there was one dominant 4-digit allele; see S5 Table for an overview. We applied the algorithm to the 32 peptides used for validation in  [10]  to sample the posterior probability distribution on the hypothesis space (sample size 250,000). The output was interpreted as described above and peptide:HLA combinations that were predicted to be immunogenic were matched against entries in the LANL A-list of HIV-1 epitopes and the general LANL database.
Validation on Burkholderia dataset In order to test the BIITE algorithm, we used CD4 + T cell ELISpot data from a small cohort of exposed sero-positive individuals (N = 38). All overlapping 20-mer peptides from the BPSL2096 protein were tested (see Methods). In Fig  2A , we present the marginal posterior distributions of the immunogenicity (solid lines) for all HLA molecules in combination with a single, representative peptide (BPSL2096, 1-20), calculated using a uniform prior. Especially for rare alleles (see S1 Table ), the distributions are rather flat and uninformative. We also show the marginal posterior distributions, now assuming a non-uniform prior as described above (Fig 2A  , dashed lines ). The posterior distributions are now markedly more pronounced. This is of course driven by the prior assumptions, especially for the rare alleles.
Validation on Pseudomonas dataset We performed an additional analysis on a Pseudomonas dataset, comprised of N = 58 subjects. The AUCs we found were 68. 8% (without predictive prior), 79. 3% (with predictive prior) (see  Fig 3). The subjects in the Pseudomonas study were divided into two groups, based on the number of positive sputum tests they had delivered. In this study, individuals with a high number of sputum cultures positive for PA had a lower immune responsiveness (lower number of peptide that elicited a T cell response) compared to individuals that were never sputum culture positive for PA, indicating that there were two different clinical groups with different peptide immunogenicities. Hence, we applied our algorithm to each of the groups separately, and predicted that the algorithm would perform poorly on the data from group 2 (who had very low response rates in their ELISpot assays).
Determinants of performance Cohort size. To test the effect of the sample size of the ELISpot dataset, we modelled 17 in silico peptides based on a population of 200 subjects whose HLA class II haplotype was drawn (with replacement) from the Burkholderia cohort. We set peptide immunogenicity such that the odds ratio of having a positive ELISpot was similar to that in the Burkholderia dataset (S2 Fig). We then applied the algorithm to this new population and to subpopulations thereof. To validate the performance of the model, one needs to first decide which of the pHLA combinations are considered immunogenic. We did so by choosing a cut-off on the immunogenicity value E j of each simulated pHLA combination; different cut-offs were considered (see Methods). For each of these cut-offs, we construct ROC curves and compute the AUC.
cut-off. It is interesting to note that in our original Burkholderia dataset, where sample size was 38, we reached an AUC of 71%, which is comparable to the results found for sample sizes 30 and 50 here (72% and 75% respectively). Crossreactivity. We also investigated how the crossreactivity of the peptide affected the performance of the BIITE algorithm. Intuitively, we expect that is it easier to determine the HLA ligands of a peptide that is poorly reactive (ie, not many positive ELISpots). This is easiest to understand in the extreme case of immunogenicity values E j 2{0,1}: each negative ELISpot would indicate that none of the HLAs of the subject is immunogenic; while a positive ELISpot result only informs us that (at least) one of the subject's HLAs is immunogenic in combination with the peptide.
In this extreme case, when all the subjects produce positive ELISpots, it is impossible to attribute these positive outcomes to any HLA. To confirm the effect of the number of positive ELISpots, we ordered the peptides in the Burkholderia dataset based on the success rate (i. e. how many positive results they produced) in the ELISpot assay in our cohort, and grouped the peptides accordingly (see S3A Fig). We can now compute the performance of the algorithm on each of these subsets. Following the reasoning above, we predict that the AUC will be negatively correlated with the success rate (S3B Fig) ;.
this is indeed the case (S3C Fig). Note that if very few positive ELISpots are observed, the algorithm might also underperform: in the extreme case of observing one positive ELISpot, there are many candidate HLAs in that subject that could be presenting the peptide. If these HLA molecules are present at similar frequencies in the cohort, the algorithm is unable to distinguish which of the possible pHLA complexes is responsible for triggering the CD4 + T cell response. Comparison to RATE. To our knowledge, there exists only one other algorithm, RATE  [10] , which uses ELISpot data to infer immunogenicity of pHLA complexes. In contrast with our Bayesian approach (which allows for inclusion of prior data and considers all HLAs simultaneously), the RATE software uses the frequentist exact Fisher test to compare the odds ratios (and the relative frequency, RF) of having a positive ELISpot in the HLA+ and HLA-groups.
We first applied RATE to the Burkholderia dataset; it returned two potential pHLA restrictions with high RF: peptide5:DRB1 Ã 01 and peptide5:DQB1 Ã 04. Transgenic mouse data is only available for the first combination and was negative. Both combinations have a high (>0. 9) posterior mode in our analysis, but lack power (Kullback-Leibler divergence below Beta cut-off) as both occur only twice in our cohort. The overall AUC based on RF was 53. 6% (compared to 71. 3% for BIITE). When applied to the Pseudomonas dataset, RATE returned 54 pHLA combinations with RF greater than 2; unfortunately, there is no transgenic mouse data available for any of these combinations. Based on the pHLA combinations that were tested in transgenic mice, we found that RATE reached an AUC of 49% (compared to 68. 8% for BIITE).
In the two subcohorts of the Pseudomonas dataset, RATE reached AUCs of 55% in group 1 (compared to 69% or 73% for BIITE, depending on the inclusion of a prior based on binding predictions, see above) and 29% in group 2 (compared to 32% or 69% for BIITE). Comparison to NetMHCIIPan. Immunogenicity and binding affinity are correlated and so it is also instructive to compare our algorithm with the best available affinity predictor, namely NetMHCIIPan  [9, 17]. We used NetMHCIIPan to construct ROC curves, based on the binding threshold, for the pHLA combinations we had previously considered. For the Burkholderia data, this yielded an AUC of 67. 7% (compared to 71. 3% from our analysis). Since we also obtained experimental binding data for several pHLA combinations from the Burkholderia dataset, we were interested in the cases where the binding assay and transgenic mouse models gave discordant results.
As for our analysis, 16 combinations were informative (reached the Kullback-Leibler cut-off, S3C Table ); of these, two (12. 5%) were incorrectly predicted to be immunogenic. This indicates that BIITE can successfully distinguish between binding and immunogenicity. Another important benefit of BIITE compared to NetMHCIIPan is that the results of BIITE are cohort-specific. That is, immunogenicity in two cohorts (e. g. controllers and non-controllers) infected with the same pathogen can be compared using BIITE. BIITE may find the same HLA:peptide combination to be immunogenic in one cohort but not in another allowing an analysis of differing immune control between the cohorts whereas NetMHCIIPan can only be used to determine whether a particular HLA:peptide combination is ever immunogenic. This is illustrated by our work on the PA dataset.
In that dataset, we could distinguish clinically between two groups which were marked by different ELISpot response rates (group 2 i. e. "non-controllers" were poorly responsive with few positive ELISpots). Since NetMHCIIPan does not use cohort data, it produces the same peptide:HLA binding predictions (which in turn serve as a proxy for immunogenicity) for both groups and hence fails to capture the real differences in immunogenicity between the two groups. If we use transgenic mouse data as the validation dataset for the poorly responsive group 2 then NetMHCIIPan reached an AUC of 81. 3%, compared to BIITE's AUC of 68. 8% (S4 Fig); however this analysis is misleading. The cohort produces very few positive ELISpot responses and, accordingly, BIITE identifies very few HLA:peptide combinations as being immunogenic compared to NetMHCIIPan.
Arguably, BIITE's prediction of very few immunogenic peptide:HLA combinations compared to NetMH-CIIPan is a more accurate representation of immunogenicity in this poorly responsive group. Effect of HLA characteristics on model outcomes. Two concerns might arise regarding the performance of our model: firstly, the model might prefer to assign higher immunogenicity values to HLAs which are more common, as they might 'hide' less common but immunogenic alleles in their hosts. Secondly, we want to ensure our analysis goes beyond a simple 'counting' algorithm, in which one considers immunogenic exactly those pHLA combinations for which the highest percentage of HLA carriers produce positive ELISpots. In order to confirm that the model is able to detect immunogenicity of peptide:HLA combinations where the HLA is rare in the population, we plotted the predicted immunogenicity (posterior mode) of each pHLA combination in the Burkholderia data set against the number of carriers of the HLA.
Validation on an HIV-1 dataset As described in the methods, we also analysed the HLA-I dataset that was analysed in  [10]. For each peptide, HLAs were ranked by descending posterior mode provided by BIITE. To validate the results, we compared the outcome to the LANL A-list  [18]  of best defined epitopes in HIV-1. This is not ideal, since a peptide:HLA combination can be immunogenic without being featured on this A-list (i. e. we will tend to overestimate false positives); nevertheless, we used this list to determine lower bounds on the AUC. Both BIITE (AUC = 81%) and RATE (AUC = 77%) performed well and were able to identify A-list peptide:HLA combinations (S6 Table ). However, RATE's 'concise output file' (S7 Table) included a number of peptide:HLA combinations which are most likely not immunogenic but appear to be so due to linkage in the HLA locus (highlighted in red in S7 Table.
ELISpot assays use ex vivo PBMCs from exposed hosts to test peptidic immunogenicity. In theory, this approach includes effects of peptide presentation pathway idiosyncrasies, binding affinities between peptide and HLA, and immunodominance. As a trade-off however, we lose the clarity of the experimental approaches where each HLA is studied independently, and it is difficult to assess which peptide:HLA combinations cause the observed ELISpot outcomes. Here, we have presented a Bayesian framework to perform this analysis. We have shown its accuracy on both simulated and experimental data, and have determined the effect of sample size. While higher sample sizes (100-200 subjects) would have yielded better results, the analysis on a very limited dataset of 38 Burkholderia-sero-positive exposed subjects performed as expected (AUC = 71%) from simulated data.
A caveat should be given regarding the composition of the data set: analysis on data sets with high rates of positive ELISpots suffer from poor identifiability. Nevertheless it is remarkable that with such a small dataset relatively high specificity and sensitivity could be achieved; while most algorithms for predicting immunogenicity or binding require training datasets with hundreds of data points per HLA allele, we typically have less than 10 data points per allele. Furthermore, unlike prediction algorithms, BIITE explicitly uses cohort data, which makes it suitable to interrogate differences in immunogenicity between patient groups responding differently to the same pathogen; this would be impossible for a binding prediction algorithm. This tool is available online as the R package BIITE and is, in essence, not ELISpot or HLA class II-specific.
In general, it can be used to attribute Boolean outcomes (such as therapeutic success or disease progression) to allelic variation on multiple genes. As an example, we have applied BIITE to an HLA class I ELISpot dataset (HIV-1). In general, to apply the algorithm to a new problem it is necessary to consider carefully the source of the priors if these are to be used; establish the length of the MH chain needed to obtain a representative sample of the posterior distribution; and to consider issues of sample size and collinearity in the dataset. It also allows for the addition of prior information ad libitum: we can envisage a scenario where not only binding predictions (from NetMHCIIPan or other binding prediction algorithms) but also binding data and transgenic mouse data are included in the prior.
Of course, conflicting binding information would result in different priors, and hence influence the outcome of the algorithm. However, the effect of the prior is dependent on sample size; in a suitably large dataset, the influence of the prior is outweighed by the likelihood factor. Alternatively, confidence in the prior can be expressed in the prior itself: the more 'flat' a prior, the less it will influence the posterior distribution. In summary, we have developed a general approach which can be used for analysing complex biological data. Whether or not a peptide:HLA combination is immunogenic involves a number of interconnected immunological processes. To sidestep this complexity, reductionist experimental approaches are often taken, e. g. in vitro stimulation with APCs expressing a single HLA molecule; these approaches do not recapitulate a number of, potentially important, biological details.
Acknowledgments We have used the high performance cluster at  Imperial College  for this project. The authors thank the  Respiratory Biomedical Research Unit  and patients at  Royal Brompton  and  Harefield National Health Service Foundation Trust  funded by  National Institute for Health Research. We thank  Professor Robert Wilson ,  Professor Diana Bilton  and  Dr. Michael Loebinger  at  Royal Brompton Hospital London  for their help recruiting the bronchiectasis/PA clinical cohorts. We thank the staff members and patients at the  Blood Transfusion Centre, Khon Kaen Hospital ,  Khon Kaen , Thailand.
This is a Discovery Program and funded by the  National Institutes of Health (NIH)  Contract number:  HHSN272200900046C  (Boyton and Altmann). KJQ was supported by an  MRC  PhD studentship and CJR was supported by the  Welton Foundation  ( P14475 ,  Boyton ). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
There are evidences that ncRNAs may regulate important processes, such as pathogenesis, iron metabolism, and quorum sensing  [4, 11, 12]. Novel ncRNAs are difficult to detect by conventional biochemical screenings  [13]. As an alternative, in silico approaches associated to a functional analysis validation have proved to be effective in the identification of ncRNAs  [14] [15] [16] [17]. In AT-rich genomes, the ncRNA genes show a relatively higher GC-content  [18]  and therefore, compositional-based analyses that scan for local GCcontent have had success in detecting ncRNAs. The algorithm Single Genome ncRNA Search (SIGRS)  [19]  is a whole-genome eukaryote predictor that uses these features. Given that most functional RNAs rely on a stable secondary structure, prediction of the minimum free energy of a transcript is also used as a means of detecting ncRNA genes  [20].
Target prediction is the main step in understanding bacterial sRNA function. Recently, computational target prediction methods had their performance improved by inclusion of RNA accessibility and conservation information  [21] [22] [23] [24]. Interacting RNA (IntaRNA) and RNAplex are reliable sRNA target prediction software  [25]. IntaRNA uses the energy score of the interaction, which is calculated as the sum of the free energy of hybridization and the free energy required for making the interaction sites accessible  [22]. RNAplex  [21]  is a refinement of the RNAhybrid software and uses a simplified algorithm to reduce the time needed to localize putative hybridization sites, mainly by neglecting intramolecular interactions and by using a slightly simplified energy model. The RNAplex tool is combined with the RNAup tool and can find high-confidence targets, with only a slight loss of sensitivity.
Results Global screening for small RNAs Knowledge related to the presence and role of small RNAs in mycoplasma remains limited, therefore a genome-wide screen for sRNAs was conducted using in silico prediction approaches. Only sRNAs present in the intergenic regions (IGRs) were searched as the input file, since all regions marked as coding sequence (CDS) and known ncRNAs, such as rRNA, tRNA, RNase P and others, were masked. These IGRs have a GC content of 21 % for M. hyopneumoniae, 23 % for M. flocculare and 21 % for M. hyorhinis representing, respectively, 13, 12 and 14 % of the total genome. The segments with a high GC cumulative, comparable to the known ncRNAs, were considered sRNA candidates. The analysis of the M. hyopneumoniae genome using the SIGRS software allowed the identification of 17 previously known ncRNAs and 26 predicted putative novel sRNAs.
However, the SIGRS outputs associate with distinct regions, defined some of them as unique ncRNA candidates. Therefore, a fragmentation algorithm (FraPS) was applied, resulting in 25 previously known ncRNAs (representing 71 % of the recovered input) and 47 putative novel sRNAs, named sRNA_1 to sRNA_47 (Additional file 1: Table  S1 ). The sequences of these predicted sRNAs including their information related to genome location, length, GC content, and free energy of the secondary structure of the sRNA are supplied in Additional file 2: Table  S2. The RNAspace software recovered 87 % of the SIGRS/FraPS predictions supporting the previous SIGRS/FraPS results for the sRNA genes. M. hyopneumoniae sRNAs have an average length of 128 bp, ranging from 61 to 424 bp and 40. 8 % of GC content. Furthermore, the minimum fold energy (ΔG) among the putative sRNAs varies from -0.
Nevertheless, some targets are ubiquitous: for example the gene AAZ53944. 1, which encodes the exonuclease protein, showed interactions with six predicted sRNAs. Another gene with multiple sRNA interactions is AAZ54018. 1 gene, which codes for the prolipoprotein p65, which may interact with three different sRNAs. The three novel sRNAs without target prediction were sRNA_27, sRNA_28 and sRNA_37. Detailed analysis of the interaction predictions revealed that these sRNAs have putative target genes found by the two algorithms, however the energy interaction in the IntaRNA software was higher than -13 kcal/mol. The sRNA_37, experimentally analyzed, has two putative target genes: MHP7448_0312, encoding glycine cleavage system H protein, and the MHP7448_0704, encoding a hypothetical protein, with energy interactions around -12 kcal/mol for both of them.
The M. flocculare sRNAs without target prediction were sRNA_F5 and sRNA_F15. Detailed analysis of the interaction predictions shows these sRNAs with putative target genes found by the two algorithms, however the energy interaction in the IntaRNA software was higher than -13 kcal/mol. Focusing in these in silico interactions, the sRNA_F5 was able to bind to genes rpsk (30S ribosomal protein S11), trxA (thioredoxin), and MF01218 (P37-like ABC transporter substrate binding lipoprotein). Furthermore, sRNA_F15 could interact with MF01377 and MF01379 (both hypothetical proteins) and also with MF00736 (ABC transport permease protein). The interaction energies for both sRNAs were around -10 kcal/mol.
In silico analysis of small RNAs The prediction of sRNAs was performed in M. hyopneumoniae 7448 (NSDC AE017244. 1), M. hyorhinis HUB-1 (INSDC CP002170. 1) and M. flocculare ATCC 27716 (INSDC AFCG00000000. 1) using two software: Single Genome ncRNA Search (SIGRS)  [19]  and RNAspace  [54]. SIGRS is a nucleotide contrast-based tool, which screens an input genome and indicates regions with similar nucleotide composition to known ncRNA sequences. SIGRS computes a scoring scheme that allows the transformation of the nucleotide genome sequence into a numeric one. Subsequences that aggregate a partial sum above a significant threshold are considered ncRNA gene candidates. RNAspace uses a similar strategy, screening for rich atypical GC regions, and only the regions with a GC value above the mean plus twice the standard deviation for the whole genome are considered as atypical.
This method detects signals intrinsic to ncRNAs, differentiating them from other elements in the genome by exploiting the compositional bias between ncRNAs and other regions of the genome. Initially, the SIGRS method was used as the Mycoplasmas sRNA predictor with a set of known ncRNAs to guide the search for new ncRNAs with a similar nucleotide composition profile and structural-based features. Known ncRNA sequences of Mollicutes (rRNA, tRNA, RNase P and other functional ncRNAs) were obtained from the Bacterial sRNA Database (BSRD)  [55]  and from the bacterial genome annotation. The genomes used for the new sRNAs search were masked in all regions marked as coding sequence (CDS) in the annotation files of M. hyopneumoniae, M. flocculare and M. hyorhinis. Therefore, the search was performed within the intergenic regions, which potentially harbor the sRNAs.
The set of known ncRNAs and the masked genomes were provided as input to SIGRS, which creates a scoring system based on the nucleotide composition of the known ncRNAs to transform the genome in a numerical sequence. The segments with a high cumulative sum are thus considered as sRNA candidates. However, SIGRS was unable to distinguish different ncRNA candidates when the distance between them was too small. To solve this problem, a segmentation of the numerical sequence that represents the ncRNA candidates was required in order to identify the largest local slopes in a given sequence. Therefore, the algorithm by Kadane  [56]  to find these slopes was adapted, allowing for the fragmentation of the output into the correct number of candidates. This methodology was defined as Fragmentation of SIGRS Predictions (FraPS) and the adapted algorithm and script are available in Additional file 11: Figure  S1.