dataesr/scientific-paragraphs-categorization huggingface dataset:
We have already demonstrated that additional identifications obtained when searching reduced databases actually derive from a flaw in TDC with respect to the reduced database size, and how they can be largely avoided by applying alternative FDR control procedures, such as BH. We will now show that straightforward searches against reduced databases followed by BH-based FDR control nonetheless yield smaller protein groups and less ambiguous protein identifications, thus regardless of additional identifications or protein inference methods. Concretely, we compared identifications obtained from the full or reduced database searches followed by BH-based FDR control. The total number of identifications, at the spectrum, peptide, and protein level, was comparable (Figure  4A ). For the number of protein-level identifications, we used the number of protein groups, as defined by the Proline software.
Protein groups include the unambiguous identification of a single protein (single-protein groups) and groups of indiscernible proteins identified by the same sets of peptides (multi-protein groups) (see "Proteome analysis" in Methods). Interestingly, the proportion of single-protein groups was considerably higher for the reduced database (Figure  4B ), meaning that protein identifications were less ambiguous. We further characterized the ambiguity of protein identifications using the graph's connected components. Briefly, we first represented peptide-to-protein mappings on bipartite graphs, with peptides and proteins as vertices and with edges featuring peptideto-protein membership: this representation provides an easy picturing of the complex structures generated by shared peptides. Then, we calculated the connected components (CCs), i.
e. , the largest subgraphs in which any two vertices are connected to each other by a path, and not connected to any of the other vertices in the supergraph. Proteins sharing one or more peptides are thus gathered in the same CC (multi-protein CCs), whereas unambiguous protein identifications are represented by CCs with a single protein vertex (single-protein CCs) (Figure  4C ). As such, CCs constitute a peptide-centric strategy to represent ambiguous protein identifications and their shared peptides, and should not be confused with the classical protein-centric protein grouping strategy. We observed that, while the total number of CCs obtained was comparable, there was a considerably higher proportion of single-protein CCs in the graphs derived from the results of the reduced database search.
Along with the reduction in protein group size, this is further evidence of decreased protein identification ambiguity (Figure  4D ). Consistently, we also observed a greater proportion of specific peptides -and a correspondingly lower proportion of shared peptides -from the reduced database search (Figure  4D ). Within multi-protein CCs, the ratio between the number of protein members and the corresponding number of their encoding genes was also lower for the reduced database, suggesting that at least part of the initial ambiguity occurred between proteins encoded by different genes (Figure  4E , Additional File 1: Table  S4 ). In addition to a decrease in protein identification ambiguity, our observations indicated that searches against reduced databases were associated with lower ambiguity at the PSM level, although to a lesser extent (see Additional File 3: Supplementary Note 2, Additional File 2: Fig.
S10 ). Finally, we adopted an alternative strategy to enhance proteomics by transcriptomics, which consists in an MS/MS search against the full database, followed by post-hoc filtering to remove proteins for which the corresponding transcript was not expressed and no specific peptide was identified (Figure  5A , Additional File 2: Fig. S11 ). The driving principle was to remove ambiguous protein identifications not supported by specific peptides or by transcriptomics, and thus to reduce ambiguity resulting from shared peptides. Overall, we observed similar results to those obtained with reduced transcriptome-informed database searches and post-hoc filtering. First, a similar number of spectra and peptide identifications were obtained, comparable to that of the full database search (Figure  5B ); second, these searches yielded a similarly increased proportion of single-protein CCs and specific peptides compared to full database searches (Figure  5C ), indicating less ambiguous protein identifications.
Post-hoc filtering is a transparent and easily interpretable approach, and we believe that it is highly suitable for use in studies seeking to enhance protein inference. While a few software tools already exist to generate reduced protein databases, we developed a specific toolbox of R scripts to perform the post-hoc filtering described. This toolbox also allows very efficient calculation of the CCs -which we have proposed as a means to quantify and compare ambiguity of protein identifications -, visualization of CCs of interest, and their manual inspection before and after post-hoc filtering.
DISCUSSION In this article, we provide guidance for the mindful use of reduced transcriptomeinformed protein databases for MS/MS searches. This type of reduced database stems from the attempt to counter excessive database inflation in proteogenomics studies, when variant or novel proteoforms identified from sequencing data are added. Indeed, increased database size complicates the task of discriminating between correct and incorrect matches. When using TDC-based FDR control, inflated target databases come with an inflated number of decoys, and consequently a higher probability of high-scoring decoy matches. This has mainly been thought to reduce sensitivity of identifications in two ways. First, decoy matches may score better than correct target matches, thus out-competing them in spectrum-peptide assignment ("out-competing decoys") and resulting in a decreased number of identifications.
Second, decoys may have a higher probability of matching than incorrect targets, which violates the so-called Equal Chance Assumption of the TDC procedure and leads to an overestimated FDR, once again decreasing the number of identifications. As the main raison d'être of proteogenomics is to maximize the number of identifications, including variants or non-canonical peptides, much effort has been expended to avoid loss of sensitivity linked to excessively large databases, for example by reducing their size. While issues relating to the use of excessively large databases have been abundantly discussed, fewer authors have pointed out that excessively small databases may also be problematic, as they can also affect TDC estimations  31, 34, 35. With excessively small databases, TDC provides inaccurate FDR estimates, as they can only be asymptotically accurate  34, 36, 37.
Furthermore, with too few (high-scoring) decoys, the probability of matching a decoy might be lower than the probability of matching an incorrect target, which once again violates the Equal Chance Assumption, but leads in this case to FDR underestimation and to an artifactual increase in identifications. In this study, we showed that the increased number of identifications obtained by searching reduced transcriptome-informed protein databases is most likely a statistical artifact due to the use of TDC on excessively small databases. We demonstrated how TDC estimates a lower score cutoff for 1% FDR control on the reduced databases compared to the full database search results. Consequently, some invalid PSMs in the full database will be retained as valid additional identifications only in the reduced database.
We confirmed this observation at various levels of FDR control (0. 5%, 1%, 5%) and for two different human-derived samples -a tissue (Lung) and a cell line (Jurkat) -representing different levels of proteomic complexity and coming with a different number of spectra. Fewer spectra were available for the Jurkat sample, which, interestingly, also presented a greater difference in score cutoffs between the full and reduced databases. Indeed, not only reduced database sizes but also a smaller number of spectra is believed to affect the ability of TDC to accurately estimate the FDR  34. We suggest that additional identifications obtained when searching such reduced databases are at least doubtful, and that it is unwise to employ reduced transcriptome-informed protein databases in an attempt to increase the number of identifications.
Indeed, the additional identifications obtained had quite low scores and did not result from removal of out-competing decoys, a known cause of missed identifications in excessively large databases; rather, they represented identical PSMs in the two database searches but that were only accepted in the reduced database due to the lower score cutoff for the same level of FDR control. Most importantly, only a negligible number of additional identifications was generated from the reduced database search when using a method for FDR control that is known to be stable with respect to database size, such as BH. Indeed, using BH, the score cutoffs estimated for the full and reduced database searches, at the same level of FDR control, were almost identical. Thus, the BH procedure constitutes an interesting alternative to TDC for stable FDR control whatever the database size.
However, many alternative approaches have been recently developed to cope with the weaknesses of classical TDC  [38] [39] [40] [41] [42] [43]. It is important that proteogenomics researchers use one of these alternatives to avoid risking the propagation of statistical artifacts in their data. By doing so, they will no longer benefit from the hypothetical sensitivity increment assumed up to now, but this seems to be the necessary price to pay for rigorous control of the FDR. Reduced transcriptome-informed protein databases are nonetheless useful in bottom-up proteomics to reduce ambiguity in protein identifications due to the presence of shared peptides. In particular, we showed that searching a reduced database yielded a higher proportion of specific peptides and unambiguously identified proteins (i.
Therefore, a lower proportion of shared peptides represents more information available for quantification. Finally, we showed that full database searches followed by post-hoc filtering of proteins for which no transcript is expressed provided comparable proteomic identifications to reduced database searches. This filtering approach similarly reduced ambiguity of protein identifications, while being more transparent and interpretable. We developed an R code to implement this type of post-hoc filtering strategy. The code allows users to visualize ambiguous protein identifications and their peptides via bipartite graphs, to prune them based on transcript expression, and to manually inspect how this transcriptomebased post-hoc filtering strategy reduces ambiguity. Ambiguous protein identifications are represented and quantified using graph CCS, which constitute here subgraphs of proteins connected by shared peptides.
This representation provides the following advantages: it is transparent, interpretable, non-redundant with respect to shared peptides, and independent of any strategy developed to define protein groups. Results from this study are of interest also beyond proteogenomics. Indeed, database reduction is widely called for in proteomics, even though little attention is paid to the limitations of TDC when using excessively small databases. For example, some authors proposed to limit database size based on peptide detectability  30  , and it has been more generally claimed that "mass spectrometrists should only search for peptides they care about"  44. The statistical TDC artifacts observed with excessively small databases are an issue that also affects multi-step search strategies implemented by some proteomics search engines  45  or developed in metaproteomics  46, 47.
Proteogenomic datasets description Two samples for which matched transcriptome and proteome were publicly available were analyzed: healthy lung tissue (referred to as Lung) and the Jurkat cell line (referred to as Jurkat). The lung sample was from a dataset by Wang et al. 33  , which includes 29 histologically healthy human tissues generated to describe mRNA and protein expression levels throughout the human body. The lung transcriptome dataset was obtained by paired-end RNA sequencing on an Illumina HiSeq 2000/2500 system generating 2×100 base-long reads. The matching proteome dataset was obtained by quantitative label-free LC-MS/MS using an on-line nanoflow liquid chromatography system coupled to a Q Exactive Plus mass spectrometer, operating in data-dependent mode. Sample preparation included peptide fractionation via hSAX (hydrophilic strong anion) chromatography.
Raw transcriptome and proteome data were downloaded from the EBI SRA (ArrayExpress accession: E-MTAB-2836; run accession: ERR315346) and the ProteomeXchange (dataset identifier: PXD010154; sample identifier: P013163) repositories, respectively. The Jurkat cell line dataset was produced as part of a study by Sheynkman et al. 24. The Jurkat transcriptome dataset was obtained by paired-end RNA sequencing on an Illumina HiSeq 2000 system generating 2×200 bases long reads. The matched proteome dataset was obtained by quantitative label-free LC-MS/MS using a nanoAquity LC system chromatography system coupled to a Velos-Orbitrap mass spectrometer, operating in data-dependent mode. Sample preparation included peptide fractionation via high-pH LC separation. Transcriptome and proteome raw data were downloaded from NCBI's Gene Expression Omnibus (GEO; accession: GSE45428) and the PeptideAtlas repositories (accession: PASS00215), respectively.
Transcriptome analysis Raw reads were downloaded from public repositories and processed on the Galaxy platform available at https://usegalaxy. org/ 48 using common workflows for read preprocessing and alignment to identify expressed transcripts (Additional File 1: Table  S5 ). First, sequencing adapters and low-quality (Phred score < 20) read ends were trimmed off using TrimGalore (https://www. bioinformatics. babraham. ac. uk/projects/trim_galore/), and reads shorter than 20 bp after trimming were discarded. Then, preprocessed reads were aligned against the human reference genome (assembly GRCh38) by the splice-aware STAR aligner  49  in default mode, using the Ensembl reference gene model for splice junctions. Only reads mapped to a proper pair, and passing platform quality controls were retained.
Reads corresponding to optical or PCR duplicates were removed, as were non-primary and supplementary alignments. Initially, two common strategies for transcriptome assembly and quantification were applied: StringTie 50 and Cufflinks  51. Both programs were run to identify reference transcripts (and no novel transcripts), and two comparable sets of expressed transcripts were obtained. Unless otherwise specified, StringTie output was used for downstream analyses. when peptides are shared by different proteins, but they are easily represented using bipartite graphs. CCs from the graph -defined as the largest subgraphs in which any two vertices are connected to each other by a path and not connected to any other of the vertices in the supergraph -were then used to quantify the level of ambiguity of protein identifications.
To build bipartite graphs of proteomic identifications, first a tab-separated file containing identified peptides and all proteins they map on (one protein per line) was generated based on the output of PSM validation by Proline software. This file was then converted into an incidence matrix, with proteins along the columns and peptides along the rows, using the crosstab function from the GNU datamash program (http://www. gnu. org/software/datamash). The cross-product of the incidence matrix was used to produce the corresponding adjacency matrix, which describes protein-to-protein connections, based on shared peptides. Finally, CCs were calculated using the connComp() function from the "graph" R package applied to the adjacency matrix. Two types of CCs were defined: i. those containing a single protein (single-protein CCs), with only specific peptides, which constitute unambiguous protein identifications; ii.
those containing multiple proteins sharing peptides (multi-protein CCs), which represent ambiguous protein identifications. Ambiguous protein identifications can be visually inspected by taking the CC of interest, extracting all specific and shared peptides mapping to the CC protein members from the incidence matrix and plotting peptide-to-protein mappings as bipartite graphs, using the "igraph" R package. To decrease the computational cost when dealing with very large datasets or if computational resources are scarce, an alternative strategy was also developed to calculate CCs (Additional File 2: Fig. S12 ). First, the incidence matrix was reduced by removing all proteins not sharing peptides and all peptides unique to these proteins. Then, the corresponding adjacency matrix was generated as the cross-product of the incidence matrix.
CCs can be more rapidly calculated on this reduced adjacency matrix. In this case, only multi-protein CCs are obtained, since protein identifications with only specific peptides, corresponding to single-protein CCs, were removed from the incidence matrix. While multi-protein CCs are those of interest when investigating ambiguous protein identifications from shared peptides, single-protein CCs can still be easily retrieved from the original incidence matrix if necessary. A companion R code is provided implementing all the above-described steps, including: i. generating the adjacency matrix; iii. calculating CCs; iii. visualizing CCs as bipartite graphs.
Transcriptome-informed post-hoc filtering As an alternative to searching a reduced transcriptome-informed database, a transcriptome-informed post-hoc filtering strategy was tested. First, peptide identifications were obtained by searching the full reference protein database, and validated using Proline software, as described in the "Proteome analysis" section. An incidence matrix was generated to encode peptide-to-protein mappings (see the "Bipartite peptide-protein graphs and connected components" section). Then, the sample-matched transcriptome was analyzed to identify the set of transcripts expressed. Finally, the peptide-to-protein incidence matrix was filtered by removing proteins for which no expressed transcript and no specific peptide were detected. The one-to-one transcript-to-protein correspondence is guaranteed by the use of Ensembl as the reference protein database in proteomics and for genome annotation in transcriptomics.
The filtered incidence matrix was then converted to an adjacency matrix to calculate CCs, as previously described (see "Bipartite peptideprotein graphs and connected components" section). For this study, the post-hoc filtering strategy on PSMs obtained from searching the target-only full Ensembl protein database was used, followed by the BH procedure for FDR control, to allow comparison with the approach searching target-only reduced transcriptome-informed protein databases, followed by the BH procedure for FDR control. Indeed, the BH procedure was used as an alternative to TDC after searching reduced transcriptome-informed protein databases to obtain accurate FDR control, as explained in the Results section (see "Reduced transcriptome-informed database search does not increase sensitivity if FDR is accurately controlled").
B. Size and overlap of the reference human Ensembl protein database (full protein DB) and the sample-specific reduced transcriptome-informed protein databases (reduced protein DB). C. Number of spectra (left) or peptides (right) exclusively identified in the reduced database ("additional in reduced DB", in blue) or exclusively identified in the full database ("lost in reduced DB", in red) searches. The net difference between identifications additional and lost when searching the reduced database is also reported on top of each bar ("net").
36e associated nominal FDR level is reported (calculated as (d+1)/t, with d and t being the number of valid decoys and targets, as suggested in Levitsky et al. Proteome Res, 201636. Match in the reduced database search for spectra matching valid targets or valid decoys in the full database. D. Score cutoffs obtained by TDC or by BH procedure for FDR Number of valid identifications obtained from the full (red) or reduced (blue) target-only database searches, followed by BH procedure for 1% FDR control. The number of valid spectra, peptide and protein identifications are reported. Protein groups, as defined by the Proline software, represent protein identifications and include: i. proteins unambiguously identified by only specific peptides (single-protein protein groups); ii.
Funding This work was supported by grants from the  French National Research Agency :  ProFI  project ( ANR-10-INBS-08 ),  GRAL  project ( ANR-10-LABX49-01 ), DATA@ UGA  and  SYMER  projects ( ANR-15-IDEX-02 ) and  MIAI @ Grenoble Alpes  ( ANR-19-P3IA-0003 ).
Availability of data and materials All datasets analyzed in this study correspond to publicly-available data from the works of Wang et al. 33  and Sheynkman et al. 24. The transcriptome datasets were downloaded from the EBI SRA with ArrayExpress accession: E-MTAB-2836 (run accession: ERR315346), and from the NCBI Gene Expression Omnibus (GEO) with accession GSE45428. The proteome datasets were downloaded from PeptideAtlas with accession PASS00215 and from the ProteomeXchange repository with dataset identifier: PXD010154 (sample identifier: P013163). Scripts used to perform these analyses are available as a platform-independent R package, under GPL-3 license, at https://github. com/laurafancello/net4pg. Guidelines for installation and usage are documented in the GitHub repository.
Transcriptome-informed reduced databases yield less ambiguous protein identifications. A. 36e associated nominal FDR level is reported (calculated as (d+1)/t, with d and t being the number of valid decoys and targets, as suggested in Levitsky et al. Proteome Res, 201636. Match in the reduced database search for spectra matching valid targets or valid decoys in the full database. D. Score cutoffs obtained by TDC or by BH procedure for FDR Number of valid identifications obtained from the full (red) or reduced (blue) target-only database searches, followed by BH procedure for 1% FDR control. The number of valid spectra, peptide and protein identifications are reported. Protein groups, as defined by the Proline software, represent protein identifications and include: i. proteins unambiguously identified by only specific peptides (single-protein protein groups); ii.
transcriptome-informed post-hoc filtering C. same valid target in reduced D spectra A. A. valid PSMs in reduced DB at 1% FDR Jurkat simulated valid PSMs in reduced DB if using full DB cutoff at 1% FDR 81,243 80,273 proteome database search full protein DB valid targets in full DB Lung 391,775 378,249 C. total number of CCs different valid target match in reduced DB same valid target 97. 84% 6,453 peptides Jurkat Lung 6,667 6,335 non-target and/or non-valid match 8,707 9,341 8,456 protein groups peptides 53,992 53,638 7,448 6,969 PSM validation ii. FDR control peptide length >= 7aa i. prefilters: pretty rank singlePSM per rank valid PSMs in full DB at 1% FDR 112,132 106,530 11,197 9,692 transcriptome valid decoys in full DB BH 2. 06% 2,500 5,000 7,500 0. 1% non-decoy and/or non-valid match different valid decoy 2.