dataesr/scientific-paragraphs-categorization huggingface dataset:
Introduction Determining the 3D structure of proteins is of great importance to many research fields, encompassing cancer drug discovery  (Ren et al. 2023; Borkakoti and Thornton 2023) , pesticide development, and crop improvement  (Koesoema 2022). It also plays a crucial role in the design of sensors and enzymes  (Pereira et al. 2021) , as well as numerous other applications, as reviewed by  Pearce and Zhang (2021). Traditionally, protein structures have been solved using laborious techniques, such as X-ray crystallography, resulting in just under 200,000 structures in over 50 years of communal effort  (Berman et al. 2000; Subramaniam and Kleywegt 2022). Resolved structures are routinely deposited in the Protein Data Bank (wwPDB consortium 2019). The demanding experimental process has motivated the development of computational tools as a less burdensome alternative for structure prediction.
In its simplest form, a similarity between a single query and a match from the PDB serves as the base for projecting the match's structure onto the query. The inaugural software MODELLER  (Sali and Blundell 1993 ) and other tools that followed have made use of this principle -see  Pearce and Zhang (2021)  for a review. Given the limited size of the PDB and its bias towards model organisms  (Orlando et al. 2016) , detecting remote sequence homology is crucial. To that end, increasingly sensitive search methods have been developed. The first step forward was taken by algorithms like BLAST  (Altschul et al. 1990 ), which directly compare the query sequence to the reference database. PSI-BLAST  (Altschul et al. 1997)  improved upon this by computing a multiple sequence alignment (MSA) of the query and its best BLAST hits and calculating a position-specific scoring matrix from that.
This generalization of the query is used for a sensitive search of the reference. This approach was further refined by using probabilistic hidden Markov models (HMMs)  (Krogh et al. 1994)  in tools like HMMer  (Eddy 2011). Another significant advancement came with HHsearch  (Söding 2005) , which expressed both query and reference as HMMs, markedly improving search sensitivity. This underpinned the success of HHpred  (Hildebrand et al. 2009 ) in the CASP9 challenge  (Moult et al. 2011). A further development, HHblits  (Remmert et al. 2011; Steinegger et al. 2019a ) accelerated the HMM-HMM comparison allowing to query databases with millions of HMMs like the Uniclust30  (Mirdita et al. 2017) , a clustered version of the Uniprot (UniProt Consortium 2023) to generate diverse query MSAs. Due to its unprecedented sensitivity, HHpred has transformed CASP in two ways.
Despite its success, AlphaFold2's prediction accuracy is not without its limitations. Most notably, it relies on its input MSA  (Mirdita et al. 2022 ) diversity, experiencing a significant drop in prediction accuracy when the median number of diverse sequences in the MSA is 30 or less  (Jumper et al. 2021 ). This finding is in agreement with previous studies on the importance of distant homologs to structure prediction  (Kuhlman and Bradley 2019; Ashkenazy et al. 2009). However, the ability to construct a deep MSA depends not only on the sensitivity of search algorithms, such as HHblits, but also on the potential pool of sequences, i. e. , the reference database. Metagenomics allows for sequencing uncultivable organisms directly from the environment, significantly expanding the repertoire of protein sequences deposited in scientific databases.
In recent years, metagenomic sequences have shown great potential in increasing the fraction of proteins, whose structure can be modeled accurately  (Söding 2017; Ovchinnikov et al. 2017; Yang et al. 2021; Wang et al. 2019). Of note, the largest metagenomic database used in these studies is the IMG/M, which contains 27 tera base pairs  (Chen et al. 2023b). It is therefore not surprising that the top scoring servers in the most recent CASP15 challenge were based on AlphaFold2 and included metagenomic sequences in their constructed MSAs (Table  1 ). An example for such a server is ColabFold  (Mirdita et al. 2022). ColabFold takes as input a query protein sequence(s), whose structure is to be predicted. Its first step, denoted here as CF-search, implements a procedure for collecting homologs of the query using MMseqs2  (Steinegger and Söding 2017).
CF-search starts by querying the input against the UniRef30 database  (Mirdita et al. 2017 ) and computing profiles from the hits. Next, CF-search queries these profiles against one of two metagenomic databases, which were constructed as part of the ColabFold release: BFD/MGnify and ColabFoldDB (the default reference database). As detailed in Table  2 , BFD/MGnify contains 513 million non-redundant proteins from the union of the BFD  (Jumper et al. 2021)  and MGnify  (Richardson et al. 2023)  databases. ColabFoldDB expanded the BFD/MGnify with various environmental proteins, resulting in ~740 million proteins. Following the search, an MSA is computed from the detected homologs and finally, in a step denoted here as CF-predict, the MSA is provided as input to the AlphaFold2 models. In this study we examined different strategies to improve protein structure prediction along three axes.
The first two focused on adding homologs to MSAs used for protein structure prediction and the third -on utilizing advanced features of CF-predict. The first and main axis is the breadth of the search, where we studied the impact of a much more systematic inclusion of metagenomic sequences on prediction accuracy. Over 37 peta base pairs are publicly available through the Sequence Read Archive (SRA), the world's largest metagenomic database  (Katz et al. 2022). Recently, Serratus, a tool for a high-throughput search of the SRA, was introduced  (Edgar et al. 2022). Here, we constructed MSAs based on Serratus-mined homologs of the CASP15 targets and merged them with the default MSAs produced by CF-search. In the second axis of this study, we further enhanced the merged MSAs by searching for distant homologs of their sequences using HHblits against the BFD.
The third axis concerns tuning the advanced parameters of ColabFold to control the use of templates and multimer models and the number of recycles. We then provided MSAs produced by each of these strategies as input to CF-predict and compared the resulting prediction accuracy to that measured with default CF-search MSAs as well as to the leading CASP15 servers. For fair comparison, we ensured all databases used in this study excluded any sequences deposited after the start of CASP15 competition (May 2022). Our results show that adding SRA-mined homologs improves prediction accuracy for 61% of the examined targets. Tuning advanced features of CF-predict, especially adding more recycles, also contributed to better prediction. By combining the different strategies, ColabFold's CASP15 ranking among servers on non-easy template targets increased from 11th to 3rd place, indicating the vast potential of large-scale sequence exploration for better structure prediction.
Thousand times broader search Our next goal was to expand the search beyond ColabFoldDB and explore the SRA. With its 37 peta base pairs of publicly available data, the SRA is orders of magnitude bigger than any previously-used metagenomic resource, including ColabFoldDB (Table  2 ). We queried the 77 CASP15 targets using Serratus against over half of the publicly available SRA, comprising 22 peta base pairs, organized in over five million SRA runs. Reads aligned to the CASP15 protein sequence queries, were examined in the context of their run and assembled using rnaviralSpades  (Meleshko et al. 2021) , mounting to over a hundred gigabytes of assembled data. Each CASP15 target was then queried against a reference protein database created from its Serratus-produced assembled proteins using MMseqs2  (Steinegger and Söding 2017).
The identified homologs were then aligned by MMseqs2 to create an MSA (Fig. 1 ② ), denoted as sra MSA. As a first indication of the tremendous capacity of the SRA, we found that more than half of the targets, which were composed solely of TBM-easy domains, could be matched with at least 1,000,000 homologs, some even exceeding 100,000,000 (Fig. 2A ). Processing MSAs with millions of sequences poses a heavy computational burden. Thus, we opted to exclude from this study targets, which only contained TBM-easy domains, focusing on the remaining 46 targets, which had 62 non-TBM-easy domains: 39 FM, 8 FM/TBM, and 15 TBM-hard. On average, the number of homologs detected per domain doubled from 106,586 in the CFDB to 274,231 when including the SRA results (Fig. 2A ). 3. The score of this study is from the Model1 in Table  3.
To filter each input MSA based on the sequence identity (seqid) between its members and the query, we utilized a newly introduced filter module called filtera3m in MMseqs2, which implements ColabFold's MSA filtering strategy. We removed unlikely homologs (seqid < 0. 2), non-informative homologs (seqid > 0. 95) and redundant sequences, keeping the most diverse and informative set of sequences in the MSA. The filtered cfdb MSAs had on average 2,395 sequences, sra_cfdb MSAs -5,731 and hh_sra_cfdb MSAs -8,133. We used HHmake  (Steinegger et al. 2019a)  to compute the number of effective sequences (N eff ), where higher values indicate less similarity between the sequences and more diverse MSAs. Here, a more moderate increase was observed with the average N eff score, rising from 4. 87 for cfdb MSA to 5.
The effect of homologs on structure prediction In order to measure the effect of the various homolog collection strategies on structure prediction, we provided CF-predict with different input MSAs. These included the sra_cfdb and hh_sra_cfdb MSAs as well as their controls, which did not include Serratus-detected homologs from the SRA: cfdb and hh_cfdb MSAs. The hh_cfdb MSAs were produced in a similar manner to hh_sra_cfdb MSAs, using cfdb MSAs, rather than sra_cfdb MSAs, as the input to HHblits. For each strategy, five models were produced and the best one was selected according to its computed predicted local distance difference test (pLDDT) score  (Jumper et al. 2021 ). For the selected models, we measured the domain level accuracy, using the GDT_TS score  (Zemla 2003). Compared to using cfdb MSAs, sra_cfdb MSAs significantly improved GDT_TS scores (Table  3 ), increasing scores for 38 out of 62 domains (Fig.
Tuning parameters In addition to homolog collection strategies, we examined the impact of three advanced features of CF-predict: using templates, using multimer models, and increasing the number of recycles. The strategies corresponding to these features were built on the MSAs constructed in previous steps and are denoted: sra_cfdb_temp, sra_cfdb_recyc and sra_cfdb_mult and their control: sra_cfdb. Other strategies, taken by the leading CASP15 servers are detailed in Table  4.
Multimer modeling We also tested the impact of setting on the "multimer" option (Fig. 1 ⑥ ). This changes the default behavior of CF-predict from treating the input query sequence as a monomer to considering it to be a part of a complex. This has the potential of stabilizing the structure, thereby improving prediction accuracy. We applied multimer modeling only for the homo-oligomer targets, based on the stoichiometry provided by CASP and used the multimer model weights from AlphaFold-multimer version 2  (Evans et al. 2022). Using multimer modeling had a diverse range of effects. Overall, it did not make a significant improvement over sra_cfdb MSAs (Table  3 ). However, it drastically increased or decreased prediction performance for certain targets. Two notable examples stand out: T1178-D1 achieved the highest improvement, with its GDT_TS score soaring from 28.
Adding more recycles Through the "recycle" parameter (Fig. 1 ⑦ ), CF-predict allows setting the number of iterations in which a prediction will be re-fed to the AlphaFold2 models. By default, this value is set to 3, but additional recycles have the potential to improve prediction accuracy  (Mirdita et al. 2022). Thus, when exploring this option, we set it to 12. The recently released version 3 of AlphaFold-multimer  (Evans et al. 2022 ) uses up to 20 recycle iterations, with early stopping if a model has already converged. Increasing recycles significantly improved the prediction accuracy compared to the control sra_cfdb with default parameters (Table  3 ). As depicted in Figure  2C , sra_cfdb_recyc MSA scored higher than cfdb MSAs and sra_cfdb MSAs in 34 domains and achieved high-accuracy (GDT_TS > 70) in 72% of the 62 domains.
Strategy selection and comparison with CASP15 servers Finally, among the seven examined strategies, we selected the one with the highest pLDDT for each target domain, denoted here as Model1. We then compared the performance of Model1 with the leading server groups in CASP15, including the original ColabFold server, which is similar to this study's cfdb MSAs (Fig. 2D ). Model1 resulted in an average GDT_TS of 75. 56 and sum of Z-scores of 40. 56, increasing 9. 76 and 23. 47 units from the baseline cfdb MSAs, respectively. Notably, 46 out of 62 domains (74%) of Model1 achieved high-accuracy scores (GDT_TS > 70), compared to 52% of the cfdb MSAs. When comparing with other server groups based on the sum of Z-scores, Model1 would have ranked 3rd, outperforming the ColabFold original server, which ranked 11th in CASP15 among server-only groups on non-easy targets.
Concluding remarks In this study, we have shown the importance of a comprehensive inclusion of metagenomic sequences from the SRA for improving protein structure prediction. Our results highlight the large variation in the number of homologs found for different targets. For example, over 100 million environmental homologs were found for T1137s7, T1195, T1196, and T1197 -the same order of magnitude as the entire UniProt database. However, there are some targets for which few matches were detected, possibly due to the limited sensitivity of the mining procedure. Serratus, the tool used for mining the SRA has impressive capabilities, but also significant constraints. It is limited to detecting homologs, which have about 50% sequence identity to the query, missing the full potential of homologs from the twilight zone  (Rost 1999).
Acknowledgments MS acknowledges the support by the  National Research Foundation of Korea , grants [ 2020M3-A9G7-103933 ,  2021-R1C1-C102065 ,  2021-M3A9-I4021220 ],  Samsung  DS research fund and the  Creative-Pioneering Researchers Program  through  Seoul National University. MM acknowledges support by the  National Research Foundation of Korea  (grant  RS-2023-00250470 ). Computing resources were provided by the  University of British Columbia Community Health and Wellbeing Cloud Innovation Centre , powered by AWS. RC was supported by  ANR  grants [ ANR-19-CE45-0008 ,  ANR-22-CE45-0007 ,  PIA/ANR16-CONV-0005 ,  ANR-19-P3IA-0001 ], and  H2020   Marie Skłodowska-Curie  grants [No  956229  and  872539 ]. AK acknowledges the support of the  US National Institute of General Medical Sciences (NIGMS/NIH) , grant  R01GM100482.
Study design and data collection This was a retrospective, multicenter, registry-based analysis of transplants performed in centers affiliated to the EBMT. Eligible MM patients had undergone a second line of treatment with KRd induction followed by a second ASCT between January 2016 and December 2018. Patients who received fewer than two cycles of KRd were excluded. Clinical data were collected using ProMISe (Project Manager Internet Server), the international database coordinated by EBMT. Patient outcome data were collected on-site using individual patient records. A total of 51 patients from 22 EBMT registered transplant units were included in the overall analysis. This study was conducted in accordance with the Declaration of Helsinki. All patients gave informed consent for the retrospective use of their clinical data.
Another point is that only 47% of patients in our cohort received a maintenance therapy, while all patients had a continuous treatment in ASPIRE and POLLUX studies. Altogether, DRd remains the preferred treatment option in patients with lenalidomide    [22]. A prospective, randomized trial would therefore be needed to assess the efficacy of intensified first relapse MM therapy in combination with stateof-the-art triplet induction regimens, with or without maintenance therapies. We chose the day of ASCT as the starting point for this study because, using the EBMT database, we identify patients who actually received an ASCT but cannot identify patients who were scheduled to receive one but who did not proceed to ASCT for any reason (i. e. , infections or relapse). Starting our analyses on the first day of KRd would introduce a major bias.
Therefore, to measure reproducibility between the peaks detected in both replicates, we use the Jaccard index, which is the size of the intersection between two datasets divided by the size of their union, indicated in the figure as Peak agreement. In human eIF4A3 eCLIP, the Jaccard index was only 0. 17 (Fig. 1b ), indicative of poor experimental reproducibility (Additional file 2: Table  S1 ). To test if this poor reproducibility was eIF4A3 specific, we determined the Jaccard index of 84 different human RBPs for which PureCLIP analyses of eCLIP data are publicly available  [31]. The Jaccard indexes vary from almost zero to 0. 42 with a median value of 0. 20 (Fig. 1c ). The Jaccard index obtained for eIF4A3 is thus well within the range of values across previous eCLIP datasets, suggesting that low reproducibility is a general feature in eCLIP datasets.
The peak agreement between the two replicates corresponds to the calculated Jaccard index (JI). c A barplot showing the distribution of Jaccard index values for peaks detected using PureCLIP analysis of replicates of eCLIP datasets for 84 different RBPs from ENCODE. The median value of the distribution is 0. 2 (dotted line) humans  [34]. More significantly, the Drosophila genes have a simpler gene architecture with fewer exons and introns (Additional file 1: Fig. S2a  and S2b ). In addition, Drosophila genes are subjected to fewer alternative splicing events than human genes, thereby sharply reducing the total number of exon-exon junctions  [26]. We followed the same experimental strategy as in HeLa cells by fusing a 3xHA tag to the endogenous eIF4A3 expressed in S2R+ Drosophila culture cells.
The tagged protein is expressed at a level similar to the untagged protein in the wild-type cell (Additional file 1: Fig. S2c ). We performed three eIF4A3 eCLIP replicates and two SMInputs. We acquired 2-10-fold more usable reads from these cells than from human cells (Additional file 1: Fig. S1b ). Considering that the Drosophila transcriptome is less than half the size of the human transcriptome, the coverage of Drosophila eCLIP is significantly higher than that of human cells. A meta-exon plot of the 5′ extremity of Drosophila eIF4A3 eCLIP reads shows a major accumulation of crosslinking sites 27 nts upstream of exon junctions, as in the human eIF4A3 eCLIP (Fig. 2a ). Therefore, EJC positioning by the splicing machinery is well conserved between the two organisms. In Drosophila, the larger peak-to-background ratio indicates lower noise levels compared to human data (Figs.
1a  and 2a ). Moreover, the sharper profile observed in the -27 peaks, with a decrease in upstream signal compared to the human meta-exon, can be attributed to two factors. First, it may be a consequence of fewer read-through events occurring in Drosophila. Second, it could be due to a more accurate mapping of eCLIP reads onto the Drosophila transcriptome due to its simplicity, resulting in a more precise alignment of the binding sites. These observations indicate that the increased coverage achieved in Drosophila resulted in a significant increase in the signal-to-noise ratio. PureCLIP analyses of Drosophila eCLIP datasets yielded between 50 and 150 thousand called peaks (Fig. 2b , Additional file 1: Fig. S1c ). Remarkably, the different replicates gave Jaccard index values between 0. 6 and 0.
Stop rate difference, a reproducible metric for eCLIP peak detection Based on the above results, we decided to focus our efforts on Drosophila. To move from quantifications at the metagene level to the detection of individual crosslinked sites and exons, we implemented an approach relying on the detection of RT-arrests induced by crosslinking ('stop rate scores'). RT-arrests were quantified via JACUSA2 software. JACUSA2 rt-arrest was originally employed to map RNA modifications  [35]. This method is centered on calculating RT-arrests per position and normalizing them by coverage per position. Then the stop rate score per position was further normalized by the stop rate score of the input, giving the stop rate difference (SRD) (see the "Methods" section) (Fig. 3a ). Posteriorly, an additional exon-centered score was established by calculating the average SRD score of all significant positions within a 10-bp window around the 27th nucleotide upstream of the exon junction, giving an exon-level SRD score.
Combination of eCLIP and ipaRt data analysis establishes universal EJC loading on exon junctions Given the observed constraints of eCLIP, we complemented our EJC mapping by re-analyzing a previous transcriptome-wide EJC interactome established in Drosophila by an RNA IP strategy independent of UV crosslinking  [27]. In this study, the authors developed the method ipaRt (isolation of protein complexes and associated RNA targets) in which Drosophila extracts are treated with a protein-protein crosslinking agent to stabilize EJCs onto RNA before isolation of EJC-bound RNA fragments. In comparison to eCLIP, examination of IGV coverage revealed a very clear signal enrichment upstream of exon junctions for the ipaRt method, characterized by lower background noise (Additional file 1: Fig. S4a ). While ipaRt does not, inherently, provide data at single nucleotide resolution (in contrast to eCLIP), we found that when we considered the middle of each read as a proxy for the whereabouts of the EJC, we observed a sharp peak 23 nt upstream of the exon junction (Fig.
4a ). The slight deviation from the -27-position observed with eCLIP data (Figs. 1a  and 2a ) is most likely because the position of the middle of EJC-bound reads purified by ipaRt does not exactly correspond to the crosslinked nucleotide mapped by eCLIP analysis. Nevertheless, the ipaRt dataset consistently confirms the highly conserved and precise binding site of the EJC onto the mRNA. To enable exon-level binding analysis for ipaRt, we created a score based on an enrichment ratio between the median coverage per base in the EJC binding site window of ipaRt compared to the same window in the mRNA-seq (see the "Methods" section). To calibrate the threshold based on which exon was considered to harbor an EJC, we utilized the fact that the signal was by large absent in the last exon and hence considered the 95th percentile of signal in the last exon as a minimal threshold for considering an exon as loaded (Additional file 2: Table  S4 ).
This disadvantage is not shared by the eCLIP approach. To further improve the robustness of our analysis, we integrated the results from both methods. To do so, we first selected genes that were expressed and detected in both datasets. We next classified the exons from these genes into three categories: those detected by both methods  (11, 318) , those detected by only one method (2377), and those not detected by either method (2377). Exons detected in only one method were mainly from ipaRt since, as previously mentioned, this method has a higher sensitivity than eCLIP. The reproducibly unloaded exons, as anticipated, were primarily associated with lower expression levels (Fig. 4c ). Subsequently, we performed a combined analysis of both methods, considering all exons detected by either one or both approaches.