dataesr/scientific-paragraphs-categorization huggingface dataset:
These families can be distributed among what authors call 'macro-classes', which correspond to the level of description given by the other datasets. A total of 306,016 sequences are present in the dataset, making it the largest state-of-the-art dataset. In 2021, Sutanto et al. [37]  curated a new dataset with sequences from Rfam 14. 1  [46]. Duplicates were removed, as well as sequences that could not be processed with the tool they use to predict secondary structure, as authors are interested in secondary structure motifs. Then, CD-HIT  [42]  was used to remove redundancy in the dataset. In total, the dataset comprises 59,723 RNAs representing 12 classes. This dataset is not available publicly. The most recent dataset, from 2023, was proposed by Lima et al. [32]  along with the tool NCYPred.
It was built following the same protocol as nRC's dataset but with updated data. Using the release 14. 3 of Rfam  [46] , authors were able to multiply by 5 the number of ncRNA in the dataset, obtaining 45,447 ncRNAs split into a training set and a test set. Note that in this dataset, classes are not balanced (some are represented by more than 5,000 examples, while others have less than 500). Another difference with nRC's dataset is that authors excluded two classes (scaRNA and IRES), citing the poor results of their method on these classes. Instead, two new classes were added (Y RNA and Y RNA-like). This dataset was used by one other recent method, MFPred. The comparison between state-of-the-art datasets can be summed up to three key points: origin of data, redundancy and diversity of sequences, and class definition.
As one of the largest databases on ncRNAs, all datasets use Rfam as their starting point, with more recent releases containing more sequences. imCnC's dataset  [30]  also integrates other databases for epigenetic data. Three of the five datasets (  [24, 37, 32] ) use the CD-HIT tool to remove redundant sequences. CD-HIT uses a predefined threshold to create groups of similar sequences, and removes all but one sequence per group from the dataset. In all cases, the threshold was set to 80%. While this step restricts the size of the dataset, it only removes examples that would not have been informative, while maintaining diversity. Models trained on these datasets could therefore be more efficient without sacrificing performance. Some datasets remove sequences containing degenerate nucleotides (  [27, 32] ), while the others do not.
As degenerate nucleotides can be present in sequences obtained from RNA-sequencing, it can be interesting to represent them in these curated datasets as well. As for the classification task, most datasets represent around 13 commonly-used ncRNA classes, that group ncRNAs that share a function while not being too specific. Only ncrna-deep's dataset  [27]  differs, taking into a larger amount of classes. While providing a more precise functional annotation, these classes have the disadvantage of not all being on the same level of detail, with some classes being more broad and others very specific. Classes are more equal in that regard in the other proposed classification.
Benchmark This section presents a comparison of different DL tools for ncRNA classification in terms of accessibility and performance. We select state-of-the-art datasets to perform this comparison. Out of the five datasets described, three have been made available publicly: nRC's dataset, NCYPred's dataset, and ncrna-deep's dataset. We exclude the latter as we want to show classification results on different well-known ncRNA classes, and classes in this dataset are too specific and numerous. As the only two datasets that were used by more than one state-of-the-art tool, we keep nRC's dataset and NCYPred's dataset in our benchmark, referring to these as Dataset1 and Dataset2. As stated, Dataset1 presents an issue of data leakage, which we fix by removing from the test set the 347 sequences that were present in both training and test sets.
We also confirmed that the same issue was not present in Dataset2. Most state-of-the-art approaches propose an online version of their tool, as presented in Table  2. Either source code is shared and has to be installed by users, or web servers are created to be used directly. Sharing source code has the advantage of allowing users to re-train the model to adapt it to their own dataset. However, executing the code involves several steps that could cause issues, and can require a certain expertise. With web servers, users can upload their ncRNAs of interest, and a class prediction will be returned, without re-training the method. Web servers are easier to use as they require no installation, but they might be less effective on new data.
Experimental protocol In the following, we assess the quality of prediction of six methods: nRC  [24] , RNAGCN  [36] , ncrna-deep  [27] , ncRDense  [29] , NCYPred  [32]  and MFPred  [35]. RAG+DL  [37]  and RPC-snRC  [28]  are not available and thus cannot be tested. For a fair comparison, we need to be able to compare tools on the same datasets -this cannot be done for imCnC  [30] , the only tool to require epigenetic data, thus it is not included in this comparison. Finally, for tools that were developed by the same group, ample comparisons between old and new versions are already available. This concerns on one hand ncRDeep  [26]  and ncRDense  [29] , and on the other hand ncRFP  [31] , ncDLRES  [33] , ncDENSE  [34]  and MFPred  [35]. In both cases, we chose to use the latest version: ncRDense and MFPred.
Out of the methods included, nRC, RNAGCN, ncrna-deep and MFPred were re-trained before predicting each dataset. Moreover, we tested different hyperparameters for nRC, RNAGCN and ncrna-deep, as listed in GitHub repository: https://github. com/ck-fighting/MFPred. Dataset preparation has to be done separately with scripts provided. Many files are in the repository, but the documentation is lacking, making it difficult to know how to use the tool. Table  2 : Description of the availability and ease-of-use of ncRNA classification tools. Each cell indicates if the tool is accessible through a web server, if source code can be downloaded, and if the tool can be re-trained. All links have been checked at the time of writing (September 2023). directions are given on how to change them or what to change.
Method Hyperparameters nRC Range of size of substructures {(2-4), (3-5) (1,2) }, number of kernels in the first {10 (2) , 20 (1) } and the second {10, 20 (1,2) } convolutional layer. RNAGCN Size {64 (2) , 80 (1) , 128} and type {MPNN, GIN (2) , GCN, GAT (1) } of graph convolutions. ncrna-deep Sequence encoding: {1-mer (1,2) , 2-mer, snake, hilbert, morton}. Table  3 : Overview of tested hyperparameters. Parameters that gave the best results on Dataset1 are denoted by (1) , and same with (2) for Dataset2. Default values were used for parameters not listed. Predictions were made on the provided test sets for Dataset1 and Dataset2. All samples were used, except in the case of NCYPred and MFPred, which cannot handle degenerate nucleotides, meaning that some samples had to be removed from Dataset1 for these tools.
Overall performance Figure  2  presents the results obtained on both datasets, according to the six metrics defined previously. As a general trend looking at both datasets, a divide can be observed between tools that were re-trained (nRC, RNAGCN, ncrna-deep and MFPred) and those only accessible as web servers (ncRDense and NCYPred). The former all have consistent performances across datasets, with more recent methods performing the best. For the latter, performance is dependent on the dataset they were trained on. ncRDense was trained on Dataset1, on which it is the third-best performing. However, on the previously unseen Dataset2, it is outperformed by all other methods. NCYPred obtains an average performance on Dataset2, on which it was trained, but it cannot classify the unknown Dataset1 as well as the other methods.
Overall, ncrna-deep and MFPred consistently reach the highest scores, across the different metrics and for the two datasets. Both datasets represent similar sncRNA classes, yet results are, in general, much better on Dataset2. This can be due to multiple factors. The first is that Dataset2 is much larger than Dataset1. Models are trained on a larger variety of ncRNAs, making the prediction of new examples easier. Another reason is that sequences in Dataset2 do not contain degenerate nucleotides, which can be more difficult to handle for certain models. As these can be present in sequences obtained from RNA-seq experiments, depending on the use case, it might be beneficial to use a classifier that is robust to these types of nucleotides. Finally, as the description of ncRNA classes is an open problem, the set of classes that should be included in datasets is not fixed.
While Dataset1 and Dataset2 are mostly comparable, each includes two classes not present in the other dataset. This can impact reported performance: for example, the two classes included solely in Dataset1 seem more difficult to predict, thus lowering overall scores. Therefore, while results on Dataset2 are quite satisfying, it is important to keep in mind that they relate to one set of ncRNA classes, which does not include all existing ncRNA classes. Each tool in this benchmark was presented in a publication that showed experimental results. We compare reported performance to the one we measured, and illustrate the difference in Figure  3. All publications use the same formula for multi-class MCC, therefore, we base our comparison on this metric. nRC, RNAGCN, ncrna-deep, MFPred, ncRDense and NCYPred all use Dataset1 (albeit the biased version with data leakage).
Since it is more recent, only MFPred and NCYPred show results on Dataset2. Results on Dataset1 were significantly worse in our experiments than what was originally reported by the various tools. These drops in performance are likely due, in part, to the correction of the data leakage problem. Interestingly, after our hyperparameter search, the results we obtain for RNAGCN are slightly better than the ones reported by the method. The performance of NCYPred is significantly worse in our experiment. This could be due in part to model training: in their benchmark, authors re-trained their tool on Dataset1, which we could not do, only having access to the web server version. This means that two classes from Dataset1 are unknown by the model and cannot be predicted. Moreover, sequences containing degenerate nucleotides cannot be predicted by NCYPred, further lowering the scores.
On Dataset2, reported results seem more reliable as we only observe negligible differences in MCC. Figure  3 : Difference between MCC reported by each state-of-the-art method and MCC measured in our benchmark. A negative value of -x% signifies that reported MCC was x% higher than the value we measured. We re-trained nRC, RNAGCN, ncrna-deep and MFPred, while for ncRDense and NCYPred (marked with an asterisk) the web server was used.
Performance on individual classes A great challenge in ncRNA classification is achieving high predictions across classes. When evaluating multi-class ncRNA classification tools, it is important to analyze class-specific predictions and see how the method fares on different classes. Figure  4  presents the F1-scores obtained for each class on both datasets using the formula: F1-score = T P T P + 0. 5(F P + F N ) Figure  4  emphasizes that more recent methods make more accurate predictions: ncrna-deep and MFPred make very accurate predictions for almost all classes. RNAGCN is also able to reach high performances on Dataset2, however, it struggles for some classes on Dataset1, perhaps needing a larger amount of examples to characterize them. The difference in F1-score between classes is quite striking for nRC, which is able to predict some classes quite well, such as Intron-gpI and 5S-rRNA, but cannot characterize well enough some other classes like miRNA (especially on Dataset1) or HACA-box.
Aside from the classes it was not trained on (IRES and scaRNA), NCYPred's performances are on par with that of other tools on Dataset1. The same cannot be said for ncRDense, which does not only fail to predict the classes it was not trained on (Y RNA and Y RNA-like), but also obtains relatively poor performances for most other classes in Dataset2. This emphasizes the point that not allowing users to re-train a tool can be detrimental. It is interesting to see that, although recent tools make generally very accurate predictions, the more problematic classes are the same as those for earlier methods: miRNA, IRES or HACA-box to name a few.
Computation time In the following, we compare the computation time of benchmark methods, which we measure when running the codes. nRC, RNAGCN, ncrna-deep and MFPred were executed on the same machine (NVIDIA GeForce RTX 3090), with GPU acceleration for all methods but nRC which only supports CPU computation. These are presented in Table  4. (Train)  and prediction (Pred) times. Times are computed for the hyperparameter sets selected in Table  3. Times are given for 1,000 ncRNAs, averaged over both datasets. Note that preprocessing cannot be separated from training / prediction in nRC and cannot be computed separately. State-of-the-art methods require preprocessing -i. e. , transforming input sequences to a format that can be handled by the model. This is an additional step to be done by users before using the tool, except for nRC for which preprocessing is included in the pipeline and cannot be done separately from training / prediction.
Finally, there is an additional consideration regarding the evolution of ncRNA classes. The definition of the different classes is not fixed and is still being refined. As we discover new types of ncRNAs or redefine the characteristics of existing classes, ncRNA classification datasets are bound to evolve. In this context, it is important to propose tools that can be applied to and evaluated on new datasets. This means developing tools that are accessible, easy-to-use, and well-documented, and that have the option to be re-trained. Furthermore, the ability to discover new classes, or detect similarities between existing classes, could even be integrated into new ncRNA classification approaches.
Conclusion The combination of a growing interest in ncRNAs over the past 30 years and recent outstanding performances of deep learning methods led to the development of DL-based methods for ncRNA classification. Novel architectures were proposed in the past six years, most being based on CNNs or RNNs. Most datasets represent ncRNAs by their sequence, and can include different ncRNA classes. In this publication, we review the different state-of-the-art contributions of the past six years. We also compare performance in an unbiased benchmark, including overall and class-specific prediction, as well as runtime. Finally, comparing the different architectures and their results allows us to identify potential recommendations for the future of computational ncRNA classification.
com/emalgorithm/ncRNA-family-prediction. The method is easy to use, as the process is documented. However, to use the tool on a new dataset, an outside tool has to be used to obtain secondary structures, which then need to be formatted cor- rectly before using RNAGCN. ncRFP Web server: X -Source code: ✓-Training: ✓ [31] Though not mentioned in the article, a GitHub repository exists: https://github. com/ linyuwangPHD/ncRFP. imCnC Web server: X -Source code: ✓-Training: ✓ [30] GitHub repository: https://github. com/BoukeliaAbdelbasset/imCnC. No instructions or examples of how to use the tool are given. ncRDeep Web server: ✓-Source code: X -Training: X [26] Web server: https://nsclbio. jbnu. ac. kr/tools/ncRDeep/. Although not mentioned on the site, there is a limit of 1,000 ncRNA per prediction task.
Moreover, it is not possible to export results. ncrna-deep Web server: X -Source code: ✓-Training: ✓ [27] GitHub repository: https://github. com/bioinformatics-sannio/ncrna-deep. Dataset preparation has to be done separately, but the process is explained and illustrated in the documentation and in the code. RAG+DL Web server: X -Source code: X -Training: X [37] The repository containing the dataset and source code no longer exists. ncRDense Web server: ✓-Source code: X -Training: X [29] Web server: https://nsclbio. jbnu. ac. kr/tools/ncRDense/. Although not mentioned on the site, there is a limit of 1,000 ncRNA per prediction task. Moreover, it is not possible to export results. ncDLRES Web server: X -Source code: ✓-Training: ✓ [33] GitHub repository: https://github. com/linyuwangPHD/ncDLRES.
No examples of how to use the tool, but very basic instructions are given. RPC-snRC Web server: X -Source code: X -Training: X [28] The GitHub repository linked in the paper no longer exists. NCYPred Web server: ✓-Source code: ✓-Training: ✓ [32] Web server: https://www. gpea. uem. br/ncypred, GitHub repository: https://github. com/ diegodslima/NCYPred.
The simulated orientation with the highest dot product is assigned to that point. The software used to perform DI was EMsoft 5. 0  [30]. To obtain accurate information on the phases and orientations from DI, the Efit package in EMsoft was employed  [30]. Here, a Zr reference pattern from the studied dataset with a high band to background contrast was refined. The pattern centre, scintillator distance and detector pixel size were refined from the initial AZtec derived values. Prior to indexing, the patterns were binned four times to reduce the pattern noise, ensuring that pattern noise was not affecting the indexing and effective indexing of the hydride patterns could be obtained. The crystal structures were assigned as face centre cubic (FCC) with space group Fm3m lattice parameter a o = 4.
OR analysis was performed using the AZtecCrystal OR analysis tool inspired by Une et al. [47]. This technique was expanded allowing ORs across the whole micrograph to be visualised such that inferences about the order of hydride precipitation could be made. hydrides located on the fringe adhere to γ and δ-Minor OR, respectively, whilst the bulk δ phase follows the δ-Major OR. The fraction of boundaries adhering to each OR was investigated (Table  1 ), representing a dataset of ~100 Zr grains and ~10 hydride clusters. The greatest percentage of hydride-matrix boundaries comes from γ-Zr OR (38%), followed by the δ-Minor OR (36%). It is evident that the conventional {0001} α ||{111} δ-Major and 〈1120〉 α ||〈110〉 δ-Major OR still populates around 26% of the hydride phase boundaries. To understand the origin and distribution of the δ-Major OR, analysis of the Zr grains where this OR is present was conducted.
Acknowledgements The Authors would like to thank  Framatome  for funding and providing the material for the present study. P. F. , R. T. and M. P. acknowledge support from  MIDAS   EPSRC programme grant  (( EP/ S01702X/1 )). The authors thanks  Jean-Paul Vassault  and  Julien Augereau  for sample preparation and follow-up experiments,  Mariano Marini  for hydriding the samples, and  Sylvain Duval  for reorientation tests. MDG acknowledges financial support from the  National Science Foundation  (grant  DMR-2203378 ) as well as use of the computational resources of the Materials Characterization Facility at  Carnegie Mellon University  supported by grant  MCF-677785. The research used  UKAEA's Materials Research Facility , which has been funded by and is part of the  UK's National Nuclear User Facility and Henry Royce Institute for Advanced Materials.
Study design The LVIS Etude post-inscription (LEPI) study was a prospective, observational, and multicenter study that included 17 French centers, conducted between February 2018 and December 2019. The aim of the study was to evaluate the morbidity, mortality, and efficacy of LVIS stents in an intention-to-treat population. All centers treated enrolled patients with LVIS and LVIS Jr consecutively and exhaustively. The study received the required national regulatory authorization from a French national Ethics Committee (Comit e de Protection des Personnes OUEST III). The LEPI study was registered in ClinicalTrial. gov under NCT 03553771. All enrolled patients agreed to the use of their anonymized data collection in the study frame. The appropriate Ethics Committee reviewed and approved the Informed Consent Form for this study.
The study of protein molecular surfaces enable to better understand and predict protein interactions. Different methods have been developed in computer vision to compare surfaces that can be applied to protein molecular surfaces. The present work proposes a method using the the Wave Kernel Signature : Protein LOcal Surficial Similarity Screening (PLO3S). The descriptor of the PLO3S method is a local surface shape descriptor projected on a unit sphere mapped onto a 2D plane and called Surface Wave Interpolated Maps (SWIM). PLO3S allows to rapidly compare protein surface shapes through local comparisons to filter large protein surfaces datasets in protein structures virtual screening protocols.
Benchmarking Dataset The benchmarking dataset is a subset of the Protein Shape Retrieval Contest track dataset of SHREC19  [20]. The subset contains 14 protein shape classes, based on the protein level of the SCOPe classification  [10]. Each protein shape class contains between 20 and 30 conformations for a total of 403 protein conformations. The IDs used are in the following format : PDBID A X, "PDBID" being the id of the PDB with 4 letters, "A" being the protein chain and the "X" being the conformation number according to the PDB file. For each structure of the dataset, the Solvent Excluded Surface (SES) is computed using EDTSurf  [47]  with default parameters. EDTSurf output triangular meshes are stored as. ply file, converted to. off and. pcd formats, required by the different shape comparison methods.
Results The evaluation of the performance of PLO3S in enrichment has been performed on a protein shapes dataset derived from the Protein Shape Retrieval Contest of the SHREC 2019 community benchmark  [20]. The performance of PLO3S is compared with spectral, geometry-based shape comparison methods and protein structure comparison methods. For all the shape comparison methods, blind all-to-all dense comparison have been performed on the whole dataset. Dense comparison is performed when all-to-all descriptor points are compared during the computation of the surface shape similarity.
Illustration of PLO3S method with selected examples Three proteins were selected from the SHREC19 dataset, herein used as the benchmarking dataset, to illustrate the PLO3S method functioning and outputs. The selection was made to include two proteins presenting similar SWIMs, ubiquitin and thioredoxin, and one protein presenting a dissimilar SWIM compared to the two former proteins, Macrophage Inflammatory Protein (MIP, Figure  4 ). Ubiquitin and thioredoxin display a globally spherical shape whereas MIP is more elongated. The SWIMs of MIP are less dense, as illustrated by the large blue area representing values of the WKS close to 0. The area containing more discriminating information, the red area, is located at the bottom of the SWIM (close to the pole) and then distorted because of the mapping of the unit sphere to the 2D plane.
Evaluation of PLO3S in enrichment The overall results for PLO3S are shown in figure  7. Precision decreases steadily from 0. 9 to 0. 2 for the PLO3S method (Figure  7a ) as the recall increases from 0. 05 to 1. The recall increases up to 0. 9 when considering the first 62% of the results, and then increases steadily up to 1. This indicates a compromise between downsizing the dataset and removing a maximum of true negatives. Negative Predictive Value (NPV) ranges from 0. 93 to 0. 99 with a peak at 37% of the dataset size (Figure  7c ). This peak corroborates the recall value stabilizing at 62% of the dataset size as the NPV is determined by the negatives while the recall is based on the positives. computation time of the descriptor SWIM and of the comparison of two SWIM descriptors are shown in Table  1.