dataesr/scientific-paragraphs-categorization huggingface dataset:
Data availability Source data for the results presented in this paper are available on GitHub (https:// github. com/mwang87/ReDU-MS2-GNPS/tree/master/examples). All curated sample information can be downloaded from the ReDU homepage (https:// redu. ucsd. edu/) by selecting 'Download Database'. The current version of the ReDU information used to generate the results in this paper is archived in the GNPS/MassIVE repository (http://gnps. ucsd. edu). The accession number is MSV000084206 (https://doi. org/10. 25345/C5407D).
Methods ReDU content. The homepage of ReDU (https://redu. ucsd. edu/) is the launch point for different analyses, centered around ' Analyze Your Data' or ' Analyze Public Data'. It also links to 'Documentation' , 'How to Contribute Data' , 'ReDU Sample Information Validator' , 'Download Database' and 'File Query-Sample Information'. The 'Documentation' option (Supplementary Fig. 5a ) links to the ReDU documentation, and the 'How to Contribute Data' option (Supplementary Fig. 5b ) links to the subsection of documentation that lists the steps necessary to contribute data to ReDU. The 'ReDU Sample Information Validator' (Supplementary Fig. 5c ) links to a drag-and-drop validator (https://redu. ucsd. edu/ReDUValidator) that verifies that the sample information template required for data contribution adheres to the required formatting and terms in a controlled vocabulary (additional terms must be submitted via GitHub at https://github.
com/mwang87/ReDU-MS2-GNPS/issues). Supplementary Fig. 5d  links to a text field into which a file name can be queried and any associated metadata are displayed. 'Download Database' (Supplementary Fig. 5e ) downloads all the sample information included in ReDU in a tab-separated text file. 'Download Annotations' (Supplementary Fig. 5f ) downloads all the MS/MS annotations. Links to specific analyses are detailed below. The ReDU server is built using the Python flask framework, SQLite and a Vue. js front end. Data and sample information contribution. Data files (. mzXML or. mzML) and a ReDU-validated sample information (metadata) table are necessary for inclusion of data in ReDU and must be uploaded to a public MassIVE dataset. A sample information template and validator are provided. Detailed step-by-step instructions can be found in the ReDU documentation (https://mwang87.
github. io/ReDU-MS2-Documentation/HowtoContribute/). Chemical annotations based on MS/MS reference library matches. MS/MS data were reanalyzed in a consistent manner to provide chemical annotations based on spectral library matches. The search was performed on the MS/MS product ion scans in files located in MassIVE de novo (that is, original MS/MS data and not the reported results) using GNPS' default parameters. The resulting MS/MS spectral matches (that is, annotations) were counted per file and tabulated; multiple hits to the same CCMSLIB ID in the same file were counted once. All annotation information was downloaded from ReDU (Supplementary Fig. 5f ) and processed in R. Script is available on GitHub in the examples folder (https://github. com/mwang87/ ReDU-MS2-GNPS/tree/master/examples).
Supplementary Table  1  displays the number of MS/MS reference library spectra available in each library in GNPS (for example, GNPS-LIBRARY) and the total number of annotations in ReDU per library. Further information can be found at https://proteomics2. ucsd. edu/ProteoSAFe/ result. jsp?task=ba6a5b6a1c0946b3a641c67ad59fb2df&view=production_library_ sizes#%7B%22table_sort_history%22%3A%22main. number_spectra_dsc%22%7D. Principal-component analysis. PCA was performed on the counts of each chemical annotation from GNPS spectral library matching using GNPS' default parameters (https://mwang87. github. io/ReDU-MS2-Documentation/). PCA was performed in Python with scikit-learn. The eigenvector matrix was retained and used to calculate the location of the projected points. Multivariate analysis of public data.
Emperor (https://github. com/biocore/ emperor) was used to generate interactive visualizations using the results from PCA (Supplementary Fig. 5g ). Emperor has many plotting options (including the axes and the color of points based on sample information) and filtering options and can rescale data. Clicking on any of the points in the plot causes the file name to be displayed in the bottom-left corner. The plot can be saved as an image file. Additional instructions on Emperor can be found in its online documentation (http://emperor. microbio. me/uno/).
Introduction Ensemble-based supervised classification consists in training a combination of many classifiers learnt from various algorithms and/or sub-samples of the initial dataset. Some of the most prominent ensemble learning frameworks include Boosting, with the seminal elegant Adaboost  (Freund and Schapire 1997) , which has been the inspiration of numerous other algorithms, Bagging  (Breiman 1996)  leading to the successful Random Forests  (Breiman 2001 ), but also Multiple Kernel Learning (MKL) approaches  (Sonnenburg et al. 2006; Lanckriet et al. 2004)  or even the Set Covering Machines  (Marchand and Taylor 2003). Most of these approaches are founded on theoretical aspects of PAC learning  (Valiant 1984 ). Among them, the PAC-Bayesian theory studies the properties of the majority vote that is used to combine the classifiers  (McAllester 1999)  according to the distributions among them.
2009; Parrado-Hernández et al. 2012; Dziugaite and Roy 2018; Germain et al. 2015). In this paper, we revisit one of such algorithms, MinCq  (Germain et al. 2015) , which focuses on the minimization of the C-bound and comes with PAC-Bayesian guarantees. The C-bound, introduced in Lacasse et al. (2006), bounds the true risk of a weighted majority vote based on the averaged true risk of the voters, coupled with their averaged pairwise disagreement. According to the C-bound, the quality of each individual voter can be compensated if the voting community tends to balance the individual errors by having plural opinions on "difficult" examples. Although MinCq has state of the art performance on many tasks, it computes the output distribution on a set of voters through a quadratic program, which is not tractable for more than medium-sized datasets.
To avoid overfitting, it considers a restriction on the posterior distribution named quasi-uniformity  (Germain et al. 2015) , and adds an equality constraint on the first moment of the margin. Beneficially, it is ensured to perform as well in train as in generalization, according to Corollary 1 and Theorem 2. The main drawback of MinCq is its computational training time: its algorithmic complexity is O(m × n 2 + n 3 ) , which prevents it from scaling up to large datasets. CqBoost 2  (Roy et al. 2016 ) Like MinCq, CqBoost is an algorithm based on the minimization of the C-bound. It was designed to accelerate MinCq and has proven to be a sparser C-bound minimizer, hence enabling better interpretability and faster training. CqBoost is based on a column generation process that iteratively builds the majority vote.
It is similar to boosting in the way that, at each iteration t, the choice of the voter to add is made by greedily optimizing the edge criterion  (Demiriz et al. 2002). Once a voter has been added to the current selected set, CqBoost finds the optimal weights by minimizing the C-bound similarly to MinCq, solving a quadratic program. The sparsity of CqBoost is explained by its ability to stop the iterative vote building early enough to avoid overfitting. Nevertheless, even though CqBoost is faster and sparser than MinCq, it is still not applicable to large datasets because of its algorithmic complexity is O(m × T 2 + T 3 ) , where T is the number of boosting iterations.
Experiments 4 In order to experimentally study CB-Boost, we first compare it to CqBoost  (Roy et al. 2016) , MinCq  (Germain et al. 2015)  and GB-CB-Boost (Sect. 3. 2), focusing on efficiency, sparsity and performance on chosen datasets featuring various properties. Then, we compare it to Adaboost  (Freund and Schapire 1997 ) and other ensemble methods: Random Forest  (Breiman 2001) , Bagging  (Breiman 1996) , and Gradient Boosting  (Friedman 2001 ).
Protocol -Datasets All the datasets used in this experiment are presented in Table  1 , an indepth presentation is made in Appendix E. -Classifiers For each classifier, hyper-parameters were found with a randomized search with 30 draws, over a distribution that incorporates prior knowledge about the algorithm, but is independent from the dataset. They were validated by a 5-folds cross-validation and each experiment was run ten times, with the result in Table  2  being the mean and standard deviation over them. These results are not statistically significant, but multiplying the experiments helps avoiding an outlier split that could bias the results.
Results Table  2  shows that, for all datasets except Ionosphere and MNist-5v6, CB-Boost is performing at least as well as the other C-bound algorithms, considering the standard deviation. It is quite clear that MinCq has the best zero-one loss on nearly all the datasets, as it uses all the available hypotheses. However, CB-Boost is competitive with both CqBoost and MinCq, and is even better on three datasets. On the more complex Animals with Attributes, CB-Boost is the best of the C-bound algorithms. As expected, GB-CB-Boost shows no significant improvement of the accuracy, and is even less stable on australian, MNist-7v9 and 5v6. CB-Boost is competitive with both the quadratic C-bound minimization algorithms 11 times out of 13, and has the best mean 3 times. Considering its shorter computational time, it is more efficient at extracting the information.
Sparsity conservation Protocol We analyze here the convergence speed of the five previously studied algorithms with grid-searched optimal hyper-parameters, when needed (MinCq's margin hyper-parameter set to 0. 05, and CqBoost's margin and error parameters, respectively set to 0. 001 and 10 -6 ). The chosen dataset is MNIST 0v9 as it is more complex than UCI datasets but still small enough for CqBoost and MinCq to have reasonable computing time on it. We use the same hypotheses, training and testing sets as previously. Results Figure  2  shows that, even if CB-Boost converges a bit less quickly than Adaboost and CqBoost on the training set, its performance on the test set is slightly better than all the other algorithms. Moreover, it is visible that CqBoost's and CB-Boost 's sparsity prevent them from the slight overfitting of MinCq.
Performance comparison: noise robustness equivalence In this section, we compare CB-Boost and Adaboost considering accuracy. The aim here is (1) to quantify how far CB-Boost is from Adaboost and (2) to evaluate their compared robustness to noise. Please note that the version of Adaboost used for these experiments is Adaboost. SAMME  (Zhu et al. 2006) , which is more robust to noise than the original. Protocol We use the same framework as in Sect. 5. 2 regarding datasets, train-test splits and hyper-parameter optimization. A Gaussian distribution is used to add noise to the data (see Appendix F. 1 for a detailed protocol and an example). The MNist dataset requires a special pre-processing to reduce its contrast in order for the noise to have an effect on classification. See Appendix F. 2 for details.
Results In Fig. 3 , we show a view of the results with each matrix presenting the comparison between Adaboost and CB-Boost for each dataset (rows), and noise level Fig. 3  Zero-one loss of CB-Boost and Adaboost. For each dataset (rows), we used several levels of noise (columns). An orange square means that Adaboost is better than CB-Boost in zero-one-loss ( L(Ada) < L(CB-B) ), and the difference between their scores is printed inside. A blue square means that they are equivalent, when considering their standard deviation (columns). It shows that, except for the four first noise levels of ionosphere where the difference is noteworthy, no significant loss of accuracy is observed when using CB-Boost, as the maximum difference between the other scores is 2. 5%. In Appendix F. 3, the numerical results are provided.
Performance comparison: ensemble methods equivalence We present the result of the performance comparison between CB-Boost and four other ensemble methods : Adaboost  (Freund and Schapire 1997) , Bagging  (Breiman 1996) , Random Forests  (Breiman 2001) , and Gradient Boosting (a) for which we use the same protocol as in Sect. 5. 4. This experiment differs from Sect. 5. 4 in the fact that we do not mean the results on each dataset but instead, in Fig. 4  we plot one dot for each train/test split on each dataset, for a matter of legibility. Results In Fig. 4 , the distance to the x = y line represents the difference in zero-one loss. So, a dot under the line means that the ensemble method has lower loss and one over the line means it has higher loss. One can see that CB-Boost is similar to the state of the art for most of the datasets and train/test splits.
C-bound approximation characterization Here, we aim at empirically analyzing the approximation made by greedily minimizing the C-bound instead of using a quadratic program solver. To do so, at each iteration of CB- Boost, we run MinCq on the selected subset of voters to compute its optimal C-bound. Protocol We use the MNist 4v9 dataset with the same sets as previously and predefined hyper-parameters: CB-Boost 's maximum number of iterations is set to 200, and MinCq's margin parameter to 0. 05. Results Figure  5  shows that there is a slight, but noticeable difference between the C-bound of CB-Boost 's majority vote and MinCq's one. However, this difference only has a small impact on CB-Boost 's performance for the first 30 iterations in train and 80 iterations in test. So even if the expected difference in C-bound optimality is noteworthy, it does not impact the performance of CB-Boost.
C Appendix: intuitive understanding of direction choice on a toy example Here, we present a toy example that is meant to give the reader an intuition on the difference between gradient boosting bound minimization, and CB-Boost 's C-bound minimiza- tion process. To do so, we use a toy loss of two variables L ∶ x, y ↦ z and analyze how the two approaches tackle the optimization problem. In Fig. 7 , we plot it in two dimensions, a 3D plot is available here. 5 This loss is convex and defined as Fig. 6  The empirical (training) C-bound for CB-Boost and GB-CB-Boost on the four MNist datasets presented in Table  1  over 75 boosting iterations with c 0 , c 1 , c 2 constants. In Fig. 7 , we can see that when starting at the same place on the loss function's surface (the "Start" point), Gradient Boosting (GB) will choose the direction where the gradient is the lowest (x), and even with a perfectly tuned step size, will have to use two iterations to reach the loss' minima, as in this direction, it is not attainable.
So We subtract So, combining with I t 's expression found in Theorem 3, On every dataset, we generated 10 pairs of complementary decision stumps for each attribute to build our hypothesis space. For each experiment, used one half of the dataset to train our algorithms and the other half to test on unseen data. C F t+1 = 1 - �∑ m i=1 [y i (F I t (x i ) + I t h I t (x i )] � 2 ∑ m i=1 [y i (F I t (x i ) + h I t (x i )] 2 1 m. C F t = 1 - �∑ m i=1 [y i F I t (x i )] � 2 ∑ m i=1 F I t (x i ) 2 1 m. N t+1 = 1 m 2 m ∑ i=1 [y i (F I t (x i ) + I t h I t (x i ))] 2 , D t+1 = 1 m m ∑ i=1.
E. 2 MNist To be able to analyse CB-Boost 's behaviour on larger data, we used M-Nist (LeCun and Cortes 2010) 7  in which we selected four difficult couple of classes : (0,9), (5,3), (5,6) and (7,9). As it has 784 attributes, we used 1 pair of complementary decision stumps for each attribute for the hypothesis space and 4% of the dataset ( ∼ 471 ex. ) to train and the remain- ing examples ( ∼ 11000 ex. ) to test.
E. 3 Animal with attributes Finally, to have a dataset with a large number of attributes, we used Animals with Attributes (AwA)  (Lampert et al. 2009)   8  by fusing two descriptors (surf-hist and cq-hist) generating 4688 attributes. As each one on its own is barely relevant, we learned 2000 decision trees of depth 3 as the hypothesis space. They where trained on randomly subsampled attributes ( 60% of the original dimension, with replacement). We selected two classes: tiger and wolf, that are very confused and provide some challenge to differentiate (we denote this dataset AwA-TvW). And one half of the dataset is used to train (546 ex. ) and the other half to test (546ex. ).
F. 1 Data noising In order to noise our datasets, we use a normal distribution, centred in 0 and of variable standard deviation. This distribution is then scaled with the attribute's range, added to the data and capped to the limits of the attribute. For example, for z a black and white pixel attribute, varying in [0, 255], to generate a noise level of 0. 5, a 0-centered normal distribution with standard deviation of 0. 5, N(0, 0. 5) was used. For each attribute (column) of each dataset, we used its upper and lower limit to generate an adequate noise.
I t ) F I t -(F I t ) (F I t , h I t ) -(F I t , h I t ) 2. I t ) F I t -(F I t ) (F I t , h I t ) -(F I t , h I t ) 2. I t ) F I t -(F I t ) (F I t , h I t )We used the following datasets, from the UCI repository(Dua and Graff 2017)6 to test CB-Boost 's capacity to solve simple problems with few examples and/or a low dimension. We chose Australian (690 × 14) , Balance (625 × 4) , Bupa (345 × 6) , Cylinder (540 × 35) , Hepatitis (155 × 19) , Ionosphere (351 × 34) , Pima (768 × 8) , Yeast (1484 × 8).
I t ) F I t -(F I t ) (F I t , h I t ) -(F I t , h I t ) 2. I t ) F I t -(F I t ) (F I t , h I t ) -(F I t , h I t ) 2. I t ) F I t -(F I t ) (F I t , h I t )We used the following datasets, from the UCI repository(Dua and Graff 2017)6 to test CB-Boost 's capacity to solve simple problems with few examples and/or a low dimension. We chose Australian (690 × 14) , Balance (625 × 4) , Bupa (345 × 6) , Cylinder (540 × 35) , Hepatitis (155 × 19) , Ionosphere (351 × 34) , Pima (768 × 8) , Yeast (1484 × 8). E Appendix: datasets 2. E. 1 UCI datasets Similarly, D t + c � D t = F I 2 I t + 2 2. So, N t c � -D t c D t + c � D t = - I t I t F I t 2 I t + 2 2 Then so for iteration t N t = D t = F I t F I t C F t -C F t+1 = 1 -1 m 2 1 m = N t + c m ∑ i=1 N t D t D t + c � -[y i F I t (x i )] -1 -N t D t 2 N t + c , D t + c � = D t c -N t c � (D t + c)D t I t (h 2 , N t = (F I t ) 2 , = F I t F I t C F D.
We analyze the influence and interactions of 60 largest world banks for 195 world countries using the reduced Google matrix algorithm for the English Wikipedia network with 5 416 537 articles. While the top asset rank positions are taken by the banks of China, with China Industrial and Commercial Bank of China at the first place, we show that the network influence is dominated by USA banks with Goldman Sachs being the central bank. We determine the network structure of interactions of banks and countries and PageRank sensitivity of countries to selected banks. We also present GPU oriented code which significantly accelerates the numerical computations of reduced Google matrix.
Hence G pr doesn't provide much information about direct and indirect links between selected nodes. The most interesting role is played by G qr , which takes into account all indirect links between selected nodes appearing due to multiple pathways via the global network nodes N (see  [19, 20] ). The matrix G qr = G qrd +G qrnd has diagonal (G qrd ) and non-diagonal (G qrnd ) parts with G qrnd describing indirect interactions between nodes. The exact formulas for all three components of G R are given in  [19, 20]. The numerical computation methods of all three components of G R and numerical REGOMAX code developed in  [19]  are given in  [26]. In this work we also ported the original REGO-MAX code to NVIDIA CUDA technology using VexCL library  [27, 28] , with two notable modifications. First, we reordered the global Google matrix using Cuthill-McKee method  [29]  in order to reduce the bandwidth of the matrix, and thus improve cache locality of the code.
This resulted in about 30% reduction of the computational time. Second, the computation of the reduced matrix was performed in batches of 8-20 columns, which allowed us to further improve the performance of the method by another 50%. The computations were run at the OLYMPE CALMIP cluster  [30]  using NVIDIA Tesla V100 GPUs. Overall, we managed to achieve an approximately 8x speedup with respect to the original OpenMP implementation which was performed on a dual socket Intel(R) Xeon(R) CPU E5-2620 v2 @ 2. 10GHz system (2 × 6 cores). The REGOMAX-GPU code is available at  [31]. With the matrix G R and its components we can analyze the PageRank sensitivity with respect to specific links between N r = 255 nodes. This approach already demonstrated its efficiency as reported in  [22, 32].
Networks of banks and countries At present the network presentation of link interactions between nodes gained a significant popularity due to its compact presentation of interactions (see e. g. [33] ). In directed networks we will say that a node j has a friend i if j points to i and i has a follower j. Such a description has been used G R friends • CN • CA • IN • UK • DE • FR • US • 47 • 41 • 40 • 34 • 33 • 32 • 31 • 28 • 22 • 20 • 19 • 17 • 15 • 14 • 13 • 12 • 11 • 10 • 9 • 8 • 7 • 6 • 5   2 , and their group color defined in Table  1 ; countries are marked by their α 2 ISO 3166-1 country code and the color indigo. • 4 • 3 • 2 • 1 ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ Goldman Sachs ■ ■ ■ ■ ■ HSBC ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ Deutsche Bank ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ State Bank of India ■ ■ ■ ■ ■ ■ ■ ■ ■ ICB China G R followers • ST • TV • GQ • GM • MV • WS • GN • BN • BT • SB • QA • NA • JO • AF • VN • IE • TH • MY • BE • CH • MX • JP • CN • AU • IN • UK • DE • FR • US • 60 • 59 • 56 • 55 • 53 • 49 • 48 • 47 • 43 • 42 • 41 • 40 • 39 • 37 • 35 • 32 • 30 • 28 • 27 • 26 • 25 • 24 • 22 • 21 • 20 • 19 • 15 • 14 • 12 • 8 • 7 • 6 • 4 • 1 ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ Goldman Sachs ■ ■ ■ ■ ■ HSBC ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ Deutsche Bank ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ State Bank of India ■ ■ ■ ■ ■ ■ ■ ■ ■ ICB China for interactions between politicians  [20] , countries  [21]  and interactions between infectious diseases and countries  [32].
Cross-species scoring identifies orthologous melanoma enhancers Next, we asked whether the human-trained model DeepMEL can be used to predict MEL and MES enhancers in other species. We started with the dog genome as a test case, because the differential ATAC-seq peaks between the MEL (Dog-OralMel-18249) and MES (Dog-IrisMel-14205) dog cell lines can serve as true positives (Fig. 4A ). DeepMEL reached similar performance in human and dog for predicting MEL and MES regions, and this accuracy is significantly higher compared to using cis-regulatory module (CRM) scoring with PWMs (Fig. 4A ). Having confirmed that the human model can identify enhancers in the dog genome, we predicted MEL and MES enhancers across all six species. This furthermore allowed us to order all samples according to the MEL-MES axis (Fig.
Despite their proven advantages, sequence-based comparative approaches have limited power to identify orthologous regula-tory regions in distant species, in part because of the rapid evolution of distal enhancers  (Dermitzakis and Clark 2002; Lindblad-Toh et al. 2011). Methods, such as enhancer element locator (EEL), try to tackle this question by aligning TF binding sites to identify conserved enhancer elements  (Hallikas et al. 2006)  or by calculating the co-occurrence of sequence patterns  (Arunachalam et al. 2010 ). However, these methods are either supervised because they require user-provided PWMs  (Hallikas et al. 2006) , or it is difficult to extract the important biologically relevant features from these methods  (Arunachalam et al. 2010). In addition, the identification and exact localization of important (de novo) TF binding sites within enhancers is complex because motif discovery tools are often dependent on user-provided databases and motif-specific thresholds.