dataesr/scientific-paragraphs-categorization huggingface dataset:
e. expansion), and network wide metrics (ground truth comparisons). The project has seen no activity in the last three years, and offer no documentation of available functionalities. It provided 17 algorithms; although some of them require the installation of additional tools and the compilation of additional code, due to the calling of external methods. • SNAP 14  : Stanford Network Analysis Platform is a general purpose network analysis and graph mining library. The core is written in C++ and there is also a python version built as a wrapper around the C++ one. The project is active, provides two crisp algorithms (the Girvan-Newman and the Clauset-Newman-Moore method) and an overlapping one Affiliation Graph Model fitting (AGMfit): it requires the installation of additional tools. • CommunityALG 15  : is a set of Matlab functions and algorithms for the community detection in networks that expands the BrainConnectivity toolbox 16.
This toolbox is widely used by brain-imaging researchers, and has been ported to many projects. • CDTB 17  : the Community Detection ToolBox is a MATLAB toolbox which can be used to perform community detection. The CDTB contains several functions and includes graph generators, clustering algorithms (12 CD methods) and finally clustering evaluation functions. Furthermore, CDTB is designed in a parametric manner so that the user can add his own functions and extensions. The project has not been maintained in the last five years. • Leidenalg 18  : this package implements the Leiden algorithm in C++ and exposes it to python. It scales well, and can be run on graphs of millions of nodes (as long as they can fit in memory). It works with direct, weighted and multiplex graphs and provides some support for community detection on bipartite graphs.
It provides implementations for six different methods derived from Leiden algorithm; the methods currently implemented are: modularity, Reichardt and Bornholdt's model using the configuration null model and the Erdös-Rényi null model, the constant Potts model (CPM), Significance, and Surprise. • GraphTool 19  : is a Python module for manipulation and statistical analysis of networks. The core data structures and algorithms are implemented in C++; this confers it a level of performance that is comparable to that of a pure C/C++ library, both in memory usage and computation time. Many algorithms are implemented in parallel which provides excellent performance on multi-core architectures. In term of community detection, it supports only methods based on bayesian inference of stochastic block models.
CDLIB include these methods by handling conversion to/from this library. In order to compare the selected libraries we identify the following set of features: • Programming Language. The choice of a programming language • CD algorithms. The set of CD algorithm implemented in the library, with the distinction from crisp and overlapping models. • Project Status. Whether the project is currently developed or if its support ceased. Table  5  reports a characteristization of the selected libraries. We can observe that half of the analyzed libraries are not actively maintained at the moment of our survey. Moving to library related characteristics, we observe a clear pattern: all libraries offer out of the box support for crisp CD algorithms while few libraries, natively support overlapping ones.
With this aim, we introduced CDLIB, a python library for community discovery that takes care of supporting its users from the model definition to the analysis/comparison and visualization of results. CDLIB primary effort focuses on providing a simple abstraction layer over the data model and in designing input/output specifications shared by both methods and evaluation functions. CDLIB is an ongoing open project: we plan to further extend it by integrating novel algorithms (contributions are welcome), by supporting alternative clustering definitions (i. e. , multiplex, evolving,. ) and by integrating evaluation/visualization facilities.
e intention of improving the single-trial classification performance ranges from increasing the database, adding artificial trials  [9] , where original data are deformed to create new artificial patterns, to EEG source reconstruction or transfer learning methods, where data from multiple users are used to train classifiers to improve the BCI system  [10, 11]. e single-trial approach saves time, but the signal-to-noise ratio (SNR) is very low, making ERP detection difficult. On the contrary, when the stimuli are repeated to enhance the ERP detection, these repetitions may become tedious and tiring for the user, and the averaging technique decreases the speed of the spelling. Another problem related to the average of repetitions is the assumption of stationarity. Latency jitter, amplitude variability, or phase artifacts between single trials can cause a flattening up to the elimination of transient characteristics  [12, 13].
Denoising Based on the Similarity in the Channels. We propose to denoise EEG signals considering the information of all channels based on their phase and correlations within the DWT transform. Let X be the matrix containing the whole dataset, and let x c (t) be the signal recorded by the c th electrode, c ∈ 1,. , C { }, at time t, t ∈ 1,. , T { }. e matrix of the recorded EEG signals can be defined as X ∈ R T×C. e denoising, through thresholding, can be done using the MRL coefficients, i. e. , using coefficients obtained for all channels (equation (  6 )), instead of the individual channel coefficients. We prune all the coefficients that are below a given threshold τ d to zero in order to reconstruct the signal using the filtered wavelet coefficients. e MRL computation is done through the combination of the phase angles of the real and imaginary parts of the wavelet decomposition.
Semblance-Based ERP Window Selection over Channels. A variation of this model is used to perform the semblance-based ERP window selection over channels (SEWS-2), as summarised in Algorithm 3, where the same temporal window is selected for all channels. To do this, the only difference in the process is to compute the grand average through all of the channels using GA T (t) � 1 |C| 􏽘 C c�1 A T c (t), GA N (t) � 1 |C| 􏽘 C c�1 A N c (t). (9) Further information about algorithms and methods can be found in  [41]. 2. 6. Databases 2. 6. 1. UAM Dataset. We employ a P300 dataset  [42]  recorded by the Neuroimaging Laboratory of Universidad Autónoma Metropolitana (UAM), Mexico, using the P300 speller  [4]  included in the BCI2000 platform  [43]. e dataset (http://akimpech. izt. uam. mx/p300db/; https:// tinyurl.
EPFL Dataset. is dataset (for further details about subjects, see http://mmspg. epfl. ch/BCI_datasets) [45]  was recorded by the Ecole Polytechnique Fédérale de Lausanne utilising six different images to evoke a P300 response. e images were individually and randomly flashed for 100 ms with an interstimulus interval of 400 ms. 32 electrodes were recorded (Fp1, AF3, F7, F3, FC1, FC5, T7, C3, CP1, CP5, P7, P3, Pz, PO3, O1, Oz, O2, PO4, P4, P8, CP6, CP2, C4, T8, FC6, FC2, F4, F8, AF4, Fp2, Fz, and Cz) using a BioSemi ActiveTwo system at a sampling rate of 2048 Hz.
When comparing with the baseline filter results, the combination of our two independent algorithms significantly improves the ERP detection in a single trial and only incurs a small computational cost compared to the time required for the training and classification phases. In particular, SEWS-1 obtains the best maximum and minimum accuracy, but, on average, both window selection algorithms are competitive. Besides, a better SNR makes the classification easier and implies that useless features removed by SEWS have less impact on the decision boundaries. e experiment using the EPFL dataset validates our techniques for single-trial ERP detection not only because it is a different dataset with a different display matrix and patients but also because it uses the same parameters used for the UAM dataset.
Turbulence Intensity-Based Jensen Model (TI-JM) This section gives a brief overview of TI-JM, which is used for developing the coordinated control strategies. The detailed methodology for developing TI-JM and validation using real-time data is given in  [7]. TI-JM modifies the wake decay coefficient (k) of the standard Jensen model  [25, 26]  using wake-added turbulence intensity. Wake decay coefficient presents how quickly the wake diffuses depending on hub height of the wake generating turbine (z) and the surface roughness length (z 0 ) as given in Equation (1)  [25, 26]. TI-JM has all the characteristics of the standard Jensen model  [25, 26]  except for the constant k. k = 1/[2 ln(z/z 0 )] (1) The Jensen model has widely been used for developing farm control strategies due to its processing efficiency  [2, 4, 7, 8, [39] [40] [41] [42]  and is also part of many industry standard software such as WindFarmer  [7]  and WindPRO  [31].
Results and Analyses Wake effects are negligible in above-rated conditions  [5] , hence only below-rated conditions were assumed in the simulations in this section. Efficiencies based on WindPRO and real-time SCADA data in full or near-full wake conditions were used as benchmarks. WindPRO direction bin was kept at 10 • , which is the finest possible. WindPRO uses the standard onshore and offshore values of k given in  [71] , for wake estimation. TI-JM used SCADA data for Brazos-row and SMV for tuning the initial value of k according to the conditions. For TI-JM and SCADA data, the directional resolution was maintained at 1 •. TI-JM was first compared with SCADA data and WindPRO using efficiencies based on greedy control. The optimal control strategies were then evaluated by comparing them with greedy control using TI-JM.
Contour plots of the three wind farm case studies in full wake conditions were used for depicting a comparison of conventional and coordinated control strategies. Data filtering was applied to ensure that only operational turbines were analysed. It was observed that the maximum efficiency in no-wake conditions for Brazos-row and SMV wind farms was not 100%. Instead it was 82% for Brazos-row and 86% for SMV. These discrepancies may have been caused by anomalies in SCADA data or other unknown operational issues. WindPRO and TI-JM did not consider any anomalies or issues with data, taking only wake effects for wind deficit estimation. Therefore, the maximum efficiency (82% for Brazos-row and 86% for SMV) was shifted to 100% by simply adding the difference (18% for Brazos-row, 14% for SMV) to the efficiency at all points.
• Full wakes (worst case) = 90 • ± 10 The shifted efficiency in Figure  5  shows that efficiency can drop to as low as 58% in full wake conditions. WindPRO and TI-JM estimated that efficiency can be as low as 30% and 62% in full wake conditions. The standard Jensen model available in WindPRO was used with a constant k = 0. 07, for such an onshore terrain. WindPRO uses the WAsP model  [31]  for analysing the impact of terrain on wake effects. On the other hand, TI-JM did not use any terrain model, rather it varied k according to wind conditions, instead of keeping it constant as discussed in Section 3. TI-JM used k up to 0. 25 for estimating wind speed deficits inside the farm. The initial value of k was the standard k for such terrains. Wind direction was derived from the data obtained from met mast.
The availability of bacterial pangenome data grows exponentially, requiring efficient new methods of analysis. Currently popular approaches for the fast comparison of genomes have the drawback of not being based on explicit evolutionary models of diversification. Making sense of bacterial genome evolution, and notably in the accessory genome, requires however to take into account the complex processes by which the genomes evolve. Here we present the Pantagruel bioinformatic software pipeline, which enables the construction of a complete bacterial pangenome database geared towards the inference of gene evolution scenarios using gene tree/species tree reconciliation. Pantagruel is a modular pipeline that combines state-of-the-art external software with unique new methods. It can be executed with no supervision to perform a standard pangenome analysis, or be configured by advanced users to integrate methods of choice.
A relational database underlies its data structure, allowing efficient retrieval of the large-scale data generated by integrative analyses of pangenome evolutionary history. From the reconstructed gene evolution scenarios, two main outputs are derived: firstly the gene tree-aware assignation of orthology, allowing the fine analysis of gene gain and loss history over the species phylogeny, and secondly a network of gene-to-gene association based on correlated events in scenarios of gene evolution, leading to the definition of co-evolved gene modules.
By contrast, associations are unlikely to result from contingent causes if they were conserved over long periods of time during which purifying selection acted on the genomes. Moreover, genes that remain associated via events of co-transfer -which are unlikely under neutral conditions, due to random sampling of transferred genes and frequent genome rearrangements or gene losses -reflect the effect of strong selection for maintained gene linkage  (Lassalle et al. 2017). This indicates that long-term co-conservation patterns are relevant signatures of selection that can be applied to study the evolution of accessory genes. We introduce the bioinformatic software pipeline Pantagruel, a modular suite for the building of a complete bacterial pangenome database, including a reference species tree, homologous gene family alignments, gene trees and most importantly, gene evolution scenarios inferred using gene tree/species tree reconciliation.
Pantagruel is implemented as a script that will deploy modules executing the various methods and third party-software as required. These modules can be used automatically in order for the integral building of a pangenome database, or independently by advanced users who desire to analyse intermediate output and/or to provide their own intermediary input files. Throughout the computations, results are gathered into a relational database which provides a naturally scalable interface for interconnecting the different aspects of the dataset and studying their patterns of association.
User interface The Pantagruel interface relies on a single command-line executable pantagruel that can execute subprograms, or tasks, that are to be executed in order: init followed by tasks 0 to 9. The init task sets up the parameters for the rest of the pipeline execution. Parameter values are passed on using command-line options and are stored in an environment file, a shell script defining environment variables that will be loaded at the beginning of every subsequent task. The environment file can be modified manually between tasks to adjust parameters, even though this is not recommended as it may create issues due to dependencies between tasks. General usage is described using the -h option and in more detail on the code repository web page https://github. com/flass/pantagruel.
0) Fetching of public genome assembly data and annotation of custom genome assemblies Input genomic data can be provided in several ways: either as a list of NCBI Assembly accession numbers to be automatically downloaded from the NCBI FTP site, or directly as a collection of genome assembly files (Figure  1 ). The preferred format for assemblies is a full set of files as can be downloaded from NCBI Assembly database (RefSeq format), including genomic FASTA (contigs), genomic GFF (annotation), but also coding sequences (CDS) and protein FASTA files. Alternatively, input can simply consist of genome assembly contig files (FASTA), for instance when using newly sequenced genomes. Unless a custom genome annotation is provided in a specific format mimicking the NCBI Assembly format (see manual), genome annotation will be automatically made using Prokka  (Seemann 2014 ).
5) Estimating the reference tree The reference tree is estimated from the concatenate of core genome gene alignments. The set of strictly core genes can be small when genome datasets are large and/or diverse, which would lead the derived tree to not be representative of the whole genome history. It is possible to relax the definition of this reference set to a pseudo-core gene set, defined as all gene families that are present in exactly one copy in almost all genomes in the dataset (e. g. 95%). Based on a concatenated alignment of these genes, a maximum-likelihood reference phylogeny is inferred using RAxML  (Stamatakis 2014 ) under the GTRCATI model (25 site categories), and branch support are estimated based on parametric bootstraps. The tree root is inferred using the maximum ancestor deviation (MAD) technique  (Tria et al.
2017). LSD  (To et al. 2016 ) is used to generate an ultrametric version of the reference tree (dated in arbitrary units). In order to correct for the bias in sampling of the phylogenetic diversity of the dataset and decrease the computational complexity associated with large phylogenies, we delineate clusters of very similar genomes, hereafter called populations, based on the (non-ultrametric) reference tree. We consider populations that can be nested, reflecting the recent emergence and evolution of bacterial clones from one another. Thus, while most often populations cover monophyletic sets of genomes, they are sometimes paraphyletic. The population delineation criterion relies on strong support of the stem branch and a long enough stem branch relative to the length of branches within the cluster; values of parameters can be modified through command line options.
Thus, the respective parts of gene sequence evolution that result from these two separate processes have to be identified, so that diversification can be modelled and analysed separately, under a phylogenetic framework and a population genetic framework, respectively. Therefore, gene trees are estimated in several steps, with two objectives: (1) separating recent vs. ancient diversification for separate analysis; and (2) decreasing the complexity of data given as input to parameter-heavy phylogenetic inference methods. In short, trees are first estimated with RAxML  (Stamatakis 2014). Clades of gene sequences with low topological support (including groups of identical sequences) are iteratively collapsed, leaving only one representative leaf per clade. For each collapsed clade (CC), the frequency of each represented population (i.
This inferred ancestral population is then used to relabel the leaf representing the CC. In certain cases, clusters of gene sequences belonging to monophyletic groups of populations suggest some sequences within the CC likely evolved vertically; in that case a subtree of population is grafted to the gene tree in place of the leaf representing the CC. These leaf relabelling or replacement operations are made on each tree of the Bayesian sample, thus preserving the diversity of topologies in the samples. Records of these operations are stored in the relational database initiated in task 3.
7) Gene tree/species tree reconciliation Dated gene tree/species tree reconciliation is conducted for each gene family to infer their scenarios of evolution, with events of gene duplication, transfer and loss (DTL) using ALE  (Szöllősi, Tannier, et al. 2013 ). Gene DTL event rate parameters are set free and estimated by a maximum-likelihood (ML) approach, and then 1,000 gene family evolution scenarios and associated reconciliations are sampled using a Bayesian approach  (Szöllősi, Rosikiewicz, et al. 2013). The output of ALE is then parsed to export inferred events to the relational database initiated in task 3, allowing to relate events involving CCs to their original constituent gene sequences.
Within the resulting matrix of pairwise SJEP scores, entries with high values were used to build a network of associations. The tightest hubs in this network reveal co-evolving gene modules, whereas connection between such hubs highlight recurring associations between modules, for instance between the large core genome gene module and smaller accessory gene modules. The specificity of the SJEP score is that it describes a pairwise association between gene lineages, and can therefore detect localized co-evolution in a pair of gene trees, without the assumption that co-evolution must have involved the whole gene family. This allows a much more sensitive detection of gene associations, but introduce the problem of high-dimensionality of testing the association for the many combinations of gene lineages that can be enumerated within the dataset.
For example, in a dataset of 1,000 genomes made of 5,000 genes, the number of pairwise comparisons is of the order of 10 13. This computational challenge is addressed by an efficient use of relational database (SQL) queries in the search of significant matches between gene lineage evolutionary scenarios and by imposing filters on the probability of events that are supporting lineage matches (> 0. 1), and on the co-evolution scores to report (> 1. 0). Even with these filters, the resulting network can be very dense. This potentially large number of association links is however redundant. In this gene lineage network representation, gene lineages from a same gene family are not independent as they share a significant fraction of their ancestral history. This may result in repeat association of closely related genes to the same set of genes.
Redundant many-to-many relationships between two groups of close homologs can be simplified into a simple link between those groups. For this reason, we aimed to group genes from a homologous gene family into sub-clusters. We define subgroups based on the orthology relationship (see task 8), as it ensures a fairly close relatedness between members and limits the size of a group to the size of the genome dataset. For each pair of OGs, we filter association links between member gene lineages to those with the highest score for each member gene lineage (best lineage hit) and report the mean of these best lineage hit scores as the OG-OG association score.
Software implementation Pantagruel is implemented as a master shell script commanding the task routines, written mostly in bash language. Each task script may itself call third-party programs or utility programs originally developed for Pantagruel-specific procedures, including phylogenetic tree manipulation and database query and dynamic management; these utility programs are mostly written in Python 2. 7, as well as in R, bash and SQL languages. This script can easily be used to run the whole pipeline using a command-line interface. The full package is available on GitHub at https://github. com/flass/pantagruel. Pantagruel was designed to run on a Linux server computer, and has been successfully tested on virtual machines 8 parallel cores, and 64 GB memory, as made available by the UK Medical Research Council Cloud Infrastructure for Microbial Bioinformatics (MRC CLIMB)  (Connor et al.
2016). A script is also provided to automatically install the dependencies. This script is fully compatible with Debian-type Linux operating systems such as Ubuntu; it may also be used on other Linux or Unix systems but some manual steps may be required for complete installation. Computationally intensive pipeline tasks such as gene tree estimation and gene tree/species tree reconciliation (tasks 6 and 7, respectively) spawn many independent computation jobs (one per gene family) that naturally lend themselves to parallelization. For these tasks, specific scripts are provided to execute them on a high-performance computer (HPC) cluster to speed-up the execution; only PBS/Torque job submission systems are supported so far. In addition, another version of the package is in development using the Bistro framework (https://github.