dataesr/scientific-paragraphs-categorization huggingface dataset:
The equitability indices were far from being normally distributed and their differences between organic vs. conventional systems were therefore tested using a non-parametric Kruskal-Wallis test. We calculate the Shannon and the equitability indices using both all the data across the 4-year period and the across-year average. Since we did not find any effect due to the variation over time, we finally kept the calculation done using the across-year average. All the analyses were performed in R Open 3. 3. 2 (MRAN 2016), using the "lme4" package for mixed models  58  , the "rcompanion" package for non-parametric models  59  , the "FSA" package to evaluate the significance of the effects  60  , and the "vegan" package for descriptive community ecology  61.
Acknowledgements We are grateful to the authors of the 77 studies whose extensive field work provided the data for this analysis. We would like to thank  FiBL colleagues Helga Willer  and  Julia Lernoud  for providing land-use data,  Verena Seufert  for providing part of the literature database and for valuable discussion and feedback on the manuscript,  Gail Wagman  for improving the English, and  Laurent Augusto ,  David Makowski ,  Laura Armengot ,  Adrien Rusch  and  Maya Gonzalez  for help with the statistical analysis. We are also grateful to the anonymous Reviewers and Editor for their valuable comments. This work was funded by the  Bordeaux Sciences Agro School and INRA's GloFoodS metaprogramme.
| State of the art in internet This section overviews the state of the art and emerging trends in Internet cooperation and discusses how a reputation system can contribute to it. Threat intelligence is already one of the emerging trends in the Internet security today. Predictably, the ability to share threat intelligence will be critical to the next generation of the security solutions. Networking community has already made efforts with creation of the frameworks, such as vulnerability databases (i. e. Common Vulnerabilities and Exposures 53 ), Open Indicators of Compromise,  54  and CERT coordination centers. 8 These developments aim to improve the Internet's responsiveness to threats, by creating frameworks for sharing threat information. However, the traditional methods of processing threat intelligence are no longer practical, since they require human intervention to encode threat information into firewall rules or written reports.
The key in the Internet security today is to automate as much as possible by designing a workflow that automatically detects, responds, and contains attacks. This paper briefly presents our approach that seeks to make an effort towards automating the process of detecting, sharing, and leveraging the threat intelligence to improve Internet security. Open Indicators of Compromise is one such format for recording, defining, and sharing the threat information in a machine-digestible format, minimizing the need for human intervention. Similarly, vulnerability databases aim to provide a baseline for sharing security vulnerabilities. For example, Common Vulnerabilities and Exposures is an industry standard for sharing common names of publicly known vulnerabilities, between security products and services.
The host-based solutions typically use a firewall on endsystems and leverage cloud computing to process threat intelligence from firewalls on Internet hosts, around the world. The cloud processing provides the security vendors with a global view of emerging threats and thus enhances preparedness. 56 Typically, threat recognition leverages vulnerability databases as well as searching of attack signatures in the processed traffic. The use of cloud computing offers advantages in saving end system computing resources and storage, provides the global threat overview, and thus quicker threat mitigation. But, for mobile networks, the use of host-based approach has several downsides, because it allows unwanted traffic to travel all the way to mobile device which (1) clutters the radio interface; (2) impacts bandwidth availability of network for legitimate users and network operations; and (3) disturbs sleep cycle of resource-constrained battery-powered terminals.
e. to create botnets or to receive payoffs in the resulting Internet. As a result, general users will be better off, since (1) amateur-or intermediate-level hackers that rely on the classical Internet weaknesses or on bot rentals will suffer from deployment of the Internet-wide reputation; and (2) the advanced hackers will preferably focus on high value targets, without being too active to cross the detection threshold for being blacklisted. The reputation system can offer advantages over state of the art, for example, in handling attacks on zero-day vulnerabilities, where classical methods such as vulnerability databases would fail to contribute. In comparison, the reputation system could contain the damage by demoting the reputation of hosts or networks that frequently originate malicious traffic, and thus limit the extent of exploitation of zero-day vulnerabilities.
Taking a step further, the misbehavior evidences aggregated against a host identity shall cause the network of the host to restrict the host's access (for example, to well-known public Internet domains only, such as Google, etc). This shall prevent a malicious host from exploiting zeroday vulnerabilities on weak Internet hosts that typically reside in the private Internet domains, and thus affect the bot-rental business that thrives by exploiting such vulnerabilities on legacy hosts and less security aware users. Similarly, the reputation system could contribute to the security of new/emerging services, where vulnerability databases have little to contribute initially.
The CES can contribute to meet the energy challenge of 5G 58 : facilitating longer battery lifetimes for wireless devices, such as mobile hosts or sensors, by allowing them to sleep as long as possible. 20 This is because the hosts in the private realm will have an improved network-based firewall, which can filter the malicious flows before they reach the end devices, and because hosts are only reachable via a policy. We have implemented a demonstrator of the Customer Edge Switch on Linux and have made it available for the community at 1 study. 17 The CES is designed to collect all the evidences and pass them on for trust processing. The CES makes the basic act of communication receiver-friendly, establishes identities for the hosts, and contributes to eliminate address spoofing and distributed denial of service attacks.
Transcription factor (TF) databases contain multitudes of binding motifs (TFBMs) from various sources, from which non-redundant collections are derived by manual curation. The advent of highthroughput methods stimulated the production of novel collections with increasing numbers of motifs. Meta-databases, built by merging these collections, contain redundant versions, because available tools are not suited to automatically identify and explore biologically relevant clusters among thousands of motifs. Motif discovery from genome-scale data sets (e. g. ChIP-seq) also produces redundant motifs, hampering the interpretation of results. We present matrix-clustering, a versatile tool that clusters similar TFBMs into multiple trees, and automatically creates non-redundant TFBM collections. A feature unique to matrix-clustering is its dynamic visualisation of aligned TFBMs, and its capability to simultaneously treat multiple collections from various sources.
We demonstrate that matrix-clustering considerably simplifies the interpretation of combined results from multiple motif discovery tools, and highlights biologically relevant variations of similar motifs. We also ran a large-scale application to cluster ∼11 000 motifs from 24 entire databases, showing that matrix-clustering correctly groups motifs belonging to the same TF families, and drastically reduced motif redundancy. matrix-clustering is integrated within the RSAT suite (http://rsat. eu/), accessible through a user-friendly web interface or command-line for its integration in pipelines.
INTRODUCTION Transcription factor binding motifs (TFBMs, simply called motifs below) are models describing the binding specificity of a transcription factor (TF). Such motifs are generally obtained by aligning the sequences of several binding sites, and summarizing the nucleotide frequencies per position. Motifs are commonly represented as position-specific scoring matrices (PSSMs)  (1)  and visualized as sequence logos  (2). Although the adequacy of PSSMs has been questioned for some particular TF classes  (3) (4) (5) (6) , e. g. in cases of dependencies between adjacent nucleotides, they are still the most widely used method to represent the binding specificity of a TF. Thousands of PSSMs are available in private or public databases, such as JASPAR  (7) , TRANSFAC  (8) , Cis-BP  (9) , FootprintDB  (10) , HOCOMOCO  (11) , which constitute key resources to interpret functional genomics results.
A well-known issue with these databases is motif redundancy  (12) , resulting from various causes: (i) for a given TF, multiple PSSMs can be built from different collections of sites characterized with alternative methods (i. e. DNase-Seq, SELEX, protein-binding microarrays (PBMs), ChIPseq); (ii) the binding specificity is often conserved between TFs of the same family; (iii) some databases contain PSSMs obtained from orthologous TFs in different organisms; (iv) some unrelated TFs recognize similar DNA motifs. In addition to this intra-database redundancy, interdatabase redundancy and the exponential growth of motif collections are becoming major issues. Indeed, the development of high-throughput methods to characterize genomewise TF binding locations (e. g. ChIP-seq, ChIP-exo) has led to an explosion of motifs, with a fast expansion of databases (e.
g. JASPAR almost doubled in size between 2014 (590 motifs) and 2016 (1092 motifs))  (12). In parallel, some individual studies targeting many TFs  (13, 14)  produce motif collections as large as the classical reference databases. This constant increase in the number of motifs and redundant e119 Nucleic Acids  Research, 2017, Vol. 45, No. 13  PAGE 2 OF 13 Table  1. Features of software tools available to perform clustering of PSSMs collections represents a real challenge for the community. Which collection to use? How important is the overlap between the different collections? Efforts to collect and integrate numerous up-to-date collections into a single metadatabase like FootprintDB  (10)  or cis-BP  (9)  are critical for the community. These metadatabases however do not deal yet with the redundancy issue, and keep increasing in size.
This now constitutes a bottleneck, by drastically increasing the time needed to compare motifs or to scan sequences with a complete motif database. Analysis of high-throughput datasets (e. g. from ChIP-seq experiments) also produces sets of redundant motifs. It is common practice to simultaneously run multiple de novo motif discovery tools, in order to benefit from their complementarity  (15) (16) (17) (18). While some motifs could be discovered exclusively by a given tool, most will be found independently by different tools, hence producing redundant motifs with small variations in length and/or nucleotide frequencies at some positions. Some of these variations may be important biologically, but remain undetected when inspecting unordered collections of motif logos. Motif redundancy can be automatically reduced by identifying sets of similar motifs and clustering them.
However, each of these tools presents some limitations: analysis based on a single metric, restricted number of input motifs, static visualization interfaces. We have developed matrix-clustering within the RSAT suite  (39) , motivated by the crucial need for a tool to cluster similar motifs, align them to facilitate visual comparison, explore each cluster in a dynamic way, and reduce redundancy either automatically or in a supervised, user-friendly way. We first show with two case studies that matrix-clustering simplifies the interpretation of motif discovery results, and that a dynamic view of aligned logos can reveal biologically relevant motif variants. We then consider two applications encompassing complete databases, which show that the program regroups motifs bound by TFs of the same family, and can be used to explore the complementarity between multiple motif collections.
Input formats and processing time matrix-clustering receives as input one or several collections of PSSMs (provided as separate files) with an associated 'collection name' (e. g. several PSSM collections obtained from different analyses or databases). This program supports different file formats: TRANSFAC (default), MEME, HOMER, JASPAR, etc. , and has no restriction on the number of input PSSMs, but users should be aware that the processing time increases drastically with the number of motifs (time complexity is approximately quadratic, see Supplementary Figure  S1 ). For small collections of motifs, the running time enables matrix-clustering usage via the website (e. g. 7 min for the first case study with 66 motifs). Large datasets can be treated with a stand-alone installation of the RSAT suite.
PSSM comparison Similarity between each pair of input PSSMs is calculated with the RSAT tool compare-matrices  (38, 39) , which can compute multiple similarity metrics in a single run: Pearson correlation (cor), sum of squared distances (SSD), Mutual Information, Information correlation (Icor), Euclidean Distances (dEucl), Sandelin-Wasserman Similarity (SW), as well as width-normalized versions of some metrics obtained by dividing the total length of the alignment by the number of columns where the two PSSMs overlap: normalized correlation (Ncor), normalized information content correlation (NIcor), normalized Euclidian distance (NdEucl) (see Supplementary Notes for the mathematical formulae). Each possible offset is tested for each pair of PSSMs in both orientations, and the program returns the best matching alignment.
Identification of motif clusters by tree partitioning As the RSAT program compare-matrices  (38)  can return several metrics simultaneously, any combination of these can be selected to define thresholds for the partitioning step, thereby enabling to combine their respective advantages. The global tree is traversed in a bottom-up way and, for each intermediate node, the selected metrics values are computed from all pairs of descendent leaves according to the chosen linkage rule (single, average, complete). Whenever an intermediate node fails to satisfy any of the threshold values, a new cluster is created by separating its two children branches.
PSSM datasets of the case studies Case studies 1 and 2: in order to illustrate the clustering of ab initio discovered motifs, we used 359 PSSMs obtained with the RSAT tool peak-motifs  (15, 40)  from 12 TF ChIPseq peak-sets  (41). We also collected the PSSMs obtained by analysing one ChIP-seq peak set with MEME-ChIP  (16)  and Homer  (42). Case studies 3 and 4: for full database clustering, we analysed 24 taxon-specific collections from 18 databases (Supplementary Table  S1 ): vertebrates (JASPAR  (7) , HO-COMOCO mouse and human  (11) , Cis-BP (9), Jolma 2013 'HumanTF' (4), Jolma 2015 'HumanTF dimers'  (13) , Uniprobe  (43) , Fantom5 'novel' motifs  (44) , hPDI  (45) , epigram  (46) , Homer  (42) , Encode (47)), plants (JASPAR, Athamap  (48) , Cis-BP, ArabidopsisPBM  (49)  and Cistrome (  14 )) and insects (OntheFly (50), JASPAR, dmmpmm and idmmpmm  (51) , Cis-BP (9), FlyFactorSurvey  (52) , DrosphilaTF (  53 )).
Choice of the default clustering parameters Parameters of matrix-clustering were chosen based on a detailed comparison between clusters of 374 PSSMs from HOCOMOCO human TFBMs  (11)  and their classification in 21 families taken from the TFClass database  (55). We tested four alternative similarity metrics (cor correlation, Ncor normalized correlation, Icor information correlation, and NIcor normalized information correlation), three linkage rules (single, average or complete), incremental series of partitioning threshold values on each metric (by step of 0. 05), as well as combined thresholds applied on a metric and its normalized version (Ncor + cor or NIcor + Icor). Based on this study, we defined the default parameters: the motif-to-motif similarity matrix is computed using Ncor with a minimal alignment width of 5 columns, the motif tree is built with the average linkage rule, and the partitioning criterion combines thresholds on two metrics: cor ≥ 0.
Case study 1: identification of TF binding motif variants within motifs discovered with multiple tools in ChIP-seq datasets It is common practice to perform ab initio motif discovery with several algorithms and to consider the motifs found by several approaches as robust predictions. Yet, some motif variants can be found only by a particular algorithm. This first case study aims at comparing motifs detected in ChIPseq peaks with three motif discovery tools: RSAT peakmotifs, Homer and MEME-ChIP. We re-analysed the ChIPseq peaks for the Oct4 TF (also named Pou5f1) in mouse embryonic stem cells (ESC) from Chen et al  (41). Altogether, the three tools produced 66 motifs: 22 discovered by RSAT peak-motifs, 25 by MEME-ChIP and 19 by Homer. matrix-clustering separated these 66 PSSMs into 13 clusters (Supporting website).
Case study 3: complete database analyses highlights relationships between motif clusters and TF families We evaluated whether a clustering of complete motif databases enables (i) to identify redundancy between motifs, and (ii) to regroup PSSMs from the same TF family. TFs are classified in families according to their DNA-binding domains  (55, 61) , which usually recognize similar binding sites. TFs belonging to the same families are thus often associated with similar TFBMs, which constitute a source of redundancy. We clustered the complete set of taxon-specific motifs from JASPAR (vertebrates and insects), and species-specific motifs from HOCOMOCO (human and mouse). The clustering of JASPAR insects (133 motifs grouped in 35 clusters) reveals a large cluster of 70 PSSMss (Figure  4A ; Supporting website) encompassing almost half of the database.
This corresponds to homeodomain-containing TFs, whose TF-BMs are characterized by the core consensus 5 -TAAT-3  (62). The dynamic browsing capabilities of matrix-clustering enable to manually reduce these 70 PSSMs to 10 distinct motifs (Figure  4B ). The numerous members of this family in the insect database reflect a bias in the content, as most of these PSSMs result from a single analysis covering many homeodomain TFs analyzed with bacterial one-hybrid system  (62). By contrast in vertebrates, the 641 human PSSMs of HOCOMOCO are reduced to 127 small clusters (Figure  4C ). We obtained similar results for JASPAR vertebrates and HOCOMOCO mouse collections (Supplementary Figure  S2A  and B , Supporting website). As HOCOMOCO includes the information about TF families imported from TFclass (55), we analyzed the correspondence between clusters produced by matrix-clustering and these TF families.
This dispersion is perfectly consistent with the well-known properties of these TFs: the sequence bound by each Zinc finger domain is determined by the four specific amino acids entering in contact with the DNA  (63). As above mentioned, we explored the impact of clustering parameters on the correspondence between clusters of PSSMs from Human HOCOMOCO  (11)  and the families of the bound TFs (see section 'Choice of the default parame- ters' and Supplementary Notes). The best correspondence, as measured by both Adjusted Rand Index (ARI) and geometric accuracy (Acc), was achieved with Ncor as matrixto-matrix comparison metric, average linkage rule to build the tree, and a partitioning combining thresholds on Ncor (≥0. 4) and cor (≥0. 6) (Figure  4F ). This case study demonstrates how matrix-clustering can handle large collections of PSSMs and automatically reduce their redundancy within a database, while correctly regrouping motifs from TFs belonging to the same family.
Case study 4: comparison and integration of multiple motif databases To evaluate inter-database redundancy and to automatically produce a non-redundant motif set, we clustered 24 motif collections and measured their cross-coverage (see Supplementary Table  S1  and Material and Methods for the complete list of collections). We first merged these public databases to obtain three taxon-specific collections for insects (7 databases; 1895 PSSMs), plants (5 databases; 1590 PSSMs) and vertebrates (12 databases; 7781 PSSMs), respectively. We then applied matrix-clustering and obtained 354 clusters for insects (19% of the total merged PSSM collection), 306 for plants (19%) and 1757 for vertebrates (33%) (Supporting website). In order to obtain non-redundant motifs whilst preserving specificity, we used more stringent partitioning criteria than the default (cor ≥ 0.
8 and Ncor ≥ 0. 65): the threshold on correlation ensures that the clustered motifs are highly similar and the additional threshold on normalized correlation selects the alignments covering most of the motif lengths, in order to separate composite motifs (e. g. , bound by a TF dimer) from their elementary components. We then explored the mutual overlap between the original collections by computing the cross-coverage (Figure  5 ). For the insect databases, Cis-BP, OnTheFly, FlyFactorSurvey and JASPAR are the most similar to each other, while DrosophilaTF is drastically different from all of them (Figure  5A ), likely because this collection was built by selecting motifs discovered exclusively on Drosophila promoters, and whose binding factors are unknown  (53). For the plant databases, JASPAR and Cis-BP are most similar to each other (Figure  5B ), which is coherent with Cis-BP being an integrative motif collection encompassing other public collections (including JASPAR).
The three other databases focus on sets of motifs characterized by specific experimental methods (PBM for ArabidopsisPBM, binding sites curated from literature for Athamap, DAP-seq for CisTrome). Regarding vertebrates, five databases have a similar content (HOCOMOCO human and mouse, JASPAR, cis-BP, Jolma 2013 'HumanTF'), which is explained by the integration of HOCOMOCO and JASPAR in Cis-BP, as well as by the similarity of the original datasets used to build the TFBMs (mostly public ChIP-seq, Selex-seq and PBM), yet with different algorithms (Figure  5C ). Note that the crosscoverage is not reciprocal since the number of motifs and the motif diversity differ among these databases. For example, JASPAR includes 62% of the content of Cis-BP, whereas the latter encompasses 86% of JASPAR motifs (Figure  5D ).
We observed that the motif diversity is not proportional to the database size (e. g. the 641 JASPAR vertebrate PSSMs cover 82% of the 1800 Cis-BP Human PSSMs). In contrast, the contents of the remaining databases differ considerably according to the different methods and data used to build the motifs: a single type of data (Uniprobe, derived from PBMs only), restricted numbers of sites (hPDI, 17 sequences per motif on average), data from ChIP-seq experiments targeting histone marks in different cell types (epigram), or motifs modelling TF dimers (Jolma 2015 'Hu-manTF dimers'). The low cross-coverage of Fantom5 collection of 'novel' motifs is consistent with the definition if this database, which is restricted to motifs without any match in reference databases  (44). In summary, this case study highlights how matrixclustering can be used to automatically reduce motif redundancy across multiple databases into non-redundant taxon-wise motif collections (available as Supporting files 1-3 and on the Supporting website) encompassing several thousands of PSSMs.