dataesr/scientific-paragraphs-categorization huggingface dataset:
For P. aeruginosa amikacin resistance, the third subgraph (min q = 2. 21 × 10 -6 ) represented a 10,000 bp plasmid acquisition. Using the NCBI nucleotide database  [32] , most of the unitigs in this subgraph mapped to the predicted prophage regions of an integrative and conjugative plasmid, whose structure corresponds to a plasmid, pHS87b, recently described in the amikacin resistant P. aeruginosa HS87 strain  [33].
For k-mer-based methods, annotations were retrieved by mapping unitig/k-mer sequences to the resistance and Uniprot databases (see Interpretation of significant unitigs (step 3) subsection of the Methods section), and completed when needed by Blast on NCBI Nucleotide database. Green cells correspond to resistance determinants already described in the literature. Grey cells represent unknown determinants. Within each category, annotations are ordered by increasing minimum p/q-values. p/q-values are reported only for the most significant annotations. For each method, the annotation with the lowest p/q-values is underlined. 'NC' means noncoding region and 'tnp' transposase. https://doi. org/10. 1371/journal. pgen. 1007758. t002 and DBGWAS methods ranked the causal rpsL determinant first, pyseer and HAWK reported their lowest p/q-values for the false positive katG determinant.
DBGWAS is fast, memory-efficient, and scales to very large panels To assess the scalability of DBGWAS to large datasets, we retrieved 5,000 genomes from M. tuberculosis, 9,000 genomes from S. aureus and 2,500 genomes from P. aeruginosa, as described in the Large panels subsection of the Methods section. We present in S9 Fig the runtime and memory usage performances for these panels. All 180 runs took less than 5 days and 250 GB of RAM on 8 cores. Both the computational time and memory usage increase log-linearly with the panel size. Moreover, at equal panel size, DBGWAS performance also depends on the genome complexity, requiring less computational resource for more clonal genomes such as M. tuberculosis. We also compared the computational performance of DBGWAS with pyseer and HAWK. The benchmark was performed on 13 datasets, including one large dataset of 2,500 genomes for each of the 3 species (see the Datasets subsection in the Methods section for details).
Detailed results are presented in S2 Table. DBGWAS was the fastest tool in 11 out of 13 experiments, always taking less than 2 hours. HAWK ran in less than 10 hours in 12 out of 13 experiments, and was a little faster than DBGWAS on two of the large-scale datasets. pyseer took from 13 to 53 hours on 9 experiments, and failed on the 4 others: one exceeded the disk space limit of 1TB, three exceeded the runtime limit of five days. It was brought to our attention during the reviewing process that piping the output of fsm-lite through gzip would decrease the disk space usage. HAWK was more parsimonious in memory usage than DBGWAS on the large scale panels. This can be explained by the fact that the 0. 8. 3-beta version of HAWK which we are using does not take into account the population structure, and as such does not have to compute an n × n covariance matrix, providing it a large gain in memory usage-and, to a lesser extent, runtime-for large panels.
The interpretability of the extracted subgraphs could also be improved by training a machine learning model to predict which types of event they represent  [43]. This automated labelling could guide users in their interpretation and allow them to search for specific events, such as SNPs in core genes or rearrangements. Several recent studies describe in silico models for defining a genomic antibiogram and hopes are high that such technologies will complement the classic phenotypic methods  [44]. Several studies have already demonstrated that in some cases, genomic antibiograms can be at least as good as phenotypic ones  [30, [45] [46] [47]. Contrary to our approach, these studies require extensive resistance marker databases. DBGWAS will surely contribute to the extension of such databases or to the development of agnostic genomic antibiograms.
This procedure is efficient because it relies on constant time operations. Firstly, we use GATB's Minimal Perfect Hash Function (MPHF)  [52]  to retrieve the index of a given k-mer, and then we use the previously computed association between k-mer and unitig indices to know which unitigs the given genome contains. Since these two operations take True positives are unitigs mapping to genuine variants described in resistance databases for the studied drugs  [7]. In both cases, the value of k leading to the best AUC is k = 31. https://doi. org/10. 1371/journal. pgen. 1007758. g005 constant time, producing this vector representation for a genome takes linear time on the size of the genome. It is important to note that the GATB's MPHF can be successfully applied here because we always use the same list of k-mers, i.
This method also offers the possibility to test for lineage effects, by calculating p-values for association between the columns of the matrix representing the population structure, and the phenotype  [5]. DBGWAS optionally provides bugwas lineage effect plots when the user specifies a phylogenetic tree using the -newick option. An example of the generated figures is available at http://pbil. univ-lyon1. fr/datasets/DBGWAS_support/ full_dataset_visualization/. Formally, the LMM represents the distribution of the binarized phenotype Y i , given the j-th minor allele pattern X ij and the population structure represented by a set of factors W 2 R n�p , by: Y i ¼ X ij b þ W T i a þ ε ij ; j ¼ 1;. ; p: ð1Þ β is the fixed effect of the tested candidate on the phenotype, a � N ð0; s 2 a Þ, s 2 a > 0 is the random effect of the population structure, and ε ij � iid N ð0; s 2 Þ are the residuals with variance σ 2 > 0.
The unitig selection can be either based on the FDR (q-value threshold) or on a number of presence/ absence patterns ordered by increasing q-values. Practically, this is done in DBGWAS using a Significant Features Filter (SFF). For a selection based on a FDR threshold, the SFF value is set between 0 and 1, while any integer value > 1 defines the number of patterns to consider. In our experiments, we choose not to apply a fixed FDR threshold, even though DBGWAS offers this option. Different datasets lead to different q-values, even by several orders of magnitude, and a single FDR threshold would lead to selecting a large number of unitigs generating more than 1,000 subgraphs on some of them (e. g. S. aureus ciprofloxacin) as shown in S8 Table. Instead, we retain the 100 patterns with lowest q-values.
Although arbitrary, this choice is tractable for all datasets and provides satisfactory results in our experiments. It does not provide and explicit control of the FDR: only the q-value provides an estimation of the proportion of false discoveries incurred when considering patterns below this value. Checking the q-values of the selected unitigs is therefore essential to assess their significance. If the default SFF = 100 is not satisfactory, it is also possible to re-run the third step only, with a more suitable SFF value. Graph neighbourhoods. We define the neighbourhood of each significant unitig u (defined by the SFF) as the set of unitigs whose shortest path to u has at most ne = 5 edges. Users can modify the ne value using the -nh option. The objects returned by DBGWAS are the connected components of the graph induced by the neighbourhoods of all significant unitigs in the cDBG.
Reducing the graph complexity can also be done by decreasing the ne value, using the -nh option. Representing metadata with coloured DBGs. The subgraphs are enriched with metadata to make their interpretation easier. We use the node size to represent allele frequencies, i. e. , the proportion of genomes containing the unitig sequence. We describe the effect β of each unitig as estimated by the LMM using colours, in the spirit of the coloured DBGs  [19]. Colours are continuously interpolated between red for unitigs with a strong positive effect and blue for those with a strong negative effect. Annotating the subgraphs. DBGWAS can optionally integrate an automated annotation step using the Blast suite  [60]  (version 2. 6. 0+) on local user-defined protein (-pt-db option) or nucleic acid (-nt-db option) sequence databases.
We annotate the subgraphs of interest by blasting each unitig sequence to the available databases. Users can then easily retrieve the annotations which are the most supported by the nodes in the subgraph, or with the lowest Evalue. Importantly, DBGWAS works with any nucleotide or protein Fasta files as annotation databases straight away. However, users can customize the annotation databases by changing the Fasta sequences headers to make DBGWAS results more interpretable. A common example is compacting the annotation in the summary page by using abbreviations or gene class names, and expanding them to full names in the subgraph page. Other custom fields can also be included in the annotation table by adding specific tags to the headers. A detailed explanation on how to customize annotation databases for DBGWAS can be found in https://gitlab.
com/leoisl/dbgwas/wikis/Customizing-annotation-databases. We also provide on the DBGWAS website a resistance determinant database built by merging the ResFinder, MEGA-Res, and ARG-ANNOT databases  [61] [62] [63] , and a subset of UniProt restricted to bacterial proteins  [24]. Subgraphs discussed in the Results section were annotated using these databases. Interactive visualisation. DBGWAS produces an interactive view of the enriched and annotated subgraphs, allowing the user to explore the graph topology together with information on each node: allele and phenotype frequencies, q-value, estimated effect, and annotation. The view is built using HTML, CSS, and several Javascript libraries, the main one being Cytoscape. js [64]. Results can be shared and visualised in a web browser. As a large number of components can be produced in one run of DBGWAS, we provide a summary page allowing users to preview and filter the subgraphs.
Filtering can be based upon the minimum q-value of all unitigs in the component (min q ), or based on the annotations. A complete description of the DBGWAS interactive interface is available in https://gitlab. com/leoisl/dbgwas/wikis/ DBGWAS-web-based-interactive-visualization. Re-running from step 2 or step 3. It is possible to re-run a part of the analysis if a first run with the default values was unsatisfactory. The -skip1 option allows to re-run from the second step, for instance to compute the lineage effects (adding the -newick option). It is also possible to re-run only the third step by using the -skip2 option, for instance when the default SFF and nh values generated highly connected graphs, or if the annotation was incomplete.
Datasets We used in our experiments genome sequences from three bacterial species with various degrees of genome plasticity, from more clonal to more plastic: M. tuberculosis, S. aureus, and P. aeruginosa. We also built large datasets with random phenotypes for these 3 species, and used them only for time performance and memory usage assessment. All panels are summarised in Table  4. TB panel. M. tuberculosis (TB) is a human pathogen causing 1. 7 million deaths each year  [66]. This species is known for its apparent absence of horizontal gene transfer (HGT) and, accordingly, most of the reported resistance determinants are chromosomal mutations  [67]  in core genes or gene promoters. Intergenic regions are also described to be instrumental in multidrug-resistance (MDR) and extensively drug-resistant (XDR) phenotypes  [9].
tuberculosis  (5, 504) , S. aureus  (9, 331) , and P. aeruginosa  (2, 802)  available on the NCBI RefSeq bacterial genome repository  [11] , and removed poor quality genomes. For each panel, We selected 3 bacterial species with distinct levels of genome plasticity, and with antibiotic resistance phenotypes available for several drugs. For each species, we also created large datasets by computing random phenotypes for all available genome assemblies from NCBI RefSeq. https://doi. org/10. 1371/journal. pgen. 1007758. t004 we generated random binary phenotypes. For a detailed time and memory assessment, we built several sub-panels from these three large panels at size points of 100, 250, 500, 1,000, 2,500, 5,000 and 9,000 genomes. To build these sub-panels, we sampled genomes uniformly from the panels.
k-mer-based GWAS pyseer. We installed pyseer  [6, 36]  commit ID d17602500a4530b0e68a679ed675 fdb12942f56f (9 commits ahead of pyseer v1. 1. 1). pyseer pipeline is composed of four steps: 1) k-mer counting; 2) population structure estimation; 3) running pyseer; 4) downstream analysis. To use the correct parameters, we followed the pyseer tutorial (https://pyseer. readthedocs. io/en/master/tutorial. html). For k-mer counting, we used fsm-lite (https://github. com/nvalimak/fsm-lite), filtering out all k-mers with a minor allele frequency smaller than 1%. For population structure estimation, we used Mash v2. 0  [74]. To run pyseer, we used 8 cores and a LRT p-value threshold of 0. 05. Downstream analysis involved getting the k-mers which exceeded the significance threshold (which can be found using the scripts/ count_patterns.
py script), sorting them by LRT p-value, blasting them against the two databases presented in the Interpretation of significant unitigs (step 3) subsection, and keeping the best hit for each k-mer. For reproducibility purposes, the scripts we used to run pyseer can be found at https://gitlab. com/leoisl/DBGWAS_support/tree/master/scripts/pySEER. HAWK. We firstly ran HAWK  [13]  v0. 9. 8-beta, as it allows correcting for population structure. Unfortunately, it was unable to find the known causal variants reported for P. aeruginosa levofloxacin and M. tuberculosis streptomycin resistances by other methods (see Tables  2  and 3 ). We therefore kept in our benchmarks an earlier version, HAWK v0. 8. 3-beta, which presented better qualitative performance for these two evaluated panels. HAWK pipeline is composed of five steps: 1) k-mer counting with a modified version of jellyfish  [75] ; 2) running HAWK; 3) assembling significant k-mers with ABYSS  [76] ; 4) getting statistics on the assembled sequences; 5) downstream analysis.
The first four steps were performed as described in HAWK's github page. However, in the first step, we had to remove the lower-count cutoff in jellyfish dump (parameter -L), since we are working with contigs and not reads. The last step was performed similarly as the one described for pyseer. For reproducibility purposes, the scripts we used to run HAWK v0. 8. 3-beta can be found at https://gitlab. com/leoisl/DBGWAS_ support/tree/master/scripts/HAWK_0_8_3_beta.
S8 Fig. Effect of k on the five first subgraphs obtained for P. aeruginosa amikacin resistance. When k varies, the plasmid (yellow) and the mercury reductase and transposase (blue) remain among the five top-rated subgraphs. However, k has an effect on the aggregation of subgraphs corresponding to different genetic events: the mutation on aac(6') gene (blue frame) always appears in the first subgraph but is merged with the large mercury reductase and transposase subgraph for k = 27, 39 and 41. The order of the subgraphs also varies with k: up to four ranks for some subgraphs, and others leave the top-5 list. (PDF) S9 Fig. Large scale analysis on computational resources usage. This figure describes how DBGWAS scales in terms of time and memory usage for large datasets, containing up to 9,000 genomes.
All runs presented in this table were executed with the default parameters, without optional steps (lineage effect analysis nor annotation of subgraphs), on a single Intel(R) Xeon(R) CPU E5-2620 v3 @ 2. 40GHz core. The datasets are described in the Datasets subsection of the Methods section. DBGWAS ran in less than 2,5 hours for all experiments in our benchmark. The maximum memory load (given between parenthesis in the Runtime column) was 11 GB of RAM. The panel size and genome length (given between parenthesis in the Panel column) did not drive alone the running performances; the genome complexity played an important role as well. To view the gain in performance of DBGWAS when running on multiple (8) cores, see S2 Table. (PDF) S2 Table. Benchmarking DBGWAS, pyseer and HAWK: Comparison of time and maximal memory load.
The total execution time is presented with the maximal memory consumption in parenthesis, in order of GBs. For pyseer and HAWK, the time and memory for each step is also detailed. All tools were ran on a same machine with 8 Intel(R) Xeon(R) CPU E5-2620 v3 @ 2. 40GHz cores, 315 GB of RAM and 1 TB of disk space. Each execution used all the 8 available cores. The datasets are described in the Datasets subsection of the Methods section. However, for the three large panels (Large TB, Large SA, and Large PA), here we just chose a random 2,500-genome sub-panel. Moreover, DBGWAS was ran with the default parameters, without optional steps (lineage effect analysis nor annotation of subgraphs). The parameters for pyseer and HAWK were the ones described in the k-mer-based GWAS subsection of the Methods section.
The last column presents the corresponding subgraphs found by DBGWAS, with their min q. (XLS) S8 Table. Number of subgraphs generated using different significance thresholds. This table shows the number of subgraphs generated when defining the significant unitigs as the ones with the 100 lowest q-values (default SFF = 100, 'top 100') or when using a 5% false discovery rate (FDR) threshold (SFF = 0. 05, '5% FDR'). Different datasets lead to different q-values, even by several orders of magnitude. For instance, a single FDR threshold leads to selecting a large number of unitigs generating several hundreds subgraphs for SA (S. aureus) panel. (PDF) S1 Appendix. Evaluation of association models. (PDF).
Acknowledgments The authors thank  Jean-Baptiste Veyrieras ,  Sarah Earle ,  Chieh-Hsi Wu  and  Daniel Wilson , as well as  Jean-Pierre Flandrois ,  Manolo Gouy ,  Stephane Schicklin  and  Ghislaine Guigon  for their insightful comments. The authors also thank the reviewers for their accurate comments and suggestions, which helped to improve the quality of the manuscript.
MJ, MT, PM and AvB are employees of bioMerieux. LL is funded by the  Conselho Nacional de Desenvolvimento Cientifico e Tecnologico -CNPq, Brazil , under the  Science Without Borders scholarship grant  process number  203362/2014-4. VL is funded by the  Agence Nationale de la Recherche   ANR-12-BS02-0008  (Colib'read) and  ANR-16-CE23-0001  (ASTER). LJ is funded by the  Agence Nationale de la Recherche   ANR-14-CE23-0003-01  ( MACARON ) and  ANR-17-CE23-0011-01  ( FAST-BIG ). This work was performed using the computing facilities of the  CC LBBE/PRABI. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
70 × 10 -31 ) rpsL (4. 85 × 10 -55 ) rpsL (5. 72 × 10 -47 ) rrs (5. 40 × 10 -8 ) rrs (2. 86 × 10 -9 ) rrs (1. 63 × 10 -14 ) rrs (3. 45 × 10 -20) Determinant described for other antibiotics katG (2. 61 × 10 -30 ) katG (1. 06 × 10 -28 ) katG (2. 12 × 10 -71 ) katG (1. 44 × 10 -57 ) espG1 (1. 20 × 10 -3 ) NC near tnp/PE (1. 13 × 10 -19 ) NC near tnp/PPE (2. 93 × 10 -57 )This table presents the annotation of the features identified by the tested methods with default parameters. The total number of reported features, as well as the execution time and memory load (in Gigabytes) are given in the header. For k-mer-based methods, annotations were retrieved by mapping unitig/k-mer sequences to the resistance and Uniprot databases (see Interpretation of significant unitigs (step 3) subsection of the Methods section), and completed when needed by Blast on NCBI Nucleotide database.
Since the pioneering work of Schumpeter in the 1940s, economists have agreed that a large component of modern economic growth has been driven by "innovation", that is, the arrival of new ideas. Nowadays, most papers on the relationship between scientific research and innovation use citation data to measure the production of new ideas in science and patent data to measure the creation of new potentially successful commercial ideas. Patents have become particularly important in this context for three reasons (Agrawal and Henderson 2002): • The patenting process requires that inventors' names, dates, assignee institutions, locations and detailed descriptions of the invention's claims be recorded. Innovation-related details are rarely recorded systematically outside of patent records. • Innovations that are patented are expected to be commercially useful.
Review of Complex Networks Over the past 30 years the methods developed by physicists for studying networks in statistical mechanics have been adapted to analyzing other types of networks including the world-wide web  (Broder et al. 2000; Albert 1999 Albert , 2000)) , power grids (Watts and Strogatz 1998), telephone call grids  (Abello et al. 1998 ) and airline timetables  (Amaral et al. 2000). Newman (2001)  and  Barabasi et al. (2002)  both studied citation networks in which the authors were the nodes in the network and a link was formed between two authors when they co-authored a paper. Newman (2001)   Although the databases went back earlier Newman limited his study to the window from 1995 to 1999 in order to obtain a good static photo of the conditions at that time. In contrast  Barabasi et al.
(2002)  studied the evolution over time of patterns of collaboration in two specific fields: mathematics and neuro-science, over the period from 1991 to 1998, using databases consisting of 70,975 different authors and 70,901 papers for mathematics and 209,293 authors 210,750 papers for neuroscience. By 2000, theoretical and empirical studies had uncovered three important results: firstly, most networks have the so-called small-world property which means that the average separation between nodes is rather small; secondly, real networks display a higher degree of clustering than expected for purely random networks and finally, the degree distribution follows a scale-free power-law form  (Barabasi et al. 2002). Initially it had been expected that the Web would be a random network like those characterized by  Erdos and Renyi (1959).
Acknowledgements Many people have kindly answered our questions and provided feedback at seminars. The main ones are listed here in alphabetical order:  Denis Allard ,  Hélène Beucher ,  Romain Bizet ,  Rich Chambers ,  Jean-Marc Chautru ,  Joao Felipe Costa ,  Sebastien Delamarre ,  Brigitte Doligez ,  Peter Dowd ,  Xavier Emery ,  Daiane Folle ,  Gaëlle Le Loc 'h,  Ute Mueller ,  Dean Oliver ,  Guiseppe Raspa ,  Christian Ravenne ,  Philippe Renard ,  Denis Schiozer ,  Olinto de Souza Gomes  and  Jeffrey Yarus.
Given the growing demand for studies dealing with natural disasters, the research fields of emotion and social cognition require validated picture stimuli of natural hazards. Such material is essential for studying perceptual processes and behaviors of exposed individuals, and it could find practical applications, such as the improvement of communication strategies during crises. We present the Natural Disasters Picture System (NDPS), a database of pictures of natural hazards, with an emphasis on volcanic threats, and their impact on the environment and humans. We first describe in detail the picture selection and database creation. We then report the validation procedure. One hundred twenty participants rated the pictures on the basis of four dimensions: valence, arousal, dominance and certainty.