dataesr/scientific-paragraphs-categorization huggingface dataset:
(b) Matrix strategy The BFs from all the datasets are concatenated column-wise to obtain a matrix. A row in the matrix roughly corresponds to a k-mer (more precisely, to the position indicated by the hash value of a k-mer). In the original BIGSI approach (  12 ), all BFs have exactly the same size. In COBS  (45) , datasets of comparable cardinality are grouped into bins, leading to a collection of matrices of different sizes. (b) ( ) Tree strategy A query is one or more k-mers. Conceptually, one starts at the root node and then explores down the tree, always checking all the children of a node before moving to another node (breadth-first strategy). A counter of absent k-mers is maintained for each node. If it exceeds a certain threshold, the search does not propagate further down the subtree of that node.
(b) Matrix Strategy In BIGSI, each k-mer corresponds to a single BF location. The corresponding rows in the matrix are then extracted, and summed column by column to obtain a vector where each element contains the number of k-mers occurring in the corresponding dataset. and COBS also have low RAM consumption. An advantage of HowDeSBT is the small size of the index on disk. This demonstrates that highly-diverse datasets, in terms of k-mer contents, can also be efficiently stored in variants of SBTs.
Indexing human genome sequencing data To the best of our knowledge, only two methods (BEETL-Fastq and Population BWT) have been applied to the representation of full read information from cohorts of whole human genomes. BEETL-Fastq represented 6. 6 TB of human reads in FASTQ format in 1. 7 TB of indexed files. Population BWT managed to index (in a lossy way) 87 Tbp of data, corresponding to 922 billion reads from the 1,000 genome project. After read correction and trimming, the authors obtained a set of 53 billion distinct reads (4. 9 Tbp) and indexed it with a BWT stored with 464 GB on disk (requiring 561 GB of main memory for query). Metadata (e. g. , sample information for each read) was stored in a 4. 75 TB database. Given the apparent difficulty to perform large-scale experiments on human datasets, we conclude that this is not yet a mature operation.
Some cDBG im-   1. Overview of best-performing tools in terms of space and time requirements to build indices. These results are extrapolated from several publications. BIGSI and VARI-Merge results were also included during evaluations. The data processing time column refers to the time necessary to convert the original sequence files to the k-mer set indices (Bloom filters / CQF / Othello). The maximum external memory column corresponds to the peak disk usage when building the index. The time column is the time required to build the set of k-mer sets index (on one processor). The index size column is the final index size. plementations (Vari and Vari-Merge) support additional features such as bubble-calling and graph traversal. New query types should also be considered. For instance, the possibility to obtain the k-mer counts instead of presence or absence would greatly assist in gene expression studies.
SBTs and COBS do not need to load the entirety of the index into memory at query time since the query iteratively prunes irrelevant datasets. That is why SBTs are more suitable for short queries, while for large queries, kmer look-ups become a bottleneck. For very large queries (e. g. the k-mers from a whole sequencing experiment), only AllSomeSBT (47) has an efficient specialized algorithm. Moreover, the fraction of k-mers matching a set query, which is mainly used as a similarity proxy in those works, could be further explored from a biological point of view. For instance, a single substitution in a base is covered by k different k-mers. If the indexed sequences differ from the query on that substitution, these k-mer will not be found in the structure, and the match could be missed. The effect of changing the size of k has not been sufficiently assessed, nor has the biological impact on the results been evaluated when tuning the θ parameter (with the exception of RNA-seq quantification (8)).
g. , SRA) or private ones. An essential next step would be to have user-friendly tools. User friendliness can be seen from different perspectives. First, one may try to cast more concrete biological questions into simplified k-mer queries that can then be asked to the indices. Second, the results of queries could be presented in a manner that is more suitable to biologists rather than their current form, consisting mainly of the output of k-mer queries. For instance, a list of reads contained in the indexed datasets could be output for further investigation. However, indexing reads is more challenging, and this direction would require new developments for the structures to scale. Third, special attention given to user interfaces could help broaden the usage of these methods. Web interfaces 5 are challenging to maintain in the long run; thus another solution could be to provide offline pre-computed indices.
From these tables, DBG operations such as going forward, backward from a given node are shown to be possible in (  26 ), but we do not describe them here. where reads shorter than 20 bases are discarded. Despite using the same dataset, the studies vary slightly: they filter low frequency k-mers slightly differently  (11, 38) , and hardware used in experiments also differs. The data processing time column refers to the time necessary to convert the original sequence files to the k-mer set indices (computation of Bloom filters, CQF (Squeakr), Othello). The maximum external memory column corresponds to the peak disk usage when building the index. The time column is the time required to build the set of k-mer sets index (on one processor). The index size column is the final index size. BIGSI is not a compressed index, but the authors had explored the possibility to compress using snappy (https://google.
github. io/snappy/). Parameters used for the different methods were θ = 0. 9 and BF size of 2. 10 9 for the SBT methods, k = 20 as the k-mer size for all methods, and 34 "log slots" for Mantis from the estimation of their paper. Supplementary Box 6. Details on the hash-based strategies in color aggregative methods Here we give details on the set of k-mer sets implementations in the different hash-based strategies. In the Figure above, the terms compacted DBG and unitigs are defined in Supplementary Box 2. Minimal Perfect Hash functions (MPHFs) are functions that map a fixed set of keys to the range of consecutive integers from 0 to the number of keys. They allow to implement memory efficient hash tables  (55). For example in Pufferfish, MPHFs are used to associate a k-mer to its position in the graph unitigs.
In BLight  (42) , they associate a k-mer to its position in a bucket. Some of the techniques use minimizers. While there exist multiple definitions in the literature, here we will say that a minimizer is the smallest l-mer that appears within a k-mer, with l < k. "Smallest" should be understood in terms of lexicographical order. For example in the k-mer GAACT, the minimizer of size 3 is AAC, as all other l-mers (GAA, ACT) are higher in the lexicographical order than AAC. Minimizers are used here to create partitions of k-mers, which roughly have the same size. Such partitioning techniques reduce the footprint of position encoding. 16,000 Salmonella strains (NCBI BioProject PRJNA18384). Note that the Vari-Merge benchmark has more genomes than the COBS benchmark, but it possibly contains a lower variability in k-mer content.
(45): the DREAM-Yara index is built by interleaving the bits of each Bloom filters. Bits of the same rank are grouped together in bins of size n. These groups are concatenated to obtain a large vector of size n × (t + 1). Middle: BIGSI matrix built for each dataset (color dots), gray boxes represent BFs. Right: RAMBO is represented as columns of merged BFs. A query corresponds to a set of rows (in BIGSI) or a set of bins (in DREAM-Yara), which are then combined to obtain the result. Space and time required to build indices on bacterial datasets (extrapolated to 4,000 datasets). The table combines results from a benchmark done in the COBS article(45)(denoted by a in the table) and one from the Vari-Merge article (16) (denoted by b in the table). The COBS benchmark contains 1,000 microbial DNA files , consisting of various bacterial, viral and parasitic WGS datasets.
ACKNOWLEDGEMENTS This work was supported by  ANR Transipedia  ( ANR-18-CE45-0020 ) and  INCEP-TION  ( PIA/ANR-16-CONV-0005 ). This material is based upon work supported by the  National Science Foundation  under Grants No. 1453527  and  1439057  to PM. The authors are grateful to  Jan Holub  for feedback on the BOSS section, and to  Daniel Gautheret  for his comments and corrections.
Details on compression To efficiently represent a n × c color matrix, over n k-mers across c datasets, different schemes have been proposed. A color class is a set of colors. It can also be seen as a bit vector, and a line in the color matrix. Supp. Box 5 presents examples of the different techniques: the delta-based encoding used in Mantis+MST (c), the RRR/Elias-Fano coding (e) used e. g. in Mantis and VARI, the lossy compression using BF from Metannot (d), the BRWT principle (f), and the three strategies used in SeqOthello (g).
Note that the above description of BFT does not capture the full complexity of the data structure, and should only be used to build an initial intuition. k-mer aggregative methods In Supp. Box 8, we present indexing and query details, in a similar fashion than Boxes 4 and 5 in the main text, but more in depth. We show the index construction and query steps in SBT, BIGSI, and shows how COBS improves on BIGSI's representation while keeping the core idea.
Supplementary Box 4. BOSS graph structure In this Box we describe the BOSS data structure, as per its original flavor ((  26 )). Let's assume we want to build the BOSS from the two sequences CAGCCGA and CAGTCGA with k = 3 in the figure below, part  (1). Part (  2 ) is the De Bruijn graph from these sequences (no reverse-complements are considered here). In this representation, each vertex contains a 3-mer, and an edge represents a 4-mer existing in the original sequences, the label of the edge being the last nucleotide of this 4-mer. (3) represents the same information, but with the constraint that any nodes not containing $ must be preceded by k vertices (3 vertices) and must have at least an outgoing edge. Thus supplementary nodes are added. (  4 ) is the list of (k + 1)-mers in the (3) graph.
Supplementary Box 5. Details on building blocks in color aggregative methods. BF is updated with the presence of its k-mers. This creates the necessary redundancy in the structure. Since several datasets can be assigned to a same BF become union BF by informing for the presence/absence of k-mers in more than one dataset. Query is performed on the rows, each union BF giving a row-wise union of sets where the query could be present. The final sets containing the query are deduced by intersection of the different set unions. Table  S1. Time complexities for the construction and query of the main approaches. N is the total number of distinct k-mers, n is the number of datasets, Q the query size (number of k-mers). We denote by b the number of bits in a Bloom filter, and h the number of datasets that contain at least θ% of Q k-mers.
We consider k as a relatively small constant. * This time is derived from Theorem 1 in  (48)  with the assumption that the size of the Bloom filter b is roughly O(N/n). Note that there may be an additional complexity cost for building the topology of the tree through clustering. We note that in the worst case (majority of k-mers present in all datasets), the query complexities of SBTs would be O(Q × n). Type of query output Method Datasets containing at least θ% of the queried k-mers SBT, BIGSI, SeqOthello, Mantis, BFT Fraction of query k-mers present in each experiment SeqOthello Bubble calling from query k-mers VARI Given a color, list of k-mers having that color VARI-Merge.
Index construction In SBT ((a) in Figure  below ), all BFs in leaves are initialized to the same uncompressed size, which means that the structure is initialized according to the estimated total number of distinct k-mers to index (which is, in practice, estimated by k-mer counting techniques applied on a random sample of datasets). As it can be seen in the example Figure, union BFs in parent nodes can thus be simply deduced by applying a logical OR on the children BF pair. In BIGSI ((b) in Figure  below ), all initial BFs must have the same size, so this parameter is usually adjusted according to the sample with the highest k-mer cardinality. It is also required that the same hash functions would be used during the construction (in practice 2-5 functions, which allows rapid query). COBS (c) uses the same principle but Bloom filters do not all have the same size.
Memory (GB) Time (h, wall clock) Peak RAM (GB) Index Size (GB) SBT N/A 7. 7 a 44 a 79 a SSBT N/A BF 32 a 6. 1 a 13 a AllsomeSBT HowDeSBT BIGSI COBS N/A N/A 110 a 12 a how det all/sim some/rem 4. 0 a 82 a 5. 0 a 0. 05 a 29 a 430 a 1,000 a 11 a 85 a 7. 6 a -- Vari 1,000 b 11 b 136 b 51 b Vari-Merge BF 1,000 b 12 b BF 52 b 51 b Rainbowfish how 1,000 b 11 b how 136 b 51 b BFT det 900 b 52 b det 120 b 99 b Multi-BRWT Mantis+MST all/sim some/rem 1,300 b 36 b 42 b 12 b all/sim some/rem 156 b 52 b 1,300 b 51 b BF how det all/sim some/rem. DREAM-YARA BIGSI RAMBO n datasets. BF size. n. B < BF size Bloom filter arrays.
ACKNOWLEDGMENTS We thank the  Bloomington Drosophila Stock Center, Kyoto Drosophila Genomics and Genetic Resources, Vienna Drosophila Resource Center, Flybase, Developmental Studies Hybridoma Bank , and  A. Bardin ,  H. Bellen ,  J. P. Couso ,  A. Debec ,  N. Frankel ,  M. Dominguez ,  B. Edgar ,  D. Stern , and  N. Tapon  for providing fly stocks and reagents. We are grateful to  M. Bou Sleiman ,  Z. Zongzhao  and members of the  FP lab  for discussions and critical reading of the manuscript,  J. Favier  and  A. Destenabes  for help with transgenics, as well as  S. Bosch  for imaging. SAH was supported by  Fondation pour la Recherche Médicale and Communauté des Communes de Dannieh-Université Libanaise. This work was supported by  Agence Universitaire de la Francophonie (PCSI AUF-BMO) ,  École Doctorale des Sciences et de Technologie-Université Libanaise ,  The Federation of European Biochemical Societies, American University of Beirut URB ,  Agence Nationale de la Recherche  ( ANR, ChronoNet ) and  Fondation pour la Recherche Médicale  ( DEQ20170336739 ).
Data analyses The number of alleles, observed and expected heterozygosity  (Nei, 1987)  were calculated for each population using GENEPOP software  (version 4. 1, Rousset 2008 ). The Hardy-Weinberg equilibrium was tested using GENEPOP. Alleles were classified as private alleles when they were observed only in one population. The same software was used to determine the proportion of genetic variation partitioned among populations (F ST ). Arlequin software (version 3. 5,  Excoffier and Lischer, 2010)  was used to determine the proportion of genetic variation partitioned within populations (F IS ). Mann-Whitney tests were used to compare the results of wild populations and farmed stocks. To identify different genetic subgroups and to infer the genetic ancestry of individual animals to a given population, multi-locus genotypes were analyzed by a model-based clustering algorithm with the Structure 2.
Complexité et incertitude médicales sont source de retards diagnostiques et de mauvaises stratégies thérapeutiques et donc d'augmentation des coûts et des erreurs médicales. Leurs mécanismes sont multiples et liés aussi bien au patient et à sa pathologie qu'aux modes de prises en charge. Parmi les nombreuses solutions pour y faire face, l'utilisation de bases de données dynamiques et la collaboration entre professionnels sont particulièrement appréciées par les praticiens. C'est ce que nous avons mis en place avec la plateforme collaborative AdviceMedica, développée initialement sur un mode associatif et académique. Elle est parfaitement fonctionnelle et utilisée quotidiennement. Elle utilise depuis peu les outils d'analyse du langage naturel. L'allergologie devient la première spécialité en France à disposer d'un réseau d'entre aide collaborative, bientôt renforcé par des analyses automatiques à la pointe de l'intelligence artificielle.
Les experts peuvent cependant commettre des erreurs, particulièrement s'ils ne font pas assez attention aux détails et prennent des raccourcis (1), d'où l'intérêt d'une analyse collective. L'analyse de bases de données dynamiques telles qu'Allerdata® (http://allerdata. com/) pour la gestion des résultats d'IgE moléculaires est une aide précieuse également. Développée par des médecins allergologues et piloté par le Dr Hervé Masson, cette application assure non seulement une veille scientifique dans ce domaine mais aussi fournit un petit algorithme d'information / décision vis vis des allergènes croisant ou pas. Telle est également le cas pour la base de données EASSAFE® (http://www. eassafe. com/) pilotée par le Dr Jean-François Fontaine, en ce qui concerne la composition des produits agro-alimentaires, cosmétiques et chimiques.
Intelligence artificielle, données volumineuses de santé et pratique médicale Le monde de la santé est sûrement l'un des secteurs où les enjeux de l'intelligence artificielle (IA) sont les plus importants  (10). Depuis la définition restrictive en 1956 par John McCarthy comme « la science et l'ingénierie de la fabrication de machines intelligentes », l'IA est devenue un pan complet de recherche regroupant plusieurs technologies basées sur des algorithmes mathématiques appliqués à de nombreuses méthodes dont l'analyse profonde ou data mining, l'apprentissage automatique par renforcement ou machine learning, l'analyse du langage naturel ou natural language processing, les réseaux neuronaux ou neuronal networks et a bénéficié de progrès considérables à partir des années 2010, grâce à l'arrivée de nouveaux algorithmes, de la mise à disposition de volumineuses bases de données (Big data) et l'augmentation des puissances de calcul.
Par ailleurs, afin d'améliorer les soins, les établissements hospitaliers se regroupent en Groupements Hospitaliers de Territoire (GHT), et la mutualisation de leurs systèmes d'information prévue par la loi du 26 janvier 2016 a eu lieu le 1er janvier 2018. Elle sera exécutée 3 ans après par la convergence officielle de tous les systèmes d'information  (13). Les données volumineuses provenant de différents hôpitaux vont donc constituer une richesse d'information médicale sans précédent, donnant la possibilité d'extraire un savoir ou une connaissance  (14). Ces bases de données médicales, souvent hétérogènes, devront être adaptées à l'IA et d'accès fluidifié  (10). Elles devront également être complétées par les données provenant de la médecine libérale (cliniques et cabinets de soignants) et du monde digital citoyen (informations issues des réseaux sociaux et requête Internet, capteurs esanté et objets connectés), ce qui conduira à l'accumulation de données massives  (15).
Remerciements Ces recherches ont bénéficié du soutien financier de la société UPL France pour les expérimentations en conditions contrôlées (2014)(2015),  du métaprogramme INRA SMaCH-Sustainable Management of Crop Health (action Presume ) pour le projet « GeDuNem » ( 2012-2016 ) et du label du GIS PICLeg. Les auteurs remercient les partenaires et les professionnels agricoles qui ont permis la réalisation de ces études.
Dans un premier temps, il est possible d'interroger des bases de données afin de déterminer à quelle fréquence ce variant est retrouvé dans une large population. Il est, par exemple, possible d'analyser la base de données gnomAD (The Genome Aggregation Database) qui inclut les données de 123,136 séquençages de l'exon et 15,496 séquençages du génome d'individus non apparentés [54]. En effet, si le variant est fréquemment retrouvé, il est peu probable qu'il soit causal car cela signifierait qu'une large part de la population serait atteinte par cette pathologie. En général, il est admis qu'un variant peut être considéré comme intéressant si sa fréquence est inférieure à 1% [55]. Un autre critère de sélection d'un gène est son profil d'expression qui doit être compatible avec le phénotype étudié.
Recueil des informations Une étude rétrospective monocentrique a été réalisée à partir des prescriptions effectuées Pour chaque patient, le sexe, l'âge au moment de l'introduction du traitement, l'association ou non d'autres thérapies antipsychotiques lors de l'instauration, les NL prescrits en plus de la clozapine, la NFS et plus particulièrement les PNN ont été relevés. La neutropénie légère est définie par une valeur de PNN < 2000/mm 3. On parle de neutropénie modérée quand les PNN sont inférieurs à 1500/mm 3 et de neutropénie sévère quand ils sont inférieurs à 500/mm 3. L'agranulocytose, qui engage le pronostic vital, est définie par un score de PNN < 500/mm 3 en présence de signes infectieux ou par une valeur isolée de PNN < 100/mm 3  [6]. Pour les patients présentant une numération de PNN < 2000/mm 3 nous avons recueilli des données supplémentaires : l'âge lors de la survenue de la neutropénie, le délai de survenue post-instauration du traitement (jours), la posologie de la clozapine, le nombre et le type de traitements concomitants au moment de la neutropénie (classe pharmaceutique et caractère neutropéniant selon les données disponibles dans la base de données publique des médicaments  [7] ) et la présence ou non d'une monothérapie antipsychotique.
Cancer de la prostate Le pronostic du cancer de la prostate oligométastatique est proche de celui de celui non métastatique  [9]. l'étude issue des registres SEER (Surveillance, Epidemiology, and End Results (SEER)-Medicare insurance program-linked database), qui représente 28% de la population américaine, a montré que les patients atteints d'un cancer de la prostate métastatique unique avaient une survie globale et une survie sans progression métastatique significativement plus longue que ceux atteints d'au moins deux sites métastatiques  [10]  : survie globale médiane de 24 mois (un site) contre 15 mois (≥ 2 sites ; p<0. 001) ; survie sans progression métastatique de 33 mois (1 site) contre 21 mois (≥2 sites ; p<0. 001). Outre le nombre de lésions au stade métastatique, le site métastatique était également un facteur pronostique de la survie globale  [10].
Un cadre juridique adapté Au plan juridique, du fait de l'existence d'une législation européenne harmonisée, le cadre de recours à l'utilisation des données de santé à des fins de recherche semble balisé pour ce qui concerne la notion de propriété attachée aux données, et plus précisément aux bases de données. Cette législation reconnaît des droits aux producteurs de bases de données et la capacité dont ils disposent de concéder des droits d'exploitation sur ces bases aux bénéfices de divers utilisateurs. D'autres législations harmonisées au niveau européen trouvent à s'appliquer à l'IA afin de garantir les droits fondamentaux des citoyens européens : • En tout premier lieu la protection de la vie privée et donc des données personnelles de chacun. Le règlement général sur la protection des données (RGPD) entré en vigueur le 25 mai 2018 poursuit cet objectif au-delà des frontières de l'Union européenne.