dataesr/scientific-paragraphs-categorization huggingface dataset:
The genome sequence dataset The dataset used in this study gathered 47 Xylella genomes sequences, including 46 X. fastidiosa and one X. taiwanensis specimen (Table  1 ). The X. fastidiosa subspecies tashke could not be included as no strain or genome sequence are available. In some analyses, the three strains belonging to X. fastidiosa subsp. sandyi were separated into two groups, containing either the original strain Ann-1 (sandyi) or the more recently discovered relatives CO33 and CFBP 8356 (sandyi-like) both belonging to the unusual sandyi ST72  [17]. The CFBP 8073 strain, described as an atypical X. fastidiosa subsp. fastidiosa strain  [11]  was either included or not for analyses of this subspecies. The X. fastidiosa genome sequences involve 39 publicly available ones and seven newly individuals.
The strains sequenced in this work were selected based on their country of isolation, genetic diversity and host range, inferred from their belonging to the subspecies fastidiosa (CFBP 7969, CFBP 7970, CFBP 8071, CFBP 8082, and CFBP 8351), sandyi (CFBP 8356) and multiplex (CFBP 8078). Genome sequence characteristics are described in Table  2. Evaluation of strain synonymy using k-mers CFBP 7970, the X. fastidiosa and X. fastidiosa subsp. fastidiosa type strain  [31] , has various synonymous names (ATCC 35879, DSM 10026, LMG17159, http://www. straininfo. net/strains/901514 or http://www. bacterio. net/xylella. html) in other collections, as a result of strain exchanges between the American (ATCC), German (DSMZ), Belgian (LMG), and French (CIRM-CFBP) collections  [32]. As no genome sequences were available at the beginning of this study for any of these strains, CFBP 7970 was included in our dataset and its genome was sequenced.
These sequences shared high identity levels with a plasmid found in multiple Xylella subspecies, known as pXF-De Donno (subsp. pauca De Donno strain), pXF-RIV5 (subsp. multiplex RIV5 strain), pXF-FAS01 (subsp. fastidiosa M23 strain) or present but unnamed elsewhere (like in subsp. pauca CoDiRO strain and subsp. multiplex Dixon strain)  [2, [33] [34] [35]. But the blast analysis suggested its possible presence, as partial Data mining of the 16S rRNA SILVA database to assign occurrences of Xylella sp. The availability of the 47 Xylella genome sequences renders possible the analysis of the allelic diversity of the 16S rRNA marker gene. This housekeeping marker is widely used in bacterial phylogeny and taxonomy studies for various reasons including its vertical inheritance and ubiquity in prokaryotes.
It is also commonly used to survey microbial communities and as such is a marker to survey largely the environment. A total of 74 16S rRNA gene sequences was retrieved from our dataset, these either being present in one (n = 20) or two copies (n = 27) in X. fastidiosa and X. taiwanensis (Table  4 ). We detected 19 SNPs (over 1547 nucleotides, 1. 22%) specific to X. taiwanensis PLS 229 16S rRNA (Additional file 4). Specific mers were searched for within the Xylella genus (i. e. , the in-group included the 74 Xylella 16S rRNA copies; the out-group included all the SSU sequences from the Silva database other than Xylella-tagged) and the X. fastidiosa species (i. e. , the genome sequence of X. taiwanensis PLS 229 strain was included in the out-group). Five long-mers (referred to as LongXyl#1 to #5) specific to Xylella genus and four long-mers (referred to as LongXylefa#1 to #4) specific to the X.
fastidiosa species were identified. LongXyl  (23-43 nt)  and LongXylefa  (23-31 nt)  mers located between positions 212 and 866, and positions 202 and 1013, respectively, which include the V3-V4 hypervariable regions widely used in community profiling approaches (Additional file 5). Specific signatures obtained from eight nucleotide positions in the 16S rRNA alignment discriminated alleles from X. fastidiosa subsp. fastidiosa, multiplex, morus, sandyi and pauca (Table  5 ). The occurrence of these specific mers in 16S rRNA nucleotide sequences was investigated within the SILVA rRNA database (Additional file 4). A large proportion of the sequences retrieved from the SILVA database (n = 118/195) covered less than half of the total gene length, and only a minority (n = 70/195) covered at least three fourth of the length.
Reconstruction of the parental origin of the subspecies morus The subspecies morus was proposed to group strains pathogenic on Morus that derived from largescale intersubspecific homologous recombination events between ancestors from at least subspecies fastidiosa and multiplex. This assumption is based on the analysis of seven  housekeeping genes  [16]. We challenged it with whole-genome sequence datasets to further understand the contribution of the subspecies fastidiosa, multiplex and others in the parenthood of the subspecies morus. We used SkIf to identify mers specific of the morus group (i. e. two strains, Mul-MD and Mul0034) plus one of each of the other groups (Table  6B ; Additional file 6). The highest level of specific mers was found for the combination morus x fastidiosa x sandyi: the highest mer cumulated size represented 5% of the Mul0034 genome in size (which does not mean that all the 5% are unique to Mul0034).
Discussion While tools based on k-mers are mainly used to improve genome assembly  [36, 37] , SkIf (https://sourcesup. renater. fr/wiki/skif/) was developed to quickly extract information from genomic datasets from already assembled genomes. It allows to decipher genomic fragments associated with traits shared by a group of sequences of interest. This strategy is applicable to any scientific questions requesting the comparison of user-defined groups of sequences. Because management of X. fastidiosa outbreaks in France depends on the subspecies of X fastidiosa, it is of major importance to precisely define these subspecies, understand the robustness of these groupings and their meaning in terms of shared and specific genetic material. In order to detect genomic regions specifically associated with a group of organisms (i.
e. a subspecies) we applied SkIf to gain a better understanding of X. fastidiosa clusterings in subspecies. This tool was also used to mine large databases as a first step to evaluate worldwide dispersion of X. fastidiosa in natural settings. The phylogeny provided by shared k-mers was highly similar to the one based on ANIb, a reference method for analyzing phylogeny of bacteria  [38, 39]. However, phylogenies were much more quickly constructed using shared k-mers than were ANI calculations in JSpecies. Here, k-mers of 22 nt were used while ANIb and TETRA are calculated from k-mers of 1020 and 4 nt,   The current grouping of X. fastidiosa in five subspecies is inappropriate and is not supported by genomic data. This is obvious regarding the phylogenies reconstructed using shared-k-mers and ANIb (Fig.
An alternative scenario might be that the plasmid-like sequences have been integrated into the chromosome of ATCC 35879 whilst the plasmid was lost, and in this case, only CFBP 7970 and ATCC 35879 are indeed synonymous. The definite answer will come from an analysis of the raw reads of ATCC 35879 to check for the presence/absence of the plasmid (raw reads are currently not available in SRA) or from a plasmid extraction from the specimen stored at ATCC. Occurrences of X. fastidiosa found in Silva rRNA database were assigned to subspecies that are coherent with sample designation. Specific mers of the genus Xylella and of the various subspecies of X. fastidiosa were retrieved in the V3-V4 region of the 16S rRNA encoding genes. One use of these tools is to taxonomically assign at the subspecies level the occurrences of X.
fastidiosa from large database. The sample description and especially the name of the plant species of isolation allow to validate the assignation provided by specific mers. It should however be noticed, that some plant species like almond, olive tree, oleander, coffee tree, or citrus may be host of several X. fastidiosa subspecies (www. pubmlst. org/xfastidiosa) [52]  and in consequence this a posteriori validation will not always be possible. Long read sequencing will generate more full length 16 s rRNA gene sequences which will facilitate subspecies discrimination. Another tempting use of these tools could be to survey large metagenome database for occurrences of these markers. This is however currently not feasible due to an astonishingly too long time required to download data or incapacity to browse those databases using our markers.
Conclusions Skif is a freely available, bioinformatic tool dedicated to the identification of specific mers. Although the results presented here were applied in the context of the emerging plant pathogen Xylella fastidiosa in Europe, this software is useful to answer many other questions beyond this scope. It is adapted to mine various group of sequences (gene, protein, genome, metagenome databases) defined by the user to identify specific or shared features. In the context of X. fastidiosa it allowed to i) refine the current grouping in subspecies that are not supported by genomic data; ii) trace the origin of the subspecies morus, a plasmid from Hib4 strain and the extent of synonymy among specimen representing the same initial strain in microbial collections; and iii) design markers that are specific to each subspecies of X.
Genome sequencing, assembling, and annotation For sequencing, diluted libraries (4 nM) were denatured as described (Illumina Preparing DNA libraries for Sequencing on the MiSeq protocol), resulting in 20pM denatured DNA. The final DNA concentration used for sequencing was 12pM in a 600 μl volume containing 1% of PhiX control. The sample was deposited in a V3 cartridge. The seven X. fastidiosa genomes were sequenced with the Illumina MiSeq v3 600 cycles technology at the ANAN plateform, SFR QuaSav, Angers, Fr. Genome assembly was performed using a combination of Velvet  [54] , SOAPdenovo and SOAPGapCloser  [55]  assemblers. Structural and functional annotations were conducted with Eugene-PP algorithm  [56] , using a concatenation of the Swissprot database and the publicly available X. fastidiosa 9a5c  [57]  and Temecula1  [58]  genomes.
By comparing mer length and the positions of the various occurrences, SkIf concatenates the overlapping mers into long-mer using the script "getLongestKmersNC. pl" (option: -k 22; available with Skif ). A list of located mers or long-mers specific to the group of interest was hence obtained. Finally, we developed a wrapper for accessing this process in a user-friendly Galaxy tool (https://iris. angers. inra. fr/galaxypub-cfbp). Hence, SkIf allows to extract all specific mers of a dataset. The optimal size of the mer was fixed to 22 nt to optimize the ratio of in-group to out-group specific sequences, after a comparison of a range from 18 to 26 nt was done (data not shown).
See figure on previous page. ) Fig. 6 Inter-and intrasubspecies comparisons of ANIb and shared k-mers values. a Boxplot of the ANIb values calculated from our genome dataset. b Boxplot of the shared k-mer values. c Dot plot of the ANIb and shared k-mer mean values. Linear regression and its corresponding r 2 is indicated. For intrasubpecies comparisons, the number of plotted values corresponds to [(number of genome) 2 -number of genome]. For intersubspecies comparisons, it corresponds to [(2 * number of genome subspecies A * number of genome subspecies B)]. Number of genomes: fastidiosa (13), morus (2), sandyi (3), multiplex (10), pauca (18).
Acknowledgements We thank  Muriel Bahut  ( ANAN technical facility, SFR QUASAV, Angers, FR ) for genome sequencing,  CIRM-CFBP (Beaucouzé, INRA, France ; http://www6. inra. fr/cirm_eng/ CFBP-Plant-Associated-Bacteria ) for strain preservation and supply, and the  CATI BBRIC  for the galaxy tools allowing assembling the reads and annotating the sequences, and using Blast2GO. We acknowledge  Charles Manceau  ( Anses, Angers, FR ) for his contribution while applying for funding. We thank  Matthieu Barret  for fruitful discussions and critical reading of the manuscript.
Funding ND salary was funded by the regional program "Objectif Végétal, Research,  Education and Innovation in Pays de la Loire" , project  SapAlien   2015-2017 , supported by the  French Region Pays de la Loire ,  Angers Loire Métropole  and the  European Regional Development Fund. This work received support from the  European Union 's  Horizon 2020 research and innovation program  under grant agreement  635646  POnTE (Pest Organisms Threatening Europe). The present work reflects only the authors' view and the  EU  funding agency is not responsible for any use that may be made of the information it contains.
The SkIf pipeline is free to use online (https://iris. angers. inra. fr/galaxypub-cfbp). All the sequence files related to the k-mer analyses (raw data, alignments, blast, GO enrichment tests) are submitted together with the manuscript as Additional files. Genome sequences were deposited at NCBI under the following accessions numbers: PHFQ00000000 (CFBP 7969), PHFR00000000 (CFBP 7970), PHFP00000000 (CFBP8071), PHFS00000000 (CFBP 8078), PHFT00000000 (CFBP 8082), PHFU00000000 (CFBP 8351), and PHFV00000000 (CFBP 8356). Related data and reannotation of the public genomes (fasta, genbank and gff3 formats) are available upon request.
( See figure on previous page. ) Fig. 6 Inter-and intrasubspecies comparisons of ANIb and shared k-mers values. a Boxplot of the ANIb values calculated from our genome dataset. b Boxplot of the shared k-mer values. c Dot plot of the ANIb and shared k-mer mean values. Linear regression and its corresponding r 2 is indicated. For intrasubpecies comparisons, the number of plotted values corresponds to [(number of genome) 2 -number of genome]. For intersubspecies comparisons, it corresponds to [(2 * number of genome subspecies A * number of genome subspecies B)]. Number of genomes: fastidiosa (13), morus (2), sandyi (3), multiplex (10), pauca (18).
Comamonas testosterone, Pseudomonas aeruginosa, Klebsiella pneumoniae and Bordetella petrii, but the largest fragments cover no more than 60% of the X. fastidiosa long-mer (Additional file 9). Two large regions of 32,804 bp (long-mer4538 in Additional file 6) and 16,015 bp (long-mer4596) localized onto the plasmid pXF64-HB. Together with smaller specific long-mers (long-mer4536 to 4596), they accounted for 60,224 bp over the 64,251 bp total size of this plasmid https://www. ncbi. nlm. nih. gov/nuccore/NZ_CP009886. 1). dismutase (Additional file 6). The screen (blastn) of the Nucleotide collection (nr/nt) database revealed that it shared high identities (> 90%) with sequences of Cupriavi- dus sp. , 34,148 bp (long-mer2422 in Additional file 6) appeared to be chromosomic. It contained 34 genes coding for 12 hypothetical/conserved proteins, 8 conjugal transfer pro- teins (including TraG), 3 membrane proteins, 2 methyl- transferases, and one acriflavine resistance protein B, DEAD/DEAH box helicase, DSBA oxidoreductase, hemo- lysin secretion protein D, integrating conjugative element protein pill (pfgi-1), lytic transglycosylase, multidrug transporter, RAQPRD family plasmid, and superoxide.
0055/1. 26e-5 4. 79e-5/4. 37e-8 5. 94e-4/3. 44e-6 4. 40e-36/5. 38e-39 1. 36e-4/7. 91e-7 35879, DSM 10026, CFBP 7969, CFBP 7970, CFBP 8071, CFBP 8082, CFBP 8351, EB92-1, GB514, M23, Stag's Leap, Temecula1. FAS2 (subsp. fastidiosa, 13): All the members of the group FAS, plus CFBP 8073. MUL (subsp. multiplex, 10): ATCC 35871, BB01, CFBP 8078, CFBP 8416, CFBP 8417, CFBP 8418, Dixon, Griffin-1, M12, Sy-VA. SAN (subsp. sandyi, 1): Ann-1. SAN2 (subsp. sandyi-like, 2): CO33 and CFBP 8356. PAU (subsp. pauca, 18): 32, 3124, 11,399, 6c, 9a5c, CFBP 8072, CoDiRO, COF0324, COF0407, CVC0251, CVC0256, Fb7, Hib4, J1a12, OLS0478, OLS0479, Pr8x, U24D 1Complete datasets are provided in Additional file 7 2 Top line: number of GO-associated CDSs in the list of CDSs harboring specific mers (query) / number of GO-associated CDSs in the CDSs of reference genome that do not harbor specific mers.
High Resolution Melting Analysis (HRMA) of mcyJ amplicons Amplification of the mcyJ gene was performed using the HRM primers described in the literature  (Kim et al. , 2010). PCR amplification was conducted using a 96 FLX Touch TM thermal cycler (Bio-Rad, California, USA). Two microliters of DNA extracts from each sample (ca. 50 ng DNA) were applied to the MeltDoctor HRM Master Mix (Applied Biosystems, California, USA) with a final reaction volume of 20 μL. Cycling conditions were 2 min at 50 °C, 15 min at 95 °C and 40 cycles of 15 s at 94 °C, 30 s at 60 °C and 30 s at 72 °C. To obtain the HRM melting profiles, RFU (relative fluorescence units) obtained from each sample within a melting region from 65 °C to 95 °C at 0. 02 °C/s increases, were recorded. HRM data were acquired using Bio-Rad Precision Melt Analysis (Bio-Rad, California, USA) and each sample was run in duplicate.
Acknowledgements We thank  Asociación Honoraria de Salvamentos Marítimos y Fluviales  (ADES) and  Comisión Técnico-Mixta de Salto Grande  ( CTM-Salto Grande ) for their valuable help to perform samplings. This study was supported by grant  ANII-ALGAS  ( Laboratorio Tecnológico del Uruguay ), by  PEDECIBA-Biología  and by the  Agencia Nacional de Investigación e Innovación of Uruguay.
Introduction Quality Estimation (QE) is a term used in machine translation (MT) to refer to methods that measure the quality of automatically translated text without relying on human references  (Bojar et al. , 2016 (Bojar et al. , , 2017)). In this study, we address QE for summarization. Our proposed model, SUM-QE, successfully predicts linguistic qualities of summaries that traditional evaluation metrics fail to capture  (Lin, 2004; Lin and Hovy, 2003; Papineni et al. , 2002; Nenkova and Passonneau, 2004). SUM-QE predictions can be used for system development, to inform users of the quality of automatically produced summaries and other types of generated text, and to select the best among summaries output by multiple systems. SUM-QE relies on the BERT language representation model  (Devlin et al.
, 2019). We use a pre-trained BERT model adding just a taskspecific layer, and fine-tune the entire model on the task of predicting linguistic quality scores manually assigned to summaries. The five criteria addressed are given in Figure  1. We provide a thorough evaluation on three publicly available summarization datasets from NIST shared Q1 -Grammaticality: The summary should have no datelines, system-internal formatting, capitalization errors or obviously ungrammatical sentences (e. g. , fragments, missing components) that make the text difficult to read. Q2 -Non redundancy: There should be no unnecessary repetition in the summary. Q3 -Referential Clarity: It should be easy to identify who or what the pronouns and noun phrases in the summary are referring to. Q4 -Focus: The summary should have a focus; sentences should only contain information that is related to the rest of the summary.
Q5 -Structure & Coherence: The summary should be well-structured and well-organized. The summary should not just be a heap of related information, but should build from sentence to sentence to a coherent body of information about a topic. Figure  1 : SUM-QE rates summaries with respect to five linguistic qualities  (Dang, 2006a). The datasets we use for tuning and evaluation contain human assigned scores (from 1 to 5) for each of these categories. tasks, and compare the performance of our model to a wide variety of baseline methods capturing different aspects of linguistic quality. SUM-QE achieves very high correlations with human ratings, showing the ability of BERT to model linguistic qualities that relate to both text content and form. 1.
Being recall-based, ROUGE correlates well with Pyramid but poorly with linguistic qualities of summaries. Louis and Nenkova (2013)  proposed a regression model for measuring summary quality without references. The scores of their model correlate well with Pyramid and Responsiveness, but text quality is only addressed indirectly. 2 uality Estimation is well established in MT  (Callison-Burch et al. , 2012; Bojar et al. , 2016 Bojar et al. , , 2017;; Martins et al. , 2017; Specia et al. , 2018). QE methods provide a quality indicator for translation output at run-time without relying on human references, typically needed by MT evaluation metrics  (Papineni et al. , 2002; Denkowski and Lavie, 2014). QE models for MT make use of large postedited datasets, and apply machine learning methods to predict post-editing effort scores and quality (good/bad) labels.
Datasets We use datasets from the NIST DUC-05, DUC-06 and DUC-07 shared tasks  (Dang, 2006a,b; Over et al. , 2007). Given a question and a cluster of newswire documents, the contestants were asked to generate a 250-word summary answering the question. DUC-05 contains 1,600 summaries (50 questions x 32 systems); in DUC-06, 1,750 summaries are included (50 questions x 35 systems); and DUC-07 has 1,440 summaries (45 questions x 32 systems). The submitted summaries were manually evaluated in terms of content preservation using the Pyramid score, and according to five linguistic quality criteria (Q1,. , Q5), described in Figure 1, that do not involve comparison with a model summary. Annotators assigned scores on a fivepoint scale, with 1 and 5 indicating that the summary is bad or good with respect to a specific Q.
ROUGE: This baseline is the ROUGE version that performs best on each dataset, among the versions considered by  Graham (2015). Although ROUGE focuses on surface similarities between peer and reference summaries, we would expect properties like grammaticality, referential clarity and coherence to be captured to some extent by ROUGE versions based on long n-grams or longest common subsequences. Language model (LM): For a peer summary, a reasonable estimate of Q1 (Grammaticality) is the perplexity returned by a pre-trained language model. We experiment with the pre-trained GPT-2 model  (Radford et al. , 2019) , and with the probability estimates that BERT can produce for each token when the token is treated as masked (BERT-FR-LM). 4 Given that the grammaticality of a summary can be corrupted by just a few bad tokens, we compute the perplexity by considering only the k worst (lowest LM probability) tokens of the peer summary, where k is a tuned hyper-parameter.
Experiments To evaluate our methods for a particular Q, we calculate the average of the predicted scores for the summaries of each particular contestant, and the average of the corresponding manual scores assigned to the contestant's summaries. We measure the correlation between the two (predicted vs. manual) across all contestants using Spearman's ρ, Kendall's τ and Pearson's r. We train and test the SUM-QE and BiGRU-ATT versions using a 3-fold procedure. In each fold, we train on two datasets (e. g. , DUC-05, DUC-06) and test on the third (e. g. , DUC-07). We follow the same procedure with the three BiGRU-based models. Hyper-perameters are tuned on a held out subset from the training set of each fold.
Results Table  1  shows Spearman's ρ, Kendall's τ and Pearson's r for all datasets and models. The three fine-tuned BERT versions clearly outperform all other methods. Multi-task versions seem to per-form better than single-task ones in most cases. Especially for Q4 and Q5, which are highly correlated, the multi-task BERT versions achieve the best overall results. BiGRU-ATT also benefits from multi-task learning. The correlation of SUM-QE with human judgments is high or very high  (Hinkle et al. , 2003)  for all Qs in all datasets, apart from Q2 in DUC-05 where it is only moderate. Manual scores for Q2 in DUC-05 are the highest among all Qs and years (between 4 and 5) and with the smallest standard deviation, as shown in Table  2. Differences among systems are thus small in this respect, and although SUM-QE predicts scores in this range, it struggles to put them in the correct order, as illustrated in Figure  3.