dataesr/scientific-paragraphs-categorization huggingface dataset:
Additionally, this could generate uncertainty in resource allocation at the hospital, which represents a limitation of our study. An additional limitation is that the beginning and end of treatment dates were not recorded prospectively; this would guarantee a greater accuracy of information. The strategy we used to approximate the dates was to record the date of the first consultation in the period observed (2010 to 2014). We consider that this provides an accurate estimate of the period of treatment. A potential limitation is the magnitude of bias due to the underestimate of the quantity of supplies and services that were actually used by each patient who received CC treatment. The extent to which the data in the medical records accurately describes the resources consumed by a given patient is unknown.
Statistical methods Data were processed and analyzed using Microsoft Excel 2013 and GraphPad Prism version 7 (GraphPad Software, La Jolla, California USA). Only apparent prevalence were calculated for each species. Binary logistic regression analyses were carried out to determine which variables (Locality, sex and Species) were significantly associated with detection of HEV antibodies. Multivariate logistic regression was performed using R software version 2. 13. 0 (https://www. r-project. org/). P < 0. 05 was considered significant. The lower and upper limits of the 95% confidence interval (CI) for a proportion were also calculated.
In addition, HEV in milk seems not to be inactivated by pasteurization  (Huang et al. , 2016). However, recent data suggest in cattle a potential antigenic cross-reaction of the antibodies with anti-HEV responsiveness with a related, but as yet unknown agent  (Yugo et al. , 2019). We thus must be very cautious about the possible role of cattle in HEV transmission until new methods like deep sequencing of transcriptome or proteome of these "HEV positive" cattle may allow to identify the real antigen identity. In this study, a prevalence of 12. 0% (9/75) was recorded in sheep. This was in accordance with prevalence from similar studies in Nigeria and China  (Geng et al. , 2010a,b,c; Junaid et al. , 2014). Higher prevalence has been recorded with 29. 0% to 35. 2% in China  (Wu, Kang, Bai, & Bai, 2010; Wu, Si, Jiang, Li, & Jin, 2015) , and 31.
Bioinformatic Analysis Raw reads were pre-processed following the pipeline published in  Ilias et al. (2015). Initially, we used Scythe [https://github. com/vsbuffalo/scythe] to identify adapter contamination setting prior contamination rate set in 0. 1 "-p 0. 1, " Sickle to trim low quality reads (parameters "pe -g -t sanger -q 20l 45"), Trimmomatic  (Bolger et al. , 2014)  (parameters "PE -phred33 ILLUMINACLIP:adapter_file. fa:2:30:10 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:25 MINLEN:45 CROP:99") and Prinseq  (Schmieder and Edwards, 2011)  to remove poly A/T tails and low complexity reads. The two new assemblies were produced using Trinity v2. 0. 6  (Grabherr et al. , 2011)  with default parameters (default kmer 25, min length 200 nucleotides). Downstream assembly filtering was conducted using the same criteria as in  Manousaki et al.
(2014)  (IsoPct < 1and FPKM < 0. 3). To assess the quality of all five assemblies we used three criteria: (1) assessing the percentage of mapped reads of each species back to the assembly using Bowtie2  (Langmead and Salzberg, 2012)  and Samtools  (Li et al. , 2009) ; (2) examining the similarity between the Trinity assembly and Swissprot database (UniProt Consortium, 2015) using blastx  (Altschul et al. , 1990)  searches within the parallel NOBLAST program  (Lagnel et al. , 2009) ; and (3) quantifying the completeness of each assembly using BUSCO (search for "vertebrates" in OrthoDBv8)  (Simão et al. , 2015). The newly obtained assemblies were then functionally characterized as described in  Tsakogiannis et al. (2018). Differential expression (DE) analysis was performed at gene level by pairwise comparisons between male and female for brain and gonad samples in each species separately.
The pre-processed reads of each species were aligned to the respective assembly with Bowtie2  (Langmead and Salzberg, 2012)  and abundance was estimated with RSEM  (Li and Dewey, 2011) , as implemented in the Trinity script align_and_estimate_abundance. pl. The estimated expected counts for each sample, when the sum of counts in each row >15, were extracted  (Bourgon et al. , 2010). To accomplish the same consensus in differential expression (DE) analysis among all species, we re-analyzed sharpsnout seabream to bring into line with the other four species. Count data were imported in R (version 3. 2. 0) and the analysis of differential expression conducted in DESeq2 (version: 1. 8. 1)  (Love et al. , 2014) , an R bioconductor package  (Gentleman et al. , 2004). Each species brain and gonadal samples were grouped according to sex and expression was compared (male vs.
female), following the developers' manual (false discovery rate, FDR, threshold of 0. 05). Genes with an adjusted p-value <0. 05 were considered significantly differentially expressed. Principal component analysis (PCA)  (Jolliffe, 2002)  and the heatmap. 2 function in the gplots package  (Warnes et al. , 2009)  were used to visualize global similarities and differences among the gonadal samples. Each species PCA plot was produced including all DE genes (Differentially Expressed genes) in the filtered (sum of counts in each row >15) count matrix. Functional-enrichment analysis was performed to identify gene ontology (GO) terms and metabolic pathways significantly enriched in DE genes in gonads. Here, the term 'enrichment' stands for the comparison between the DE genes of the female (test-set) against the male (reference-set) and the gene-set was restricted to only those with |log 2 FC| >2.
Following an in-house built pipeline, using perl scripts and the Vennt tool (http:// drpowell. github. io/vennt/) we combined the expression data of D. dentex, S. aurata, P. pagrus, P. erythrinus, and D. puntazzo gonadal orthologous genes. We performed a BLASTn between each pair of the five transcriptomes (e-value threshold 10 -10 ), keeping the best reciprocal hits (bidirectional best blast hits), an approach followed also in  Tsakogiannis et al. (2018). We then selected all transcripts that have orthologs in all five species. Next, we combined the expression information with the existing orthologous genes. Following that, we visualized and analyzed the data using the freely available Vennt tool  (version v0. 8. 4, GPL v. 3 , http://drpowell. github. io/vennt), which generates dynamic Venn diagrams for differential gene expression.
Based on the Vennt results, we concluded that an FDR value lower than 0. 05 produces robust results. Finally, we plotted a Venn diagram for all five species using the R package "venn" (version 1. 5, GPL v. 3, https://cran. r-project. org/web/ packages/venn/index. html). A more detailed description of parameters used in bioinformatic analysis can be found in Supplementary Excel Table  SET1.
Pre-processing Treatment of Reads and Assembly Construction Illumina sequencing of common dentex and gilthead seabream yielded 512,882,392 and 440,315,096 reads, respectively. The raw sequence data has been submitted to the National Center for Biotechnology Information (NCBI) Sequence Read Archive (SRA) database (common dentex: BioProject ID PRJNA481721 and gilthead seabream: BioProject ID PRJNA415944). Reads' pre-processing led to 272,911,232 and 368,540,942 high-quality paired reads for common dentex and gilthead seabream, respectively. High-quality reads were used to construct the initial transcriptome assemblies, consisting of 247,693 transcripts for common dentex (N50: 1,891 nucleotides, mean length: 870 nucleotides) and 292,100 for gilthead seabream (N50: 1,939 nucleotides, mean length: 872 nucleotides).
Following transcript quality control, the initial assemblies were limited to 129,012 contigs (N50: 2,592; mean length: 1,259 nucleotides) corresponding to 98,747 genes for common dentex and 119,677 contigs (N50: 2,806; mean length: 1,321 nucleotides) corresponding to 92,160 genes for gilthead seabream. The restricted final datasets combined with the respective datasets of the previously published sparids are shown in a comparative representation in Figure  1. To evaluate the accuracy of the assembled transcripts, clean reads were aligned against the transcripts, reaching relatively high percentages in both common dentex (85%) and gilthead seabream (80%). Then, a sequence similarity search between each species and UniProtKB/SwissProt database disclosed 19,429 FIGURE 1 | Pipeline for building assemblies and differential expression profiles in brains and gonads of the two species sequenced here in addition to sharpsnout seabream  (Manousaki et al.
Transcriptome Annotation A blastx search of both assembled transcripts against UNIPROT (UniProt Consortium, 2015; e -value cut-off <10 -5 ) returned 26,509 and 24,326 genes with at least one hit for common dentex and gilthead seabream, respectively, and 24,248 and 23,961 genes with a protein domain match against InterProScan  (Quevillon et al. , 2005) , respectively. To identify possible functions, Gene Ontology (GO) assignments were used to classify the sequences. A total of 21,705 and 23,169 common dentex and gilthead seabream genes, respectively, were attributed with at least one GO term. The potential enzymes were characterized based on the chemical reaction they catalyze using the prediction of Enzyme Commission (EC) numbers for each sequence. In the case of common dentex 4,616 genes were classified to 1,037 different EC numbers, while in the case of gilthead seabream, 6,050 genes were classified to 1,215 different EC numbers (Table  2 ).
Expression Profile of Reproduction-Related and Key-Candidate Genes in the Gonads In order to identify genes that play an important role in reproduction we searched the annotated brain and gonad datasets for relevant GO categories and checked if they were differentially expressed. More particularly, DE genes related with broad biological processes terms, such as "reproduction" and "reproductive process" were searched in gonads of both sparids. A large number of "reproduction"related and few "reproductive process"-related genes were found in the gonads of both species. In common dentex, 133 female-biased and 77 male-biased genes belonging to the GO term "reproduction," and four female-biased and 14 male-biased genes belonging to "reproductive process" were detected. In gilthead seabream, 191 female-biased and 166 male-biased genes belonging to the GO term "reproduction," and 19 female-biased and 20 male-biased genes belonging to "reproductive process" were detected (Supplementary Excel Tables  SET4-12 ).
In an effort to identify the molecular machinery employed by each species to express the functional male or the female phenotype of the gonad, we aimed at analyzing and comparing the expression patterns of orthologous genes known for their involvement in the fish and the vertebrate sex determination/differentiation cascade. We selected 114 candidate genes in total. Combining data from the two newly sequenced species and the three sparids analyzed in  Manousaki et al. (2014)  and  Tsakogiannis et al. (2018) , we obtained a comparative view of expression of those genes in all five species (Table  3 ). Out of this list, 56 genes were present in the transcriptomes of all five species. Table  3  includes the 69 mostly discussed genes in other fish or vertebrates that were differentially expressed in at least one of the five species, and compares their expression pattern between them.
Comparative Analysis of Sex-Biased Gene Expression in the Gonads of All Five Sparid Species Apart from the 114 genes that were identified based on literature and public datasets, we looked for sex-biased differences using a comparative whole-transcriptome approach, based on the orthology assignment of the annotated transcriptomes of the gonads of all the species (Figure  4 ) (homologous genes between the five studied species in Supplementary Excel Table  SET13 ). We compared the expression data of the annotated gonadal orthologous genes for the five species. Those multi-group comparisons helped us to identify common (in two, three, four, and all five species) and unique (species-specific) male-and female-biased genes in gonads. To gain greater insight into the genes that consistently play a role in gonadal identity we focused on genes that were differentially expressed in all five species (common/shared genes).
Ctnnb1 showed a consistent pattern with a clear female-biased expression in these sparids thus proposing a conserved and acute role of Ctnnb1 in preserving gonad's female identity in this fish lineage. Similarly, the role of retinoic acid (RA) signaling pathway in gonadal differentiation has been well-stated  (Yamaguchi and Kitano, 2012; Rodríguez-Marí et al. , 2013; Feng et al. , 2015). The implication of this pathway in fish sex determination remains largely unknown, and data from different species show that the enzymes responsible for RA catabolism may vary  (Feng et al. , 2015). In the current study, two pathway members, Cyp26a and Stra6 showed a female-biased expression in all studied sparids. On the other hand, a search for genes encoding RA receptors revealed two retinoic acid receptors (Raraa and Rarb) that were male-biased in all studied sparids.
ACKNOWLEDGMENTS We would like to thank  Prof. Michalis Pavlidis  for his very helpful support and advice. We wish to thank  Mrs. Margaret Eleftheriou  for proof-reading and English editing of the manuscript. We would also like to thank  Haris Apostolidis ,  Constantinos Vatikiotis , and  Irini Sigelaki  for their valuable help in sampling. Financial support for this study has been provided by the  Ministry of Education and Religious Affairs , under the Call ARISTEIA I of the National Strategic Reference Framework 2007-2013 (SPARCOMP, # 36 ), co-funded by the  EU  and the  Hellenic Republic  through the  European Social Fund. Last, the authors express gratitude to the reviewers for their thorough reading of our manuscript and their intuitive comments and suggestions.
AVAILABILITY OF SUPPORTING DATA The datasets generated and/or analyzed during the current study are available from the corresponding author on request. The raw sequence data has been submitted to the National Center for Biotechnology Information (NCBI) Sequence Read Archive (SRA) database (common dentex: BioProject ID PRJNA481721, and gilthead seabream: BioProject ID PRJNA415944). The transcriptomic sequences can be retrieved under the following accession numbers: SRR7536480-SRR7536499, SRR7536500-SRR7536502 for common dentex and SRR6223527-SRR6223532, SRR6223535-SRR6223542 for gilthead seabream.
Usage similarity estimation addresses the semantic proximity of word instances in different contexts. We apply contextualized (ELMo and BERT) word and sentence embeddings to this task, and propose supervised models that leverage these representations for prediction. Our models are further assisted by lexical substitute annotations automatically assigned to word instances by context2vec, a neural model that relies on a bidirectional LSTM. We perform an extensive comparison of existing word and sentence representations on benchmark datasets addressing both graded and binary similarity. The best performing models outperform previous methods in both settings.
, 2009). Usim differs from a classical Semantic Textual Similarity task  (Agirre et al. , 2016)  by the focus on a particular word in the sentence. We evaluate on this task word and context representations obtained using pre-trained uncontextualized word grass clippings can be brought out to the landfill at anytime for no *charge* and may not be placed in city cans. the tag consists of a tiny chip , [about the [size of a match head that serves ] as a ] portable database. this is at least 26 weeks by the [week in [which the approved match with the child ] is made ]. grass clippings can be brought out to the [landfill at [anytime for no charge and may not ] be placed in city cans ] Figure  1 : We use contextualized word representations built from the whole sentence or smaller windows around the target word for usage similarity estimation, combined with automatic substitute annotations.
embeddings (GloVe)  (Pennington et al. , 2014) , with and without dimensionality reduction (SIF)  (Arora et al. , 2017) ; context representations obtained from a bidirectional LSTM (context2vec)  (Melamud et al. , 2016) ; contextualized word embeddings derived from a LSTM bidirectional language model (ELMo)  (Peters et al. , 2018)  and generated by a Transformer (BERT)  (Devlin et al. , 2018) ; doc2vec  (Le and Mikolov, 2014)  and Universal Sentence Encoder representations  (Cer et al. , 2018). All these embedding-based methods provide direct assessments of usage similarity. The best representations are used as features in supervised models for Usim prediction, trained on similarity judgments. We combine direct Usim assessments, made by the embedding-based methods, with a substitutebased Usim approach.
Building up on previous work that used manually selected in-context substitutes as a proxy for Usim  (Erk et al. , 2013; Mc-Carthy et al. , 2016) , we propose to automatize the annotation collection step in order to scale up the method and make it operational on unrestricted text. We exploit annotations assigned to words in context by the context2vec lexical substitution model, which relies on word and context representations learned by a bidirectional LSTM from a large corpus  (Melamud et al. , 2016). The main contributions of this paper can be summarized as follows: • we provide a direct comparison of a wide range of word and sentence representation methods on the Usage Similarity (Usim) task and show that current contextualized representations can successfully predict Usim; • we propose to automatize, and scale up, previous substitute-based Usim prediction methods; • we propose supervised models for Usim prediction which integrate embedding and lexical substitution features; • we propose a methodology for collecting new training data for supervised Usim prediction from datasets annotated for related tasks.
We test our models on benchmark datasets containing gold graded and binary word Usim judgments  (Erk et al. , 2013; Pilehvar and Camacho-Collados, 2019). From the compared embeddingbased approaches, the BERT model gives best results on both types of data, providing a straightforward way for word usage similarity calculation. Our supervised model performs on par with BERT on the graded and binary Usim tasks, when using embedding-based representations and clean lexical substitutes.
Our method relies on substitute annotations proposed by the context2vec model  (Melamud et al. , 2016) , which uses word and context representations learned by a bidirectional LSTM from a large corpus (UkWac)  Baroni et al. (2009). Table  1 : Example pairs of highly similar and dissimilar usages from the Usim dataset  (Erk et al. , 2013)  for the nouns paper (Usim score = 4. 34) and coach. n (Usim score = 1. 5), with the substitutes assigned by the annotators (GOLD). For comparison, we give the substitutes selected for these instances by the automatic substitution method (context2vec) used in our experiments from two different pools of substitutes (AUTO-LSCNC and PPDB). More details on the automatic substitution configurations are given in Section 4. 2.
The LexSub and Usim Datasets We use the training and test datasets of the SemEval-2007 Lexical Substitution (LexSub) task  (McCarthy and Navigli, 2007) , which contain instances of target words in sentential context handlabelled with meaning-preserving substitutes. A subset of the LexSub data (10 instances x 56 lemmas) has additionally been annotated with graded pairwise Usim judgments  (Erk et al. , 2013). Each sentence pair received a rating (on a scale of 1-5) by multiple annotators, and the average judgment for each pair was retained. McCarthy et al. (2016)  derive two additional scores from Usim annotations that denote how easy it is to partition a lemma's usages into sets describing distinct senses: Uiaa, the inter-annotator agreement for a given lemma, taken as the average pairwise Spearman's ρ correlation between ranked judgments of the annotators; and Umid, the proportion of midrange judgments over all instances for a lemma and all annotators.
In our experiments, we use 2,466 sentence pairs from the Usim data for training, development and testing of different automatic Usim prediction methods. Our models rely on substitutes automatically assigned to words in context using context2vec  (Melamud et al. , 2016) , and on various word and sentence embedding representations. We also train a model using the gold substi-tutes, to test how well our models perform when substitute quality is high. Performance of the different models is evaluated by measuring how well they approximate the Usim scores assigned by annotators. Table  1  shows examples of sentence pairs from the Usim dataset  (Erk et al. , 2013)  with the GOLD substitutes and Usim scores assigned by the annotators. The Usim score is high for similar instances, and decreases for instances that describe different meanings.
The Concepts in Context Corpus Given the small size of the Usim dataset, we extract additional training data for our models from the Concepts in Context (CoInCo) corpus  (Kremer et al. , 2014) , a subset of the MASC corpus  (Ide et al. , 2008). CoInCo contains manually selected substitutes for all content words in a sentence, but provides no usage similarity scores that could be used for training. We construct our supplementary training data as follows: we gather all instances of a target word in the corpus with at least four substitutes, and keep pairs with (1) no overlap in substitutes, and (  2 ) minimum 75% substitute over-lap. 2 We view the first set of pairs as examples of completely different usages of a word (DIFF), and the second set as examples of identical usages (SAME). The two sets are unbalanced in terms of number of instance pairs  (19,060 vs.
2,556). We balance them by keeping in DIFF the 2,556 pairs with the highest number of substitutes. We also annotate the data with substitutes using context2vec  (Melamud et al. , 2016) , as described in Section 4. 2. We apply an additional filtering to the sentence pairs extracted from CoInCo, discarding instances of words that are not in the con-text2vec vocabulary and have no embeddings. We are left with 2,513 pairs in each class (5,026 in total). We use 80% of these pairs (4,020) together with the Usim data to train our supervised Usim models described in Section 4. 3. 3.
The Word-in-Context dataset The third dataset we use in our experiments is the recently released Word-in-Context (WiC) dataset (Pilehvar and Camacho-Collados, 2019), version 0. 1. WiC provides pairs of contextualized target word instances describing the same or different meaning, framing in-context sense identification as a binary classification task. For example, a sentence pair for the noun stream is: ['Stream of consciousness' -'Two streams of development run through American history']. A system is expected to be able to identify that stream does not have the same meaning in the two sentences. WiC sentences were extracted from example usages in WordNet  (Fellbaum, 1998) , VerbNet  (Schuler, 2006) , and Wiktionary. Instance pairs were automatically labeled as positive (T) or negative (F) (corresponding to the same/different sense) using information in the lexicographic resources, such as presence in the same or different synsets.