dataesr/scientific-paragraphs-categorization huggingface dataset:
Neural Embeddings (Skip-Gram). Our main results in Section 6 are produced using neural embeddings. These types of vectors have proven useful for a wide variety of tasks in recent literature, and in our early experiments with tensor, learning provided the best results on the RELPRON development data and other composition tasks. For our methods, in addition to the word vectors, we required holistic vectors to use as targets for training the verb and relative pronoun matrices and tensors. The holistic vectors were also obtained using neural embeddings. Word vectors were built using a re-implementation of skip-gram with negative sampling  (Mikolov et al. 2013 ) on a lemmatized 2015 download of Wikpedia. We used a window of ten words on either side of the target, with ten negative samples per word occurrence, and 100-dimensional target and context vectors.
V S-→ s + V O-→ o ex. eat S -→ cat + eat O----→ mouse ( 18 ) Verb matrices were learned using L 2 ridge regression  (Hoerl and Kennard 1970) , also known as L 2 -regularized regression. For a given verb V, the training data for V S consisted of (subject vector, holistic subject-verb vector) pairs, and the training data for V O consisted of (object vector, holistic verb-object vector) pairs. In addition, each pair was weighted by the logarithm of its number of occurrences in the corpus, which we found improved performance on the RELPRON development set. Regression was implemented in Python using standard NumPy operations, and we optimized the regularization parameter on the RELPRON development set, using MAP scores and the SPLF composition method described in Section 5. 5. 3. A single regularization parameter was used globally for all verbs and set to 75 after tuning.
The regularization parameter was optimized on the RELPRON development set using MAP scores and set to 80 for both tensors. The power of the tensor comes from the fact that it captures the interaction between the head noun and the composed verb-argument phrase, and hence models the relative pronoun in a meaningful way that composition methods such as vector addition, which omits the pronoun entirely, do not. However, capturing these interactions leads to a large number of parameters, and the sparsity of training data is therefore a challenge in training the full tensor model. Our corpus of 72M sentences contained only around 800K occurrences of relative clauses, compared with over 80M verb-subject and verbobject occurrences (4M types); and this was heavily unbalanced across subject and object relative clauses, with 771K subject relative clause occurrences (222K types) compared with 21K object relative clause occurrences (only 8K types).
The composition of the relative clause vector is shown in Equations (  27 )-(  28 ), for subject and object relative clauses, respectively. R S n -→ n + R S va (V O -→ a ) ex. that S n ----→ device + that S va (detect O ----→ planet) (subject) (27) R O n -→ n + R O va (V S -→ a ) ex. that O n ----→ device + that O va (play S ------→ shepherd) (object) (28) The relative pronoun matrices (R S n , R S va ) and (R O n , R O va ) are learned by weighted L 2 ridge regression, where the training data consists of (noun vector, holistic relative clause vector) pairs for (R S n , R O n ), and (holistic verb-argument vector, holistic relative clause vector) pairs for (R S va , R O va ). The regularization parameter was optimized on the RELPRON development set and set to 75 for all four matrices. 5. 5.
Error Analysis In this section we examine in detail the performance of some of the main composition methods on the RELPRON development data. We are interested in four main points. First, we investigate whether the grammatical function (subject or object) of the relative clause plays a role in the MAP score. We find that most of the methods are relatively well-balanced across grammatical functions, but FPLF shows a lower MAP on object properties, where the amount of training data is substantially lower. Second, we investigate whether the different head nouns in the data set have any effect on MAP score. We find that the scores are relatively well-balanced across head nouns, but that more concrete terms and their properties may be easier to model. However, based on qualitative observations, a more important factor seems to be term polysemy.
Accuracy by Grammatical Function An obvious first step in understanding the RELPRON results is to break them down by grammatical function. Because subjects and objects are asymmetric in their interaction with the verb-in particular, verbs show stronger selectional preferences for objects than for subjects  (Marantz 1984; Kratzer 1996 Kratzer , 2002)) -we might expect that relative clauses with different extraction sites are not equally easy to compose. In addition, for the methods that learn an explicit representation of the relative pronoun (FPLF in this analysis), the substantial difference in the amount of training data might affect the result. The MAP scores for each grammatical function are shown in Table  7. The first row shows the subject results, in which only the subject properties from the development data are ranked by similarity for each term.
Common Errors and Evaluation with Properties as Queries In addition to poor performance on polysemous terms (Section 7. 2), we observed two common sources of error on RELPRON. The first is lexical overlap between terms and properties, a type of confounder that we deliberately included in the data set (see Section 3. 5). For example, the top two ranked properties for batter for all three methods are player that walks batter (pitcher) and player that strikes batter (pitcher); and the top ranked property for balance for all three is document that has balance (account). FPLF, despite its more sophisticated modeling of the relative pronoun, suffers from lexical overlap about as much as SPLF and Addition. However, we note that the RPTensor method is able to overcome this problem to a small extent, with the lexically confounding properties ranked a few positions below the top.
We used Mean Reciprocal Rank (MRR) as the evaluation measure, because there is only one correct term per property. MRR is given by the formula: MRR = 1 M M p=1 1 rank(p) ( 35 ) where M is the number of properties in the data set and rank(p) is the rank at which the correct term t is found for property p. The reasoning behind the MRR evaluation is that even if a property is applicable to more than one term, there is always a single term which is most similar to it. The results are shown in Table  11. For all three methods, the correct term appears on average at approximately rank 2. The MRR evaluation is also another use of RELPRON that can be exploited in future work.
Conclusions RELPRON was designed to test compositional distributional semantic methods on a phrase type of intermediate size and complexity: more complex than two-word combinations, and involving a function word, but less complex than full sentences. The data set is challenging, but results are promising with current techniques; a MAP score of 0. 5 corresponds to every other property being ranked correctly, given a term. Again, in line with a growing body of literature, we observe that vector addition performs extremely well, but we are able to match its performance with a more complex method based on the Practical Lexical Function model. Based on the qualitative analysis, our contention is that, in order to improve substantially on the results presented in this article, methods more linguistically sophisticated than vector addition will be required.
We base this contention on a few factors. First, vector addition has little room for improvement, with potential gains limited to those that come from improving the overall quality of the vectors. It seems unlikely that vector addition can achieve a MAP much higher than its current one of around 0. 5. We also note that the relative clauses used in this article are well below the tenword length at which the quality of composed representations with vector addition has been suggested to degrade, so future data sets focusing on more complex linguistic constructions may require alternative methods. For the more complex methods, on the other hand, there are likely to be readily achievable gains on RELPRON from increasing or changing the training data. This article represents the first large-scale implementation of methods that learn explicit categorial representations for relative pronouns, but the training data in the form of holistic vectors for relative clauses was still observed to be relatively small compared with the number of parameters learned by the methods, especially for object relative clauses.
Larger corpora or different sources of data-for example, including dictionary definitions or paraphrase data sets in the training data-may improve the models. Beyond the issue of training data, the qualitative analysis showed that the various methods were sometimes unable to integrate the semantic contributions of the head noun and the remainder of the relative clause. Vector addition will not be able to capture this relationship any better than it already does, but other learning methods (e. g. , involving non-linear models) may be able to do so. It is also possible that a learning method designed more specifically for the RELPRON ranking task would perform better, especially because many of the training examples in our existing data are non-restrictive relative clauses-although this would require an alternative data source, and RELPRON itself is too small to provide training data.
Acknowledgments We are grateful to  Mehrnoosh Sadrzadeh  for many detailed and constructive discussions on the topic of relative clause composition, and comments on earlier drafts of this article. We would like to thank  Bob Coecke ,  Dimitri Kartsaklis ,  Daoud Clarke ,  Jeremy Reffin ,  Julie Weeds ,  David Weir , and the anonymous reviewers for their detailed and helpful comments. Laura Rimell  and  Stephen Clark  were supported by  EPSRC  grant  EP/I037512/1. Jean Maillard  is supported by an  EPSRC   Doctoral Training Grant  and a  St. John's Scholarship. Laura Rimell ,  Tamara Polajnar , and  Stephen Clark  are supported by  ERC   Starting Grant DisCoTex  ( 306920 ).
Koi herpesvirus disease (KHVD) is an emerging disease that causes mass mortality in koi and common carp, Cyprinus carpio L. Its causative agent is Cyprinid herpesvirus 3 (CyHV-3), also known as koi herpesvirus (KHV). Although data on the pathogenesis of this deadly virus is relatively abundant in the literature, still little is known about its genomic diversity and about the molecular mechanisms that lead to such a high virulence. In this context, we developed a new strategy for sequencing full-length CyHV-3 genomes directly from infected fish tissues. Total genomic DNA extracted from carp gill tissue was specifically enriched with CyHV-3 sequences through hybridization to a set of nearly 2 million overlapping probes designed to cover the entire genome length, using KHV-J sequence (GenBank accession number AP008984) as reference.
CyHV-3 genome enrichment and sequencing About 2 µg of total DNA were used to establish the genomic libraries of each sample, using the TruSeq DNA sample preparation kit (Illumina). DNA was sonicated using a Bioruptor device (Diagenode). The average insert size used for sequencing was ∼300 bp. After adapter ligation using the Truseq kit, libraries were subjected to a viral-specific sequence capture using the SeqCap EZ library SR technology (Roche-Nimblegen). A large set of 50-105 mer CyHV-3 specific probes was designed (Roche-Nimblegen) to theoretically capture all DNA sharing more than 90% identity with the published CyHV-3 genome of strain J (GenBank #AP008984). This probe set (Roche Diagnostics design OID 41307, IRN 4000012980; Roche) is available upon demand. A pre-capture ligation-mediated PCR (LM-PCR) consisting of 8 cycles of PCR was first performed according to the manufacturer's instructions (Illumina).
Genome assembly/mapping and detection of variations Raw sequences (fastq files) were stored in the public Sequence Read Archive (SRA) repository under the SRP study accession SRP059764 as well as in the NG6 repository  (Mariette et al. , 2012)  of GenoToul (Toulouse, France) for analysis purposes. A first quality control consisted in the visual inspection of the different graphs produced by fastqc (http://www. bioinformatics. babraham. ac. uk/projects/fastqc/), as well as in the alignment of the reads against classical contaminants such as Escherichia coli, yeast and phages. All the raw reads which passed this quality control were aligned to the CyHV-3 J strain genome (accession number AP008984) using BWA MEM (version bwa-0. 7. 5a)  (Li & Durbin, 2009). Low quality alignments, i. e. , reads aligning more than once or with poor sequencing quality inducing alignment quality lower than 30, were filtered out.
, 1997)  against the NCBI nucleotide collection (nr/nt) database. Figures representing the sequencing depth and the number of small nucleotide variants were created with the R packages ggplot2 and limma, respectively. A de novo assembly of the reads was concomitantly performed for each sample using ABySS, and Cap3 programs. Briefly, fastq files were trimmed with Fastq trimmer (version 1. 0. 0) (Blankenberg et al. , 2010)  using the default parameters and the resulting reads were assembled in paired-end by ABySS v1. 5. 0  (Simpson et al. , 2009). The obtained contigs were subjected to a new assembly using Cap3  (Huang & Madan, 1999) , and resulting assemblies were mapped on a CyHV-3 reference genome (strain U, accession number DQ657948) using Bowtie 2 (version 2. 2. 1) program  (Langmead et al.
Genome comparisons Read alignments were also subjected to a genetic distance analysis. Variant calling files of individual specimens were compared through an identity-by-state calculation using PLINK, with the distance option  (Chang et al. , 2015). Reads were artificially created for the three reference genomes U, I and J in order to include them in the analyses. For this purpose, 100-base reads were artificially produced every five bases along the reference genome with a simple python script. These reads were then randomly shuffled to form a set of reads with an even 20× depth, using samtools faidx (version 0. 1. 19). Then a hierarchical cluster analysis was computed with a complete-linkage clustering method, using the standard parameters, and dendrograms were plotted with the hclust R package.
A visual representation of the distribution of nucleotide variants (SNPs + indels) was also created with Circos software  (Krzywinski et al. , 2009) , using the genome of CyHV-3 J strain as reference. For clarity, only the genomes of PoB3_1, J1_101110 and I09_2i3 were represented. The seven consensus sequences were aligned together with the three already sequenced genomes (J, U and I strains) using the online Multiple alignment program for amino acid or nucleotide sequences (Mafft program version 7, http://mafft. cbrc. jp/alignment/server/). The resulting alignments were visualized and manually corrected with GeneDoc version 2. 7  (Nicholas, Nicholas & Deerfield, 1997). Then the evolutionary history was inferred by the Neighbor-Joining method  (Saitou & Nei, 1987) , and evolutionary distances were computed with MEGA6  (Tamura et al.
Nucleotide sequence accession numbers The raw sequences (fastq files) of the whole project are available under the SRP study accession SRP059764 (http://trace. ncbi. nlm. nih. gov/Traces/sra/?study=SRP059764). It is organized in 7 biosamples and 10 experiments, as shown in Table  1. Sequences corresponding to the 7 consensus genomes have also been submitted to GenBank and can be accessed under the following accession numbers: KX544842, KX544843, KX544844, KX544845, KX544846, KX544847 and KX544848 (for PoB3, J2_101110, CB4_181110, J1_101110, I_09_2i3, I_10_3 and PP3_070411, respectively).
Sequence depth along this reference genome was variable both across and within specimens. It was >2,000× on average for all specimens with an estimated number of viral copies >10 5 , and >200× for all other specimens (Table  2  and Fig. S1 ). For most samples, there was an over representation of reads at positions 226,373-226,831 and 228,015-228,163 (according to KHV-J coordinates, GenBank # AP008984). A Blast search revealed that these regions shared 97% and 94% identity with sequences of Cyprinus carpio genome (GenBank # LN591727 and LN591823, respectively). However, this over representation was attenuated in samples with high initial viral loads (J1_101110, CB4_181110 and PP3_070411). A higher coverage (×30) was also noticed for J2_101110 specimen in the region corresponding to positions 45,500-47,700; yet this portion of CyHV-3 genome did not return any significant blast hit.
In addition, the high measurement frequency of this original device makes it possible to calculate the rates of total production of fermentative aromas  [4, 33]. Biomass was also measured (off-line), allowing for the calculation of specific rates of production. Such rates represent metabolic fluxes and are essential for understanding yeast metabolism. With this dataset, we performed PCA using three parameters (maximum specific rate of total production: SRmax, sugar consumption at the maximal specific rate of production: PointSRmax and total production of each volatile compound: Max) to obtain an overview of the effect of fermentation conditions and strains on aroma synthesis (Fig. 3 ). The first two PCA axes accounted for 78. 4 % of the total variation. The dispersion of fermentation conditions was greater for the fermentations performed with 330 mg/L of assimilable nitrogen and for the Affinity ™ ECA5 strain.
5b ) and isobutanol/ isobutyl acetate (data not shown) by Lalvin EC1118 ®. The corresponding bioconversion yields remained constant despite the additions. By contrast, the addition of phytosterols dramatically lowered the bioconversion of higher  alcohols for Affinity ™ ECA5. Indeed, after each lipid addition, the yield was divided by two (Fig. 5b ). To better understand the underlying mechanisms, we performed a transcriptomic analysis of the two strains. Cells were sampled at 35 and 70 g/L of CO 2 released (arrows in Fig. 5a ). For Affinity ™ ECA5, the first sample was collected before the change in bioconversion yield of isoamyl alcohol to its acetate ester and the second after this shift. A sparse PLS-DA analysis was performed with the normalized transcriptomic data and the production of all fermentative aromas.
Fermentation conditions Fermentations were performed in 300 ml fermenters with musts containing three levels of assimilable nitrogen (70, 200 and 330 mgN/L) and three levels of phytosterols (2, 5 and 8 mg/L) at three temperatures  (20, 24 and 28 °C). The data presented in the box plot (Fig. 1 ) were obtained in a previous study on Lalvin EC1118 ®  [29]. The same Box-Behnken design  [29]  was used to study the response of Affinity ™ ECA5 to changes in fermentation conditions. The concentrations of volatile compounds in the liquid phase were measured by GC-MS using the method described in  [29]. Fermentations were run in 10 L stainless steel tanks at 24 °C. The amount of CO 2 released was measured accurately and automatically with a gas mass flow meter to calculate the rate of CO 2 production (dCO 2 /dt).
Data processing and statistical analyses Statistical analysis was performed with R software, version 3. 1. 1 [52]. We obtained three datasets, in which each variable of interest is a curve along the time (h) that we expressed in terms of consumed sugar (g/L). We chose to summarize these three datasets by modeling each curve with an adequate model and then extracting criteria of interest. First, for each condition, the biomass was modeled using a Weibull model with the drc package  [53]. The one four-parameter Weibull model is written as follows: This four-parameter ascending function is asymmetric with an inflection point at time e. For each modeled function, we extracted several criteria of interest: µmax, defined as the maximal of the ratio f '(t)/f(t) for each t, expressed in h -1 ; the inflection point, expressed in terms of consumed sugar g/L; and the maximum biomass, expressed in 10 6 cells.
For these two parametric models, the normality of residual distributions and homogeneity of variance were studied with standard diagnostic graphs; no violation of the assumptions was detected. Each volatile compound under each condition was then modeled using a non-parametric model using the cell-Growth package  [54]. The model used is a local regression and allows for the extraction of the inflection point expressed in consumed sugar (g/L), the maximal production in mg/L and the maximal rate (maximum of the first derivative in mg/L. h). To calculate the specific rate, we divided the first derivative of the model (the rate) by the population, as estimated above. Finally, we recorded the maximum specific rate (SRmax) and the time at which this maximum was reached, expressed in consumed sugar (g/L) (PointSRmax).
To provide an overview of the dataset, principal component analysis (PCA) was carried out with the Facto-MineR package  [56]. Multivariate factorial analysis (MFA) was then performed for the two strains (Lalvin EC1118 ® and Affinity ™ ECA5) at two levels of nitrogen (70 and 330 g/L) and two levels of phytosterols (2 and 8 mg/L). This analysis allowed for the study of links between the consumption of AA and volatile compound production  [55].
Fluorescent cRNAs were synthesized from 100 ng of total RNA using the One color RNA Spike-In kit (Agilent Technologies). Labeled cRNA was purified with the RNeasy Kit (Qiagen). Microarrays were hybridized for 17 h at 65 °C in a rotating hybridization oven (Corning) with the Gene Expression Hybridization kit (Agilent). The hybridization signal was detected with a GenePix 4000B laser Scanner (Axon Instruments). The limma package  [56]  was used to import and normalize the global microarray data (quantile method for normalization between arrays). The entire dataset is available in the "Gene Expression Omnibus Database" (No. GSE68354). Transcriptomic data were analyzed by two different methods. For each level of CO 2 released (35 and 70 g/L) and based on this normalized dataset of 6200 expression data for the two strains, we used sparse partial least squaresdiscriminant analysis (sPLS-DA), which is an exploratory approach in a supervised context, to select the most important transcripts relative to the four samples  [57].
We tuned the number of dimensions of the sPLS-DA to two and the number of variables to choose on these two dimensions to 500 (250 for each). Functional analysis was performed on the selected transcripts by time point to highlight significant functional groups according to the gene ontology (GO) process terms using the Genecodis program  [58]  via the FDR method at a p value cutoff of 0. 05  [59]. For each time point, MFA was then performed to obtain an overview of the dataset, which consisted of 513 variables measured for the two strains (Lalvin EC1118 ® and Affinity ™ ECA5) and for the two sampling times. The dataset included a set of individuals described by two types of variables: the normalized expression of the 500 transcripts selected by the sPLA-DA according to the two strains and the 13 compounds (or ratios) produced during fermentation by the two strains.
The MFA took the structure of the two groups of data into account and balanced the effect of each group of variables, enabling the study of links between expression data and volatile compounds production  [55]. To determinate the differential gene expression between experimental conditions, a modified t-test was performed by filtering on confidence at p < 0. 05, using the Benjamini and Hochberg false discovery rate as multiple testing corrections of the t-test p values  [59]. The genes with different levels of expression were grouped according to gene ontology (GO) process terms using the Genecodis program  [58].