dataesr/scientific-paragraphs-categorization huggingface dataset:
In HGPRTs and PNPs, the short-lived oxocarbenium ions are protected from the solvent by the enzymes, and it has been proposed that they are stabilized by the Ser-33 in human PNP and by the 5ЈOH of PRPP in human HGPRT  (11, 22). In hAPRT, we identified Tyr-105 as a potential stabilizer of the reaction. Also, in the ADE-PRPP-Mg 2ϩ -hAPRT structure, the distance between the 4Ј and 5Ј oxygen atoms is 2. 9 Å with a geometry similar to that observed in ImmGP-human HGPRT structure (PDB ID: 1BZY). However, the involvement of 5ЈOH in stabilization of an oxocarbenium ion is probably prevented by electrostatic interaction between the 5ЈOH atom and the C8 hydrogen, as predicted in QM/MM calculations (Fig. 5D ). In addition, the reaction coordinate separation (the distance from the N9 to the attacking oxygen atom of PP i in the reverse reaction and vice versa) is 4.
6 Å in the ADE/PRPP complex and 4. 5 Å in the AMP/PP i complex. This is between 0. 6 to 1Å shorter than the values for HGPRT (5. 2 Å) (PDB ID: 1TC2) and for PNP (5. 5 Å) in the Michaelis complex. These shorter distances are in favor of an S N 2-type mechanism, even though the atomic excursion of the anomeric carbon (C1Ј) of the ribosyl moiety is around 2 Å from substrate to product, similar to the value for HGPRT (2. 1 Å) and greater than the value for PNP (1. 7 Å)  (23). Together, these observations demonstrate that a pure S N 1-type reaction is not possible for hAPRT and, instead, an S N 2 mechanism is mainly involved. After the binding of PRPP, the hood region and the flexible loop are preferentially in an open conformation. The interaction with an adenine molecule will select for the closed conformation of the flexible loop in which the conserved Glu-104 and Tyr-105 interact with both substrates through hydrogen bonds and van der Waals contacts.
5C ). Instead, in all PRPPbound hAPRT WT structures (PDB IDs: 6FCH, 6FCI, 6FD4, 6HGQ), including the Hx-PRPP-Mg 2ϩ -hAPRT WT structure, the magnesium ion has an expected geometry (ϳ2. 25 Å coordinating distance and ϳ90°angle between atoms) except for the values of the angle between O1Ј-Mg 2ϩ -O 2 Ј and O 2 Ј-Mg 2ϩ -O3Ј (Table  S1 ). A value of ϳ70°was observed for these angles because of the geometry of the sugar moiety, which leads to constraints on the chelating system. However, in the ADE-PRPP-Mg 2ϩ -hAPRT Y105F structures (PDB IDs: 6FD5 and 6FD6)  (6) , which show the transformed products AMP and PP i , the Mg 2ϩ -O1Ј (from PP i ) and Mg 2ϩ -O 2 Ј (from AMP) coordinating bonds have distances of 1. 8 and 2. 7 Å, respectively (Fig. S4 ). These values are ϳ0. 4 Å from the ideal value. This leads to a 0.
The ribose seems to have rotated around the C4Ј atom, culminating to a maximum displacement of 0. 7 Å for the O 2 Ј atom (Fig. S6 ). This AMP movement would be because of the presence of the PP i -Mg 2ϩ complex in the active site, which would force the O 2 Ј and O3Ј atoms of the ribose moiety to coordinate the magnesium ion. The N6 atom of AMP would then carry along the Val-25 carbonyl, which would lead to the displacement of the hood region, as observed in ADE-PRPP-Mg 2ϩ -hAPRT Y105F structure (PDB ID: 6FD5) (Fig. S6 ). Finally, we noticed that the closest pyrophosphate oxygen atom was only 3. 0 Å away from the C1Ј atom in the ADE-PRPP-Mg 2ϩ -hAPRT Y105F structures. This would be a favorable geometry for the attack of the C1Ј atom by the pyrophosphate group. Therefore, we propose that disruption of the conventional magnesium geometry is used in a template effect chemical reaction, to initiate the reverse enzymatic reaction  (26, 27).
Data collection, processing, and structure refinement Four diffraction data were obtained at SOLEIL (beamline PX1) and ESRF (beamlines ID14 -4 and ID29) (Table  2 ). They were integrated with iMOSFLM  (35)  or XDS  (36)  and scaled with AIMLESS  (37). The structures were solved by molecular replacement with the program PHASER using phases from the PDB ID 6FCH structure  (6). Complex structures were then refined with REFMAC5 (38) using the CCP4i software suite  (37)  and the building program Coot  (39).
QM/MM calculations A computational model of the ADE-PRPP-hAPRT complex was prepared based on our X-ray structure (PDB ID: 6FCI)  (6). The model of the Hx-PRPP-hAPRT complex was prepared from the model of the ADE-PRPP-hAPRT complex by mutating ADE into Hx. A model very similar to the X-ray structure of our Hx-PRPP-hAPRT complex was obtained. This was performed to place both systems initially in the same minima of the PRPP-hAPRT potential energy surface. Hence, the differences observed between the two calculated reaction paths are mainly associated with the chemical reactivity of ADE and Hx. Missing hydrogen atoms were added and optimized using the Protein Preparation Wizard in Maestro  (25). For both models, the QM region included the purine base (ADE or HX), the co-substrate PRPP, the magnesium ion, and five water molecules and six side chains in close contact with these two substrates (namely, Arg-67, Lys-88, Glu-104, Tyr-105, Asp-127, Asp-128).
We introduce a set of nine challenge tasks that test for the understanding of function words. These tasks are created by structurally mutating sentences from existing datasets to target the comprehension of specific types of function words (e. g. , prepositions, wh-words). Using these probing tasks, we explore the effects of various pretraining objectives for sentence encoders (e. g. , language modeling, CCG supertagging and natural language inference (NLI)) on the learned representations. Our results show that pretraining on language modeling performs the best on average across our probing tasks, supporting its widespread use for pretraining state-of-the-art NLP models, and CCG supertagging and NLI pretraining perform comparably. Overall, no pretraining objective dominates across the board, and our function word probing tasks highlight several intuitive differences between pretraining objectives, e.
D Today there are more than 300,000. → Today there are less than 300,000. X Table  1 : Examples of sentences and sentence pairs corresponding to each of our probing datasets. The highlighted words are those that are relevant to the phenomena targeted by each set. the output representations. We isolate the effects of different pretraining objectives by holding the model architecture constant. • We study function words, which have been under-studied in previous works on representation learning, but are critical to language understanding. • We release nine new datasets, 1 qualitycontrolled by both linguists and non-linguist annotators, to facilitate ongoing work and follow-up analysis. 2 Function Word Probing Tasks.
Approach We introduce nine new probing tasks aimed at evaluating models' understanding of function words. We focus on function words because although they are key building blocks of compositional meaning and are highly frequent, they have received relatively little attention in the probing 1 The datasets are released as part of the Diverse Natural Language Inference Collection (DNC,  Poliak et al. , 2018b) , available at http://decomp. io. literature and in the distributional semantics literature. Each task targets the understanding of a specific type of function word; illustrative examples are given in Table  1. Our expectation is that different pretraining objectives (see Section 3. 2) will yield sentence representations which measurably differ in their performance on these probing tasks.
We use two different formats for our probing tasks: acceptability judgment and natural language inference (NLI). The former uses a binary classification approach (acceptable/unacceptable) for probing a single sentence vector, in line with works such as  Conneau et al. (2018)  and  Adi et al. (2017). The latter uses an entailment-based approach similar to  White et al. (2017)  and  Poliak et al. (2018b) , which is a ternary classification task (entailment, contradiction, neutral) over sentence pairs. The format is selected based on the suitability to the particular function word type in question. To generate our probing datasets, we make structural modifications to sentences drawn from existing corpora, targeting a particular type of function word. We heuristically apply modifications which we believe are likely to produce a specific label, and then recruit human annotators in order to produce the final labels used in our evaluations.
Our dataset consists of sentences that contain one of the six wh-words listed above. Half of these sentences are mutated versions of the original which are generated by replacing the original wh-word with a different wh-word randomly selected from the remaining five options. Definite-Indefinite Articles The definiteness task probes the understanding of definiteness that arises by the use of the definite article (the) versus indefinite articles (a and an). We find sentences containing multiple occurrences of the or multiple occurrences of a, and, for half of them, swap all such occurrences (i. e. , replacing the with a 2  or vice-versa). This gives us four types of sentences: unchanged sentences with multiple definite articles, unchanged sentences with multiple indefinite articles, sentences with all definite articles replaced by the indefinite article, and sentences with all indefinite articles replaced by the definite article.
End-of-Sentence The end-of-sentence (EOS) task tests a model's ability to identify semantically coherent chunks (i. e. , sentences) in running text. In written text this is often indicated by punctuation marks such as periods, but humans are able to easily identify sentences even without overt markers. Thus, we take pairs of sentences from the same paragraph of the WikiText-103  (Merity et al. , 2017)  test set and remove all punctuation marks and capitalization, and concatenate each sentence pair to create a line of running text. 3 Half of the dataset consists of a pair of valid sentences, and the other half consists of a pair of potentially invalid sentences generated from an incorrect segmentation of the running text, where the incorrect segmentation index is obtained by sampling from a Gaussian distribution centered around the correct index (σ = 2) and rounding to the nearest integer.
NLI-Based Tasks Our NLI-based probing tasks ask whether the choice of function word affects the inferences licensed by a sentence. These tasks consist of a pair of sentences-a premise p and a hypothesis hand ask whether or not p entails h. We exploit the label changes induced by a targeted mutation of the sentence pairs taken from the Multi-genre Natural Language Inference dataset (MNLI,  Williams et al. , 2018). The rationale is that, if a change to a single function word in the premise changes the entailment label, that function word must play a significant role in the semantics of the sentence. Prepositions We manually curate a list of prepositions (see Appendix D) that are likely to be swapped with each other without affecting the grammaticality of the sentence. We generate mutated NLI pairs by finding occurrences of the prepositions in our list and randomly replacing them with other prepositions in the list.
Spatial Expressions The spatial expressions task probes the understanding of words that denote spatial relations between entities. Changing the spatial configuration often leads to different inferences; for instance, A is to the left of B implies that B is to the right of A, but not that A is to the right of B. We select a set of words that describe spatial configurations which are not necessarily prepositions (e. g. , left, right, close, far). Again, we find MNLI pairs containing these words and negate to generate mutated pairs. Negation This task probes whether models are able to understand negations, in particular explicit negation using the word not, lexical negation using antonyms, and the interaction between them. We first identify premise-hypothesis pairs from the MNLI dataset that contain antonym pairs (e.
Annotation We recruit human annotators on Amazon Mechanical Turk to produce the final labels for the heuristically-generated datasets described above. We collect three labels per sentence (or per pair of sentences for EOS and NLI probing sets). We use the majority label in our final dataset, and discard examples on which there is no majority consensus. For more details about our annotation protocol, including compensation, refer to Appendix C. Acceptability Tasks Human annotators are presented with a single (mutated or unmutated) sentence and are given the options {natural, unnatural, neither}. We discard sentences in which the majority label does not agree with our expected label. That is, we only include mutated sentences with a majority label of unnatural and unmutated sentences with a majority label of natural.
We collect around 500 annotated examples with balanced label ratio for each probing set. We release our sentences in small batches until we have approximately 250 unnatural examples per task. To create the final dataset, we pool all answers from all batches and take a subset of the natural sentences so that the label ratio is balanced, prioritizing examples with perfect inter-annotator agreement. Natural Language Inference Tasks For the NLI tasks, we collect common-sense entailment judgments from annotators on a 5-point Likert scale on which 1 denotes 'definitely contradiction' and 5 denotes 'definitely entailment', following  Zhang et al. (2017). This finer-grained scale is intended to avoid confounds arising from borderline cases. Except for the use of scaled judgments, our instructions follow the MNLI guidelines.
Pretraining Tasks Our main experiments compare seven pretraining tasks which we believe capture different aspects of linguistic meaning and which yield reasonable performance when used on a benchmark task such as MNLI. 7 For our purposes, a task is a datasettraining objective pair. We attempt to select a set of tasks diverse enough to highlight performance differences due to pretraining objectives. We additionally report results using BERT  (Devlin et al. , 2019) (base, uncased)  to demonstrate that our probing sets prove challenging even for state-of-the-art models.
Language Modeling We train a left-to-right word-level language model on BWB, which was successfully used by  Peters et al. (2018)  for pretraining sentence encoders. Because language modeling is trivial for a bidirectional LSTM, we follow  Peters et al. (2018)  by training separate forward and backward two-layer 1024d language models and concatenate their hidden states as token representations. Skip-Thought Drawing from  Kiros et al. (2015)  and  Tang et al. (2017) , we train a sequence-tosequence model on skip-thought, which is a task of generating the next sentence in the discourse given the previous sentence. We use the learned encoder as our sentence encoder. Since this objective requires running text, we use sentences from WikiText-103 as training data. CCG Supertagging We train a model to predict the Combinatory Categorial Grammar (CCG) supertag for each word, with sentences from CCG-Bank  (Hockenmaier and Steedman, 2007).
Discourse (DisSent) We train a model on Dis-Sent  (Jernite et al. , 2017; Nie et al. , 2017) , which is an unsupervised task of predicting the discourse marker (e. g. , and, because, or so) that connects two clauses. We train our model on a dataset created from WikiText-103 following Nie et al. (2017)'s protocol, which involves extracting pairs of clauses with a specific dependency relation. Natural Language Inference Inspired by  Conneau et al. (2017) , we use the MNLI dataset for NLI pretraining. The task is to predict the entailment label for premise-hypothesis pairs; the possible labels are entailment, contradiction, neutral.
Classifiers for Probing Tasks To probe the sentence encoders pretrained on the different objectives, we freeze the weights of the encoder after pretraining and train an additional model using the outputs of the fixed encoder as inputs. We describe the implementation details for the NLI and acceptability probing sets below. NLI Tasks For NLI-type probing, we train an NLI model on top of the representations produced by the pretrained sentence encoder that uses an attention mechanism inspired by  Seo et al. (2017)  that computes attention between all pairs of words in the two sentences (described in more detail in Section 3. 1). We train this component on MNLI and evaluate directly on our NLI probing datasets with no further dataset-specific training. Acceptability Classification Tasks For all acceptability tasks except the EOS task, we take the sequence of hidden state outputs from the pretrained encoder as the sentence representation.
Overall Performance Figure  1  shows the performances of models trained on each pretraining task on our probing datasets. We also provide comparison with a randomly initialized encoder with no pretraining, which is known to be a strong baseline  (Bowman et al. , 2018). We observe that different pretraining tasks have different strengths and weaknesses; there is no single pretraining task that achieves the best (or worst) performance across the board. This implies that even the best encoders, such as BERT, are unable to capture function word semantics fully, and suggests further research into combining advantages of different tasks. Furthermore, most models are far from human performance, with only a few exceptions (e. g. , BERT on conjunctions). This demonstrates that our probing datasets serve as useful challenge sets, in addition to permitting fine-grained analysis.
However, we note that CCG, our most syntactic task, achieves competitive results with the smallest number of training examples (an order of magnitude less data than NLI and two orders less than language modeling), suggesting that incorporating syntactic knowledge might yield more efficient learning of function word semantics. We furthermore see that many of our probing sets are challenging even for BERT-although BERT substantially improves performance on many probing sets, and obtains superhuman performance on conjunctions and EOS, 11  it also shows clear weaknesses in several probing sets (e. g. , wh-words and definite/indefinite articles) where it is outperformed even by a randomly initialized baseline with no pretraining.
We hypothesize that this is due to the duality of preposition semantics; most prepositions have both concrete and abstract senses, and the image model is biased to focus on the former. To illustrate, consider the preposition below, which can denote a spatial configuration (e. g. , the boots end below the knee) or an abstract rela-tion (numeric or qualitative comparison; e. g. , her score is below sixty). In the preposition dataset, below occurs 17 times, 11 of which are spatial and 6 abstract. For the spatial usage, both MNLI and image-caption models have 64% accuracy (7/11). The NLI model shows 50% accuracy for pairs containing abstract uses (3/6), but the imagecaptioning model answers none of them correctly (0/6). Here is an example of a numeric usage of below that the NLI model answered correctly but the image model answered incorrectly: P: Only those whose incomes do not exceed 125 percent of the federal poverty level qualify.
Data Size and Genre Effects As can be seen from the varying sizes of the pretraining dataset reported in Figure  1 , seeing more data during pretraining does not imply better performance on probing tasks. Also, as noted before, the fact that pretraining can hurt performance suggests that if the task is not the "right" task, adding more datapoints during pretraining can lead models to learn counterproductive representations. Another potential confound is vocabulary overlap between pretraining and probing task datasets. Since all pretraining task datasets have different sets of vocabulary, the variance in the results could be attributed to the number of words in the probing set that were already seen in pretraining. To investigate this possibility, we compute the ratio of overlapping words between the pretraining and probing datasets.
Samples were considered positive at a cycle threshold (Ct) value of < 40 and characteristic amplification curves. Analysis of genetic diversity of Anaplasma ovis gltA and msp4 primers design and qualitative PCR gltA and msp4 primers were designed in the present study using sequences available on GenBank. Sequences were aligned using MAFFT and conserved regions identified  [32]. Primers targeting the conserved regions of the genes were then designed by Primer-BLAST  [33]. Primer properties were further analysed using OligoAnalyzer  [34]. Selected primers were then synthetized (Eurofins Scientific Laboratories, Paris, France) (Table  1 ). Amplifications were achieved with Phusion High-Fidelity DNA Polymerase (Thermo Fisher Scientific, Waltham, MA, USA) and a Doppio thermocycler (VWR, Radnor, PA, USA).
Sequence alignment and phylogenetic analysis gltA homologous sequences were retrieved from GenBank to represent different species of Anaplasmataceae, genera Anaplasma, Ehrlichia and Neorickettsia, as follows: Anaplasma ovis (GenBank: KJ410285), A. marginale (AF304139), A. centrale (AF304141), A. capra (KM206274), A. platys (AB058782), A. phagocytophilum (AY464138), A. bovis (KU586317), Ehrlichia ruminantium (DO513397), E. ewingii (DQ365879), E. canis (AY647155), E. chaffeensis (AF304142), E. muris (AF304144), Neorickettsia sennetsu (AF304148) and N. risticii (AF304147). For msp4, homologous sequences of different species belonging to the genus Anaplasma were retrieved from GenBank as follows: A. ovis (KU497712), A. marginale (KU497715), A. centrale (AF428090), A. capra (KM206277) and A. phagocytophilum (EU008082).
Sequences were translated into proteins using the ExPASy translate tool  [35]. For strain analysis, msp4 nucleotide and MSP4 protein sequences belonging to A. ovis strains from all over the world were selected from GenBank as shown in Additional file 1: Figure  S1  and Additional file 2: Figure  S2 , respectively. For each gene, sequences were aligned with MAFFT (v. 7) configured for the highest accuracy using the scoring matrix 200PAM/kD2, alignment strategy MAFFT-FFT-NS-I, gap opening penalty 1. 53 and offset value 0. 123. Non-aligned regions were removed using Molecular Evolutionary Genetics Analysis (MEGA) v. 6 software  [36]. The best-fit model of sequence evolution was selected based on the corrected Akaike information criterion (cAIC) and Bayesian information criterion (BIC) implemented in MEGA.
Anaplasma ovis strain diversity analysis using MSP4 sequences Following theoretical translation of all msp4 nucleotide sequences into proteins, six different sequences were identified and named strains Corsica 1-6 (hereafter COR1-6, GenBank: MH121148-MH121152). Protein sequence alignment revealed amino acid substitutions and indels that explained the genetic diversity among these strains (Fig. 5a ). The strain COR1 was the most frequently found and therefore it was considered as the reference strain in further comparisons. COR2 differed from the reference sequence in a substitution from cysteine (C) to arginine  (R). Compared to COR1, strains COR3 and COR4 also had one amino acid substitution from alanine (A) to valine (V) and from isoleucine (I) to phenylalanine (P), respectively. Remarkably, strain COR5 had one deletion and one insertion of a glycine (G) compared to COR1-4 and COR6 and, in addition, a substitution from leucine (L) to proline (P) (Fig.
Comparison between Anaplasma ovis strains in Corsica and other regions Among the six A. ovis strains identified in Corsica, the msp4 nucleotide and MSP4 protein sequences of COR1 had 100% identity with strains previously reported in Cyprus (GenBank gene/protein accession numbers FJ460454/ACJ74223), Mongolia (GenBank gene/protein accession numbers LC141078/BAU79725) and other countries. None of the other strains (COR2, COR3, COR4, COR5 and COR6) were 100% identical to any previously reported A. ovis strains. To study the evolutionary history of the A. ovis strains of Corsica, phylogenetic trees were constructed using msp4 nucleotide and MSP4 protein sequences. Sequences that were 100% identical between them were excluded from the analysis. Except for COR2 and COR6 that were closely related between them (Fig.