dataesr/scientific-paragraphs-categorization huggingface dataset:
SER was calculated by exact matching 6 https://github. com/OpenNMT/OpenNMT-py 7 https://github. com/tuetschek/ e2e-metrics slot values in the candidate texts, SER = S + D + I N , where S is a number of substitutions, D is a number of deletions, I is a number of insertions, and N is a total number of slots in the reference. The resulting SER is an average of SER for each prediction. While computing SER for the dialogue slot-based E2E corpus is straightforward (the binary slot familyFriendly was excluded), it results in some noise for WebNLG where subjects and objects are numerous (3,648 vs. 79 values in E2E) and where they were rephrased in references (cf. also Section 2.
Results and Discussion We compared the output of the sequence-tosequence model with attention described in Section 2. 5 on two datasets (WebNLG and E2E) and considering eight different configurations depending on how rare words are handled (without delexicalisation, with delexicalisation, with a copy-andcoverage mechanism and with both delexicalisation and a copy-and-coverage mechanism) and on how the train/dev/test partition is constructed (unconstrained vs. constrained). As pointed out in Section 2. 6, automatic scores are reported using the whole test sets whereas human evaluation is based on shared MR instances between the non-constrained and constrained test sets (40 instances for E2E and 44 for WebNLG). The results are summarised in Table 5 (E2E) and 6 (WebNLG). Some example predictions are shown in Tables  7  and 8.
For instance, for the E2E data, restau-rant names (which had not been seen in the training data) were not copied over in the constrained setting. In most cases, the input restaurant name was replaced by a restaurant name that is frequent in the training data. For example, given the MR name[Cocum], eatType[coffee shop], near[The Rice Boat], the text Near The Rice Boat there is a coffee shop called Fitzbillies was generated, where Fitzbillies, a frequently occurring restaurant name in the training data (2,371 instances), was generated instead of the input restaurant name Cocum.
Constrained vs. Unconstrained Setting There is a clear difference in terms of relative performance in the constrained vs. the unconstrained setting between the two datasets. For the E2E dataset, since the constrained dataset is defined with respect to slot values (name and near) which are delexicalised, the constrained setting is in fact similar to the unconstrained setting. And indeed the scores are similar (e. g. , unconstrained vs. constrained, D, E2E: ∆BLEU: -0. 05, ∆SER: -1. 1 and ∆right: +0. 54%). When using copying however, the results are lower in the constrained setting again suggesting that copying underperforms for items that have rarely been seen at training and development time (e. g. , unconstrained vs. constrained, C, E2E: ∆BLEU: 0. 11, ∆SER: -9. 84 and ∆right: 12. 9%). For the WebNLG data, the difference between constrained and unconstrained setting is much stronger for both delexicalisation and copying.
For instance, for copying the BLEU score in the un-constrained setting is 0. 61 vs. 0. 34 in the constrained setting. Semantic adequacy also drops noticeably (unconstrained: 83%, constrained: 41%). This is in line with  Aharoni and Goldberg (2018) 's observation that in the unconstrained setting, the model learns to memorise association between facts and entities and thereby fails to generate text that adequately captures the meaning of the input data. The low scores for the copying mechanism also confirm the observation made above that copying underperforms for rare data fragments. This difference between datasets is further discussed in the next paragraph. Semantic Adequacy As mentioned above, the manual and automatic evaluation metrics we used to assess semantic adequacy strongly correlate.
They both show that semantic adequacy is much lower for the WebNLG data (higher SER, higher proportion of wrong and missed items). This is not surprising, since the WebNLG dataset contains a much higher number of distinct values (3,648 against 79 in the E2E dataset) and exhibits a greater mismatch between input and output value names 8. That is, the delta shows that the efficiency of copying and delexicalisation varies depending on the variety and content of the dataset. The two datasets also differ with respect to the proportion of added slots which is higher for the E2E dataset and suggests an overfitting effect due to a skewed distribution in favour of inputs containing more than 3 attributes. Thus, the human evaluation shows that the majority of cases with added slots are cases where the input consists of three slots (the minimal number of attributes in E2E).
The overgeneration can be explained by the restricted number of three-slot inputs in the E2E dataset (only 2. 5% MRs out of the whole corpus). That claim is also supported by predictions produced by adversarial examples. While inputting dialogue moves consisting of 2 slots (the nonexistent number of attributes in E2E), all eight E2E models tend to overgenerate by predicting texts with 3 or 4 slot-value pairs. Fluency As mentioned in Section 2. 6, while annotating the data for semantic adequacy, we found that almost all systems outputs were wellformed English sentences. The only exception was the WebNLG model with copy mechanism where stutterings were spotted in half of the examined instances. Despite those repetitions, it was always possible to detect the subject-predicateobject structure (e.
Conclusion We investigated the impact of copying and delexicalisation on two datasets and using two different   ways of splitting the data into train, dev and test. The results show some regularities and highlight some interesting differences. Overall, the results indicate that delexicalisation outperforms copying. Furthermore, they show that copying underperforms on rare items. Since delexicalisation is a somewhat ad hoc process, an interesting direction for future research would be to devise copying methods that are more accurate and that can better handle rare data items. Another direction for future research would be to further investigate how the content and train/dev/test split of a dataset impact learning. Our results suggest two ways in which these may induce overfitting. In the WebNLG dataset, strong associations between entities and facts seem to result in generation models that memorise facts with entities rather than generate a text that adequately verbalises the input.
This is highlighted in the manual evaluation by the high number of wrong and missed data items observed both in the constrained and in the unconstrained setting. In the E2E dataset, on the other hand, we saw that added facts are frequent and manual evaluation suggests that this is due to an overfitting effect whereby, because most inputs consists of more than three slot-value pairs, the models tend to overgenerate by predicting texts that verbalise four or more slot-value pairs. In both cases, the copy-and-coverage mechanism does not suffice to ensure correct output and the results further decrease in the constrained setting. It would therefore be interesting to see to what extent better methods can be devised both for creating datasets and for devising train/dev/test splits that adequately test the ability of models to generalise.
WebNLG dataset (D: Delexicalisation, D+C: delexicalisation and copying, C: copy and coverage, NIL: Neither copy nor delexicalisation). The upper half of the table presents automatic evaluation results; the lower half-human evaluation results. Best scores are in bold. * -word repetitions present in predictions. Delexicalisation and Copying vs. Standard Encoding-Decoding A first observation is that, when neither delexicalisation nor copying is used, there is a strong drop in semantic adequacy. In the worst case, the SER increases by 25. 4 for the E2E dataset (constrained setting, NIL vs. D+C) and by 47. 05 points (constrained setting, NIL vs. D+C) in the WebNLG dataset. Similarly, the proportion of correcly predicted items (right) decreases by up to 23. 66 points for the E2E dataset (constrained setting, NIL vs.
Picked mutant clones were screened in a 96-well plate. The response to external L-arabinose concentration was measured, as explained in the 'fluorescence measurements' section, and the resulting measurement data was processed as explained in the 'data processing' section. The resulting data is presented in Supplementary Fig. 2  and 3. Fluorescence measurements. E. coli strain MK01  22  harbouring DNA constructs, were inoculated from a glycerol stock into EZ Rich Defined medium with glucose (Teknova, Hollister, CA, USA, cat. nr. M2105) and appropriate antibiotics. These cultures were grown at 37 °C until early exponential growth phase, and then diluted to a final optical density (OD) of 1 × 10  Fitness proxy. We considered a variable environment characterised by two input values x 1 and x 2 , leading to the respective cascade outputs y 1 and y 2.
Phylogenetic analyses were performed using Bayesian Inference (BI) using MrBayes 3. 2. 1. (Huelsenbeck & Ronquist, 2001). We generated 10 7 topologies, sampling every 1,000 th topology and discarding the first 10% topologies as burn-in. The stationarity of the posterior distributions for all model parameters was verified in Tracer v1. 5  (Rambaut & Drummond, 2009). From the MCMC output, we generated the final consensus treemaximum clade credibility tree using Tree Annotator v1. 6. 2 (part of the Beast software package). For visualization and edition of the consensus maximum clade credibility tree, we used the program Figtree v. 1. 3. (Rambaut, 2009). We used a Poisson tree processes (PTP) model  (Zhang et al. , 2013)  to infer the most likely number of species in our dataset, as implemented in the bPTP server (http://species.
Convergence of MCMC chain was confirmed visually. To ensure that the lineages detected using PTP presented high genetic distance (>3. 0%, sensu  (Fouquet et al. , 2007a; Fouquet et al. , 2007b)  we calculated uncorrected p-distance using the 16S mtDNA  (Vences et al. , 2005)  in the program MEGA 7. 0  (Kumar, Stecher & Tamura, 2016). To generate a dated tree in Beast 2. 0  (Drummond & Rambaut, 2007) , we selected one representative individual per species. We used a birth and death prior, GTR+I+G evolution model and calibrated the tree using normal distribution following the divergence time estimates of  Fouquet et al. (2012a) : crown age of Hyloidea (mean = 77. 0 ± 10 Ma); basal divergence time of Bufonidae (mean = 67. 9 ± 12 Ma); divergence of Atelopus + Oreophrynella vs. other Bufonidae (mean = 60.
0 ± 11 Ma); Nannophryne vs. other Bufonidae (mean = 47. 0 ± 8 Ma); Rhaebo vs. other crown Bufonidae (mean = 40. 8 ± 7 Ma) and Dendrophryniscus vs. other crown Bufonidae (mean = 52. 1 ± 9). We generated 10 7 topologies, sampling every 1,000 th topology and discarding the first 10% topologies as burn-in. Stationarity of the posterior distributions for all model parameters was verified in Tracer v1. 5  (Rambaut & Drummond, 2009). From the MCMC output, we generated the final consensus maximum clade credibility tree using Tree Annotator v1. 6. 2 (part of Beast software package). For visualization and edition of the consensus tree, we used the program Figtree v. 1. 5 (Rambaut, 2009).
Environmental analyses The environmental analyses were undertaken in order to test if delimited species occur in distinct climatic environments  (Soberón & Peterson, 2005). We retrieved high resolution bioclimatic layers (30 arc-seconds ∼1 km, present environmental conditions) using the Community Climate System Model (CCSM4) from the WorldClim project (http://www. worldclim. org/) (Hijmans et al. , 2005). To avoid geographic pseudoocurrence of points, localities were filtered using the program Geographic Distance Matrix Generator 1. 2. 3. (Ersts, 2014)  considering a threshold of 1 km between localities. The localities of each lineage used for analyses are in Appendix S3, Table  S3A. To identify environmental variables that were most informative and test the classification of specimens into mtDNA lineages using ecological variables, we performed Principal Component Analysis (PCA) and Discriminant Function Analysis (DFA) separately for each lineages/species of the eastern and western clades.
In the eastern clade specimens of each lineage can be successfully separated based on morphometric traits using PCA (Fig. 3A ). The first two principal components extracted by the PCA account for 57. 7% of the variation found in the dataset. The first component (PC1) explained 37. 48% of the total variation and the second component (PC2) explained 20. 29% of the variation. Using DFA a total of 80% of specimens were correctly classified to phylogenetic groups. The number of individuals correctly assigned to each clade by DFA are presented in Table  5. The DFA showed that the variables that contributed the most to the morphometric separation were snout length, tarsal length, and head width. Head measurement traits (head width, head length, snout length, and intranasal distance) explained 93% of the classification by the first two discriminant axes (Appendix S8, Figs.
6 ± 0. 3 3. 7 ± 0. 3 4. 6 ± 0. 4 4. 6 ± 0. 5 3. 9 ± 0. 4 3. 9 ± 0. 3 4. 9 ± 0. 2 5. 0 ± 0. by the PCA account for 52. 37% of the variation found in the dataset. The first component (PC1) explained 33. 2% of the total variation and the second component (PC2) explained 19. 17% of the variation. Using the DFA a total of 68% of specimens were correctly assigned to phylogenetic groups. The number of individuals correctly assigned to each clade by DFA are presented in Table  5. The DFA showed that the variables that most contributed to the morphometric separation were eye diameter, hand length, head width and foot length. Head traits (head length, eye diameter and intranasal distance) and hand traits (hand length) were the variables that explained 78% of the classification by the first two discriminant axes (Appendix S8, Figs.
In the eastern clade specimens of each lineages can be successfully separated based on environmental traits using PCA (Fig. 3C ). The first two principal components extracted by the PCA account for 87. 71% of the variation found in the dataset. The first component (PC1) explained 73. 28% of the total variation and the second component (PC2) explained 14. 43% of the variation. A total of 65% of specimens were correctly classified to their lineage. The numbers of individuals correctly assigned to each clade by DFA are presented in Table  6. The environmental variables that most contributed to separating lineages were mean temperature of the coldest quarter (bio11), maximum temperature of the warmest month (bio5), mean diurnal temperature range (bio2) and isothermality (bio3) (Appendix S8, Figs.
A. minuta (70%) A. minuta sp1 (0%) A. matses 7 0 0 0 0 0 0 A. javierbustamantei 0 6 2 0 0 0 0 A. moisesii sp. nov. 0 0 5 0 0 0 0 A. amazonicola 1 0 0 6 0 0 0 A. siona sp. nov. 0 0 0 0 8 2 1 A. minuta 0 0 1 1 7 0 A. aff. minuta sp1 0 0 0 0 1 1 0 In the western clade specimens of each lineages can be successfully separated based on environmental traits using PCA (Fig. 3D ). The first two principal components extracted by the PCA account for 95. 55% of the variation found in the dataset. The first component (PC1) explained 95. 37% of the total variation and the second component (PC2) explained 0. 18% of the variation. A total of 81% of specimens were correctly assigned to their candidate species. The numbers of individuals correctly assigned to each clade by DFA are presented in Table  6. The environmental variables that most contributed to group separation were annual mean temperature (bio1), mean diurnal temperature range (bio2), mean temperature of the warmest quarter (bio10) and mean temperature of the wettest quarter (bio8) (Appendix S8 and Figs.
, 2013; Caminer & Ron, 2014; Fouquet et al. , 2014; Gehara et al. , 2014; Lourenço et al. , 2015). These discoveries imply that public data deposited in, for example GenBank, Gbif or IUCN, are often flawed and that the numerous metaanalyses  (Godinho & Da Silva, 2018)  based on such data may be imprecise or even inaccurate. As a consequence of not recognizing true taxonomic diversity of anurans, macroecological studies will fail to recognize actual patterns of geographic structuring, and ultimately will not contribute to our understanding of the evolutionary and ecological processes that lead to and are maintaining this diversity. Our results suggest that the genus harbors more than twice as many species as current estimates. In the last several years the systematics and taxonomy of the genus Amazophrynella has begun to be elucidated  (Ávila et al.
, 2012). The use of additional nuclear markers is generally recommended the use of these unliked markers has the potential to improve the accuracy of phylogenetic reconstructions and species delimitation. In spite of having used only mtDNA loci, our study also provides an extensive new morphological dataset, bioacoustic data and accurate collecting locality information which allowed us to associate environmental data with each specimen. All these additional data support and reinforce the inference based on the mitochondrial genes. Our phylogenetic analysis also reveals a striking biogeographic pattern with a basal eastern and western divergence followed by a northern and southern split within both eastern/western clades (Fig. 2 ). Our basal east-west pattern dated to the Miocene and match similar patterns and divergence times detected in other groups of frogs  (Symula, Schulte & Summers, 2003; Noonan & Wray, 2006; Funk et al.
ACKNOWLEDGEMENTS We would like to thank  Ariane Silva , from the herpetological collection of the  Instituto Nacional de Pesquisas da Amazônia (INPA) in Manaus, Brazil , to  Fernando Ayala  from the  Pontificia Universidad Católica de Ecuador (PUC(E) in Quito-Ecuador , and to  O Aguilar  and  R Orellana  ( MUBI ) for providing administrative support and part of the material for this study. RR Rojas thanks  Alexander Almeida ,  Ian Pool Medina ,  Richard Naranjito Curto  for field support and  Mario Nunes  for laboratory assistance.
Funding This work was supported by  Coordenação de Aperfeiçoamento de Pessoal de Nivel Superior (CAPES) , by the  Program Incentivo para la Publicación Efectiva de Artículos Científicos en Revistas Indexadas 2016 -1 • corte , ( RDE 036 ,  20. 04. 2016 ) del  Consejo Nacional de Ciencia, Tecnología e Innovación Tecnológica de Perú (CONCYTEC -FONDECYT (Cienciactiva) ), by  Concelho Nacional de Pesquisa (CNPq/SISBIOTA-BioPHAM  grant no. CNPq 563348/2010 ), and an  Investissement d'Avenir grant  managed by  Agence Nationale de la Recherche  (CEBA, ref. ANR-10-LABX-25-01 ),  France. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
, 2015; Jang et al. , 2017; Martin et al. , 2005; Yang et al. , 2010) , transcriptomic analysis was also performed on infected macrophages treated with LPS. We focused our analysis on genes that were modulated with more than 3-fold change and had a signal threshold above 5 Reads Per Kilobase of transcript per Million mapped reads (RPKM) in at least one sample when comparing the wild-type and Dgra18 mutant strains. Filtered data are presented in Supplementary file 3 (data are accessible through NCBI GEO, accession number GSE103113). Thirty-eight genes were significantly and differentially regulated with most corresponding to genes up-regulated upon macrophage infection by wild-type parasites, but not with the Dgra18 strain as revealed by hierarchical clustering (Figure  5A ). Complementation of Dgra18 mutation with GRA18-HF under the control of the GRA1 promoter (Dgra18, GRA18 +++ ) restored the expression pattern to levels observed with wild-type parasites or even higher, in strong support of the GRA18-dependent induction of those genes.
To confirm the transcriptomic profile, we focused on the chemokines Ccl17, Ccl22, and Ccl24 previously identified as selectively up-regulated genes upon T. gondii infection  (Hammoudi et al. , 2015; Melo et al. , 2013). Quantitative RT-PCR analysis reproducibly demonstrated a pronounced decrease in Ccl17, Ccl22, and Ccl24 expression from cells infected with Dgra18 mutant parasites when compared to wild-type Pru or Pru Dgra18, GRA18 +++ parasites (Figure  5E ). Similar data were obtained with the 76K strain, another type II cystogenic strain, highlighting that T. gondii reproducibly regulated those chemokines in a GRA18-dependent manner. GRA18 clearly plays a major role in regulating CCL17 and CCL22 synthesis and secretion since its loss significantly lowered protein levels in the supernatant of mouse macrophages (Figure  5F ).
7 macrophages when compared to mock -that is, mCherry -transfected cells (Figure  6D  and E) whereas Ccl17 and Ccl24 were no longer induced in the b-catenin lacking ones. The slight reduction of Ccl22 expression in the GRA18-expressing mutant RAW264. 7 macrophages (Figure  6E ) suggests a different mode of regulation for the three chemokines. Consistent with b-catenin upregulation, similar data were obtained with Ifnb1 (Figure  6E ), a known b-catenin target gene in macrophages  (Ma et al. , 2014; Rathinam et al. , 2010; Yang et al. , 2010). To ascertain the b-catenin requirement for GRA18 activity under more physiological conditions, chemokine expression was monitored upon infection of RAW264. 7 cells disrupted or not for Ctnnb1 with wild-type and Dgra18 mutant parasites. While b-catenin was required for full induction of the Ccl17 chemokine, Ccl22 expression remained unaffected in Ctnnb1 -/-mutant host cells (Figure  6F ), again consistent with a different mode of regulation orchestrating Ccl17 and Ccl22 expression.
Parasite and cell culture T. gondii tachyzoites were maintained by serial passage on human foreskin fibroblast (HFF) monolayers. The strains used in this study were RH ku80, Pru ku80, and 76K-GFP-LUC (gift of M. Grigg, National Institutes of Health, Bethesda, MD). HFF primary cells, T-Rex-293 (RRID:CVCL_D585), L929 (Sigma-Aldrich Cat# 85011425), J774 (J774A. 1, Sigma-Aldrich Cat# 91051511), and RAW264. 7 (ATCC Cat# TIB-71, RRID:CVCL_0493) cell lines were cultured in Dulbecco's modified Eagle's medium (DMEM, Invitrogen) supplemented with 10% heat-inactivated FBS (Invitrogen), 10 mM Hepes buffer, pH 7. 2, 2 mM L-glutamine, and 50 mg/ml penicillin and streptomycin (Thermo Fisher Scientific). Cells were incubated at 37 ˚C in 5% CO 2. Stable transgenic parasites or recombinants were selected with 25 mg/ml mycophenolic acid and 50 mg/ml xanthine or 1 mM pyrimethamine.
Yeast two-hybrid screen Full-length GRA18 II (aa 27 to 648) cloned in pB27 (N-LexA-bait-C fusion) was used in a ULTImate Y2H screen against a human Human Placenta_RP5 complementary DNA Gal4-activating domainfusion library (Hybrigenics, Paris, France). The construct was checked by sequencing. Prey fragments of positive clones were amplified by PCR and sequenced at their 5 0 and 3 0 junctions and the resulting sequences were used to identify corresponding interacting proteins in the GenBank database via a fully automated procedure. A confidence score (Global Predicted Biological Score, Global PBS) was attributed to each interaction as previously described  (Formstecher et al. , 2005). The interaction domain mapping was performed using Full-length GRA18 II (aa 27 to 648) and GRA18 fragments (aa 26 to 233, aa 233 to 440, aa 420 to 648, aa 152 to 350, and aa 350 to 585) that were PCR-amplified and cloned in frame with the LexA DNA binding domain (DBD) by Gaprepair into plasmid pB27, derived from the original pBTM116  (Vojtek and Hollenberg, 1995).