dataesr/scientific-paragraphs-categorization huggingface dataset:
We then present the QATS shared task and the associated dataset, which we use for our experiments. Finally we compare all methods in a reference-less setting and analyze the results. 2 Existing evaluation methods 2. 1 Using MT metrics to compare the output and a reference TS can be considered as a monolingual translation task. As a result, MT metrics such as BLEU  (Papineni et al. , 2002) , which compare the output of an MT system to a reference translation, have been extensively used for TS  (Narayan and Gardent, 2014; Štajner et al. , 2015; Xu et al. , 2016). Other successful MT metrics include TER  (Snover et al. , 2009) , ROUGE  (Lin, 2004)  and METEOR  (Banerjee and Lavie, 2005) , but they have not gained much traction in the TS literature. These metrics rely on good quality references, something which is often not available in TS, as discussed by  Xu et al.
Methodology Our goal is to compare a large number of ways to perform TS evaluation without a reference. To this end, we use the dataset provided in the QATS shared task. We first compare the behaviour of elementary metrics, which range from commonly used metrics such as BLEU to basic metrics based on a single low-level feature such as sentence length. We then compare the effect of aggregating these elementary metrics into more complex ones and compare our results with the state of the art, based on the QATS shared task data and results.
6 The code is available on Github at https: //github. com/facebookresearch/ text-simplification-evaluation 7 We used PCA instead of feature selection because it performed better on the validation set. The number of component was tuned on the validation set as well. 8 Regressors: Linear regression, Lasso, Ridge, Linear SVR (SVM regressor), Adaboost regressor, Gradient boosting regressor and Random forest regressor. 9 Classifiers: Logistic regression, MLP classifier (with L2 penalty, alpha=1), SVC (linear SVM classifier), Knearsest neighbors classifier (k=3), Adaboost classifier, Gradient boosting classifier and Random forest classifier. Meaning preservation It is not surprising that meaning preservation is best evaluated using MT metrics that compare the source sentence to the output sentence, with in particular smoothed BLEU, BLEU 3gram and METEOR.
Regarding meaning preservation, semantic analysis or paraphrase detection models would also be good candidates for a deeper analysis. Warning note We should be careful when interpreting these results as the QATS dataset is relatively small. We compute confidence intervals on our results, and find them to be non-negligible, yet without putting our general observations into  question. For instance, METEOR, which performs best on grammaticality, has a 95% confidence interval of 0. 36 ± 0. 08 on the training set. These results are therefore preliminary and should be validated on other datasets.
Conclusion Finding accurate ways to evaluate text simplification (TS) without the need for reference data is a key challenge for TS, both for exploring new approaches and for optimizing current models, in particular those relying on unsupervised, often MT-inspired models. We explore multiple reference-less quality evaluation methods for automatic TS systems, based on data from the 2016 QATS shared task. We rely on the three key dimensions of the quality of a TS system: grammaticality, meaning preservation and simplicity. Our results show that grammaticality and meaning preservation are best assessed using n-grambased MT metrics evaluated between the output and the source sentence. In particular, METEOR and smoothed BLEU achieve the highest correlation with human judgments. These approaches even outperform metrics that make an extensive use of external data, such as language models.
This shows that a lot of useful information can be obtained from the source sentence itself. Regarding simplicity, we observe that counting the number of characters, syllables and words provides the best results. In other words, given the currently available metrics, the length of a sentence seems to remain the best available proxy for its simplicity. However, given the small size of the QATS dataset and the high variance observed in our experiments, these results must be taken with a pinch of salt and will need to be confirmed on a larger dataset. Creating a larger annotated dataset as well as averaging multiple human annotations for each pair of sentences would help reducing the variance of the experiments and confirming our findings. In future work, we shall explore richer and more complex features extracted using syntactic and semantic analyzers, such as those used by the SAMSA metric, and paraphrase detection models.
OriginalAll three were arrested in the Toome area and have been taken to the Serious Crime Suite at Antrim police station. goodgood good good syntactic Simple All three were arrested in the Toome area. All three have been taken to the Serious Crime Suite at Antrim police station. Examples from the training dataset of QATS. Differences between the original and the simplified version are presented in bold. This table is adapted fromŠtajner et al. (2016b).
Implementation and Experiments We have implemented our method using the FILIB++ C++ library  [23]  for interval computations, the FADBAD++ 2  package for automatic differentiation, and (a slightly modified version of) the aaflib 3  library for affine arithmetic. Let us first consider the running example, with order 2 Taylor models, and an integration step size of 0. 05. Figure  1  left presents the results until t = 2 (obtained in 0. 03 s) compared to the analytical solution (dashed lines): the solid external lines represent the outer-approximating flowpipe, the filled region represents the inner-approximating flowpipe. Until time t = 0, the DDE is in its initialization phase, and the conservativeness of the outer-approximation is due to the abstraction in affine arithmetic of the set of initialization functions.
Acknowledgments. The authors were supported by  ANR  project  MALTHY ,  ANR-13-INSE-0003 ,  DGA  project " Complex Robotics Systems Safety " and the academic chair " Complex Systems Engineering" of Ecole Polytechnique-ENSTA-Télécom-Thalès-Dassault-DCNS-DGA-FX-FDO-Fondation ParisTech.
Recent LHC W/Z vector boson production data in proton-lead collisions are quite sensitive to the heavier flavors (especially the strange PDF), and this complements the information from neutrino-DIS data. As the proton flavor determination is dependent on nuclear corrections (from heavy target DIS, for example), LHC heavy ion measurements can also help improve proton PDFs. We introduce a new implementation of the nCTEQ code (nCTEQ++) based on C++ which has a modular strucure and enables us to easily integrate programs such as HOPPET, APPLgrid, and MCFM. Using ApplGrids generated from MCFM, we use nCTEQ++ to perform a fit including the p-Pb LHC W/Z vector boson data.
PoS(DIS2018)009 PDF Flavor Determination and the nCTEQ PDFs The nCTEQ Collaboration data and the proton data, as illustrated schematically in Fig. 1. This enables us to investigate if observed tensions between data sets could potentially be attributed to the nuclear corrections. The details of the nCTEQ program are presented in Ref. [1]. The analysis includes Deeply Inelastic Scattering (DIS), lepton pair production (Drell-Yan), and pion production from a variety of experiments totaling 740 data (after cuts) and 19 nuclei. The computed PDFs compare favorably to other determinations from the literature  [3] [4] [5]. In this report, we will focus on W ± /Z production as this process is sensitive to the heavier flavors. In Fig. 3  we display the correlations between W + and W -cross sections for proton-lead interactions calculated with different input PDFs and assumptions  [6].
The nCTEQ++ Program & Fast Grid Computations The original nCTEQ project was based on the Fortran 77 proton PDF framework. [7]  As the scope of this project has grown, it was necessary to restructure the nCTEQ code to make it more modular. The new nCTEQ++ program is C++ running on top of legacy Fortran; the evolution is performed by a modified version of HOPPET  [8]  (extended to accommodate grids of multiple nuclei), and the output of the fit is exported in YAML format and then processed by Python Jupyter notebooks. If we are to include complex NLO processes into our PDF fit in an efficient manner, it is essential to make use of the grid tools such as APPLgrid,  [9]  and we have implemented these techniques into our new nCTEQ++ framework. Specifically, nCTEQ++ uses pre-computed APPLgrid grids generated by the MCFM Fortran program  [10]  to perform fast NLO and NNLO W ± /Z calculations inside the Minuit fitting loop.
We demonstrate the advantages of these features below. In Figure  4 -a) we validate our grid implementation for nuclear W ± /Z production. The FEWZ cross section  [11]  (red histogram) was computed using a modified version of the standard FEWZ program to accommodate the proton-lead initial state. [6]  Although such modifications are technically straightforward, their systematic implementation and cross-checking is time consuming nevertheless. In contrast for the APPLgrid calculation (blue histogram), the MCFM program was used (in protonproton mode) to generate a collection of grids. If sufficient statistics are used, the combined grids are independent of the initial PDF. Thus, we can generate grids in proton-proton mode, but then use them also for proton-nucleus or nucleus-nucleus processes! The fact that the red and blue histograms match within statistical accuracy validates this approach.
PDF Flavor Determination and the nCTEQ PDFs The nCTEQ Collaboration the ∼1000 processes included in MCFM. 2  We now illustrate the utility of this flexible modular framework by performing a (preliminary) fit including the LHC W ± /Z heavy ion data. The individual data sets are identified by the ID number corresponding to those in Ref. [6]. The LHC W ± /Z data is displayed in green and have ID numbers corresponding to 62XX; in this fit, we have allowed a floating normalization of 1 σ. Now that we can compute fast NLO (or NNLO) W ± /Z cross sections, we can include these directly into our PDF fits. A sample comparison is displayed in Figure  4-b ). The data for CMS W + are shown as blue squares with error bars, and the theory in the red circles. For comparison, we also present the theory with a 7% normalization shift applied; the quoted luminosity uncertainty is 3.
Conclusion The goal of the nCTEQ project is to obtain the most precise PDFs using the full collection of both proton and nuclear data. In this brief report we have observed that the LHC heavy ion data can help determine nuclear corrections for large A values in a kinematic {x, Q 2 } range very different from those provided by fixed-target measurements. The W ± /Z pPb data are particularly sensitive to the heavier quark flavors (especially the strange PDF), so this provides important information on the flavor decomposition. The new nCTEQ++ framework, which integrates the NLO grid calculations, allows us to easily include these processes into the PDF fit. Improved information on the nuclear corrections from the LHC lead data can also help reduce proton PDF uncertainties as (at present) fixed-target nuclear data is essential for distinguishing the individual flavors  [13].
The International Mouse Phenotyping Consortium (IMPC) is building a catalogue of mammalian gene function by producing and phenotyping a knockout mouse line for every protein-coding gene. To date, the IMPC has generated and characterised 5186 mutant lines. One-third of the lines have been found to be non-viable and over 300 new mouse models of human disease have been identified thus far. While current bioinformatics efforts are focused on translating results to better understand human disease processes, IMPC data also aids understanding genetic function and processes in other species. Here we show, using gorilla genomic data, how genes essential to development in mice can be used to help assess the potentially deleterious impact of gene variants in other species. This type of analyses could be used to select optimal breeders in endangered species to maintain or increase fitness and avoid variants associated to impaired-health phenotypes or loss-of-function mutations in genes of critical importance.
2009; Smith and Eppig 2015) , which is actively developed to capture phenotypes of mutant mouse lines by Mouse Genome Informatics (MGI) based at Jackson Laboratory. All raw data, results of statistical pipelines and curated phenotype data are made publicly available through the IMPC website. The data are further integrated with other resources, including OMIM, MGI and Ensembl. The IMPC database is searchable by gene name, phenotype and disease, allows batch queries and the download of all data, dedicated reports, graphs and images. To date, 5186 mutant lines have been phenotyped (data release 7. 0), with an average of 163 parameters measured on any given mouse, represented by over 128,000 knockouts and 35,000 wildtype or control mice. In addition, embryonic lethal mouse lines are analysed in a specialized embryonic development pipeline that utilizes high-resolution 3D imaging to understand structural changes  (Dickinson et al.
2016). These data allow the IMPC to identify the physiological systems that are disrupted when a gene is disabled and make new gene-phenotype associations. Evolutionary conservation of fundamental processes governing development and support of metazoan life allows functional knowledge gained in one species to be translated to others  (Kirschner and Gerhart 2006; Liao et al. 2006; Saenko et al. 2008; Bellen et al. 2010; Greek and Rice 2012). The IMPC uses its new gene-phenotype associations to identify models for human disease based on phenotypic similarity scores using PhenoDigm  (Smedley et al. 2013) , which establishes a link between IMPC mouse phenotypes mapped to the Mammalian Phenotype Ontology and the clinical descriptions of human diseases, as featured in OMIM and Orphanet, mapped to terms of the Human Phenotype Ontology  (Kohler et al.
Essential genes in mice and humans: mining wildlife genomes for LoF gene variants to identify basis of reduced fitness-a pilot study A previous analysis of IMPC's high-throughput mammalian embryonic phenotype data for 1751 knockout mouse lines resulted in 24, 11 and 65% of the lines being associated to a lethal, subviable and viable phenotype, respectively; this led to the conclusion that, in mice, approximately 35% of the genes are essential for organism viability  (Dickinson et al. 2016 ). We hypothesize that these genes are essential in other mammalian species, and variants causing loss of function in these critical genes might, therefore, be undesirable. To test this hypothesis, we first compared genes identified as essential in IMPC mice with those identified in humans based on cell viability.
We defined a set of core essential genes comprising 1568 genes (9%) which were essential for viability in over 50% of the cell lines where the gene was studied. To understand how gene essentiality compares between human cells and mice, we inferred mouse-to-human orthologues and looked at their distribution in the IMPC and human-viability categories. We obtained a dataset containing 4115 IMPC mouse-to-human orthologues (see Supplementary Methods), of which 4026 were included in the human cell studies (Supplementary Datafile S2). We found that 36% of the mouse genes identified as embryonic lethal (i. e. , essential) corresponded to genes identified as essential in the human cell lines, while 64% corresponded to genes that are non-essential in cells. In the case of genes identified as embryonic viable in mice (i.
When bringing in IMPC and MGI phenotypes, the data become increasingly complex and more difficult to interpret. For the 191 mouse orthologues, there was phenotype information for 62% of the genes, and 34% of the genes were associated with an embryonic lethal phenotype (Fig. 1b ). It is important to note that any given gene may not be associated with a lethal phenotype in all mouse lines, but be linked to a variety of health-impaired phenotypes that do not cause lethality in additional lines (Datafile S3). For example, homozygotes of Chd2 investigated in three genetic backgrounds resulted in postnatal lethality in two of them and in viable individuals in the other, but with healthimpaired phenotypes associated to growth, the skeleton and the hematopoietic system. These effects can be due to potential differences in genomic modifiers between different strains used to generate the knockouts.
2015). Indeed, cardiovascular disease is an important cause of death for gorillas in captivity  (McManamon and Lowenstine 2012). Mouse orthologues were obtained for all except one gene, of which 5 have IMPC phenotype information (F13A1, HNF4A, KLKB1, NPC1, SHH), two associated to a cardiovascular phenotype, one to a skeleton phenotype and two to pre-weaning lethality in homozygotes, respectively (Datafile S6). The MGI database included 20 of these genes, of which 11 were associated to cardiovascular phenotypes and 5 to the hematopoietic system. In the IMPC, there are predictions for the association of 585 mouse orthologues of Western gorilla genes to cardiovascular phenotypes, and 844 to hematopoietic system phenotypes (Fig. 2a ), potentially constituting an important resource for understanding cardiomyopathies in gorillas.
A study on speciation and adaptation in polar bears (Ursus maritimus) identified 20 genes as strong candidates to have been positively selected in polar bears, in what is a prime example of speciation through adaptation to an extreme environment  (Liu et al. 2014). The authors reported disease associations in humans and other mammalian model organisms, including mice, suggesting a function for 11 of these genes associated to adipose tissue development and fatty acid metabolism (APOB), cardiovascular function (APOB, ABCC6, ALPK3, ARID5B, CUL7, EHD3, TTN, VCL, XIRP1) or white fur pigmentation (LYST, AIM1), which may be advantageous in the Arctic. Information derived from the IMPC and MGI databases (Datafile S7) supported these predictions and provided evidence for new roles. An exception was AIM1(currently CRYBG1).
IMPC data indicates no association with coat colour or pigmentation. Homozygotes for AIM1 were not viable, and the heterozygotes presented phenotypes associated with vision and the nervous system, but the fur was normal. In addition, there were phenotypic associations for 3 genes out of the 9 for which no function was reported in the paper. Knockouts for COL5A3, LAMC3 and SH3PXD28 presented a variety of phenotypes, including associations with adipose tissue, the cardiovascular and the immune systems and homeostasis, which are functions that the authors indicated might be relevant in adapting to the Arctic. A recent study on grey wolves (Canis lupus) from North America aimed to identify candidate genes under selection and environmentally driven functional variation  (Schweizer et al. 2016a, b).
In this study, nonsynonymous mutations were significantly correlated with environmental variables in genes associated to lipid metabolism (APOB, LIPG), immunity (DLA-DQA, DLA-DRB1), olfaction (OR4S2, OR5B17, OR6B1), vision and hearing (PCDH15, USH2A), and pigmentation (TYR, TYRP1), where 4 genes had variants with predicted deleterious impact LIPG, OR4S2, OR5B17, USH2A  (Schweizer et al. 2016a ). Information derived from the IMPC database for 6 of these genes and the MGI database for 23 of them reproduces previous findings and provides evidence for new roles (Datafile S8). LIPG is reported to be associated with the metabolic and cardiovascular systems, USH2A with the nervous system but also with vision and hearing, and no information is available for genes OR4S2 and OR5B17, potentially related to olfaction.
We identified at least one study focusing on wild species, giant and red pandas Ailuropoda melanoleuca and Ailurus fulgens  (Hu et al. 2017 ) and two on cattle  (Kadri et al. 2014; Biase et al. 2016 ) that have used the mouse knockout database information to further characterize genes or processes. The panda species are predicted to have independently acquired adaptations in 70 genes to a bamboo-rich diet, including a pseudothumb (limb development genes DYNC2H1 and PCNT) and features related to digestion and nutrient utilization (in particular genes GIF, CYP4F2, ADH1C and CYP3A5). The IMPC database complements data collected by MGI by providing information for 5 additional genes (Datafile S9). In the two cattle studies, mouse knockout data informed about processes related to infertility  (Kadri et al.
Outlook Here we show how viability data collected by the IMPC are defining a set of essential genes that are likely also relevant in other species, particularly mammals. Identifying deleterious mutations is important for the design of captive breeding strategies  (Bosse et al. 2015) , and we encourage exploring the potential of the analyses presented here to identify critical functional variants. An assessment of human and mouse genes orthologous to gorilla genes containing homozygous LoF variants indicates that a number of them are strong candidates to compromise fitness and, therefore, further investigation of the phenotypes of gorillas with these variants will be required. The phenotypic effects of LoF or any other variants will manifest under certain genetic conditions (genetic background) or environmental conditions.
In addition, integration of phenotype data is expected to become prominent in evolutionary studies. In order to produce databases that are computationally tractable and that allow for cross-species integrations, as well as to avoid loss of information, adhering to standards and persistent genetic identifiers (e. g. , Ensembl, HGNC or MGI identifiers), as well as applying purpose-oriented ontologies, will be critical. In evolutionary biology and phylogenetic systematics, efforts to computationally integrate genetic, phenotypic and anatomical data include the 'Phenotype And Trait Ontology' (PATO;  Mabee et al. 2007 ) and the Phenoscape project  (Dahdul et al. 2010 ) but improvements in this area will certainly be needed  (McMurry et al. 2017). Animal models have proved useful to develop assisted reproductive technologies for endangered species, including lessons learned from oocyte and embryo culture in domestic animals and humans, and oncofertility techniques applied to humans  (Comizzoli et al.
Currently, methods based on genomic data are being put forward to improve breeding strategies of wild species to attempt to minimize the impact of undesirable genetic variants while maintaining acceptable levels of genetic diversity  (Bosse et al. 2015; Irizarry et al. 2016)  and rapid advances in CRISPR/Cas9 technology in animal models to reduce the risk of off-target mutagenesis opens up opportunities to eliminate deleterious mutations in zygotes. In the case of wild species, such methods would allow the persistence of fitness-linked alleles and the avoidance of deleterious mutations without the risks associated with inbreeding or breeding between two similar species. The combined accumulation of gene function annotation by the IMPC and their advances in the use of CRISPR/Cas9 technology will be able to assist in future conservation efforts.
We present a significant extension of the Brain Imaging Data Structure (BIDS) to support the specific aspects of magnetoencephalography (MEG) data. MEG measures brain activity with millisecond temporal resolution and unique source imaging capabilities. So far, BIDS was a solution to organise magnetic resonance imaging (MRI) data. The nature and acquisition parameters of MRI and MEG data are strongly dissimilar. Although there is no standard data format for MEG, we propose MEG-BIDS as a principled solution to store, organise, process and share the multidimensional data volumes produced by the modality. The standard also includes well-defined metadata, to facilitate future data harmonisation and sharing efforts. This responds to unmet needs from the multimodal neuroimaging community and paves the way to further integration of other techniques in electrophysiology.