dataesr/scientific-paragraphs-categorization huggingface dataset:
Data For our model and baselines, we consider candidate substitutions from three datasets. The first is WordNet  (Miller, 1995) , a lexical network encoding manually identified semantic relationships between words, such as synonymy, hypernymy and hyponymy. This resource has been widely used in substitution tasks  (McCarthy and Navigli, 2007). We also use paraphrases extracted from the Paraphrase Database (PPDB) and the Simple Paraphrase Database (SimplePPDB). PPDB is a collection of more than 100 million English paraphrase pairs  (Ganitkevitch et al. , 2013). These pairs were extracted using a bilingual pivoting technique  (Bannard and Callison-Burch, 2005) , which assumes that two English phrases that translate to the same foreign phrase have the same meaning. PPDB was updated by  Pavlick et al.
, 2013b,a). The context C is the set of context embeddings generated by skip-gram for words appearing within a fixed-width window of the target t in a sentence. We use a context window of 1; while this seems counter-intuitive, this is the best-performing window found by  (Cocos et al. , 2017) , and we also confirm this result remains true in Section 5. 2. We use the AddCos implementation of  Cocos et al. (2017)   5  , and 300-dimensional word and context embeddings trained over the 4 billion words in the AGiga corpus  (Napoles et al. , 2012)  using the gensim word2vec package  (Mikolov et al. , 2013b,a; Řehůřek and Sojka, 2010). 6 n our experiments, candidate substitutes for a target word are its paraphrases in the PPDB and SimplePPDB resources. The model needs to select among these candidates the ones that best carry the meaning of target words in specific contexts.
Complex Word Identification Datasets for training and evaluating Complex Word Identification (CWI) systems were created and released in the SemEval 2016 competition  (Paetzold and Specia, 2016)  but we decided not to use them for several reasons. Although this was a CWI task, surprisingly only 4. 7% of the words in the test data were identified as complex, and all the other words were viewed as simple. As a consequence, none of the systems that participated in the SemEval task managed to beat the accuracy of the "All Simple" baseline which labeled all words in the test set as simple (0. 953). As noted by  Paetzold and Specia (2016) , the inverse problem is present in the corpus developed by  Shardlow (2013b) , where the "All Complex" baseline achieved higher accuracy, recall and F-scores than all other tested systems, suggesting that marking all words in a sentence as complex is the most effective approach for CWI.
Another problem in the SemEval-2016 dataset is that although the number of complex words is much higher in the training data (32%), 18% of all words were annotated as complex by only one out of 20 annotators and considered as complex. In addition to the highly different number of complex words in the training and test data, the two datasets are also imbalanced in terms of size, with only 2,237 training instances and 88,211 testing instances. These factors make this dataset a dubious choice for system evaluation. Comparison to the participating systems is also extremely difficult, since the best systems are ones that label most of the data as simple. For these reasons, we decided to create and use our crowdsourced data for training and evaluation. 7  We compare the performance of an SVM classifier with only word features (SVM-word) to one that exploits both word and context features (SVM-context).
We use 5-fold cross validation on unique words from the training data collected through Mechanical Turk (see Section 4. 1). We also compare a Random Forest classifier with only word features (RF-word) to one with word and context features (RF-context). We consider three baselines: • labeling all words as complex (All-Complex). • thresholding for word length (Token Length), considering longer words as complex; the length threshold with the best performance was 7. • thresholding for word frequency using Google n-gram counts (n-gram Frequency), considering more frequent words as simple; the frequency threshold with the best performance was 19,950,000. The results of this experiment are shown in Table 2. While the Token Length and n-gram Frequency baselines have higher recall, both of our models show substantial improvements in terms of precision and increases overall accuracy and F-score, with SVM outperforming Random Forest.
• AddCos-PPDB: We extract all PPDB synonyms for a complex word and rank them using the AddCos model described above. The performance of AddCos with SimplePPDB paraphrases (AddCos-SimplePPDB) in the lexical simplification task is compared to performance of the baselines in Table  3. For each model, we calculate Top 1 and Top 5 accuracy scores, which show how often the gold-standard simple word was proposed as the best fitting or among the 5 highestranked paraphrases. In addition, we calculate the upper bound performance for each dataset (PPDB, SimplePPDB and WordNet), i. e. how often the gold-standard simple word was found as a paraphrase of the target word in the dataset. This is useful in telling us how well we could potentially do, if we could perfectly rank the paraphrases. When performing this experiment, we also evaluated the impact of the context window size on the quality of the proposed substitutions.
The AddCos implementation we used in this work does not rely on syntactic annotations and can be easily applied to new languages. In future work, we plan to experiment with syntactic substitution models and with syntax-based word embeddings like the ones used in the initial AddCos implementation  (Melamud et al. , 2015). We expect syntactic information to further enhance the quality of the proposed substitutions, ensuring the functional similarity of the lexical substitutions to the target word. Furthermore, we intend to integrate lexical and syntactic simplification, both crucial steps towards text simplification.
Acknowledgements We would like to thank the anonymous reviewers for their helpful comments and feedback on this work, and Anne Cocos for sharing with us her implementation of the AddCos model with PPDB paraphrase substitutes. This material is based in part on research sponsored by DARPA under grant number FA8750-13-2-0017 (the DEFT program) and HR0011-15-C-0115 (the LORELEI program). The U. S. Government is authorized to reproduce and distribute reprints for Governmental purposes. The views and conclusions contained in this publication are those of the authors and should not be interpreted as representing official policies or endorsements of DARPA and the U. S. Government. The work has also been supported by the French National Research Agency under project ANR-16-CE33-0013. Finally, we gratefully acknowledge the support of NSF-SBIR grant 1456186.
Target species Elytrigia repens (L. ) Desv. and Brachypodium pinnatum (L. ) Beauv. are two common perennial species in the grasslands of western Europe. Both species grow laterally by producing sympodial plagiotropic rhizomes of which the apexes develop into erected shoots (ramets) (CLO-PLA database,  Klimešová & de Bello, 2009). Both species present intraspecific variation in their clonal and aerial traits, particularly in response to resource availability and competition (de  Kroon & Knops, 1990; van der Werf, Visser, Schieving, & Lambers 1993; Pottier & Evette, 2009; Bittebiere & Mony, 2015).
Acknowledgements This project benefited from two grants from the  French National Agency for Research   ANR-08-SYSC-012  and the Systerra call  ANR-10-STRA-002  for data collection. The Diversify  FP7 FET Project , the  ERC  Grant agreement 647038 [BIODESERT], and the  Juan de la Cierva-Formación  fellowship from the  Spanish Ministry of Economy and Competitiveness  ( FJCI-2015-26782 ) financed the postdoctoral position of H. Saiz. We are very grateful to  T. Fontaine ,  V. Gouesbet , and  F. Nassur  for providing technical assistance with the maintenance of the experimental design and to  P. Boulard  for providing assistance with the experiment and data collection. We also thank  American Journal Experts  for editing the English.
The implementation of Limbo follows a policy-based design  (Alexandrescu 2001 ) that leverages C++ templates: this allows it to be highly flexible without the cost induced by classic object-oriented designs  (Driesen and Hölzle 1996)  (cost of virtual functions). The regression benchmarks 1 show that the query time of Limbo's Gaussian processes is several orders of magnitude better than the one of GPy (a state-of-the-art Python library for Gaussian processes 2 ) for a similar accuracy (the learning time highly depends on the optimization algorithm chosen to optimize the hyper-parameters). The black-box optimization benchmarks 3 demonstrate that Limbo is about 2 times faster than BayesOpt (a C++ library for data-efficient optimization, (Martinez-Cantin 2014)) for a similar accuracy and data-efficiency.
In practice, changing one of the components of the algorithms in Limbo (e. g. , changing the acquisition function) usually requires changing only a template definition in the source code. This design allows users to rapidly experiment and test new ideas while keeping the software as fast as specialized code. Limbo takes advantage of multi-core architectures to parallelize the internal optimization processes (optimization of the acquisition function, optimization of the hyper-parameters of a Gaussian process) and it vectorizes many of the linear algebra operations (via the Eigen 3 library 4 and optional bindings to Intel's MKL). To keep the library lightweight, most of the optimizers in Limbo are wrappers around external optimization libraries:   (Hansen and Ostermeier 1996 )) • a few other algorithms that are implemented in Limbo (in particular, RPROP  (Riedmiller and Braun 1993) , which is a gradient-based optimization algorithm) The library is distributed under the CeCILL-C license 7 via a GitHub repository, 8 with an extensive documentation 9 that contains guides, examples, and tutorials.
The code is standard-compliant but it is currently mostly developed for GNU/Linux and Mac OS X with both the GCC and Clang compilers. New contributors can rely on a full API reference, while their developments are checked via a continuous integration platform (automatic unit-testing routines). Limbo is currently used in the ERC project ResiBots 10 , which is focused on data-efficient trial-and-error learning for robot damage recovery, and in the H2020 projet PAL.
All surveys include observational biases, which makes it impossible to directly compare properties of discovered trans-Neptunian Objects (TNOs) with dynamical models. However, by carefully keeping track of survey pointings on the sky, detection limits, tracking fractions, and rate cuts, the biases from a survey can be modelled in Survey Simulator software. A Survey Simulator takes an intrinsic orbital model (from, for example, the output of a dynamical Kuiper belt emplacement simulation) and applies the survey biases, so that the biased simulated objects can be directly compared with real discoveries. This methodology has been used with great success in the Outer Solar System Origins Survey (OSSOS) and its predecessor surveys. In this chapter, we give four examples of ways to use the OSSOS Survey Simulator to gain knowledge about the true structure of the Kuiper Belt.
Examples include: the orbital evolution of Neptune (e. g. Malhotra 1993 ), the large scale re-ordering of the Solar System (e. g. Thommes et al. 1999; Gomes et al. 2005) , the process of planetesimal accretion (e. g. Stern 1996; Davis & Farinella 1997) , the production of cometary size objects via collisional processes (e. g. Stern 1995)  and their delivery into the inner Solar System  (Duncan et al. 1988) , and the stellar environment in which the Sun formed (e. g. Brunini & Fernandez 1996; Kobayashi & Ida 2001). Our goal as observers is to test these models and their consequences by comparison to the Solar System as we see it today. Given the sparse nature of the datasets and the challenges of detecting and tracking trans-Neptunian objects (TNOs), a strong statistical framework is required if we are to distinguish between these various models.
WELL-CHARACTERIZED SURVEYS A well-characterized survey is one in which the survey field pointings, depths, and tracking fractions at different magnitudes and on-sky rates of motion have been carefully measured. The largest wellcharacterized TNO survey to date is the Outer Solar System Origins Survey (OSSOS;  Bannister et al. 2016) , which was a large program on the Canada-France-Hawaii Telescope (CFHT) carried out over five years, and was specifically planned with the Survey Simulator framework in mind. OSSOS builds on the methodology of three previous well-characterized surveys: the Canada-France Ecliptic Plane Survey (CFEPS;  Jones et al. 2006; Kavelaars et al. 2008b; Petit et al. 2011) , the CFEPS high-latitude component  (Petit et al. 2017) , and the survey of  Alexandersen et al. (2016).
The time separation between imaging successive camera fields inside each observing block must be long enough that significant motion against background stars has occurred for TNOs, but not so long that the TNOs have moved too far to be easily recovered by eye or by software. OSSOS used triplets of images for each camera field, taken over the course of two hours. The exposure times are chosen as a careful compromise between photometric depth and limiting trailing 1  of these moving sources. The images are searched for moving objects by an automated and robustly tested moving object pipeline (CFEPS and OSSOS used the software pipeline described in  Petit et al. 2004) , and all TNO candidates discovered by the software are visually inspected. Prior to searching the images, artificial sources (whose flux and image properties closely mimic the real sample) are inserted into the images.
The Basics of a Survey Simulator A Survey Simulator allows models of intrinsic Kuiper Belt distributions to be forward-biased to replicate the biases inherent in a given well-characterized survey. These forward-biased simulated distributions can then be directly compared with real TNO detections, and a statement can be made about whether or not a given model is statistically consistent with the known TNOs. One particular strength of this approach is that the effect of non-detection of certain orbits can be included in the analysis. Methods that rely on the inversion of orbital distributions are, by their design, not sensitive to a particular survey's blind spots. Directly comparing a model with a list of detected TNOs (for example, from the Minor Planet Center database), with a multitude of unknown detection biases, can lead to inaccurate and possibly false conclusions.
The OSSOS Survey Simulator follows this basic model, but takes into account more realities of survey limitations. It is the result of refinement of this Survey Simulator software through several different well-characterized surveys: initially the CFEPS pre-survey  (Jones et al. 2006) , then CFEPS  (Kavelaars et al. 2008a; Petit et al. 2011) , then  Alexandersen et al. (2016) , and finally OSSOS  (Bannister et al. 2016). The OSSOS Survey Simulator software and methodology are now robustly tested, and are presented below.
The Details of the OSSOS Survey Simulator While the methodology presented here is specific to the OSSOS Survey Simulator, by measuring on sky pointings, magnitude limits and tracking fractions, a Survey Simulator can be built for any survey. The Survey Simulator for the OSSOS ensemble of well-characterized surveys is available as a package 3  , and the list of observed characteristics of the TNOs discovered in these surveys is published in  Bannister et al. (2018). To forward-bias a distribution of objects to allow statistical comparison with the real TNO discoveries in the surveys, the OSSOS Survey Simulator uses the following steps. The instantaneous on-sky position, rate of motion, and apparent magnitude are computed from an orbit, position, and H-magnitude, and can be written to a file that contains the "drawn" simulated objects.
How these are generated is free for the user to decide. The user may edit the source software to include generation of orbits and absolute magnitudes within the Survey Simulator, but it is recommended that a separate script is used in conjunction with the unedited Survey Simulator. The software package comes with a few examples, and details are provided in the following paragraph showing how we on the OSSOS team have implemented the generation of simulated TNOs. The orbital elements of an object can be determined in a variety of ways. The Survey Simulator can choose an orbit and a random position within that orbit, either from a list of orbits (as would be produced by a dynamical model) or from a parametric distribution set by the user. Orbits from a list can also be easily "smeared," that is, variation is allowed within a fraction of the model orbital elements, in order to smooth a distribution or produce additional similar orbits (however, one must be careful that the distribution is dominated by the original list of orbits, and not by the specifics of the smearing procedure).
The four emplacement simulation scenarios analyzed here include the following conditions: 4. The four giant planets and an additional 2 Earth mass rogue planet that started with a = 35 AU and q = 30 AU, which was ejected after ∼200 Myr (simulation from  Gladman & Chan 2006). The papers which have recently proposed the presence of a distant undiscovered massive planet (popularly referred to as "Planet 9";  Trujillo & Sheppard 2014; Batygin & Brown 2016)  rely on published detections of large semimajor axis (a), high pericenter distance (q) TNOs, which have extreme biases against detection in flux-limited surveys 4. These large-a, high-q TNOs are drawn from different surveys, with unpublished and thus unknown observing biases. The Minor Planet Center (MPC) Database, which provides a repository for TNO detections, does not include information on the pointings or biases of the surveys that detected these TNOs.
Sample selection caused by survey depth and sky coverage can be non-intuitive, caused by weather, galactic plane avoidance, and telescope allocation pressure, among other possibilities  (Sheppard & Trujillo 2016; Shankman et al. 2017 ). Thus the assumption can not be made that a collection of TNOs from different surveys will be bias-free. Further, the detected sample of objects in the MPC Database provides no insight into the parts of the sky where a survey may have looked and found nothing, nor to strong variation in flux sensitivity that occurs due to small variations in image quality. Here we demonstrate how to use the results of well-characterized surveys to compare real TNO detections to the output of a dynamical model, comprising a list of orbits. We visualize this output with a set of cumulative distributions.
Using a Parametric Population Distribution: Biases in Detection of Resonant TNOs In this section we show how to use the Survey Simulator with a resonant TNO population in order to demonstrate two important and useful points: how to build a simulated population from a parametric distribution, and showing how the Survey Simulator handles longitude biases in nonuniform populations. For this demonstration, we build a toy model of the 2:1 mean motion resonance with Neptune, using a parametric distribution built within the Survey Simulator software (though we note that this parametric distribution could just as easily be created with a separate script and later utilized by the unedited Survey Simulator). The parametric distribution here is roughly based on that used in  Gladman et al. (2012) , but greatly simplified and not attempting to match the real 2:1 TNO distributions.
The Survey Simulator chooses from this parametric distribution with a within ±0. 5 AU of the resonance center (47. 8 AU), q = 30 AU, i = 0 • , 6  and libration amplitudes 0 -10 •. There are three resonant islands in the 2:1 resonance, and the libration center φ 21 is chosen to populate these three islands equally in this toy model. For the purposes of our toy model, we define these angles as: leading asymmetric φ 21 = 80 • , symmetric φ 21 = 180 • , and trailing asymmetric φ 21 = 280 •. The resonant angle φ 21 in the snapshot is chosen sinusoidally within the libration amplitude. The ascending node Ω and mean anomaly M are chosen randomly, then the argument of pericenter ω is chosen to satisfy the resonant condition φ 21 = 2λ TNO -λ N -TNO , where mean longitude λ = Ω + ω + M, longitude of pericenter = Ω + ω, and the subscripts TNO and N denote the orbital elements of the TNO and Neptune, respectively.
While the intrinsic distributions in this model are evenly distributed among the three islands by design (33% in each), the detected TNOs are not. This is because TNOs on eccentric orbits are most likely to be detected close to pericenter, and in this toy model, the pericenters are highly clustered around three distinct positions on the sky. 8 he red points in Figure  2  show the simulated TNOs that are detected by the OSSOS Survey Simulator. Just because of the positions of survey pointings on the sky, the detections are highly biased toward the leading island, which has over 2/3 of the detections, with fewer detections in the trailing island, and only 10% of all the detections in the symmetric island. Without knowing the on-sky detection biases (as is the case for TNOs pulled from the MPC database), one could easily (but erroneously) conclude that the leading island is much more populated than the trailing island, when in fact the intrinsic populations are equal.
The relative fraction of n:1 TNOs that inhabit different resonant islands has been discussed in the literature as a diagnostic of Neptune's past migration history  (Chiang & Jordan 2002; Murray-Clay & Chiang 2005; Pike & Lawler 2017). Past observational studies have relied on n:1 resonators from the MPC database with unknown biases, or the handful of n:1 resonators detected in well-characterized surveys to gain weak (but statistically tested) constraints on the relative populations. Upcoming detailed analysis using the full OSSOS survey and making use of the Survey Simulator will provide stronger constraints on this fraction, without worries about unknown observational biases (Chen, Y. -T. , private communication).
Constraints from Non-Detections: Testing a Theoretical Distant Population Here we show a perhaps unintuitive aspect of the Survey Simulator: non-detections can be just as powerful as detections for constraining Kuiper Belt populations. Non-detections can only be used if the full pointing list from a survey is published along with the detected TNOs. An examination of the orbital distribution of TNOs in the MPC database makes it clear that there is a sharp dropoff in the density of TNO detections at a 50 AU and q 40 AU (Figure  2  in  Sheppard & Trujillo 2016, demonstrates this beautifully). Is this the result of observation bias or a real dropoff? Without carefully accounting for survey biases  (Allen et al. 2001 (Allen et al. , 2002)) , by using the MPC database alone, there is no way to know.
2016) , dropouts from meanmotion resonances during Neptune's orbital circularization phase  (Pike & Lawler 2017) , interactions with a distant giant planet  (Gomes et al. 2015; Batygin & Brown 2016; Lawler et al. 2017) , a stellar flyby  (Morbidelli & Levison 2004) , capture from a passing star  (Jílková et al. 2015) , and perturbations in the Solar birth cluster  (Brasser & Schwamb 2015). Here we have highlighted only a small number of the inconsistencies between models and real TNO orbital data. The level of detail that must be included in Neptune migration simulations has increased dramatically with the release of the full OSSOS dataset, containing hundreds of TNOs with the most precise orbits ever measured. The use of the Survey Simulator will be vital for testing future highly detailed dynamical emplacement simulations, and for solving the lingering mysteries in the observed structure of the Kuiper belt.