dataesr/scientific-paragraphs-categorization huggingface dataset:
Funding: MW acknowledges funding by the  EU H2020  research and innovation programme  MSCA-RISE-2016  under grant agreement No. 734439  INFERNET. This work has been undertaken partially in the framework of CALSIMLAB and supported by the public grant  ANR-11-LABX-0037-01  overseen by the  French National Research Agency (ANR)  as part of the  "Investissements d'Avenir" programme  ( ANR-11-IDEX-0004-02 ). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
On-line and off-line monitoring of the pilot plant Different on-line sensors were placed in the HRAP, which allowed continuous data acquisition. The on-line sensors used in this study were: (i) a pH-T transmitter (METTLER TOLEDO InPro ® 4260 SG), (ii) a DO probe (METTLER TOLEDO InPro ® 6800 G Amperometric Oxygen Sensor) and (iii) an irradiation sensor (Skye PAR Quantum Sensor) for measuring the PAR. Besides the on-line process monitoring, samples were taken from the influent and the effluent streams to assess the performance of the biological processes. The concentrations of the following parameters were determined: total and soluble COD (COD T and COD S , respectively), N T , P T , inorganic nutrients (NH 4 + , NO 2 -, NO 3 -and PO 4 3-), total suspended solids (TSS) and volatile suspended solids (VSS).
Multivariate projection analysis Statistical analyses based on multivariate projection were performed to assess potential key state indicators based on pH and DO measurements affecting the HRAP performance. The mixOmics package (http://www. mixOmics. org) implemented in the R statistical software version 3. 2. 3 (http://www. R-project. org) was used for this purpose. Firstly, the set of calculated variables shown in Table  2  was statistically analyzed through principal component analysis (PCA) for screening and validation of potential key state indicators for real-time HRAP monitoring. PCA reduces the dimensionality of the data by creating uncorrelated artificial variables called principal components (PCs) that combine linearly the original variables, retaining as much information as possible  (Wold et al.
Statistical identification of the key state indicators Initially, PCA and PLSR analyses were carried out using all the available data (from whole experimental period). The obtained results are presented in Figure  5a  and Figure  5b. Two PCs accounted for a cumulative explained variance of 75 %, indicating that the given results represent most of the contained information. The results of Figure  5a  further confirm the conclusions from the previous section, since two clusters were observed, which corresponded to data from day 1 to day 19 (bacteria community was predominant), and data from day 20 to day 36 (microalgae-bacteria community). These results confirm also the different behaviour of the HRAP plant with respect to the microalgal activity. Days 1 and 10 were considered outliers of the PCA.
Acknowledgements The authors acknowledge the financial support of the  French National Research Agency (ANR)  for the " Phycover "  project  (project  ANR-14-CE04-0011 ), the  Spanish Ministry of Economy and Competitiveness (MINECO , Projects  CTM2014-54980-C2-1-R  and  CTM2014-54980-C2-2-R ) jointly with the  European Regional Development Fund (ERDF) , and the  European Climate KIC  association for the "MAB 2. 0" project ( APIN0057_2015-3. 6-230_P066-05 ). Ángel Robles  would also like to acknowledge the financial aid received from  Generalitat Valenciana  via a  VALi+d post-doctoral grant  ( APOSTD/2014/049 ). Gabriel Capson-Tojo  is grateful to the  Xunta de Galicia  for his postdoctoral fellowship ( ED481B-2018/017 ).
11 ndeed, the issue of reproducibility for NLP is critical in a clinical setting, where the quality of the NLP processes (eg, for information extraction and concept normalization) may directly impact the secondary use of the extracted data (eg, the results of public health studies). 12 Subsequently, health policies supported by the results of public health studies relying on this data may also be misled. The field of bioinformatics  13  is well acquainted with similar issues around reproducibility. Four levels of reproducibility have been articulated by 2 as: Repeat, when an experiment can be performed again on the same computational environment and yield the same results; Replicate, when the experiment can be performed in a new setting and still yield the same results; Reproduce, when an experiment validates the same scientific hypothesis by using a different method, or a different dataset; Reuse, when a different experiment is performed with similarities to the original experiment (eg, the same methods are applied to a range of datasets and support the original conclusion).
The initial step towards this deployment is the ability to effectively replicate workflows with identical tools, datasets, and outcomes. In this study, we focus our interest on clinical NLP deployed in the hospital and the dimensions of reproducibility associated with the production of data (and not secondary use). We identify a need for actionable reproducibility from the inception of experiments to the final results integrated in clinical practice. With the advent of text use in clinical artificial intelligence algorithms, we must be in a position to assess the impact of changes anywhere in the processing pipeline to provide supporting evidence for algorithm outputs and to facilitate hospital interoperability. We do not seek to invent new formalisms but to integrate existing definitions and adapt them to our specific use cases.
CLAMP, Clinical Language Annotation Modelling and Processing (2017) CLAMP  33  is currently in use by 509 organizations. CLAMP is a graphical user interface or command line clinical NLP solution which relies on the UIMA model. The latest release 1. 4. 0 is freely available for research use upon submission of a request form. CLAMP architecture is based on 3 pillars:  38  First, the NLP Pipeline components allow the creation of a pipeline by a simple drag and drop of CLAMP NLP components; second, a set of NLP pipelines is supplied for common clinical applications (eg, extraction of smoking status, colorectal cancer, etc). The machine learning and hybrid approach pillar supplies alternative methods (eg, support vector machine, rule-based methods, etc) which can be retrained on a new corpus. The third pillar is a corpus management and annotation tool that provides a built-in text annotation interface based on brat.
Moreover, GATE integrates existing tools, such as LingPipe  40  or OpenNLP. 41  ScispaCy, spaCy pipeline, and models for scientific documents (2019) ScispaCy  35  is based on spaCy 42 but is dedicated to scientific projects. ScispaCy proposed 4 named entity recognition (NER) models dedicated to English trained on 4 different corpora, including a biomedical one (BIONLP13CG). ScispaCy also retrained 3 spaCy mod- els for part-of-speech (POS) tagging, dependency parsing, rule-based tokenizer, and syntactic parser trained on biomedical datasets. Scis-paCy also proposed additional pipeline components such as Abbre-viationDetector and an EntityLinker linked to the UMLS, RxNorm, GeneOntology, Medical Subject Headings (MeSH), and the Human Phenotype Ontology. ScispaCy leverages external libraries, such as tensorflow, for deep learning models.
ScispaCy (similarly to spaCy) relies on coding skills (ie, not on graphical user interface). While resources for languages other than English are available in spaCy, ScispaCy does not offer resources outside of English. TextFlows, text mining workflows platform (2015) TextFlows,  36  a fork of ClowdFlows,  43  is an open source web interface and focuses on smoothing the construction, execution, and shareability of NLP workflows through a web interface. TextFlows was not designed for clinical narratives, and its last update was December 1, 2017. TextFlows NLP tools are called "widgets" (eg, lem-matization or POS) and are assembled to produce workflows. Text mining widgets embed several software packages (eg, NLTK  44  and scikit-learn  45  ) and cover a wide range of NLP analysis (eg, tokenization, stop word removal, POS tagging, stemming, and lemmatization).
Moreover, users can extend TextFlows by adding their own widget. TextFlows is developed as a Django Python web interface and can be run on a cluster of servers. OpenMinTed, Open Mining Infrastructure for TExt and Data (2015) OpenMinTed 28 emphasized the use of text and data mining in the context of open access scientific publication. OpenMinted is built around 3 building blocks. The first is named "Content resources" and represents open access articles used as input for text and data mining software. The second building block, "Text and data mining software," performs basic NLP tasks (eg, tokenization, sentence splitting, NER, etc) split between component and applications. The last building block, "Ancillary knowledge resources," supports the management of ontologies, terminologies or machine learning models.
Furthermore, OpenMinTed dispenses OMTD-SHARE, a metadata schema which covers description of resources and text and data-mining software to enhance interoperability. OpenMinTed is developed in Java; workflows are executed within Galaxy. Grid galaxy, the language application grid, and galaxy flavor (2016) LAPPS Grid 27 aspired to ease the creation of workflows from hundreds of NLP tools indexed in a library and facilitate the sharing of data. The LAPPS Grid Project is a web interface which contains a wide range of NLP tools (eg, sentence splitters, tokenizers, etc). Tools came from multiple providers, such as Stanford CoreNLP,  46  Apache open NLP,  47  and Gate. 37 The LAPPS Grid Project released a Galaxy flavor called Grid Galaxy in March/April 2018, available as Docker containers. Table  4  illustrates the adherence of the frameworks to the reproducibility features.
It is worth noting that the community of users has made an attempt to enrich the system. For example,  30  have proposed ProvCare. OpenMinTed has its own metadata schema called OMTD-SHARE. Standardization of data interaction through the Galaxy framework aims at ensuring replicability and reproducibility of analysis. As an example, each datum uploaded in Galaxy is associated to an identifier and a type (BAM, FastQ, etc). Furthermore, each datum produced by a tool is also associated with the unique identifier of the tool. Note that recommendation R13 to R16 (URI and standard vocabulary), are also recommended in the FAIR principles. LAPPS Grid fully integrated their pipeline in Galaxy and provided to the community a Galaxy flavor as a container solution. Also, Open-MinTeD pipelines are executed with Galaxy.
Remarks on clinical data sharing Clinical corpus sharing is restricted for confidentiality reasons, and FAIR principles cannot always be applied easily. In many cases, corpora cannot be shared outside the hospital that hosts them. As a result, shared datasets are limited to American English narratives (eg, based on MIMIC  49  ) or substitutes to clinical corpora, such as clinical case descriptions (eg, https://temu. bsc. es/cantemist/ 50 ), MED-LINE biomedical literature (eg,  51  ), or micro blogs (eg, SSM4H  52  ). Data access issues include the attrition or modification of dataset content over time when data elements (such as twitter posts and MEDLINE citations) are distributed in the form of identifiers. NLP researchers and tool developers are bound by data sharing constraints. However, these limitations should be taken into account when considering reproducibility.
For example, reporting results on a shareable dataset, even with limitations such as size or scope, could be helpful. Could WMS be used in NLP? NLP tools could easily be adapted to be used through WMS. The main requirement for using WMS is a clear definition of inputs and outputs. A large number of NLP tools already fulfill this requirement; clinical NLP shared tasks (eg, offered at n2c2, CLEF, SemEval) have encouraged the adoption of well-defined formats implemented in the shared datasets. 53 The availability of the tools as a command line executable (ie, not as graphical user interfaces) also simplifies the integration process. For example, QuickUMLS  54  can easily be integrated into the WMS Nextflow  16  (eg, https://github. com/equipe22/QuickUMLS_Nextflow). Moreover, the combination of single tools in WMS is simple.
From replicability to reusability As a motivation for this study, we outlined the importance of achieving replicability (reiterating experiments in identical settings to yield the same results) to pave the way to reusability (conducting similar experiments with selected variations in methods and/or datasets to yield consistent conclusions). The core of clinical NLP research investigates methods and the right combination of NLP methods and clinical data to achieve clinical outcomes. From the perspective of replication, methods are not investigated as such, since the goal is to apply existing tools, models, and methods on corpora without modification (ie, without retraining machine learning models, recreating deep semantic representations, or information extraction rules). In this context, traceability offers the means to facilitate the comparison to original experiments and identify discrepancies.
Traceability of performance at each step in a workflow helps identify the cause for overall variation in a workflow. Moving on to reproducibility and reusability, changes to corpora or methods can be introduced. The importance of traceability is therefore heightened to keep track of the new experiments specifics as well as to better control for workflow variations. In this context, parameters specific to machine learning methods need to be tracked, such as corpus partitions, random seeds, learning rate, and size of hidden layers. Likewise, resources, such as language models or terminologies used by NLP methods, can be customized or retrained, which needs to be recorded. Additionally, performance obtained on a given dataset with a specific set of parameters should also be recorded-ideally, using a shareable dataset for this purpose.
Source https://github. com/ apache/ctakes https://clamp. uth. edu/getclamp. php https://github. com/ GateNLP/gate-core https://github. com/allenai/scispacy https://github. com/ xflows/textflows https://github. com/openminted/registry/ https://github. com/lappsgrid-incubator/galaxyappliance Demo https://ctakes. apache. org/ downloads. cgi https://clamp. uth. edu/ clampdemo. php https://gate. ac. uk/demos/ https://scispacy. apps. allenai. org/ http://textflows. org/ https://github. com/openminted/install-tutorial https://galaxy. lappsgrid. org/ Tools format UIMA UIMA GATE scpaCy widget Galaxy tool Galaxy tool Container https://github. com/tmills/ ctakes-docker https://github. com/ choyiny/ctakes-server https://github. com/Cog-Stack/gate-nlp-service https://github. com/allenai/scispacy https://github.
We briefly mention that this formalism has afforded us to develop algorithm for spike train analysis  [100, 101, 102]  leading to the software PRANAS  [103]  freely available at https://team. inria. fr/biovision/pranas-software/, although we do not develop along these lines in this paper. We focus then on a specific question. When dealing with a model of spiking neurons, how much of the intrinsic dynamics of neurons, their interaction via synapses, and the influence of stimuli, constrain the collective spatio-temporal spike statistics? Neurons are (nonlinear) entities evolving in a concerted way (as they interact via synapses) and responding to external stimuli. The theoretical analysis of this high dimensional systems can be made thanks to mathematical methods (dynamical systems, bifurcations theory, stochastic processes, partial differential equations) or theoretical physics (statistical physics, nonlinear physics).
Acknowledgments: The authors thanks  Antonio Galves ,  Godofredo Iommi  and  Ignacio Ampuero  for valuable suggestions. R. C. was supported by  Fondecyt Proyecto 11181072  and  CONICYT   REDES  Project No. 180151. CM acknowledges  CONACYT (Mexico)  for financial support through project No. A1-S-15528  and the  CONICYT   REDES  Project No. 180151  which partially supported him. CM also wishes to thank the CIMFAV for their wonderful hospitality and facilities during his stay. B. C. and R. C. acknowledge the  INRIA program  "Associated teams" for funding part of the project via the associated team.
Existential state At the level of an individual organism, a fundamental measure of existence is to be alive or not (i. e. dead or unborn). We will therefore begin by demonstrating that one of the oldest and most popular inferential tools in wildlife ecology, the Cormack-Jolly-Seber (CJS) model of survival  (Williams et al. , 2002) , is a special case of an HMM. The CJS model estimates survival probabilities (φ) from capture-recapture data. Capture-recapture data consist of n sequences of encounter histories for marked individuals collected through time, where for each individual the observed data are represented as a binary series of ones and zeros. For the CJS model, X t = 1 indicates a marked individual was alive and detected at time t, while X t = 0 indicates non-detection. Marked individuals can either be alive or dead at time t, but the "alive" state is only partially observable and the "dead" state is completely unobservable.
In this example, the HMM can be regarded as a clustering scheme which maps observed input data to unobserved underlying classes with biological interpretations roughly corresponding to "resting", "minimal activity" (e. g. preening), "moderate activity" (e. g. walking, digging), and "flying". Complete details of this analysis, including each step of the workflow and example R (R Core Team, 2019) code, can be found in the Supplementary Tutorial.
Four behavioural modes were identified and biologically interpreted to be associated with resting (yellow), minimal activity (orange), moderate activity (blue), and flying (green). 2009;  Rakhimberdiev et al. , 2015). Although too technical to be described in detail here, geolocation HMMs can be particularly useful for inferring individual location from archival tag data  (Basson et al. , 2016). These HMMs have even been extended to include stateswitching behaviours such as those described in Section 3. 1. 2 (Pedersen et al. , 2008 (Pedersen et al. , , 2011b)). Animal movement behaviour HMMs have also been extended to accomodate partially-observed location data common to marine mammal satellite telemetry studies  (Jonsen et al. , 2005; McClintock et al. , 2012).
Software Recent advances in computing power and user-friendly software have made the implementation of HMMs much more feasible for practitioners. However, the features and capabilities of the software are varied, and it can be challenging to determine which software may be most appropriate for a specific objective. We briefly describe some of the HMM software currently available, limiting our treatment to freely available R (R Core Team, 2019) packages and standalone programs that we believe are most accessible to ecologists and non-statisticians. While most HMM packages in R include data simulation, parameter estimation, and state decoding for an arbitrary number of system states, they differ in many key respects (Table  2 ). Some of the more general packages provide greater flexibility for specifying state-dependent probability distributions  (Visser & Speekenbrink, 2010; Jackson, 2011; Harte, 2017; McClintock & Michelot, 2018).
One of the earliest and most flexible HMM packages, depmixS4  (Visser & Speekenbrink, 2010) , can accommodate multivariate HMMs, multiple observation sequences, parameter covariates, parameter constraints, and missing observations. Similar to depmixS4 in terms of features and flexibility, momentuHMM  (McClintock & Michelot, 2018)  can also be used to implement mixed HMMs  (DeRuiter et al. , 2017) , hierarchical HMMs (Leos-  Barajas et al. , 2017a; Adam et al. , 2019a) , zero-inflated probability distributions  (Martin et al. , 2005) , and partially-observed state sequences. In addition to the R packages presented in Table  2 , there are numerous R and stand-alone software packages that are less general and specialise on particular HMM applications in ecology, as well as programs with which these types of models can be relatively easily implemented by users with minimal statistical programming experience (see Appendix B in Supplementary Material).
We limit our treatment to freely available R (R Core Team, 2019) packages and stand-alone programs that we believe are most accessible to ecologists and non-statisticians. The Comprehensive R Archive Network (https://cran. r-project. org) currently hosts 26 packages that include "hidden Markov" in their description. While most HMM packages in R include data simulation, parameter estimation, and state decoding for an arbitrary number of system states, they differ in many key respects (see Table  2  in main text). Most of the packages are focused on categorical sequence analysis and are therefore limited in the statedependent probability distributions that can be implemented  (Himmelmann, 2010; Bartolucci et al. , 2017; Helske & Helske, 2019; Wilkinson, 2019). However, some of the more general packages provide greater flexibility for specifying state-dependent probability distributions, including commonly used discrete (e.
, 2017; Adam et al. , 2019) , zero-inflated probability distributions  (Martin et al. , 2005) , and partially-observed state sequences. However, unlike depmixS4 and other packages such as mhsmm  (O'Connell & Højsgaard, 2011)  and HiddenMarkov  (Harte, 2017) , momentuHMM does not currently support custom-coded state-dependent probability distributions. To our knowledge, only hsmm  (Bulla & Bulla, 2013)  and mhsmm (O'Connell & Højsgaard, 2011) can currently implement hidden semi-Markov models  (Barbu & Limnios, 2009). Many R packages are less general and specialise on specific HMM applications within individual-or population-level ecology. The marked package  (Laake et al. , 2013)  implements many of the popular capture-recapture HMMs described in Section 3. 1. Packages that specialise in animal movement behaviour HMMs for telemetry data, such as those described in Section 3.
1. 2, include bsam  (Jonsen et al. , 2005) , moveHMM  (Michelot et al. , 2016), and momentuHMM (McClintock & Michelot, 2018). The package HMMoce  (Braun et al. , 2018)  is specifically catered for HMMs that infer location from archival tag data (e. g. light levels, depth-temperature profiles) such as those described in Section 3. 1. 3. Using telemetry and count data, kfdnm  (Schmidt et al. , 2015)  can fit HMMs for population abundance and related demographic parameters such as those described in Section 3. 2. The package DDD  (Etienne & Haegeman, 2019)  implements HMMs for macroevolutionary inference about diversification rates from phylogenetic trees such as those described in Section 3. 2. 2. The package openpopscr  (Glennie et al. , 2019)  can fit spatial capture-recapture HMMs that account for unobserved animal movements when estimating population-level density and survival, such as those described in Section 3.
, 2011). Program PRESENCE  (Hines, 2006)  has many of the features of MARK and E-SURGE but focuses solely on presence-absence data, including models for species co-occurrence dynamics  (MacKenzie et al. , 2018). The RMark (Laake, 2013) and RPresence (MacKenzie & Hines, 2018) packages have been developed as R interfaces for Programs MARK and PRESENCE, respectively. Although not intended specifically for HMMs, it is worth noting that there are a number of software programs with which these types of models can be relatively easily implemented by users with minimal statistical programming experience. For Bayesian inference using MCMC sampling  (Gelman et al. , 2004) , these include WinBUGS/OpenBUGS  (Lunn et al. , 2009; Kéry & Schaub, 2011; Lunn et al. , 2012) , JAGS  (Plummer, 2003) , and Stan  (Gelman et al.