dataesr/scientific-paragraphs-categorization huggingface dataset:
Specifically, the weight of each study was taken as the ratio between the experimental group's sample size and the number of behavioral scales available. We also ran the analysis using the OLS method to assess the impact of the weights on the results. The second linear method applied was the LASSO, which naturally incorporates variable selection into the linear model thanks to ℓ 1 -norm applied on the coefficients. A coefficient not set to zero means that the associated factor has an influence on NFB efficacy and its sign indicates the direction of the effect. The last method used to determine factors influencing NFB was the decision tree (85), a hierarchical and non-linear method. This breaks down a dataset into smaller and smaller subsets using, at each iteration, a variable and a threshold chosen to optimize a simple Mean Square Error (MSE) criterion  (86).
Factors Influencing Neurofeedback Description and analysis of different types of NFB implementation were subject to several studies  (1, 54, (59) (60) (61) (62). However, to the best of our knowledge, none of these studies has implemented a systematic multivariate approach to associate factors to clinical endpoints therefore exposing their univariate analysis to a greater extend to the presence of a confounding factor. Two observations were detected as outliers and so removed from the dataset before performing the SAOB: Bazanova et al. 's (51) individualized NFB and individualized NFB coupled with EMG-Biofeedback groups. Indeed, these two groups presented very large within-ES (-3. 41 and -3. 95), even bigger than those reported in the literature on medication treating ADHD  (109). These large values broke our working hypothesis, so in order to be able to conclude on the results obtained by the SAOB, an outlier rejection was implemented.
Conceivably, the temporal threshold of efficacy was passed for all included studies, making the identification of this factor by the three methods unlikely on this dataset. However, the two methods that identified this factor both agreed that as expected, the more sessions performed, the more efficient the NFB tends to be. Interestingly, Minder et al. (110)  suggests that the subject location of the NFB training may also be an important contributory factor to clinical effectiveness. However, this has been challenged by a recent study  (110)  showing that performing NFB at school or at the clinic has no significant impact on treatment response. The type of NFB protocol was not identified by any method, and did not appear to influence the NFB results. This minimal importance granted by the methods to the NFB protocols is counter-intuitive given the centrality of the protocols in the neurophysiological mode of action and subsequent expected impact on therapeutic effectiveness  (61).
ACKNOWLEDGMENTS We would like to thank  Jean-Arthur Micoulaud-Franchi, MD, PhD  for his helpful comments and ideas stimulating this work. We also want to thank  Samuele Cortese, MD, PhD  for his help while replicating his meta-analysis. Finally, we also would like to thank the  Association Nationale Recherche et Technologie (ANRT)  for partially funding the PhD thesis of the first author as well as the  EU H2020  NEWROFEED grant number  684809.
Future use The Wines of Portugal best practice manual can serve for more than just the application of guidelines and procedures during a winery visit. It provides ViniPortugal with the opportunity to create a committee or an assembly of wine and wine tourism professionals dedicated to improving and enhancing their wine tourism offer. Regular and ongoing meetings to discuss wine tourism issues, strategies, and report back on the application of best practice guidelines would give further confidence in the recommendations outlined in the manual and create a supportive community dedicated to making Portugal one of the best wine tourism destinations. A best practice manual can also provide the framework for workshops and seminars, or similar to the "Enotourism Technician for the Sherry Wine Region", an educational workshop offered by the Sherry Council, ViniPortugal could adapt the material presented in the best practice manual to design a comprehensive training program and/or professional certification.
We present here Nested_fit, a Bayesian data analysis code developed for investigations of atomic spectra and other physical data. It is based on the nested sampling algorithm with the implementation of an upgraded lawn mower robot method for finding new live points. For a given data set and a chosen model, the program provides the Bayesian evidence, for the comparison of different hypotheses/models, and the different parameter probability distributions. A large database of spectral profiles is already available (Gaussian, Lorentz, Voigt, Log-normal, etc. ) and additional ones can easily added. It is written in Fortran, for an optimized parallel computation, and it is accompanied by a Python library for the results visualization.
I. INTRODUCTION Nested_fit is a general purpose parallelized data analysis code for the evaluation of Bayesian evidence and parameter probability distributions for given data sets and modeling function. The computation of the Bayesian evidence is based on the nested sampling algorithm  [1] [2] [3] , for the integration of the likelihood function over the parameter space. This integration is obtained reducing the J-dimensional volume (where J is the number of parameters) in a one-dimensional integral by a clever exploration of the parameter space. In Nested_fit, this exploration is obtained with a search algorithm for new parameter values called lawn mower robot, which has been initially developed by L. Simons  [4]  and modified here for a better exploration of multimodal problems. Nested_fit has been developed over the past years to analyze several sets of experimental data from, mainly, atomic physics experiments.
II. GENERAL STRUCTURE OF THE PROGRAM The general structure of the program is represented in Fig. 1. The main input files are two: the file nf_input. dat, where all computation input parameters are included, and the data file, which name is indicated in the parameter input file. The function name in the input file indicates the model to be used for the calculation of the likelihood function. Several functions are already defined in the function library for modelling spectral lines: Gaussian, Log-normal, Lorentzian, Voigt (Gaussian and Lorentzian convolution), Gaussian convoluted with an exponential (for asymmetric peaks), etc. Additional functions can be easily defined by the users in the dedicated routine (USERFNC). Differently from the version presented in Ref. [5]  (V. 0. 7), in the new version discussed here (V.
2. 2) non-analytical or simulated profile models can be implemented. In this case, one or more additional files have to be provided by the users. These external data, which can have some noise like the case of simulated data, are interpolated by B-splines using FITPACK routines  [6]. The B-spline parameters are stored and used as profile/model with the total amplitude and a possible offset as free parameters. An additional feature of this new program version, is the possibility to analyze data with error bars. This option has to be indicated in the input file. Several data sets can be analyzed at the same time by selecting the option "set of data: YES". This is particularly important for the correct study of physically correlated spectra at the same time, eg. background and signal-plus-background spectra.
This is done using a global user-defined function with common parameters of specific models for each spectrum. In the case of multiple data files, the program read an additional input parameter file nf_input_set. dat for the additional datafile names to analyze and data ranges to consider. The exploration of the parameter space and the corresponding evaluation of the likelihood function is done implementing the nested sampling algorithm  [1] [2] [3]. If the data are in the format (channel, counts), a Poisson distribution for each channel is assumed for the likelihood function. If the data has error bars (channel, y, δy), a Gaussian distribution is assumed (new feature in V. 2. 2). The main analysis results are summarized in the output file nf_output_res. dat. Here the details of the computation (n.
e. with the presence of distinguished regions of the parameter space with high values of the likelihood function. Other search algorithms are based on Markov chain Monte Carlo (MCMC) methods  [10]  and the recent Galilean Monte Carlo  [11, 12] , particularly adapted to explore the regions close to the boundary of V L>Lm volumes. Nested_fit program is based on an improved version of the lawn mower robot method, originally developed by L. Simons  [4]  and presented in details in the next section.
V. AN APPLICATION TO LOW-STATISTICS DATA To show the capabilities of Nested_fit, we present in this section its implementation on a particular critical case corresponding to a debated experiment. In 2008 it was observed an unexpected modulation in the two-body electron capture decay of single H-like 142 61 Pm ions to the stable 142 60 Nd bare nucleus, with a monochromatic electron-neutrino emission  [13]. The same modulation frequency, but with much smaller amplitude, was found in 2010 data  [14]  but not in the latest campaign in 2014  [15]  where much more events have been recorded. The unstable ions are produced by collision with a solid target and then injected in a storage ring where they are cooled down. In the storage ring, the decay time of single ions is measured from changes of the Schottky noise frequency induced by the ion revolution.
The difficulties to deal with these multiple likelihood maxima pushed in fact the creation of the improved lawn mower robot algorithm. In Fig. 3  (top) we present the collected data together with the exponential and modulated exponential functions corresponding to the most probable parameter set. The output result from Nested_fit are presented in Tab. I where model 1 and 2 represent the absence of presence of modulation. For each model, values of the evidence, Bayesian complexity and extracted information are provided, as well as model probabilities. The uncertainty of the probabilities is related to the uncertainty of the evidence. As example of probability distribution, we present in Fig. 3  (bottom) the joint probability of the amplitude a and pulsation ω of the modulation in model 2. The 2D histogram (obtained with Python nested_res.
VI. CONCLUSIONS We presented here the program Nested_fit, a general purpose parallelized data analysis code for the evaluation of Bayesian evidence and other statistically relevant outputs. It uses the nested sampling method with the implementation of the improved lawn mower robot algorithm for the evaluation of the Bayesian evidence. Nested_fit has been developed over the past years for the analysis of several sets of atomic experimental data that strongly contribute to the code evolution. We would like to mention in particular the analysis of low-statistics X-ray spectra of He-like uranium  [5, 17] , X-ray spectra of pionic atoms  [18, 19] , electron photoemission spectra from nano-particles  [20, 21] , single-ion decay spectra  [15]  and response function of crystal X-ray spectrometers (in progress).
Motivation: Most currently available text mining tools share two characteristics that make them less than optimal for use by biomedical researchers: they require extensive specialist skills in natural language processing and they were built on the assumption that they should optimize global performance metrics on representative datasets. This is a problem because most end-users are not natural language processing specialists and because biomedical researchers often care less about global metrics like F-measure or representative datasets than they do about more granular metrics such as precision and recall on their own specialized datasets. Thus, there are fundamental mismatches between the assumptions of much text mining work and the preferences of potential end-users. Results: This article introduces the concept of Agile text mining, and presents the PubAnnotation ecosystem as an example implementation.
The system approaches the problems from two perspectives: it allows the reformulation of text mining by biomedical researchers from the task of assembling a complete system to the task of retrieving warehoused annotations, and it makes it possible to do very targeted customization of the pre-existing system to address specific end-user requirements. Two use cases are presented: assisted curation of the GlycoEpitope database, and assessing coverage in the literature of pre-eclampsia-associated genes. Availability and implementation: The three tools that make up the ecosystem, PubAnnotation, PubDictionaries and TextAE are publicly available as web services, and also as open source projects. The dictionaries and the annotation datasets associated with the use cases are all publicly available through PubDictionaries and PubAnnotation, respectively.
Introduction These are exciting times for biomedical research, and that excitement is reflected in the volume of publications available in PubMed/ MEDLINE (http://www. pubmed. gov/), the US National Library of Medicine's database of biomedical publications-at the time of writing, contained 28 million articles. But, that very volume creates a problem for researchers: staying on top of relevant literature is important, as is targeted reading in order to interpret results and evaluate potential follow-ups to experimental findings. Unfortunately, the enormous volume of publications can make digesting even the most relevant literature in a typical scientist's field essentially impossible. Consider e. g. (Alper et al. , 2004 ) calculated that physicians trained in epidemiology would need an estimated 627.
, 2016; Rak et al. , 2012 Rak et al. , , 2014;; Roeder et al. , 2010; Yoshinobu et al. , 2009 Yoshinobu et al. , , 2011)). Other families of approaches have focused on usability, building interfaces that are meant to provide graphical user interface-like tools for constructing a conceptually clear-cut text mining pipeline  (Condie and Urbanski, 2014; Garten and Altman, 2009; Mu ¨ller et al. , 2004). Although the uptake of tools like Textpresso within the model organism database community (see the previous citations) has been impressive, adoption of these tools and the associated text mining technology by bench scientists has been negligible-it is notable that usability issues for text mining tools were pointed out as early as ten years ago  (Altman et al. , 2008). All of these approaches have something in common: they focus on addressing the problem of end-user access to text mining by making it easier for users to interact with the tools.
The PubAnnotation ecosystem: design principles PubAnnotation supports an Agile approach to text mining by instantiating software components that allow for decomposed parallel development, while also facilitating continuous integration. Three components in particular are used to implement the iterations of Agile development: 1. A manual annotation tool allows for the specification of the language processing version of a user story: an annotation model. 2. A dictionary-based annotator allows push-button evaluation. 3. A storage component facilitates regression testing. Thus, the iterative development/testing cycle of an Agile text mining sprint can be carried out as follows: (i) modify a dictionary, (ii) manually reannotate, (iii) automatically reannotate and benchmark. Additionally, for cases where no gold standard dataset is available for benchmarking, PubAnnotation allows random sampling of the automatically annotated text for manual evaluation.
g. databases or ontologies). PubAnnotation (http://pubannotation. org) is originally implemented as a persistent and shareable storage of corpus annotation  (Kim and Wang, 2012). Since then, its functionality is substantially extended to facilitate production of annotations, and now it supports following tasks which are required to produce high-quality annotations: Preparation of target texts, Automatic annotation, Manual correction and Searching annotation. To simplify the task of preparing documents for annotation, PubAnnotation implements text sequencers, which process documents to make them ready for annotation. Concretely, a text sequencer turns a document into a sequence of characters, so that positions in the document can be specified unambiguously by character offsets. Currently, PubAnnotation has two sequencers for PubMed and PMC.
). PubAnnotation will then send all the texts within the project to PubDictionaries, either sequentially or in a batch mode, and store the results within the project. The back-end database of PubAnnotation, implemented on PostgreSQL, facilitates a stable and scalable storage for annotations. (The URL can be easily obtained from the web site of PubDictionaries. ) After a set of annotations are produced through an automatic annotation process, the next step would be to evaluate the quality to determine if it is good enough or if it needs improvement. In an ideal situation, there may be a benchmark dataset (a. k. a. gold standard or gold annotation) prepared for evaluation of the annotation. If this is the case, the benchmark dataset can be registered to PubAnnotation as a project and compared with any other project.
PubAnnotation offers an easy interface to do so, and the result is reported in recall, precision and F1-score. More often, however, there may be no benchmark dataset. In the case, a set of annotations may be evaluated through manual inspection. PubAnnotation provides several tools to ease this task, most notably the SPARQL-based search interface, as well as standard sorting and a simple keyword-based search interface powered by Lucene indexing (http://lucene. apache. org/). To enable the SPARQL-based search interface over a project, the annotations in the project need to be converted to resource description framework (RDF) statements (https://www. w3. org/RDF/), which is a one-click process on PubAnnotation. Once the RDFization is done, SPARQL (https://www. w3. org/TR/sparql11-overview/) queries may be used for searching specific patterns over the annotations, even across multiple projects (e.
In this case, the improvement may be replicated to future (dictionary-based) annotations. In the case of annotation revision, the improvement will not be replicated, but the revised annotation may be regarded as the correct one, and this option may be chosen to develop a benchmark dataset of the annotation for future evaluation. As discussed in the beginning of this section, PubAnnotation works as a public repository of annotations, as well as a platform for annotation development. So, annotations developed on PubAnnotation can be shared. To date, PubAnnotation contains almost 200 projects, making it one of the largest repositories of publicly available biomedical text annotations. In 2018, on average, the repository was accessed by 832 unique users per month.
Additionally, an instance of TextAE can be embedded in a HTML document as a div element. Figure  2  shows three TextAE instances embedded in one document. TextAE works as a REST client to get the annotation from a REST server. So, while annotations are stored in dedicated databases like PubAnnotation, specific portions of them may be freely included in documents in the web. Note that TextAE works not only as a visualizer but also as an editor, so that annotations can be easily edited within the generated The SPARQL search interface of PubAnnotation, which was discussed in Section 2. 5, uses TextAE to display the results. In case the results of a search are annotations, they are shown in TextAE, so that users can immediately read or sometimes edit them, when necessary. Figure  2  shows a part of the results, 3 out of 410, of the SPARQL query shown in Figure  3.
Use Case 1: GlycoEpitope database curation Glycan analysis has been an important topic of focus within a wide range of life science research and biotechnology. Glycans are carbohydrate sugar chains identified as part of every cell type; they interact with various proteins, viruses, bacteria and antibodies. They are often found on the cell surface, serving as 'switches' in toggling various cellular functions  (Hakomori, 1984). A large number of polyclonal or monoclonal antibodies have been used to analyze their expression and function. The GlycoEpitope database integrates a variety of useful information on carbohydrate antigens and their antibodies  (Kawasaki et al. , 2006). To aid in the curation of locational information for glycan epitopes, the PubAnnotation ecosystem was used. First, an annotation dataset, GlycoBiology-MAT, was produced in following way: 1.
5. The dictionary was revised to remove the FPs and FNs found at Step 4, and the process went back to Step 3 for re-annotation. The loop was continued until the user was satisfied with the annotation. The final dictionary was ended up with 559 entries (The dictionary is available at http://pubdictionaries. org/diction aries/MAT. ), which was then used to produce the final annotation dataset with 5241 annotation instances. Another dictionary, GlycoEpitope, was created using the entries from the GlycoEpitope database. It was then used to produce an annotated dataset, GlycoBiology-Epitope. The two datasets were examined using the SPARQL interface of PubAnnotation. In Figure  4 , the white bars show the statistics on probable anatomical locations of epitopes indicated by the annotated dataset.
To accomplish this, we first Fig. 3. A pre-defined SPARQL template for the Preeclampsia project. SPARQL is powerful for searching across datasets and annotation categories, but difficult to learn. PubAnnotation's support for pre-defined templates allows uses to search for arbitrarily specific entities without learning SPARQL identified PE-associated DEGs (n ¼ 17 322) via meta-analysis of 12 experiments downloaded from the Gene Expression Omnibus. Then, we utilized the PubAnnotation ecosystem to help determine how many of the PE-associated DEGs had literature evidence. First, an annotation dataset, PE was produced, in following way: 1. A PubDictionaries dictionary, PE, was created, with the three entries, PE, pre-eclamptic toxemia and pre-eclamptic toxemia, which were curated from the PE ontology.
After the revision, the process went back to Step 3 for re-annotation. After a new term is added, the precision of the term was evaluated in a random sample. The loop was continued until the user was satisfied with the annotation. The final version of the dictionary was ended up with 22 entries, which was then used to produce the final annotation dataset with 68 708 annotation instances. In a similar way, a second dictionary, preeclampsia_genes, was created, using the PE-associated DEGs (names as labels with corresponding HGNC identifiers), which was then used to produce another annotation dataset, preeclampsia_genes, in PubAnnotation. Of the 17 322 PE-associated DEGs identified from the metaanalysis, 417 had literature evidence in at least 1 article in PubAnnotation. Of these 417 genes, PGF (n ¼ 216), TYK2 (n ¼ 203), VEGFC (n ¼ 169), VEGFA and VEGFB (n ¼ 167), INS (n ¼ 133) and ENG (n ¼ 130) were found to be annotated in the greatest number of articles.