dataesr/scientific-paragraphs-categorization huggingface dataset:
Relevant studies are added to the database as they are found, but the annual update to the systematic review is necessary to ensure completeness. A two-stage process is employed to screen records and identify relevant studies. Titles and abstracts are read to assess eligibility of studies for inclusion in the review (stage 1). Full texts of potentially relevant articles are obtained to assess for inclusion (stage 2). Studies are eligible for inclusion if they have applied methodology for determining which outcomes or outcome domains should be measured in clinical trials or other forms of health research. Relevant studies therefore describe the development of a COS, regardless of any restrictions by age, health condition or setting. The inclusion and exclusion criteria are described in more detail in the original systematic review  (8).
We encounter challenges in undertaking this comprehensive approach, such as the variability in free text terms and index terms used for COS development, further confounded by the absence of a specific index term or Medical Subject Heading (MeSH) main heading for this study type  (9). The term 'core outcome set' has not been commonly used until recently, and is still not consistently applied with many variations employed to describe this type of study (e. g. core domain set, core measurement set, minimum outcome set), and they do not appear to be categorized consistently across different databases. Furthermore, no single database specializes in this type of methodological research and it is likely to be found across a wide range of literature. Finally, the search is not limited by condition or disease, setting, study type or intervention.
A wide range of methods have been proposed for this kind of screening prioritization, including Support Vector Machines, Naive Bayes, Voting Perceptrons, Lambda-Mart, Decision Trees, EvolutionalSVM, Waode, kNN, Rocchia, hypernym relations, ontologies, Generalized Linear Models, Convolutional Neural Networks, Gradient Boosting Machines, Random Indexing and Random Forests  (14, 16, 21, 24). Several screening prioritization systems are publicly available, including Eppi-Reviewer, Abstrackr, Swift-Review, Rayyan, Colandr and RobotAnalyst  (13, 16, 23, 25, 27). Comparing the relative performance of different methods is difficult since most methods have been evaluated on different datasets, under different settings, and with different metrics. There have been attempts to compare previous methods by replicating reported methods on the same datasets, but the replication of published methods is often difficult or impossible due to insufficient reporting  (20).
Stopping after screening 25% of the candidate references in the retrospective evaluation would have identified 317 out of the 324 deduplicated references with abstracts in the four review updates (97. 7%) (Figure  4 ). In the dataset, 3840 out of 45 602 articles lacked abstracts (8. 4%), of which 8 were included in the systematic review. The performance of the model was unreliable when evaluated on these. The area under the curves were visibly lower (Figure  5 ). Correspondingly, stopping after screening 25% of references would only have identified 2/6 references in the prospective evaluation, and 22/29 references (75. 9%) in the retrospective evaluation. The model performed substantially worse when evaluating on only titles (Figure  5 ). A model trained using all prior references, but trained and evaluated only on titles would on average have identified 86% of the relevant references after screening 25% of the candidates (Figure  5 , bottom).
A model trained on titles and abstracts from all prior references, but evaluated only on titles would on average have identified 90% of the relevant references after screening 25% of the candidates (Figure  5 , bottom). Using a more conservative threshold would not have helpedseveral of the relevant references were identified only at the end of the simulated screening. However, only 3840 out of 45 602 references in the dataset lacked abstracts (8. 4%). These references constitute <300 references in each review, corresponding to a workload of <5 hours of screening per reviewer.
However, 97. 8% of articles (317/324) could still be identified retrospectively and 97. 7% of articles (126/129) could be identified prospectively if a different position was selected for the cut-off point for screening. Other methods of identifying relevant studies are employed in the update of the systematic review of COS, such as handsearching, reference checking, relevant database alerts for key words and references, as well as checking with known experts. These other methods of identifying relevant papers increase the likelihood that all eligible studies will continue to be identified, and mean that a balance can be struck between managing the workload and identifying all eligible studies. The results of this study showed that the screening automation can be reliable, provided both titles and abstracts are available, but that the automated ranking cannot reliably identify included references based only on titles.
Conclusions Based on the results in this study we determined that stopping after screening the first 25% of the candidate studies would result in a loss of roughly 2% of the relevant studies, which we deemed an acceptable trade-off in this systematic review. However, the same stopping criterion would have resulted in a loss of over 10% of the relevant studies without abstracts. Balancing the risk of missing relevant references against the limited number of such references, we opted to screen all references without abstracts manually. We are currently using this system based on logistic regression to identify Core Outcome Sets published in 2018 for the fifth update of the Comet database. The database searches were performed in March 2019 and the screening is currently ongoing. The prospective use of these methods will further validate these results and this model of automated screening.
Mesophotic coral ecosystems (MCEs) and temperate mesophotic ecosystems (TMEs) occur at depths of roughly 30-150 m depth and are characterized by the presence of photosynthetic organisms despite reduced light availability. Exploration of these ecosystems dates back several decades, but our knowledge remained extremely limited until about a decade ago, when a renewed interest resulted in the establishment of a rapidly growing research community. Here, we present the 'mesophotic. org' database, a comprehensive and curated repository of scientific literature on mesophotic ecosystems. Through both manually curated and automatically extracted metadata, the repository facilitates rapid retrieval of available information about particular topics (e. g. taxa or geographic regions), exploration of spatial/temporal trends in research and identification of knowledge gaps.
The repository has since progressed to form the current iteration of 'mesophotic. org' (http://mesophotic. org): a comprehensive and curated repository of scientific literature on MCEs and TMEs (hereafter referred to as 'mesophotic ecosystems'). The main goals of this repository are to (i) provide a comprehensive and dynamic record of all the published scientific information on these ecosystems and make it queryable through a single web portal, (ii) allow for the exploration of spatial/temporal trends and identification of knowledge gaps through curated metadata for each publication and (iii) act as an institution-independent platform and as a shared resource for and by the research community to accelerate our understanding regarding the occurrence, composition and functioning of these ecosystems.
Database information and usage The current release provides a comprehensive collection of peer-reviewed, scientific articles up to the end of 2018 and incorporates efforts from several other systematic literature reviews  (6, 10, 29). The primary contents of 'mesophotic. org' are therefore currently peer-reviewed publications, although the proportion of secondary scientific literature including conference proceedings, technical reports, museum publications, book chapters and (post)graduate theses is growing (as we continue to add these). Each catalogued publication is first identified as 'mesophotic', depending on whether it presents data from or a discussion on the mesophotic depth zone (defined by the most commonly considered depth range of 30-150 m depth; 30) or whether it is relevant despite not occurring in this depth range.
Next, where relevant, the publication is tagged either as 'mesophotic coral ecosystem' or 'temperate mesophotic ecosystem' to facilitate separate searches and summaries for these distinct systems. The publications are identified by type (scientific, technical and popular), format (article, review, report, chapter, book and thesis) and whether they present original data and/or report new species. In addition to standard citation information, additional metadata are manually extracted from each publication (an up-to-date overview of fields and categories can be found on http://mesophotic. org/metadata), including the minimum and maximum study depth, the geographic location(s) (each defined by a latitude and longitude), the research platform(s) (e. g. 'rebreather'), the research field(s) (e. g.
'physiology') and the focal taxa (e. g. 'Scleractinia'). The abstract and full text are automatically indexed to extract the occurrence (frequency) of custom keywords (e. g. 'mesophotic'), species known to occur at mesophotic depths (through a curated list: http://mesophotic. org/species) and metadata categories (as mentioned above). The article is also summarized through a 'word cloud' (eliminating undesired stop words through a custom 'stop list'). This also allows keywords in the abstract and word cloud to be hyperlinked to other pages on the website. Full-text contents are stored in the database for indexing and searching purposes but are not accessible to comply with copyright regulations. An advanced search function is available to search across publications and their associated metadata, and the results can be browsed through the website or downloaded as a commaseparated values (CSV) file.
If a search term is queried, publications are ordered by the number of occurrences of that term in the full text, which can be helpful in identifying the most relevant references. In addition, each metadata field has a separate entry page to summarize literature by category (e. g. geographic region, research platform or focus taxon). New publications are added by the content editors but can be suggested (as well as any corrections) by any website visitor through email (info@mesophotic. org). After extracted metadata are entered, they are considered 'validated' when verified by at least two different content editors using an internal validation system. If a content editor changes any of the metadata, the system will expire existing validations prompting the other content editors for revalidation.
In addition to scientific publications, the website maintains a moderated list of people working on mesophotic ecosystems, a gallery of member photos (with the option of sharing photos for reuse with the Creative Commons License; CC BY-NC 4. 0) and a regular blog featuring scientists and their publications in 'Behind the Science' posts. The current release of all the validated metadata associated with publications can be downloaded in CSV format from anywhere in the 'mesophotic. org' repository. Data can be directly imported into R for analysis by using the 'read. csv' option (www. r-project. org) and pointing to the URL of the dynamically generated CSV files on the website (e. g. http://mesophotic. org/publications. csv). Alternatively, CSV files of specific queries can be downloaded, saved and imported in the same way.
An electronic notebook with usage examples, including reproductions of the figures included with this manuscript, can be found on http:// mesophotic. org/tutorial. Metadata can also be explored through the Statistics page (http://www. mesophotic. org/ stats), to provide general summaries of the data, such as the number of publications over time (Figure  1 ) and over depth (Figure  2 ) and the most commonly studied focal taxa, research fields, geographic locations or research platforms (Figure  3 ). a wide range of Ruby gems and JavaScript packages that are detailed in the README file of the code repository. The database structure is visualized through an Entity Relationship Diagram (ERD; Figure  4 ), with the most recent version also accessible through the README file. Issues and bugs can be reported through the https://github.
We provide an automatic classification method of the three near-Earth regions, the magnetosphere, the magnetosheath and the solar wind in the streaming in-situ data measurement that outperforms the previous methods of automatic region classification. The method was used to identify 14186 magnetopause crossings and 16192 bow shock crossings in the data of 10 different spacecrafts of the THEMIS, ARTEMIS, Cluster and Double Star missions and for a total of 79 cumulated years. These multi-missions catalogs are non ambiguous and can be automatically enlarged with the increasing quantity of data and their elaboration paves the way for additional massive statistical analysis of the two near-Earth boundaries. The development of these algorithms is a promising step towards their usage for the onboard selection of data of interest.
org/10. 5194/angeo-2019-149 Preprint. Discussion started: 5 November 2019 c Author(s) 2019. CC BY 4. 0 License. spacecraft coupled with the solar wind conditions provided by WIND,  Jelínek et al. (2012)  established a threshold based method to identify the three near-Earth regions and eventually build crossings lists from this classification but the efficiency of such a method is unknown and it is not sure how this method could be adapted to the data of other missions especially the one having a non-equatorial or nightside orbit. Additionally, finding the perfect set of features and the optimal associated thresholds can appear to also be time-consuming and not flexible enough with regard to the large variability expected from the data. A way to improve this solution would stand in the use of supervised machine learning algorithms that have the advantage of rapidly finding the intrinsic differences between different labeled points in a dataset.
The use of these algorithms to classify time series into several categories is not new in the field of space physics. They have especially made their proof to classify the solar wind into several categories  (Camporeale et al. , 2017)  or to determine if an interval of data actually did contained a Flux Transfer Event (FTE)  (Karimabadi et al. , 2009). In this paper, we use the magnetic field and plasma moments of the THEMIS mission to establish a fast and reproducible automatic detection of the three near-Earth regions. After a presentation of the data and the algorithm we have been using, we adapt the method to the data of 3 different missions: Cluster, Double Star and Artemis. We compare the performance of our method to a threshold based method for the different missions. We finally use our method to automatically elaborate boundary crossings catalogs.
Data and label We used plasma moments and magnetic field data from the THEMIS B spacecraft for the whole mission period (between April 2007 and October 2009) during the dayside, dawn and dusk operation phases. The magnetic field data were provided by the Fluxgate Magnetometer  (Auster et al. , 2008)  with a temporal resolution of 3s. Concerning the plasma moments, we used the reduced mode of the data provided by the electrostatic analyzer  (ESA, McFadden et al. (2008) ). The gaps in the data were filled with the plasma onboard moments  (MOM, McFadden et al. (2008) ) and the full mode of ESA. The remaining gaps were filled by time interpolation in order to have regular streaming data through the whole 2007-2009 period. Due to the important differences existing between the different missions in the specificities of the distribution functions and particle energy or pitch angle spectrograms, we chose not to consider those products and focused on moments only.
The obtained dataset then consists in 8 input variables: the ion bulk velocity components, V x , V y , V z , the magnetic field and its components, B x , B y , B z , the ion density N p and the temperature T. The data was then resampled to a 1 minute resolution. Each time step was then associated with a label indicating the region in which spacecraft was: 0 for the magnetosphere, 1 for the magnetosheath and 2 for the solar wind. To avoid manual labelling of each regions over the whole dataset, we initiated our process with 2 hours of each regions. We then increased our dataset and our labels with the successive, eventually corrected, predictions made by our algorithm on our training set. The typical labelling of the three regions is shown on the last subplot of Figure  1  where the theoretical label, shown in blue has been shifted for visual purpose.
Following this process, our dataset is made of 59798 points of magnetosphere, 48056 points of magnetosheath and 150415 points of solar wind. We selected data within dawn, dayside and dusk operation phases of THEMIS and thus expect a good coverage of both magnetopause and shock surfaces. This is confirmed by the actual spatial coverage of our labelled dataset shown in Figure  2. We then expect the method to be robust enough to the variability one can find in the data through the three different THEMIS operation phases. In the following, we will designate the subset that has been used to fit our algorithm by training set. We will designate by test set the remaining subset of data that is used to evaluate the performance of our model. For each of the configurations we have been testing our algorithm with, the training set represents 70% of the dataset while the test set represents the remaining 30% of the dataset.
Consequently, performing this split only partially shows the reproducibility of our method and how well it would perform on really unknown data. To ensure there were not any temporal overfit due to this split and that our method was truly reproducible, we trained and evaluated our model 3 times by splitting our dataset temporally instead of randomly. For this, we considered our training set to be a time interval that represented 2/3 of our dataset and left the remain to be the training set. As the average AUC we obtained in this case, also shown in Table  1 , has very few variations compared to the random split, this ensures the temporal independance of our method as well as our reproducibility. Last but not least, the greatest part of our label has been obtained after successive train and prediction of our model on the parts of the dataset that had not been labelled yet.
Thus, the label can eventually contain errors that could affect the quality of our prediction and high AUC would then not indicate the classification ability or our model but its ability to learn from an erroneous label. To figure this out, we led trainings and evaluations of the algorithm by voluntarily mislabelling 25% of our dataset. If our model completely followed the indicated label in the training set, we would once again expect a high AUC. This is not the case, as shown in Table  1. This shows the real capacity of our algorithm to classify the three near-earth regions as well as the reliability of our label. 3 Adaptability of the model: from a mission to the other Having developed an automatic detection method of the three near-earth regions with high reliability, one could think of its adaptation to the data provided by additional spacecraft that went through these regions.
Cluster In the case of Cluster, we used the available data from Cluster 1 between the 1st of January 2001 and the 1st of January 2013. The magnetic field data were provided by the Fluxgate Magnetometer with a temporal resolution of 4s  (Balogh et al. , 2001). The plasma moments were provided by the Hot Ion Analyzer instrument  (Rème et al. , 2001)  when the instrument was working under the magnetosphere or the magnetosheath mode. Following what we did for THEMIS, the data were resampled to a 1 minute resolution. A typical representation of such data is shown in Figure  4. To check out the adaptability of the model we trained in the previous section, we labelled 50277 points of magnetosphere, 76468 points of magnetosheath and 22017 of solar wind between the years 2005 and 2006. The spatial distribution of this labelled dataset is shown in the Appendix A.
Intuitively, one could think that the efficient model we have trained in the previous section on THEMIS would also perform well on Cluster data. Nevertheless, the lower AUC we have in Table  1  without retraining indicate the adaptability is not this obvious. This is mostly due to the spatial coverage of THEMIS that has a much more equatorial orbit than Cluster. The data provided by the two missions can therefore be substantially different and an algorithm that has only seen equatorial data could perform better on unseen polar data. To cope with it, we refitted the model trained on THEMIS with a training set we picked in our labelled Cluster data. The resulting model was then evaluated on the remaining test set and the operation was repeated for 10 different train-test split configurations in a similar way than what was done for THEMIS.
The spatial distribution of our labelled data is also shown in Appendix A. As TC1 also has an equatorial orbit, we expect the model trained on THEMIS to perform well even without having to be retrained and this is the main reason why we do not have an entire coverage of the (Y-Z) plane in the specific case of Double Star. We have this confirmation with the value of the AUC without retraining in Table  1. Refitting the model would then allow a finer detection that would be specific to the quality of the data provided by TC1 in comparison to the THEMIS data and this is indicated with the increasing AUC with a retraining of the model with our labelled data.
Artemis The mission ARTEMIS actually corresponds to the THEMIS B and C spacecrafts when they moved from a terrestrial to a lunar orbit at the end of the year 2009. The data we used in this case are then the one provided by the same instruments of THEMIS B than the one we mentioned in section 2 between the 1st of January 2010 and the 1st of June 2019. With the change of orbit, it is necessary to take into account the moments in the data for which the spacecraft will be in the moon's wake. This new region will then be considered as an additional possible region (and thus another label with the value 3) for the data. A typical representation of the data that includes a passage in the moon's wake is shown in Figure  6. As, in this case, the spacecraft spends the greatest part of its orbit in the solar wind, and because our dataset consists of a quasi complete solar cycle, we do here expect much more variability of the data specific to this class than what we could have had for the three previous missions.
To cope with it, we labelled a month per year of our dataset for a total of 26560 magnetosphere points, 131656 magnetosheath points, 429283 solar wind points and 15070 points of moon's wake which spatial distribution is shown in Appendix A. Additionally, the greatest part of the data here corresponds to the distant night side for which the transition from the magnetosheath to the solar wind is much less obvious than on the dayside and where small velocity, density and temperature variations could be wrongly interpreted as a boundary crossing. To cope with it, we then added the spacecraft GSM coordinates as a feature of the dataset which will then consist in 11 input variables. Having a different dataset and a different number of classes, it is here no use of using the model we trained in the previous section and we will then focus on the specific model we trained for this mission.