dataesr/scientific-paragraphs-categorization huggingface dataset:
Errata service Due to the experimental protocol and the inherent complexity of projects like CMIP6, it is important to record and track the reasons for dataset version changes. Proper handling of dataset errata information in the ESGF publication workflow has major implications for the quality of the metadata provided. Reasons for version changes should be documented and justified by explaining what was updated, retracted and/or removed. The publication of a new version of a dataset, as well as the retraction of a dataset version, has to be motivated and recorded. The key requirements of the errata service (https://errata. es-doc. org/, last access: 4 May 2020) are to provide timely information about newly discovered dataset issues (as errors cannot always entirely be eliminated, the errata service provides a centralized public interface to data users where data providers directly describe problems as and when they are discovered) provide information on known issues to users through a simple web interface.
Synda Synchronize Data (http://prodiguer. github. io/synda/, last access: 4 May 2020) (Synda) is a data discovery, download and replication management tool built and developed for and by the ESGF community. It is primarily aimed at managing the data synchronization between large data centres, though it could also be used by any CMIP user as a tool for data discovery and download. Given a set of search criteria, the Synda tool searches the metadata catalogues at nodes across the ESGF network and returns results to the user. The search process is highly modular and configurable to reflect the end user's requirements. Synda can also manage the data download process. Using a database to record data transfer information, Synda can optimize subsequent data downloads. For example, if a user has retrieved all data for a given variable, experiment and model combination and then a new search is performed to find all the data for the same variable and experiment but across all models, Synda will not attempt to retrieve data from the model it has already completed data transfers for.
Dashboard The ESGF also provides support for integrating, visualizing and reporting data downloads and data publication metrics across the whole federation. It aims to provide a better understanding about the amount of data and number of files downloaded -i. e. the most downloaded datasets, variables and models -as well as about the number of published datasets (overall and by project). During the data challenges as described in Sect. 5, it became apparent that the dashboard implementation in use at that time was not feasible for operational deployment. Since the close of the data challenges, work has continued on the dashboard development. A new implementation of the dashboard (https://github. com/ ESGF/esgf-dashboard/, last access: 11 September 2020) is now available and has been successfully deployed in production (http://esgf-ui.
cmcc. it/esgf-dashboard-ui/, last access: 4 May 2020). Figure  5  shows how the dashboard statistics are collected. The deployed metrics collection is based on industrystandard tools made by Elastic (https://www. elastic. co/, last access: 4 May 2020), namely Filebeat and Logstash. Filebeat is used for transferring log entries of data downloads from each ESGF data node. It sends log entries (after filtering out any sensitive information) to a central collector via a secured connection. Logstash collects the log entries from the different nodes, processes them and stores them in a database. Then, the analyser component extracts all the relevant information from the logs to produce the final aggregated statistics. The provided statistics fulfil the requirements gathered from the community on CMIP6 data download and data publication metrics.
Data challenge 1: publication testing The aim of the first phase was to verify that each participating site could complete tasks 1-5: 1. install (or update) the ESGF software stack 2. run quality control on primary data 3. publish primary data 4. publish replica data 5. verify search and download were functional. These tasks constitute the basic functionality of ESGF data publication, and it is therefore essential that these steps are able to be performed by all sites using the most recent ESGF software stack. In order for each ESGF site participating in this data challenge to test the publication procedure, a small amount (≈ 10 GB) of pseudo-CMIP6 data provided by modelling centres was circulated "offline", i. e. not using the traditional ESGF data replication methodology. These data were prepared specifically as test data for this phase of the data challenge.
The data contained pseudo-data prepared to look as if they had come from a few different modelling centres and were based on preliminary CMIP6 data. In normal operations ESGF nodes publish data from their national modelling centre(s) as "primary data"; these are also referred to as "master copies". Some larger ESGF sites act as primary nodes for smaller modelling centres outside their national boundaries. Once data records are published, other nodes around the federation are then able to discover these data and replicate them to their local site. This is typically done using the Synda replication tool (see Sect. 3. 3. 5). The replicating site then republishes the data as a "replica" copy. In this data challenge, using only pseudo-data, sites published data as a primary copy if the data had come from their national centre; otherwise they were published as a replica copy.
Data challenge 2: publication testing with an integrated system The aims of the second phase were for each participating site to complete steps 1-5 from phase 1 and additionally complete the following steps: 6. register data with the PID assignment service 7. verify citation service DOIs for published data 8. populate further_info_url through ES-DOC scanning. This second phase of the data challenge was expanded to include additional ESGF sites; additionally a larger subset (≈ 20 GB) of test data were available. Some of the test data were the same as in the first data challenge, though some of the new test data were early versions of real CMIP6 data. The increase in the volume and variety of data at each stage in the data challenges is essential to continue to test the software to its fullest extent.
Data challenge 3: publication with data replication The aims of the third phase were for each participating site to complete steps 1-8 from phases 1 and 2 and additionally complete the following steps: 9. replicate published data 10. apply the "test suite" (described below) 11. verify the metrics collection for the dashboard 12. register data errata with the errata service. The third data challenge introduced a step change in the complexity of the challenge with the introduction of step 9: "replicate published data". The replication of published data was done using Synda (see Sect. 3. 3. 5), mimicking the way in which operational CMIP6 data are replicated between ESGF sites. In the first two challenges, test data were published at individual sites in a stand-alone manner, where data were shared between sites outside the ESGF network, thus not fully emulating the actual CMIP6 data replication workflow.
This third phase of the data challenges was the first to test the full data replication workflow. The added complexity in this phase also meant that the phase took longer to complete; all sites had to first complete the publication of their primary data before they could search for and replicate the data to their local sites and then republish the replica copies. This was further complicated as the volume of test data (typically very early releases of CMIP6 data) was much larger than the previous phases, with approximately 400 GB of test data. Sites also tested whether they could register pseudo-errata with the errata service. As errata should be operationally registered by modelling centres, this step required contact with the modelling centres' registered users. An additional test was included in this phase which was to run the test suite (https://github.
Data challenge 4: full system publication, replication with republication and new version release The aims of the fourth phase were for each participating site to complete steps 1-12 from phases 1-3 and additionally complete the following steps: 13. retract a version of a dataset 14. publish a new version of a dataset. This data challenge was performed with approximately 1. 5 TB of test data. This increase in volume increased the complexity and the time taken for sites to publish their primary data and the time taken to complete the data replication step. This larger volume and variety of data (coming from a variety of different modelling centres) was essential to continue to test the publication and quality control software with as wide a variety of data as possible. The inclusion of the two new steps, the retraction of a dataset and the publication of a new version, completes the entire CMIP6 publication workflow.
When data are found to have errors, the data provider logs the error with the errata service and the primary and replica copies of the data are retracted. After a fix is applied and new data are released, they are published with a new version number by the primary data node; then new replica copies are taken by other sites, who then republish the updated replica versions. No issues were found in the retraction and publication of new versions of the test data. Not all sites participated in the deployment of the dashboard. Those that did participate provided feedback on its performance, which identified that a new logging component on the ESGF data node would be required for the dashboard to be able to scale with the CMIP6 data downloads. Given the difficulties in sites deploying the dashboard, it was recommended that it be optional until the new service was deployed.
Data challenge 5: full system publication, replication with republication and new version release on the real infrastructure The aims of the fifth phase were for each participating site to complete steps 1-14 (step 11, deployment of the dashboard, was not required) from phases 1-4 and additionally complete the following steps: 15. ensure homogeneity across the ESGF web user interface 16. move testing onto production environment (i. e. operational software, hardware and federation). The search service for CMIP6 is provided through the ESGF web user interface, which cannot be fully configured programmatically. For this reason, on sites that host search services the site administrators had to perform a small number of manual steps to produce an ESGF landing page that was near identical at all sites (with the exception of the site-specific information); this ensured consistency across the federation.
While this is a relatively trivial step, it was important to make sure that the end users would be presented with a consistent view of CMIP6 data search across the federation. Finally, but most importantly, all the steps of the data challenge were tested on the production hardware. In all of the previous four phases the data challenges were performed using a test ESGF federation, which meant that a separate set of servers at each site were used to run all the software installation, quality control, data publication and data transfer. This was done to identify and fix all the issues without interrupting the operational activity of the ESGF network. The ESGF continued to supply CMIP5 and other large programme data throughout the entire testing phase. In this final phase, the CMIP6 project was temporarily hidden from search, and all testing steps were performed within the operational environment.
Summary and conclusions In this paper the preparation of the infrastructure for the dissemination of CMIP6 data through the ESGF has been described. The ESGF is an international collaboration in which a number of sites publish CMIP data. The metadata records are replicated around the world, and a common search interface allows users to search for all CMIP6 data irrespective of where their search is initiated. During CMIP5 only an ESGF node installation and publication software were essential to publishing data. The ESGF software is far more complex for CMIP6 than for CMIP5, with the ESGF software required to interact with a software ecosystem of value-added services required for the delivery of CMIP6 -such as the errata service, the citation service, Synda and the dashboard -all being new in CMIP6.
The quality of service of the network of the nation where CMIP6 data node resides and the physical distance between the CMIP6 data node and the testing site are both factors that need to be considered when distributing the CMIP6 to end users. The distinctive nature of the ESGF can be seen by comparing it with other international data exchange projects. The SeaDataNet (https://www. seadatanet. org/, last access: 25 November 2020) project is a European Research Infrastructure, though it also serves a global community. The use of standards data which are specified in a machineinterpretable form gives them an efficient framework for enforcing standards using generic tools. This is not possible in the current ESGF system; however the data volumes handled by SeaDataNet are substantially smaller, and the degree of human engagement with each published dataset is consequently larger.
Data request: https://www. earthsystemcog. org/projects/wip/ CMIP6DataRequest/ (last access: 14 January 2021; Juckes et al. , 2020) -Citation service: http://cmip6cite. wdc-climate. de/ (last access: 14 January 2021; Stockhause and Lautenschlager, 2017) -Synda: http://prodiguer. github. io/synda/ (last access: 14 January 2021; Ben Nasser et al. , 2021b).
Acknowledgements. The authors would like to thank the two anonymous reviewers for their reviews and helpful comments and the members of the  WGCM Infrastructure Panel  for their support and assistance during the CMIP6 data challenges work. The authors also acknowledge the support of the CMIP programme, a project of the  World Climate Research Programme  co-sponsored by the  World Meteorological Organization , the  International Oceanographic Commission of UNESCO  and the  International Science Council. Ruth Petrie  would like to thank  Bryan Lawrence  for his guidance and comments on the manuscript. Financial support. This international collaborative work was funded through various agencies. Co-authors at  Lawrence Berkeley National Laboratory  were funded under contract no. DE-AC02-05CH11231 , and co-authors at  Lawrence Livermore National Laboratory  under contract  DE-AC52-07NA27344  with the  US Department of Energy.
European co-authors were supported by the  European Union Horizon   2020 IS-ENES3  project (grant agreement no. 824084 ). CNRM participants were additionally funded by the  French National Research Agency  project  CONVERGENCE  (grant  ANR-13-MONU-0008-02 ). Co-authors from  NCI  were supported by the  National Collaborative Research Infrastructure Strategy (NCRIS)-funded National Computational Infrastructure (NCI) Australia  and the  Australian Research Data Commons (ARDC). Review statement. This paper was edited by  Juan Antonio Añel  and reviewed by two anonymous referees.
Data request: https://www. earthsystemcog. org/projects/wip/ CMIP6DataRequest/ (last access: 14 January 2021; Juckes et al. , 2020) -Citation service: http://cmip6cite. wdc-climate. de/ (last access: 14 January 2021; Stockhause and Lautenschlager, 2017) -Synda: http://prodiguer. github. io/synda/ (last access: 14 January 2021; Ben Nasser et al. , 2021b) -Errata service: -Client: https://github. com/ES-DOC/esdoc-errata-client/ (last access: 14 January 2021; Ben Nasser and Lev- avasseur, 2021) -Web service: https://github. com/ES-DOC/ esdoc-errata-ws/ (last access: 14 January 2021; Ben Nasser et al. , 2021a) -ESGF front-end web page: https://github. com/ EarthSystemCoG/COG/ (last access: 14 January 2021; DeLuca et al. , 2013) -Dashboard: https://github. com/ESGF/esgf-dashboard/ (last ac- cess: 14 January 2021; ESGF Data Statistics Service, 2021).
Participants and Recruitment Data were collected from users of the KANOPEE application, downloaded voluntarily between April 22 and October 26, 2020. Respondents were anonymous and had to meet the following inclusion criteria: be of legal age (18 years old and over), have used tobacco or alcohol at least once in the past 12 months, and have completed the screening interview for tobacco and alcohol use disorder [five-item Cigarette Dependence Scale (CDS-5) and/or "Cut Down, Annoyed, Guilty, Eye-opener" (CAGE) questionnaire]. Figure  1  summarizes the recruitment procedure.
The proportion of the explained variance of multivariate models was examined with the R 2 (U) value. Univariates analyses were also conducted to identify variables associated with completing follow-up assessments and finding the application and diary helpful. Lastly, to investigate factors associated with acceptance and trust, we conducted univariate analyses with Pearson correlation analyses between two continuous variables (age, risk of SUD, and familiarity with technologies) and performed mean comparisons (t-test or analysis of variance) to analyze the variation in AES and ETQ scores regarding categorical variables (gender and educational level). Analyses were performed using the JMP Pro 14 R software. The significance threshold was set at p ≤ 0. 05.
We noted that user characteristics had only a limited influence on trust and acceptance of the application, suggesting that KANOPEE was appreciated by all, regardless of user's condition. Some limitations must be acknowledged. First, this study was conducted in a sample recruited through a mobile application and not through face-to-face data collection. As a result, it is possible that some of the data could be erroneous (e. g. , voluntarily false answers and multiple registrations by same individual). Nevertheless, previous studies from generalpopulation internet-based surveys have shown that the data generated may be reliable  (40). Furthermore, our data are consistent with current knowledge. Second, the application did not collect information on quantity or frequency of tobacco and alcohol use before the epidemic for objective comparison with current use.
Objective To review biomarker discovery studies using omics data for patient stratification which led to clinically validated FDA-cleared tests or laboratory developed tests, in order to identify common characteristics and derive recommendations for future biomarker projects. Design Scoping review. Methods We searched PubMed, EMBASE and Web of Science to obtain a comprehensive list of articles from the biomedical literature published between January 2000 and July 2021, describing clinically validated biomarker signatures for patient stratification, derived using statistical learning approaches. All documents were screened to retain only peer-reviewed research articles, review articles or opinion articles, covering supervised and unsupervised machine learning applications for omics-based patient stratification.
In order to guide future projects on suitable method choices, there is a need for dedicated studies on the key determinants of previous translational successes in ML-based omics biomarker development. As part of an EU project on 'Personalised Medicine Trials' (PERMIT  18  ), funded within the H2020 framework, we have therefore investigated the current methodological practices for personalised medicine, covering ML approaches for omics-based patient stratification as a major focus area. While a broader series of questions was established and examined for the overall scoping review,  19  for this manuscript, we focused our analysis on biomarker discovery studies that have led to successful, clinically validated FDA-cleared tests or laboratory developed tests (LDTs), to determine their shared and distinctive characteristics compared with studies with no clinical translation.
METHODS We conducted a scoping review following the methodological framework suggested by the Joanna Briggs Institute. 28 his framework consists of six stages: (1) identifying the research questions, (2) identifying relevant studies, (3) study selection, (4) charting the data, (5) collating, summarising and reporting results and (6) consultation. The scoping review approach was considered most suitable to respond to the broad scope and the evolving nature of the field. Compared with systematic reviews that aim to answer specific questions, scoping reviews present a general overview of the evidence pertaining to a topic and are useful to examine emerging trends, to clarify key concepts and identify gaps. 29 30 Before conducting the review, a study protocol was published on the online platform Zenodo.
Eligibility criteria We included peer-reviewed methodology articles, review articles, opinion articles on supervised and unsupervised ML methods for omics disease prediction and stratification and associated statistical cross-validation (CV) and multicohort validation methods (addressing accuracy, robustness and clinical relevance). Only approaches tested on real-world biomedical omics data were reviewed, while studies relying purely on simulated data were excluded. We also excluded papers on biomarker methods without a demonstrated biomedical application, and those with insufficient sample size (ie, removing studies covering less than 50 samples per group for the main conditions studied, unless a dedicated power calculation was presented) or statistical validation (ie, lack of clear descriptions of CV or external testing methodology, performance metrics and test statistics).
These exclusion criteria were not specified in the generic review protocol, but they were agreed among the authors prior to the screening process. To cover both data from original research papers and prior systematic reviews, we extracted information from three main article types: (1) applied research papers, (2) methodology articles with demonstrated applications and (3) review articles on methods, applications and validation approaches. Apart from these inclusion and exclusion criteria, for the final result presentation, the statistical investigations covered all selected articles, whereas the detailed discussion of study characteristics focused on the studies that Figure  1  Keyword based search strategy for the scoping review. Four categories of keywords were defined to retrieve relevant articles from the biomedical literature on machine learning analyses of omics data for personalised medicine, which include a validation study (highlighted by the coloured boxes in the centre).
For each category relevant keywords were determined, including controlled vocabulary terms from the Medical subject Headings (MeSH) thesaurus by the US National Library of Medicine (upper and lower boxes with frames coloured according to the corresponding category). As indicated by the keyword 'and' in the centre, a conjunctive search was conducted, that is, every retrieved article had to contain at least one keyword from each category. This strategy was adapted for searching the other databases. on January 2, 2022 by guest. Protected by copyright. http://bmjopen. bmj. com/ BMJ Open: first published as 10. 1136/bmjopen-2021-053674 on 6 December 2021. Downloaded from Open access led to clinically validated biomarker signatures tested on multiple cohorts with large sample sizes (ie, studies using a power calculation to demonstrate the adequacy of the chosen sample sizes, or covering hundreds or thousands samples per studied subject group).