dataesr/scientific-paragraphs-categorization huggingface dataset:
The largest group of respondents identified themselves as falling in the category of 'involvement in data analysis, computation software or infrastructure' (55. 30%, n=73), followed up by 'clinicians, clinical researchers, or principal investigators' (36. 36%, n=48) and finally 'data managers or curators' (8. 33%, n=11). Around a fourth (33 out of 128) of the respondents belong to institutions participating in PhenoMeNal, meaning that the survey reached its goal of scanning the needs of the PhenoMeNal community but also those of the Metabolomics community at large. The key findings of the H2020 Phenomenal Survey on standards are the following: • Most respondents indicated they used spreadsheet solutions (Excel) for collecting sample metadata and data (Q24/46/76:77. 5%, n=55), followed by custom LIMS solutions (Q24/46/76:31%, n=22).
4%, n=12). • The majority of clinical metabolomics profiling appears to be occurring in the context of clinical interventions (Q5/61/97:57. 3%, n=47). In 34. 1% (Q5/61/97:n=28) of such cases, the reported study design underlying the study was of the repeated measure type with crossover. This type of design is frequently used in nutrition studies and studies focusing on metabolic syndromes associated with Type 2 Diabetes. Accounting for over a third of all study types indicated, and owing to its importance in the realm of nutrition, PhenoMeNal needs to consider this type of study design with the right level of attention and offer meaningful data management support. • In terms of volume of sample processing and data productions, the weight of the large phenome centres, which are being established is showing clearly with 24.
5% of respondents (Q17/39/69:n=12). This indicates processing at industrial scale with more than 10,000 samples processed. 65. 3% of data producers fell into the range of more than 1,000 but less than 10,000 samples a year (Q17/39/69:n=32). This is an extremely valuable insight as it provides a very good estimate of the volume of data PhenoMeNal needs to plan for and be able to cope with. Efficient processing has to be done for supporting current datasets retrospectively and, more critically, PhenoMeNal needs to define methods and processes to enable prospective data management. • Data deposition either to public repositories or regulatory agencies remains the exception rather than the norm. Of all respondents, only a third had deposited data to public repositories (Q19/41/71: 35. 9%, n=29) and only one respondent indicated experience with regulatory submissions (Q21/43/73: in other, 1.
Final Remarks and Summary A survey was carried out under the auspices of and serving the H2020 PhenoMeNal project and in collaboration with the Metabolomics Society. It was supported by key stakeholders such as major publishers and leading data journals (NPG, BMC with Scientific Data and GigaScience) and targeted the wider metabolomics as well as the clinical phenotyping community. The results highlight the need to bridge an awareness divide for clinical standards and instrumentation related standards. While the latter are well understood by personnel in charge of data production centres, patient-centric metadata standards are less known to that audience. The survey revealed two areas in dire need of standardisation. The first is that of analysis reporting. Indeed two-thirds of participants to the survey (Q32/54/84: 67.
1%, n=47) indicated that description of the data processing was only captured in textual form. This entails inevitably poor capability of automatic high-throughput review, quality assurance and reproducibility of the findings. This is precisely what the PhenoMeNal program aims to have most impact on, as it intends to provide all the necessary infrastructure to re-enact clinical data processing within computational workflows stored in machine readable and runnable format. The second salient point is a gap between ELSI awareness and implementation. Respondents indicate a good level of awareness around requirements for patient data protection and privacy. However, based on the survey results the actual level of data integrity checks and encryption are extremely low. This points again to the need to implement out of the box solutions in the various APIs beings developed, to provide native checksumming on data files and options to provide robust encryption routines for data and metadata files to comply with current regulatory requirements for clinical omics datasets.
• Coordinate between namespace borders in order to avoid overlapping, nonorthogonal ontologies and standards in order to save users from heterogeneous annotations. The next key deliverable is on D8. 3 "nmrML, mzML data exchange formats and associated terminologies for instrument raw, with reference implementation guidelines and validation rules" at M18, and D8. 2 "Modularized ISA model and format: biospecimen centric schema, corresponding xml schemas, reference implementation guidelines and validation rules" at M24. Work towards these has already begun (progress can be found in Pivotal tracker), with much work towards the reference implementations and tooling underway in collaboration with WP9. The current state of WP8 tasks is documented in the below screenshots taken from the PhenoMeNal Pivotal tracker, which records live progress.
CONCLUSION The work done towards T8. 1, "Use cases and state-of-the-art of communication standards" and reported on in this deliverable includes the commissioning and execution of a community-wide survey on metabolomics data needs, the PhenoMeNal Data Infrastructures Survey, and the collation of the state-of-the-art in relevant data standards, the PhenoMeNal Data Standards Collection. Much insight was found in the survey results helping to shape the data standards collection, but also to help inform the future work that needs to be carried out in WP8. In particular, the upcoming tasks and deliverables now focus on standards development and adoption within the PhenoMeNal use cases, with ISA, mzML, and nmrML being further developed for use by the metabolomics community.
B. 1. Survey Question Reference Note: Choice answers presented to the survey respondent are not illustrated here for the sake of brevity. For the further details, please refer to the full dataset provided. The headings 'Clinical-related', 'Data manager', and 'Software and infra' refer to the different groups of respondents (as identified by themselves in Q3) and a 'X' indicates of the question was asked to that group. The question references where multiple question numbers are referenced indicate where the same question was asked, but in the survey logic it appears in different places. For example, Q4/60/96 is the same question, but Q4 appears in the clinical track, Q60 in the data manager track, and Q96 in the software and infrastructure track.
The ISBER Biospecimen Science Working Group has released a "Standard PREanalytical Code" (SPREC), which is a proposal for a standard coding of the preanalytical which have been adopted, in order to track and make explicit preanalytic variations in collection, preparation and storage of specimens. Each biospecimen is assigned a seven-element-long code that correspond to seven preanalytical variables and contains a string of letters(different for fluid or for solid tissues) in a defined order, separated by hypens A specific extension/module in the HL7 model to capture patient consent. http://wiki. hl7. org/index. p hp?title=FHIR_Consent_ Directive_Implementatio n_Guide#Examples http://hl7-fhir. github. io/ http://hl7-fhir. github. io/valueset-consent-category. html The Clinical Document Electro Released Domain Status Releas ed Releas ed Domai n Status Clinical Trial Data Regulat ory Submis sion Released as Productio n Ready Clinical Trial Pharma cogeno mics Data Released as Draft Standard for Trial Use Data Collecti on, Clinical Trial.
using a LIMS, what vendor/manufacturer provides the system? Answer Options Response Count 11 answered question 11 skipped question 385 Q26/48/78 If using a LIMS, what formats and what type of information can you export? Answer Options Respons Respon e se Percent Count CSV or TSV files 86. 7% 26 JSON 20. 0% 6 XML 36. 7% 11 Analytical Information Markup Language (AniML) Format 3. 3% 1 LIMS specific API 13. 3% 4 Database dump 13. 3% 4 Other (please specify) 10. 0% 3 answered question 30 skipped question 366 Q27/49.
/79 If using a LIMS, what patient and sample information do you record? Answer Options Respons e Percent Respon se Count Fully integrated to a clinical database 10. 7% 3 Basic information (patient id, patient age, patient gender', 17. 9% 5 sample id and sample type) Customized for each project, highly variable 57. 1% 16 None, only an autogenerated specimen ID used as key 17. 9% 5 Other (please specify) 14. 3% 4 answered question skipped question Q28/50/.
A new global estimate of surface turbulent fluxes, latent heat flux (LE) and sensible heat flux (H ), and gross primary production (GPP) is developed using a machine learning approach informed by novel remotely sensed solarinduced fluorescence (SIF) and other radiative and meteorological variables. This is the first study to jointly retrieve LE, H , and GPP using SIF observations. The approach uses an artificial neural network (ANN) with a target dataset generated from three independent data sources, weighted based on a triple collocation (TC) algorithm. The new retrieval, named Water, Energy, and Carbon with Artificial Neural Networks (WECANN), provides estimates of LE, H , and GPP from 2007 to 2015 at 1 • × 1 • spatial resolution and at monthly time resolution. The quality of ANN training is assessed using the target data, and the WECANN retrievals are evaluated using eddy covariance tower estimates from the FLUXNET network across various climates and conditions.
g. , the Priestley-Taylor or Penmann-Monteith equation in the case of evapotranspiration (ET), or a light-use efficiency model in the case of GPP) informed by remote sensing information (e. g. , vegetation indices, infrared temperature, microwave soil moisture), often in combination with reanalysis meteorological forcing data  (Fisher et al. , 2008; Miralles et al. , 2011a; Mu et al. , 2007; Zhang et al. , 2016b; Zhao et al. , 2005; Zhao and Running, 2010). These approaches are sensitive to the assumptions and imperfections of the underlying flux models. The second approach, uses machine learning (e. g. , a model tree ensemble) to determine LE, H , and GPP from meteorological drivers and optical remote sensing data  (Tramontana et al. , 2016). Like all supervised machine learning models, this approach relies on a training dataset to determine the nonlinear statistical relationships.
In this case, in situ turbulent flux and GPP estimates from eddy covariance towers are used  (Beer et al. , 2010; Jung et al. , 2011). Such an approach relies implicitly on an assumption that a long temporal record of these variables at a small number of sites captures the full range of behavior and sensitivities of terrestrial ecosystems around the globe. In addition, extreme and therefore rare events may be difficult to capture based on the limited data availability. Alternatively, one can use a machine learning approach, such as an artificial neural network (ANN), trained on globally representative but imperfect estimates of the fluxes (such as those from models) to parameterize the nonlinear statistical relationships between remote sensing observations and surface fluxes. This approach has been successfully used for global soil moisture retrieval  (Aires et al.
, 2012; Kolassa et al. , 2013 Kolassa et al. , , 2016;; Rodriìguez-Fernández et al. , 2015)  and surface heat flux retrieval  (Jiménez et al. , 2009). Such ANNs require a target dataset for training. Climate model simulations of the relevant geophysical variable are usually used as the training dataset to facilitate subsequent data assimilation efforts  (Aires et al. , 2012; Kolassa et al. , 2013 Kolassa et al. , , 2016)). However, the downside of this approach is that the resulting fluxes estimated by the ANN often exhibit some of the same biases as the simulations used to train the network  (Rodriìguez-Fernández et al. , 2015) , even if improvements can be achieved such as a more realistic seasonal cycle as it is informed by the seasonal cycle of the remote sensing data  (Jiménez et al. , 2009).
In early 2011, drought conditions developed in the southern US, particularly in the states of  Texas and Louisiana (Luo and Zhang, 2012). By April, most of Texas, Oklahoma, Louisiana, and Arkansas was classified in the D4 drought condition (exceptional drought), and the situation continued throughout the summer and fall of 2011 as reported by the US Drought Monitor  (Svoboda et al. , 2002). As Fig. 11  reveals, the same spatial pattern is pronounced in the monthly anomalies derived from WECANN retrievals, emphasizing massive reduction in LE and GPP accompanied by high H over the region. Finally, an intense drought in the central US, particularly in the Corn Belt, occurred in 2012 and reduced maize yields by about 25 % and increased prices by 17-24 %  (Boyer et al. , 2013; USDA, 2013). By mid-September 2012 almost twothirds of the continental US was covered by drought, and different parts of the US Corn Belt were categorized as either D3 (extreme drought) or D4 (exceptional drought) conditions.
The evaluation results compared with training datasets show that our retrieval has similar correlation with the three products, while it has the smallest RMSD with FLUXNET-MTE for LE (RMSD = 6. 42 W m -2 ), H (RMSD = 7. 84 W m -2 ), and GPP (RMSD = 0. 88 gC m -2 day -1 ), which is believed to be one of the most realistic global datasets. It also has the lowest RMSE based on our TC error estimates (Fig. S4-S6 ), despite its reported underestimated interannual variability due to the use of climatological values for several meteorological drivers  (Miralles et al. , 2014a (Miralles et al. , , 2016)). Such a tendency can also be summarized from the global difference maps, which show that FLUXNET-MTE has the best agreement with WECANN retrievals. The WECANN and FLUXNET-MTE approaches are both based on machine learning, although the FLUXNET-MTE retrievals use a regression tree rather than an ANN.
Nevertheless, this commonality of methods may also contribute to the greater correspondence between these two datasets. The retrieval maps indicate that LE, H , and GPP have similar seasonal variability and distribution, which are determined by the annual phenological cycle in energy-limited northern latitude regions, dryness in Mediterranean and monsoonal climates, and by light availability in rainforests. Seasonal radiation has a great impact for all three variables on some regions, such as the eastern US, Europe, and East Asia, which have wet conditions, are highly vegetated and located in midlatitudes. Conversely, the seasonal variability in LE, H , and GPP in some low-latitude and wet condition regions, such as Amazon rainforest, southern Africa, and Southeast Asia, as well as some low-latitude arid regions, such as the southwestern US, Western Australia, North Africa, and western Asia, are not significant, as there is less seasonal solar radiation variability in the aforementioned regions.
Scientific communities in different domains have started to act in an attempt to address this problem. Prestigious conferences (such as two major conferences from the database community, namely, VLDB 1  and SIGMOD  2  ) and journals such as PNAS  3  , Biostatistics  [8] , Nature  [9]  and Science  [10] , to name only a few, encourage or require published results to be accompanied by all the information necessary to reproduce them. However, making their results reproducible remains a very difficult and extremely time-consuming task for most authors. In the meantime, considerable efforts have been put into the development of scientific workflow management systems. They aim at supporting scientists in developing, running, and monitoring chains of data analysis programs. A variety of systems (e.
Reproducibility needs Workflow run-time environment maintenance. A given workflow in INCa usually embeds a large variety of tools and makes use of several databases, evolving quickly. Each version change implies the update, development and validation of a new stable version of the workflow. In addition to the classical workflow maintenance, the workflow environment has to be captured again (packaged) for the workflow to be concretely in use again in the laboratories. Sharing sample data. Reuse of workflows follows very strict rules in particular because given accreditations have to be obtained (following specific norms in the representation and the documentation of the workflow such as CE-IVD or ISO 13485 norms). As part of the requirements to permit reuse, workflows designed in InCA should be accompanied with a gold standard providing a sample data set to be used as input of the workflows and the expected outputs, so that the workflow can be tested.
Context This use case takes place in the context of the "Phenome" 4  project where high-throughput phenotyping platforms (Figure  1 ) are used to help assess the genetic variability of plant responses to environmental conditions by allowing growing and automatic extraction of traits of a large number of plants. In such platforms, 1,650 plants are imaged from 12 lateral and one apical view, producing 20,000 images per day and about 3 Terabytes of raw data per experiment. This raw data (images) should be processed to extract traits information (size of the leaves, number of leaves. ) and then further analyzed with a response model to simulate the behavior of plants under climate change. Each of these steps combines tools from very different fields of research including image analysis, plant physiology, and crop modeling.
Computational tools used All computational experiments performed in PhenoArch are designed using the OpenAlea  [19]  workflow system, widely used in the Plant modeling community. Workflows analyze images and combine raw data with numerous and heterogeneous data sets coming from various external databases or produced by previous workflow runs. Both raw data sets and data sets analyzed by workflows are stored in a dedicated database. As for the tools used in the current workflows, they are parts of two versioned Python packages: phenomenal 5  for image analysis and alinea 6  for eco-physiological simulations. These packages are platform-independent and can be downloaded and installed to replay an analysis on a local desktop.
It is also required to share and reuse site-specific datasets to assess the relevance of the data analysis pipeline. However, designing and sharing RNA-seq analyses is particularly challenging both in terms of computing time, massively produced data and multiplicity and diversity of tools available  [23]. In this context, SyMeTRIC is a federated project in Systems Medicine, launched to standardize RNA-seq data analysis pipelines and allow the reuse of RNA-Seq experimental results.
Computational tools used The SyMeTRIC project relies on two kinds of solutions to represent in silico experiments. First, Galaxy is used to design and evaluate a common RNA-seq workflow in the context of multi-centric cancer studies. Its Webbased platform is useful to the SyMeTRIC users who want to learn bioinformatics tools and re-execute them on reference datasets. Second, when large-scale data analysis campaigns with hundreds of biological samples have to be run, Make-like systems (similar to NextFlow) are being used. While using such approaches requires more technical expertise, they undoubtedly offer versatility and scalability in the tuning of large-scale experiments  [24].
Reproducibility needs Annotation on workflows and runs. Workflows implemented using scripts are generally not annotated with any metadata. As for Galaxy workflows, no standardized metadata is currently in use to annotate workflows either. The same occurs at the execution level: no provenance module is available yet to inspect or query execution traces based on standard schemas or vocabularies. This situation has two kinds of consequences. First, querying workflows or workflow runs based on keywords/metadata reflecting the nature of data processing tools or the nature/role of processed data is impossible. Second, trusting new experiments (without any comparison point with previous experiment) remains difficult. Reusing parts of executed workflows. SyMeTRIC users may want to reuse a workflow which has been already run in part, to continue the execution with new processing tools, without re-executing the full workflow (but only the new part).
Discussion on the concepts associated with reproducibility Reproducibility has obviously been studied in science in larger contexts than computational reproducibility, in particular where wet experiments are involved, and especially in the context of preclinical studies (e. g. ,  [25, 26, 27, 28] , see  [29]  for a review on this topic). A plethora of terms are used including repeat, replicate, reproduce, redo, rerun, recompute, reuse and repurpose etc. to name a few. We introduce the terminology we will follow in this paper based a review of the various definitions provided in the literature  [2, 30, 17, 31, 16, 32]. In such definitions, we need to distinguish the results of an in-silico experiment, which are concrete datasets produced, from the conclusion that can be drawn from the interpretation of such data sets.
Companion tools and standards 5. 1. 1. Workflow interoperability To promote workflow reuse, a number of initiatives has been launched to deal with the heterogeneity of workflow languages (introduced in Section 4. 1). In particular, in the context of the SHIWA project  [35] , Plankensteiner et al. proposed a workflow specification language called IWIR  [36]  to allow portability between major workflow management systems. To annotate (and complement) IWIR workflows, Cerezo and Montagnat  [37]  proposed a conceptual workflow language for describing the high-level scientific tasks. To promote interoperability, a consortium composed of multiple organizations and workflow system vendors has recently proposed the Common Workflow Language (CWL)  [38]   7  , with the objective to promote workflow specification portability.
Workflow repositories To promote workflow reuse, and ultimately reproducibility, a number of scientific workflow repositories, e. g. , CrowdLabs  [42] , SHIWA  [43] , the repositories offered with Kepler  [44]  and Galaxy  [45]  and myExperiment  [46] , have been launched to allow scientists to share, publish their workflows for the benefit of the scientific community. As well as workflow repositories, there exists nowadays software catalogs, such Bio. tools [47]  and Biocatalogue  [48]  for sharing information about the software products that can be used to implement the steps of workflows. Both kinds of catalogs are accompagnied with annotations, using free texts and/or controlled vocabularies, that describe the workflows/software products. For example, Bio. tools utilizes con-cepts from domain ontologies, e.
Other approaches like ReproZip  [50]  and CDE  [51]  allow capturing the command line history and associated input and output files, and package them in the case of Reprozip in the form of workflows. Scientific-wide packaging solutions. Other solutions directly address scientific reproducibility, e. g. , Research Objects (RO)  [52]  or Investigation/Study/Assay (ISA)  [53]  which capture and expose auxiliary information about the hypothesis investigated and the conclusions made by the scientists. Such approaches create self-contained and query-able packages capturing the required i) data, ii) metadata (including provenance) and iii) protocol/workflow specifications.