dataesr/scientific-paragraphs-categorization huggingface dataset:
Discussion In the present study, transcriptional profiling of eight independent data sets was conducted to decipher the intricate process of gene regulation in peach and to reveal meaningful biological signatures. DETs were grouped into 45 co-expression modules undergoing similar changes in their expression patterns. Unlike conventional clustering methods (such as k-means and hierarchical clustering), which are based on geometric distances, WGCNA is a graph-based approach relying on network topology as inferred from the correlation among expression values  (Li et al. , 2018). In our hands, the WGCNA algorithm robustly and accurately defined modules within a complex multicondition dataset. Discerning regulatory signals from blocks of co-expressed genes is a common presumption used to identify functional genomic elements.
We reasoned that low complexity sequences might be linked to repetitive stretches of DNA, extensively present in plant genomes  (Yu et al. , 2015). Interestingly, when tuning the promoter upstream length to a tract of [-500 bp, +200 bp] relative to the TSS, these low complexity motifs were limited to module M2. It would seem that long upstream promoter regions unbalance the signal-to-noise ratio exacerbating the identification of such AT motifs. Along the same lines, we observe a dependence of (AT)-rich sites on the dataset size. Indeed, AT-low-complexity motifs were only detected in the first six modules, which contain from 560 to 1795 upstream sequences. In light of these considerations, we believe that in our study case, they may result in part due to the properties of DNA sequences (both upstream region length and dataset size) rather than the performance of the chosen algorithms.
Input data and processing Eight peach RNA-sequencing datasets were downloaded from the European Nucleotide Archive (https://www. ebi. ac. uk/ena) and were used as raw reads for this project. This comprehensive dataset includes data of various peach cultivars, from various tissues (root, leaf, stigma and fruit), different stress conditions and developmental stages. A detailed list of the project IDs and metadata is provided respectively in Table  1  and Table S 1. A. The obtained reads were quality-processed and trimmed  using FASTQC v. 0. 11. 5 and Trimmomatic v. 0. 36 (Bolger et al. , 2014) , to discard adaptors and lowquality sequences with mean Phred score (Q < 30) and window size of 4:15. The first nucleotides were then head-cropped to ensure a per-position A, C, G, T frequency near to 0.
25. Following the trimming, only sequences longer than 36 bp were retained for further analysis. The complete workflow is shown in Figure  1 (see step 1 ). The high quality reads from each RNA-seq project were quantified separately using the pseudo-aligner kallisto v. 0. 43. 1 for fast and accurate transcripts count and abundance  (Bray et al. , 2016). Kallisto was run in two steps: i) a transcriptome index was built from all cDNA transcripts of Prunus persica v2, from Ensembl Plants release  39 (Verde et al. , 2017; Howe et al. , 2020). ii) Each sample was pseudo-aligned against the index. Transcript level abundance was estimated and normalized to transcripts per million (TPM) using 100 bootstraps (-b 100) to ascertain the technical variation. For single-end read mode, average fragment length and standard deviation were additionally required and were set to (-l 200) and (-s 50), respectively.
De novo cis regulatory sequences discovery using RSAT::Plants Gene modules resulting from network analysis were subjected to an ab-initio motif discovery pipeline using the RSAT::Plants standalone (Figure  1 , step 2). For each module, the analysis initiates by generating as negative control 50 random clusters of the same size for each module as described previously  (Contreras-Moreira et al. , 2016). Sequences with four different boundaries around the TSS were retrieved from the genes in the co-expressed modules, random clusters and Prunus persica genome v2. The upstream sequences were defined as intervals of i) -1. 5 kb to +200 bp ii) -500 bp to +200 bp and iii) two segments around the TSS: -500 bp to 0 and 0 bp to +200 bp. Note that the 0 to +200 interval corresponds to the 3' UTR region, which is already downstream.
RSAT peak-motifs was run under the differential analysis mode, where module's upstream sequences served as the test set and all upstream sequences from peach genome were considered as the control set to estimate the background model (a background model was created for each upstream stretch)  (Thomas-Chollier et al. , 2012). Two discovery algorithms were used: i) oligo-analysis, which is based on the over-representation of k-mers in upstream regions, and ii) dyad-analysis, which looks for over-represented spaced pairs of oligonucleotides  (Defrance et al. , 2008). For each run, up to five motifs were returned per algorithm and were retained to compare their statistical significance with the 50 random clusters considered as negative control. Candidate motifs were chosen based on their significance (log E-value) compared to negative control and were subsequently annotated by comparison to the footprintDB collection of plant curated motifs (http://floresta.
eead. csic. es/footprintdb) (Sebastian and Contreras-Moreira, 2014) using the compare-matrix tool in RSAT  (Nguyen et al. , 2018)  requiring a normalized correlation score Ncor ≥ 0. 4. Finally, selected motifs were scanned along the stretch [-1500 bp, +200 bp] to predict their corresponding binding site positions, using as background model a Markov chain of order 1 (m=1) and a cutoff P-value ≤ 1e -4. To ensure the clarity and reproducibility of this strategy, a repository including the source code, links to the results and a tutorial explaining how to reproduce a similar analysis on any species is available at https://eead-csic-compbio. github. io/coexpression_motif_discovery/peach.
Transcription factor prediction and Gene Ontology analysis Hereafter, the analysis was restricted to modules with significant detected signals. Firstly, genes coding peach TFs were predicted and classified using the iTAK database (http://itak. feilab. net/cgi-bin/itak/index. cgi, last accessed January 2020). Protein sequences of TFs were subsequently submitted to footprintDB to predict their interacting DNA-binding site. To functionally interpret the co-expressed modules, Gene Ontology (GO) enrichment was conducted on PlantRegMap / PlantTFDB portal v5. 0 (http://planttfdb. gao-lab. org/, last accessed January, 2020)  (Tian et al. , 2019). P-value of 0. 01 was set to retain the significant GO terms.
Validation of the pipeline by detecting a priori known motifs in Arabidopsis thaliana To assess the impact of upstream region lengths on the identification of relevant motifs, we used sets of experimentally validated binding sites of 10 Arabidopsis thaliana TF families. Sequences of the proven sites were downloaded from JASPAR database  (Fornes et al. , 2020)  and were locally aligned with BLASTN against the A. thaliana     Gene profiles along the different conditions are provided at (https://eead-csic compbio. github. io/coexpression_motif_discovery/peach). See supplementary Table  S 1  for the abbreviations. TFs showing sequence similarity between their footprintDB and RSAT predicted motifs are labeled with a star. S3. An interactive report with source code is accessible at https://eead-csic-compbio.
This work was partly funded by the  Spanish Ministry of Economy and Competitiveness  grants  AGL2014-52063R ,  AGL2017-83358-R (MCIU/AEI/FEDER/UE ); and the  Government of Aragón  with grants  A44  and  A09_17R , which were co-financed with FEDER funds. N. Ksouri  was hired by project  AGL2014-52063R  and now funded by a PhD fellowship awarded by the  Government of Aragón.
This study provides an observational constraint on the latter uncertainty. As the vast majority of the CMIP6 and CMIP5 models do not yet represent vertically resolved deep soil carbon in permafrost or peatlands, we focus our constraint on carbon change in the top 1 metre of soil. To ensure a fair like-for-like comparison we also exclude the two CMIP6 models that do represent verticallyresolved soil carbon (CESM2 and NorESM2), although this has a negligible effect on our overall result. Our study therefore applies to soil carbon loss in the top 1 metre of soil only. Below we show that it is possible to significantly reduce the uncertainty in this key feedback to climate change using current-day spatial data to constrain the sensitivity to future warming.
Results and discussion Proof of concept. For each ESM, we begin by calculating the effective τ s using time-averaged (1995-2005) values of C s and R h at each grid-point, and applying our definition of τ s = C s /R h. We do likewise for observational datasets of soil carbon in the top 1 metre  19, 20  and time-averaged (2001-2010) heterotrophic respiration  21  , as shown in Fig. 2. Figure  2c  shows the map of inferred values of τ s from these observations, with a notable increase from approximately 7 years in the warm tropics to over 100 years in the cooler high northern latitudes. Similar maps can be diagnosed for each of year of data, for each ESM, and for each future scenario, giving time and space varying values of τ s for each model run. This allows us to estimate ΔC s,τ , via the last term on the right of Eq.
6, RCP4. 5 and RCP8. 5 (CMIP5)  23. In all cases ΔC s,τ is negative, which is consistent with the soil carbon turnover time decreasing with warming. The more surprising thing to note is the huge range in the projections, with a spread at 2 °C global mean warming of approximately 400 PgC, regardless of future SSP/RCP scenario. Figure  1b  plots the fractional change in soil carbon ΔC s,τ /C s,0 , showing that there is a large range of effective q 10 sensitivities between the model projections. Unfortunately, we do not have time-varying observational datasets of C s and R h that might allow us to directly constrain this projection uncertainty. Instead we explore whether the observed spatial variability in τ s (as shown in Fig. 2c ) provides some observational constraint on the sensitivity of τ s to temperature.
In doing so, we are motivated by Chadburn et al. 15  who used the correlation between the observed geographical distributions of permafrost and air temperature to constrain projections of future permafrost area under global warming. Similarly, we use ESMs to test whether the spatial variation in τ s reveals the sensitivity of soil carbon turnover to temperature. The spatial patterns of τ s in CMIP5 simulations and observations were previously shown in Koven et al. 16  , and here we test whether such relationships can be used to estimate the response of soil carbon to future climate change, using a combination of CMIP6 and CMIP5 models. Figure  3a  is a scatter plot of log τ s against temperature, using the τ s values shown in Fig. 2c     WFDEI dataset over the period 2001-2010  24. The thick blackdotted line is a quadratic fit through these points.
We apply a standard statistical approach  27, 28  to estimate the probability density function of the y-axis variable (model ΔC s,τ ),  The table presents the sensitivity of the emergent constraint on ΔC s,τ to model ensemble: CMIP5, CMIP6 or CMIP5 and CMIP6 combined (columns), and to the observational dataset for heterotrophic respiration (rows). The penultimate row presents the constraint using the observational τ s v T fit and model R h,0 , opposed to observational R h,0 , to isolate the uncertainty reduction from these different components. For comparison, the last row shows the mean and standard deviation of the unconstrained model ensemble. accounting for both this observational spread and the quality of the emergent relationship. To test the robustness to the choice of observations we have repeated the analysis with different datasets that represent heterotrophic respiration, which produces stronglyoverlapping emergent constraints, and completing the analysis with both CMIP6 and CMIP5 models shows that the result is also robust to the choice of model ensemble (see Table  1 ).
Then, monthly model output data was time averaged over this period, for the output variables 'soil carbon content' (C s ) in kg m -2 , 'heterotrophic respiration carbon flux' (R h ) in kg m -2 s -1 , and 'air temperature' in K. The variables C s and R h were used to obtain values for soil carbon turnover time (τ s ) in years, using the equation τ s = C s /(R h × 86400 × 365). The model temperature variable units were converted from K to °C. For each model, these values of log τ s were plotted against the corresponding spatial temperature data to obtain the spatial log τ s -temperature plot. Then, quadratic fits (using the python package numpy polyfit) are calculated for each model, which represent the spatial log τ s relationship and sensitivity to temperature. These model specific relationships are shown by the coloured lines in Fig.
3a  in the main manuscript, and in Supplementary Fig. 2  for CMIP6 and in Supplementary Fig. 3  for CMIP5. Obtaining spatial relationships for observations. Following Koven et al. 16  , we estimated observational soil carbon data (to a depth of 1 m) by combining the Harmonized World Soils Database (HWSD)  19  and Northern Circumpolar Soil Carbon Database (NCSCD)  20  soil carbon datasets, where NCSCD was used where overlap occurs. To calculate soil carbon turnover time, τ s , using the following equation: τ s = C s /R h , we require a global observational dataset for heterotrophic respiration. In the main manuscript, CARDAMOM (2001-2010) heterotrophic respiration (R h ) is used  21. We completed a sensitivity study on the choice of observational heterotrophic respiration dataset, see below.
The WFDEI dataset is     24. Then, these datasets can be used to obtain the observational log τ s -temperature relationship, using the same quadratic fitting as with the models. This represents the 'real world' spatial temperature sensitivity of log τ s , and is shown by the thick-dotted-black line in Fig. 3a  of the main manuscript. A comparison of the derived observational relationships can be seen in Supplementary Fig. 6. Observational sensitivity study. We completed a sensitivity study to investigate our constraint dependence on the choice of observational heterotrophic respiration dataset (CARDAMOM (2001-2010)  21  ). The other observational datasets considered are as follows: NDP-08 'Interannual Variability in Global Soil Respiration on a 0. 5 Degree Grid Cell Basis' dataset (1980-1994)  29  , 'Global spatiotemporal distribution of soil respiration modelled using a global database'  30  , and MODIS net primary productivity (NPP) (2000-2014)  31.
Supplementary Fig. 4  shows scatter plots showing one-to-one comparisons of these observational datasets against one another, and Supplementary Fig. 5  shows the corresponding comparisons of the equivalent log τ s values calculated from each dataset. The CARDAMOM R h dataset is used in the main manuscript for the following two main reasons: firstly, we calculate τ s using heterotrophic respiration which allows for consistency between models and observations, and secondly, the dataset does not use a prescribed q 10 sensitivity 21. Instead, the CARDAMOM R h dataset was derived by explicitly assimilating observations into a process-based diagnostic land-surface model. To test the robustness of our results, we also repeated our analysis with MODIS NPP and Raich 2002, for both CMIP6 and CMIP5 together, and as separate model ensembles.
Supplementary Fig. 6  shows the observational log τ s -temperature relationships, derived using each of these observational datasets. The results are presented in Table  1  which shows the constrained values of ΔC s,τ at 2 °C global mean warming. We decided not to complete the paper analysis using the Hashimoto dataset since not only is it inconsistent with the three other datasets considered, it also shows an arbitrary maximum respiration level (Supplementary Fig. 4 ), which likely results from the assumed temperature-dependence of soil respiration in this dataset which takes a quadratic form  30. The quadratic form is justified based on a site-level study in which it is used to fit temporal dynamics. However, the parameters for the quadratic function that are fitted in the Hashimoto study are very different from those in the site-level study, which therefore suggests that the same relationship does not apply to the global distribution of mean annual soil respiration.
Anomaly correction for future temperature projections. To remove uncertainty due to errors in the models' historical simulation, a spatial future temperature anomaly was projected using each model and each respective future SSP/RCP scenario separately. To calculate this, the temperature at the reference time frame (1995-2005), which overlaps the WFDEI observational temperature data time frame (2001-2010), is subtracted from the future temperature profile for each model (as calculated above), to calculate the temperature change. Then, this temperature anomaly is added onto the observational temperature dataset to give a model-derived future 'observational' temperature for each model. Proof of concept for our method. Our method relies on the idea that the spatial temperature sensitivity can be used to project and constrain the temporal sensitivity of τ s to temperature, and subsequently global warming.
Then this together with relationship-derived historical τ s deduced using the observational temperature dataset, can be used to calculate Δτ s. Finally global ΔC s,τ can be obtained by multiplying Δτ s by the observational dataset for R h,0 (using Eq. (  4 )), and then calculating a weighted-global total. As each model-derived future 'observational' temperature is considered separately, we obtain a range of projected observational-constrained ΔC s,τ values. We have now obtained a set of x and y values, corresponding to the relationship-derived and modelled values of ΔC s,τ , respectively, for each ESM. Where we have an x and y value for each model, representing the modelled ΔC s,τ (y values), and the model specific relationship-derived ΔC s,τ (x values). We also have an x obs value representing the mean observational-constrained ΔC s,τ value, and a corresponding standard deviation due to the uncertainty in the modelled spatial profiles of future temperatures.
Acknowledgements This work was supported by the  European Research Council  ' Emergent Constraints on Climate-Land feedbacks in the Earth System (ECCLES) ' project, grant agreement number  742472  (R. M. V. and P. M. C. ). S. E. C. was supported by a  Natural Environment Research Council independent research fellowship , grant no. NE/R015791/1. P. F. was supported by the  European Union 's  Framework Programme Horizon 2020 for Research and Innovation  under Grant Agreement No. 821003 ,  Climate-Carbon Interactions in the Current Century  ( 4C ) project. E. J. B. was funded by the  European Commission 's  Horizon 2020 Framework Programme , under Grant Agreement number  641816 , the ' Coordinated Research in Earth Systems and Climate: Experiments, Knowledge, Dissemination and Outreach (CRESCENDO) ' project ( 11/2015-10/2020 ) and the  Met Office Hadley Centre Climate Programme  funded by  BEIS and Defra.
C. D. K. acknowledges support from the  U. S. Department of Energy, Office of Science, Office of Biological and Environmental Research , through the  RUBISCO SFA  and the  Early Career Research Program. G. H. was supported by the  Swedish Research Council  ( 2018-04516 ) and the  EU H2020  project  Nunataryuk  (  773421 ). We also acknowledge the  World Climate Research Programme 's Working Group on Coupled Modelling, which is responsible for CMIP, and we thank the climate modelling groups (listed in Tables 2 and 3 in the "Methods" section) for producing and making available their model output.
Data availability The datasets analysed during this study are available online: CMIP5 model output [https://esgf-node. llnl. gov/search/CMIP5/], CMIP6 model output [https://esgf-node. llnl. gov/search/cmip6/], The WFDEI Meteorological Forcing Data [https://rda. ucar. edu/ datasets/ds314. 2/], CARDAMOM Heterotrophic Respiration [https://datashare. is. ed. ac. uk/handle/10283/875], MODIS Net Primary Production [https://lpdaac. usgs. gov/ products/mod17a3v055/], Raich et al. 2002 Soil Respiration [https://cdiac. ess-dive. lbl. gov/epubs/ndp/ndp081/ndp081. html], Hashimoto et al. 2015 Heterotrophic Respiration [http://cse. ffpri. affrc. go. jp/shojih/data/index. html], and the datasets for observational Soil Carbon [https://github. com/rebeccamayvarney/soiltau_ec].
To improve our categorization, identify exciting patterns, and promote data investigation by the readers of this report, we have deployed an interactive online survey browser available at https://trustmlvis. lnu. se We expect that our results will support new research possibilities for different groups of professionals: • beginners/non-experts who want to get acquainted with the field quickly and gain trust in their ML models; • domain experts/practitioners of any discipline who want to find the appropriate visualization techniques to enhance trust in ML models; • model developers and ML experts who investigate techniques to boost their confidence and trust in ML algorithms and models; and • early-stage and senior visualization researchers who intend to develop new tools and are in search of motivation and ideas from previous work.
Background: Levels of Trustworthiness of Machine Learning Models First, we present some earlier definitions of trust that are subsequently adapted to the context of our research. We also discuss Figure  1 : The overview of our STAR with regard to the methodology, main results, and corresponding sections of the manuscript. Color coding is used for grouping related activities and results (purple for the background information and key concepts, blue for the literature search, green for the paper categorization, orange for the data analyses, and yellow for the manuscript); italic font is used for intermediate activities; and bold font is used for the items discussed explicitly in this STAR. The marks S1 -S8 refer to supplementary materials. qualitative data gathered from an online questionnaire that we distributed among ML experts and practitioners.
Issues about reliability of the data and of the collection process can jeopardize, from the very start, the ML process and diminish the TLs set by users. If those issues remain undetected, they can spoil the later phases, according to the classic "garbage in, garbage out" principle. For instance in the case of unreliable labels  [FV14] , reported error rates are also unreliable. This is becoming more relevant with the growing attention given to adversarial machine learning, an ML research field which focuses on adversarial inputs designed to break ML algorithms or models [GMP18, LL10]. • Data labeling & feature engineering (TL2). The next group of categories has its focus beyond the raw data and into feature engineering and labeling of the data. This is also partially related to data wrangling.
Target Variable In ML, the target (or response) variable is the characteristic known during the learning phase that has to be predicted for new data by the learned model. In classification problems, it can take a binary value for two-class problems, a single label for multi-class problems, or even a set of labels for multi-label problems. In regression problems, it is generally a continuous variable. On the one hand, Krause et al. [KDS * 17] propose a workflow to help practitioners to examine, diagnose, and explain the decisions made by a binary classifier. In their approach, instance-level explanations are obtained based on local feature significance measures that explain single instances. With these findings as a basis, they develop visualizations that lead the users to important areas of investigation.