dataesr/scientific-paragraphs-categorization huggingface dataset:
3%) compared to the non-enriched animals (40. 7%). This result suggests that enrichment might actually improve replicability, although the reason for this remains elusive. The heterogenization concept was not received with outright rejection in the meeting, perhaps surprisingly in light of the importance usually prescribed to standardization. Notably, it was argued that strict "universal" standardization of laboratory and test environment is unfeasible, and that widespread adoption of few standard protocols, apparatuses and test environments diminishes, rather than enriches, understanding of the assayed behavior. Any attempt to repeat an experiment can never perfectly replicate the original conditions, but this is probably a good thing since it will increase generalizability of findings. Phenotyping databases (e.
Replicability across laboratories: can it be ensured? The issue of replicability across laboratories, an immediate form of GxE, is one of the most critical in mouse phenotyping, because modern science does not normally accept experimental results that are idiosyncratic to a certain location, even if they are replicable within this location. This is why the results and conclusions of the  Crabbe et al. (1999)  report were disturbing for many researchers in the field, and in other fields as well. As previously discussed in Section 6 there is currently no consensus even over the proper criteria to evaluate replicability across laboratories. Studies on the subject are few because they typically require collaboration of several laboratories and institutions, although they are becoming more and more common, thanks to data sharing and community efforts (Section 3).
Unknown sources of interaction are unavoidable (e. g. ,  Würbel, 2002). How can the interaction with the laboratory be estimated? One possibility is using as a surrogate the variability across time within a single laboratory  (Karp et al. , 2014)  or heterogenization (Section 7). However, controlled heterogenization uses effects we know about, while true interaction with laboratory might involve factors we are not aware of at all. Another proposal  (Kafkafi et al. , 2017)  is to make use of multi-lab data from large and evolving phenotyping databases, such as the MPD and the IMPC. Such a database can calculate the interaction and publish it for use by scientists running phenotyping experiments in their own laboratories. This calculation has to be repeated for each phenotypic measure separately.
A website was demonstrated in which any researchers conducting a phenotyping experiment can upload their results, get an updated estimate of the interaction for the relevant phenotypic measure, perform a GxL-adjustment and get an adjusted p-value. The researchers are then given an option to leave their data in the database, thus enriching it and providing a better estimate, based on more laboratories, for future users. As in other file-sharing and community-effort strategies, GxL-adjustment has ethical implications: by employing previous data of the same phenotypes from other laboratories and other research questions, instead of replicating the experimental study, it may eventually reduce the number of experimental animals without compromising replicability. Ultimately, the replicability of a phenotyping discovery can be guaranteed only by testing it in additional laboratories.
Even in these simple cases, ways to quantify replicability, such as the "r-value", are still in need of development and acceptance by the scientific community. In cases when testing was performed in a single lab only, it may still be possible to enhance replicability, or estimate its prospects in a better way. Several directions were proposed: heterogenizing the experimental setup, splitting the duration of the experiments to different time units, and using external estimates of the interaction from phenotyping database. All these may serve to get more realistic estimates of the replicability standard deviation, and better (not too optimistic) estimates of the relevant standard errors.
, 2014). Combinatorial integration of these gene sets revealed a single QTL positional candidate gene common to both preference and withdrawal. This QTL seems to have a replicable phenotypic effect -it was confirmed by a validation study in knockout mice mutated for this locus. Since discoveries in this strategy can be based on multiple studies across laboratories, species, and phenotyping protocols, they have a better chance to be replicable, generalizable, and translatable. However, the complex integration of multiple data sets in this strategy makes it difficult to construct statistical models for estimating how much the replicability may be improved. Addressing validity issues might be critical when deciding how to share and integrate behavioral data, e. g. , when constructing "controlled vocabularies" and "ontologies" used for querying and searching phenotyping databases.
In databases storing multiple phenotypic measures from multiple studies and laboratories, the implications of these approaches for replicability and reproducibility may for the first time be methodically investigated. It was further suggested to consider replicability as a "gold standard", and use behavioral data across several laboratories in order to explicitly redesign behavioral constructs and ontologies for increased replicability  (Benjamini et al. , 2010). In this strategy, the issue of replicability in behavioral results is turned into an asset rather than a liability -it enables researchers to methodically improve the definition of key behavioral constructs by using the statistical level of replicability as a benchmark, filtering out behavioral results and representations that are less replicable  (Kafkafi et al.
, 2003; Lipkind et al. , 2004). Another type of validity that is highly relevant to preclinical testing is "predictive validity"  (Willner, 1984 (Willner, , 1986) )  or "pharmacological validity" -a response of the phenotypic measure to multiple psychiatric drugs that predicts the response of human disorders and syndromes to the same drugs. Here too it is possible to use behavioral databases in order to explicitly design behavioral measures for high predictive validity, as well as for replicability across laboratories  (Kafkafi et al. , 2014). A requirement for such a strategy, however, is storing detailed and high-quality low-level data, e. g. , locomotor path coordinates in the above examples, rather than merely animal final means. A potentially even more powerful approach may be to explicitly design behavioral measures and constructs to increase generality across species and taxonomic groups.
Any proposed solutions, therefore, should likely be tailored to the particularities of the field. Phenotypic databases, community efforts and other methods of data sharing play important roles, as they can be employed to efficiently assess the severity of the issue, as well as the performances of different proposed solutions. Correct handling of the genotype-environment interaction (GxE) is a key to proper methodology, and depends on the context and objective of the research. GxE can typically be understood and decreased through careful observation of the animals, redesigning the rearing and testing conditions to eliminate adverse effects of irrelevant confounding factors. Especially when testing a hypothesis of a genotype effect, ignoring or mishandling the relevant form of GxE is likely to result in replicability issues and other severe methodological issues.
Extreme standardization of rearing and testing conditions is probably not by itself feasible or helpful as a strategy to eliminate GxE, and might limit the generalizability of any conclusions. Ultimately, the replicability and generalizability of any phenotyping discovery can be guaranteed only by replicating it across additional studies, laboratories and experimental conditions. Even when studies are repeated, there is no single well-established method to quantify the replicability of the results, but large and consistent genotype effect sizes can usually be agreed upon as replicable. In the more common situation where results are available from only one study, it may still be possible to enhance replicability, or estimate its prospects in a better way. Several directions were proposed and discussed at the meeting, including heterogenizing the experimental setup, splitting the duration of the experiments to different time units, and employing external estimates of the interaction from phenotyping database, effectively suggesting an expansion of the field of experimental design.
Acknowledgments This work is supported by  European Research Council  grants FP7/2007-2013  ERC  agreement no [ 294519 ] PSARPS (YB, NK, IG, IJ), by  NIH   AA018776  (EJC), by  NIH   DA045401  (YB, NK) and by the  Spanish Ministry of Economy and Competitivity (Severo Ochoa Center of Excellence  programme  SEV2013-0317  start-up funds to AGM). We thank  Prof. Allan Kalueff  for many useful comments. This paper is the result of a scientific meeting in  Tel Aviv University. Videos of the full lectures can be accessed at the following links.
Conversely, poor hatching rates were observed in crosses involving eggs of C. robusta (16. 6% ± 15. 1% with C. intestinalis and 5% ± 5. 7% with C. roulei). And no hatching was observed when C. edwardsi was involved except for one cross made with oocytes of C. edwardsi and sperm of C. intestinalis but with low success (8. 5% of hatching). Survival and growth rate. For subsequent development, the geographic origin of C. robusta was not distinguished as results were identical (data not shown). Survival was first examined at D + 10 by comparing the number of juveniles with the number of post-larvae settled at D + 3. At D + 10, the three series were composed of 12, 10 and 17 crosses respectively, with variable categories of crosses within each. A non-parametric Kruskal-Walis test showed significant differences among series (p = 0.
All adults (N = 33) used in this experiment were sequenced on the cytochrome c oxidase subunit I (COI) mitochondrial gene. The COI fragment was amplified using primer sequences and protocols obtained from Nydam and Harrison 11. We also sequenced four additional individuals of C. edwardsi collected at the same time but which never reach sexual maturity. Purification of PCR products and sequencing reactions were performed by Eurofins. All PCR products were sequenced in both directions. Sequences obtained were edited using CodonCode Aligner v. 4. 0. 2 (CodonCode Corporation, MA). In addition, we included in the dataset seven sequences (all different) of C. savignyi that were obtained as part of a side study. This species was used as outgroup of the targeted Ciona spp. Finally, we included six and 11 sequences (all different) obtained by Bouchemousse, et al.
17   S1  which also provides Genbank accession numbers. The sequences were aligned using BioEdit v. 7. 1. 981  33 leading to a 717 base pairs final alignment (all sequences were 737 bp except for the sequences of C. roulei from Nydam and Harrison 11 shorter by 20 base pairs at the 5' start of the sequence). A tree was built with MEGA7  34  with complete deletion of missing data. Goodness of fit with various evolution model was first tested: based on BIC criteria, the best model explaining the data is the Tamura 3-parameter (T92) model with 67% of the sites evolutionarily invariable. Using AIC criteria, the Best model (ΔAIC > 10) is the General Time Reversible (GTR) model with 63% of the sites evolutionarily invar-(I). The two models were used in subsequent analyses. Results were identical; only results based on T92 are shown.
Acknowledgements We are very grateful to the divers of the  Marine Operations Departments of the Station Biologique of Roscoff and of the laboratory Arago in Banyuls-Sur-Mer , for sampling in the field. We are also grateful to  Pascal Romans  and  Michael Fuentes  for additional sampling and tank installation and to  Nancy Trouillard  for algae production at the  laboratory Arago in Banyuls-Sur-Mer. The project was supported by the  ANR  project  HySea  (no. ANR-12-BSV7-0011 ) and  EMBRC-France. MM acknowledges a PhD grant from the  University Pierre et Marie Curie (Sorbonne Universités ).
Although considerable progress has been made since  Pitman (2003) , land evaporation remains to be one of the most uncertain components of the global hydrological cycle to date (e. g. Fisher et al 2017). It is hence not surprising that a recent ensemble product of global ET estimates still reveals substantial uncertainties, which are comparable to the magnitude of uncertainties in precipitation estimates  Mueller et al (2013). To thoroughly analyse extreme events within the ISIMIP2a framework, it is thus a prerequisite to precisely assess the magnitude of common ensemble statistics (mean, median and interquartile ranges, IQRs) of presently available ET estimates across datasets/models and sectors, and to further attempt to identify potential causes for differences between these estimates. Uncertainties in estimates of ET can be due to multiple interrelated factors, including (but not limited to) the choice of the model or the forcing, the data category (model-based estimates from a specific sector, diagnostic or reanalysis-based estimates), the number of soil layers in the model and the ET scheme.
Studies from the recent past have mainly focussed on the latter issue: the difficulties of choosing the most appropriate ET scheme for specific applications (e. g. Kay et al 2013 , Bormann 2011)  Given these uncertainties, it is highly appropriate to assess in more detail whether the differences in ET schemes govern the overall differences across available ET estimates, or whether some of the other factors prevail. Here we make use of the ISIMIP2a framework to assess selected hydrological extreme events by taking uncertainties into account that arise from difficulties in simulating global land ET. The analysis makes use of a large ensemble of ET datasets including models participating in the the global water, biomes and agriculture sectors, and also including other (independent) estimates from the following sources (introduced in more detail in section 2): i.
An ensemble of diagnostic datasets, ii. the EartH2Observe This variety of datasets allows us to assess the full range of uncertainties captured by simulated or observation-based historical records of ET, and to identify strengths and weaknesses of individual groups of datasets with respect to their potential to reflect regional ET anomalies during extreme events. The ensemble is particularly suitable to draw valuable conclusions on structural differences among the datasets (e. g. to measure the influence of the meteorological forcing dataset employed in each of the model-based estimates). The remainder of this paper is structured as follows. After a brief introduction to all input datasets in section 2, we present our methodological approach (section 3). This is followed by the presentation and discussion of spatial and spatio-temporal variabilities across the analysed datasets by means of a cluster analysis (section 4.
Data Table 1 lists all datasets used in this study. In total, we assess 11 diagnostic datasets, nine models from E2O, 12 models from the ISIMIP2a agriculture sector, seven models from the ISIMIP2a biomes sector, 11 models from the ISIMIP2a global water sector, two land reanalyses and one composite dataset (the latter consisting of four different realizations). Note that ET estimates from all datasets except from the ISIMIP2a crop model simulations (labelled with asterisks ( * ) in table 1) correspond to estimates of total ET at monthly resolution (hereafter denoted as ET tot ). Besides other information, the table also shows the category that each dataset is associated with. In the following, we provide some more information for each of the data categories. Please refer to the publications listed in table 1 for more detailed information on individual datasets.
Model simulations and land reanalyses We analyse ET from historical simulations of the ISIMIP2a project using all models from the global water, biomes and agriculture sectors that simulate ET using the model's default ET scheme. The simulations used in this analysis are based on three distinct meteorological forcing datasets: the Global Soil Wetness Project Phase 3 (GSWP3, http://hydro. iis. u-tokyo. ac. jp/GSWP3/), the Princeton Global Meteorological Forcing Dataset version 2  (PGMFD v. 2, Sheffield et al 2006)   in table 1, ISIMIP2a crop model simulations are provided as pure crop runs (i. e. assuming a specific crop is growing all over the world), implying that the ET output from these models corresponds to the amount of water evaporated or transpired from a specific crop type under a given irrigation scenario.
Diagnostic estimates We use an ensemble of recent diagnostic datasets suitable for identifying potential biases in model-based estimates of ET (see table  1 ). This ensemble consists of some well established ET estimates from the recent past (a subset of those diagnostic datasets has also been used to generate the ensemble of diagnostic ET in LFE, see section 2. 3), but also includes more recent datasets such as GLEAM v3. 1 Martens et al  (2016). Please refer to the references listed in table 1 for further details on the individual datasets. Please note that the version of Fluxnet-MTE employed here uses a modified temperature and precipitation forcing compared to the previous version presented in  Jung et al (2009).
Composite estimates: LandFlux-EVAL LandFlux-EVAL (LFE) is an ensemble based land ET product that itself is based on individual diagnostic, model and reanalysis products available in the early 2010s Mueller et al  (2013) , which we use here as a reference dataset (96 citations as of September 14, 2017, according to Web of Science). Although we cannot argue that LFE is free of biases, we can make the conservative assumption that its ensemble statistics (in particular the provided quantile statistics) are a suitable estimate for quantifying the probable range of ET over land. Please note that some of the diagnostic and model datasets used in the LFE ensemble are also included as individual datasets in this analysis, and hence there is some degree of dependency between the diagnostic (model based) estimates and both LFE DIAGNOSTIC (LFE LSM) and LFE ALL.
Study areas Besides global land (excluding Antarctica), we focus on the following regions (see In approach (a), we first reform the elements of the four-dimensional input field (latitude, longitude, time and dataset) to a two-dimensional matrix X of size m by n (where n corresponds to the number of all non-missing observations) and transform this into anomalies (based on all time steps remaining) ΔX. To only preserve the dominant modes of ensemble variability, a singular value decomposition (SVD) is performed on ΔX and truncated to t = 9 modes (which is well within the range of suitable truncation values found in  Sanderson et al (2015) ). The corresponding loadings matrix U (size m by t) can now be used to derive the difference matrix D svd by calculating pairwise Euclidean distances as 𝛿 𝑖𝑗 = { 𝑡 ∑ 𝑙=1 [𝐔 (𝑖, 𝑙) -𝐔 (𝑗, 𝑙)] 2 } 1∕2.
In approach (b), we simply calculate the temporal average (denoted 'tavg') and reform the result to a two-dimensional matrix Y of size m by p (where p corresponds to the number of remaining grid cells, which is the same for all datasets). We then calculate pairwise Euclidean distances  ij in between datasets i and j, resulting in the difference matrix D tavg (dimensioned m by m): 𝛿 𝑖𝑗 = { 𝑝 ∑ 𝑙=1 [𝐘 (𝑖, 𝑙) -𝐘 (𝑗, 𝑙)] 2 } 1∕2. The distance matrices D svd and D tavg are then subjected to hierarchical clustering by applying the hclust function in R statistics software with Ward's clustering criterion  (Ward 1963, ward. D2). The results are visualized in R using the Complex Heatmap package (version 1. 14. 0, available at https://bioconductor. org/packages/release/bioc/html/ ComplexHeatmap. html).
Spatio-temporal patterns of global land ET Figure  1  shows global patterns of time averaged total ET (i. e. excluding crop models) grouped by data category. Spatial patterns of the ensemble averages are mostly similar among data categories and delineate the average location of the governing hydroclimatic regimes, such as the Intertropical Convergence Zone. The ensemble spread (as expressed by the IQR, i. e. the difference between the 75th and 25th percentile) in tropical rainforest regions is generally largest for the ISIMIP simulations and LFE ALL, whereas the uncertainties (in absolute values) in this region are considerably less pronounced in the diagnostic and E2O ensemble. Similarly, the spatial patterns of the relative ensemble spread (shown by means of the quartile coefficient of dispersion, QCD, which is a relative measure of dispersion, Bonett 2006) in the diagnostic ensemble are very well represented by both ISIMIP sectors and by LFE ALL.
Although the magnitude of the uncertainties in the diagnostic ensemble is smaller than in the ISIMIP sectors, it is still higher than the relative ensemble spread in E2O. Note, however, that these small inter-model differences are linked to the fact that the E2O models are all forced by WFDEI, while the ISIMIP ensembles consist of three different forcings. The results of the SVD-based cluster analyses are shown in figures 2 (ET tot ) and figure 3 (ET maize ). As the associated clusters explain most of the variability among the assessed datasets (i. e. incorporating variability in both their spatialand temporal domains), we treat those as our main results, but occasionally draw comparisons to the clustering results based on time-averaged ET (visualized in figures A3 and figure  A4 ). In the following, we highlight important details of the cluster diagrams by discussing individual parameters potentially having a measurable influence on the differences among the analysed ensemble.
At first glance, we notice a few apparent outliers in the displayed difference matrices, most strikingly for WB-MTE (ET tot ), PM-MU CSIRO, MPI-HM and the crop models LPJ-GUESS and ORCHIDEE-CROP (ET maize only). As the differences among the contributing models are affected not only by the modelling structure but also the calibration process taken, it is not surprising to see such apparent differences between the non-calibrated models LPJ-GUESS and ORCHIDEE-CROP and all other datasets (the less pronounced signal for ORCHIDEE-CROP is potentially due to the fact that this model includes Nitrogen cycling). We also see that the long-term average global land mean ET of these models is not an outlier, suggesting that the differences are mainly due to anomalous patterns in the spatial domain. This is also supported by the clustering results of time-averaged ET, where outliers in the dendrogram mostly coincide with anomalies in the long-term mean (e.
g. WANG-ET for ET tot , figure  A3  or PEGASUS for ET maize , figure  A4 ). The meteorological forcing has, in general, a very noticeable impact on ET estimates when considering variability across both space and time. There are a number of clusters whose members are almost exclusively based on the same forcing (e. g. all PGMFD-forced simulations form a single cluster for ET tot ; cluster 2 in figure  2 ), underlining the dominant role of this parameter. However, the forcing apparently only has very minor influence on the spatial variability of timeaveraged ET tot (figure  A3 ), indicating that most of the differences in between the employed forcing datasets are due to differences in the temporal evolution of ET. For ET maize , there are a few more models for which the forcing dominates the differences, arguably due to a higher sensitivity of these models to diurnal forcings and other parameters which play a more important role when considering crop-specific ET aggregated over growing seasons.