dataesr/scientific-paragraphs-categorization huggingface dataset:
g. , between radiation and temperature). The contribution of uncertainties in historic weather data sets on crop model skill is to be evaluated elsewhere and is not part of the objectives here. All input and harmonization targets are supplied at a regular grid with 0. 5 • resolution. Weather data are supplied at daily resolution. Some models use a different spatial or temporal resolution for which they had to find individual solutions. See text and Table  S2  in the Supplement for further detail. Each modeling group is asked to use their own soil data and parameterization  (Elliott et al. , 2015). Yield simulations are conducted for the four major crops wheat, maize, rice and soybean depending on model capacities. Some groups could not supply data for all crops or harmonization settings (see Table  2 ).
Each modeling group supplied data for each crop for all land grid cells (up to 62 911 grid cells) with separate simulations for purely rain-fed conditions and for conditions with full irrigation. Full irrigation does not necessarily imply the absence of water stress in all models, if, e. g. , the atmospheric water vapor pressure deficit exceeds the plant's physical capacity to transpire water. Model irrigation is triggered on demand (Supplement Table  S2 ) independent of the availability of irrigation water  (Elliott et al. , 2015). Following FAO reporting standards, we are not reporting simulated yield data as calendar aggregates but as a time series of annual growing seasons. In this way, we avoid that individual calendar years can have two harvests (one shortly after 1 January and one shortly before 31 December) and others with zero harvest, which would greatly increase the variability in the reported simulated crop yields and would be inconsistent with FAO data.
Reference data We use two different data sets for the evaluation of the GGCMs. The FAO data (FAOstat data, 2014) are used for national and global-scale model evaluation and are available at these scales from 1961 to 2013. For some countries, production data and/or harvested areas have been estimated by the FAO rather than reported  (FAOstat data, 2014). For spatially resolved detail we use the data published by  Ray et al. (2012, henceforth "Ray2012") , as that allows for direct comparison with the regression model analysis of  Ray et al. (2015, henceforth "Ray2015"). The Ray2012 data span 1961-2008 and were aggregated from its original resolution of 5 arcmin to the 0. 5 • GGCMI standard resolution, weighted by production. Both production and harvested area data are collected at sub-national level for 51 countries in the Ray data and changes in productivity thus reflect both dynamics in area and production.
Metrics used In the analysis we largely focus on time series correlation of simulated and reference crop yields, given that the main application of gridded crop models at the global scale is related to studies on climate change impacts, where we expect models to respond reasonably to changes in atmospheric conditions (weather, climate). The main metric used is therefore the time series correlation analysis, employing the Pearson's product moment correlation coefficient (henceforth "correlation coefficient"). Significance levels (p values) are reported based on a t distribution with length(x) -2 • of freedom. Given difficulties in attributing sequences of growing periods to the calendar year in both FAO statistics 2  and in simulated data where groups also interpreted the reported standards differently, we test if the time series correlation can be substantially improved by shifting the times series by 1 year.
To disentangle the contribution of the spatial vs. the temporal variability to the Taylor diagram, we also compute two variants of these diagrams, which focus on temporal or spatial variability only. For the temporaldynamics-only variant, we remove the national means from all de-trended time series so that all national time series have a mean of zero and thus display no differences in this respect. For the space-dynamics-only variant, we average time series so that we compute the metrics with one national mean value per country only, ignoring possible changes in data quality over the time series. For plotting Taylor diagrams, we use the taylor. diagram function of the R package plotrix  (Lemon, 2006) , which we have modified to allow for weighted correlation and for testing of significance levels.
3 ) and two others achieve significant correlations under fullharm (EPIC-BOKU, PEPIC; Fig. S5 ), but none of the models reach statistical significance under the harm-suffN setting (Fig. S6 ). PRYSBI2's correlation improves substantially (from 0. 53* to 0. 83***) if trends are maintained. Of the 13 GGCMs that submitted data for soybean (Table 2), seven achieve significant correlation to FAO statistics of variations in global soybean productivity (correlation coefficients between 0. 64 and 0. 41). Under harmonization, two more models reach statistical significance levels (LPJ-GUESS, PEPIC; Figs. S7-S8), and PRYSBI2 reaches significant correlations (0. 57**) if trends are not removed. 2 ). The (sb) flag indicates that the time series had been shifted backwards by a year to achieve a better match.
Pixel scale At the pixel scale, reference data uncertainty increases substantially, as the two available data sets are essentially modeland observation-based interpolations of (sub-)national yield  statistics, and neither of the two is independent from FAO national data. Differences between the two gridded yield reference data sets  (Iizumi et al. , 2014b; Ray et al. , 2012)  are expressed via a time series correlation analysis after removing trends via a moving average (see Methods, Fig. 9 ). Independent of the harmonization setting, the GGCMI model ensemble (selecting the best correlation per pixel across the different GGCMs and harmonization settings) finds statistically significant correlations (p < 0. 1) with Ray2012 in most of the currently cropped areas for all four crops analyzed here (Fig.
Gray areas depict areas where there is no statistically significant correlation between the two data sets (p > 0. 1), white areas have no yield data for that crop in at least one of the two data sets. Panel (a) shows coefficients of determination (R 2 ) for maize, (b) for wheat, (c) for rice, (d) for soybean. Figure  10. Analysis of time series correlation between the GGCM ensemble simulations for maize (selecting best correlation across the GGCMs per grid cell) and the Ray2012 reference data set after removing trends via a moving average (see methods). Gray areas depict areas where none of the GGCMs find a statistically significant correlation; white areas have no yield data for that crop in Ray2012 data sets. Panel (a) shows coefficients of determination (R 2 ) for the default setting, (b) for the fullharm setting, (c) for the harm-suffN setting, and (d) shows the original coefficients of determination as reported by  Ray et al.
(2015) , "Ray X ensemble" is as Ray2015 but only for the area where also the GGCMI ensemble reports significant correlation coefficients. The distribution is weighted by production, following the Ray2012 data set. Numbers at the top describe the fraction of the total harvested area for which significant correlations could be found, which ranges between 96 % (ensemble best, default), 63 % and 19 % for the individual GGCMs, and 68 % for  Ray et al. (2015). biases often do not affect the model's skill to respond to interannual variation in weather conditions. However, it also often leads to greater variance in the time series (orange symbols move outwards relative to blue symbols in Figs. S28, S31, S34, S37). The effect of harmonization is dependent not only on the individual GGCM's sensitivity to these assumptions but also to the difference between the default and the harmonized settings with respect to growing season and fertilizer input.
5) to positive (r > 0. 5) for PEGASUS fullharm maize simulations in China (Figs. 6  and S12 ). We acknowledge that some of this is owing to the relatively vague definition of how FAO yields are attributed to calendar years and how this matches with assumed growing periods in the GGCM simulations. However, this seems to be an important improvement to be achieved by future global crop modeling studies. The GGCMI phase I protocols request that data are reported as a series of growing season harvests  (Elliott et al. , 2015)  rather than calendar years to avoid complications with harvest year attribution if harvest occurs around the end of the calendar year. Moreover, years are removed from the record if sowing occurred during the spinup; i. e. , part of the growing season is not within the supplied weather input.
g. , eddy flux tower measurements that can help to get good simulation results for good reasons. Therefore, it remains crucial to also test global-scale models against detailed data from experiments to build trust in the underlying mechanisms. This point-scale evaluation of models has been performed for several of the GGCMs engaged here and is not the subject of this study (e. g. ,  Gaiser et al. , 2010; Izaurralde et al. , 2006; Jones et al. , 2003). We propose that future global or large-scale gridded crop models are tested against the GGCMI model ensemble and the reference data used here to establish a benchmark for model evaluation and future model development. This cannot overcome the shortage in suitable reference data, but it provides a first benchmark against which global gridded crop models can be tested.
Figure11. Box plot of R 2 distribution for each GGCM-harmonization setting for maize. Boxes span the interquartile range (25-75 percentiles); whiskers expand to the most remote value within 1. 5 times the interquartile range. Values outside this range are considered outliers and are depicted as dots. The "ensemble best" shows the GGCMI skill-based (correlation coefficient) ensemble, "ensemble X Ray" is the same but only for those pixels where, and both are not independent from FAO national data also report significant correlations, "Ray2015" is the distribution of values as published by Ray et al. (2015) , "Ray X ensemble" is as Ray2015 but only for the area where also the GGCMI ensemble reports significant correlation coefficients. The distribution is weighted by production, following the Ray2012 data set.
Acknowledgements. We acknowledge the support and data provision by the  Agricultural Intercomparison and Improvement Project (AgMIP). This work was completed in part with resources provided by the  University of Chicago Research Computing Center. C. Müller  acknowledges financial support from the  MACMIT  project ( 01LN1317A ) funded through the  German Federal Ministry of Education and Research (BMBF). A. Arneth  and  T. A. M. Pugh  were supported by the  European Commission 's  7th Framework Programme  under grant agreement no. 603542  ( LUC4C ) and by the  Helmholtz Association  through its research program ATMO. P. Ciais acknowledges support from the  European Research Council Synergy  grant  ERC-2013-SyG-610028 IMBALANCE-P. Edited by: M. -H. Lo Reviewed by: two anonymous referees.
Code and data availability. The code of the processing scripts is available via github at https://github. com/RDCEP/ggcmi. The evaluation pipeline is available at https://mygeohub. org/tools/ ggcmevaluation and can be used by crop modelers to evaluate their model performance in the same way as in this paper. Model output data is available via the GGCMI data archive at http://www. rdcep. org/research-projects/ggcmi.
Numerical experiment We have implemented our interval B&B algorithm with dominance contractor in C++ using the RealPaver  ( Granvilliers & Benhamou, 2006  ) API, which features routines for constraint propagation and contractors, and using the Gaol  ( Goualard, 2006 )  library for interval arithmetic. These experiments have been run on a computer under Linux Ubuntu version 14. 10 64-bit, with processor Intel i7-4702MQ 2. 20 gigahertz and 8 gigabyte of RAM. The implementation follows the description given in Section 4. The portion ρ used in the dominance divider and dominance full divider is set to 0. 125, which proved to be an appropriate default setting for our benchmark. The improvement factor used to interrupt long-running fix-point constraint propagation is set to 0. 75 and also considers the prescribed precision on the decision variables given by the termination criterion in order to avoid unnecessary pruning of small enough variable domains.
Acknowledgment The authors would like to thanks  Brice Chevalier , whose earlier work on interval B&Bin nonlinear multiobjective optimization during his master thesis has been the foundation of the developments proposed here. The first author is also grateful to the  Portuguese Foundation for Science and Technology  for having partially granted this work through the project  PROCURE  (  Probabilistic Constraints for Uncertainty Reasoning in Science and Engineering Applications ), ref. PTDC/EEI-CTP/1403/2012.
Proposed System The proposed system is based on a human-scale, realtime VE which supports the resolution of interactive 3D-object layouts using ongoing communication with an efficient constraint-solver (Gecode) written in C++ language. The selection of objects and constraints as well as a user's 3D manipulation is converted into queries sent to the solver. The solver's outputs (positions of objects) allow real-time automatic reconfiguration of the 3D scene. Figure  1  illustrates the general resolution process and the link between the 3D environment and the solver. The events exchange is managed by a Manager class which creates and sends the corresponding event to the visualization module (Process 3D) and the resolution module (Process Solver). Each module sends the requests to the appropriate thread.
To ensure real-time interaction, each module runs on a different thread. The request sent to the Thread 3D is to display the selected objects. The Thread Solver class creates a CSP in which the variables are the positions of the center of selected objects, and the constraints are those initially specified. The Gecode solver is called to find a solution to the proposed problem. The Thread Solver class will be informed about the results and will transmit the corresponding request to the upper level (Process Solver). Thus the Process Solver class informs the Manager class that a solution has been found (through the event "Solution found"). The resulting 3D positions of each object are then forwarded level by level until the VE reconfigures itself by updating the 3D scene.
EXPERIMENTAL STUDY Previous studies did not provide detailed analysis of the human performance, and particularly, nor the comparison between manual (without solver) and assisted (with solver) 3D-layout tasks. There are multiple issues to be considered when integrating a constraint solver in a VE: data exchange between the constraint solver and VE, user interaction, and effective and sufficient user assistance. An experimental study was conducted to evaluate the effectiveness of a combination of a solver and a VE, with special focus on the interaction technique. Various data were collected, including the effects on task completion time and placement accuracy, as well as subjective preference from users. The study compared two approaches, namely manual and assisted, with two different levels of layout difficulty, simple and complex.
The following hypotheses were proposed: 1. H1: the participants are satisfied with the solver assistance. 2. H2: the interaction with the solver during the layout task is easy. 3. H3: the participants solve the layout task faster with the solver. 4. H4: the participants make fewer placement errors with the solver. A NASA-TLX questionnaire was used to access user's satisfaction (H1) and the ease of interaction with the solver (H2). Task completion data was collected for H3. The satisfaction of the hypothesis H4 is relatively obvious since a solver does not make errors. However, it is important to confirm it in the interest of the solver integration.
Collected Data To investigate the effect of the solver integration on task performance, several measures were utilized that can be categorized as subjective preference (via questionnaire) and task performance in terms of completion time and layout errors. • Satisfaction. Using a seven-points Likert scale (i. e. , 1 for "very low" to 7 for "very high"), subjects were asked to rate their satisfaction of the layout process and the solution proposed by the solver. It should be noted that the satisfaction concerns the final proposed solution and not the user interaction. • Assistance. Using the same seven-points Likert scale, subjects were asked to express their opinions about the assistance provided by the solver. The aim was to know to what extent the solver's integration assisted subjects and facilitated the layout tasks.
• Task Performance. Although the main objective of the study was to assess to what extent the subjects appreciated the assistance provided by the solver, objective data (completion time and number of errors) was also collected. Completion time is defined as the time between the first pressing on the B button (Wiimote TM ) and the moment when the participant considers the task achieved. Layout error can be evaluated by two parameters: 1. Placement Error: This type of error is not related to any of the constraints previously described. The system checks whether or not each object was included in the 3D scene (large 3D cube) based on the x, y, and z coordinates. The number of occurrences of this error was calculated for each experimental condition. 2. Distance Error: Since the Minimum Distance constraint exists in all conditions, the error generated by the dissatisfaction of this constraint was calculated.
(2016). The IP is an excellent test case for remote sensing of SSM, as its two characteristic climate regimes (oceanic and Mediterranean) result in a strong contrast in soil water content. Furthermore, SSM is a critical variable regarding water resources, especially in the IP, which makes this study even more necessary. The data from SMOS and the LSMs used in this paper will be presented in the next section. A methodology section follows describing the data filtering and sampling processes carried out, together with the analysis performed to compare TBs. Afterwards, results will be presented. First, modelled and measured TBs will be compared. Second, their difference will be characterized spatially and temporally and certain hypotheses to explain the differences found in the TB comparison will be analysed.
For instance, its first layer's depth is of the order of millimetres, while H-TESSEL's is of centimetres. In order to evaluate the role of these differences in the vertical discretization and the. 391, 0. 978, 2. 151, 4. 497, 9. 189, 18. 570, 37. 340, 74. 880, 150, and 200; 21, 72, and 189. * Layer depths of H-TESSEL's hydrological scheme (cm): 7, 21, 72, and 189. LSMs, we performed the sensitivity analysis detailed in the next paragraph. In addition to the CMEM simulations performed to model TB OR and TB HT using the configurations indicated in Table 1, the following simulations were carried out to test whether parameterization assumptions could affect the resulting TBs. -Simulation 1: TB HT(VC) , where the subscript "VC" stands for "vegetation cover". Vegetation cover is a key input. Since this parameter is directly related to land-surface emissivity, the effects of a different vegetation cover were tested on TB HT.
For this matter, a new set of TBs was modelled using H-TESSEL's state variables with the same configuration as detailed in Table  1 , except for the vegetation cover input, where H-TESSEL's prescribed vegetation  (Boussetta et al. , 2013)  was considered. One of the differences between this input and the ECOCLIMAP database (used in the original configuration) is that the former consists of 20 vegetation types, while the latter considers 7. -Simulation 2: TB OR(SD) , where the subscript "SD" stands for "soil discretization". The impact of a coarser soil representation on modelled TBs was tested by recomputing TB OR using OR-CHIDEE's state variables averaged to 3 soil layers: upper (9 cm), medium (66 cm), and lower (125 cm). -Simulation 3: TB OR(FW) , where the subscript "FW" stands for "Fresnel Wigneron".
As for the radiative balance, the available energy at the surface is one of the major drivers of LST. We chose to verify this variable in this study for two reasons. First, it provides a good summary of the surface energy balance. Second, it is a key parameter in CMEM's estimation of TB. Therefore, its analysis will indicate whether the LSM thermodynamics shows biases with spatio-temporal characteristics similar to those from TBs. The independent datasets used for validation are -P from the E-OBS dataset  (Haylock et al. , 2008) , and -LST provided by the LandSAF product (http://landsaf. meteo. pt). It should be noted that these products have errors which must be taken into account when used. For example, E-OBS data can be over-smoothed depending on the station network density  (Hofstra et al.
LST and precipitation errors Precipitation and LST data are used to explore possible causes of the difference between measured and modelled TBs. Errors are calculated with respect to independent datasets. The dominant error pattern of each variable is computed via EOF analysis and compared with the dominant pattern of the ORCHIDEE and H-TESSEL TB errors. If similarities can be identified, then possible causal links between these variables and the TB error can be explored. The precipitation error is calculated as the difference between the P provided by the ERA-Interim forcing and the E-OBS independent dataset. The LST errors are computed as the difference between modelled LST (from ORCHIDEE or H-TESSEL) and the EUMETSAT LandSAF product (http: //landsaf. meteo. pt).
Instrumental issues from SMOS could also explain the differences in TB spatial structures, in case these are not of a climatological or geophysical nature. For example, one of the most important causes of noise in SMOS surface soil moisture is radio-frequency interferences (RFIs). Daganzo-Eusebio et al. (2013)  describe their effect on SMOS data. Some of them are difficult to detect and, thus, RFIs may not be properly filtered out. For instance,  Dente et al. (2012)  identified an irregular angular pattern in the TBs affecting data from the L1C product used to retrieve soil moisture. In their opinion, this was caused by weak RFIs which were not correctly filtered. Another explanation could be antenna pattern errors as SMOS TB seasonal and latitudinal drifts detailed in  Oliva et al. (2013).
Conclusions TBs of the SMOS Level 1C product were compared to two sets of modelled TBs. The latter were obtained using simulated state variables (from the ORCHIDEE and H-TESSEL LSMs) and a radiative transfer model, CMEM. The study was carried out over the Iberian Peninsula (IP) for the period 2010 to 2012. On the one hand, a temporal correlation analysis between measured and modelled data shows that there is a good agreement in their temporal evolution. However, this diagnostic is mainly driven by the TB's signal synoptic variability, as occurs with SSM  (Polcher et al. , 2016). On the other hand, a spatial correlation analysis detected a large mismatch between the TB spatial structures provided by models and observations. An EOF analysis of the error between modelled and measured TBs suggests that the inconsistency is not limited to a particular LSM.