dataesr/scientific-paragraphs-categorization huggingface dataset:
g. Soriano-Disla et al. , 2014). They have also shown that the predictions of some attributes, such as the soil's pH and the contents of plant nutrients, cannot be predicted consistently  (Stenberg et al. , 2010). But there has been much duplication of objectives and rather few publications describing significant advances or novel applications. It seems that the adoption of vis-NIR in laboratories is, in practice, only incremental and still waiting to happen. These are reasons for the growing interest in an international database of vis-NIR spectra linked to information on the soil's composition. The database might then be used to further the research on soil vis-NIR spectroscopy and for the prediction of the soil's attributes, condition and functions where measurements of those qualities are lacking and would be too expensive to make using conventional laboratory methods  (Viscarra Rossel et al.
applications at a range of spatial scales, in the laboratory, in the field and from the air (Fig. 1 ). The scientists in this core group (Table  1 , names in bold text) discussed how the project might proceed and produced the general guidelines, standards and protocols for the project and for the consistent measurement of spectra in the laboratory (see Appendices A, B). Here we report on this global effort and our findings thus far. We describe the development of the global vis-NIR spectral library. We show that the information it contains can be used to characterize soil and its variability and diversity globally, and that by deriving a spectral classification we can describe the associations between spectra, soil, land cover and geography. We also show the usefulness of the global database for predicting soil attributes, such as soil organic and inorganic carbon, clay, silt, sand and iron contents, cation exchange capacity and pH.
The global vis-NIR spectral library For spectra to be included in the database, we requested that they be from air dry ≤ 2 mm soil and in the range between 350 and 2500 nm recorded in intervals of one, two, five or 10 nm. We requested a minimum set of analytical data, geographic location and metadata, but they were not always supplied. We asked contributors that the spectra be representative of the variation in their spectroscopic databases and if possible the variation of soil in their countries. Contributors were provided with guidelines, minimum requirements and the measurement protocol for consistent measurement of spectra in the laboratory. These are shown in Appendices A and B. To date, 23 631 soil spectra have been contributed to the database by around 45 soil scientists and researchers from 35 institutions.
The contributors to the database so far, by country and continent are listed in Table  1. The number of spectra by both country and continent are given in Table  2. The global database has spectra from 92 countries, representing seven continents (Table  2 ). It includes spectra from soil in the World Soil Information (ISRIC) collection, which were recorded by the World Agroforestry Centre (ICRAF)  (ICRAF, 2015). It also contains spectra with corresponding soil attribute data from other published, multi-national, national, regional and local databases. They are listed in Table  1. The geographic locations of the spectra in the database are shown in Fig. 3. There are many large gaps in Fig. 3 , and many countries are poorly represented with only very few spectra. We encourage participation from colleagues who work in regions where we have these gaps, particularly, we would like contributions from counties in Central and South America, Mexico, Canada, Russia and Eastern Europe, Africa and Asia.
Around 15% of the spectra have information on the soil horizons from where they originate, 95% are assigned a soil classification using the FAO-WRB system (IUSS Working  Group WRB, 2006). Land cover is recorded for around 80% of the samples. The number of samples in the database by WRB major soil groups and land cover type are listed in Table  3. Around 80% of the samples have measurements of soil organic carbon and clay content, 65% have pH measured in water, around 50% have measurements of silt and sand contents, 30% have pH measured in calcium chloride and cation exchange capacity (CEC) and 20% have measurements of CaCO 3 and extractable iron contents (Table  4 ). Around 25% of the measurements have records of the laboratory method used in the analyses. Table  5  lists by continent and overall, the number of data, the soil attributes and their statistics.
The mean of the organic C data in the global database is 2. 16%, the distribution is positively skewed and the median is 1. 00% (Table  5 ). The distributions of the data on CaCO 3 and extractable Fe content is also positively skewed. The median CaCO 3 content is 2. 10% and the median extractable Fe content is 1. 00% (Table  5 ). The average CEC of the samples is 17. 5 cmol c kg -1 , average pH Water is 6. 57 and on average the samples in the database have more sand than clay and silt (Table  5 ). The ranges of the soil attribute distributions are wide and their coefficients of variation large, underlining the varied and diverse origin of the samples (Table  2 ).
Denoising, compression and variable selection with wavelets We used the discrete wavelet transform to denoise and compress the spectroscopic database, and thereby produce a more parsimonious representation of the spectra for the modelling. For this, we followed the approach described by Viscarra  Rossel and Lark (2009). The decomposition was made using a Daubechies's wavelet with two vanishing moments  (Daubechies, 1988). Once the wavelet decomposition was performed, we wanted to retain only those coefficients that produced the most parsimonious representation of the global spectra and would be useful in the spectroscopic modelling. Selection was based on the variance of the coefficients, regardless of the wavelet scale to which they belonged  (Viscarra Rossel and Lark, 2009). The rationale is that coefficients with larger variances, which can occur at different wavelet scales, contain the systematic information in the spectra that is useful for regression, while coefficients with small variances are less likely to be useful and can be discarded.
Spectroscopic modelling for prediction of soil attributes To train and then validate the spectroscopic models so that they could be used with confidence, the dataset for each soil attribute separately were split into a training and an independent validation set (roughly 75:25 split) by simple random sampling. To develop the spectroscopic models on the training set, we used the machine learning algorithm, Cubist. Quinlan (1992)  provides detail on the algorithm, the construction of the trees and the quantification of their errors. Briefly, Cubist partitions the response data into subsets in which their characteristics are similar with respect to their spectra and other predictors that might be used. A series of rules derived using if-then conditions defines the partitions, and these rules are arranged in a hierarchy.
Estimation and uncertainty To quantify the uncertainty in the spectroscopic models that we used to estimate the soil properties, we followed the approach using the nonparametric bootstrap described in Viscarra  Rossel and Hicks (2015). It involves modelling of the soil attributes with Cubist, as above, but using 100 bootstrap samples  (Hastie et al. , 2009; Viscarra Rossel, 2007). We assumed that the spectra in the global database would be independent and could be used with the bootstrap to measure the uncertainty in our analysis. We modelled each bootstrap realisation independently with Cubist and derived cumulative distribution functions for the predictions on the validation dataset, which we used to compute the mean estimates, their standard errors and 95% confidence limits to describe their uncertainty.
Harmonising the global soil database We remodelled each soil attribute using 100 bootstraps and Cubist as described above, but this time using all of the available spectra with matching analytical data. These models were used to predict onto the entire database (N = 23 361) for each of the nine soil properties. For each set of predictions, as above, we calculated the average estimates from the bootstraps, their standard errors and their 95% confidence limits. Thus for each of the nine soil properties in the database, we produced a complete set of soil attribute data that was harmonised by the spectroscopic method. All of the spectroscopic and statistical analyses and modelling described above were performed using the R software (R Development Core Team, 2008).
The exception is the spectra from Argentina and Uruguay where Phaeozems and Vertisols occupy large areas. Their spectra are mostly represented by classes 1, 4, 5 (Table  10 ). Antarctica is associated with classes 4 and 6 (bottom left quadrant of Fig. 7c ), which represent soil containing goethite and 2:1 clay minerals, respectively. The Antarctic soils in the database appear not to be deeply weathered and to contain small amounts of organic matter. They do not have samples in classes 2 and 3 and five, respectively (Table  10 ). The average spectrum from soil of the Ross Dependency in Antarctica (Fig. 4 ) shows a much younger age of the soil with broad absorption characteristics of goethite near 1000 nm and those near 1400, 1900, and 2200 nm that might be attributed to micaceous minerals such as illite and swelling smectitic clays (Fig.
North America plots in the bottom right quadrant of Fig. 7c , as does Oceania but more towards the centre of the graph. Its samples are evenly represented by all six classes, pointing to the diversity of the Australian soil in the database. Similar interpretations can be made for the soil from the countries in the database (Fig. 7d ). We note however, that our interpretations are based only on the soil spectra that are in the database and acknowledge that the composition of soil globally is likely to be more varied.
Harmonising the global soil attribute database We used the spectroscopic models, described above, to predict onto the entire dataset. These predictions provide a harmonised set of soil attribute data because they were derived using a single method, vis-NIR spectroscopy, and come with estimates of uncertainty that are described by 95% confidence limits. Descriptive statistics for the harmonised data are given in Table  13. The statistics in Table  13  are summarised by continent as an indicative means to provide continental information and to show that our predictions for each attribute produced sensible values. In doing so, we acknowledge that the sampling over many of the worlds regions is sparse and strongly biased so that statistical comparisons of continental means and variances may not be entirely appropriate.
Relative accuracy of the global predictions The accuracies of the global spectroscopic models predictions were comparable to those reported in the literature for prediction at different scales (Fig. 13 ; Tables 14-20 with the review data in Appendix C). The accuracies of our estimates are similar to those of other studies conducted at continental and global scales (Fig. 13 ). In some cases they are better and in other a little worse, although direct comparisons are difficult because there are no other studies made using a global dataset that is as large or diverse as the one we have here. Our results show that vis-NIR spectroscopy can be used to predict soil attributes using historical soil spectroscopic databases, developed by different people and for different applications. Filtering and standardising the global spectra with wavelets helped account for the inconsistencies in sample preparation, different measurement protocols and instruments used in the many laboratories.
Modelling of the global spectra with a data mining machine learning algorithm helped to find local relationships in the data to produce relatively accurate predictions of the soil attributes studied (Fig. 9 ; Table  12 ; Fig. 13 ). Soil spectroscopy is a highly reproducible analytical method so the inaccuracies in the spectroscopic models are largely due to the inconsistencies of the reference soil analyses. We have not tested the use of the global database for predictions of soil attributes at local scales (e. g. Ramirez-Lopez et al. , 2013; Guerrero et al. , 2016). This was not our objective here, however, we believe that more research is needed to optimally use large spectroscopic databases Fig. 12. Boxplots of the predicted harmonised soil properties by spectral class (see Fig. 5 ). Class 1 represents soil with smectitic mineralogy and with some carbonates, classes 2 and 3 represent weathered soil rich in kaolinite, hematite and sand, classes 4 and 6 represent soil with goethite and illite and soil in class 5 represents soil with organic material.
The global database for soil mapping, modelling and monitoring Both the models and the harmonised data and uncertainties, may be used in different applications and for different purposes. For instance, the harmonised soil attribute data, with estimates of uncertainty, could be used to complement and help to improve regional, continental and global soil resource assessment at those scales. The spectra and spectroscopic models could be used to make predictions of soil attributes, mineralogy and soil type, where these measurements are lacking and would be too expensive to make by conventional laboratory means. The global spectroscopic database should also help to reduce the number of soil samples that need to be measured with the reference analytical method, thereby making the assessments of soil more affordable.
The accuracies of spectroscopic estimates made from using large continental spectroscopic databases, have been shown to be sufficient for large scale mapping of soil mineralogy  (Viscarra Rossel et al. , 2010; Viscarra Rossel, 2011) , organic carbon  (Viscarra Rossel et al. , 2014)  and other soil attributes across the whole of Australia and Africa  (Viscarra Rossel et al. , 2015; Vågen et al. , 2016). The largest uncertainty in the use of soil information is in predicting soil behaviour from measured soil attributes. Future efforts might therefore be best focused on relating soil behaviour and management responses directly to soil spectra, which as we have shown here, provide a reliable and composite measure of mineral and organic composition.
Linking the global database with proximal and remote sensing Spectroscopy, in the laboratory and in the field by proximal and remote sensing, has become an indispensable tool for soil, earth and environmental scientists who need soil information. There is significant advantage to be gained by combining the use of laboratory spectra with proximal and remote sensing. Spectra measured in the laboratory provide a useful basis for proximal and remote sensing measurements. The advantages of proximal sensing are that one can measure the soil with minimal preparation, no interferences (e. g. from atmosphere, clouds or vegetation) and to depth. Remote sensing enables measurements over larger areas (and scales) and at potentially finer temporal resolutions. There are examples of laboratory, proximal, and remote vis-NIR spectroscopic sensing research, but few that combine their use  (Ben-Dor and Banin, 1995; Ben-Dor et al.
, 2002; Gomez et al. , 2008; Stevens et al. , 2008). The reason might be that there are significant challenges posed by the inherent differences between the standardised laboratory measurements and those made under natural conditions, in the field. Approaches are being developed to account for the effects of water and other environmental factors on proximally sensed spectra so that they may be used together with spectroscopic databases measured in the laboratory (e. g. Minasny et al. , 2011; Ji et al. , 2015). Accounting for these effects on remote sensing spectra are fundamentally more difficult because of the interferences mentioned above. But also because of biogenic and physical crusting, dust  (De Jong, 1992; Ben-Dor et al. , 2003) , and the natural roughness of the soil surface, which creates anisotropic patterns that cast shadows and affect the measurements (e.
g. Pinty et al. , 1989; Chappell et al. , 2006; Croft et al. , 2009). The global database might help to develop methodologies that bridge the gap between laboratory, proximal and remote sensing. The global spectra might form the basis for new developments in hyperspectral remote sensing of soil, or at least, they may enable appropriate validation of the reflectance information extracted from the remote sensing products of current and future airborne and satellite hyperspectral spectrometers. The spectra might also be used for downscaling of the coarse resolution remote sensing images to help with regional assessments of soil condition. As we have shown here, vis-NIR spectra measure the inherent composition of the soil, so using the global spectra as baselines and proximal and remote sensors for monitoring changes in the spectra at those locations, might form a strong base for the development of an effective global soil monitoring network.
This allowed us to analyze a fairly inconsistent dataset, with spectra and soil attributes that were measured in different laboratories using different spectrometers and methods, and derive spectroscopic models that were parsimonious and robust. We found that globally, modeling a diverse set of spectra with a machine learning algorithm can, for most attributes, find the local relationships in the data to produce accurate predictions. Modeling regionally did not always help, except for some attributes (e. g. soil organic C) where grouping the spectra into more homogeneous spectral classes (irrespective of geographical position) improved the modeling by removing bias in the predictions. Our results show that the global spectral library can be used to accurately estimate soil organic and inorganic C and extractable Fe and fairly accurately estimate CEC, clay and silt contents and pH.
Using these spectroscopic models, we derived a harmonized global soil attribute dataset, which might facilitate research on soil and biogeochemical cycles at regional and global scales. Soil vis-NIR spectroscopy is a versatile tool that can provide harmonized data with sufficient accuracy for different applications. It can help to overcome the worldwide shortage of soil data and it could also help to assess and monitor global changes in soil condition. We encourage other contributions from people developing spectral libraries in regions where we have few or no spectra (Fig. 3 ). This work and our collaboration might then progress to develop a dynamic and easily updatable database with better global coverage. We hope that this work will reinvigorate our community's discussion of scientific practice towards larger, more coordinated collaborations.
Acknowledgements We thank the following people for helping to organise and contribute soil samples and/or spectra to the global library:  H. Abbaslou  for spectra from Iran,  V. Barrón  and colleagues for the soil samples from  Andalucia, Spain ,  D. Brunet  for the spectra from  Senegal Madagascar  and  Martinique ,  C. Castilla  for the spectra from  Colombia ,  J. Eriksson  for the spectra from  Sweden ,  S. T. Drummond  and  N. R. Kitchen  for the spectra from the  USA ,  T. Kemper  and  S. Sommer  for the spectra from the Natura 2000 sites in Italy,  B. Kusumo  for the spectra from  New Zealand ,  A. Ringrose-Voase  for the spectra from the  Philipinnes ,  S. Shibusawa  and  M. Kodaira  for the spectra from  Japan ,  A. Sila  for the spectra from  Kenya , and  A. Thomsen  for the spectra from  Denmark.
The blue line in Figure  8  is the R obtained for t 0 = t V LDmin , the first minimum of the VLD2 signal following the preceding ELM. We can then see that R > 0. 3 for δt < 5ms before the ELM occurs, and systematically increases as we approach the ELM occurrence time. Within this time interval the standardized angular variance σ φ is small, it gradually increases with δt as the phases become more disordered. The p-statistic remains small for times δt < 15ms indicating that the distribution of phases remains far from circular. However this is not a smooth trend, there are short intervals (for example around δt ∼ 6ms where p ∼ 0. 05. We have found that for δt > 5ms the details of where short-lived fluctuations in R, σ φ and p-value occur are not robust, they vary with the dataset and with the detailed parameters of how the Hilbert transform is computed.
Appendix: Surrogate time series and null hypotheses The full flux loop signals can be seen in the time-series to have intervals where there is a clear sinusoidal component, with a characteristic period of ∼ 10ms and the ELM waiting times have time structure; they are not random. The analysis is performed on a restricted sized sample. We now test a series of null hypotheses that capture scenarios where the phase alignment that we report above could occur by coincidence. We will use the same circular statistics as above to distinguish the likelihood of coincidental occurrence in a quantitative manner. We use Hilbert transform window (a) and the same dataset as in the main paper, plasma 83771, to construct the surrogates. We have repeated this analysis for all the other JET plasmas in this sequence, and we obtain the same results.
Acknowledgments We acknowledge the  EPSRC  for financial support. This work has been carried out within the framework of the  EUROfusion Consortium  and has received funding from the  European Unions   Horizon 2020 research and innovation programme  under grant agreement number  633053 , and from the  RCUK Energy Programme  [grant number  EP/I501045 ]. The views and opinions expressed herein do not necessarily reflect those of the  European Commission. We acknowledge  J. Davidsen  and participants in the 2014 MPIPKS workshop on Causality,  Information Transfer  and  Dynamical Networks  for discussions, and  A. J. Webster  for.
Background: Making forecasts about biodiversity and giving support to policy relies increasingly on large collections of data held electronically, and on substantial computational capability and capacity to analyse, model, simulate and predict using such data. However, the physically distributed nature of data resources and of expertise in advanced analytical tools creates many challenges for the modern scientist. Across the wider biological sciences, presenting such capabilities on the Internet (as "Web services") and using scientific workflow systems to compose them for particular tasks is a practical way to carry out robust "in silico" science. However, use of this approach in biodiversity science and ecology has thus far been quite limited. Results: BioVeL is a virtual laboratory for data analysis and modelling in biodiversity science and ecology, freely accessible via the Internet.
BioVeL includes functions for accessing and analysing data through curated Web services; for performing complex in silico analysis through exposure of R programs, workflows, and batch processing functions; for on-line collaboration through sharing of workflows and workflow runs; for experiment documentation through reproducibility and repeatability; and for computational support via seamless connections to supporting computing infrastructures. We developed and improved more than 60 Web services with significant potential in many different kinds of data analysis and modelling tasks. We composed reusable workflows using these Web services, also incorporating R programs. Deploying these tools into an easy-to-use and accessible 'virtual laboratory' , free via the Internet, we applied the workflows in several diverse case studies.