dataesr/scientific-paragraphs-categorization huggingface dataset:
To determine the significance of the linear fit, we also mel = melilite, sp = spinel, px = pyroxene (ranging from Al-Ti-rich pyroxene to diopside), an = anorthite, grs = grossite, hib = hibonite, pv = perovskite, bdl = below detection limit. K 2 O was measured but was below the detection limit in all analyses. Mineralogy information for CAIs Mesquite, TS23 and TS68 can be found in  Hertwig et al. , 2021; Simon et al. , 1994 Simon et al. , , 1999. Ak = 100*Mg/(Mg + Al/2) (mol. %). * We re-classified or sub-classified these chondrites, see Supplementary Materials. calculate the uncertainty of the MSWD. For a linear regression with two free parameters (slope and intercept), the value of the MSWD, if the scatter of points is due to measurement errors with the reported uncertainties, is expected to lie within the range 1 ± 2r MSWD , where: r MSWD ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 ðN À 2Þ s If the fit yields an MSWD outside this 2-sigma limit (either too high or too low), there is a less than $5% probability that the data conform to a simple linear model and yet exhibit an MSWD so far from 1; scientific inferences based on that model are statistically inappropriate  (Wendt and Carl, 1991).
1, 4,  and 5 ). The isochron regressed from the FUN CAI CMS-1 data is the only one that yields an initial 10 Be/ 9 Be that does not agree with this range; we consider it separately below and focus here on the other Fig. 2. 26  Al-26 Mg isochrons for 5 CAIs, with insets in two cases to show the data more clearly for phases with low Al/Mg ratios. All uncertainties are 2r. 24 CAI isochrons. As can be seen in Fig. 4 , there is not a significant or systematic difference between the 10 Be/ 9 Be ratios inferred for CV3 ox , CV3 red , CR2, CO3 or CH/CB CAIs, or between different petrologic types of CAIs (i. e. , condensates versus igneous, or Type A versus Type B). Because we are testing whether 10 Be was distributed homogeneously or heterogeneously among CAIs, with the goal of determining the origin of 10 Be, we assess whether our CAI data are primarily consistent with a single initial 10 Be/ 9 Be ratio.
One approach is to evaluate statistically if the data are part of a single population. We find that the weighted average 10 Be/ 9 Be value from our 24 CAI isochron regressions is (7. 5 ± 0. 3) Â 10 -4 with an MSWD = 0. 8. Considering that n = 24, the maximum MSWD for these data to define a single population is 1. 5, which is higher than the MSWD of the data group. This indicates that all CAIs analyzed here to obtain these 24 CAI isochron regressions are indeed part of the same population characterized by a homogeneous initial 10 Be/ 9 Be value. Another approach to quantifying the 10 Be/ 9 Be homogeneity, or lack thereof, in the CAI dataset is to determine its probability density distribution. If only one probability density peak is apparent, then the data conform to a single value. If we consider our full dataset of 10 Be/ 9 Be values based on 25 CAI isochron regressions, the probability density distribution, calculated using MATLAB, defines a sin-gle peak at 10 Be/ 9 Be = 7.
4 Â 10 -4 (FWHM = 4. 1) (Fig. 5 ). Including (n = 25) or not including (n = 24) the data for the FUN CAI CMS-1 in the probability density function does not affect the outcome. There are no other significant peaks on this curve. There is a slight deviation around 10 Be/ 9 Be $10 Â 10 -4 , however this does not alter the interpretation that these data are statistically distributed around a single value.
4. The 10 Be/ 9 Be ratios inferred from 54 CAI isochron regressions (this study;  McKeegan et al. , 2000; Sugiura et al. , 2001; Marhas et al. , 2002; MacPherson et al. , 2003; Chaussidon et al. , 2006; ; Liu et al. , 2009 Liu et al. , , 2010;; Wielandt et al. , 2012; Gounelle et al. , 2013; Srinivasan and Chaussidon, 2013; Sossi et al. , 2017; Fukuda et al. , 2019; Mishra and Marhas, 2019; Fukuda et al. , 2021a). Data for FUN and UN CAIs and PLACs are shown on the left-most side of the plot; other data are grouped according to the different carbonaceous chondrite classes that the CAIs come from. The vertical light gray boxes highlight these different groups of CAIs: F(UN), CH/CB, CV3 ox , CV3 red , CR2, and CO3. The upward pointing arrow indicates CAI SA301, which has a high 10 Be/ 9 Be of (6.
1 ± 2. 8) Â 10 À3  (Fukuda et al. , 2019) , well above the maximum value on this plot. The horizontal dark gray bars indicate the CAI weighted mean 10 Be/ 9 Be value for the normal group ( 10 Be/ 9 Be = (7. 1 ± 0. 2) Â 10 À4 ) and the FUN and PLAC group range ( 10 Be/ 9 Be $ (3-5) Â 10 À4 ). Some CAI data points are labeled. Uncertainties are 2r. The data from our study are distributed about a single peak with 10 Be/ 9 Be = 7. 4 Â 10 -4 (black curve). The combined data from this study and the literature are predominantly distributed about a single peak at 10 Be/ 9 Be = 7. 1 Â 10 -4 , however there are smaller peaks at 3. 1, 4. 0, and 5. 1 Â 10 -4 (gray curve). The combined data from this study and the literature, but not including FUN CAIs, are also distributed about a single peak at 10 Be/ 9 Be = 7.
As discussed in section 4. 1, to determine if 10 Be is distributed homogeneously in CAIs among the integrated dataset, we plotted the probability density distribution of the combined data from this study and from the literature (using only 10 Be/ 9 Be values based on regressions with acceptable MSWD values). Including the 25 isochron regressions from this study, a total of 54 CAI regressions are represented in Fig. 5. The data are predominantly distributed about a single peak at 10 Be/ 9 Be = 7. 1 Â 10 -4 (FWHM = 3. 1), however, there are smaller peaks at 3. 1, 4. 0, and 5. 1 Â 10 -4 (gray curve in Fig. 5 ). The three smaller peaks represent individual FUN CAIs and are distinguishable because data for these CAIs have unusually small uncertainties. We note that we only analyzed one FUN CAI while the literature data includes 10 Be/ 9 Be values from seven FUN CAI and PLAC isochron regressions.
, 2020). Of the two other anomalous CAIs, CAI Y20-1X1 has canonical 26 Al/ 27 Al and a fractionated group II pattern, indicating that it condensed from a gas formed by evaporation  (Hu et al. , 2021). CAI SA301 is 26 Al-poor and from the CH parent body. Additionally, these two CAIs include spots with some of the lowest boron concentrations in the datasets, down to [B] $5 ng/g  (Fukuda et al. , 2019 (Fukuda et al. , , 2021b)). It will be necessary to find and study additional CAIs with high 10 Be/ 9 Be values to further constrain the origin of this poorly defined set of CAIs. There are two CAIs, 31-2 and CH-C4, that fit into the main group of CAIs with 10 Be/ 9 Be $ 7. 1 Â 10 -4 , but their other isotopic signatures are distinct. The UN CAI 31-2, which has 10 Be/ 9 Be = (14. 6 ± 6. 6) Â 10 -4 , is 26 Al-poor, and 16 O-rich  (Simon et al.
It is possible that the CH/CB parent body, which underwent a high-energy collision event, could have introduced conditions to alter initial CAI 10 Be/ 9 Be ratios; an impact plume from the irradiated surface of a CH/CB parent body precursor could have mixed with the original CAI and chondrule material. CH CAI SA301 has low 26 Al and could have been altered by the impact plume; however, oxygen isotopes and other chemical and isotopic measurements are necessary to confirm this. The FUN CAIs, PLACs, and CO3 CAI Y20-1X1 from the combined dataset show group II, III, and distillation REE patterns. We did not measure a CAI with a group II pattern, so cannot confirm a possible correlation between initial 10 Be/ 9 Be ratios and group II REE patterns  (Fukuda et al. , 2021a). These are speculative ideas that need to be tested, but the larger point is that unusual CAIs with isotopic anomalies, low 26 Al, or evidence of additional processing (such as formation on the CH/CB parent body and/or the presence of group II REE patterns), are the only ones that show 10 Be/ 9 Be values distinct from those of normal CAIs that have 10 Be/ 9 Be $ 7.
ACKNOWLEDGEMENTS We thank associate editor  Dr. Thorsten Kleine  for editorial handling and  Dr. Andrew Davis  and two anonymous reviewers for careful reviews that greatly improved the manuscript. We are grateful to  Dr. Lynda Williams  for her assistance in the  ASU SIMS lab. We thank  Dr. Ryan Ogliore  and  Dr. Chris Coath  for useful discussions about statistics and ratio determination. We acknowledge the use of facilities within the  Eyring Materials Center at ASU , supported in part by  NNCI-ECCS-1542160 , with the assistance of  Dr. Axel Wittmann. We acknowledge  ASU's Buseck Center for Meteorite Studies collection  for providing samples of  NWA 5028 ,  NWA 6991  ( CAI B4 ),  Leoville, Allende  ( CAI CMS-1 ),  NWA 7891 , and  NWA 7892 ; the  Field Museum of Natural History for Allende CAIs TS23A  and  TS68 ;  Dr.
Introduction Although they have become the focus of intense research activity in recent times, with more papers published in the 5 years prior to 2018 than all years before that combined, machine learning (ML) techniques applied to geoscience problems date back to the middle of the last century (see  Van der Baan and Jutten, 2000, and Dramsch, 2020, for reviews). ML applications relate to a whole range of geological and geophysical problems, but many of these studies face common challenges due to the nature of geoscientific data sets. Karpatne et al. (2017)  summarise the principal challenges as follows: i. Objects with amorphous boundaries. The form, structure and patterns of geoscience objects are much more complex than those found in the discrete spaces that ML algorithms typically deal with, consisting of changes in Published by Copernicus Publications.
In this study we specifically focus on six of these challenges by providing a database of 1 million 3D geological models and resulting gravity and magnetic fields. We address the spatio-temporal structure of the system by using a kinematic modelling engine that converts a sequence of deformation events into a 3D geological model. We address high dimensionality by generating a very large database of possible 3D geological models. This represents a fundamental point of difference from many ML targets such as those studying consumer preference, movie ratings or facial recognition. Although of course every human face is different, with few exceptions we share the same number of features (eyes, ears, noses), and these features' sizes and relative positions vary only within small bounds. The number, geometry, composition and relative position of features in the subsurface have very wide bounds, and this represents a major hurdle to the application of ML to characterising 3D geology.
Modern ML training sets for popular subjects such as the human face may contain tens of millions of examples  (Kollias and Zafeiriou, 2019). A search of the Kaggle database of training data sets (https://kaggle. com, last access: 27 January 2022, which contains over 63 000 distinct data sets at the time of writing) had only 151 data sets with geoscience in the keywords, and only seismic catalogues featured as geophysical data. Similarly, only 59 data sets contained 3D data, and none were related to the geosciences. Finally, we address the spatial and temporal paucity of ground truth by publishing over 1 million models for which the full 3D lithological and petrophysical distribution is provided in a labelled form for comparison with resulting gravity and magnetic fields. This challenge is also faced by geophysical inversion methods.
Work on the automating of modelling workflows may allow us to explore the model uncertainty space more efficiently  (Jessell et al. , 2021). In this study, we have created a massive open-access resource consisting of 1 million three-dimensional geological models using the Noddy modelling package  (Jessell, 1981; Jessell and Valenta, 1996). These are provided as the input file that define the kinematics, together with the resulting voxel model and gravity and magnetic forward-modelled response. The models are classified by the sequence of their deformation histories, thus addressing a temporal paucity of ground truth. This resource is provided to anyone who would like to train a ML algorithm to understand 3D geology and the resulting potential-field response or to anyone wishing to test the robustness of their geophysical inversion techniques.
Guo et al. (2022)  used the same modelling engine to produce more than 3 million models of a more restricted range of parameters to train a ML convolutional neural network system to estimate 3D geometries from magnetic images. In this study we aim to provide a much broader range of possible geological scenarios as the starting point for a more general exploration of the geological model space. The Noddy software has been used in the past for a range of studies due to its ease in producing "reasonable-looking" geological models with a low design or computational cost. A precursor to this study used 100 or so manually specified models as a way of training geologists in the interpretation of regional geophysical data sets by providing a range of 3D geological models and their geophysical responses  (Jessell, 2002).
Similarly,  Clark et al. (2004)  developed a suite of ore deposit models and their potential-field responses. The automation of model generation using Noddy was first explored using a genetic algorithm approach to modifying parameters as a way of inverting for potential-field geophysical data, specifically gravity and magnetics  (Farrell et al. , 1996). Wellmann et al. (2016)  developed a modern Python interface to Noddy to allow stochastic variations of the input parameters to be analysed in a Bayesian framework. Finally  Thiele et al. (2016a, b)  used this ability to investigate the sensitivity of variations in spatial and temporal relationships as a function of variations in input parameters. In this study we draw upon the ease of generating stochastic model suites to build a publicly accessible database of 1 million 3D geological models and their gravity and magnetic responses.
Model construction The Noddy package  (Jessell, 1981; Jessell and Valenta, 1996)  provides a simple framework for building generic 3D geological models and calculating the resulting gravity and magnetic responses for a given set of petrophysical properties. The 3D model is defined by superimposing user-defined kinematic events that represent idealised geological events, namely base stratigraphy (STRAT), folds (FOLD), faults (FAULT), unconformities (UNC), dykes (DYKE), plugs (PLUG), shear zones (SHEAR-ZONE) and tilts (TILT), which can be superimposed in any order, except for STRAT, which can only occur once and has to be the first event. The 3D geological models are calculated by taking the current x, y, z position of a point and unravelling the kinematics (using idealised displacement equations) until we get back to the time when the infinitesimal volume of rock was formed, whether defined by the initial stratigraphy, the time of formation of a stratigraphy above an unconformity or an intrusive event.
In this study, we use only the resulting voxel representation of the 3D geological models; however it is possible to produce iso-surface representations of the predeformation location of points in an implicit scheme. We have used this tool as it is rapid, taking under 15 s to generate 200 × 200 × 200 voxel models with both geological and geophysical representations combined using an Intel ® Xeon ® Gold 6254 CPU at 3. 10 GHz, and produces "geologically plausible" models that may occur in nature. Given that the final 3D model depends on the user's choice of geological history, Noddy can be thought of as a kinematic, semantic, implicit modelling scheme. As opposed to  Wellmann et al. (2016) ,  Thiele et al. (2016)  and  Guo et al. (2022) , who used a Python wrapper to generate stochastic model suites, in this study we have modified the C code itself to simplify use by third parties, although the philosophy of model generation is an extension of, as well as very similar to, these earlier studies.
In the case of densities this may result in occasional negative values; however since the gravity field is only sensitive to density contrasts, this does not invalidate the calculation. Some rock types have bimodal petrophysical properties to reflect real-world empirical observations, so we draw from a Gaussian mixture in these cases. The petrophysical data are drawn from aggregated statistics (mean and standard deviation of one or two peaks) of the approximately 13 500-sample British Columbia petrophysical database  (Geoscience BC, 2008). The parameters which can be varied for each type of event, together with the range of these parameters, are shown in Table 1. These parameters can be grouped by the shape, position, scale and orientation of the events, and for a fivestage deformation history, the random selection is required of a minimum of 23 parameters for a STRAT-TILT-TILT-TILT-TILT model and up to 69 parameters for a STRAT-TILT-UNC-UNC-UNC model where each stratigraphy has five units.
Applications The same logic of using millions of Noddy models was first applied by generating a massive 3D model training set and used to invert real-world magnetic data  (Guo et al. , 2022). Earth Syst. Sci. Data, 14, 381-392, 2022  https://doi. org/10. 5194/essd-14-381-2022 That study used a model suite consisting of only FOLD, FAULT and TILT events and only one of each to predict 3D geology using a convolutional neural network (CNN). This approach corresponds to a use case where prior geological knowledge of the local geological history has been used to limit the model search space, and formal expert elicitation could provide an important precursor step to support the generation of sensible and tractable problems. In addition to the CNN training demonstrated by  Guo et al. (2022) , we can envisage three broad categories of studies that could build upon the 3D model database we present here.
), and (3) the clustering (k-means, hierarchical methods, DBSCAN/OPTICS) or classification methods (random forests, support vector machines, linear classifiers). A study of geophysical image variability using a simple 2D correlation or maximal information coefficient between pairs of images of different histories would be illuminating. Do we have images which are the same as each other (or at least very similar and within the noise tolerance of the geophysical fields) but belong to very different histories? If these exist, the ambiguity of the histories can be examined, and we then know where we would expect poor performance from ML techniques which rely on easily discriminated images. The systems of equations characterising geophysical inverse problems often have a non-unique solution. In ML research, if we only use magnetic data or gravity data for inversion, we will be troubled by the non-uniqueness of the solution.
Comparison between training schemes of ML systems. We see potential applications of deep learning techniques (e. g. convolutional neural network and generative adversarial networks) where the series of models we propose may also be complemented by other data sets. In this broad topic we would seek to understand which ML techniques are suitable and effective in mapping geophysical data back to the geology or geological parameters. We can see potential for investigating which techniques minimise the number of data necessary to obtain a good constraint; i. e. what are the model structures that most successfully capture geological expert knowledge? This could be framed as an open challenge to allow different groups to use their preferred approach to the inversion problem. 3. Validation of the robustness of geophysical inversion schemes.
Discussion In this study we have produced a ML training data set that attempts to address six recognised limitations of applying ML to geoscientific data sets, namely spatio-temporal structure; high dimensionality; small sample size; paucity of ground truth; multi-resolution data; and noise, incompleteness and uncertainty in data. Contrary to usual practice, the work for the generation of a comprehensive suite of geological models did not depend on the manual labelling of data. We relied solely on geoscientific theory and principles while remaining computationally efficient. While realistic-looking suites of geological models have been generated using generative adversarial networks  (Zhang et al. , 2019) , these generally represent a limited range of geological scenarios and lack extensive training samples.
Small sample size The total number of models sounds impressive; however once we divide that number by the 343 different event sequences, we are left with between 905 and 8245 models per sequence, which, whilst still large, is by no means exhaustive. There is no fundamental problem with building 10 or 100 million models, and if this is found to be necessary to provide useful ML training data sets, we can certainly do so at the expense of an increased computation time: these models were built in around a week on a computer using 20 processor cores. We can also try to apply a metric, such as model topology, to analyse how well sampled the model space is. Thiele et al. (2016b)  analysed the topology of stochastically generated Noddy models and found that after 100 models for small perturbations around a starting model, the number of new topologies dropped off rapidly.
Paucity of ground truth The primary goal of this study was to build a large data set to provide a wide range of possible models for use in training ML systems and to test more traditional geophysical inversion systems. The models here, whilst simpler than the large test models mentioned earlier, represent to our knowledge the largest suite of 3D geological models with resulting potential-field data and tectonic history, which have their own utility. This usage applies equally well to classical geophysical inversion codes, which have traditionally been tested on only a handful of synthetic models prior to being applied to real-world data, for which there is no ground truth available.
-Include magnetic remanence and anisotropy effects. At present we only model scalar magnetic susceptibility, but the Noddy modelling engine can calculate variable remanence and anisotropic magnetic susceptibility as well. -Allow linked deformation events. At the moment every event is independently defined; however we could allow parallel fault sets or dyke swarms, situations which commonly occur in nature. -Predict different types of geophysical fields. For example, the SimPEG package  (Cockett et al. , 2015)  could easily be linked to this system to predict electrical fields  (Cockett et al. , 2015). -Model larger volumes. Modelling larger volumes would be an improvement as large or deep features cannot currently be modelled due to the 4 km model dimensions. -Build more models. We in no way believe we have explored the range of possible models in the present model suite, and if we include more events or more complex event definitions, we will certainly have to generate many more models, perhaps orders of magnitude more, in order to provide robust training suites and inversion scenarios.
Code and data availability A DOI (https://doi. org/10. 5281/zenodo. 4589883) provides access to a GitHub repository which contains the following elements  (Jessell, 2021) : 1. the source code (C language) for the version of Noddy adapted to producing random models; 2. a readme. md file with a link to the windows version of the Noddy software, plus a link to 343. tar files, 1 for each event history ordering of the model suite; 3. a Jupyter notebook (Python code) for sampling from and unpacking the models; 4. a link in the same readme. md file to the equivalent https://mybinder. org (last access: 27 January 2022) version of the notebook so that no code installation is required to sample from and view the model suite https://mybinder. org/v2/gh/Loop3D/noddyverse/ HEAD?labpath=noddyverse-remote-files-1M.