dataesr/scientific-paragraphs-categorization huggingface dataset:
, 2019 (Sathyendranath et al. , , 2021)). We used the Version 4. 0 (1997-2019) and Version 5. 0 (2019-2020) datasets produced as part of the ESA Climate Change Initiative Ocean Colour project. Comparisons against in situ measurements show a global mean bias in log 10 (Chl a) of -0. 04 and RMSD of 0. 34  (Sathyendranath et al. , 2019). Data from the NOAA Level 4 Analysis Climate Data Record (Table  1 , dataset no. 23), produced by merging AVHRR satellite data with measurements from ships, buoys and Argo floats using an optimal interpolation scheme, provide gap-free global daily SST on a global 0. 25 • grid  (Huang et al. , 2021; Banzon et al. , 2016; Reynolds et al. , 2007). We used the Version 2. 0 dataset from September 1981 to December 2019 and Version 2. 1 for 2020. As the analysis uses both night and day observations, it cannot be considered foundation sea surface temperature and includes some diurnal warming effects.
Data from the Coriolis Ocean Dataset for Reanalysis (CORA) dataset (Table  1 , dataset no. 24), produced from the merging of many different sources collected by Coriolis data centre in collaboration with the In Situ Thematic Centre of the Copernicus Marine Service (CMEMS INSTAC), acquired both by autonomous platforms (Argo profilers, fixed moorings, gliders, drifters, sea mammals), research or opportunity vessels (CTDs, XBTs, ferrybox), provide monthly temperature and salinity on a global 0. 5 • grid  (Szekely et al. , 2019). CORA is a 4-dimensional dataset, and we only used the temperature and salinity from the first level (1 m depth). We used the Version 5. 2 dataset from 1990 to 2019. Data from the In Situ Analysis System (ISAS) dataset (Table 1, dataset no. 25), produced from the merging of the Argo network of profiling floats and other in situ sources using an optimal interpolation scheme, provide monthly temperature and salinity on a global 0.
5 • grid at several standard depth levels  (Kolodziejczyk et al. , 2021; Gaillard et al. , 2016). We used only the salinity from the first level (1 m depth). We used the ISAS15 v7 dataset from 2002 to 2015 and ISAS20_ARGO v7 (Argo floats only) from 2016 to 2020. Data from the OceanSODA-ETHZ dataset (Table  1 , dataset no. 26), produced by ETH Zurich from surface ocean observations (SOCAT, GLODAP), using the newly developed Geospatial Random Cluster Ensemble Regression (GRaCER) method, provide monthly C T , A T , pCO 2w and pH on a global 1 • grid  (Gregor and Gruber, 2021). We used the C T , A T , pCO 2w , pH, temperature and salinity variables provided in the v2020b dataset from 1990 to 2018. For the open ocean, the estimated pCO 2w and A T show global nearzero biases and root mean squared errors of 12 µatm and 13 µmol kg -1 , respectively.
Taking into account also the measurement and representation errors, the total uncertainty increases to 14 µatm and 21 µmol kg -1 , respectively. Comparisons against direct observations from GLODAP show surface ocean pH and C T global biases of near zero and root mean squared errors of 0. 023 and 16 µmol kg -1 , respectively  (Gregor and Gruber, 2021). Data from the Copernicus Marine Service (CMEMS) Global Ocean Surface Carbon dataset (Table  1 , dataset no. 27) is a reconstruction of monthly surface ocean pCO 2w , air-sea fluxes of CO 2 and pH with associated uncertainties on a global 1 • grid. The product is obtained from an ensemble-based feedforward neural network, mapping SOCAT in situ surface ocean fugacity, salinity, temperature, sea surface height, Chl a, mixed layer depth and atmospheric CO 2 mole fraction.
Preprocessing Before grouping in situ data into ROIs, the different in situ datasets must first be merged into a single collated dataset and sorted into date order. The largest in situ dataset (SO-CAT) is first divided into yearly subsets. If the number of stations (unique sampling locations and times) in a year exceeds a threshold of 10 5 , it is subdivided into monthly subsets, and if a month exceeds 10 5 stations, it is further subdivided into daily subsets. Each subset is then sorted, first by date and time, then by latitude and finally by longitude. Each station is labelled with its data source and version (in this case SOCATv2020); its estimated uncertainty; and its QC flag, if available. For most datasets the latter is a World Ocean Circulation Experiment (WOCE) flag, but in the case of SOCAT this is always 2, so the QC flag is categorised from classification A (the best) to D (the worst included in the final SO-CAT product); see  Lauvset et al.
(2018)  for details. The next dataset (LDEO) is then similarly subdivided, sorted and then merged into the first (after continuing to subdivide until both datasets have the same temporal resolution) to form a single dataset, continuing to subdivide where a yearly or monthly subset expands beyond 10 5 data points. SOCAT and LDEO have many measurements in common, and in case of matching stations (defined by a separation < 1 km and < 30 s), the LDEO station is discarded. This completes the main global pCO 2w datasets. The next dataset to be merged is GLODAP, the main global A T , C T and pH dataset, using only stations with sampling depth less than 10 m. Further variables from GLODAP that have been included in the MDB are pH at 25 • C, dissolved oxygen, apparent oxygen utilisation, nitrate, nitrite, silicate, phosphate, total and dissolved organic carbon, nitrogen, and Chl a.
Again, the WOCE flag is always set to 2 in GLODAP, so in this case the QC flag is a classification indicating whether secondary QC has been performed on the data (1. 0) or not (0. 0). All other datasets use the WOCE flag where available, quoted as an integer, so the three types of QC flag can be distinguished. All other in situ datasets are then merged in the same way, discarding pCO 2w measurements if they match spatiotemporally with SOCAT or LDEO measurements already in the dataset and A T , C T or pH measurements if they match with GLODAP measurements. It is possible that higher-quality laboratory pCO 2w measurements not included in SOCAT are excluded from the MDB due to proximity to underway pCO 2w measurements in SOCAT on the same platform, but we have not checked for this. Next, ancillary data are added to each station in the collated dataset.
These are estimated as their mean and standard deviation from 100 runs of the SeaCarb package (version 3. 2. 12,  Gattuso et al. , 2021) , with A T and C T values taken from a Gaussian distribution with a mean and standard deviation equal to the measured value and its estimated uncertainty. Nutrients were left at their SeaCarb defaults. In each run, the following SeaCarb options are also selected randomly from those with ranges of validity of temperature and salinity appropriate to the given data point, hence including the component of uncertainty arising from these choices: -k1k2 is selected from "m10", "m06", "l" and "r". kf is selected from "dg" and "pf". ks is selected from "d" and "k". b is selected from "l10" and "u74". It is of course possible to calculate the carbonate system from other pairs of key variables (a total of six possible combinations, some of which calculate pH and/or saturation states better than others; e.
Creating the radial in situ data The next step is to group stations into cylindrical spatiotemporal ROIs, each of which consists of a circle on the Earth's surface (all points with a great circle distance from the centre less than a given radius, assuming a spherical Earth) between two dates and times. For the global MDB, the ROIs are restricted to a maximum radius of 50 km (diameter 100 km) and a maximum temporal extent of 10 d. These limits are adjustable; for example, smaller values might be more appropriate in a regional dataset with high spatiotemporal variability. A ROI is the smallest spatiotemporal cylinder that can contain all of its associated in situ stations. The procedure is as follows: 1. Define the first ROI centred on and containing the first station. A ROI containing a single station is infinitesimally small.
The mean of pH variables is calculated geometrically, i. e. from the mean of [H + ], but it should be noted that the standard deviation is that of pH not of [H + ]. In addition to pCO 2w treated normally, a further dataset is processed consisting of pCO 2w corrected at each measurement to the mean SST of the ROI using pCO 2w at SST ROI = (pCO 2w at SST) exp[0. 0433(SST ROI -SST) -4. 35 × 10 -5 (SST 2 ROI -SST 2 )] (  Takahashi et al. , 2009). All data sources of each measurement type in the ROI are listed along with the number of measurements contributed by each source; for example, a ROI might contain 10 pCO 2w measurements from SOCAT and 1 from LDEO. ROIs are stored in NetCDF files using the trajectory format for ungridded data. Sequential processing of all in situ data is a major task that would take several weeks to complete on a normal personal computer, a situation that is likely to worsen as data volume continues to increase.
Creating the OceanSODA-MDB matchup database Felyx is a tool created to extract data from along-track, swath or gridded datasets such as Earth observation (EO) data over defined ROIs (https://felyx. gitlab-pages. ifremer. fr/ felyx_docs/, last access: 20 February 2023). Felyx is a free software solution, written in Python, the aim of which is to provide Earth observation data producers and users with an open-source, flexible and reusable tool to allow scientific analysis and performance monitoring of scientific data through subsetting over specific areas or matching up with in situ measurements. The development of felyx is supported by Copernicus, the European Union's Earth observation programme. Felyx is used in this context to extract EO, model, climatology and reanalysis data within maximum-sized ROIs centred on the in situ ROIs.
Given the time and location of a ROI, felyx is able, for each EO data source, to extract observation subsets within a 50 km radius and ±5 d from the ROI's centre time and location (Fig. 18 ). Hence matchup data are all extracted over the same size regions centred on matching in situ data. This methodology ensures that all data being compared (e. g. satellite and in situ observations) are treated as consistently and equally as possible, allowing all uncertainties in all observations to be included within the analysis. The resultant radial matchup data (the output from the felyx system) are stored in the same NetCDF files used for the in situ data. For each averaged parameter, the mean, median, standard deviation, minimum, maximum, interquartile range and sample count of the observations found within the ROI's search area and time frame are calculated and provided.
Data overview The collated input dataset contains 34 912 843 individual stations, of which 34 839 413 (99. 8 %) contain pCO 2w , 24 474 (0. 07 %) contain A T , 27,032 (0. 08 %) contain C T and 21 924 (0. 06 %) contain pH (note that stations may contain more than one carbonate system parameter). Based on the ROI definition of 100 km radius and 10 d duration, this collated dataset resulted in 285 822 ROIs, of which 272 753 (95. 4 %) contain pCO 2w , 13 595 (4. 8 %) contain A T , 15 041 (5. 3 %) contain C T and 19 613 (6. 9 %) contain pH. Dates range from https://doi. org/10. 5194/essd-15-921-2023 Earth Syst. Sci. Data, 15, 921-947, 2023     The recent reduction in A T and C T measurements may be associated with the situation that, unlike pCO 2w and pH, all A T and C T measurements in the dataset are performed in the laboratory, resulting in a delay in data submission.
Figures  3 to 6  show the mean, standard deviation (where a ROI contains more than one measurement) and number of measurements in each ROI over the whole dataset. Note that the standard deviation is only over a 10 d period and so does not show variability on longer timescales, such as seasonality or interannual variability. Note also that these plots include ∼ 270 000 points in the case of pCO 2w , and more recent measurements overlay older ones, so specific features seen in these plots should be checked in more detail. Variability is greatest in coastal regions and in parts of the Arctic, for example, the Beaufort, East Siberian and Laptev seas and between Greenland and Svalbard. Figures 7 to 10 show the mean in each ROI divided into seasons. Strong pCO 2w seasonality is evident in the Northern Hemisphere, with high values in the northern Pacific and Atlantic in January-March and in the subtropics in July-September.
Example application To illustrate the use of the MDB, we present a reparameterisation of the Takahashi et al. (2014) (T14) global algorithm for potential alkalinity (PA), which is equal to A T plus nitrate. A parameterisation of pCO 2w , which has much higher data density, would better illustrate the advantages of the MDB approach, but we could find no simple, global pCO 2w algorithm that could be reparameterised as T14 can. For this we use in situ A T and SSS and WOA monthly climatological nitrate, since this is what would be needed to produce synoptic maps of A T , for example, from satellite or model SSS. The T14 algorithm is clearly not intended for application to extreme coastal waters, of which there are many in the MDB, and algorithm uncertainties are expected to increase if we include waters with coastal or benthic influence, but T14 offer no criteria to distinguish these.
Discussion In order to stay relevant, the MDB must be regularly updated. Updates to the MDB can be made by adding new data to the existing ROIs, only creating new ROIs where necessary. This enables the MDB to be updated much more quickly and easily that recalculating all ROIs from the beginning. As well as inclusion of new data, this process allows updating of data already in the MDB, for example, if an existing dataset is reprocessed. The scale of the ROIs (100 km diameter, 10 d duration) was chosen based upon our knowledge of the datasets used, the likely spatial and temporal scales of changes in these data and conditions, and the desire to ensure we have matchups between the in situ ROIs and the other datasets. These scales are adjustable, and regional MDBs may be created on smaller spatiotemporal scales in regions where this scale is inappropriate.
, 2016)  will be similarly masked by averaging over the top 10 m within the current MDB. The MDB approach might be inappropriate if the subject of study is highly dependent on sub-ROI effects, such as a study of the depth variation of pCO 2w in the surface layer, and in such cases it would be better to use individual measurements. It should also be borne in mind that averaging will not remove biases (e. g. regional or seasonal) from the in situ data, though it will reduce stochastic noise. The reparameterisation described in Sect. 5, though successful, could have been done with the original A T measurements from the original sources, probably with similar results because the data density of A T measurements is typically low. Over 50 % of MDB regions with A T contain only one A T measurement, and only 1 % contain 10 or more.
The real distinction between the MDB approach and use of individual data points comes when comparing data with very high and low densities, which are more likely to be found in parameters that can be measured at high frequency, such as pCO 2w. To illustrate, consider the hypothetical example of a little-studied sea to which we wish to apply a simple model of https://doi. org/10. 5194/essd-15-921-2023 Earth Syst. Sci. Data, 15, 921-947, 2023 constant pCO 2w. We have data from two cruises, one in winter making a transect of 10 discrete samples and the other in summer with an underway system making a transect of 1000 samples. The actual pCO 2w has a mean of 400 µatm but with a seasonal cycle from 375 to 425 µatm not accounted for by our simple model. If the 10 winter measurements average to 375 µatm and the 1000 summer measurements to 425 µatm, simple averaging of all data gives an estimate of just under 425 µatm, while if the 1000 samples are binned into 10 regions of 100 measurements each, the correct average is found.
Although this example is unrealistic, it is representative of the problems that can occur when fitting models to unevenly distributed data. The models are more complex and the deviations less obvious, but if there are systematic effects not captured by the model, and the data density is greater towards one side of the distribution of these effects, then the model becomes biased. The best way to overcome this is to identify the biasing effects and account for them (e. g. a seasonal split in the example), but some will always remain. The MDB approach lessens the effect of unaccounted biases by evening out differences in data density as consistently as possible. validation, evaluation and performance assessment of satellite data, reanalysis or model datasets, as well as model-data intercomparisons, by simply adding these data into the existing MDB.
Conclusions Here we present a global dataset created using a novel methodology for combining different dataset types (e. g. in situ, model, satellite) onto the same spatial and temporal scales using "regions of interest" (ROIs). This method gives advantages over previous methods, which predominantly grid data by latitude and longitude, as it provides a uniform spatiotemporal resolution across the globe and minimises biases created by differences in data density. We have collated a large global dataset comprised primarily of publicly available in situ, satellite and climatological data; processed it using this methodology; and presented summary statistics describing the results. The resulting matchup database (OceanSODA-MDB) is suitable for reparameterising empirical algorithms, as demonstrated here, but would also enable.
(2020) SST 33HQ20170826 12 A T , C T , SSS, In situ data HLY1103 2011 Arctic Mathis et al. (2016a) SST 13 A T , C T , SSS, In situ data 316n20090614 2009 Arctic Cross et al. (2019) SST 14 A T , C T , SSS, In situ data 33HQ20080703 2008 Arctic Mathis et al. (2016c) SST 15 A T , C T , SSS, In situ data 33HQ20080329 2008 Arctic Mathis et al. (2016b) SST 16 A T , C T , SSS, In situ data Kara Sea dataset 1993-2004 Arctic Wallhead et al. (2017), Polukhin (2019) SST 17 A T , pH, SSS, In situ data Eurasian Arctic Ocean 2006-2009 Arctic Pipko et al. (2017) SST 18 SST Satellite cci_sst (2. 1) 1981-2020 Global (0. 05 • Good et al. (2019), Merchant et al. (2019) daily) 19 SSS Satellite cci_sss (2. 31) 2010-2019 Global (25 km Boutin et al. (2021, 2020) 7 d) 20 SSS Satellite arctic_sss (3. 1) 2011-2019 Arctic (25 km Martínez et al.
The design of a e-mail: marc. labiche@stfc. ac. uk (corresponding author) these arrays very much relies on a good determination of the interaction position in the crystal and the tracking algorithm efficiency. While the former is difficult to simulate, modern Monte-Carlo radiation transport codes are great tools to simulate the interaction in a material and to optimise the tracking algorithm and the overall array geometry. Therefore, right from the start of both the AGATA and GRETA projects, just over 20 years ago, a simulation code was developed using the CERN-based GEANT4 for radiation transport code  [3, 4] , which had just started to replace the widely used GEANT3 toolkit. The AGATA code was born. Since then, the AGATA code has been developed tremendously, following along the evolution of GEANT4.
The AGATA collaboration continuously maintains the AGATA simulation code with new releases of GEANT4 and provides user support for its installation and execution. The code is now distributed via a GIT repository  [5]. All advanced features presented in this article have been produced with GEANT4 10. 5 and are currently being tested with the latest versions. The structure of the AGATA code still follows the structure of a native GEANT4 application. However, many new classes for new event generators and ancillary detectors have been added to take advantage of new capabilities in GEANT4 or extend the scope of the simulation code. With most event generators, the simulation output has remained in the ASCII format, which leaves the choice of data analysis tool to the users. The code is distributed with a version of the two tracking methods, the back-tracking algorithm (MGT) and the Orsay forward-tracking algorithm (OFT)  [6, 7] , however, the responsibility is left to the users to acquire the latest versions.
Event generators New built-in event generator Back in 2004, for the toolkit release 6. 0, the GEANT4 collaboration produced a new event generator class called G4GeneralParticleSource (GPS), which gives users a realistic method to simulate the radiation decay of nuclides. The new class allows users to specify spectral, spatial and angular distribution of the primary source particles via macros or command lines. The simulation of the decay is based on ascii data file taken from available nuclear database, which can be completed, corrected or even created directly by users. This feature has been added in the AGATA code as a new event generator option, on top of the original built-in generator G4ParticleGun and the AGATA external event generator class which simply reads as input a list of primary particles defined by their nature, position and momentum as input.
Event generation for the AGAPRO analysis package: The introduction of a " wall-clock time" into the AGATA simulation code gives the possibility to generate events in a format that can be read by the AGAPRO package (see Stézowski et al. in this issue). A code has been written that creates AGATA data format (adf) files that emulates experimental data files after the Pulse-shape analysis step. The wall-clock time step in the output of the GEANT4 simulation is converted into a 10 ns timestamp corresponding to the 100 MHz clock of the.
Event builder Tracking Event merger Further analysis MakeADFEvents AGATA geant4 sim. Fig. 3  Tool chain for producing adf event from AGATA GEANT4 simulations. ADF files containing the positions and energies of γ -ray interactions are feed into the event builder of the agapro package and analysed as if from experimental data. Data from ancillary detectors are, as is the case for experimental data, added at the event merger level AGATA DAQ. Each crystal is treated independently, where the energy of the core is given as the sum of all interactions within the assumed shaping time of the amplifier (typically a few microseconds). The same is true for the energies of the segments. The used segmentation is that of the AGATA simulations, which mimics closely but not exactly the real segmentation (the segmentation is defined in an input file).
Then for every interaction point, a pulse shape response is deduced by linear interpolation of the AGATA Data Library (ADL) signal basis in 2 mm grid. The pulse shape responses from one detector are summed up in every event, and a certain noise level is added to generate the simulated pulse shape data. The simulated pulse shape data is then processed through the normal PSA algorithm in the same way as experimental data to obtain the analysed interaction positions. The position resolution of every grid point is extracted by comparing the PSA analysed positions with the GEANT4 simulated ones around this grid point. The position resolution obtained from this method is in general smaller than the 2 mm (σ ) average resolution reported for the experimental data  [14] , which is expected, since there are effects like neutron damage, cross talk and time alignment in experimental signals making the signal shape deviated from the ADL basis, leading to a worse position resolution.