dataesr/scientific-paragraphs-categorization huggingface dataset:
In this case, it might be beneficial to build a Bloom filter for the k-mers present in the union of the datasets, and maintain multiple Set-Min sketches to represent k-mer counts in each dataset. Note that Set-Min sketches can be also used for long-term storage and transmission of the k-mer composition of a dataset augmented with count information. The k-mers of the dataset can be reassembled into simplitigs  [7]  with a Set-Min sketch storing the (approximated) frequencies. The full count table can be restored from the simplitigs and the sketch.
Conclusions We presented Set-Min sketch -a novel sketching method inspired by the Count-Min sketch. Its primary use is to associate keys to labels without explicitly storing the former. In this paper, we demonstrated the performance of Set-Min sketch for storing k-mer counts information, where the distribution of labels (k-mer counts) follows a power-law. Under this assumption, we proposed simple bounds for a Set-Min sketch that guarantee the total error sum to be within an ε fraction of the total number of k-mers in the dataset. We showed that Set-Min sketch allows us to save space compared to the raw output of the popular KMC k-mer counting tool when applied to labels following a skewed distribution, at the price of a very modest error rate. Memory efficiency is particularly significant in case of whole-genome data and large values of k, where it can reach reductions of two orders of magnitude.
Point cloud segmentation into individual grains is performed using a watershed algorithm applied here to 3D point clouds. Once the grains are individualized into several sub-clouds, each grain-scale morphology is determined by fitting a 3D geometrical model applied to each sub-cloud. If different geometrical models can be conceived and tested, including cuboids and ellipsoids, this study focuses mostly on ellipsoids. A phase of results checking is then performed to remove grains showing a best-fitting model with a low level of confidence. The main benefits of this automatic and non-destructive method are that it provides access to 1) an un-biased estimate of surface grain-size distribution on a large range of scales, from centimeters to meters; 2) a very large number of data, only limited by the number of grains in the point-cloud dataset; 3) the 3D morphology of grains, in turn allowing to develop new metrics characterizing the size and shape of grains; and 4) the in-situ orientation and organization of grains and grain clusters.
Method G3Point is a Matlab program which aims at measuring the size, shape, and orientation of a large number of individual grains as detected from any type of 3D point clouds describing the topography of surfaces covered by sediments. The main functions of G3Points are described in the following and summarized by Figure  1. A 3D point cloud represents a topographic surface defined by a set of points associated to a 3D coordinate system. Compared to 2D digital elevation models where elevation  is https://doi. org/10. 5194/egusphere-2022-75 Preprint. Discussion started: 5 April 2022 c Author(s) 2022. CC BY 4. 0 License. defined as a function of 2 horizontal coordinates (, ), 3D point clouds can include several points located at the same horizontal position (e. g. , the face above and below a grain), allowing a better description of geomorphological features such as grains.
Results: method validation and application to synthetic or natural environments In addition of its robustness and efficiency, an algorithm dedicated to extract granulometric information from point clouds must be able to manage various sources of data, including SFM and LiDAR. In the following, we therefore test the newly developed algorithm against "ground truth" datasets of grain size, obtained in synthetic or natural environments. For each data set, we compare the distribution obtained with G3Point to the grain-size distribution measured by hand. The first experiment consists in 39 black pebbles, bought in a hardware store, laying in a horizontal position over a planar surface of 0. 5 x 0. 5 m (Fig. 4a ). This synthetic experiment was captured by pictures to generate a 3D point cloud by SFM.
The fluvial pebbles along the Hérault River are smaller, with a median b-axis of 75+/-12 mm (Table  1 ). At each site, we took about a hundred of pictures with a Nikon D3500 that covered a few squared meters to build a 3D point cloud by SFM. Data were processed with Agisoft Metashape and the resulting point clouds have a native resolution of ~ 1 point per millimeter. We subsampled the point clouds with CloudCompare to ~ 1 point per 2 to 3 mm to reduce calculation duration. G3Point is then applied to the resulting point clouds with parameters defined after a trial-and-error series of tests so that the segmentation of the grains is visually satisfying (Fig. 5 ). With this approach, a large number of grains is detected (342, 901 and 831 for Chateau Renard Site 1, Site 2 and the Hérault river, respectively, Table  1 ) and each segmented grain is fitted with two different ellipsoidal fits, DLSF and IE.
Acknowledgments. We thank  Thomas Croissant ,  Edwin Baynes ,  Benjamin Bruneau ,  Benjamin Guillaume ,  Lucas Pelascini ,  Romy David  and  Simon Abel  for their help acquiring data. This project has received funding from the  European Research Council (ERC)  under the European Union's  Horizon 2020 research and innovation program  (grant agreement No  803721 ). We also acknowledge support by  Université Rennes 1 and CNRS.
The long-term record of Umkehr measurements from four NOAA Dobson spectrophotometers was reprocessed after updates to the instrument calibration procedures. In addition, a new data quality-control tool was developed for the Dobson automation software (WinDobson). This paper presents a comparison of Dobson Umkehr ozone profiles from NOAA ozone network stations -Boulder, the Haute-Provence Observatory (OHP), the Mauna Loa Observatory (MLO), Lauder -against several satellite records, including Aura Microwave Limb Sounder (MLS; ver. 4. 2), and combined solar backscatter ultraviolet (SBUV) and Ozone Mapping and Profiler Suite (OMPS) records (NASA aggregated and NOAA cohesive datasets). A subset of satellite data is selected to match Dobson Umkehr observations at each station spatially (distance less than 200 km) and temporally (within 24 h).
Continuous Umkehr datasets provide a reference for testing consistency among shorter satellite and remote sensing methods and are used to validate combined records  (Petropavlovskikh et al. , 2005a; Kramarova et al. , 2013). The Umkehr method is based on measuring the difference in zenith-sky intensities selected from two spectral regions (centered on 311. 5 and 332. 4 nm, so-called C-pair spectral channels) over a range of solar zenith angles (SZAs). The longest records are those from Dobson and Brewer spectrophotometers. The logs of the ratio of the observed radiances (also called N values) increase with increasing SZA, and at about 86 • SZA reverses and starts to decrease up to 90 • SZA, which grants the observation its name since "Umkehr" means reversal or change in German. Using the Umkehr effect for calculating vertical ozone distribution was first described by  Götz et al.
, 2013)  aided in assessment of offsets between individual SBUV(/2) instrumental records due to satellite drifting orbit  (Kramarova et al. , 2013). Because of its long-term measurement record, Umkehr data are regularly used for scientific assessments of ozone depletion  (Harris et al. , 1998). They were first used in the early 1980s  (Reinsel et al. , 1984)  to estimate changes in stratospheric ozone depletion over long-term stations in the US, India, Australia, Canada, and Europe. Some global locations that host a Dobson instrument have been providing routine, morning and afternoon, Umkehr observations to the World Ozone and UV Radiation Center (WOUDC) database, including a number of stations hosting Umkehr time series that start in the late 1950s. This renders the Umkehr ozone profiles the longest ozone profile time series  (Bojkov et al.
The Umkehr data collection is automated by the NOAA WinDobson operational software  (Evans et al. , 2017)  that schedules zenith-sky observations at C-pair spectral channels during the morning and afternoon hours. During the automation, the observational process (i. e. , frequency of observations, signal-to-noise ratio, cloud clearance, etc. ) is changed. The software uses the near-IR cloud detector to screen the Umkehr data for clear-sky conditions, interpolates screened observations to 12 nominal SZAs, adds total column ozone information, processes data, and checks retrieved ozone profiles for quality flags and against station climatological variability (±2 standard deviations). This process results in the improved quality assurance of observations and reduces cloud-induced anomalies in the Umkehr data.
The Umkehr observations are archived at the WOUDC (https: //www. woudc. org, last access: 18 October 2018), operated by the Environment Climate Change Canada, where the centralized data processing is done by Python-based version of the UMK04 processing software (https://github. com/woudc/ woudc-umkehr, last access: 18 October 2018). The content of the files at the NOAA and WOUDC archives is the same as that for the operational Umkehr ozone profile record, but the format differs.
nodc. noaa. gov/ cgi-bin/iso?id=gov. noaa. ncdc:C01562 (last access: 3 October 2021). In this paper, we are using ozonesonde data from Boulder, USA; Hilo, USA; and Lauder, New Zealand. The data for the first two stations are taken from the NOAA GML archive and are homogenized versions  (Sterling et al. , 2018). The Lauder ozonesonde data prior to 2018 were provided by Richard Querel of NIWA, New Zealand, for the use in the LOTUS report  (Petropavlovskikh et al. , 2019). This dataset is not homogenized, and the data are the same as those archived at NDACC (http://www. ndaccdemo. org/, last access: 8 July 2021). We extended Lauder ozonesonde data with the unhomogenized 2018-2020 data downloaded from the NDACC archive (last accessed in April 2021). The OHP ozonesonde data were homogenized in 2020.
SBUV and OMPS ozone profile records NASA and NOAA have produced satellite measurements of ozone profiles through the SBUV and related instruments (Nimbus 4 and 7) providing nearly 40 years of continuous data  (1978-present). The use of the common-design singleinstrument dataset eliminates many homogeneity issues including varying vertical resolution or instrumentation differences. Version 8. 6 SBUV data incorporate additional calibration adjustments beyond the version 8 release  (McPeters et al. , 2013; Bhartia et al. , 2013). Small but evident biases remain  (Kramarova et al. , 2013). The Suomi National Polar-orbiting Partnership (S-NPP) satellite of the Joint Polar Satellite System (JPSS) was launched in October 2011  (Flynn et al. , 2006). It carries the Ozone Mapping and Profiler Suite nadir profiler (fur-ther referred to as OMPS) sensor that collects high spectrally resolved solar backscattered radiance in the sunlit part of the globe  (Seftor et al.
, 2014). OMPS makes measurements from 250 to 310 nm with a 1. 1 nm resolution. It has a 16. 6 • cross-track FOV and 0. 26 • along-track slit width, but several spectrums are combined to cover a footprint of 250 × 250 km. The ozone profile retrieval is very similar to the Rodgers optimal statistical method deployed in the SBUV and Umkehr retrieval techniques. Validation of the NOAA operational OMPS ozone profile products is described in  Flynn et al. (2014). Evaluation of the OMPS NASA V8. 6 algorithm products for trend analyses is described in  McPeters et al. (2019). In this paper, we used two satellite combined records. The first record is the NASA aggregated dataset (further referred to as AGG), which is comprised of SBUV, SBUV/2, and OMPS profiles from all (Nimbus 4 through NOAA 19) overlapping satellites and using the NASA version 8.
6 processing  (McPeters et al. , 2013). The AGG station overpass data are selected from all daily records that are found within the ±2 • latitude, ±20 • longitude box centered on the station location and averaged using 1 / distance weighting to the station location. The dataset for Boulder station is available at https://acd-ext. gsfc. nasa. gov/anonftp/toms/sbuv/ AGGREGATED/sbuv_aggregated_boulder. co_067. txt (last access: 10 March 2021). The AGG overpass records for other Umkehr stations can be found in the same directory. Sometimes, there are two or three satellite overpass data found for a single day. For the purpose of comparisons with Umkehr data, all daily records are averaged. The second record is the NOAA COHesive (COH) dataset that combines records data from the SBUV/2 and OMPS (NOAA processing, further referred to as OMPS_NOAA) instruments on the many satellites using correlation-based adjustments providing an overall bias adjustment plus an ozone-dependent factor  (Petropavlovskikh et al.
Aura MLS profiles The Microwave Limb Sounder (MLS) measured ozone profiles from the Upper Atmosphere Research Satellite (UARS) and Aura satellite platforms  (Waters et al. , 1999). We use Aura MLS version 4. 2 data  (Livesey et al. , 2020)  for comparisons with Umkehr observations during the 2005-2020 period. MLS version 5. 1 was not available at the time of analysis; the ozone product is not expected to differ significantly between the two versions  (Livesey et al. , 2020). Retrieved ozone values are computed for 12 levels per decade change in pressure. That is, 12 levels are selected between 1 and 10 hPa; another 12 levels are selected between 10 and 100 hPa, and also between 100 and 1000 hPa. The spacing is linear in log pressure. The vertical resolution of the MLS averaging kernel (AK) is about 2.
The CTM's tropospheric physical processes include convection, boundary layer turbulent transport, wet scavenging in convective updrafts, wet and dry deposition, lightning NO x production, and anthropogenic, natural, and biogenic emissions. The chemical mechanism uses JPL-2015 rates and currently has 119 species and more than 400 kinetic and photolytic reactions; it is an updated version of the mechanism described in  Duncan et al. (2008). Customized GMI CTM simulation outputs were created for the three NOAA Dobson Umkehr stations for 1979-2017 to assist in the assessment of the instrumental offsets and to develop instrument-specific corrections to homogenize the Umkehr record. GMI CTM data at the NDACC sites (including six NOAA Umkehr sites) are available at https://www. ndacc. org (last access: 8 July 2021).
, 2019) , we decided to use its ozone data as a reference for the Umkehr optimization. In addition, we are using the GMI CTM output for assessment of changes in the optimized Umkehr record and for evaluation of ozone variability represented by two modeling records. The M2GMI ozone profile output is subsampled for the Boulder, OHP, MLO (or Hilo), and Lauder Dobson station geolocations (selected from the grid closest to the station location) and is matched within 30 min to the Umkehr observation (local time for the averaged Sun elevation between 70 and 90 • SZA). The ozone profiles are provided on the constant pressure levels that are converted to Dobson units (DUs) and smoothed with Umkehr AKs to created Umkehrlike layers. This is the version of data that is used as a reference dataset for Umkehr optimization.
FG11 and QBO a priori FG11 (further referred to as fg11ap) is a climatological ozone dataset  (McPeters and Labow, 2012)  that describes typical ozone variability with latitude (5 • zonal averages) and season (12 months). This is based on the Aura MLS and ozonesonde records measured between 2005 and 2010. Note that the ozone profile on any day of the year is the same in each year of the record. Thus, ozone in each Umkehr layer only changes seasonally. The QBO a priori (further referred to QBOap) is an ozone climatology developed for analyses of the SBUV records to improve soft calibrations for the MOD ozone record  (Ziemke et al. , 2021). In addition to the seasonally and latitudinally dependent climatology, the method empirically modifies ozone profiles based on the phase of the QBO cycle.
Combined MLS and ozonesonde record The Aura MLS record (described in Sect. 2. 3. 2 above) is matched with ozonesonde profile by date (±12 h) and location (±5 • in longitude and ±5 • in latitude). The approach to the combining of MLS and ozonesonde record is described in  McPeters and Labow (2012). We use this method to extend MLS station overpass ozone profile below 100 hPa with the ozonesonde profiles. The time series of MLS-ozonesonde combined profiles between 2005 and 2020 is created for the Boulder station. The extended dataset is indicated by SND_MLS in the figures and is used in the homogenization process. 3 Optimization of Umkehr stray light corrections.
Aura MLS bias is also like the COH bias. The M2GMI model comparisons show the small-est bias except for the largest negative bias in layer 2 found in both time periods and an increased positive bias in layer 5 in 2005-2020 period. The GMI CTM biases in layers 6-9 are similar in magnitude to the M2GMI biases, whereas they grow larger in layers 3-5. Moreover, in the 2005-2020 period (right panel), both M2GMI and GMI CTM biases in layers 3-5 increase relative to 1994-2004 (left panel) comparisons, and GMI CTM biases become the largest positive biases among all datasets. Ozonesondes have the lowest bias in layers 3 and 6, high bias in layers 4 and 5, and large negative bias in layer 2. The models have lower bias in layers 6-8 as compared to observations (satellite and ozonesonde) and larger bias in layers 2-4.
The mean offset is calculated by averaging results from all datasets except SAGE II (six datasets before and after 2005), and horizontal bars represent the standard deviation of the mean bias values. Figure  1b  shows comparison of the same datasets, but Umkehr profiles are processed using standardized stray light corrections (SLCs,  Petropavlovskikh et al. , 2011). It is found that SLC reduces bias in layers 7, 8, and 9, increases biases in layers 4 and 3 (GMI bias in layer 3 becomes the largest among all layers), whereas biases in layers 5, 6, and 1 do not change significantly. Figure  1c  and d summarize the uncertainty of the bias calculated for operational and SLC Umkehr profiles, respectively. The solid (dashed) lines show results for the  2005-2018 (1995-2004)  comparison periods. There is no large difference found between standard deviations (SDs) in two time periods, and they are larger than 5 % in layers 2, 3 and 4.
The measurement uncertainty of typical Umkehr observations ranges between 0. 5 N value at 70 • SZA and up to 1. 2 N value at 90 • SZA (i. e. , standard deviations of the error co-variance matrix). The empirical corrections appear to agree within the observation uncertainty. However, the largest negative correction at 86. 5 • SZA varies between -0. 1 and -1. 4 N values depending on the reference dataset. Also all corrections exhibit similar shape with respect to the SZA. Figure  2b  summarizes distribution of Umkehr N-value residuals calculated with M2GMI reference profiles. The width of the distributions increases with SZA. The largest deviations are found when the QBOap dataset is used as a reference (not shown) and the lowest deviations are found when the M2GMI ozone profiles are used as a reference.
Discussion of optimization results In order to select the most effective empirical adjustment for the Boulder Umkehr data processing in 2008-2015, Umkehr ozone profiles retrieved with multiple empirical corrections are compared to the MLS station-overpass ozone profiles (Fig. 2c ). The goal is to have a zero bias through the difference profile comparisons with MLS. Empirical optimizations minimize ozone bias in comparisons to the MLS profiles; however, optimized Umkehr profiles still show ±5 % bias, even when the MLS profiles are used as a reference (see results for the MLS and sonde combined profile, SND_MLS). There are some differences between optimized datasets, but they all agree within the uncertainty of each empirical correction. Results show the wave-like distribution of biases that change from negative bias in the upper stratosphere to positive in the middle, then again to the negative bias in the lower stratosphere and to the positive bias in the troposphere.
Some of these biases are due to the Rodgers optimal estimation technique that relies on the vertical ozone profile smoothing and a priori covariance that assumes cross correlations between adjoining layers  (Rodgers, 1990 (Rodgers, , 2000)). There is also a limitation in Umkehr observations that makes it difficult to clearly separate ozone information between tropospheric and stratospheric layers. Since no Aura MLS data are available prior to 2004, we select the M2GMI dataset to develop optimized corrections for the entire Umkehr record. The M2GMI correction is derived separately for each calibration time period of the Dobson record in Boulder (Table  1 ). We note that the M2GMIbased optimized correction produces a small (±5 %) but significant bias in retrieved Umkehr ozone profiles relative to the MLS profiles averaged over the 2005-2018 period (see Fig.
Changes in mean and seasonal biases After reprocessing of the Umkehr data with optimization corrections, the changes to vertical profiles are verified through comparisons against independent ozone observations that are matched with Umkehr record. For verification of changes in Umkehr data at Boulder, satellite overpass data are used for comparisons (Table  3 ) and coincident ozonesonde profiles. Figure  5  shows Boulder Umkehr data comparisons for the two time periods  : 1994-2004 and 2005-2020. The reference data are the same as those in Fig. 1. In comparison to the results shown in Fig. 1  (Umkehr operational and SLC versions), the optimization process significantly reduces satellite/Umkehr biases in the upper stratosphere (Umkehr layers 7-9). For example, a small bias (< 2 %) is found between Umkehr and AGG, COH, and MLS profiles above 30 hPa.
In the upper troposphere-lower stratosphere (UTLS) (layer 2, 250-125 hPa), positive biases (5 %-10 %) are found in comparisons against  COH (1994 COH ( -2020) )  and  MLS (2005 MLS ( -2020) )  satellite data. However, these biases are within the uncertainty of the Umkehr retrieval in layer 2 (∼ 10 %). Comparisons of optimized Umkehr profiles with the date-/timematched ozonesonde data in Boulder show that prior to 2004, the bias from optimized Umkehr is ∼ 5 % in layers 3-5, but after 2005 the bias is largely reduced. For the validation of Umkehr optimization, it is important to track changes in the reference records (especially models) before and after 1998, and around 2005, as the discontinuity can lead to an introduction of the artificial trend in homogenized Umkehr time series. The step change in GMI CTM ozone record in 2005 was discussed in several recently published papers (i.
The plots show the monthly percent differences averaged from 2000 to 2018. The optimized version of the Umkehr data shows a significant reduction in biases as compared to the operational version in the layers above 10 hPa, while small positive and somewhat seasonally varying bias (up to 5 %) remains between 100 and 10 hPa. The largest negative bias is found near 200 hPa in winter months. It could be related to the 5 %-10 % differences in the a priori data used for satellite and Umkehr retrievals (Fig. 6b ) and limitations in both systems to the sensing of ozone variability in the lowest stratosphere. To summarize, optimization of the Umkehr observational record in Boulder over the 2000-2018 period reduced mean biases between Umkehr and COH ozone in the upper and middle stratosphere. Since SBUV/2 and OMPS ozone information in lower layers is strongly influenced by a priori data, and has little independent information there  (Kramarova et al.