dataesr/scientific-paragraphs-categorization huggingface dataset:
X-and W-band Doppler spectrograms In this section, we present the experimental dataset used for the implementation of the second part of the pipeline. Measurements were collected during the 2021 ICE GENESIS campaign, a joint ground-based and airborne field experiment that was conducted in the Swiss Jura in January 2021, and is fully described in  Billault-Roux et al. 2022 (under review). Data from Xand W-band vertically-pointing Doppler radars are used, which were located at les Eplatures airport. The X-band radar, further on referred to as ROXI  (Viltard et al. , 2019) , is a high-sensitivity 9. 48 GHz Doppler spectral profiler with 1. 8 Then, the spectrograms of both radars are remapped to a common grid, by averaging in time (with a resolution of 20 s), interpolating in range (resolution of 50 m), and average-binning the velocity to the same bins as the synthetic dataset, i.
Aircraft in-situ measurements In addition to the ground-based measurements, the ICE GENESIS campaign included scientific aircraft overpasses with remotesensing and in-situ instruments. The airborne in-situ data is particularly valuable for the quantitative evaluation of the microphysical retrieval, presented in Sect. 5. Airborne measurements used in this work were collected during three flights of the Safire ATR-42 (Jan. 22, Jan. 23, Jan. 27) as the aircraft was performing overpasses over the ground site, with data from several probes. First, the Counterflow Virtual Impactor (CVI,  Anderson et al. , 1994; Schwarzenboeck et al. , 2000)  provides a measurement of total water content (TWC), and the Cloud Droplet Probe (CDP) of liquid water content (LWC); from those measurements, an estimate of IWC can be obtained as IW C = T W C -LW C.
Decoder training The NN is trained on the synthetic dataset described in Sect. 3. 1, which is split into training, validation and testing sets (80% -10% -10%). The input is normalized using the statistics of the training dataset, i. e. with the mean and standard deviation of each variable. For certain variables, the natural logarithm is used instead of the original value, in order to have more homogeneously spread distributions: this is the case for IW C, a m and α a. The network is trained using the Adam optimizer  (Kingma and Ba, 2015) , with mean square error (MSE) as a loss metric, and with Xavier normal initialization of weights and biases  (Glorot and Bengio, 2010) ; in addition, the learning rate is periodically decreased with a scheduler. The network and training hyperparameters are summarized in Table  3.
Encoder training The encoder is trained using the radar data presented in Sect. 3. It is important to note that the decoders' parameters are frozen at this step: only the encoder is being learned; this differs from classical autoencoder models, for which the decoder and encoder are trained simultaneously. Estimates of the latent features are then obtained from the encoder's output, after inverse normalization and exponential transform for those variables whose logarithm was used as input to the decoder (see Section 4. 1. 2). To prevent occasional convergence toward unrealistic values in the latent space (e. g. D 0 < 0), an additional constraint was incorporated to the loss term to penalize latent values outside of a manually defined range: for a given feature x with realistic bounds x min and x max , this secondary loss reads L sec (x) = 1 (x<xmin) × (x -x min ) 2 + 1 (x>xmax) × (x -x max ) 2.
When looking only at spectrograms with moderate apparent turbulence (e. g. 0. 5 km -1. 5 km), and strict unimodality, the overlap metrics are similar to the decoder. Finally, it is also empirically observed that in some cases (e. g. peak of the spectrograms, which is a propagation of the decoder behavior. As a safety check, it was also verified that running PAMTRA on the latent variables also led to spectrograms close to the original ones, as displayed on Fig. 4 c ). Let us point out at this stage that unlike most machine learning models, which are trained on a dedicated dataset and then implemented on independent data, the encoder is here trained directly on the data of interest. Indeed, the aim is not to create a generic model that can be used to retrieve microphysical variables from any dual-frequency spectrogram: the aim is to find the latent variables which minimize the reconstruction error on specific measurements; the encoder can learn any relevant feature from the input data to achieve this goal.
Training set limitations Another aspect of our framework which could cause a bias in the retrieval is if the training set is too narrow. While special attention was paid to this potential issue as the microphysical parameters were sampled from the MASC database, there is likely still room for improvement. In particular, the size cutoff for good-quality images in the MASC is quite high and very few particles with a diameter below 0. 5 mm are accurately captured. For reference, Fig. 13  illustrates the histogram of D 0 derived from MASC measurements  (Grazioli et al. , 2022) , and by the aircraft 2D-S and PIP probes during the ICE GENESIS campaign. The limitation is apparent: the aircraft is able to capture much smaller particles, but not beyond a certain size, while the MASC can detect large snowflakes but very few small particles.
Scattering model Surely one of the strongest hypotheses on which the pipeline was built is the parameterization of the scattering model in forward simulations. As explained in Sect. 3. 1. 1 and Appendix B2, the default version of PAMTRA was used which to this date (28/06/2021) assumes constant values for certain parameters of the SSRGA, and allows to change two coefficients (κ and β, cf. Hogan and Westbrook (2014)  and online documentation  (PAMTRA team, 2015) ). Several studies suggest however that more parameters are needed, and moreover that their values can vary significantly (e. g. Leinonen et al. , 2018a; Ori et al. , 2021)  depending on particle type, shape, etc. To get an empirical sense of how this could affect the retrieval, the approach described hereafter was followed. First, a few time and range gates were randomly selected from the dataset.
The corresponding retrieved values were then modified by adding a D 0 offset ranging from -1. 5 to +2 mm, and PAMTRA simulations were run on the microphysical parameters obtained, using the same settings as in Sect. 3. 1. 1. Parallel to that, slight modifications of the PAMTRA code were conducted to allow for modification of the four literature coefficients of the self-similar Rayleigh-Gans approximation (κ, β, γ, ζ); new simulations were run for the selected time and range gates, keeping the retrieved microphysics unchanged but randomly changing the SSRGA parameters within ±10% of their original values. As seen in Appendix B2, this is well within the typical variability of the coefficients calculated from simulating various types of particles (e. g. Leinonen et al. , 2018a, , Fig. 5  shapes but mostly affected the amplitude of the spectra: hence, the influence of the changes were measured by the change in the scalar "total" reflectivity at X-and W-band, then in the dual-frequency ratio.
59 (with this direct inversion). Similar behaviors are observed for the other retrieved variables. Let us mention an alternative approach that could be used, which lies half-way between classical OE and the proposed method. The notations used here are those of Sect. 2. Once a differentiable approximation of the forward model is known ( f ), another way to look for X r is to find the minimizing argument of || F(X) -Y r || 2 using gradient descent; a regularizing term can be added to ensure for instance the spatial continuity of X or to enforce some degree of spatio-temporal smoothness. This requires only one deep learning model instead of two, and could thus seem more appealing, but the first approach was preferred. Indeed, by actually learning an approximation of the inverse mapping G, and doing so on a large dataset, the risk of reaching a local minimum in X is reduced.
The "decoder": a differentiable emulator of PAMTRA The first part of the framework consists in developing a differentiable emulator of PAMTRA, by designing a deep learning model and training it on the synthetic dataset (Sect. 3. 1). If viewed in perspective with the technique of auto-encoders, this 11 https://doi. org/10. 5194/amt-2022-199 Preprint. Discussion started: 21 July 2022 c Author(s) 2022. CC BY 4. 0 License. consists in learning the "decoder", which maps the latent space -containing the physical variables-to the high-dimensional measurement space -the spectra. It was chosen to train separately the X-band and W-band decoders rather than use a single algorithm emulating both simultaneously; indeed, the two frequencies may have slightly different smoothness or amplitude features, which justifies the use of distinct architectures.
3. 2. The spectrograms are normalized using the statistics used for normalization in the decoder part (means and standard deviations from the synthetic spectra). Rather than using the entire Doppler spectrograms (N rg = 100) as one training item, chunks of the spectrograms are used with N rg = 25, and are sampled in the following way: the first chunk corresponds to i rg = 0. 24, the second chunk to i rg = 5. 29, the third to i rg = 10. 34, etc. ; i rg being the range gate index. The dataset is then randomly shuffled at each epoch during training. Rearranging the dataset this way makes training both more tractable -thanks to the smaller size of each item-and more robust -due to the data augmentation, which helps avoid local minima during the training process. During training, the encoder output is passed as input to the X-and W-band decoders, which then output reconstructed spectrograms.
Instead of learning an inverse mapping, it simply learns to emulate the forward model: taking microphysical and atmospheric (i. e. latent) variables (x s https://doi. org/10. 5194/amt-2022-199Preprint. Discussion started: 21 July 2022 c Author(s) 2022. CC BY 4. 0 License. asinput, it outputs Doppler spectra (y s. This model, which we hereafter refer to as the "decoder" and denote f , is thus a differentiable emulator of PAMTRA. When applied not to a single set of microphysical descriptors, but to a stack (e. g. of multiple range gates) at once, it is denoted with upper-case F. This synthetic dataset should include a wide range of realistic parameters, to not induce bias in the further steps. -In a second step, we shift our attention to a real (i. e. not synthetic) dataset of full dual-frequency Doppler spectrograms Y r ; the aim is to retrieve the underlying profiles of latent variables X r.
Code availability. The code for generating the training set using PAMTRA, and for training the decoder and encoder models, is available at https://github. com/annecbroux/DeepSpectralRetrieval (latest version). Data availability. The data of the ICE GENESIS campaign will be made available on the Aeris platform (https://en. aeris-data. fr/). The entire dataset will be made fully public, after an embargo period and the necessary post-processing time, at the beginning of 2023. Individual access can be granted before this time upon contact with the authors.
4. 1 1 The "decoder": a differentiable emulator of PAMTRA The first part of the framework consists in developing a differentiable emulator of PAMTRA, by designing a deep learning model and training it on the synthetic dataset (Sect. 3. 1). If viewed in perspective with the technique of auto-encoders, this 11 https://doi. org/10. 5194/amt-2022-199 Preprint. Discussion started: 21 July 2022 c Author(s) 2022. CC BY 4. 0 License. consists in learning the "decoder", which maps the latent space -containing the physical variables-to the high-dimensional measurement space -the spectra. It was chosen to train separately the X-band and W-band decoders rather than use a single algorithm emulating both simultaneously; indeed, the two frequencies may have slightly different smoothness or amplitude features, which justifies the use of distinct architectures.
Experiments were done in a batch reactor, and samples were removed at specific time points from the reactor over the course of one run. Fifteen repetitions (runs) were done (9 were reported before, 6 additional experiments are included here). The full experimental details can be found in Gómez  Ruiz et al. (2018). These 15 experiments/repetitions are analyzed here. Compared to the original paper  (Gómez Ruiz et al. , 2018) , data were recalculated as follows: time in seconds was recalculated to hours and concentration in M to mM. This was done to bring the numerical values in the same order of magnitude to avoid numerical difficulties in the MCMC (Markov Chain Monte Carlo) procedure to approximate the posterior distributions (see below). This has no effect on the modelling, only on the units of the resulting parameter estimates.
Software The calculations, plots and writing were done in RStudio (version 1. 4. 1103) using the R package papaja. The R package brms (bayesian regression models using stan, version 2. 16. 1) was used for regression  (Bürkner, 2017 (Bürkner, , 2018)). brms uses the probabilistic programming language Stan in the background for the MCMC calculations, see the website from the Stan development team https://mc-stan. org/users/docum entation/, see also  Carpenter et al. (2017). Graphs were produced using the R package ggplot. MCMC sampling requires thorough checks to be sure that the algorithm converged. In all cases described here, four chains were run simultaneously with at least 4000 simulations of which half was discarded as warm-up. The standard checks for convergence are: trace plots showing graphically whether or not the chains have converged, R (Rhat) that should be very close to 1 (showing whether the chains have mixed well), the number of effective simulations showing how many of the simulations were effective (there is no clear-cut number for this parameter but if it is at least half of the total number of simulations that should be fine).
Analysis of the kinetics of the normalized, averaged data using the first-order model Gómez  Ruiz et al. (2018)  handled the observed variation in the data in two ways: i) by analyzing each dataset separately and studying the variation in obtained estimated rate constants (to be found in the Supplement of their article), and ii) by eliminating the variation in initial concentration by dividing the momentary concentration by the initial concentration for each experiment, c t /c 0. The initial concentration used for normalization was estimated from the individual regressions. Next, they averaged the results obtained per sampled heating time from each run and performed nonlinear least-squares regression (i. e. , in the frequentist way) of these normalized, averaged concentrations as a function of heating time according to a first-order model (equation (  2 )).
With complete pooling, the model is not informed that the experiments come from different runs. Variation is only accounted for in the residual standard deviation/variance, capturing the remaining variation not explained by the model. This is shown in Table  2  by σ e , which is estimated as a parameter with its own uncertainty. The parameter estimates are sometimes mentioned as being at the "fixed effect level," fixed because they are supposed to describe the behaviour at the population level. The question is, though, whether or not fixed effect parameters obtained by complete pooling of data are a good indication for what is going on. That is what will be investigated next in this case study. The opposite of complete pooling is to use no pooling and to apply regression on each dataset separately.
Model comparison Besides posterior predictive checks, models can be compared in their predictive performance. The recommended technique by statisticians is to use loo-cv (leave-one-out-cross-validation)  (Vehtari, Gelman, & Gabry, 2017). This is a technique in which one sample is left out from the data set, the model is then refit without this one sample and it is evaluated how well the left-out observation is predicted. This is repeated for each data point. Because this is computationally expensive, an approximation is used called 'Pareto-smoothed importance sampling crossvalidation' (PSIS in short). Details can be found in  Gabry, Simpson, Vehtari, Betancourt, and Gelman (2019)  and  Vehtari et al. (2017). This technique was applied here, the software for doing this (the R package loo) is taken up in the R package brms.
This result also goes to show that the well-known modeling rule called Ockham's razor (cut down the number of parameters as much as possible) may not always be true in the case of multilevel models. According to McElreath (2020), the most important aspect to consider is finding the right balance between overand underfitting, and multilevel modeling does exactly that. A further advantage of the loo-cv-criterion is that it alerts for possible "problematic observations"; this happens if the "pareto-shape parameter k" is higher than 0. 7. What this basically means is that the software is able to pinpoint observations that are very influential on the outcome. When applying this to the 4 models tested, the output for the two singlelevel, completely pooled models did not show a warning and reported that "All Pareto k estimates are good (k < 0.
Briefly, the ADI as external toxicity reference value is translated into a corresponding HBM-GV by means of the following mass balance equation, assuming, as an example, one major metabolite as an exposure biomarker and steady-state conditions (i. e. , that a balance exists between the intake of the active substance and the internal concentration/excretion of the metabolite used as a biomarker)  [20] : HBM -GV(GenPop) = ADI • MW(Metabolite)•Fue HBM GV GenPop = • • (1) The ADI and molecular weights (MW) were extracted from the most recent EFSA or ECHA assessment for each pyrethroid, or from other expert reviews, such as JMPR (Joint FAO/WHO Meeting on Pesticide Residues). The value for the "fraction of urinary excretion" (Fue) was selected from published data covering toxicokinetic studies in humans (see Section 2.
2 for details and Section 3. 1 for the respective results). For the HBM4EU project, average daily urinary volumes adjusted to bodyweight of 0. 03 and 0. 02 L/kg bw/d for children and adults, respectively, as proposed by the German HBM Commission have been assumed  [20]. Following Equation (  1 ), a screening level was derived for 3-PBA based on conservative approaches, complemented with provisional HBM-GVGenPop established for a number of pyrethroids. In addition, published HBM-GV for deltamethrin and cyfluthrin  [24]  were used for risk assessment. Aggregated HBM data (percentiles and their confidence intervals for the full dataset) obtained from HBM4EU-aligned studies were used. These data are reported in Govarts et al. [31]. The 95th percentiles were selected as conservative estimates of population exposure levels for steps S and I.
The Risk Characterisation Ratio (RCR) was estimated as the The ADI and molecular weights (MW) were extracted from the most recent EFSA or ECHA assessment for each pyrethroid, or from other expert reviews, such as JMPR (Joint FAO/WHO Meeting on Pesticide Residues). The value for the "fraction of urinary excretion" (Fue) was selected from published data covering toxicokinetic studies in humans (see Section 2. 2 for details and Section 3. 1 for the respective results). For the HBM4EU project, average daily urinary volumes adjusted to bodyweight of 0. 03 and 0. 02 L/kg bw/d for children and adults, respectively, as proposed by the German HBM Commission have been assumed  [20]. Following Equation (  1 ), a screening level was derived for 3-PBA based on conservative approaches, complemented with provisional HBM-GV GenPop established for a number of pyrethroids.
In addition, published HBM-GV for deltamethrin and cyfluthrin [24] were used for risk assessment. Aggregated HBM data (percentiles and their confidence intervals for the full dataset) obtained from HBM4EU-aligned studies were used. These data are reported in Govarts et al. [31]. The 95th percentiles were selected as conservative estimates of population exposure levels for steps S and I. The Risk Characterisation Ratio (RCR) was estimated as the ratio between the 95th percentile and the corresponding screening or HBM-GV GenPop value; RCRs higher than 1 represent potential concerns requiring refinement. For the probabilistic refinement, full distributions were reconstructed from the percentiles and used for Monte Carlo analysis using the R-program tool developed by EFSA (https://r4eu. efsa. europa.
Data and Information Sources Pyrethroid metabolites were measured in HBM4EU-aligned studies in adults (age range 20-39 years) and children (4-11 years old) from different European countries, and Israel. These included four studies on adults (DE ESB, FR ESTEBAN, CH HBM4EU study, IL RAV MABAT, ), with an age range of 20-74 years, from France, Germany, Israel, and Switzerland, conducted in the period 2014-2021; and six studies on children (BE 3xG, FR Esteban, CY Organiko, IL RAV MABAT, SI SLO CRP, NL SPECIMEN, ), with an age range of 6-11 years, from Belgium, Cyprus, France, Israel, Slovenia, and The Netherlands, conducted in the period 2014-2020. Table  1  provides summary information on each study and additional details are available in previous HB4EU publications  [31] [32] [33]. Table  1.
Study The ADIs, and additional information on toxicity and toxicokinetics were retrieved from the European Food Safety Authority (EFSA). Conclusions and related documents published in the EFSA Journal https://efsa. onlinelibrary. wiley. com/journal/18314732 (accessed on 25 May 2022) or publicly available through the Open-EFSA web https://open. efsa. europa. eu/ (accessed on 25 May 2022); and from the European Chemicals Agency (ECHA) assessments on biocides https://echa. europa. eu/es/information-on-chemicals/ biocidal-active-substances (accessed on 25 May 2022). References mentioned in these reports, including JMPR and other evaluations, were also considered as needed. A literature search in the databases Web of Science and SCOPUS was conducted for retrieving toxicokinetic information with the focus on studies in human volunteers.
Theoretically, it should be possible to include also in the probabilistic assessment differences in the ADIs. However, as the differences do not represent variability in the dataset, but real differences in the potency and toxicity of the different active substances, this approach should be used in the risk characterisation phase, not for setting screening HBM-GVGenPop values. The available information for deriving a screening value for 4-FPBA is limited to a single study with a single volunteer. Considering that this single value is within the range observed for 3-PBA, the worst-case value of 9% was considered as the best alternative for setting Tier I screening values of 16. 0 µg 4-FPBA/L for children and of 24. 7 µg/L for adults based on the ADI for cyfluthrin of 0. 01 mg/kg bw. Theoretically, it should be possible to include also in the probabilistic assessment differences in the ADIs.
However, as the differences do not represent variability in the dataset, but real differences in the potency and toxicity of the different active substances, this approach should be used in the risk characterisation phase, not for setting screening HBM-GV GenPop values. The available information for deriving a screening value for 4-FPBA is limited to a single study with a single volunteer. Considering that this single value is within the range observed for 3-PBA, the worst-case value of 9% was considered as the best alternative for setting Tier I screening values of 16. 0 µg 4-FPBA/L for children and of 24. 7 µg/L for adults based on the ADI for cyfluthrin of 0. 01 mg/kg bw.