dataesr/scientific-paragraphs-categorization huggingface dataset:
shom. fr. These stations provide time series of hourly observations qualified as "valid" and spanning over at least 30 years. The observations are referenced to the zero tide gauge, which corresponds to the Chart datum (hydrographic datum). Nevertheless, the time series used present some gaps, and the average data availability is ~89% (from the first year of measurement to 2018) for Brest station, ~58% for Cherbourg station, ~59% for Le Havre station, and ~81% for Dunkirk station (Figure  2 ). Sea level data were provided by the French national hydrographic service (Shom), and data are available on data. shom. fr. These stations provide time series of hourly observations qualified as "valid" and spanning over at least 30 years. The observations are referenced to the zero tide gauge, which corresponds to the Chart datum (hydrographic datum).
Nevertheless, the time series used present some gaps, and the average data availability is ~89% (from the first year of measurement to 2018) for Brest station, ~58% for Cherbourg station, ~59% for Le Havre station, and ~81% for Dunkirk station (Figure  2 ). With the aim of studying how extreme surges can be impacted by climate variability, two indices were chosen to conduct this work: SCAND and NAO. Monthly North Atlantic Oscillation data were provided by the NCAR (https://climatedataguide. ucar. edu/climatedata/hurrell-north-atlantic-oscillation-nao-index-station-based, accessed on 1 June 2019), and SCAND monthly data were provided by the CPC (https://www. cpc. ncep. noaa. gov/ data/teledoc/scand. shtml, accessed on 1 June 2019).
Methodology 2. 2. 1. Pre-Processing of Data In order to extract surges, harmonic analyses were performed to calculate the harmonic components of the astronomic tide with a software developed and used by the French Hydrographic Office (Shom). This program, called MAS, allows the calculation of 143 harmonic components, defined by the phase and range of each tidal harmonic, using the least-squares method. Then, a prediction can be made and is subtracted from observation data to obtain surges (i. e. , the nontidal residual). Due to the temporal variability of harmonic components where the phase and range are not constant over the time, dephasing between the prediction and measurements can be observed, which induces a periodicity of 12 h and 25 min in the surges. In order to get rid of this bias, authors regularly consider skew surges, which are the difference between the maximum observed sea level and the nearest high tide  [10].
Analysis of the Time-Frequency Variability of Monthly Maxima Surges and R tionship with SCAND and NAO A continuous wavelet transform was conducted to identify typical timescales o iability in monthly maxima surges. The application of this method produced a time contour diagram with time indicated on the x-axis, period or scale on the y-axis, and plitude or power or variance on the z-axis. More details, as well as explanations o methodology and applications for environmental signals, can be found in  [30] [31] [32] [33]. As for many (but not all) spectral methods, it is necessary to get data without To maximize the temporal lengths of these time series and to prevent the loss of information, the start years used to analyze these datasets were not equal in this study. Dunkirk station was analyzed between 1957 and 2018, Le Havre station between 1972 and 2018, Cherbourg station between 1975 and 2018, and Brest station between 1846 and 2018.
Analysis of the Time-Frequency Variability of Monthly Maxima Surges and Relationship with SCAND and NAO A continuous wavelet transform was conducted to identify typical timescales of variability in monthly maxima surges. The application of this method produced a timescale contour diagram with time indicated on the x-axis, period or scale on the y-axis, and amplitude or power or variance on the z-axis. More details, as well as explanations on the methodology and applications for environmental signals, can be found in  [30] [31] [32] [33]. As for many (but not all) spectral methods, it is necessary to get data without gaps. In this work, gaps were filled by linear interpolation to address this issue. The Matlab package «Grinsted Wavelet Coherence» was used to perform the continuous wavelet transform  [34].
This is illustrated later using the monthly maxima of surges. Finally, for each wavelet detail, we estimated a percentage of energy that represents the importance of the detail in the total variability. The relationship between monthly maxima surges and SCAND or NAO was detected with wavelet coherence. This allowed us to identify regions in the time-frequency space where signals covariate  [30]. This method describes the intensity of the relation on a scale from 0 to 1. If the value is 0, the signals are not correlated. If the value is 1, a linear correlation exists between the two signals at time t and scale a. The Matlab package «Grinsted Wavelet Coherence» allowed us to conduct wavelet coherence and estimate the dephasing of two signals according to the timescale. Simultaneously, we also used the MODWT to visualize the covariability between corresponding climate indices and monthly maxima surges timescales.
When this happened for several years, we observed an increase in the amplitude of the ~2-3-year variability (e. g. , for years around 1990). Indeed, between 1987 and 1991, there were numerous storm events of exceptional strength (e. g. , the hurricane (1987), Daria (1990), Meteo France database (http://tempetes. meteofrance. fr/, accessed on 1 June 2019)). At Dunkirk, the ~2-3-year variability appeared to be particularly intense before the 1990s and then showed a decrease in amplitude after the 1990s. A similar phenomenon was found for the ~5-7-year variability.
Moreover, the spots of increased variability appeared over the same periods for different stations (e. g. , for the ~5-7-year variability from around 1980 for stations in the English Channel and Brest). Therefore, these common patterns might have been generated by the same large-scale atmospheric drivers that we investigate in the next section. Brest, Cherbourg, Le Havre, when the winter monthly maxima were particularly high. When this happened for several years, we observed an increase in the amplitude of the ~2-3-year variability (e. g. , for years around 1990). Indeed, between 1987 and 1991, there were numerous storm events of exceptional strength (e. g. , the hurricane (1987), Daria (1990), Meteo France database (http://tempetes. meteofrance. fr/, accessed on 1 June 2019)). At Dunkirk, the ~2-3-year variability appeared to be particularly intense before the 1990s and then showed a decrease in amplitude after the 1990s.
In this manuscript, we discuss the capabilities of a deep learning algorithm implemented with the Conventional Neural Network concept to characterize the hydraulic properties of aquifers. The algorithm called CNN-HT is designed to predict the inverse operator of hydraulic tomography using a synthetic training dataset in which the hydraulic head data associated with pumping tests are linked to hydraulic transmissivity field. This approach relies on an adaptation of the SegNet network that was initially developed to process image segmentation. The SegNet is composed of encoders and decoders networks. In the encoder, sequential operations with multiple filters, as convolution, batch normalization, max-pooling are performed to identify feature maps of the input data. In the decoder, the up-sampling, convolution, batch normalization and regression operations are used to prepare the output by recovering the loss of spatial resolution that occurred in the encoder process.
Laloy et al. (2018)  used the Generative Adversarial networks to generate randomly in 2D and 3D binary hydraulic conductivity fields and MCMC to perform the inversion process. In geophysics realm,  Wu and Lin (2018)  applied CNN with encoder-decoder networks for mapping the subsurface velocity from seismogram data. Puzyrev and Swidinsky (2021) , used the CNN network to determine the vertical subsurface heterogeneity of electrical conductivity by training electromagnetic data. In this article, we discuss the use of the convolution neural networks formed according to the SegNet architecture to process hydraulic tomography. This technique was initially designed for the semantic segmentation of objects learned on the images  (Badrinarayanan, et al. 2017). To illustrate the inversion principle with the CNN-HT structure, we organize this manuscript with the following outline: the first section will be devoted to the generation of data used in training, validation and test, and the second to the introduction of concept of the CNN-SegNet structure.
Preparation of training data In deep learning algorithm, the first step concerns the construction of training data that will condition the accuracy of the result obtained from the predicted inversion operator (see Fig. 1 ). Given the impossibility of having a real dataset connecting the spatial distribution of hydraulic properties of aquifers and the piezometric responses, we use synthetic models in which the hydraulic parameter fields are generated geostatistically and their corresponding piezometric responses are obtained by solving the numerical groundwater equation in steady state (forward problem). F(m) : { ∇. (T∇h) = q p δ ( x p -x ) , h = h 0 , Γ d ( 2 ) where m is the negative of logarithm of transmissivity T = 10 -m randomly generated and distributed on the studied domain. h is the hydraulic head responses due to water extraction represented by a point source term q p δ ( x px ) defined at pumping well locations x p (δ is Dirac delta function).
The pumping tests will be done sequentially on several wells. The forward problem is solved by finite element technique using the Comsol software with a constant hydraulic level h 0 imposed as Dirichlet boundary at all limit of the domain Γ d that are positioned far from the studied area for reducing their impact. As the nature of the training models conditions the result of the CNN-HT inversion, we have to use the prior information on the studied field to establish a cluster of models for training and targeting a type of solution with certain characteristics. In this paper, prior information is expressed in statistical terms, with a Gaussian distribution wherein the mean and covariance, are held constants during model generation. Once the parameters of the distribution are defined, we launch the generation of thousands of hydraulic transmissivity fields using the SGeMS code implemented in Matlab  (Remy et al.
These realizations aim to evaluate the effectiveness of the CNN-SegNet approach in the reconstruction of hydraulic transmissivity field and to analyze the sensitivity of this Fig. 3. SegNet Convolutional Neural Networks with encoder-decoder structure. Modified from  (Badrinarayanan, et al. , 2017). Here, the piezometric data can be resized to the same dimension of the output field using the projection method. reconstruction to the size of the training dataset and the amount and uncertainties of the hydraulic head data collected during the pumping tests.
Effect of training data size In the first analysis, we examine the impact of training data size on the reliability of the inverse operator prediction and determine the amount of data required to obtain a satisfactory prediction. For this purpose, we performed three assessments with different training data sizes: T1 = 1000, T2 = 3000, T3 = 10000, the validation was done with 1000 models and the test with 500 models different from those used in the learning phase (training and validation). During the optimization process, we followed the quality of the minimization of loss functions with the training and validation data until they reached optimal values to stop the process manually or wait until the maximum number of epochs was met (see Fig. 5 ). In this way, we avoid the problem of overfitting which appears especially when we tried to build the network with a limited number of training data (T1 = 1000).
On the other hand, the adjoint state technique requires less computation, by iteratively solving an adjoint operator with the form of the forward problem on the number of observation wells, but its implementation is quite complex (for more details see  Sykes et al. , 1985; Cardiff & Kitanidis, 2008). In this section, we test the implementation of Gauss-Newton algorithm to invert the synthetic hydraulic data determined with a hydraulic transmissivity field shown in    2. These data were then contaminated with noise with a variance of 5 cm 2. The inversion of these datasets was constrained, as explained in the introduction, by geostatistical parameters such as covariance and mean. The results of the inversions with the GN algorithm using the finite difference and adjoint state methods are presented in Fig.
In particular, the DL method becomes faster when we reduce the amount of training data to T = 500 and T = 2500 (20 min and 1 h41 min), because even with these small samples, the transmissivity field can be well reconstructed well in a reasonable amount of time, as demonstrated by the comparison of real and inverted fields (Fig. 10 ). At the end of this comparison, we conclude that both the CNN-HT and Gauss Newton codes are based on the forward problem for which the computation time depends on the numerical method and the degree of mesh refinement used to solve the groundwater flow equation. The machine learning technique uses this forward problem to generate a training database to build the CNN network. The time spent in this construction can be equivalent or less to the time spent in finding a local minimum with the Gauss Newton using the finite difference method to calculate the Jacobian matrix over a large number of unknown parameters.
However, it is difficult to have an idea in advance of the time that will be spent in building this database because the size needed to get accurate results is related to the degree of complexity of the input-output relationship and the type of network. The optimal size as well as the rest of the network parameters (epochs, number of encoders) are only established by a trial-and-error analysis, which can take a considerable amount of time. Therefore, the data size used in this paper cannot be generalized to all cases treated by this network or others. Both methods also share their dependence on prior information. In the Gauss-Newton algorithm, this prior information is introduced as a constraint to guide the optimization towards realistic models and reduce the uncertainties associated with the non-uniqueness of the solution.
On the other hand, the CNN-HT algorithm uses it to generate the training models. The CNN-HT offers the possibility of incorporating multiple prior models into the construction of the training dataset, which is difficult to perform in a deterministic algorithm where the mathematical forms of these constraints must be differentiable. In addition, both methods are sensitive to the uncertainties associated with the measured data. The CNN-HT allows simultaneous learning with data affected by several degrees of uncertainty by adding noise to the training data, then combining this set with the original to double the size of the training data and take into account the impact of noise. Regarding the quantification of the prediction uncertainties, in the Gauss Newton method, we use the a posteriori covariance formulation to get an idea of the uncertainty interval concerning the local minimum  (Tarantola and Valette, 1982).
This network ends with a regression layer to evaluate the performance of the inverse operator approximation. The quality of this approximation is highly dependent on the amount of data used in the training with an accurate approximation when the dataset is large. This makes the assembly of the training data the most expensive step in time, requiring several hours compared to the learning process that in this case does not exceed 30 min. In practice, it is preferable to launch the learning with few data and check whether or not the desired accuracy is achieved before generating large amounts of data unnecessarily. In terms of comparison, the time spent in forming a training database can be equivalent to the time spent to reach a local minimum with Gauss Newton using the finite difference method to compute the Jacobian matrix In addition to the amount of training data, the nature of this data also controls the prediction results of the network.
Regarding the advantages, we point out that: • The CNN-HT method uses the forward problem as a black box to generate the training data with user-selected transmissivity fields, thus avoiding numerical instability and outliers in hydraulic head simulations related to its solving. These numerical problems can be encountered when using conventional inversion methods (stochastic and deterministic) if they are ill-constrained. • Once the inverse operator is trained, the network permits to interpret the piezometric data in a few seconds without the user having to add new parameters. • The CNN-HT code provides predictions that require any initial hydraulic transmissivity models or computation of sensitivity matrix, as is the case with deterministic methods. This advantage makes CNN-HT an effective inversion tool for dealing with highly nonlinear inverse problems with large-scale parameterization that are difficult to handle with deterministic codes, especially when the computation of sensitivity is complex.
• The CNN-HT code performs reconstructions of the transmissivity field that depend on the a priori model used in the generation of transmissivity field for the training data. This dependence to the a priori model is a property shared with other inversion methods. However, in this CNN-HT tool, the mathematical form of this a priori model is not subject to any condition, which is not the case with deterministic methods where the model must have a quadratic and differentiable form. • The CNN-HT algorithm also provides great flexibility for the user to incorporate multiple prior models, especially when in doubt about which a priori information to use. • The proposed method is based on a simple network that can be generalized and applied to handle hydraulic tomography in 3D or in time domain in order to identify the storativity coefficient as well, by only modifying the training dataset.
It is also easily adaptable to deal with hydraulic tomographies with complex heterogeneities having for example multiple discontinuous hydrofacies with contrasting transmissivities, or parameterized in discrete fracture network mode. These types of problems can be treated as a classification task by replacing the regression layer of the network with the Softmax layer. In this way, the network can identify fracture geometries or hydrofacies shapes. • The code allows the incorporation of uncertainties on the boundary conditions imposed in the numerical solving of the forward problem by randomly choosing their values within a confidence interval when building the training data set. • This code, like other deep learning algorithms that use the convolutional neural network technique are able to better reduce the effects of noise in the data.
This is thanks to the convolution concept that processes the input data with some local connectivity and not in a pointwise manner as in other conventional inversion methods  (Vu and Jardani, 2021). The impact of noise could be minimized and accounted for in the inversion results by contaminating the hydraulic data used in the training data with a noise signal. Regarding the disadvantages of this method, we mention that • It is based on a repetitive and time-consuming numerical resolution of the forward problem, which is involved in the construction of the training database. The size of this database conditions the quality of the predictions, and its optimal size can only be obtained by a long trial-and-error procedure. • The method requires important computing resources to carry out the learning.
• The code also uses a large number of parameters that intervene in the construction of the networks (such as: size of the decoder-encoder, number of filters and their size), and in the optimization algorithm used in the training phase (number of epochs, regularization parameters, initialization of filters, Batchsize, Dropout). The determination of all these parameters implies a tedious sensitivity analysis with several training operations to choose the best parameters. • The method does not provide an uncertainty assessment of the predicted transmissivity fields. This is a major weakness of this method compared to classical approaches. There are some attempts in the literature to estimate the uncertainty of the prediction determined with DL algorithms by performing the learning in a probabilistic way  (Kendall and Cipolla, 2016; Gal and Ghahramani, 2016).
In Deep Learning (DL), a subfield of Machine Learning (ML) that aims to build learning capabilities in computers, these challenges of reproducibility are not different and can be more complex  (Heil et al. , 2021; McDermott et al. , 2021; Pineau et al. , 2020a; Renard et al. , 2020). There are additional complexities to handle during a DL experiment, such as the large number hyperparameters of the training process and the size of data (with possibly missing datasets and changes to the data), changes to some of the algorithms, and consequently versioning control to deal with the many iterations during training. In summary, these sources of "variability" can be due to: (a) the data set, (b) the DL architecture, (c) the optimization procedure, (d) the hyperparameters for the optimization, and (e) the implementation and infrastructure  (Renard et al.
The ML and DL communities such as Paperswithcode (2021) and journals such as Association for Computing Machinery (2020) are working to reward and motivate the adoption of R & R practices. Recommendations have been proposed to better organize the steps of a ML workflow, such as DOME-ML (Data, Optimization, Model and Evaluation in Machine Learning) which aims to create standards for supervised ML validation in biology  (Walsh et al. , 2020). There are a growing number of workflow tools such as Apache Airflow  (Kotliar et al. , 2019) , MLFlow  (Zaharia et al. , 2018) , Collective Knowledge technology  (Fursin, 2020)  and Whole Tale  (Brinckman et al. , 2019) , the last mentioned capturing the prospective provenance of all data products generated during a study. Other resources have been proposed such as PRIMAD  (Freire et al.
Then the data are divided into training, validation and test datasets following a "cross validation strategy". 3. Deep Learning experiment. First, the DL architecture and hyperparameters are configured, that is, the designer selects an architecture to use (by choosing a known DL model, designing a new architecture itself, or using a network architecture searcher through optimization). A typical architecture describes the number of nodes, the number of layers and their types (fully connected, foldable, pooling, dense, etc. ) and the connectivity between layers. Second, the training process takes place. In this phase, some experiments can be planned to find the optimal set of parameters within the DL model. All the hyperparameters (including learning rate, batch size, etc. ) must be carefully described in order to run the model.
Examination of Reproducibility of an Experiment Three aspects are involved in the evaluation of the reproducibility of an experiment reported in a paper: the information provided in the paper, the specificity of the DL experiment, and the "FAIRness" of the study. To assess the adequacy of the information provided in the paper about the experiment we used the Machine Learning checklist proposed by  Pineau (2020b) , noticing that the paper and the code are two separate research artifacts, each with their own checklist. Second, we assessed the steps of the DL workflow using the recommendations of  Renard et al. (2020) , focusing on the sources of potential difficulties ("variabilities") in each case study. Finally, we assessed the "FAIRness" of the DL code using sub-principles following the work of  Hartley and Olsson (2020).