dataesr/scientific-paragraphs-categorization huggingface dataset:
Additional datasets We validate our results against other observational and synthetic datasets. The sea surface salinity (SSS) CCI dataset is distributed at 1/4 • horizontal grid resolution from 2010 to 2019  (Boutin et al. , 2021). We do not use SSS as an input variable for several reasons. SSS satellite observations only cover the period 2010-2019, and its quality is questionable at high latitudes and in cold water  (Boutin et al. , 2021). As our prediction only depends on the input variables, it is risky to rely on data with systematic errors. Moreover, we tested an architecture with SSS as input, and results were not improved significantly: only the surface salinity was slightly better. The relative importance algorithm also showed that SSS was not used significantly in predictions. It has the same order of importance than geostrophic currents (see Sect.
A section of OSnet is compared to hydrographic section AT20 sampled along 52. 3 • W by the research vessel Atlantis from 1 to 11 May 2012  (McCartney, 2012). Finally, we use T -S data sampled at moorings of the Line W array, deployed in April 2004 between Cape Cod and Bermuda. We use profiles from the third mooring located at 69. 11.
Method The method is composed of three steps (Fig. 2 ). Firstly, a neural network is trained to predicts T -S profiles and MLD from satellite data. Secondly, an adjustment of predicted T , S and MLD corrects the vertical shape of the profiles towards a physically consistent solution. Finally, an operational phase uses the trained network and MLD adjustment to predict T , S and MLD on daily grids from the satellite data. The procedure is coded in Python with the help of several useful modules. The NN algorithm is coded with Tensorflow  (Abadi et al. , 2016)  and the Keras application programming interface  (Chollet and Others, 2015). It is explained with Shap  (Kaur et al. , 2020). Xarray  (Hoyer and Hamman, 2017) , Dask  (Rocklin, 2015)  and Numba  (Lam et al. , 2015)  are used for fast computation and the management of large datasets.
The color palettes for maps are from Cmocean  (Thyng et al. , 2016)  and Colorcet  (Kovesi, 2015). All the codes to build OSnet models are available at https: //github. com/euroargodev/OSnet (last access: 27 June 2022,  Pauthenet et al. , 2022b) , and the models and prediction tools specific to the Gulf Stream region are available at https://github. com/euroargodev/OSnet-GulfStream (last access: 27 June 2022,  Pauthenet et al. , 2022c).
The NN uses 12 surface inputs that are listed in Table  1  to predict profiles of temperature (T ), salinity (S) and MLD mask (K). T -S profiles are then adjusted using the profile K for a better prediction of the MLD. linear activation function (ReLU) for the hidden layers, linear for T and S output, and a sigmoid for the MLD output. We tested more complex architectures (additional layers, convolutional layers, bottleneck architecture) but could not improve the accuracy of the results with them. A simple architecture is advantageous for its lower computation time. The input consists of 12 values, listed in Table  1:   We then run this model on the test dataset (data not seen in training or validation) to confirm the good generalization of the model (i. e. , training and test errors are similar, Fig.
3 ). Given a NN architecture with good generalization properties, we retrain a NN using all 67 767 profiles (Fig. 3 , orange) as training data. The training of one model takes ∼ 20 min on an eight-core CPU with 32 Go of RAM. To further improve the prediction performance and assess the associated confidence intervals, we exploit a bootstrapping scheme  (Breiman, 1996). More precisely, we bootstrap the training procedure 15 times using a different initialization and training dataset each time. Indeed, because of the instability of the prediction method, bootstrapping can give substantial gains in accuracy. Overall, given 15 trained models, we compute the mean T , S and K profiles for all input data and their standard deviation (Fig. 3 , grey). The latter delivers an estimate for the confidence interval.
Prediction of the mixed-layer depth We define the mixed-layer depth H with a density deviation from the surface method, as proposed by de Boyer  Montegut et al. (2004). It is the depth at which the potential density referenced to the surface, σ 0 , exceeds by a threshold of 0. 03 kg m -3 the density of the water at 10 m, σ 0 (z = -H ) = σ 0 (z = -10 m) +0. 03 kg m -3. This definition is chosen for its simplicity of application but tends to overestimate deep winter MLDs compared to the more sophisticated hybrid algorithm of  Holte et al. (2017). These regions of deep winter MLD are rarely observed in our dataset (1145 profiles with MLD deeper than 300 m, i. e. , 1. 7 % of the dataset), comforting our choice of using a simple density threshold. For the NN, the mixed layer is represented in the form of a unitless profile K of size Z = 51 that is filled with zeros between the surface and the mixed-layer depth H and with ones from H to -1000 m.
We first modify the K into a new mask K * as Ocean Sci. , 18, 1221-1244, 2022 https://doi. org/10. 5194/os-18-1221-2022 SST is added with a purple dot and a horizontal bar for its mapping uncertainty. follows: K * z =    Kz if Kz < λ, 2 -Kz if λ > Kz > 1, Kz if Kz = 1, (2) with λ = 0. 57 the value of K corresponding to the MLD. The calibration of λ is done by a cross-validation procedure according to the estimation bias between T at the sea surface and the SST value (Fig. 5 ). In other words, λ = 0. 57 allows us to adjust the MLD while keeping null the mean difference between T and SST (green in Fig. 5 ). We expect this value (λ) to be specific to our region and NN parameterization. It would likely require a new calibration for another study. After computing the K * profiles, we reconstruct iteratively T -S profiles with the gradients multiplied by K * , starting from the bottom value (at z = 1000 m) where K * = 1 (because the deepest MLD never reaches 1000 m, the maximum observed in the CORA dataset is H = 628 m for our region).
The change in the output is representative of the importance of the input for predicting the output. Shapley regression values  (Shapley et al. , 1953)  represent the impact of an input on the output by removing it from the input set and retraining this model with the subset of inputs. This being computationally expensive, it is possible to obtain an approximation of the effect of removing a variable from the model by integrating over samples from the training dataset using the Shapley sampling values  (Štrumbelj and Kononenko, 2014). It produces an "importance" value for each particular prediction. The importance value is positive or negative, indicating the direction in which the input influences the output relative to the averaged output. The SHAP algorithm being computationally expensive, it was not possible to run it over the full dataset.
Accuracy of the predictions compared to observations Table  2  presents several metrics to evaluate OSnet, Armor3D and Glorys12 relative to CORA. Each dataset is predicted or subsampled at the locations of CORA profiles (in longitude, latitude, depth and time). The first two lines indicate the number and size of vertical density inversions. They indicate that the MLD adjustment of OSnet suppresses almost all density inversions, from 17. 3 % to 0. 32 % of profiles. Meanwhile, Armor3D has about 50 % of profiles with density inversions and Glorys12 almost none (0. 01 %). Regarding the amplitude of density inversions, the MLD adjustment suppresses sufficiently large inversions and decreases by 1 order of magnitude the mean amplitude of inversions, from 10 -3 to 10 -4 kg m -3. The root mean square error (RMSE) of the MLD (Table 2) indicates that the MLD adjustment improves the MLD RMSE of OSnet from 50 to 40 m, which is the same order of magnitude as Glorys12 (38.
Temperature and salinity maps Let us examine a daily map of temperature and salinity at 1/4 • resolution (Fig. 6 ). We chose a date in the pre-Argo era to illustrate the generalization skill of the OSnet product. All the maps reveal coherent horizontal structures. At the surface, the warm Gulf Stream detaches from Cape Hatteras and meanders further east, transforming into the North Atlantic Current (Fig. 6a, b ). The surface confidence intervals are maximum for the cold and fresh waters near the edge of the continental slope and inside the cold and fresh core eddies and meanders (Fig. 6c, d ). On average, confidence intervals highlight cold waters north of the Gulf Stream (Fig. 7a, b ), which is consistent with the error of prediction maps presented in Buongiorno Nardelli (2020). This could be due to the lack of profiles containing these cold waters in our dataset.
Time series of surface properties To assess the accuracy of OSnet through time, we analyze the time series of the spatially averaged surface temperature and compare it to the observed SST time series as well as the Glorys12 and Armor3D products. We average data over the region after removing values on the shelf (bathymetry > 1000 m). The long-term trends are obtained by applying a seasonal-trend decomposition based on local regression (STL)  (Cleveland et al. , 1990). STL is a filtering procedure that extracts three components: (i) the variations in the data at the seasonal frequency (Fig. 13 ), (ii) the lowfrequency variation together with nonstationary and longterm changes (Fig. 12 ) and (iii) a remaining high-frequency component. OSnet follows best the long-term trend of SST with a linear trend of 0.
197 • C/decade, close to the SST trend of 0. 190 • C/decade (Fig. 12a ). For comparison, the surfaceaveraged temperature of Armor3D is warmer in the pre-Argo years, probably due to their background global average that is mostly composed of Argo floats. This is a validation that OSnet generalizes well and is not biased towards the recent years of in situ observations. Regarding surface salinity longterm trends, no significant trend is captured over the 1993-2019 period (Fig. 12b ). There is no clear agreement between the different datasets, except in the last period, 2010 to 2019, where all averages increase like the SSS signal. Armor3D mean surface salinity drops significantly during the last 2 years, 2018 and 2019, out of the SSS error margin. Note that OSnet does not include the areas over the continental shelf and does not predict deeper than 1000 m.
A significant part of the climatic signal takes place in the coastal regions  (Ezer et al. , 2013; Davis et al. , 2017) , and an improvement of OSnet would be to deal with profiles of different lengths in order to include these regions. The mean seasonal variation of surface temperature, salinity and MLD is presented in Fig. 13. We compute it by averaging the signal by day of the year (week of the year for Ar-mor3D). The surface temperature seasonal signal is well reproduced for each dataset, as expected considering that SST is included in the input of OSnet and used to produce both Armor3D and Glorys12 (Fig. 13a ). A close observation of the curves in Fig. 13a  shows that the seasonal cycle of OSnet surface temperature is too cold between May and September (Fig. 13c ). This is due to the MLD adjustment, since the direct prediction of surface temperature gives a very precise seasonal cycle of SST (red line in Fig.
Comparison to observed data One major feature of OSnet is the possibility of estimating T and S profiles at any location, given that surface data are available. Here we predict T -S at the location of a mooring of the line W (Fig. 14 ) and along the hydrographic section AT20 (Fig. 15 ). Temperature and salinity at 1000 m are plotted in comparison to mooring data for a period of 3 years  (10 May 2004 to 11 March 2007). OSnet corresponds well to observations but with a slight warm and fresh shift in the first year (Fig. 14 ). A warm-core eddy crosses the location of the mooring in 2006 (see the SLA map in Fig. 14a   We compare the OSnet T -S structure along the hydrographic section AT20 sampled in May 2012 by the research vessel Atlantis (Fig. 15 ). The OSnet prediction is done at the exact locations of the conductivity-temperature-depth (CTD) profiles, interpolated linearly on the maps of input data (Table  1 ).
The comparison is also done for Glorys12 but by collocating the profiles with a nearest neighbor method. Both Glorys12 and OSnet predictions are coherent, but we note two specific biases in Glorys12. First, Glorys12 displays a deep patch of MLD around 41 • N, north of the Gulf Stream, that is not observed by the CTDs nor predicted by OSnet (Fig. 15d ). This deep MLD could be due to the nearest neighbor selection of the profile that is not exact in the case of Glorys12, or it could be an artifact of their model. Indeed, deep patches of MLD are also visible on daily MLD maps of Glorys12 but are absent on the OSnet daily MLD maps (Fig. 8 ). Second, the salinity reconstruction of Glorys12 differs more from the observed data north of the Gulf Stream (Fig. 15g ). OSnet performs very well along this section in comparison to Glorys12.
Prediction intervals (PIs), also called "coverage probability", could be computed as a complement of confidence intervals  (Khosravi et al. , 2011). While the confidence intervals (Fig. 7 ) give the range of variation of a set of NNs with different initializations and training datasets, the PI gives the probability of the observation associated with the prediction being in a range of values. The PI can be obtained by adding output variables to the proposed architecture. Here, the PI could be represented as temperature and salinity profiles around the prediction, representing the 95 % interval. It means there would be a 95 % probability of the true value associated with the prediction lying within the interval. Given the very promising results of our study, obvious future work would be to apply OSnet in other more challenging regions, with fewer data or more complicated vertical structures and different dynamics.
Conclusions We proposed a method to estimate the ocean stratification from surface data using a neural network trained from in situ historical data. The originality of this study is the attention we gave to the vertical coherence of T -S profiles, in particular the accuracy of MLD predictions and the absence of unrealistic vertical density inversions. The global nRMSEs of T and S are better than a state-of-the-art ocean re-analysis (Glorys12) but worse than Armor3D predictions. However, OSnet predictions do not have any unrealistic density inver-sions, while Armor3D does. Each OSnet profile is predicted independently, but OSnet produces coherent horizontal patterns on a 1/4 • daily grid, especially for MLD. In addition, the pre-Argo years are well reconstructed, which supports the good generalization skill of OSnet.
Confidence intervals issued from the bootstrapped method provide an estimate of the prediction variability. Confidence is lower in the cold surface waters north of the Gulf Stream and in the jet at depth, which corresponds to the most variable areas. The reconstructed surface temperature reproduces the observed warming trend. The seasonal cycle of surface salinity matches best the one of SSS compared to Glorys12 and Armor3D. One convenient feature of OSnet is the possibility of estimating profiles at any location, given that the surface data are provided. This allows us to compare predicted profiles at the exact location of the observed CTD, for example. It is computationally inexpensive to run, and we encourage anyone who needs to predict ocean stratification from surface data https://doi.
latitude, longitude, day of the year (cosine and sine), bathymetry, MDT, SST, SLA, four geostrophic velocities U , V and both anomalies. Inputs are linearly interpolated at the in situ location of each profile. The outputs are predictions of three vectors of 51 depth levels (temperature, salinity and the MLD mask). The depth levels are the ones presented in the data section, on which the CORA profiles are interpolated. The dataset is split (randomly with no replacement) as follows: 20 % of the profiles are set aside for testing. In the remaining 80 %, we use 80 % as training data and 20 % as validation data. Validation data are used to avoid overfitting by assessing the performance of the trained model after each epoch (one epoch sees all the training data). Be aware that the train and test data are not truly independent: the selection is random without accounting for spatial and temporal autocorrelation.
Acknowledgements. Etienne Pauthenet is funded by the  Euro-Argo RISE  project of the  European Union 's  Horizon 2020 research and innovation programme  (grant no. 824131 ). Anne-Marie Tréguier  is supported by  CNRS  and by the  MEDLEY  project, funded by  JPI Climate and JPI Oceans  under the 2019 joint call. Etienne Pauthenet  would like to thank  Tanguy Szekely ,  Camille Lique ,  Claude Talandier  and  Alexandre Supply  for the useful discussions around the study and  Sean Tokunaga  for his inspiring preliminary work on this subject. Financial support. This research has been supported by  Euro-Argo RISE  project of the  European Union 's  Horizon 2020 research and innovation programme  (grant no. 824131 ). Review statement. This paper was edited by  Anna Rubio  and reviewed by  Michel Crepon  and one anonymous referee.
Code and data availability. The complete code to process input data and to develop a fully trained OSnet model is available at https://github. com/euroargodev/OSnet (Pauthenet et al. , 2022b). A simpler version, focusing on making predictions with OSnet, is also available at https://github. com/euroargodev/OSnet-GulfStream (Pauthenet et al. , 2022c). The OSnet gridded temperature and salinity daily fields of the 0-1000 m Gulf Stream region from 1993 to 2019 are available at https://doi. org/10. 5281/zenodo. 6011144 (Pauthenet et al. , 2022a). The CORA hydrographic profiles are available at https://www. seanoe. org/data/00351/46219/ (last access: 27 June 2022,  Szekely et al. , 2022). MDT CNES-CLS2018 is available at https://www. aviso. altimetry. fr/en/data/products/ auxiliary-products/mdt/mdt-global-cnes-cls18.
html (last access: 27 June 2022, Mulet et al. , 2022b). The SST dataset is available at https://doi. org/10. 5285/62c0f97b1eac4e0197a674870afe1ee6 (Good et al. , 2019). SLA and derived variables are available through the CMEMS portal at https://www. copernicus. eu/en/access-data/copernicus-services-catalogue/ global-ocean-gridded-l4-sea-surface-heights-and-derived (last access: 27 June 2022, CMEMS, 2022). The SSS CCI dataset is available at https://catalogue. ceda. ac. uk/ uuid/4ce685bff631459fb2a30faa699f3fc5 (last access: 27 June 2022,  Boutin et al. , 2022). Armor3D is available through the CMEMS portal at https://doi. org/10. 48670/moi-00052 (Mulet, 2022a). Glorys12 is available through the CMEMS portal at https://doi. org/10. 48670/moi-00021 (Drévillon et al. , 2022). Bathymetry ETOPO1 can be found at https://www.
ngdc. noaa. gov/mgg/global/ (last access: 27 June 2022, NOAA, 2009). The Line W mooring data are available at https://scienceweb. whoi. edu/linew/ (last access: 27 June 2022,  Toole et al. , 2022). The hydrographic section AT20 is accessible at https://cchdo. ucsd. edu/cruise/33AT20120419 (McCartney, 2012).
Algorithm Framework The weight factor transformation approach to the multi-objective problem is as follows: Assign (I = 1,2, …,n) to every sub function (I = 1,2,…,n), where ω i means the corresponding importance in multi-objective optimization, then u = ∑ω i f(x i ). It adds up the linear weights of all the destination functions. This convenient and efficient method will receive a feasible solution which can be used as an initial solution and applied to other methods. It can be seen from the mathematical model that by assigning weights, the multi-objective problem is transformed to a single-objective one, and then is solved by use of genetic algorithm  [24] [25] [26]. Specific solutions are as follows. Step 1: To collect data. Berth position and cranes are assigned to Ship i based on the data collected.
• Reduced order models based on the integration of PV modules in operating cycles for different power types generation, e. g. , thermodynamic cycles simulation software, Matlab/Simulink. • Statistical  [116] , ML and AI models for an increased prediction accuracy (forecasting) and thus higher confidence in the PV system continuous production and performance, e. g. , enhanced solar energy management and conversion. • CFD and optimisation tools for enhanced efficiency of PV systems at the components level, i. e. , materials, cooling technologies for energy storage, thermal isolation and regulation for thermal energy storage. Bourdoucen et al. [117]  presented the analytical modelling and simulation of PV panels considering solar cells' different physical and connectivity parameters, such as temperature, irradiance, and array configuration.
Campana et al. [118]  developed a dynamic modelling tool for PV water pumping systems (PVWPS). The tool allows dynamic simulations for an optimal design of PVWPS depending on operating conditions and dynamic water demand. Vogt et al. [119]  developed models to simulate the c-Si PV modules' performance. They coupled the semiconductor modelling via FEM to account for thermal conduction, convection and radiation. Their physical model allows the prediction of the percentage of the sun's intensity that becomes parasitically absorbed within different solar modules under a specific spectrum. Neelagam et al. [120]  developed ML models for predicting the solar radiation for solar systems. Their numerical techniques were based on NN models with different backpropagation algorithms. Vinod et al. [121]  developed a solar PV modelling and simulation tool on top of Matlab/Simulink that considers the performance of PV modules at actual metrological data such as irradiance and weather temperature.
With the advancement of large onshore and offshore wind engineering farms, there is also a large-scale deployment of sensors to monitor the systems' performance. ML can be used for processing large datasets produced by sensors. A few examples of ML future applications for the wind sector include massive (fine-grain) fluid flow simulations around wind farms and geographical terrains that uncover complex flow features, which are difficult to discover through traditional visualisation. In addition, ML can classify physical patterns and identify correlations. Furthermore, ML could also be used as a surrogate model. Surrogate models can be used both for present and future time predictions. In addition, Recurrent Neural Networks (RNN) are suited for dynamic models. Although they are complex in terms of training, they will play a more critical role in the future.
, 2014; Skofronick-Jackson et al. , 2017)  Mission that is actually operating in conjunction with other passive microwave platforms, the performance of satellite precipitation estimates has significantly evolved along the years and numerous satellite-based precipitation products, with different characteristics (i. e. , spatial, temporal, among others), have been developed and improved for distinct application purposes, such as for climatological, hydrological, agricultural and monitoring studies. Numerous observational daily precipitation products, including satellite estimates, are now available in a common 1•x1• grid format through the Frequent Rainfall Observations on GridS (FROGS) database  (Roca et al. , 2019). FROGS database enables easier intercomparisons and applications, based on the selection of a single to multiple products considering their category (i.