dataesr/scientific-paragraphs-categorization huggingface dataset:
Pouyet et al. also used selected areas of an historical object, a Tibetan thangka dated from the beginning of the 19th century, as part of the training data  [58]. They have built their input sets from multiple sources, including single-pigment pellets, pigment mixture mock-ups, and the historical tangka, altogether presenting 12,000 spectra. Striova et al. applied the network based on an oil painting Madonna of the Rabbit by É. Manet (c. 1856) to improve the visibility of pentimenti and underdrawing  [60]. 50k randomly selected pixels from the analytical data acquired formed their input dataset. The above-mentioned data types and labelling methods are mix-used in some studies to build a more reliable training dataset. Both historical data and modern replica data were used to build the input datasets in  [52, 58].
A large simulated dataset and a smaller mock-up dataset were used in different learning stages in the transfer learning approaches  [50, 59]. However, historical data are more frequently used for the test of the developed network with the ultimate goal to study the material composition and distribution of historical artworks as illustrated for: one illuminated folio from the 15th century Book of Hours in  [56] ; two impressionistic paintings, the Poèmes Barbares by Paul Gauguin (1896) and The Bathers by Paul Cezanne (1899-1904) in  [59] ; a set of 11 paintings by the late Portuguese artist Amadeo de Souza-Cardoso in  [49] ; and an early Renaissance painting, Saint Michael Triumphs over the Devil by Bartolomé Bermejo  [50]. In  [63] , a pre-trained model was tested on damaged murals of a Buddhist temple (c.
For RIS data, a spectrally moving average filter that reduced the wavelength channels by a factor of 4 was used in  [57] , and a Savitzky-Golay filter was applied in  [49, 56]  to reduce the high-frequency noise. For the XRF dataset, spatial median filtering with a 3 × 3 pixels kernel and spectral binning of 5 were employed  [51]. On the other hand, noise was introduced to the artificial XRF spectra to increase the robustness of the dataset  [59]. Data dimensionality reduction techniques, including PCA and MNF, are applied in some cases to centralise the information in fewer bands  [44, 54, 55, 63 ]. An inverse PCA transformation was applied in  [54]  after processing the first component by high-pass filtering. A pipeline of statistical algorithms, including t-SNE (t-stochastic distributed neighbourhood embedding), DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering, HySime (hyperspectral signal subspace identification by minimum error), SISAL (simplex identification via split augmented Lagrangian), and SAM, was built in  [27]  to fully automate the pigmentidentification process and the results were fed into the NN.
Self-Organizing MAP (SOM) A self-organizing map (SOM) is a type of unsupervised learning algorithm, i. e. , it does not require a labelled dataset. The algorithm takes the pixel-level spectra as input and maps them onto a 2D layer, the SOM  [85]. SOMs thus contain only a single layer of neurons, in which every neuron is associated with a weight vector, with the same length as that of the input vectors. The training takes on the competitive learning strategy, where each neuron competes with its neighbours to be the "winner" for a given input. The winning neuron and its neighbours adjust their weights to better match the input. The SOM NN has a single input parameter-the number of neurons, which should be larger than the number of clusters. This type of clustering method naturally allows for a visualisation of the high-dimensional spectral data in 2D space in the form of single-cluster maps.
Shallow MLP As an early application, a shallow MLP was applied in  [44]  to predict reflectance spectral curves. The input datasets are reflectance spectra reconstructed using linear techniques, such as pseudoinverse, PCA, and cubic-spline interpolation. The neural network was composed of an input layer of size 3 × N (where N is the number of channels), a hidden layer of 85 neurons per channel, and an output layer of a single neuron per channel. Linear activation functions were used in the hidden and output layers. The root mean squared error (RMSE) was the metric employed. The neural network parameters were optimised with the aid of the Levenberg-Marquardt algorithm  [87, 88]. The training was set to stop when RMSE fell below a certain threshold, which was fixed to a small arbitrary value.
The performance of the model was measured with a categorical cross-entropy loss function, while the stochastic gradient descent optimizer was applied to minimise the loss function. For the learning process, the model was trained with batch sizes of 50 and epochs of 30, and evaluated on a validation set of 10% of the training data. The training started with a learning rate of 0. 01, which was decreased if after four epochs (cycle through full training dataset) the validation loss did not decrease. up to 1. The final class is assigned by defining a threshold; at 0. 99, a number of pixels remain unclassified. By decreasing the threshold from 0. 99 to 0. 85, part of the unclassified areas was assigned a class with rising in false positive identifications. The performance of the model was measured with a categorical cross-entropy loss function, while the stochastic gradient descent optimizer was applied to minimise the loss function.
For the learning process, the model was trained with batch sizes of 50 and epochs of 30, and evaluated on a validation set of 10% of the training data. The training started with a learning rate of 0. 01, which was decreased if after four epochs (cycle through full training dataset) the validation loss did not decrease. [53]. The input layer (shown in blue) takes as input the VNIR spectrum, followed by a series of hidden layers (shown in green) that perform feature extraction using convolutional kernels (shown in orange). The output layer (shown in yellow) produces the final classification results.
CNN with Transfer Learning Transfer learning (TL) is a learning strategy in which a model trained on one set of data is reused as the starting point to train a new model on data with slightly different properties. Both models adopt the same architecture but possess different parameters. The low-level features learned by the pre-trained model are maintained when training a new model by freezing all parameters on certain layers, commonly the earliest layers which tend to learn more general structures from the input set. The model, thus, only adjusts a smaller portion of parameters to adapt to the new data, making it possible to achieve high accuracy with much less training data and computational resources. TL is especially useful when the available data is limited while the NN has a rather complex architecture, as the model can leverage the knowledge gained from the larger dataset to the new task.
In CH, two works have adopted the TL strategy to exploit large artificial datasets  [50, 59]. In  [50] , a CNN model was applied to classify XRF spectra into pigment classes (15 classes were used in this study). The network was composed of three convolutional layers with max-pooling layers in between, and the output layer predicts the label for the corresponding pigment. The ReLU activation function was used for each layer except the final layer, which used the softmax activation function. The model is pre-trained on synthetic XRF data, and then fine-tuned with spectra acquired on real pigment samples, by freezing the weights in all layers in the network except the final layer. The performance of the model was evaluated before and after the fine-tuning, and the model with transfer learning showed higher accuracy.
The loss was calculated to optimise the performance of the model by averaging the binary cross entropy of each predicted class. The model was pre-trained with the artificial dataset using randomly initialised weights. It was further fine-tuned with data acquired on mock-ups, in two training steps: first, the pre-trained weights were fixed in all convolutional layers, whereas only the fully connected layers were fine-tuned. Next, all layers were trainable and were further fine-tuned with the mock-up dataset.
Due to insufficient training data, the authors resorted to transfer learning by pre-training the BDCN model on a publicly available dataset of natural scenes. The network parameters were then fine-tuned on a target cultural relics dataset. In  [49] , Chen et al. achieved pure pigment identification with a multi-class classification problem using a three-branch deep-learning (DL) model. Unlike the 1D-CNN where a single spectrum is used as input, in this case, the first two branches of the NN take as input a small sub-cube around a central pixel, thus allowing for a combined spectral and spatial investigation. The first branch of the neural network consists of five sets of 3D convolutional filters with ReLU activation functions. It takes a 9 × 9 sub-cube of the hyperspectral cube around a central pixel as input.
The architecture of the second branch is identical to that of the first one using the derivative of the same sub-cube as input. The third branch of the model takes the reflectance spectrum of the central pixel as input and computes the spectral correlation map between the pixel and a reference pigment database. This result is fed to a shallow fully connected feed-forward NN, which analyses the error between the reflectance spectrum of the pixel and the spectral signatures in the reference pigment database. The outputs of the three branches are flattened, concatenated in a single vector, and fed into a fully connected feed-forward deep NN of five hidden layers with ReLU activation functions. One of these layers is a dropout layer, intended to minimise overfitting. The output layer consists of 17 neurons-16 for the pure pigments, and 1 neuron which predicts whether the reflectance spectrum belongs to a mixture or not.
Deep Belief Network (DBN) Deep belief networks (DBNs) are a type of NN that consist of multiple layers. In a DBN, the lower layers consist of the so-called restricted Boltzmann machines  [99, 100] , typically used for unsupervised learning and designed to learn the underlying probability distribution of a dataset, and the top layer is usually fully connected. The training process of DBN is different from MLP and CNN, which involves pre-training the lower layers using unsupervised learning and then fine-tuning the top layers using supervised learning through backpropagation  [101]. DBN can be used to solve unsupervised learning tasks to reduce the dimensionality of features, as well as for supervised learning tasks to build classification or regression models. In  [54] , a DBN-initialised neural network was used to predict the degree of flaking.
Model Performance Evaluation The testing of the generated output on an unseen dataset is accompanied by different evaluation criteria. For the test sets, the ground truth is already known by the methods described in Section 4. 2 and the quantitative evaluation of the prediction accuracy is performed  [27, [53] [54] [55] [56] 59, 61, 62]. As summarised in Table  1 , it is measured by calculating sensitivity, overall accuracy (OA), average accuracy (AA), standard deviation (STD), R-squared coefficient of determination, root mean square error (RMSE), or using the individual per-class accuracy. For those tested on a new set of historical paintings, the performance is evaluated more qualitatively. The output results of the networks are commonly cross-validated with other analytical instruments, involving the manual check of spectral features of selected points using XRF, FORS, FTIR or micro-Raman, and optical microscopy  [49] [50] [51] 56, 58, 60].
Furthermore, the training datasets also present a large complexity, varying in spectral ranges, input data types, and preparation methods, from artificial data to historical data, and from modern model paintings to historical artworks of different origins. Table  2. A summary of the neural network models analysed in this review. The distinct characteristics of the NN models are presented, such as the spectral input utilised, the type and size of the dataset employed, the assigned task, and their applications to historical objects. Over the last few years, various algorithms were developed with the aim of capturing complex features from HSI data for an in-depth analysis of the rich spectral features  [102]. However, as observed in the remote sensing field, the scale of the HSI training data is limited due to the cost, complexity, and labelling constraints in their development  [103].
Architecture Supervised NN models represent a major part of the models developed for classification or unmixing purposes (85. 7%). However, due to the costly and labour-intense labelling of HSI data, the supervised models suffer from very limited labelled input datasets that constrain the effective learning of the massive spectral parameters required for accurate classification or regression results  [104, 105]. Extensive use of those models will soon represent a bottleneck that will significantly inhibit their practical application for wider HSI data analysis. Building new HSI benchmark datasets to meet the training needs of the NN model is expensive and time-consuming  [106]. Several HSI datasets have been assembled over the last few years for diverse applications, and multiple open-source databases of diffuse reflectance reference spectra flourished simultaneously  [107] [108] [109].
Similarly, several works address this issue by expanding the dataset through an augmented dataset approach  [49] , and a mixed use of various dataset types (artificial, modern, historical)  [50, 52, 58, 59]. In CH, a general trend has been observed with NN algorithms becoming more powerful and widely applied, leading to increasingly complex architectures to tackle more difficult tasks. This trend is driven by the increasing complexity of data and the improvement in computing power. However, it is important to balance the complexity of the network with the size of the dataset in order to achieve the best performance and efficiency. It is generally recommended to start with a simple network configuration and gradually increase complexity to find the optimal architecture for a given dataset, and to reach the optimal model performance in terms of accuracy and computing power.
For large and complex datasets, a more advanced network with more hidden layers and complex structures is preferred, while a simpler network is sufficient for smaller datasets and simpler tasks. This practice is observed in the CH literature where relatively simpler models (SOM and MLP) have an averaged training set size of around 80 k spectra, while for more complex architectures (CNN, ENDEC and DBN), the average size is 215 k spectra (Table  2 ). As an example,  [50, 59]  adopted a CNN architecture and transfer learning strategy, respectively, while using the same input range and data type. As the model used in  [59]  is much more complex than in  [50] , the size of the training data is also significantly increased.
Perspectives In conclusion, spectral imaging instruments have become a common tool in CH studies accompanied with a natural extension of dataset and data range. The use of neural networks (NNs) is likely to become increasingly important for assisting the interpretation and information extraction of hyperspectral dataset. As such, the approach will soon grow and provide more examples, available to the community. In this context potential research axes arise from this review. Many researchers have indeed recognized the limitation of current practices in terms of poor generalisation ability and applicability to other datasets, highlighting the need for new, larger and more complex reference hyperspectral datasets. One way to address this issue is to continue to increase the dataset by including a wider range of data from larger pigment databases and more variable painting structures.
In order to facilitate this process and increase model transferability, a move towards open-source algorithms and publicly available datasets is indispensable. Similarly, the combination of NN models with complex physical models (four-flux KM approximation  [110, 111] , or the radiation transfer equation  [111, 112] ) needs to be explored, providing more diverse and accurate artificial data to feed NN models. In parallel to data augmentation and synthetic data generation, thatstarted to be addressed in the field, NN models that can efficiently utilise a limited number of labelled samples have to be tested. As such, transfer learning or weakly supervised methods should be further explored to mitigate the demand for training samples  [113].
[63]t type I, II, and III refers to artificial data, modern data, and historical data described in Section 4. 2. **The three different input sets serve as separate training cases in ref. [64]. ***Ref. [63]used a pre-trained model developed by a previous study; thus, the application does not include the training process. **** Ref. [61] used a publicly available dataset containing 500 images to pre-train their model, which is the only case and does not belong to our defined categories.
[63]t type I, II, and III refers to artificial data, modern data, and historical data described in Section 4. 2. **The three different input sets serve as separate training cases in ref. [64]. ***Ref. [63]used a pre-trained model developed by a previous study; thus, the application does not include the training process. **** Ref. [61] used a publicly available dataset containing 500 images to pre-train their model, which is the only case and does not belong to our defined categories. Spectral Input Input Type * Dataset Size (Spectra) Task Application Ref. SOM XRF RIS (SWIR) III II, III 41,327 NA Clustering Clustering Peruvian watercolour painting (c. 1860) Excavated bronze fragments (B. C. ) [51] [52] RIS (VNIR) III NA Classification Mural paintings (17th-20th century) [55] RIS (VNIR) I 143,000 Classification II, illuminated folio (15th century) [56] MLP RIS (VNIR) I 110,000 Unmixing II [57] RIS (SWIR) II, III 12,000 Unmixing Tibetan thangka (19th century) [58] RIS (SWIR) II NA Classification II [27] XRF I II 16,224 1320 Unmixing II, impressionistic paintings (c.
Electrode microstructures for LBM simulations The first step for performing LBM simulations is to construct the porous electrodes. 3D micro X-ray tomography of an uncalendered and of a calendered electrode were used. This is because the calendering of the electrodes has a significant impact on the electrochemical performance of the LIB cells. For more details of this topic please see our previous publication  [19]. The geometries are imported into the FIJI software for slicing and segmentation, then into the Geodict software  [33]  to extract pores using the watershed algorithm. The complete setup used is reported in detail in our previous publication  [14]. We opted for a volume of 50 × 50 × 50μm 3 for the different electrode microstructures used in this study, to have a good compromise between representativeness of the electrode geometry and reasonable LBM simulation time.
Average LBM simulation time is 1 h per condition, on an Intel® Core™ i7-12700 with a 32 GB of RAM. The porosity and the tortuosity factor of the uncalendered and the calendered electrodes are reported in Table  1. The porosity was calculated by using the Geodict software [33] and the tortuosity factor by using the TauFactor software  [34]. The major difficulty of the electrolyte infiltration process is how to choose a set of design parameters that will produce an optimal wetting, as measured by the saturation, or some function of the saturation. The parameter design space has too many dimensions for exhaustive exploration, given the computational cost of a LBM simulation and the convergence difficulties for certain parameter combinations. This is the motivation for seeking a surrogate model, as described later.
Classification and clustering models In this section, two different models were evaluated in order to categorize the data considering the values of their saturation, time of saturation and the completion of the saturation. First, an unsupervised clustering model was fitted, providing an accurate partition of the data set without any labels. The k-means algorithm used here, found patterns, and assigned successfully the observations to the corresponding category by taking into account both the completion and the speed of the saturation. Afterwards, we used a support vector machine (SVM) technique, a supervised classification machine leaning model that takes target variables inspired by the k-means outcomes and uncovers the relationships between the electrolyte variables and the targetsthat are the wettability descriptors classes.
Unsupervised learning (k-means) The unsupervised, k-means analysis aims to identify k groups in the data set by optimizing the distances between each point and the data with similarities and maximizing cluster separations using the metrics defined below. The Within-cluster Sum of Squares is defined as, WSS = ∑ N i=1 ∑ x∈Ci d(x, x Ci ) 2 , ( 10 ) where C i refers to the i th cluster, N the size of the cluster and x Ci is the cluster centroid. The compactness of the clustering is measured using the formula I = BSS/TSS. ( 11 ) Here, BSS is the Between-cluster Sum of Squares, defined as BSS = ∑ N i=1 ⃒ ⃒ C i ⃒ ⃒ d(x Ci , x) 2 , ( 12 ) where x is the cluster mean and TSS stands for Total Sum of Squares. A good partition is supposed to minimize the distances between the points belonging to the same cluster and maximize the separation between the clusters.
Acknowledgments A. A. F. , A. E. M. and A. S. acknowledge the  European Union 's  Horizon 2020 research and innovation program  for the funding support through the  European Research Council  (grant agreement  772873 , " ARTISTIC " project). A. A. F. and O. A. acknowledge partial funding from the  Région Hauts de France  and from the  European Union 's  Horizon 2020 research and innovation program  under grant agreement  957189  ( BIG-MAP ). A. A. F. acknowledges  Institut Universitaire de France  for the support.
Secondly, the efficient pyramid squeeze attention (EPSA) net is integrated into EvoNorm-S0 and the spatial pyramid pool (SPP) layer to obtain the hand joint position information. Lastly, the D-S theory is used to fuse two classifiers, support vector machine (SVM) and random forest (RF), to produce a mixed classifier named S-R. Eventually, the synergetic effects of our algorithm are shown by experiments on self-created datasets with a high average recognition accuracy of 89. 6%.
Gesture Recognition Methods Based on Machine Learning Dominio et al. [30]  proposed a gesture recognition scheme based on depth data collected by depth cameras. Four different sets of feature descriptors were extracted from the data, considering the distance from fingertips to the palm and palm plane, the curvature of the hand contour, and the geometry of the palm area. Finally, a multi-class SVM classifier was used to recognize the performed gestures, which obtained a very high accuracy on standard datasets, and specialization ones collected for experimental evaluation. Their implementation without optimization was able to achieve about 10 fps. A light-intensityinvariant technique for gesture recognition  [31] , taking advantage of the principle that one skin tone looks different under changing light intensity but different skin tones may look the same under changing light intensity, used directional histograms to identify unique features to recognize the features of gestures.