dataesr/scientific-paragraphs-categorization huggingface dataset:
, 2015). Two receptors are used to compare with the observed concentrations on each side of the street: OSPM-R1 and OSPM-R2. The height of the receptors is taken as 2 m to correspond roughly to the height at which observations were performed. They are compared to the concentrations simulated at the first two levels of MUNICH-hete, which are representative of the concentrations at 1 m and 3 m. MUNICH and OSPM use the same input data to estimate the street concentrations. The meteorological parameters originate from WRF simulations  (Skamarock et al. , 2008)  and the background concentrations are simulated using the Urban Background Model (UBM)  (Berkowicz, 2000b). Traffic emissions are generated using the OML-Highway model  (Olesen et al. , 2015)  allowing for precise information for each street segment.
0  (Kim et al. , 2022)  and in several sensitivity studies  (Lugon et al. , 2021b; Sarica et al. , 2022). The street network represents a district of Le Perreux-sur-Marne, a suburb 13 km east of Paris, France. It is composed of 577 street segments (see Fig. 8 ). Simulations are performed from 22 March to 15 June 2014 with input data (emissions, meteorological parameters and background concentrations) from the reference simulation SCN0 of  Sarica et al. (2022). Observational data are available for the whole simulation period for NO 2 , NO x , PM 2. 5 , PM 10 and black carbon (BC) at a segment of Boulevard d'Alsace Lorraine (see brown cross on Fig. 8 ). They were performed at a height of about 2 m and they are thus to be compared to the concentrations of the first two levels (i=1 and i=2) of MUNICH-hete.
Hyperspectral imaging (HSI) has become widely used in cultural heritage (CH). This very efficient method for artwork analysis is connected with the generation of large amounts of spectral data. The effective processing of such heavy spectral datasets remains an active research area. Along with the firmly established statistical and multivariate analysis methods, neural networks (NNs) represent a promising alternative in the field of CH. Over the last five years, the application of NNs for pigment identification and classification based on HSI datasets has drastically expanded due to the flexibility of the types of data they can process, and their superior ability to extract structures contained in the raw spectral data. This review provides an exhaustive analysis of the literature related to NNs applied for HSI data in the CH field.
Introduction Hyperspectral imaging (HSI) has become widely used in the field of cultural heritage (CH) for painting analyses supporting the identification of original and degraded paint compounds, and revealing underdrawings  [1] [2] [3] [4]. The non-invasive and non-destructive analysis tool can be applied in situ in either reflectance, transmittance, or fluorescence mode  [5] [6] [7]. It registers a spatial map and collects a spectrum at each pixel position, in the wavelength range varying from infrared to X-ray. The contiguous set of 2D images collected through the energy range of interest produces a 3D data cube containing millions of spectra, which poses an analytical and computational challenge. Thus, the effective processing of heavy spectral datasets remains an active research area with the primary challenge being to efficiently process data and extract rich information from the inherently complex and delicate painting materials  [8, 9].
A range of cutting-edge data processing techniques has particularly been applied in the CH domain to reduce the dimensionality of the dataset, classify and unmix spectral signature to map paint components. Thus, numerous approaches were developed, starting from the conventional multivariate analyses and statistical methods (e. g. , spectral angle mapper (SAM)  [10] [11] [12] [13] , fully constrained least square (FCLS)  [14] [15] [16] , principal component analyses (PCA)  [17] [18] [19] [20] , minimum noise fraction transform (MNF)  [21, 22] , and k-means clustering  [23] [24] [25] ), to the more advanced machine learning algorithms  [42]. This relies on the large flexibility of the NN architectures available (number of neurons, type of layers, and depth), making NNs adaptive for a wide range of tasks and input data  [37].
A systematic and exhaustive search was conducted in the time frame up to December 2022  [27, , which included database searches on Web of Science  [41]  and manual searches of relevant journals, conference proceedings, and PhD theses. The inclusion criteria were set on implementing NN techniques for processing HSI data in the CH field, with the main focus on solving paint component mapping problems. After defining the basic terminology related to NN in Section 2, we describe the two main applications involving the use of NN for the analyses of data acquired on historical paintings (Section 3). In Sections 4 and 5 we present adaptations of the dataset and NNs architecture to the specific constraints of ancient and historical materials. Finally, Section 6 discusses and questions the prominent trends that may lead to the future novel implementation of NN algorithms for historical paintings.
Record count of the scientific publications related to the use of NNs to process hyperspectral datasets from CH paint-based materials over the last twenty years (Web of Science, December 2022  [41] , using keywords "paint", "spectral imaging" and "neural network"). In this circumstance, the rapid rise of NN-based strategies has outlined a variety of data analysis workflows available to the community. However, the flexibility in the architecture designs and the automated learning process (often referred to as "black-box") cause difficulties in application, and very rare studies have documented their choice on the architecture and appropriately described the use of specific features. In this context, the present article proposes a comprehensive overview of the applications of NNs for HSI of historical paintings.
A systematic and exhaustive search was conducted in the time frame up to December 2022  [27, , which included database searches on Web of Science  [41]  and manual searches of relevant journals, conference proceedings, and PhD theses. The inclusion criteria were set on implementing NN techniques for processing HSI data in the CH field, with the main focus on solving paint component mapping problems. After defining the basic terminology related to NN in Section 2, we describe the two main applications involving the use of NN for the analyses of data acquired on historical paintings (Section 3). In Sections 4 and 5 we present adaptations of the dataset and NNs architecture to the specific constraints of ancient and historical materials. Finally, Sensors 2023, 23, 2419 3 of 25 Section 6 discusses and questions the prominent trends that may lead to the future novel implementation of NN algorithms for historical paintings.
There are several learning strategies that have so far been applied in the field of CH: supervised learning, unsupervised learning, and transfer learning  [65, 79, 80]. In the first case, the available dataset consists of data points and their labels, and the NN algorithm learns a function which maps the input to the output by learning from available labelled examples, as observed in most cases  [27, [43] [44] [45] [46] [47] [48] [49] [53] [54] [55] [56] [57] [58] 60, 62, 63]. The second type of learning, unsupervised learning, is used to analyse and cluster unlabelled data [51,  52, 64]. In the case of transfer learning, the model is pre-trained with a large dataset and then finetuned with a separate smaller dataset  [50, 59, 61]. This type of learning is discussed in detail in Section 5.
NN Application to the Study of Artistic Painting The ability of NNs to efficiently learn high-level features from vast data has gradually attracted the interest of CH scholars in assisting with spectral data processing. The preliminary applications of the NN models for CH were implemented on multispectral imaging (MSI) datasets twenty years ago, and focused on the improvement of spectral data resolution and the digital reconstruction of reflectance curves  [43] [44] [45]. Super-resolution spectral reconstruction was first studied in the computer vision field with primary attention on the device-independent digital image reproduction using spectral reflectance as the intermediate space for colour management  [17, 81]. The very first example of using NNs on multispectral data in CH applications was conducted by Ribés and Schmitt in 2003  [46] [47] [48] , to achieve high fidelity and illumination invariant The basic units of a neural network are called "neurons".
• Number of epochs and batch size. Epochs are the number of times the training data appears to the network and batch size determines after how many input sub-sets the network should update its parameters  [72]. In a feed-forward NN, data flows through the network in only one direction, from the input layer to the output layer, without looping back, as in recurrent neural networks (RNNs)  [73, 74]  or generative adversarial networks (GANs)  [75, 76]. There are different types of hidden layers used according to the specific architecture, such as fully connected layers, convolutional layers, or pooling layers  [77, 78]. The various NN architectures with different choices of hidden layer types are detailed in Section 5. After defining the NN architecture and hyperparameters, the datasets are usually divided into a training, validation, and test set.
By comparing the output result with the ground truth, an error signal is computed for each neuron in the output layer, indicating the direction and magnitude to adjust. Second, during the backpropagation, the error signals are transmitted backwards from the output layer to each node in the last hidden layer, and repeatedly propagate backwards layer by layer. After the model has been trained, it is tested on a previously unseen test set to evaluate its performance and the ability to generalise to unseen data. There are several learning strategies that have so far been applied in the field of CH: supervised learning, unsupervised learning, and transfer learning  [65, 79, 80]. In the first case, the available dataset consists of data points and their labels, and the NN algorithm learns a function which maps the input to the output by learning from available labelled examples, as observed in most cases  [27, [43] [44] [45] [46] [47] [48] [49] [53] [54] [55] [56] [57] [58] 60, 62, 63].
NN Application to the Study of Artistic Painting The ability of NNs to efficiently learn high-level features from vast data has gradually attracted the interest of CH scholars in assisting with spectral data processing. The preliminary applications of the NN models for CH were implemented on multispectral imaging (MSI) datasets twenty years ago, and focused on the improvement of spectral data resolution and the digital reconstruction of reflectance curves  [43] [44] [45]. Super-resolution spectral reconstruction was first studied in the computer vision field with primary attention on the device-independent digital image reproduction using spectral reflectance as the intermediate space for colour management  [17, 81]. The very first example of using NNs on multispectral data in CH applications was conducted by Ribés and Schmitt in 2003  [46] [47] [48] , to achieve high fidelity and illumination invariant colour reproduction of fine art paintings.
They developed a non-linear model to learn the spectral reflectance curves from multispectral images. Osorio-Gomez et al. later proposed an alternative method for similar tasks on oil canvas paintings  [44]. Recently, a similar approach by Shi et al. [43]  further optimised faithful print reproduction of oil paintings to model the bidirectional mapping between the spectral reflectance and the 3D printer ink stack layouts. In luminescence MSI development, NNs have also been applied by Chane et al. to learn the radiometrically calibrated emission spectra with resolution in the nanometre scale from 15-band multispectral data  [45]. With the significant development and spread of HSI instruments over the last decades, the fast and high-resolution acquisition of the spectral data on artworks has overcome the limitation of multi-spectral sampling.
In this approach, the spectra are first classified into areas with similar features and then the pigments present are hand identified. The proposed strategy has been later applied on visible and near-infrared (VNIR-RIS) data for bronze corrosion products' identification in a large museum collection  [52]. As opposed to the simplified model assumption of a single layer of pure pigments, historical paintings usually feature a more complex mixture of pigments or even have a multi-layered structure. Therefore, a multi-labelling model is necessary for a robust solution to correctly identify the multiple pigments present within every single pixel. Kleynhans et al. have developed a non-linear model to produce a qualitative multi-label map of the intimate pigment mixtures in a single step  [53]. Their training dataset is based on VNIR-RIS spectra extracted from well characterised 14th century illuminated manuscripts to make accurate predictions of the materials used by the same art school.
In addition to the one-step solution exploiting the hand-labelled database, Grabowski et al. have developed alternative data processing pipelines for a fully automatic pigment classification task  [27]. They have broadened the applicability of the algorithms to paintings with unknown material compositions, as they first obtain the labels through automatic identification with several pigment libraries, and then use NNs to classify the entire painting and create a multi-label map. NNs have also been applied to detect the degradation areas of murals. Mural paintings often suffer from several diseases, such as flaking, deep loss, mud besmirch and sootiness, Sensors 2023, 23, 2419 6 of 25 in various degrees. The documentation of those damaged areas and the identification of their composition is essential for mural conservation and restoration.
Pouyet et al. adopted the same architecture and applied it to datasets acquired in the short-wave infrared (SWIR-RIS) domain  [58]. Another study by Fukumoto et al. applied the encoder-decoder (ENDEC) NN model (fully described in Section 5) that estimates the pigment concentration as an intermediate output and includes a further spectral remix step to reconstruct the predicted spectrum from the estimated concentration  [62]. Apart from paint component mapping, NN implementation has also been explored to solve other spectral unmixing problems related to the layered structure of paintings, such as thickness estimation and paint segmentation. Xu et al. have targeted the multi-layered structure of the painting, andapplied a NN to fully automate the pigment identification process  [59]. Their work is based on XRF data and has simulated XRF spectra with threelayered pigments to train the model.
Dataset Preparation The spectral input sources, data types, pre-processing steps, and labelling methods are crucial for the dataset preparation. Figure  3  summarises the general workflow across the literature and all the alternatives in this preparation process. After collecting and pre-processing the available data, the obtained dataset is split into two or three subsets: training, validation, and/or test set. A common data splitting ratio is 80% for the training and 20% for the validation and/or test  [54, 58, 60, 62]. The alternatives are 64% training, 16% validation, and 20% test according to  [56] , or 70% training and 30% validation as in  [49].
Spectral Inputs As introduced in Section 3, the early studies were mainly conducted on MSI data for spectral reflectance curves reconstruction. These data were mostly acquired or simulated in reflectance mode, with one exception utilising a 15-band luminescence acquisition in the 450-740 nm range  [45]. The reflectance data mainly fall into the VIS domain, either collected with seven narrow-band interferential filters in the range of 480-650 nm  [44] , or 31 bands with high-dynamic range (HDR) multispectral capture within the spectral range of 420-720 nm  [43]. Multispectral data extended to the SWIR range was also inspected, as presented by Striova et al. in 2018, who acquired 32 narrow-band images with spectral coverage from 390 up to 2500 nm  [60]. Furthermore, Ribés et al. proposed using artificial data that simulate seven-channel camera responses over the 400-760 nm range  [46, 48].
Artificial Data In the shortage of adequately labelled datasets, physical models can be applied to generate a library of synthetic spectra with a sufficient amount of training data. Two approaches have been used in accordance with different spectral inputs: the fundamental parameters (FP) method for XRF spectra and the Kubelka-Munk (KM) theory for RIS data. For these data, the ground truth is known, favouring not only the multi-label classification, but also quantitative estimation or even unmixing in multi-layer structures. The FP method is based on Sherman's equations that describe the theoretical relationship between the measured XRF signal intensity of the element and its concentration in the sample  [82]. With defined pigments and concentrations, simulated spectra responses are generated.
Jones et al. applied this method to create a training dataset containing 3000 spectra of 15 pigment classes  [50] , modelled as a simplified case that only accounted for the primary fluorescence of single-layered samples. Xu et al. [59]  have employed a more complex model that simulates XRF spectra for three-layer structures in mixtures of 12 different pigment classes, resulting in 16,224 spectra in total. Both studies have adopted a transfer learning strategy that further fine-tunes the NN model with a smaller quantity of experimental data, after the initial training with the large synthetic dataset. For the reflectance data, Kubelka-Munk (KM) theory is widely used to estimate the interaction between the incident light and paint layers  [83, 84]. This physical model measures a sequence of absorption and scattering coefficients to predict the reflectance spectra from the composition of pigment mixtures.
The KM theory can be applied to decompose the measured spectra and thus estimate the pigment concentrations via spectral unmixing. Moreover, it can be used for mixing models that generate many patterns of synthetic spectral data, with given pigment types and concentrations. Rohani et al. have applied a nonlinear KM mixing function to generate simulated spectra of pigment mixtures  [56, 57]. All possible two/three mixtures out of 11 pure pigments  [57]  and 12 pigments  [56]  were modelled and 500 random combinations of coefficients/concentration values were selected for each mixture, obtaining 110,000 spectra and 143,000 spectra in total, respectively. Fukumoto et al. have used a similar approach to build the synthetic dataset based on 19 reference pigments  [62]. Two datasets were prepared: the first one that served as a trial to validate the method contained 1771 spectra of only one combination and the second one contained 1,445,136 spectra which included all combinations of 4 colours out of 19, including white with 5% mixing steps.
Furthermore, Shitomi et al. synthesized the spectral responses of layered pigments with a given layer thickness and mixing ratio  [64]. Three pigments of primary colours are used for the simulation with thickness varying from 0 to 3. 2 µm, generating in total 35,700 spectra. Chen et al. used an alternative method to create an artificial spectral database  [49]. They generated augmented pure pigment samples by adding random noise and amplitude offset to the reference pigment database, and synthesised simulated colour mixtures using an ideal subtractive mixing function. All possible mixtures of 3 pigments out of a selected set of 16 pigments were generated with increments of 10%, yielding a total of 21,240 spectra.
Most generally, mock-up samples are prepared to imitate the historical painting materiality (binder, pigment(s), preparation layer, and support). The pigments selected to create the mock-ups are commonly based on analysis of the historical artworks and the range of pigment fractions, binder ratios, and layer thicknesses are consistent with those in the historical objects  [27, 58, 59, 64]. The spectral data acquired on the mock-ups covered both pure pigment areas and two to three pigment mixtures, with variances in paint thickness in some cases. The input datasets have an average size of 80 k spectra. In some cases, the mock-ups were used to test the performance of the NNs previously trained on artificial datasets. Rohani et al. created mock-up paintings with selected combinations of mixtures and pure pigment examples to test their model  [56, 57].
Historical Data Historical data are those acquired directly on historical objects. Those data are either included in the training process by forming the input dataset or serving as a test case to verify the performance of the proposed model. The types of art objects and the sizes of the dataset used present a large variability. Within the historical set, both labelled and unlabelled data are used. Kogou et al. used an unlabelled dataset composed of 41,327 XRF spectra acquired on a Peruvian watercolour painting (c. 1860) by Francisco Pancho Fierro  [51]. With the aid of an unsupervised NN model, the large number of XRF spectra was reduced to 13 distinct clusters of unlabelled material compositions. Liggins et al. applied the same model to two excavated bronze fragments, classifying the SWIR and VNIR images with hundreds of thousand spectra (exact size unreported) into three corrosion areas  [52].
They also built a reference spectra database composed of 17 samples of powdered and fragmented ancient Chinese bronze extensively characterised by other analytical instruments. The identification was performed by manually comparing the obtained averaged spectra in each cluster with the reference. On the contrary, supervised networks require labelled input data, which is quite challenging for historical objects that are usually unknown and heterogeneous materials. The main labelling method proposed in the literature relies on the visual inspection of the spectral features, and manual annotation to mark areas with a certain presence of pigments or degradation features. To confirm this manual identification, complementary analyses can be used, such as RIS, XRF, fibre optics reflectance spectroscopy (FORS), X-ray diffraction (XRD), Raman and Fourier Transform InfraRed spectroscopies  [27, 51, 52, 54, 55].
However, this labelling method is very time-consuming. Thus, usually only small inherent areas with limited pixels are selected, resulting in a much smaller dataset size. Moreover, the obtained labels are not suitable for the quantitative assessment of concentrations. In some cases, the historical objects in question were extensively studied in previous reports and have been well characterised for all chemical components, available as the ground truth maps. Kleynhans et al. used selected areas on four paintings from the illuminated manuscript Laudario of Sant'Agnese (c. 1340) to form their training dataset  [53]. In total, 25 classes of paints were identified, and 16,683 individual spectra were collected across all four paintings. Two out of the four paintings were also used to test the performance of the model.