# Non-Invasive Painting Pigment Classification Through Supervised Machine Learning
## 5 Results  
To provide a reliable and comparative evaluation of all classifiers, a stratified five-fold cross-validation protocol was adopted. This procedure was repeated twice (i.e., 2 \(\times\) 5-fold), with random shuffling of the dataset prior to each repetition. Within each fold, 80% of the data were used for training and 20% for validation. The same data partitions were consistently applied across all evaluated models to ensure comparability. As a result, each sample appeared once in a validation fold and four times in a training set during each repetition. Performance metrics, including accuracy, precision, recall, and F1 score, were computed for each fold. The final reported values represent the mean scores across all ten validation subsets, accompanied by standard deviation values to reflect result variability. The classification results from this cross-validation procedure are presented in Table 2. Among all tested models, the random forest classifier achieved the highest overall performance, with an average accuracy of 99.30 \(\pm\) 0.15%, along with consistently high values of precision, recall, and F1 score, all exceeding 99.29%. The K-Nearest Neighbors (KNN) algorithm demonstrated similarly strong performance, with an accuracy of 99.26 \(\pm\) 0.12%. The Gaussian naive Bayes classifier also performed well, reaching 96.60 \(\pm\) 0.15%, which suggests a notable generalization ability despite the assumption of feature independence. The support vector machine (SVM) achieved a lower accuracy of 90.00 \(\pm\) 0.43% but still provided competitive results. Other classifiers yielded more variable outcomes. The extra trees classifier achieved relatively good performance ((94.25 \(\pm\) 0.37%)), while logistic regression resulted in \(69.03\pm 0.48\%\) and Adaboost performed substantially worse (54.26 \(\pm\) 2.88%). Several algorithms, including the decision tree algorithm (5.83 \(\pm\) 0.02%), stochastic gradient de  
Figure 5: False-color representations of the painting shown in Figure 4: (**a**) ultraviolet false color (UVFC), (**b**) infrared false color (IRFC). Images were acquired using different optical filters, while the illumination setup remained constant. scent (SGD) (23.82 \(\pm\) 3.95%), and bagging (44.05 \(\pm\) 2.96%), exhibited low accuracy and high variability, indicating limited suitability for the classification task addressed in this study. In addition to evaluating the classifiers themselves, two labeling strategies were considered. In the unified-class strategy, all five preparation variants of a given pigment--transparent; opaque; and mixtures with lead, zinc, or titanium white--were grouped into a single class label. In contrast, the subclass-based strategy assigned a distinct class to each variant, enabling more detailed categorization. While both approaches produced the same top-1 accuracy of 25.0%, the subclass-based model consistently outperformed the unified strategy in top-3 and top-10 accuracy. This suggests that more granular class definitions improve the model's ability to exclude implausible candidates and rank relevant pigments more effectively. To further assess the model's applicability in conservation-oriented contexts, a real-world validation study was conducted using 41 annotated pigment samples selected from four modern reference paintings. These artworks were created to emulate the techniques of 14 th- and 15 th-century Italian panel painting, using egg tempera as a binder and well-documented pigments obtained from Kremer Pigments GmbH & Co. KG. All pigment compositions were known in advance based on documentation, and no additional instrumental analyses were performed prior to annotation. Ground-truth labels for selected regions of interest (ROIs) were assigned by domain experts based on this prior knowledge. The validation focused on visually homogeneous regions, where the assumption of a single dominant pigment is more likely to hold. Segmentation was performed using 32 \(\times\) 32 pixel patches, consistent with the model's training configuration.