# Machine Learning Models for Artist Classification of Cultural Heritage Sketches
## 4 Discussion  
Artist attribution suggestions for artwork sketches have proven to be a challenging problem even for modern computer vision machine learning models. To address such issues we have developed a flow for previously authenticated sketches that first clean up the images from artifacts and image degradation allowing for models to focus on the artist's drawings and not on degradations. Then, we started implementing our analysis system that contains three machine learning models with different architectures. The first architecture, using reinforcement learning with deep convolution networks, we focused on the award winning VGG topology. This approach applied agent learning or QLearning for the classification issue allowing the deep network to accommodate to our problem at hand. The results were quite promising allowing for acceptable accuracy values for a suggestion system. As the system evolved under the agent's pressure there was a specific issue related to model interpretability and this led us towards using the Captum library for interpretable machine learning. The next Faster R-CNN architecture considered for our work is the pre-trained Residual Network (ResNET) model with COCO annotations. Such models provided the best results for classification problems but required extra training steps. The first step is to create the annotations for each image. Our initial approach was to annotate using custom algorithms using the PascalVOC annotation style but later we converted it to the COCO annotation style when we chose the best model from the TensorFlow model garden. The results, in term of accuracy, provided the best values for our problem and this type of architecture is one of the best in terms of classification problems. The third architecture is common for recommendation systems so we converted the classification problem to a recommendation one and employed a vector embeddings database. We focused on this type of application as the new developments in terms of knowledge representation within large language models employ this technique. This methodology does not provide us with an accuracy metric but rather use the semantic representation in terms of vectors from which a distance provides the most similar elements from the database. In order to be able to measure its performance against the classic models, we needed to apply a ratio-based accuracy metric. For the vector embeddings encoder we also employed the standard VGG model (similar to the one from the reinforcement learning model). The results provided a very good accuracy level (much higher than the reinforcement learning and the Faster R-CNN model). See Table 8 for measured accuracy levels. Vector embeddings and residual deep convolution networks proved to be two powerful techniques for our classification problem. ResNet, as a traditional deep learning model, is more suited in accuracy and feature learning but can be computationally expensive and requires large labeled datasets. While using such models no validation accuracy is available as the training is not a standard one but rather a feature extraction, thus the n/a from the Table 8. Vector embeddings, on the other hand, offer efficiency, flexibility, and semantic understanding, making them more appropriate for zero-shot learning. While their effectiveness depends on factors like embedding quality and computational resources, combining both approaches can often yield superior results. \begin{table}
\begin{tabular}{l l l l} \hline
**Nr. Crt.** & **Model** & **Architecture** & **Val. Acc. [\%]** & **Test Acc. [\%]** \\ \hline
1 & Model 1 & Reinforcement Learning & \(58.8\pm 0.017\) & \(61.86\pm 8.03\) \\
2 & Model 2 & Faster R-CNN (RetinaNET/ResNET50) & \(84.48\pm 6.82\) & \(95.84\pm 7.76\) \\
3 & Model 3 & Vector Embeddings & n/a & \(96.21\pm 8.18\) \\ \hline \end{tabular}
\end{table} 
Table 8: Accuracy evaluation metrics (from the test data). Recent developments include Transformer architecture [38] that can also be applied for object detection purposes.