# Machine Learning Models for Artist Classification of Cultural Heritage Sketches
## 2 Material and Methods
### Image Preprocessing  
While our dataset consisted of only sketches, several processing steps were implemented for optimal results (by filtering out degradation) and to enhance the overall image contrast. The data processing framework was implemented using a tabular format with standard Python processing tools [18; 19]. Once a pointer reference file (csv) was created for each file, we started the processing flow with a thread-intensive workload. This workload was based on the popular OpenCV [20] library, which first loaded the file and normalized the arrays, leading to standard Numpy [21], which could be easily ingested by our model. While most available algorithms limit themselves to these steps before the model training phase, in our case, the degradation required special attention for the discrimination of the background from the actual image. To overcome this issue, in our case--where the sketches had a limited color palette--we identified this as being an application of classical principal component analysis (PCA) [22] for two classes (content and background) and 10 iterations. As our attention was driven toward the edges, we converted the images into gray-scale and dilated them (by applying a maximum filter to enhance the filtered edges) to overcome the sharpening specific to the PCA step. While some architectures, like the Faster R-CNN one, usually do not require such preprocessing, in our case, the model can be misled by the straight edges within the existing image (e.g., for Kentridge) resulting in wrong identification. Beside these situations, by ensuring a clean input image, all models interpret the data thus making the model comparison unbiased. The next step, required by Faster R-CNN, was the performance of image segmentation, which involved a technique that extracted pen/pencil strokes from the actual images in order to correctly draw the model attention to them. To obtain optimal results, we employed the Canny Edge Detection algorithm [23] due to its good and reliable edge and contour detection (areas from within the normalized image where the local gradient reached its maximum value). For specific areas, the Canny Edge Detection algorithm employed non-maximum suppression to thin out the edges and compute a gradient strength with a double threshold for edge classification (weak and strong). At the same time, during the last step, it used a hysteresis threshold, thus highlighting only the strongest edges. The resulting edges were then filtered out using a minimal area of 50 pixels (px), thus further excluding potential noise. This technique allowed the model to focus mainly on significant strokes (with actual artist-specific features). These extracted features were exported using a Pascal VOC [24] file format. This type of annotated file allowed for the generation of testing, training, and validation datasets using the common TensorFlow Data (tfrecords) format. We used this format due to its \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Nr. Crt** & **Usage** & **Image Count** & **Disk Size [Mb]** & **Percentage [\%]** \\ \hline
1 & Training & 772 & 271.53 & 72 \\
2 & Testing & 196 & 71.31 & 18 \\
3 & Validation & 96 & 34.09 & 10 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The image count in the original dataset. versatility and non-sequential access, allowing for faster training and evaluation workflows. These files are extensively used for standard regional convolutional neural network models. Upon searching for the best way to accommodate the dataset with our hardware capabilities, the best approach for our dataset was to enforce an image size with a shape of (512, 512, 3), meaning that all images were 512 \(\times\) 512 pixels and had three color layers (blue, green, and red). While images with better resolutions often had specificities in terms of their size and orientation, many of them had dimensions that exceeded this limitation. Our approach was to split the images (without losing pixels) into multiple images.