## 2 Background and Related Work
### Transfer Learning
Transfer learning allows the VGG16 model, which is trained on large datasets such as ImageNet, to be efficiently fine-tuned with new data, reducing the training time and computational resources (Figure 1) [30]. VGG16 has a depth of 16 layers consisting of 13 convolutional layers and three fully connected layers with 3 \(\times\) 3 filters and max pooling for dimensionality reduction and robustness. The network ends with fully connected layers and a SoftMax layer for classification. Despite its effectiveness, VGG16's 138 million parameters pose a computational challenge [31]. Through the technique of transfer learning, VGG16 capitalizes on the foundational logic already embedded within pre-existing models, expediting the training process and requiring fewer data for effective retraining [17]. In this approach, the architecture of a pre-trained model is utilized to generate a new dataset from input images. Specifically, the convolutional and pooling layers of the model are imported, excluding the \"top portion\" containing the fully connected layer. By passing images through these layers of the VGG16 model, a feature stack representing detected visual features is obtained. Subsequently, this three-dimensional feature stack is converted into a NumPy array, facilitating its seamless integration with various modeling techniques (Figure 2) [32; 33]. Figure 1: The VGG16 model. Figure 2: After removing the “top” portion, the model convolves the pre-trained output layers, resulting in a 3D stack of feature maps. The pre-trained layers of VGG16 were utilized for feature extraction, in which the image dataset underwent processing through the convolutional layers and weights to generate transformed visual features. No direct training was performed on these pre-trained layers. While the frozen pre-trained layers maintained their usual convolution of visual features, the non-frozen (or 'trainable') pre-trained layers were adapted to our custom dataset, updating in accordance with predictions from the fully connected layer (Figure 3) [30]. In a pre-trained model like VGG16, we only train additional classifier layers, which is advantageous for several reasons as follows. Transfer learning efficiency: It uses pre-trained weights to extract general features and increases transfer learning efficiency. Overfitting prevention: Freezing pre-trained layers provides better generalizations and reduces the risk of overfitting on smaller datasets. Training speed: It reduces the overall training time and computational costs by focusing computational resources on new layers. Therefore, instead of retraining the entire model, we only tune additional classifier layers, as this is more time-efficient and requires fewer computational resources. The VGG16 model was selected due to its proven track record in image classification tasks and its deep architecture which enables it to capture complex features in images. Using VGG16 pre-trained weights on the ImageNet dataset provides a solid foundation that allows transfer learning to adapt to specific tasks with a smaller dataset. This significantly reduces the training time and required computational resources while also increasing the model's ability to generalize from a relatively small dataset by leveraging the features learned from a large and diverse set of images. Resnet was also used in our preliminary work. Resnet is a pre-trained transfer learning model like VGG16. VGG16 was preferred due to the low performance of Resnet.