# Deep Learning Methods for Bug Bite Classification: An End-to-End System
## 2 Materials and Methods
### CNN Variants Used for Bug Bite Classification
#### 2.6.1 Convolutional Neural Network Architectures
Regarding the VGG16 architecture, it contained thirteen convolutional layers, five max pooling layers, and three dense layers, of which sixteen were weight layers (with trainable parameters). The base model layers were divided in five blocks. Tje first two blocks consisted of two convolutional layers and one max pooling (MaxPooling2D) layer, and the next three blocks contained three convolutional layers and one MaxPooling2D layer each. To customize and fine-tune the pre-trained model, we unfroze the top layers of the frozen model base, and then, we jointly trained both the newly-added custom layers and the last layers of the base model. In fact, we dropped-out the 3 fully connected layers and the last layer that makes a prediction at the top of the base model. The parameters of the first 10 layers were frozen, and the other layers were retrained. The output of the last layer, which was MaxPooling2D, was flattened and was fed into a dense layer with 512 neurons along with the ReLU activation function. To reduce over-fitting, dropout with a probability of 20% was used. Another dense layer with 256 neurons and the ReLU activation function was then added. The last layer was a dense layer with 9 neurons and a softmax activation function. The final model had 25,958,409 trainable parameters and 1,735,488 non-trainable parameters (Table 4). During the training process, the Adam optimizer with a categorical cross-entropy loss function and a learning rate of \(10^{-5}\) were used. The best model weights were saved using the callback function based on the max validation accuracy over 50 epochs. **Inception-v3. ** Inception-v3 is an image recognition model widely used in computer vision. Compared to its predecessors, it is a deeper network with uncompromised speed characterized by higher efficiency. It has achieved accuracy greater than 78% on the ImageNet dataset, while having fewer parameters and lower memory and resource consumption [43]. This architecture has proven to be more computationally efficient in comparison to VGGNet, both in terms of the number of parameters and resource costs [44]. In fact, generally speaking, it was created to reduce the computational burden of deep neural networks and to maintain the state-of-art performance at the same time. This is the reason why we chose this architecture for the bug bite classification task. Inception-v3 contains symmetric and asymmetric building blocks that include convolutions, average pooling, max pooling, dropouts, concatenations, and fully connected layers, along with the usage of batch normalisation and the softmax loss function. The Inception-v3 implementation used in this study was taken from the Keras Python library [38]. It has a total of 21,802,782 parameters across 311 layers. Weights learned on the ImageNet dataset were used for transfer learning. We concatenated four additional layers \begin{table}
\begin{tabular}{c c c} \hline \hline
**Layer** & **Output Shape** & **Number of Parameters** \\ \hline Block5 pool—MaxPooling2D & (None, 7, 7, 512) & 0 \\ Flatten & (None, 25,088) & 0 \\ Dense & (None, 512) & 12,845,568 \\ Dropout & (None, 512) & 0 \\ Dense & (None, 256) & 131,328 \\ Dense & (None, 9) & 2313 \\ \hline Total parameters: 27,693,897 & & \\ Trainable parameters: 25,958,409 & & \\ Non-trainable parameters: 1,735,488 & & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Summary of the custom layers added to pre-trained VGG16 model. to the model's output. Namely, we added a flatten layer to the output, a dense layer with 1024 neurons and the ReLU activation function, a dropout layer with a 0.2 dropout rate, and finally, a dense layer with 9 neurons and a softmax activation function that represents a probability for each class. Additionally, we tried using only certain layers of the Inception-v3 model instead of using all 48. Two cases were tested: one using all the layers from the beginning of the model until the layer named'mixed4' and the other, similarly, until the layer annotated with'mixed7'.