# Deep Learning Methods for Bug Bite Classification: An End-to-End System
## 2 Materials and Methods
### CNN Variants Used for Bug Bite Classification
#### 2.6.1 Convolutional Neural Network Architectures
The same additional layers mentioned before were added in all three cases (mixed4, mixed7 and the case when all the layers were considered) and are summarized in Table 5. The resulting model using all Inception-v3 layers has 52,439,049 trainable and 21,802,784 non-trainable parameters, the model using all layers until the'mixed4' layer has 113,256,457 trainable and 3,445,920 non-trainable parameters, and finally, the model using all layers until the'mixed7' layer has 113,256,457 trainable and 8,975,264 non-trainable parameters (Table 6). The training of the model was carried out by freezing all the parameters of the layers taken from the Inception-v3 model and updating the weights of the aforementioned added layers. Categorical cross-entropy was used as a loss function during training, as well as the Adam optimizer with a \(10^{-4}\) learning rate. The training was carried out using 50 epochs for all three models. **DenseNet169. ** DenseNets have the advantage of increasing the accuracy by increasing the number of parameters without reducing the performance and without the risk of overfitting in the case of training with smaller datasets [45]. In fact, they demonstrate the ability to learn deeper and more discriminative features from images [46]. Because of these properties, we include this CNN in our research. DenseNet-169 has a depth of 169 layers and a relatively low number of parameters. In fact, it shows similar or higher performance on the ImageNet dataset achieved with half the number of parameters compared to other architectures [45]. The DenseNet169 architecture was adjusted for the current problem by modifying and adding new layers. More precisely, we used the DenseNet169 model from the Keras library, modified with a custom input layer together with pooling, batch normalization, dropout and fully connected layers on top of the pre-trained network. The batch normalization was incorporated in order to constrain the output of a layer in a specific range, with a mean of zero and a standard deviation of one, which works as regularization. With that, the stability of the neural network was increased, and the training was faster [47]. The dropout was also used as a way of regularization, achieved by inhibiting neurons. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Total Parameters** & **Trainable Parameters** & **Non-Trainable Parameters** \\ \hline All layers & 74,240,808 & 52,438,024 & 21,802,784 \\ Mixed 4 & 116,701,352 & 113,255,432 & 3,445,920 \\ Mixed 7 & 122,230,696 & 113,255,432 & 8,975,264 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Number of parameters in all three Inception-v3 based models. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Layer** & **Output Shape** & **Number of Parameters** \\ \hline Flatten & (None, 51200) for Inception-v3 all layers & 0 \\  & (None, 110592) Inception-v3 mixed4 and mixed7 & \\ Dense & (None, 1024) & 52,429,824 \\ Dropout & (None, 1024) & 0 \\ Dense & (None, 8) & 8200 \\ \hline Total parameters: 27,693,897 & \\ Trainable parameters: 25,958,409 & \\ Non-trainable parameters: 1,735,488 & \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Summary of the custom layers added to the pre-trained Inception-v3 models. In order to experiment with the architecture and improve it, a custom model was made on which several different variants of layers and techniques were applied. This includes an input layer with a custom shape, global average pooling, batch normalization, a dropout of 20%, a dense layer with 2048 neurons, ReLU, a dropout of 20%, a dense layer with 512 neurons, ReLU, a dropout of 20%, a dense layer with 128 neurons, ReLU, a dense layer with 9 neurons, fully connected layer(s) with dropout, ReLU activation, a fully connected output layer, and softmax activation. A learning rate of \(10^{-5}\) and 70 epochs were used for training. The architecture of the best-performing variant is given in Table 7.