# A Computer Vision Model to Identify the Incorrect Use of Face Masks for COVID-19 Awareness
## 4 Results and Discussion  
### Phase I: Testing Deep Learning Classification Models with Different Optimizers  
In this section, the results obtained in the two-stage pipeline are presented. Special attention is given to the classification stage, since the pre-trained RetinaFace model performed the face detection part, requiring no fine-tuning. Therefore, the image classification task concerning COVID-19 masks was the pivotal part of this study. The principal objective was to gain an insight into the performance of the DL models while solving an urgent real-life problem. We implemented all the variants of ResNet and one variant of ResNeSt using transfer learning. The initial weights were borrowed from the pre-trained model with the ImageNet dataset. Then, all the models were trained using the FMLD dataset, with RBG input images of size \(W\times H\) pixels as inputs, where \(W\) and \(H\) were \(\geq\) 10 to avoid losing the activation maps during the pooling operations. The dataset was randomly divided into training, validation, and test sets. An overview of the resulting datasets can be found in Table 2.  
The cross-entropy was adopted as a loss function. At the same time, the following optimizers were considered: (1) SGD with a learning rate equal to 0.001 and a reduction factor of 0.1 after seven epochs, (2) SGD with a momentum value of 0.9, (3) Adam with a learning rate of 0.001, (4) RMSprop with a learning rate of 0.01, and (5) Adagrad with a learning rate of 0.01. The models were trained with 15 epochs, each with a batch size equal to 16. The behavior of the accuracy and loss during the training and validation stages of ResNet-34 with the Adagrad optimizer led to this being selected as the optimal configuration. The ResNet-34 with Adagrad results can be found in Figures 4-7. RMSprop, Adam, and SGD were less effective optimizers in either the training or validation datasets, since they resulted in the highest losses and lowest accuracy. On the other hand, SGD with momentum and Adagrad optimizers resulted in the highest accuracy and lowest loss, but their differences were minimal. The details are given as follows:  
1. **ResNet-18 and ResNet-152:** The best performance was obtained with the Adagrad optimizer for both the training and validation datasets. 2. **ResNet-34, ResNet-50, ResNet-101, and ResNeSt-200:** In these models, the best result on the training dataset was obtained with the Adagrad optimizer, and for the validation dataset, SGD with momentum was more robust. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset** & \multicolumn{2}{c}{**Classes**} \\  & _Compliant_ & _Non-Compliant_ \\ \hline _Training_ & 7028 & 17,525 \\ _Validation_ & 1757 & 4382 \\ _Test_ & 4795 & 137,50 \\ _Total_ & 13,580 & 35,657 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: An overview of the dataset split used to train the different classification models. Figure 4: ResNet-34 accuracy on training dataset. Figure 5: ResNet-34 loss on training dataset. Figure 6: ResNet-34 accuracy on validation dataset. Figure 7: ResNet-34 loss on validation dataset. However, it is necessary to analyze the results obtained on the test dataset to verify the performance of each model. Therefore, the experiments were performed using 18,545 images (4745 Compliant and 13,750 Non-Compliant), and the accuracy results are presented in Table 3.  
As can be seen, all the models achieved an accuracy greater than 94% except ResNeSt-200 with the RMSprop optimizer, which obtained 92.41%. The more accurate models are those trained with the SGD with momentum and Adagrad optimizers. ResNet-34 with Adagrad is the model configuration with the highest accuracy, followed by ResNet-152 with SGD with momentum and ResNet-152 with Adagrad. All the models, ordered by their accuracy percentages, can be seen in Figure 8. The lowest accuracy models are those trained with the RMSprop optimizer. To analyze the results obtained in the test dataset, we chose ResNet-34 with the Adagrad optimizer as the model for performing the predictions.