# Deep Learning Seehive Monitoring System for Early Detection of the Varroa Mite
## 5 Experimental Scenarios and Results
### Scenario III: Evaluation of Varroa Mite Detection Step  
In this scenario, 200 photos were used to detect the Varroa mite on bee contours derived from the previous object detection step of the CNN models. This scenario aims to test the accuracy and precision of the Varroa mite detection process on bee objects. One hundred of the detected contours depicted bees with Varroa, while the rest depicted bees without Varroa. Upon the application of the HSV masking and Hough transformation detection step for Varroa mites on bee objects, the measurements in Table 6 were obtained. Table 6 shows the accuracy and precision calculated using the following Equation (5) for accuracy and precision accordingly.  
\[\begin{split} Acc=\frac{TP+TN}{TP+FP+FN+TN},\;Prec=\frac{TP}{TP+ FP},\\ Recall=\frac{TP}{TP+FN},\;F1\_Score=\frac{2*Recall*Prec}{Recall+ Prec}\end{split} \tag{5}\]  
The variables used for the validation process are: _TP_ (True Positive) describes the number of photos in which the mite is present and detected successfully, _FN_ (False Negative) is the number of photos that had Varroa but were not detected, _FP_ (False Positive) is the number of images that did not have Varroa but was detected, and finally the variable _TN_ (True Negative) is the number of images that did not have the Varroa mite and was not detected.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{\begin{tabular}{c} **CNN** \\ **Models** \\ \end{tabular} } & \multirow{2}{*}{\begin{tabular}{c} **Mean Detection** \\ **Accuracy** \\ **(MDA)** \\ \end{tabular} } & \multirow{2}{*}{\begin{tabular}{c} **CNN Model** \\ **mAP** \\ \end{tabular} } & \multirow{2}{*}{\begin{tabular}{c} **End Node** \\ **Device** \\ **SF** \\ \end{tabular} } & \multirow{2}{*}{\begin{tabular}{c} **Cloud Server** \\ **Version 1** \\ **SF** \\ \end{tabular} } \\ \hline \begin{tabular}{c} Mobilenet \\ V2 \\ \end{tabular} & 0.887 & 0.481 & 0.003 & 0.17 & 0.28 \\ \hline \
\begin{tabular}{c} Mobilenet \\ V3 \\ \end{tabular} & 0.677 & 0.496 & 0.003 & 0.14 & 0.29 \\ \hline
\begin{tabular}{c} ResNet-50 \\ FPN \\ \end{tabular} & 0.821 & 0.467 & 0.001 & 0.1 & 0.25 \\ \hline \end{tabular}
\end{table}
Table 5: Mean Detection Accuracy (human bee counting verification) over models’ mAP.  
In addition, using the measurements of Table 6 along with the measurements of scenario II (Table 5), the total accuracy of the whole Varroa detection process can be inferred through the formula: \(Total_{Acc}=Acc*MDA\). In this way, it can be determined which CNN, together with the use of image processing, can give the best results for the detection of Varroa. This is because the accuracy rates of Table 6 depend to a large extent on the number of images that go through the process of detecting the Varroa mite. The MobileNet V2 model has the best total accuracy, which corresponds to 68% (8% more than the ResNet-50 FPN and 24% more than the MobileNet V3).  
It should be commented at this point that the authors have not used the Recall and F1-Score metrics. The recall metric refers to the sensitivity percentage of the varroa mite detection algorithm. The degree of sensitivity depends directly on the size of the masked color space to recognize the Varroa color hue. This space also depends on the brightness and sharpness of the photos that go through this process. For this reason, the F1-Score for the calculation of which recall in part has not been taken into account. Instead, only precision and accuracy metrics have been used.  
Using the same set of photos for all system modes of operation (online version 1, online version 2, offline end-node device), there were no differences in the execution times of the detection process, since, in all three cases, the average detection time is close to 4 ms. Moreover, there was no change in the size of the memory used by the three device cases during the Varroa detection step, which is close to 0.7 MB.