# Deep-Learning-Incorporated Augmented Reality Application for Engineering Lab Training
## 4 Experimental Results
### Evaluation Metrics  
* Precision: The percentage of positive detections that were correct is referred to as precision. If a model produces no false positives, it has a precision of 1.0. Equation (2) describes precision as the True Positive divided by the sum of the True Positive (TP) and False Positive (FP). TP is defined as a correct prediction of the positive class, whereas FP is an incorrect prediction of negative class as the positive class. \[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}}\] (2)
* Recall: The percentage of true positives that were correctly identified by the model. A model with a recall of 1.0 produces zero false negatives. Recall can be computed as a ratio of True Positives predictions and the sum of TP and False Negatives (FN), as shown in Equation (3). FN is defined as an incorrect prediction of the positive class as the negative class. \[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}}\] (3)
* Intersection over Union (IoU): It is also known as the Jaccard index used for measuring the similarity and diversity of sample sets. In an object detection task, it describes the similarity between the predicted bounding box and the ground truth bounding box. Equation (4) expresses IoU in terms of area of the prediction and ground truth bounding boxes. \[\text{IoU}=\frac{\text{Area of Overlap}}{\text{Area of Union}}\] (4)  
Figure 9: AR training scenario: object tracking and superimposition of a 3D object. It is important to define a threshold to define what is the correctness of the prediction for IoU. T \(\leftarrow\) Threshold IoU \(\geq\) T \(\rightarrow\) Correct IoU \(<\) T \(\rightarrow\) Incorrect
* mean Average Precision (mAP): It takes under consideration both precision and recall. It is also the area beneath the precision-recall curve. The mAP can be computed by \[\text{mAP}=\frac{\sum_{k=1}^{n}AP_{k}}{n}\] (6) where \(AP_{k}\) is the average precision of class \(k\), and \(n\) is the number of classes. Figure 10 shows our model during inference when the new dataset was fed to the model. It demonstrates the correct detection of four types of lab equipment in a single shot when the confidence threshold value (i.e., the threshold related to the confidence score to determine whether the detection is an object of interest or not. Confidence scores of the predicted bounding boxes above the threshold value are considered as positive boxes, or vice versa) is greater or equal to 90%. The experimental results shown in Table 2 support that our model has a high mAP. In practice, the DL model can recognize all of the electrical lab equipment that has been pre-selected. Table 2 shows the average precision and average recall for a given IoU score and mAP. The IoU is a range between 0.50 and 0.95. Using 193 testing images, the average precision of our proposed model achieved a mAP of 81.4% and an average recall of 85.3%. Some failure cases were due to the low ambient lighting and a lack of training datasets with varying lighting conditions. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Description** & **IoU** & **mAP** \\ \hline Average Precision & 0.50:0.95 & 0.814 \\  & 0.50 & 0.976 \\  & 0.75 & 0.954 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Mean average precision (mAP) with different IoU scores of the trained MobileNet-SSD V2 model. Figure 10: Example of automatic equipment detection by the DL model. The number above each green bounding box indicates confidence score of the model, which is the probability that a bounding box contains an object of interest, for the detection. The DL model deployed on a mobile device uses CPU resources of the device to infer and predict objects. We ran the DL model on two different mobile devices to evaluate performances of devices for real-time prediction. We used the frame per second (FPS) unit, which measures the number of images that the mobile device screen displays every second.