# Augmented Reality and Machine Learning Incorporation Using YOLOv3 and ARKit
## 4 Object Detection  
Object detection using a convolutional neural network can be categorised into two different types: region nomination and regression. Region nominations, such as R-CNN [27], SPP-Net [28], Fast R-CNN [29], and Faster R-CNN [30], use step-by-step detection strategy algorithms. They first extract the proposal regions from the image using selective search and then classify the image within the proposal regions. The output accuracies of these models are consistently above 80%. However, the frame per second (FPS) rate reduces dramatically. Only 7 FPS is possible for Faster R-CNN, which is one of the fastest models but is still far from the real-time FPS standard. On the other hand, the YOLO model can reach 45 FPS, which is suitable for real-time detection tasks, especially for AR applications. YOLO [31] uses the regression method to predict the object bounding box and class name instead of using the proposed region method. However, due to the simpler network architecture, the detection accuracy reduces when the frame rate increases. **YOLOv3 deep neural network model**. The main principle of YOLO is using the entire image as the input to the network and directly returning the bounding box coordinates and corresponding class name. YOLOv3 [32] is the next generation of YOLOv2 [33] and contains significant improvements. YOLOv2 uses Darknet-19 as its backbone and an additional 11 object detection layers. However, YOLOv2 struggles with detecting small objects, whereas YOLOv3 provides a state-of-the-art performance by using residual blocks, skip connections and upsampling, as shown in Figure 11. It uses Darknet-53 as the backbone, which is reported to be more efficient than Darknet-19, ResNet-101 and ResNet-152 [34]. Figure 10: Examples of formats representing coordinates of a bounding box in YOLO training. The main difference between YOLOv3 and its ancestors is the prediction of three different scale levels. Each of the input images is downsampled by 32, 16 and 8, respectively. The detection is first made at the 82 nd layer after the downsampling process of the original input from the previous 81 layers. The 81 st layer has a stride of 32, meaning that if we have an image of 416 \(\times\) 416, then the resultant feature map would be 13 \(\times\) 13 \(\times\) depth. Then the feature map from 79 th layer upsampled by two to dimensions of 26 \(\times\) 26 \(\times\) depth is depth concatenated with the feature map from the 61 st layer. The combination feature maps are subjected to a few convolutional layers before reaching the 94 th layer, where the second detection occurs. The same procedure is executed again, where the feature map at the 94 th layer is subjected to a few convolutional layers and depth concatenated with the feature map from the 36 th layer. The final detection occurs at the 106 th layer, yielding the feature map of 52 \(\times\) 52 \(\times\) depth. Detections at three different scale levels help to address the limitations around detecting small objects in YOLOv2. The 82 nd prediction layer is responsible for detecting large-scale objects. The last prediction layer is responsible for detecting the small-scale objects, whereas the 94 th prediction layer is suitable for medium-scale objects. The predictions are made by applying 1 \(\times\) 1 detection kernels to the feature map, as shown in Figure 12. At the tensor procedure level, the YOLOv3 network divides the input image into an S \(\times\) S grid of cells. Each cell is responsible for predicting bounding boxes \(B\) and class probabilities \(C\) of the potential objects whose centres are inside the grid cell. Each bounding box has five attributes: four bounding box coordinates (\(t_{x}\), \(t_{y}\), \(t_{h}\), \(t_{w}\)) and an object confidence score \(P_{o}\). The confidence score represents the probability of a box containing an object and usually falls between 0 and 1.