# Augmented Reality and Machine Learning Incorporation Using YOLOv3 and ARKit
###### Abstract  
Augmented reality is one of the fastest growing fields, receiving increased funding for the last few years as people realise the potential benefits of rendering virtual information in the real world. Most of today's augmented reality marker-based applications use local feature detection and tracking techniques. The disadvantage of applying these techniques is that the markers must be modified to match the unique classified algorithms or they suffer from low detection accuracy. Machine learning is an ideal solution to overcome the current drawbacks of image processing in augmented reality applications. However, traditional data annotation requires extensive time and labour, as it is usually done manually. This study incorporates machine learning to detect and track augmented reality marker targets in an application using deep neural networks. We firstly implement the auto-generated dataset tool, which is used for the machine learning dataset preparation. The final iOS prototype application incorporates object detection, object tracking and augmented reality. The machine learning model is trained to recognise the differences between targets using one of YOLO's most well-known object detection methods. The final product makes use of a valuable toolkit for developing augmented reality applications called ARKit. augmented reality; artificial intelligence; synthetic data annotation; deep learning; object detection; YOLOv3; iOS application; ARKit +
Footnote †: journal: Applied sciences
the real world and transfer them to an artificial digital environment. These immersive experiences are usually applied to training, education and video games. AR lies between reality, or the natural world, and VR to integrate virtual information into the actual physical world's live view. It provides a unique experience by merging the actual environment with digital information, allowing users to engage and absorb knowledge in a much more efficient manner [6,7]. While VR became more popular with investors during the 1970s and 1980s, AR was forgotten due to hardware limitations and the lack of potential application ideas. However, we do not need to wait long for AR to reappear again due to the significant growth of the technology in the early 1990s [8,9]. Boeing researcher [PERSON] introduced the term \"Augmented Reality\" in his research on mounting cables in airplanes [10]. In the late 1990s, the Columbia University Computer Science research group demonstrated the prototype of an interaction wearable device [11]. Since the expansion of the smartphone market in the beginning of the 21 st century, AR concepts have been gaining more public attention, increasing the number of supported technologies and research. Google was one of the first competitors to introduce to the public their own designed AR equipment, called \"Google Glass\" (Figure 3a). The Google Glass is intended to be a mini wearable HMD that allows the users to experience AR via the glass optics. In 2016, Microsoft introduced the HoloLens [12], which enables users to experience AR and interact with the virtual environment with their own hands (Figure 3b). Apple went one step further in 2017 by re-configuring their iPhone processing chip and introducing the software development kit (SDK), \"ARKit\", to enable AR experiences on mobile devices (Figure 3c). Simultaneously, Google also launched their SDK, called \"AR Core\", for their Android devices. The battle between the major companies is becoming more intense; Apple continues upgrading its hardware while Google optimises its SDK. Figure 1: The head-mount displays (HMD) introduced by [PERSON]’s research team were the first HMD prototype (**left**). The night vision device introduced by the US Army during the Vietnam War allows the viewing of targets in low light and range estimation (**right**). Figure 2: [PERSON] and [PERSON]’s Mixed Reality Continuum. However, there are still many challenging issues that are waiting to be discovered and improved in AR related fields.