# Augmented Reality and Machine Learning Incorporation Using YOLOv3 and ARKit
## 3 Synthetic Data Generation  
Developing accurate object detection requires high-quality training data and can be a challenging task in a real-world environment, especially when developing a classifier where the data and environments are uncommon; one example is book images that are used as target markers for AR applications. Therefore, very little data exist that we can use for training the deep neural network model in this instance. In these rare scenarios, synthetic data appear to be a helpful method for generating high-quality and diverse training data in a minimal amount of time. To generate the synthetic dataset, we firstly need to collect all possible images used as the target markers in the AR application. Identification of all possible natural environment situations, such as lighting, orientation and backgrounds, is the most significant task and this helps to improve the quality of the training dataset. For example, if the application will be used primarily by students and teachers, then the backgrounds should be indoors, and the lighting should be warm and moderately bright. Once all of the necessary environmental components are identified, they can be imported into a game engine, such as Unity, to generate the image dataset used for deep neural network training and testing purposes, as shown in Figure 8.  
**Image targets**. We integrated approximately 22 different rectangular-shaped images in this study. Each of these images has a different design, and some of them have a very similar colour. This setup is believed to help us qualify how good the dataset is in terms of identifying the similarity objects. The images were from four different categories: (1) trading/business cards; (2) posters; (3) children's educational books; and (4) food advertisements. The details of target markers are presented in Table 2 and Figure 9.  
Figure 7: Machine learning and augmented reality incorporation module where the object detection algorithm using the deep neural network (**left**) can easily find the marker in the scene and produce the augmented reality experience (**right**). **Virtual environments**. The performance of the deep neural network model is often susceptible to the background, orientation and lighting conditions. Hence, the training dataset must include backgrounds and external conditions similar to what is expected in future application use. The following aspects are used during the proposed synthetic images generated procedure:  
* The number of distinct AR markers appearing in each scene is chosen at random from the pool of trainable classes. * The likelihood of seeing one or more AR markers in the same scene is set at 50%. * The AR marker's scaling range is set randomly from 20% to 40% of the scene area. * The AR marker's rotation angle is set randomly from 0\({}^{\circ}\) to 360\({}^{\circ}\) in respect of the scene \(z\)-axis. * A variety of lighting sources generated by the random camera views to present different real-world illumination effects. These variations provide a wide range of real-life scenarios and external influences that might increase marker detection accuracy. The GPU acceleration allows us to generate over 40,000 high-definition pictures with bounding box coordinates in less than six hours. Human error in data labelling and noise can be minimised by applying this new approach. Figure 8: The synthetic dataset generated process is started with different images of a training class (**panel a**) that are augmented on top of the diverse backgrounds (**panel b**). Several different image processing filters such as orientations and light illusions from various views are applied to the virtual scene (**panel c**) to generate the output images (**panel d**) that can present a wide range of real-world situations. Figure 9: The target marker figures. **First row:** aut, coragunk, tangela, marill, pokemon, match mattax, liverpool fc. **Second row:** milkshake, pokemon online, book food page, book santa page, aut last page.