# An Aerial Mixed-Reality Environment for First-Person-View Drone Flying
## 2 Related Work
### Object Recognition and Immersion Measurement in Mixed Reality  
To create an MR environment based on the real world, it is crucial to accurately match the locations of virtual objects to be merged with those of real objects. In other words, real objects, which are the criteria for positioning virtual objects to be rendered, must be recognized accurately. As an example, virtual objects, such as trees and obstacles, can be dynamically created in the MR environment when a drone recognizes flags or water bottles in real scenes. Furthermore, a stable MR system must interact with users in real time to eliminate a sense of difference with the real world. In fact, users are more sensitive to visual mismatches than to virtual objects not moving in the same manner as real objects because the matching errors of virtual objects result in an unnatural MR. Moreover, the MR environment for drone flights from an FPV requires faster and more accurate identification when deciding the locations of virtual objects to be mixed with real scenes because flying drones change their locations and directions more dynamically in each moment than human vision.  
Object recognition has developed rapidly in the field of computer vision because it can be used in major applications, such as intelligent vehicle control, monitoring, and advanced robots. The technology for object recognition has been investigated mainly for pedestrian detection [30] and face recognition [31]. This technology extracts the features of objects using scale invariant feature transformation [32] and histogram of oriented gradient [33] methods, and uses them for identification. However, methods for solving classification problems are slowly being developed, and their performance has improved owing to the use of ensemble or complementing past studies [34; 35]. Recently, [PERSON] et al. [36] achieved a significant performance improvement in the ImageNet Large-Scale Visual Recognition Challenge using AlexNet based on a convolutional neural network (CNN). Subsequently, CNNs for object recognition have been investigated actively. Consequently, region-based CNNs (R-CNNs) using selective search have been proposed, and fast R-CNN [37] and faster R-CNN [38] with higher speeds have been released. A fast R-CNN cannot perform real-time object recognition because of its slow processing speed of the region proposal method operated outside the CNN. However, the faster R-CNN improves the accuracy and speed by using the region proposal algorithm inside the CNN. Consequently, real-time recognition with high accuracy is enabled. In this study, the faster R-CNN proposed by [PERSON] et al. [38], which can perform accurate object recognition in real time, was used as the object recognition method.  
Previous studies have not precisely defined user experience components or the method to measure them in an immersive virtual environment; however, they have been studied indirectly in flight simulators [39] and game fields [40]. Simulator sickness among aircraft pilots has been mainly investigated. The simulator sickness questionnaire (SSQ) is the current standard for measuring simulator sickness. It is recognized as an excellent single measure because it provides each indicator of overall symptoms. The game engagement questionnaire (GEQ) has been widely used as a measure of user engagement in games. Based on these, a more accurate method of measuring immersion in a virtual environment [41] was proposed, in which the reliability of the questionnaire was improved using 10 rating scales and 87 questions. Questions corresponding to immersion, presence, emotion, flow, judgment, usability, and experience were posed and answered based on 10 scales. Furthermore, user immersion was measured with a simple questionnaire that combined the GEQ [42] and SSQ [39] to verify the efficiency of the proposed framework.