# An Aerial Mixed-Reality Environment for First-Person-View Drone Flying
## 2 Related Work
### Virtual and Augmented Reality for Drones  
Creating virtual and augmented reality (AR) environments for drones is a recent challenge. Drone manufacturers, including DJI and Parrot, have released low-priced easy-to-control drones, and low-priced VR/AR headsets have recently been launched. Hence, many interesting applications combining drones and VR/AR technologies have been developed. Using drones with cameras and HMDs, a user's view can be expanded to places that are difficult or impossible to capture. [PERSON] et al. [23] suggested using a see-through display for flight control in a narrow or limited environment. This system simulates X-ray vision in a walled environment to enable users to naturally view and explore hidden spaces. [PERSON] et al. [24] proposed a system that allows users to interactively control a drone and visualize the depth data of photographed parts in real time using a point cloud, which can be viewed on a VR HMD.  
Research has been conducted that mixed real scenes from drone cameras with virtual objects. For large-scale AR, [PERSON] et al. [25] proposed a hardware configuration and virtual object overlay methods for aerial photography using a multidirectional camera and a large airship. [PERSON] et al. [26] presented a haptic augmented interface to detect nuclear sources outdoors and verified their locations using a fixed ground camera. [PERSON] et al. [27; 28] proposed a drone navigation method for 3D reconstruction of an area of interest using a drone equipped with a high-resolution camera and AR. This is an AR system that complements a supervisor's view through relevant information for its autonomous driving plan and real-time flight feedback. [PERSON] et al. [29] developed a video augmentation system using differential global positioning system (DGPS) sensors for remote disaster monitoring. In AR research regarding construction environment monitoring, a system for the midair 3D reconstruction of a construction site has been presented, where data of interest are captured and then overlaid with those of an actual site on a mobile device.  
However, in the abovementioned studies, the locations of augmented objects are occasionally mismatched with the locations in the physical environment, which can confuse users. This is because virtual object mixing using a mono camera or GPS can easily generate errors from the sensors. Furthermore, their main objective is not to provide an immersive experience for users but to provide information to users through a two-dimensional (2D) screen interface or 3D point-cloud-based VR. As previously mentioned in Section 2.1, drones with stereo cameras have been investigated for autonomous driving [16; 17; 18] and 3D reconstructions [19; 20; 21; 22], and [PERSON] et al. [18] proposed a similar immersive setup using a VR HMD but only for a ground control purpose. In addition, the recent studies [25; 26; 27; 28] discussed in Section 2.2 combined advanced AR technologies with drones to superimpose valuable graphic data over the images that come from the cameras of the drones. MR is a hybrid of physical and virtual worlds, encompassing both AR and augmented reality via immersive technology. Stereo cameras are appropriate for generating an immersive MR environment; however, no previous study exists in which stereo cameras mounted on drones are used to create an aerial MR environment and expand the ground control experience of drones using a VR HMD.