User-Aware Audio Marker Using Low Frequency Ultrasonic Object Detection and Communication for Augmented Reality  
[PERSON]  
1 School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Gwangju 61005, Korea; [EMAIL_ADDRESS]  
[PERSON]  
2 Future Infrastructure Research Center, Korea Institute of Civil Engineering and Building Technology, Goyang-si, Gyeonggi-do 10223, Korea; [EMAIL_ADDRESS]  
[PERSON]  
1 School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Gwangju 61005, Korea; [EMAIL_ADDRESS]  
[PERSON]  
3 Department of Electrical and Computer Engineering, City University of New York (CUNY), New York, NY 10017, USA; [EMAIL_ADDRESS]  
Correspondence: [EMAIL_ADDRESS]; Tel. : +82-62-715-2228; Fax: +82-62-715-22043330999  
who have different interests within the same space. For indoor guidance purposes, an AR marker, which is an image marker based on a combination of coloring patterns and quick response (QR) codes, was used for indoor AR services, because the detection and recognition mechanism in this method was robust with commercial mobile hardware, even at a long distance [3]. In particular, this marker was used for establishing indoor guidance systems for the visually impaired [3]. However, if such image markers were concentrated in a small area, many of the markers that were not interested in the view of the AR user might be observed, which caused an interference with the user's immersive experience. Although inpainting techniques have been proposed to remove image markers from the user's sight [4], the computational burden could be somewhat heavy for most mobile AR devices, limiting its applications [4]. Other visual marker approaches have been proposed as alternatives to image marker-based approaches. For example, the shape of human fingertips was utilized to replace an image marker as the role of visual cues in an AR device [5]. In other words, each fingertip acted as a virtual marker to interact with virtual objects at close distances. Despite its usefulness for some AR applications, this approach could be considerably distracting under visual conditions affected by illumination or occlusion [6; 7; 8]. While recent AR applications tried to recognize visual environments without using a marker, in order to deploy virtual objects in natural ways [9], such attempts required substantial computations of the AR device; thus, their use was limited to a certain AR scenario [10]. Visual marker limitations could also be avoided by using other marker types, such as gyroscope and global positioning system (GPS) sensors [11; 12], but unfortunately, such markers were difficult to establish in indoor environments [11]. For example, while motion detection sensors, e.g., pyroelectric infrared (PIR) sensors, could work as indoor markers for some AR applications [13], PIR sensors failed to send rich information regarding virtual objects or events to the AR device. Instead, our previous work proposed an audio marker based on low frequency ultrasound (LFU) [14]. There were many benefits to this approach. Since the frequency response of the LFU-based audio marker ranged from 18 to 20 kHz, the transmitted signal was inaudible to human ears, and its transceivers could be made of the most commercial off-the-shelf (COTS) loudspeakers and microphones [15]. Thus, operating the LFU-based audio marker did not require any additional hardware other than a generic device equipped with an audio interface, e.g., a smart phone. Therefore, the LFU-based audio marker could be useful for some AR service scenarios where conventional image markers were difficult. For example, audio markers could present virtual objects to the user although the user might not see an image marker. This was because the LFU spread across all directions within the room, regardless of the user's visual focus or line-of-sight with the marker.