# Training-Free Acoustic-Based Hand Gesture Tracking on Smart Speakers
## 1 Introduction  
Hand gesture recognition is a convenient and natural Human-Computer Interaction (HCI) technology for users to control smart devices. The report has shown that the market of hand gesture recognition will reach CNY 48 billion by 2024 [1], applying to various fields such as virtual reality, smart home, and health care. Therefore, effective hand gesture recognition will be an important way for humans to interact with machines in the future. In recent years, diverse hand gesture recognition systems have been developed to serve HCI applications. They can be classified as device-based systems and device-free systems. Among device-based systems, a range of solutions has been proposed based on accelerometers [2], gyroscopes [3], and magnetometers [4]. While promising in recognition accuracy, this series of solutions requires the user to wear or carry devices, which brings inconvenience to their daily life. Among device-free systems, camera-based solutions [5] have issues in regard to occlusion and privacy. Wireless sensing technologies have been successfully employed to sense hand gestures, including Wi-Fi [6], LoRa [7], visible light [8], and radar [9]. Wireless sensing relies on analyzing the wireless signals reflected from the target to obtain information of hand gestures. However, there are several limitations restricting the wide deployment of these solutions. Wi-Fi-based systems are severely affected by environmental interference. Radar-based systems are promising, but they require dedicated high-cost hardware, as do LoRa and visible light. Acoustic signals [10] have attracted extensive attention recently. Compared with hand gesture recognition systems that employ other wireless signals, acoustic-based systems have two unique advantages. On one hand, speakers and microphones are widely appliedon electronic devices in our daily life, such as smart TVs and smartphones. On the other hand, due to the low propagation speed in the air (340 m/s), acoustic signals have higher sensing precision. Most acoustic-based hand gesture recognition systems adopt machine learning, which requires many data for training. However, there are no publicly available datasets, and it is difficult for us to collect data from diverse users. Inspired by a prior study [11], we adopt acoustic-based hand tracking to intuitively recognize gestures. While promising in many aspects, the microphones of present commodity microphone arrays are usually spaced at several centimeters to optimize speech recognition. Since the wavelength of our using inaudible acoustic signal is much shorter than the spacing of these microphone arrays, the spatial sampling rate is inadequate, which leads to localization angle ambiguity of hand tracking. Therefore, it is particularly challenging for acoustic-based hand gesture recognition with a commodity microphone array. In this paper, we propose a novel system, TaGesture, which is designed for device-free and training-free hand gesture recognition based on the inaudible acoustic signal at the frequency band of 17 kHz\(\sim\)23 kHz (audible sound is in 20 Hz\(\sim\)16 kHz [12]). Specifically, TaGesture utilizes a commercial speaker and microphone array to track hand trajectories for gesture recognition. To achieve TaGesture, we need to address the following challenges. (1) Interference caused by multipath: The interference can cover up the hand information and substantially reduce the sensing precision. (2) Localization angle ambiguity of hand tracking: Due to the large spacing of the microphone array, localization has the angle ambiguity, which leads to hand tracking with a significant error. (3) Training-free gesture recognition: Considering the user diversity, it is challenging to achieve high accuracy and robust gesture recognition without training. To address the first challenge, we apply background subtraction to obtain clean signals with highlighted target information.