# FlickPose: A Hand Tracking-Based Text Input System for Mobile Users Wearing Smart Glasses
## 1 Introduction  
In recent years, virtual space technologies such as XR and the metaverse have been rapidly advancing, leading to the widespread adoption of head-mounted displays (HMDs). Among HMDs, smart glasses designed for mobile users have garnered significant attention. However, text input methods for smart glasses in mobile environments are not yet well developed, and improvements in usability and accuracy are required. Currently, text input methods for HMDs, including smart glasses, can be broadly classified into physical keyboards, voice recognition, and virtual keyboard projection. Physical keyboards allow users to input text similarly to conventional PCs, but they are inconvenient for mobile use due to the need to carry and place them somewhere ([1]). Voice recognition eliminates the need for a keyboard, but it has privacy concerns and is susceptible to background noise, making it difficult to use in public spaces. [PERSON] et al. [2] found that input via speech recognition is subject to noise and social acceptability, as well as privacy restrictions when entering passwords and private messages. Due to these limitations, virtual keyboard projection has attracted attention as a promising input method for HMDs. Virtual keyboards can be projected onto a flat surface, such as a desk, or in midair, and users can select keys using controllers, eye tracking, or hand tracking. Traditionally, controllers have been the most common input method, but they are not ideal for mobile use due to portability issues. Eye tracking requires high-precision eye trackers, which are expensive and have not yet been widely adopted. Hand tracking, on the other hand, is relatively cost-effective and has improved in accuracy, as many recent HMDs are equipped with advanced hand-tracking capabilities. However, two major challenges remain--the accuracy limitations of hand tracking and the lack of haptic feedback--both of which contribute to input errors. Input errors caused by hand tracking occur due to discrepancies between the detected hand position and its actual location, leading to unintended key selections. [PERSON] et al. [3] state that the use of a virtual keyboard in the air is challenged by the lack of haptic feedback and the inability to place part of the hand or arm on a physical plane. They also state that tracking errors and delays exacerbate these inaccuracies. One potential solution to reduce input errors is to enlarge the keys; however, in QWERTY layouts, increasing key sizes takes up more screen space and requires larger hand movements, which can lead to fatigue. Thus, alternative approaches, such as reducing the number of keys and supplementing key selection with additional gestures, are being explored. This study proposes a new text input method called \"FlickPose,\" with the objectives of developing a text input method optimized for mobile environments and evaluating its effectiveness in comparison to standard methods. FlickPose is based on the flick input keyboard layout and allows users to select keys through a touch-panel method and a raycast method. Additionally, left-hand gestures are used to determine the characters to be input. Here, the focus of this research is on smart glasses, which are devices intended for use in a mobile environment. However, currently available smart glasses do not have a hand tracker with sufficient accuracy. Therefore, this study implements the system on Meta Quest3, a VR/MR HMD as an alternative device, and incorporates a machine learning model to estimate the hand pose of the left hand, thereby improving input accuracy. FlickPose addresses the limitations of conventional input methods and seeks to provide a more intuitive and comfortable text entry experience. The contributions of this research are as follows:  
1. Proposal of FlickPose: A novel text input method combining flick input and bimanual interaction for head-mounted displays (HMDs), including smart glasses.