# FlickPose: A Hand Tracking-Based Text Input System for Mobile Users Wearing Smart Glasses
## 6 Limitations and Future Work  
In this study, we proposed FlickPose, a novel text input method that integrates hand posing with key selection. Experimental results suggest that it can improve performance for regular HMD users. However, several limitations and future challenges remain. The input experiment resulted in some erroneous inputs, which are reflected in both the total and consecutive deletion counts. Identifying the causes of these errors could help improve system performance. In the text input experiment, the input logs are collected, and it is possible to see which characters are deleted. However, the causes of errors are difficult to determine because the number of patterns is very large (for example, if the key selection was correct and the hand pose was wrong, it is difficult to determine whether the subject made a hand pose mistake, the machine learning model made a mistake, or the task sentence was misread). Therefore, it is not possible to quantitatively calculate the breakdown of error causes. However, there were some issues that were pointed out by several subjects as the cause of the error. With respect to hand pose determination, more misclassifications of Pose 1 and Pose 2 occurred than for other hand poses. Although a machine learning model was used to identify hand poses in this study, the evaluation of the training results did not show many misjudgments of Pose 1 and Pose 2. Therefore, it is believed that the main cause of misjudgment is that the subjects in the evaluation experiment had different skeletal and joint characteristics than the five subjects for whom data was collected. Therefore, it is necessary to improve the responsiveness of the model by increasing the number of people for whom data is collected. Efforts to improve the robustness of the model by increasing the dataset require further study. Regarding hand poses, it was noted that other poses should be adopted. Specifically, it was pointed out that Pose 4 placed a heavy burden on the hands, especially the middle finger. It was also pointed out that Pose 1 should be performed with all fingers extended, because the load is smaller when fingers are extended than when fits are clenched. In our study, we employed Poses 1-5 as simpler poses than sign language and finger writing, but as subjects pointed out, there is room for improvement in hand poses. In addition, subjects were confused by the hand poses when they first started using FlickPose, but they became somewhat familiar with them during the input experiment. Considering this, the emphasis should be placed on minimizing the burden on the hand rather than forcing the hand pose to be more regular. In addition, a mistake was made in selecting the blank key by mistake when trying to select the delete key, especially on the touch-panel type. Subjects made the mistake of pressing the blank key when they tried to touch the key directly because the blank key is directly below the delete key. This problem can be attributed in part to the lack of tactile feedback. We believe that adjusting the size of the keys according to how often they are used could be a possible countermeasure. Since FlickPose relies on the device's built-in hand tracker, its accuracy may decrease in environments where tracking performance is limited. This issue could be mitigated by using time-series data to estimate hand poses and correct finger misalignment. Recently, EMG-based wristbands (e.g., Mudra Band, [[https://mudra-band.com/pages/mudra-band-main](https://mudra-band.com/pages/mudra-band-main)]([https://mudra-band.com/pages/mudra-band-main](https://mudra-band.com/pages/mudra-band-main)), accessed on 23 June 2025) have been developed, offering accurate hand recognition with less effort than traditional controllers. We believe incorporating such devices could further enhance accuracy. Although FlickPose relies less on tactile feedback, especially in the raycast form, the lack of tactile feedback remains a challenge. To address this issue, there are previous works ([PERSON] et al. [13], [PERSON] et al.