# FlickPose: A Hand Tracking-Based Text Input System for Mobile Users Wearing Smart Glasses
## 4 Machine Learning Model to Estimate Hand Pose
### Comparison with Other Learning Models  
We compare the classification performance of the constructed MLP machine learning model with other machine learning models and other hand pose classification methods. Before the comparison, we describe the Random Forest, LightGBM and rule-based architecture, which are the subjects of performance comparisons in this study. The hyperparameters selected during the construction of the Random Forest were as follows. The number of features used for branching is selected as sqrt, which is the square root of the number of input dimensions. Since the input data for the Random Forest is 15-dimensional numerical data, as in MLP, the number of features is 3, truncated to the decimal point of the square root of 15. * Number of decision trees (n_estimators): 100
* Number of features used for branching (max_features): 3
* Depth of decision trees (max_depth): 5  
The hyperparameters used to construct LightGBM are listed below. Given that the target of the training was a relatively simple hand pose estimation, default values were \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Evaluation Index** & **Accuracy** & **Precision** & **Recall** & **Macro-F1** \\ \hline Value & 0.9988 & 0.9987 & 0.9985 & 0.9986 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Training results of model with all joint points as input. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Evaluation Index** & **Accuracy** & **Precision** & **Recall** & **Macro-F1** \\ \hline Value & 0.9970 & 0.9966 & 0.9962 & 0.9964 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Training results for a model with only the joint point of the fingertip of each finger as input. Figure 9: Architecture of model with all joint points as input. used for many of the hyperparameter values. Note that, as with MLP and Random Forest, the input for LightGBM is also 15-dimensional numerical data. * Objective function (objective): multiclass
* Boosting algorithm (boosting_type): gbdt
* Evaluation index (metric): multi_logloss
* Learning rate (learning_rate): 0.1
* Maximum number of leaves per tree (num_leaves): 31
* Depth of decision tree (max_depth): 5  
In the rule-based algorithm, since the same process is performed with each finger, the processing with the index finger is used as an example. Four joints of the index finger are used, starting from the fingertip, and the vectors to the adjacent joints are calculated. Next, the cosines are computed for each combination of these vectors. When the fingers are extended, the vectors point in the same direction, so the cosine value approaches 1. Therefore, if the value obtained by multiplying all three calculated cosines is greater than a predetermined threshold value, the finger can be judged to be extended. By performing this series of processes for each finger, Poses 1-5 can be determined in the form of a combination of the bending and stretching of each finger. Next, the classification accuracy of each method is compared. The five hand poses collected are used for the comparison, but since all methods except for the rule-based one are trained in advance, the classification accuracy of human hands not used for training is also important for the implementation scenario. Therefore, MLP, Random Forest, and LightGBM were trained five times by taking four out of five datasets and dividing them 9:1 into training and validation data, respectively, and using the remaining dataset for one person as test data. Cross-validation was then performed to find the average of the results obtained from each training. The classification results obtained with each method are shown in Table 4. From Table 4, it can be seen that the accuracy is particularly high for MLP and LightGBM. Here, in a situation where a character input system is implemented, it is important to be able to maintain high accuracy even with hand data that is not included in the training data.