# Machine Learning-Based Smartphone Grip Posture Image Recognition and Classification
## 5 Discussion  
The present study developed a machine learning-based model to detect grip posture images while using a smartphone UI and to classify grip postures based on the number of fingers on each side of the smartphone. Previous studies on detecting and classifying grip postures manually differentiated grip postures rather than using machine learning algorithms or focused on grip postures in non-smartphone usage scenarios [1, 8, 63]. Additionally, methods such as using hand gloves or attaching markers made it difficult to induce natural hand movements in users. The grip posture recognition model developed in the present study ensures the user's natural hand movements by utilizing web cameras without requiring additional devices on the user's hand, employing machine learning methods to process and classify the image data. To identify various grip posture types, the grip posture experiment utilized nine different smartphone mock-ups with varying  
Figure 16: Learning curve for Inception V3. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Grip Posture** & **Accuracy (\%)** & **Precision** & **Recall** & **F1-Score** \\ \hline Class 1: L2-B1-R1-K1 & 93.2 & 0.95 & 0.93 & 0.94 \\ Class 2: L2-R1-K2 & 97.3 & 0.93 & 0.97 & 0.95 \\ Class 3: L2-T1-B1-R1 & 100 & 1.00 & 1.00 & 1.00 \\ Class 4: L3-B1-R1 & 98.6 & 0.97 & 0.99 & 0.98 \\ Class 5: L3-R1-K1 & 95.1 & 0.96 & 0.95 & 0.96 \\ Class 6: L3-R1-T1 & 92.9 & 0.98 & 0.93 & 0.95 \\ Class 7: L4-R1 & 98.1 & 0.98 & 0.98 & 0.98 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Classification accuracies for the seven classes of the test data using the ensemble model. lengths, widths, and weights, along with four different usage scenarios, collecting data from participants with a range of hand sizes and hand widths. In the preprocessing phase, utilizing a combination of hand contour and skin color improved the accuracy of grip posture classification. Additionally, classifying grip postures based on the number of fingers positioned on different sides of the smartphone enabled detailed recognition of grip postures at the finger position level. The image detection and classification model developed in the present study is effective in identifying the user's grip posture, with an accuracy of 95.9%. The overall accuracies of the individual models used in the present study were 90.6% for MobileNetV2, 94.3% for ResNet-50, and 85.9% for Inception V3. The ensemble classification model achieved an accuracy of 95.9%, higher than that of the individual models. The ensemble technique combines the classification accuracies of individual models to attain improved accuracy in image classification [8, 10, 63]. [PERSON] et al. [12] used three models (VGG19, ResNet-50, and MobileNetV2) to detect and classify 29 hand gestures in American Sign Language (ASL). The accuracies of the individual models were 99.4% for VGG19, 98.2% for ResNet-50, and 88.1% for MobileNetV2. When the models were ensembled, the accuracy increased to 99.7%, demonstrating improved accuracy with the ensemble technique compared to the individual models. The RGB-based algorithm for tracking hands using skin color has the advantage of accurately tracking hands when there is no background noise or when the skin color of the data used is consistent. However, challenges arise when the background contains colors similar to skin tones or when there is variability in skin color, necessitating adjustments to the algorithm's settings [64]. The data preprocessing method in this study focuses on identifying hand contours (hand shapes) through skin color, allowing the model to be applied regardless of skin color. Although Inception V3 was trained for more epochs than the other models, it demonstrated relatively lower classification accuracy. This may be attributed to its complex multi-branch convolutional architecture and large number of parameters, which likely require larger datasets to generalize effectively.