# Machine Learning-Based Smartphone Grip Posture Image Recognition and Classification
###### Abstract
Lastly, computer vision and model-based detection techniques involve detecting the hand by removing the background, identifying the hand contour based on the captured image, and then classifying the grip posture using a developed classification model. [PERSON] et al. (2017) used image detection techniques to remove the background from hand images and then employed contour extraction and the Haar cascade classifier to distinguish seven hand postures (2-finger, 3-finger, 4-finger, 5-finger, palm, fist, and swipe). The classification accuracy was reported to be between 85% and 95% (2-finger: 94%, 3-finger: 93%, 4-finger: 92%, 5-finger: 92%, palm: 95%, fist: 95%, and swipe: 85%) [10]. [PERSON] et al. [11] detected the overall image of the hand using the RGB form of OpenCV to identify skin color. The convex hull technique was applied to detect finger angles and identify hand shapes, which were classified into 10 forms (A, B, C, D, E, F, G, H, I, and J) using a convolutional neural network (CNN) technique, achieving a reported classification accuracy of 90%. In smartphone grip posture classification, inducing natural hand grip postures and achieving high model performance are considered important, with few studies applying machine learning techniques to classify smartphone grip posture in usage scenarios. Previous studies on identifying grip posture have involved attaching markers to the hand or using gloves; however, wearing such equipment may hinder natural grip postures [1; 12]. [PERSON] et al. (2020) ensured natural hand movements by utilizing web cameras without wearable equipment, manually classifying grip postures from videos [1]. However, this method relies on subjective judgment and requires significant effort and time. While many studies have detected natural grip posture based on 2D images, few have focused on smartphone grip posture. [PERSON] et al. (2019) used a hidden Markov model (HMM) to classify hand gestures into five categories related to smart light control: switch smart light on, switch smart light off, increase saturation of smart light, decrease saturation of smart light, and change smart light's color [13]. While ensuring natural hand movements by not attaching any equipment to the hand, there remains a need to identify grip posture in smartphone usage scenarios. Ensemble models have demonstrated their ability to ensure diversity among models and achieve better predictive accuracy compared to single models [14,15]. [PERSON] et al. (2022) utilized an ensemble of ResNet-50 and MobileNetV2 models to recognize 32 Arabic hand gestures, achieving an accuracy of 97.5% for ResNet-50 alone and 97.1% for MobileNetV2 alone; the ensemble of both models reached an accuracy of 98.2% [16]. The present study aims to propose a model for detecting and classifying smartphone hand grip posture images using machine learning techniques. To identify smartphone grip posture, an experiment was conducted with participants of various handedness, hand lengths, and hand widths. Grip postures performed during tasks with smartphones of different sizes were recorded using video cameras. Grip posture images were extracted from the recorded videos through frame captures and preprocessed based on skin color. The proposed model detected and classified grip postures, and the model's accuracy was evaluated in the study. The rest of this article is organized as follows. Section 2 reviews related work on hand recognition and its applications. Section 3 describes an experimental protocol, as well as the structures and training procedures of the proposed models. Section 4 presents experimental results, and Section 5 discusses the key findings, their implications, the limitations of the study, and future research directions in detail. Lastly, Section 6 concludes the article with the major findings and a brief summary of the study's limitations and directions for future research.