# SMS-Net: Bridging the Gap Between High Accuracy and Low Computational Cost in Pose Estimation
## 1 Introduction  
Human pose estimation (HPE) is a computer vision technique for identifying and classifying key joints of the human body in images or videos. In traditional pose estimation methods, various types of equipment (such as sensors) are attached to the subject to directly acquire physical movement information, which is then used to estimate the pose. These methods can accurately capture human movements in real time but require significant time and resources; thus, they are mainly used in restricted environments, like laboratories. Recently, with rapid advancements in deep learning technology and the emergence of large-scale pose estimation datasets, there has been continuous research based on deep learning, focusing on accurate HPE without special equipment. Thus, there have been breakthroughs in overcoming the HPE limitations. As the accuracy and robustness of pose estimation improve, its applications are expanding to various fields, such as posture correction, behavior recognition, abnormal behavior detection, and augmented reality [1]. Various pose estimation models are considerably accurate, from convolutional neural network-based models, such as stacked hourglass network [2] and HRNet [3], to the recently introduced transformer-based models, such as ViTPose [4] and TransPose [5]. However,owing to the high complexity of these models, high-speed inference in low-power and low-performance environments, such as mobile devices, is limited. Generally, high-performance HPE models are characterized by high memory usage and computational costs during training and inference, necessitating hardware equipped with high-performance graph processing units (GPUs). However, with the proliferation of smartphones and various Internet of Things devices, coupled with the rising popularity of edge computing, the demand for deep learning models capable of high-speed pose estimation even on devices with limited computational capability, memory, and power has increased. Therefore, this study proposes SMS-Net based on the encoder-decoder structure of the hourglass network, which applies various lightweight techniques to enable high-speed pose estimation while requiring minimal storage space and computation. Specifically, to improve the internal structure of the encoder and decoder constituting the individual hourglass networks, a building block is proposed for extracting key features necessary for pose estimation and generating heatmaps that indicate the likely regions where key points exist. Subsequently, we conducted experiments to demonstrate its effectiveness. Lightweight convolution operations, including depthwise convolution, were applied to improve the execution complexity of conventional convolution operations in the encoder. Additionally, instead of performing convolution operations on all channels of the input feature map, the input channels were halved, and the convolution operations were performed only on the first half; the results were reconnected with the second half without channel loss, followed by channel shuffle. This reduces the overall computation while preventing performance degradation in keypoint prediction. Different types of spatial information included in the input image to restore meaningful feature maps necessary for keypoint extraction were captured for the decoder. Particularly, a multi-dilation block composed of depthwise convolutions with various dilation values was applied to maintain computational efficiency while integrating spatial information at different scales. This approach prevented overfitting while using fewer weights compared to simply increasing the kernel size. Consequently, by applying the aforementioned methods, the proposed SMS-Net model significantly reduced floating-point operations and memory access costs. Additionally, the inference time was improved while maintaining the pose estimation accuracy by minimizing the number of weights and expanding the receptive field of the decoder.