# Human Activity Recognition by the Image Type Encoding Method of 3-Axial Sensor Data
## 7 Conclusions  
This paper proposes an image encoding method using 3-axial sensor data of acceleration and gyro and a human activity recognition (HAR) model based on it. By visualizing the raw sensor data from the WISDM dataset, strong visual features of the data waveform could be extracted, which improved recognition accuracy and categories. To augment the 1D raw sensor data, we divided it into time intervals calculated based on the \"walking\" activity, which is one of the fundamental human activities, and normalized the representation range of the segmented 1D sensor data to values between 0 and 255. This enabled clustering of the finely represented sensor data into a larger range, making it possible to remove noise caused by fine changes, such as shaking. The data with the modified representation range creates a 2D image through the matrix dot product of the acceleration and gyro data, and this image includes areas of strong brightness and weak brightness depending on the position of the data waveform. However, this can show overly geometric patterns, which can actually degrade the performance of the model. Therefore, a second processing step is used to generate a standardized visual image.  
The standardized visual image shows a propeller shape with different curves and brightness areas of the wings depending on the sensor data waveform, creating visual feature differences in similar types of human activities. Moreover, due to the clear input data, the hierarchical structure of the HAR model could be simplified to a relatively shallow eight layers compared to previous studies. In addition, it was possible to recognize 18 categories of human activity, which is three times higher than in previous HAR studies, and achieve a high accuracy of 98.15%.  
Our proposed algorithm is a method for detecting various types of human body activities on a single device. Through this, we were able to recognize 18 categories of body activities. In future research, additional experiments are needed to recognize more types of body activities, and comparison and analysis with previous studies that use dimension expansion concepts such as image encoding will be necessary. Additionally, analysis of the correlation between increased computational load due to data expansion and changes in encoding images based on data waveforms will be needed.  
If we design a self-big-data-measurement device for detecting human body activities and collecting the measured data, we can expect its usefulness in the development of customized healthcare services based on lifelogging.  
Writing--original draft, C.K.; Writing--review & editing, W.L. All authors have read and agreed to the published version of the manuscript.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Ref.** & **Model** &
\begin{tabular}{c} **No. of** \\ **Activities** \\ \end{tabular} & **Layer** & **F1 Score (\%)** & **Accuracy (\%)** \\ \hline
[36] & Baseline & 6 & 10 & - & 89.55 \\
[37] & VGG16 & 6 & 23 & - & 89.32 \\
[38] & Inception-V3 & 6 & 313 & - & 91.54 \\
[39] & Xception & 6 & 126 & - & 90.17 \\
[40] & EfficientNet B0 & 6 & 233 & - & 89.11 \\
[23] & CNN & 6 & 6 & - & 93.32 \\
[41] & Multichannel CNN-GRU & 6 & 9 & 96.39 & 96.41 \\
[42] & U-Net & 6 & 11 & 96.50 & 96.40 \\
[43] & CNN with an attention & 6 & 6 & - & 96.40 \\  & mechanism & & & & \\ - & Proposed Model & 18 & 8 & 98.00 & 98.15 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Evaluation of the proposed model in comparison with models based on CNN.  
**Funding:** This research was supported by Sangji University Research Fund, 2020. This research was supported by \"Regional Innovation Strategy (RIS)\" through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (MOE) (2022 RIS-005).  
**Conflicts of Interest:** The authors declare no conflict of interest.