# Activity Recognition Using Different Sensor Modalities and Deep Learning
## 4 Discussion and Conclusions

Table 3: The classification performance of ConvLSTM NN when the input set is a combination of all the data. for many people and reduce the demand for use. However, smart insoles can be thought of as an object-based device. They can be located in shoes. Thus, continuous monitoring can be provided without the need for users to feel as if they are carrying an external device. Many researchers have preferred traditional and modern machine learning methods for activity recognition. Currently, deep learning is very efficient due to its automatic feature extraction ability. This study uses a hybrid model, namely, the convolutional LSTM neural network, due to its superior performance. Various sensor modalities including foot contact states, acceleration, and gyroscope were fed into neural networks to examine the classification performance. There are many studies on activity recognition using deep learning. [PERSON] et al. proposed a hybrid of CNN and bidirectional long-short-term memory [43]. The accuracy of the UCI-HAR, WISDM, and PAMAP2 datasets achieved 96.37%, 96.05%, and 94.29%, respectively. [PERSON] et al. proposed a model with CNN combined with a gated recurrent unit and obtained accuracy of 96.20%, 97.21%, and 95.27% on UCI-HAR, WISDM, and PAMAP2 datasets, respectively [44]. [PERSON] et al. proposed a design including inertial sensors and pressure sensors embedded into the smart insoles. They set the sampling frequency as 200 Hz. Their study included six different activities including walking downstairs, sitting, sit to stand, standing, walking upstairs, and walking. The average accuracy was 94.66% [45]. In our study, the sampling rate is 40 Hz for lower battery consumption and the total duration of the collected data was 1600 min. Moreover, the number of activities was eight. When compared with these studies, our study presents less accuracy; however, the results are important to provide more generalized scores based on the number of performed activities, number of placed sensors, and number of attended volunteers. Moreover, a high sampling rate may result in high power consumption for the device. The results showed that the combination of all the data presented the best performance compared with other categories for activity recognition. Moreover, the neural network provided better results for foot contact states compared to the combination of the accelerometer and gyroscope data. Considering the ease of use and performance of the designed smart insole, it is seen that it may be sufficient for the specified activity recognition types alone. Improving the accuracy of results is possible by adding more FSRs into the smart insole or attaching more IMU devices on the human body, including the shank part of the lower limb. However, accelerometers are sensitive sensors and stable positions are important to affect less noise. Hence, placing these sensors in stationary positions is important to improve the success rate. However, more data does not always mean high performance. They can cause an increment in the training duration for the neural networks and do not guarantee more accuracy. In our future work, we are planning to conduct a study about sports activity recognition such as playing football, basketball, tennis, etc., using different sensor modalities including blood pressure, electromyography, heart rate, etc. Moreover, traditional and modern machine learning techniques can be used to make a comparison of the performance. Conceptualization, G.A. and Y.S. ; methodology, G.A. and Y.S. ; software, G.A. and Y.S. ; validation, G.A. and Y.S. ; formal analysis, G.A. and Y.S. ; investigation, G.A. and Y.S. ; resources, G.A. and Y.S. ; data curation, G.A. and Y.S. ; writing--original draft preparation, G.A. and Y.S. ; writing--review and editing, G.A. and Y.S. ; visualization, G.A. ; supervision, Y.S. ; project administration, Y.S. All authors have read and agreed to the published version of the manuscript. This research received no external funding.