# Human Activity Recognition Based on Deep Learning Regardless of Sensor Orientation
###### Abstract  
In recent years, the continuous progress of wireless communication and sensor technology has enabled sensors to be better integrated into mobile devices. Therefore, sensor-based Human Activity Recognition (HAR) has attracted widespread attention among researchers, especially in the fields of wearable technology and ubiquitous computing. In these applications, mobile devices' built-in accelerometers and gyroscopes have been typically used for human activity recognition. However, devices such as smartphones were placed in users' pockets and not fixed to their bodies, and the resulting changes in the orientation of the sensors due to users' habits or external forces can lead to a decrease in the accuracy of activity recognition. Unfortunately, there is currently a lack of publicly available datasets specifically designed to address the issue of device angle change. The contributions of this study are as follows. First, we constructed a dataset with eight different sensor placement angles using accelerometers and gyroscopes as a prerequisite for the subsequent research. Second, we introduced the Madgwick algorithm to extract quaternion mode features and alleviate the impact of angle changes on recognition performance by fusing raw accelerometer data and quaternion mode features. The resulting study provides a comprehensive analysis. On the one hand, we fine-tuned ResNet and tested its stability on our dataset, achieving a recognition accuracy of 97.13%. We included two independent experiments, one for user-related scenarios and the other for user-independent scenarios. In addition, we validated our research results on two publicly available datasets, demonstrating that our method has good generalization performance. human activity recognition; sensor data fusion; sensor orientation +
Footnote †: journal:  
greater recognition accuracy, but is inconvenient and costly, while the latter is low-cost and convenient, but may not be as accurate. Another issue involves device angle changes during data collection. In [16], for example, the sensors were not easily fixed on the human body when participants were performing tasks such as running. When only a single mobile phone was placed in the participant's pocket, the issue of device angle change was augmented due to the sensor not being physically fixed to the human body. The issue of device angle change in sensors which can affect the accuracy of HAR tasks has always been present. Our goal here is to find a more robust method that can eliminate the impact caused by device angle changes. At present, there are several publicly available HAR datasets on the market, such as WISDM [14], UCI HAR [17], Opportunity [18], and others. These datasets are comprehensive, but lack the different angle orientation data needed to simulate real-life scenarios accurately. For example, users may have different habits when using the devices, and are able to freely adjust the angles and positions of the devices according to their preferences. Additionally, device angle fluctuations may vary among different users during movement [19]. Existing research has mainly focused on machine learning or deep learning model architectures, with limited emphasis on the sensor data specifically. This means that important sensor data information, such as gyroscope readings that indicate angle information, may be underutilized. In other words, these studies often input only raw data into their models and apply the same feature extraction strategy to all sensors. Several studies have pointed out that the angle information in gyroscopes is rich and valuable [20]. Typically, combining gyroscopes with accelerometers yields better results than using accelerometers alone. Therefore, effectively utilizing the angle information from gyroscopes is crucial for HAR tasks. In addition to the HAR field, gyroscope data play an important role in fields such as inertial navigation [21], virtual reality [22], robotics [23], and more.