# LIMUNet: A Lightweight Neural Network for Human Activity Recognition Using Smartwatches
## 3 Materials and Methods
### LIMUNet Algorithm Design
#### 3.2.3 Lightweight IMU Feature Extraction Method Based on Dual Attention Mechanism  
The attention mechanism can reduce relevant interference items by discarding the practice of incorporating all information into the reference range, instead focusing on information that is more valuable to the task. The attention mechanism, as a relatively independent network module, can also be well integrated into the previously designed lightweight neural network. In the scenario of user behavior recognition based on smartwatches, the diversity of input data is strong, and the model faces significant challenges with new users, new activity areas, and new devices. The feature extraction capability of convolutional neural networks is very strong, but the weights of each channel and spatial region in the trained network are fixed. When the input data change, the representational capability of each channel and spatial region is the same, which limits the model's generalization ability. This paper combines the different representational capabilities of multi-channel sensor data and employs an attention mechanism for sensor feature extraction and fusion. By learning the weights that influence the classification results for each channel and each type of sensor data, the network can perform fine-grained enhancement and suppression at the neuron level when processing specific inputs. This helps the neural network to focus more on the parts beneficial for the user behavior recognition task, reducing sensitivity to redundant or irrelevant information and improving the model's adaptability. To improve the generalization ability of the residual bottleneck layer, this study adds a channel attention module [31]. This module learns to obtain the attention scores for each channel and then uses these scores as weights to enhance and suppress the channels of the  
Figure 3: Residual bottleneck layer. feature map. The specific implementation of the channel attention mechanism is shown in Figure 4.  
In the channel attention mechanism, to calculate the attention for each input channel, it is first necessary to compress the global spatial information of the feature map into channel descriptors. This paper uses an average pooling layer for compression, and the calculation method is shown in Equation (4):  
\[z_{c}=F_{sq}(\mathbf{u}_{c})=\frac{1}{W\times H}\sum_{i=1}^{H}\sum_{j=1}^{W}\mathbf{u}_{ c}(i,j) \tag{4}\]  
where \(C,F_{sq},\mathbf{u}_{c},W\), and \(H\) denote the channel number of the feature map, the average pooling compression function, the input of the neural network at this layer, the width of the feature map, and the height of the feature map, respectively. In order to obtain the attention information of each channel in the channel descriptor, it is necessary to fully capture the dependencies between channels. The input will be fed into a single-layer fully connected network, and in order to reduce computational complexity and increase operational speed, the ReLU functionzz [47] is used as the activation function, making it more suitable for lightweight requirements. Through the processing of the fully connected network, channel attention scores can be obtained, as shown in Equation (5):  
\[\mathbf{s_{1}}=F_{\text{ex}}\left(\mathbf{z},\mathbf{W}\right)=\delta(\mathbf{W}\mathbf{z}+\mathbf{b}) \tag{5}\]  
where \(\delta\), \(\mathbf{W}\), and \(b\) denote the ReLU function, the weights of the fully connected neural network, and the biases of the fully connected neural network, respectively. Finally, applying the attention scores to each channel yields the feature map processed by the channel attention mechanism, calculated as shown in Equation (6):  
\[\mathbf{x}_{c}=F_{\text{scale}}\left(\mathbf{u}_{c},s_{1c}\right)=\mathbf{u}_{c}\times s_{ 1c} \tag{6}\]  
In recent cutting-edge research on lightweight activity recognition, lightweight convolutional neural networks apply the same weights for feature extraction across all sensor data, but this approach overlooks the specificity of IMU sensors.