# Merging-Squeeze-Excitation Feature Fusion for Human Activity Recognition Using Wearable Sensors
## 2 Related Work
\begin{table}
\begin{tabular}{p{142.3 pt} p{142.3 pt} p{142.3 pt}} \hline \hline
**Symbol** & **Definition** \\ \hline \(\mathbf{\hat{g}}^{(d,n)}\in\mathbb{R}^{C}\) & A vector of channel-wise statistics according to the feature maps \(\mathbf{\hat{P}}^{(d-1,n)}\) at the \(n\)-th branch in the \(d\)-th MSE feature fusion block. \\ \(\mathbf{\hat{h}}^{(d,n)}\in\mathbb{R}^{C}\) & A vector of channel-wise statistics according to the addition of \(\mathbf{\hat{g}}^{(d,n)}\) and \(\mathbf{\hat{u}}^{(d)}\) at the \(n\)-th branch in the \(d\)-th MSE feature fusion block. \\ \(\mathbf{\hat{g}}^{(d,n)}\in\mathbb{R}^{C}\) & A vector of channel weights for the feature maps \(\mathbf{\hat{P}}^{(d,n)}\) at the \(n\)-th branch in the \(d\)-th MSE feature fusion block. \\ \(\mathbf{\hat{u}}^{(d)}\in\mathbb{R}^{C}\) & A vector of channel-wise statistics according to the merged and channel-weighted feature maps \(\mathbf{\hat{Y}}^{dep,(d)}\) in the \(d\)-th MSE feature fusion block. \\ \(\mathbf{A}^{(n)}\in\mathbb{R}^{1\times L\times C}\) & A 3D array of local feature maps at the \(n\)-th branch. \\ \(\mathbf{B}\in\mathbb{R}^{1\times L\times C}\) & A 3D array of pre-merged feature maps. \\ \(\mathbf{P}^{(d,n)}\in\mathbb{R}^{1\times L\times C}\) & A 3D array of channel-weighted feature maps according to the feature maps \(\mathbf{\hat{P}}^{(d-1,n)}\) at the \(n\)-th branch in the \(d\)-th MSE feature fusion block. \\ \(\mathbf{X}^{(n)}\in\mathbb{R}^{1\times W\times C}\) & A 3D array of sensor data at the \(n\)-th branch (obtained from the \(n\)-th IMU). \\ \(\mathbf{\hat{Y}}^{\text{mse},(d)}\in\mathbb{R}^{1\times L\times C}\) & A 3D array of channel-weighted and merged feature maps, which is the output of the weighted feature merging in the \(d\)-th MSE feature fusion block. \\ \(\mathbf{\hat{Y}}^{dep,(d)}\in\mathbb{R}^{1\times L\times C}\) & A 3D array of channel-weighted and merged feature maps, which is the output of the \(d\)-th MSE feature fusion. \\ \hline \hline \end{tabular}
\end{table} 
Table 2: List of main symbols used in Section 5.3. \begin{table}
\begin{tabular}{p{142.3 pt} p{142.3 pt} p{142.3 pt} p{142.3 pt}} \hline \hline
**Year** & **Ref.** & **Dataset** & **Device** & **DL Model** \\ \hline
2021 & [21] & UCI HAR, WISDM & Smartphone & CNN with an SE block \\
2022 & [17] & HASC, UCI HAR, WISDM & Smartphone & State-of-the-art CNNs with SE blocks \\
2022 & [18] & HAPT, MobiAct v2.0 & Smartphone & CNN with a residual block, an SE block, and BiGRU \\ \hline \hline \end{tabular}
\end{table} 
Table 3: A summary of DL models with SE blocks for HAR using sensor data. data are obtained initially. These sets of sensor data are inputted into a multi-branch DL architecture. Each branch will receive each set and extract the corresponding features by using the same DL model independently [19, 22, 23]. In order to obtain a set of feature maps on each branch, [PERSON] et al. [19] applied a series of convolutional layers and max pooling layers, [PERSON] et al. [22] implemented stacked convolutional layers, and [PERSON] et al. [23] employed a CNN model with residual blocks. The feature maps of all branches are fused (i.e., feature fusion) by using concatenation. In the second category, we apply a set of sensor data to a multi-branch DL architecture where each branch uses a different DL model and results in a different set of features. Three-branch DL architectures were proposed in [20, 21, 24, 25, 26, 27, 28], where a CNN model [20, 24, 27, 28], a hybrid of a CNN model, and a bidirectional long short-term memory (LSTM) layer [25], as well as a CNN model with an SE block [21], and a hybrid of convolutional layers and gated recurrent unit (GRU) layers [26] were used on each branch to extract a set of features. The differences among these branches are the kernel sizes of the convolutional layers [20, 21, 24, 25, 26, 27, 28] and the number of layers [20]. Similarly, the output feature sets are combined by using concatenation.