# Merging-Squeeze-Excitation Feature Fusion for Human Activity Recognition Using Wearable Sensors
## 4 Proposed Architecture
### Merging-Squeeze-Excitation Feature Fusion
#### 4.2.2 Squeeze and Excitation  
In the squeeze-and-excitation step, we recalibrate each set of feature maps \(\textbf{A}^{(n)}\) such that the informative feature maps will be emphasized and ambiguous feature maps will be suppressed by using channel weights, which will be computed according to both \(\textbf{A}^{(n)}\) and **B**. First, we obtain the channel-wise statistics \(\textbf{u}\in\mathbb{R}^{C}\) by passing **B** to a 1D GAP layer and the channel-wise statistics \(\textbf{g}^{(n)}\in\mathbb{R}^{C}\) by passing \(\textbf{A}^{(n)}\) to another 1D GAP. The statistics **u** are expressed as \(\textbf{u}=[u_{1},u_{2},\ldots,u_{c}]^{T}\) and the statistics \(\textbf{g}^{(n)}\) are expressed as \(\textbf{g}^{(n)}=[g_{1}^{(n)},g_{2}^{(n)},\ldots,g_{C}^{(n)}]^{T}\), where \([\cdot,\cdot,\ldots,\cdot]^{T}\) is the transpose, \(u_{c}\) is obtained by averaging the values in the \(c\)-th 1D feature map of **B**, and \(g_{c}^{(n)}\) is obtained by averaging the values in the \(c\)-th 1D feature map of \(\textbf{U}^{(n)}\). Specifically, we have \[u_{c}=\frac{1}{W}\sum_{w=1}^{W}b_{w,c},\] (8) and \[g_{c}^{(n)}=\frac{1}{W}\sum_{w=1}^{W}a_{w,c}^{(n)}.\] (9) Thereafter, we obtain a channel-wise statistics \(\textbf{h}^{(n)}\in\mathbb{R}^{C}\) from \[\textbf{h}^{(n)}=\textbf{u}+\textbf{g}^{(n)}.\] (10) The statistics \(\textbf{h}^{(n)}\) are expressed as \(\textbf{h}^{(n)}=[h_{1}^{(n)},h_{2}^{(n)},\ldots,h_{C}^{(n)}]^{T}\).  
Next, a set of channel weights \(\textbf{s}^{(n)}\in\mathbb{R}^{C}\), where \(\textbf{s}^{(n)}=[s_{1}^{(n)},s_{2}^{(n)},\ldots,s_{C}^{(n)}]^{T}\), for individual \(\textbf{A}^{(n)}\), is obtained by using two fully connected (FC) layers with the ReLU activation after the first FC layer and the Sigmoid activation after the second FC layer [16]:  
\[\textbf{s}^{(n)}=\sigma\Bigl{(}\textbf{W}_{2}^{(n)}\delta\bigl{(}\textbf{W}_{ 1}^{(n)}\textbf{h}^{(n)}\bigr{)}\Bigr{)}, \tag{11}\]  
where \(\sigma(\cdot)\) is the Sigmoid activation function, \(\delta(\cdot)\) is the ReLU activation function, \(\textbf{W}_{1}^{(n)}\in\mathbb{R}^{\frac{C}{\tau}\times C}\) is the weight matrix of the first FC layer, \(\textbf{W}_{2}^{(n)}\in\mathbb{R}^{C\times\frac{C}{\tau}}\) is the weight matrix of the second FC layer, and \(r\) is the reduction ratio which is used to reduce the first FC layer's output dimension.  
Finally, we recalibrate the feature maps \(\textbf{A}^{(n)}\) according to the channel weights \(\textbf{s}^{(n)}\) to emphasize useful feature maps and suppress ambiguous feature maps and, then, obtain a set of channel-weighted feature maps \(\textbf{P}^{(n)}\in\mathbb{R}^{1\times W\times C}\). The feature maps \(\textbf{P}^{(n)}\) can be expressed as \(\textbf{P}^{(n)}=[\textbf{P}_{1}^{(n)};\textbf{P}_{2}^{(n)};\ldots;\textbf{P} _{C}^{(n)}]\), where \(\textbf{p}_{c}^{(n)}=[p_{1,c}^{(n)},p_{2,c}^{(n)},\ldots,p_{W,c}^{(n)}]\) is a 1D feature map and obtained from multiplication between the channel weight \(s_{c}^{(n)}\) and the 1D feature map \(\mathbf{a}_{c}^{(n)}\): \[\mathbf{p}_{c}^{(n)}=s_{c}^{(n)}\mathbf{a}_{c}^{(n)}\] (12) For a future usage, we denote the squeeze-and-excitation operation to compute \(\mathbf{P}^{(n)}\) as \[\mathbf{P}^{(n)}=\mathbb{F}_{SE}\big{(}\mathbf{A}^{(n)},\mathbf{B}\big{)}.\] (13)