# Merging-Squeeze-Excitation Feature Fusion for Human Activity Recognition Using Wearable Sensors
## 2 Related Work
We summarize the aforementioned multi-branch DL architectures in Table 4.  
Recently, the SE mechanism (i.e., squeeze and excitation operations) has been applied to feature fusion in multi-branch DL architectures where feature maps from each branch will be recalibrated before fusing them together. [PERSON] et al. [29] proposed a model called the temporal-spectral-based squeeze-and-excitation feature fusion network (TS-SEFFNet) to classify motor imagery tasks by using electroencephalography (EEG) signals. The TS-SEFFNet receives EEG signals and uses two branches with different DL models called the deep-temporal convolution block and the multi-spectral convolution block to extract two different sets of feature maps. The feature maps of each set are recalibrated by using the SE mechanism. The TS-SEFFNet combines the outputs of these two branches by using concatenation. Instead of using sensor data from one modality, multimodal classification [30] receives data from multiple modalities and has gained a significant amount of attention [31, 32, 33]. Essentially, a multi-modal classification model will be implemented based on a multi-branch architecture where each branch will receive different modal data and extract the corresponding features. These features obtained from various modalities will be combined and sent to the classification process. Since the feature maps from each modality contribute information unequally, an efficient fusion method must be investigated [31, 32, 33]. In addition, several SE-based feature fusion methods were extensively investigated in multi-modal classification. [PERSON] et al. [34] proposed a feature fusion method called the \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Year** & **Ref.** & **Dataset** & **Device** & **Category** & **DL Model** & **Feature Fusion** \\ \hline
2018 & [19] & Opportunity, & IMUs & Multiple Inputs & CNN & Concatenation \\  & & Order Picking, PAMAP2 & & & & \\
2019 & [20] & UCI HAR, WISDM & Smartphone & Multiple DL Models & CNN & Concatenation \\
2020 & [22] & DG, DSAD, & IMUs & Multiple Inputs & CNN & Concatenation \\  & & PAMAP2, & & & & \\  & & RealWorld-HAR & & & & \\
2020 & [24] & WISDM & Smartphone & Multiple DL Models & CNN & Concatenation \\
2020 & [25] & MHEALTH, WISDM & IMUs, & Multiple DL Models & CNN and LSTM & Concatenation \\  & & Smartphone & & & & \\
2021 & [21] & UCI HAR, WISDM & Smartphone & Multiple DL Models & CNN with & Concatenation \\  & & UCI HAR, WISDM & Smartphone & Multiple DL Models & an SE block & Concatenation \\
2021 & [26] & PAMAP2, UCI HAR, & IMUs, & Multiple DL Models & CNN and GRU & Concatenation \\  & & WISDM & Smartphone & & & \\
2021 & [27] & Self-Recorded Data, & IMUs, & Multiple DL Models & CNN & Concatenation \\  & & UCI HAR & Smartphone & & & \\
2022 & [28] & PAMAP2, UCI HAR, & IMUs, & Multiple DL Models & CNN & Concatenation \\  & & WISDM & Smartphone & & & \\
2023 & [23] & Opportunity, PAMAP2, & IMUs, & Multiple Inputs & CNN and & Concatenation \\  & & UnMiMiB-SHAR & Smartphone & & Residual Blocks & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: A summary of multi-branch DL architectures for HAR using sensor data. multi-modal SE feature fusion module to combine feature maps from EEG signals and feature maps from electroocologram (EOG) signals for sleep-staging classification. Unlike [16; 29] where each branch computes the weights for the feature-map calibration in the excitation operation separately, the multi-modal SE feature fusion module will calculate the channel weights based on the channel-wise statistics from both EEG feature maps and EOG feature maps. [PERSON] et al. [35] proposed a DL model called the expansion-squeeze-excitation fusion network (ESE-FN) for elderly activity recognition using RGB videos and skeleton sequences. The ESE-FN applies two successive fusion modules (modal fusion and channel fusion) to combine RGB features and skeleton features properly.