## 4 Non-Vision-Based HAR
### Multimodal HAR
The outputs from these streams were then concatenated and processed through densely connected layers [61]. In 2021, [PERSON] et al. proposed the use of ensemble learning with Multiple CNNs [68]. They compared the performance of different ensembles of three 1D CNN models with the performance of the single models and found that the ensembles had a higher accuracy. Also in 2021, [PERSON] et al. proposed a Multimodal Human Activity Recognition Ensemble Network (MMHAR-EnsemNet) which uses multiple streams to process information from the skeleton and RGB data in addition to the accelerometer and gyroscope data [67]. They trained and validated their model on the UTD-MHAD and Berkeley-MHAD datasets. They compared the model's performance using different combinations of the inputs, and the model had the highest accuracy when all four modalities were used. In 2022, [PERSON] et al. proposed a three-stream model that uses reinforcement learning for data fusion [69]. Their model took as input data the RGB image, skeleton and depth data from the NTU RGB and HMDB51 datasets. They proposed that each modal datum cannot be weighed equally because they have different representation abilities for various actions [69]. Thus, using reinforcement learning to determine the fusion weights would account for this variability. They trained and tested their model on the NTU RGB and HMDB51 datasets and performed ablation studies to show the effectiveness of their reinforcement learning multimodal data fusion method. [PERSON] et al. proposed the combination of cameras and movement sensors for the health monitoring of people with mobility disabilities [57]. Their model took as inputs the skeleton sequence from the camera and the inertial sequence from the movement sensors. Two component models, ALSTGCN and LSTM-FCN, were used to process the skeleton sequence and inertial sequence, respectively. An Adaptive Weight Learning (AWL) model was used to fuse the skeleton and inertial features optimally. They tested their model on two public datasets, C-MHAD and UTD-MHAD, and on M-MHAD, which is a novel dataset created by them. They compared the model with other state-of-the-art single-modality models, and it performed better [57]. Some researchers have also proposed combining non-vision inputs in multiple stream models. [PERSON] et al. proposed a multistream HAR model called 1 DCNN-Att-BiLSTM, which combines a one-dimensional CNN, an attention mechanism, and a bidirectional LSTM [66]. The sensor inputs are processed in parallel and fed into the 1 DCNN and BiLSTM models connected in series. An attention mechanism was used to select important features. Then, a fully connected layer was used for the fusion of the feature data obtained from each channel. A softmax layer was used for the final output. They tested their model on the Shoaib AR, Shoaib SA and HAPT datasets. Their focus was on the combination of three sensor types: accelerometer, gyroscope, and magnetometer. They compared the performance of their model to other models and demonstrated its superior performance. In their work, they also compared the performance of various sensor combinations, and showed that the accelerometer and gyroscope sensor combination performed better than the accelerometer and magnetometer or gyroscope and magnetometer combination [66]. [PERSON] et al. proposed a position-aware HAR model using a multimodal Deep Convolutional Neural Network (DCNN) [70]. They proposed that forming a body network using data from sensors on different positions on the body would improve activity recognition. They treated each position independently by passing them through an ensemble of Convolutional Neural Networks (CNNs) before fusion. The fusion block is made up of fully connected layers that concatenate the outputs from the CNN blocks. Using the Leave-Subjects-Out Hold Out (LSOHO) method, they trained and validated their model on the SHO [71] and mHealth [72] datasets. Their result showed an overall improvement compared to simple multiposition and single-position models. [PERSON] et al.