# MLPs Are All You Need for Human Activity Recognition
## 2 Related Work  
Four main categories of deep-learning architecutres have been used in HAR, convolution-based architectures, recurrent networks, hybrid models, and attention-based models [9]. Evaluation is performed on benchmark HAR datasets, including Opportunity [10], Daphnet Gait [11], PAMAP2 [12], Skoda Checkpoint [13], WISDM [14], MHEALTH [15], and UCI-HAR [16]. With the recent success of CNNs in feature detection, [PERSON] et al. [17] first proposed using CNNs in HAR, but they only used a basic CNN on a single accelerometer. Next, [PERSON] et al. [18] thoroughly investigated CNN use in HAR and established its viability. However, good performance requires large CNN models; this increases the computational cost, constraining their use on low-power devices. To solve this, [PERSON] et al. [19] looked into the performance and viability of an efficient CNN that uses a tiny Lego filter inspired by [PERSON] et al. [20]. The paper investigated a resource-constrained CNN model for HAR on mobile and wearable devices, achieving an \(F_{1}\) score of 91.40% and 86.10% in the PAMAP2 and Opportunity datasets, respectively. However, this work had the drawback of having slightly worse performance when compared to conventional CNNs when using small Lego filters instead of traditional filters. Recurrent networks are good at capturing long-term dependencies, and because of their architecture, they can pick up temporal features in sequenced data. [PERSON] et al. [18] took advantage of these benefits and proposed three LSTM models: two uni-directional LSTM and a bi-directional LSTM model, which trains on both historical and upcoming data. The models were trained and evaluated on the PAMAP2, Opportunity, and Dapnet Gait datasets. This work described how to train similar recurrent networks in HAR and introduced a brand-new regularisation method. The bi-LSTM model outperformed state-of-the-art models in the Opportunity Gestures dataset, achieving an \(F_{1}\) score of 92.7%. [PERSON] et al. [21] showcased the performance of uni-directional, bi-directional, and cascaded LSTM models. The bi-direction LSTM performed best on the Opportunity dataset, with an accuracy of 92.5%. The cascaded LSTM performed the best on Daphnet, with an accuracy of 94.1%. However, the work did not evaluate the models on extensive and complex human activities; additionally, resource efficiency was not considered when designing the model. CNNs effectively extract spatial features from a local area; however, these models do not have \"memory\", making it hard to learn long-term dependencies between different samples. RNNs, on the other hand, due to their specific structure, have memory allowing them to learn long-term dependencies; however, they are challenging to train. Researchers have created hybrid deep learning models to address the shortcomings of both CNN and RNN neural networks. Recently, attention mechanisms have been applied in models to improve performance in HAR. Attention mechanisms allow the model to learn what to focus on in the dataset and understand the relationship between each input element. [PERSON] et al. [22] combined attention mechanisms with a CNN-GRU. This architecture provides the benefits of CNNs, GRUs,and attention, enabling spatial and temporal understanding of the dataset. The model had good performance on all the datasets explored. However, the model is unsuitable for low-powered devices due to the computational complexity of combining all these models. [PERSON] et al. [23] combined temporal and sensor attention in residual networks using a novel dual attention technique to enhance the capacity for feature learning in HAR datasets. The temporal attention focuses on the target activity sequence and chooses where in the sequence to concentrate, whereas the sensor attention is vital in selecting which sensor to focus on, obtaining accuracy scores of 82.75% and 93.16% on Opportunity and PAMAP2, respectively. Although this model performed well, it was constrained by the shortage of labelled multimodal training samples.