# MLPs Are All You Need for Human Activity Recognition
## 2 Related Work
### MLP Architectures  
In a different area of study, with the arrival of the MLP-Mixer, pure deep MLP architectures have started appearing in computer vision tasks. The MLP variants have similar structures to the MLP-Mixer, usually with only the internal layers being modified to improve the model. These MLPs work by using a \"token-mixing\" or/and \"channel-mixing\" layer to capture relevant information from the input, followed by stacking these layers \(N\) times. The MLP-Mixer achieved competitive results in computer vision tasks; however, CNNs and Transformer-based models such as Vision Transformers (ViT) [24] outperform the Mixer. To overcome this, [PERSON] et al. [25] proposed a new MLP model called gMLP that introduces a spatial gating unit into MLP layers to enable cross-token interactions. The gMLP performs spatial and channel projections similar to the MLP-Mixer; however, there is no channel-mixing layer. The gMLP has 66% fewer parameters than the MLP-Mixer yet has a 3% performance improvement.  
Another method involves using only channel projections. Removing the token-mixing layer prevents MLPs from gaining context from the input and stops the tokens from interacting with one another. Instead, to regain context, the feature maps are spatially interacted with using channel projections after being shifted to align them between the various channels [24]. [PERSON] et al. [26] proposed the \(S^{2}\)-MLP. This model uses spatial shift operations to communicate between patches. This method is computationally efficient with low complexity. This model achieves high performance even with its simplicity, outperforming the MLP-Mixer and remaining competitive with ViT. Finally, [PERSON] et al. [27] proposed ActiveMLP. This is a token-mixing mechanism that enables the model to learn how to combine the current token with useful contextual information from other tokens within the global context of the input. This mechanism allows the model to learn diverse patterns successfully in vision-based tasks, achieving an accuracy of 82% in ImageNet-1K.  
The token-Mixer uses static operations. This prevents the token-Mixer from adapting to the varying content contained in the different tokens. Methods have been proposed to add adaptability, allowing the varying information in the tokens to be mixed [24]. [PERSON] et al. [28] try to overcome the static token-mixing layer by viewing each token as an amplitude and phase-varying wave. The phase is a complex number that controls the influence of how tokens and fixed weights are related in the MLP, whereas the amplitude is a real number that represents each token's content. The combined output of these tokens is affected by the phase difference between them, and tokens with similar phases tend to complement one another. WaveMLP limits the fully connected layers to only tokens connected within a local window to address the issue of input resolution sensitivity; however, this prevents the MLP from taking global context across the entire input. WaveMLP is among the best MLP architectures, achieving 82.6% top 1-accuracy in ImageNet-1K. It achieves competitive results with CNNs and Transformers but is still outperformed by them. To improve on this, [PERSON] et al. [29] proposed the DynaMixer; by considering the contents of each set of tokens to be mixed, DynaMixer can dynamically generate mixing matrices. The DynaMixer mixes the tokens row-wise and column-wise to improve the computation speed. In each iteration of the DynaMixer, feature dimensionality occurs to produce the Mixer matrices; additionally, substantially reducing the number of dimensions has little impact on the performance. These feature spaces are separated into various segments for token-mixing. The DynaMixer currently produces state-of-the-art performance among MLP vision architectures, achieving 82.7% top-1 in Imagenet-1k.