# Exploring Cutout and Mixup for Robust Human Activity Recognition on Sensor and Skeleton Data +
## 3 Methodology
First, the input time-series data is projected into a higher-dimensional space using a linear embedding layer, transforming the raw sensor readings into a suitable format for the Vision Transformer and then applying positional embeddings to keep temporal order, as shown in Equations (3) and (4):  
\[E=Embedding(X) \tag{3}\]  
\[E_{pos}=E+PositionalEncoding(T,d_{model}) \tag{4}\]  
where \(d_{model}\) is the dimentionality of the embeddings. The model consists of multiple Vision Transformer encoder layers, each comprising multi-head self-attention mechanisms and feed-forward networks:  
\[H_{l}=TransformerEncoderLayer(H_{l-1}) \tag{5}\]  
for \(l=1,\ \dots,\ L\), where \(H_{0}=E_{pos}\) and \(L\) is the number of encoder layers. The output of the final encoder layer is passed through a global average pooling layer to aggregate the sequence information into a fixed-size vector. \[z=GlobalAveragePooling1D(H_{L}) \tag{6}\]  
The pooled vector is then passed through fully connected layers to produce the final activity classification. The cross-entropy loss function is used to train the model, comparing the predicted activity probabilities \(\hat{y}\) with the mixup-augmented labels \(\widetilde{y}\):  
\[\mathcal{L}=-\sum_{k}\widetilde{y}_{k}\text{log}\,\hat{y}_{k} \tag{7}\]  
This methodology of implementing mixup on sensor-based HAR data is empirically validated through experiments on four different HAR datasets, comparing the results to baseline models without mixup augmentation. The findings demonstrate that mixup augmentation leads to improved accuracy and robustness in transformer-based HAR models. After applying mixup for sensor-based HAR, we turn our focus to skeleton-based HAR, where we utilize the cutout data augmentation technique to improve model robustness by randomly masking portions of the skeleton data. Figure 4 shows that the skeleton keypoints are randomly masked as a data augmentation technique. Given a skeleton sample \(X\in\mathbb{R}^{T\times J\times D}\), where \(T\) is the sequence length, \(J\) is the number of joints in the skeleton, and \(D\) is the dimensionality of each joint, cutout for skeleton data can be defined as randomly selecting a set of joints \(\mathcal{I}_{cutout}\subseteq\{1,\ 2,\ 3,\ \}\) to be masked, where the size of \(\mathcal{I}_{cutout}\) is controlled by a hyperparameter \(p\), representing the proportion of joints to be cutout. Then, for each selected joint \(J\in\mathcal{I}_{cutout}\), set its coordinates to zero or an alternative value, such as the mean of the dataset across all frames, such that \(\text{x}_{\text{tj}}=0\) for all \(t\in\{1,\ 2,\ \dots,\ T\}\), \(j\in\mathcal{I}_{cutout}\). The masking function \(M\) can be represented as shown in Equation (8):  
\[\text{M}(\text{x})_{t,j}=\left\{ \begin{array}{l}\text{x}_{\text{tj}}\ \text{if}\ j\
ot\in \mathcal{I}_{cutout},\\ 0\ j\in\mathcal{I}_{cutout}\end{array} \right. \tag{8}\]  
This cutout augmentation technique effectively masks or \"removes\" selected joints from the skeleton data, simulating missing or occluded parts of the body, as shown in Figure 5. By doing so, the model is trained to recognize activities using only the remaining visible joints, encouraging it to focus on broader movement patterns rather than relying on specific joints. This enhances the model's robustness, improving its ability to generalize to real-world scenarios where sensor data might be incomplete or noisy.