# Exploring Cutout and Mixup for Robust Human Activity Recognition on Sensor and Skeleton Data +
## 3 Methodology  
Our proposed method applies both sensor-based and skeleton-based Human Activity Recognition (HAR) models, incorporating mixup and cutout data augmentation techniques. The datasets used in this study consist of time-series sensor data collected from devices such as accelerometers and gyroscopes, and skeleton data extracted from visual sensors like RGB cameras or depth sensors. Each dataset contains sequences of sensor readings from subjects performing various activities for sensor-based HAR, while skeleton-based HAR tracks the positions of important body joints over time. The continuous sensor and skeleton data are segmented into fixed-length windows to create consistent input samples for the models, with each segment labeled according to the corresponding activity. This research applies the mixup data augmentation technique for time-series sensor data to address data scarcity and improve model robustness. Mixup generates new training samples by linearly interpolating between pairs of existing sensor data samples and their corresponding labels, enhancing the diversity of the data. For skeleton-based HAR, we employ the cutout data augmentation technique, which involves masking random parts of the skeleton data to improve model robustness by forcing it to focus on incomplete or missing joint information, ultimately enhancing the generalization capabilities of the model. Building on this approach, we first apply the mixup data augmentation technique to the sensor-based HAR data, which generates new training samples by blending existing data points and their labels. For each training iteration, pairs of time-series samples \((X_{i},\,y_{i})\) and \((X_{j},\,y_{j})\) are randomly selected from the training dataset. A mixing coefficient \(\lambda\;\epsilon\;[0,\;1]\) is sampled from Beta distribution \(\lambda\;\sim\;Beta(\alpha,\;\alpha)\). The selected pairs are linearly combined to generate new samples:  
\[\widetilde{X}=\lambda X_{i}+(1-\lambda)X_{j} \tag{1}\]  
\[\widetilde{y}=\lambda y_{i}+(1-\lambda)y_{j} \tag{2}\]  
Here, \(X_{i}\) and \(X_{j}\) are time-series matrices of the shape \((T,\;C)\), where \(T\) is the sequence length and \(C\) is the number of sensor channels. This implies \(X_{i},\;X_{j}\in\mathbb{R}^{T\times C}\) and \(y_{i},\;y_{j}\in\mathbb{R}^{K}\), where \(K\) is the number of activity classes. The labels \(y_{i}\) and \(y_{j}\) are one-hot encoded vectors representing the activities. The graphs in Figures 1 and 2 display sensor readings from a human activity sample before (Figure 1) and after (Figure 2) applying the mixup technique. The data are presented across six separate channels, each representing different sensor readings over time. The x-axis of each plot denotes time, while the y-axis indicates the sensor reading values. For each channel, the signal exhibits a combination of patterns derived from two different samples, reflecting the linear interpolation characteristic of mixup. The mixup augmentation technique blends the time-series data from two separate activities, creating synthetic samples that enhance the training dataset's diversity. The representation of each class after dimensionality reduction with t-SNE [59], similar to [22], can be seen in Figure 3. We can see in the figure that mixup provides additional data points to improve model generalization and provide additional data points for classes with a lower number of samples. We employ transformer-based models proposed by [PERSON] et al. [22] and by [PERSON] [60]. Their transformer architecture is a customized form of the Vision Transformer [54], with some modifications to adapt it to HAR data. For the sake of clarity,  
Figure 3: Human activity sample from RealWorld dataset after mixup. Figure 2: Human activity samples from RealWorld dataset before mixup. we simplify the model architecture and explain its components below.