## 1 Introduction and Background  
[PERSON]'s initial vision of ubiquitous computing foresaw computers seamlessly integrating into daily life, operating inconspicuously to enhance human experience without intrusion [1]. Today, this vision has materialized, with computers seamlessly incorporated into personal smart devices. Pervasive computing, a key aspect, achieves non-intrusive functionality by allowing computers to detect and respond to user actions implicitly based on their environment. The emergence of wearable sensor devices represents a significant development in the ongoing activity recognition research landscape, enabling individuals to monitor their physical activity seamlessly. Smartphones, smart watches, or bands are the most common wearable sensors. They are embedded with motion sensors such as accelerometers, gyroscopes, and magnetometers. Accelerometers gauge the acceleration of an object for the rate of change in its velocity, measured in meters per second (m/s). Gyroscopes determine the orientation and angular velocity, measured in degrees per second (\({}^{\circ}\)/s), and magnetometers measure the magnetic field strength at a specific position, denoted in tesla units (T) [2]. The activity recognition chain [3] compounds the framework for activity classification applied to machine learning. It summarily involves data reading from sensors, preprocessing, segmentation, feature selection, and classification as can be seen in Figure 1. While most HAR studies rely on this protocol, its effectiveness can be stymied by validation methods, which are techniques used to estimate how well they will generalize to unseen data by evaluating them on different subsets of the available data [4]. Studies in HAR have typically reported high accuracies [5; 6]. They have all also been evaluated using questionable k-fold cross-validation. K-fold is a common validation approach in HAR systems, and it achieves high accuracies in generalization [7]. The approach randomly splits the data into training and testing subsets, repeating the process in the specified \(k\) times. For systems that are trained on participants' continuous data, this can cause a leak, where participant samples exist in both training and testing sets, leading to classification accuracy, as these data have already been seen [8; 9]. Ensuring trained models are tested on unseen samples should be the true test of their predictive accuracy. This work investigates the impact of validation methodologies on classification algorithms for HAR systems in wrist-worn wearables. It analyzes how different HAR models trained on feature vectors generalize to k-fold cross-validation and LOSO. The performance of models trained on raw data will also be examined. In summary, this study examines the generalization performance of HAR systems trained on extracted features and validated on k-fold and LOSO cross-validation. We highlight the inappropriate splitting and data leakage in k-fold cross-validation leading to higher accuracies over LOSO and compare the testing performance of HAR systems trained on raw data to those trained on extracted features, validating the importance of feature vectors. We propose a combination of training models on extracted feature vectors as well as validating with LOSO for better-generalized HAR systems. Background  
Previous studies have highlighted the inappropriateness of the traditional train-test split and k-fold cross-validation. Ref. [10] emphasized the tendency for clinical studies that utilize data segmented from the same subject data and employ k-fold cross-validation to frequently overestimate the accuracy of predictions. They suggest the possibility that overlapping frames likely capture identical activity within a similar context; consequently, these adjacent segments are highly correlated. Ref.