## 3 Materials and Methods
### Experimental Setup
These adjustments were crucial for managing memory, preventing overfitting, and improving the model's ability to capture temporal dependencies in the data. Similarly, TCN were optimized by varying the number of layers, filters, kernel sizes, and learning rates to effectively capture temporal patterns and ensure robust training dynamics. GRU models required adjustments in the number of GRU layers, units, dropout rates, optimizers, and learning rates. The GRU's simpler structure compared to LSTM networks makes it computationally efficient while still retaining the ability to capture essential temporal dependencies in sequential data. The Transformer model was fine-tuned by varying the number of layers, hidden units, attention heads, dropout rates, and learning rates. The Transformer's self-attention mechanism allows it to focus on relevant parts of the input sequence, making it effective in handling long-range dependencies and interactions within the activity data. Each model's parameters were chosen through nested cross-validation to ensure they were well-suited for the data and provided generalizable performance across different configurations. The inclusion of a range of traditional and deep learning models, including DT, KNN, RF, SVC, XGBoost, GRU, LSTM, TCN, and Transformer, provided a comprehensive exploration of diverse techniques, enhancing the robustness of our activity recognition framework. These hyperparameter optimizations were evaluated using a five-fold outer cross-validation and three-fold inner cross-validation approach, as shown in Table 5. This ensured an unbiased estimate of model performance while reducing the risk of overfitting. \begin{table}
\begin{tabular}{c l l} \hline \multicolumn{1}{c}{**Model**} & \multicolumn{1}{c}{**Hyperparameters**} & \multicolumn{1}{c}{**Values**} \\ \hline \multirow{4}{*}{DT} & criterion & [gini, entropy] \\  & max\_depth & [None, 10, 20, 30] \\  & min\_samples\_split & [2, 5, 10] \\  & min\_samples\_leaf & [1, 2, 4] \\ \hline \multirow{4}{*}{KNN} & n\_neighbors & [1, 3, 5, 7, 9] \\  & weights & [uniform, distance] \\  & \(p\) & [1, 2] \\ \hline \multirow{4}{*}{RF} & n\_estimators & [50, 100] \\  & criterion & [gini, entropy] \\  & max\_depth & [None, 4, 5] \\  & min\_samples\_split & [2, 5] \\  & min\_samples\_leaf & [1, 2] \\ \hline \multirow{4}{*}{SVC} & C & [0.1, 1, 10] \\  & kernel & [linear, rbf] \\  & gamma & [scale, auto] \\  & class\_weight & [None, balanced] \\ \hline \multirow{4}{*}{XGBoost} & learning\_rate & [0.01, 0.1] \\  & n\_estimators & [50, 100] \\  & max\_depth & [3, 6] \\  & min\_child\_weight & [1, 3, 5] \\  & gamma & [0, 0.1] \\  & reg\_alpha & [0, 0.1, 1] \\  & reg\_lambda & [0, 1, 10] \\ \hline \multirow{4}{*}{GRU} & GRU Layers & [1, 2, 3] \\  & GRU Units & [32, 64, 128] \\  & Dropout Rate & [0.2, 0.3,..., 0.5] \\  & Optimizer & [Adam, Adamax, RMSProp, SGD] \\  & Learning Rate & [0.0001, 0.001,..., 0.1] \\ \hline \multirow{4}{*}{LSTM} & LSTM Layers & [1, 2, 3] \\  & LSTM Units & [32, 64, 128] \\  & Dropout Rate & [0.2, 0.3,..., 0.5] \\  & Optimizer & [Adam, Adamax, RMSProp, SGD] \\  & Learning Rate & [0.0001, 0.001,..., 0.1] \\ \hline \multirow{4}{*}{TCN} & TCN Layers & [1, 2, 3] \\  & Number of Filters & [32, 64,..., 128] \\  & Kernel Size & [2, 3, 4] \\  & Optimizer & [Adam, Adamax, RMSProp, SGD] \\  & Learning Rate & [0.0001, 0.001,..., 0.1] \\ \hline \multirow{4}{*}{Transformer} & Transformer Blocks & [2, 4, 6, 8] \\  & Head Size & [8, 16,..., 256] \\  & Number of Heads & [2, 4,..., 16] \\  & FF Dim & [4, 8,..., 64] \\  & Dropout Rate & [0.1, 0.2,..., 0.6] \\  & Number of MLP Layers & [1, 2, 3] \\  & MLP Units & [32, 64,..., 256] \\  & MLP Dropout & [0.1, 0.2,..., 0.6] \\  & Optimizer & [Adam, Adamax, RMSProp, SGD] \\  & Learning Rate & [0.0001, 0.001,..., 0.1] \\ \hline \end{tabular}
\end{table} 
Table 4: Hyperparameters Used to Tune Nine Different Models for Optimal Performance.