## 6 Results and Discussion
### Performance Evaluation Using K-Fold Cross-Validation Training and Subsequent Compression  
Next, we present the performance analysis of the six architectures on four datasets, namely, MHEALTH (Table 13), USCHAD (Table 14), WHARF (Table 15), and WISDM (Table 16), using cross-validation training with the subsequent methods of compression discussed in Section 5.2. The hyperparameters for training and compression were tuned as discussed in Sections 5.1 and 5.2. We used the scikit learn library, which has kfold.split method, to perform the five-fold validations to obtain the training and validation sets for each fold in an iterative manner. We report the mean train accuracy, mean validation accuracy across five folds, test accuracy, F1 score (normalized %), and model size (kilobytes) for this set of experiments. We chose 16 hidden units for this set of training experiments since 8 was small and 32 was large for running a five-fold training. For this method, two kinds of data generation methods were employed, namely FNOW and SNOW. Model sizes were the same for both data generation schemes and are thus reflected as a common column (Col. 6) in the next four tables (Tables 13-16). \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**RNN Cell**} & \multirow{2}{*}{**Hidden Units**} & \multicolumn{6}{c}{_Opportunity_} & \multicolumn{6}{c}{_Pamap2_} \\ \cline{3-10}  & & & **Test Acc. (\%)** & **F1 Score** & **Train Time (min)** & **Model Size (KBi)** & **Test Acc. (\%)** & **F1 Score** & **Train Time (min)** & **Model Size (KBi)** \\ \hline Basic RNN & 8 & 0.84 & 0.25 & 12.14 & 6.70 & 0.45 & 0.39 & 6.46 & 2.83 \\ \hline  & 16 & 0.86 & 0.34 & **4.19** & 8.70 & 0.60 & 0.52 & **8.91** & 3.98 \\ \hline  & 32 & 0.87 & 0.38 & 9.27 & 12.70 & 0.59 & 0.54 & 6.31 & 6.30 \\ \hline FastGRNN & 8 & 0.85 & 0.29 & 13.25 & 3.92 & 0.71 & 0.64 & 8.69 & **2.99** \\ \hline  & 16 & 0.86 & 0.34 & 21.71 & 5.30 & 0.73 & 0.65 & 7.48 & 4.59 \\ \hline  & 32 & 0.86 & 0.42 & 14.89 & **8.05** & 0.73 & 0.67 & 8.05 & 6.93 \\ \hline FastRNN & 8 & 0.86 & 0.33 & 15.99 & 4.70 & 0.57 & 0.63 & 7.72 & 4.96 \\ \hline  & 16 & 0.85 & 0.39 & 15.77 & 6.23 & 0.62 & 0.65 & 9.21 & 5.55 \\ \hline  & 32.00 & 0.87 & 0.44 & 18.66 & 9.29 & 0.69 & 0.67 & 8.49 & 8.62 \\ \hline UGRNN & 8 & 0.85 & 0.34 & 23.32 & 4.41 & 0.75 & 0.69 & 12.65 & 4.08 \\ \hline  & 16 & 0.86 & 0.38 & 14.36 & 6.29 & **0.81** & 0.77 & 9.48 & 6.08 \\ \hline  & 32 & 0.86 & 0.44 & 26.49 & 8.77 & 0.69 & 0.66 & 8.49 & 8.62 \\ \hline GRU & 8 & 0.86 & 0.37 & 19.45 & 7.05 & 0.69 & 0.63 & 10.24 & 4.73 \\ \hline  & 16 & **0.87** & 0.42 & 31.89 & 10.34 & 0.78 & 0.72 & 12.02 & 7.39 \\ \hline  & 32 & **0.87** & **0.47** & 30.15 & 16.90 & 0.80 & **0.80** & 22.82 & 11.55 \\ \hline LSTM & 8 & 0.86 & 0.39 & 20.72 & 5.48 & 0.67 & 0.59 & 19.78 & 4.42 \\ \hline  & 16 & **0.87** & 0.46 & 16.37 & 10.16 & 0.57 & 0.50 & 16.51 & 7.17 \\ \hline  & 32 & 0.86 & 0.46 & 26.24 & 17.16 & 0.59 & 0.53 & 19.28 & 12.67 \\ \hline  & Best Performance & \multicolumn{6}{c}{GRU, LSTM with 1632 hidden units} & \multicolumn{6}{c}{GRU with 32 hidden units} \\ \hline Smallet Model & \multicolumn{6}{c}{FGRNN} & \multicolumn{6}{c}{FRNN, FGNN, BasicRNN} \\ \hline Memory Savings via Fast Gates & \multicolumn{1}{c}{2.09\(\times\) with respect to GRU32, 1.26\(\times\) with respect to LSTM16} & \multicolumn{1}{c}{2.03\(\times\) with respect to UGRNN16, 3.86\(\times\) with respect to GRU32} \\ \hline Performance Deviation via Fast Gates & \multicolumn{1}{c}{up to 5\%} & \multicolumn{1}{c}{around 13–16\%} \\ \hline \hline \end{tabular}
\end{table} 
Table 12: Performance of Recurrent Neural Networks across Opportunity and PAMAP2 datasets with 8, 16, and 32 hidden units. Numbers in bold font represent the best results under each column, numbers in blue font represent figures that deviate from the best results under each column. F1 scores are low for the Opportunity dataset as it is highly unbalanced.