# Real-Time Physical Activity Recognition on Smart Mobile Devices Using Convolutional Neural Networks
## 4 Results
### Performance Evaluation on the WISDM Dataset  
#### 4.4.1 Original Train/Test Split  
In Table 8, the accuracy scores of the two models for a window size of 50 and 200 data points respectively are presented after running the training and testing of the original paper. More specifically, the first 26 users were used as a training set, and the remaining 10 were used as a test set. It is evident that although the parameters of the model are considerably fewer than those of the reference one, it not only matches, but surpasses the accuracy of the reference model from 0.35% to 1.04%, for window sizes of 50 and 200, respectively. The mean and standard deviation of the accuracy for the DCNN model were calculated by running 10 trainings with different initializations. There is no standard deviation--or average accuracy--for the RCNN model, since for this comparison, we used the results published in their paper [35]. We can see that the standard deviation is quite small and ensures that the accuracy is reliably higher for the DCNN model.  
In Figure 2, there is an overview of the precision scores of both models for every class. Both models struggle mostly with recognizing the Upstairs and Downstairs activities, which was expected, but DCNN manages to achieve considerably higher precision in both of these activities. It is, however, less precise in identifying the Standing activity.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **Throughput (I/s)** & **Model Size (MB)** & **Parameters (M)** \\ \hline  & **50** & **200** & **50** & **200** & **50** & **200** \\ \hline RCNN [35] & 206.88 & 52.283 & 10.7 & 40.4 & 2.667 & 10.092 \\ DCNN & 405.80 & 115.27 & 2.2 & 5.5 & 0.547 & 1.383 \\ \hline \hline \end{tabular}
\end{table}
Table 7: On-device throughput and size of the models.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Average Accuracy** & **Standard Deviation** \\ \hline  & **50** & **200** & **50** & **200** \\ \hline RCNN [35] & 90.42\% & 93.32\% & - & - \\ DCNN & 90.77\% & 94.36\% & 0.12\% & 0.11\% \\ \hline \hline \end{tabular}
\end{table}
Table 8: Average accuracy scores and their standard deviation on the original train/test split of the WISDM dataset.  
Similar conclusions can be also deduced from the results of Figure 3, the recall scores. The performance is considerably better in the Downstairs and Upstairs activities, while slightly lower recall in the Standing one. The recall is also better in the Walking activity.  
The f1-score results in Figure 4 validate the above findings, since the score is higher for the Downstairs, Upstairs, and Walking activities, while lower for the Standing one. This means that DCNN achieves better results for the dynamic activities. This probably happens because they are periodic, and the two-layer structure offers a wider receptive field.  
Figure 4: F1-scores on the original train/test split of the WISDM dataset.  
Figure 3: Recall scores on the original train/test split of the WISDM dataset.  
Finally, we can see the confusion matrices for window sizes of 50 and 200, respectively, in Table 9. The model can easily distinguish between stationary and dynamic activities, except for a small confusion between the Upstairs and stationary activities. This is reduced when using a wider window size, which is expected since it ensures that the whole periodic pattern of the dynamic movement is included in the window multiple times, and thus, the difference between the stationary (and non-periodic) and dynamic activities is more pronounced. There is naturally a common misclassification of Standing and Sitting activities between each other. Surprisingly, Downstairs is more usually confused with Walking and vice versa and not the Upstairs activity. Jogging is mostly confused with the Walking activity, but it is rarely confused. Using a window size of 200 helps decrease the confusion among all of the classes, except of the one between the stationary activities.