# Comparing Human Activity Recognition Models Based on Complexity and Resource Usage
## 5 Discussion
### Performance  
Looking at the performance of the best models of each model type depicted in Table 12, a few interesting points become apparent. The first is that the best performing model in terms of F1-Score or accuracy is not one of the deep neural nets, but one of the machine learning models, the SVM with RBF kernel with an F1-Score of 96.02% followed closely by the SVM with linear kernel with 95.62%. Whereas the deep neural nets only achieve an F1-Score of 91.98% for the CNN and 90.34% for the RNN. Of course, considering that the machine learning models need highly engineered features in contrast to the deep neural nets, they can still achieve very good performances with just the preprocessed sensor data. What also can be seen is that the KNN works very well (F1-Score of 92.07%) considering its simple approach of using the closest available samples to predict a new sample. Unfortunately, with this simple approach also comes a very big drawback with its high memory usage (see Section 3.1.2). Interestingly, the Random Forest is not really able to keep up with the performances of the other models. We attribute this to the way trees can only split very specific ranges in a node which in turn makes it hard for the different trees to split the very unstructured data we have also seen in the data analysis section (see Section 3.3.3). Of course, trees could perfectly learn to model the training data in this way and overfit, but generalizing well for this kind of data seems difficult. Coming back to the results of the deep neural nets seen in Table 12, where our CNN (F1-Score of 91.98%) performs better than our RNN (F1-Score of 90.34%) on this particular dataset. But the difference is small and some of it could also be attributed to the fact that the training of RNNs takes much longer than the training of CNNs, which in turn limited the amount of hyperparameter tuning due to the limited resources in Google Colab. Another reason could be that the relevant relations and patterns of values within a sample are better identified using convolutions with different kernels of size 9 than what the LSTM layers are able to find with their long and short term relations. Would we now need to decide on one model to use in production purely based on these performances, then we would choose one of the SVMs as they show the best performance. However, instead of just going with the best model in terms of performance, \begin{table}
\begin{tabular}{l c} \hline \hline
**Hyperparameter** & **Value** \\ \hline Number of Epochs & 10 \\ \hline Batch Size & \(2^{7}\) \\ \hline Optimizer & RMSprop \\ \hline Loss Function & Categorical Crossentropy \\ \hline Number of Units (LSTM Layer 1) & \(2^{7}\) \\ \hline Number of Units (LSTM Layer 2) & \(2^{7}\) \\ \hline Dropout 1 & 0.0 \\ \hline Dropout 2 & 0.2 \\ \hline Number of Units (Dense Layer 1) & \(2^{7}\) \\ \hline Activation Function (Dense Layer 1) & ReLU \\ \hline Activation Function (Dense Layer 2) & Softmax \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Used hyperparameters for the training of the best RNN. the SVM with RBF kernel, we would choose the SVM with linear kernel. This is due to the fact that the linear SVM only shows a marginal decrease in the performance, but when looking at the hyperparameters, the SVM with RBF kernel has a much higher cost (64 compared to 3 for the linear SVM) and also has a very low gamma, which should make it more likely to overfit than the linear SVM. To put the results of this work in perspective, Table 13 shows a set of results of related work on the exact same dataset. Looking at the SVM by [PERSON] et al. (2018), the performance is nearly the same with them reporting an accuracy of 96% in comparison to our achieved performance of the two SVMs of around 95.6% to 96%. Unfortunately, there is no mention of the used kernel or any of the other available hyperparameters and therefore only the performance without further context can be related. For our best performing deep neural net architecture, the CNN, there are two data points to relate to.