# Comparing Human Activity Recognition Models Based on Complexity and Resource Usage
## 4 Results
### Deep Learning
#### 4.2.1 Hyperparameter Evaluation  
As a short recap, the architecture of the CNN we saw in Table 6, consists of two one dimensional convolutional layers each followed by a dropout layer and a max pooling layer. In the end, a flattening layer puts everything in one dimension and two dense layers reduce to the output shape of our six labels. Figure 10 shows the mean performances of models, trained with a certain hyperparameter, with its standard deviation on the test data. Starting with the plot for num_units_conv1, the number of units of the first convolutional layer, it can be seen that there is a trend towards more units providing better peak performance. Interestingly, for the value of 64, the models tend to have a lower spread of performances visible in the lower standard deviation bars. Leading to the conclusion that for this number of units, the tradeoff between underfitting and overfitting is the best as there are fewer outlier performances and the models seem to consistently perform well. For the number of the units of the second convolutional layer num_units_conv2 this general trend of increasing peak performances is also visible. In this case, the value of 128 seems like the best available tradeoff as it performs better than the lower values but does not bring too much more complexity to the model that could make it overfit more. The same value also seems favourable for the number of units of the dense layer, num_units_dense1. The two dropout layers that should reduce the possibility for the CNN to overfit by randomly switching off a certain part of the neurons also display interesting results. The two parameters controlling the rate at which each of the layers switches off the neurons, dropout1 and dropout2, clearly show that adding for example a dropout of 0.2 or 20% allows models to be able to generalize better in some cases but unfortunately also to perform worse in some other cases. In order to be sure not to overfit the data, we see the value of 0.2 for both dropouts as the most favourable. Another parameter that tries to limit overfitting is the kernel regularizer for the first dense layer, _kernel_regularizer_dense1_. In contrast to the dropout, in this case, the potential of the regularizer does not seem to warrant the use as the models tend to get more worse than they get better, which is why it is not used going forward. For the kernel size of the two convolutional layers, kernel_size_conv1 and kernel_size_conv2, that affect how many values are considered for each step of the convolution, the bigger value of 9 is favourable even though it does not have that much impact. Lastly, the pool sizes of the max-pooling layers, pool_size_maxpooling1 and pool_size_maxpooling2. They do not have any effect and therefore 2 is used going forward. To tune the RNN architecture, as previously explained in Table 7, the five hyperparameters visible in Figure 11 have been evaluated as they are expected to change most of the behaviour. The leading two hyperparameters in terms of influence on the performance of the models are num_units_lstm1 and num_units_lstm2, the amounts of units of the two LSTM layers. For both of these parameters, there is a very interesting trend visible at the highest evaluated value of 512, where the trained models have the potential for very high high hits but also for very low lows as can be seen by the comparably lower mean values and the comparably high standard deviations. This indicates that the models using these higher values tend to be too complex to generalize well enough as they can accidentally perform very well for the test set or just the opposite. Given this fact, we view 128 as the most promising value for the number of units of both LSTM layers. In order for the model to be less likely to overfit, two dropout layers that disable a certain rate of the neurons of the previous layers have been added after each of the LSTM layers.