# Comparing Human Activity Recognition Models Based on Complexity and Resource Usage
## 2 Related Work
### Deep Learning  
Compared to typical machine learning methods, deep learning methods can train more complex models to model more complex data. With ML models, the practitioner needs to help it understand the data by extracting more representable features. Deep models on the other hand can have so many trainable parameters through their different layers and various amounts of units to be able to understand raw data. The more trainable parameters a deep model has, the more complex data it can represent. The tradeoff between overfitting and underfitting data with a model is present for both approaches, but for deep models, the focus can shift from applying domain knowledge and extracting the most useful features to balancing that tradeoff by using different architectures, learning parameters and regularization methods. Given enough data, deep models outperform typical models like support vector machines in many problem domains. This section will highlight approaches for the domain of HAR using various architectures for deep models. As mentioned in the introduction, finding the best performing architecture and hyperparameters is key to modeling complex data and still achieving good recognition performances. [PERSON] et al. [15] focus on this exact problem by evaluating several architectures with ranges of hyperparameters regarding learning, regularization and architectural constraints. Their chosen architectures consist of a deep feed forward neural net (DNN), a CNN and three different RNNs. For the RNNs, they use Long Short Term Memory (LSTM) setups denoted LSTM-F, LSTM-S and b-LSTM-S. The LSTM-F and LSTM-S are deep feed forward LSTMs with recurrent cells only connected forwards in time whilst the b-LSTM-S is connected forwards and backwards using two parallel recurrent layers that are followed by a layer to combine them. The second distinguishing feature of those LSTMs is the way the input is fed into them. With the LSTM-F using predefined timespans of sensor readings and the LSTM-S and the b-LSTM-S only using one single sensor reading at a time [15]. [PERSON] et al. [15] use fANOVA analysis to estimate the impact each of architectural, learning, regularization parameters and beneficial interactions between those three on the performance of the model. The hyperparameters for each trained model were chosen at random. On average, architectural parameters and learning parameters are equally important whilst the regularization, to reduce overfitting, is less important. But a very significant portion of the variance comes from beneficial interactions of certain parameters in combination with each other, which indicates that focusing on one single type is not as important as finding the best combination of different kinds of parameters [15]. In contrast to the work by [PERSON] et al. [15], most of the research focuses on one single deep learning approach and one or more datasets to evaluate its performance. [PERSON] and [PERSON] [16] for instance use a CNN on the dataset also used in our work. They propose CNNs as a good solution for HAR due to their ability to exploit local temporal relations within time-series signals and cancel out smaller translations in the sensor readings. They report a peak performance of 94.79% accuracy for a model consisting of three convolutional and pooling layers followed by a fully connected layer and a softmax layer. They also report their evaluation results of the performance increase received by changing the number of convolutional layers, the kernel size of the convolutional layers and the pooling size of the pooling layers. While the pooling size had no effect on performance, the kernel size shows major performance increases up to a kernel size of 9. For the input of the complex raw sensor data, the amount of convolutional layers is quite important. Going from one to two convolutional layers, the performance increases drastically, with a small improvement available when adding a third layer.