# Context-Aware Complex Human Activity Recognition Using Hybrid Deep Learning Models
## 2 Literature Review
### Machine Learning for Complex HAR
In a related study conducted to detect anomalies in time series in [51], authors demonstrated the ability of the LSTM networks to maintain long-term memory and learn sequences of data containing long-term patterns of unspecified length. This capability of LSTM makes it a suitable candidate for the complex activity recognition system. Often, complex activities take longer to detect compared to their simple counterparts [15; 16]. LSTM models are well-suited for sequence learning as they allow to input sequence data into the network thereby helping to make more accurate predictions using the time steps in the sequence data [52]. By splitting the window sequences into subsequences, we can reuse the same CNN model when reading each subsequence of data separately. We achieved this by wrapping the CNN layers in a time-distributed wrapper and applying the entire model once per input subsequence. The extracted features are then flattened and fed to the LSTM model, extracting its features before a final mapping to activity. Apart from those early works, more recent works have explored the combination of an LSTM with a CNN to address the identified challenges resulting in a CNN-LSTM or ConvLSTM model [21; 22; 53; 54; 55; 56; 57; 58; 59]. CNN-LSTM hybrid models have been considered as effective in other domains apart from the HAR domain. For example, [PERSON] et al. [57] use the hybrid CNN-LSTM model to predict people's personality using user-generated contents and emojis. This combination of CNNs and RNNs is an emerging methodology adopted lately by researchers to replace the previous conventional independent learning with multiple related activities. One such work by [53] implemented the hybrid model: a combination of CNN- and RNN-based LTSM recurrent layers to classify activities from Kinect v2 sensors. Although they conclude that the combination of both CNN and LSTM produced better results than individual deep neural networks, human activity recognition using videos is considered intrusive. In [55], the authors present a novel approach using a combination of both the CNN and LSTM by including an attention mechanism by merging the output of the upstream layer and then filtering the important representation by redistributing the weights of feature representations. However, this work did not consider the importance of context information as we did in our proposed solution. Another recent and similar work is presented by [PERSON] et al. [56] where multilayer features were extracted from sensory data using a hybrid of the CNN and BLSTM networks. This is a similar work in that the sensory data were infused with location information to enhance recognition accuracy. However, we used two forms of context information, namely light conditions, and environmental noise levels, to enhance the recognition accuracy of hybrid HAR models. The results obtained by these recent works conclude that combining the CNN network with the LSTM network can achieve higher generalization accuracy than just using either of them alone. These works provide the basis for the research although not one of these studies explores the combination of rich contextual information provided by the smartphone ambient sensors to augment the inertial sensing signals. The work presented in this article is unique in that it tried to improve recognition accuracy significantly by exploring these contextual dependencies. More specifically, the proposed solution only uses sensor data from smartphones as opposed to other models that require multiple wearable and nonwearable sensors, such as video cameras, which are far from ideal and quite intrusive.