# Context-Aware Complex Human Activity Recognition Using Hybrid Deep Learning Models
###### Abstract  
Smart devices, such as smartphones, smartwatches, etc., are examples of promising platforms for automatic recognition of human activities. However, it is difficult to accurately monitor complex human activities on these platforms due to interclass pattern similarities, which occur when different human activities exhibit similar signal patterns or characteristics. Current smartphone-based recognition systems depend on traditional sensors, such as accelerometers and gyroscopes, which are built-in in these devices. Therefore, apart from using information from the traditional sensors, these systems lack the contextual information to support automatic activity recognition. In this article, we explore environmental contexts, such as illumination (light conditions) and noise level, to support sensory data obtained from the traditional sensors using a hybrid of Convolutional Neural Network and Long Short-Term Memory (CNN-LSTM) learning models. The models performed sensor fusion by augmenting low-level sensor signals with rich contextual data to improve the models' recognition accuracy and generalization. Two sets of experiments were performed to validate the proposed solution. The first set of experiments used triaxial inertial sensing signals to train baseline models, while the second set of experiments combined the inertial signals with contextual information from environmental sensors. The obtained results demonstrate that contextual information, such as environmental noise level and light conditions using hybrid deep learning models, achieved better recognition accuracy than the traditional baseline activity recognition models without contextual information. simple activities; complex activities; context awareness; deep hybrid learning; sensors; smart devices; human activity recognition +
Footnote †: journal:  
signals of dissimilar activities exhibit similar patterns. Because of these similar signal patterns, discriminating between simple and complex activities becomes increasingly more difficult for HAR systems. To address the interclass similarity problem, not just the use of more sensory data is required but also the use of algorithms capable of discriminating patterns in these activity signals, including the consideration of contexts in which such activities occur [11]. To emphasize the interclass problem in HAR and other related problems, let us consider the following scenarios to illustrate typical situations where a mix of simple and complex activities can be misclassified if they exhibit interclass similarity without considering contextual information. A driver, for example, might be distracted by a personalized mobile content recommendation because a model confuses _driving_ with _running_, where both activities involve movements. Another example is _lying in bed_ and _sunthathing at the beach_. These are two examples of activities that involve _lying down_, but the latter is a complex activity typically involving a substantial amount of sunshine (illumination) and sometimes the level of noise/sound in a specific location, usually on a beach. A final example is _walking_ and _mountain climbing_. Again, these are two activities that are quite similar but contextually different. _Mountain climbing_ involves _walking_, but at the same time, the contexts are quite different. Usually, factors such as high altitudes are associated with low temperature and humidity. Therefore, to effectively discriminate between these similar activities, information, such as spatial-temporal and environmental contexts could be exploited. The examples above demonstrate clearly that simple and complex activity recognition from sensor data does pose significant challenges. First, the underlying activities of a complex activity are mostly dependent [14]. Particularly, in one complex activity, the temporal relatedness of several simple activities often manifests in different forms. Consequently, the complicated temporal combinations often have semantic issues for understanding a complex activity [15].