# Real-Time Physical Activity Recognition on Smart Mobile Devices Using Convolutional Neural Networks
## 1 Introduction  
Human Activity Recognition (HAR) is an active area of research, since it can automatically provide valuable knowledge and context about the actions of a person using sensor input. Its importance is evidenced by the variety of the areas it is applied to, including pervasive and mobile computing [1; 2], context-aware computing [3; 4; 5], sports [6; 7], health [8; 9], elderly care [9; 10], and ambient assisted living [11; 12; 13]. In recent years, there has been a growing interest in the field, because of the increase of the availability of low-cost and low-power sensors, especially the ones embedded in mobile devices, along with the improvement of the data processing techniques. This growing interest can also be attributed to the increasingly aging population [14] in the case of health and elderly care applications. At the same time, several datasets have been collected over the years, including a range of modalities [15]. There has been a great contribution from human activity recognition video datasets; some of the most important include the 20 BN-something-something Dataset V2 [16], VLOG [17], and EPIC-KITCHENS [18]. However, using video datasets for activity recognition entails potential pitfalls. Compared to using inertial sensors, there is a greater risk of violating personal data; they require a camera setup or a person actively recording the video at runtime, and their processing is heavier. The ubiquity of mobile devices and their embedded sensors have produced a multitude of datasets, which include accelerometer, gyroscope, magnetometer, and ECG data, such as the MHEALTH [19; 20], the OPPORTUNITY Activity Recognition [21], the PAMAP2 [22], the USC-HAD [23], the UTD-MHAD [24], the WHARF [25], the WISDM [26], and the Actitracker [27] datasets. A wide range of sensors has beenalso used to create Adaptive Assisted Living (AAL) HAR datasets, including reed switches, pressure mats, float sensors, and Passive Infrared (PIR) sensors [28]. In the context of human physical activity recognition, competitive performance can also be achieved using only the input data from a mobile device's sensors. This offers potential unobtrusiveness and flexibility, but also introduces a number of challenges [29]. The motion patterns are highly subject dependent, which means that the results are heavily affected by the subjects participating in the training and testing stages. The activity complexity increases the difficulty of the recognition either because of multiple transitions between different motions or because of performing multiple activities at the same time. Energy and resource constraints should also be considered, since the capacity of mobile devices is limited, and demanding implementations will drain their battery. Localization is important to understand the context of a situation, but implementing it using GPS is problematic in indoor environments, while inferring distance covered with motion sensors usually results in the accumulation errors over time. Of course, it is also crucial to adapt research implementations to real-life problems, such as elderly care, workers' ergonomics, youth care, and assistance for disabled people. At the same time, the privacy of sensitive data collected from the subjects must always be respected and protected, even at the expense of a solution's performance. The conventional method to extract knowledge from these datasets is with machine learning algorithms such as decision tree, support vector machine, naive Bayes, and hidden Markov models [30]. They perform well in specific cases, but rely on domain-specific feature extraction and do not generalize well easily [31]. Contrary to machine learning methods, deep learning methods perform the feature extraction automatically, and specifically, Convolutional Neural Networks (CNN), which are also translation invariant, have achieved state-of-the-art performance in many such tasks [15].