A Thermal Camerabased Activity Recognition Using Discriminant Skeleton Features and RNN Recognizing human activities from sensor data is one of the key areas of image processing, computer vision, and pattern recognition researches today. The target of human activity recognition (HAR) is usually to detect and analyze distinguished activities from the data acquired via different sensors (e.g. thermal cameras). This work proposes a HAR approach from videos recorded via a thermal camera. Skeletons of human bodies are extracted from thermal frames using an opensource deep convolutional neural network (CNN)based approach named OpenPose. It is generally applied on videos of typical color cameras. However, this work adopts OpenPose on thermal images to extract useful features so that the HAR system can be deployed in environments with low lights as well. Once skeletons of human silhouettes are obtained from the thermal images, robust spatiotemporal features are extracted followed by discriminant analysis. Finally, the discriminant features are fed into a deep recurrent neural network (RNN) for activity training and recognition. The proposed HAR method can be applied to monitor the users such as elderly in both bright and dark environments to prolong their independent life, unlike other typical color cameras which are generally applied in bright environments.

Smart Phone Based Human Activity Recognition Human Activity Recognition (HAR) is a field that uses collected data to classify different human actions. One simple and general approach to HAR is to use the sensor data from a mobile device to recognize different patterns behind complex motions. Early studies show promising results on simple activities using manually selected features from accelerometer readings. As newer publicly available datasets include more complex data and activities, manual feature selection have become cumbersome, impractical and face limitations in finding the optimal feature sets for HAR. In this paper, we present an empirical approach to defining models of 3D tensor data structures from 2D time series data obtained from multiple sensors on a smart phone, and a new Convolutional Neural Network (CNN) model, which uses the tensor data and performs automatic feature extraction and classification for HAR. We use the public benchmark dataset, MobiAct v2.0, to train and validate our model, which achieved an overall better performance in classifying 11 Activities of Daily Living (ADL) than the stateoftheart approaches. Compared to the approach presented by Chatzaki et al. which has a very high rate of misclassifications for carstep out (CSO), carstep in (CSI), sit to stand (CHU), and stand to sit (SCH) classes, our proposed approach has 15% higher sensitivity for each of these activities with the optimal number of training epochs being only 25.