###### Abstract  
Numerous methods and applications have been proposed in human activity recognition (HAR). This paper presents a mini-survey of recent HAR studies and our originally developed benchmark datasets of two types using environmental sensors. For the first dataset, we specifically examine human pose estimation and slight motion recognition related to activities of daily living (ADL). Our proposed method employs OpenPose. It describes feature vectors without effects of objects or scene features, but with a convolutional neural network (CNN) with the VGG-16 backbone, which recognizes behavior patterns after classifying the obtained images into learning and verification subsets. The first dataset comprises time-series panoramic images obtained using a fisheye lens monocular camera with a wide field of view. We attempted to recognize five behavior patterns: eating, regarding a smartphone, operating a laptop computer, and sitting. Even when using panoramic images including distortions, results demonstrate the capability of recognizing properties and characteristics of slight motions and pose-based behavioral patterns. The second dataset was obtained using five environmental sensors: a thermopile sensor, a CO\({}_{2}\) sensor, and air pressure, humidity, and temperature sensors. Our proposed sensor system obviates the need for constraint; it also preserves each subject's privacy. Using a long short-term memory (LSTM) network combined with CNN, which is a deep-learning model dealing with time-series features, we recognized eight behavior patterns: eating, operating a laptop computer, operating a smartphone, playing a game, reading, exiting, taking a nap, and sitting. The recognition accuracy for the second dataset was lower than for the first dataset consisting of images, but we demonstrated recognition of behavior patterns from time-series of weak sensor signals. The recognition results for the first dataset, after accuracy evaluation, can be reused for automatically annotated labels applied to the second dataset. Our proposed method actualizes semi-automatic annotation, false recognized category detection, and sensor calibration. Feasibility study results show the new possibility of HAR used for ADL based on unique sensors of two types.  
activities of daily living; behavior pattern; deep learning; human activity recognition; long short-term memory +
Footnote †: journal: Medical  
applications to a security service [7] and biometric authentication [8] in a public environment, vision-based systems have been proposed for identifying suspicious people based on characteristics of facial features, facial expressions [9], and body motions [10]. Moreover, development of a vision-based system [11] and an invisible sensor system [12] for bed-leaving detection and fall prevention at hospitals and nursing care sites has been reported. Such systems emphasize abnormality detection from the identification of behavior patterns.  
In human living environments, real-time recognition of abnormal behavior patterns related to falls, injuries, and collisions is set as a high priority for reasons of safety and security [13]. Physical body motions in our daily life are classified roughly into three categories: standing, sitting, and lying down [14]. Standing motions are used during cooking, cleaning, household chores of various types, etc. Sitting motions are used during meals, when reading, playing games, watching television (TV), listening to music, and operating a smartphone, tablet, or laptop computer, etc. Classification and recognition of daily life behavior patterns based on image analyses is regarded as a challenging task for computer vision studies because the range of motion of represented features is limited and often inclusive of vague or meticulous movements, especially when examining sitting motions. Moreover, numerous systems use wearable devices [15] with embedded acceleration and gyro sensors without using a camera. They emphasize privacy in private environments such as living rooms.