## 3 Vision-Based Approach Using a Fisheye Lens Monocular Camera  
### Proposed Sensor System and Recognition Method  
Figure 1 depicts procedures used for our first proposed method. The procedures are divided roughly into three steps. Time-series panoramic images are obtained using a fisheye lens camera in the first step. For this measurement, we fixed the camera at a position \(x\) m distant from a subject. The obtained images were downsampled to \(n\) fps with a linear transformation to reduce the computation time and memory size.  
The second step uses OpenPose [190] to extract feature vectors of pose motions. Herein, human two-dimensional (2D) pose estimation represents a challenging task for computer vision studies. Numerous methods [191; 192; 193; 194; 195; 196; 197; 198; 199] have been proposed to detect 2D poses from multiple people in a single image. Based on an earlier study reported by [PERSON] et al. [200], we used OpenPose [190] because of its simple and easy implementation, high-speed processing, and high estimation accuracy. Regarding the accuracy verification of OpenPose, [PERSON] et al. [201] employed 17 inertial sensors to evaluate 12 motion tasks: upright standing, trunk flexion 30\({}^{\circ}\), sitting on a stool trunk flexed and rotated, placing an object on top, kneeling above the head, arms crossed, holding a box, legs crossed, sitting at the desk, simple lifting, and complex lifting.  
OpenPose [190] employs a bottom-up representation of association scores via part affinity fields (PAFs) [202]. A set of 2D vector fields is encoded for the location and orientation of limbs over an image domain. To detect body parts, confidence maps are predicted from two-branch CNN. As an open library, OpenPose was trained using two benchmark datasets for multi-person pose estimation: The Max Planck Institut Informatik (MPII) human pose dataset [203] and the common objects in context (COCO) key-points challenge dataset [204]. For our implementation, we used the OpenPose library, which was pre-trained using the latter dataset.  
As examined for behavior pattern classification and recognition used for labeling, we used the VGG-16 [159] backbone, which comprises 13 convolutional layers and three fully connected layers. The original VGG-16 was designed as a deep architecture to use \(3\times 3\) convolution filters for each pixel. Using feature vectors produced from OpenPose, we applied transfer learning [205] to the upper layers of VGG-16, which were trained in advance using ImageNet [206]. Although ResNet [161] and its derivative models such as WideResNet [207], ResNeXt [208], Res2 Net [209], and Inception-resnet (Inception-v4) [210] have already been proposed, we used the VGG-Net backbone because of our overall considerations of GPU time, speed, and memory consumption [211]. Actually, CNN backbones can be switched easily according to specifications and applications.  
Figure 1: Comprehensive system architecture of the proposed procedure.  
Table 3 presents the major parameters of VGG-16. Based on our earlier study [212], the number of generations, batch size, and validation split were set, respectively, to 50 epochs, 4, and 0.2. The number of the input layer corresponds to the resized image size: 224 pixels \(\times\) 224 pixels \(\times\) 3 channels. The numbers of the output layer were set to two, three, and five, according to the number of target categories. For the optimization algorithm, we used Adam [213], which is an optimizer combined with Momentum [214] and RMSProp [215].