# Discrete HMM for Visualizing Domiciliary Human Activity Perception and Comprehension
## 2 Human Activity Comprehension Framework
### Feature Extraction  
The RGB-D sensor has built-in functions for 3D positions of 25 skeleton joints, including head, neck, spine-shoulder, shoulder, spine-mid, elbow, spine-base, wrist, hand, hand tip, thumb, hip, knee, ankle, and foot for the human profile shown in Figure 2. Assume the real-world 3D positions of each skeleton joint, \(i\), can be represented by the camera reference system for each frame, \(t\), of a sequence shown in Equation (1) as:  
\[j_{i}(t)\ =\ (x_{i}(t),\,y_{i}(t),\,z_{i}(t)) \tag{1}\]  
The original point (\(x_{i}=0,y_{i}=0\), and \(z_{i}=0\)), in three dimensions, is located at the center of the binocular camera, wherein \(x_{i}\) is pointed toward the sensor's left, \(y_{i}\) is pointed up, and \(z_{i}\) is pointed in front of the sensor. Let \(N_{i}\) be the number of skeleton joints, which consists of the posture of the skeleton at frame \(t\) in \(3N_{i}\) dimension, as represented in Equation (2):  
\[u(t)\ =\ \left[j_{1}(t)j_{2}(t)j_{3}(t)\ldots\ldots j_{N_{j}}(t)\right] \tag{2}\]  
For an activity sequence composed of \(N_{f}\) frames, \(N_{f}\) feature vectors are extracted that can be built as a feature matrix for the whole sequences as Equation (3):  
\[A^{mat}\ =\ \left[u(1),u(2),\ldots\ldots,\ u\left(N_{f}\right)\right] \tag{3}\]  
Then, this matrix is represented as the variations of the joint positions over time. Each size of the feature matrix is defined as \(3A_{i}^{mat}\times A_{f}^{mat}\). To effectively apply the joint information for pose detection, our findings show that only 10 of the 25 joints are sufficiently beneficial for the proposed work, owing to the intrinsic noise of the sensor and the peculiarities of the human body, particularly, the  
Figure 2: Twenty-five defaulted joints of black-or-white circle points are provided by a third-party sensor in **(a)**, Only ten white circle points were selected for the relative position connection and distance calculation by nine red lines shown as D1, D2...... D9, from centroid (Spine Mid) to the other 9 joints in **(b)**, The remaining 15 unselected points labeled on the black circles points show no affection during activity detection in **(b)**. remaining unselected joints which are trivial, reflecting the dynamic variation of the activity. For example, the poses of some trivial joints are redundant (e.g., wrists and ankles) due to adjacent joints (e.g., hands and feet), in addition, some joints are completely irrelevant for human activity recognition (i.e., neck and hip). Therefore, the remaining set of joints are chosen for human activity recognition and categorized as: head, spine-mid, elbows (left and right), hands (left and right), knees (left and right), and ankles (left and right). The set \(J\) is defined as Equation (4):  
\[J\ =\ \{j_{i}|i\ =\ 1,\ldots\ldots\ldots,\ 10\} \tag{4}\]  
Relative joint position is used because it is an intuitive and efficient way to represent human motions. Consider, for example, the action \"victory gesture\", which can be interpreted as \"arms raised up over the shoulder\" and is significantly useful to be characterized by the relative positions. However, several essential factors are carefully considered when representing the local position for each joint, owing to the fact that the feature descriptor should be invariant in terms of the position, the orientation of the sensor, and the variance in height and weight among different people. [PERSON] et al. [28] showed that using relative positions between joints rather than using absolute positions originated at the sensor were less dependent on perspectives.