# Enhancing Recognition of Human-Object Interaction from Visual Data Using Egocentric Wearable Camera
## 3 Proposed System
### Feature Extraction and Description  
After preprocessing, we extracted 18 time-domain features, as outlined in Table 2, from each matrix, i.e., HJM\({}^{\prime}_{\text{Normalized}}\). These features represented the statistical attributes of the hand joints data and were useful for the pattern recognition task that involved movement classification. We extracted these features for the HOIR because of their efficient performance involved hand movement data from motion sensors. Also, these features entailed low computational cost compared with the deep learning-based feature extraction methods. Moreover, these features provided signal traits that had great significance in recognizing hand motion patterns.  
\begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt}} \hline \hline
**Feature** & **Mathematical Description** \\ \hline Maximum amplitude & \(X_{\text{max}}=\text{max}(S_{a})\) where \(S\) can be either accelerometer or gyroscope data, and \(a\) represents the axis \\ \hline Minimum amplitude & \(X_{\text{min}}=\text{min}(S_{a})\) \\ \hline Arithmetic mean & \(\mu=\frac{1}{N}\sum_{i=1}^{N}S_{i}\) where \(N\) is the total number of samples \\ \hline Standard deviation & \(\sigma=\sqrt{\frac{1}{N}\sum_{i=1}^{N}(S_{a_{i}}-S_{a})^{2}}\) where \(S_{a}\) represents the mean value \\ \hline Kurtosis & \(K=\frac{E\left[(S_{a}-S_{a})^{4}\right]}{E\left[(S_{a}-S_{a})^{2}\right]^{2}}\) where \(E\) represents the expected value \\ \hline Skewness & \(Sk=E\left[\left(\frac{S_{a}-S_{a}}{\sigma}\right)^{3}\right]\) \\ \hline Peak-to-peak signal value & \(S_{a}(p2p)=X_{\text{max}}-X_{\text{min}}\), where \(p2p\) is the peak-to-peak value of the signal \\ \hline Peak-to-peak time & \(T_{a}(pp4)=X_{\text{max}}+X_{\text{min}}\) \\ \hline Peak-to-peak slope & \(S_{a}(pp8)=\frac{S_{a}(p2p)}{T_{a}(pp4)}\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: List of statistical features extracted for HOIR.  
We extracted these features against each column of the matrix, which produced a feature vector of size \([1\times(21\times 7)]=[1\times 126]\) per frame. We then applied mean pooling to aggregate these features over the entire video snippet, which resulted in the finalized feature vector of size \(1\times 126\). Figure 4 illustrates the process of feature extraction and description.  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Feature** & **Mathematical Description** \\ \hline Maximum latency & \(nX_{\text{max}}=\{n\mid S_{a}(n)=X_{\text{max}}\}\) \\ \hline Minimum latency & \(nX_{\text{min}}=\{n\mid S_{a}(n)=X_{\text{min}}\}\) \\ \hline Absolute latency-to-amplitude ratio & \(\text{ALAR}=\left|\frac{nX_{\text{max}}}{X_{\text{min}}}\right|\) \\ \hline Mean of absolute values of first difference & \(\mu_{\
abla}=\frac{1}{N}\sum|aS_{a}(n)|\), \\  & where \(\Delta S_{a}(n)=S_{a}(n)-S_{a}(n-1)\) \\ \hline Mean of absolute values of second difference & \(\mu_{\
abla\
abla}=\frac{1}{N}\sum|S_{a}(n+1)-2S_{a}(n)+S_{a}(n-1)|\) \\ \hline Normalized mean of absolute values of first difference & \(\mu_{n}\
abla=\frac{1}{N}\sum|S_{a}(n)-S_{a}(n-1)|\), \\  & where \(n\) is the normalized signal \\ \hline Normalized mean of absolute values of second difference & \(\mu_{n}\
abla\
abla=\frac{1}{N}\sum|S_{a}(n+1)+2S_{a}(n)+S_{a}(n-1)|\), \\  & where \(n\) is the normalized signal \\ \hline Energy & \(E_{n}=\frac{1}{N}\sum_{i=1}^{N}S_{a}(i)^{2}\) \\ \hline Normalized energy & \(E_{N}=\frac{E_{n}}{N}\) \\ \hline Entropy & \(e=\sum_{n=1}^{N}e\log(e)\), where \(e=\frac{S_{a}}{\sum_{i=1}^{N}S_{a}(i)}\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: _Cont._  
Figure 4: Feature extraction and description process for proposed framework (where “\(\theta\)” represents the process of hand joints extraction, “\(\tau\)” represents the process of the 3D to 7D transformation, and “\(f\)” represents the process of feature extraction).