# Robust Hand Shape Features for Dynamic Hand Gesture Recognition Using Multi-Level Feature LSTM
## 3 Proposed Idea
### Hand Skeleton Normalization  
The hand skeletal data are received from various camera sensors, the pose of the camera, as well as the performer. Therefore, we need to normalize the data to the reference pose and size to prevent over-fitting of the method with respect to the environmental elements.  
First, we choose the reference hand \(\mathbf{S}_{ref}=[\mathbf{x}_{1},\mathbf{x}_{2},...,\mathbf{x}_{N_{j}}]^{T}\) in the dataset with the status of open and in front of the camera, as in Figure 3. Then, we transform the reference hand so that the palm joint is at the root coordinate and scale the hand size to fit in the unit sphere as follows:  
\[\mathbf{S}_{norm} =\mathbf{S}_{ref}-\mathbf{x}_{palm} \tag{4}\] \[scale =\max\left\|\mathbf{x}_{i}\right\|_{2},\text{ where }\mathbf{x}_{i} \in\mathbf{S}_{norm}\] (5) \[\mathbf{S}_{norm} =\frac{\mathbf{S}_{norm}}{scale} \tag{6}\]  
For every skeletal data sequence, we find the optimal rotation \(\mathbf{R}\) and translation \(\mathbf{t}\) using [38] between the hand skeletal data at the first frame \(\mathbf{S}_{t_{1}}\) and the reference hand \(\mathbf{S}_{norm}\) based on the seven hand joints corresponding to the 3D points (palm, wrist, from the thumb to pinky base joints) as the least squares error minimization problem:  
\[E=\sum_{i=1}^{N_{b}}\left\|\mathbf{R}\mathbf{x}_{i}^{t_{1}}+\mathbf{t}-\mathbf{x}_{i}^{norm} \right\|_{2}\longrightarrow 0 \tag{7}\]  
where \(\mathbf{x}_{i}^{t_{1}}\in\text{base joints}\left(\mathbf{S}_{t_{1}}\right)\), \(\mathbf{x}_{i}^{norm}\in\text{base joints}\left(\mathbf{S}_{norm}\right)\), and \(N_{b}=7\) is the number of base joints (palm, wrist, from the thumb to pinky base joints).  
To solve Equation (7), we find the center of base joints \((S_{t_{1}})\), base joints \((S_{norm})\), respectively:  
\[\mathbf{x}_{center}^{t_{1}} =\frac{1}{n_{t_{1}}}\sum_{i=1}^{n_{t_{1}}}\mathbf{x}_{i}^{t_{1}} \tag{8}\] \[\mathbf{x}_{center}^{norm} =\frac{1}{n_{norm}}\sum_{i=1}^{n_{norm}}\mathbf{x}_{i}^{norm} \tag{9}\]  
where \(n_{t_{1}}=\left|\text{base joints}\left(S_{t_{1}}\right)\right|\), and \(\mathbf{x}_{i}^{t_{1}}\in\text{base joints}\left(S_{t_{1}}\right)\); similarly, \(n_{norm}=\left|\text{base joints}\left(S_{norm}\right)\right|\), and \(\mathbf{x}_{i}^{norm}\in\text{base joints}\left(S_{norm}\right)\). Then, we calculate the Singular Value Decomposition (SVD) of the co-variance matrix \(\mathbf{H}\) to find the rotation transformation \(\mathbf{R}\) as follows:  
\[\mathbf{H} =\left(\text{base joints}\left(S_{t_{1}}\right)-\mathbf{x}_{center}^{t _{1}}\right)\left(\text{base joints}\left(S_{norm}\right)-\mathbf{x}_{center}^{ norm}\right)^{T} \tag{10}\] \[\mathbf{U},\mathbf{S},\mathbf{V} =SVD\left(\mathbf{H}\right)\] (11) \[\mathbf{R} =\mathbf{V}\mathbf{U}^{T} \tag{12}\]  
We address the reflection case of the SVD results by checking the determinant \(\left|R\right|<0\) and fixing again as the equation below:  
\[\mathbf{U},\mathbf{S},\mathbf{V} =SVD\left(\mathbf{R}\right) \tag{13}\] \[\mathbf{V}\left[column3\right] =-1*\mathbf{V}\left[column3\right]\] (14) \[\mathbf{R} =\mathbf{V}\mathbf{U}^{T} \tag{15}\]  
Finally, the translation is calculated as below:  
\[\mathbf{t}=\mathbf{x}_{center}^{norm}-\mathbf{R}\times\mathbf{x}_{center}^{t_{1}} \tag{16}\]