# A Four-Stage Mahalanobis-Distance-Based Method for Hand Posture Recognition
## 4 Experiments and Discussion
### Ablation Study  
The classification accuracies of the whole method compared to versions with particular stages removed are presented in Table 3. Additionally, for the SPAS dataset, the confusion matrices for the reduced method are presented in Figure 6.  
As one can see, stages II and III are important and their removal leads to lower effectiveness of the method on both datasets. However, the greatest deterioration of the results (about 20 percentage points) is visible in the case of MU-ASL-digits.  
Based on the confusion matrix of the algorithm without stage III, we can see that the most frequently confused letter pairs are S-T and O-S. The rules applied in stage III are crucial to greatly reduce the number of such mistakes. Stage II, in turn, affects mainly the recognition of the letter U. It is probably due to the unusual hand shape of this letter, which sometimes makes it difficult for MediaPipe to detect all hand joints correctly. Even with some inaccurately detected joints, the algorithm is able to filter most of the remaining classes based on Mahalanobis distance to their representative samples.  
The classification times of the whole method compared to versions with particular stages removed are presented in Table 4. The times were measured on a laptop with an Intel Core i5 8300H CPU. The average classification time of a gesture from the SPAS dataset is 4.2 ms with a standard deviation of 2.2 ms. The average time of a simple classification using only stage I and IV of the algorithm (data preprocessing and kNN) is 18.2 ms with a standard deviation of 4.6 ms. The whole algorithm is much faster because stage IV has the longest computation time and it can be greatly reduced by excluding a majority of classes in stage II and, in some cases, many skeletal joints in stage III. The gain from the reduction in stage IV computation time is greater than the combined time of stage II and stage III.  
However, it should be mentioned that the presented times were measured without detecting a hand and creating skeletal data by MediaPipe. The average hand detection time in our experiments with the SPAS dataset was 50 ms with a standard deviation of 10 ms.  
The learning time of our method on the entire SPAS dataset considered as a training set is 0.8 s. This means that learning is almost instantaneous for datasets of this size, which can be considered another advantage of our method.  
Figure 6: Confusion matrices of the proposed method with particular stages removed on the SPAS dataset: (**a**) stages I, II, and IV and (**b**) stages I and IV.  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **SPAS** & **MU-ASL-Digits** \\ \hline whole algorithm (stages: I, II, III and IV) & 94.69/3.54 & 97.44/3.07 \\ stages: I, II, and IV & 90.62/4.63 & 83.6/5.25 \\ stages: I and IV & 87.08/6.08 & 76.64/5.72 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Recognition rates/stanard deviations [%] of the whole algorithm compared to versions with particular stages removed.  
Our method can run as a real-time video streaming application with more than 10 frames per second and high effectiveness, even if a person showing gestures is not present in the training set, which confirms the correctness of the validation tests performed.