# Multi-Class Confidence Detection Using Deep Learning Approach
## 6 Experiments & Results  
The purpose of this research is to provide an effective framework for the detection and recognition of various human gestures that people perform while communicating with others. Before training our model, we divide our dataset into a training and a testing dataset with the ratio of 70% and 30%, respectively. After splitting the dataset, we have a total of 2817 images in the training dataset and 1208 images in the testing dataset. The dataset contains four categories for classification; therefore, for each class, Table 3 shows the result of evaluation metrics of the training dataset. Table 3, shows the results in terms of Precision, F-measure, Recall, and Support for each category. Due to unbalancing of the dataset, confidence achieves highest results whereas the uncomfortable class presents lower results than the other categories. The reason is that, when a class has an imbalanced dataset, then the model will have a slight bias for the class with the maximum number of samples due to unequal misclassification costs. However, the accuracy of this framework collectively is 90.48%, which represents better results compared to existing frameworks. The Confusion matrices represented in Figure 6 for the training data shows the predicted and expected outcomes. The proposed architecture is validated by the results shown in Table 4 corresponding to the testing data: Precision, Recall, F-measure, and Support; whereas Figure 7 presents the Confusion matrices for testing the visual data. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Gesture Class** & **Precision** & **Recall** & **F-Score** & **Support** \\ \hline Confidence & 96\% & 92\% & 94\% & 814 \\ Cooperation & 95\% & 94\% & 95\% & 841 \\ Uncomfortable & 88\% & 90\% & 89\% & 567 \\ Unconfident & 93\% & 97\% & 95\% & 595 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Training Data Result for Human Gesture Classification. Figure 6: Training Data Confusion Matrix. The experimental study involves the comparison of customized neural network + LSTM architecture with the existing pre-trained models, including K-nearest neighbor(KNN), Naive Bayes, SVM, VGG-19 on the collected dataset which shows the leading accuracy of the proposed architecture. In Table 5, the comparison of accuracy, precision, f-measure, and recall is presented with the proposed architecture. Figure 8 shows the comparison of Accuracy, Precision, Recall, and F-measure for overall results of human-emotion-recognition based on gesture classes. While Figures 9-11 indicate the comparison of these evaluation metrics based on four different gesture classes. Although CNNs perform well in image classification tasks by learning spatial feature hierarchies from raw data, they do not perform well in sequence data as they do not learn temporal dependencies. Since our model takes images extracted from videos as input, a combination of CNN and LSTM performs better in our case. In real-life scenarios, our model can be used to detect the confidence of a person, learning both spatial and temporal dependencies during training. The VGG-19 model shows slightly better precision for the \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Machine Learning Method** & **Accuracy** & **Precision** & **Recall** & **F-Score** \\ \hline KNN & 79\% & 81.5\% & 81.75\% & 81.25\% \\ Naives Bayes & 43\% & 37.5\% & 50.5\% & 35.75\% \\ SVM & 67\% & 50.5\% & 69.3\% & 70\% \\ VGG-19 & 86\% & 89.3\% & 87.75\% & 88.5\% \\ Proposed Method (CNN + LSTM) & 90.5\% & 90.46\% & 90.48\% & 90.46 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison Results with Other Machine Learning Methods. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Gesture Class** & **Precision** & **Recall** & **F-Score** & **Support** \\ \hline Confidence & 92\% & 93\% & 93\% & 368 \\ Cooperation & 91\% & 90\% & 91\% & 352 \\ Uncomfortable & 87\% & 85\% & 86\% & 248 \\ Unconfident & 90\% & 93\% & 92\% & 240 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Testing Dataset Result for Human Gesture Classification.