# Smart Rings vs. Smartwatches: Utilizing Motion Sensors for Gesture Recognition
## 4 Evaluation and Results
### Watch Session Results
#### 4.1.2 Smart Ring Observations  
Figure 15 shows the results for the 12 different models that were trained with the smart ring data. The smart ring was controlled remotely by the smartwatch and the activation mechanism on the ring was temporarily disabled to ensure that the data are independent of the activation method. Each model was cross-validated with the same feature vector sizes that were used in the watch data section and tested with the held-back test set.  
The model representative is chosen in the same way as in the smartwatch section, by comparing all three models of the same model type and using the highest accuracy score as a criterion. However, in this case, the scores for the SVM models are equal for the medium and large feature vector sizes. Since a lower number of features leads to lower complexity models, the model that was trained on the medium sized feature vector is chosen. The best and the worst performing model representatives are analyzed in detail by comparing their confusion matrices.  
The RF representative performed the best with 96.6% accuracy while the NB representative performed the worst with a score of 88.6%. The difference between these two model representatives is 8.0%, which is rather significant. The KNN and NB model representatives are very close to each other with 89% accuracy and 88.6% accuracy, respectively. However, both are unable to satisfy the defined minimum requirement of 92% accuracy.  
Figure 14: Confusion matrix for the worst performing model representative trained by the watch data set of the watch session. The KNN model achieved an accuracy of 94.6%, while utilizing the medium sized feature vector.  
The RF and the NB classifiers seem to be able to handle different feature vector sizes well compared to the SVM and KNN classifiers. The difference between the worst model and the best model for different feature vector sizes is 1.2% for the RF classifier and 3.5% for the NB classifier. In contrast, the difference for the KNN and SVM classifiers is 17.3% and 14.3% respectively. However, these differences originate from the low score for the small feature vector while the scores for larger vectors are almost identical. This suggests, that these classifiers require at least a certain number of features in order to perform good on the data set. The KNN model representative is the only model that gains an advantage by using the large feature vector over the medium vector.  
The confusion matrix in Figure 16a shows the individual classifications for the best performing model. Most of the gestures are perfectly classified, hence, the high accuracy of the model. The worst performing gestures are the _Clap Twice_ and the _Flick Left_ gestures with a true positive rate of 90%. The _Clap Twice_ gesture is confused with the _Clap Once_ gesture with a false positive rate of 10% which is reconcilable due to the similarity between them.  
Figure 16b shows the confusion matrix for the worst performing model representative. Similar to the RF model, the worst performing gesture for the NB model is the _Clap Once_ gesture with a true positive rate of 55%. It is frequently confused with the _Clap Twice_ and the _Rotate Twice_ gesture. Due to the low overall accuracy, several gestures are misclassified with true positive rates ranging from 80% to 95%. The distribution in the matrix suggests that the NB model has problems differentiating directional gestures from circle gestures.  
Figure 15: Accuracy scores of all 16 models trained by the ring data set of the watch session. Each model type was trained with three different feature vector sizes and tested against the held-back test data set.