# Smart Rings vs. Smartwatches: Utilizing Motion Sensors for Gesture Recognition
## 4 Evaluation and Results
### Ring Session Results  
The data for the ring session was recorded only by the smart ring, while utilizing the force sensitive resistor as the activation method. The advantage of this method is that the ring can be activated with just one hand by using a finger that is adjacent to the ring. Another usability difference is that the button is pressed for the duration of the gesture and it can be released to finish the gesture at an earlier time for convenience reasons. Smart Ring Observations  
The data from the ring session are evaluated the same way as the data from the watch session. Figure 17 shows the 12 models that were cross-validated with different feature vector sizes and tested against the held-back test set. For each model type, a model representative is selected, with the highest accuracy score as the criterion. Since two of the SVM models and two of the KNN models share the same score, the model based on the smaller feature vector is preferred. The best and worst performing representatives are compared in further detail utilizing their confusion matrix. The RF representative performed the best with 93.2% accuracy, while the NB representative performed the worst with 85.9% accuracy. The two model representatives differ in 7.3%, which is consistent with the representatives from the watch session. The representative for the SVM classifier is close to the best model with 92.2% and both satisfy the defined minimum requirement of 92% accuracy. Only the RF classifier is able to handle the small feature vector without a significant loss in accuracy. The difference between the worst and best performing models of the RF classifier is only 1.5%, while the KNN classifier shows a difference of 9.9%. This makes the RF the most flexible classifier in regard to the feature size when compared with the results of the watch session, since it is the only classifier that consistently shows this size resistant behavior. The SVM and KNN classifiers show the same scores for medium and large vectors and could be further refined by adding new feature vectors with additional sizes between 50% and 90%. Figure 16: Confusion matrices illustrating results. (**a**) Confusion matrix for the best performing model trained by the ring data set of the watch session. The random forest (RF) model achieved an accuracy of 96.6%, while utilizing the medium sized feature vector. (**b**) Confusion matrix for the worst performing model representative trained by the ring data set of the watch session. The naive Bayes (NB) model achieved an accuracy of 88.6%, while utilizing the medium sized feature vector. The confusion matrix for the best RF model in Figure 18a shows the _Clap Twice_ gesture as the top misclassified gesture with a true positive rate of 69%. However, it is confused only with the _Clap Once_ gesture, resulting in a false-positive rate of 31%. Half of the gestures are perfectly classified. The remaining gestures range between true positive rates of 81% and 95%. The distribution of the matrix suggests that the directional gestures _Flick Left_ and _Flick Right_ as well as the _Flick Up_ and _Flick Down_ gestures are frequently confused with each other. Figure 17: Accuracy scores of all 16 models trained by the ring data set of the ring session. Each model type was trained with three different feature vector sizes and tested against the held-back test data set. Figure 18: Confusion matrices illustrating results. **(a)** Confusion matrix for the best performing model trained by the ring data set of the ring session. The RF model achieved an accuracy of 93.2%, while utilizing the medium sized feature vector. **(b)** Confusion matrix for the worst performing model representative trained by the ring data set of the ring session. The NB model achieved an accuracy of 85.9%, while utilizing the medium sized feature vector. Figure 18b shows the confusion matrix for the model representative of the NB classifier. More than half of all gestures do not reach a true positive rate higher than 88%, hence, the low overall score.