# Smart Rings vs. Smartwatches: Utilizing Motion Sensors for Gesture Recognition
## 4 Evaluation and Results
### Watch Session Results
#### 4.1.1 Smartwatch Observations  
Figure 12 shows the results for 12 different models. Each model was trained with data from the smartwatch using cross-validation and compared against the held-back test set to estimate realistic accuracy scores. The held-back test was split apart from the data set in advance to the model training stage. Therefore, testing against this test set provides a realistic estimate for the model performance on unseen data. The training process was repeated for each of the three feature vector sizes, where the small feature vector contains only the most important features and the large vector misses only the weakest features. The different vectors contain 10%, 50% and 90% of the best features, respectively. For each type of model, a model representative is selected by comparing the scores originating from the three different feature vector configurations. From the four resulting representatives, the best and worst models are further inspected in detail.  
The random forest (RF) representative performed best with 98.8% accuracy, while the k-nearest neighbor (KNN) representative performed the worst with 94.6% accuracy. The difference between the best and the worst representatives is only 4.4%.  
The naive Bayes (NB) and the RF classifier seem to be able to handle different feature vectors about equally well considering that the maximum difference in accuracy between feature vectors sizes is only 1.1% for the RF and 2.0% for the NB. In contrast, the difference for the support vector machine (SVM) and the k-nearest neighbor (KNN) classifiers is 9.6% and 8.4%, respectively. Generally, the results show that feature vectors containing more than 10% of the best features lead to better results and that feature vectors with more than 50% of the best features lead to worse results. This suggests, that the models could be optimized further by introducing additional feature vector sizes around the medium sized category.  
The differences between the best and the worst performing model representatives can be compared in further detail with their confusion matrices. Figure 13 shows the matrix for the best performing RF model. The misclassified gestures for the RF model are the _Clap Once_ and the _Rotate Once_ gestures with a true positive rate of 90% and 95%, respectively. The confusion matrix suggests that the only difficulty for the RF is to decide whether the gesture is a clap or a rotation gesture and how often it is performed.  
Figure 12: Accuracy scores of all 16 models trained by the watch data set of the watch session. Each model type was trained with three different feature vector sizes and tested against the held-back test data set.  
Figure 13: Confusion matrix for the best performing model trained by the watch data set of the watch session. The RF model achieved an accuracy of 98.8%, while utilizing the medium sized feature vector.  
The confusion matrix in Figure 14 for the best KNN model shows many misclassifications. The most misclassified gesture is the _Rotate Once_ gesture with a true positive rate of 85%. Only four gestures are perfectly classified, and the remaining gestures are recognized with a true positive rate of either 90% or 95%. The distribution of the false positives suggests that the model has difficulties deciding between circle gestures and the _Flick Up_ gesture.