# A Smart Motor Rehabilitation System Based on the Internet of Things and Humanoid Robotics
## 5 Performance Evaluation
### Results
Specifically, we analyzed the prediction accuracy and F1-score for movement **A**, movement **B**, and the combination of both motions across different time windows, from 125 milliseconds (ms) to 2 s. In the smallest time window of 125 ms, accuracy is close to 100%, while there is a performance loss for a time window of 2 s. The results indicate a clear correlation between increased time windows and lower accuracy rates for motion classification. Table 4 presents the average inference time for each tested algorithm. All techniques showed inference times under 1 s. Furthermore, excluding SVM (which is the slowest), all models achieved inference times under 100 ms, demonstrating that the tested set of techniques is suitable for real-time scenarios. Figure 8: Comparison of F1-Score for evaluating FFNN performance using different signals on a variable number of devices. Figure 7: Comparison of Accuracy for evaluating FFNN performance using different signals on a variable number of devices. Figure 9: Accuracy and F1-Score for predicting motion using FFNN across different time windows. As previously mentioned, during the activity, participants were asked to wear a smart ring to monitor their heart rate and oxygen saturation levels. Figure 10 illustrates the patient's heart rate fluctuations over a predetermined period of time while performing the AL exercise. It is evident that exercise may cause an increase in heart rate variability, highlighting the physiological differences between a male and female subject during a 24 min exercise session, with the female subject's heart rate peaking at **120-130 ppm**, while the male subject's heart rate remains just below **90 bpm**. The oxygen saturation levels collected during the experiments remained stable, showing no notable patterns or significant variations. The final analysis aimed to predict the number of repetitions a patient would perform for a given exercise, based on the algorithm described in Section 4.2. Figure 11 illustrates the peak identification performed by the repetition counter algorithm on the data collected during the execution of the AR exercise. To analyze performance across four different exercises and four different devices, we calculated the peaks for each exercise separately, and then averaged the results for each device. This approach allowed us to derive predicted peaks, which are the average of \begin{table}
\begin{tabular}{c c c} \hline
**Model** & **Mean** & **Std** \\ \hline Support Vector Machine & 0.6887 & 0.0063 \\ Decision Tree & 0.0010 & 0.0001 \\ Random Forest & 0.0170 & 0.0050 \\ Gradient Boosting & 0.0549 & 0.0006 \\ Feed Forward Neural Network & 0.0152 & 0.0159 \\ \hline \end{tabular}
\end{table} 
Table 4: Inference Time (in seconds) of each algorithm. Figure 10: Heart rate comparison of two subjects—AL exercise. the peaks detected across all devices. We collected four separate data sets, one for each exercise, to accurately count the number of repetitions performed by the subject. Since the algorithm uses threshold and distance parameters to identify peaks, varying these values can significantly affect the number of peaks detected. Therefore, we explored different parameter values to understand their influence on the results, which are presented as confidence intervals. Figure 12 shows the results of the repetition counter algorithm. While the predicted repetitions are close to the actual counts, there are still some fluctuations due to the different parameters used in the algorithm, which significantly influence the number of peaks detected in the signal.