# Data-Driven Safe Deliveries: The Synergy of IoT and Machine Learning in Shared Mobility
## 5 Results and Discussion
### Classification Model
#### 5.2.3 Model Training and Selection  
To identify the most appropriate classifier for the model, various algorithms, including Support Vector Machine (SVM), Decision Tree (DT), Naive Bayes (NB), and K-Nearest Neighbor (KNN), were trained and compared based on accuracy and F1-score. After rigorous evaluation, the Decision Tree classifier exhibited outstanding performance with an accuracy of 0.95 and an F1-score of 0.94, making it the optimal choice for further analysis. The Decision Tree model underwent further optimization through hyperparameter tuning. Initially, given the unequal distribution of data samples across various classes, we employed the class weight parameter, setting it to \"balanced\" during the model's training. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Maneuver Type** & **Number of Samples** & **Percentage** \\ \hline Acceleration on a curve & 3803 & 4.20\% \\ Acceleration on a straight line & 10,975 & 12.13\% \\ Fall-like maneuver & 32,520 & 35.94\% \\ Harsh braking on a straight line & 8846 & 9.78\% \\ Degraded track & 10,480 & 11.58\% \\ Fall in a curve & 6226 & 6.88\% \\ Fall in the roundabout & 5266 & 5.82\% \\ Fall on a slippery section & 5875 & 6.49\% \\ Fall with leaning of the motorcycle & 6480 & 7.16\% \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Percentage of samples for each maneuver type. This ensured that all classes were treated with equal importance during the classification process with weights assigned inversely proportional to the number of samples in each class. Furthermore, to evaluate the model's performance, we divided the dataset into training and testing subsets, employing the conventional 80-20 split ratio. Our primary objective was to determine the optimal number of tree nodes within the decision tree, thereby avoiding overfitting. This was accomplished through the experimentation with different values of the \"minimum samples split\" parameter, which is the minimum number of data samples required to split an internal node in the decision tree. For each value of minimum samples, we split the dataset multiple times and recorded the average number of tree nodes in the decision tree along with the average training and testing accuracies. The relationship between the number of nodes and the corresponding accuracy scores was visualized in Figure 4. As shown in the figure, the average training and testing accuracy graphs are close to one another when the testing accuracy is 91%. Based on this analysis, a minimum sample split of 115, resulting in 637 tree nodes, was selected, as it showed the closest average training and testing accuracies. To enhance the model efficiency, recursive feature elimination with cross-validation was employed to extract feature importance. Consequently, the following features were omitted due to their negligible importance: gyro_jerk_y, gyro_jerk_z, gyro_mean_jerk, acc_jerk_x, acc_jerk_y, acc_jerk_z, mean_acc_jerk, overall_acc_jerk, acc_x_std, acc_y_mean, acc_y_std, acc_y_median, acc_y_min, acc_y_max, acc_y_sem, acc_z_std, gyro_x_mean, gyro_x_std, gyro_x_max, gyro_x_sem, gyro_y_mean, gyro_y_std, gyro_y_min, gyro_y_sem, gyro_std, and gyro_z_sem. The refined model, post-dimensionality reduction, demonstrated promising performance with a testing accuracy of 0.91, an F1-score of 0.88, a recall of 0.9, a precision of 0.88, and a [PERSON]'s kappa coefficient of 0.89. The confusion matrix of the refined model is presented in Figure 5, and it provides a detailed breakdown of the classifier's performance for each class. The labels on the confusion matrix stand for the classes acceleration on a curve, acceleration on a straight line, fall-like maneuver, harsh braking on a straight line, degraded track, fall in a curve, fall in the roundabout, fall on a slippery section, and fall with leaning of the motorcycle, respectively. The classifier correctly identified many instances of acceleration on a curve, acceleration on a straight line, and degraded tracks, with relatively low numbers of false positives and false negatives.