## 5 Performance Evaluation
### Implementation and Evaluation
#### 5.3.3 Part III-Recommendation System  
The goal of this part is to evaluate the performance of the recommendation system introduced in Section 3. In order to get an accurate result, we implemented and run Algorithm 1 ten times, and then calculated an average result. Every time, 100 drivers sent their requests. For every request, the segment number and requested time interval were selected randomly. To have more accurate results, we consider only day time (i.e., from 8:00 to 18:00). Inspiring from results of Parts I and II of this section, we used the simple but effective DT algorithm in our recommendation system. The results show that 77% of drivers selected L1 or low availability segments. It means, without using the recommendation system, 77% of drivers can not find a free spot in their selected segment and, therefore, they have to search for an available place in other segments. If we assume that finding an  
\begin{table}
\begin{tabular}{l l l l l l l} \hline \hline
**VC** & **Accuracy** & **Precision** & **Recall** & **f1\_Score** & **Training (s)** & **Testing (s)** \\ \hline DT,CNB,SLP,SVM,MLP & 77.48 & 79.70 & 77.48 & 75.26 & 0.6209 & 0.0227 \\ DT,RF,CNB,SLP,MLP & 79.37 & 81.38 & 79.37 & 78.04 & 0.8037 & 0.0264 \\ DT,RF,CNB,SLPSVM & 79.31 & 81.33 & 79.31 & 77.79 & 0.2530 & 0.0399 \\ DT,RF,CNB,SVM,MLP & 79.50 & 81.59 & 79.50 & 77.92 & 0.8303 & 0.0414 \\ DT,RF,SLPSVM,MLP & 77.75 & 79.87 & 77.75 & 75.86 & 2.1170 & 0.0416 \\ KNN,CNB,SLPSVM,MLP & 77.45 & 79.74 & 77.45 & 75.23 & 0.6299 & 0.0236 \\ KNN,DT,CNB,SLP,MLP & 79.26 & 81.25 & 79.26 & 77.95 & 0.5945 & 0.0088 \\ KNN,DT,CNB,SLPSVM & 79.19 & 81.28 & 79.19 & 77.67 & 0.0410 & 0.0229 \\ KNN,DT,CNB,SVM,MLP & 79.42 & 81.57 & 79.42 & 77.84 & 0.6210 & 0.0238 \\ KNN,DT,RF,CNB,MLP & 79.92 & 81.76 & 79.92 & 79.14 & 3.4872 & 0.0276 \\ KNN,DT,RF,CNB,SLP & 79.62 & 81.48 & 79.62 & 78.90 & 0.2269 & 0.0278 \\ KNN,DT,RF,CNB,SVM & 79.93 & 81.79 & 79.93 & 79.10 & 0.2534 & 0.0421 \\ KNN,DT,RF,SLP,MLP & 79.50 & 81.39 & 79.50 & 78.75 & 0.8036 & 0.0277 \\ KNN,DT,RF,SLP,SVM & 79.56 & 81.41 & 79.56 & 78.74 & 0.2505 & 0.0411 \\ KNN,DT,RFSVM,MLP & 79.77 & 81.63 & 79.77 & 78.91 & 0.8329 & 0.0428 \\ KNN,DT,SLPSVM,MLP & 77.64 & 79.68 & 77.64 & 75.76 & 0.6264 & 0.0239 \\ KNN,RF,CNB,SLP,MLP & 79.29 & 81.21 & 79.29 & 77.97 & 0.8033 & 0.0280 \\ KNN,RF,CNB,SLPSVM & 79.16 & 81.07 & 79.16 & 77.72 & 0.2518 & 0.0412 \\ KNN,RF,CNB,SVM,MLP & 79.42 & 81.50 & 79.42 & 77.85 & 0.8425 & 0.0429 \\ KNN,RF,SLP,SVM,MLP & 77.62 & 79.63 & 77.62 & 75.77 & 2.9616 & 0.0426 \\ RF,CNB,SLPSVM,MLP & 77.48 & 79.83 & 77.48 & 75.27 & 0.8413 & 0.0418 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Ensemble Learning Algorithms (combinations of five algorithms).  
\begin{table}
\begin{tabular}{l l l l l l l} \hline \hline
**VC** & **Accuracy** & **Precision** & **Recall** & **f1\_Score** & **Training (s)** & **Testing (s)** \\ \hline RF,SLP,MLP & 76.79 & 78.72 & 76.79 & 75.02 & 5.8749 & 0.0244 \\ RF,SLPSVM & 76.60 & 78.79 & 76.60 & 74.33 & 5.3220 & 0.0380 \\ RF,SVM,MLP & 76.92 & 79.05 & 76.92 & 74.65 & 0.8238 & 0.0392 \\ SID,SVM,MLP & 75.32 & 77.49 & 75.32 & 72.67 & 0.6250 & 0.0207 \\ \hline \hline \end{tabular}
\end{table}
Table 5: _Cont._available place in another segment takes 5 min, for our 1000 requests, in total, the drivers should spend 1000 \(\times\) 0.70 \(\times\) 5 min to find a place. In this case, our recommendation system can save this time by predicting (with accuracy around 94%) selected L1 segments, and then suggesting a L3 or L2 segment.