# 5G/B5G Service Classification Using Supervised Learning
## 4 Simulation Results
A database containing 165 rows was kept, with five additional columns corresponding to the KQI parameters. We used the same functions to create and train the ML algorithms, resulting in the same SL algorithms. Again, we used the K-Folds cross-validation technique with K = 10 [32] to validate the ML algorithm and we obtained the results showed in Table 5. Figure 3 shows the confusion matrix obtained for each model in this simulation. We obtained the performance metrics for the predictive models based on the newly generated confusion matrices. The results were as follows Table 6: \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**SL Algorithms** & **Accuracy (\%)** & **Precision Macro (\%)** & **Recall Macro (\%)** & **F1-Score Macro (\%)** & **MCC (\%)** \\ \hline Decision Tree & 93.9 & 93.5 & 94.7 & 93.1 & 93.19 \\ Random Forest & 93.9 & 94.7 & 94.7 & 93.8 & 93.17 \\ SVM & 96.9 & 96.3 & 98.4 & 96.9 & 96.6 \\ KNN & 78.8 & 70 & 79.9 & 71.8 & 76.89 \\ MLPC & 87.8 & 82.4 & 85.1 & 82.7 & 86.58 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Model metric resuts for the first simulation (KPIs). Figure 3: Confusion matrices for the second simulation (KPIs + KQIs). CV: Connected Vehicles; IE: Immersive Experience; IA: Industry Automation; SG: Smart Grid; UHD: Video Streaming; VS: Video Surveillance; VO: Vo5G; eH: e-health. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **SL Algorithms** & **K-Folds (K = 10)** & **Cross-Validation Results** \\ \hline  & \multicolumn{3}{c}{Decision Tree} & 97.69 \\  & \multicolumn{3}{c}{Random Forest} & 98.52 \\  & \multicolumn{3}{c}{SVM} & 91.59 \\  & \multicolumn{3}{c}{KNN} & 83.35 \\  & \multicolumn{3}{c}{MLPC} & 90.11 \\ \hline \hline \end{tabular}
\begin{tabular}{c c c c} \hline \hline  & \multicolumn{3}{c}{CV ITS E IA 86 vs UHD V0 \#H} & \multicolumn{3}{c}{CV ITS E IA 86 vs UHD V0 \#H} \\  & \multicolumn{3}{c}{CV [3 0 0 0 0 0 0 0 0 0]} \\  & \multicolumn{3}{c}{1 TS [0 6 0 0 0 0 0 0 0 0]} & \multicolumn{3}{c}{1 TS [0 6 0 0 0 0 0 0 0 0]} \\  & \multicolumn{3}{c}{IE [0 0 7 0 0 0 0 0 0 0]} & \multicolumn{3}{c}{IE [0 0 7 0 0 0 0 0 0 0]} \\  & \multicolumn{3}{c}{IA [0 0 0 0 5 0 0 0 0 0]} & \multicolumn{3}{c}{IA [0 0 0 5 0 0 0 0 0 0]} \\  & \multicolumn{3}{c}{so [1 0 0 0 2 0 0 0 0]} & \multicolumn{3}{c}{so [1 0 0 0 2 0 0 0 0]} \\  & \multicolumn{3}{c}{vs [0 0 0 0 0 0 3 0 0 0]} & \multicolumn{3}{c}{vs [0 0 0 0 0 3 0 0 0]} \\  & \multicolumn{3}{c}{UHD [0 0 0 0 0 0 0 2 0 0]} & \multicolumn{3}{c}{UHD [0 0 0 0 0 0 0 2 0 0]} \\  & \multicolumn{3}{c}{vo [0 0 0 0 0 0 0 3 0 0]} & \multicolumn{3}{c}{vo [0 0 0 0 0 0 3 0 0]} \\  & \multicolumn{3}{c}{eH [0 0 0 0 0 0 0 0 0 0 1]} & \multicolumn{3}{c}{eH [0 0 0 0 0 0 0 0 0 1]} \\ \end{tabular}
\end{table} 
Table 5: Results of the accuracy in cross-validation stage for the second simulation (KPIs + KQIs). From Table 6, it is possible to know that the KNN model does not apply to our problem because it had an inadequate accuracy. Furthermore, we can see that the other models increased their metrics in this second simulation, and the best metrics obtained are Decision Tree, Random Forest, and SVM. To verify if the predictive model was satisfactory, we created a function to compare the accuracy obtained in the cross-validation stage versus the accuracy of the testing stage. We considered the model was acceptable if the difference does not exceed 5%. The result obtained for the SVM had a difference of 8.41%, so we conclude that this model may be overfitting. The result obtained had a difference of 0.69% and 1.62% in the Decision Tree and Random Forest. This result means that the predictive model generated by the Decision Tree and the Random Forest algorithms are not overfitting. If the predictive model is overfitting, we choose the third option mentioned above, for example, making a Random Forest with maximum depth. We determined that we can use both Decision Tree or Random Forest to solve the service classification problem presented, as the authors obtained in [11].