# 5G/B5G Service Classification Using Supervised Learning
## 4 Simulation Results  
To determine whether if the inclusion of KQIs improves the predictive service classification model, we perform two simulations. The first considers only the KPIs, while the second also incorporates the KQIs. We first explain and define the scenario and conditions used in the simulations. The necessary elements are the SML algorithms, a programming language, a development platform, the 5G services to be classified, and the parameters of their KPIs and KQIs. For the validation scenario and to simulate the proposed system, we used SL algorithms: Decision Tree, Random Forest (with five trees), Support Vector Machine (SVM) with a linear kernel, K-Nearest Neighbors (KNN, K = 3) and Multi-Layer Perceptron Classifier (MLPC), using the Python language, and Anaconda Navigator platform with Jupyter Notebook as IDE. We considered nine essential 5G services to be classified: Ultra High Definition (UHD) video streaming, immersive experience, connected vehicles, e-health, industry automation, video surveillance, smart grid, Intelligent Transport Systems (ITS) and Voice over 5G (Vo5G). The selected KPI parameters were E2E latency, jitter, bit rate, packet loss rate, peak data rate DownLink (DL), peak data rate UpLink (UL), mobility and, service reliability. The KQI parameters were service availability, user experience data rate DL/UL, survival time and interruption time. In the first simulation, we worked with the KPIs. The dataset had dimensions of \(165\times 9\), where the first eight columns represented the KPIs, and the last contained the labels of the services. We divided the database into two parts, where 80% (132) of the data (_Xtrain_) were used to train the algorithms created and, once trained, generated the predictive model. The remaining 20% (_Xtest_) were used to test the model. The models may be prone to underfitting or overfitting, meaning that it will work perfectly for the training data (_Xtrain_) that are already known, but its accuracy may be lower for new services (_Xtest_). According to [29; 31], there are two possible approaches to avoid overfitting: increasing the volume of the database or reserving additional data by dividing the dataset into three parts (training, validation and testing). Increasing the amount of data was difficult because there were insufficient known data from the 5G service; hence, additional data were reserved, and the K-Folds cross-validation technique was applied using K = 10 [32] and we obtained the results showed in Table 3. It should be noted that all of the data formed part of the original dataset, and did not constitute three new datasets. Figure 2 shows the confusion matrices obtained during the testing process for each model in the first simulation, in which we considered only the KPIs. The main diagonal shows the number of correct predictions made by the predictive model. Values outside the main diagonal represent predictions in which the model was wrong. We applied Equations (1)-(5) to the metrics obtained from the confusion matrix to evaluate the performance of the predictive models. The results were as follows Table 4: \begin{table}
\begin{tabular}{c c} \hline \hline
**SL Algorithms** & **K-Folds (K = 10) Cross-Validation Results** \\ \hline Decision Tree & 99.23 \\ Random Forest & 99.23 \\ SVM & 92.42 \\ KNN & 59.83 \\ MLPC & 87.08 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Results of the accuracy in cross-validation stage for the first simulation (KPIs). Figure 2: Confusion matrices for the first simulation (KPIs). CV: Connected Vehicles; IE: Immersive Experience; IA: Industry Automation; SG: Smart Grid; UHD: Video Streaming; VS: Video Surveillance; VO: Vo5G; eH: e-health. In the second simulation, we incorporated the user quality parameters (KQIs) and repeated the procedure (with a few differences from the previous simulation). The KQI parameters considered here were service availability, user experience data rate DL/UL, survival time, and interruption time.