# Crowdsourcing Based Performance Analysis of Mobile User Heterogeneous Services
## 5 Use Cases
### Use Case 3: Impact of Radio Parameters on the Video Metrics and User QoE
The next operation consists of the final prediction step, in which the best configuration for each ML method is used to implement the regression method that predicts three calculated MOS scores and two considered video KQI (bitrate and launching time). Concerning the three MOS scores prediction, the results are given in Table 12. We report in this figure the Mean Absolute Error (\(MAE\)) [31] and Mean Absolute Percentage Error (\(MAPE\)) [32] and Pearson correlation rate (\(r\)) [8]. \begin{table}
\begin{tabular}{c c} \hline \hline
**ML Method** & **Configurations** \\ \hline Decision Tree (DT) & \(type=DTregressor\) \\  & \(h\in\{2,3,4,5,6\}\) \\  & \(th:\ 0.1\) to \(0.9\) \\ \hline k-Nearest & \(type=regression\) \\ Neighbours (KNN) & \(n\_neighborsint\in\{1\dots 200\}\) \\ \hline Random Forest (RF) & \(type=RF\) \\  & \(n\_estimators\in\{10\dots 1000\}\) \\  & \(max\_depth\in\{10,20,50,\ 100\}\) \\ \hline DT-based Adaptive & \(base=DTC(DecisionTreeRegressor)\) \\ Boosting (AdaBoost) & \(n\_estimators\in\{10\dots 1000\}\) \\  & \(learning\_rate:\ 0.1\) to \(0.9\) \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Used configurations in the ML tuning step. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Method/Performance**} & \multicolumn{3}{c}{**MOS Based on Buffer Information**} & \multicolumn{3}{c}{**MOS Based on Bitrate**} & \multicolumn{3}{c}{**Global MOS (User)**} \\ \cline{2-10}  & **MAE** & **MAPE** & **r** & **MAE** & **MAPE** & **r** & **MAE** & **MAPE** & **r** \\ \hline RF & 0.16 & 0.06 & 0.88 & 0.21 & 0.32 & 0.88 & 0.17 & 0.07 & 0.90 \\ \hline DT & 0.28 & 0.09 & 0.31 & 0.20 & 0.05 & 0.27 & 0.29 & 0.09 & 0.30 \\ \hline KNN & 0.07 & 0.03 & 0.92 & 0.10 & 0.14 & 0.91 & 0.35 & 0.16 & 0.63 \\ \hline GTB & 0.06 & 0.02 & 0.81 & 0.13 & 0.03 & 0.73 & 0.07 & 0.02 & 0.80 \\ \hline \hline \end{tabular}
\end{table} 
Table 12: MOS scores prediction performance results using 80% for training and 20% for testing with 5-fold for cross validation. From Table 12, we notice that all the considered models, except the DT method, performed reasonably well on the task of MOS score prediction and showed high degrees of accuracy with at least 81% in the case of buffer-based MOS, 73% in the case of bitrate-based MOS and 63% in the case of buffer-based MOS. According to the results, we see also that the \(KNN\)_method_, with 11 \(n\_neighbors\) and _square inverse distance weight_, is the best one in the prediction of buffer-based MOS and bitrate-based MOS with mean error of 0.07 and 0.1. This confirms the efficiency of the \(KNN\)_method_ in the context of QoE prediction as achieved in [25]. Furthermore, and concerning the user's MOS prediction, we find that the two ensemble methods, \(RF\) and \(GBT\), achieve the best prediction performance result with mean error rate equal to 0.17, respectively, and 0.7. In fact, the correlation performance of \(GBT\), with 90%, is close to the values reported by other researchers in the literature. The results of predicting video KQI are given in Table 13.  
According to the results, the behavior of the ML models is not the same. Indeed, we observe that the ensemble methods (RF and GBT) give better results than the classical methods (DT and KNN). These are justified by the strong of ensemble ML methods compared to classical ones, where a classical ML method (DT or KNN) is built on a complete data set, using all characteristics/variables of interest, while ensemble methods (RF or GBT) select observations/lines and specific characteristics/variables to build multiple predictors from which the results are then averaged. In fact, the ensemble ML methods are more suitable when we have large interval values of the targets: bitrate and launch time. As the range of values for these targets are large, both the MAE and MAPE metrics do not necessarily lead to a good interpretation.