# Network Traffic Type-Based Quality of Experience (QoE) Assessment for Universal Services
## 4 Results
### Traffic Classification Performance  
The confusion matrix of classification results is presented in Figure 2, where the tick label annotations are OA: on-demand audio; WB: web browsing; IM: instant message; FS: file sharing; OG: online game; LV: live video; LO: location; LA: live audio; OV: on-demand video; MV: meeting video; and VC: voice call. Further, Table 4 shows the Precision, Recall Rate, and F1 Score of all the network service types, where Precision, Recall Rate, and F1 Score are defined as  
\[Precision=\frac{TP}{TP+FP} \tag{12}\]  
\[Recall=\frac{TP}{TP+FN} \tag{13}\]  
\[F1 Score=\frac{2\times Precision\times Recall}{Precision+Recall} \tag{14}\]  
where TP (True Positive) is the number of instances correctly classified as X, FP (False Positive) is the number of instances incorrectly classified as X, and FN (False Negative) is the number of instances incorrectly classified as Not-X. Precision means the proportion of the true positive samples that are determined by the classifier to be positive; the recall rate reflects the proportion of true positive samples that are correctly determined to the total positive samples. The F1 score is the harmonic average of the precision and the recall rate, when only both metrics are high, the F1 score will achieve a high value.  
On the test dataset of almost all network traffic types, our approach can achieve a competitive average classification accuracy of nearly 90%. From Figure 2, we find that the misclassification error rate between voice call and meeting video is significantly higher than that of other services; the most likely reason is that some of the voice call and meeting video use the same network transmission protocols, such as RTP, H.323, and SIP. Moreover, in some meeting video services, the voice call is integrated as one of the basic services, which means the two service flows have overlaps, thus higher classification error occurs.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **APP Number** & **File Number** & **Total Size** \\ \hline on-demand video & 10 & 357 & 17.9 GB \\ on-demand audio & 10 & 285 & 4.05 GB \\ live video & 9 & 300 & 16.6 GB \\ live audio & 11 & 354 & 1.41 GB \\ web browsing & 10 & 368 & 2.87 GB \\ instant message & 10 & 342 & 0.41 GB \\ file sharing & 10 & 331 & 12.49 GB \\ online game & 10 & 194 & 0.95 GB \\ location & 10 & 351 & 0.56 GB \\ meeting video & 10 & 244 & 9.12 GB \\ voice call & 10 & 255 & 2.47 GB \\ \hline
**Total** & **110** & **3781** & **67.83 GB** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Details of APP flow data collection.  
We compare the classification performance between our applied random forest and other popular classifiers, e.g., support vector machine (SVM), feedforward neural network (NN), and top-K nearest neighbors (top1-NN and top5-NN); the results shown in Figure 3 suggest that the random forest classifier outperforms the other classifiers in Precision, Recall Rate, and F1-Score.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Precision** & **Recall** & **F1-Score** \\ \hline on-demand audio & 87.98\% & 79.65\% & 83.61\% \\ web browsing & 87.68\% & 81.25\% & 84.34\% \\ instant message & 98.34\% & 95.69\% & 96.99\% \\ file sharing & 91.49\% & 90.94\% & 91.21\% \\ online game & 87.10\% & 83.51\% & 85.26\% \\ live video & 91.03\% & 91.33\% & 91.18\% \\ location & 92.02\% & 95.56\% & 93.79\% \\ live audio & 85.19\% & 90.96\% & 87.98\% \\ on-demand video & 75.57\% & 84.03\% & 79.58\% \\ meeting video & 73.80\% & 69.26\% & 71.46\% \\ voice call & 66.67\% & 78.43\% & 72.07\% \\ \hline
**Average** & **88.90\%** & **90.11\%** & **89.51\%** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Classification performance of all the network traffic types.  
Figure 2: Confusion matrix of the classification results.