# On Predicting Ticket Reopening for Improving Customer Service in 5G Fiber Optic Networks
## 4 A Bayesian Network Classifier Trained on a Further Reduced Dataset
### Model Performance Evaluation and Discussion on the Results  
For the purpose of evaluating the model performance, we adopted the same approach as discussed at the beginning of Section 3.2. However, in order to further test the robustness of the Bayesian network classifier, we also carried out the experiment of creating 1000 random pairs of training/test sets according to the 70/30 rule, i.e., 70% of the \(X^{\prime}_{red}\) dataset was used for training purposes and the rest for testing. The measured KPIs were very similar, thus testifying the effectiveness of the Bayesian network classifier as well as its robustness. The average values of the KPIs measured throughout these tests are reported below in Table 6 in order to compare them with the results of the different classifiers trained in Section 3. The validation process was completed by comparing the accuracy measure achieved by the Bayesian network classifier against the performance of the other classifiers. From Table 6, it is clear that the Bayesian Network classifier trained on \(X^{\prime}_{red}\) outperforms the classifiers introduced in Section 3. Among the trained classifiers, according to the accuracy and AUC measures, the Bayesian Network classifier proves to be the most effective at minimizing the error (8). From the results obtained, we also infer the combinations of features in \(X^{\prime}_{red}\) that are most probably the reason for ticket reopening: namely, they are the _events_ listed below:  
1. [label=(0)]
2. Ticket close code (d) AND Constant bitrate (downstream) in bin 2 AND Power (upstream) in bin 4 AND Attenuation (upstream) in bin 3;
3. Ticket close code (k) AND Constant bitrate (downstream) in bin 1 AND Power (upstream) in bin 1 AND Attenuation (downstream) in bin 3;
4. Ticket close code (i) AND Constant bitrate (downstream) in bin 4 AND Power (upstream) in bin 3 AND Attenuation (upstream) in bin 3;
5. Ticket close code (i) AND Constant bitrate (downstream) in bin 2 AND Power (upstream) in bin 2 AND Attenuation (upstream) in bin 3;
6. Ticket close code (i) AND Constant bitrate (downstream) in bin 3 AND Power (upstream) in bin 2 AND Attenuation (upstream) in bin 1;
7. Ticket close code (p) AND Constant bitrate (downstream) in bin 2 AND Power (upstream) in bin 3 AND Attenuation (upstream) in bin 3; \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Trained Classifier** & **Accuracy** & \
\begin{tabular}{c} **Gini** \\ **Coefficient** \\ \end{tabular} &
\begin{tabular}{c} **Youden** \\ **Index** \\ \end{tabular} & **AUC** \\ \hline Decision Tree trained on \(X_{red}\) & 79\% & 0.1 & 0.2 & 0.55 \\ Random Forest trained on \(X_{red}\) & 80\% & 0.9 & 0.83 & 0.95 \\ Gradient Boosting trained on \(X_{red}\) & 79\% & 0.8 & 0.84 & 0.90 \\ Logistic Regression trained on \(X_{red}\) & 81\% & 0.12 & 0.25 & 0.56 \\ ANN trained on \(X_{red}\) & 67\% & \(-\)0.12 & 0.06 & 0.46 \\ Support Vector Machine trained on \(X_{red}\) & 69\% & 0.78 & 0.78 & 0.89 \\ Bayesian Network classifier trained on \(X^{\prime}_{red}\) & 96\% & 0.90 & 0.79 & 0.95 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Numerical results for the second dataset. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Variable** & **Bin 1** & **Bin 2** & **Bin 3** & **Bin 4** & **Bin 5** \\ \hline Constant bitrate (downstream) [bits per sec] & \textless{}36,200 & [36,200; 54,000] & [54,000; 64,200] & [64,200; 87,100] & \textgreater{}87,100 \\ Attenuation (upstream) [dB] & \textless{}10.2 & [10.2; 16.1] & [16.1; 21.1] & \textgreater{}21.1 & - \\ Power (upstream) & \textless{}\textless  
* Ticket close code (p) AND Constant bitrate (downstream) in bin 3 and Power (upstream) in bin 4 AND Attenuation (upstream) in bin 4;
* Ticket close code (p) AND Constant bitrate (downstream) in bin 4 AND Power (upstream) in bin 1 AND Attenuation (upstream) in bin 2. Table 7 reports them with the corresponding number of occurrences of such combinations of QoS parameters.