## 4 Case Study
### Pattern Recognition and Prediction Algorithms  
As noted in Section 2, there are a variety of machine learning algorithms. However, each has its own weaknesses and strengths. As a result, a variety of classification and regression algorithms were selected for the prediction and pattern recognition algorithms in the proposed architecture. 1. **Classification:** Five classification algorithms were considered and compared: classification decision tree, logistic regression, SVM, naive Bayes and random forest classification tree. In order to apply classification algorithms, an additional variable called IS_DISRUPTED was added to the data. This variable takes a value of 0 when a terminal status flag (the status flag is generated using a type of error checking algorithm) is equal to 0 (not disrupted), taking a value of 1 when a terminal status flag is set to any positive value different than 0 (disrupted). This approach was used because classification algorithms need labelled data to predict the class into which a data entry might fall. By following this approach, it was possible to use the values of the network variables to classify an entry as either disrupted or not disrupted. Accuracy (Equation (2)) and precision metrics were tools that were used to evaluate performance. Precision (Equation (6)) indicates the proportion of true positives (Equation (3)) among classified as positive, while recall (Equation (7)), another applied tool, indicates true positives (Equation (3)) among all positive instances. These performance matrices examine different criteria. While the accuracy cannot reflect how well the classifier performed for positive classification, precision and recall can. Table 2 shows that SVM has a quite high accuracy, but very low precision and recall, which means that the method classified all instances as not disrupted. Naive Bayes has high recall, but low \begin{table}
\begin{tabular}{c c c} \hline \hline
**SUB_ROUTE_GROUPS** & **Network** & **Number of Terminals** \\ \hline SUB\_GRP\_1 & NET\_1 & 36 \\ SUB\_GRP\_2 & NET\_2 & 432 \\ SUB\_GRP\_3 & NET\_3 & 7 \\ SUB\_GRP\_4 & NET\_4 & 1 \\ SUB\_GRP\_5 & NET\_5 & 15 \\ SUB\_GRP\_6 & NET\_6 & 4 \\ SUB\_GRP\_7 & NET\_7 & 13 \\ SUB\_GRP\_8 & NET\_8 & 13 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Showing sub route groups, networks, and number of terminals. and precision, which indicates large number of false positives. Logistic regression and random forest algorithms have low recall and therefore have numerous false negatives. \[\mathcal{A}=\frac{Number\ of\ correct\ predictions}{Total\ number\ of\ predictions\ made} \tag{2}\]  
\[\text{True\ Positive\ Rate}=\frac{Number\ of\ True\ Positives}{Number\ of\ False\ Negatives +Number\ of\ True\ Positives} \tag{3}\]  
\[\text{True\ Negative\ Rate}=\frac{Number\ of\ True\ Negatives}{Number\ of\ True\ Negatives +Number\ of\ False\ Positives} \tag{4}\]  
\[\text{False\ Positive\ Rate}=\frac{Number\ of\ False\ Positive}{Number\ of\ True\ Negatives +Number\ of\ False\ Positives} \tag{5}\]  
\[\text{Precision}=\frac{True\ Positives\ Rate}{True\ Positive\ Rate+False\ Positive\ Rate} \tag{6}\]  
\[\text{Recall}=\frac{True\ Positives\ Rate}{True\ Positive\ Rate+False\ Negative\ Rate} \tag{7}\]  
A confusion matrix is shown for all algorithms in Table 3. In this work, the classification decision tree algorithm was considered as a good method for terminal data analysis because it obtains high accuracy, appropriate precision and recall, and good results in confusion matrix, compared to other algorithms, and therefore was considered for further analysis. Results of using the classification decision tree technique are presented in Figure 5. By following the tree, one can see which terminal variables lead to possible disruptions. For example, the highlighted node in red in Figure 5 indicates that if a terminal's ACTIVE_X1 is greater than 0.5, MIS_X7 (a variable in the dataset indicating equipment error) is greater