# Ensemble Classifiers for Network Intrusion Detection Using a Novel Network Attack Dataset +
## 5 Experiments
### Machine Learning Algorithm Performance Comparison  
The experiment in this section had two phases. In the first phase, we compared the performance of the classifiers using the GTCS dataset with the full extracted. To evaluate the performance of the ML classifiers, we used precision, recall, and F1 score, which are the most common measures for evaluating the performance of anomaly detection models. Precision refers to the portion of relevant instances among the retrieved instances. Recall refers to the portion of relevant retrieved instances in the total number of relevant instances. F1 score is the harmonic mean of precision and recall. These three measures depend on the confusion matrix, where four possible situations can be defined, as shown in Table 3. \begin{table}
\begin{tabular}{c c c} \hline
**Class** & **Training Set** & **Occurrence Percentage** \\ \hline
**Normal** & 139,186 & 26.98\% \\ \hline
**Botnet** & 93,021 & 17.97\% \\ \hline
**Brute Force** & 83,858 & 16.20\% \\ \hline
**DDoS** & 131,211 & 25.35\% \\ \hline
**Infiltration** & 70,202 & 13.56\% \\ \hline
**Total** & 517,478 & 100.00\% \\ \hline \end{tabular}
\end{table} 
Table 2: Number of samples for normal and attack classes. \begin{table}
\begin{tabular}{c c} \hline
**Measure Metric** & **Explanation** \\ \hline
**True Positive Rate (TP)** & Correctly classified anomalous instances as an anomaly \\ \hline
**True Negative Rate (TN)** & Correctly classified normal instances as normal \\ \hline
**False Negative Rate (FN)** & Wrongly classified anomalous instances as normal \\ \hline
**False Positive Rate (FP)** & Wrongly classified normal instances as an anomaly \\ \hline
**Precision** & _TP_* (_TP_ + _FP_) \\ \hline _Recall_ & _TP_* (_TP_ + _FN_) \\ \hline _F1 score_ & (_2_* Precision_ * _Recall_) (_Precision_ + _Recall_) \\ \hline \end{tabular}
\end{table} 
Table 3: Evaluation metrics. The results of this phase are summarized in Table 4.  
In the second phase, we applied the InfoGainAttributeEval algorithm with Ranker [75] to reduce the dimensions of the dataset. InfoGainAttributeEval evaluates the worth of an attribute by measuring the information gain with respect to the class. \[\text{InfoGain(Class,Attribute)}=\text{H(Class)}-\text{H(Class|Attribute)}\]  
Tables 4 and 5 present a comprehensive comparison of the classifiers in terms of classification accuracy, precision, recall, TPR, FPR, _F_-measure, and ROC area. Table 6 presents the accuracy of each classifier in classifying different classes in the GTCS dataset in the two phases. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Classifier** & **Class** & **Phase I** & **Phase II** \\ \hline \multirow{4}{*}{**NB**} & Normal & 86.00\% & 86.90\% \\  & Botnet & 87.90\% & 88.80\% \\  & Brute Force & 80.00\% & 80.90\% \\  & DDoS & 83.80\% & 84.70\% \\  & Infiltration & 78.96\% & 79.86\% \\ \hline \multirow{4}{*}{**Logistic**} & Normal & 92.50\% & 93.40\% \\  & Botnet & 92.80\% & 93.70\% \\  & Brute Force & 84.80\% & 85.70\% \\  & DDoS & 91.58\% & 92.48\% \\  & Infiltration & 81.30\% & 82.20\% \\ \hline \multirow{4}{*}{**MLP**} & Normal & 93.60\% & 94.50\% \\  & Botnet & 93.50\% & 94.40\% \\  & Brute Force & 90.00\% & 90.90\% \\  & DDoS & 98.89\% & 99.79\% \\  & Infiltration & 82.30\% & 83.20\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Classifiers’ accuracy detection for different classes of attacks. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Classifier** & **Accuracy** & **TPR** & **FPR** & **Precision** & **Recall** & _F_**-Measure** & **ROC Area** \\ \hline
**NB** & 86.81\% & 0.869 & 0.116 & 0.912 & 0.864 & 0.885 & 0.921 \\ \hline
**Logistic** & 91.88\% & 0.936 & 0.097 & 0.923 & 0.931 & 0.924 & 0.930 \\ \hline
**MLP** & 92.90\% & 0.944 & 0.085 & 0.932 & 0.939 & 0.931 & 0.937 \\ \hline
**SMO** & 92.13\% & 0.937 & 0.093 & 0.925 & 0.932 & 0.926 & 0.917 \\ \hline
**IBK** & 94.23\% & 0.948 & 0.055 & 0.943 & 0.945 & 0.942 & 0.942 \\ \hline
**J48** & 94.29\% & 0.951 & 0.054 & 0.949 & 0.945 & 0.944 & 0.943 \\ \hline \hline \end{tabular}
\end{table}