# A Local Feature Engineering Strategy to Improve Network Anomaly Detection +
## 6 Strategy Validation
### Metrics  
We adopted three different metrics, in the validation process, to evaluate the performance of the proposed LFE strategy: (i) the _specificity_[102]; (ii) the Matthews Correlation Coefficient (MCC) [103; 104]; and (iii) the Area Under the Receiver Operating Characteristic curve (AUC) [105].  
The first two are based on the results related to the confusion matrix described in Section 3.2, whereas the last one is related to the Receiver Operating Characteristic (ROC) curve. The combination of these metrics provides us a reliable evaluation of the strategy effectiveness, regardless to data imbalance (i.e., the numerical difference between normal and intrusion events), according to the literature in this research area.  
* The formalization of the specificity metric, which is aimed to assess the performance in terms of the system capability to detect intrusions (i.e., the true negative rate), is shown in Equation (6), according to the formal notation provided in Section 4. Denoting as \(\hat{E}\) the unclassified instances, TN reports the number of events classified as intrusion, correctly, and FP reports the number of intrusion events classified as normal, erroneously. \[Specificity(\hat{E})=\frac{TN}{(TN+FP)}\] (6)
* The formalization of the MCC metric, which respect to other confusion-matrix metrics is able to operate even with unbalanced data, is shown in Equation (7). It provides a result in the \([-1,+1]\) range, where the correctness of all the performed classification leads to \(+1\), whereas the incorrectness of all of them leads to \(-1\), and \(0\) indicates the typical performance reached by a random classifier. \[MCC(\hat{E})=\frac{(TP\cdot TN)-(FP\cdot FN)}{\sqrt{(TP+FP)\cdot(TP+FN)\cdot( TN+FP)\cdot(TN+FN)}}\] (7)
* The formalization of the AUC metric, which is able to evaluate the capability of a classifier to differentiate the normal from the intrusion events, regardless their numerical imbalance, is shown in Equation (8). Given set \(E\), denoting as \(\kappa\) all the possible score comparisons related to each event \(e\), the final result (in the \([0,1]\) range) is given by the average of them, where \(1\) indicates the best classification performance.  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Class** & **Description** & **Target** \\ \hline \multirow{4}{*}{RAA} & \multirow{4}{*}{_Remote Access Attack_} & Gets an access to a remote system by using several \\  & & techniques (e.g., Guess\_password, Ftp\_write, Imp, Phf, \\  & Multihop, Warezmaster, Mock, Xsnoop, Snmpguess, \\  & Snmpgetattack, Httptunnel, Sendmail, and Named). \\ \hline \multirow{2}{*}{NNA} & \multirow{2}{*}{_Normal Network Activity_} & This is the class we used in order to identify the normal \\  & & network events). \\ \hline \hline \end{tabular}
\end{table}
Table 8: _Cont._  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & **PEA** & **DSA** & **RSA** & **RAA** & **NNA** \\  & **Events** & **Events** & **Events** & **Events** \\ \hline _Total_ & 119 & 53,387 & 14,077 & 3879 & 77,053 \\ \% & 0.08 & 35.95 & 9.48 & 2.61 & 51.88 \\ \hline \hline \end{tabular}
\end{table}
Table 9: Event categories numerical overview.  
\[AUC=\frac{1}{I_{+}\cdot I_{-}}\sum_{1}^{|I_{+}|}\sum_{1}^{|I_{-}|}\kappa(i_{+},i_{ -})\ \ \text{with}\ \ \kappa(i_{+},i_{-})=\left\{\begin{array}{ll}1,&if\ i_{+}>i_{-}\\ 0.5,&if\ i_{+}=i_{-}\\ 0,&if\ i_{+}<i_{-}\end{array}\right. \tag{8}\]