# Efficacy Improvement of Anomaly Detection by Using Intelligence Sharing Scheme
## 5 Experimental Evaluation and Discussion
### Improvement of Per-Flow Anomaly Scores  
In this section, we explore the possibility of using a global reputation table to improve the anomaly scores in individual networks (the third arrow in Figure 2). The reputation model was used in the manner described in Section 4.3. There are two ways in which this procedure might improve classification. 1. Smoothing: Since many values are averaged together to arrive at the reputation score, much of the noise of the detectors will be filtered out. 2. Information sharing: Many different networks participate in the creation of the reputation table, which is beneficial as discussed in Section 1. If smoothing is the main benefit of the reputation table, it may be possible to simply build a network-local reputation table and use that--no information sharing is necessary. Because of this, smooth evaluation and information sharing separately is necessary to know whether information sharing is beneficial or not. Three types of anomaly detection algorithms were evaluated:  
1. Plain: The anomaly scores corresponding to individual flows; no reputation table is used. 2. Local: Anomaly scores are enhanced by a network-local reputation table. Since this reputation table is built using only one network, there are no normalization, combination or missing data issues, and this reputation table is a simple per-hostname average of anomaly scores. 3. Global: Anomaly scores enhanced by a global reputation table. The global reputation table can be generated by either of the four different algorithms described in Section 5.3. Due to their poor performance, norm-all and norm-isect were not attempted; instead focus was on average and pairwise algorithms. By discarding the norm-isect algorithm, the full set of 310 networks was used. The global and pairwise reputation models were re-computed by using all 310 networks and these updated models were utilized in the following experiments. The rationale behind evaluating local vs. global is that most of the smoothing effect will be present in both of them; the strength of information sharing (mostly) alone can be assessed by comparing their performance. If the global algorithms significantly improve classification strength over both plain and local algorithms, information sharing is beneficial. In order to test the effects of information sharing on HTTP and HTTP(S) flows separately, the performance of the algorithms both in isolation and combined will be tested. This gives us four different classification algorithms over three distinct datasets (per network), for a total of 3720 distinct performance measures. This number of different results makes the ROC curves too granular; instead, only the AUC scores will be used to evaluate the classification strengths. More than half of the networks do not contain even a single flow labeled as anomalous. For these, the AUC score is not meaningful and they are excluded from the comparison, leaving just 1432 performance measures. We expect the classification performance of \"plain\" to be significantly worse for HTTP(S) traffic than for HTTP traffic because of the lower information content discussed in Section 2. The reputation table should then improve on HTTP(S) traffic more than on HTTP traffic because of the sharing of information gathered by MITM attacks in some networks. \begin{table}
\begin{tabular}{c c} \hline \hline
**Algorithm Names** & **AUC Score** \\ \hline average & 86.6\% \\ pairwise & 86.2\% \\ norm-all & 80.8\% \\ norm-isect & 76.8\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Global reputation table of area under the ROC curve (AUC) score comparison on the biggest 100 networks. The performance of all tested algorithms is visualized as a box plot in Figure 8A,B. Numerical comparison of AUC per-flow anomaly score is presented in Table 4. It is visible from both Figure 8 and Table 4 that the largest differences between different algorithms are in the lowest percentiles.