# Efficacy Improvement of Anomaly Detection by Using Intelligence Sharing Scheme
## 5 Experimental Evaluation and Discussion
### Improvement of Per-Flow Anomaly Scores
The 5 th percentile (displayed in the Figure 8 as lower whiskers) is a robust indicator of the behavior of networks with poor classification strength. In this percentile, both global reputation algorithms improve substantially upon the base case of \"plain\" with \"pairwise\" offering the best performance. Local also improves the 5 th percentile, but not nearly as much as the global algorithms. The median does not differ very considerably or predictably between algorithms with the exception of HTTP(S) traffic, where \"local\" and \"average\" give inferior results. Interestingly, the pairwise algorithm did not suffer from this effect nearly as much. Another interesting feature of the results is the distribution of outliers (displayed as circles) as given Figure 8B, which shows that the pairwise algorithm eliminated all outliers with AUC < 88%, but every other algorithm suffered from at least some networks with severely degraded performance. This indicates the high robustness of the pairwise reputation model. Of all the tested algorithms, \"pairwise\" exhibited the best AUC mean, minimum, 5 th and 10 th percentiles, and its standard deviation was smallest for all data sets. Other performance characteristics (including the global reputation table ROC curve) were never substantially worse than for other algorithms. These results suggest that \"pairwise\" is an effective algorithm for building a global reputation model GRM. Surprisingly, not much difference is immediately obvious between HTTP and HTTP(S) results despite the very different characteristics of their underlying data. The Wilcoxon signed-rank test was used [41; 42; 43] to test the significance of differences between all pairs of HTTP and HTTP(S) measurements: plain, local, global and pairwise. Figure 8A,B shows that Plot (A) differs from plot (B) only in the scale of the y-axis to highlight different parts of the plots. Boxes span between the 25 th and 75 th percentiles, whiskers span between the 5 th and 95 th percentiles, and circles represent all the networks outside the range of the whiskers. The median is marked by a horizontal line. AUCs are expressed as percentages in Table 4. The highest value in each row is bold with the exception of the standard deviation. However, the differences are very small in some rows (median) while being very large in others (minimum, 5 th percentile). Higher percentiles are not displayed because their differences were mostly negligible. The maximum was in all cases equal to 100.0%. In Figure 9, the top chart contains AUCs computed by the plain algorithm and the bottom chart contains AUCs computed by the pairwise algorithm. In both cases, the AUC scores are not linearly dependent on the amount of network traffic. Networks with both great and poor performance are found in all the parts of the size spectrum. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Data**} & \multirow{2}{*}{**Algorithm**} & \multirow{2}{*}{**Mean**} & \multicolumn{2}{c}{**Standard**} & \multirow{2}{*}{**Minimum**} & **5 th** & **10 th** & **25 th** & **Median** \\  & & & & **Deviation** & & **Percertile** & **Percertile** & **Percertile** & **Median** \\ \hline \multirow{4}{*}{**Dataset**} & \multirow{2}{*}{**Mean**} & Plain (\%) & 97.3 & 4.4 & 69.1 & 88.5 & 94 & 97.4 & **98.6** \\  & & Local (\%) & 97.4 & 4 & 65.2 & 91.3 & 94.3 & 97 & 98.2 \\  & & Average (\%) & 97.3 & 4.8 & 51.9 & 92.8 & 94.5 & 96.6 & 98.4 \\  & & Pairwise (\%) & **98.1** & 2 & **88.2** & **94.7** & **96.4** & **97.5** & 98.5 \\ \hline \multirow{4}{*}{**Dataset**} & \multirow{2}{*}{**Mean**} & Plain (\%) & 97.1 & 4.8 & 72.7 & 88.2 & 91.9 & 97 & 98.7 \\  & & Local (\%) & 97.2 & 6.4 & 33.8 & 91.3 & 94.6 & 96.9 & 98.7 \\  & & Average (\%) & 98.3 & 2.4 & 85.9 & 94.1 & 95.9 & **98.1** & **99.2** \\  & & Pairwise (\%) & **98.4** & 2.2 & **Figure 8: Comparison of AUCs (**A**) per-flow anomaly score (**B**) all pairwise algorithm. For neither of these were there any significant differences between their population mean ranks (\(p<0.01\)).