# Models versus Datasets: Reducing Bias through Building a Comprehensive IDS Benchmark
## 4 Experiments, Results, and Analysis
### Top Contributing Features  
This section presents the aggregated knowledge of important features identified by each of the eight DL classifiers from all eight datasets. Our approach was to extract the top 20 features and is divided into three sub-categories, (a) Table 4 provides a list of features identified by every classifier. These features must be considered the most important features (b) Table 5 provides a list of features identified by six or more classifiers, and (c) Table 6 provides a list of features identified by five or fewer classifiers. Each classifier reports features sorted by top contributing to low contributing; however, we are not reporting the ordering sequence here because it will require a very big table size if we report the individual output of eight classifiers on eight datasets. Our goal is to present an aggregated knowledge and report those features to which all or the majority of classifiers have been given some importance in the top 20 list regardless of the ordering sequence. \begin{table}
\begin{tabular}{l l} \hline \hline
**Dataset** & **Feature Name** \\ \hline BoT\_IoT & mean, min, N\_IN\_Conn\_P\_DstIP, N\_IN\_Conn\_P\_SrcIP, Pkts\_P\_State\_P\_Protocol\_P\_DestIP, proto\_number, seq, state\_number, stddev, stime, TnP\_Per\_Dport, dur, TnP\_PDstIP \\ \hline CICIDS2017 & ACK Flag Count, Destination LP, Fwd Packet Length Max, Init\_Win\_bytes\_forward, Min Packet Length, Source I.P., Source Port \\ \hline NSL-KDD (*a) & count, dst\_host\_count, dst\_host\_error\_rate, dst\_host\_same\_src\_port\_rate, dst\_host\_same\_srv\_rate, dst\_host\_server\_rate, dst\_host\_srv\_count, service\_http, dst\_host\_srv\_server\_rate, flag\_S0, logged\_in, same\_srv\_rate, serv\_error\_rate \\ \hline NSL-KDD (*b) & count, dst\_host\_count, dst\_host\_error\_rate, dst\_host\_same\_src\_port\_rate, dst\_host\_same\_srv\_rate, dst\_host\_server\_rate, dst\_host\_srv\_count, same\_srv\_rate, service\_http, srv\_server\_rate, logged\_in \\ \hline UNSW\_NB15 (*a) & ct\_dst\_src\_ltm, ct\_srv\_dst, ct\_srv\_src, ct\_state\_ttl, dmean, dttl, service\_dns, service\_-, sttl \\ \hline UNSW\_NB15 (*c) & ct\_dst\_port\_ltm, ct\_dst\_src\_ltm, ct\_src\_dport\_ltm, ct\_srv\_dst, ct\_state\_ttl, dmeansz, dttl, service\_dns, service\_-, sttl, swin \\ \hline N-BaIoT & MI\_dir\_L0\_1\_weight \\ \hline KDD CUP 99 & count, dst\_host\_count, logged\_in, protocol\_type\_icmp, dst\_host\_same\_src\_port\_rate, dst\_host\_srv\_count, service\_cer\_i, same\_srv\_rate, service\_http \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Top features identified by all classifiers. \begin{table}
\begin{tabular}{l l} \hline \hline
**Dataset** & **Feature Name** \\ \hline BoT\_IoT & flgs\_number, litme, max, sum, Pkts\_P\_State\_P\_Protocol\_P\_SrcIP, TnP\_PerProto, TnP\_P\_SrcIP \\ \hline CICIDS2017 & Protocol, Average Packet Size, Destination Port, PSH Flag Count, Fwd IAT Std \\ \hline NSL-KDD (*a) & dst\_host\_diff\_srv\_rate, dst\_host\_srv\_error\_rate, error\_rate, service\_private \\ \hline NSL-KDD (*b) & diff\_srv\_rate, dst\_host\_diff\_srv\_rate, dst\_host\_srv\_error\_rate, error\_rate, flag\_SF, error\_rate \\ \hline UNSW\_NB15 (*a) & id, smean, ct\_src\_dport\_ltm, ct\_src\_ltm, proto\_tcp \\ \hline UNSW\_NB15 & ct\_srv\_src, smeansz, proto\_tcp, proto\_udp \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Top features identified by six or more classifiers. Table 4 reveals that some features in each of the datasets carry more importance than others. Every classifier picked such features in making decisions. Similarly, as shown in Table 5, some other features were given a little less importance by only one out of eight classifiers who did not think it was an important feature. Lastly, many other features were considered important by five or fewer classifiers. These features in the third category still carry importance but not as much compared to the first two categories. Feature selection has an important role in machine learning.