# Models versus Datasets: Reducing Bias through Building a Comprehensive IDS Benchmark
## 3 Proposed Framework
\begin{table}
\begin{tabular}{c l l l l l} \hline \hline \multirow{2}{*}{**Classifier**} & \multirow{2}{*}{**Dataset**} & \multicolumn{1}{l}{**Trainable**} & \multicolumn{1}{l}{**Training**} & \multicolumn{1}{l}{**Model**} & \\  & & **Parameters** & **Time** & **Size** & **Accuracy** \\ \hline MLP & Bot-IoT & 32,581 & 7 min & 434 KB & 100.00\% \\  & CICIDS2017 & 38,255 & 5 min & 500 KB & 99.90\% \\  & NSL-KDD (*a) & 43,205 & 1 min & 193 KB & 78.20\% \\  & NSL-KDD (*b) & 43,205 & 0.5 min & 560 KB & 98.90\% \\  & UNSW\_NB15 (*a) & 52,890 & 0.5 min & 673 KB & 38.90\% \\  & UNSW\_NB15 (*c) & 53,786 & 5 min & 684 KB & 97.80\% \\  & N-BaIoT & 42,411 & 2 min & 550 KB & 90.80\% \\  & KDD CUP 99 & 42,693 & 1 min & 553 KB & 99.90\% \\ CNN & Bot-IoT & 12,901 & 11 min & 198 KB & 100.00\% \\  & CICIDS2017 & 18,495 & 9 min & 264 KB & 100.00\% \\  & NSL-KDD (*a) & 32,525 & 1 min & 322 KB & 80.10\% \\  & NSL-KDD (*b) & 23,525 & 1 min & 322 KB & 98.60\% \\  & UNSW\_NB15 (*a) & 33,170 & 1 min & 436 KB & 37.30\% \\  & UNSW\_NB15 (*c) & 34,066 & 15 min & 447 KB & 97.70\% \\  & N-BaIoT & 22,683 & 5 min & 313 KB & 90.90\% \\  & KDD CUP 99 & 23,013 & 2 min & 316 KB & 99.90\% \\ LSTM & Bot-IoT & 3,100,261 & 162 min & 36 MB & 100.00\% \\  & CICIDS2017 & 3,188,495 & 248 min & 37 MB & 100.00\% \\  & NSL-KDD (*a) & 3,270,245 & 23 min & 38 MB & 73.60\% \\  & NSL-KDD (*b) & 3,270,245 & 20 min & 38 MB & 98.70\% \\  & UNSW\_NB15 (*a) & 3,423,930 & 23 min & 39 MB & 51.50\% \\  & UNSW\_NB15 (*c) & 3,438,266 & 532 min & 39 MB & 98.00\% \\  & N-BaIoT & 3,256,011 & 121 min & 37 MB & 90.80\% \\  & KDD CUP 99 & 3,262,053 & 61 min & 37 KB & 99.90\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Individual DL classifiers summary. Figure 2: Autoencoder + TCN architecture. The TCN block in Figure 2 is the layered implementation of the dilated causal convolution of the TCN architecture. Figure 3 reflects the details of this model. TCN block takes three parameters as follows:  
_TCN block (filters, kernel_size, dialation_rate)_  
where \"filters\" are similar to units in LSTM, and they affect the model size. A larger filter size is preferred. It helps to train the model in parallel and faster, unlike RNN and LSTM, where predictions must wait for the predecessor results [79]. In TCN, a longer input sequence can be processed as a whole instead of sequentially. The \"kernel_size\" parameter determines the size of each filter in each convolution layer. It helps to calculate how much of the input is used to calculate each value in the output. A larger kernel_size helps detect complex features. The \"dialation_rate\" parameter represents a fixed step between two adjacent filters. A \"dialation_size=1\" is the same as a regular convolution in a CNN network. A larger dilation rate captures a bigger input range on the top convolution layer, making a TCN more receptive [79]. Our TCN structure starts with a CONV1D dilated causal convolution layer, followed by a batch normalization layer to obtain high accuracy values and increase model training. We then implemented a Rectified Linear Unit \"Relu\" to allow quick network convergence. We then added a \"Dropout\" layer to avoid over-fitting and added regularization by randomly dropping 30% of weights connected to certain