# Models versus Datasets: Reducing Bias through Building a Comprehensive IDS Benchmark
## 3 Proposed Framework  
Many research studies have been performed on anomaly detection where researchers report their model settings and performance measures. Our proposed methodology overcomes some of the important limitations in previous research studies, where researchers do not report their proposed model's output itself but what features contributed to the model to make certain predictions or influence the overall model performance. Similarly, many previous research studies generate bias results due to training the models only on a single dataset consisting of a limited network traffic pattern. A single dataset does not classify a vast range of attack classes and may lack recent traffic patterns [11]. This paper explores the output of various DL models by implementing SHAP and LIME, analyzing predictions, finding commonalities to avoid bias, improving classifier quality and reliability, and extracting top contributing features that influenced the model predictions most. Figure 1 depicts the proposed framework overview. To avoid the bias issue of a single dataset, we used various benchmark datasets. The details of these datasets are presented in Section 2.1. Datasets are then trained on eight different deep learning models known for processing sequential data and producing good results in anomaly detection. In our approach, we first implemented single classifiers and then hybrid classifiers to find the impact on classification. Detailed summary results were captured with numerous valuable information such as model settings, trainable parameters, training time, and model size, etc. The novelty of this work is reducing bias results and improving model interpretability by using LIME to explain local predictions and SHAP to explain both local and global model output. The top 20 important contributing features were extracted, which were then used to enhance the DL models by improving model settings, performance and choosing the best classifier for a resource constraint environment. Figure 1: Overview of the proposed framework. Our experiments involve various datasets and DL models. To preserve the space, we only present the architecture of one of our hybrid models, \"Autoencoder + TCN,\" in Figure 2. The same architecture has been used for other hybrid model implementations shown in Figure 1. In other implementations, we replaced the \"TCN Structure\" block in Figure 2 with LSTM, BRNN, or BLSTM. Model-specific settings are reported in Table 2 (individual classifiers) and Table 3 (hybrid classifiers). In this architecture, first, the autoencoder takes the input shape of the dataset, reduces its dimensions to a smaller size (also known as latent space) and then reconstructs the original data from the compressed representation. Autoencoders generate reconstruction errors and guarantee very high accuracy with low latency detection [78]. The output of autoencoders is passed into a TCN structure. The pooled outputs of TCN block layers are flattened into a one-dimensional array and are passed to a fully connected layer. The last layer is passed with total output classes using the \"softmax\" activation function to make predictions on each label.