# Research on DoS Traffic Detection Model Based on Random Forest and Multilayer Perceptron

Authors: Hongyan He; Guoyan Huang; Bing Zhang; Zhangqi Zheng
Journal: Security and Communication Networks
 (2022)

---

## Abstract

Denial of service (DoS) attack is a typical and extremely destructive attack, which poses a serious threat to the Internet security and is highly concealed, making it difficult to detect. In response to this problem, the paper proposes an efficient DoS attack traffic detection method, Random Forest and Multilayer Perceptron hybrid network attack detection algorithm (RF-MLP). At first, it is adopted that the random forest algorithm can be used for feature selection and the optimal threshold can be determined by drawing a learning curve; therefore the optimal feature subset is determined. Then the optimal feature subset is used as the input of the multilayer perceptron for training. We will analyze the experimental results obtained using different configurations by varying the number of training neurons and the number of hidden layers of the multilayer perceptron network in order to improve the accuracy and reduce the number of false results. Using the real network traffic CICIDS2017 dataset and UNSW-NB15 dataset to evaluate the method in this paper, the results show that the model can effectively detect and classify DoS attacks, the accuracy rate can reach 99.83% and 93.51%, and there is also a significant reduction in the false alarm rate, verifying the effectiveness of the method and its ease of use.


---

## Body

## 1. Introduction

With the Internet occupying an increasingly important position in our lives, in recent years, website users have become increasingly threatened by DoS. As one of the oldest and most common attacks types on the Internet, the real purpose of a DoS attack is to disrupt and organize access to services to legitimate users by overconsuming and wasting network bandwidth and computing resources [1]. From 1983 until now, DoS attack is still a malicious attack which does great harm to the network. DoS attack brings great threat to the rapid development of Internet security.The principle of a DoS attack is very simple, as shown in Figure1. It is generated by overwhelming a large amount of unnecessary network traffic or by forcing computing resources to waste unnecessary task processing or storage capacity and is aimed at undermining service operation and reliability. A malicious user sends multiple authentication requests to the server to make it full, and the return addresses of all requests are forged. When the server attempts to return the authentication results to users, it will not be able to find these users. In this case, the server has to wait until the connection times out before closing the connection. During this period, the attacker will continue to send false requests until the server is overloaded and unable to provide normal services.Figure 1
Principle of DoS attack.DoS attacks are implemented in three parts: the attacker, the master, and the agent. These three work together to complete the entire attack process.Attacker. The computer is the main control platform of the attack and can make any host on the network. The attacker can manipulate the complete attack process, which sends instructions to the main controller to attack.Main console. The attacker illegally invades and controls some hosts, and a large number of proxy machines are controlled by these hosts, respectively. Install a specific program on the host through the backdoor, so that it can accept special instructions sent by the attacker and at the same time can send these instructions to the proxy host [2].Agent. Some hosts are invaded and controlled by an attacker. They are running attack programs, accepting and running commands from the host. The proxy host acts as an executor for the attacker. It is possible to launch an attack directly against the target host.In complex network environments, IDS play an integral and very important role in network security systems, where security is key to the operation of system. A variety of techniques have been implemented in IDS to detect DoS attack traffic, including misuse-based and anomaly-based approaches. One of the misuse-based methods aims to detect known attack types by using signatures of network attack traffic. Anomaly-based detection is possible by monitoring network traffic to discover traffic deviations. With regard to IDS, there has been some research into machine learning (ML) to detect DoS attacks, with algorithms such as decision tree, Parsimonious Bayes, SVM, and KNN being very popular [3]. There are many types of DoS attacks, each in its own way, but they all have the same aim, which is to prevent authorized users from accessing their authorized services. However, faced with massive and large scale data in real network environments, the efficiency of classification is relatively low and cannot achieve the requirements of intelligent and efficient analysis of massive network data and prediction of high-dimensional network feature learning, so in this case, a high degree of accuracy and efficiency is required to detect DoS attacks to prevent serious damage.The other parts of the article are organized as follows: Section2 summarizes the current research literature. Section 3 introduces the main framework of the article. Section 4 shows the features of the data set, feature extraction, preprocessing applied to capture traffic, and experimental results. Section 5 gives the conclusion of this article and the outlook for the future work.


## 2. Literature Survey

At present, many research scholars have proposed a series of methods to study DoS attacks. Deep learning algorithms are widely used by a wide range of researchers and academics to improve performance with IDS, and a steady stream of related research results is being produced. Detection methods classified by detection mode: detection based on misuse and detection based on anomaly. The detection based on misuse method mainly utilizes the methods of feature matching, model inference, state transition, and expert system, which is to collect various attack features of existing DoS attacks and compare them. If the features match, DoS attacks are found. Snort Intrusion Detection System is an IDS that can be deployed on almost all major operating systems and uses a rule-based intrusion detection that is fast. However, a major problem with this method is that it cannot detect new types of attacks, only those already in the rule base. Hackers’ attack methods are constantly changing and new attack types are often created, which are often more damaging, and this method also suffers from a high false alarm rate. Anomaly-based DoS attack detection is to monitor the abnormal situation of the system usage on the system audit record, and it can detect security violations. Currently, most DoS attack detections are anomaly detections. Anomaly-based DoS detection methods are largely dependent on the detection model established, as different detection models correspond to different detection methods, which include four detection methods: artificial intelligence detection, pattern prediction, statistical detection, and machine learning detection.Tang et al. [4] proposed a density-based application space clustering algorithm (SADBSCAN), which groups network traffic and uses cosine similarity to determine whether the group of network traffic contains DoS attacks. Wang et al. [5] used the CNN algorithm to learn the spatial features of network traffic to realize the classification and identification of malicious network traffic through image classification methods. Torres et al. [6] convert network traffic features into characters that can be computed and then learn temporal features by RNN algorithms for detecting malicious and aggressive network traffic. Zhang et al. [7] used BP neural networks to obtain the basic probability assignment values for each attack type and then obtained the basic probability assignment values through a modified D-S evidence theory, which combines neural networks and D-S evidence theory together to implement a DoS attack detection method. Osorio et al. [8] proved the effectiveness of the method by studying two technologies, GMM and UBM, to detect DoS/DDoS network attacks. Reddy et al. [9] combined the Objection-Based Learning (OBL) and the crow search algorithm (CSA) to design an opposing crow search algorithm (OCSA) DoS attack traffic detection system, which uses the OCSA algorithm to select features and then hands off a subset of the features to an RNN classifier. Jinhui et al. [10] proposed a power number correlation check method, and the experimental results showed that the method can improve the detection rate of malicious nodes and effectively reduce the impact of hybrid DoS attacks on network traffic when the network is subjected to hybrid DoS attacks. Ait Tchakoucht et al. [11] proposed an IDS for probe and DoS attack types. The most important and most relevant feature subsets were selected by correlation feature (CFS) and information gain (IG) methods and packaged with four machine learning methods (Naive Bayes (NB), REPTree Random forest (RF), and C4.5). The validation was performed with KDD99 with good results. Perez-Diaz et al. [12] constructed a modular architecture, using 6 machine learning (ML) models (MLP, random tree, support vector machine (SVM), REP tree, RF, and J48) to train IDS, which improves the detection rate of DoS. Kshirsagar et al. [13] proposed an effective feature reduction method for detecting DoS attacks. Feature reduction methods combine information gain ratio, correlation, and relief. Latah et al. [14] proposed a combination of packet-based and stream-based approaches to achieve a reduced false alarm rate in intrusion detection systems. Hwoij et al. [15] built a detection model using five rule-based machine learning classifiers (PART, ZeroR, Decision Table, OneR, and JRip) to distinguish DoS attacks from ordinary traffic.Motivation for this paper is as follows:(i)
Traffic data can be collected and analyzed from the web gateway, so DoS attack defense can be implemented before the attack traffic reaches the target computer being attacked.(ii)
As the network flow data is host and operating system independent, the DoS attack classifier can be used on any web server without any configuration changes at the server level.(iii)
In a real network environment, the huge volume and complexity of network traffic data is a problem that we cannot ignore [16]. Due to the high-dimensional nature of network traffic data, there are some redundant or irrelevant features in the network traffic data, which has a major hold on the speed of classification and even on the accuracy of the classification that cannot be underestimated.Therefore, this paper proposes a DoS attack traffic detection model based on RF-MLP, which combines the extracted network traffic statistical characteristics to calculate its importance, select the best subset of variables, and then use the best set of variables for MLP to conduct DoS attack classification research.


## 3. Based on RF-MLP Detection Method

In this section the model proposed in this paper, RF-MLP, will be described in detail. In the network anomaly detection method, reasonable analysis of traffic characteristics reveals the characterization effect of different traffic characteristics on attack behavior and studies the adaptability of different traffic characteristics to the network environment, so as to realize the rapid detection of attack behavior in the network environment. This paper uses the good performance of random forest in feature selection and the outstanding advantages of multilayer perceptron in training prediction. This paper integrates random forest and multilayer perceptron into RF-MLP model. Figure2 shows the method model. The experiments in this paper divide the captured network traffic data into a training set and a testing set for the study. Firstly, the network traffic is preprocessed, including missing value processing, numerical processing of labeled data, and data normalization processing. Secondly, based on the feature set, the random forest algorithm is used to calculate the importance of the network traffic features, and the features with high feature importance threshold and that are good for classifier training as well as ensuring the detection performance of the classifier are retained [17]. Finally, a subset of the best retained features is used as input data for training the model to complete the detection of DoS attack traffic.Figure 2
Flowchart of DoS abnormal traffic detection based on RF-MLP.


### 3.1. Random Forest Feature Selection

Each network traffic feature contains information that can reflect the variability of the network. However, irrelevant features in a network traffic dataset can place a severe load on computing resources and thus affect the performance of the system, so feature selection is an important and key step in finding the best set of features for an IDS, i.e., filtering out features that are irrelevant to the detection result and filtering the useful ones. Feature selection plays a very important role in the classification problem by attempting to map information from high-dimensional network data to low-dimensional data in order to remove redundant or irrelevant features [16]. Feature selection allows the model to be trained effectively in the training phase so that more accurate detection results can be obtained during the detection process.Random forest is constructed by multiple decision trees. At first, the features are sampled, then the current Gini index is calculated, and then the full splitting process is performed. The nonleaf nodes of each tree have a Gini index. After a tree is established, the importance of each node of the tree can be obtained. By sorting them according to the Gini index as the feature correlation, multiple decision trees are built at a time and generate multiple feature relevance rankings and finally select the average value of these features to obtain the final ranked feature importance ranking.The importance score is expressed by VIM, and the Gini index is expressed by GI. Assuming there arem features X1, X2, X3, ⋯, Xm, calculate the Gini index score VIMjGini, VIM for each feature Xj. The calculation formula of Gini index can be defined as(1)GIm=∑k=1k∑k′≠kpmkpmk′=1−∑k=1kpmk2.It can be seen that equation (1) represents K categories, and Pmk represents the proportion of category K in node m. It is the probability that two samples are randomly drawn from node m with inconsistent category labels.The importance measure of featureXj at node m, i.e., the change in Gini index of node m before and after branching, is expressed as(2)VIMjmGini=GIm−GIl−GIr.In equation (2), GIl and GIr denote the Gini indices of the two new nodes after node m branches, respectively.If the node in the decision treei in which feature Xj appears is in the set M, then the importance of Xi in the ith countable is expressed as(3)VIMijGini=∑m∈MVIMjmGini.Suppose there aren trees in RF; then(4)VIMjGini=∑i=1nVIMijGini.Finally, the importance scores of all features are normalized so that the sum of the importance of all features is equal to 1.(5)VIMj=VIMj∑i=1cVIMi.


### 3.2. MLP Classification

The MLP is a feedforward neural network. The main idea of MLP is to build multilayer neural networks by expanding the number of layers of hidden layers in the network. The model is a one-way multilayer structure, that is, from input to output one-way transmission. At the same time, it is also a nonlinear model, which consists of input layer, hidden layer, and output layer. Due to the nonlinear mapping and good self-learning ability of the multilayer perceptron, it has been widely used as a comprehensive classifier in many fields. By separately collecting normal traffic and DoS attack traffic data, the output of the multilayer perceptron is divided into 6 types: normal traffic that has not suffered from DoS attacks and 5 types of abnormal traffic that have suffered DoS attacks. The components of the MLP model are shown in Figure3. The network traffic sample to be detected is used as input to the MLP and the label of the type of network traffic attack is used as output, i.e., the estimated probability of the DoS attack for the input sample [1]. The output layer classifies traffic into benign and DoS attacks.(i)
Input layer: It consists ofn nodes, where the best subset of features in the network traffic is sent as input to the hidden layer, without any processing power.(ii)
Input layer: It consists ofn nodes. It consists of several neural system elements, which are computed to send their computational results to the next layer.(iii)
Output layer: It consists ofm nodes, each of which is computed as the actual output of the neural network, which is the estimated probability of the type of DoS traffic attack.Figure 3
MLP model used for DoS classification.Assume that the multilayer perceptron has L layers, where the first layer is the input layer,x1,x2,…,xn is the input data, and the Lth layer is the output layer, and y1,y2,…,ym is the output data. For the lth hidden layer, suppose there are nl neurons, and h1l,h2l,…,hnll is the output data of the l layer. Let ϖijl be the weight from the jth neuron of the l-1th layer to the ith neuron of the lth layer, bil is the bias of the ith neuron of the lth layer, and f⋅ is the activation function. Using the activation function can add nonlinear functions to the network, making the network’s fitting ability stronger. The output formulas of the ith neuron oil in the l layer and the ith neuron yi in the output layer are as follows:(6)hil−1=foil−1,oil=∑j=1sl−1ϖijlhjl−1+bil,yi=f∑j=1sL−1ϖijLhjL−1+biL.The error backpropagation algorithm then uses the output sensitivity of the neural network to quickly calculate the hyperparameters of the layers of the neural network, which include thresholds and weights, in order to achieve the goal of ensuring that the output values of the neural network are as close to the true values as possible [18]. Denote the training data set as x1,y1,x2,y2,…,xn,yn,yk∈−1,1, then for a certain training sample xk,yk, the output of the multilayer perceptron is y^i, and then the error function of the data set is defined as follows:(7)E=1n∑k=1nyk−y^k2.The weightW and bias B in the multilayer perceptron can be updated iteratively according to the following formula, where α is the learning rate and its value range is (0, 1).(8)Wl=Wl−α∂E∂Wl,Bl=Bl−α∂E∂Bl.By modifying the weight and bias of the network, the multilayer perceptron is trained to complete some complex tasks.


## 3.1. Random Forest Feature Selection

Each network traffic feature contains information that can reflect the variability of the network. However, irrelevant features in a network traffic dataset can place a severe load on computing resources and thus affect the performance of the system, so feature selection is an important and key step in finding the best set of features for an IDS, i.e., filtering out features that are irrelevant to the detection result and filtering the useful ones. Feature selection plays a very important role in the classification problem by attempting to map information from high-dimensional network data to low-dimensional data in order to remove redundant or irrelevant features [16]. Feature selection allows the model to be trained effectively in the training phase so that more accurate detection results can be obtained during the detection process.Random forest is constructed by multiple decision trees. At first, the features are sampled, then the current Gini index is calculated, and then the full splitting process is performed. The nonleaf nodes of each tree have a Gini index. After a tree is established, the importance of each node of the tree can be obtained. By sorting them according to the Gini index as the feature correlation, multiple decision trees are built at a time and generate multiple feature relevance rankings and finally select the average value of these features to obtain the final ranked feature importance ranking.The importance score is expressed by VIM, and the Gini index is expressed by GI. Assuming there arem features X1, X2, X3, ⋯, Xm, calculate the Gini index score VIMjGini, VIM for each feature Xj. The calculation formula of Gini index can be defined as(1)GIm=∑k=1k∑k′≠kpmkpmk′=1−∑k=1kpmk2.It can be seen that equation (1) represents K categories, and Pmk represents the proportion of category K in node m. It is the probability that two samples are randomly drawn from node m with inconsistent category labels.The importance measure of featureXj at node m, i.e., the change in Gini index of node m before and after branching, is expressed as(2)VIMjmGini=GIm−GIl−GIr.In equation (2), GIl and GIr denote the Gini indices of the two new nodes after node m branches, respectively.If the node in the decision treei in which feature Xj appears is in the set M, then the importance of Xi in the ith countable is expressed as(3)VIMijGini=∑m∈MVIMjmGini.Suppose there aren trees in RF; then(4)VIMjGini=∑i=1nVIMijGini.Finally, the importance scores of all features are normalized so that the sum of the importance of all features is equal to 1.(5)VIMj=VIMj∑i=1cVIMi.


## 3.2. MLP Classification

The MLP is a feedforward neural network. The main idea of MLP is to build multilayer neural networks by expanding the number of layers of hidden layers in the network. The model is a one-way multilayer structure, that is, from input to output one-way transmission. At the same time, it is also a nonlinear model, which consists of input layer, hidden layer, and output layer. Due to the nonlinear mapping and good self-learning ability of the multilayer perceptron, it has been widely used as a comprehensive classifier in many fields. By separately collecting normal traffic and DoS attack traffic data, the output of the multilayer perceptron is divided into 6 types: normal traffic that has not suffered from DoS attacks and 5 types of abnormal traffic that have suffered DoS attacks. The components of the MLP model are shown in Figure3. The network traffic sample to be detected is used as input to the MLP and the label of the type of network traffic attack is used as output, i.e., the estimated probability of the DoS attack for the input sample [1]. The output layer classifies traffic into benign and DoS attacks.(i)
Input layer: It consists ofn nodes, where the best subset of features in the network traffic is sent as input to the hidden layer, without any processing power.(ii)
Input layer: It consists ofn nodes. It consists of several neural system elements, which are computed to send their computational results to the next layer.(iii)
Output layer: It consists ofm nodes, each of which is computed as the actual output of the neural network, which is the estimated probability of the type of DoS traffic attack.Figure 3
MLP model used for DoS classification.Assume that the multilayer perceptron has L layers, where the first layer is the input layer,x1,x2,…,xn is the input data, and the Lth layer is the output layer, and y1,y2,…,ym is the output data. For the lth hidden layer, suppose there are nl neurons, and h1l,h2l,…,hnll is the output data of the l layer. Let ϖijl be the weight from the jth neuron of the l-1th layer to the ith neuron of the lth layer, bil is the bias of the ith neuron of the lth layer, and f⋅ is the activation function. Using the activation function can add nonlinear functions to the network, making the network’s fitting ability stronger. The output formulas of the ith neuron oil in the l layer and the ith neuron yi in the output layer are as follows:(6)hil−1=foil−1,oil=∑j=1sl−1ϖijlhjl−1+bil,yi=f∑j=1sL−1ϖijLhjL−1+biL.The error backpropagation algorithm then uses the output sensitivity of the neural network to quickly calculate the hyperparameters of the layers of the neural network, which include thresholds and weights, in order to achieve the goal of ensuring that the output values of the neural network are as close to the true values as possible [18]. Denote the training data set as x1,y1,x2,y2,…,xn,yn,yk∈−1,1, then for a certain training sample xk,yk, the output of the multilayer perceptron is y^i, and then the error function of the data set is defined as follows:(7)E=1n∑k=1nyk−y^k2.The weightW and bias B in the multilayer perceptron can be updated iteratively according to the following formula, where α is the learning rate and its value range is (0, 1).(8)Wl=Wl−α∂E∂Wl,Bl=Bl−α∂E∂Bl.By modifying the weight and bias of the network, the multilayer perceptron is trained to complete some complex tasks.


## 4. Experimental Results and Analysis

### 4.1. Experimental Environment

The algorithm in the paper is implemented in Python language. The operating system used for the experiments is Windows 10, 64 bit. The hardware environment is an Inter(R) Core (TM) i5-7200U CPU@ 2.50 GHz with 8G RAM.


### 4.2. Data Collection

The CICIDS2017 [19] dataset used in this paper was published by the Canadian Cyber Security Institute. The UNSW-NB15 [20] dataset was created by the Australian Centre for Cyber Security (ACCS) Laboratory. Both datasets reflect the characteristics of contemporary network intrusion detection traffic. The CICIDS2017 dataset is the largest intrusion detection dataset currently available on the Internet, and the dataset contains 11 of the most important features, namely, attack diversity, available protocols, complete captures, metadata, complete interactions, heterogeneity, complete network configurations, feature sets, complete traffic, anonymity, and tagging [6, 21]. The anomalous traffic behavior of the UNSW-NB15 dataset is divided into 9 categories containing a total of 42 features which are classified into 5 categories, i.e., basic features, additional generated features, content features, flow features, and temporal features [22]. The network traffic records and descriptions are shown in Table 1. In this paper, anomalous traffic and normal traffic of DoS attacks were collected from this dataset. 70% of the random subset of the CICIDS2017 dataset was selected for training, and the remaining 30% was used for testing. The UNSW-NB15 dataset has been distributed with a good training and testing set. Table 2 shows a detailed description of the dataset.Table 1
General characteristics of datasets.
DatasetUNSW-NB15CICIDS2017Year of publication20152017Total number of records1753412830743Number of features4283Number of types1015Table 2
Datasets description.
CICIDS2017LabelAttack typeCharacterizationNumber0BenignNatural transaction data, that is, normal behavior data23580361DoS GoldenEyeGoldenEye is a Layer 7 denial of service testing tool developed in Python102932DoS HulkHULK is a denial of service attack tool for the web. It is capable of generating many single spoofed flows on a web server that can bypass the engine’s cache and therefore attack the server’s resource pool directly [23]2310733DoS SlowHTTPTestSlowHTTPTest is a Slow HTTP DoS attack tool that relies on the actual HTTP protocol, which requires all requests from the server to be fully received before they can be processed54994DoS SlowLorisSlowLoris is an attack via HTTP denial of service, which affects threaded servers, thus causing the server thread pool to be depleted and the server to be unable to reply to other servers57965HeartbleedThe attacker inserts malicious information into the memory of the cryptographic library Open SSL via the Open SSL protocol, allowing the attacker to gain unauthorized access to useful information11UNSW-NB15LabelAttack typeCharacterizationTrainingTesting0NormalNatural transaction data56000370001DoSDenying users access to network resources or servers, typically by temporarily disconnecting or suspending service to an Internet host122644089


### 4.3. Evaluation Index

The confusion matrix is used as a matrix to evaluate the overall performance of the classifier. According to the prediction result of the Dos attack type, a confusion matrix can be obtained. Evaluating whether network traffic is subject to DoS attacks belongs to the category of two classification problems; evaluating which type of DoS attacks network traffic is subject to belongs to the category of multiple classification problems. In this regard, a series of evaluation indicators are needed. Among the many indicators, this study selects the most representative ones as the standard to measure the detection ability of the attack traffic distributed detection model, namely, precision and accuracy.When conducting multiclass classification anomaly detection research, we use recall rate as an evaluation indicator. Because the category with more data samples has a high accuracy rate, and the category with a few data samples has a low accuracy rate but still can get a higher overall accuracy rate, it cannot describe the performance of the classifier very well. Therefore, when performing multiclass classification, we choose to use recall rate and F1 score as evaluation indicators. The confusion matrix of the classification results is shown in Table3. The F1 value is a harmonic average of the model accuracy and recall.Table 3
Confusion matrix of the classification results.
Predict resultTrue resultDoSNormalDoSTPFNNormalFPTNAccuracy and FAR are, respectively, defined as(9)accuracy=TP+TNTP+TN+FP+FN,recall=TPTP+FN,precision=TPTP+FP,F1=2×precision×DRprecision+DR.


### 4.4. Data Preprocessing

#### 4.4.1. Label Digitization

Mark the benign traffic “Benign” of the CICIDS2017 data set as “0,” and mark the 5 types of DoS attacks as “1–5.” The normal traffic “Normal” of the UNWS-NB15 data set is marked as “0,” and the DoS attack type is marked as 1, as shown in the label column of Table2 in Section 4.2.


#### 4.4.2. Data Normalization

In order to reduce the problem of inconsistent impact weights between different dimensions of the data, this paper uses a min-max normalization method to normalize the traffic data. The aim is to perform a linear transformation on the original data so that the results fall into the interval [0, 1]. The conversion function for the min-max normalization method is as follows:(10)X+=X−XminXmax−Xmin.Among them,Xmin is the minimum value of all the sample data. Xmax is the maximum value of all the sample data. X is the original sample data before conversion. X+ is the data after the conversion [24].


### 4.5. Feature Selection Threshold Determination

Establish the priority of intrusion detection data features and filter out the more important features. Establishing feature engineering to make the data have a good form of expression can make the machine learning model achieve better results. The most effective features need to be selected for the needs of the attack data classification to achieve a better expression of the data.Calculate the importance metric of each feature according to Section3.1, and then perform feature selection on this basis. The specific steps are shown below.(1)
Train a random forest on the original data set and calculate the variable importance scores; sort all features in descending order according to the variable importance scores of each feature. To ensure the reliability of the ranking results, a 5-fold cross-validation method was used. The variable importance score obtained in the iteration with the largest classification accuracy rate in the iteration is used as the basis for feature ranking.(2)
The process of selecting feature subsets is done by refining the classification accuracy rate to select the best threshold to determine the optimal feature subset. The determination of the optimal threshold for feature selection is illustrated in Figures4 and 5. The optimal feature subset is shown in Table 4.Figure 4
CICIDS2017 dataset feature selection determination of thresholds.Figure 5
UNSW-NB15 dataset feature selection determination of thresholds.Table 4
Selected feature display.
DatasetsCICIDS2017UNSW-NB15Threshold0.0007050.00395Feature number total8342Feature selection number6139


### 4.6. Classification Performance Evaluation

This paper evaluates the performance of the RF-MLP model, by adjusting the configuration of the algorithm parameters and changing the structure of the multilayer perceptron, i.e., the number of neurons in the first hidden layer and the second hidden layer of this neural network. Table5 shows the accuracy of varying the model parameters. We observe that the multilayer perceptron structure is (10, 10, 10) and the excitation function is “rule,” and the model performs best on the intrusion detection dataset with accuracies of 0.9983 and 0.9352, respectively.Table 5
Accuracy of different MLP structure.
MLP structureAccuracyNumber of hidden layersNumber of units per hidden layerCICIDS2017210, 200.9978220, 200.9982220, 400.9978310, 10, 100.9983310, 20, 200.9966410, 10, 10, 100.9921UNSW-NB15 datasets210, 200.7352210, 100.9337220, 200.9063310, 10, 100.9352320, 10, 100.9107410, 10, 10, 100.8887According to the comparison of several groups of parameters, we can see that the accuracy rate is the highest when the parameter is (10,10,10), so the parameter (10,10,10) is selected as the optimal parameter of the model and marked in bold in Table5.Figures6 and 7 show the accuracy and loss plots for the training and testing sets of the CICIDS2017 and UNSW-NBA15 data, respectively. We set the “Y” axis to represent the loss or accuracy values and the “X” axis in the plots represents the epoch values [25]. The model has used 30 epochs; the testing set and training set achieved similar accuracy.Figure 6
CICIDS2017 datasets. (a) Accuracy on the train and test datasets. (b) Loss on the train and test datasets.
(a)(b)Figure 7
UNSW-NB15 datasets. (a) Accuracy on the train and test datasets. (b) Loss on the train and test datasets.
(a)(b)The performance of our model was evaluated by looking at three metrics: accuracy, recall rate, and F1 score, the results of which are shown in Table6. Accuracy rate and recall rate play a crucial role for each attack type. A low accuracy for a class indicates a high rate of false positives and therefore benign samples are unnecessarily flagged as intrusions, so benign samples will be flagged as intrusions. Also, low recall means that the model may miss genuine intrusions. Therefore, the values of both accuracy and recall must be high enough to ensure that the model works best. Through the result display, we can conclude that RF-MLP model is effective and feasible for DoS attack traffic classification.Table 6
The performance of the methods in the paper on different categories.
IndicatorsBenignDoS GoldenEyeDoS HulkDoS SlowHTTPTestDoS SlowLorisHeartbleedCICIDS2017Precision1.001.001.000.990.991.00Recall1.000.981.000.990.991.00F11.000.991.000.990.991.00UNSW-NB15NormalDoSPrecision0.990.61Recall0.930.95F10.960.74In order to better validate the algorithm proposed in this paper, the performance of the method in this paper is compared with other algorithms. This is shown in Table7. In order to observe the results more intuitively, the proposed method RF-MLP is compared with other algorithms and the results are presented in the form of a bar chart as shown in Figure 8. It is found that the proposed method has improved in terms of accuracy recall compared to other algorithms. Therefore, the effectiveness of this paper’s method is verified.Table 7
Performance comparison of the algorithm proposed in this paper with other algorithms.
Algorithm modelPrecisionRecallF1CICIDS2017AdaBoost0.770.840.77NB0.880.040.04RF0.980.970.97MLF0.770.830.76RF-MLP0.990.990.99UNSW-NB15KNN0.78——SVM0.72——RF—0.10—RF-MLP0.800.940.85Figure 8
Performance comparison of the algorithm proposed in this paper with other algorithms.


## 4.1. Experimental Environment

The algorithm in the paper is implemented in Python language. The operating system used for the experiments is Windows 10, 64 bit. The hardware environment is an Inter(R) Core (TM) i5-7200U CPU@ 2.50 GHz with 8G RAM.


## 4.2. Data Collection

The CICIDS2017 [19] dataset used in this paper was published by the Canadian Cyber Security Institute. The UNSW-NB15 [20] dataset was created by the Australian Centre for Cyber Security (ACCS) Laboratory. Both datasets reflect the characteristics of contemporary network intrusion detection traffic. The CICIDS2017 dataset is the largest intrusion detection dataset currently available on the Internet, and the dataset contains 11 of the most important features, namely, attack diversity, available protocols, complete captures, metadata, complete interactions, heterogeneity, complete network configurations, feature sets, complete traffic, anonymity, and tagging [6, 21]. The anomalous traffic behavior of the UNSW-NB15 dataset is divided into 9 categories containing a total of 42 features which are classified into 5 categories, i.e., basic features, additional generated features, content features, flow features, and temporal features [22]. The network traffic records and descriptions are shown in Table 1. In this paper, anomalous traffic and normal traffic of DoS attacks were collected from this dataset. 70% of the random subset of the CICIDS2017 dataset was selected for training, and the remaining 30% was used for testing. The UNSW-NB15 dataset has been distributed with a good training and testing set. Table 2 shows a detailed description of the dataset.Table 1
General characteristics of datasets.
DatasetUNSW-NB15CICIDS2017Year of publication20152017Total number of records1753412830743Number of features4283Number of types1015Table 2
Datasets description.
CICIDS2017LabelAttack typeCharacterizationNumber0BenignNatural transaction data, that is, normal behavior data23580361DoS GoldenEyeGoldenEye is a Layer 7 denial of service testing tool developed in Python102932DoS HulkHULK is a denial of service attack tool for the web. It is capable of generating many single spoofed flows on a web server that can bypass the engine’s cache and therefore attack the server’s resource pool directly [23]2310733DoS SlowHTTPTestSlowHTTPTest is a Slow HTTP DoS attack tool that relies on the actual HTTP protocol, which requires all requests from the server to be fully received before they can be processed54994DoS SlowLorisSlowLoris is an attack via HTTP denial of service, which affects threaded servers, thus causing the server thread pool to be depleted and the server to be unable to reply to other servers57965HeartbleedThe attacker inserts malicious information into the memory of the cryptographic library Open SSL via the Open SSL protocol, allowing the attacker to gain unauthorized access to useful information11UNSW-NB15LabelAttack typeCharacterizationTrainingTesting0NormalNatural transaction data56000370001DoSDenying users access to network resources or servers, typically by temporarily disconnecting or suspending service to an Internet host122644089


## 4.3. Evaluation Index

The confusion matrix is used as a matrix to evaluate the overall performance of the classifier. According to the prediction result of the Dos attack type, a confusion matrix can be obtained. Evaluating whether network traffic is subject to DoS attacks belongs to the category of two classification problems; evaluating which type of DoS attacks network traffic is subject to belongs to the category of multiple classification problems. In this regard, a series of evaluation indicators are needed. Among the many indicators, this study selects the most representative ones as the standard to measure the detection ability of the attack traffic distributed detection model, namely, precision and accuracy.When conducting multiclass classification anomaly detection research, we use recall rate as an evaluation indicator. Because the category with more data samples has a high accuracy rate, and the category with a few data samples has a low accuracy rate but still can get a higher overall accuracy rate, it cannot describe the performance of the classifier very well. Therefore, when performing multiclass classification, we choose to use recall rate and F1 score as evaluation indicators. The confusion matrix of the classification results is shown in Table3. The F1 value is a harmonic average of the model accuracy and recall.Table 3
Confusion matrix of the classification results.
Predict resultTrue resultDoSNormalDoSTPFNNormalFPTNAccuracy and FAR are, respectively, defined as(9)accuracy=TP+TNTP+TN+FP+FN,recall=TPTP+FN,precision=TPTP+FP,F1=2×precision×DRprecision+DR.


## 4.4. Data Preprocessing

### 4.4.1. Label Digitization

Mark the benign traffic “Benign” of the CICIDS2017 data set as “0,” and mark the 5 types of DoS attacks as “1–5.” The normal traffic “Normal” of the UNWS-NB15 data set is marked as “0,” and the DoS attack type is marked as 1, as shown in the label column of Table2 in Section 4.2.


### 4.4.2. Data Normalization

In order to reduce the problem of inconsistent impact weights between different dimensions of the data, this paper uses a min-max normalization method to normalize the traffic data. The aim is to perform a linear transformation on the original data so that the results fall into the interval [0, 1]. The conversion function for the min-max normalization method is as follows:(10)X+=X−XminXmax−Xmin.Among them,Xmin is the minimum value of all the sample data. Xmax is the maximum value of all the sample data. X is the original sample data before conversion. X+ is the data after the conversion [24].


## 4.4.1. Label Digitization

Mark the benign traffic “Benign” of the CICIDS2017 data set as “0,” and mark the 5 types of DoS attacks as “1–5.” The normal traffic “Normal” of the UNWS-NB15 data set is marked as “0,” and the DoS attack type is marked as 1, as shown in the label column of Table2 in Section 4.2.


## 4.4.2. Data Normalization

In order to reduce the problem of inconsistent impact weights between different dimensions of the data, this paper uses a min-max normalization method to normalize the traffic data. The aim is to perform a linear transformation on the original data so that the results fall into the interval [0, 1]. The conversion function for the min-max normalization method is as follows:(10)X+=X−XminXmax−Xmin.Among them,Xmin is the minimum value of all the sample data. Xmax is the maximum value of all the sample data. X is the original sample data before conversion. X+ is the data after the conversion [24].


## 4.5. Feature Selection Threshold Determination

Establish the priority of intrusion detection data features and filter out the more important features. Establishing feature engineering to make the data have a good form of expression can make the machine learning model achieve better results. The most effective features need to be selected for the needs of the attack data classification to achieve a better expression of the data.Calculate the importance metric of each feature according to Section3.1, and then perform feature selection on this basis. The specific steps are shown below.(1)
Train a random forest on the original data set and calculate the variable importance scores; sort all features in descending order according to the variable importance scores of each feature. To ensure the reliability of the ranking results, a 5-fold cross-validation method was used. The variable importance score obtained in the iteration with the largest classification accuracy rate in the iteration is used as the basis for feature ranking.(2)
The process of selecting feature subsets is done by refining the classification accuracy rate to select the best threshold to determine the optimal feature subset. The determination of the optimal threshold for feature selection is illustrated in Figures4 and 5. The optimal feature subset is shown in Table 4.Figure 4
CICIDS2017 dataset feature selection determination of thresholds.Figure 5
UNSW-NB15 dataset feature selection determination of thresholds.Table 4
Selected feature display.
DatasetsCICIDS2017UNSW-NB15Threshold0.0007050.00395Feature number total8342Feature selection number6139


## 4.6. Classification Performance Evaluation

This paper evaluates the performance of the RF-MLP model, by adjusting the configuration of the algorithm parameters and changing the structure of the multilayer perceptron, i.e., the number of neurons in the first hidden layer and the second hidden layer of this neural network. Table5 shows the accuracy of varying the model parameters. We observe that the multilayer perceptron structure is (10, 10, 10) and the excitation function is “rule,” and the model performs best on the intrusion detection dataset with accuracies of 0.9983 and 0.9352, respectively.Table 5
Accuracy of different MLP structure.
MLP structureAccuracyNumber of hidden layersNumber of units per hidden layerCICIDS2017210, 200.9978220, 200.9982220, 400.9978310, 10, 100.9983310, 20, 200.9966410, 10, 10, 100.9921UNSW-NB15 datasets210, 200.7352210, 100.9337220, 200.9063310, 10, 100.9352320, 10, 100.9107410, 10, 10, 100.8887According to the comparison of several groups of parameters, we can see that the accuracy rate is the highest when the parameter is (10,10,10), so the parameter (10,10,10) is selected as the optimal parameter of the model and marked in bold in Table5.Figures6 and 7 show the accuracy and loss plots for the training and testing sets of the CICIDS2017 and UNSW-NBA15 data, respectively. We set the “Y” axis to represent the loss or accuracy values and the “X” axis in the plots represents the epoch values [25]. The model has used 30 epochs; the testing set and training set achieved similar accuracy.Figure 6
CICIDS2017 datasets. (a) Accuracy on the train and test datasets. (b) Loss on the train and test datasets.
(a)(b)Figure 7
UNSW-NB15 datasets. (a) Accuracy on the train and test datasets. (b) Loss on the train and test datasets.
(a)(b)The performance of our model was evaluated by looking at three metrics: accuracy, recall rate, and F1 score, the results of which are shown in Table6. Accuracy rate and recall rate play a crucial role for each attack type. A low accuracy for a class indicates a high rate of false positives and therefore benign samples are unnecessarily flagged as intrusions, so benign samples will be flagged as intrusions. Also, low recall means that the model may miss genuine intrusions. Therefore, the values of both accuracy and recall must be high enough to ensure that the model works best. Through the result display, we can conclude that RF-MLP model is effective and feasible for DoS attack traffic classification.Table 6
The performance of the methods in the paper on different categories.
IndicatorsBenignDoS GoldenEyeDoS HulkDoS SlowHTTPTestDoS SlowLorisHeartbleedCICIDS2017Precision1.001.001.000.990.991.00Recall1.000.981.000.990.991.00F11.000.991.000.990.991.00UNSW-NB15NormalDoSPrecision0.990.61Recall0.930.95F10.960.74In order to better validate the algorithm proposed in this paper, the performance of the method in this paper is compared with other algorithms. This is shown in Table7. In order to observe the results more intuitively, the proposed method RF-MLP is compared with other algorithms and the results are presented in the form of a bar chart as shown in Figure 8. It is found that the proposed method has improved in terms of accuracy recall compared to other algorithms. Therefore, the effectiveness of this paper’s method is verified.Table 7
Performance comparison of the algorithm proposed in this paper with other algorithms.
Algorithm modelPrecisionRecallF1CICIDS2017AdaBoost0.770.840.77NB0.880.040.04RF0.980.970.97MLF0.770.830.76RF-MLP0.990.990.99UNSW-NB15KNN0.78——SVM0.72——RF—0.10—RF-MLP0.800.940.85Figure 8
Performance comparison of the algorithm proposed in this paper with other algorithms.


## 5. Conclusions and Future Work

A DoS attack disrupts network performance because it deprives legitimate users of a specific amount of time to reach network resources. A problem with current intrusion detection is that the false positive and false negative rates are high, and most existing intrusion detections rely on rule-based expert systems, which cannot detect new attacks. The RF-MLP model method proposed in this paper analyzes and evaluates network traffic and establishes a security prediction model that can accurately identify DoS attacks in network traffic.From the obtained precision, recall, and F1 scores, it can be concluded that the model proposed in this paper is effective and feasible for the detection of DoS traffic. The CICIDS2017 dataset and the UNSW-NB15 dataset used in this paper are also the largest and most authoritative intrusion detection datasets available, which also demonstrates the robustness of our model. Although the classifier achieves high accuracy, there is still much room for improvement and further research. However, it is important to evaluate and benchmark model with captured real network traffic. Intrusion detection systems monitor and analyze network traffic data or various audit data generated by host systems to detect illegal intrusions or attacks in networks and systems and then provide early warning and take appropriate measures to protect against them. The core of an intrusion detection system lies in the framework of the system and the detection algorithms used.As future research, we plan to extend this work by deploying the experimental results to corresponding software systems to observe the performance of the software in real network environments. A combination of misuse and anomaly detection is used to ensure system performance. Misuse-based intrusion detection is easier and more accurate to detect but often misses, while anomaly-based intrusion detection can identify unknown novel attacks but usually tends to judge normal traffic behavior as anomalous attacks, and the false alarm rate increases as the system detection rate increases. The two approaches are combined to provide a more comprehensive model to detect and prevent DoS attacks.



---

*Source: 2076987-2022-04-29.xml*