# Towards Effective Network Intrusion Detection: A Hybrid Model Integrating Gini Index and GBDT with PSO

Authors: Longjie Li; Yang Yu; Shenshen Bai; Jianjun Cheng; Xiaoyun Chen
Journal: Journal of Sensors
 (2018)

---

## Abstract

In order to protect computing systems from malicious attacks, network intrusion detection systems have become an important part in the security infrastructure. Recently, hybrid models that integrating several machine learning techniques have captured more attention of researchers. In this paper, a novel hybrid model was proposed with the purpose of detecting network intrusion effectively. In the proposed model, Gini index is used to select the optimal subset of features, the gradient boosted decision tree (GBDT) algorithm is adopted to detect network attacks, and the particle swarm optimization (PSO) algorithm is utilized to optimize the parameters of GBDT. The performance of the proposed model is experimentally evaluated in terms of accuracy, detection rate, precision, F1-score, and false alarm rate using the NSL-KDD dataset. Experimental results show that the proposed model is superior to the compared methods.


---

## Body

## 1. Introduction

With the prompt development of the Internet, a variety of network security problems has continued to occur. At the same time, many security mechanisms have been implemented to maintain the security of computer systems. Among these mechanisms, intrusion detection systems (IDSs) play a vital role in protecting computing infrastructures from attackers and intruders.The primary task of an IDS is to discover and prohibit abnormal connections from network traffics. In general, intrusion detection approaches can be divided into two categories:misuse-based detection and anomaly-based detection [1–3]. A misuse-based detection method stores in advance signatures of already known intrusive behaviors in a database and determines whether a network connection is an attack by matching its characteristic with the signatures. If matching one signature, this connection is an attack. These methods are effective in identifying the well-known network attacks with low false alarm rate. However, they will fail when meeting unknown intrusions whose properties do not match any signature in the database [4]. To address this problem, a misuse-based method needs to regularly update its database. Whereas, obtaining the signatures of new intrusive behaviors is usually expensive. Conversely, an anomaly-based detection method trains models for normal behavior and detects network intrusions by identifying traffics that significantly deviate from normal profile [5]. The basic hypothesis in anomaly-based detection methods is the characteristic of an abnormal connection is far from those of normal connections. In an anomaly-based detection system, we no longer need to store signatures of attack patterns. The anomaly-based detection methods can recognize not only unknown intrusive behaviors but also unknown future attacks. That is an advantage over misuse-based methods. Nevertheless, they may misclassify some normal traffics that are located in the boundaries between normal and abnormal behavior. Because new attacks keep emerging, anomaly-based methods have captured more attention from researchers.To achieve reliable detection results, great efforts have been made by researchers. In the early stages, rule-based expert systems [6] and statistical methods [7] were applied in intrusion detection systems. But the worse performance of these approaches when dealing with large-scale network traffics relegates their application to small datasets. In nature, anomaly-based intrusion detection is a classification problem. Hence, researchers have sought to develop solutions for IDS by utilizing various machine learning techniques [8], such as decision trees, support vector machines (SVMs), naive Bayes, and artificial neural networks (ANNs). When developing an IDS, the primary goal is to achieve the best possible accuracy. To this end, hybrid methods have been largely proposed to further enhance the accuracy of intrusion detection when compared to using individual machine learning approaches [4, 9, 10]. The basic idea behind a hybrid model is to significantly improve the detection performance by means of combining several machine learning techniques. In general, the styles of existing hybrid models mainly include ensemble classifier [9, 11, 12], clustering plus classification [13–15], and feature selection plus classification [3, 4, 16–21].As mentioned above, hybrid models show a new way for network intrusion detection. In this paper, a novel hybrid model, namely,GINI-GBDT-PSO, is proposed to put forward a solution for IDS with high accuracy. The proposed method integrates the Gini index, the GBDT (gradient boosted decision trees) algorithm (GBDT has some other names, e.g., GBRT (gradient boosted regression tree), MART (multiple additive regression tree), and tree net.) [22], and the PSO (particle swarm optimization) algorithm [23] together. Feature selection, which extracts an optimal subset of features to represent the whole dataset, is a key step in intrusion detection because training a classifier based on the optimal subset can not only reduce the learning time but also improve its accuracy [19]. In the proposed method, we adopt the Gini index, which has been used for dimensionality reduction in the study of text mining [24–26], to extract significant features from original datasets. After feature selection, the proposed method uses the GBDT algorithm to train a prediction model on the optimal feature space. The GBDT algorithm is a powerful supervised learning method, which integrates the gradient boosting framework and decision tree technique into one ensemble model [22]. Due to its successful applications in many research fields, such as disease modeling [27, 28], web-search ranking [29, 30], and travel time prediction [31], we argue that the GBDT algorithm will be quite competent to the work of intrusion detection. The parameters of GBDT, for instance, learning rate ν, are optimized by the PSO algorithm. Finally, we experimentally study the performance of the GINI-GBDT-PSO method on the NSL-KDD dataset [32], compared with several baselines including single classification methods and hybrid models. Experimental results show that the proposed hybrid method improves the detection performance in comparison with baselines.The rest of the paper is organized as follows. Section2 gives the criteria used to evaluate the performance of intrusion detection models. The detail of the proposed hybrid method is described in Section 3. In Section 4, the performance of the proposed method is experimentally evaluated compared with several baselines. Section 5 briefly reviews some hybrid intrusion detection models. Finally, Section 6 concludes this work.


## 2. Evaluation Criteria

Table1 shows a confusion matrix that is used to represent the information related to the actual and predicted classifications performed by a detection model. In Table 1, TP is the number of abnormal connections that are correctly detected; TN is the number of normal connections that are correctly identified; FP is the number of normal connections that are incorrectly classified as abnormal; FN is the number of abnormal connections that are incorrectly judged as normal.Table 1
Confusion matrix.
Predicted
Abnormal
Normal
Actual
Abnormal
TP
FN
Normal
FP
TNIn this paper, the performance of intrusion detection models is evaluated by five widely used measures: accuracy, precision, detection rate (DR), F1-score, and false alarm rate (FAR) [14, 33, 34]. The calculations of these evaluation measures are defined as follows:
(1)accuracy=TP+TNTP+TN+FP+FN,(2)DR=TPTP+FN,(3)precision=TPTP+FP,(4)F1‐score=2×precision×DRprecision+DR,(5)FAR=FPFP+TN.The accuracy and detection rates evaluate the capability of an intrusion detection model to correctly predict connections and detect abnormal events, respectively. The precision reflects the proportion of real abnormal events in all returned abnormal connections. The F1-score is the harmonic mean of precision and DR, which balances the precision and DR of a detection model. The false alarm rate measures the ratio of normal connections incorrectly classified as abnormal to total normal connections. Therefore, higher values of accuracy, detection rate, precision, and F1-score and lower values of false alarm rate show better detection performance for network intrusion detection models.


## 3. The Proposed Method

As mentioned in the Introduction, this work aims to improve the performance of network intrusion detection by constructing a hybrid model. To this end, we propose theGINI-GBDT-PSO method that combines the Gini index, the GBDT algorithm, and the PSO method into one framework. In the proposed model, we use Gini index to select important features from original datasets and then adopt GBDT as a classification approach to detect abnormal events on the feature filtered datasets. Because manually configuring the parameters of GBDT is not an easy work, we use PSO as an optimizer to find the high-quality parameters for GBDT.For the sake of clarity, the stages of the system framework are listed as follows:(1)
Dataset preprocessing
(2)
Featuring a selection using the Gini index
(3)
Training a prediction model with GBDT optimized by PSO
(4)
Identifying network intrusions by the prediction model


### 3.1. NSL-KDD Dataset

In this study, the NSL-KDD dataset [32] is used as benchmark. NSL-KDD is an improved version of the popular KDDCup99 dataset (http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html), which solves some inherent problems existing in KDDCup99 dataset. Each instance in NSL-KDD dataset is a TCP/IP connection record depicted by 41 different features and classified as one of the following classes: normal event, denial of service (DoS) attack, probe attack, user to root (U2R) attack, and remote to local (R2L) attack. The detail description about the NSL-KDD dataset can be found in [35–37]. Table 2 lists the number of instances in the training and testing sets. As seen in Table 2, the number of instances in the training and testing sets of NSL-KDD is in the reasonable range, which makes it affordable to conduct experiments on whole dataset. However, researchers have usually run experiments on randomly selected small portion of the KDDCup99 dataset, which may cause inconsistent evaluation results. In NSL-KDD dataset, it is free of redundant records in training set and duplicate records in testing set, so the classifiers will not be biased towards more frequent records.Table 2
Number of instances in NSL-KDD.
Normal
DoS
Probe
U2R
R2L
Total
Training set
67,343
45,927
11,656
52
995
125,973
Testing set
9711
7458
2421
200
2754
22,544There are three symbolic, two binary, and 36 continuous features among the 41 features in NSL-KDD dataset. To use the proposed method, symbolic features should be converted into numeric features. In this paper, the simple scheme used in [12] is adopted to handle symbolic features. The scheme maps symbolic values to integer values with range from 1 to M, where M is the number of distinct symbols for a feature. For class labels, normal is mapped to 1, DoS to 2, probe to 3, U2R to 4, and R2L to 5. Furthermore, we normalize the values of each feature into [0, 1].


### 3.2. Gini Index

Usually, the dataset for network intrusion detection contains many features. However, not every feature contributes to the task of detecting intrusion. Feature selection, which can remove redundant or irrelevant features, is a crucial step for intrusion detection. Based on the optimal feature space, we can not only enhance the speed of training a classifier for network intrusion detection but also improve its detection performance.The goal of feature selection is to get a group of significant features from the whole dataset, such that these selected features are very important for training a classification model.To do that, we employ in this paper theGini index to undertake the mission of feature selection. The Gini index, which was developed by Corrado Gini, an Italian statistician and sociologist, in 1912, was originally used to measure the statistical dispersion of income distribution across different population sectors. Nowadays, in the research of data mining, Gini index has been widely used in text mining [24–26].Given a datasetD, which includes instances from l classes C1,…,Cl, let f be a feature that has k distinct values in D. According to the values of f, we divide D into k disjoint subsets D1,…,Dk. The score of feature f measured by Gini index is define as follows:
(6)Gf=∑i=1kDiD1−∑j=1lpi,j2,where pi,j is the probability that an instance in Di belongs to class Cj.The range ofG(f) is [0, 1]. The smaller G(f) is, the more important feature f is. For any Di, if all instances in it belong to the same class, then Plj=1p2i,j = 1, and G(f) = 0. At this time, feature f has the strongest discrimination.


### 3.3. GBDT Algorithm

Thegradient boosting is a machine learning technique for classification and regression problems, which produces a strong ensemble model by combining a series of weak prediction models in an iterative fashion. The GBDT (gradient boosted decision trees) algorithm is a powerful ensemble learning algorithm, which extends and enhances the classification and regression tree model according to gradient boosting [22]. The GBDT algorithm iteratively constructs decision trees. In each iteration, a decision tree is trained from the residuals of the previous tree. Then, the accumulation of predicted results of all trees is the final result.Given the training samplesxi,yii=1n, where xi is a sample and yi denotes the label of sample xi. Let Fx be a prediction model and Ly,Fx be a loss function. For any sample xi,Fxi is the prediction of xi, and Lyi,Fxi is the lossness between Fxi and yi. The goal of GBDT is to learn an optimal model F∗x such that ∑i=1nLyi,Fxi is minimized for a specified loss function Ly,Fx.To do that, the GBDT algorithm first builds an initial decision treeF0x, then iteratively constructs m new trees. In each iteration, a new tree hx is added to reduce the residuals, which are obtained by the given loss function Ly,Fx. Therefore, the optimal model F∗x of GBDT can be defined as follows:
(7)F∗x=F0x+v∗∑t=1mρt∗htx,where m is the number of iterations, and v0<v<1 is the shrinkage parameter that controls the learning rate of GBDT. htx is the tree trained in the tth iteration and ρt is the weight of htx.The procedure of GBDT algorithm is shown in Algorithm1. In line 1, tree F0x is initialized. In this paper, we simply set F0x=0. Line 3 calculates the pseudoresiduals, which represent the difference between real value and predicted value. In GBDT, the loss function is applied to obtain residuals at each iteration. Next, a new tree is trained according to the pseudoresiduals in line 4 and the weight of this tree is determined in line 5. Then, the prediction model is updated in line 6.In GBDT algorithm, the least absolute deviation function, the least square function, and the Huber function are commonly used as the loss function. In this paper, we adopt the least square function that is defined as follows:(8)Ly,Fx=12y−Fx2.


### 3.4. PSO Algorithm

Inspired by the population behavior of bird flocking and fish schooling, Kennedy and Eberhart proposed thePSO (particle swarm optimization) algorithm in 1995 [38]. This algorithm is a derivative-free, zero-order method and thus can be applied to solve a variety of optimization problems by simulating social behavior [12]. Compared to other optimization algorithms, the PSO technique is easy to implement, robust, and scalable and can quickly find approximately optimal solutions [34, 39]. In the research of IDSs, PSO is a popular optimization technique [12, 34, 40, 41].Algorithm 1: GBDT algorithm.
Require:
Training set:xi,yii=1n; number of iterations: m
Loss function:Ly,Fx; learning rate: ν
Ensure: The predict model: Fx
1.F0x=0;
2.fort=1→mdo
3.yi′=−∂Lyi,Fxi∂FxiFx=Ft−1x,i=1,2,…,n;
4.   Train a new treehtx according to xi,yi′i=1n
5.ρt=argminρ∑i=1nLyi,Ft−1x+ρ∗htx;
6.Ftx=Ft−1x+v∗ρt∗htx;
7.end for
8.returnFmx;At the outset, a group of particles, which represent a population of candidate solutions, is generated with random positions and velocities in the problem space. Then, each particle flies through the problem space by following the particle that is on the best-known position. To define how best a position is, a fitness value is assigned to a particle according to its position. Thus, the optimization problem becomes to find the best position that has the best fitness value. In this work, we define the detection accuracy (see (1)) as fitness function. To obtain the optimization goal, the PSO algorithm iteratively updates the position and velocity of each particle in the problem space guided by the best position identified so far by itself as well as the best coordinate tracked by entire particle swarm.Formally, in then-dimensional problem space, the position and velocity of the ith particle at the tth iteration is defined as Xit=xi1t,xi2t,…,xint and Vit=vi1t,vi2t,…,vint, respectively. Then, the particle changes its position and velocity according to its position and velocity at current stage, the best-known position identified by itself so far (denoted as Pbesti), and the global best-known position identified by entire particle swarm so far (denoted as Gbest). Equations (9) and (10) show the calculation of the velocity and position, respectively.
(9)Vit=ωVit−1+c1r1Pbesti−Xit−1+c2r2Gbest−Xit−1,(10)Xit=Xit−1+Vit,where ω represents the inertia weight. c1 is the particle acceleration factor, and c2 is the population acceleration factor. r1 and r2 are two independently positive random numbers between 0 and 1.Algorithm 2: PSO algorithm.
1.for each i∈1,Ndo
2.  Randomly initialize the positionXi and velocity Vi of current particle.
3.  Mark the position of current particle as its best-known position,Pbesti.
4.end for
5. Calculate the global best known positionGbest.
6.for each t∈1,Tdo
7.for each i∈1,Ndo
8.   Update the position and velocity of theith particle based on (9) and (10).
9.if fitnessXit > fitnessPbestithen
10.Pbesti=Xit;
11.end if
12.if fitnessPbesti > fitnessGbestthen
13.Gbest=Pbesti;
14.end if
15.end for
16.end forThe detailed process of the PSO algorithm is described in Algorithm2. In Algorithm 2, N is the total number of particles in the population, and T is the total number of iterations. fitness(·) is the function that gives the fitness value of a position.


## 3.1. NSL-KDD Dataset

In this study, the NSL-KDD dataset [32] is used as benchmark. NSL-KDD is an improved version of the popular KDDCup99 dataset (http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html), which solves some inherent problems existing in KDDCup99 dataset. Each instance in NSL-KDD dataset is a TCP/IP connection record depicted by 41 different features and classified as one of the following classes: normal event, denial of service (DoS) attack, probe attack, user to root (U2R) attack, and remote to local (R2L) attack. The detail description about the NSL-KDD dataset can be found in [35–37]. Table 2 lists the number of instances in the training and testing sets. As seen in Table 2, the number of instances in the training and testing sets of NSL-KDD is in the reasonable range, which makes it affordable to conduct experiments on whole dataset. However, researchers have usually run experiments on randomly selected small portion of the KDDCup99 dataset, which may cause inconsistent evaluation results. In NSL-KDD dataset, it is free of redundant records in training set and duplicate records in testing set, so the classifiers will not be biased towards more frequent records.Table 2
Number of instances in NSL-KDD.
Normal
DoS
Probe
U2R
R2L
Total
Training set
67,343
45,927
11,656
52
995
125,973
Testing set
9711
7458
2421
200
2754
22,544There are three symbolic, two binary, and 36 continuous features among the 41 features in NSL-KDD dataset. To use the proposed method, symbolic features should be converted into numeric features. In this paper, the simple scheme used in [12] is adopted to handle symbolic features. The scheme maps symbolic values to integer values with range from 1 to M, where M is the number of distinct symbols for a feature. For class labels, normal is mapped to 1, DoS to 2, probe to 3, U2R to 4, and R2L to 5. Furthermore, we normalize the values of each feature into [0, 1].


## 3.2. Gini Index

Usually, the dataset for network intrusion detection contains many features. However, not every feature contributes to the task of detecting intrusion. Feature selection, which can remove redundant or irrelevant features, is a crucial step for intrusion detection. Based on the optimal feature space, we can not only enhance the speed of training a classifier for network intrusion detection but also improve its detection performance.The goal of feature selection is to get a group of significant features from the whole dataset, such that these selected features are very important for training a classification model.To do that, we employ in this paper theGini index to undertake the mission of feature selection. The Gini index, which was developed by Corrado Gini, an Italian statistician and sociologist, in 1912, was originally used to measure the statistical dispersion of income distribution across different population sectors. Nowadays, in the research of data mining, Gini index has been widely used in text mining [24–26].Given a datasetD, which includes instances from l classes C1,…,Cl, let f be a feature that has k distinct values in D. According to the values of f, we divide D into k disjoint subsets D1,…,Dk. The score of feature f measured by Gini index is define as follows:
(6)Gf=∑i=1kDiD1−∑j=1lpi,j2,where pi,j is the probability that an instance in Di belongs to class Cj.The range ofG(f) is [0, 1]. The smaller G(f) is, the more important feature f is. For any Di, if all instances in it belong to the same class, then Plj=1p2i,j = 1, and G(f) = 0. At this time, feature f has the strongest discrimination.


## 3.3. GBDT Algorithm

Thegradient boosting is a machine learning technique for classification and regression problems, which produces a strong ensemble model by combining a series of weak prediction models in an iterative fashion. The GBDT (gradient boosted decision trees) algorithm is a powerful ensemble learning algorithm, which extends and enhances the classification and regression tree model according to gradient boosting [22]. The GBDT algorithm iteratively constructs decision trees. In each iteration, a decision tree is trained from the residuals of the previous tree. Then, the accumulation of predicted results of all trees is the final result.Given the training samplesxi,yii=1n, where xi is a sample and yi denotes the label of sample xi. Let Fx be a prediction model and Ly,Fx be a loss function. For any sample xi,Fxi is the prediction of xi, and Lyi,Fxi is the lossness between Fxi and yi. The goal of GBDT is to learn an optimal model F∗x such that ∑i=1nLyi,Fxi is minimized for a specified loss function Ly,Fx.To do that, the GBDT algorithm first builds an initial decision treeF0x, then iteratively constructs m new trees. In each iteration, a new tree hx is added to reduce the residuals, which are obtained by the given loss function Ly,Fx. Therefore, the optimal model F∗x of GBDT can be defined as follows:
(7)F∗x=F0x+v∗∑t=1mρt∗htx,where m is the number of iterations, and v0<v<1 is the shrinkage parameter that controls the learning rate of GBDT. htx is the tree trained in the tth iteration and ρt is the weight of htx.The procedure of GBDT algorithm is shown in Algorithm1. In line 1, tree F0x is initialized. In this paper, we simply set F0x=0. Line 3 calculates the pseudoresiduals, which represent the difference between real value and predicted value. In GBDT, the loss function is applied to obtain residuals at each iteration. Next, a new tree is trained according to the pseudoresiduals in line 4 and the weight of this tree is determined in line 5. Then, the prediction model is updated in line 6.In GBDT algorithm, the least absolute deviation function, the least square function, and the Huber function are commonly used as the loss function. In this paper, we adopt the least square function that is defined as follows:(8)Ly,Fx=12y−Fx2.


## 3.4. PSO Algorithm

Inspired by the population behavior of bird flocking and fish schooling, Kennedy and Eberhart proposed thePSO (particle swarm optimization) algorithm in 1995 [38]. This algorithm is a derivative-free, zero-order method and thus can be applied to solve a variety of optimization problems by simulating social behavior [12]. Compared to other optimization algorithms, the PSO technique is easy to implement, robust, and scalable and can quickly find approximately optimal solutions [34, 39]. In the research of IDSs, PSO is a popular optimization technique [12, 34, 40, 41].Algorithm 1: GBDT algorithm.
Require:
Training set:xi,yii=1n; number of iterations: m
Loss function:Ly,Fx; learning rate: ν
Ensure: The predict model: Fx
1.F0x=0;
2.fort=1→mdo
3.yi′=−∂Lyi,Fxi∂FxiFx=Ft−1x,i=1,2,…,n;
4.   Train a new treehtx according to xi,yi′i=1n
5.ρt=argminρ∑i=1nLyi,Ft−1x+ρ∗htx;
6.Ftx=Ft−1x+v∗ρt∗htx;
7.end for
8.returnFmx;At the outset, a group of particles, which represent a population of candidate solutions, is generated with random positions and velocities in the problem space. Then, each particle flies through the problem space by following the particle that is on the best-known position. To define how best a position is, a fitness value is assigned to a particle according to its position. Thus, the optimization problem becomes to find the best position that has the best fitness value. In this work, we define the detection accuracy (see (1)) as fitness function. To obtain the optimization goal, the PSO algorithm iteratively updates the position and velocity of each particle in the problem space guided by the best position identified so far by itself as well as the best coordinate tracked by entire particle swarm.Formally, in then-dimensional problem space, the position and velocity of the ith particle at the tth iteration is defined as Xit=xi1t,xi2t,…,xint and Vit=vi1t,vi2t,…,vint, respectively. Then, the particle changes its position and velocity according to its position and velocity at current stage, the best-known position identified by itself so far (denoted as Pbesti), and the global best-known position identified by entire particle swarm so far (denoted as Gbest). Equations (9) and (10) show the calculation of the velocity and position, respectively.
(9)Vit=ωVit−1+c1r1Pbesti−Xit−1+c2r2Gbest−Xit−1,(10)Xit=Xit−1+Vit,where ω represents the inertia weight. c1 is the particle acceleration factor, and c2 is the population acceleration factor. r1 and r2 are two independently positive random numbers between 0 and 1.Algorithm 2: PSO algorithm.
1.for each i∈1,Ndo
2.  Randomly initialize the positionXi and velocity Vi of current particle.
3.  Mark the position of current particle as its best-known position,Pbesti.
4.end for
5. Calculate the global best known positionGbest.
6.for each t∈1,Tdo
7.for each i∈1,Ndo
8.   Update the position and velocity of theith particle based on (9) and (10).
9.if fitnessXit > fitnessPbestithen
10.Pbesti=Xit;
11.end if
12.if fitnessPbesti > fitnessGbestthen
13.Gbest=Pbesti;
14.end if
15.end for
16.end forThe detailed process of the PSO algorithm is described in Algorithm2. In Algorithm 2, N is the total number of particles in the population, and T is the total number of iterations. fitness(·) is the function that gives the fitness value of a position.


## 4. Experiments

This section experimentally studies the detection performance of the proposed hybrid model (i.e., GINI-GBDT-PSO), compared with six baselines. The six baselines contain three individual classification algorithms, that is, SVM [42], random forests (RF) [43], and C4.5 [44], and three hybrid models, that is, FC-ANN [14], CFA-DT [19], and IGCR-ANN [3]. The NSL-KDD dataset [32] is the benchmark. All experiments are implemented in the MATLAB 7.0 environment.


### 4.1. Setting of GINI-GBDT-PSO

For the proposed method, before training an intrusion detection model, the first task is to extract the optimal subset of features. The Gini index was originally used to determine the degree of inequality of income distribution across different population sectors. The international community has usually used 0.4 as the warning line for the gap between rich and poor [45]. In our experiments, we consider a feature whose score of Gini index is less than or equal to 0.4 as important feature. As a result, 18 important features are selected from the NSL-KDD dataset, which are listed in Table 3.Table 3
Important features selected from NSL-KDD.
Id
Data type
Feature
f1
Continuous
Duration
f2
Symbolic
protocol_type
f3
Symbolic
Service
f4
Symbolic
Flag
f5
Continuous
src_bytes
f6
Continuous
dst_bytes
f7
Binary
Land
f8
Continuous
wrong_fragment
f12
Binary
logged_in
f23
Continuous
Count
f25
Continuous
serror_rate
f26
Continuous
srv_serror_rate
f29
Continuous
same_srv_rate
f32
Continuous
dst_host_count
f33
Continuous
dst_host_srv_count
f34
Continuous
dst_host_same_srv_rate
f38
Continuous
dst_host_serror_rate
f39
Continuous
dst_host_srv_serror_rateIn order to train the best possible prediction model, parameters of the GBDT algorithm are optimized by PSO. However, the PSO algorithm also has several parameters (see (9)). Some papers discussed the values of ω, c1, and c2 [34, 46, 47]. In this paper, we set the values of ω, c1, and c2 as suggested in [47]. The details of parameter setting for PSO are outlined in Table 4. According to the parameter setting of PSO, the optimized parameters for GBDT are obtained, which are presented in Table 5. In Table 5, the last parameter (i.e., minimum number of leaf) is set for the decision trees used in GBDT.Table 4
Parameters for PSO.
Parameter
Value
Number of particles (N)
20
Number of iterations (T)
200
Inertia weight (ω)
0.6
Particle factor (c1)
0.103
Population factor (c2)
2.897Table 5
Values of parameters for GBDT.
Parameter
Value
Learning rate (ν)
0.2
Number of iterations (m)
701
Minimum number of leaf
13


### 4.2. Results and Discussion

At first, we analyze the performance of SVM, random forest, C4.5, and FC-ANN with respect to different features. These four methods do not integrate feature selection; thus, we run them with all 41 features and the 18 selected features in Table3, respectively. The results are listed in Table 6. For a method M, M41 and M18 in Table 6 denote M with the 41 and 18 features, respectively. It is evident from Table 6 that the performance of each method with the 18 features is better than that with all features. These results are easy to understand, since the 18 features have more discrimination than others. In the following of this part, the results of these four methods are all with the 18 features.Table 6
Performance of four methods with different features.
Method
Accuracy
Precision
DR
F1-score
FAR
SVM41
71.40%
94.51%
55.64%
70.04%
4.27%
SVM18
73.41%
96.89%
66.28%
78.72%
2.81%
RF41
75.92%
96.85%
59.64%
73.82%
2.57%
RF18
76.57%
96.86%
60.83%
74.73%
2.61%
C4.541
80.27%
96.44%
67.84%
79.65%
3.31%
C4.518
80.58%
95.08%
69.49%
80.30%
4.75%
FC-ANN41
72.56%
97.96%
52.91%
68.71%
1.48%
FC-ANN18
75.80%
96.95%
59.36%
73.64%
2.47%Next, the detection performance of the proposed method in comparison with the six competing methods are evaluated. Table7 exhibits the overall performance of different intrusion detection approaches in terms of various criteria. The best result for each criterion is highlighted in boldface. Clearly, our model yields the highest accuracy, detection rate, and F1-score, which markedly outperforms the counterparts. It can be seen from Table 7 that all methods achieve very high precision. Although the precision of FC-ANN is the highest, its detection rate is the lowest. That makes its F1-score is only 73.64%, which is very low compared with others. For the false alarm rate, the proposed method does not perform well. In the future, we need to further reduce the false alarm rate of the proposed method.Table 7
Performance comparison of various methods in detection of abnormal connections.
Method
Accuracy
Precision
DR
F1-score
FAR
SVM
73.41%
96.89%
66.28%
78.72%
2.81%
RF
76.57%
96.86%
60.83%
74.73%
2.61%
C4.5
80.58%
95.08%
69.49%
80.30%
4.75%
FC-ANN
75.80%
96.95%
59.36%
73.64%
2.47%
CFA-DT
72.24%
93.42%
73.58%
82.32%
3.45%
IGCR-ANN
77.83%
96.85%
63.10%
76.41%
2.71%
GINI-GBDT-PSO
86.10%
96.44%
78.48%
86.54%
3.83%After introducing the overall performance of seven intrusion detection models, then we present the detection performance of these models for four attack types in Figure1. We can observe from Figure 1(a) that the proposed method outperforms all compared methods in terms of detection rate for all attack types. For the two low-frequency attack types, that is, R2L and U2R, the corresponding values of the proposed method are 34.44% and 14.66%, respectively. Whereas, all compared methods, especially SVM and FC-ANN, exhibit poor performance when meeting U2R and R2L attacks. The proportions of samples of these two attacks in training set are very low (see Table 2) which is a reason that gives rise to this scenario. However, the main reason is that the capability of these models to detect low frequency attack is insufficient. Figure 1(b) reports the precision of each detection model for four attack types. In Figure 1(b), the proposed method gets the highest precision for R2L attack; however, its precision in detection of DoS, probe, and U2R attacks is not good enough. The RF method yields the highest precision for DoS and probe attacks, but its precision for U2R and R2L is very low. To fairly evaluate the capabilities of all methods for individual attack type, Figure 1(c) depicts the F1-scores obtained by all methods. Figure 1(c) shows that the proposed model performs the best for DoS, U2R, and R2L attacks and gets the approximate best F1-score for probe attack. RF shows the best for probe attack and the second best for Dos attack, but its F1-scores about U2R and R2L attack are very low.Figure 1
Performance comparison of various methods for four attack types.
(a)
Detection rate
(b)
Precision
(c)
F1-scoreIn Table8, we show the harmonic mean of each measure for different attack types. Since some values are 0 (see Figure 1), we add a small constant, 0.01, to each value when computing harmonic mean. It is clear from Table 8 that the proposed method ranks first in terms of both DR and F1-score and third with respect to precision. For DR and F1-score, our method outnumbers the compared methods. Although, the precision of our method is lower than that of C4.5 and CFA-DT, it is far better than that of the other four.Table 8
Harmonic mean of each measure for four attack types.
Method
DR
Precision
F1-score
SVM
0.02%
0.02%
0.02%
RF
0.61%
0.64%
0.61%
C4.5
13.80%
46.94%
21.27%
FC-ANN
0.02%
3.01%
0.02%
CFA-DT
8.19%
32.42%
13.06%
IGCR-ANN
0.04%
0.04%
0.04%
GINI-GBDT-PSO
33.18%
22.18%
26.59%Finally, we compared our method with the 18 selected features as well as all 41 features. The result in detection of abnormal connections is described in Figure2. Clearly, the performance with the 18 features is better than that with 41 features. This scenario indicates that feature selection is an important technique in intrusion detection. Consequently, the combination of Gini index, GBDT algorithm and PSO technique are a competitive detection model.Figure 2
Performance comparison of the proposed methods with different features.In summary, the above experimental results demonstrate that our hybrid model can give overall better performance than single classification methods (i.e., SVM, C4.5, and RF) and some hybrid models (i.e., FC-ANN, CFA-DT, and IGCR-ANN) when detecting network intrusion. Specifically, for the low frequency attacks, our model substantially outnumbers the others. Consequently, the proposed model has a strong applicability for network intrusion detection.


## 4.1. Setting of GINI-GBDT-PSO

For the proposed method, before training an intrusion detection model, the first task is to extract the optimal subset of features. The Gini index was originally used to determine the degree of inequality of income distribution across different population sectors. The international community has usually used 0.4 as the warning line for the gap between rich and poor [45]. In our experiments, we consider a feature whose score of Gini index is less than or equal to 0.4 as important feature. As a result, 18 important features are selected from the NSL-KDD dataset, which are listed in Table 3.Table 3
Important features selected from NSL-KDD.
Id
Data type
Feature
f1
Continuous
Duration
f2
Symbolic
protocol_type
f3
Symbolic
Service
f4
Symbolic
Flag
f5
Continuous
src_bytes
f6
Continuous
dst_bytes
f7
Binary
Land
f8
Continuous
wrong_fragment
f12
Binary
logged_in
f23
Continuous
Count
f25
Continuous
serror_rate
f26
Continuous
srv_serror_rate
f29
Continuous
same_srv_rate
f32
Continuous
dst_host_count
f33
Continuous
dst_host_srv_count
f34
Continuous
dst_host_same_srv_rate
f38
Continuous
dst_host_serror_rate
f39
Continuous
dst_host_srv_serror_rateIn order to train the best possible prediction model, parameters of the GBDT algorithm are optimized by PSO. However, the PSO algorithm also has several parameters (see (9)). Some papers discussed the values of ω, c1, and c2 [34, 46, 47]. In this paper, we set the values of ω, c1, and c2 as suggested in [47]. The details of parameter setting for PSO are outlined in Table 4. According to the parameter setting of PSO, the optimized parameters for GBDT are obtained, which are presented in Table 5. In Table 5, the last parameter (i.e., minimum number of leaf) is set for the decision trees used in GBDT.Table 4
Parameters for PSO.
Parameter
Value
Number of particles (N)
20
Number of iterations (T)
200
Inertia weight (ω)
0.6
Particle factor (c1)
0.103
Population factor (c2)
2.897Table 5
Values of parameters for GBDT.
Parameter
Value
Learning rate (ν)
0.2
Number of iterations (m)
701
Minimum number of leaf
13


## 4.2. Results and Discussion

At first, we analyze the performance of SVM, random forest, C4.5, and FC-ANN with respect to different features. These four methods do not integrate feature selection; thus, we run them with all 41 features and the 18 selected features in Table3, respectively. The results are listed in Table 6. For a method M, M41 and M18 in Table 6 denote M with the 41 and 18 features, respectively. It is evident from Table 6 that the performance of each method with the 18 features is better than that with all features. These results are easy to understand, since the 18 features have more discrimination than others. In the following of this part, the results of these four methods are all with the 18 features.Table 6
Performance of four methods with different features.
Method
Accuracy
Precision
DR
F1-score
FAR
SVM41
71.40%
94.51%
55.64%
70.04%
4.27%
SVM18
73.41%
96.89%
66.28%
78.72%
2.81%
RF41
75.92%
96.85%
59.64%
73.82%
2.57%
RF18
76.57%
96.86%
60.83%
74.73%
2.61%
C4.541
80.27%
96.44%
67.84%
79.65%
3.31%
C4.518
80.58%
95.08%
69.49%
80.30%
4.75%
FC-ANN41
72.56%
97.96%
52.91%
68.71%
1.48%
FC-ANN18
75.80%
96.95%
59.36%
73.64%
2.47%Next, the detection performance of the proposed method in comparison with the six competing methods are evaluated. Table7 exhibits the overall performance of different intrusion detection approaches in terms of various criteria. The best result for each criterion is highlighted in boldface. Clearly, our model yields the highest accuracy, detection rate, and F1-score, which markedly outperforms the counterparts. It can be seen from Table 7 that all methods achieve very high precision. Although the precision of FC-ANN is the highest, its detection rate is the lowest. That makes its F1-score is only 73.64%, which is very low compared with others. For the false alarm rate, the proposed method does not perform well. In the future, we need to further reduce the false alarm rate of the proposed method.Table 7
Performance comparison of various methods in detection of abnormal connections.
Method
Accuracy
Precision
DR
F1-score
FAR
SVM
73.41%
96.89%
66.28%
78.72%
2.81%
RF
76.57%
96.86%
60.83%
74.73%
2.61%
C4.5
80.58%
95.08%
69.49%
80.30%
4.75%
FC-ANN
75.80%
96.95%
59.36%
73.64%
2.47%
CFA-DT
72.24%
93.42%
73.58%
82.32%
3.45%
IGCR-ANN
77.83%
96.85%
63.10%
76.41%
2.71%
GINI-GBDT-PSO
86.10%
96.44%
78.48%
86.54%
3.83%After introducing the overall performance of seven intrusion detection models, then we present the detection performance of these models for four attack types in Figure1. We can observe from Figure 1(a) that the proposed method outperforms all compared methods in terms of detection rate for all attack types. For the two low-frequency attack types, that is, R2L and U2R, the corresponding values of the proposed method are 34.44% and 14.66%, respectively. Whereas, all compared methods, especially SVM and FC-ANN, exhibit poor performance when meeting U2R and R2L attacks. The proportions of samples of these two attacks in training set are very low (see Table 2) which is a reason that gives rise to this scenario. However, the main reason is that the capability of these models to detect low frequency attack is insufficient. Figure 1(b) reports the precision of each detection model for four attack types. In Figure 1(b), the proposed method gets the highest precision for R2L attack; however, its precision in detection of DoS, probe, and U2R attacks is not good enough. The RF method yields the highest precision for DoS and probe attacks, but its precision for U2R and R2L is very low. To fairly evaluate the capabilities of all methods for individual attack type, Figure 1(c) depicts the F1-scores obtained by all methods. Figure 1(c) shows that the proposed model performs the best for DoS, U2R, and R2L attacks and gets the approximate best F1-score for probe attack. RF shows the best for probe attack and the second best for Dos attack, but its F1-scores about U2R and R2L attack are very low.Figure 1
Performance comparison of various methods for four attack types.
(a)
Detection rate
(b)
Precision
(c)
F1-scoreIn Table8, we show the harmonic mean of each measure for different attack types. Since some values are 0 (see Figure 1), we add a small constant, 0.01, to each value when computing harmonic mean. It is clear from Table 8 that the proposed method ranks first in terms of both DR and F1-score and third with respect to precision. For DR and F1-score, our method outnumbers the compared methods. Although, the precision of our method is lower than that of C4.5 and CFA-DT, it is far better than that of the other four.Table 8
Harmonic mean of each measure for four attack types.
Method
DR
Precision
F1-score
SVM
0.02%
0.02%
0.02%
RF
0.61%
0.64%
0.61%
C4.5
13.80%
46.94%
21.27%
FC-ANN
0.02%
3.01%
0.02%
CFA-DT
8.19%
32.42%
13.06%
IGCR-ANN
0.04%
0.04%
0.04%
GINI-GBDT-PSO
33.18%
22.18%
26.59%Finally, we compared our method with the 18 selected features as well as all 41 features. The result in detection of abnormal connections is described in Figure2. Clearly, the performance with the 18 features is better than that with 41 features. This scenario indicates that feature selection is an important technique in intrusion detection. Consequently, the combination of Gini index, GBDT algorithm and PSO technique are a competitive detection model.Figure 2
Performance comparison of the proposed methods with different features.In summary, the above experimental results demonstrate that our hybrid model can give overall better performance than single classification methods (i.e., SVM, C4.5, and RF) and some hybrid models (i.e., FC-ANN, CFA-DT, and IGCR-ANN) when detecting network intrusion. Specifically, for the low frequency attacks, our model substantially outnumbers the others. Consequently, the proposed model has a strong applicability for network intrusion detection.


## 5. A Brief Review of Hybrid Models

Recent years have witnessed a surge of research efforts about the application of hybrid techniques for intrusion detection. This section briefly lists some recent studies.Aburomman and Reaz [12] developed an ensemble method based on the SVM and k-nearest neighbors (k-NN) classifiers, as well as the PSO algorithm to enhance the detection accuracy. Their approach trains six k-NN experts with different ks and six SVM experts with different parameters on the same dataset and then generates two new ensembles by combining the opinions of 12 experts using the PSO and metaoptimized PSO, respectively. The difference of both ensembles is in the setting of the PSO behavioral parameters. In the first ensemble, the behavioral parameters are manually selected, and in the second one, the behavioral parameters are optimized by using local unimodal sampling (LUS) [48]. Additionally, they also gave an ensemble method by weighing each expert using the weighted majority algorithm (WMA) [49], for the purpose of comparison. The experimental results on five randomly selected subsets from the KDDCup99 dataset show that the LUS-based ensemble achieves the best accuracy.De la Hoz et al. [18] applied statistical techniques and the self-organizing map (SOM) approach to detect network anomalies. Two machine learning techniques, principal component analysis (PCA) and Fisher discriminant ratio (FDR), were considered in their work to remove irrelevant features and noises. Then, the probabilistic SOM (PSOM), a fuzzy version of the classical SOM, is used to model and classify the filtered feature space. Experiments run on the NSL-KDD dataset show that the PCA + FDR + PSOM model has the best performance compared with the contrasted hybrid models.De la Hoz et al. [17] also introduced another hybrid model into the network intrusion detection problem with the purpose of diminishing model complexity and promoting detection performance. In that paper, feature selection is carried out by a multiobjective optimization, and both anomaly detection and attack classification are fulfilled by the growing hierarchical self-organizing maps (GHSOMs) [50]. The multiobjective approach is implemented by the NSGA-II algorithm [51] in which Jaccard coefficient is employed to measure the performance of a classifier for the corresponding class. Experimental results on the NSL-KDD dataset illustrate that their proposed model is better than other approaches in terms of detection accuracy and detection rate.Wang et al. [14] proposed a hybrid model by uniting the fuzzy clustering and ANN. First, their model partitions the training set into several different groups using fuzzy clustering technique. Then, different ANN models are trained based on different groups. Finally, a fuzzy aggregation module is generated by aggregating all ANN models’ results. Experimental results on the KDDCup99 dataset indicate that this model outperforms the decision trees, naive Bayes, and backward propagation neural network (BPNN).Eesa et al. [19] presented a hybrid method, which combines the cuttlefish optimization algorithm (CFA) and the decision tree classifier, to detect network intrusions. In their model, the CFA is used to select significant attributes, and the decision tree algorithm is utilized to identify types of abnormal events. The performance of this hybrid model was evaluated on the KDDCup99 dataset. The results demonstrate when the number of attributes is less than 25, the detection accuracy and detection rate have obvious improvement.Guo et al. [2] proposed a two-level hybrid approach for intrusion detection by exploiting the strengths of misuse-based and anomaly-based detection methods. This approach is composed of two anomaly detection components (ADCs) and one misuse detection component (MDC). In stage 1, the ADC 1 detects abnormal connections using the ADBCC method [2]. Then, the declared abnormal and normal connections are, respectively, sent to the ADC 2 and the MDC in parallel to further evaluated by k-NN. This hybrid approach was experimentally tested on the KDDCup99 and the Kyoto University Benchmark dataset. The results show that this approach is effective in detection abnormal connections with a low false alarm rate.Akashdeep et al. [3] developed an intelligent system by means of feature selection and ANN classifier. The proposed system first ranks features according to information gain and correlation and then selects useful features by combining their ranks using a novel approach. Experimental results on the KDDCup99 dataset illustrate that the system is really encouraging.


## 6. Conclusion

Motivated by the successes of Gini index and GBDT algorithm in other fields, this paper proposed the GINI-GBDT-PSO method, a novel hybrid intrusion detection model to improve the performance of network intrusion detection systems. The proposed model first extracts the optimal subset of features from whole dataset by using the Gini index. Then, the GBDT algorithm, which is a gradient boosting approach, is adopted to detect abnormal connections. In addition, the PSO algorithm is employed to optimize parameters of the GBDT algorithm in the proposed model. This model can not only enhance the overall performance for network intrusion detection effectively but also improve the detection performance for each type of attack.In order to validate the performance of our method, we performed experiments on the NSL-KDD dataset compared with six baselines. Five evaluation criteria are introduced to conduct fair comparisons, which are accuracy, detection rate, precision, F1-score, and false alarm rate. The experimental results demonstrated that the proposed model performs the best on the whole in comparison with baselines. These results indicate that the proposed model is an accurate and effective solution for network intrusion detection systems.



---

*Source: 1578314-2018-03-26.xml*