<figure><img src="image_1.png" /></figure>

HAL
open science

# DAFL: Deep Adaptive Feature Learning for Network Anomaly Detection

Shujian Ji, Tongzheng Sun, Kejiang Ye, Wenbo Wang, Cheng-Zhong Xu

► To cite this version:

Shujian Ji, Tongzheng Sun, Kejiang Ye, Wenbo Wang, Cheng-Zhong Xu. DAFL: Deep Adaptive Feature Learning for Network Anomaly Detection. 16th IFIP International Conference on Network and Parallel Computing (NPC), Aug 2019, Hohhot, China. pp.350-354, 10.1007/978-3-030-30709-7_32. hal-03770566

HAL Id: hal-03770566

https://inria.hal.science/hal-03770566v1

Submitted on 6 Sep 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_3.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# DAFL: Deep Adaptive Feature Learning for Network Anomaly Detection

Shujian Ji¹,², Tongzheng Sun¹, Kejiang Ye*¹, Wenbo Wang³, and Cheng-Zhong Xu⁴

¹ Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences,
Shenzhen 518055, China

² University of Chinese Academy of Sciences, Beijing 100049, China

³ Khoury College of Computer Sciences, Northeastern University, Seattle WA 98109,
USA

⁴ Faculty of Science and Technology, University of Macau, Macau, China
sj.ji, tz.sun, kj.ye@siat.ac.cn, wang.wenbo@husky.neu.edu,
czxu@um.edu.mo

**Abstract.** With the rapid development of the Internet and the growing complexity of the network topology, network anomaly has become more diverse. In this paper, we propose an algorithm named Deep Adaptive Feature Learning (DAFL) for traffic anomaly detection based on deep learning model. By setting proper feature parameters $\theta$ on the neural network structure, DAFL can effectively generate low-dimensional new abstract features. Experimental results show the DAFL algorithm has good adaptability and robustness, which can effectively improve the detection accuracy and significantly reduce the detection time.

**Keywords:** Network Anomaly Detection · Deep Learning · Feature Learning.

## 1 Introduction

Network attack is a serious problem in the Internet environment. With the rapid development of the Internet and the growing complexity of the network topology, network anomaly has become more diverse. Network anomaly detection is an effective way to deal with different network attacks [1].

Machine learning is a common method for anomaly detection in the network environment, such as Naive Bayes, Support Vector Machine and other shallow learning technologies [2,3]. Although these technologies have improved the detection accuracy to a certain extent, they also face some limitations. For example, expert knowledge is required for data processing, and a large amount of time is needed for data training. Recently, deep learning based methods [4–6] are proposed for anomaly detection due to the better feature learning ability. However, they improved the detection accuracy, without taking into account the training time and execution time in high-speed networks.

<footnote>* Corresponding author</footnote>
In this paper, we propose an algorithm named Deep Adaptive Feature Learning (DAFL) which can utilize the feature learning ability of deep learning and the advantages of transfer learning. The contributions of this paper are summarized as follows: (i) the algorithm can determine the structure of the neural network according to the dimension of data. (ii) By combining deep learning with shallow machine learning, DAFL improves the classification performance of anomaly detection and greatly reduces the training time.

## 2 DAFL Algorithm

we design the DAFL algorithm to determine the number of layers of the network hidden layer and the number of neurons in each layer according to the dimensions of the input data, and construct a pre-trained learning model that can adapt to the dimension of data features, as shown in Algorithm 1.

<caption>**Algorithm 1 Deep Adaptive Feature Learning**</caption>

**Require:** training sample $v$, feature parameter $\theta$, learning rate $\eta$, list $N$
**Ensure:** pre-train model $xW + b$
1: $D$ = data dimension of $v$, layer number: $l = \lceil D/5 \rceil$
2: initial $n_1 = D$, calculate neurons number of each layer: $n_l = \lceil \theta * D \rceil$
3: **for** $i = 2$ to $l-1$ **do**
4:     neurons: $n_i = \lceil D/i^2 \rceil + \lceil \theta * D \rceil$, save $n_i$ to $N$
5:     **end for**
6:     **for** $i = 1$ to $l$ **do**
7:         use $N$ to build network with $l$-th layers and $n_i$ neurons.
8:         output layer: $S(x) = \frac{1}{1+e^{-x}}$
9:     **end for**
10:    **for all** $v_i$ **do**
11:        calculate the actual output of the neuron $v_i'$
12:        $\delta_k = v_i'(1 - v_i')(v_i - v_i')$
13:        hidden layer $h$ error gradient: $\delta_h = v_h'(1 - v_h')W_h\delta_k$
14:        update the weights: $W_{ij} = W_{ij} + \Delta W_{ij}$, $\Delta W_{ij} = \eta O_i \delta_j$, update the bias: $b_j = b_j + \eta \delta_j$
15:    **end for**

In order to balance the training speed and accuracy of the deep learning model, we design a feature parameter $\theta$ (from 0.1 to 1) as the control value in the DAFL algorithm to make the high hidden layer generate abstract features of different dimensions. As shown in Fig. 1, combining the deep network structure based on DAFL with different conventional shallow machine learning classifiers can be used as the detection model.
<figure><img src="image_4.png" /><figcaption>(a) Pre-train DBN model by using (b) Use the remain structure to generate new feature</figcaption></figure>

<caption>Fig. 1. The design of DAFL</caption>

## 3 Experiment

We conduct experiments with NSL-KDD [7] dataset to evaluate our proposed algorithm. By comparing the performance of the original data and the data processed by the DAFL algorithm on the classifier, we can verify the validity of DAFL.

<table><caption>Table 1. Models Performance in NSL-KDD Dataset</caption><thead><tr><th>Model</th><th>Accuracy</th><th>Precision</th><th>Recall</th><th>F<sub>1</sub> - score</th><th>Time(s)</th></tr></thead><tbody><tr><td>Support Vector Machine(SVM)</td><td>97.26%</td><td>98.03%</td><td>96.06%</td><td>97.19%</td><td>91.73s</td></tr><tr><td>DAFL SVM(θ=0.8)</td><td><strong>99.17%</strong></td><td><strong>99.49%</strong></td><td><strong>98.74%</strong></td><td><strong>99.15%</strong></td><td><strong>8.72s</strong></td></tr><tr><td>K-Nearest Neighbors(KNN)</td><td>99.02%</td><td>99.34%</td><td>98.88%</td><td>99.15%</td><td>107.19s</td></tr><tr><td>DAFL KNN (θ=0.8)</td><td><strong>99.21%</strong></td><td><strong>99.41%</strong></td><td><strong>98.89%</strong></td><td><strong>99.19%</strong></td><td><strong>13.62s</strong></td></tr><tr><td>Logistic Regression(LR)</td><td>95.05%</td><td>95.26%</td><td>94.05%</td><td>94.98%</td><td>1.98s</td></tr><tr><td>DAFL LR(θ=0.8)</td><td><strong>99.15%</strong></td><td><strong>99.36%</strong></td><td><strong>98.82%</strong></td><td><strong>99.13%</strong></td><td><strong>0.40s</strong></td></tr><tr><td>Decision Tree(DT)</td><td>98.94%</td><td>98.98%</td><td>98.75%</td><td>98.93%</td><td>1.19s</td></tr><tr><td>DAFL DT(θ=0.8)</td><td><strong>99.67%</strong></td><td><strong>99.72%</strong></td><td><strong>99.58%</strong></td><td><strong>99.65%</strong></td><td><strong>0.47s</strong></td></tr><tr><td>Naive Bayes(NB)</td><td>88.82%</td><td>86.72%</td><td>89.24%</td><td>88.66%</td><td>0.10s</td></tr><tr><td>DAFL NB(θ=0.8)</td><td><strong>98.77%</strong></td><td><strong>95.25%</strong></td><td><strong>99.75%</strong></td><td><strong>97.70%</strong></td><td><strong>0.06s</strong></td></tr></tbody></table>

Experiments show that the classifier achieves the best result when the feature parameter is set to 0.8. Table 1 shows the changes of classifier performance metrics when DAFL is applied to the classifier on the NSL-KDD dataset. It is worth noting that the accuracy in the NB classifier increased from 88.92% to 98.77%, and the recall increased from 89.24% to 99.75%. In terms of detection time, the classifier that has been processed by the DAFL algorithm has a significant reduction in detection time. The most obvious change is that the time of the SVM classifier is reduced from 91.73s to 8.72s, and the detection time of the KNN classifier is reduced from 107.19s to 13.62s. Fig. 2 shows the accuracy and time saving percentage on NSL-KDD.
<figure><img src="image_5.png" /><figcaption>(a) Classifiers' accuracy perfor- (b) NSL-KDD time saving after
mance comparison on NSL-KDD DAFL</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>(a) PCA visualization for NSL-KDD (b) PCA visualization for NSL-KDD after DAFL</figcaption></figure>

<caption>Fig. 3. PCA visualization for NSL-KDD before and after DAFL</caption>

We perform data scatter visualization by PCA method for normal traffic and abnormal traffic in the dataset in Fig. 3. It is obvious that the DAFL algorithm can separate the normal traffic and abnormal traffic.

4 Related work

There are a lot of work on network anomaly detection. Ibrahimi *et al.* used classification algorithms such as linear discriminant analysis (LDA) and principal component analysis (PCA) to classify abnormal network traffic [8]. Alrawashdeh *et al.* used Restricted Boltzmann Machine (RBM) to perform unsupervised feature reduction [9]. Potluri *et al.* proposed an accelerated DNN structure for identifying network data anomalies [10]. Kang *et al.* proposed an intrusion detection system based on deep neural network [11]. Our research group also proposed different algorithms and tool for network anomaly detection [1, 4–6].
## 5 Conclusion

In this paper, we propose a DAFL algorithm for network anomaly detection that can determine the number of hidden layers and the number of neurons in each hidden layer according to the dimension of the original data. Using the idea of transfer learning, we remove the output layer of the neural network and use the residual structure to generate new data with abstract features as input of other machine learning classifiers. The experimental results show that the method achieves good results, and has a certain degree of robustness and adaptability.

**Acknowledgment.** This work is supported by the National Key R&D Program of China (No. 2018YFB1004804), National Natural Science Foundation of China (No. 61702492), Shenzhen Discipline Construction Project for Urban Computing and Data Intelligence, and Shenzhen Basic Research Program (No. JCYJ20170818153016513).

## References

1. P. Lin, K. Ye, and C.-Z. Xu, "Netdetector: an anomaly detection platform for networked systems," in *IEEE International Conference on Real-time Computing and Robotics*. IEEE, 2019.

2. T. Shon, Y. Kim, C. Lee, and J. Moon, "A machine learning framework for network anomaly detection using svm and ga," in *Proceedings from the sixth annual IEEE SMC information assurance workshop*. IEEE, 2005, pp. 176-183.

3. N. B. Amor, S. Benferhat, and Z. Elouedi, "Naive bayes vs decision trees in intrusion detection systems," in *Proceedings of the 2004 ACM symposium on Applied computing*. ACM, 2004, pp. 420-424.

4. P. Lin, K. Ye, and C.-Z. Xu, "Dynamic network anomaly detection system by using deep learning techniques," in *International Conference on Cloud Computing*. Springer, 2019, pp. 161-176.

5. M. Zhu, K. Ye, Y. Wang, and C.-Z. Xu, "A deep learning approach for network anomaly detection based on amf-lstm," in *IFIP International Conference on Network and Parallel Computing*. Springer, 2018, pp. 137-141.

6. M. Zhu, K. Ye, and C.-Z. Xu, "Network anomaly detection and identification based on deep learning methods," in *International Conference on Cloud Computing*. Springer, 2018, pp. 219-234.

7. "Nsl-kdd," https://iscxdownloads.cs.unb.ca/iscxdownloads/NSL-KDD", 1999.

8. K. Ibrahimi and M. Ouaddane, "Management of intrusion detection systems based-kdd99: Analysis with lda and pca," in *2017 International Conference on Wireless Networks and Mobile Communications (WINCOM)*. IEEE, 2017, pp. 1-6.

9. K. Alrawashdeh and C. Purdy, "Toward an online anomaly intrusion detection system based on deep learning," in *2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA)*. IEEE, 2016, pp. 195-200.

10. S. Potluri and C. Diedrich, "Accelerated deep neural networks for enhanced intrusion detection system," in *2016 IEEE 21st International Conference on Emerging Technologies and Factory Automation (ETFA)*. IEEE, 2016, pp. 1-8.

11. M.-J. Kang and J.-W. Kang, "Intrusion detection system using deep neural network for in-vehicle network security," *PloS one*, vol. 11, no. 6, p. e0155781, 2016.