This is the post-print version of the following article: Loc Gia Nguyen and Kohei Watabe, ``A Method for Network Intrusion Detection Using Flow Sequence and BERT Framework,'' in Proceedings of 2023 IEEE International Conference on
Communication (ICC 2023), 2023. The original publication is available at https://doi.org/10.1109/ICC45041.2023.10279335 (DOI: 10.1109/ICC45041.2023.10279335). (c) 2023 IEEE. Personal use of this material is permitted. Permission fromIEEE must be obtained for all other users, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted componentsof this work in other works.

A Method for Network Intrusion Detection Using
Flow Sequence and BERT Framework
Loc Gia Nguyen

Kohei Watabe

Graduate School of Engineering
Nagaoka University of Technology
Nagaoka, Niigata, Japans203145@stn.nagaokaut.ac.jp

Graduate School of Engineering
Nagaoka University of Technology
Nagaoka, Niigata, Japank watabe@vos.nagaokaut.ac.jp

Abstract—A Network Intrusion Detection System (NIDS) is atool that identifies potential threats to a network. Recently, different flow-based NIDS designs utilizing Machine Learning (ML)algorithms have been proposed as solutions to detect intrusions
efficiently. However, conventional ML-based classifiers have notseen widespread adoption in the real world due to their poor domain adaptation capability. In this research, our goal is to explore
the possibility of using sequences of flows to improve the domain
adaptation capability of network intrusion detection systems.
Our proposal employs natural language processing techniquesand Bidirectional Encoder Representations from Transformers
framework, which is an effective technique for modeling datawith respect to its context. Early empirical results show that ourapproach has improved domain adaptation capability compared
to previous approaches. The proposed approach provides a newresearch method for building a robust intrusion detection system.
Index Terms—Network security, NIDS, deep learning, BERT,CIDDS-001

I. I NTRODUCTION
As more of our daily lives depend on computer networks,
safeguarding the information contained within these systems
and maintaining service availability has become a necessity.
Systems that handle financial transactions, personal information, or those that serve as social platforms are prime targets
for bad actors seeking to gain financial assets or influence
over society. Currently, many different cybersecurity measures
are deployed in tandem to counter these threats, and intrusion
detection is an important component of these measures.
Intrusion detection is the process of monitoring the eventsoccurring in a computer system or network and analyzing themfor signs of malicious activities. Network traffic is captured ineither packet-based or flow-based format. Packet-based datacontain complete payload information. Flow-based data are
aggregated packet-based data and typically contain various
properties of the collection of packets and metadata fromnetwork connections [2]. NIDSs, therefore, can be separated
into two categories: packet-based and flow-based. The former
analyzed individual packet payloads and header information,
An earlier and short version of this paper was presented at the 18th International Conference on emerging Networking EXperiments and Technologies
Student Workshop (CoNEXT-SW ’22) [1].
This work was partly supported by JSPS KAKENHI Grant Number
JP20H04172.

while the latter only analyzed the aggregated properties of acollection of packets. The speed at which flow-based NIDSs
analyze network traffic has made it the preferred solution formodern network security systems.
Changes in the users’ behavior or appearances of new
attacks manifest as shifts in the distribution of flow features.
Therefore, NIDSs need to adapt to the new benign flows
while still being able to detect malicious ones. Flow-based
NIDSs analyze the properties of the packet collections using
machine learning algorithms, such as K-Nearest Neighbors andRandom Forest. However, these machine learning algorithms
often adapt poorly to changes in data distributions, which arecommon in real-world deployment, which in turn limits theirusefulness [3], [4]. To improve the domain adaptability of
NIDSs, Camila et. al. [5] proposed the Energy-based Flowclassifier (EFC) - a statistical approach where the NIDS
would model the statistical distribution of benign flows, thenclassifies flow that deviates from this distribution as malicious.
Although it improves the domain adaptability of NIDSs, there
are still limitations on certain data distributions. We assume
that the reason for the limitations of current algorithms is theuse of singular flows as input data, as the classifier can only
model the distribution of features within a flow.
In this paper, we propose the use of a sequence of flows,i.e., a collection of flows that are temporally close together,and Bidirectional Encoder Representations from Transformers(BERT) framework to obtain the representation of a network
flow in a sequence. A Multi-layer Perceptron (MLP) then processes this vector representation to produce the classification
result of the flow. To the best of our knowledge, this is thefirst example of using BERT to improve the domain adaptation
capability of NIDS. This paper evaluates the performanceof our proposal on the entirety of the CIDDS data sets, as
opposed to a subset of data in an earlier version [1].
In summary, the contribution of this study is as follows.
We investigate the possibility of using information froma sequence of network flows to improve the domain
adaptation capability of the network intrusion detection
system.
• We propose an NIDS that uses BERT for feature extraction and MLP for classification.
•

We demonstrate the domain adaptation capability of the
proposed system using authentic and synthetic data sets.
The rest of the paper is organized as follows: In section II,
we present an overview of prominent studies in the literature
on flow-based NIDSs. Section III provides the basic conceptof network flow and network intrusion detection system, thedata sets used in this study, and an introduction to the BERT
framework. Section IV illustrates the details of the proposal
and its evaluation method. Performance evaluation results ofour proposal and related discussion are provided in Section V.
Finally, Section VI concludes this paper.
•

II. R ELATED W ORKS
The use of machine learning for implementing NIDSs has
been extensively researched [6], with deep learning methods
being employed to further improve upon machine learning
methods by using deep learning models to extract featuresfrom flow data. A popular technique for feature extraction in
NIDS is the autoencoder. It works on the idea of matching theoutput as close the to input as possible by learning the best
features. Shone et al. [7], Yan et al. [8] and Al-Qatf et al. [9]showed that autoencoders can retain important information
about a flow while reducing its dimensions. This approach
reduces the processing time needed by machine learningclassifiers and boosts their classification capability. Andresini
et al. [10] trained different autoencoders for each type of flow
and uses the representation produced by these autoencoders totrain a classifier. Khan et al. [11] showed that autoencoders
can also be used as a scoring function that takes the featuresof a flow as input and returns a score. The autoencoder in thiscase is optimized to assign a low score for normal flow and ahigh score for suspicious flows.
Thapa et al. [12] conducted an experimental comparison of
NIDSs based on various machine learning and deep learningtechniques. Their performance was evaluated on CIDDS001 and CIDDS-002 data sets. The study includes k-nearest
neighbors, XGBoost, and decision tree classifiers as machinelearning techniques. For deep learning techniques, Embedding
Layer, Convolutional Neural Network, and Long Short Term
Memory were considered.
A model is only as good as the data used to train it. The
preparation of data for training NIDSs is also an important aspect of NIDS research. Verma et al. [13] performed a statisticalanalysis of the labeled flow-based CIDDS-001 dataset using
k-nearest neighbor classification and k-means clustering algorithms. Abdulhammed et al. [14] proposed several techniques
to handle imbalanced training data and improve NIDSs performance. The effectiveness of sampling methods on CIDDS-001
is studied and experimentally evaluated through deep neuralnetworks, random forest, voting, variational autoencoder, andstacking machine learning classifiers.
All of the previously mentioned papers focused on improving the detection of malicious activities and did not evaluate
the performance of the approaches concerning changing data
distributions. Camila et al. [5] stated that machine learning
approaches adapt poorly to changing data distributions and

TABLE IATTRIBUTES WITHIN THE CIDDS-001 AND CIDDS-002 DATA SETS

#
1
2
3
4
5
6
7
8
9
10

Name
Date first seen
DurationProto
Src IP
Src PtDst IP
Dst PtPackets
Bytes
Flags

Description
Start time flow first seen
Duration of the flow
Transport Protocol (e.g. ICMP, TCP, or UDP)
Source IP Address
Source Port
Destination IP Address
Destination Port
Number of transmitted packets
Number of transmitted bytes
OR concatenation of all TCP Flags

proposed Energy-base Flow Classifier (EFC). This algorithm
is based on inverse statistics to infer a statistical model basedon benign flows. The authors showed that the algorithm could
adapt to different data distributions, making it suitable fordetecting zero-day threats.
III. M ATERIALSIn this section, first, we introduce the concept of the subjectof our research, network traffic flow data. Second, we present
the data sets used to evaluate NIDS performance. Finally, we
provide an overview of the BERT framework.
A. Network traffic flow data
A network flow is a sequence of packets carrying information between two hosts where packets have common
properties. All packets within a flow share the same 5-tuple(Src IP, Src Pt, Dst IP, Dst Pt, Proto). All inter-packet times areless than an arbitrary flow expiry timeout value. Other than the
5-tuple or 3-tuple, a flow can include other flow keys, suchas the number of transmitted packets, bytes, etc. There aremany standards for flow data formats, including NetFlow and
OpenFlow.
B. Benchmark data sets
CIDDS-001 [15] and CIDDS-002 [16] are relatively recent
intrusion detection benchmark data sets containing unidirectional NetFlow data. The CIDDS-001 data set contains two
sets of flows, one is captured within a simulated OpenStackenvironment, and the other is captured from a server deployedon the Internet. The CIDDS-002 data set contains one set offlows from a simulated OpenStack environment.
In this study, CIDDS-001 OpenStack, CIDDS-001 External
Server, and CIDDS-002 OpenStack environments are used toevaluate the domain adaptation capability of our proposal.
Within the CIDDS-001 data set, a change from the simulated
OpenStack environment to the external server environment
is considered a domain change, as the distribution of flow
features differs between the environments. Similarly, a change
from CIDDS-001 to CIDDS-002 OpenStack environment isalso considered a domain change. We utilize these domainchanges to evaluate the domain adaptation capability of our
proposal.

The network flows within CIDDS-001 and CIDDS-002
are all labeled. In the OpenStack environment of CIDDS001 and CIDDS-002, benign flows are labeled normal, whilemalicious flows are labeled dos, scan, portScan, pingScan,or bruteForce, according to the type of attack. The External
Server environment contains both simulated flows from the
OpenStack environment and real flows from the internet. The
real flows were labeled unknown for traffic to port 80 and
443 and suspicious for the remaining traffic. Flows labeledunknown are likely to be traffic from normal users and areconsidered benign in this study, and those labeled suspicious
are considered malicious.
Table I shows an overview of the attributes within the
CIDDS-001 and CIDDS-002 data sets. Source IP address,
destination IP address, and Date first seen were not used forclassification in this study as they are arbitrary and carry no
useful information.
C. Bidirectional Encoder Representations from Transformers
Bidirectional Encoder Representations from Transformers(BERT) [17] is a transformer-based machine-learning technique for NLP developed by Google. The BERT frameworkcomprises two steps: pre-training and fine-tuning. In pretraining, the BERT model trains on unlabeled data. For finetuning, the model is initialized using the pre-trained parameters
and then trained using labeled data from the downstream tasks.
BERT is pre-trained with two unsupervised tasks: Masked
Language Modeling (MLM) and Next Sentence Prediction(NSP). In MLM, some words in a sentence are replaced witha different token. The objective is to predict the original valueof the masked words based on other unmasked words in thesentence. In NSP, BERT takes sentence pairs as input. The
objective is to predict whether the second sentence in thepair is the next in the document. For fine-tuning, task-specificinputs and outputs are added to a pre-trained BERT model.
IV. M ETHOD
In this section, we first highlight the benefit of using asequence of flows as input for our NIDS. Next, we describe
the detail of our proposed system architecture. Finally, weelaborate on the evaluation methods used in this study.
A. Conditional information from flow sequence
A sequence of flows are flows that appear temporally nearto one another. We can determine the appearance probabilityof the next flow in the network based on past flows. Table IIshows two flow sequences where a flow can be distinguishedbetween malicious or benign based on the information provided by the sequence. The flows in bold have similar port
numbers and flags, however, they are malicious in the first
sequence and benign in the second sequence. If we were toclassify these flow base on their features alone, these flows
would all be classified as either benign or malicious. In thefirst sequence, these flows appear multiple times in a row andexhibit rapid changing of port numbers, this can be a pattern ofan attacker retrying attacks at port 22. In the second sequence,

TABLE IIE XAMPLE OF TWO SEQUENCES OF FLOWS FROM CIDDS-001 DATA SET
Duration
9.588
9.588
9.555
9.555
10.412
10.412
0
0
11.03
11.03
0.054
0.054
0.077
0.077
526.089526.089
0
0
0.07
0.07

Proto
TCP
TCP
TCP
TCP
TCP
TCP
UDP
ICMP
TCP
TCP
TCP
TCP
TCP
TCP
TCP
TCP
TCP
TCP
TCP
TCP

Src Pt
22
47695
54731
22
22
57489
56475
0
33056
22
8000
52252
8000
52253
22
59862
53213
23
8000
52243

6-dim vector

6 features

Src Pt: 445
Dst Pt: 48888
Proto+Flags: TCP.AP...
# Packets: 1# Bytes: 108
Duration: 0.000

Dst Pt
47695
22
22
54731
57489
22
19
3.3
22
33056
52252
8000
52253
8000
59862
22
23
53213
52243
8000

Packets
19
15
15
19
19
15
1
1
19
21
7
6
7
6
178
180
1
1
7
6

768-dim vector(128 x 6)

768-dim vector(128 x 6)[0.1, -0.2, ...][-0.0, -0.2, ...]
[0.3, 0.9, ...]
[0.7, -0.2, ...]
[0.8, -0.3, ...]
[0.2, 0.1, ...]

[0.3, 0.4, ...]
[0.7, -0.3, ...][-0.9, 0.7, ...]
[0.5, -0.8, ...][-0.3, -0.3, ...]
[0.4, 0.6, ...]

[12,
43,
23,
54,
85,
23]

sequenceof flows

...

...

...

Bytes
3185
2163
2163
3185
3185
2163
46
57
2383
3369
556
515
702
586
24253
13471
46
40
702
586

Flags
.AP.SF.AP.SF.AP.SF.AP.SF.AP.SF.AP.SF......
......
.AP.SF.AP.SF.AP.SF.AP.SF.AP.SF.AP.SF.AP.SF.AP.SF....S.
.A.R..
.AP.SF.AP.SFClass
suspicious
suspicious
suspicious
suspicious
suspicious
suspicious
suspicious
suspicious
suspicious
suspicious
normalnormalnormalnormalnormalnormalsuspicious
suspicious
normalnormal2-dim vector

Linearlayer
768 to 2

BERT[0.99,
0.01]

benignbenign 0.99
malicious 0.01

benign...

...

predictions
probabilities

outputpredictions

Fig. 1. Proposed system architecture

the flows appear isolated. In cases where flow features arenearly identical, the extra information provided by examining
the flows surrounding the target flow helps to distinguish them.
The patterns that arise in a sequence of flows are featureagnostic, that is the same pattern can appear with arbitrary
features. Therefore, information from flow sequence is robust
between different domains, where features within flows mightvary. In summary, with the use of sequences of flows, extra
information can be made available to the classifier, and thisinformation is independent of the features within flows.
B. System architecture
We propose a method for detecting network intrusion basedon the BERT and MLP models. The BERT model functions
as a feature extractor, transforming network traffic flows
into vectors. While the MLP model function as a classifier,classifying network traffic flows into benign and malicious
flows.
We first organize network traffic flows into sequences, witha flow being considered analogous to a word and a sequenceof flows to a sentence. Similar to how natural languages
are processed, the sequences of flows are then encoded intosequences of tokens. Training of the proposed model consistsof two steps. First, we pre-train the BERT model using theMLM task and only the benign flows. Since there is noinherent order to flows, we opt not to use positional encoding
and special starting and ending tokens as used in the typical
BERT framework. Second, we replace the MLM output layer

TABLE III
L ABELS WITHIN CIDDS ENVIRONMENTS
CIDDS-001 OpenStackLabel#normal28051906
dos
2959027
portScan
265918
pingScan6090
bruteForce
4992

CIDDS-001 External ServerLabel#normal134240unknown
77923
suspicious
437911
portScan18719
bruteForce
2448

TABLE VU PPER VALUE OF BINS FOR FEATURE DISCRETIZATIONCIDDS-002 OpenStackLabel#normal15598543
scan
562640FeatureDurationProtocolSrc PtDst PtBytes
Packets
Flags

Upper limit of each bin
0.001, 0.002, 0.003, 0.004, 0.005, 0.006, 0.01, 0.04, 1, 10, 100, ∞TCP, UDP, GRE, ICMP, IGMP
50, 60, 100, 400, 500, 40000, 60000, ∞50, 60, 100, 400, 500, 40000, 60000, ∞50, 60, 70, 90, 100, 110, 200, 300, 400, 500, 700, 1000, 5000, ∞2, 3, 4, 5, 6, 7, 10, 20, ∞{(f0 , f1 , f2 , f3 , f4 , f5 ) |fi ∈ {0, 1}}

TABLE IV
N UMBER OF LABELS IN EACH DATA SET USED IN THIS STUDY

BenignMalicious
Total

CIDDS-001
internal
28051906
323602731287933

CIDDS-001
balanced
3236027
32360276472054

CIDDS-001
external
212163459078671241

CIDDS-002
15598543
56264016161183

with an MLP classifier and perform fine-tuning. In this step,
we used all the flows in the data set and their labels to train the
BERT model and MLP classifier. Preserving the distributionof flows within a sequence is important; therefore, the flows in
the training data set are not shuffled. We use cross-entropy as
the training criterion for both steps. It maximizes the likelihoodof guessing the correct flow features given the context in thefirst step and the correct label in the second step.
V. E VALUATION
A. Model configuration
The overall architecture of the system is illustrated in
Figure 1. Each flow contains six features, these features are
encoded as numbers (flowi ). BERT decodes each number into
a 128-dimension vector, concatenates them to form a 768dimension vector (eflowi ), and processes it to represent theflow as a different 768-dimension vector (hflowi ). The output
of BERT is then passed through a Multilayer Perceptron classifier (a linear layer with softmax output), which reduces thedimension from 768 to 2. This 2-dimension vector represents
the probability of each class (benign and malicious). The final
prediction label is chosen to be the class with the higher
probability.
B. Benchmarks
We compare our proposal with EFC - an approach thataims at better domain adaptability. Other ML-based NIDS
methods included in this comparison are Decision Tree (DT),K-Nearest Neighbors (KNN), Multilayer Perceptron (MLP),
Naive Bayes (NB), and Support Vector Machine (LinSVM),
AdaBoost (AB) and Random Forest (RF). We deployed the
ML-based classifiers with their default scikit-learn (version
1.1.2) configurations. We created the proposed model using
PyTorch. Flow features are discretized for all classifiers, as
most ML algorithms perform better with discrete than continuous features. We use the discretization thresholds described in
EFC’s paper [5]. Table V details the value of the discretizationthresholds.
We chose the CIDDS-001 and CIDDS-002 data sets toevaluate the domain adaptation of our proposal due to theirrecency and the inclusion of multiple network environments.
Three different environments (domains): CIDDS-001 OpenStack, CIDDS-001 External Server, and CIDDS-002 OpenStack are used to evaluate the domain adaptation capability ofour proposal compared to other classifiers. The classifier will
be tested on anomaly detection, which is a binary classification
task. Therefore, the labels will be grouped into benign and malicious. Labels within each environment are shown in Table III.
Flows labeled normal and unknown are considered benign,
while those labeled otherwise are considered malicious. The
composition of the data sets used in the experiments is shownin Table IV.
We conducted two experiments to evaluate the domain adaptation capability of our proposal against other classifiers. In
the first experiment, training was performed on the entirety offlow data from the CIDDS-001 OpenStack environment with
CIDDS-001 internal data set, while testing was performed on
the CIDDS-001 External Server environment with CIDDS-001
external data set and CIDDS-002 OpenStack environment with
the CIDDS-002 data set. As label imbalance in the training
data set can impair the performance of some classifiers [4], asecond experiment in which the training data set is balanced isconducted. In the second experiment, testing was performed on
the same data sets as in the first experiment. While training
was performed on a subset of flow data from the CIDDS001 OpenStack environment. This subset, called CIDDS-001
balanced, is created by keeping all the malicious flow fromthe CIDDS-001 OpenStack environment while reducing thenumber of benign flows to match its number. The benign flows
were sampled in a way that preserves the order of flows fromthe original data set.
C. Metrics
The performance of the proposed method is measured
using Accuracy, F1-score, Recall, and Precision, which arewidely utilized metrics in NIDS studies [6]. These metrics
are computed from TP (True Positive, i.e., malicious trafficclassified as malicious), TN (True Negative, i.e., benign trafficclassified as benign), FP (False Positive, i.e., benign trafficclassified as malicious), and FN (False Negative, i.e., malicious
traffic classified as benign) values. The first metric, accuracy,
is defined as the ratio of correctly classified flows to the total
number of flows.
Accuracy =TP + TNTP + TN + FP + FN

(1)

TABLE VI
E VALUATION RESULTS - T RAINING PERFORMED ON CIDDS-001
INTERNAL , TESTING PERFORMED ON CIDDS-001 EXTERNALClassifier
Proposal
EFC
DT
KNN
LinSVM
MLP
NB
AB
RF

TABLE VIIE VALUATION RESULTS - T RAINING PERFORMED ON CIDDS-001
INTERNAL , TESTING PERFORMED ON CIDDS-002

Train CIDDS-001 internal
Test CIDDS-001 external
Accuracy
F1-score
RecallPrecision
0.90780.9311
0.91200.95110.86590.90440.92780.88220.84910.88000.80880.96490.79780.82700.7067
0.99670.67840.69400.53330.99330.43800.3254
0.19820.90870.31610.00000.00000.99370.4606
0.38120.24300.88450.81770.85100.76130.9647

Precision is defined as the ratio of correctly classified malicious flows to all of the flows classified as malicious.
Precision =TP
TP + FPThe second metric, F1-score, is the harmonic mean of precision and recall. In other words, it is a statistical technique for
examining the accuracy of a system by considering both recall
and precision of the system.
Precision × Recall
Precision + Recall

TABLE VIIIE VALUATION RESULTS - T RAINING PERFORMED ON CIDDS-001
BALANCED , TESTING PERFORMED ON CIDDS-001 EXTERNAL(2)

Recall is defined as the ratio of correctly classified malicious
flows to all of the malicious flows. It is also referred to as
Detection Rate.
TP
Recall =(3)
TP + FN

F1-score = 2 ×Classifier
Proposal
EFC
DT
KNN
LinSVM
MLP
NB
AB
RF

(4)

D. Evaluation Results
We implemented a BERT model [17] with one layer, one
attention head, and a hidden size of 768. During training, we
use a batch size of 512 and a sequence length of 128 flows(equivalent to 65536 flows per iteration). Adam optimizer witha constant learning rate of 10−5 . In both experiments, we train
the BERT model with the MLM task for 400 iterations andthen fine-tune freeze the parameters of the BERT model andonly train the MLP classifier for 1100 iterations, finally, we
unfreeze the BERT model and trained both BERT and MLP
for 400 iterations. In total, the model was trained for 2000
iterations. During testing, the sequence length is increased to1024 flows.
Table VI and Table VII show the results of the first experiment, where training was performed on CIDDS-001 internal.
The proposed method presents the best performance in termsof accuracy (0.9078 and 0.9913) and F1-score (0.9311 and
0.8578) in both CIDDS-001 external and CIDDS-002 test sets.
EFC ranks second in the CIDDS-001 external test set both interms of accuracy (0.8659) and F1-score (0.9044). However,
EFC performs the worst in terms of accuracy (0.9084) and thethird worst in terms of the F1-score (0.3317) in the CIDDS002 test set.

Train CIDDS-001 internal
Test CIDDS-002
Accuracy
F1-score
RecallPrecision
0.99130.85780.7531
0.99620.90840.33170.65340.22230.98800.79480.66550.98640.98790.79240.66560.97890.95030.10420.08300.14000.98670.77220.64790.95540.96380.00060.00030.00720.98370.71480.58730.91310.98810.79610.66700.9871

Classifier
Proposal
EFC
DT
KNN
LinSVM
MLP
NB
AB
RF

Train CIDDS-001 balanced
Test CIDDS-001 external
Accuracy
F1-score
RecallPrecision
0.85810.90020.93580.86730.8566
0.89850.92780.87100.84960.88050.80980.96460.81280.84340.73740.98500.81230.86310.86500.86120.6671
0.69100.54410.94630.22790.02820.01640.10130.47420.42680.28620.83860.83570.86790.78930.9639

Table VIII and Table IX show the results of the second
experiment, where training was performed on CIDDS-001
balanced. The proposed method presents the best performancein terms of accuracy (0.8581) and F1-score (0.9002) for
CIDDS-001 external. For CIDDS-002, RF performs the bestin terms of accuracy (0.9875) and F1-score (0.7891). EFC,
again, exhibits strong performance in CIDDS-001 external
but underperformed in CIDDS-002. DT, KNN, LinSVM, and
MLP accuracy and F1-score on CIDDS-001 external improved

TABLE IXE VALUATION RESULTS - T RAINING PERFORMED ON CIDDS-001
BALANCED , TESTING PERFORMED ON CIDDS-002

Classifier
Proposal
EFC
DT
KNN
LinSVM
MLP
NB
AB
RF

Train CIDDS-001 balanced
Test CIDDS-002
Accuracy
F1-score
RecallPrecision
0.98700.7717
0.63150.99210.90880.33270.65340.22320.98740.78710.6676
0.9586
0.9874
0.78670.66980.9532
0.53880.02600.17670.01400.98650.78320.70210.88540.79080.00070.00210.00040.98170.71700.66530.77740.98750.78910.66920.9613

slightly when trained on a balanced data set compared toan imbalanced one with more benign flows, as CIDDS-001
external is slightly biased toward malicious flow. The same
effect is not observed on the CIDDS-002 test set due to it
being biased toward benign flow, which benefits the classifier
with a low false positive rate.
The extra benign flows present in the CIDDS-001 internal
data set compared to the CIDDS-001 balanced data set do
not have much effect on the performance of EFC, DT, KNN,
AB, and RF. However, they improved the performance of the
proposed method in terms of accuracy and F1 score. The
proposed method benefited the most when trained on the
CIDDS-001 internal data set. We hypothesize that the extra
benign flows provide more contextual information to the BERT
model, improving its ability to identify benign flows, while notskewing the classification result toward the class with more
samples.
In cases where the proposed model is trained with CIDDS001 internal (Table VI, VII), its Precision is much worse than
KNN, the best performing method. Our proposal detected agreater amount of anomalous flows, and as a result, increases
the number of normal flows flagged as malicious (false positives), which negatively impacts Precision. Whereas KNN
achieved better Precision by sacrificing the number of flows
flagged as malicious, and therefore, a lot of malicious flows
remained undetected (false negatives).
Overall, the proposed method, together with DT, RF, andKNN was able to maintain performance across two different
test sets, exhibiting good domain adaptation capability. The
results for DT, RF, and KNN reflect the good performance
reported in prior researches [7], [13], The proposed method
was able to outperform all other classifiers at both test data
sets when trained on CIDDS-001 internal, and CIDDS-001
external test sets when trained on CIDDS-001 balanced.
From the test result, we notice that while EFC performs
well in CIDDS-001 external (same data set used in EFC’s
original paper [5]), its F1-score for the CIDDS-002 test set(not used in EFC’s original paper) was significantly worse.
We ran an experiment with smaller balanced test sets (with10000 benign and 10000 malicious flows each) sampled from
CIDDS-002 [1]. We observed that in the smaller test setting,
EFC achieved results comparable to DT, RF, and KNN. We
found that EFC performed well in small and balanced test sets
(as reported by Camila et al.) but its performance degrades
significantly in a large imbalanced test set heavily skewered
toward benign flows (the CIDDS-002 test set).
VI. C ONCLUSION
We proposed an NIDS that utilizes a sequence of flows
as input and the BERT model. We theorize that because
conventional ML-based NIDSs only use information from asingle flow, they exhibit poor domain adaptation capability.
The training of our proposed model consists of two steps. In
the first step, we trained the BERT model to predict a flow
given its context using unlabeled data. In the second step, an
MLP classifier is added to the output of the BERT model andtrained on labeled data. Early experimental results on CIDDS001 and CIDDS-002 showed the proposed method achieving
consistent results across different domains.
In this study, we used a feedforward neural network toclassify benign and malicious flows, which requires labeled
flow to train the model. In the future, we plan to model the
distribution of benign flow sequences and use the deviations
from this distribution to detect anomalous flows.
R EFERENCES[1] L. G. Nguyen and K. Watabe, “Flow-based network intrusion detection
based on BERT masked language model,” ser. CoNEXT-SW ’22. New
York, NY, USA: Association for Computing Machinery, 2022.
[2] M. Ring, S. Wunderlich, D. Scheuring, D. Landes, and A. Hotho,“A survey of network-based intrusion detection data sets,” Computer
Security, vol. 86, pp. 147–167, 2019.
[3] H. Li, Z. Chen, R. Spolaor, Q. Yan, C. Zhao, and B. Yang, “DART: Detecting unseen malware variants using adaptation regularization transfer
learning,” in 2019 IEEE International Conference on Communications(ICC), 2019, pp. 1–6.
[4] M. Zolanvari, M. A. Teixeira, and R. Jain, “Effect of imbalanced datasetson security of industrial iot using machine learning,” in 2018 IEEE
International Conference on Intelligence and Security Informatics (ISI),
2018, pp. 112–117.
[5] C. F. T. Pontes, M. M. C. de Souza, J. J. C. Gondim, M. Bishop,and M. A. Marotta, “A new method for flow-based network intrusion
detection using the inverse potts model,” IEEE Transactions on Network
and Service Management, vol. 18, no. 2, pp. 1125–1136, 2021.
[6] Z. Ahmad, A. Shahid Khan, C. Wai Shiang, J. Abdullah, and F. Ahmad,“Network intrusion detection system: A systematic study of machinelearning and deep learning approaches,” Transactions on Emerging
Telecommunications Technologies, vol. 32, no. 1, Jan. 2021.
[7] N. Shone, T. N. Ngoc, V. D. Phai, and Q. Shi, “A deep learning approachto network intrusion detection,” IEEE Transactions on Emerging Topics
in Computational Intelligence, vol. 2, no. 1, pp. 41–50, 2018.
[8] B. Yan and G. Han, “Effective feature extraction via stacked sparseautoencoder to improve intrusion detection system,” IEEE Access, vol. 6,
pp. 41 238–41 248, 2018.
[9] M. Al-Qatf, Y. Lasheng, M. Al-Habib, and K. Al-Sabahi, “Deep learning
approach combining sparse autoencoder with svm for network intrusion
detection,” IEEE Access, vol. 6, pp. 52 843–52 856, 2018.
[10] G. Andresini, A. Appice, N. D. Mauro, C. Loglisci, and D. Malerba,“Multi-channel deep feature learning for intrusion detection,” IEEE
Access, vol. 8, pp. 53 346–53 359, 2020.
[11] F. A. Khan, A. Gumaei, A. Derhab, and A. Hussain, “A novel two-stage
deep learning model for efficient network intrusion detection,” IEEE
Access, vol. 7, pp. 30 373–30 385, 2019.
[12] N. Thapa, Z. Liu, D. B. KC, B. Gokaraju, and K. Roy, “Comparisonof machine learning and deep learning models for network intrusion
detection systems,” Future Internet, vol. 12, no. 10, 2020.
[13] A. Verma and V. Ranga, “Statistical analysis of CIDDS-001 dataset
for network intrusion detection systems using distance-based machinelearning,” Procedia Computer Science, vol. 125, pp. 709–716, Dec.
2017.
[14] R. Abdulhammed, M. Faezipour, A. Abuzneid, and A. AbuMallouh,“Deep and machine learning approaches for anomaly-based intrusion
detection of imbalanced network traffic,” IEEE Sensors Letters, vol. 3,
no. 1, pp. 1–4, 2019.
[15] M. Ring, S. Wunderlich, D. Gruedl, D. Landes, and A. Hotho, “Flowbased benchmark data sets for intrusion detection,” in Proceedings of the
16th European Conference on Cyber Warfare and Security (ECCWS).
ACPI, 2017, pp. 361–369.
[16] ——, “Creation of flow-based data sets for intrusion detection,” Information Warfare, vol. 16, 2017, to appear.
[17] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pretraining of deep bidirectional transformers for language understanding,”
in Proceedings of the 2019 Conference of the North American Chapterof the Association for Computational Linguistics: Human Language
Technologies, Volume 1 (Long and Short Papers).
Minneapolis,
Minnesota: Association for Computational Linguistics, Jun. 2019, pp.
4171–4186.