FLAG: Flow Representation Generator based on
Self-supervised Learning for Encrypted Traffic
Classification
Wenting Wei

Tianjie JuB

Han Liao

Xidian University

Xidian University
Shanghai Jiaotong University

Southwest University
Jianghan University

Weike Zhao

Huaxi Gu

Xidian University

Xidian University

ABSTRACTengineering to improve network performance and quality-ofservice. However, encryption procedure turns raw flows intoa pseudo-random-like format [12], which poses considerable
challenge for traffic classification.
Existing traffic classification methods can be divided into
three categories: rule-based methods, statistical learning
methods and deep learning (DL) based methods. With theincreasing adoption of dynamic assignments in new application, rule-based methods do not work well due to theirdependence on deterministic rules and expert knowledge[17][16]. Especially with the development of traffic encryption technologies, it is extremely difficult to construct rules
for encrypted traffic by domain knowledge. Traditional statistical learning methods rely heavily on statistical or timeseries features extracted by humans, but how to design aneffective traffic feature set is still an open question.
Inspired by recent improvement of artificial intelligence,
DL provides a way to traffic identification [2][20]. DL methods can extract high-level features automatically for trafficclassification by learning the raw input layer by layer, without constructing features based on domain knowledge [3].
With sufficient datasets and computing resources, DL-based
encrypted traffic classifiers employing common DL algorithms, such as SAE and CNN [1], can already achieve good
performance. However, as users’ awareness of privacy protection increases, it is always time-consuming to obtain sufficient data with ground-truth label for training DL models
in most network scenarios, resulting in a poor performanceand unexpected behaviors of some apps with the increase of
training time [19]. Thus, a robust DL-based classifier without the demand of large-quantity labeled data needs to be
designed urgently.
In this paper, FLAG is proposed to classify encrypted traffic. It can be divided into four modules: Data Processing,Phrase Expanding, Bidirectional Encoder and Finetuning.
Specifically, Raw flows in pcap format are transfered into
bytecodes during Data Processing. N-phrase Extraction isused in Phrase Expanding to reconstruct training samples in

Due to its excellent ability in learning features from large
scale raw data, deep learning (DL) has attracted much attention for encrypted traffic classification. However, mostDL-based traffic classifiers usually rely on enormous labeled
samples. Motivated by this, we investigate a self-supervised
traffic classifier (FLAG) without sacrifice of identification
accuracy, only depending on small labeled traffic samples
and highly available unlabeled traffic samples. Specifically,
focusing on local short-term characteristics of traffic, we
design a preprocessing algorithm, termed as N-phrase Extration, to convert unlabeled raw traffic dataset into sequencesof high-frequency phrases as input of Bidirectional Encoder.
On account of their significance, potential timing characteristics from input sequences are mined by Bidirectional
Encoder and embedded into robust representations with
distributed vectors to enhance classifier’s performance significantly. Our comprehensive experiments indicate FLAG
can achieve 98.65% in 100% of dataset and 98.07% in 10%of dataset in terms of true positive rate in UNB ISCX VPNnonVPN dataset, which are better than p-FP, FS-Net andDeep Packet.

1

INTRODUCTION

Network traffic classification is playing an increasingly significant role in network optimization applications. An accurate classifier for network traffic will contribute to traffic
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bear
this notice and the full citation on the first page. Copyrights for componentsof this work owned by others than ACM must be honored. Abstracting withcredit is permitted. To copy otherwise, or republish, to post on servers or toredistribute to lists, requires prior specific permission and/or a fee. Request
permissions from permissions@acm.org.
APNet 2021, June 24–25, 2021, Shenzhen, China, China© 2021 Association for Computing Machinery.
ACM ISBN 978-1-4503-8587-9/21/06. . . $15.00
https://doi.org/10.1145/3469393.3469394
1

APNet 2021, June 24–25, 2021, Shenzhen, China, China

Wenting Wei, Tianjie Ju, Han Liao and Weike Zhao, Huaxi Gu

bytecodes and extract meaningful phrases as new featuresfrom unlabeled dataset. It traverses unlabeled flows to countfrequency of each N-phrase (N = 1, 2, 3) and convert raw
flows using Greedy Algorithm. Then, BERT [4] is used in
Bidirectional Encoder to embed traffic data into dense vector
form for pre-training, in which Transformer [18] Encoder
digs deep and robust sequence features. In addition, dynamic
masking technology [10] is applied to improve Bidirectional
Encoder, thereby enhancing the performance of trained vector. Finally, softmax layer is added in Finetuning module tofine-tune the whole model for supervised learning, so thatthe whole model can be applied to our classification task.
Compared to previous work, small labeled samples andhighly available unlabeled samples are used for training withsignificant improvements in performance. We summarize
our contributions as follows.
(1) FLAG is an end-to-end traffic classification solution forencrypted traffic, which independently learns vital featuresand contextual information from traffic, without requiringexpert knowledge for feature extraction.
(2) Self-supervised learning is used to extract informative
phrases and generate robust and high-quality flow representations, so that FLAG can achieve high accuracy with smalllabeled samples and sufficient unlabeled samples.
(3) Compared with other work, we carried out encrypted
traffic classification work on UNB ISCX VPN-nonVPN
dataset and achieved breakthroughs in various indicators
regardless of whether labeled data is sufficient.
The rest of the paper is organized as follows. Section 2 describes the preliminaries. Section 3 introduces the model we
proposed, namely FLAG. Section 4 conducts comprehensive
experiments between FLAG and current models. We havecarried out detailed experiments and analysis on various
indicators. Section 5 summarizes the paper and introduces
possible future work directions.

2

Our proposal is first pretrained on a large number of unlabeled flows and subsequently fine-tuned using labeled flows.

2.2

The Transformer network[18] is one of the most popular neural networks to model sequences, which consists of Encoder
and Decoder. Transformer uses the attention mechanism
to replace LSTM [5] and Positional Encoding to solve theword order problem. It has made huge breakthroughs in Natural Language Processing (NLP). Transformer is an Encoderand-Decoder architecture. The encoder consists of N = 6
individual layers. Each layer has two sub-layers, one is multihead mechanism layer and another is position-wise fully connected feed-forward layer. Eesidual connection and layer normalization are connected between each two sub-layers. Thatis, the output of sub-layers is LayerNorm(x + SubLayer(x)).
The decoder also consists of N = 6 individual layers. However, there is a third sub-layer in decoder which performs
multi-head attention over the output of the encoder stack.

2.3

Attention Mechanism

A simple encoder-and-decoder model won’t focus on different features when predicting results, which means eachfeature has the same contribution. Attention[18] mechanism
can focus on some important raw features in order to generate high-quality sequences.
The Attention mechanism is essentially an addressing
process. Given a task-related "Query" vector, it calculates
the Attention Value by calculating the attention distributionof vector "Key" and attaching it to vector "Value". Actually,an attention function is a set of key-value pairs. To specific,
the attention function of Transformer network is "MultiHead Attention", which contains several attention heads toallow each head to focus on different features. The formulaof multi-head attention mechanism is as follows:
Q

PRELIMINARIES

Q i = QWi , Ki = KWiK , Vi = VWiV , i = 1, ..., nIn this section, we first define the traffic classification problem we need to solve. Then we briefly introduce the Transformer framework and self-attention mechanism.

2.1

Transformer Network

Q KT

headi = so f tmax( √i i )Vi , i = 1, ..., n(2)MultiHead(Q, K, V ) = Concat(head 1 , ..., headn )W 0

(3)

dk

Problem Definition(1)

where n denotes the number of heads, W is the weight vectorof training models, and Q, K and V are the vector "Query","Key" and "Value" respectively.

Traffic classification is mainly to identify different categoriesof network traffic. Assume that there are N samples and C
classes in our dataset. The k t h sample can be represented
as x k = {l 1(k ) , l 2(k ) , ..., ln(kk) }, where nk is the length of the k thsequence and lp(k ) is the packet value in time step i. We aregoing to build an end-to-end model ψ (xp ) to predict a label Âk that is exactly the real label Ap . Our dataset can be
represented as D = {(x 1 , A1 ), (x 2 , A2 ), ..., (x N , A N )}.

3

PROPOSED METHOD

In this section, we will introduce our proposed end-to-end
model FLAG in detail. The architecture of FLAG can be seenin Figure 1, which can be divided into four phases.
2

FLAG: Flow Representation Generator based on Self-supervised Learning for Encrypted Traffic
Classification

APNet 2021, June 24–25, 2021, Shenzhen, China, China

Figure 1: The architecture of FLAG.

3.1

m is a parameter to control the length of our dictionary. To
specific, our task is to discover m informative phrases tofill in our dictionary. With such a dictionary, we can build
a mapping which map raw flows to sequences composedof n-phrases. By traversing the unlabeled flow in unlabeleddataset, the m phrases with the highest frequency can be
added to the dictionary. Each phrase in the dictionary has
an index.
For example, the raw sequence of x k can be represented as(k ) (k )x k = [l 1(k ) , l 2(k) , ..., li(k) , li+1
, li+2 , ..., ln(kk) ]. We use the greedy algorithm to determine whether it is a phrase in the dictionary
from left to right. If 3-phrase sequence дt = [l 1(k ) , l 2(k ) , l 3(k ) ]is in the dictionary, then the raw sequence is transfered toyk = [дt , l 4(k) , l 5(k ) , ..., ln(kk) ]. If not, we will check 2-phrase and1-phrase sequence sequentially. The priority of each phrase
is 3 − phrase > 2 − phrase > 1 − phrase. The detailed stepsof proposed N-phrase Extraction are shown in Algorithm 1
Finally, N-phrase Extraction is selected for this paper. Compared with bytecodes, N-phrase Extraction takes into account some important 2-phrase sequences and 3-phrase sequences, which considers more effective contextual information and makes the converted sequences more like sentencesin natural language.

Data Processing

The dataset we use (ISCXVPN2016) [7] is captured at datalink layer. It consists of labeled network traffic, including full
packets in pcap format. Considering that data preprocessing
is not the focus of our paper, our preprocessing method issimilar to (Mohammad et al. 2020) [12] and we make certain
adjustments. The specific process is as follows.
1. Ethernet headers are removed since they are useless forclassification task.
2. Zeros are injected to the end of UDP segment’s headers
to make them equal length with TCP headers.
3. IP is masked and irrelevent packets are removed.
4. Raw packets are converted into bytes. Unlike (Mohammad et al. 2020)[12], we do not limit the length of bytecodes.

3.2

Phrase Expanding

For Bidirectional Encoder, we need to generate a phrase
dictionary W corresponding to the flows. A simple method
is to use bytecodes in raw flows as a dictionary. However,this dictionary only has 256 kinds of phrases. With such asmall number of phrases, it is difficult for our Bidirectional
Encoder to learn enough useful representations. Therefore,
N-phrase Extraction Algorithm is proposed to expand thephrase dictionary.
N-phrase Extraction can extract informative byte phrases
throuth unlabeled dataset. It is used to convert raw flows intosequences composed of n-phrases (n = 1, 2, 3) where n is thelength of the phrase in bytes. Consider the whole dataset,the frequency of a phrase дt is f (дt ). A phrase dictionary
D = {д1 , д2 , ..., дm } contains all of these n-phrases, where

3.3

Bidirectional Encoder

Embedding is an important concept in Natual Language Processing. It is actually a function which maps original vector
space into a dense and low-dimension (semantic) space. We
use Bidirectional Encoder [4] to better embed flows. Our
encoder is a stack of Transformer encoder layers [18] which
3

APNet 2021, June 24–25, 2021, Shenzhen, China, China

Wenting Wei, Tianjie Ju, Han Liao and Weike Zhao, Huaxi Gu

Algorithm 1 N-phrase Extraction

epoch, training data is duplicated 10 times so that each sequence is masked in 10 different ways over the 40 epochs oftraining.
After masking, the pre-training model will try to predictthe phrase that is masked or replaced. It compares predictedvalue with true value to calculate the loss of the model, anduses backpropagation method to update the weights of themodel. Finally, thanks to the attention mechanism, the vector[CLS] can learn a robust representation of the entire flow.
We can apply the vector [CLS] to the next phase.

Input: Origin flow set X = [x 1 , x 2 , ..., x i , ..., x N ].
Output: Transfered flow set Y = [y1 , y2 , ..., yi , ..., y N ].
Initialize the phrase dictionary D.
// Calculate the frequency of each phrase.
for i = 1 to N dofor j = 1 to nk − 2 dof [l (i ) ] ← f [l (i ) ] + 1
jjf [l (i ),l (i ) ] ← f [l (i ),l (i ) ] + 1
j

j+1

j

j+1 j+2

jj+1

f [l (i ),l (i ) ,l (i ) ] ← f [l (i ),l (i ) ,l (i ) ] + 1j

j+1 j+2

3.4

end forend for// Generate the phrase dictionary D = {д1 , д2 , ..., дm }.
D ← reverseSort(D)[1 : m]// Transfer X to Y according to the dictionary D.
for i = 1 to N doj ← 0, yi ← [] // yi is a list to store transfered flow.
while j < lenдth(x i ) − 2 do// Using Greedy Algorithm, the priority is 3 −phrase > 2 − phrase > 1 − phrase.
(i) (i)if [l j(i) , l j+1, l j+2 ] ∈ D thenClassification Layer

After Bidirectional Encoder, the vector [CLS] is feed into
the classification layer for supervised learning. The previous
phases have enabled the model to learn enough importantcharacteristics of raw flows. Therefore, in the Classification
Layer, we do not use complex neural networks (such as RNN[21]) but softmax layer to fine-tune our model, which can
avoid the risk of overfitting and reduce the complexity of ourmodel. The softmax function is used to generate the label
distribution of i th sample:
exp(x i )P(x i ) = ÍN(i = 1, ..., N )j=1 exp(x j )(i) (i)yi ← yi + [l j(i) , l j+1, l j+2 ], j ← j + 3
else(i)if [l j(i) , l j+1] ∈ D then(4)

where N is the total number of classes. We can use the outputof the softmax layer to represent the probability of encrypted
traffic belonging to each category and compare it with thetrue value. Then cross entropy function is used to calculate
loss and backpropagate it to update weights.

(i)yi ← yi + [l j(i) , l j+1], j ← j + 2

else
yi ← yi + [l j(i) ], j ← j + 1
end ifend ifend while
end for

4

EVALUATIONIn this section, we have a brief introduction on our experimental settings. Then, we present our experiment comparison results to evaluate the efficiency of our proposed model.

4.1
consists of multiple self-attention "heads" [14]. We use aTransformer architecture with 12 layers. Each block uses 6
self-attention heads and 768 hidden dimensions.
We feed the processed unlabeled flows into Bidirectional
Encoder with special tokens: {[CLS], д1 , д2 , ..., дn , [EOS]}
where n is a parameter that controls the maximum sequencelength during training, [CLS] is the representation of thissequence and [EOS] marks the end of the sequence.
Bidirectional Encoder uses dynamic masking technology[10] as its training task. Dynamic masking technology makes
our encoder learn more from different masking scheme to enrich encoder’s represent ablity. A random sample of phrases
in the input sequence is selected to replace with a special
token [MASK], another phrase or the phrase itself. To avoid
using the same mask for each training instance in every

Experimental Settings

In this paper, we utilize a representative dataset of real-world
traffic in ISCX which is rich enough in diversity and quantity[8]. According to (Mohammad, 2020)[12], it is divided into
12 traffic categories. They are Chat, Email, File Transfer,
Streaming, Torrent, VoIP, VPN: Chat, VPN: Email, VPN: File
Transfer, VPN: Streaming, VPN: Torrent and VPN: VoIP.
We test our FLAG using Python 3.7 and implement with
TensorFlow. All of simulations were carried out on an Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz processor with
64 GB of memory. As for Bidirectional Encoder, we use aTransformer architecture with 6 layers. Each block consistsof 6 self-attention heads and 768 hidden dimensions.
To better evaluate performance of our proposed model,
True Positive Rate (TPR) and False Positive Rate (FPR) areused as indicators referring to [15][12]. TPR stands for the4

FLAG: Flow Representation Generator based on Self-supervised Learning for Encrypted Traffic
Classification

APNet 2021, June 24–25, 2021, Shenzhen, China, China

percentage of the flows that are rightly classified among total
flows. FPR is the percentage of the flows that are wrongly
classified among total flows. Fractional combination of TPR
and FPR (FTF) shows the overall accuracy. These metrics aredescribed mathematically as follows.
C

T PR AV E =1 ÕT PRi ∗ FlNi ,N i=0

F PR AV E =1 ÕF PRi ∗ FlNi ,
N i=0

(5)

C

FT F =

C
Õ

wi

i=0

T PRi
,
1 + F PRi

(6)

(7)

FlNi
,
(8)
N
where FlN i is the number of the i t h flows class and w i is theradio between FlN i and the total flows number N .
wi =

4.2

Figure 2: Comparison of Bidirectional Encoder using
N-phrase Extraction and raw flows.

Results Comparison

Consider that most traffic in reality is unlabeled, 40% of thedataset has the label removed for self-supervised learningand 60% of the dataset for supervised learning. Labeled data
can be divided into training set, validation set and test set
according to the ratio of 6:2:2. To prevent overfitting during
training, the model with the highest accuracy on verification
set is selected, so as to ensure that the test set would not betouched during training process. Finally, results are shownin the test set.
Several works had been conducted on encrypted trafficclassification. We compare our model with FS-Net [9], Deep
packet [12] and p-FP [13], where Deep packet employed two
deep neural network structures, namely stacked autoencoder(SAE) and convolution neural network (CNN) and p-FP useda basic algorithm, multi-Layer perceptron (MLP) classifier.
In our work, we adopt 1) CBOW [11] and Text-CNN [6] and2) FLAG which is expounded detailedly in section 3.
The main reason for the excellent performance of current
deep learning algorithms is that they utilize large amountsof labeled data for training. Therefore, in order to verify
the performance of each algorithm in the case of sufficient
or insufficient labeled data, we conducted two groups ofcontrolled experiments. The first experiment trains the wholelabeled data (60% of the total dataset). And for another, we
randomly sample 10% of our labeled data, which is 6% of thetotal dataset. Comparison results are listed in Table 1.
N-phrase Extract algorithm is innovatively utilized in
CBOW and FLAG by converting raw flows into semantic
sequences before pretraining. To investigate its effects on
encrypted traffic classification, we conduct an ablation study,(a) FLAG(b) FS-Net

Figure 3: Normalized confusion matrices using (a)FLAG (b) FS-net model on 10% labeled dataset.
which use the same model except the use of N-phrase Extract algorithm. Fig.2 shows the ablation experiment comparision results. Experiments show that N-phrase Extraction canslightly improve the overall performance of FLAG (98.65%
with N-phrase Extraction in TPR and 98.53% with bytecodes
in TPR), and has obvious advantages in identifying categories
with less data such as Chat and Email.
To conduct a further study on FLAG performance, normalized confusion matrixs are made to compare with FS-net on
10% labeled dataset, which is shown in Fig. 3(a) - Fig. 3(b).
Finally, we further analyze the inference delay of FLAG.
In our experiment (a NVIDIA GeForce GTX 1080 Ti), it takes
323.37s to infer 1% datasets (5218 flows in total) on FLAG,
It means that each flow takes an averge of 0.062s, which isacceptable as the inference delay.
5

APNet 2021, June 24–25, 2021, Shenzhen, China, China

Wenting Wei, Tianjie Ju, Han Liao and Weike Zhao, Huaxi Gu

Table 1: Evaluation Result
FS-NetFPR

Deep Packet: SAE
TPR
FPR

Chat
Email
File Transfer
Streaming
Torrent
VoIP
VPN: Chat
VPN: Email
VPN: File Transfer
VPN: Streaming
VPN: Torrent
VPN: VoIP
Total
FTF

0.27190.00030.36780.0016
0.95880.00650.95140.00040.97040.00040.99020.0371
0.53290.00020.71140.00000.96340.00120.98800.00000.98350.0001
0.99680.00330.96480.0032
0.94960.72030.0016
0.58410.0004
0.97690.00540.98050.00030.96350.00000.98860.01920.62520.00030.63490.00000.98610.00120.98140.00000.99760.00030.99750.0021
0.97820.00200.9692Chat
Email
File Transfer
Streaming
Torrent
Voip
VPN: Chat
VPN: Email
VPN: File Transfer
VPN: Streaming
VPN: Torrent
VPN: Voip
Total
FTF

0.45190.00120.47930.00210.96070.00320.98150.0006
0.95520.00000.99260.03270.69040.00000.91240.00000.98860.00100.99320.00030.99700.00000.99670.00100.97050.00270.95830.53120.00210.29620.00040.96990.00790.96010.00070.95640.00020.98710.02870.68770.00120.50510.00010.93270.0012
0.99420.0004
0.97430.00020.98660.0016
0.96890.00280.9560TPR

Deep Packet: CNN
p-FP
TPR
FPR
TPR
FPR
Data volume: 100%0.74580.00090.57670.00050.60880.00030.68040.00130.97550.00420.97010.00190.99330.0001
0.98170.00020.98420.00000.97020.0002
0.99310.02070.99570.02500.96600.00020.81890.0001
0.98330.00000.92700.0001
0.99200.00010.97930.00080.99830.0001
0.99490.00000.99910.00000.99390.00030.99820.00020.99650.0011
0.98210.00160.97800.00200.9734
0.9688
Data volume: 10%
0.58460.00210.47060.00110.46920.00090.49380.0011
0.96370.00430.96390.0031
0.97300.00040.96460.00040.96730.0001
0.97320.00030.98940.0312
0.99230.03220.78180.0006
0.64650.00070.81040.00010.64930.00020.96580.00070.96420.00150.99320.00030.99370.00000.99840.00010.98530.0004
0.99290.00100.99080.00180.97110.00260.97010.00270.95900.9579Among these experiments, we can draw following conclusions from the results.
Firstly, N-phrase Extraction (used in CBOW and FLAG)
can indeed generate important representations in raw flows.
Ablation experiments indicate that N-phrase Extraction isan effective phrase expanding algorithm which considers
contextual information of each byte, so that the sequence
after phrase expansion has stronger timing characteristics
and semantic richness. Secondly, FLAG always outperforms
FS-Net with about 0.5% - 1.5% improvement and CBOW with
about 0.4% improvement in TPR, which proves that Bidirectional Encoder can better obtain sequence representation.
Finally, It is not difficult to find from the normalized confusion matrices that FLAG’s ability to recognize "Chat", "Email"and "VPN Chat" is significantly better than FS-Net. However,
FLAG still often misjudges "Chat" and "Email" as "VoIP".

5

CBOW
TPR
FPR

TPR

FLAG

0.79040.00050.67380.0002
0.97200.00130.99620.00000.98960.00000.99770.02300.98380.00000.98590.00000.99740.00010.99930.00000.99980.00000.99940.00000.98320.00150.97510.85930.00050.77960.00020.97810.00230.99890.00000.99320.00000.99540.01730.99180.00000.99530.00000.99900.00010.99980.00001.00000.00000.99990.00010.98650.00120.97970.74190.00240.21980.0001
0.96260.00080.98910.00000.98710.0001
0.99810.03020.96960.00030.98100.00010.98820.00020.99590.00000.99940.00000.99880.0002
0.97620.00220.96590.77910.00090.66420.00050.97130.00430.99230.00020.98240.0001
0.99160.0226
0.97920.00010.99050.00000.99720.0001
0.99710.00000.99970.00000.99900.00020.98070.00000.9713

FPR

Thanks to pre-training with unlabeled samples, FLAG can
reduce the overhead of labeling data while ensuring highclassification performances. N-phrase Extraction is design toextract phrases with more contextual information than bytes
from raw traffic data. In addition, Bidirectional Encoder isused to generate robust and high-quality vectors for finetuning. Experiments showed that FLAG can always achieve
the best performance compared to several current models,
regardless of the lack of labeled data.
In the future, more comprehensive ablation experiments
will be conducted to prove each component (N-phrase Extraction, Bidirectional Encoder, etc.) contributes to the successof FLAG while the delta between FLAG and other type of
methods like rule-based methods will be studied in detail. In
addition, we will focus on proving the importance of eachcomponent theoretically, not just intuitively.

REFERENCES

CONCLUSION[1] Giuseppe Aceto, Domenico Ciuonzo, Antonio Montieri, and Antonio
Pescapé. 2018. Mobile Encrypted Traffic Classification Using Deep
Learning. In IEEE/ACM Network Traffic Measurement and Analysis

In this paper, a self-supervised encrypted traffic classifier,termed as FLAG, is proposed in the small labeled sample case.
6

FLAG: Flow Representation Generator based on Self-supervised Learning for Encrypted Traffic
Classification

APNet 2021, June 24–25, 2021, Shenzhen, China, China[20] WangXin, ChenShuhui, and SuJinshu. 2020. Real Network Traffic
Collection and Deep Learning for Mobile App Identification. WirelessCommunications and Mobile Computing (2020).
[21] Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent
Neural Network Regularization. Eprint Arxiv (2014).

Conference (TMA’18).
[2] G. Aceto, D. Ciuonzo, A. Montieri, and A Pescapé. 2020. Toward Effective Mobile Encrypted Traffic Classification through Deep Learning.
Neurocomputing 409 (2020).
[3] G. Aceto, D. Ciuonzo, A. Montieri, and A Pescapé. 2021. DISTILLER:
Encrypted traffic classification via multimodal multitask deep learning.
Journal of Network and Computer Applications (2021).
[4] Jacob Devlin, Ming Wei Chang, Kenton Lee, and Kristina Toutanova.
2018. BERT: Pre-training of Deep Bidirectional Transformers for
Language Understanding. (2018).
[5] S Hochreiter and J Schmidhuber. 1997. Long Short-Term Memory.
Neural Computation 9, 8 (1997), 1735–1780.
[6] Yoon Kim. 2014. Convolutional Neural Networks for Sentence Classification. Eprint Arxiv (2014).
[7] Arash Habibi Lashkari, Gerard Draper-Gil, Mohammad Saiful Islam
Mamun, and Ali A. Ghorbani. 2016. Characterization of Encrypted
and VPN Traffic Using Time-Related Features. In The International
Conference on Information Systems Security and Privacy (ICISSP).
[8] Arash Habibi Lashkari, Gerard Draper-Gil, Mohammad Saiful Islam
Mamun, and Ali A Ghorbani. 2017. Characterization of tor traffic using
time based features.. In ICISSp. 253–262.
[9] C. Liu, L. He, G. Xiong, Z. Cao, and Z. Li. 2019. FS-Net: A Flow Sequence
Network For Encrypted Traffic Classification. In IEEE INFOCOM 2019
- IEEE Conference on Computer Communications. 1171–1179. https://doi.org/10.1109/INFOCOM.2019.8737507[10] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi
Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.
2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach.
(2019).
[11] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. Computer
Science (2013).
[12] Mohammad, Lotfollahi, Mahdi, Jafari, Siavoshani, Ramin, Shirali, Hossein, Zade, and Mohammdsadegh and. 2020. Deep packet: a novel
approach for encrypted traffic classification using deep learning. Soft
Computing 24, 3 (2020), 1999–2012.
[13] Se Eun Oh, Saikrishna Sunkam, and Nicholas Hopper. 2019. p1-fp:
Extraction, classification, and prediction of website fingerprints withdeep learning. Proceedings on Privacy Enhancing Technologies 2019, 3(2019), 191–209.
[14] Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. A Primer in
BERTology: What We Know About How BERT Works. Transactions of
the Association for Computational Linguistics 8 (2020), 842–866. https://doi.org/10.1162/tacl_a_00349[15] M. Shen, M. Wei, L. Zhu, and M. Wang. 2017. Classification of Encrypted Traffic With Second-Order Markov Chains and Application
Attribute Bigrams. IEEE Transactions on Information Forensics and Security 12, 8 (2017), 1830–1843. https://doi.org/10.1109/TIFS.2017.2692682[16] Hongtao Shi, Hongping Li, Dan Zhang, Chaqiu Cheng, and Xuanxuan
Cao. 2018. An Efficient Feature Generation Approach based on Deep
Learning and Feature Selection Techniques for traffic classification.
Computer Networks 132, FEB.26 (2018), 81–98.
[17] Hamid Tahaei, Firdaus Afifi, Adeleh Asemi, Faiz Zaki, and Nor Badrul
Anuar. 2020. The rise of traffic classification in IoT networks:: A survey.
Journal of Network and Computer Applications 154 (2020), 102538.
[18] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion
Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. arXiv preprint arXiv:1706.03762 (2017).
[19] Q. Wang, A. Yahyavi, B. Kemme, and W. He. 2015. I know what you didon your smartphone: Inferring app usage over encrypted data traffic. In
2015 IEEE Conference on Communications and Network Security (CNS).
7