# SAD: Website Fingerprinting Defense Based on Adversarial Examples

Authors: Renzhi Tang; Guowei Shen; Chun Guo; Yunhe Cui
Journal: Security and Communication Networks
 (2022)

---

## Abstract

Website fingerprinting (WF) attacks can infer website names from encrypted network traffic when the victim is browsing the website. Inherent defenses of anonymous communication systems such as The Onion Router(Tor) cannot compete with current WF attacks. The state-of-the-art attack based on deep learning can gain over 98% accuracy in Tor. Most of the defenses have excellent defensive capabilities, but it will bring a relatively high bandwidth overhead, which will seriously affect the user’s network experience. And some defense methods have less impact on the latest website fingerprinting attacks. Defense-based adversarial examples have excellent defense capabilities and low bandwidth overhead, but they need to get the complete website traffic to generate defense data, which is obviously impractical. In this article, based on adversarial examples, we propose segmented adversary defense (SAD) for deep learning-based WF attacks. In SAD, sequence data are divided into multiple segments to ensure that SAD is feasible in real scenarios. Then, the adversarial examples for each segment of data can be generated by SAD. Finally, dummy packets are inserted after each segment original data. We also found that setting different head rates, that is, end points for the segments, will get better results. Experimentally, our results show that SAD can effectively reduce the accuracy of WF attacks. The technique drops the accuracy of the state-of-the-art attack hardened from 96% to 3% while incurring only 40% bandwidth overhead. Compared with the existing proposed defense named Deep Fingerprinting Defender (DFD), the defense effect of SAD is better under the same bandwidth overhead.


---

## Body

## 1. Introduction

Anonymous communication system can protect the users’ information through data encryption and multi-hop proxy. The Onion Router (Tor) [1], a kind of anonymous communication software, is widely used worldwide. However, inherent defense of Tor is difficult against WF attacks. In WF attacks, encrypted traffic is collected by the WF attacker between the client and guard node of Tor. Then, the website name can be inferred from the traffic obtained by attackers. It is the goal of the defense model that reduces the accuracy of WF attacks Many hackers carry out network attacks through Tor. We need some tools or technologies to take computer forensics for these illegal acts [2]. WF attack is also a computer forensics technology for Tor.In the past, the attacker extracts a variety of packets level features from encrypted traffic, such as number, order, direction, and unique length, and then uses machine learning for classification [3–5]. In recent years, deep learning has been recognized in various fields such as image classification [6] and speech recognition [7]. Similarly, it is widely used in the field of computer security [8]. Different from traditional machine learning, automatic extraction of features from original data is one of the advantages of deep learning. WF attacks based on deep learning have been proposed such as Automated Website Fingerprinting (AWF) [9], Var-CNN [10], and Deep Fingerprinting (DF) [11]. DF can gain more than 98% accuracy only using the direction sequence of traffic.In the meantime, some WF defenses have been proposed such as WTF-PAD [12], walkie-talkie (W-T) [13], and Deep Fingerprinting Defender (DFD) [14]. In addition, researchers have found that deep learning models are vulnerable to adversarial examples [15]. It can make a classifier based on deep learning misclassification [1]. Therefore, some WF defenses based on adversarial examples are proposed such as WF-GAN [16] and Mockingbird [17]. But, WTF-PAD, W-T, and DFD have expired defense capability and suboptimal bandwidth overhead. WF-GAN and Mockingbird are the most recently proposed WF defense, which is based on adversarial examples. They have excellent defensive capabilities and low bandwidth overhead. But it can only use the complete traffic of the website to generate the corresponding defense data. The situation is unrealistic.In order to achieve excellent defensive performance, low bandwidth overhead, and availability in real-time traffic, we propose SAD based on adversarial examples and head rate. SAD consists of two components: an adversarial examples generation model and a WF attack model-based deep learning. The former can generate adversarial examples according to the direction sequence of flow. The attack model is an adversary of the generated model similar to GAN [18]. The experiments show that SAD effectively reduces the attack success accuracy of DF, Var-CNN, and AWF attack models with slight bandwidth overhead. SAD has excellent defense capabilities whether facing unknown attack or adversarial training. The main contributions of this article are as follows:(1)
This article proposes a WF defense, named SAD, using segmented processing with head rate and adversarial examples. Compared with some previous studies, SAD does not require global traffic sequence information when generating adversarial examples. Therefore, it can apply in live traffic.(2)
SAD can be easily generated adversarial examples for different attack models based on deep learning. The input and output of SAD are the original direction sequence data rather than burst-based data. This is the same data format required by the attack model.(3)
SAD can effectively reduce the attack success rate, even facing adversarial training or unknown attack. In particular, we obtain better results with head rate at low bandwidth overhead. For the three existing WF attack models, DF, Var-CNN, and AWF, SAD can reduce these attack success rates from 96% to 1%. Compared with DFD, we have better defense whether in open-world or close-world.


## 2. Related Work

### 2.1. Website Fingerprinting Attack

WF attacks initially required manual feature extraction. Instead, deep learning models based convolutional neural network (CNN) are now mainly used, as it only requires packet direction information. Table1 shows simple information about website fingerprinting attacks, which are recognized by researchers.Table 1
WF attacks based on traditional machine learning or deep learning.
AttackClassifierFeaturesWeaknessK-NN [3]K-nearest neighborsThe number of incoming packets, the number of outgoing packets, the sum of incoming packet sizes, the sum of outgoing packet sizes, etc.Manual feature extraction from network traffic.CUMUL [4]SVMK-FP [5]Random forestsAWF [9]CNNDirection sequence of Tor cellsRequires large amounts of training data.Var-CNN [10]DF [11]Herrmann et al. [19] first applied the WF attack on Tor in 2009, but the accuracy was only 3% in close-world. In 2011, Panchenko et al. [20] proposed a WF attack based on support vector machines (SVMs). It improves the accuracy of WF attacks. But, the computational cost of these WF attack classifiers is too high and not practical. With the development of computers, researchers have proposed more effective attacks. In 2014, the K-nearest neighbors (K-NN) classifier was proposed by Wang et al. [3]. The attack achieved a higher accuracy with shorter training time and test time. In addition, Panchenko et al. [4] used the cumulative sum of packet lengths as features and SVM as a classifier, called CUMUL, which achieved good results in close-world. Hayes et al. [5] proposed a new website fingerprinting attack K-fingerprinting (K-FP) based on random forest. K-FP achieved 91% accuracy in a closed-world setting. These methods are all based on traditional machine learning. What they have in common is that they require manual feature extraction, which is directly related to the accuracy rate. The previous work was more on how to extract more representative features.With the successful application of deep learning in other fields, many researchers are using it for website fingerprinting attacks. The obvious benefit is that it does not require manual feature extraction. To skip the process that manually extracting features, Rimmer et al. [9] proposed a deep learning-based WF attack AWF. They collected a lot of training data and tried to use three different neural network structures, Stacked Denoising Autoencoder (SDAE), CNN, and Long Short-Term Memory (LSTM). The experimental results show that the model-based CNN can obtain the highest accuracy of over 96%. Sanjit [10] proposed Var-CNN based on ResNets [21] deep learning. It has a better recognition effect when the amount of training data is small, especially in close-world. Sirinam et al. [11] proposed DF, which is based on CNN. The model has reached a high level of attack ability in both close-world and open-world. The above three deep learning-based models for WF attacks require a very simple form of data (packet direction), but it requires more training data compared to traditional machine learning.


### 2.2. Website Fingerprinting Defense

WF defense against WF attacks by inserting dummy packets. The approach brings bandwidth overhead. Tor’s bandwidth resources are precious. Researchers have been focusing on reducing bandwidth overhead and improving defensive capabilities. Different methods have different insertion rules, but they can be broadly sourced in two ways: generated by the authors based on their own expertise, and utilized to generate insertion rules from attack methods, as shown in Table2.Table 2
Differences in the rules for inserting dummy packets for WF defense.
DefenseDefensive RulesWeaknessBuFLO [22]Authors’ predetermined insertion rules, such as inserting dummy packets regularly.Large bandwidth overhead or poor defense against current WF attacks.Tamaraw [23]CS-BuFLO [24]WTF-PAD [3]W-T [13]DFD [14]WF-GAN [16]Insertion rules learned against website fingerprinting attacksDifficult to apply to practical scenarios. But it has low bandwidth overheadMockingbird [17]Buffered Fixed-Length Obfuscation (BuFLO) [22] sends packets of a fixed size at fixed intervals, using dummy packets to both fill in and (potentially) extend the transmission. But, it brings unreasonable bandwidth overhead. Tamaraw [23] and CS-BuFLO [24] extend BuFLO. The difference is that the download and upload rates can be set to different rates. But their bandwidth overhead exceeds 100%. WTF-PAD was proposed by Juarez et al. [3]. The defense attempts to fill a large time gap between two data packets. Whenever the gap is large, WTF-PAD will send a series of dummy packets. It reduces the accuracy of K-NN attacks. However, WTF-PAD was defeated by the WF attack DF. Walkie-talkie (W-T) [13] makes the attack success rate of DF only 49.7% with 31% bandwidth overhead. W-T changed the communication mode of both parties from full duplex to half duplex. The change affects the user’s network experience. For WF attacks based on deep learning, some researchers proposed WF defense based on adversarial examples, WF-GAN [16] and Mockingbird [17]. They have excellent performance, but they are difficult to be applied in live traffic due to the requirements of model input data.The latest defense method (DFD) [14] injects a dummy burst sequence according to the previous burst sequence in the real-time traffic. When the injection method is the server-side injection method, the attack success rate using CNN as the attack model is reduced from 99.93% to 5% with 85.56% bandwidth overhead. However, it did not test the existing WF attacks. In addition, the DFD cannot stably insert dummy packets. In some cases, the number of inserted dummy packets is small. When bandwidth overhead is small, the expected defense effect cannot be achieved.


## 2.1. Website Fingerprinting Attack

WF attacks initially required manual feature extraction. Instead, deep learning models based convolutional neural network (CNN) are now mainly used, as it only requires packet direction information. Table1 shows simple information about website fingerprinting attacks, which are recognized by researchers.Table 1
WF attacks based on traditional machine learning or deep learning.
AttackClassifierFeaturesWeaknessK-NN [3]K-nearest neighborsThe number of incoming packets, the number of outgoing packets, the sum of incoming packet sizes, the sum of outgoing packet sizes, etc.Manual feature extraction from network traffic.CUMUL [4]SVMK-FP [5]Random forestsAWF [9]CNNDirection sequence of Tor cellsRequires large amounts of training data.Var-CNN [10]DF [11]Herrmann et al. [19] first applied the WF attack on Tor in 2009, but the accuracy was only 3% in close-world. In 2011, Panchenko et al. [20] proposed a WF attack based on support vector machines (SVMs). It improves the accuracy of WF attacks. But, the computational cost of these WF attack classifiers is too high and not practical. With the development of computers, researchers have proposed more effective attacks. In 2014, the K-nearest neighbors (K-NN) classifier was proposed by Wang et al. [3]. The attack achieved a higher accuracy with shorter training time and test time. In addition, Panchenko et al. [4] used the cumulative sum of packet lengths as features and SVM as a classifier, called CUMUL, which achieved good results in close-world. Hayes et al. [5] proposed a new website fingerprinting attack K-fingerprinting (K-FP) based on random forest. K-FP achieved 91% accuracy in a closed-world setting. These methods are all based on traditional machine learning. What they have in common is that they require manual feature extraction, which is directly related to the accuracy rate. The previous work was more on how to extract more representative features.With the successful application of deep learning in other fields, many researchers are using it for website fingerprinting attacks. The obvious benefit is that it does not require manual feature extraction. To skip the process that manually extracting features, Rimmer et al. [9] proposed a deep learning-based WF attack AWF. They collected a lot of training data and tried to use three different neural network structures, Stacked Denoising Autoencoder (SDAE), CNN, and Long Short-Term Memory (LSTM). The experimental results show that the model-based CNN can obtain the highest accuracy of over 96%. Sanjit [10] proposed Var-CNN based on ResNets [21] deep learning. It has a better recognition effect when the amount of training data is small, especially in close-world. Sirinam et al. [11] proposed DF, which is based on CNN. The model has reached a high level of attack ability in both close-world and open-world. The above three deep learning-based models for WF attacks require a very simple form of data (packet direction), but it requires more training data compared to traditional machine learning.


## 2.2. Website Fingerprinting Defense

WF defense against WF attacks by inserting dummy packets. The approach brings bandwidth overhead. Tor’s bandwidth resources are precious. Researchers have been focusing on reducing bandwidth overhead and improving defensive capabilities. Different methods have different insertion rules, but they can be broadly sourced in two ways: generated by the authors based on their own expertise, and utilized to generate insertion rules from attack methods, as shown in Table2.Table 2
Differences in the rules for inserting dummy packets for WF defense.
DefenseDefensive RulesWeaknessBuFLO [22]Authors’ predetermined insertion rules, such as inserting dummy packets regularly.Large bandwidth overhead or poor defense against current WF attacks.Tamaraw [23]CS-BuFLO [24]WTF-PAD [3]W-T [13]DFD [14]WF-GAN [16]Insertion rules learned against website fingerprinting attacksDifficult to apply to practical scenarios. But it has low bandwidth overheadMockingbird [17]Buffered Fixed-Length Obfuscation (BuFLO) [22] sends packets of a fixed size at fixed intervals, using dummy packets to both fill in and (potentially) extend the transmission. But, it brings unreasonable bandwidth overhead. Tamaraw [23] and CS-BuFLO [24] extend BuFLO. The difference is that the download and upload rates can be set to different rates. But their bandwidth overhead exceeds 100%. WTF-PAD was proposed by Juarez et al. [3]. The defense attempts to fill a large time gap between two data packets. Whenever the gap is large, WTF-PAD will send a series of dummy packets. It reduces the accuracy of K-NN attacks. However, WTF-PAD was defeated by the WF attack DF. Walkie-talkie (W-T) [13] makes the attack success rate of DF only 49.7% with 31% bandwidth overhead. W-T changed the communication mode of both parties from full duplex to half duplex. The change affects the user’s network experience. For WF attacks based on deep learning, some researchers proposed WF defense based on adversarial examples, WF-GAN [16] and Mockingbird [17]. They have excellent performance, but they are difficult to be applied in live traffic due to the requirements of model input data.The latest defense method (DFD) [14] injects a dummy burst sequence according to the previous burst sequence in the real-time traffic. When the injection method is the server-side injection method, the attack success rate using CNN as the attack model is reduced from 99.93% to 5% with 85.56% bandwidth overhead. However, it did not test the existing WF attacks. In addition, the DFD cannot stably insert dummy packets. In some cases, the number of inserted dummy packets is small. When bandwidth overhead is small, the expected defense effect cannot be achieved.


## 3. Preliminary

### 3.1. Threat Model

Tor protects browsing website information of the user through data encryption and multi-hop proxy. Its working principle is shown in Figure1. In general, the user randomly selects three relay nodes, guard node, middle node, and exit node, from global active relay nodes. These selected nodes will form a communication link. All traffic will be encrypted when through the link. The purpose of the WF attacker is to obtain the website name visited by users from encrypted traffic.Figure 1
WF attack mode.To be specific, the attacker monitors and saves traffic between the guard node and user. Then, the attacker uses the traffic to infer the website name using deep learning. The attacker generally needs the direction sequence of traffic packet. The outgoing (client to server) and incoming (server to client) packets are represented as+1 and −1, respectively. For a piece of data, the total number of −1 and 1 is the effective length of the data. However, the attacker did not own all the data of the website. We mark target websites as monitored sites and other sites as unmonitored sites. To this end, WF attack and WF defense are evaluated in two different experimental environments: close-world and open-world. Close-world means that users can only access monitored websites. The open-world is closer to the real scene than the close-world. It allows the user to access any website.


### 3.2. Defense Model

The goal of the defender is to reduce the accuracy of WF attacks. One of the best ways to reduce the accuracy of WF attack is to change the input data of WF attack model. For this, there are two operations: inserting dummy packets and delay packets. Delayed packets have a greater negative impact on the network than inserting dummy packets; the latter usually is adopted by WF defense. In addition, there are two scenarios for WF defense: black-box or white-box. Black-box means that the structure and parameters of the WF attack model are unknown. We only know the correspondence between the input and output of the model. White-box means that the structure and parameter information of the attack model can be obtained. We can interact deeply with the attack model.


### 3.3. Adversarial Examples

Adversarial examples are carefully designed input data for deep learning model, which will cause the model to output incorrect prediction results. It was first discovered by the research in the field of image classification [25]. Fast Gradient Sign Method (FGSM) [15] is a classic adversarial examples generation method based on gradient. It can be expressed as x′=x+αSign∇xLθ,x,y, x′ is adversarial examples, θ is the parameter of deep learning model, x is the input of the model, y is the correct category for x, Lθ,x,y is loss function, ∇xLθ,x,y is the gradient of the loss function with respect to x, and α is super parameter. The WF defense generates adversarial examples, which is similar to FGSM. WF-GAN [16] and Mockingbird [17] modify burst-based features to generate adversarial examples. Due to the practical significance of the feature, the feature value can only increase but not decrease. Adversarial examples can be expressed as b1+δ1,b2+δ2,…,bn+δn, where bi is burst sequence of original flow and δi is perturbation size and nonnegative number that means cannot drop packet.


## 3.1. Threat Model

Tor protects browsing website information of the user through data encryption and multi-hop proxy. Its working principle is shown in Figure1. In general, the user randomly selects three relay nodes, guard node, middle node, and exit node, from global active relay nodes. These selected nodes will form a communication link. All traffic will be encrypted when through the link. The purpose of the WF attacker is to obtain the website name visited by users from encrypted traffic.Figure 1
WF attack mode.To be specific, the attacker monitors and saves traffic between the guard node and user. Then, the attacker uses the traffic to infer the website name using deep learning. The attacker generally needs the direction sequence of traffic packet. The outgoing (client to server) and incoming (server to client) packets are represented as+1 and −1, respectively. For a piece of data, the total number of −1 and 1 is the effective length of the data. However, the attacker did not own all the data of the website. We mark target websites as monitored sites and other sites as unmonitored sites. To this end, WF attack and WF defense are evaluated in two different experimental environments: close-world and open-world. Close-world means that users can only access monitored websites. The open-world is closer to the real scene than the close-world. It allows the user to access any website.


## 3.2. Defense Model

The goal of the defender is to reduce the accuracy of WF attacks. One of the best ways to reduce the accuracy of WF attack is to change the input data of WF attack model. For this, there are two operations: inserting dummy packets and delay packets. Delayed packets have a greater negative impact on the network than inserting dummy packets; the latter usually is adopted by WF defense. In addition, there are two scenarios for WF defense: black-box or white-box. Black-box means that the structure and parameters of the WF attack model are unknown. We only know the correspondence between the input and output of the model. White-box means that the structure and parameter information of the attack model can be obtained. We can interact deeply with the attack model.


## 3.3. Adversarial Examples

Adversarial examples are carefully designed input data for deep learning model, which will cause the model to output incorrect prediction results. It was first discovered by the research in the field of image classification [25]. Fast Gradient Sign Method (FGSM) [15] is a classic adversarial examples generation method based on gradient. It can be expressed as x′=x+αSign∇xLθ,x,y, x′ is adversarial examples, θ is the parameter of deep learning model, x is the input of the model, y is the correct category for x, Lθ,x,y is loss function, ∇xLθ,x,y is the gradient of the loss function with respect to x, and α is super parameter. The WF defense generates adversarial examples, which is similar to FGSM. WF-GAN [16] and Mockingbird [17] modify burst-based features to generate adversarial examples. Due to the practical significance of the feature, the feature value can only increase but not decrease. Adversarial examples can be expressed as b1+δ1,b2+δ2,…,bn+δn, where bi is burst sequence of original flow and δi is perturbation size and nonnegative number that means cannot drop packet.


## 4. Defense Design

In this section, we present the design details of SAD, that is,, method of processing data, detailed design of deep learning model, and the process of training the model.Figure2 represents the overall process of SAD. xi is a sequence of the segment of different lengths of flow, yi is the dummy packets being inserted, and adversarial examples x′=x1,y1,x2,y2,…,xi,yi,… are the output of SAD. The head rate line means that we only generate adversarial examples for the first head rate percent of each data item. We define head rate=tl/el, where x is sequence data +1,−1,+1¯,1,…, el is effective length of x, and tl is the number of tagged data in x.Figure 2
SAD defense process.


### 4.1. SAD Model Design

#### 4.1.1. Effective Length Alignment atHead Rate

We do this to train the model quickly, even if the effective length of each data is different. In Algorithm1, we describe the detailed processing; that is, each element of the data is shifted to the right by a different distance for the entire dataset. Finally, we have a uniform stop segmentation point. This is the key to speed up the model training.Algorithm 1: Batch data effective length alignment at head rate line.
(i)
Input: a batch sequence flow data: bf; head rate: hr(ii)
ml⟵ predefined maximum flow length// usually it is 5000(iii)
gei⟵ floor(hr∗ml)// global end index(iv)
forf∈BFdo(v)
el⟵ effective length of f // number of non-filled values(vi)
// the length of the current flow to be processed(vii)
hrl⟵el∗hr(viii)
step⟵gei - hrl// number of steps to move backward(ix)
// all elements off move to the right step times(x)
f⟵ moveToRight(f, step)(xi)
end


#### 4.1.2. Sequence Data Segmentation

In the existing public dataset, each piece of data is a packet direction sequence generated during the complete access process of a certain website. We do this to simulate the real packet transmission. Similarly, we describe in detail in Algorithm2. The segmentation refers to cutting the sequence into different lengths according to different segmented packet numbers spn. SPN set spns consists of several different spn. The segmented adversarial examples yi is inserted after each segment of original xi packets until the spi≥gei. Finally, the segmented sequence and the segmented adversarial examples are sequentially connected to form complete adversarial examples.


#### 4.1.3. Generating Adversarial Examples

SAD contains multiple submodels with similar structures, the generation of adversarial examples is shown in Figure3. The number of submodels is equal to the number of preset spns elements. In other words, each submodel has a corresponding spn as the input feature dimension size. The submodel structure is a simple multilayer fully connected neural network. Its activation function is Relu, as in formula (1). In order to shorten the model training time and improve the robustness of the model, Drop out and Batch Normalization are used. The input to the submodel is a segment of sequence data. The features dimension of the submodel output is ⌊spn∗sir∗hr⌋ for strictly control the bandwidth overhead, where sir is segment injection rate and hr is head rate. Although we take this means, the actual bandwidth overhead is a little bit smaller than the preset one. We tend to drop the last segment to ensure that the adversarial examples is valid. Significantly, the output of the submodel only contains −1 or +1. We applied Tanh and Sign to the model output, TanhSignas shown in formula (1). Finally, the output of the submodel represents the direction of dummy packets. The main function of the submodel is to generate adversarial examples corresponding to the segmented sequence. The output of multiple submodels is spliced with segmented sequence to generate complete adversarial examples, and the process is shown in Algorithm 2:(1)Relux=max0,x,Tanhx=ex−e−xex+e−x,Signx=−1,x<0,1,x≥0.Figure 3
SAD generates adversarial examples.Algorithm 2: SAD generate adversarial examples.
(i)
Input: a batch flow sequence: bf; submodel set: sms; segment packet number set: spns; head rate: hr(ii)
Output: Adversarial examples: ae(iii)
ml⟵ predefined maximum flow length// Usually it is 5000(iv)
gei⟵ floor(hr∗ml)// global end index(v)
spi⟵ 0// split point index(vi)
res⟵ a empty list// store final results(vii)
whileTruedo(viii)
spn⟵ randomChoice(spns)// random choice a element from spns(ix)
ifspn + si≥geithen(x)
break(xi)
end(xii)
csd⟵bf[:,spi:spi + spn]// select the data of the current segment(xiii)
res.append(css)(xiv)
sm⟵ choiceSubmodel(spn)// select submodel according spn(xv)
//sm output segment adversarial examples according css(xvi)
sae⟵sm(csd)(xvii)
sae⟵ Sign(sae)(xviii)
spi⟵spi + spn(xix)
all value ofsae reset to 0 if csd effective length is 0(xx)
res.append(sae)(xxi)
end(xxii)
res⟵ sequential splicing of elements of res(xxiii)
ae⟵ clear elements in res that have a valueof 0(xxiii)
ae⟵ fill each data length to ml at the endThe input dimension size of the submodel in thesms corresponds to the spns elements one-to-one. First, we randomly select spn from spns. Second, we choose the corresponding submodel smi that input dimension size is spn. Finally, the subsequence xilength isspn is selected from f and input into the selected submodel to obtain the output yi. In other words, the first segmented sequence x1 is put into the submodel sm1 to get output y1. The input of the submodel sm2 selected for the second time is x2, and the corresponding output is y2. By analogy, the input of the submodel smi is xi, and the output is yi. Finally, the adversarial examples generated by all submodels are spliced with the segmented sequence to generate adversarial examples y, in the form of x1,y1,x2,y2,…,xi−1,yi−1,xi,yi,….


#### 4.1.4. SAD Model Training

First, SAD outputs the adversarial examples according to the original data. Then,loss will be generated when the adversarial examples are fed to the attack model, and the loss function is Cross Entropy Loss. The difference is that our goal is to make the loss larger. Moreover, we only update the parameters of SAD. In this way, the accuracy of the attack model will decrease with adversarial sample generated by SAD. The attack model can be any WF attack model based on deep learning. Significantly, we use Sign to map the SAD output to −1 or +1. But this function is not diversified, the input gradient of Sign as the output gradient.


## 4.1. SAD Model Design

### 4.1.1. Effective Length Alignment atHead Rate

We do this to train the model quickly, even if the effective length of each data is different. In Algorithm1, we describe the detailed processing; that is, each element of the data is shifted to the right by a different distance for the entire dataset. Finally, we have a uniform stop segmentation point. This is the key to speed up the model training.Algorithm 1: Batch data effective length alignment at head rate line.
(i)
Input: a batch sequence flow data: bf; head rate: hr(ii)
ml⟵ predefined maximum flow length// usually it is 5000(iii)
gei⟵ floor(hr∗ml)// global end index(iv)
forf∈BFdo(v)
el⟵ effective length of f // number of non-filled values(vi)
// the length of the current flow to be processed(vii)
hrl⟵el∗hr(viii)
step⟵gei - hrl// number of steps to move backward(ix)
// all elements off move to the right step times(x)
f⟵ moveToRight(f, step)(xi)
end


### 4.1.2. Sequence Data Segmentation

In the existing public dataset, each piece of data is a packet direction sequence generated during the complete access process of a certain website. We do this to simulate the real packet transmission. Similarly, we describe in detail in Algorithm2. The segmentation refers to cutting the sequence into different lengths according to different segmented packet numbers spn. SPN set spns consists of several different spn. The segmented adversarial examples yi is inserted after each segment of original xi packets until the spi≥gei. Finally, the segmented sequence and the segmented adversarial examples are sequentially connected to form complete adversarial examples.


### 4.1.3. Generating Adversarial Examples

SAD contains multiple submodels with similar structures, the generation of adversarial examples is shown in Figure3. The number of submodels is equal to the number of preset spns elements. In other words, each submodel has a corresponding spn as the input feature dimension size. The submodel structure is a simple multilayer fully connected neural network. Its activation function is Relu, as in formula (1). In order to shorten the model training time and improve the robustness of the model, Drop out and Batch Normalization are used. The input to the submodel is a segment of sequence data. The features dimension of the submodel output is ⌊spn∗sir∗hr⌋ for strictly control the bandwidth overhead, where sir is segment injection rate and hr is head rate. Although we take this means, the actual bandwidth overhead is a little bit smaller than the preset one. We tend to drop the last segment to ensure that the adversarial examples is valid. Significantly, the output of the submodel only contains −1 or +1. We applied Tanh and Sign to the model output, TanhSignas shown in formula (1). Finally, the output of the submodel represents the direction of dummy packets. The main function of the submodel is to generate adversarial examples corresponding to the segmented sequence. The output of multiple submodels is spliced with segmented sequence to generate complete adversarial examples, and the process is shown in Algorithm 2:(1)Relux=max0,x,Tanhx=ex−e−xex+e−x,Signx=−1,x<0,1,x≥0.Figure 3
SAD generates adversarial examples.Algorithm 2: SAD generate adversarial examples.
(i)
Input: a batch flow sequence: bf; submodel set: sms; segment packet number set: spns; head rate: hr(ii)
Output: Adversarial examples: ae(iii)
ml⟵ predefined maximum flow length// Usually it is 5000(iv)
gei⟵ floor(hr∗ml)// global end index(v)
spi⟵ 0// split point index(vi)
res⟵ a empty list// store final results(vii)
whileTruedo(viii)
spn⟵ randomChoice(spns)// random choice a element from spns(ix)
ifspn + si≥geithen(x)
break(xi)
end(xii)
csd⟵bf[:,spi:spi + spn]// select the data of the current segment(xiii)
res.append(css)(xiv)
sm⟵ choiceSubmodel(spn)// select submodel according spn(xv)
//sm output segment adversarial examples according css(xvi)
sae⟵sm(csd)(xvii)
sae⟵ Sign(sae)(xviii)
spi⟵spi + spn(xix)
all value ofsae reset to 0 if csd effective length is 0(xx)
res.append(sae)(xxi)
end(xxii)
res⟵ sequential splicing of elements of res(xxiii)
ae⟵ clear elements in res that have a valueof 0(xxiii)
ae⟵ fill each data length to ml at the endThe input dimension size of the submodel in thesms corresponds to the spns elements one-to-one. First, we randomly select spn from spns. Second, we choose the corresponding submodel smi that input dimension size is spn. Finally, the subsequence xilength isspn is selected from f and input into the selected submodel to obtain the output yi. In other words, the first segmented sequence x1 is put into the submodel sm1 to get output y1. The input of the submodel sm2 selected for the second time is x2, and the corresponding output is y2. By analogy, the input of the submodel smi is xi, and the output is yi. Finally, the adversarial examples generated by all submodels are spliced with the segmented sequence to generate adversarial examples y, in the form of x1,y1,x2,y2,…,xi−1,yi−1,xi,yi,….


### 4.1.4. SAD Model Training

First, SAD outputs the adversarial examples according to the original data. Then,loss will be generated when the adversarial examples are fed to the attack model, and the loss function is Cross Entropy Loss. The difference is that our goal is to make the loss larger. Moreover, we only update the parameters of SAD. In this way, the accuracy of the attack model will decrease with adversarial sample generated by SAD. The attack model can be any WF attack model based on deep learning. Significantly, we use Sign to map the SAD output to −1 or +1. But this function is not diversified, the input gradient of Sign as the output gradient.


## 4.1.1. Effective Length Alignment atHead Rate

We do this to train the model quickly, even if the effective length of each data is different. In Algorithm1, we describe the detailed processing; that is, each element of the data is shifted to the right by a different distance for the entire dataset. Finally, we have a uniform stop segmentation point. This is the key to speed up the model training.Algorithm 1: Batch data effective length alignment at head rate line.
(i)
Input: a batch sequence flow data: bf; head rate: hr(ii)
ml⟵ predefined maximum flow length// usually it is 5000(iii)
gei⟵ floor(hr∗ml)// global end index(iv)
forf∈BFdo(v)
el⟵ effective length of f // number of non-filled values(vi)
// the length of the current flow to be processed(vii)
hrl⟵el∗hr(viii)
step⟵gei - hrl// number of steps to move backward(ix)
// all elements off move to the right step times(x)
f⟵ moveToRight(f, step)(xi)
end


## 4.1.2. Sequence Data Segmentation

In the existing public dataset, each piece of data is a packet direction sequence generated during the complete access process of a certain website. We do this to simulate the real packet transmission. Similarly, we describe in detail in Algorithm2. The segmentation refers to cutting the sequence into different lengths according to different segmented packet numbers spn. SPN set spns consists of several different spn. The segmented adversarial examples yi is inserted after each segment of original xi packets until the spi≥gei. Finally, the segmented sequence and the segmented adversarial examples are sequentially connected to form complete adversarial examples.


## 4.1.3. Generating Adversarial Examples

SAD contains multiple submodels with similar structures, the generation of adversarial examples is shown in Figure3. The number of submodels is equal to the number of preset spns elements. In other words, each submodel has a corresponding spn as the input feature dimension size. The submodel structure is a simple multilayer fully connected neural network. Its activation function is Relu, as in formula (1). In order to shorten the model training time and improve the robustness of the model, Drop out and Batch Normalization are used. The input to the submodel is a segment of sequence data. The features dimension of the submodel output is ⌊spn∗sir∗hr⌋ for strictly control the bandwidth overhead, where sir is segment injection rate and hr is head rate. Although we take this means, the actual bandwidth overhead is a little bit smaller than the preset one. We tend to drop the last segment to ensure that the adversarial examples is valid. Significantly, the output of the submodel only contains −1 or +1. We applied Tanh and Sign to the model output, TanhSignas shown in formula (1). Finally, the output of the submodel represents the direction of dummy packets. The main function of the submodel is to generate adversarial examples corresponding to the segmented sequence. The output of multiple submodels is spliced with segmented sequence to generate complete adversarial examples, and the process is shown in Algorithm 2:(1)Relux=max0,x,Tanhx=ex−e−xex+e−x,Signx=−1,x<0,1,x≥0.Figure 3
SAD generates adversarial examples.Algorithm 2: SAD generate adversarial examples.
(i)
Input: a batch flow sequence: bf; submodel set: sms; segment packet number set: spns; head rate: hr(ii)
Output: Adversarial examples: ae(iii)
ml⟵ predefined maximum flow length// Usually it is 5000(iv)
gei⟵ floor(hr∗ml)// global end index(v)
spi⟵ 0// split point index(vi)
res⟵ a empty list// store final results(vii)
whileTruedo(viii)
spn⟵ randomChoice(spns)// random choice a element from spns(ix)
ifspn + si≥geithen(x)
break(xi)
end(xii)
csd⟵bf[:,spi:spi + spn]// select the data of the current segment(xiii)
res.append(css)(xiv)
sm⟵ choiceSubmodel(spn)// select submodel according spn(xv)
//sm output segment adversarial examples according css(xvi)
sae⟵sm(csd)(xvii)
sae⟵ Sign(sae)(xviii)
spi⟵spi + spn(xix)
all value ofsae reset to 0 if csd effective length is 0(xx)
res.append(sae)(xxi)
end(xxii)
res⟵ sequential splicing of elements of res(xxiii)
ae⟵ clear elements in res that have a valueof 0(xxiii)
ae⟵ fill each data length to ml at the endThe input dimension size of the submodel in thesms corresponds to the spns elements one-to-one. First, we randomly select spn from spns. Second, we choose the corresponding submodel smi that input dimension size is spn. Finally, the subsequence xilength isspn is selected from f and input into the selected submodel to obtain the output yi. In other words, the first segmented sequence x1 is put into the submodel sm1 to get output y1. The input of the submodel sm2 selected for the second time is x2, and the corresponding output is y2. By analogy, the input of the submodel smi is xi, and the output is yi. Finally, the adversarial examples generated by all submodels are spliced with the segmented sequence to generate adversarial examples y, in the form of x1,y1,x2,y2,…,xi−1,yi−1,xi,yi,….


## 4.1.4. SAD Model Training

First, SAD outputs the adversarial examples according to the original data. Then,loss will be generated when the adversarial examples are fed to the attack model, and the loss function is Cross Entropy Loss. The difference is that our goal is to make the loss larger. Moreover, we only update the parameters of SAD. In this way, the accuracy of the attack model will decrease with adversarial sample generated by SAD. The attack model can be any WF attack model based on deep learning. Significantly, we use Sign to map the SAD output to −1 or +1. But this function is not diversified, the input gradient of Sign as the output gradient.


## 5. Experiment and Analysis

### 5.1. Dataset

In this article, the experimental dataset is collected by Sirinam et al. [11], which uses tor-browser-crawler [26] to simulate the process of users visiting the website.


#### 5.1.1. Plain Data

They visited Alexa 100 sites in the close-world; each website was visited 1250 times. Next, they only store the data for at least 1000 valid visits after filtering out the invalid data. In the final result, 95 websites are stored. In the open-world, they visited 50000 Alexa websites, excluding the 100 websites in the close-world dataset. Finally, they got 40716 unmonitored website traffic data after filtering invalid data.


#### 5.1.2. DFD Data

In order to evaluate the defense performance of DFD and compare it with SAD, plain data are used to generate defense data usingtwo−way and asymmetric injection with 100% client injection according to the algorithm in DFD. The average bandwidth overhead range of these defense data is 0,1.


#### 5.1.3. W-T and WTF-PAD Data

For WTF-PAD, Sirinam et al. [11] protect traces by padding them according to the defense protocols, using the scripts and simulators provided by Juarez et al. [12]. For W-T, Sirinam et al. [11] performed a new crawl with a Tor Browser in half-duplex mode. They collected datasets of size similar to the plain data. W-T requires padding the bursts in the half-duplex traces, so they followed the mold padding strategy as described in the W-T paper [13].


#### 5.1.4. Data Representation

These traffic can be expressed as a sequence of (times tamp, packet_size, direction), where direction:+1−1 means outgoing(incoming) packet. On this foundation, Sirinam et al. [11] showed that using packet length does not get an attractive improvement in the accuracy of the attack. Therefore, all data are represented by a sequence of packet directions. The sequence contains only −1 and +1.


### 5.2. Performance Index

We define three performance index: attack success rateASR, defense success rate DSR, and bandwidth overhead bo, respectively, N1/N, N2/N, Itotal/OVtotal, where N is the number of browsing records contained in the dataset, N1 is the number of data correctly identified by the WF attack model, N2 is the number of error identification of the WF attack model, Itotal is the number of inserted dummy packets, and OVtotal is the number of valid packets of the source data.


### 5.3. Result and Discussion

#### 5.3.1. WF Attack Model Evaluation in Different Sequence Lengths

Deep learning-based WF attacks use the direction sequence of flow as input data. The strategy adopted by WF defense is to insert dummy packets into normal traffic. The most obvious change brought by this strategy is that the effective length of the sequence becomes longer. In previous WF attacks based on deep learning, the length was usually set to 5000. If the length is less than 5000, fill it with 0; if it exceeds, it will be truncated directly. To obtain a credible result, we trained multiple models on different sequence lengths, even if the effective data are the same. Considering bandwidth overheadbo∈0.1x|1≤x≤7,x∈Z and sequence length sl∈50001+x|x∈bo. In this experiment, three WF attack models, DF, Var-CNN, and AWF, are used in both close-world and open-world.It can be seen from Figure4 that in close-world, different sequence lengths have stable ASR. When the sequence length is 7500, ASR of DF reaches the highest, 98.14%. When the sequence is 7000, ASR of Var-CNN reaches the highest, 97.56%, which is in line with the ability performance in original paper. In open-world, ASR of Var-CNN, AWF, and DF reaches the highest, 96.39%, 91.28%, and 96.04%, respectively. In comparison, DF and Var-CNN are relatively stable. Therefore, in the subsequent experiments related to AWF, ASR except AWF in open-world is between 96% and 97%. Finally, we can consider that the performance of the three attack models is stable in the current sequence length range. Appending meaningless data at the end of the sequence to proceed does not reduce the capability of the model. These three attack models are stable in accuracy over different sequence lengths. The padding sequences do not affect the attack models much. This ensures that the results of subsequent experiments are not affected by the length of the input data.Figure 4
Evaluation of website fingerprint attack model based on deep learning.


#### 5.3.2. Defense Model Evaluation in Different Bandwidth Overheads

In this experiment, segment packet numberspn∈13+6x|0≤x≤13,x∈Z and bandwidth overhead bo∈0.05x|1≤x≤14,x∈Z. Because different combinations of segment injection rate and head rate will bring the same real bandwidth overhead, here we have chosen the best result. In different environments, the relationship between bo and DSR is shown in Figure 5. The solid line represents the data of this experiment. It can be seen from Figure 5 that DSR has been significantly improved with the increase of bo. In close-world, DSR for three attack models is above 90% when bo is 30%. DSR is close to saturation when bo is 40%. Although ASR of the three attack models in this experiment is 96%, there are differences in the defense capabilities of SAD for the three. With the same bo, DSR in open-world is higher. It can be seen that it is more beneficial to the attacker in close-world. It is more beneficial to the defender in open-world. In addition, DSR cannot be significantly increased when bo reaches a threshold. In practice, we can ignore unmonitored website and only confuse monitoring websites. This can save computing costs. In open-world, the attacker marks the unmonitored sites as additional tags, which appear only once in the experimental data. When the defender obfuscates only the commonly used sites, the attacker’s traffic classification for the targeted sites results in either other targeted sites or unmonitored sites. In this way, the attacker is also unable to deduce valid information from the results.Figure 5
SAD defense effect.


#### 5.3.3. Different Combinations of Segment Injection Rate and Head Rate

In our design, the actual bandwidth overhead is determined by both the segmented injection rate and head rate. Therefore, the DSR may also be different for the same bandwidth overhead. In Table3, we explore this situation in detail. In this experiment, segment injection rate sir∈0.1x|1≤x≤10,x∈Z, head rate hr∈0.05x|1≤x≤20,x∈Z, and 0.05≤sir∗hr≤0.70. In the table, we have marked (bold) the largest DSR with the same bandwidth overhead and 0.05≤sir∗hr≤0.40.Table 3
Different combinations of the head rate and the segment injection rate.
DSRHR0.100.200.300.400.500.600.700.800.901.0SIR0.05—————————0.360.10————0.410.460.470.520.560.560.15———0.450.540.580.610.660.700.710.20——0.470.560.620.660.720.780.770.820.25-0.360.520.610.710.740.770.820.840.880.30—0.390.580.680.750.800.840.890.890.910.35—0.440.620.750.810.840.880.910.910.940.40—0.460.670.780.840.880.910.940.930.960.45—0.510.690.820.890.920.930.950.950.970.500.230.540.740.860.910.940.950.960.960.980.550.240.570.780.870.930.950.960.970.970.980.600.240.590.810.900.940.960.960.970.970.980.650.260.620.820.900.950.960.970.980.980.980.700.260.640.830.910.950.970.970.980.980.990.750.270.650.850.930.960.970.980.980.98—0.800.270.660.870.940.970.980.980.98——0.850.280.670.870.940.960.960.980.98——0.900.290.710.900.940.960.970.98———0.950.300.730.890.940.970.980.98———1.000.330.740.900.950.970.980.98———The greaterDSR is seen where the head rate is smaller. It means that a large amount of injection in the early stages of network traffic transmission will have better results for defense. According to the general flow, SAD inserts a fixed percentage of dummy packets based on the length of each segment, that is, hr=1. In Table 3, DSR=0.33 when sir=0.1 and hr=1 (bo=0.1); that is, each segment will be processed. However, DSR=0.62 when sir=0.5 and hr=0.2 (bo=0.1). The reason for such a large gap at the same bandwidth overhead is that dummy packets that should have been inserted at the back are inserted at the front. A web page contains many different files. When a browser loads a web page, the files are downloaded in a relatively fixed order; for example, html files are always loaded first. This means that the first segment of web traffic contains more unique packet patterns. So we speculate that the deep learning model pays more attention to the front part of the data. And our behavior causes more drastic perturbation to the data in the forward part. On the other hand, in a real scenario loading two pages sequentially, it is difficult for an attacker to separate the first page from the second page from the mix. The approach in [27] will be to extract only the traffic that contains the first page. This traffic is used to infer the name of the site on the first page. SAD may get even better results by adding more dummy packages to the first half of the segment.


#### 5.3.4. Compared with DFD

We use the algorithm that injecting dummy packets proposed by DFD to generate defense data. These defensive data contain only monitored websites. In open-world, the training data of the attack model contains both unmonitored and monitored dataset. Then, we calculate the average of bandwidth overhead based on the defense data and DF is selected as the attack model. The results are shown in Figure5, dashed lines. It is obvious that SAD is superior to DFD in all conditions of this experiment.The performance of the DFD is somewhat different from the results expressed in the original paper. In DFD, the attack model being detected is not the latest AWF, Var-CNN, and DF, but is created by itself, although its CNN-based attack model can reach the same level of accuracy as DF. On the other hand, the valid length of the obfuscated data was not increased during testing, and the model input length was set to 5000, which can lead to the loss of valid data in some data back segments. In SAD, all data will be retained.


#### 5.3.5. Adversary Training

Attackers can obtain defense data when the defense is public. In response to this situation, SAD can adjust bandwidth overhead to slow down the decline of DSR. We simulate the situation in this experiment. First, we train the DF attack model on the defense dataset until ASR reaches 90%. Then, the attack model will verify the effectiveness of multiple adversarial examples. When the defense data sequence and the input feature dimension of the attack model are not the same, if it is not enough, it is filled with 0, and if it is too long, it will be directly truncated. Furthermore, WTF-PAD and W-T are the most recognized defense methods, so we have conducted experiments using them as well, and the result is shown in Table4.Table 4
Adversary training in WTF-PAD and W-T.
DefensesOverheadaccuracy of DFWTF-PAD64%0.905 1W-T31%0.493 8In Figure6, when DF uses the adversarial examples generated by SAD for training, its ASR can still reach more than 90% with the same bandwidth overhead. The greater the gap between bandwidth overhead setting of defender and attacker, the higher DSR. Therefore, SAD has excellent defense ability against the WF attack model after adversary training. Although deep learning can automatically extract features, it requires that the dimensional size of the training and test data be the same. Defenders can employ a variety of bandwidth overheads to increase the difficulty for attackers. But, it can be seen from Table 4 that WTF-PAD and W-T have different defense abilities in the face of confrontation training, and W-T is better. WTF-PAD uses dummy packets filled according to a certain rule. W-T is to try to convert traces to another traces that already exist. SAD is almost ineffective against adversarial training, although excellent defense can be obtained by modifying the bandwidth overhead. The WF attack model can still extract the relevant features. W-T modifies the original full-duplex communication. It tries to fit already existing traces when inserting dummy packets.Figure 6
Adversarial training attack models face defense data with different bandwidth overheads.


#### 5.3.6. Black-Box Defense

The purpose of this experiment is to evaluate the robustness of SAD against unknown attack models. Unlike experiments (2)-(5), this experiment is a black-box defense. In the process of training and testing, three WF attack models can be selected, one of which is selected as the training model of SAD, and the other two is used as the test model.In Figure7, the dotted line and solid line represent DSR under the white-box and black-box, respectively. The black-box defense did not cause the SAD’s defense performance to fluctuate too much. As far as we know, the deep learning model is transferable. The three models in this experiment are the three most commonly used WF attack models, which have higher performance than other attack models and do not require manual feature extraction. Reference [9] has compared the models of SDAE, CNN, and LSTM structures. The experiments show that the model based on CNN structure achieves the best results. The basic structure of AWF, Var-CNN, and DF is CNN. We speculate that the features automatically extracted by the three models are similar, but the regions of focus are different. This leads to better results for black boxes at low bandwidth overhead. Although SAD is optimized based on only one attack model, it remains valid for other unknown models.Figure 7
SAD defense success rate in black-box defense mode.


## 5.1. Dataset

In this article, the experimental dataset is collected by Sirinam et al. [11], which uses tor-browser-crawler [26] to simulate the process of users visiting the website.


### 5.1.1. Plain Data

They visited Alexa 100 sites in the close-world; each website was visited 1250 times. Next, they only store the data for at least 1000 valid visits after filtering out the invalid data. In the final result, 95 websites are stored. In the open-world, they visited 50000 Alexa websites, excluding the 100 websites in the close-world dataset. Finally, they got 40716 unmonitored website traffic data after filtering invalid data.


### 5.1.2. DFD Data

In order to evaluate the defense performance of DFD and compare it with SAD, plain data are used to generate defense data usingtwo−way and asymmetric injection with 100% client injection according to the algorithm in DFD. The average bandwidth overhead range of these defense data is 0,1.


### 5.1.3. W-T and WTF-PAD Data

For WTF-PAD, Sirinam et al. [11] protect traces by padding them according to the defense protocols, using the scripts and simulators provided by Juarez et al. [12]. For W-T, Sirinam et al. [11] performed a new crawl with a Tor Browser in half-duplex mode. They collected datasets of size similar to the plain data. W-T requires padding the bursts in the half-duplex traces, so they followed the mold padding strategy as described in the W-T paper [13].


### 5.1.4. Data Representation

These traffic can be expressed as a sequence of (times tamp, packet_size, direction), where direction:+1−1 means outgoing(incoming) packet. On this foundation, Sirinam et al. [11] showed that using packet length does not get an attractive improvement in the accuracy of the attack. Therefore, all data are represented by a sequence of packet directions. The sequence contains only −1 and +1.


## 5.1.1. Plain Data

They visited Alexa 100 sites in the close-world; each website was visited 1250 times. Next, they only store the data for at least 1000 valid visits after filtering out the invalid data. In the final result, 95 websites are stored. In the open-world, they visited 50000 Alexa websites, excluding the 100 websites in the close-world dataset. Finally, they got 40716 unmonitored website traffic data after filtering invalid data.


## 5.1.2. DFD Data

In order to evaluate the defense performance of DFD and compare it with SAD, plain data are used to generate defense data usingtwo−way and asymmetric injection with 100% client injection according to the algorithm in DFD. The average bandwidth overhead range of these defense data is 0,1.


## 5.1.3. W-T and WTF-PAD Data

For WTF-PAD, Sirinam et al. [11] protect traces by padding them according to the defense protocols, using the scripts and simulators provided by Juarez et al. [12]. For W-T, Sirinam et al. [11] performed a new crawl with a Tor Browser in half-duplex mode. They collected datasets of size similar to the plain data. W-T requires padding the bursts in the half-duplex traces, so they followed the mold padding strategy as described in the W-T paper [13].


## 5.1.4. Data Representation

These traffic can be expressed as a sequence of (times tamp, packet_size, direction), where direction:+1−1 means outgoing(incoming) packet. On this foundation, Sirinam et al. [11] showed that using packet length does not get an attractive improvement in the accuracy of the attack. Therefore, all data are represented by a sequence of packet directions. The sequence contains only −1 and +1.


## 5.2. Performance Index

We define three performance index: attack success rateASR, defense success rate DSR, and bandwidth overhead bo, respectively, N1/N, N2/N, Itotal/OVtotal, where N is the number of browsing records contained in the dataset, N1 is the number of data correctly identified by the WF attack model, N2 is the number of error identification of the WF attack model, Itotal is the number of inserted dummy packets, and OVtotal is the number of valid packets of the source data.


## 5.3. Result and Discussion

### 5.3.1. WF Attack Model Evaluation in Different Sequence Lengths

Deep learning-based WF attacks use the direction sequence of flow as input data. The strategy adopted by WF defense is to insert dummy packets into normal traffic. The most obvious change brought by this strategy is that the effective length of the sequence becomes longer. In previous WF attacks based on deep learning, the length was usually set to 5000. If the length is less than 5000, fill it with 0; if it exceeds, it will be truncated directly. To obtain a credible result, we trained multiple models on different sequence lengths, even if the effective data are the same. Considering bandwidth overheadbo∈0.1x|1≤x≤7,x∈Z and sequence length sl∈50001+x|x∈bo. In this experiment, three WF attack models, DF, Var-CNN, and AWF, are used in both close-world and open-world.It can be seen from Figure4 that in close-world, different sequence lengths have stable ASR. When the sequence length is 7500, ASR of DF reaches the highest, 98.14%. When the sequence is 7000, ASR of Var-CNN reaches the highest, 97.56%, which is in line with the ability performance in original paper. In open-world, ASR of Var-CNN, AWF, and DF reaches the highest, 96.39%, 91.28%, and 96.04%, respectively. In comparison, DF and Var-CNN are relatively stable. Therefore, in the subsequent experiments related to AWF, ASR except AWF in open-world is between 96% and 97%. Finally, we can consider that the performance of the three attack models is stable in the current sequence length range. Appending meaningless data at the end of the sequence to proceed does not reduce the capability of the model. These three attack models are stable in accuracy over different sequence lengths. The padding sequences do not affect the attack models much. This ensures that the results of subsequent experiments are not affected by the length of the input data.Figure 4
Evaluation of website fingerprint attack model based on deep learning.


### 5.3.2. Defense Model Evaluation in Different Bandwidth Overheads

In this experiment, segment packet numberspn∈13+6x|0≤x≤13,x∈Z and bandwidth overhead bo∈0.05x|1≤x≤14,x∈Z. Because different combinations of segment injection rate and head rate will bring the same real bandwidth overhead, here we have chosen the best result. In different environments, the relationship between bo and DSR is shown in Figure 5. The solid line represents the data of this experiment. It can be seen from Figure 5 that DSR has been significantly improved with the increase of bo. In close-world, DSR for three attack models is above 90% when bo is 30%. DSR is close to saturation when bo is 40%. Although ASR of the three attack models in this experiment is 96%, there are differences in the defense capabilities of SAD for the three. With the same bo, DSR in open-world is higher. It can be seen that it is more beneficial to the attacker in close-world. It is more beneficial to the defender in open-world. In addition, DSR cannot be significantly increased when bo reaches a threshold. In practice, we can ignore unmonitored website and only confuse monitoring websites. This can save computing costs. In open-world, the attacker marks the unmonitored sites as additional tags, which appear only once in the experimental data. When the defender obfuscates only the commonly used sites, the attacker’s traffic classification for the targeted sites results in either other targeted sites or unmonitored sites. In this way, the attacker is also unable to deduce valid information from the results.Figure 5
SAD defense effect.


### 5.3.3. Different Combinations of Segment Injection Rate and Head Rate

In our design, the actual bandwidth overhead is determined by both the segmented injection rate and head rate. Therefore, the DSR may also be different for the same bandwidth overhead. In Table3, we explore this situation in detail. In this experiment, segment injection rate sir∈0.1x|1≤x≤10,x∈Z, head rate hr∈0.05x|1≤x≤20,x∈Z, and 0.05≤sir∗hr≤0.70. In the table, we have marked (bold) the largest DSR with the same bandwidth overhead and 0.05≤sir∗hr≤0.40.Table 3
Different combinations of the head rate and the segment injection rate.
DSRHR0.100.200.300.400.500.600.700.800.901.0SIR0.05—————————0.360.10————0.410.460.470.520.560.560.15———0.450.540.580.610.660.700.710.20——0.470.560.620.660.720.780.770.820.25-0.360.520.610.710.740.770.820.840.880.30—0.390.580.680.750.800.840.890.890.910.35—0.440.620.750.810.840.880.910.910.940.40—0.460.670.780.840.880.910.940.930.960.45—0.510.690.820.890.920.930.950.950.970.500.230.540.740.860.910.940.950.960.960.980.550.240.570.780.870.930.950.960.970.970.980.600.240.590.810.900.940.960.960.970.970.980.650.260.620.820.900.950.960.970.980.980.980.700.260.640.830.910.950.970.970.980.980.990.750.270.650.850.930.960.970.980.980.98—0.800.270.660.870.940.970.980.980.98——0.850.280.670.870.940.960.960.980.98——0.900.290.710.900.940.960.970.98———0.950.300.730.890.940.970.980.98———1.000.330.740.900.950.970.980.98———The greaterDSR is seen where the head rate is smaller. It means that a large amount of injection in the early stages of network traffic transmission will have better results for defense. According to the general flow, SAD inserts a fixed percentage of dummy packets based on the length of each segment, that is, hr=1. In Table 3, DSR=0.33 when sir=0.1 and hr=1 (bo=0.1); that is, each segment will be processed. However, DSR=0.62 when sir=0.5 and hr=0.2 (bo=0.1). The reason for such a large gap at the same bandwidth overhead is that dummy packets that should have been inserted at the back are inserted at the front. A web page contains many different files. When a browser loads a web page, the files are downloaded in a relatively fixed order; for example, html files are always loaded first. This means that the first segment of web traffic contains more unique packet patterns. So we speculate that the deep learning model pays more attention to the front part of the data. And our behavior causes more drastic perturbation to the data in the forward part. On the other hand, in a real scenario loading two pages sequentially, it is difficult for an attacker to separate the first page from the second page from the mix. The approach in [27] will be to extract only the traffic that contains the first page. This traffic is used to infer the name of the site on the first page. SAD may get even better results by adding more dummy packages to the first half of the segment.


### 5.3.4. Compared with DFD

We use the algorithm that injecting dummy packets proposed by DFD to generate defense data. These defensive data contain only monitored websites. In open-world, the training data of the attack model contains both unmonitored and monitored dataset. Then, we calculate the average of bandwidth overhead based on the defense data and DF is selected as the attack model. The results are shown in Figure5, dashed lines. It is obvious that SAD is superior to DFD in all conditions of this experiment.The performance of the DFD is somewhat different from the results expressed in the original paper. In DFD, the attack model being detected is not the latest AWF, Var-CNN, and DF, but is created by itself, although its CNN-based attack model can reach the same level of accuracy as DF. On the other hand, the valid length of the obfuscated data was not increased during testing, and the model input length was set to 5000, which can lead to the loss of valid data in some data back segments. In SAD, all data will be retained.


### 5.3.5. Adversary Training

Attackers can obtain defense data when the defense is public. In response to this situation, SAD can adjust bandwidth overhead to slow down the decline of DSR. We simulate the situation in this experiment. First, we train the DF attack model on the defense dataset until ASR reaches 90%. Then, the attack model will verify the effectiveness of multiple adversarial examples. When the defense data sequence and the input feature dimension of the attack model are not the same, if it is not enough, it is filled with 0, and if it is too long, it will be directly truncated. Furthermore, WTF-PAD and W-T are the most recognized defense methods, so we have conducted experiments using them as well, and the result is shown in Table4.Table 4
Adversary training in WTF-PAD and W-T.
DefensesOverheadaccuracy of DFWTF-PAD64%0.905 1W-T31%0.493 8In Figure6, when DF uses the adversarial examples generated by SAD for training, its ASR can still reach more than 90% with the same bandwidth overhead. The greater the gap between bandwidth overhead setting of defender and attacker, the higher DSR. Therefore, SAD has excellent defense ability against the WF attack model after adversary training. Although deep learning can automatically extract features, it requires that the dimensional size of the training and test data be the same. Defenders can employ a variety of bandwidth overheads to increase the difficulty for attackers. But, it can be seen from Table 4 that WTF-PAD and W-T have different defense abilities in the face of confrontation training, and W-T is better. WTF-PAD uses dummy packets filled according to a certain rule. W-T is to try to convert traces to another traces that already exist. SAD is almost ineffective against adversarial training, although excellent defense can be obtained by modifying the bandwidth overhead. The WF attack model can still extract the relevant features. W-T modifies the original full-duplex communication. It tries to fit already existing traces when inserting dummy packets.Figure 6
Adversarial training attack models face defense data with different bandwidth overheads.


### 5.3.6. Black-Box Defense

The purpose of this experiment is to evaluate the robustness of SAD against unknown attack models. Unlike experiments (2)-(5), this experiment is a black-box defense. In the process of training and testing, three WF attack models can be selected, one of which is selected as the training model of SAD, and the other two is used as the test model.In Figure7, the dotted line and solid line represent DSR under the white-box and black-box, respectively. The black-box defense did not cause the SAD’s defense performance to fluctuate too much. As far as we know, the deep learning model is transferable. The three models in this experiment are the three most commonly used WF attack models, which have higher performance than other attack models and do not require manual feature extraction. Reference [9] has compared the models of SDAE, CNN, and LSTM structures. The experiments show that the model based on CNN structure achieves the best results. The basic structure of AWF, Var-CNN, and DF is CNN. We speculate that the features automatically extracted by the three models are similar, but the regions of focus are different. This leads to better results for black boxes at low bandwidth overhead. Although SAD is optimized based on only one attack model, it remains valid for other unknown models.Figure 7
SAD defense success rate in black-box defense mode.


## 5.3.1. WF Attack Model Evaluation in Different Sequence Lengths

Deep learning-based WF attacks use the direction sequence of flow as input data. The strategy adopted by WF defense is to insert dummy packets into normal traffic. The most obvious change brought by this strategy is that the effective length of the sequence becomes longer. In previous WF attacks based on deep learning, the length was usually set to 5000. If the length is less than 5000, fill it with 0; if it exceeds, it will be truncated directly. To obtain a credible result, we trained multiple models on different sequence lengths, even if the effective data are the same. Considering bandwidth overheadbo∈0.1x|1≤x≤7,x∈Z and sequence length sl∈50001+x|x∈bo. In this experiment, three WF attack models, DF, Var-CNN, and AWF, are used in both close-world and open-world.It can be seen from Figure4 that in close-world, different sequence lengths have stable ASR. When the sequence length is 7500, ASR of DF reaches the highest, 98.14%. When the sequence is 7000, ASR of Var-CNN reaches the highest, 97.56%, which is in line with the ability performance in original paper. In open-world, ASR of Var-CNN, AWF, and DF reaches the highest, 96.39%, 91.28%, and 96.04%, respectively. In comparison, DF and Var-CNN are relatively stable. Therefore, in the subsequent experiments related to AWF, ASR except AWF in open-world is between 96% and 97%. Finally, we can consider that the performance of the three attack models is stable in the current sequence length range. Appending meaningless data at the end of the sequence to proceed does not reduce the capability of the model. These three attack models are stable in accuracy over different sequence lengths. The padding sequences do not affect the attack models much. This ensures that the results of subsequent experiments are not affected by the length of the input data.Figure 4
Evaluation of website fingerprint attack model based on deep learning.


## 5.3.2. Defense Model Evaluation in Different Bandwidth Overheads

In this experiment, segment packet numberspn∈13+6x|0≤x≤13,x∈Z and bandwidth overhead bo∈0.05x|1≤x≤14,x∈Z. Because different combinations of segment injection rate and head rate will bring the same real bandwidth overhead, here we have chosen the best result. In different environments, the relationship between bo and DSR is shown in Figure 5. The solid line represents the data of this experiment. It can be seen from Figure 5 that DSR has been significantly improved with the increase of bo. In close-world, DSR for three attack models is above 90% when bo is 30%. DSR is close to saturation when bo is 40%. Although ASR of the three attack models in this experiment is 96%, there are differences in the defense capabilities of SAD for the three. With the same bo, DSR in open-world is higher. It can be seen that it is more beneficial to the attacker in close-world. It is more beneficial to the defender in open-world. In addition, DSR cannot be significantly increased when bo reaches a threshold. In practice, we can ignore unmonitored website and only confuse monitoring websites. This can save computing costs. In open-world, the attacker marks the unmonitored sites as additional tags, which appear only once in the experimental data. When the defender obfuscates only the commonly used sites, the attacker’s traffic classification for the targeted sites results in either other targeted sites or unmonitored sites. In this way, the attacker is also unable to deduce valid information from the results.Figure 5
SAD defense effect.


## 5.3.3. Different Combinations of Segment Injection Rate and Head Rate

In our design, the actual bandwidth overhead is determined by both the segmented injection rate and head rate. Therefore, the DSR may also be different for the same bandwidth overhead. In Table3, we explore this situation in detail. In this experiment, segment injection rate sir∈0.1x|1≤x≤10,x∈Z, head rate hr∈0.05x|1≤x≤20,x∈Z, and 0.05≤sir∗hr≤0.70. In the table, we have marked (bold) the largest DSR with the same bandwidth overhead and 0.05≤sir∗hr≤0.40.Table 3
Different combinations of the head rate and the segment injection rate.
DSRHR0.100.200.300.400.500.600.700.800.901.0SIR0.05—————————0.360.10————0.410.460.470.520.560.560.15———0.450.540.580.610.660.700.710.20——0.470.560.620.660.720.780.770.820.25-0.360.520.610.710.740.770.820.840.880.30—0.390.580.680.750.800.840.890.890.910.35—0.440.620.750.810.840.880.910.910.940.40—0.460.670.780.840.880.910.940.930.960.45—0.510.690.820.890.920.930.950.950.970.500.230.540.740.860.910.940.950.960.960.980.550.240.570.780.870.930.950.960.970.970.980.600.240.590.810.900.940.960.960.970.970.980.650.260.620.820.900.950.960.970.980.980.980.700.260.640.830.910.950.970.970.980.980.990.750.270.650.850.930.960.970.980.980.98—0.800.270.660.870.940.970.980.980.98——0.850.280.670.870.940.960.960.980.98——0.900.290.710.900.940.960.970.98———0.950.300.730.890.940.970.980.98———1.000.330.740.900.950.970.980.98———The greaterDSR is seen where the head rate is smaller. It means that a large amount of injection in the early stages of network traffic transmission will have better results for defense. According to the general flow, SAD inserts a fixed percentage of dummy packets based on the length of each segment, that is, hr=1. In Table 3, DSR=0.33 when sir=0.1 and hr=1 (bo=0.1); that is, each segment will be processed. However, DSR=0.62 when sir=0.5 and hr=0.2 (bo=0.1). The reason for such a large gap at the same bandwidth overhead is that dummy packets that should have been inserted at the back are inserted at the front. A web page contains many different files. When a browser loads a web page, the files are downloaded in a relatively fixed order; for example, html files are always loaded first. This means that the first segment of web traffic contains more unique packet patterns. So we speculate that the deep learning model pays more attention to the front part of the data. And our behavior causes more drastic perturbation to the data in the forward part. On the other hand, in a real scenario loading two pages sequentially, it is difficult for an attacker to separate the first page from the second page from the mix. The approach in [27] will be to extract only the traffic that contains the first page. This traffic is used to infer the name of the site on the first page. SAD may get even better results by adding more dummy packages to the first half of the segment.


## 5.3.4. Compared with DFD

We use the algorithm that injecting dummy packets proposed by DFD to generate defense data. These defensive data contain only monitored websites. In open-world, the training data of the attack model contains both unmonitored and monitored dataset. Then, we calculate the average of bandwidth overhead based on the defense data and DF is selected as the attack model. The results are shown in Figure5, dashed lines. It is obvious that SAD is superior to DFD in all conditions of this experiment.The performance of the DFD is somewhat different from the results expressed in the original paper. In DFD, the attack model being detected is not the latest AWF, Var-CNN, and DF, but is created by itself, although its CNN-based attack model can reach the same level of accuracy as DF. On the other hand, the valid length of the obfuscated data was not increased during testing, and the model input length was set to 5000, which can lead to the loss of valid data in some data back segments. In SAD, all data will be retained.


## 5.3.5. Adversary Training

Attackers can obtain defense data when the defense is public. In response to this situation, SAD can adjust bandwidth overhead to slow down the decline of DSR. We simulate the situation in this experiment. First, we train the DF attack model on the defense dataset until ASR reaches 90%. Then, the attack model will verify the effectiveness of multiple adversarial examples. When the defense data sequence and the input feature dimension of the attack model are not the same, if it is not enough, it is filled with 0, and if it is too long, it will be directly truncated. Furthermore, WTF-PAD and W-T are the most recognized defense methods, so we have conducted experiments using them as well, and the result is shown in Table4.Table 4
Adversary training in WTF-PAD and W-T.
DefensesOverheadaccuracy of DFWTF-PAD64%0.905 1W-T31%0.493 8In Figure6, when DF uses the adversarial examples generated by SAD for training, its ASR can still reach more than 90% with the same bandwidth overhead. The greater the gap between bandwidth overhead setting of defender and attacker, the higher DSR. Therefore, SAD has excellent defense ability against the WF attack model after adversary training. Although deep learning can automatically extract features, it requires that the dimensional size of the training and test data be the same. Defenders can employ a variety of bandwidth overheads to increase the difficulty for attackers. But, it can be seen from Table 4 that WTF-PAD and W-T have different defense abilities in the face of confrontation training, and W-T is better. WTF-PAD uses dummy packets filled according to a certain rule. W-T is to try to convert traces to another traces that already exist. SAD is almost ineffective against adversarial training, although excellent defense can be obtained by modifying the bandwidth overhead. The WF attack model can still extract the relevant features. W-T modifies the original full-duplex communication. It tries to fit already existing traces when inserting dummy packets.Figure 6
Adversarial training attack models face defense data with different bandwidth overheads.


## 5.3.6. Black-Box Defense

The purpose of this experiment is to evaluate the robustness of SAD against unknown attack models. Unlike experiments (2)-(5), this experiment is a black-box defense. In the process of training and testing, three WF attack models can be selected, one of which is selected as the training model of SAD, and the other two is used as the test model.In Figure7, the dotted line and solid line represent DSR under the white-box and black-box, respectively. The black-box defense did not cause the SAD’s defense performance to fluctuate too much. As far as we know, the deep learning model is transferable. The three models in this experiment are the three most commonly used WF attack models, which have higher performance than other attack models and do not require manual feature extraction. Reference [9] has compared the models of SDAE, CNN, and LSTM structures. The experiments show that the model based on CNN structure achieves the best results. The basic structure of AWF, Var-CNN, and DF is CNN. We speculate that the features automatically extracted by the three models are similar, but the regions of focus are different. This leads to better results for black boxes at low bandwidth overhead. Although SAD is optimized based on only one attack model, it remains valid for other unknown models.Figure 7
SAD defense success rate in black-box defense mode.


## 6. Limitations and Future Work

There are still many limitations to SAD. Our ultimate goal is to deploy SAD in real-world scenarios. However, we have to consider how long it takes to generate the adversarial examples. We counted the time spent by SAD to generate the adversarial examples; the information is presented in Table5. SAD generates a segment of adversarial examples in less than 2 milliseconds. It takes about 200 milliseconds for a complete trace, that is, the number of times SAD generates adversarial data T=⌈traces length/segment length⌉, and the time complexity is On. As far as we know, it takes 1 to 2 seconds to load a simple search engine home page. On the face of it, the SAD time overhead is reasonable. In fact, our results are obtained with the GPU in use. It takes 1 to 2 minutes to generate a piece of adversarial examples under the same conditions, with only CPU. This will seriously affect the web browsing. WTF-PAD, W-T, and DFD do not have such troubles because they do not use deep learning. According to the current development of deep learning, in order to get faster speed on the CPU, it is necessary to reduce the complexity of the model. So in the subsequent work, we can start from two aspects: reduce the model complexity and maintain the performance, and use nondeep learning methods to achieve the same purpose.Table 5
SAD generates data time statistics. Segment data generate: generating a segment of data; avg. segment: average number of data segments; complete data generate: SAD generates completed adversarial examples. The unit of data is milliseconds.
Bandwidth OverheadSegment data generationAvg. segmentComplete data generationMaxminMeanStdMaxminMeanStd0.17.631.661.880.2193.64205.19162.84184.679.390.26.141.671.860.1893.77216.87155.31182.2611.300.38.291.661.850.1892.86202.50157.09179.857.980.47.421.661.840.1893.28196.94159.27179.658.800.58.101.661.890.3394.32242.65161.43186.9214.320.67.811.651.850.2093.32203.19160.22181.379.350.77.921.661.890.3493.27230.50160.94184.3814.41Adversarial examples can reduce the classification performance of the deep learning classifier. SAD is a model that can generate adversarial examples. We can assume that SAD transforms trace into another class. Ilyas et al. [28] demonstrate that adversarial examples can be directly attributed to the presence of nonrobust features: features (derived from patterns in the data distribution) that are highly predictive, yet brittle and (thus) incomprehensible to humans. But this transformation can be recognized by the model. Using the adversarial samples as the training data for the model, the model can extract nonrobust features as the basis for classification. However, deep learning-based classifiers have a maximum accuracy of only 50% on data generated by W-T. Currently, SAD has no such capability. Our future work is to make SAD have W-T’s better ability to face the adversarial training.


## 7. Conclusion

In this article, we proposed a novel website fingerprinting defense, called Segment Adversarial Defense, against deep learning-based WF attacks. SAD can perform segmentation processing on live traffic and injects dummy packets after each segment to complete the defense. At the same time, we can limit the range of segmentation to get more diverse adversarial examples. By this, SAD can flexibly tune the actual bandwidth overhead. The operation effectively balances the defense success rate and bandwidth overhead. In addition, SAD can deal with adversarial training and black-box defense. The experimental results show that in a close-world and open-world, the SAD defense success rate can reach up to 99%. It only needs 30% bandwidth overhead to achieve 90% defensive success rate. Remarkably, it is possible to get over 40%DSR with only 5% bandwidth overhead. In summary, SAD can effectively against WF attacks based on deep learning.



---

*Source: 7330465-2022-04-07.xml*