Title: DiffuPac: Contextual Mimicry in Adversarial Packets Generation via Diffusion Model
Venue: NeurIPS 2024
Accepted: yes
Keywords: Network Intrusion Detection System

Adversarial Machine Learning

Cybersecurity

Adversarial Sample Generation
Abstract: In domains of cybersecurity, recent advancements in Machine Learning (ML) and Deep Learning (DL) have significantly enhanced Network Intrusion Detection Systems (NIDS), improving the effectiveness of cybersecurity operations. However, attackers have also leveraged ML/DL to develop sophisticated models that generate adversarial packets capable of evading NIDS detection. Consequently, defenders must study and analyze these models to prepare for the evasion attacks that exploit NIDS detection mechanisms. Unfortunately, conventional generation models often rely on unrealistic assumptions about attackers' knowledge of NIDS components, making them impractical for real-world scenarios. To address this issue, we present DiffuPac, a first-of-its-kind generation model designed to generate adversarial packets that evade detection without relying on specific NIDS components. DiffuPac integrates a pre-trained Bidirectional Encoder Representations from Transformers (BERT) with diffusion model, which, through its capability for conditional denoising and classifier-free guidance, effectively addresses the real-world constraint of limited attacker knowledge. By concatenating malicious packets with contextually relevant normal packets and applying targeted noising only to the malicious packets, DiffuPac seamlessly blends adversarial packets into genuine network traffic. Through evaluations on real-world datasets, we demonstrate that DiffuPac achieves strong evasion capabilities against sophisticated NIDS, outperforming conventional methods by an average of 6.69 percentage points, while preserving the functionality and practicality of the generated adversarial packets.

DiffuPac: Contextual Mimicry in Adversarial Packets Generation via Diffusion Model

Abstract

In domains of cybersecurity, recent advancements in Machine Learning (ML) and Deep Learning (DL) have significantly enhanced Network Intrusion Detection Systems (NIDS), improving the effectiveness of cybersecurity operations. However, attackers have also leveraged ML/DL to develop sophisticated models that generate adversarial packets capable of evading NIDS detection. Consequently, defenders must study and analyze these models to prepare for the evasion attacks that exploit NIDS detection mechanisms. Unfortunately, conventional generation models often rely on unrealistic assumptions about attackers' knowledge of NIDS components, making them impractical for real-world scenarios. To address this issue, we present DiffuPac, a first-of-its-kind generation model designed to generate adversarial packets that evade detection without relying on specific NIDS components. DiffuPac integrates a pre-trained Bidirectional Encoder Representations from Transformers (BERT) with diffusion model, which, through its capability for conditional denoising and classifier-free guidance, effectively addresses the real-world constraint of limited attacker knowledge. By concatenating malicious packets with contextually relevant normal packets and applying targeted noising only to the malicious packets, DiffuPac seamlessly blends adversarial packets into genuine network traffic. Through evaluations on real-world datasets, we demonstrate that DiffuPac achieves strong evasion capabilities against sophisticated NIDS, outperforming conventional methods by an average of 6.69 percentage points, while preserving the functionality and practicality of the generated adversarial packets.

1 Introduction

Network Intrusion Detection Systems (NIDS) play a pivotal role in safeguarding the vast array of digital devices and infrastructures that permeate our lives. As of 2023, the global count of active IoT devices is expected to reach approximately 15.14 billion, with projections suggesting a rise to 30 billion by 2030 (Statista [2023]). This explosive growth, fueled by applications spanning from consumer electronics to industrial automation and healthcare, presents formidable security challenges. To meet these challenges, advancements in Machine Learning (ML) and Deep Learning (DL) have significantly bolstered the efficacy of NIDS in monitoring IoT traffic and detecting malicious activities (Talaei Khoei and Kaabouch [2023]; Talaei Khoei et al. [2023]).

However, the rapid evolution of generative AI technologies has ushered in a new era of cybersecurity threats, notably through the creation of adversarial packets designed to evade detection by even the most sophisticated NIDS. These AI-driven attacks can emulate and synthesize legitimate network behaviors, presenting an unprecedented challenge to existing security paradigms. Generative models, particularly those trained on extensive datasets of genuine network traffic, can generate adversarial packets that blend malicious functionalities within seemingly normal packet sequences, thus effectively camouflaging their malicious intents.

In response to these evolving threats, it is critical for cybersecurity defenders to deepen their understanding of these generative AI models. By scrutinizing the mechanisms through which these models generate adversarial packets, defenders can better anticipate and counteract adversarial tactics that compromise the detection capabilities of NIDS (Ibitoye et al. [2019]; Khazane et al. [2024]). The urgency to develop innovative solutions that can adapt to and preempt these adversarial tactics is paramount, ensuring the reliability and robustness of NIDS in an increasingly complex threat landscape.

Traditional methods for generating adversarial packets have primarily relied on direct engagement with NIDS or the use of surrogate classifiers, often assuming unrealistic levels of attacker access to NIDS configurations. Table 1 summarizes recent literature on adversarial packet generation, highlighting the limitations of these approaches. Studies using techniques such as Network Emulator (NetEM) and Metasploit (Homoliak et al. [2018]), Generative Adversarial Network (GAN) and Particle Swarm Optimization (PSO) (Han et al. [2021]), and Reinforcement Learning (RL) (Hore et al. [2023]) demonstrate that while these methods successfully modified the packets behavior, their

efficacy in evading detection in real-world conditions remains suboptimal. The reliance on detailed knowledge of NIDS models is flawed, as attackers typically operate with limited information about the underlying security infrastructure. This gap underscores the necessity for more practical and effective adversarial generation methods that align with the realistic constraints faced by attackers.

To address these challenges, we introduce DiffuPac: Contextual Mimicry in Adversarial Packet Generation via Diffusion Model , a novel solution that leverages the combined strengths of Bidirectional Encoder Representations from Transformers (BERT) and diffusion model. This innovative fusion not only promises high accuracy in generating adversarial packets but also operates under the realistic assumption that attackers lack direct access to NIDS models. DiffuPac leverages the extensive contextual understanding provided by BERT, which has been trained on diverse datasets representing a wide range of network behaviors, along with the generative capabilities of diffusion models. This fusion results in a sophisticated adversarial tactics where the elements of the attack are seamlessly integrated into the network traffic, making them indistinguishable from legitimate data. This capability represents a significant leap forward, offering a stealthy approach that outmaneuvers current NIDS through advanced mimicry rather than direct confrontation.

In summary, the principal contributions of this paper are as follows: (a) we have pioneered the integration of BERT and diffusion models to create DiffuPac, marking a first in the cybersecurity domain. This novel methodology sets a precedent in the field by blending the advanced contextual comprehension of network traffic with sophisticated generative capabilities to produce adversarial packets that are both stealthy and indistinguishable from genuine traffic; (b) we introduce a unique concatenation strategy coupled with targeted noising techniques. These innovations ensure that the adversarial packets not only blend seamlessly into the network environment but also dynamically adapt to evade modern detection systems; (c) DiffuPac advances a classifier-free approach to adversarial packet generation. This approach challenges traditional dependency on surrogate classifiers, offering

a new paradigm that more accurately reflects the constraints and capabilities of real-world attackers; (d) Lastly, our extensive experimental evaluations, conducted on real-world datasets, demonstrate that DiffuPac significantly outperforms existing methods in terms of evasion effectiveness, establishing new benchmarks for the generation of adversarial packets. These contributions collectively push the boundaries of what is possible in the realm of network security, paving the way for more resilient cybersecurity defenses and a deeper understanding of adversarial tactics in network environments.

2 Related Works

Adversarial Attacks on ML/DL-based NIDS . NIDS are crucial for protecting digital infrastructures by monitoring network traffic and identifying potential threats through signature-based and anomalybased detection paradigms. Signature-based NIDS use pattern matching against predefined threat databases, while anomaly-based NIDS employ machine learning models to detect deviations from benign traffic patterns, providing an advantage in detecting sophisticated threats. The transition to DL in anomaly-based NIDS, as highlighted by Ahmad et al. [2021], has enhanced threat detection due to the capability to learn abstract patterns, though this shift has introduced a vulnerability to adversarial attacks. These attacks modify network data subtly to evade NIDS, a phenomenon first noted in computer vision (Szegedy et al. [2014]) and now challenging DL-based NIDS. Adversarial attack tactics vary based on attacker knowledge, ranging from white-box attacks with full system knowledge to black-box attacks with no classifier knowledge, as outlined by McCarthy et al. [2022]. Black-box scenarios are common in real-world threats since attackers generally lack direct NIDS access, necessitating sophisticated modifications of data to exploit DL model vulnerabilities and blur the distinction between normal and malicious traffic.

Adversarial Attacks Evading NIDS . In the study of NIDS, researchers explore feature-level and packet-level attacks to enhance robustness. Feature-level attacks modify input network features using methods like GANs to mislead classifiers without direct knowledge of their mechanisms. For instance, Yang et al. [2018] used transfer-based and score-based attacks to deceive a Deep Neural Network (DNN) model on the NSL-KDD dataset, while Sheatsley et al. [2022] developed an Augmented JSMA (AJSMA) to ensure realistic feature modifications. However, these still fail to generate executable malicious packets since they do not provide a method for converting modified features into packet sequences. Conversely, packet-level attacks, which modify network packets directly to maintain their malicious intent while evading detection, are particularly effective. As detailed in Table 1, studies such as Hashemi et al. [2019] show successful modifications using non-payload based and mimicking operations. In the survey He et al. [2023], authors emphasized that the practicality and replayability of packet-level attacks make them more dominant, ensuring adversarial packets evade detection while remaining executable. Given this efficacy, our research model, DiffuPac, is designed to excel in this domain, outperforming previous models in robustness and detection evasion, and setting a new standard for NIDS efficacy in adversarial cybersecurity.

Diffusion Models . Represent a breakthrough in generative modeling, offering a novel framework for understanding data as a dynamic and stochastic process. Based on the works of Sohl-Dickstein et al. [2015] and Ho et al. [2020], these models conceptualize data points z 0 ∈ R d (where d is a positive integer) as the end result of a reverse Markov chain. In the forward diffusion process, data points z 0 are gradually transformed into a noise distribution . This transformation is modeled by a Markov chain starting from the data distribution q ( z 0 ) and ending in a Gaussian distribution z T ∼ N ( 0 , I ) , where T represents the total number of timesteps in the forward diffusion process. The transitions are defined q ( z t | z t -1 ) = N ( z t ; √ 1 -β t z t -1 , β t I ) for 1 ≤ t ≤ T , where β t as the variance scale. In the reverse process, the model f θ (usually a U-Net or a Transformer) learns to reconstruct the original data z 0 from the noised data z T . It does so by iteratively estimating the parameters p θ ( z t -1 | z t ) = N ( z t -1 ; µ θ ( z t , t ) , σ 2 t I ) , where µ θ and σ 2 t predict the mean and variance of the distribution.

BERT and Diffusion Models Approaches in Cybersecurity and Other Fields . The application of pre-trained models, such as ET-BERT (Lin et al. [2022]) and NetGPT (Meng et al. [2023]), has revolutionized understanding of network traffic by capturing the intricacies of language and interpreting complex network patterns. Meanwhile, diffusion models have emerged as powerful generative tools capable of producing high-fidelity data for applications like image generation and notably, synthetic network traffic generation. These models leverage a unique denoising training objective to closely mimic the original data distribution, making them promising for generating realistic network

Pcap Preprocessing

Pre-training

Fine-tuning with Diffusion Model

IP 192.168.1.1.52255

> 40.103.10.38.443:

Flags [.], seq

2667572664:2667574

014, ack 3107627624,

win 4096, length 1350

Pcap files

Src-to-dst &

Dst-to-src

splitting

Session

IPsrc

192.168.1.1,

IPdst

40.103.10.38,

TCPsrcPort

52255,

TCPdstPort

443 ….

IPsrc

40.103.10.38,

IPdst

192.168.1.1,

TCPsrcPort

443,

TCPdstPort

52255 ….

Bidirectional

Flow-based

splitting

5056C05A75085….

A81406C0A900….

Hex number

representation

5056 C05A 7508 ….

A814 06C0 A900 ….

Tokenization

Token

Embedding

Position

Embedding

Src-to-dst

…

…

…

…

…

…

𝐸

!

𝐸

!

𝐸

"

𝐸

"

𝐸

"

Segment

Embedding

Dst-to-src

𝐸

#

𝐸

$

𝐸

%&'

𝐸

%

𝐸

%('

𝐸

)

𝐸

*+*,

𝐸

-+*.

𝐸

[!01]

𝐸

[%.!3]

𝐸

*+*,

𝐸

1."

ℎ

["#$]

𝐸

[-4!]

𝐸

'

𝐸

!

ℎ

["#$]

ℎ

&'&(

ℎ

&'&(

ℎ

"'&)

ℎ

"'&)

ℎ

[$*+]

ℎ

[$*+]

ℎ

[,)$-]

ℎ

&'&(

ℎ

&'&(

ℎ

[,)$-]

ℎ

[+).]

ℎ

[+).]

…

…

…

…

ℎ

[-4!]

ℎ

*+*,

ℎ

-+*.

ℎ

[!01]

ℎ

*+*,

ℎ

[%.!3]

ℎ

[1."]

…

…

BERT

ℎ

[-4!]

ℎ

*+*,

ℎ

-+*.

ℎ

[!01]

ℎ

*+*,

ℎ

[%.!3]

ℎ

[1."]

…

…

Masked Unidirectional

Same Sequence-

origin Prediction

Normal

packet

sequences

…

Malicious

packet

sequences

Packet Seq.

Concatenation

Contextually

relevant pairing

𝑧

!

𝑧

"

𝑧

#$%

𝑧

#

…

…

𝑝

&

(𝑧

#$%

|𝑧

#

)

𝑞(𝑧

#

|𝑧

#$%

)

Forward

Process

Reverse

Process

𝑆

'()

Normal (benign)

Packet Seq.

𝑆

*+,

Malicious

Packet Seq.

Targeted Noising

𝐸

"

traffic for testing and analysis. Despite these advancements, combining the contextual understanding of pre-trained models with the generative prowess of diffusion models remains underexplored in adversarial packet generation. This integration offers a synergistic approach, merging the nuanced comprehension of network behaviors by pre-trained models with the high-quality data generation of diffusion models. By leveraging these technologies together, cybersecurity researchers can enhance the realism of simulated network environments and develop more advanced evasion tactics.

3 DiffuPac

In this section, we introduce DiffuPac, a first-of-its kind adversarial packet generation model. Figure 1 illustrates the overall framework of the proposed model, consisting of three main phases: pcap pre-processing, pre-training and fine-tuning with diffusion model.

3.1 Data pre-processing

In real network environments, traffic contains diverse flows from various applications, protocols, and services, complicating the learning of stable representations. Therefore, we first split pcap (packet capture) files into sessions (bidirectional flows) based on IP addresses, port addresses, and protocols. To refine the training of the BERT model, we further split sessions into unidirectional flows, categorizing them as either source-to-destination (src-to-dst) packet sequence or destination-tosource (dst-to-src) packet sequence. This categorization is crucial for the BERT model's pre-training. Network traffic varies due to diverse protocols and network services, resulting in different formats and patterns. To handle these variations and encoding requirements, we convert each byte to its corresponding hex number and tokenize using WordPiece (Wu et al. [2016]). Each token ranges from 0 to 65535, with a dictionary size of 65536. We also incorporate special tokens ❬❈▲❙❪ , ❬❙❊/a80❪ , ❬/a80❆❉❪ , and ❬▼❆❙❑❪ for training tasks. ❬❈▲❙❪ is used at the beginning of each sequence and helps in classification tasks. ❬❙❊/a80❪ separates different sequences or segments within the same input. ❬/a80❆❉❪ ensures sequences are of uniform length and satisfy the minimum length requirement. ❬▼❆❙❑❪ is used in pre-training task, where it temporarily substitutes tokens to be predicted.

3.2 Pre-training

As shown in Figure 1, the input tokens are processed using an embedding strategy that involves the sum of three types of embeddings: token embeddings, positional embeddings, and segment embeddings. Each embedding has a dimension of 768. Token Embeddings are high-dimensional

vectors that uniquely represent each token, acting as their exclusive identifiers. Positional Embeddings are used to capture the temporal relationships of tokens, ensuring that the model learns to focus on the order of data transmission. Segment Embeddings differentiate packets within a single flow, as packets may not inherently share semantic associations, and preserving the order of packets to maintain the temporal sequence of events in a session flow.

We design our pre-training tasks based on the approach described in Devlin et al. [2019]. Our two proposed pre-training tasks aim to capture the contextual relationships between traffic bytes. The first task involves predicting masked tokens to learn the underlying patterns and dependencies within the traffic data. The second task predicts the transmission order by determining whether packets belong to src-to-dst or dst-to-src sequences, thereby capturing the directional flow of the network traffic.

Masked Unidirectional Flow Model . Inspired by BERT's Masked Language Model (MLM) in natural language processing, we adapt this approach for network traffic analysis through our Masked Unidirectional Flow Model. This model is designed to understand and predict the semantic patterns within bidirectional network flows; that are src-to-dst sequence and dst-to-src sequence. During pre-training, each token in the input sequence is masked with a probability of 15%. Amongst these masked tokens 80% are replaced with a ❬▼❆❙❑❪ token, 10% are replaced with a random token from the vocabulary, and 10% are left unchanged. This introduces variability that mimics real-world data inconsistencies. The training objective is to minimize the negative log-likelihood of correctly predicting the original tokens at the masked positions. Formally, the loss function L MUM for this task is defined as:

where n is the total number of tokens in the sequence, 1( m i ) is an indicator function that is 1 if the token v i was masked (0 otherwise). m i indicates the masking status of the i -th token and V masked is the masked sequence. v i is the actual token at position i and P ( v i | V masked ; θ ) is the probability of predicting the original token v i for given masked sequence and model parameters θ . The transformer encoder, characteristic of BERT, processes V masked to predict each masked token. This architecture leverages self-attention mechanisms to capture dependencies and context effectively, which is crucial for understanding complex patterns in network traffic.

Same Sequence-origin Prediction . Inspired by Lin et al. [2022], this task employs a dedicated binary classifier to determine the directional origin of network traffic, specifically whether packets in a sequence originate from src-to-dst or dst-to-src. This classification enhances the model's understanding of network flows and the contextual relationships between packets, which is crucial for recognizing communication patterns and dependencies in network traffic. For this task, each packet a r is paired with another packet a s . 50% of the time, a s is the next logical packet in the flow (src-to-dst or dst-to-src); the other 50% of the time, it is a randomly chosen packet from the opposite flow. The classifier then predicts whether a s follows a r in the correct directional sequence. This pairing strategy improves the model's capability to discern patterns in packet flows. Let A = { ( a r , a s ) } be the set of packet pairs, where each pair is labeled with b w ∈ { 0 , 1 } (1 if a s follows a r in the correct flow, 0 otherwise). The loss function L SSP for this task can be formulated as:

where K is the total number of packet pairs and θ represents the trainable parameters of the classifier. P ( b w = 1 | a r , a s ; θ ) is the probability predicted by the classifier that a s correctly follows a r in the given network flow direction.

In summary, the final pre-training objective is the sum of the above two losses, which can defined as:

Fu et al. [2021] mentioned that the number of packets within each flow can vary significantly. Given the constraints imposed by packet size and the potential volume of network traffic, computational efficiency is a paramount concern. In addressing this problems, as demonstrated in Dai et al. [2023] that the initial packets in a flow contain the most significant information, we limit our analysis to the first three packets of each heavy flow. This means that for each session, we analyze a total of six packets-three from the src-to-dst flow and three from the dst-to-src flow. This strategy ensures a comprehensive view of the session while maintaining efficiency.

3.3 Fine-tuning with Diffusion Models

Forward Process with Packet Sequences Concatenation Strategy . In the fine-tuning phase, the goal is to train the model so that malicious packets can mimic normal packets to bypass NIDS. This involves using packet sequences that are only from the src-to-dst, reflecting the adversary's control over the packets being sent (Hore et al. [2023]). The forward process of our diffusion model begins by embedding both normal (benign) packet sequences S ben and malicious packet sequences S mal . This transformation of discrete packet data into a continuous feature spaces uses an embedding function adapted from Li et al. [2022]. Building upon the groundwork of DiffuSeq (Gong et al. [2023]) approach, which typically involves random merging, our model innovates by leveraging the deep contextual insights provided by the pre-trained BERT model (Details in A.2). This allows us to strategically pair normal and malicious sequences with strong contextual alignments that show similarity in network behavior patterns. These contextually aligned pairs are crucial for mimicking normal traffic and increasing the chances of bypassing NIDS. By integrating these contextually aligned pairs into our diffusion process, the model extends the original forward chain to a new Markov transition q ϕ ( z 0 | S ben ⊕ mal ) = N ( EMB ( S ben ⊕ mal ) , β 0 I ) , where EMB ( S ben ⊕ mal ) symbolizes the embedding transformation and concatenation of normal and malicious packet sequences. While S ben ⊕ mal denotes the initial concatenated sequence, the forward process gradually perturbs this initial state z 0 through a series of transitions, producing latent variables z 1 , z 2 , . . . , z t .

Targeted Noising . To enhance our diffusion model's capability for adversarial packet generation, we simplify the model's state transitions by defining z t = x t ⊕ y t , where x t and y t correspond to the portions of z t that belong to S ben and S mal , respectively. This setup allows us to strategically inject noise into only the malicious packet sequences (represented by y t ), rather than the entire state z t , during each forward step q ( z t | z t -1 ) . This targeted noising, inspired by DiffuSeq, is pivotal for adapting conventional diffusion models for targeted modification.

Reverse Process With Normal Packet Guidance . Building upon the foundational principles of DiffuSeq, our model introduces an innovative reverse process tailored for more nuanced handling of network traffic data. This process distinctively utilizes normal packet sequences as a guiding framework, enabling a sophisticated denoising technique that treats the concatenated sequences of normal and noise-added malicious packets as a unified unit. This approach effectively 'teaches' the model to perceive these malicious elements as integral parts of the normal traffic pattern. A key aspect of this reverse process is the use of the pre-trained BERT model as the denoising engine during packet reconstruction. In this phase, the BERT model undergoes fine-tuning within the diffusion framework, ensuring that it is specifically adapted for reconstructing packets. As a result, the BERT model is not only responsible for contextual understanding but also plays a central role in recovering malicious packets that have been integrated into normal traffic patterns. The conditional denoising process effectively employs Bayesian inference to parameterize the transition probabilities between states, ensuring precise control over each step in the reverse process. These parameterizations are mathematically articulated through the equations: p θ ( z 0: T ) := p ( z T ) ∏ T t =1 p θ ( z t -1 | z t ) and p θ ( z t -1 | z t ) = N ( z t -1 ; µ θ ( z t , t ) , σ θ ( z t , t ) , where µ θ ( · ) and σ θ ( · ) model the mean and variance necessary for the stochastic reverse transition from z t to z t -1 , respectively. To rigorously ensure that the malicious packets are not only recovered but also convincingly mimic normal packets, our model optimizes a specially formulated variational lower bound L V LB . This objective underscores the model's effectiveness in blending malicious packets within normal packets:

This objective is particularly focused on accurately reconstructing the initial state z 0 from the noised states, with a distinct emphasis on ensuring that the malicious components are seamlessly integrated into the normal packets pattern.

Preserving Packets' Integrity . We propose an innovative approach that utilizes a parallel data structure called 'model dictionary' that clearly distinguishes between mutable and immutable fields within each packet. By categorizing fields into mutable (e.g. TCP flags, TTL, and window size) and immutable (including the critical 5-tuple information and payload), we ensure that modifications during training do not compromise the packet's integrity. The model dictionary serves a dual purpose: it retains the original values of immutable fields and establishes a link to their mutable counterparts. This approach allows for dynamic modification of mutable fields during training without affecting

the core attributes of the packet. After training, any changes made to mutable fields are seamlessly integrated with the preserved immutable fields. This recombination ensures that the modified packets maintain their operational integrity and are indistinguishable from genuine traffic in real-world scenarios. By bypassing traditional domain constraints-which typically complicate training and pose convergence challenges-our method simplifies the training process and obviates the need for additional compensatory loss functions (Letcher [2021]). The outcome is a highly effective generation of adversarial packets that are capable of evading detection without compromising the essential characteristics of the original packets.

4 Experiments

We conduct experiments to validate the performance of DiffuPac on 6 types of attacks, against 6 classifiers.

4.1 Experimental Setup

Dataset . The datasets used for this model are Kitsune Dataset (Mirsky et al. [2018]) and CICIDS2017 Dataset (Sharafaldin et al. [2018]). Initially, pre-training of the BERT model utilizes a large subset of unlabeled network traffic to leverage the model's capability to capture diverse traffic patterns, accounting for 60% of the total data. Fine-tuning phase , the focus shifts to a smaller, labeled dataset, which constitutes 20% of the total data. This dataset is distinctly partitioned into malicious and normal packets, which is crucial for training the model to mimic malicious packets as normal. Training the classifier and NIDS then utilizes another 10% of the total data consisting of labeled portion. Testing phase is conducted with the remaining 10% of the data, reserved exclusively for evaluating the model's efficacy. This phase includes testing both original and mimicked malicious packets to rigorously assess the model's real-world applicability and its capability to generalize across unseen data.

Baseline Models . We evaluate two baseline models-Traffic Manipulator (Han et al. [2021]) and TANTRA (Sharon et al. [2021])-both of which represent current strategies for evading NIDS but share a significant limitation in their reliance on specific operational assumptions. Traffic Manipulator formulates the evasions as a bi-level optimization problem, where the first level uses GAN to identify adversarial features closely mimicking the original network features, and the second level utilizes PSO to modify packet characteristics to exploit vulnerabilities in NIDS. TANTRA employs a three-step process involving an LSTM trained to model benign traffic by predicting inter-packet delays. This LSTM then adjusts malicious traffic to mimic these benign patterns before deployment. Both models, however, assume a degree of accessibility to NIDS configurations or feature extractors that is often unrealistic in practical scenarios. This reliance on specific knowledge about NIDS configurations creates a critical dependency that can limit the deployment and scalability of these evasion techniques in diverse operational environments. Conversely, DiffuPac operates under the assumption of zero prior knowledge about NIDS configurations or feature extractors.

Implementation Details . DiffuPac leverages a BERT model with 12 transformer blocks, each featuring 12 attention heads and an embedding dimension of 768, to capture nuanced relationships in network traffic. This setup supports the maximum sequence length of 512. The diffusion model incorporates time step embedding to provide temporal context, enhancing the model's understanding of the reverse process. To manage the computational demands, particularly during the sampling phase, a projection layer reduces the embedding dimension to 128. Additionally, DiffuPac utilizes the FAISS library (Douze et al. [2024]), renowned for its capability to handle large-scale similarity search and clustering of dense vectors.

Experimental Details . Our experimental framework integrates advanced feature extraction tools and a diverse set of machine learning classifiers, building on methodologies established in the Traffic Manipulator research. We employ 2 feature extractors: AfterImage (Mirsky et al. [2018]), which provides detailed packet statistics, and CICFlowMeter (Draper-Gil et al. [2016]), which assesses connection-level metrics. Recognizing the documented shortcomings of the original CICFlowMeter, our research employs a revised version that integrates enhancements and corrections proposed in recent studies (Engelen et al. [2021], Liu et al. [2022]). These modifications are crucial in ensuring more accurate and reliable feature extraction for our analysis. AfterImage and CICFlowMeter offer a comprehensive view of network traffic, capturing both packet-level and flow-level data essential

(a) Botnet

for nuanced analysis. These feature extractors are integral to NIDS detection as they preprocess network data into structured features that are then analyzed by ML classifiers. This preprocessing step is crucial for transforming raw network traffic into a format that classifiers can effectively interpret, thereby enhancing the accuracy of the anomaly detection. We utilize a variety of machine learning classifiers for anomaly detection, including KitNET (Mirsky et al. [2018]), an ensemble of autoencoders; Multi-Layer Perceptron (MLP) for deep learning; Logistics Regression (LR) , Decision Tree (DT) , and Support Vector Machine (SVM) for traditional approaches; and Isolation Forest (IF) for outlier detection. Our experiments cover 6 types of attacksMan-in-the-Middle (MITM) , Botnet , Brute Force , DDoS , Port Scan , and Infiltration -utilizing data from Kitsune and the CICIDS-2017 dataset.

Evaluation . To assess the effectiveness of the mimicked packets in evading the 6 classifiers, we employed a singular, highly illustrative metric from the Traffic Manipulator: the Malicious traffic Evasion Rate (MER). This metric is calculated using the formula: MER = 1 -( N adv /N mal ) , where N adv and N mal represent the number of detected adversarial and detected malicious packet sequences, respectively. This equation captures the percentage of adversarial packet sequences that goes undetected as compared to the original malicious packet sequences that was detected, effectively measuring the evasion capability of the adversarial packet sequences.

We employed the two-sample Kolmogorov-Smirnov (K-S) test, a powerful non-parametric method used to determine the probabilistic differences between two data samples. Specifically, we compare the empirical cumulative distribution functions (eCDFs) of both the original malicious packets and the adversarial packets generated by our model. The K-S test calculates the maximum distance (K-S statistic, D ) between these two eCDFs. This statistic measures the greatest deviation between the distribution of our adversarially modified packets and the original malicious packets (Hore et al. [2023]; Gretton et al. [2012]). A D value exceeding the critical threshold at a chosen significance level suggests a rejection of the null hypothesis-that is, the two samples are not derive from the same distribution. We demonstrated adversarial packets that were found to be out-of-distribution (OOD) at 95% significance level.

Additionally, we used Wireshark, a widely recognized network protocol analyzer to ensure that during the modifications were made, the immutable fields crucial for the packet's attack functionality were preserved. This visual verification via Wireshark confirms that our adversarial packets maintain their structural integrity and functionality, successfully mimicking genuine network traffic while evading detection (Due to limited space, results and analysis are in Appendix B.1).

We also evaluated DiffuPac's efficacy in generating adversarial packets that maintain their malicious functionality in a controlled test environment using UTM hypervisor technology. This setup included two virtual machines: one running Kali Linux (attacker) and another running Ubuntu 24.04 (victim). The isolation ensured a realistic yet controlled network environment. We conducted the evaluation on two type of attacks: Port Scan and Brute Force . (Due to space constraints, results and analysis of the two attacks are in Appendix B.3.1 and B.3.2).

4.2 Results and Analysis

Evasion Rate . We evaluated the evasion rates of the 6 attacks using DiffuPac and two baseline models, and compared the results in Table 2 (Due to space constraints, the other 4 attacks are demonstrated in Appendix B.2). As shown in Table 2, we conclude that DiffuPac is capable of generating adversarial packets that achieves comparable or even higher in evasion rate compared with both Traffic Manipulator (GAN & PSO) and TANTRA (LSTM). Importantly, DiffuPac achieves this high performance without relying on NIDS components, surrogate classifiers or specific insights into the NIDS's feature extraction methods, highlighting its effectiveness in realistic settings where attackers lack access to such insider information. The evasion performance varied significantly across different classifiers, reflecting the inherent variability in their robustness and detection capabilities.

Flow-based NIDS exhibited greater resilience against our attacks compared to packet-based NIDS, likely due to DiffuPac's focus on packet-level modifications. Interestingly, the Traffic Manipulator model performed well with KitNet's AfterImage feature extractor, likely because it was trained directly on these features, enhancing its capability to modify them to evade detection. This specific alignment with NIDS features, while effective, does not typically reflect real-world attacker capabilities or constraints.

TANTRAalso demonstrated high evasion rates, particularly with AfterImage, thanks to its use of LSTM models trained on benign traffic from the targeted network. This training allows TANTRA to reshape the timing between packets.

This specific adjustment of interpacket delays directly impacts the time-based features extracted by AfterImage, making it more difficult for the classifiers to distinguish between normal and malicious packets.

Quantitatively, DiffuPac outperformed Traffic Manipulator by an average of 9.12 percentage points and TANTRA by an average of 4.26 percentage points across all attack types. The overall average improvement of DiffuPac over both baselines is 6.69 percentage points, demonstrating its superior capability to generate adversarial packets that effectively evade detection.

Based on the results of all the attacks, we can conclude that ML classifier are more robust than DNNs to adversarial attacks. Traditional ML models, like DT and IF, have simpler and more interpretable decision boundaries, making them less susceptible to subtle adversarial modifications (Sauka et al. [2022]). In contrast, DNNs with their complex and high-dimensional decision boundaries are more easily misled by the nuanced modifications introduced by DiffuPac.

Statistical Difference (K-S test) . The percentage of successful adversarial packets that were found to be OOD are demonstrated in Table 3. From the result, we can obeserve that Brute Force and DDoS attacks show the highest percentage of OOD samples. The grounded reasons for these kind of attacks are due to the inherent nature of their traffic patterns. The nature of Brute Force attacks involves numerous failed login attempts, leading to highly irregular sequence numbers (Javed and Paxson [2013]). Even after modifications by DiffuPac, these attempts will stand out due to their frequency and pattern, resulting in significant deviations in the eCDFs. The sheer volume and repetitive nature of DDoS traffic (Haseeb-ur rehman et al. [2023]) make it difficult to disguise effectively. Despite efforts of intelligent mutation process, the high volume and distinctive traffic patterns will cause substantial deviations in the eCDFs as well. On the other hand, the MITM show the lowest percentage of OOD samples. Adjusting TTL values is straightforward as they can be set to typical ranges seen in normal packets without significantly altering packet behavior, resulting in minimal deviation in the eCDFs.

5 Limitations

Our evaluations demonstrated DiffuPac's strong evasion capabilities, enabling the generation of adversarial packets that convincingly mimic legitimate network traffic, particularly in packet header fields. While these results highlight DiffuPac's ability to evade detection by NIDS, there remain areas for further study.

In particular, while we assessed the malicious functionality of generated adversarial packets in a controlled environment using UTM hypervisor technology, this evaluation was limited to Port Scan and Brute Force attacks. Although DiffuPac successfully retained the malicious intent of these attacks, further evaluations are required to determine its effectiveness across a broader range of attacks, especially those with more complex behaviors such as DDoS or Botnet attacks.

We also noted that DiffuPac's evasion performance is not uniform across all scenarios, notably DDoS and Brute Force. This variability is due to the distinct nature of each attack's traffic patterns, which may not be fully captured by the model, especially in highly repetitive or anomalous behaviors. To enhance the model's adaptability and generalizability, we plan to enrich the training dataset with a broader spectrum of real-world attack scenarios.

Moreover, the dual-use nature of adversarial generation models like DiffuPac presents significant ethical and legal challenges. While designed to improve security defenses, these technologies could be misused for malicious activities, such as facilitating cyber-attacks or disrupting services. To mitigate these risks, we have opted not to publicly release saved model checkpoints, aiming to prevent exploitation by malicious entities and ensure that our advancements in adversarial packet generation are used responsibly and within ethical bounds.

6 Conclusion and Future Directions

In this study, we introduced DiffuPac, an intelligent generative model that successfully generates adversarial packets capable of evading advanced NIDS while maintaining attack functionality. Unlike previous research, which often assumes attacker access to NIDS, DiffuPac operates under constraints of limited attacker knowledge, reflecting more realistic scenarios. Here, we present the in-depths insights obtained from this study: (a) DiffuPac uniquely combines normal and malicious packet sequences using contextual alignments, ensuring seamless integration into genuine traffic while employing these normal packet sequences to guide the denoising process. Through the performance evaluations with various NIDS, our model achieved an average improvement of approximately 6.69 percentage points in evasion rate compared to the two baselines across all attack types. (b) Evasion rates varied notably across attack types, with DDoS and Brute Force attacks showing higher probabilistic differences due to their complex, repetitive nature. This highlights potential areas for further refinement in DiffuPac's approach to handling voluminous attacks. (c) Simpler ML models like DT and IF displayed surprising resilience due to their less complex decision boundaries, limiting the effectiveness of DiffuPac's modifications. In contrast, DNNs, with their intricate decision boundaries, were more vulnerable, underscoring the complexities inherent in designing robust adversarial tactics. (d) DiffuPac's capability to balance sophisticated attacks with operational stealth makes it especially suitable for environments where attackers lack comprehensive NIDS configurations, enhancing its utility in realistic defense testing.

As future studies, we will compare DiffuPac against a broader generative adversarial packets model and extend testing across more diverse NIDS to better understand its relative strengths and limitations. Next, we will further expand the range of attack types included in the malicious functionality evaluation. Moreover, adversarial defenses are a key focus of our ongoing research, particularly in relation to DiffuPac's capabilities. One promising direction is the development of adversarial purification using diffusion models, a technique successfully applied in image processing. This approach treats adversarial perturbations as noise, utilizing the diffusion model's reverse process to restore network packets to their original state before analysis by NIDS. We believe this method has significant potential for enhancing network security and aim to further develop and test it as a novel defense strategy.

References

Statista. Number of Internet of Things (IoT) connected devices worldwide from 2019 to 2023, with forecasts from 2022 to 2030. ❤/a116/a116♣/a115✿✴✴✇✇✇✳/a115/a116❛/a116✐/a115/a116❛✳❝♦♠✴/a115/a116❛/a116✐/a115/a116✐❝/a115✴✶✶✽✸✹✺✼✴ ✐♦/a116✲❝♦♥♥❡❝/a116❡❞✲❞❡✈✐❝❡/a115✲✇♦/a114❧❞✇✐❞❡✴ , 2023. Accessed: 2024-4-20.

Tala Talaei Khoei and Naima Kaabouch. Machine learning: Models, challenges, and research directions. Future Internet , 15(10):332, 2023. doi: 10.3390/fi15100332.

Tala Talaei Khoei, Hadjar Ould Slimane, and Naima Kaabouch. Deep learning: Systematic review, models, challenges, and research directions. Neural Computing and Applications , 35(31):2310323124, 2023. doi: 10.1007/s00521-023-08957-4.

Olakunle Ibitoye, Rana Abou Khamis, Ashraf Matrawy, and M. Omair Shafiq. The threat of adversarial attacks on machine learning in network security - A survey. arXiv , 2019. URL ❤/a116/a116♣✿✴✴❛/a114①✐✈✳♦/a114❣✴❛❜/a115✴✶✾✶✶✳✵✷✻✷✶ .

Hassan Khazane, Mohammed Ridouani, Fatima Salahdine, and Naima Kaabouch. A holistic review of machine learning adversarial attacks in IoT networks. Future Internet , 16(1):32, 2024. doi: 10.3390/fi16010032.

Ivan Homoliak, Martin Teknøs, Martîn Ochoa, Dominik Breitenbacher, Saeid Hosseini, and Petr Hanacek. Improving network intrusion detection classifiers by non-payload-based exploitindependent obfuscations: An adversarial approach. EAI Endorsed Transactions on Security and Safety , 5(17), 12 2018. doi: 10.4108/eai.10-1-2019.156245.

Mohammad J. Hashemi, Greg Cusack, and Eric Keller. Towards evaluation of nidss in adversarial setting. In Proceedings of the 3rd ACM CoNEXT Workshop on Big DAta, Machine Learning and Artificial Intelligence for Data Communication Networks , 2019.

Aditya Kuppa, Slawomir Grzonkowski, Muhammad Rizwan Asghar, and Nhien-An Le-Khac. Black box attacks on deep anomaly detectors. In Proceedings of the 14th International Conference on Availability, Reliability and Security (ARES 2019) , 2019.

Dongqi Han, Zhiliang Wang, Ying Zhong, Wenqi Chen, Jiahai Yang, Shuqiang Lu, Xingang Shi, and Xia Yin. Evaluating and improving adversarial robustness of machine learning-based network intrusion detectors. IEEE Journal on Selected Areas in Communications , 39(8):2632-2647, 2021. doi: 10.1109/JSAC.2021.3087242.

Yam Sharon, David Berend, Yang Liu, Asaf Shabtai, and Yuval Elovici. TANTRA: Timing-based adversarial network traffic reshaping attack. IEEE Transactions on Information Forensics and Security , 17:3225-3237, 2021. doi: 10.1109/TIFS.2022.3201377.

Soumyadeep Hore, Jalal Ghadermazi, Diwas Paudel, Ankit Shah, Tapas K. Das, and Nathaniel D. Bastian. Deep PackGen: A deep reinforcement learning framework for adversarial network packet generation. arXiv , 2023. URL ❤/a116/a116♣/a115✿✴✴❛/a114①✐✈✳♦/a114❣✴❛❜/a115✴✷✸✵✺✳✶✶✵✸✾ .

Zeeshan Ahmad, Adnan Shahid Khan, Cheah Wai Shiang, Johari Abdullah, and Farhan Ahmad. Network intrusion detection system: A systematic study of machine learning and deep learning approaches. Transactions on Emerging Telecommunications Technologies , 32(1):e4150, 2021. doi: https://doi.org/10.1002/ett.4150.

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In Yoshua Bengio and Yann LeCun, editors, Proceedings of the 2nd International Conference on Learning Representations (ICLR 2014) , 2014.

Andrew McCarthy, Essam Ghadafi, Panagiotis Andriotis, and Phil Legg. Functionality-preserving adversarial machine learning for robust classification in cybersecurity and intrusion detection domains: A survey. Journal of Cybersecurity and Privacy , 2(1):154-190, 2022. doi: 10.3390/ jcp2010010.

Kaichen Yang, Jianqing Liu, Chi Zhang, and Yuguang Fang. Adversarial examples against the deep learning based network intrusion detection systems. In Proceedings of 2018 IEEE Military Communications Conference (MILCOM 2018) , pages 559-564, 2018. doi: 10.1109/MILCOM. 2018.8599759.

Ryan Sheatsley, Nicolas Papernot, Michael J. Weisman, Gunjan Verma, and Patrick McDaniel. Adversarial examples for network intrusion detection systems. Journal of Computer Security , 30 (5):727-752, 2022. doi: 10.3233/JCS-210094.

Ke He, Dan Dongseong Kim, and Muhammad Rizwan Asghar. Adversarial machine learning for network intrusion detection systems: A comprehensive survey. IEEE Communications Surveys & Tutorials , 25(1):538-566, 2023. doi: 10.1109/COMST.2022.3233793.

Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning , volume 37 of Proceedings of Machine Learning Research , pages 2256-2265, Lille, France, 07-09 Jul 2015. PMLR.

Jonathan Ho, Ajay Jain, and P. Abbeel. Denoising diffusion probabilistic models. In Proceedings of Advances in Neural Information Processing Systems 33 (NeurIPS 2020) , pages 6840-6851, 2020.

Xinjie Lin, Gang Xiong, Gaopeng Gou, Zhen Li, Junzheng Shi, and Jing Yu. Et-bert: A contextualized datagram representation with pre-training transformers for encrypted traffic classification. In Proceedings of the ACM Web Conference 2022 (WWW 2022) . ACM, 2022. doi: 10.1145/3485447. 3512217.

Xuying Meng, Chungang Lin, Yequan Wang, and Yujun Zhang. Netgpt: Generative pretrained transformer for network traffic. arXiv , 2023. URL ❤/a116/a116♣/a115✿✴✴❛/a114①✐✈✳♦/a114❣✴❛❜/a115✴✷✸✵✹✳✵✾✺✶✸ .

Yonghui Wu, Mike Schuster, Z. Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason R. Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Gregory S. Corrado, Macduff Hughes, and Jeffrey Dean. Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv , 2016. URL ❤/a116/a116♣/a115✿✴✴❛/a114①✐✈✳♦/a114❣✴❛❜/a115✴✶✻✵✾✳✵✽✶✹✹ .

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 17th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019) , pages 4171-4186. ACL, 2019. doi: 10. 18653/v1/N19-1423.

Chuanpu Fu, Qi Li, Meng Shen, and Ke Xu. Realtime robust malicious traffic detection via frequency domain analysis. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (CCS 2021) , pages 3431-3446. ACM, 2021. doi: 10.1145/3460120.3484585.

Jianbang Dai, Xiaolong Xu, and Fu Xiao. Glads: A global-local attention data selection model for multimodal multitask encrypted traffic classification of IoT. Computer Networks , 225:109652, 2023. doi: https://doi.org/10.1016/j.comnet.2023.109652.

Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. DiffusionLM improves controllable text generation. In Proceedings of Advances in Neural Information Processing Systems 35 (NeurIPS 2022) , pages 4328-4343, 2022.

Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. DiffuSeq: Sequence to sequence text generation with diffusion models. In Proceedings of the 11th International Conference on Learning Representations (ICLR 2023) , 2023.

Alistair Letcher. On the impossibility of global convergence in multi-loss optimization. arXiv , 2021. URL ❤/a116/a116♣/a115✿✴✴❛/a114①✐✈✳♦/a114❣✴❛❜/a115✴✷✵✵✺✳✶✷✻✹✾ .

Yisroel Mirsky, Tomer Doitshman, Yuval Elovici, and Asaf Shabtai. Kitsune: An ensemble of autoencoders for online network intrusion detection. In Proceedings 2018 Network and Distributed System Security Symposium (NDSS 2018) , 2018.

Iman Sharafaldin, Arash Habibi Lashkari, and Ali A. Ghorbani. Toward generating a new intrusion detection dataset and intrusion traffic characterization. In Proceedings of the 4th International Conference on Information Systems Security and Privacy (ICISSP 2018) , pages 108-116. INSTICC, SciTePress, 2018. doi: 10.5220/0006639801080116.

Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. The faiss library. arXiv , 2024. URL ❤/a116/a116♣/a115✿✴✴❛/a114①✐✈✳♦/a114❣✴❛❜/a115✴✷✹✵✶✳✵✽✷✽✶ .

Gerard Draper-Gil, Arash Habibi Lashkari, Mohammad Saiful Islam Mamun, and Ali A. Ghorbani. Characterization of encrypted and vpn traffic using time-related features. In International Conference on Information Systems Security and Privacy (ICISSP 2016) , 2016. doi: 10.5220/0005740704070414.

Gints Engelen, Vera Rimmer, and Wouter Joosen. Troubleshooting an intrusion detection dataset: the cicids2017 case study. In 2021 IEEE Security and Privacy Workshops (SPW) , pages 7-12, 2021. doi: 10.1109/SPW53761.2021.00009.

Lisa Liu, Gints Engelen, Timothy Lynar, Daryl Essam, and Wouter Joosen. Error prevalence in nids datasets: A case study on cic-ids-2017 and cse-cic-ids-2018. In 2022 IEEE Conference on Communications and Network Security (CNS) , pages 254-262, 2022. doi: 10.1109/CNS56114. 2022.9947235.

Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research , 13:723-773, 2012.

Kudzai Sauka, Gun-Yoo Shin, Dong-Wook Kim, and Myung-Mook Han. Adversarial robust and explainable network intrusion detection systems based on deep learning. Applied Sciences , 12(13), 2022. doi: 10.3390/app12136451.

Mobin Javed and Vern Paxson. Detecting stealthy, distributed SSH brute-forcing. In Proceedings of the 2013 ACM SIGSAC Conference on Computer and Communications Security (CCS 2013) , pages 85-96, New York, NY, USA, 2013. ACM. doi: 10.1145/2508859.2516719.

Rana M. Abdul Haseeb-ur rehman, Azana Hafizah Mohd Aman, Mohammad Kamrul Hasan, Khairul Akram Zainol Ariffin, Abdallah Namoun, Ali Tufail, and Ki-Hyung Kim. High-speed network ddos attack detection: A survey. Sensors , 23(15), 2023. doi: 10.3390/s23156850.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of Advances in Neural Information Processing Systems 30 (NIPS 2017) , pages 6000-6010, 2017.

Zhe Zhao, Hui Chen, Jinbin Zhang, Xin Zhao, Tao Liu, Wei Lu, Xi Chen, Haotang Deng, Qi Ju, and Xiaoyong Du. Uer: An open-source toolkit for pre-training models. EMNLP-IJCNLP 2019 , page 241, 2019.

A Model Architecture

A.1 BERT

The core of DiffuPac is the BERT model, specifically adapted to serve as the denoising component in the diffusion process. The architecture consists of 12 bidirectional Transformer blocks and each block incorporates 12 attention heads within its self-attention layers (Vaswani et al. [2017]). These layers are crucial for capturing the intricate and implicit relationships between traffic bytes. Each input token has a dimension of 768, and the model can process sequences with up to 512 tokens. This setup allows the BERT model to effectively handle extensive sequences of traffic data, making it well-suited for the task of denoising during the diffusion process.

To implement of our pre-trained BERT model, we utilized the codebase from the UER (Universal Encoder Representations)-py project (Zhao et al. [2019]). UER-py is a versatile toolkit designed for pre-training and fine-tuning various NLP models, including BERT. In UER-py, the MLM is designed to predict masked words within a sentence. We adapted this approach to our masked unidirectional flow model, which focuses on predicting masked tokens within network traffic sequences. Instead of textual tokens, our model deals with traffic units from src-to-dst and dst-to-src sequences. We modified the MLM implementation to handle these traffic tokens, ensuring that each token is masked with a probability of 15%. Our model then predicts the original token from the masked sequence, capturing semantic patterns within bidirectional network flows.

The Next Sentence Prediction (NSP) task in UER-py involves prediction step to determine whether one sentence follows another. We transformed this concept into the same sequence-origin prediction task to determine the origin of directional network traffic. We adapted the binary classifier used in NSP to classify whether packets are originated from src-to-dst or dst-to-src. By pairing packets and predicting the directional flow, our model learns the contextual relationships and patterns in network traffic, similar to how NSP learns sentence relationships.

Table 4 lists the parameters are set to run our pre-trained BERT model. The remaining parameters were set to their default values.

We also conducted a comprehensive evaluation of different BERT model configurations to identify the most efficient option that still maintains high performance. We tested four BERT model variants: Tiny, Small, Medium, and Large, comparing their specifications, training time, loss, and accuracy. The results are summarized in Table 5. The Medium BERT model was chosen as the optimal configuration for DiffuPac because it offers a balanced trade-off between accuracy and loss metrics.

A.2 Fine-tuning with Diffusion Model

For the fine-tuning phase of our model, we leveraged the DiffuSeq codebase (DiffuSeq's code). The DiffuSeq framework, originally designed for text data, required several significant adaptations to handle network traffic data and to meet the objectives of our diffusion model for adversarial packet generation. The primary adaptation to the DiffuSeq codebase involves the data preparation phase,

specifically the concatenation strategy used to blend normal and malicious packet sequences. This adaptation is crucial for the model to learn how to generate adversarial packets that can mimic normal packets and evade detection by NIDS.

In our concatenation strategy, we identified pairs of normal and malicious packet sequences that exhibit strong contextual alignments unlike DiffuSeq, which, randomly merge the text sequences. This means that the patterns and behaviors within these sequences are similar, ensuring that the concatenated sequences are blended seamlessly. We concatenated these contextually aligned normal and malicious packet sequences to form a new sequence, S ben ⊕ mal . The strategy leverages the inherent patterns and behaviors within normal packet sequences, which is crucial for the mimicking process of malicious packet sequences.

We utilized the pre-trained BERT embeddings in our concatenation strategy. BERT is designed to capture deep semantic relationships within sequences. This design makes BERT making it highly effective for analyzing complex patterns in network traffic. By employing the pre-trained embeddings, DiffuPac can accurately transform discrete packet data into continuous feature spaces, capturing the intricate dependencies between packets. This transformation is critical for the subsequent similarity analysis and pairing process. The pre-trained embeddings offer a rich representation of the packet sequences, so that the model can discern subtle contextual similarities between normal and malicious packet sequences. The detailed of the concatenation strategy is demonstrated in Algorithm 1.

Here, we set the similarity threshold, ϵ to 0.9. This is a strategic decision aimed at balancing the need for high contextual alignment with the practicalities of ensuring a sufficient number of matched pairs. Setting ϵ to 0.9 indicates that we are looking for pairs of packet sequences that are highly similar, capturing nearly all the essential contextual information shared between normal and malicious packet sequences. This high threshold ensures that the concatenated sequences are blended seamlessly, maintaining the realistic traffic patterns and behaviors required to evade detection by NIDS. From a practical standpoint, a threshold of 0.9 is chosen because it strikes an optimal balance between precision and recall. A higher threshold (closer to 1.0) would result in fewer matched pairs, as only the most similar sequences would be selected. This could limit the model's ability to generate a diverse set of adversarial packets. Conversely, a lower threshold would increase the number of matched pairs but might include sequences that are not contextually well-aligned, reducing the effectiveness of the adversarial packets in mimicking normal traffic.

During the reverse process, we introduced a guidance mechanism using normal packet sequences. This mechanism treats the concatenated sequences of normal and noise-added malicious packets as a unified entities, allowing the model to denoise the malicious packets in the context of the normal packet sequences. We recalibrated the variational lower bound L V LB to emphasize the integration of normal sequence guidance. The objective focuses on accurately reconstructing the initial state, z 0 from the noised states, so that the malicious components are seamlessly integrated into the normal packet sequences pattern.

Table 6 lists the parameters used to run our diffusion model. The other parameters are set according to the default values of DiffuSeq. All training, experimentation, and sampling processes are executed on a single NVIDIA AD102 (GeForce RTX 4090) GPU. The training process for BERT required approximately 23 hours, while fine-tuning with the diffusion model took roughly 10 hours.

Algorithm 1 Finding contextually relevant packet sequences.

Require: Normal packet sequences S ben , Malicious packet sequences S mal , Model parameters θ , Similarity threshold ϵ Ensure: Matched pairs of normal and malicious packet sequences ( S ben i , S mal j ) 1: Initialize: Load pre-trained BERT model with parameters θ 2: Embed Normal Sequences: 3: for each sequence S ben i in S ben do 4: Compute embedding E ben i ← EMB ( S ben i ; θ ) 5: end for 6: Embed Malicious Sequences: 7: for each sequence S mal j in S mal do 8: Compute embedding E mal j ← EMB ( S mal j ; θ ) 9: end for 10: Find Contextually Aligned Pairs: 11: for each embedding E mal j in E mal do 12: Initialize best match b j ← None 13: Initialize highest similarity σ max ←-1 14: for each embedding E ben i in E ben do 15: Compute similarity σ ij ← cosine_similarity ( E mal j , E ben i ) 16: if σ ij > σ max then 17: Update best match b j ← S ben i 18: Update highest similarity σ max ← σ ij 19: end if 20: end for 21: if σ max > ϵ then 22: Add pair ( b j , S mal j ) to matched pairs 23: end if 24: end for 25: Return Matched Pairs: Return all matched pairs ( S ben i , S mal j )

B Experimental Results

B.1 Wireshark Results

?

?

No.

Time

<H>

Source

Destiation

Protocol LenghInfo

No.

Time

Source

Destination

Protocol|Lenth Info

2 0.00042

40.003936

5 0.031765

1 0.0000

7 0.100297

80.101245

30.003745

6 0.032415

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

192.168.10.50

192.168.10.50

192,168.10.50

192.168.10.50

192.168.10.50

192,168.10.50

192,168,10.50

192.168.10.50

FTP

FTP

TCP

TCP

TCP

88 Request: USER iscxtap

6653924 21[FIN,ACK]Seq15Ack=1Win=229 Len=

66 53926 21[FIN,ACK] Seq15 Ack=1Win=229 Len=

80 Request: USER iscxtap

80 Request:USER iscxtap

74 53944 - 21 [SYN] Seq=8 Win=292088 Len=8 MSS=146e S

86 Request: PASS ee86\tashmoreg

2 0.000042

1 0.00000

30.003745

60.032415

80.101245

40.003936

50.031765

7 0.100297

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

192.168.10.50

192.168.10.50

192.168.10.50

192,168.10.50

192.168.10.50

192.168.10.50

192.168.10.50

192.168.10.50

TCP

FTP

80 Request:USERiscxtap

8Request:USERiscxtap

80 Request:USER iscxtap

86Request:PASSe086\tashoreg

66 53924 → 21 [FIN, ACK] Seq=15 Ack=1 Win=2]

74 53944 - 21 [SYN] Seq=@ Win=2920e Len=θ M

11 0.1243

90.104336

100.105400

172.16.0.1

172.16.0.1

172.16.0.1

192,168.10.50

192.168,.10.50

192,168,10.5

TCP

66 53946 ~ 21 [ACK] Seq=1 Ack=1 Win=29312 Len=0 TSva

11 0.106243

100.105400

172.16.0.1

172.16.0.1

172.16.0.1

192.168.10.50

192.168.10.50

192.168.10.50

665394421[ACK]Seq1 Ack=21Win=29312

Frae8:66bytere(52bits)66byecaptured(5b）

）

Ethernet II，Src:Cisco_14:eb:31（00:c1:b1:14:eb：31)，Dst:Del_0a：69f1(00:19:b9:a:69:f1)

TransmissionControlProtoco,SrcPort:53944,DstPort:21,Seq:1,Ack:1,Len:

Internet Protocol Version 4, Src: 172.16.0.1, Dst: 192.168.10.50

Transmission ControlProtocol, Src Port:53944,Dst Port:21,Seq:1,Ack:1,Len:

Internet Protocol Version 4, Src: 172.16.0.1, Dst: 192.168.10.50

Flags: exb10 (ACK)

Urgent Pointer:

Source Port:53944

Acknowledgnent nunber （raw)）：1568395363

Destination Port: 21

[Stream index:3]

Acknowledgment Nusber: 1

1e..=Header Length: 32 bytes (8)

Window: 236

[Mindw size scaling factor: 128]

[Checksu Status:Unverified]

[Calculated window size: 30208]

Checksun: exe524 [unverified]

[Conversation completeness: Inconplete (29）]

[Next Sequence Nunber: 1

(relative sequence nunber)

(relative sequence nunber)]

(relative ack nunber)

>Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Tinestanps

Flags: exe1e (ACK)

Source Port:53944

Acknowledgment Nunber:1

Destination Port: 21

[Conversation completeness:Incomplete （29）]

Sequence Number (raw）:3463631349

Checksun: exe52b [unverified]

Urgent Pointer: @

[Strean index: 3]

[TCP Segment Len: 0]

[Next Sequence Number: 1

Acknowledgment nunber (raw): 156e395363

Window: 229

[Checksum Status: Unverified]

[Calculated window size: 29312]

[Window size scaling factor: 128]

(relative sequence nunber)

(relative sequence nunber)]

(relative ack nunber)

Options(12bytes），NeratinNP)NOperatin(NOP)，Tiesta

>[Tinestamps]

>[Timestamps]

2038

2040

3020

0019 b9@a69 f100 c1 b114 eb 3108 08 45 00

000020

Qa 32 d2b8 e0 15ce 72 d1f5 5d 01 be 63 8010

be e6

4@>

..·$.

......].....

6028

0010

0040

e019b9ea69f100 c1b114eb31 08 00 45 00

e0341b 1c48003e 06

be e6

31019313

4.0·>

-r

..]..

1-·E

(a) Botnet attack before in Wireshark.

(b) Botnet attack after in Wireshark.

To demonstrate that our model intelligently alters specific fields to seamlessly blend malicious packets into normal traffic, we analyzed the packets using Wireshark. Our analysis covered 6 attack types: MITM , Botnet , Brute Force , DDoS , Port Scan , and Infiltration . The results showed that different specific fields were modified in the packets corresponding to each type of attack. This variation in field alteration confirms that our model adapts its modifications according to the nature of the attack, enhancing the stealthiness of the malicious packets. In Figure 2, we can see that there is a change in the window size of a Botnet packet. Botnet attack can be described as a network of infected devices (bots) controlled by an attacker to perform various malicious activities. Botnet traffic often shows abnormal patterns in window size due to automated control and data bursts. Here, we can conclude that DiffuPac made the intelligent decision by itself in modifying the window size to fall within typical user traffic patterns. This helps to camouflage the botnet activity.

MITM, on the other hand, demonstrated attacks whereby an attacker intercepts communication between two parties without their knowledge, often to eavesdrop or alter the data being sent. Abnormal TTL values (time-to-live field) can reveal this kind of interception and redirection. Here, once again, our model intelligently adjusts TTL value to match the expected values in aiding the masking of MITM presence. As shown in Figure 3, DiffuPac adjusts the TTL values to effectively conceal signs of interception. This ensures that the adversarial packets blend seamlessly with normal data exchanges, mimicking legitimate communication between parties.

DDoS attacks involve a large number of packets with similar characteristics, often reusing or predictably incrementing the IP Identification (ID) field. In Figure 4, it can be shown that DiffuPac addresses this by randomizing the IP IDs to prevent detection. When we analyzed the traffic using Wireshark, we observed varied and less predictable IP IDs in the modified packets, aligning with normal traffic patterns. This randomness helps avoid forming detectable patterns and evades detection systems that rely on identifying repetitive IP ID sequences. Thus, DiffuPac successfully masks the attack's presence while preserving the attack functionality.

Port Scan often involve specific flags, such as SYN, to probe for open ports. This behavior is distinct from normal traffic, which uses a variety of flags. In Figure 5, we found that DiffuPac diversifies the use of flags to include SYN, ACK, and FIN, similar to normal connections. By normalizing the traffic in this way, DiffuPac helps disguise scanning activity. For example, instead of sending many SYN

Aply adsplay fter  <j>

PrteelLenghilo

Aopy a dsplyfter <R)>

10 87.314204

11 87.314204

13 87.314308

14 87.31433

15 87.314485

16 87,11445172.160.1

12 87.314262

7 4.436634

8 4.436637

9 4.475174

38.083961

53.396459

6 4.31584

2 0.0916

4 3.395788

172.16.0.1

172.16.0.1

172.16.0.1

172,16.0.1

172,16.0.1

12.1.1

172.16.0.1

172.1.8.1

172.16.0.1

172.16.0.1

172.16.0.1

172,.16.0.1

172,16.0.1

172.16.0.1

192,168.10.50

152,18.18.50

132,16.18.50

152.16.18.50

132,16.18.50

102,158.1.5

192.168.10.50

192,.168.10.50

192,168.18.5

192,168.10.50

192,168.10.56

192,158,10.50

192.168.10.50

192,168,10.50

TCP

74 5218 - 21[SN]SeqMin292 LeneMS1460 SACX_ER1TSval475779 TSecr M5=128

65

7 Reguest:PAS51234

74524292

4 56

66511[A]eAcin29312LTv475779TSec154

66 518821[AX]SeqAck21Min29312Len TSval4757 TSecr211541

72 Request:5YST

66 518821[AX]eq2Ac97Win29312LTl4768TSec11769

74 52114 21[S]eqin29 Len546 SAK_PRa4976TSe M128

74 52116 21[S]eqin29 LeMS46SAK_ERa4976TSe M128

74 52118 21[S]eqin29 Len546 SAK_PRa4976TSe M128

452122SSqi29S4AT97Te2

Request: USER iscxtap

13 87.31438

15 87.314485

16 87,31445172.16.0.1

11 87.314204

12 87.314262

14 87.314332

30.003961172.16.0.1

6 4,35548

1887.314204

2 0.000916

5 3.396459

7 4,.436634

8 4,436637

4 3.395788

9 4,475174

172,16.0.1

12.16.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172,.16.0.1

172.16.0.1

172.16.0.1

1592,1618.50

192,168.10.50

192,168,18.50

192.16.10.50

192.166.18.50

192,166.18.58

192,166.18.58

192,168.10.50

192,168.10.50

192,168. 18.5

192,.168.10.50

192,.168.10.50

192.168.18.50

192.168.18.50

192.168.10.50

TP

77 Reguest:PA551234

4524T2

618AA293127579T1154

618AA2T3

6182AA2T

452149

4512949

6521882[AX]SeqAck21Wi2312LTva47578TSec2115]

80 Request:USER iscxtap

72 Request: SYST

66 52188 21[ACK] 5eq32 Ack=97 Win29312 Len TSval476898 TSecr2117649

7452112 21[N]5eqWin292 LenMS5146eSAK_E1TSval49767TSecr 5128

74 52124  21[S]SeqWin292 LenMS46e SACX_PE1TSva497688TSecg S128

74_52126- 21[NSeoMin292lenMS1SAX PE1Ta98TSer18

Fra3:6beve8its)6bye catred ts

InteetPrtersi 4,r:1.16. st:92.8.

e80 ... = Version: 4

Frane3:6byenve(58its)）66bescapturedbs）

Etheet I1r:Cis_14:b:31(b4：31）st:e1a691(:19：:9f1)

Intemet Protoco Version 4, Src: 172.16.8.1, Dst: 192.168.18.58

f

TotalLength:52

180 .... = Version: 4

Hdeh2bes （5）

Identifictio:e (18664)

FlagsaBnfraent

Fft

Tine to Live: 62

Pretocol: TCP (6)

FlasDntfrt

ee  Fragment offset:

Time to Live: 85

Protocol: TCP (6)

Destination Adress: 192.168.18.58

[Header checksun status: Inverified]

Source Adress: 172.16.0.1

Destintnd9

SourceAdress172.16..1

36 7d

08 0a 00 07 42 84 81 42

6

4H-0-1

-1--E

36 7d

a 32 cb 8c 0 15 19 2a

a6615时ee10

08 0a 00 0742 84 01 42

64 el ac 18 00 81 c8 a8

4H-0-- d-

(a) MITM attack before in Wireshark.

Aoply a dsplay fter<R)

21 2.548381

18 0.501282

11 0.601586

16 0.6968

20 2.5476

22 3.153457

12 0.602438

14 0.605530

15 0.607189

17 e.615635

18 2.545847

19 2.545671

6 0.352317 172.16.0.1

96.58161

13 0.685527

2 6.247674 12.16.6.1

5 0.34994

TneSoure

3.0.247893

4 0.348224

8 0.352949

1 0.0172.16.0.1

7 0.352320

172.16.0.1

172.16.0.1

172.16-0.1

172.16-0.1

12.16.0.1

172.16.0.1

172.6.1

172.16-0.1

172.16.0.1

172.16.0.1

172.16,.0.1

172.16,6.1

172.16-.1

172.16,6.1

172.16.6.1

172.16.6.1

172.16.6.1

172.16.6.1

192,16818.50

192.15818.50

192.16,.18.56

192,168.18.580

192.156.10.50

192,168.18.50

192.16,1.50

192.08.18-50

192.06.18.56

192.168.18.50

192.168.1.50

192,168.18.50

192.166.10.50

192.16,18.50

192.168.18.50

192.168,.18.50

192.168.10.50

PrtolLenthifo

TOP

FP

FTP

TCP

TCP

FP

82RequetPAS 321929

Reguest:uSeRisocta

85RequestA5S33395383

Request：USERiscxtap

745762

66 5761221[AX]Seq1AckWin29312LenTval1124894TSec2176568

66 5761221[A]SeqAck21Win29312TSv124894TSec217656

ReqA5399

Request:USERiscxtap

745761421[S]eqi292Le54605ACT24957Te5128

66 57614  21[ACX] Seq1Ack21 in2312 Le TSval1124958 TSecr21765712

Request：uSERisxtap

85 Requst:PASS33c4ta

Request：USERiscxtap

Request:uUERisctap

84 Request: PASS3412866741

8 Request: u5SER iscxtap

80 Request: USER iscxtap

[Apply a displayfiter _ <M)

11 0.601586

21 2.548381

10 0.501282

12 0.62438

13 0.685527

14 0.685538

15 0.687189

16 0.689968

17 8.615635

18 2.545847

19 2.54561

20 2.547676

10.000172.16-0.1

2 0.247674 172.16.0.1

3 0.247893172.16.01

60.352317172.16.0.1

8 0.352949

9 0.58161

4 0.348224

5 0.349094

70.352320

TmeSource

172,16.0.1

172,16.0.1

172,.16.0.1

172,.16.0.1

172,.16.0.1

172,.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.8.1

172.16.0.1

Destratfion

192.16.18.50

192.168,.18.50

192,166.18.50

192.166.18.50

192.18.10.50

192.168,.18.50

192,.168,180.50

192,.168.10.50

192,.16.18.50

192,168.10.50

192,168,10.50

192,168.10.50

192,.168,10.50

192,.168,.10.50

192,.168.10.50

192,.168.10.50

192,168.10.50

192.168.18.50

192,168,18.50

192,168,18.50

192,168,18,50

PrtoolLenghIfo

1

TCP

TCP

TCP

FTP

FTP

FTP

FTP

FTP

FTP

82RequestPA5321929

8Request：PASS33533

65A2T

74576124AT21

57594Ac2

745761429S4T5

94Request：PASS-329ttr

8 Request：USER isoxtap

B Request:usER iscxtap

66 57612  21[ACK] Seq Ack21 Wi2312 LnTval1124894TSecr21765648

6571421ACK]SeAck29312LTa1957TSe1511

BRequest:usERiscxtap

84 Request:PAS5342866741

Request:USERisctap

66 57612 + 21 [ACK] Seq1 Ack1 Win29312 Leng TSva1124894 TSecr21765648

9eRequest:PAS33796363396

8 Request: U5ER Iscxtap

66 57614  21[ACK] Se1 Ack21 Vin29312 Le TSval124958 TSecr21765712

85 Request: PASS33c4detda

8 Request: U5ER Iscxtap

88 Request: USER iscxtap

23 3.154287

172.16.0.1

192.168.18.50

8Request:ASS33iitncin

22 3.153457

23 3.154287

172.16.0.1

172,.16.0.1

192,168,.18.50

192.168,.18.50

RequestPAS3tnin

88 Request: USER iscxtap

rbsbad

Ethmetr:Cisc_141(1:3）Ost: De:9:19::1

Frane6: 66 bytes n vire (28 bits),66bytes captured(528 bits）

Ethmet I,r:Cisc_14：31(b::3),st:e1:9:9:91)

18..  Versin: 4

Inteet Prtc ersn 4Sr:17.16.1,Ost:192.18.18.5

018 .. Version: 4

Differentiated Services Field: exoe (DSCP: CS, EO: Net-EC)

Flas: Dnt frant

Total Length: 52

Idefcin feb （6455

if

Total Length: 52

Ientificatlon: ex7d7b (3213)

Tine to Live: 62

）Flas:x4Dnt fraent

Pretecol: TCP (6)

Tine to Live: 62

Protocol: TCP (6)

Transin Contrlrtl, SrcPer:57612, Ost Port: 21,Seq:1, Ack: 2, Ln:

SourceAdre:172.6.6.1

DestinstiAddress:192.168.18.9

Hede Checksu: ex85d[valildatindisbIed]

Source Adress: 172.16.8.1

[Header checksun status: UInverified]

1e 18

fc 2b

100112 101 40

2.

--5,

0818

19 b9a69f10c1 b114eb3180450

1e 10

88 6a 00 11 2a 1e 01 4c

...

(a) DDoS attack before in Wireshark.

packets, it includes a mix of SYN, ACK, and FIN packets, which helps blend the scan into typical network behavior and evade detection.

Brute force attacks generate numerous login attempts, leading to irregular sequence numbers due to the repeated connections. By analyzing the traffic with Wireshark, we observed that DiffuPac intelligently normalizes sequence numbers to mimic regular user connections. Normal user login attempts typically show sequentially increasing sequence numbers. In Figure 6, it shows that DiffuPac adjusts these numbers to follow a similar pattern, thereby camouflaging the brute force activity. This adjustment helps blend the attack packets seamlessly into benign traffic, making it harder for NIDS to spot the abnormality.

Infiltration attacks involve moving laterally within a network, often generating unusual acknowledgment numbers as the attacker accesses various systems. In Figure 7, it demonstrated that DiffuPac

< o    .  + + + . . = e e a

Acoly a displa

10.00

2 0.074947

Destination

192.168.18.54

192,168.10.50

PeticolLnghl

RequestA5582466

Request：uER isoxta

1 0.00172.160.1

2 3.047445 172.16.61

192.18.18.5

Detintion

192.168.0.50

PrtoclLengh

印

86Request:PA5S542782446

BRequest:u5ER iscxtap

13 6.18626

14 0.186936

5 8.12341

80.175313

172.16.0.1

192,168.10.58

192.168.10.50

192,168,10.50

TOP

TOP

Regst：uSERisoxtp

RetStah

8Reust：PA55521879

5ReqestPA559

74 5942  21[5N]5eq= Vin

0 Request: usER iscxtap

2920ACTT2

13 0.18294

150.184453

14 0.186936

180.175313

90.14384

5 0.075197

60.122750

172.16.01

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

192,16,18.50

192.16.10.50

192.168.18.50

192.168.18.50

Regest：uSRiscxtap

5RstAe

Regest：uERisoxtap

5422 21[FN,AX] Se15 Ack i229 Leng Tva3

74594422[SN]e292LeM551466SACK_PETSva33231TSe528

2ReqestPA5552809

88 Request: u5ER iscxtap

170.3016861

19 6.302528

18 0.301906

172.16.8.1

172.16.0.1

192,168.10.50

192,168.10.58

192,168.10.50

TO

TCP

94133978

RePA12

0 Request: uSER iscxtap

E

170.284045

180.285523

172.16.0.1

172.16.0.1

172.16.0.1

192.16.10.50

192.166.18.50

TOP

745944 21[5Seqi292 Le5514SACK_PTa1332328TerM5128

66 59424 - 21 [FIN, ACK] Seq=15 Ack1

E

21 0.334306

24 0.4388

25 0.438611

200.334859

172.16.0.1

172.16.6.1

172.16.6.1

192,16.1.50

192,168,10.50

192.18.1.50

192,168.1.58

292,18.10.50

192,168,10.50

9

694462AC]qAc292T13336T5e219738

66 5446  21[A] Seq Ack21 Min2932LeT3327TSec2193118

8RequestAS551976

Request:usER isoxtap

80 Request: u5ER iscxtap

uUSER isota

2 8.38252

27 8.43811

22 0.33458

26 0.438688

172.1-0.1

172.16.0.1

172.16.0.1

172.16.0.1

12,16.0

192.16.18.50

2

4AcT3691

88 Request: USER iscxtap

74 59446 = 21 [SYN]

Si292LeMS1460SACK_ERTSva133236TSecr528

3 3.03445

28 2.958613

29 2,958825

32 3.833658

三

172.16.6.1

172.16.8.1

172.6.8.1

323 15-4.1

192,168.1.58

192,16,.10.50

192,168.10.58

192,168,.10.50

TOP

FTP

2221

8R

6ReqsA55t

6542A5

Reqst：A5554253991492

BRequest：USER isoxtap

0 Request: usER iscxtap

t: UsER isoxtap

uSER isoxta

31 2.958613

33 3.03387

35 3.043445

29 2.95822

34 3.833658

37.3.143828

30 2.958825

32 2.959217

36 3.043674

28 0.439257

三

172.16-6.1

172,16.0.1

172.16.0.1

172.16.6.1

192,166.38-50

192.166.10.50

192,168,18.50

TCP

Reest：uSRiscxtap

Reguest:uSERiscxtap

87Request:A5558253993\82

86 Request1

8 Request:PASst34

( e( a s

Internet Protocol Version 4, Src: 172.16.e.1, Dst1 192.168.18.58

t

DestintinPert:21

irce Pert: 59444

nce Nunber: 8

(relative sequece suber)]

010

00 01 03

8307

(a) Port Scan attack before in Wireshark.

(b) Port Scan attack after in Wireshark.

<h  oy

1 0.172.16.0.1

TimeSouoe

Desinfon

192.16.80.50

PetolLengthil

8Request:U5ER 1scxtap

Apply a display fiter <Kj>

10.00172.16.0.1

TmeSource

Destiration

192,168.10.50

PrtoolLengthllo

Request:USERiscxtap

2 0.625172.16.0.1

4 0.64073

3 0.639978

172.16.86.1

172.16.6.1

192.168.18.50

89 Reqest:PASS81

94RequestPASSjettadstl9

2 0.80625172.16.0.1

3 0.639970172.16.0.1

4 0.648733

172.16.0.1

192,168.10.5

192.16,.10.50

192,160.1,.50

89 Rquest: PA55 kirt9182001

80 Request: USER iscxtap

9ReestPA5jt

5 0.645387

6 0.646131

7 0.863174

12.16.6.1

172,.16.0.1

172,16.0.1

192.16.18.50

192.166.18.50

192.160.18.50

PP

FTP

FTP

Request:USER iscxtap

88Request: PA5S1jeffs\t7ashs

Request:U5R iscoxta

5 0.645387172.16.0.1

6 0.646131

7 0.863174

172.16.0.1

172.16.0.1

192,.16.10.50

192,168.10.50

192,168.10.50

FTP

80 Request:USER iscxtap

88Request:ASSjeffsths

8 Request: USER iscxtap

18 0.87715

80.86338

9 0.876522

172.16.0.1

172.16.0.1

172.16.8.1

192.168.18.50

192.168.18.50

192.18.18.50

TCP

187Reqt:PASS22

6555625157

8 Request：USER iscxtap

10 0.877150

80.86338

9 0.876522

172.16.0.1

172.16.0.1

172.16.0.1

192,168.10.50

192,168.10.5

192,166.1.50

TOP

18R

8 Request:USER iscxtap

66556 21[FIN,ACK]Seq5 AckWin29LenTSval1451TSecr2155487

15 8.908294

201.033190

21 1.033465

2.856767

23 2.856815

11 0.91283

12 0.913639

130.963524

14 0.964196

16 0.96829

17 0.984718

19 1.832879

18 1.832566

172.16.0.1

17,.16.6.1

172.16.60.1

172.16.8.1

172.16.0,.1

172.16.6.1

172.16.0.1

172.16.86.1

172.16.6.1

172,16.0.1

172.16.0.1

172,16.0.1

172.16.6.1

192,168.3.50

192,168.8.50

192.16,.18.50

192.16.18.50

192.1686.18.50

192,16.18.50

192.168.18.50

192.16.18.50

192.18.18.50

192.168.18.50

192.18.18.50

TCP

TCP

74554A62

93Reqst5ak

Request:u5ERIisctap

80 Request: USER iscxtap

88 Reqest:PASS nusn(t2

665781[A]SeqAckMin29312LnTSv91476TSec255543

6655781AC]Ack21i2912 LTl4TSec15537

86Reqst:PASSjer

RequestPASsjuarttt

Request: USER isoxtap

Request:USER isoxtap

8 Request: USER iscxtap

12 0.913639

13 0.963524

17 0.98478

19 1.032879

20 1.83319

21 1.033465

22 2.856767

23 2.,856815

11 0.91293

15 0.968294

16 0.968297

18 1.832566

14 0.964196

172,16.0.1

172.16.0.1

172.16.0.1

172,16.0.1

172.16.6.1

172.16.60.1

172.16.0.1

172.16.6.1

172.16.6.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

192,160.10.50

192.168.18.58

192,.168.10.50

192,16.1.50

192.168.1.58

192,168.10.50

192,168.18.50

192,168.10.50

192.166.1,.50

192,.16,.10.50

192.168.1.50

192,168,.18.50

192.168.10.50

TOP

TOP

8 Request:USER iscxtap

86Reguest：PASjptker

RequestuSERiscatap

Reguest：USERiscatap

ReguestPA5sjttt

88 Request: PASSeInusniv/t5892

66 55788~ 21[ACX]5eq1Ack Win29312 Len TSval914676TSecr255433

8e Request: USER iscxtap

93 Request: PASSe1axxpupjt13p2rk

8 Request: USER iscxtap

6655768  21 [FIN, ACK] Seq15 Ack]

Frane 5: 8bytes on vire (648bits),0bytes captured (64 bits)

rae5:bee6btbcaudbs）

EtheetIr:Cis_14::3(b4：31)st：ela69f1(19:：9f1)

Intemet Protocol Verslon 4, Src: 172.16.8.1, Ost: 192.168.18.5

TasisCntPrt,cPrt:562stPt:2,eq:1,Ack:1,L:4

arngth3bytes （8)

Cheksa:ac5lerfel

SourcePort:55762

DestinatinPort:21

[5tren inden2]

Ceticlesle ]

Seqence Ner: 1(rlative seqene er)

Flags: bx818 (PSH, ACK)

[TCP Segpent Len: 14]

Vindov: 29

[Calcslated vindo size: 229]

(relatie sequence nbe]

Transnissin Controt Pretoc1, Src Pert: 55762, Ost Port: 21, Seq: 1, Ack: 1, Len: 14

Ethemet I1,r:Cisc_14::3(b:4:b：31）st:Del69f(:9:91)

Flags:ex18(PSH，,ACX)

Source Pert: 55762

Windov: 229

[Calcvlated vindo siz: 229]

[Strean inde: 2]

Acknovledgnet Nuber: 1

[Cnversation copleteness: Incoplete (28)]

[Next Sequence Nunber: 15

[indv size scaling facter: 1 (unknan)]

(relative sequence suber)

(relative ack nunber)

{relative sequence nunber)]

Chcksn tausUeffedl

etfo: (1 bte). No-frtin IP. No-oration NO1. Tins

[Checksun Status: Unverified]

Urgent Pointer: 8

Det ions: (12 bvtes1.

1. Ne-Oeeratin (NDP). Noeratin (NP] Tintaes

882

2848

0 P0 02 T9 2 8L 9 6L 69 0225 5# E5 55 66 99

-SER 1 soxtap

0030

0 0 0L 19 L8L E9 6L 69 0225 S E5 5 66 89

-USER I scoxta-

-d x.F-

(a) BruteForce attack before in Wireshark.

(b) BruteForce attack after in Wireshark.

intelligently modifies acknowledgment numbers to follow expected sequences. In normal communication, acknowledgment numbers increment predictably based on the received data. DiffuPac ensures that infiltration attempts have acknowledgment numbers that follow these normal patterns, effectively masking the lateral movement within the network and blending the attack traffic with normal traffic, making it difficult for detection systems to identify the anomaly.

DiffuPac demonstrates its ability to modify specific fields intelligently in the packet headers of different attack types, such as the window size for botnet traffic and TTL for MITM, to blend malicious packets into normal packets. This capability is consistently proven through detailed packet analysis with Wireshark. Indeed, the detailed packets analysis shows that critical fields defining packet identity and practicality (e.g., IP addresses, ports, payload) remain intact after modification, ensuring the functionality of the attack while evading detection by sophisticated NIDS. The comparisons

plya dsplay fiter  <RJ>

E

E

E

26 8.211856

29 8.21184

12 8.0864859

15 8.165843

19 8:218287

31 0.212066

18 0.218199

20 e.218227

23 0.218881

30 e.211822

2 0.0629

90.059086

3 0.087486

6 0.048639

10.00172.16.0.1

TeSsur

172.16-0.1

172.16-0.1

172.16.0.1

172.16.0.1

172.16-0.1

172.16.0.1

172.16-0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172,16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172,16.0.1

172.16.0.1

172.16.0.1

192.1686.10.50

192,168.18.50

192.168.10.50

192,18.10.50

192.166.10.50

192,168.18.58

192.168.10.50

192.168.10.50

192,168.10.50

192,168.10.50

192,168.10.50

192,168.10.50

192, 168.18.50

192,166.10.50

192.168.10.50

192,168.10.50

192,.168.10.50

192, 168.10.50

192.168.10.50

192.168.10.50

192,168.10.50

192,168.10.50

192.168.10.50

192,168.10.50

192,168.10.50

192.168.18.50

EE.

3lntedcke）

38Ln：Eyedket

13Client：Encrypted packet（le）

13 Client: Encrypted packet (len64)

13Client:Encrypted packet（le64）

178lin：ypedketle）

178CLen：Eyedkete）

2Lient：Encryted packet（le1）

82 CLient:Encrypted packet (

66 4642 - 22[ACX] Seq993 Ack1889Win259Len Tva925967TSe2556679

Clint：Enptedpaet（l）

82 CLient: Encrypted packet (1en16)

66 46834 - 22[AC]Seq353Ack=913Win259 LenTSval4925967 TSecr256679

664684822[A]Seq353Ack13in259LeTval92597TSec255679

66 46834 - 22 [ACX] Seq=273 Ack=849 Win=259 Lenne TSval=4925928

82 Client: Er

66 46838 - 22 [ACX] S

66 4

38 Client：Encrypted pach

46848 - 22 [A

46836 22 [AOX] S

Encrypted packet (

[AK] s

5eq993 Ack1889win259 Len Tva4925967 TSecr25566679

5eq81Ack65Win259 LenTval4925967TSecr255679

Seq273Ack849 in259 LeTSva4925932TSecr255633

（len16)

(len16)

Aply a displey fiter<)>

E

31 6:212866

16 6.165967

22 6.218243

25 6.211833

27 e.2110862

28 8.211378

33 0.386346

11 0.06877

14 0.164494

15 0.16544

17 0.167486

20 0.21822

23 0.218801

26 e.211856

30 0.21182

32 e.212881

90.8596

100.065589

21 e.218227

24 e.218887

29 e.211884

30.887486

5 0.823284

6 0.040639

2 0.806209

4 0.887813

7 0.044852

80.052612

TmeSouce

172.16.0.1

172.16.0.1

172.16.8.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.0.1

172.16.6.1

172.16.6.1

172.16.0.1

172.16.6.1

172.16,0.1

172.16.0.1

192.166.18.50

192.166.18.50

192,168.10.5

192.166.108.50

192.166.18.50

192,166.10.50

192,168.10.50

192,.168.10.50

192.168.10.50

192.168.10.50

192.168.10.50

192.168.10.50

192.168.10.50

192.168.10.50

PretocolLenghlo

EEE

3led

76litEnytedacketl

3litEncytedpacket(le2）

13ClitEnat（）

138 Client： Encrypted p

178 Client: Encrypted packet (len112)

82 Client: Encrypted packet (len16)

66 4834 22[ACK]Seq353 Ack913 Win259 LenTSval4925967TSecr2556679

66 46835-22

66 46840 ~ 22 [ACXK] S

8ClientEnryptedpacket

82Clint:Encyptedacket()

ltEntedet（）

66 46842 ~ 22 [ACK] 5

66 46834 - 22 [ACK] Seq=273

66 46838 - 2[ACX] Seq913 Ack1825 in259 Len0 TSval4925936 TSecr266638

66 46838 - 22 [

[ACK]

[ACK]

Sq993Ack1889 in259 Lene TSval4925967Tecr2556679

Seq=913 Ack1825 Win259 Len TSval92531TSecr25566642

Seq993Ack1889 in259 LenTSa4925967TSecr25566679

Seq81 Ack65 Vin259 LeTSval4925967TSecr2556679

packet

packet

packet

packet

（len112

Ack=266e942180 Win=259 Lenne TSval=4925928 TSecr=2556662

(1en64)

(len6)

(le16)

(len64)

(len64)

(2))

Intts

36_A 220768

122-16-41

(len64)

Destint ionPrt: 2

(）s

Acknvledgent Nunber: 977

er（rav): 3352617128

(relative ack nunber)

Flags: exe (ACX)

Acknledgent Nunber: 371151894e(relative ack nunber)

t number (raw): 2769167795

in siz saling fat1

Checksun: da [uverifie]

-K)..

1-·E-

020

830

1d a4

(a) Infiltration attack before in Wireshark.

(b) Infiltration attack after in Wireshark.

for each attack type before and after the modificatinos further highlight DiffuPac's effectiveness in maintaining attack practicality while enhancing stealth.

B.2 Evasion Rate

We evaluated the evasion rates of the other 4 attacks-DDoS, Port Scan, Brute Force, and Infiltration-using DiffuPac and the baseline models, with the detailed results summarized in Table 7. According to the results, we can observed that the trends in the 4 attacks are consistent with the 2 attacks shown in Table 2 in the main paper.

B.3 Malicious Functionality Evaluation

Experimental Setup . We conducted a robust evaluation of both Port Scan and Brute Force attacks to assess their malicious functionality in controlled, isolated network environments. The primary goal of this evaluation was to determine whether adversarially modified packets retain their ability to perform malicious actions. Throughout the evaluation, we focused on analyzing the response of both the original and adversarial packets using Wireshark, carefully observing how the target system reacted to each type of attack.

B.3.1 Port Scan

Before proceeding to evaluate the generated adversarial packets of DiffuPac, we first demonstrated a a successful Port Scan attack within our controlled environment. This initial step is a fundamental step to ensure the validity and reliability of subsequent tests with both original and adversarial packets. The network configurations depicted in Figure 8(a) and Figure 8(b) confirm that both virtual machines were correctly placed within the same subnet. As demonstrated in Figure 8(c), the Port Scan successfully identified open ports on the Ubuntu, highlighting port 22 as vulnerable.

We initiated a Port Scan attack from the Kali Linux machine targeting the Ubuntu server. During the execution of this attack, we captured all packets sent from the attacker to the victim, saving them as a pcap file. This capture was timed precisely to begin as the Nmap tool launched the Port Scan, ensuring that only the relevant src-to-dst packets were recorded. This focus aligns with the operational framework of DiffuPac, which concentrates on modifying packets sent from the source.

Using the captured src-to-dst packets, we employed DiffuPac to generate adversarial Port Scan packets, which were then saved in a separate pcap file. These adversarial packets were subsequently replayed using the Tcpreplay tool, directing them towards the Ubuntu server. Concurrently, we used Wireshark on the Ubuntu machine to capture the incoming responses, allowing us to analyze the effectiveness of the adversarial packets in real-time.

To ensure a balanced evaluation, we also replayed the original Port Scan pcap file using Tcpreplay, capturing the responses in Wireshark on the Ubuntu side. This dual replay enabled a direct comparison between the packets' responses generated by the original and adversarial Port Scan packets.

As depicted in Figure 9, the response to the adversarial Port Scan packets closely mirrors that of the original Port Scan. A detailed inspection of the TCP flags reveals that packets with SYN and ACK flags targeted port 22, corroborating the initial vulnerability identified in Figure 8(c). This consistent response across both original and adversarial packets provides compelling evidence that DiffuPac successfully generates adversarial packets that retain their intended malicious functionality.

(ajkali)-[~]

etho:flagS=4163<UP,BROADCAST,RUNNING,MULTICAST>mtu 1500

inet 192.168.56.101 netmask 255.255.255.0 broadcast 192.168.56.255

ether 52:3d:74:76:90:f0 txqueuelen 1000（Ethernet)

RX packets 1108bytes 66065（64.5 KiB)

RXerrors0dropped0overruns0frame0

inet6 fe80::503d:74ff:fe76:90f0prefixlen 64 scopeid 0x20<link>

TX packets 1038 bytes 62122 （60.6 KiB)

TX errors0dropped0overruns0carrier0collisions0

lo:flagS=73<UP,LOOPBACK,RUNNING>mtu 65536

inet 127.0.0.1netmask 255.0.0.0

inet6:1 prefixlen 128 scopeid 0x10<host>

RXerrors0dropped0overruns0

TX packets 8bytes 480 （480.0 B)

RX packets 8bytes 480（480.0 B)

TX errors 0dropped0overruns0

carrier0

frame0

collisions 0

Kali Linux network configurations

aj@ajnatsu:~/Desktops ifconfig

enp0s1:flagS=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 192.168.56.102 netmask 255.255.255.0 broadcast 192.168.56.255

ether 7a:3a:4e:76:82:6d txqueuelen 1000(Ethernet)

RX errorsθdroppedθoverrunsθframe

RX packets 1097 bytes 71573 (71.5 KB)

TX packets 1113 bytes 66960 (66.9 KB)

TX errors θdropped θoverruns θcarrier θcollisions ①

lo:flags=73<UP,LO0PBACK,RUNNING> mtu 65536

inet 127.0.0.1netmask 255.0.0.0

inet6::1 prefixlen 128 scopeid 0x10<host>

RX packets 3588 bytes 256356 (256.3 KB)

RX errors0droppedoverrunsframe

TX packets 3588 bytes 256356 (256.3 KB)

TX errors θdropped θ overruns @carrier θcollisions ①

Ubuntu network configurations

Successfully executed Port Scan using Nmap tool

Transmission Control Protocol, Src Port: 22,Dst Port: 5465o, Seq: 0, Ack: 1, Len:@

tp.flags.acka1

NO.

ae55ap

Ethernet 11, Src: 7a:3a:4e:76:82:6d (7a:3a:4e:76:82:6d), Dst:52:3d:74:76:90:fo (52:3

Internet Protocol Version 4, Src: 192.168.56.102,Dst: 192.168.56.181

17 5.311367628

185.311368669

275.311395211

445.311927211

455.311929378

465.311986253

285.311404878

295.311406044

305.311406878

315.311416628

325.311424586

335.311432961

43 5.311923294

47 5.311995836

48 5.311998294

495.312000169

50 5.312015461

Time

Source

192.168.56.102

192-168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

①<

Destination

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168,56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

replay_topspeed_original.pcapng

①QQ

TC

Protocol Length info

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

5425-54650[RST,A

54 113-54650[RST, ACK]Seq=1 Ack=1 win=θ Len=g

54 2154650[RST, ACK]Seq1 Ack= win=0 Len=0

54 8054650[RST,ACK]5eq=1 Ack1 win=0 Len=0

545354650[R5T,ACK]Seq=1Ack=1 Win=0 Len

54143

54110

54 1723

54 3306

543389

54 443

54256

548888

TTTS

54139

54 993

541720

54658[RST,ACK] Seq=1 Ack=1 win=0 Len=8

54650

54650

54658

54650[RST, ACK] Seq=1 Ack=1 win=0 Len=0

54650 [RST， ACK] Seq=1 Ack=1 Win=0 Len=0

54650

54650[RST,ACK]Seq*1 Ack=1Win=0Len=0

54650

54650

54658

5465e

0000

0010

0820

0030

[RST,

[RST,ACK]

[RST,ACK]

[RST,ACK]

[RST,

[RST,ACK]

38 65 60 16 d5 7a 1d 84

000F000022 00

7d 78 d9 2e 00 00 82 84 85 b4

323d74769016

ACK]Seq=1 Ack=1 win=0 Len=0

ACK]Seq=1 Ack=1 Win=θ Len=θ

ACK]

= 0= =x =b

Seq=1 Ack=1 win=0 Len=0

b

Seq=1 Ack=1 win=0 Len=@

@=u7 0=u T=x =bas

7a 3a

4006

4e 76826d 0800 4500

48 b0 c0 a8 38 66 c0 a8

Original Port Scan Response

response_adversarial.pcapng

tcp.flags.acku1

Ethernet II,Src: 7a:3a:4e:76:82:6d (7a:3a:4e:76:82:6d),Dst:

Transmission Control Protocol, Src Port: 22, Dst Port: 54650,

Eile Edit yiew Co Capture Analyze Statistics Telephony Wireless Iools Help

（

NO.

250.000895584

100.000744875

110.000785084

120.000811292

260.080921258

310.001081250

130.000825000

14 0.000834625

150.000844875

16 0.000849375

27 0.000923375

280.000926375

29 0.000937667

300.001063125

41 0.001396125

42 0.001397959

43 0.001399542

Time

白

Source

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.102

Destination

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

①QQI

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

TCP

0010

0020

0030

52 3d 74 76 90 f0 7a 3a

002c

386560

7d 78 76 7a 00 00 02 04

54554

54 23

54135

541025

54 587

54110

541723

54113

54 21

543306

54 3389

54 80

54 443

54256

54 1720

548888

54 53

00

00

16 d5

-

40

54650[RST, ACK] Seq=1 Ack=1 Win=8 Len=

54650 [RST,ACK]Seq=1 Ack=1 Win=8 Len=(

54650[RST,ACK] Seq=1 Ack=1 win=0 Len

54650[RST,ACK]

54650

54650

54650

54650

54650[RST，ACK]Seq=1Ack=1Win=8 Le

54650[RST, ACK] Seq=1 Ack=1 Win=9 Le

54650[RST,

54650

54658

54650

0048 06

7a 91

[RST, ACK] Seq=1 Ack=1 Win=θ Len

[RST,

[RST,ACK]

[RST,

[RST,

[RST,ACK]

[RST, ACK] Seq=1 Ack=1 Win=θ Let

f9

ACK]

e0 28 9f ab ca a8 60 12

4e 76 82 6d 08 00 45 00

48 b0 c0 a8 38 66 c0 a8

ACK]

05 b4

ACK]

ACK]

Seq=1 Ack=1 win=0 Len

Seq1 Ack=1 win=e Len

Seq=1 Ack=1 win=θ Len

Seq=1 Ack=1 win=θ Len:

Seq=1 Ack=1 win=0 Lel

R=tv.

..-0-1

Be-

·Z

Adversarial Port Scan Response

B.3.2 Brute Force

For the Brute Force attack evaluation, the experimental setup followed a similar approach to that of the Port Scan, focusing on capturing and analyzing both original and adversarial packets. However, in this case, we leveraged Metasploit's ssh_login module to perform the attack. After demonstrating a successful Brute Force attack using Metasploit, we captured the relevant packets and utilized DiffuPac to generate adversarial versions of those packets. The evaluation focused on comparing the response of original and adversarial packets, observing how successful logins were achieved in both cases.

As shown in Figure 10, we utilized the Metasploit Framework to perform a Brute Force attack targeting the SSH service on the Ubuntu server. After setting the RHOSTS to the Ubuntu machine, we provided both a user file and a password file containing potential login credentials. The ssh_login module systematically attempted each combination of usernames and passwords, ultimately leading to a successful login.

The success of the Brute Force attack is demonstrated in Figure 10, where Metasploit successfully opened an SSH session on the target machine. This provided the necessary confirmation that the attack could be executed under normal conditions, laying the foundation for the subsequent evaluation of adversarial packets.

Following the successful attack, similar to the Port Scan setup, we captured the src-to-dst packets exchanged during the Brute Force attack. These packets were saved as a pcap file, which was subsequently used as input for DiffuPac to generate adversarial Brute Force packets.

Using Tcpreplay, both the original and adversarial Brute Force packets were replayed towards the Ubuntu server. As shown in Figure 11, the response to the adversarial Brute Force packets closely mirrored that of the original Brute Force packets. The packet exchange between the attacker and the target server increased significantly as the correct login credentials were identified, culminating in a successful SSH session for both the original and adversarial attacks.

To further analyze and validate the outcomes of both the original and adversarial Brute Force attacks, we utilized pam_unix(sshd) on the Ubuntu server to log and compare successful login attempts. As depicted in Figure 12, the log entries confirmed that the SSH service recorded successful logins for both the original Brute Force attack and the adversarial Brute Force attack.

FileActions Edit View Help

CreateSession

true

no

Create a new session for every successful login

DB_ALL_CREDS

false

no

Tryeachuser/passwordcouplestored in thecurrentdatab

ase

DB_ALL_PASS

false

no

Add all passwords in the current database to the list

DB_ALL_USERS

false

no

Add all users in the current database to thelist

DB_SKIP_EXISTING

none

no

Skip existing credentials stored in the current database

(Accepted:none,user，userarealm)

PASSWORD

no

A specificpassword to authenticate with

PASS_FILE

no

File containing passwords, one per line

RHOSTS

yes

using-metasploit/basics/using-metasploit.html

RPORT

22

yes

The targetport

STOP_ON_SUCCESS

false

yes

Stop guessing when a credential works for a host

THREADS

1

yes

The number of concurrent threads (max one per host)

USERNAME

no

A specificusername to authenticate as

USERPASS_FILE

no

File containing users and passwords separated by space,0

ne pairper line

USER_AS_PASS

false

no

Try the username as the password for all users

USER_FILE

no

File containing usernames, one per line

VERBOSE

false

yes

Whether to print output for all attempts

View the full module info with the info, or info -d command.

msf6auxiliary(scanner/ssh/ssh_login)>setrhosts 192.168.56.102

rhosts = 192.168.56.102

msf6 auxiliary(scanner/ssh/ssh_login)>set user_file/home/aj/Desktop/Users.txt

user_file→/home/aj/Desktop/Users.txt

msf6auxiliary(scanner/ssh/ssh_login)> setpass_file/home/aj/Desktop/Password.txt

pass_file/home/aj/Desktop/Password.txt

[*]192.168.56.102:22-Starting bruteforce

[+]192.168.56.102:22-Success:'aj:lily'‘uid=1000（aj） gid=1000（aj） groups=1000（aj),4（adm),24（cdrom),27（s

udo),30(dip),46(plugdev),101(lxd) Linux ajnatsu 6.8.0-35-generic #35-Ubuntu SMP PREEMPT_DYNAMIC Tue May 21

07:52:29UTC2024aarch64aarch64aarch64 GNU/Linux

[*] SSH session 1 opened (192.168.56.101:40985 → 192.168.56.102:22) at 2024-07-29 13:53:53 +0900

[*]

Scanned 1 of 1hosts （100% complete)

[*]Auxiliary module execution completed

original_brutefore.pcapng

adversarial_bruteforce.pcapng

Eile Edit yiew Go Capture Analyze Statistics Telephony Wireless Iools Help

Edit

View

05

Capture

Analyze Statistics Telephony Wireless Iools Help

→<>

口

→<>02x

①QQI

tcp.port = 22 && tcp.flags.push == 1

tcp.port  22 88 tcp.flags.,push  1

No.

539 101.918854715

541 101.930762924

543 181.951671215

550 105.475138759

556 105.485162259

559 185.487969589

564 105.510634425

567 165.562674759

571 165.593476384

548 101.920174507

542 101.943827799

545 101.996516632

547 101.996708549

548 181.998632674

561 105,489900384

562 185.491426634

563 185.497742717

565 185.516826342

569 105.562846634

570 185.564054384

572 105.599815842

575 186.114826551

577 186.168515342 192.168.56.102

579 106,161610217

581 106.163064426 192.168.56.102

582 186.165546676 192.168.56.102

Time

192.168.56.101

192.168.56.101

192.168.56.102

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.102

192.168.56.101

192.168.56.102

192.168.56.101

192.168.56.101

192.168.56.102

192.168.56.101

192.168.56.102

Source

192.168.56.102

192.168.56.101

192.168.56.102

192.168.56.102

192.

192.168.56.101

192.168.56.101

192.168.56.102

168.56.102

Destination

192.168.56.102

192.168.56.102

192.168.56.101

192.168.56.102

192.168.56.101

192.168.56.102

192.168.56.102

192.168.56.101

192.168.56.102

192.168.56.101

192.168.56.102

192.168.56.101

192.168.56.102

192.168.56.102

192.168.56.101

192.168.56.102

192.168.56.101

192.168,56.102

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.102

192.168.56.101

192.168.56.101

192.168,56.102

Protocol

SSHV2

SSHV2

SSHv2

SSHv2

SSHV2

SSHv2

SSHV2

SSHV2

SSHv2

SSHv2

SSHV2

SSHv2

SSHV2

SSHv2

SSHv2

SSHV2

SSHv2

SSHV2

SSHv2

SSHV2

SSHV2

SSHv2

SSHV2

SSHv2

SSHv2

SSHV2

1506 Client:

1186 Server:

1506 client:

1186 Server:

218 client:

218 client:

378 Server:

166 Client:

166 Server:

107 client:

106 Server:

378 Server:

166 Client:

166 Server:

159 Server:

166 client:

742 Server:

166 Server:

234Server:

446 Server:

166 Server:

198 client:

198 client:

90 client:

90 client:

Key Lxchange Init

Elliptic Curve Diffie-Hellman Key Exchange

Protocol (SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ul

Key Exchange Init

Elliptic Curve Diffie-Hellman Key Exchange

Elliptic Curve Diffie-Hellman Key Exchange

Key Exchange Init

qng-nqunqn tdg*6Hssuado-ez-Hss)tooogoid

Key

Exchange Init

No.

799 115.889692722 192.168.56.101

810 119.444465765

800 115.891189472

801 115.900507597

808 115.963860638

827 119.512564598

882 115.913487655

803 115.919681305

805 115.962624972

807 115.962718138

817 119.456986765

819 119.458963390

821 119.462689973

823 119.463961973

825 119.504858557

828 119.518528182

838 119.562256848

832 119.562379765

833 119.563509148 192.168.56.181

834 119.577993848

835 119.580862807

837 119.882039473

839 119.939194599 192.168.56.182

841 119.931182682 192.168.56.181

843 119.931893182 192.168.56.102

844 119.934425682 192.168.56.182

Time

192.168.56.102

192.168.56.182

192.168.56.101

192.168.56.101

192.168.56.102

192.168.56.101

192.168.56.181

192,168.56,102

192.168.56.101

192.168.56.102

Source

192.168.56.182

192.168.56.101

192.168.56.101

192.168.56.182

192.168.56.182

192.168.56.102

192.168.56.181

192.168.56.101

192.168.56.182

192.168.56.101

Destination

192.168.56.102

192.168.56.101

192.168.56.102

192.168.56.102

192.168.56.102

192.168.56.101

192.168.56.102

192.168.56.101

192.168.56.102

192.168.56.101

192.168.56.102

192,168.56,101

192.168.56.101

192.168.56.101

192.168.56.102

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.101

192.168.56.182

192.168.56.102

192.168.56.181

192.168.56.102

192.168.56.101

192.168.56.182

192.168.56.102

SSHV2

SSHV2

SSHV2

SSHv2

SSHV2

SSHV2

SSHV2

SSHV2

SSHV2

SSHV2

SSHv2

SSHV2

SSHv2

SSHV2

SSHv2

SSHV2

SSHv2

SSHV2

SSHV2

SSHV2

SSHV2

SSHV2

SSHV2

SSHV2

SSHv2

SSHV2

1186 Server:

1506 client:

1186 Server:

218 client:

378 Server:

166 Server:

198 client:

166 Server:

107 Client:

106 Server:

218 Client:

378 Server:

166 Client:

166 Server:

198 client:

166 Client:

742 Server:

166 Server:

166 Client:

234 Server:

446 Server:

166 Client:

90 client:

90 client:

Key Exchange Init

Protoco1(SSH-2.0-OpenSSH_9.6p1

Key Exchange Init

Protoco1 (SSH-2.0-0penSSH_7.6p1

Key Exchange Init

Key

Exchange Init

，Flags: 6x918 (PSH, ACK)

[Calculated window size: 31872]

Acknowledgment number （raw): 512551189

oriqinal brutefore.pcaq

Packets: 840 - Displayed: 335 (39.9%)

0020

0030

0040

0000

0010

00 f9 e7 76000001

0f e500000020 9d cb

386691

7a3a46

009802

76 82 6d 52 3d

65 80

dc 40

16 7e

00

40

06

1e

01

74 76 90 fe

2e 1f 1e 8c

47 f3 fe b

45 68 c0 al

08 0n n3 e3

Profile: Default.

，

Acknowledgment number （raw): 2185820363

Flags: 0x018 (PSH,ACK)

Acknowledgment Number: 1890

Window:249

adversarial_bruteforce.pcapng

(relative ack number)

Packets: 906 - Displayed: 443 (48.9%)

0020

0838

0810

0040.

0080

68 f9 10

Ac 3h000000102e 75

52 3d 74 76 90 f0 7a 3a

0088 5f

38

6500

89 48 68 48 06

1691

d1

000001

Profile:Default

81a2

10

01

a) Original Brute Force b) Adversarial Brute Force

=

aj@ajnatsu:~/Desktop

三

0ct 07 03:10:29 ajnatsu sshd[4256]: Failed password for invalid user postgres fron 192.168.56.101 port 39279 ssh2

0ct 07 03:10:30 ajnatsu sshd[4256]: Connection closed by tnvalid user postgres 192.168.56.101 port 39279 [preauth

Oct 07 03:19:03 ajnatsu sshd[4579]: Connectton closed by authenticating user aj 192.168.56.101 port 44193 [preaut

0ct 07 03:19:02 ajnatsu sshd[4579]: Failed password for aj fron 192.168.56.101 port 44193 ssh2

h]

Oct 07 03:10:30 ajnatsu sshd[4258]:pan_unix(sshd:auth): authentication failure; lognane= uid=θ euid=0 tty=ssh ru

0ct 07 03:19:03 ajnatsu sshd[4581]:pan_unix(sshd:auth): authentication failure; lognane= uid=0 euid=0 tty=ssh ru

ser=rhost=192.168.56.101user=aj

ser= rhost=192.168.56.101 user=aj

0ct 07 03:10:32 ajnatsu sshd[4258]: Failed password for aj from 192.168.56.101 port 41939 ssh2

0ct 07 03:19:05 ajnatsu sshd[4581]: Failed password for aj fron 192.168.56.101 port 36313 ssh2

0ct 07 03:10:32 ajnatsu sshd[4258]: Connection closed by authenticating user aj 192.168.56.101 port 41939 [preaut

]

h]

h]

Oct 07 03:10:32 ajnatsu sshd[4260]:pan_untx(sshd:auth): authentication failure; lognane= uid=θ euid=0 tty=ssh ru

0ct 07 03:19:06 ajnatsu sshd[4583]:pan_unix(sshd:auth):authentication failure;lognane= uid=0 euid=0 tty=ssh ru

ser= rhost=192.168.56.101 user=aj

ser= rhost=192.168.56.101 user=aj

0ct 07 03:10:34 ajnatsu sshd[4260]: Failed password for aj from 192168.56.101 port 46147 ssh2

0ct 07 03:19:08 ajnatsu sshd[4583]: Falled password for aj fron 192.168.56.101 port 34739 ssh2

0ct 07 03:10:34 ajnatsu sshd[4260]: Connectlon closed by authentlcatlng user aj 192.168.56.101 port 46147 [preaut

h]

h]

0ct 07 03:10:34 ajnatsu sshd[4262]:pan_untx(sshd:auth): authentication failure;lognane= uid=θ euid=0 tty=ssh ru

0ct 07 03:19:10 ajnatsu sshd[4586]:pan_unix(sshd:auth): authentication failure;lognane= uid=0 euid=0 tty=ssh ru

ser= rhost=192.168.56.101 user=aj

ser= rhost=192.168.56.101 user=aj

0ct 07 03:10:36 ajnatsu sshd[4262]: Failed password for aj from 192.168.56.101 port 45723 ssh2

0ct 07 03:19:12 ajnatsu sshd[4586]: Falled password for aj fron 192.168.56.101 port 38335 ssh2

0ct 07 03:10:38 ajnatsu sshd[4262]: Connection closed by authenticating user aj 192.168.56.101 port 45723 [preaut

0ct 07 03:19:14 ajnatsu sshd[4586]: Connection closed by authenttcating user aj 192.168.56.101 port 38335 [preaut

0ct 0703:10:38 ajnatsu sshd[4264]:pan_untx(sshd:auth):authentication fatlure;lognane= uid=0 eutd=0 tty=ssh ru

h]

0ct 07 03:19:14 ajnatsu sshd[4588]:pan_unix(sshd:auth): authenticationfailure; lognane= uid=0 euid=0 tty=ssh ru

ser= rhost=192.168.56.101 user=aj

ser=rhost=192.168.56.101user=a]

Oct 07 03:10:41 ajnatsu sshd[4264]: Connection closed by authenticating user aj 192.168.56.101 port 42359 [preaut

0ct 07 03:10:40 ajnatsu sshd[4264]: Failed password for aj fron 192.168.56.101 port 42359 ssh2

Oct 07 03:19:17 ajnatsu sshd[4588]: Connectton closed by authenttcating user aj 192.168.56.101 port 42133 [preaut

0ct 07 03:19:16 ajnatsu sshd[4588]: Failed password for aj fron 192.168.56.101 port 42133 ssh2

h]

h]

0ct 07 03:10:42 ajnatsu sshd[4266]: Accepted password for aj fron 192.168.56.101 port 38381 ssh2

0ct 07 03:19:18 ajnatsu sshd[4590]: Accepted password for aj fron 192.168.56.101 port 34549 ssh2

Oct 07 03:10:42 ajnatsu sshd[4266]: pam_unix(sshd:session): session opened for user aj(uid=1000) by aj(uid=θ)

(0=pn)e q (00t=pn)e asn o pauado uossas :(uossas:pyss)xunwed :[06s]pyss nseue 8t:6t:0 0 0

a) Original Brute Force b) Adversarial Brute Force

C Ablution Study

This ablation study assesses the impact of substituting the pre-trained BERT model with a standard transformer architecture in the denoising process of DiffuPac. The focus is on assessing each model's ability to accurately reconstruct and generate adversarial packets that can effectively evade detection.

Non-BERT-based. This variant of DiffuPac employs a standard transformer architecture for the denoising process, as outlined in the DiffuSeq methodology (Gong et al. [2023]). Notably, this model is trained from scratch without leveraging any pre-trained weights, thus relying solely on the information provided during the training phase.

BERT-based. The existing DiffuPac configuration, which incorporates a pre-trained BERT model for the denoising process, was utilized. This setup takes advantage of BERT's pre-trained weights, enabling enhanced semantic understanding and contextual relevance during packet reconstruction.

Both models were trained under identical conditions and evaluated based on their success in generating adversarial packets that evade detection. The results, as depicted in Table 8 , highlight the comparative effectiveness of each model.

As shown in Table 8, the BERT-based DiffuPac significantly outperforms the non-BERT variant, demonstrating superior ability to reconstruct packets that blend seamlessly into normal traffic. The BERT-based model excels in pairing malicious packets with normal packets that exhibit high contextual relevance. This strategic pairing allows the generated adversarial packets to blend seamlessly into normal traffic, thereby increasing their likelihood of evading detection. In contrast, the nonBERT-based model, which lacks this pre-trained knowledge, resorts to random matching of malicious and normal packets, leading to a noticeable reduction in the effectiveness of the generated adversarial traffic.

An intriguing aspect of DiffuPac is its dual utilization of the pre-trained BERT model: first, as the denoising engine during the packet reconstruction process, and second, as a critical tool for identifying contextually relevant packet pairs. To the best of our knowledge, this dual application of a

(a) Botnet

pre-trained model within a diffusion framework is unprecedented, further highlighting the innovation and effectiveness of our approach.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction clearly state the key contributions of our paper, including the development of DiffuPac, the integration of a pre-trained BERT model with diffusion model, and the novel approach to generating adversarial packets. These claims are substantiated by both theoretical analysis and experimental results presented in the paper. The scope of our research, including the focus on real-world constraints and the effectiveness of the proposed model against advanced NIDS, is accurately reflected in the paper.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have included the limitations in the main paper.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The proofs for all theorems and formulas are clearly provided both in the main paper and supplemental material.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The paper provides a detailed explanation of the model architecture and the experimental setups. This includes a thorough description of the pre-trained BERT model integration, the diffusion model, and the specific configurations used in the experiments. Additionally, all hyperparameters, dataset details, and evaluation metrics are disclosed to enable replication of the results.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: Due to ethical concerns, we have decided not to publicly share the data and code. Additionally, much of the code is based on existing codebases from other sources. However, we have provided a full and detailed explanation of the model architecture and its implementation in the paper and supplemental material. This includes comprehensive descriptions of the experimental setups, hyperparameters, and step-by-step instructions to facilitate reproducibility.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( ❤/a116/a116♣/a115✿✴✴♥✐♣/a115✳❝❝✴ ♣✉❜❧✐❝✴❣✉✐❞❡/a115✴❈♦❞❡❙✉❜♠✐/a115/a115✐♦♥/a80♦❧✐❝② ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( ❤/a116/a116♣/a115✿ ✴✴♥✐♣/a115✳❝❝✴♣✉❜❧✐❝✴❣✉✐❞❡/a115✴❈♦❞❡❙✉❜♠✐/a115/a115✐♦♥/a80♦❧✐❝② ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We have provided the full detailed explanation regarding the training process and the experimental details both in the main paper and supplemental material.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: The experiments conducted in this paper primarily focus on evaluating the success of generating adversarial packets. Metrics such as evasion rate, packet field analysis, and probabilistic differences between original and adversarial packets are used to assess the effectiveness of the proposed method.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We have detailed the types of computational resources used and the time of execution for BERT and diffusion model in the supplemental materials.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics ❤/a116/a116♣/a115✿✴✴♥❡✉/a114✐♣/a115✳❝❝✴♣✉❜❧✐❝✴❊/a116❤✐❝/a115●✉✐❞❡❧✐♥❡/a115 ?

Answer: [Yes]

Justification: We have read the Code Of Ethics and strictly follow it.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We have discussed the positive societal impacts and negative societal impacts in the main paper.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [Yes]

Justification: Due to ethical concerns of our paper, we have implemented stringent measures.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All existing assets used in this paper, including code, data, and models, have been properly credited to their original creators. We have cited the original papers that produced these assets and included URLs where applicable.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, ♣❛♣❡/a114/a115✇✐/a116❤❝♦❞❡✳❝♦♠✴❞❛/a116❛/a115❡/a116/a115 has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The paper introduces a new model, DiffuPac, which integrates a pre-trained BERT with a diffusion model to generate adversarial packets. Detailed documentation for DiffuPac is provided, including descriptions of the model architecture, training process, implementation details, and usage guidelines. This documentation is included in the main paper and supplemental materials.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper proposes an adversarial packet generation method, named DiffuPac, to evade detection. DiffuPac integrates BERT and a diffusion model to make the generated packets indistinguishable. To better fit the task, a concatenation strategy and a classifier-free approach are proposed. The experimental results are promising.

STRENGTHS:
1.	This paper proposes a novel adversarial packet generation scheme to test the evasion detection ability of NIDS. 
2.	The paper is well-written and easy to follow.
3.	The performance is good.

WEAKNESSES:
1.	The primary concern is the efficiency of DiffuPac. As a diffusion model is adopted to generate adversarial packet, it may be time-consuming. Can you conduct some experiments to validate the efficiency of your method?
2.	Can you also consider some defenses against such adversarial packets? For example, adversarial training (AT), feature selection (FS), and adversarial feature reduction (AFR) as shown in work [1]


[1] Han, Dongqi, et al. "Evaluating and improving adversarial robustness of machine learning-based network intrusion detectors." IEEE Journal on Selected Areas in Communications 39.8 (2021): 2632-2647.

QUESTIONS:
1.	I understand the effectiveness of DiffuPac in an experimental environment but in the real world, usually, an ML model is adopted to detect evasion for fast response requirements but not a DNN. As DNN is less robust to adversarial examples, will this affect the practicability of DiffuPac?

LIMITATIONS:
Mentioned in Sec.5.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-03

comments: SUMMARY:
This paper proposes a novel strategy named DiffuPac to generate adversarial packets to bypass NIDS detection. DiffuPac encompasses two critical components including the BERT, which captures the semantic meaning of packets and facilitates the embedding and contextual paring. The other one is DiffSeq which makes the malicious part seamlessly integrated into the normal packet pattern thereby improving the evasion against detection. Experiments showcase that DiffuPac outperforms the baseline and is capable of breaking practical detection through simulation.

STRENGTHS:
1. This paper is well-written, with a clear method description.

2. The proposed DiffPac is effective against bypassing detection, the proposed strategy seems reasonable with good explanation.

WEAKNESSES:
1. As stated in the limitation, whether the functionality of these adversarial packets is preserved is not verified. From my understanding, this point is quite important given the practical aspect of DiffPac. If the adversarial package can only bypass detection without functioning as intended, it would undermine the practical utility of DiffPac.

2. It would be beneficial to include additional analysis, such as evaluating the performance without applying contextual pairing and assessing the robustness against potential packet-level defenses. These analyses would provide further insights into the effectiveness and resilience of DiffPac.

QUESTIONS:
Please refer to weakness.

LIMITATIONS:
The authors have listed their limitations and have addressed most of them. One concern about functionality preservation is mentioned in the weakness part.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-11

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents DiffuPac, a novel solution that integrates Bidirectional Encoder Representations from Transformers (BERT) and diffusion models to generate adversarial packets that can evade detection by sophisticated Network Intrusion Detection Systems (NIDS).\nDiffuPac leverages the extensive contextual understanding provided by BERT, which has been trained on diverse datasets representing a wide range of network behaviors, along with the generative capabilities of diffusion models. This fusion results in a sophisticated adversarial tactic where the elements of the attack are seamlessly integrated into the network traffic, making them indistinguishable from legitimate data.\nExtensive experimental evaluations on real-world datasets demonstrate that DiffuPac significantly outperforms existing methods in terms of evasion effectiveness, establishing new benchmarks for the generation of adversarial packets.\n\nSTRENGTHS:\n* Integrates BERT and diffusion models for adversarial packet generation\n\n* Addresses limitations of traditional methods that rely on unrealistic attacker knowledge\n\n* Introduces unique concatenation and targeted noising techniques for seamless blending of adversarial packets\n\n* I really appreciate that authors consider many realistic attacker constraints (e.g., packet integrity, replayability, only control on src-to-dst flows), showing that the authors have solid domain knowledge in network security\n\nWEAKNESSES:\n* There are many unclear points about the high-level idea, methodology, and experiments; see them below. If provided with compelling evidence, I may increase my score.\n\nQUESTIONS:\n1. For the high-level idea, I admit that there have been attempts to transform packet bytes as tokens and use language models for traffic analysis (e.g., [a,b,c]). However, most of them (also including DiffuPac) do not discuss if and how this type of method can handle encrypted traffic. As you may know, modern encryption algorithms like AES can significantly reduce entropy and there will be no one-to-one relations between plaintext and ciphertext; so from my perspective, learning from the encrypted payload is meaningless. Some methods may only use the header fields for tokenization. As there lack of such details, I'm not sure how DiffuPac deals with this concern.\n2. On page 5, line 199: \"... this task employs a dedicated binary classifier to determine the directional origin of network traffic, specifically whether packets in a sequence originate from src-to-dst or dst-to-src.\" Though the idea seems reasonable, I'm curious will there be any cases where packets from both sides are very similar so that the task can be too difficult to converge (e.g., in P2P apps where there are no explicit servers or clients)?\n3. On page 5, line 217: \"the initial packets in a flow contain the most significant information, we limit our analysis to the first three packets of each heavy flow\". Will the first three packets be too few? For TCP connections, the first three packets are typically just for handshakes so the model probably won't learn any useful information. Can the authors explain this, or give some experiments on this parameter?\n4. Could the authors list the mutable and immutable fields in the appendix?\n5. For experiment setup, one of my concerns is about the data preparation. Will the pre-trained model use malicious traffic for training? For one thing, the pattern of malicious traffic can be very abnormal so will that affect the BERT's understanding of normal traffic? For another, in practice, malicious traffic is much less than in the dataset so this setting can be somewhat impractical for a pre-trained model.\n6. Another major weakness of the experiment is that there is only one baseline. However, there are other works on generating (malicious or generic) traffic data, including those generating on feature spaces ([d,e]) or generating on raw bytes ([a,b]). The authors may compare them to improve the experiment, or discuss them in related work and present their fatal limitations that hinder the direct comparison.\n7. As there are two main components in DiffuPac (BERT and diffusion model), an ablation study on the pre-trained model is needed to show the necessity of using the pre-trained representations instead of some simpler representations.\n8. In the limitations, I suggest that the authors discuss the implications of your method facing signature-based NIDS (e.g., Snort). You may conclude with a consideration of the current situation that encrypted traffic is the majority, but that will go back to question 1 again.\n9. I like the authors providing the screenshots of Wireshark as parts of the results. One weakness is that the authors use their own domain knowledge to reveal the important packet fields that should be altered to avoid NIDS. However, the ML/DL-based NIDS may use some other features as the decision reasons which can be different from yours. I think the authors may use some global explanation models (e.g., [f,g]) to verify if the adversarial fields are really the features that NIDSes care.\n\n[a] NetGPT: Generative Pretrained Transformer for Network Traffic, arxiv 2023\n\n[b] TrafficGPT: Breaking the Token Barrier for Efficient Long Traffic Analysis and Generation, arxiv 2024\n\n[c] Yet Another Traffic Classifier: A Masked Autoencoder Based Traffc Transformer with Multi-Level Flow Representation, AAAI 2023\n\n[d] Flow-based network traffic generation using Generative Adversarial Networks, SIGCOMM 2022\n\n[e] Knowledge Enhanced GAN for IoT Traffic Generation, WWW 2023\n\n[f] AI/ML for Network Security: The Emperor has no Clothes, CCS 2022\n\n[g] Interpreting Unsupervised Anomaly Detection in Security via Rule Extraction, NeurIPS 2023\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work presents DiffPac, a method that leverages BERT and a diffusion model to generate adversarial packets aimed at evading network \nintrusion detection systems. Compared to the approach by Han et al. (2021), DiffPac demonstrates superior effectiveness, with most machine learning-based classifiers trained on adversarial packets generated by DiffPac successfully evading detection across six different attack scenarios.\n\nSTRENGTHS:\n* Approach: The paper presents an interesting intuition for generating adversarial packets using a diffusion model, which is a creative approach in the field.\n\n* Methodology: The methodology is clearly presented, making it easy to understand the steps and processes involved in the proposed DiffPac method.\n\nWEAKNESSES:\n* Motivation: The research motivation could be more clearly and concretely articulated. It would be beneficial to highlight the specific niche of this study in relation to existing work, such as Network Emulator (NetEM) and Metasploit (Homo-liak et al., 2018), Generative Adversarial Network (GAN) and Particle Swarm Optimization (PSO) (Han et al., 2021), and Reinforcement Learning (RL) (Hore et al., 2023). Rather than broadly stating that \"their efficacy in evading detection in real-world conditions remains suboptimal\" (line 64), the paper should detail the unique challenges and gaps that DiffPac addresses.\n\n* Method for Preserving Packets’ Integrity: The approach to preserving packets' integrity by only mutating TCP flags, TTL, and window size, rather than the payload, is unrealistic for real-world conditions where payloads often contain the malicious behaviors. This limitation should be addressed to enhance the practical applicability of the method.\n\n* Evaluation: The evaluation settings could be more closely aligned with those used by Han et al. (2021) to make it easier for the readers to understand the differences and contributions of this work.\n\nQUESTIONS:\n* Data Pre-processing: Could you explain the rationale and advantage of transforming IP numerical values into hexadecimal representation and treating them as discrete values for workpiece segmentation? What benefits does this approach offer?\n\n* Formula (4): In Formula (4), does  S^{ben+mal}  denote z_t ? \n\n* Dataset: How many adversarial packets are included in the test set? Providing a statistical description of the datasets used would enhance clarity and help understand the dataset composition.\n\nLIMITATIONS:\nThe limitation section highlights three main concerns:\n\n* The actual malicious functionality of the generated adversarial packets has not been thoroughly examined.\n\n* There is inconsistent performance across the six attack scenarios tested.\n\n* There is a potential risk associated with making the approach open-source, which could be exploited maliciously.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes DiffuPac, an adversarial packet generation model that aims to evade Network Intrusion Detection Systems (NIDS) without depending on detailed knowledge of NIDS components. DiffuPac combines a pre-trained BERT model with a diffusion model. Experimental results show that DiffuPac outperforms traditional methods by 8.83%.\n\nSTRENGTHS:\n- This work studies an interesting and important issue about evading NIDS detection based on deep learning methods.\n- I appreciate that the author divides the traffic data into mutable fields and immutable fields for modification.\n- The experimental results support the effectiveness of the proposed method to evade detection.\n\nWEAKNESSES:\n- The paper achieves the following goals through expensive experiments (more than 30 hours of training on RTX 4090): select the normal packet sequence that is most similar to the malicious packet sequence, and modify the limited mutable fields (e.g. TCP flags, TTL, and window size) in the malicious packet sequence header according to the normal packet sequence. Given the very limited mutable fields modified within the packets and the lack of ablation studies in the experimental section, I doubt the necessity of using BERT and DiffuSeq. It seems that a simpler approach involving feature similarity matching, coupled with selective field selection and replacement, could potentially achieve the same goal without the complexity introduced by these advanced models.\n- In the diffusion model where all mutable fields of the entire malicious packet sequence are regenerated, there is a concern about whether this process affects its original functionality. Although the modification effects on specific fields of individual packets are shown in the appendix, whether the author replays these generated packets in a real network environment to verify its functionality, rather than simply viewing them through Wirshark.\n- Some minor issues: The paper needs improvement in writing quality. The connections between different components are not explicitly explained, and the specific format of the packet sequence input to the diffusion model is not detailed. In addition, the BERT-related content is very redundant in Sec. 3 and Figure 1. However, the related content is highly similar to ET-BERT and is only used to calculate the most relevant benign and malicious packet sequences in the \"fine-tuning\" stage (in fact, I think the operations in this stage do not belong to the category of \"fine-tuning\").\n\nQUESTIONS:\nPlease see the above-mentioned weaknesses. I will confirm the author's response during the rebuttal.\n\nLIMITATIONS:\nThe authors discussed the limitations in this work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}