Augmented Memory Replay-based Continual Learning Approaches for Network Intrusion Detection
Authors: Suresh kumar Amalapuram
Sumohana S. Channappayya
Bheemarjuna Tamma
Venue: NeurIPS 2023 poster (2023)
Keywords: Continual learning
Class imbalance
scalability
Network intrusion detection
and Cybersecurity
TL;DR: We present two novel contributions aimed at improving the performance of continual learning based network intrusion detection in the context of class imbalance and scalability.

Abstract: Intrusion detection is a form of anomalous activity detection in communication network traffic. Continual learning (CL) approaches to the intrusion detection task accumulate old knowledge while adapting to the latest threat knowledge. Previous works have shown the effectiveness of memory replay-based CL approaches for this task. In this work, we present two novel contributions to improve the performance of CL-based network intrusion detection in the context of class imbalance and scalability. First, we extend class balancing reservoir sampling (CBRS), a memory-based CL method, to address the problems of severe class imbalance for large datasets. Second, we propose a novel approach titled perturbation assistance for parameter approximation (PAPA) based on the Gaussian mixture model to reduce the number of \textit{virtual stochastic gradient descent (SGD) parameter} computations needed to discover maximally interfering samples for CL. We demonstrate that the proposed approaches perform remarkably better than the baselines on standard intrusion detection benchmarks created over shorter periods (KDDCUP'99, NSL-KDD, CICIDS-2017/2018, UNSW-NB15, and CTU-13) and a longer period with distribution shift (AnoShift). We also validated proposed approaches on standard continual learning benchmarks (SVHN, CIFAR-10/100, and CLEAR-10/100) and anomaly detection benchmarks (SMAP, SMD, and MSL). Further, the proposed PAPA approach significantly lowers the number of virtual SGD update operations, thus resulting in training time savings in the range of 12 to 40\% compared to the maximally interfered samples retrieval algorithm.

Augmented Memory Replay-based Continual Learning Approaches for Network Intrusion Detection
Suresh Kumar Amalapuram', Sumohana S Channappayya" and Bheemarjuna Reddy Tamma" Dept. of Computer Science and Engineering", Dept. of Electrical Engineering Indian Institute of Technology Hyderabad, India {cs19resch11001, sumohana, thr ttrrtbittthaaaee ac.in
Abstract
Intrusion detection is form of anomalous activity detection in communication network traffic. Continual learning (CL) approaches to the intrusion detection task accumulate old knowledge while adapting to the latest threat knowledge. Previous works have shown the effectiveness of memory replay-based CL approaches for this task. In this work, we present two novel contributions improve the performance of CL-based network intrusion detection in the context of class imbalance and scalability. First, we extend class balaneing reservoir sampling (CBRS), memory- based CL method, to address the problems of severe class imbalance for large datasets. Second, we propose a novel approach titled perturbation assistance for parameter approximation (PAPA) based the Gaussian mixture model to reduce the number of virtual stochastic gradient descent (SGD) parameter computations needed discover maximally interfering samples for CL. We demonstrate that the proposed approaches perform remarkably better than the baselines on standard intrusion detection benchmarks created over shorter periods (KDDCUP' 99, NSL- KDD, CICIDS- SECIDS-2017/200 2017/2018, UNSW-NB15, and CTU-13) and longer period with distribution shift (AnoShift). We also validated proposed approaches standard continual learning benchmarks (SVHN, CIFAR-10/100 100, and CLEAR-10/100) and anomaly detection benchmarks (SMAP, SMD, and MSL). Further, the proposed PAPA approach significantly lowers the number of virtual SGD update operations, thus resulting in training time savings in the range of 12 to 40% compared the maximally interfered samples retrieval algorithm.
Introduction
Learning from a continuum of data with distribution shift over time is known as continual/ifelong learning (CL) [I] (2) CL algorithms aim to preserve learned knowledge (stability) while embracing the knowledge of new tasks (plasticity). The conflict in balancing stability and plasticity often leads to the problem of catastrophic forgetting (CF) 3 4. In other words, past tasks' knowledge degrades time due to interference from the newer task. Unlike computer vision 5
6 7 8) natural language
processing 10), and speech processing [2], CL applications many real-world problems are explored, and one such exemplar is network intrusion detection (NID) in communication networks. Intrusion detection is a form of distribution shift detection that aims detect anomalous activity patterns in the network traffic 13 [4]. To ensure protection against novel cyber-attacks, network intrusion detection systems (NIDS) must evolve continuously and effortlessly lear from the newer attack data.
The concept drift/distribution shift is prevalent in cybersecurity, malicious behavior can shift either suddenly or gradually over time [5] While NID formulated as anomaly detection (AD) is typically trained normal data (zero positive learning [16] and is immune the drift of malicious
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

behavior. however, its performance can be severely affected when the distribution of normality shifts [14] (open-world setting). In the cybersecurity domain, shift in normality can occur when new patches, software updates, devices, or protocols are introduced [14] 13]. This work considers normality shifts and distribution shifts in attack data, So, we formulate our problem as a continual
learning-based supervised binary classification problem (SBCP).
Class imbalance and Task-free nature: SBCP formulation differs from the traditional classification problem as the former one possesses two challenging characteristies l..... class imbalance (CI) and class overlap. In this work, our focus is primarily on mitigating the adverse effects of class imbalance on performance. Here, CI means different number of training examples (n₂) for each class in the dataset. The majority class contains the highest number of training examples, whereas minority classes are the last few classes when sorted in non non-increasing order based on 74, Cs said to be severe when the difference between the minority class training examples (n²) is relatively high (e.g., the difference between the number of samples of DDoS attack-HOIC and SQL Injection attack (minorityy classes of the CICIDS-22 2010 18 dataset is 2.1 million). Intrusion detection generally operates in a streaming environment. As result, task-free continual learning [9] becomes a natural fit for such applications. Working in task-free CL setup assumes no information about task boundaries and the number of samples per class.
Continual learning with shallow methods: We study the feasibility of shallow methods (non-neura) networks) in the CL setting by conducting preliminary experimenting using the random forest (RF) algorithm (a popular algorithm in NIDS literature) on CICIDS-2017 and SICIDS-2015 datasets. We made the following observations: First, RF exhibits higher CF (especially on minority classes) on previous tasks while learning new tasks. Second, augmenting RF with buffer memory partially reduces the CF on previous tasks. More details on this experiment are available in the supplementary material AA.: To conclude, compared to deep learning (neural network) methods, shallow methods exhibit higher CF, thus diminishing their performance in the CL setting.
Prior art [20] has demonstrated the suitability of CL for addressing the NID problem. This work shown the adverse impact of severe class imbalance on the performance of NID algorithms. This work also demonstrated the usefulness of memory replay (MR)-based approaches for this task. Generally, class imbalance-awaan aware MR methods (e.g., class balancing reservoir sampling (CBRS) (2I) perform a periodic memory population operation to handle class imbalance. partieular, CBRS distinguishes majority and minority classes based on the respective class sample count (local information) from the buffer memory. However, this strategy has a pitfall when the count of minority class samples in the finite buffer memory appears to come from majority class. This situation could occur when there is a severe imbalance among the minority classes, especially in larger datasets. Larger datasets mean the number of training samples in the order of millions (CICIDS-2018 AnoShift). In this work, we extend the CBRS method (and call it ECBRS) to address this issue. The proposed ECBRS always prioritizes the majority class for sample replacement using additional global information about class imbalance.
Table 1: Timing comparison between the virtual SGD operations and the total training of the MIR algorithm. Each experiment is repeated five times, and mean values are reported for virtual SGD ops time and total train time.
Other families of MR techniques like maximally interfered retrieval (MIR) (30) and gradient- based memory editing (GMED)
quantify the significance of samples in the buffer memory using temporary (virtual) stochastic gradient de- scent (SGD) parameter updates. However, this computation overhead consumes nearly 40 to 50% of the total training time, as shown in Ta ble Motivated by this drawback, we propose novel approach known as perturbation assis- tance for parameter approximation (PAPA) for estimating the virtual SGD parameter (VSP) up- dates using the Gaussian mixture model (GMM) with low overhead.
(time seconds)
Datasel
VSP ops time
Total train time
VSP ops propertion
CICIDS-2017
102.9
254.3
40.44%
UNSW-NB15
34
132.4
428 2
30-9 9
CTU-13 X1
142.4
398.1
35.7%
KDDCUP'S
420.8
45.3%
AnoShin
499.4
12104
41.2%
CICIS--2IIS
3315.0
7620
43.5%
SVHN 27]
112.0
217.0
51.6%
CIFAR-10 28
58.6
118.6
49.4%
CIFAR-100
88.56
CLEAR-10
128.2
265.9
18.2%
CLEAR- 100
703.6
1490
47.2%
To summarise, our key contributions in this work are:
1. An extension to the CBRS, dubbed ECBRS, to deal with severe class imbalance by always undermining the majority class by maintaining the global information about class imbalance. Further, FCBRS can be used as a memory population policy in conjunction with the existing memory replay-based approaches.
2

---

2. A perturbation assistance for parameter approximation (PAPA) method for estimating VSP updates that significantly reduces the training time for methods like MIR and GMED, leading to improved scalability.
3. A demonstration of the improved performance of the proposed methods relative to the considered baselines using standard network intrusion detection, computer vision, and anomaly detection benchmarks (including AnoShift and CLEAR-100,, whose training data spans ten years). Further, we show that using the PAPA method results in training time savings of 12 to 40% (relative to the baselines), thus it is highly suitable for large-scale training.
2 Related Work
is one of the fundamental tools in security infrastructure safeguard against cyber-attacks at the host network (NIDS) perimeter level 34] A signature-hased NIDS uses a rule-based engine to identify known patterns, whereas anomaly-based NIDS identifies patterns that deviate from normal behavior 35 NIDS built using techniques ranging from traditional machine learning to deep learning and reinforcement learning [36]. Typically, network intrusions represent a tiny subset of network traffic (37. As result, a heavy class imbalance is observed in NID datasets, making the intrusion detection performance vulnerable. Sampling (algorithm) based approaches have been proposed in the literature to mitigate this problem [38 39 40). But, this work focuses on building an NIDS that adapts to both benign and attack data distribution shifts without any aid of sampling strategies handle the CI effect.
Intrusion detection: An intrusion detection system (IDS) 32 33
Continual learning: Recently, there has been a significant effort to expand the horizon of continual learning by mitigating the impact of CF
4 These efforts can be broadly grouped into the following
families 3 regularization-batt based [41] 42 expansion-base
43), meta-learning-boe based 44 and
rehearsal or memory replay-based approaches.
A subset of rehearsal methods that focus on class imbalance are CBRS [2] and partition reservoir sampling 45), in which samples are preserved in memory in proportion to their class strength. However, these methods do not handle severe CI between the minority samples. The proposed ECBRS addresses this CI issue. Further works like maximum interfered retrieval (MIR) [30] and sudieettbbseed sample editing (GMED) [31] use virtual SGD parameter updates to identify the most interfered samples. These are samples in the replay memory that suffer increase in loss due to previous parameter updates. However, for large-scale training, these virtual update computations add significant overhead. Unlike these, our proposed PAPA method significantly reduces this overhead, making it suitable for training over large datasets.
Manity
5022
Astea
AUSCA
Astrnd
(a) Running statistics
(b) Memory distribution
(c) CBRS
(d) ECBRS
Figure I:
Comparison between CBRS and ECBRS over an imbalanced stream from a CICIDS-2018
dataset setting with a memory size (M) = 1500. (a) Running statisties indicate the number of classwise samples seen so far. (b) Memory distribution represents the strength of each class in buffer memory at a particular instance. Upon the arrival of ten new samples from the class attack5, using CBRS, the memory distribution changes to (c). However, using ECBRS, the memory distribution changes to (d). For (c) and (d), the class with the red-colored bar is chosen for replacement, the green-colored bar class receives new samples, and the class with the blue-colored bar remains intact. Detailed illustrations of these configuration changes for CBRS and ECBRS are presented in the supplementary material (SM:A.3).
3 Methodology
In this section, we present the proposed ECBRS and PAPA methods. In the ECBRS method, we adopt two definitions introduced in the base paper of CBRS (2T). They are full and largest classes. When particular class contains the most instances among all the different classes in the memory,
3

---

we call it the largest. Two or more classes can be the largest if they have the same size. We call class full it it currently is, or has been in one of the previous time steps, the largest class. Once a class becomes full, it remains so in the future. More details of the CBRS algorithm are given in the supplementary material (SM:A.14).
3.1 ECBRS: An Extended Class Balancing Reservoir Sampling Algorithm
The legacy CBRS algorithm assigns higher probability weights to minority class samples using a weighted replay approach. However, these weights are computed based on the number of data samples in the buffer memory (local information). This approach has a limitation when the class imbalance present between the minority classes themselves is severe. This limitation is illustrated in Figure I where running statistics of each class is the corresponding class running frequency(refer (rreeeerrrffeFiuurreeerr la). and benign is the maximal (majority) class. Typically, the benign class samples must be chosen for replacement whenever new minority class samples arrive. However, CBRS treats all classes equally for a replacement to accommodate newly arrived attack5 (minorityy class samples (instead of the benign class) This is because CBRS relies on local information for sample replacement. From Figure [b] all class samples appear to be the majority classes in the buffer memory distribution. As a result, different class samples in memory are uniformly selected for sample replacement (refer[Figure lc] in which red-colored classes are chosen for sample replacement for accommodating newly arrived attack5 class samples). This choice will have an adverse impact on large-scale training, where the class imbalance between the minority classes is significant. Motivated by this, we extended the CBRS to rely on each class's s running statistics (global information) for replacement decision- making. Our method (ECBRS) will always prefer the class with higher global information until a threshold (refer Equation I). later it will choose the class with the next highest running statistic value, thereby ensuring that a majority class sample is replaced in the memory buffer. In summary, we use original class labels to organize the buffer memory to learn SBCP and choose class samples in memory with higher running statistic values for the replacement to accommodate newly arriving samples.
Further, the class imbalance among minority classes poses another challenge known as the conflict of equal weights. It will occur whenever the majority (maximal) class samples are completely replaced or underrepresented in the buffer memory. So, undermining the chosen minority class based on global information will lead to the loss of the essential samples in combating the class imbal- ance problem. This drawback can be mitigated by disallowing the undermining of the minority class beyond a certain threshold y. In ECBRS method, we choose this parameter based on global information. Thus, we guarantee that at- tack class samples can never be replaced beyond this threshold. From Figure Id it can be ob- served that the proposed ECBRS initially chose benign class samples for replacements to accom- modate newly arriving attack5 class samples, as it has the highest running statistics value. In the process of sample replacement, once it reaches the threshold of the benign class (+(-)= 262), ECBRS selects the class with the next highest running static value (attack4). Thus remaining, different class samples (colored blue) will re- main intact. We compute 77-) )) (for class i) as the expected number of samples be present based on the global information, as shown in Equa tion
Algorithm 1 Memory population for ECBRS
Input: data stream: (zi, Wo]== number of currently stored instances of class (c= Mi): Mc. number of stream instances of class y encountered so far: The
for = to n do
if memory is not filled then store (zt, 2t);
else
if y) is not full class then
select a class that is the largest, having higher running statistics value and non-zero samples with me 2 7(c) in the buffer. Otherwise, select class with the next higher running statistic value with m. 2 7(c); overwrite the selected class sample with (mi, ve);
else
sample u ~ Uniform(O, 1);
if u v ma/n.. then
pick a stored instance of class e Ma at random and replace it with wi); else
Ignore (x,, end if end if
(i) = m x w(1),
end if
end for
where i is the class index, w(1) = exp(-n.)
Σ, exp( n
m is the buffer memory capacity, 7(),, n, are the threshold and running class frequency for class i, respectively, and w(i) is the weight associated with class i, w(i) is the softmax of the negative running

---

frequencies favoring the minority classes. In our experiments, we also observe that computing single loss over the replayed and stream samples achieves better performance compared to CBRS (which involves computing two-component loss). One pragmatic reason for this behavior could be that concatenating replay samples with stream samples achieves better batch-wist class balance. We present the pseudo-code of the proposed ECBRS in Algorithm 1.
Tega u 69
(a) CICIDS-2017
(b) CICIDS-2015
(c) CIFAR-10
(d) CLEAR-10
Figure 2: 1-SNE visualization of regular SGD updates and virtual SGD updates. An ML. trained on the CICIDS-2017 and CICIDS-2018 datasets. A ResNet-18 (pretrain (46) model is trained on the CIFAR-10 and CLEAR-10 datasets.
3.2 Perturbation Assistance for Parameter Approximation (PAPA)
Memory replay-based CL techniques such as MIR and GMED select subset of samples from the buffer memory based on the loss incurred during virtual stochastic gradient descent (SGD) parameter (VSP) updates. Samples that incur higher loss preferred. These updates computed using the current batch of the incoming stream samples. Further, these updates are ignored after the subset selection. For lengthy data streams (used to train very deep models), these frequent VSP updates result significant computational overhead (due to SGD operations). Each SGD involves computing the gradients for larger weight matrices, leading to increased computational overhead.
Motivated by this problem, we try to understand the relationship between the regular and vir- tual SGD update operations. Towards this, we plot various t-SNE visualizations CICIDS- 2017, CICIDS- 2018, CIFAR-IO, and CLEAR- 10 datasets. The CIDIDS-2017 and CICIDS- 2018 datasets are used to train an MLP, and the CIFAR-10 and CLEAR-10 datasets used to train a ResNet-18 model (refer |Figure 2). We make the following observations from these plots: virtual SGD parameter update is slowly varying process. In other words, the VSP over- laps with or is scattered closely around the regu- lar SGD updates. One way to capture this notion is to sufficiently perturb the regular SGD param- eter (RSP) updates estimate the VSP updates. Then, the next question is: how to quantify the perturbation required for given task?
(a) CICIDS2017
(b) CICIDS2018
(c) CIFAR-10
To quantify the perturbation, we model the error (difference between the VSP and RSP) distribu- tion of randomly chosen model parameters on diverse set of benchmarks. These shown in Figure 3 We observe that the error values of the chosen parameters exhibit skewed Gaus- sian (heavy-tailed) distribution. We model this erтor distribution using two-component Gaus- sian mixture model (GMM). However, mod- elling individual error components with a GMM is time- mone-onnu process. So, we model the joint distribution P(O.)) of error values of all the parameters using GMM, whose marginals remain Gaussian. As a result, each parameter- level error distribution is preserved.
(d) CLEAR-I0
Figure 3: For randomly chosen model parameters, the plots show the distribution the error between the regular SGD parameter updates and virtual SGD parameter updates on various datasets. In (a) and (b), MLP is trained CICIDS-2017 and CICIDS-2018 In (c) and (d), a ResNet-18 is trained CIFAR-10 and CLEAR-10.

---

Thus, our approach for approximating the VSP is mathematically formulated as a simple additive model which is given in Equation 2
eppn -rpu +
(2)
The perturbation Z is drawn from P(O,) where P(0+) T T1 11, E1) + T2 Σg), where (11, 122, are the mean vectors and Σι, Σg are the covariance matrices of the two Gaussian components, respectively. 11, T2 are the mixing coefficients and Op is the most recent regular parameter update.
We use the MIR algorithm for the first CL task in our proposed approach to estimate the error distribution (ED). This ED is used to train the GMM once, and this GMM is used in all remaining tasks. Our empirical study also confirms that different first task (used to construct ED) has no adverse effect on the performance results (refer SM:A.10.2). The pseudo-code of the proposed PAPA method is outlined in Algorithm 2. Experiments and Analysis
GMM Training:
Datasets and Tasks Formulation: We conduct the experiments in the domain- incremental (class labels space fixed to benign and attack, whereas data space hori- zon may change) learning approach, simi- lar to the prior study (20). We created tasks by dividing the benign data and combin- ing them with attack class samples. This approach ensures that each task contains a mix of benign and attack data, maintaining the class imbalance resembling real-world network traffic. We created five tasks for KDDCUP'99 99 (25) and NSL-KDD 47). ten for CICIDS- 2EEDDS-2117/000 20201
Algorithm 2 Perturbation assistance for parameter approxi-
mation
1: Input: data stream (zs, (+); Yim initial task Ti, maximally interfered retrieval algorithm (MR((.). gaussian mixture model with two components G.M.M, buffer memory M, perturbation arry P. policy to update buffer memory function approximator f(z y θ), parameters after updating on previous batch of samples Orpu.
2: for the task Ti do
3: sample batch B, ~ Ti
4: render 95-1: parameters after updating on Bj-1
5: Run (IR(BB))
store θ, parameters after updating on B3,
7: store (6, - -1) to
22
48 nine for
8: end for
UNSW-NB 23 49 50 51 52), and ten for AnoShift (66 benchmark.
9: train a G.MM using P
10: for remaining tasks do
Existing network intrusion detection bench- short time periods and may not exhibit natural distribution shifts [3] (except AnoShift). Distribution shifts are quantified by the op- timal transport dataset distance (OTDD) (refer to Table 2). Large OTDD values in- dicate higher distribution shift. More de- tails on how OTDD values are computed for each dataset are available in SM:A.7. Due the absence of distribution shifts in NIDS datasets, the proposed approaches were validated using computer vision (CV) CL benchmarks. Furthermore, the ease of visualizing distribution shifts makes CV benchmarks suitable for validating the proposed approaches. For CV benchmarks, we randomly selected one or more classes as the attack classes. We split and distributed the attack class data among the remaining benign classes. This way of class split- ting ensures experiments formulated using vision benchmarks are identical to the intrusion detection experiments. Specifically, we created nine tasks for SVHN 27 and CIFAR-I0 28). Similarly, we created ten tasks for the CIFAR- 100 28 benchmark using the coarse/super class labels. The data spanned over ten years for the CLEAR- 10/100 (2) benchmark and contained all the classes' natural temporal evolution for each year. We created ten tasks the data spans decade. We also maintained 1:100 class imbalance ratio per task in CIFAR- 100, CLEAR- 10, and CLEAR- 100 experiments. More details about datasets, preprocessing (and feature selection), and task formulations are presented in SM:A.44 SM:A.5, and SM:A.66
11: sample batch B from current task
12:
sample Z ~ G.MMM
marks artificially created
13:
Oupn. 6 2
14:
S compute interfere samples using Cppn on M
15: Bent subsample from S based on certain criteria
16: Brew B Bist
17: train f(: +) with Baew
18: update M with using B
19:
end for
We compare the proposed ECBRS method with elastic weight consolidation (EWC) 41 synaptic intelligence (SI) [42 (regularization methods), memory((gradient)-aass algorithms like gra- dient episodic memory(GEM) [53] A-GEM (54). gradient-based sample selection (eSSeygredy) [55] and rehearsal-based methods like MIR 30 and CBRS (1). We compare the proposed PAPA with the
Baselines:
MIR algorithm in all facets. We deliberately omit partition reservoir sampling (45) and GMED 3]

---

as the former approach is designed for the multi- classification setting, and example editing of the latter one is trivial for intrusion data. Additionally, we do not consider INSOMNIA [56], this work does not operate in the explicit continual learning setup and relies on existing works to identify distribution shifts and pseudo-label generation.
Architecture details: The ResNet-18 [46] architecture (pre-trained on Imagenet) is used as the backbone for SVHN, CIFAR-I0/100 and CLEAR-100100, followed by fully connected network. We use a multi-layer, fully connected network for the NID datasets. We use RMSProp optimizer for intrusion detection and SGD with Nesterov momentum 0.9 and weight decay 10 for computer vi- sion benchmarks, respectively. A batch size of 128 is used for MNIST, SVHN, CIFAR- 10/100, 10 for CLEAR-10/1IE and 1024 for the rest of the datasets except for NSL-KDD. for which is 500. More details on batch size, architecture, hyperparameters, and implementation are presented in SM:A.11.
Table 2:
Average optimal transport dataset distance (OTDD) values of se- lective intrusion detection and computer vision benchmark datasets.
Memory size (M): We use M=13333 for the CICIDS- 2017, 2018 and AnoShift datasets. For the KDDCUP'99 dataset, MM=533,, and M==133 for the NSL-KDD dataset. M=500 for the SVHN and CIFAR-10/100 datasets. For the CLEAR-10 dataset, MM=666, and M=2666 for CLEAR- 100 datasets. 75% of the buffer memory samples are used for replay. Additional details and an ablation study on M are presented in SM:A.10.2.
Dataset
Avg. OTDD values
CICIDS-2018 [8
0.0508
CICIDS-2017
18
0.1527
C3U-13 24
4.44
CIFAR-10 28
CIFAR-IO6 (2
3139
Evaluation metrics: We chose metrics similar to 26 15
the receiver operating characteristic area under the curve (ROC-AUC) metric and the precision- recall area under the curve (PR-AUC) metrie for both benign and attack data denoted as PR-AUC (B) and PR-AUC (A), respectively.
AnoShift 226
7189
Specifically,
Table 3: Performance results comparison of the proposed ECBRS method with the baselines on network intrusion detection and computer vision benchmarks. We report the arithmetic mean with each experiment repeated five times independently. The performance results of the MIR using ECBRS memory population method are highlighted in light grey.
KDOCUP"9
NSL-KD0
CICIDS-2117
CICIDS 2018
Racliie PR.ADC
PR.AIIC.R) (B)
ROC-ATC
PR.AICC
BOC-ALY ALC
R0C-MCC
PR.ATIC
(8)
ROC-AUC
1.000
01919
11:277
0.488
01762
6EX
1,000
2913
0339
0.56
0.41
01550
0.913
FL565
0.591
0229
0.762
0
A-EEM [c3
1,000
2,912
0339
0.55%
0.35
11545
0.64
01.52
0.096
0.36
0.762
C.5
Mil 10
1.000
5.617
0.955
0.553
11918
0.265
0.788
0.5
CBRS 40
1.000
3,255
0.395
0.509
0.96
11560
0.999
0.550
0.990
2999
0,500
CO99
ECBRS
1.000
6.991
MIR ECBRS
1000
0.590
0.9%%
D156
0.565
1.00
0 1999
0.91
0.991
0.993
UNSWNBB115
CTU--13
AssSlán PR-ATXC. (R)
Raulline Mirivdd
PR A0- (A)
PR-AUTT (R)
R0C-MDC:
(RL-ATI7(AA)
TR-ATIC
ROC-AUIC
PR.ADC (A)
SO0-ADI:
(RLAI(C((A)
TR-AUC
ROC-AUC
DWWC
11925
[1831
01915
1/00
1.31
01984
01565
0.586
(1:50
D1971
(3955
2/86-
St
11985
[1980
0.590
1.33
1181
0.920
11.03
01531
0972
0.965
2/933
A.GEM
0.750
0.350
0.250
0.500
0071
0.96
CSS grredy
(1949
11.19
11959
1.00
1381
08712
01344
01355
09965
03966
11.300
11823
03961
2000
FCBRS (saer)
0.909
0.999
0.999
1
1.00
8.998
0.909
0/144
0.988
D971
0.908
D/974
CIRAR-ID 10
CIRARI 100
CLEXX--0
CLEAR- (00
Bazzline Mettods
PR-AEC
PS-ALT:
ROC-AEX?
(B-AAII:(A)
ROC-AUC
PR-AUC
PR-ALIC(D)
SOC-ADIC
PR-AIEE(A)
PR-AIIC
ROC-AUI
EWC
0.522
2044
02786
51718
GEM
0.931
0.19
0.008
0/533
0.036
0/63
0.555
0.852
0.565
0814
0.500
0/81
A-GEM
0.955
0.933
0.96
D:588
0.991
0/38
0.855
0.833
0335
0.72
0.700
0.758
MIB
11.41
10235
11559
D640
03901
5635
11380
D3577
03301
CBRS
0.583
0.385
0.500
0.573
0.588
0.523
0.961
0537
0.511
0541
0.789
EORRS Inarce
01953
[L911
0.98
0661
5651
0.97
0/336
6.554
0,807
EXTTI
MIIE ECCRRS
11919
10535
1090
8661
01880
8563
18983
6833
19942
0334
0.791
5/817
4.1 Quantitative analysis
We present the performance results of the proposed ECBRS algorithm and its comparison with the baselines on network intrusion detection and computer vision benchmarks Table 3 We make the following observations. First, we observe that ECBRS either outperforms is on par with all the baselines on all twelve benchmarks. Second, the performance results of the ECBRS enhanced over CBRS when both benign and attack data exhibit frequent distribution shifts,
ECBRS:
7

---

especially on AnoShift, SVHN, CIFAR-I0, CIFAR-100, CLEAR-10, 10, and CLEAR- 100 benchmarks. Specifically, ECBRS achieves 7% (avg) and 3% (avg) performance gains in attack and benign sample detection in terms of PR-AUC (A) and PR-AUC (B) values relative to the CBRS on aforementioned benchmarks. Third, GEM, AGEM, and GSS- ASS-greedy methods strongly compete with our approach. Specifically, for GEM and A-GEM the reason for this would be due to the usage of the ring buffer memory organization policy (57 58 that stores task-wise representative samples, which are used to reduce CF of previous tasks [53 154, thus allowing positive backward transfer by constraining directions of the gradient projections. However, such a buffer organization policy requires knowing task boundaries apriori. Despite their effectiveness, such a policy is inappropriate in online task-free continual learning setting like streaming intrusion detection in which task boundaries and the number of tasks unknown. On the contrary, our approach organizes the memory as a pool in which any memory slot can be chosen for replacement. Eventually, we observe that training time for larger datasets is high for gradient-based methods like GEM and GSS-greedy As a result, we omit the most recent gradient- based memory replay methods like online corset selection (OCS) [59] in our baselines (refer SM::A.10.J 10.1 for more details). Besides, ECBRS is efficient algorithm with respect to total training time, and it outperforms CBRS in total training time on five benchmark datasets. More discussions on training times are available in SM:A.8.
ECBRS as a memory population method: We can also integrate ECBRS as a memory population technique with other memory replay- based continual learning algorithms. Specifically, we demon- strate this for the MIR algorithm and observe an average performance gain of 12% and 14% in terms of PR-AUC (A) and PR-AUC (B) values relative to MIR. We observe that the performance gains of using ECBRS will increase with large-scale benchmarks, especially on AnoShift, CICIDS-2016 UNSW-NB15, and CICIDS-2017 benchmarks. We achieved an average gain of 30% (PR-AUC (A)) and 31% (PR-AUC (B)) on the aforementioned benchmarks. The results of the MIR+ECBRS algorithm (rows) are highlighted in light grey in Table 3 In the ECBRS experiments, we use the random memory population method for the MIR algorithm.
Threshold parameter (7): As described earlier, we are also interested in learning the distribution shifts of the benign data. Since there are two distribution shifts to adapt to, one may wonder which one is more important. The answer depends the problem context; intrusion detection requires paying more attention to the distribution shifts in attack traffic, which can be controlled using the hyperparameter w(()). We conduct extensive experiments benchmarks to understand the behavior of setting the default value for the w(.) Equation 1, We found that all the datasets' performance is consistent for the lower w(.) value, so we used fixed value of 0.1. Eventually, wx-) will be multiplied by the buffer memory size to obtain the expected number of samples per class (7) to be present in the buffer memory. Ablation studies on 7(.) are presented in SM:A.10.1.
intrusion detection and computer vision
Table 4: Performance comparison of the proposed PAPA method with other baselines on intrusion detection and computer vision benchmarks. We report the arithmetic mean of each evaluation metric with each experiment repeated five times independently.
MIR
PAPA
Training time (in soc.)
Datasets
PR-AUC (A)
(R-ALIC(R)
ROC-ALIC
PR-AUC (A)
PR-AUC(()
ROC-AUC
MIR
PAPA
Scalable efficiency
NSL-KDD
0,964
0.971
0.970
0
0.968
0.969
25.4
21.4
15.7%
CICIDS-2017 2017
0.999
0.999
0.999
0.999
0.999
0.999
316.0
188.8
40.2%
CTU-13
1.000
1.000
0.999
0.999
0.999
0,999
375.5
350.8
11.9%
KDDCUP'99
1.000
1,000
457.0
13.5%
UNSW-NB15
0.999
(1999
0.959
0.959
(.99
0.999
4991
350.6
29.7 %
AzoSbift
0.941
0.926
0.934
0.947
0.927
0,934
1300.0
900.6
30.7%
CICIDS-2018
0.994
0.992
0.958
5948.0
34.2%
CIFAR-100
0,663
0.659
0.663
0.673
0.647
0.672
115.2
76.7
33.4%
CIFAR-10
0.949
0.936
0.944
0.948
0.936
0.944
133.0
108.0
18.7%
CLEAR-10
0.953
0.943
262.9
175.4
33.2%
SVHN
0.979
0.972
0.976
0
0.974
0.978
296.0
208.0
29.7%
CLEAR-100
0.839
0.793
0.817
0.845
0.793
0.823
1706.0
1209.0
29.15%
PAPA: We present the results of the proposed PAPA algorithm and its comparison with the MIR algorithm in Table 4, In this set of experiments, we use ECBRS as a memory population algorithm for both MIR and PAPA algorithms. Results are presented in the non-decreasing order of their respective training times. Based on performance metrics, our evaluation the benchmarks demonstrates PAPA as replacement for method like MIR, which uses numerous virtual SGD updates. From Table 4 it also interesting to see that our approach's 's training time efficiency progresses with the dataset's s size. Specifically, we note that PAPA achieves nearly 29%, 30%, 33%. 34%, and 40% scalable efficiency
8

---

on the CLEAR- 100, AnoShift, CIFAR- 100, CICIDS-200 2018 and CICIDS-200 datasets, respectively. By scalable efficiency, we mean the total training time saved using the PAPA algorithm compared to the MIR method.
Table 5: Performance comparison of the number of regular and virtual SGD operations required for the MIR and proposed PAPA approach on benchmark datasets. Each experiment repeated five times independently.
MIR
PAPA
Savings
Datasets
Vir.GGD ops
Reg. SGD ops
Vir. SGD ops
Reg. SGD ops
Vir. SGD ops
Total SGD ops
NSL-KDD
1140
1578
210
1740
81.5%
28.2%
CICIDS-2017
9160
15784
10881
89.5%
54. 17%
UNSW-NB15
11915
14638
12587
94.9%
50.3%
CTU-13
13235
18007
24658
99.0%
20.6%
KDDCUP"99
19555
17525
97.5%
32.7%
AnoShift
48825
53187
1200
58421
97.5%
41.5%
CICIDS-2018
30590
41234
1200
36605
96.1%
47.3%
SVHN
3850
7143
360
6267
90.6%
39.7%
CIFAR-10
1935
2526
180
3028
90.6%
28.0%
CIFAR- 100
1700
2436
135
2253
92.0%
42.2%
CLEAR-10 500
602
40
607
92.0%
41.2%
CLEAR-100
3250
3250
320
3848
90.1%
35.8%
Virtual SGD parameter updates: Here, we present the core of the proposed PAPA algorithm in terms of the number virtual stochastic gradient descent parameter updates. In Table 5) we present the results in the non-decreasing order of the size of the datasets and make the following observations. First, the number of virtual SGD operations in MIR increases with the size of the dataset. On the contrary, the number of these operations is reduced with the benchmark size in our proposed method. Second, the number of virtual SGD operations the PAPA algorithm per benchmark always fixed. The fixed number of virtual SGD operations in the PAPA is due to using the MIR algorithm only for the first task.
Ablation studies: We conduct various experiments
part of ablation studies validate the robustness of the proposed approaches. These include the effect of gradient-based sample selection instead of random selection (refer to line 177) of the ECBRS performance, the effect of batch size and y on ECBRS performance, robustness to different task orders on the PAPA algorithm, the effect of batch size and buffer memory size (M) the PAPA algorithm. We present and discuss the results in SM:A.10. Further, limitations of the proposed methods are discussed in SM:A.13.
Additional experiments: We also validate the proposed approaches on the standard unsupervised anomaly detection (SMAP (60), SMD [6], and MSL (60]) benchmarks. Our findings the supervised dataset are equally valid in this new set experiments (refer to SM:A.9).
4.2 Impact of task similarity on virtual SGD updates
All the preceding discussions of the PAPA algorithm focus leveraging the slowly varying parameter update process to approximate virtual SGD updates. Here, we ex- plain the reason for the slowness in the virtual parameter updates using task similarity. Specifically,
opti-
mal transport dataset distance (OTDD) between consecutive tasks quantify the task similarity. Also, an error vector (with dimension equal to the number of parameters the model) whose entry ' equal to the difference
OTDO (ozel)
MNIST -CIAARO
norm (x3e')
MNIST + CIFARIO
(62
(7,.7))
(T2.
(T..
Task pairs
Error vector
tasks
(a) OTD distance between pair of tasks
(b) Error vector values shift visualization
Figure Illustration of the relationship between task similarity and slowness in the parameter update process of the MIR algorithm, using (a) optimal transport dataset distance and (b) visualization of parameter value shift using the 7, norm over the error vector.

---

in the parameter (6.) value prior and after learning particular task. We perform experiments to compute OTDD and [ noom of the error vector over the MNIST and CIFAR-10 datasets. We create nine tasks in each of these experiments. We recall that higher OTDD value signifies lower task similarity. The OTDD value zero for all the consecutive tasks in the MNIST dataset and therefore ignored in this discussion.
In the second experiment, the tasks are formulated using the CIFAR-IO 10 dataset. The third experiment creates a sequence of two tasks from the MNIST and CIFAR-10 10 datasets. From Figure 4a we observe that the OTDD value is higher for the third experiment than the CIFAR-10 experiment. The reason for this behavior is evident since the consecutive tasks in the third experiment learn from two dissimilar datasets. We discover similar behavior in the error vector which I nooo over the eπror vector of the third experiment exhibits a zig-zag pattern in Figure 4b) In other words, parameter values go back and forth between two dissimilar tasks. Further, we provided detailed empirical analysis to showcase that the modelling error distribution of the first task using the Gaussian mixture model is sufficient and useful to approximate error distributions of the subsequent tasks. In other words, the occurrence of two dissimilar tasks resulting in different error distributions that can't 't be modelled by PAPA is rare the context of NID, as demonstrated in SM:A.12. To conclude, assuming overparameterized models, parameter value shifts between the regular SGD and virtual SGD update are low, given the sequence of similar tasks.
Conclusions and Future Work
In this work, we improved the performance of online memory replay-based CL approaches for network intrusion detection while addressing class imbalance and scalability issues. Specifically, we focused on two popular algorithms viz., CBRS and MIR. The CBRS algorithm is extended (that we call ECBRS) using global information that helps it to maintain accurate class imbalance information. We showed that ECBRS outperforms all the basclines, including CBRS, by maintaining higher minority class samples in the buffer memory. Furthermore, ECBRS is also efficient in terms of the training time incurred as compared to the baselines. We proposed simple perturbation-assin parameter approximation (PAPA) method for virtual parameter updation in the MIR algorithm that helps significantly reduce the number of SGD update operations. The efficacy and scalability of these augmentation strategies have been demonstrated on standard network intrusion detection, computer vision, and anomaly detection datasets. In summary, the proposed augmentations to online memory replay-based CL approaches achieved improved performance using fewer computations and thereby at a lower training cost.
As part of future work, we plan to extend the supervised binary classification setting of the network intrusion detection problem are interested in exploring the semi-supervised techniques for intrusion detection to understand the challenges in the context of class imbalance and distribution shifts in conjunction with open-world world learning, explainability, and adversarial robustness settings.
supervised multi-class classification problem. Furthermore, we
Acknowledgments and Disclosure of Funding
We extend our appreciation to Aman Panwar and Shreya Kumar, both are undergrads at IIT Hyderabad, for their meaningful contributions during the rebuttal period. Additionally, our gratitude goes out to our former undergrads Akash Tadawai, Reetu Vinta, and Thushara Tippireddy, who played instrumental roles in shaping this work during its initial phases. Finally, we express our sincere thanks to all the anonymous reviewers for their constructive feedback and invaluable suggestions which helped a lot in improving this work. SKA thanks Google for the generous travel grant and the Ministry of Education, Government of India, for the financial support.
10

---

References
[1] Sebastian Thrun. A lifelong learning perspective for mobile robot control. In Proceedings of IEEE/RSJ International Conference Intelligent Robots and Systems (IROS pages 23-30, 1994.
[2] Bing Liu. Lifelong machine learning: a paradigm for continuous learning, Frontiers of Computer Science, 113):359-361, 2017.
[3] German Parisi, Ronald Kemker, Jose Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54-71, 2019.
[4] Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109-166 165. Elsevier, 1989.
[5] Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence. 44(7):3366-338) -388366666336..6 2022.
[6] Haoxuan Qu, Hossein Rahmani, Li Xu, Bryan Williams, and Jun Liu. Recent advances of continual learning in computer vision: An overview. arXiv preprint arXiv:2109. 1369, 2021.
[7] Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. Online continual learning in image classification: An empirical survey. Neurocomputing. 469:28-51 2022.
[8] Angelo G Menezes, Gustavo de Moura, Cézanne Alves, and André CPLF de Carvalho. Con- tinual object detection: review of definitions, strategies, and challenges. arXiv preprint arXiv:2205.1 15445, 2022.
[9] Magdalena Biesialska, Katarzyna Biesialska, and Marta R Costa-Jussa. Continual lifelong learning in natural language processing: A survey, arXiv preprint arXiv:2012.09823, 2020.
[10] Zixuan Ke and Bing Liu. Continual learning of natural language processing tasks: A survey. arXiv preprint 1.Xix:2211.12700 2022.
[11] Samik Sadhu and Hynek Hermansky. Continual learning automatic speech recognition. In Procedding of Interspeech, pages 1256-1250, 2020.
[12] Muqiao Yang, Ian Lane, and Shinji Watanabe. Online continual learning of end-to-end speech recognition models. arXiv preprint arXiv:2207.05071. 2022.
[13] Marius Dragoi, Elena Burceanu, Emanuela Haller, Andrei Manolache, and Florin Brad. Anoshift: A distribution shift benchmark for unsupervised anomaly detection. In Proceeding of 36th Conference on Neural Information Processing Systems (NeurIPS 2022), pages 32854-32867, 2022.
[14] Dongqi Han, Zhiliang Wang, Wenqi Chen, Kai Wang, Rui Yu, Su Wang, Han Zhang, Zhihua Wang, Minghui Jin, and Jiahai et al. Yang. Anomaly detection in the open world: Normality shift detection, explanation, and adaptation. In Proceeding of Network and Distributed System Security Symposium (NDSS 2023).
[15] Daniel Arp, Erwin Quiring, Feargus Pendlebury, Alexander Warnecke, Fabio Pierazzi, Christian Wressnegger, Lorenzo Cavallaro, and Konrad Rieck. Dos and don't 't of machine learning in computer security. In Proceeding of 31st USENIX Security Symposium (USENIX Security), 2022.
[16] Min Du, Zhi Chen, Chang Liu, Rajvardhan Oak, and Dawn Song. Lifelong anomaly detection through unlearning. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 1283-1297, 2019.
[17] Charu C Aggarwal. An introduction to outlier analysis. Springer, 2017.

---

[18] Lisa Liu, Gints Engelen, Timothy Lynar, Daryl Essam, and Wouter Joosen. Error prevalence in nids datasets: case study on cic-ids-2017 and cse-cic-ids-2018. In Proceedings of IEEE Conference Communications and Network Security (CNS 2022), pages 254 262, 2022.
[19] Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11254-11263, 2019.
[20] Suresh Kumar Amalapuram, Akash Tadwai, Reethu Vinta, Sumohana S. Channappayya, and Bheemarjuna Reddy Tamma. Continual learning for anomaly based network intrusion de- tection. In Proceedings of International Conference on COMmunication Systems NETworkS (COMSNETS 2022), pages 497-505, 2022.
[21] Aristotelis Chrysakis and data. In Proceeding of International Conference on Machine Learning, pages 1952-1961. PMLR, 2020.
Moens. Online continual learning from imbalanced
[22] Iman Sharafaldin, Arash Habibi Lashkari, and Ali A Ghorbani. Toward generating new intrusion detection dataset and intrusion traffic characterization. In Proceedings of the 4th International Conference Information Systems Security and Privacy (ICISSP 2018), pages 108-116, 2018.
[23] Nour Moustafa and Jill Slay. Unsw-nb15: comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In Proceeding of Military Communications and Information Systems Conference (MilCIS), pages 1-6, 2015.
[24] Sebastian Garcia, Martin Grill, Jan Stiborek, and Alejandro Zunino. An empirical comparison of botnet detection methods. Computers & Security, 45:100-1 2014.
[25] Mahbod Tavallace, Ebrahim Bagheri, Wei Lu, and Ali A Ghorbani. detailed analysis of the kdd cup 99 dataset. In Proceedings of IEEE symposium computational intelligence for security and defense applications, pages 1 1-6. IEEE, 2009.
[26] Marius Drăgoi, Elena Burceanu, Emanuela Haller, Andrei Manolache, and Florin Brad. Anoshift: A distribution shift benchmark for unsupervised anomaly detection. arXiv preprint arXiv:2206. 15476, 2022.
[27] Netzer Yuval. Reading digits in natural images with unsupervised feature learning. In Proceed- ings of the NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
[28] A. Krizhevsky. Learning multiple layers of features from tiny images. Master's ' Thesis, University of Toronto, 2009.
[29] Zhiqiu Lin, Jia Shi, Deepak Pathak, and Deva Ramanan. The clear benchmark: Continual learning on real-world imagery. In Proceeding of 35th Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2021.
[30] Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, I Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccis Online continual learning with maximal interfered retrieval. In Proceeding of 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), 2019.
[31] Xisen Jin, Arka Sadhu, Junyi Du, and Xiang Ren. Gradient-based editing of memory examples for online frre continual learning. In Proceeding of 34th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021.
[32] Stephen E Smaha. Haystack: An intrusion detection system. In Fourth Aerospace Computer Security Applications Conference, volume 44, Orlando, FL, USA, 1988.
[33] Varun Chandola, Arindam Banerjee, and Vipin Kumar, Anomaly detection: A survey, ACM Comput. Surv. 41(3), jul 2009.
[34] Hung-Jen Liao, Chun-Hung Richard Lin, Ying-Chih Lin, and Kuang- Yuan Tung. Intrusion detection system: comprehensive review. Journal of Network and Computer Applications, 36(1):16-24, 2013.
12

---

[35] Sang- Woong Lee, Mokhtar Mohammadi, Shima Rashidi, Amir Masoud Rahmani, Mohammad Masdari, Mehdi Hosseinzadch, et al. Towards secure intrusion detection systems using deep learning techniques: Comprehensive analysis and review. Journal of Network and Computer Applications, 187:103111, 2021.
[36] Ankush Singla and Elisa Bertino. How deep learning is making information security more intelligent. IEEE Security & Privacy, 17(3):56-65, 2019.
[37] David A Cieslak, Nitesh V Chawla, and Aaron Striegel. Combating imbalance in network intrusion datasets. In Proeeding of IEEF International Conference on Granular Computing, pages 732-737, 2006.
[38] Nitesh V Chawla, Kevin W Bowyer, Lawrence Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321- 357, 2002.
[39] Sotiris Kotsiantis, Dimitris Kanellopoulos, Panayiotis Pintelas, et al. Handling imbalanced datasets: A review. GESTS international transactions on computer science and engineering. 30(1):25-36 2006.
[40] Charles X Ling and Victor Sheng. Cost-sensitive learning and the class imbalance problem. Encyclopedia of machine learning, 201 11:11111233 2008
[41] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 1 114(13):3521-333 2017.
et al.
[42] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In Proceeding of International Conference on Machine Learning, pages 3987- 3995. PMLR, 2017.
[43] Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv: (16666606777 2016,,
[44] Joseph KJ and Vineeth N Balasubramanian. Meta-consolidation for continual learning. Advances in Neural Information Processing Systems, 33:14374-14386 2020.
[45] Chris Dongjoo Kim, Jinseo Jeong, and Gunhee Kim. Imbalanced continual learning with partitioning reservoir sampling. In Proceeding of European Conference Computer Vision, pages 41 1-1-4 Springer, 2020.
[46] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Proceedings of IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[47] Dhanabal and SP Shantharajah. A study on nsl-kdd dataset for intrusion detection system based on classification algorithms. International journal of advanced research in computer and communication engineering, 4(6):446-452, 2015.
[48] Iman Sharafaldin, Arash Habibi Lashkari, and Ali A Ghorbani. A Realistic Cy- ber Defense Datasel (CSE-CIC-IDS2018) 2018 https://registrry openyata.nddccccm. aws/
ise-cic-ids2012
[Aceessdd: April,2023).
[49] Nour Moustafa and Jill Slay. The evaluation of network anomaly detection systems: Statistical analysis of the unsw-nb15 data set and the comparison with the kdd99 data set. Information Security Journal: Global Perspective, 25(1-3):18-31, 2016.
2016.
[50] Nour Moustafa, Jill Slay, and Gideon Creech. Novel geometric area analysis technique for anomaly detection using trapezoidal area estimation on large-scale networks. IEEE Tronsactions on Big Data 5(4):481 184444 2017.
13

---

[51] Nour Moustafa, Gideon Creech, and Jill Slay. Big data analytics for intrusion detection system: Statistical decision-makingg using finite dirichlet mixture models. In Data analyties and decision support for cybersecurity, pages 127-156. Springer, 2017.
[52] Mohanad Sarhan, Siamak Layeghy, Nour Moustafa, and Marius Portmann. Netflow datasets for machine learning- basedd network intrusion detection systems. In Big Data Technologies and Applications, pages 1 117-135. Springer, 2020.
[53] David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. In Proceedings of s1 Conference on Neural Information Processing Systems (NIPS 2017),
2017.
[54] Arslan Chaudhry, Marc' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny, Efficient lifelong learning with a- a-gem. arXiv preprint arXiv: 2018.
[55] Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In Proceeding of 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), 2019.
[56] Giuseppina Andresini, Feargus Pendlebury, Fabio Pierazzi, Corrado Loglisci, Annalisa Appice, and Lorenzo Cavallaro. Insomnia: Towards concept-drift robustness in network intrusion detection. In Proceedings of the 14th ACM workshop artificial intelligence and security, pages 111-122, 2021.
[57] Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet Dokania, Philip HS Torr, and Marc' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv:
2019.
[58) Pietro Buzzega, Matteo Boschini, Angelo Porrello, and Simone Calderara. Rethinking ex- perience replay: bag of tricks for continual learning. Proceedings of 25th International Conference Puttern Recognition (ICPR), pages 2180-2187. IEFF, 2021.
[59] Jachong Yoon, Divyam Madaan, Eunho Yang, and Sung Ju Hwang. Online coreset selection for rehearsal-bee based continual learning. In Proceeding of International Conference on Learning Representations (ICLR 2022). International Conference on Learning Representations, ICLR, 2022.
[60] Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soderstrom. Detecting spacecraft anomalies using Istms and nonparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 387-395, 2018.
[61] Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery data mining, pages 2828 2837, 2019.
[62] David Alvarez-Melis and Nicolo Fusi. Geometric dataset distances via optimal transport. pages 21428-211 21439, 2020.
14


Reviews:
[{"review_id": "Ttg2gNc6hY", "reviewer": "Reviewer_wTtL", "summary": "This paper improves upon existing memory replay-based continual learning methods for anomaly detection. First the authors extend class balancing reservoir sampling (CBRS) and develop ECBRS, using global information in order to keep more accurate information about class imbalance. Second, the authors proposed a perturbation-assisted parameter approximation (PAPA) method for estimating virtual SGD parameter (VSP) updates, resulting in reduced training time for methods like maximally interfered retrieval (MIR). The proposed methods are evaluated on intrusion detection, computer vision, and anomaly detection datasets, and shown to increase performance while reducing training time.", "questions": "- Do the authors have an idea on why PAPA achieves better performance as compared to MIR in Table 3? Since PAPA performs an approximation, I would have expected the AUC scores to be slightly lower than those for MIR.\n- Do the authors have an insight as to why the error between regular/virtual parameter updates resembles a two-component GMM? Is it reasonable to expect this assumption to hold on other datasets?\n- What is the reason for including image datasets for evaluation. If the reason is distribution shifts, can you state why you expect these datasets to contain distribution shifts?", "limitations": "Limitations are adequately discussed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper relies on simple heuristics that are shown to be quite effective for improving upon existing CL methods. The authors have also done extensive evaluation of their methods using several datasets, including ablation studies. In addition to performance improvements, the proposed methods can also reduce training times, resulting in better scalability.", "weaknesses": "- Since the paper heavily relies on CBRS and MIR, I suggest that the authors briefly describe these methods to make the paper more standalone and to make its novelties more clear. \n- While the proposed methods are sound, I am not sure if the utilized evaluation datasets can fully support the paper's claims. The paper mainly focuses on learning from data with distribution shifts, however all but one of the evaluated datasets correspond to a short time period with little to no distribution shift.\n- Related to the above, I am not sure if the inclusion of image datasets is needed as the proposed methods are presented for network intrusion datasets. The authors mention that they include image dataset to evaluate learning on data with distribution shifts, but I am not sure if image datasets contain any distribution shift.\n- The authors should also include the standard deviation of AUC scores in Tables 2, 3.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1) Do the authors have an idea on why PAPA achieves better performance as compared to MIR in Table 3? Since PAPA performs an approximation, I would have expected the AUC scores to be slightly lower than those for MIR**.\n\nR1). We sincerely thank the reviewer for bringing up this important observation. After your feedback, we thoroughly investigated the issue and successfully identified the root cause. The problem arose from using **drop_last=False** in the data loader for specific datasets. We have addressed this issue and re-conducted the experiments. As a result of the fix, we now observe consistent and uniform behavior across all the datasets. The affected datasets, namely NSL-KDD, CICIDS2017, ANOSHIFT, and CIFAR-100, have all been rectified. Additionally, we noticed a minor discrepancy in the results of SVHN. However, this discrepancy is insignificant, as it only appears at the third decimal position and can be attributed to noise in the training process, which we can safely ignore. (refer to table below). We appreciate your diligence in reviewing our work, and your feedback has helped us improve the accuracy and reliability of the results presented in the paper.\n|Dataset|Method|PR-AUC(O)|PR-AUC (I)|ROC-AUC| Train time |\n|-|-|-|-|-|-|\n|NSL-KDD|MIR+ECBRS|0.964±0.009|0.971±0.003|0.97±0.004|27.86|\n||PAPA|0.961±0.017|0.968±0.007|0.969±0.006|20.75|\n|CICIDS2017|MIR+ECBRS|0.994±0.001|0.993±0.000|0.993±0.000|398.4|\n||PAPA|0.994±0.001|0.992±0.003|0.993± 0.003| 239.8|\n|ANOSHIFT|MIR+ECBRS|0.944±0.004|0.926±0.011|0.934±0.009|1273.4|\n||PAPA|0.947±0.005| 0.927±0.011| 0.934±0.007|982.2|\n|CIFAR-100|MIR+ECBRS|0.882±0.011|0.864±0.014| 0.871±0.013| 227.4|\n||PAPA|0.877±0.011|0.861±0.012|0.870±0.010|119.8|\n\n\n**Q2.Do the authors have an insight as to why the error between regular/virtual parameter updates resembles a two-component GMM? Is it reasonable to expect this assumption to hold on other datasets?**\n\nR2:  As mentioned previously, incorporating a two-component Gaussian Mixture Model (GMM) has emerged as one of our key empirical findings. In this context, we aimed to shed light on the fundamental nature of the two GMM components from the perspective of task similarity. The computation of task similarity relies on the optimal transport dataset distance (OTDD)[2].\n\nWe designed an experiment to learn tasks sequentially from MNIST and CIFAR-10 during this study. Before initiating the experiments, we calculated each task's mean OTDD[2] value, considering its relation to the remaining tasks. However, these values are not uniformly distributed compared to the experiments conducted solely using the CIFAR-10 dataset. This discrepancy suggests a higher dissimilarity within these experiments.\n\n| Dataset | Task1 | Task2 | Task3 | Task4 | Task5 | Task6 | Task7 | Task8 | Task9 |  Avg OTDD value across all tasks(Mean±std )|\n|-|-|-|-|-|-|-|-|-|-|-|\n| CIFAR-10| 258| 255| 220| 243| 212| 241| 225|236| 244| 237.2298±10.067|\n| MNIST+CIFAR10 | 310| **448**|310|**444**|310|**456**|310|**470**|310|374.179±**50.024**|\n\nFurthermore, during these experiments, we observed a significant disparity in the performance values between the MIR and PAPA algorithms. Notably, all these experiments are performed using 20 distinct task orders, and the reported results encompass both mean and standard deviation values.\n| Algorithm | PR-AUC (O)| PR-AUC (I)| ROC-AUC|\n|-|-|-|-|\n| MIR| **0.675 ± 0.053** |**0.700  ± 0.030** | **0.657 ±0.040**|\n| PAPA| 0.645 ± 0.047| 0.661   ± 0.099 | 0.628± 0.080|\n\nAdditionally, we aimed to validate this observation through a computational approach empirically. In this pursuit, after conducting experiments using the MIR algorithm, we calculated error values for the parameters and endeavored to fit the Gaussian Kernel Density Estimator (KDE). However, we encountered a singular matrix error while attempting the KDE fit, preventing us from generating a Gaussian approximation. It's noteworthy that this issue doesn't arise in other experimental scenarios.\n\nFrom this experience, we can deduce that the two-Gaussian Mixture Model is feasible whenever the variance in the mean OTDD value of each task concerning the remaining tasks is low.\n\n**Q3: What is the reason for including image datasets for evaluation? If the reason is distribution shifts, can you state why you expect these datasets to contain distribution shifts?**\n\nR3:  ANOSHIFT captures long-term distribution shifts (DS), but many NID benchmarks lack concept drift [1]. Without relevant cybersecurity benchmarks, the proposed methods were tested on computer vision benchmarks and modified to mimic the behavior of the IDS benchmark. The ease of visualizing distribution shifts makes CV benchmarks suitable for validating the proposed approaches. These DS in image classification can also be quantified using optimal transport dataset distance (OTDD)[2] computed between two tasks, where higher OTDD values indicate greater distribution shifts. For instance, on CIFAR-10 and CIFAR-100 datasets, mean OTDD values are 205 and 3139, respectively\n\n|Dataset| Avg.OTDD values across all the tasks|\n|-|-|\n|CICIDS2018|0.0508|\n|CICIDS2017| 0.1527|\n|CTU-13|4.44|\n|CIFAR-10|231|\n|CIFAR-100|3139|\n| ANoshift|7189|\n\n[1] Dragoi, M., Burceanu, E., Haller, E., Manolache, A., & Brad, F. (2022). AnoShift: A distribution shift benchmark for unsupervised anomaly detection. Advances in NIPS, 35, 32854-32867.\n\n[2]Alvarez-Melis, D., & Fusi, N. (2020). Geometric dataset distances via optimal transport. Advances in NIPS, 33, 21428-21439.\n\n\n**Weakness**\n\n*Please read R1 as a response to weakness one identified by the reviewer*\n\n**R1**) We will add more description about CBRS in supplementary material\n\n**R4**) We observe that the std for most experiments across all the datasets falls in the (0.001 to 0.010) range, implying low variance. So, we exclude them in the main paper.  Considering your feedback, we agree to add them to the final paper.\n"}, {"review_id": "mxjaT6c4jg", "reviewer": "Reviewer_KabC", "summary": "The authors propose techniques for improving how to select samples for\nreplacement in the memory for continual learning and how to estimate\nvirtual SGD in MIR to reduce computation.\n\nFor replacement in memory, CBRS does not keep track of class counts,\nand replacement might occur on non-majority samples in the memory.\nInstead the authors propose ECBRS (Extended Class Balancing Reservoir\nSampling) keeps track of class counts (global information) and\nreplacement prefers classes with the highest count.  Each class c has\ngamma(c) as the expected count in the memory with weights favoring\nsmaller classes.  Based on gamma(c), the next largest class might be\nchosen.\n\nThey also propose Perturbation Assistance for Parameter Approximation\n(PAPA) for MIR. In MIR sampling from the memory is informed by the\nloss from virtual SGD parameter (VSP) updates for each sample, which\nincur additional overhead.  They observe that VSP overlaps or scatter\naround regular SGD parameter (RSP) updates.  They modeled the\ndifference between VSP and RSP with a two-component Gaussian Mixture\nModel (GMM).  Then they estimate VSP from RST and GMM.  The GMM is\ntrained on one task and used in the remaining tasks.\n\nEmpirical results indicate that ECBRS generally outperforms 7 existing\non 12 datasets.  ECBRS can also improve the performance of MIR.  The\ntraining time for PAPA is lower than MIR on 11 datasets, but achieve\nsimilar accuracy.\n\nI have read the authors' response and commented on them.", "questions": "1.  Alg 1: \"select a class that is the largest, having higher running\n    statistics value and non-zero samples in the buffer. Otherwise,\n    select a class with next higher running statistic value that has\n    m_c >γ(c).\"\n\n    It seems the next highest class is selected when the largest class\n    has no samples in the the buffer.  This seems to be different from:\n\n    line 172: \"once it reaches the threshold of the benign class\n    (gamma(.)= 262), the ECBRS selects the class with\n    the next highest running static value\"\n\n2.  line 233: The GMM is trained based on one task and is used for the\n    other tasks.  This implies all the tasks has similar error\n    distributions, which does not seem to be the case in Figure 3.  Learning\n    a GMM for each task, and compare the GMMs would be helpful.\n\n3.  Even if the error distributions are similar for the tasks in the paper, \n     can other tasks have similar error distributions?  If not, a separate\n     GMM would need to be learned, which might negate the computation\n     savings in estimating VSP.\n\n4.  line 266: \"a randomly chosen single attack class and the remaining\n    nine classified as benign.\"  Would one (large) benign class with\n    the remaining (small) clases as attacks be more realistic (similar\n    to the example in Figure 1)?.\n\nMinor comment:\n\nline 174: static -> statistic", "limitations": "Limitations of the proposed approach do not seem to be mentioned.", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.  The idea of estimaing virtual SGD updates is interesting.\n\n2.  Empirical results indicate that ECBRS generally outperforms and\n    PAPA can reduce training time.", "weaknesses": "1.  While PAPA is interesting, the reasoning for the error\n    distribution from one task is applicable to another task could be\n    further discussed/explored--see questions below.\n\n2.  Font sizes for Table 2-4 and some figures are quite\n    small--difficult to read.\n\n3.  Some items can be clarified--see questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: Alg 1: \"select a class that is the largest, having higher running statistics value and non-zero samples in the buffer. Otherwise, select a class with next higher running statistic value that has m_c >γ(c).\" It seems the next highest class is selected when the largest class has no samples in the the buffer. This seems to be different from:line 172: \"once it reaches the threshold of the benign class (gamma(.)= 262), the ECBRS selects the class with the next highest running static value\"** \n\nR1)Dear reviewer,  Thanks for raising this question.  We acknowledge that this issue caused due to the typo in the presented ECBRS algorithm, and it is required to add the condition m_c >= gamma(c) in line number 169 of the paper. This condition will ensure that irrespective of the higher running statistic value of the class ‘c’,  our proposed ECBRS method will not undermine beyond the threshold gamma(c).\n\n**Q3.Even if the error distributions are similar for the tasks in the paper, can other tasks have similar error distributions? If not, a separate GMM would need to be learned, which might negate the computation savings in estimating VSP.**\n\nR2:  As mentioned previously, incorporating a two-component Gaussian Mixture Model (GMM) has emerged as one of our key empirical findings. In this context, we aim to shed light on the fundamental nature of the two- GMM components from the perspective of task similarity. The computation of task similarity relies on the optimal transport dataset distance (OTDD).\n\nDuring this experiment, we design an experiment to learn tasks sequentially from MNIST, CIFAR-10. Before initiating the experiments, we calculate the mean OTDD value for each task, considering its relation to the remaining tasks. However, these values are not uniformly distributed compared to the experiments conducted solely using the CIFAR-10 dataset. This discrepancy suggests a higher dissimilarity within these experiments.\n\n|Dataset|Task1|Task2|Task3|Task4|Task5|Task6|Task7|Task8|Task9|Avg OTDD values acros all tasks(Mean±std)|\n|-|-|-|-|-|-|-|-|-|-|-|\n|CIFAR-10|258|255|220|243|212|241|225|236|244|237.2298 ±10.067|\n|MNIST+CIFAR10|310|**448**|310|**444**|310|**456**|310|**470**|310|374.179±**50.024**|\n\nFurthermore, during these experiments, we observe a significant disparity in the performance values between the MIR and PAPA algorithms. Notably, all these experiments are performed using 20 distinct task orders, and the reported results encompass both mean and standard deviation values.\n\n|Algorithm|PR-AUC(O)|PR-AUC(I)|ROC-AUC|\n|-|-|-|-|\n|MIR|**0.675±0.053**|**0.700±0.030**|**0.657 ±0.040**|\n|PAPA|0.645±0.047|0.661±0.099|0.628± 0.080 |\n\nAdditionally, we aimed to validate this observation through a computational approach empirically. In this pursuit, after conducting experiments using the MIR algorithm, we calculated error values for the parameters and tried to fit the Gaussian Kernel Density Estimator (KDE). However, we encountered a singular matrix error while attempting the KDE fit, preventing us from generating a Gaussian approximation. It's noteworthy that this issue doesn't arise in other experimental scenarios. From this experience, we can deduce that the two-Gaussian Mixture Model is feasible whenever the variance in the mean OTDD value of each task concerning the remaining tasks is low.\n\n**Q2) The GMM is trained based on one task and is used for the other tasks. This implies all the tasks have similar error distributions, which does not seem to be the case in Figure 3. Learning a GMM for each task, and comparing the GMMs would be helpful**\n\nR2) We acknowledge the reviewer's concern about the error distribution.  In fig3, each row of subfigures refers to the error distribution of the model parameter on the particular datasets. This observation stands as a basis for developing the PAPA approach. To our knowledge, we assume stacking the error distribution figures of the different datasets might have brought ambiguity to the reviewer.  Once again, the error distribution for each dataset is displayed in a row labeled with the corresponding dataset name. Based on the response to question 3 (Q3), we advocate computing separate GMM for each task whenever task dissimilarity is high. For instance, in an experimental setting, like training jointly with MNIST and CIFAR-10 dataset.\n\n\n**Q4)Line 266: \"a randomly chosen single attack class and the remaining nine classified as benign.\" Would one (large) benign class with the remaining (small) clases as attacks be more realistic (similar to the example in Figure 1)?.**\n\n**CIFAR-100** : Following your input, we conducted experiments on the CIFAR-100 dataset, focusing on its super-class labels. Six classes were grouped to form a larger benign task, then split into ten parts. Similarly, the remaining classes (shuffled together) were divided into fifteen parts, and we formed ten tasks. This approach aimed to maintain a class imbalance ratio of 1:5 or 1:20 in combined tasks. Remarkably, these results aligned with our main paper findings. Another notable finding was that the  ECBRS and PAPA consistently performed much better than all the other methods. ECBRS outperformed baselines by a large margin as the class imbalance ratio increased from 1:5 to 1:20.\n\n\n**CLEAR-10**: We also conducted experiments similar to CIFAR-100. Specifically, we used four classes across all the ten years to create a larger benign file and used the rest of the six classes to create 16 different attack classes in such a way that class imbalance per each task will be 1:20. Further, we split the larger benign class into ten parts and create nine tasks. In this experiment, each task will contain one benign and one attack class. Interestingly, even in this setup, the performance results stayed consistent with what we had reported in the main paper.\n**Due to the space limitation, we report the results of these experiments in Table 1 of the uploaded pdf**\n\n\n\n"}, {"review_id": "uVW1MvgxLB", "reviewer": "Reviewer_ZBnA", "summary": "The paper considers the application of deep learning (DL) for network intrusion detection systems (NIDS). The paper correctly points out that, in real contexts, NIDS must be continously updated with new data-points to mitigate the impact of \"concept drift\". A way to do so is by employing \"continual learning\" (CL) methods---which, unfortunately, are affected by many issues which the paper seeks to address.\nSpecifically, the main contribution is a novel approach to deal with (i) class-imbalance problems leading to \"catatrophic forgetting\" (CF); and (ii) the computational overhead deriving from ways to mitigate CF. These contributions, whose intuitions are based on empirical evidence, are experimentally assessed on many datasets. The results show a good improvement over the baselines.", "questions": "I thank the authors for carrying out the research discussed in this paper. Ultimately, I **really** liked the paper: the research direction is crucial, the proposed methodology is \"sensible\", the evaluation is massive and (on the surface) rigorous. However -- as discussed in my review -- I have some doubts on the \"practical\" effectiveness of the proposed method. \n\nBelow is a set of questions that I invite the authors to answer. I must stress that my goal, here, is to assist the authors in realizing a work that is \"outstanding\": given the effort put by the authors in this submission, I have reason to believe that the authors are capable of this.\n\n* Q1) What is the envisioned system infrastructure in which the proposed method is meant to be deployed?\n* Q2) Can the authors, in the (short) timespan allotted for NeurIPS'23 review process, carry out additional experiments on a better dataset? (see my review)\n* Q3) Can you explain Figure 3 in detail?\n* Q4) What features are analyzed by the NIDS models?\n* Q5) Please provide convincing arguments that the paper is not affected by data-snooping -- and, if this is not the case, then provide feasible ways in which the paper can be amended.\n* Q6) Please elucidate how the \"temporal\" aspect is accounted for in the experiments. Suggestions are in [A, F, T].\n\nPlease answer these questions fairly. ", "limitations": "The limitations are stated, but only in the supplementary material. However, according to this reviewer, these limitations should be expanded significantly (the reasons are provided in my review). For instance, most assumptions are overly optimistic (i.e., immutable attack landscape, widespread deployment of deep neural networks for NID) and the data hardly reflects a real network and/or which is in constant evolution.\n\n\n## UPDATE AFTER AUTHOR's RESPONSE\n\nI am increasing my score from a 5 to a 6, as well as the \"Contribution\" (from a 2 to a 3) in light of the clarifications and experiments carried out by the authors during the discussion phase.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ Relevant problem\n+ Large evaluation\n+ Good results\n+ (somewhat) theoretically grounded\n+ Multiple repetitions\n+ Good quality of writing\n+ The supplementary material is rich with details\n\nThe paper addresses a relevant problem within the machine learning (ML) domain, which has received only limited attention in the specific context of network intrusion detection (NID). The proposed methods are theoretically grounded (especially Section 3.1), and the findings are derived by a large sets of experiments carried out on various datasets (which are not limited to NID data) and considering many baselines. The conclusions are drawn by repeating the experiments 5 times, increasing the overall soundness of the results---showing a substantial improvements over the baselines.", "weaknesses": "\n## High Level\n\n- Inappropriate datasets\n- (somewhat) confusing theoretical arguments \n- Bold assumptions\n- (risk of) data snooping\n- (some) missing details\n\n## Low Level (and suggestions)\n\nBelow is an extended description of the abovementioned weaknesses. I will also provide some actionable means to rectify such issues, as well as abundant references that can be used to improve this paper and/or support some of my critiques. \n\n### Inappropriate datasets\n\nThis is by far the biggest concern I have with the paper: the data used as basis for the experiments is inappropriate to test the hypothesis and provide a _convincing_ answer. Let me explain.\n\nFirst, the paper deals with the problem of NID. As such, _any finding that derives on data that does not pertain to NID is redundant_. This automatically makes all the evaluations carried out on, e.g., on CIFAR, SVHN, CLEAR to be of zero-value to the NID community (do note that a recent paper [D] highlighted that many papers on security problems carry out evaluations on data that does not reflect a security context, leading to skepticism by practitioners). I acknowledge that these datasets were added due to some limitations of (some) existing datasets for NID, but this is not an acceptable reason: if the authors were better aware of the NID context, they would know that there are (publicly available) datasets that allow a better representation of a real CL scenario. (more on this below).\n\nSecond, among the chosen NID datasets, there is the NSL-KDD and KDD Cup'99. These datasets are well-known to be flawed [C]; furthermore, they are also almost 25 years old, and the security community does not find them to be of any interest from a practical viewpoint [B, I]. Hence, even these experiments do not allow to provide a convincing argument in favor of the paper's conclusions.\n\nThird, the NID datasets also include the CIC-IDS17 and its enhanced variant, the CIC-IDS18. Unfortunately, also these two datasets are flawed [G, H]. Note that [G] came out in 2021, and it has already been well-received by the NID community (e.g., [A, F]), so it is concerning that this paper (which has been submitted to a top-venue such as NeurIPS) performs the experiments on the \"flawed\" variant of these two datasets---especially given that a \"fixed\" version exists (provided both in [G] and [H]). In short, these experiments are questionable; plus, what is even more questionable is that CIC-IDS17 and CIC-IDS18 are used to derive some observations that motivate the theoretical design of the proposed method.\n\nFourth, overlooking the previous two points, all the NID datasets span over a very short period of time, and do not enable any assessment that can be used to test \"time-aware\" applications of machine learning-based NIDS [F] -- or, at least, do so in a way that is appropriate for NeurIPS. The only exception could, potentially, be ANONIDS: however, such a dataset contains data-points from different networks, which raises many concerns [S, O]. Hence, even this dataset has dubious utility for the problem at hand.\nThe way to circumvent all these issues is not to use \"image datasets\", but rather to use NID datasets captured (i) over a long period of time and (ii) in the same network: _such datasets **exist**_, and a prominent example is the MCP dataset, which is built upon the well-known CTU13 dataset (see [J,K]). \n\nTo summarize: all the experiments carried out in the paper are performed on datasets that are inappropriate to test the underlying hypothesis without there being a sensible reason to do so.\n\n### Bold Assumptions\n\nThis is a \"pragmatic\" weakness, which does not invalidate the paper, but significantly limits its real-world relevance.\n\nPut simply, the paper proposes a method that is rooted in the application of \"deep learning\" (DL) for NID. The problem, however, is that real developers of ML-NIDS are very skeptical of DL in NID [L]. The reasons are many, but at the basis of this is that they are hardly explainable [M], but also that they are outperformed by \"traditional\" machine learning approaches (assuming that they can be applied). For example, [E] shows that multi-layer perceptrons are inferior to a random forest from a detection-performance perspective. Moreover, [B] reveals that fine-tuning a DL method requires months of data. Finally, the recent [F] shows that training \"shallow\" methods (such as decision trees) is very short. Note that [F] uses similar datasets as the ones used in this paper: to give some context, training a binary classifier using a decision tree on UNSW-NB15 requires less than 5s, whereas the MLP used in this paper requires 350s (looking at Table 3) -- and the experiments done in this paper entail much better hardware.\n\nIn light of this, it is questionable whether the proposed method has any relevance in reality. Yes, perhaps it provides some advantages -- but if the price of such advantages is an unacceptably high training time (which the method itself seeks to reduce) then the overall contribution of this paper to the state-of-the-art is low.\n\nTo address this issue, I invite the authors to include also \"shallow\" methods that do not entail (deep) neural networks, and show that the proposed method allows the resulting architectures to achieve a comparable degree of performance of \"shallow\" methods.\n\n\nNotwithstanding, there is another \"bold assumption\" made in the paper: the fact that the \"malicious classes\" are going to remain stable. Indeed (to the best of my understanding), the paper seeks to analyze the effect of distribution shift of the \"benign\" samples (this is explicitly stated in the introduction), and the set of malicious classes is always known beforehand. This is quite unrealistic (and it had been known since [O]), and may further decrease the value of the proposed method in practice: what would happen in case a new \"unknown\" attack appears, and is then assigned a \"new\" label? \n\nOn this note, the other \"practical\" limitation is that the paper assumes that there will be an influx of (correctly labelled) samples that are fed into the system. How is this done in practice? Abundant work (see [N] for a summary) pointed out that doing this is expensive, and especially so in NID context. As a matter of fact, what is not shown in the paper is a clear \"use-case\" that depicts how the system is designed to be deployed in practice: without such a schematic, it is difficult to determine if the proposed method is even \"conceivable\" to be deployed into real systems (as also highlighted in [F], practitioners are very interested in the \"system infrastructure\" envisioned in research papers). Note that I did look in the supplementary material, but the \"schematic\" provided in Figure 3 does not allow to answer my doubts: the \"continual learning module\" appears to be a \"black-box\" that receives its inputs from a \"training dataset\". However, how is such training data collected (given that we are in an \"continual learning\" setting)? The way I see it, it requires an enormous amount of manual effort.  \n\n\n### (somewhat) confusing technical arguments\n\nWhile I had no issues in understanding the rationale of Section 3.1 (aside from the following unclear sentence ```In our approach, we choose this parameter based on global information.```), I found that the description of Section 3.2 to be lacking in terms of clarity and soundness. Setting aside that the experiments are based on \"inappropriate\" datasets, I did not find compelling evidence of ```virtual SGD parameter update is a slowly varying process.``` -- and this may be due to Figure 3 not being introduced in any way (e.g., what do the x and y axis report? what is the line? the y-axis is also not uniform).\n\nI also did not find the \"motivation\" (mentioned in the Introduction) that leads up to Section 3.2 to be clear. To my understanding, the problem is that the implementation of memory-replay techniques is expensive from a computational perspective---but is this really the case for NID? As I mentioned above, \"shallow\" methods are much faster to train.\n\nAlso, at the end of the Introduction it stated that ```[PAPA] lead[s] to improved scalability.```. What does this mean? The key-term is \"scalability\". I do not see any evidence in Section 3.2 that makes me believe that PAPA leads to better scalability. I invite reading [P, Q].\n\n\n### Potential Data Snooping\n\nI have reason to believe that the methodology followed in the paper is affected by the \"data snooping\" problem [15]. In other words: the proposed method is designed in a way that would not be typically \"known\" in advance.\n\nThis is epitomized by Section 3.2, wherein some conclusions are drawn by analyzing CIC-IDS18 (and CIFAR), and then used to develop the proposed methodology. At the same time, this is also likely to be present in the experimental setup:\n\n> The intuition behind the memory size is to store nearly 1% of the total training sample of the benchmark dataset in the buffer memory, and 75% of the buffer memory samples will be used for replay.\n\nThe point is that a real developer has no clue about the size of the \"training dataset\": in real settings, an organization will collect some training data, and then use it to train a model; however, they would not know \"how much data will appear in the future\". It is possible to address this later issue by assuming a \"fixed amount\" (e.g., 1000 samples) which does not depend on the size og a given benchmark (this is what is done, e.g., in [N]). \n\n\n### Some additional issues:\n\n* In the caption of Figure 1, it is not stated what \"m\" refers to\n* Bad formatting on Page 6 (the text overlaps with Algorithm 2)\n* Also about formatting: tables and figures (and their captions) are almost impossible to discern from the text itself. I think the authors played too much with margins.\n* Given the importance of hardware in determining the runtime [F], the main paper should include also details on the experimental platform.\n* There is an excessive usage of acronyms in the paper. For instance, SBCP occurs only 3 times (and is strikingly similar to CBRS). Also, MR and MIR are very similar in how they appear, but they denote different concepts. HIDS is also redundant. \n* Reference [14] is incomplete\n* In Section 3.1, I had a hard time distinguishing between terms such as \"maximal\", \"majority\", \"full\", \"largest.\"\n* The \"Limitations\" are only mentioned in the supplementary material\n* The features used to train the models are not mentioned (I found such a lack to be surprising given that the paper cites [15]). Some network features have been demonstrated to be redundant for the sake of NID classification (see [G, H, S]), and not mentioning them casts doubts on the experiments done in the paper. Note that I did not find any mentioning of these also in the supplementary material\n* I was surprised of not finding [A] among the cited works, and especially among the \"comparison\" methods, given that it considers a very similar problem (and is also evaluated on the --fixed-- CIC-IDS17)\n* Please report the size of the training / test data. I couldn't find a clear mention of this in the experimental section (note that this is a well-known problem in related literature [N]). \n* The \"bread and butter\" of CL is that the performance should be measured \"over-time\". However, given the way the experiments are designed (at least based on how the results are presented), I cannot understand if this is truly the case. \n\nThe following is a list of paragraphs in the text for which I have concerns:\n\n> NIDS must evolve continuously and effortlessly learn from the limited novel attack data.\n\nWhat does \"limited novel attack data\" mean? How is such \"novelty\" determined?\n\n> Furthermore, formulating NID as a supervised binary classification problem (SBCP) will be helpful in differentiating out-of-distribution normal samples from known intrusions well.\n\nIsn't this the de-facto standard in many NIDS evaluations? I think this statement is redundant.\n\n> (e.g., the difference between the number of samples of the minority classes DDOS attack-HOIC and SQL Injection of the CICIDS-2018 [18] dataset is 0.68 million).\n\nUnclear.\n\n> However, this strategy has a pitfall when the count of minority class samples in the finite buffer memory appears to come from a majority class.\n\nUnclear. Plus, what does such \"pitfall\" lead to?\n\n> making our approach suitable for large-scale training.\n\nWhat does this mean? Plus, is this a problem in NID?\n\n> Typically, the benign class samples must be chosen for replacement whenever new minority class samples arrive.\n\nDefine \"typically\". Is it \"typical\" in research? Is it typical \"in practice\"? On what grounds is this statement made?\n\n> we use original class labels to organize the buffer memory to learn SBCP and chose class samples in memory with higher running statistic values for the replacement to accommodate newly arriving samples.\n\nThis statement is clear, but it contrasts with the underlying assumption of seeing NID as a binary classification problem. Indeed, providing (accurate) fine-grained attack labels is a tough problem in NID [N, R].\n\n> This ensured that each task contained a mix of benign and attack data, maintaining the class imbalance resembling real-world network traffic.\n\nHow is the \"resemblance\" achieved?\n\n> We created five tasks for KD249 DCUP’99 [22] and NSL-KDD [49], ten 250 for CICIDS-2017/2018 [18, 50], nine for UNSW-NB [24, 51, 52, 53, 54], and ten for ANOSHIFT [25] benchmark contains naturally occurring distribution shifts spanning over ten years of network traffic.\n\nThe last sentence does not connect with the previous ones.\n\n> We use M=13333 for CICIDS-2017/2018, Anoshift, 5333/1333 for KDDCUP’99/NSL-KDD, 500 for SVHN, CIFAR-10/100 and 666/2666 for CLEAR-10/100.\n\nI do not understand why the first settings have only \"M\", and the others have two numbers.\n\n### EXTERNAL REFERENCES\n\n[A]: Andresini, Giuseppina, et al. \"INSOMNIA: towards concept-drift robustness in network intrusion detection.\" Proceedings of the 14th ACM workshop on artificial intelligence and security. 2021.\n\n[B]: Apruzzese, Giovanni, et al. \"The role of machine learning in cybersecurity.\" Digital Threats: Research and Practice 4.1 (2023): 1-38.\n\n[C]: Kim, Daniel E., and Mikhail Gofman. \"Comparison of shallow and deep neural networks for network intrusion detection.\" 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC). IEEE, 2018.\n\n[D]: Apruzzese, Giovanni, et al. \"“Real Attackers Don't Compute Gradients”: Bridging the Gap Between Adversarial ML Research and Practice.\" 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE, 2023.\n\n[E]: Pontes, Camila FT, et al. \"A new method for flow-based network intrusion detection using the inverse Potts model.\" IEEE Transactions on Network and Service Management 18.2 (2021): 1125-1136.\n\n[F]: Apruzzese, Giovanni, Pavel Laskov, and Johannes Schneider. \"SoK: Pragmatic Assessment of Machine Learning for Network Intrusion Detection.\" IEEE EuroS&P (2023).\n\n[G]: Liu, Lisa, et al. \"Error Prevalence in NIDS datasets: A Case Study on CIC-IDS-2017 and CSE-CIC-IDS-2018.\" 2022 IEEE Conference on Communications and Network Security (CNS). IEEE, 2022.\n\n[H]: Engelen, Gints, Vera Rimmer, and Wouter Joosen. \"Troubleshooting an intrusion detection dataset: the CICIDS2017 case study.\" 2021 IEEE Security and Privacy Workshops (SPW). IEEE, 2021.\n\n[I]: Silva, João Vitor Valle, Martin Andreoni Lopez, and Diogo MF Mattos. \"Attackers are not stealthy: Statistical analysis of the well-known and infamous kdd network security dataset.\" 2020 4th Conference on Cloud and Internet of Things (CIoT). IEEE, 2020.\n\n[J]: Dietz, Christian, et al. \"DMEF: Dynamic Malware Evaluation Framework.\" NOMS 2022-2022 IEEE/IFIP Network Operations and Management Symposium. IEEE, 2022.\n\n[K]: Venturi, Andrea, et al. \"Drelab-deep reinforcement learning adversarial botnet: A benchmark dataset for adversarial attacks against botnet intrusion detection systems.\" Data in Brief 34 (2021): 106631.\n\n[L]: De Shon, Markus. \"Information Security Analysis as Data Fusion.\" 2019 22th International Conference on Information Fusion (FUSION). IEEE, 2019.\n\n[M]: Jacobs, Arthur S., et al. \"AI/ML for Network Security: The Emperor has no Clothes.\" Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 2022.\n\n[N]: Apruzzese, Giovanni, Pavel Laskov, and Aliya Tastemirova. \"SoK: The impact of unlabelled data in cyberthreat detection.\" 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P). IEEE, 2022.\n\n[O]: Sommer, Robin, and Vern Paxson. \"Outside the closed world: On using machine learning for network intrusion detection.\" 2010 IEEE symposium on security and privacy. IEEE, 2010.\n\n[P]: Hill, Mark D. \"What is scalability?.\" ACM SIGARCH Computer Architecture News 18.4 (1990): 18-21.\n\n[Q]: Luke, Edward A. \"Defining and measuring scalability.\" Proceedings of Scalable Parallel Libraries Conference. IEEE, 1993.\n\n[R]: Van Ede, Thijs, et al. \"Deepcase: Semi-supervised contextual analysis of security events.\" 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022.\n\n[S]: Apruzzese, Giovanni, Luca Pajola, and Mauro Conti. \"The cross-evaluation of machine learning-based network intrusion detection systems.\" IEEE Transactions on Network and Service Management (2022).\n\n[T]: Pendlebury, Feargus, et al. \"TESSERACT: Eliminating experimental bias in malware classification across space and time.\" Proceedings of the 28th USENIX Security Symposium. USENIX Association, 2019.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely request the reviewer ZBnA read the **responses to weakness section**  mentioned in the pdf (we could not specify here due to space limitations)  before reading our responses to questionnaires.\n\n**RESPONSES TO QUESTIONARIES**\n\n**R1**: The proposed mode does not require huge system infrastructure (memory and processing resources) as it uses a simple multi-layer perceptron (hardly with  5 to 6 hidden layers), which is a low complexity (capacity) architecture compared to large-size architecture like ResNet which has stacked convolution operations. So, it can also be deployed on edge devices.  Further, our solution is based on deep models, and we recommend using MLOps for low-latency inference in real-world deployments.\n\n**R2**:  To strengthen our findings, we carried out additional tests using CTU-13 [K], the newer CICIDS2017 and  CICIDS-2018 datasets ([G]). These results are presented in the Table2 of the PDF. Notably, our proposed methods consistently perform well on these datasets, aligning with the outcomes we previously shared in the paper. But, we could not run GSS-greedy and A-GEM because their competitiveness is equal  GEM method and also due to the short deadline. \n\n**R3)**  Memory replay-based continual learning methods, such as MIR, select specific samples from memory based on their impact on the model's parameter updates. They prefer samples with higher resulting losses. While these updates occur frequently during incoming data batches, they are only used for sample selection and not applied afterward. However, these frequent updates can introduce substantial computational overhead, especially for long data streams training deep models, due to the involvement of complex computations on large weight matrices. To address this, we investigated the relationship between regular and virtual updates of model parameters. Through t-SNE visualizations on datasets like CICIDS-2018, CIFAR-10, and CLEAR-10, we observed that virtual updates change gradually and tend to align with regular updates. To capture this phenomenon, we modified regular updates to approximate virtual ones. The challenge was quantifying this change. We approached this by examining the distribution of errors between virtual and regular updates across various model parameters, discovering a skewed Gaussian distribution. We used a two-component Gaussian mixture model (GMM) to represent this distribution, preserving individual error distributions for each parameter.\"\n\n**R4**: Here, we provide a list of features analyzed for each dataset.\n\nRegarding the newer CICIDS2017 and CICIDS-2018 datasets recommended by the reviewer, these datasets initially included more than 90 features. However, we removed flow-specific identifiers like flow ID, source and destination IPs, and timestamps, leaving approximately 51 features. Feature engineering was conducted using Pearson correlation with a threshold of 90%. Subsequently, the selected features underwent min-max normalization before training.\n\nOlder CICIDS 2017 and CICIDS-2018 (used in prior experiments): These datasets initially contained nearly 80 features. Initially, we removed flow specific features. Then we excluded redundant features like Bwd PSH Flags, Bwd URG Flags,  and another six features as they contain invalid/zero entries and followed by normalizing data using a min-max normalizer. Then, we removed duplicate and inconsistent rows (similar feature values but with different labels). Eventually, we started the training process with the remaining 70 features. \n\nCTU-13: As recommended by this reviewer, we used the dataset from [K]. We perform min-max normalization on this dataset before training.\n\nKDDCUP & NSL-KDD:  For these datasets,  \"service\",\"is_host_login\", and \"num_outbound_cmds\" features were removed as they were found to contain redundant values. \n\nUSNW-NB15:  For this dataset, we removed 'srcip','sport','dstip','dsport','Stime','Ltime' features. We performed a categorical data encoding for the features like ‘proto’, ‘state’, and ‘service’, followed by min-max normalization, after which the training process was invoked with the remaining features.\n\n**R5)**: We have clarified potential data snooping in the previous section (Addressing Weakness). Here, we extend the discussion on the training process. The test set was strictly avoided throughout our training experiments. All experiments exclusively used training or validation sets, and data normalization relied on min-max techniques with the entire dataset, ensuring no test data influence on outcomes.\nFurthermore, we acknowledge the removal of timestamp features from certain datasets, which does not qualify as temporal snooping due to the formulation of the intrusion detection problem in continual learning. Our experiments operated in a domain learning setup, where consistent labels (0, 1) are assigned to each task, with label distributions evolving over subsequent tasks. This approach targets temporal distribution levels for each task, diverging from examining within-class (benign, attack) distribution using timestamps. We suggest employing RNN and LSTM alongside continual learning to capture temporal aspects within each class distribution.\n\n**R6)** The main theme of continual learning is to learn a set of tasks with varying distributions arriving sequentially without forgetting previously learned tasks (avoiding catastrophic forgetting).  In this paradigm, the temporal aspect is considered at the distribution level of each task (class incremental, task incremental, and domain incremental) rather than the temporal aspect within the distribution. Based on this intuition,  we conducted experiments in the domain incremental learning setting. As a result, we don't consider timestamps in some of our experiments. However, we advocate using RNN and LSTM with continual learning to capture temporal aspects within and between the distributions of different classes."}]

Author rebuttal:
This pdf contains a table1 representing the results corresponding to the question (Q4) raised by the reviewer **KabC**.  Specifically, table results are obtained on CIFAR-100 and CLEAR-10 datasets.  Another table (table2) contained the results on newer datasets like CTU-13, modified CICIDS-2017 and CICIDS-2018 as suggested by the reviewer  **ZBnA**. Additionally, we also include our responses related to  weakness identified by the reviewer **ZBnA**

Meta-review: The paper proposes two novel methods to improve continual learning for network intrusion detection with class imbalance and scalability challenges, with experiments showing performance improvements on various benchmarks.

Two reviewers are positive, and one reviewer is negative, whose main concern (regarding why the error distribution from one task is applicable to another task) was largely addressed during the discussion period.

Overall, it is a good work.
Decision: Accept (poster)