SlowGuess/NeurIPS_2023_OCR huggingface dataset:
for each dataset are available in SM:A.7. Due to the absence of distribution shifts in NIDS datasets, the proposed approaches were validated using computer vision (CV) CL benchmarks. Furthermore, the ease of visualizing distribution shifts makes CV benchmarks suitable for validating the proposed approaches. For CV benchmarks, we randomly selected one or more classes as the attack classes. We split and distributed the attack class data among the remaining benign classes. This way of class splitting ensures experiments formulated using vision benchmarks are identical to the intrusion detection experiments. Specifically, we created nine tasks for SVHN [27] and CIFAR-10 [28]. Similarly, we created ten tasks for the CIFAR-100 [28] benchmark using the coarse/super class labels. The data spanned over ten years for the CLEAR-10/100 [29] benchmark and contained all the classes' natural temporal evolution for each year. We created ten tasks as the data spans a decade. We also maintained 1:100 class imbalance ratio per task in CIFAR-100, CLEAR-10, and CLEAR-100 experiments. More details about datasets, preprocessing (and feature selection), and task formulations are presented in SM:A.4, SM:A.5, and SM:A.6.
Baselines: We compare the proposed ECBRS method with elastic weight consolidation (EWC) [41], synaptic intelligence (SI) [42] (regularization methods), memory(+gradient)-based algorithms like gradient episodic memory(GEM) [53], A-GEM [54], gradient-based sample selection (GSS-greedy) [55], and rehearsal-based methods like MIR [30], and CBRS [21]. We compare the proposed PAPA with the MIR algorithm in all facets. We deliberately omit partition reservoir sampling [45] and GMED [31]
as the former approach is designed for the multi-label classification setting, and example editing of the latter one is trivial for intrusion data. Additionally, we do not consider INSOMNIA [56], as this work does not operate in the explicit continual learning setup and relies on existing works to identify distribution shifts and pseudo-label generation.
Architecture details: The ResNet-18 [46] architecture (pre-trained onImagenet) is used as the backbone for SVHN, CIFAR-10/100 and CLEAR-10/100, followed by a fully connected network. We use a multi-layer, fully connected network for the NID datasets. We use RMSProp optimizer for intrusion detection and SGD with Nesterov momentum 0.9 and weight decay  $10^{-4}$  for computer vision benchmarks, respectively. A batch size of 128 is used for MNIST, SVHN, CIFAR-10/100, 10 for CLEAR-10/100, and 1024 for the rest of the datasets except for NSL-KDD, for which it is 500. More details on batch size, architecture, hyperparameters, and implementation are presented in SM:A.11.
Table 2: Average optimal transport dataset distance (OTDD) values of selective intrusion detection and computer vision benchmark datasets.
Dataset	Avg. OTDD values
CICIDS-2018 [18]	0.0508
CICIDS-2017 [18]	0.1527
CTU-13 [24]	4.44
CIFAR-10 [28]	231
CIFAR-100 [28]	3139
AnoShift [26]	7189
Memory size  $(\mathcal{M})$ : We use  $\mathcal{M} = 13333$  for the CICIDS-2017, 2018 and AnoShift datasets. For the KDDCUP'99 dataset,  $\mathcal{M} = 5333$ , and  $\mathcal{M} = 1333$  for the NSL-KDD dataset.  $\mathcal{M} = 500$  for the SVHN and CIFAR-10/100 datasets. For the CLEAR-10 dataset,  $\mathcal{M} = 666$ , and  $\mathcal{M} = 2666$  for CLEAR-100 datasets.  $75\%$  of the buffer memory samples are used for replay. Additional details and an ablation study on  $\mathcal{M}$  are presented in SM:A.10.2.
Evaluation metrics: We chose metrics similar to [26, 15]. Specifically, we use the receiver operating characteristic area under the curve (ROC-AUC) metric and the precision-recall area under the curve (PR-AUC) metric for both
benign and attack data denoted as PR-AUC (B) and PR-AUC (A), respectively.
Table 3: Performance results comparison of the proposed ECBRS method with the baselines on network intrusion detection and computer vision benchmarks. We report the arithmetic mean with each experiment repeated five times independently. The performance results of the MIR using ECBRS as a memory population method are highlighted in light grey.
Baseline Methods	KDDCUP'99	NSL-KDD	CICIDS-2017	CICIDS-2018
PR-AUC (A)	PR-AUC (B)	ROC-AUC	PR-AUC (A)	PR-AUC (B)	ROC-AUC	PR-AUC (A)	PR-AUC (B)	ROC-AUC	PR-AUC (A)	PR-AUC (B)	ROC-AUC
EWC [41]	1.000	0.694	0.995	0.949	0.852	0.927	0.617	0.766	0.608	0.740	0.762	0.505
SI [42]	1.000	0.793	0.997	0.949	0.885	0.930	0.812	0.878	0.868	0.804	0.826	0.744
GEM [53]	1.000	0.918	0.999	0.968	0.941	0.959	0.993	0.988	0.991	0.739	0.762	0.502
A-GEM [54]	1.000	0.892	0.999	0.958	0.925	0.946	0.84	0.852	0.696	0.738	0.762	0.5
GSS-greedy [55]	1.000	0.680	0.997	0.951	0.881	0.933	0.807	0.827	0.742	0.8215	0.762	0.664
MIR [30]	1.000	0.767	0.996	0.923	0.861	0.918	0.785	0.840	0.798	0.737	0.762	0.5
CBRS [21]	1.000	0.896	0.999	0.929	0.969	0.960	0.999	0.999	0.999	0.999	0.999	0.998
ECBRS (ours)	1.000	0.982	0.999	0.970	0.967	0.969	1.00	0.999	0.999	0.999	0.999	0.998
MIR + ECBRS	1.000	0.929	0.999	0.958	0.966	0.965	1.00	1.00	0.999	0.994	0.994	0.992
Baseline Methods	UNSWNB-15	CTU-13	AnoShift	SVHN
PR-AUC (A)	PR-AUC (B)	ROC-AUC	PR-AUC (A)	PR-AUC (B)	ROC-AUC	PR-AUC (A)	PR-AUC (B)	ROC-AUC	PR-AUC (A)	PR-AUC (B)	ROC-AUC
EWC	0.925	0.823	0.913	1.00	1.00	0.999	0.543	0.566	0.550	0.971	0.953	0.964
SI	0.985	0.989	0.990	1.00	1.00	0.999	0.620	0.609	0.631	0.970	0.954	0.963
GEM	0.998	0.995	0.994	1.00	1.00	0.999	0.880	0.900	0.902	0.977	0.964	0.973
A-GEM	0.750	0.750	0.500	0.750	0.750	0.500	0.846	0.900	0.883	0.977	0.962	0.971
GSS-greedy	0.949	0.848	0.959	1.00	1.00	0.999	0.742	0.744	0.753	0.980	0.968	0.975
MIR	0.886	0.807	0.855	0.950	0.950	0.899	0.655	0.609	0.620	0.972	0.961	0.967
CBRS	0.999	0.999	0.999	1.00	0.999	0.999	0.949	0.939	0.941	0.954	0.957	0.955
ECBRS (ours)	0.999	0.999	0.999	1.00	1.00	0.999	0.949	0.944	0.948	0.978	0.968	0.974
MIR + ECBRS	0.999	0.999	0.999	1.00	1.00	0.999	0.942	0.928	0.934	0.980	0.972	0.976
Baseline Methods	CIFAR-10	CIFAR-100	CLEAR-10	CLEAR-100
PR-AUC (A)	PR-AUC (B)	ROC-AUC	PR-AUC (A)	PR-AUC (B)	ROC-AUC	PR-AUC (A)	PR-AUC (B)	ROC-AUC	PR-AUC (A)	PR-AUC (B)	ROC-AUC
EWC	0.931	0.917	0.925	0.644	0.603	0.644	0.858	0.828	0.835	0.770	0.764	0.778
SI	0.929	0.915	0.925	0.645	0.614	0.635	0.847	0.833	0.835	0.766	0.753	0.772
GEM	0.931	0.919	0.928	0.653	0.626	0.643	0.858	0.852	0.848	0.818	0.800	0.813
A-GEM	0.925	0.903	0.916	0.638	0.591	0.638	0.853	0.822	0.833	0.772	0.766	0.780
GSS-greedy	0.944	0.932	0.940	0.659	0.646	0.662	0.881	0.850	0.861	0.733	0.691	0.721
MIR	0.941	0.936	0.939	0.640	0.640	0.636	0.890	0.885	0.887	0.837	0.800	0.82
CBRS	0.883	0.889	0.890	0.572	0.598	0.572	0.941	0.927	0.931	0.841	0.789	0.82
ECBRS (ours)	0.953	0.941	0.948	0.663	0.611	0.663	0.937	0.926	0.926	0.854	0.807	0.831
MIR + ECBRS	0.949	0.936	0.944	0.663	0.659	0.663	0.953	0.933	0.942	0.839	0.793	0.817
4.1 Quantitative analysis
ECBRS: We present the performance results of the proposed ECBRS algorithm and its comparison with the baselines on network intrusion detection and computer vision benchmarks in Table 3. We make the following observations. First, we observe that ECBRS either outperforms or is on par with all the baselines on all twelve benchmarks. Second, the performance results of the ECBRS are enhanced over CBRS when both benign and attack data exhibit frequent distribution shifts,
especially on AnoShift, SVHN, CIFAR-10, CIFAR-100, CLEAR-10, and CLEAR-100 benchmarks. Specifically, ECBRS achieves  $7\%$  (avg) and  $3\%$  (avg) performance gains in attack and benign sample detection in terms of PR-AUC (A) and PR-AUC (B) values relative to the CBRS on aforementioned benchmarks. Third, GEM, AGEM, and GSS-greedy methods strongly compete with our approach. Specifically, for GEM and A-GEM, the reason for this would be due to the usage of the ring buffer memory organization policy [57, 58] that stores task-wise representative samples, which are used to reduce CF of previous tasks [53, 54], thus allowing positive backward transfer by constraining directions of the gradient projections. However, such a buffer organization policy requires knowing task boundaries apriori. Despite their effectiveness, such a policy is inappropriate in an online task-free continual learning setting like streaming intrusion detection in which task boundaries and the number of tasks are unknown. On the contrary, our approach organizes the memory as a pool in which any memory slot can be chosen for replacement. Eventually, we observe that training time for larger datasets is high for gradient-based methods like GEM and GSS-greedy. As a result, we omit the most recent gradient-based memory replay methods like online corset selection (OCS) [59] in our baselines (refer to SM:A.10.1 for more details). Besides, ECBRS is an efficient algorithm with respect to total training time, and it outperforms CBRS in total training time on five benchmark datasets. More discussions on training times are available in SM:A.8.
ECBRS as a memory population method: We can also integrate ECBRS as a memory population technique with other memory replay-based continual learning algorithms. Specifically, we demonstrate this for the MIR algorithm and observe an average performance gain of  $12\%$  and  $14\%$  in terms of PR-AUC (A) and PR-AUC (B) values relative to MIR. We observe that the performance gains of using ECBRS will increase with large-scale benchmarks, especially on AnoShift, CICIDS-2018, UNSW-NB15, and CICIDS-2017 benchmarks. We achieved an average gain of  $30\%$  (PR-AUC (A)) and  $31\%$  (PR-AUC (B)) on the aforementioned benchmarks. The results of the MIR+ECBRS algorithm (rows) are highlighted in light grey in Table 3. In the ECBRS experiments, we use the random memory population method for the MIR algorithm.
Threshold parameter  $(\gamma)$ : As described earlier, we are also interested in learning the distribution shifts of the benign data. Since there are two distribution shifts to adapt to, one may wonder which one is more important. The answer depends on the problem context; intrusion detection requires paying more attention to the distribution shifts in attack traffic, which can be controlled using the hyperparameter  $w(\cdot)$ . We conduct extensive experiments on intrusion detection and computer vision benchmarks to understand the behavior of setting the default value for the  $w(\cdot)$  Equation 1. We found that all the datasets' performance is consistent for the lower  $w(\cdot)$  value, so we used a fixed value of 0.1. Eventually,  $w(\cdot)$  will be multiplied by the buffer memory size to obtain the expected number of samples per class  $(\gamma)$  to be present in the buffer memory. Ablation studies on  $\gamma(\cdot)$  are presented in SM:A.10.1.
Table 4: Performance comparison of the proposed PAPA method with other baselines on intrusion detection and computer vision benchmarks. We report the arithmetic mean of each evaluation metric with each experiment repeated five times independently.
Datasets	MIR	PAPA	Training time (in sec.)
PR-AUC (A)	PR-AUC(B)	ROC-AUC	PR-AUC (A)	PR-AUC(B)	ROC-AUC	MIR	PAPA	Scalable efficiency
NSL-KDD	0.964	0.971	0.970	0.961	0.968	0.969	25.4	21.4	15.7%
CICIDS-2017	0.999	0.999	0.999	0.999	0.999	0.999	316.0	188.8	40.2%
CTU-13	1.000	1.000	0.999	0.999	0.999	0.999	375.5	330.8	11.9%
KDDCUP'99	1.000	0.929	0.999	1.000	0.920	0.999	457.0	395.0	13.5%
UNSW-NB15	0.999	0.999	0.999	0.999	0.999	0.999	499.4	350.6	29.7%
AnoShift	0.944	0.926	0.934	0.947	0.927	0.934	1300.0	900.6	30.7%
CICIDS-2018	0.994	0.994	0.992	0.998	0.999	0.998	9040.0	5948.0	34.2%
CIFAR-100	0.663	0.659	0.663	0.673	0.647	0.672	115.2	76.7	33.4%
CIFAR-10	0.949	0.936	0.944	0.948	0.936	0.944	133.0	108.0	18.7%
CLEAR-10	0.953	0.933	0.942	0.943	0.927	0.932	262.9	175.4	33.2%
SVHN	0.979	0.972	0.976	0.981	0.974	0.978	296.0	208.0	29.7%
CLEAR-100	0.839	0.793	0.817	0.845	0.793	0.823	1706.0	1209.0	29.1%
PAPA: We present the results of the proposed PAPA algorithm and its comparison with the MIR algorithm in Table 4. In this set of experiments, we use ECBRS as a memory population algorithm for both MIR and PAPA algorithms. Results are presented in the non-decreasing order of their respective training times. Based on performance metrics, our evaluation of the benchmarks demonstrates PAPA as a replacement for a method like MIR, which uses numerous virtual SGD updates. From Table 4, it is also interesting to see that our approach's training time efficiency progresses with the dataset's size. Specifically, we note that PAPA achieves nearly  $29\%$ ,  $30\%$ ,  $33\%$ ,  $34\%$ , and  $40\%$  scalable efficiency
on the CLEAR-100, AnoShift, CIFAR-100, CICIDS-2018 and CICIDS-2017 datasets, respectively. By scalable efficiency, we mean the total training time saved using the PAPA algorithm compared to the MIR method.
Table 5: Performance comparison of the number of regular and virtual SGD operations required for the MIR and proposed PAPA approach on benchmark datasets. Each experiment is repeated five times independently.
Datasets	MIR	PAPA	Savings
Vir.SGD ops	Reg. SGD ops	Vir. SGD ops	Reg. SGD ops	Vir. SGD ops	Total SGD ops
NSL-KDD	1140	1578	210	1740	81.5%	28.2%
CICIDS-2017	9160	15784	550	10881	89.5%	54.17%
UNSW-NB15	11915	14638	600	12587	94.9%	50.3%
CTU-13	13235	18007	120	24658	99.0%	20.6%
KDDCUP'99	19555	17525	480	24450	97.5%	32.7%
AnoShift	48825	53187	1200	58421	97.5%	41.5%
CICIDS-2018	30590	41234	1200	36605	96.1%	47.3%
SVHN	3850	7143	360	6267	90.6%	39.7%
CIFAR-10	1935	2526	180	3028	90.6%	28.0%
CIFAR-100	1700	2436	135	2253	92.0%	42.2%
CLEAR-10	500	602	40	607	92.0%	41.2%
CLEAR-100	3250	3250	320	3848	90.1%	35.8%
Virtual SGD parameter updates: Here, we present the core of the proposed PAPA algorithm in terms of the number of virtual stochastic gradient descent parameter updates. In Table 5, we present the results in the non-decreasing order of the size of the datasets and make the following observations. First, the number of virtual SGD operations in MIR increases with the size of the dataset. On the contrary, the number of these operations is reduced with the benchmark size in our proposed method. Second, the number of virtual SGD operations of the PAPA algorithm per benchmark is always fixed. The fixed number of virtual SGD operations in the PAPA is due to using the MIR algorithm only for the first task.
Ablation studies: We conduct various experiments as a part of ablation studies to validate the robustness of the proposed approaches. These include the effect of gradient-based sample selection instead of a random selection (refer to line 177) of the ECBRS performance, the effect of batch size and  $\gamma$  on ECBRS performance, robustness to different task orders on the PAPA algorithm, the effect of batch size and buffer memory size  $(\mathcal{M})$  on the PAPA algorithm. We present and discuss the results in SM:A.10. Further, limitations of the proposed methods are discussed in SM:A.13.
Additional experiments: We also validate the proposed approaches on the standard unsupervised anomaly detection (SMAP [60], SMD [61], and MSL [60]) benchmarks. Our findings on the supervised dataset are equally valid in this new set of experiments (refer to SM:A.9).
4.2 Impact of task similarity on virtual SGD updates
All the preceding discussions of the PAPA algorithm focus on leveraging the slowly varying parameter update process to approximate virtual SGD updates. Here, we explain the reason for the slowness in the virtual parameter updates using task similarity.
Specifically,
we use optimal transport dataset distance (OTDD) [62] between two consecutive tasks to quantify the task similarity. Also, an error vector (with a dimension equal to the number of parameters in the model) whose entry  $i$  is equal to the difference
(a) OTD distance between pair of tasks
(b) Error vector values shift visualization
Figure 4: Illustration of the relationship between task similarity and slowness in the parameter update process of the MIR algorithm, using (a) optimal transport dataset distance and (b) visualization of parameter value shift using the  $l_{1}$ -norm over the error vector.
in the parameter  $(\theta_{i})$  value prior and after learning a particular task. We perform experiments to compute OTDD and  $l_{1}$ -norm of the error vector over the MNIST and CIFAR-10 datasets. We create nine tasks in each of these experiments. We recall that a higher OTDD value signifies lower task similarity. The OTDD value is zero for all the consecutive tasks in the MNIST dataset and therefore ignored in this discussion.
In the second experiment, the tasks are formulated using the CIFAR-10 dataset. The third experiment creates a sequence of two tasks from the MNIST and CIFAR-10 datasets. From Figure 4a, we observe that the OTDD value is higher for the third experiment than the CIFAR-10 experiment. The reason for this behavior is evident since the consecutive tasks in the third experiment learn from two dissimilar datasets. We discover similar behavior in the error vector in which  $l_{1}$ -norm over the error vector of the third experiment exhibits a zig-zag pattern in Figure 4b. In other words, parameter values go back and forth between two dissimilar tasks. Further, we provided detailed empirical analysis to showcase that the modelling error distribution of the first task using the Gaussian mixture model is sufficient and useful to approximate error distributions of the subsequent tasks. In other words, the occurrence of two dissimilar tasks resulting in different error distributions that can't be modelled by PAPA is rare in the context of NID, as demonstrated in SM:A.12. To conclude, assuming overparameterized models, parameter value shifts between the regular SGD and virtual SGD update are low, given the sequence of similar tasks.
5 Conclusions and Future Work
In this work, we improved the performance of online memory replay-based CL approaches for network intrusion detection while addressing class imbalance and scalability issues. Specifically, we focused on two popular algorithms viz., CBRS and MIR. The CBRS algorithm is extended (that we call ECBRS) using global information that helps it to maintain accurate class imbalance information. We showed that ECBRS outperforms all the baselines, including CBRS, by maintaining higher minority class samples in the buffer memory. Furthermore, ECBRS is also efficient in terms of the training time incurred as compared to the baselines. We proposed a simple perturbation-assisted parameter approximation (PAPA) method for virtual parameter updation in the MIR algorithm that helps significantly reduce the number of SGD update operations. The efficacy and scalability of these augmentation strategies have been demonstrated on standard network intrusion detection, computer vision, and anomaly detection datasets. In summary, the proposed augmentations to online memory replay-based CL approaches achieved improved performance using fewer computations and thereby at a lower training cost.
As a part of future work, we plan to extend the supervised binary classification setting of the network intrusion detection problem to a supervised multi-class classification problem. Furthermore, we are interested in exploring the semi-supervised techniques for intrusion detection to understand the challenges in the context of class imbalance and distribution shifts in conjunction with open-world learning, explainability, and adversarial robustness settings.
We extend our appreciation to Aman Panwar and Shreya Kumar, both are undergrads at IIT Hyderabad, for their meaningful contributions during the rebuttal period. Additionally, our gratitude goes out to our former undergrads Akash Tadawai, Reetu Vinta, and Thushara Tippireddy, who played instrumental roles in shaping this work during its initial phases. Finally, we express our sincere thanks to all the anonymous reviewers for their constructive feedback and invaluable suggestions which helped a lot in improving this work. SKA thanks Google for the generous travel grant and the Ministry of Education, Government of India, for the financial support.
[51] Nour Moustafa, Gideon Creech, and Jill Slay. Big data analytics for intrusion detection system: Statistical decision-making using finite dirichlet mixture models. In Data analytics and decision support for cybersecurity, pages 127-156. Springer, 2017.
[52] Mohanad Sarhan, Siamak Layeghy, Nour Moustafa, and Marius Portmann. Netflow datasets for machine learning-based network intrusion detection systems. In *Big Data Technologies and Applications*, pages 117–135. Springer, 2020.
[53] David Lopez-Paz and Marc'Aurelio Ranzato. Gradient episodic memory for continual learning. In Proceedings of 31st Conference on Neural Information Processing Systems (NIPS 2017), 2017.
[54] Arslan Chaudhry, Marc'Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint arXiv:1812.00420, 2018.
[55] Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In Proceedings of 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), 2019.
[56] Giuseppina Andresini, Feargus Pendlebury, Fabio Pierazzi, Corrado Loglisci, Annalisa Appice, and Lorenzo Cavallaro. Insomnia: Towards concept-drift robustness in network intrusion detection. In Proceedings of the 14th ACM workshop on artificial intelligence and security, pages 111-122, 2021.
[57] Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc'Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv:1902.10486, 2019.
[58] Pietro Buzzega, Matteo Boschini, Angelo Porrello, and Simone Calderara. Rethinking experience replay: a bag of tricks for continual learning. In Proceedings of 25th International Conference on Pattern Recognition (ICPR), pages 2180-2187. IEEE, 2021.
[59] Jaehong Yoon, Divyam Madaan, Eunho Yang, and Sung Ju Hwang. Online coreset selection for rehearsal-based continual learning. In Proceedings of International Conference on Learning Representations (ICLR 2022). International Conference on Learning Representations, ICLR, 2022.
[60] Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soderstrom. Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 387-395, 2018.
[61] Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2828-2837, 2019.
[62] David Alvarez-Melis and Nicolo Fusi. Geometric dataset distances via optimal transport. pages 21428-21439, 2020.