# Resampling Imbalanced Network Intrusion Datasets to Identify Rare Attacks
## 5 Experimental Design  
The experimental design compares two approaches to solve the problem of having significantly small samples of minority data: (a) resampling or random undersampling before a stratified split (Figure 1a) and (b) resampling or random undersampling after a stratified split (Figure 1b). Since the objective of this work is to identify minority data or rare attacks, stratified sampling was used to guarantee that the training and testing data included minority data. However, the question is, when is it best to perform stratified sampling? In stratified sampling, the data are divided into groups and a certain percentage of samples are taken randomly from each group. This guarantees that there will be samples from the minority class in the data. For the purposes of this study, significantly small minority data are defined as <0.1 % of the total sample.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Type of Attack** & **Count** & **\% of Attack Data** & **\% of Benign Data** & **\% of Total Data** \\ \hline Worms & 174 & 0.054 & 0.007 & 0.006 \\ Shellcode & 1511 & 0.47 & 0.068 & 0.059 \\ Backdoors & 2329 & 0.724 & 0.104 & 0.091 \\ Analysis & 2677 & 0.833 & 0.12 & 0.105 \\ Reconnaissance & 13,987 & 4.353 & 0.63 & 0.55 \\ DoS & 16,353 & 5.089 & 0.737 & 0.643 \\ Fuzzers & 24,246 & 7.546 & 1.092 & 0.954 \\ Exploits & 44,525 & 13.858 & 2.006 & 1.752 \\ Generic & 215,481 & 67.068 & 9.711 & 8.483 \\ Total attack data & 321,283 & - & - & - \\ Benign data & 2,218,761 & - & - & 87.351 \\ Total & 2,540,044 & - & - & - \\ \hline \hline \end{tabular}
\end{table}
Table 1: UNSW-NB15: distribution of attack families [4].  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Label\_Tactic** & **Count** & **\% of Attack Data** & **\% of Benign Data** & **\% of Total Data** \\ \hline Persistence & 1 & 0.00001 & 0.00001 & 0.000005 \\ Initial\_access & 1 & 0.00001 & 0.00001 & 0.000005 \\ Defense\_evasion & 1 & 0.00001 & 0.00001 & 0.000005 \\ Resource\_development & 3 & 0.00003 & 0.00003 & 0.00001 \\ Lateral\_movement & 4 & 0.00004 & 0.00004 & 0.00002 \\ Exfiltration & 7 & 0.00007 & 0.00007 & 0.00003 \\ Privilege\_escalation & 13 & 0.00014 & 0.00014 & 0.00007 \\ Credential\_access & 31 & 0.00033 & 0.00033 & 0.00016 \\ Discovery & 2086 & 0.02247 & 0.02247 & 0.01123 \\ Reconnaissance & 9,278,722 & 99.97686 & 99.969 & 49.98646 \\ Total attack data & 9,280,869 & - & - & - \\ Benign\_data & 9,281,599 & - & - & 50.00196 \\ Total & 18,562,468 & - & - & - \\ \hline \hline \end{tabular}
\end{table}
Table 2: UWF-ZeekData22: distribution of MITRE ATT&CK tactics [5; 21].  
Basically, the two approaches differ in the preprocessing sequence prior to the machine learning model execution and have different approaches to creating training and testing data. In the first design, resampling before splitting, the data are preprocessed, and then random undersampling is performed. This is followed by a stratified split and oversampling using borderline SMOTE (B-SMOTE). Finally, these data were used for training and testing the machine learning model using random forest.  
In the second design, resampling after splitting, after the initial preprocessing steps, the data are split (using stratified sampling). This method retains the stratified ratio of the majority to minority class since the stratified split was performed on the whole data. This is followed by oversampling using BSMOTE and random undersampling. Finally, these data were used to train and test the machine learning model using random forest.