# Generating Network Intrusion Detection Dataset Based on Real and Encrypted Synthetic Attack Traffic
## 4 Hikari-2021 Generation Methodology  
In this section, we explain our methodology for producing our dataset, which we call HIKARI-2021. The process starts with creating a victim network, where background traffic is captured, and attackers generate synthetic benign traffic, using a benign profile (details in Section 4.3), and malicious traffic, using an attacker profile (details in Section 4.4). The attacker traffic is captured in the attacker network. We do this to differentiate between synthetic benign and malicious traffic. Distinguishing between benign and malicious traffic is based on several criteria (details in Section 4.4). We then process the packet traces to anonymize the background traffic and extract the features. The packet traces and extracted features, as well as the documentation, constitute the produced dataset.  
\begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & **Comp.** & \multirow{2}{*}{**Payload**} & \multirow{2}{*}{**Anonymity**} & \multirow{2}{*}{**Ground-Truth**} & **Up to Date** & \multirow{2}{*}{**Labeled**} & **Encryption** & **Practical** \\  & & & & & & & **traffic** \\ \hline KDD99 [11] & Yes & Yes & No & Yes & No & Yes & No & No \\ MAWILab [15] & Yes & No & Yes & No & Yes & Yes & Yes & Yes \\ CADIA [19] & Yes & No & Yes & No & Yes & No & No & No \\ SimpleWeb [24] & Yes & No & No & No & No & No & No & Yes \\ NSI-KDD [28] & Yes & Yes & No & Yes & No & Yes & No & No \\ IMPACT [34] & Yes & No & Yes 1 & No & Yes & No & No & No \\ UMass [18] & Yes & Yes & - & No & No & No & No & No \\ Kyoto [39] & Yes & Yes & No & No & No & Yes & No & Yes \\ ICRC [34] & Yes & Yes & No & Yes & No & Yes & No & No \\ UNSW-NB15 [P] & Yes & Yes & No & Yes & No & Yes & No & No \\ UGR’16 [10] & Yes & No & Yes & Yes & Yes & Yes & Yes & Yes \\ CICIDS-2017 [53] & Yes & Yes & Yes & No & No & Yes & Yes 2 & No \\ \hline \hline \end{tabular}  
[FOOTNOTE:*]Footnote *: footnotetext: This datasets with partial anonymization; 2 Mix data between un-encrypted data, such as HTTP, and encrypted data, such as SSH.  
\end{table}
Table 1: Comparison of IDS datasets based on the requirements in Section 3.1.  
We are focused on application layer attacks that employ HTTPS. Based on the report from the 2021 Data Breach Investigation, 80% of the attack vectors come from application-layer attacks. There are many attacks on the internet but we are not focused on how many attacks we can generate. Based on the survey from netcraft.com and websitesetup.org, WordPress, Joomla, and Drupal are among the ten most popular open-source CMSs, with the combined market share of almost 50%. Based on the information from CVE, more than 300 vulnerabilities existed for WordPress from 2006 to 2021, 92 vulnerabilities for Joomla from 2004 to 2021, and 202 vulnerabilities for Drupal from 2002 to 2021. More than half of the vulnerabilities from these three CMSs are part of Brute Force and Probing. Furthermore, the goal of this research is not in the attack diversity but in what kind of attack we can deliver in the encrypted network. We decided to focus on common application-layer attacks, such as brute force and probing. In addition, the IDS researcher may build their script based on our tool to enrich the attack, such as SQL Injection, Denial of Service, etc.