# A Systematic Review of Defensive and Offensive Cybersecurity with Machine Learning
## 3 Results
### Supervised Machine Learning Data Sets  
Most of the studies surveyed using supervised ML with defensive or offensive security mechanisms used contest-curated data sets, while a number of recent articles have also used real-world data. The listing of data sets for supervised ML is summarized in Figure 5.  
KDDCUP'99 was the most commonly used data set in the selected reviewed papers and this was followed by DARPA'99, NSL-KDD, log files, and honeypot. In Figure 5, the list of others includes MiniChallenge2, PREDICT, ADFA-LD, UCI, USPS, CDMC2012, SEA, CSIC2010, SSNET2011, ISCX, MNIST, ISOT, HIGGS1, SUSY2, Collection, Flower, NAB, MUTAG, ENZYMES, Tiki Usenet, CIDDS-001, Netresec AB 2015, RTBTE, CAIDA'07, and CAIDA'08. Figure 4: Machine Learning (ML) techniques used in offensive approaches. DARPA'99 data set was commonly used in the reviewed papers when testing IDS methods as it is considered well documented and well-studied in IDS. This data set is widely used especially in articles that focused on signature-based IDS. This type of detection involves the availability of a normal network traffic and a whole set of attack with each attack having a comprehensive information such as source and destination ports, source and destination IP addresses, attack duration, attack starting time, and other relevant information [27]. Though DARPA'99 data set is considered well documented and well-studied, many of the reviewed papers combined it with other data sets when evaluating intrusion methods [16]. This was because many researchers concluded that DARPA'99 is old and not suitable for evaluating recent IDS methods. Some of the reviewed articles either used both DARPA'99 and UNSW-NB15 Data Set or used only the UNSW-NB15 Data Set which is a recent data set generated with the aim of responding to the inaccessibility of network benchmark data set challenges [27]. The NSL-KDD data set is an updated version of KDDCUP'99 and was used in many of the articles reviewed as researchers consider KDDCUP'99 to have some innate flaws. One major flaw is the large number of unwanted records as duplicates of records were found when the KDDCUP'99 was used for IDS. This flaw causes bias in evaluation results [28]. However, many recent articles employ KDDCUP'99 when evaluating the performance of anomaly-based IDS. Also, many authors consider the KDDCUP'99 to provide non-biased evaluation results especially when used for the evaluation of anomaly-based IDS performance [19]. NSL-KDD is still regarded to have some problems, such as unrealistic data rates of normal and attack data. Nevertheless, many studies employ NSL-KDD data set when evaluating IDS approaches because its test subset and train subset records are reasonable [29]. Log files were also employed in the reviewed literature. Though log files were not as commonly used in the reviewed papers as DARPA'99, KDDCUP'99, and NSL-KDD, it was still employed in a significant number of studies. Some of the studies used log files obtained from anti-virus logs, firewall servers, and IDS [30]. Web logs as real-world data sets contained various websites and relevant information about ATTACK-DATE, HOST, PARAMETERS, URL, REFER, USER-AGENT, COOKIE, IP, POST-CONTENT, as well as other details. Various honeypot mechanisms provided as data sets were also leveraged in the surveyed research papers, which could be deployed to different networks to provide effective and practical evaluation results. Most of the reviewed papers that employed honeypots when evaluating IDS approaches made some modifications in addition to the traditional honeypots. For example, in one of the reviewed papers, the researchers deployed offensive systems that access joint botnets and malicious web servers to receive different types of commands as the traditional honeypots only receive attacks. There were also some less common real-world data sets used in some of the reviewed papers.