# Data-Driven Network Anomaly Detection with Cyber Attack and Defense Visualization
## 5 Results and Performance Evaluation  
To evaluate our prototype and mitigation strategy, we implemented our experiment inside our Data Science and Cybersecurity Center (DSC2) lab, Howard University, Washington, DC 20059, USA, on a MS Windows server 2016 Intel(R) Xeon CPU E5-2676 v3 @2.40 GHz with 4 GB RAM. We extracted a network topology from our (DSC2) computer lab as per _service discovery_ described in Section 4.4 to obtain 101 computing devices in two subnets, as summarized in Figure 2. The network module in our visual display system contains information about network devices and topology from the computer running the application. We designate 5% attack hosts launched to the network simultaneously to perform the probing and DoS attack described in Section 4.3. Each attack host simulates the attack data in a 3-low rate (0.1, 1, 10 packets/s) and 2-high rate (100, 1000 packets/s) instance, up to a period where the experiment output is stable and does not vary much, i.e., 5 min. This time interval allowed collection of sufficient data to accurately identify anomalies without overwhelming our analysis process, but any subsequent research can vary the interval appropriately. Each attack packet IP header is distinctively labeled by setting a reserved bit. Our experiment closely mimicked an actual network environment having the performance evaluation executed on a real-time environment with a finite target. We assume the network obtains no new connections once the experiment starts, but we surmise that the scenario could be different in the real world. The prevalent strategy for identifying anomaly-based cyberattacks is using outlier detection. To demonstrate a clear identification of a malicious host having undefined arbitrary connection attempts as with the case of worm attacks, Figure 3 shows a malicious sample host \(C_{1}\) detected as an outlier to seven other active hosts. The malicious host \(C_{1}\) tries maximizing the exploit by attempting sundry connections within a short period. However, the connection requests sent from \(C_{1}\) are mostly rejected, as not all hosts would be running the targeted service, resulting in a low connection success rate; meanwhile, the other non-malicious hosts attempt connection requests with near certainty for a positive response, resulting in a high success rate. Client \(C_{2}\) and \(C_{3}\) have slightly higher failure rate than Client \(C_{4}\) and \(C_{5}\) because of normal network resource contention, but they are not flagged as outliers. Figure 3: Failure rate for malicious client \(C_{1}\) and benign clients \(C_{2}\) to \(C_{5}\). Figure 2: Extracted network model through service discovery  
Therefore, with a finite target, precision, recall, and \(F_{1}\) score become crucial to our analysis based on the attack dataset. Accuracy is our first step to understanding what we are obtaining correctly or incorrectly numerically. We extract true and false positives at different time intervals when FREC-NIDS is running and calculate precision and recall as:  
\[Precision=\frac{TP}{TP+FP}\quad,\quad Recall=\frac{TP}{TP+FN} \tag{22}\]  
where \(TP\), \(FP\), and \(FN\) denote true positive, false positive, and false negative, respectively. Subsequently, precision and recall in Equation (22) help us evaluate if our model identifies most attack data without pulling many false positives, i.e., only what is relevant to the attacks. As shown in Figure 4, our false positive rate (FPR) at the beginning of the experiment is 5.1% but improves to 0.3% at the end of the experiment. TP at the beginning of the experiment was 89.8% and ended at a good ratio of 99.7%. The FPR experimental results summarized in Table 2 show a higher overall precision and recall for FREC-NIDS than the models from [15, 16, 17, 18, 19] described in Section 2. We apply the same attack dataset and experiment setup to all algorithms in the selected related works.