# Packet Injection Exploiting Attack and Mitigation in Software-Defined Networks
## 5 Evaluation
### Evaluation  
We evaluate the effectiveness and performance of PIEDefender from the following aspects: (1) Injection detection effect, (2) DoS detection effect, (3) defense effect on the SDN controller, (4) defense effect on OpenFlow switch, and (5) defense overhead. **Injection detection effect:** We evaluate the injection detection effect of the PIEDefender by comparing the number of injected and detected hosts and compare with PacketChecker [17] and DoSDefender [16]. Figure 8a shows the number of injected hosts successfully discovered. As we can see, PacketChecker can find all injected hosts than PIEDefender because it assumes that the host's MAC address in the data plane will not change. On the one hand, we think this assumption is too strict to some extent. On the other hand, this mechanism cannot distinguish between malicious and non-malicious behavior. The malicious injection has apparent intent to attack, and the purpose is to generate many puppet hosts in the SDN network topology. Otherwise, we consider it as non-malicious behavior. Figure 8b depicts the detection precision and the average detection precision of PacketChecker, DoSDefender, and PIEDefender to be 96.4%, 96.9%, and 97.8%, respectively. It can be seen that the PIEDefender are less affected by the injection scale, and the detection performance is more stable. Although DoSDefender tracks host location migration through _OFPT_PORT_STATUS_ messages, the defense mechanism can be easily bypassed due to the lack of message verification. The PIEDefender uses the threshold to verify the legality of _OFPT_PORT_STATUS_ messages, preventing attackers from bypassing to a certain extent. Therefore, considering the applicability, stability, and precision, the PIEDefender is superior to other solutions. **DoS detection effect:** As mentioned in Section 4.3, we use SVM as the classifier. To prove that SVM is more suitable for our experimental environment and scenarios, we first compared with other classification and clustering algorithms commonly used in anomaly detection, including k-Nearest Neighbors (KNN), Random Forest, Decision Tree, K-means, and BayesNet. As a comparison, we use the F1 score, precision, and recall to evaluate the performance of different algorithms and schemes, as shown in Equations (6)-(8). In Equations (6) and (7):  
(1) **TP**: The number of true positives means the classification of the simplified DoS detection module is correct, and an attack occurs. (2) **FP**: The number of false positives means the classification of the simplified DoS detection module is incorrect and that no attack occurs. (3) **FN**: The number of false negatives, which means the classification of the simplified DoS detection module is incorrect, and an attack occurs. \[Precision=\frac{TP}{TP+FP} \tag{6}\]  
Figure 8: Comparison of injection detection. (**a**) Detection hosts. (**b**) Detection precision. \[Recall=\frac{TP}{TP+FN} \tag{7}\]  
\[F1\ score=2\cdot\frac{Recall\cdot Precision}{Recall+Precision}. \tag{8}\]  
The results is shown in Table 2. As we can see, the SVM algorithm is superior to other detection algorithms in terms of precision, recall, and F1 score for this paper's data and features. Next, we compare with other proposed solutions in the same environment, including the entropy-based detection method [28], Detpro, and DoSDefender. Detpro uses machine learning methods for detection, and DoSDefender is a static detection solution. We randomly selected the attacker and the timing of the attack, and the evaluated result is shown in Table 3. As we can see, the attack detection accuracy of PIEDefender is superior to the others. **Defense effect on SDN controller:** We utilize controller CPU utilization to evaluate the protection on the control plane with the PIEDefender. Figure 9a-d represent the CPU utilization under the injection of a different numbers of hosts. As we can see, the controller CPU usage sharply increases without the PIEDefender when attacks start.