# Multi-Path Routing Algorithm Based on Deep Reinforcement Learning for SDN
## 5 Results and Validation
### Validation of the KL Trust Mechanism  
To assess the effectiveness of the KL trust value mechanism, we conducted experiments using the CICIDS2017 dataset [56], which offers insights into packet exchange patterns within a network over a specific time frame. This validation process involved simulating a botnet network with three key stages: probing, propagation, and launching DDoS attacks. These stages encompassed diverse characteristics, such as creating packets with fixed IP addresses but varying TCP destination port numbers, generating packets with a multitude of uniform source addresses but with distinct destination addresses and port numbers, introducing a significant number of unique source addresses accessing a specific destination IP address. Furthermore, our SDN was inundated with SYN packets, but the number of ACK packets did not align with these SYN packets.  
The decision to employ KL divergence as a metric for the trust value was driven by our objective to evaluate the impact of attack interference on a node's communication capability, as opposed to classifying the type of attack. KL divergence serves as a suitable measure for gauging the degree to which a node's communication capabilities are affected, making it a pragmatic choice for our purposes. Our methodology involves identifying records in which a node functions as both the source and destination IP without being subjected to any attacks, deeming these records as valid for that particular node. We randomly sampled 3000 such records for each test node. Out of the 84 distinctive features within the CICIDS2017 dataset, we classify a node as malicious if it falls into any of the  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter Name** & **Value** \\ \hline Total Episodes & 3000 \\ Discount Factory & 0.96 \\ Learning Rate of Actor & \(1\times 10^{-5}\) \\ Learning Rate of Critic & \(1\times 10^{-4}\) \\ Advantage function scaling factor \(\lambda\) & 0.90 \\ Truncation parameter \(\epsilon\) & 0.2 \\ Parameter of delay reward \(a_{delay}\) & 0.2 \\ Parameter of variation \(a_{variation}\) & 4 \\ Parameter of trust value \(a_{trust}\) & \(-400\) \\ Parameter of diversity \(a_{type}\) & 3 \\ Parameter of repeat \(a_{repeat}\) & \(-100\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Parameters of neural network and reward function.  
following categories: an attacker node itself, a node with identified port vulnerabilities, a node affected by worm infestations, or a node subject to DDoS attacks. As a result, we established four key features as our criteria for classification: source IP frequency, destination IP frequency, SYN packet frequency, and the frequency of distinct destination ports. These criteria were utilized as benchmarks for calculating the KL divergence. It is worth noting that the data volume for the number of distinct ports recorded (DP) may vary across different nodes. Therefore, we combined attack records with a portion of normal communication records in varying proportions and subsequently calculated their trust values. The weightings assigned to the parameters, namely SIP (Source IP), DIP (Destination IP), SYN, and DP (Distinct Ports), are detailed in Table 3.  
In the experiment, several nodes were tested, including one with the IP address 172.16.0.1. The test dataset included a mix of DDoS attack records and regular communication, with proportions of 20%, 40%, and 60%, respectively. The results are presented in Table 4. As can be observed, there is a positive correlation between the KL trust value and the proportion of DDoS attacks. This result demonstrates that the KL trust value can reflect the security performance of a given node.