# Multi-Path Routing Algorithm Based on Deep Reinforcement Learning for SDN
## 1 Introduction  
The advancement of communication networks has witnessed swift development in mobile communication, the Industrial Internet of Things (IIoT), and emerging internet technologies. The result is the emergence of a varied spectrum of network types molding the current communication panorama. The interplay and coexistence of these networks have grown crucial due to their deployment in unattended or hostile settings and their inherent openness, rendering them vulnerable to potential intrusions by malicious actors. Moreover, the proliferation of mobile devices connecting to networks has challenged traditional network access methods and policies, necessitating the refinement of routing strategies to meet dynamic information transmission needs. One promising solution to these evolving challenges is Software-Defined Networking (SDN). SDN is a flexible and efficient networking approach that offers a global view of the network and enables the selection of optimal paths based on real-time network conditions [1]. Its fundamental characteristic is the separation of the control plane from the data plane, where a centralized control function maintains the network's state and issues instructions to data plane devices [2]. This architecture provides clear security advantages, allowing for real-time analysis and correlation of network feedback. Also, due to the special programmability of SDN, programmablerouting based on SDN has also become quite popular in recent years [3; 4; 5; 6]. However, it also presents concerns related to open programmability and trust between network elements, particularly when using technologies like OpenFlow. OpenFlow, the most commonly used SDN technology, is a focal point of security analysis. The STRIDE threat analysis method applied to OpenFlow reveals vulnerabilities, including susceptibility to Denial of Service (DoS) attacks [7]. DoS attacks on SDN can exhaust controller resources by flooding it with fake flow table requests and disrupt communication by targeting the southbound interface between the controller and network devices. To address these security threats, various technologies like intrusion detection, authentication, and access control have been used [8]. However, these are typically passive measures against specific vulnerabilities. It is crucial to include proactive security measures in network system design to enhance immunity against threats. Recently, a trust management-based security mechanism has been proposed [9; 10; 11; 12]. Trust levels are assigned to network nodes based on direct and prior observations, guiding decisions in future interactions [13]. This approach enhances network security by avoiding nodes with security vulnerabilities during route discovery [14; 15; 16; 17]. Despite trust-based methods showing potential, their complexity, when coupled with deep learning, hinders real-time applications. The traditional Dijkstra shortest path algorithm [17] faces issues like slow convergence and responsiveness, causing network congestion, especially in dynamic environments with increasing traffic [18]. Machine learning, particularly deep reinforcement learning, has gained prominence due to its exceptional performance in data processing, classification, and intelligent decision making. This has led to the development of several popular algorithms, including the Deep Q Network (DQN) [19], Actor-Critic (AC) [20], Deep Deterministic Policy Gradient (DDPG) [21], Trust Region Policy Optimization (TRPO) [22], etc. More recently, researchers have explored integrating deep reinforcement learning into SDN routing specifications to achieve intelligent routing and fine-grained management [23; 24; 25; 26]. In their pioneering work, [PERSON] et al. [25] proposed a DQN-based deep reinforcement learning (DRL) scheme to generate dynamic traffic changes for SDN network routing. Likewise, [PERSON] et al.