# A3C-R: A QoS-Oriented Energy-Saving Routing Algorithm for Software-Defined Networks
## 1 Introduction  
In recent years, with the explosive growth of network traffic of various new network applications and services, data flows with a large number of business characteristics have been generated. For example, online teaching and video conferencing have high requirements on bandwidth, and online games and instant messaging have high requirements on delay. Therefore, in order to ensure the real-time nature of network service interaction, network services in different scenarios put forward higher requirements for quality of service (QoS) such as network bandwidth, delay, packet loss rate, and throughput [1]. At the same time, with the continuous popularization of data-intensive applications and the continuousgrowth of the number of network devices, the proportion of network infrastructure energy consumption is gradually increasing, and energy-saving flow scheduling has become a common trend [2]. In the face of huge network QoS and energy-saving demands, the disadvantages of the traditional \"best effort\" network architecture and design concept are gradually exposed, such as difficulty in controlling the network scale, implementing network management, guaranteeing network services, and updating network technologies [3]. Therefore, how to achieve more efficient QoS differentiated services and rational allocation of network energy resources under the existing available network resources is of great significance. In order to improve the openness and innovation of the traditional network architecture and promote the optimal selection of traffic scheduling strategies, software defined network (SDN) [4] technology has been well developed. SDN decouples the control plane and data forwarding plane of network equipment, which can make the utilization of network resources more fully and effectively [5]. Compared with traditional networks, SDN simplifies the complexity of network equipment, reduces network management configuration and operation requirements, effectively improves the flexibility of network control and management, and enhances the rapid support capabilities of new network technologies, new requirements, and new protocols. SDN-oriented routing planning makes the path selection flexible and changeable, which can realize the rational utilization of network resources, effectively avoid network congestion, and better improve the overall network service quality [6]. With the rise in artificial intelligence, machine learning technology has developed rapidly, and significant progress has been made in data processing and combinatorial optimization. Utilizing the convenience provided by SDN, SDN-oriented intelligent routing optimization and network resource scheduling can be realized, making network operation, maintenance, and management continue to develop in the direction of intelligence. Based on this, this paper proposes a QoS-oriented energy-saving routing optimization algorithm A3C-R in the SDN environment. A3C-R divides different types of service traffic, uses delay, bandwidth, and packet loss rate as QoS measurement indicators, and considers the energy consumption of the network. Based on the advantage of multi-threaded asynchronous update of the A3C algorithm [7], multiple agents can be executed simultaneously. It can reduce the correlation between samples in the training process through a variety of different state samples, thereby reducing the computing power demand in the process of agent and environment exploration, so as to choose the energy-saving path that meets the QoS requirements as much as possible. The main contributions of this paper are as follows:  
* We investigated the research value and significance of SDN-oriented QoS and energy-saving routing schemes, analyzed the research status of existing routing algorithms, and proposed an A3C-R intelligent routing algorithm that takes into account both QoS and energy saving.