# A3C-R: A QoS-Oriented Energy-Saving Routing Algorithm for Software-Defined Networks
## 5 Experimental Evaluation  
### Experimental Environment and Comparison Algorithms  
In this paper, the network simulation software Mininet is used to realize the QoS and energy-saving test of the A3C-R routing algorithm. The experiment process adopts the RYU controller (version 4.34) and Open vSwitch switch (version 2.9.8). The RYU controller was selected for its support of OpenFlow 1.3, which allows for dynamic flow table management and fine-grained control of network flows. Its flexible, scalable, and modular architecture facilitates seamless integration with the A3C-R algorithm, enabling QoS-aware and energy-efficient routing. We use OS3E as the network topology. OS3E network topology consists of 38 nodes, which simulate various network devices such as switches, routers, and hosts. These nodes are interconnected through 48 links. The bandwidth of each link is uniformly set to 200 Mbps, which serves as a standardized parameter for testing the performance of different routing algorithms. This setting enables a fair comparison of how algorithms affect network performance metrics like delay, throughput, packet loss rate, and energy-saving efficiency under the same bandwidth condition. The experimental hardware implementation platform is the Linux operating system Ubuntu 18.04, tensorflow 1.8.0, and Python 3.5.0. The hardware is an i5-10600 KF-CPU, with 16 GB-DDR4 memory, and two GTX-1080 8 G graphics cards. During the training process of A3C-R, the neural network uses the Relu activation function. In the pre-experiments, we tried a number of different combinations of learning rates. When the learning rate of the Actor network is too large, the model training is unstable and easy to skip the optimal solution; when it is too small, the training speed is slow and falls into the local optimum. It has been tested that a learning rate of 0.002 enables the Actor network to efficiently explore and learn the optimal policy while maintaining the training speed. For Critic networks, a learning rate that is too large leads to large fluctuations in the value assessment, while a learning rate that is too small is too slow to guide the Actor network in a timely manner. It was ultimately determined that a learning rate of 0.001 would enable the Critic network to stabilize the learning value function and provide reliable feedback to the Actor network. The algorithm trains for 100,000 steps, with the learning rates for the Actor and Critic set to 0.002 and 0.001, respectively. The batch size for training is 128, and the discount factor is 0.9. Specifically, the discount factor (0.9) is chosen based on related deep reinforcement learning (DRL) research in SDN routing optimization [20], which suggests that values between 0.8 and 0.99 balance long-term and short-term rewards effectively. We also conducted initial experiments with discount factors of 0.7, 0.8, 0.9, and 0.99, and found that 0.9 led to better convergence stability and long-term performance. Similarly, for the batch size, we tested values of 32, 64, 128, and 256, and found that a batch size of 128 achieved a good trade-off between training stability and computational efficiency. This experiment is compared with traditional routing algorithm, intelligent routing algorithm, and energy-saving routing algorithm. The specific comparison content includes (1) the traditional multi-path routing algorithm ECMP. ECMP is a widely used routing algorithm that distributes traffic across multiple equal-cost paths, serving as a baseline for traditional routing performance. Comparing with ECMP allows us to demonstrate how A3C-R improves QoS and energy efficiency beyond conventional approaches. (2) I-DQN, a fast-converging DQN routing algorithm considering multi-type QoS requirements [21]. I-DQN is a deep reinforcement learning-based routing algorithm designed to optimize multiple QoS metrics such as delay and throughput. Since A3C-R also targets QoS improvements, I-DQN serves as a strong benchmark for evaluating QoS optimization effectiveness.