###### Abstract  
In data center networks, when facing challenges such as traffic volatility, low resource utilization, and the difficulty of a single traffic scheduling strategy to meet demands, it is necessary to introduce intelligent traffic scheduling mechanisms to improve network resource utilization, optimize network performance, and adapt to the traffic scheduling requirements in a dynamic environment. This paper proposes a fine-grained traffic scheduling scheme based on multi-agent deep reinforcement learning (MAFS). This approach utilizes In-Band Network Telemetry to collect real-time network states on the programmable data plane, establishes the mapping relationship between real-time network state information and the forwarding efficiency on the control plane, and designs a multi-agent deep reinforcement learning algorithm to calculate the optimal routing strategy under the current network state. The experimental results demonstrate that compared to other traffic scheduling methods, MAFS can effectively enhance network throughput. It achieves a 1.2\(\times\) better average throughput and achieves a 1.4-1.7\(\times\) lower packet loss rate. data center network; traffic scheduling; multi-agent deep reinforcement learning; in-band network telemetry; programmable data plane +
Footnote †: journal:  
0000-0001-999-227  
learning (DRL) [3] has achieved remarkable results in this field. Compared with the traditional routing algorithms, the intelligent routing model that combines a SDN with deep reinforcement learning can automatically learn the mapping relationship between input parameters and output results by observing the empirical information provided by the environment. This not only saves time and costs but also continuously adjusts to output better routing strategies as the environment changes so as to achieve the purpose of load balancing. In recent years, the proposal of multi-agent deep reinforcement learning (MADRL) [4] has brought new hope for optimizing network performance. This method solves large-scale complex problems based on the relationship between individual agents collaborating with each other, which not only reduces the complexity of problem-solving for individual agents but also improves the overall problem-solving ability of multiple agents. In summary, by setting the network topology, the data packet processing logic is realized on the programmable data plane using P4 language, and the intelligent machine learning method is introduced on the control plane to effectively complete the routing decision and the corresponding forwarding of the data packet. Therefore, applying the MADRL method to a data center network will provide a new strategy to solve the load balancing problem. This paper proposes a traffic scheduling method called MAFS based on multi-agent deep reinforcement learning. In-Band Network Telemetry was used to collect global network state information [5], and the method customizes the environment, state space, action space, and reward function of the model. Furthermore, it designs the CTN-MADDPG algorithm based on a one-dimensional convolutional neural network (1D-CNN) [6] and long short-term memory network (LSTM) [7] to realize traffic scheduling decisions centered on the control plane. In general, the main contributions of this proposed work include the following:  
A traffic scheduling method based on multi-agent deep reinforcement learning is proposed. In the process of traffic scheduling, the CTN-MADDPG algorithm is designed, enabling individual agents to learn how to better collaborate in the network environment to complete routing strategies through the joint training of deep neural networks for agents. In the CTN-MADDPG algorithm, the 1D-CNN and LSTM are combined to design the Actor network and Critic network for each agent so that the model captures time-dependent relationships in sequential data, enhancing the modeling and representation capabilities of the model for time-series data and optimizing the selection of routing strategies.