## 3 System Scheme and Implementation  
### MAFS Traffic Scheduling System  
This paper proposes the CTN-MADDPG algorithm based on the MADDPG algorithm and builds the MAFS traffic scheduling system. The overall structure is shown in Figure 1. The core of this system model is the control plane, where the controller obtains information from the underlying network through a southbound interface to achieve centralized control function. The current network state information is obtained using INT on the data plane and transmitted to the control plane. The control plane preprocesses the network state data, using it as input for a neural network to calculate the routing strategies. Subsequently, the controller converts the optimal routing strategy into flow table rules, which are then deployed to the data plane. Figure 1: Architecture of MAFS traffic scheduling scheme. Information Preprocessing Module: This module is primarily responsible for processing the obtained information from the data plane and feeding the processed data into the traffic scheduling module. It integrates the switch load information collected by INT within the data plane as input to the state information of the Actor network and Critic network. Additionally, it consolidates the link load information received from the receiver feedback into reward values, which are utilized to guide the selection of the routing strategies for each intelligent agent. Traffic Scheduling Module: This module primarily executes the CTN-MADDPG algorithm, with each intelligent agent corresponding to an Actor-Critic framework. Each intelligent agent interacts with the data plane environment to obtain the current state information of the agent, and the distributed execution of the Actor network obtains the optimal routing strategy. In the centralized training process, the Critic network can obtain the global state information and the routing strategies adopted by all the intelligent agents. It grasps the global value function to evaluate the utility of the set of routing strategies selected by each agent to guide the update of the Actor network. Route Conversion Module: This module is primarily used to convert the routing selection results of the traffic scheduling module into the form of flow rules in order to obtain new flow table rules. It dynamically modifies the flow rules executed by the switches on the control plane and distributes them to the data plane. When a data packet enters a switch, the switch will forward it based on the corresponding forwarding rules to achieve dynamic data transmission. Compared with the previous single-agent deep reinforcement learning methods, the multi-agent deep reinforcement learning method can assign decision tasks to multiple agents for parallel learning and decision-making. Each agent can independently observe network states and make routing decisions. By sharing experiences and learning for each other, agents can improve their routing strategies, thereby improving overall network performance. The MAPS traffic scheduling method proposed in this paper designs the CTN-MADDPG algorithm in the traffic scheduling module, which is based on the MADDPG algorithm of MADRL. The MADDPG algorithm is a deep deterministic policy gradient algorithm for a multi-agent environment. Each agent corresponds to an Actor-Critic network structure, adopting the mode of centralized training and distributed execution. It combines the advantages of the action-value function and policy gradient method, utilizing the information of the action-value function to guide the policy update, thereby addressing the deterministic policy optimization problem in a continuous action space. In general, the structure of the Actor network and Critic network of the MADDPG algorithm are composed of three fully connected layers, which may not be able to capture complex feature information. In addition, the model has fewer parameters, which can easily lead to overfitting or lower learning efficiency.