## 4 Implementation of MAFS Traffic Scheduling Algorithm
### The Algorithm of MAFS  
During the operation of MAFS, the SDN controller monitors the global network states in real time by establishing a connection with the switches for data interaction. Through continuous iterative training and updating, the CTN-MADDPG algorithm can optimize its own network model and continuously learn network states to generate the optimal forwarding paths, thereby achieving load balancing. The CTN-MADDPG algorithm assumes that there are \(N\) agents, where the policy representation of each agent is \(\pi_{i}\), which is the mapping of the network state information to the routing policy. The policy set is \(\pi=\{\pi_{1},\pi_{2},\cdots,\pi_{N}\}\), and the set of policy parameters is \(\theta=\{\theta_{1},\theta_{2},\cdots,\theta_{N}\}\). The computation process of MAFS is outlined in Algorithm 1. ```
0: Data plane information collected by INT;
0: Optimal path;
1: Set the state and action; initialize Actor Network parameters \(\theta=\{\theta_{1},\theta_{2},\cdots,\theta_{N}\}\) and Critic Network parameters \(\phi=\{\phi_{1},\theta_{2},\cdots,\phi_{i}\}\) for each agent;
2:for episode \(=1\) to \(MAX\_EPISODE\)do
3: Reset the environment, obtain initial state \(s_{t}\);
4:for step \(t=1\) to \(MAX\_STEP\)do
5:for agent \(i=1\) to \(N\)do
6: action \(\leftarrow\) mafs.action(s);
7: Select scheduling path as \(a_{i}\), and controller execute the \(a_{i}\);
8:endfor
9: Observe reward \(r_{t}\) and next state \(s_{t+1}\);
10:\(s_{t}\gets s_{t+1}\);
11: Store \((s_{t-1},a_{t-1},r_{t-1},s_{t},a_{t},r_{t},s_{t+1})\) in experience replay buffer D;
12:for agent \(i=1\) to \(N\)do
13: Sample a random minibatch of S samples from D;
14: Set \(y=r_{i}+\gamma Q_{i}^{\mu^{\prime}}(s,s^{\prime},a_{1},\cdots,a_{N},a^{\prime} _{1},\cdots,a^{\prime}_{N})|_{a^{\prime}_{i}=r^{\prime}_{i}(o_{i})}\);
15: Update Critic Network by minimizing the loss \(L(\theta_{i})\);
16: Update Actor Network using the sampled policy gradient \(\
abla_{\theta_{i}}J(\mu_{i})\);
17:endfor
18: Update target network parameters for each agent \(i\): \(\theta^{\prime}_{i}\leftarrow\tau\theta_{i}+(1-\tau)\theta^{\prime}_{i}\); \(\phi^{\prime}_{i}\leftarrow\tau\phi_{i}+(1-\tau)\phi^{\prime}_{i}\);
19:endfor
20:endfor
```  
**Algorithm 1** MAFS Method  
The MAFS method updates the network model by providing a reward indicator of the current data center network load balancing based on the current network state \(s_{t}\) and the selected scheduling path \(a_{t}\). By maximizing the cumulative reward, the load in the data center network becomes more balanced. The cumulative expected reward for each agent is given by:\[J(\theta_{i})=\mathbb{E}_{s\sim p^{\pi},a_{i}\sim\pi_{\theta_{i}}}\left[\sum_{t=0} ^{\infty}\gamma^{t}r_{i},t\right], \tag{6}\]  
where \(\theta_{i}\) represents the parameters in the \(i\)th policy network, and \(r_{i}\) represents the reward value of the \(i\)th agent, which is composed of the average throughput and end-to-end delay in the current network. In the process of solving the traffic scheduling problem in the DCN, if the random policy is adopted, and the selection of the action is made according to the probability distribution, the process of policy optimization will be affected by large fluctuations and lead to the instability of the training. In contrast, the deterministic policy based on the CTN-MADDPG algorithm enables each agent to choose a specific action based on the current network environment information to make a specific routing decision, which makes the agent have better stability and convergence in the process of making a traffic scheduling decision. At the same time, the deterministic policy can more flexibly adjust the traffic scheduling strategies to adapt to different network topology environments and traffic load situations, thus improving the global network performance. Therefore, the deterministic policy is adopted in this paper. In a given network state, each agent directly outputs a complete routing path based on the current network environment information.