## 3 Proposed Methods
### Algorithmic Pseudo-Code
In other words, the two-dimensional latent observational data extracted by the encoder in this task is the important observational information that affects the action executed by the actor. However, the overall strategy remains largely unaffected across the three dimensions, as evidenced by episode returns. Furthermore, experiments validating the model based on local state transition mechanisms involved FACMAC, FACMAC+Encode, and ST-FACMAC, and the experimental results were shown in Figure 6:

FACMAC+Encode refers to the fact that the local observation state and the state related to the local state transition automatically extracted by the encoder are used as the input of the actor network; ST-FACMAC refers to the fact that only the states related to the local state transition automatically extracted by the encoder are used as the input of the actor network, i.e., the return value of the algorithm with automatically extracted local observation states that are closely related to the local transformation mechanism added to the above comparative experimental graph is much higher than that of the algorithm without automatically extracted local observation states, and the curve after convergence is smoother, that is, the ability of the agent to use the strategy in the algorithm is stable. It can be seen that the local observation states extracted based on the local state transformation mechanism model effectively improve the ability of the agent to use the strategy, and the stability is better. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Experimental Environment** & **NST-FACMAC** & **FACMAC** & **MATD3** & **MADDPG** \\ \hline Ant-v2 (2 \(\times\) 4) & 996.87 (2.73) & 891.19 (42.67) & 978.78(6.65) & 824.71 (42.70) \\ Ant-v2 (2 \(\times\) 4d) & 1001.89 (2.19) & 808.23 (100.66) & 955.83 (3.91) & 831.17 (92.02) \\ Ant-v2 (4 \(\times\) 2) & 1000.83 (2.94) & 908.38 (59.22) & 894.66 (6.53) & 900.20 (16.21) \\ Manyant (2 \(\times\) 3) & 1005.30 (13.52) & 1.86 (201.53) & 738.40 (16.43) & \(-\)6061.82 (1977.75) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The mean and standard deviation of the 10 K returned value after convergence. Figure 5: The episode returns for NST-FACMAC training with z_dim of 2, 4, and 8 in the Manyagent Ant [\(2\times 3\)]. Centralized exploration based on local state transitions was also assessed through experiments with NST-FACMAC, ST-FACMAC+NS, and ST-FACMAC, and the experimental results are shown in Figure 7:

In this experiment, NST-FACMAC, ST-FACMAC+NS, and ST-FACMAC algorithms implement three different exploration strategies based on the integration of local state transition mechanism components. NST-FACMAC adopts the centralized novelty exploration strategy proposed in this paper (Formula (10)), and ST-FACMAC only adopts the Gaussian strategy, which is the most commonly used in the multi-agent strategy gradient algorithm, for exploration. ST-FACMAC+NS adopts both the centralized novelty strategy and the Gauss strategy to observe whether further enhancing the agent's exploration capability on top of the exploration strategy proposed in this paper is conducive to the cooperation of agents to accomplish this task. The experimental results show that NST-FACMAC and ST-FACMAC+NS converge faster than ST-FACMAC; that is, the algorithm with a centralized novelty strategy converges faster than the algorithm without a centralized novelty strategy. Therefore, it can be seen that using a centralized novelty strategy can improve the learning speed of agents. NST-FACMAC ultimately receives a higher return value than ST-FACMAC+NS, which further indicates that the agent can fully explore by adopting the centralized novelty exploration strategy and does not need to explore using Gaussian noise. Figure 6: Experiments to verify the effectiveness of local state transitions. In summary, the incorporation of a local state transition model facilitates automatic extraction of relevant local states, enhancing agents' strategic capabilities.