## 1 Introduction
In the G2 RL proposed by [PERSON] [12], the same concept of regional observation as PRIMAL is used as well as the concept of short-term and long-term memory. There are also sequential inputs that use the same concept as GLAS. G2 RL has implemented and analyzed algorithms for complex environments. The results proved that G2 RL can maintain good results in different environments. The A* algorithm [13] has excellent path planning ability in one robot and can find the shortest path in a very short time. However, the A* algorithm is still limited in terms of path planning for multiple robots in the same field. It contains overlapping paths that may cause collisions, and the amount of computation may be too large. Several studies modified A* for multi-robot applications, including the windowed hierarchical cooperative A* (WHCA*) proposed by [PERSON] [14]. Various methods have also been developed to compute the optimal solution for MAPF problems including subdimensional expansion [15], compilation-based solver [16], and integer programming-based methods [17] and Conflict-Based Search (CBS) [18; 19]. However, they are associated with a long computation time [20], making it one of the difficulties in a multi-agent learning engine. In this study, A* is proposed to explore the best local path planning as an expert's guide in the design reward function. There are three main architectures for multi-agent learning, namely, fully decentralized, fully centralized, and centralized training and decentralized execution [21]. Due to the powerful agent-centered, multi-agent, and support of actor-critic methods, multi-agent learning could generate fair credit assignment and better generalizability. It also allows information to be distributed evenly across multiple agents [22]. Fully decentralized, multi-agent reinforcement learning is an environment where each agent is individualized, and observed states and actions are not shared between agents. The core architecture of reinforcement learning is that the agent obtains the state of the environment after executing an action. Even when only one agent is interacting with the environment, no problem arises, because the environment is only affected by the actions of the agent. However, with multiple agents operating in the same environment, the actions performed by each agent affect the state of the other agent. A fully decentralized structure would therefore be difficult to converge. In a fully centralized architecture, a central neural network determines the actions to be performed by all agents and waits for all agents to finish running before collecting the status and rewards from all agents. It ensures that the neural network is aware of the state of the environment, avoiding the problems of a completely decentralized framework. However, since it needs to wait for all the agents to finish their actions before data collection, a slowdown problem could occur. Therefore, a centralized learning with decentralized execution (CLDE) framework is selected to form multi-agent model-based policy optimization (MAMBPO) for better sample efficiency than the model-free multi-agent soft actor-critic [23; 24]. Centralized training coupled with decentralized execution combines the advantages of both structures [25]. In this paper, we used an actor-critic framework to avoid stochastic state changes due to agent interactions. Each agent has a policy network to execute actions and observe the state of the environment. The central review network collects all the actions performed by the agents, the observed status, and the rewards to judge the system. This ensures that there is a centralized network of reviewers who can update the overall training of the system at the time of training. When the whole system learns to converge, the policy network of all agents finds the best parameters to use. The final execution phase only requires that each agent acts through its own policy network. The important policy gradient methods of Proximal Policy optimization (PPO) are taken to improve the performance of the DQN [26].