## 2 Related Work  
Traditional studies have been conducted for formation control protocol along with changing configuration in multi-agent flight systems, mainly in distributed control strategies [1; 17]. However, these methods exhibit low accuracy and flexibility in dense environment. To address this, refs. [9; 18] proposed an affine-based mapping method for target configuration involving translation, rotation, scaling, shear, or compositions of these transformations. Ref. [9] extended the affine-based method with the stress matrix for changing the configuration within the nominal configuration set. However, these methods show the low effectiveness in transforming the configuration geometry in an unknown environment without predefining the change scale and nominal configuration. Additionally, this policy does not account for sudden changes in an agent's velocity regarding both magnitude and direction. Thus, the velocity is specifically calculated at each step of the configuration transformation, relying on prior information about the environment. To overcome this problem, many studies have brought the RL methods to model the issue as a navigation task. Refs. [19; 20] employed Q Learning (QL) to establish paths while avoiding static or dynamic obstacles in the plane. However, it hardly converges in complex and highly dynamic environments by extending to the high dimension. To enhance the ability of trajectory planning of mobile agents for dynamic configuration in dense environments, ref. [21] utilized the Deep Q Network (DQN) algorithm combined with greedy policy to support trajectory planning and optimization in a large-scale dense environment. As for the value-based Double DQN (DDQN) [22] method, ref. [23] explored a high-precision navigation technique for multi-agent systems to enhance the efficiency of target tracking. However, these methods mainly utilized discrete actions with limited agent action space, which may reduce the efficiency in real-world constraints. Ref. [24] proposed a modified Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. The primary improvement focuses on prioritized experience replay and dynamic delay updates, which reduce the impact of value estimation errors on training time. Ref. [14] introduces the Deep Deterministic Policy Gradient (DDPG) algorithm with continuous action space to address the problems of high dimension of continuous state space. Their experimental results demonstrated that the combined DDPG algorithm exhibited superior performance in generating effective path plans for mobile robots within continuous action spaces. However, the DDPG algorithm suffers from low training efficiency and the local minimum problem, which negatively impacts motion convergence due to a high proportion of illegal policies stemming from the lack of policy action filtering [25]. Therefore, recent research on proximal policy optimization (PPO) has shown great potential in multi-agent navigation. In [26], this algorithm was used to solve the trajectory tracking problem in manipulator and mobile robot areas. Ref. [27] utilized the PPO to ensure safety constraints through constrained neural networks evaluated by the cost function. In addition, ref. [28] developed a PPO-based MADRL framework in the handover and power allocation problem, which fully used the local observation to the lower bound of the objective function and policy gradient. Ref. [29] demonstrated the effectiveness of MAPPO in the Particle World environments, Starcraft II Micromanagement Tasks and the Hanabi Challenge with continuous action spaces. The possibility of applying DRL for path navigation of robots and UAVs has been confirmed in the literature. Compared with other DRL-based methods, the policy-gradient-based algorithms have more advantages in continuous action generation. In addition, the prediction module is a vital stage in motion navigation. Ref. [12] utilized a distributed nonlinear model predictive controller (DNMPC) combined with an estimator module to achieve an unrigid configuration. Ref.