## 1 Introduction  
Formation flight with various changing configurations has presented a significant challenge, demanding the algorithm design the control protocol to change the configuration continuously while effectively avoiding obstacles [1, 2, 3, 4]. The traditional control methods have shown strong significant results in the area of maintaining and changing the geometry rigid configuration [3, 4]. Meanwhile, the formation flight system is essential for the multi-agents in the constrained environment. However, a rigid configuration maintenance can lead to high system complexity and stability issues [3, 5]. Traditional algorithms have mainly focused on position-, displacement- and distance-type studies, based on the types of sensed and controlled variables used thus far. Two key challenges limit the practical application of configurations: (a) The preset trajectory sequence of each agent within the formation restricts configuration changes to time, scale and orientation variations in the target configuration [6, 7, 8], especially for the rigid configuration control level [6, 7, 8, 9]. (b) The formation would lead to reconstruction failures from the undesirable situation by unknown obstacles and the agents' breakdowns with much communication consumption [9; 10]. Various studies exist on the formation control protocol along with changing configurations in a multi-agent flight system, such as the rigid configuration control level [3; 4; 5; 6; 7; 8]. While these methods are valued for their stable structure in formation flight, they need to preserve the relative positions (displacement/distance/bearing-based [1; 3; 11]) and facilitate transitions between various formation shapes. Preserving collinearity and distance ratios would strongly rely on the predefined change scale and nominal configuration. Alternatively, some methods focus on model predictive control (MPC) [12; 13], utilized by constructing a model to meet safety and flexibility requirements only in a strong connected approach. While this policy can ensure unknown obstacle avoidance and dynamically adjusts geometry by modulating velocity values and directions based on current costs to some extent, it heavily depends on strong communication among agents, making it susceptible to reconstruction failures if one or more agents experience breakdowns [13; 14]. However, the Multi-Agent Deep Reinforcement Learning (MADRL) technique has shown less research in the area of configuration control. Based on the practical challenges, the objectives of continuous configuration control can be summarized as follows: an adaptable module for adjusting the local trajectory while maintaining configuration, a predictable module that smoothly responds to sudden changes in values and a versatile module for forming flexible configurations in dense environments. In our previous work [15], proposing a method named as Affine-based Maneuver Control on Optimal Flight System (AMCOPS), we focused solely on the first objective above, based on the local observation of each agent, addressing the conflict between the obstacle avoidance in the local frame and the rigid configuration maintenance in the global one. An integrated flight framework of the global and local control law was designed as a nonlinear optimization problem. However, the fmincon nonlinear optimization cannot be extended to the higher dimension caused by the large calculation [16], where the computation of Hessian matrix after sampling all the points' values in every iteration limits the extension capability. In this paper, we address the prediction and flexibility challenges during the formation interaction with the environment by proposing a novel Deep Reinforcement Learning (DRL) integrated with the distributed model predictive control (DMPC) method. Our framework establishes an online learning policy of integrating local and global control. The problem of multi-agent flight system has been modeled as a fully cooperative DRL problem.