## 1 Introduction
In [11], particle swarm optimization, ant colony optimization, and game theory were applied in a cooperative air combat framework, and the results were compared. Artificial intelligence methods mainly include the use of neural network methods [12; 13] and reinforcement learning methods [3; 14; 15; 16]. Air combat maneuvering decisions based on artificial neural networks are extremely robust and they can be learned from a large number of air combat samples. However, air combat samples comprise multiple sets of time series as well as the results of air combats. Owing to confidentiality, air combat samples are difficult to obtain, and the label of such samples are extremely sparse. Furthermore, manual work required to label samples at every sampling moment. Therefore, the method of generating air combat maneuvering strategy based on neural networks has the problem of insufficient samples. Owing to the aforementioned limitations, air combat maneuvering decisions that are based on reinforcement learning are now garnering significant attention. This is because, with this learning method, the sample insufficiency problem, which exists in the case of supervised learning, can be avoided. In [3], an algorithm that is based on a deep Q-network and a new training method were proposed to reduce training time and simultaneously obtain sub-optimal but effective training results. A discrete action space was used, and the air combat maneuvering decisions of an enemy UCAV were considered. Single-agent reinforcement learning has been successfully applied in video games, board games, self-driving cars, etc. [17]. However, UCAV 1-vs.-1 WVR air combats happen in unstable environments, and the Markov decision process (MDP) is not suitable for modelling them. In most articles [3; 6; 8], the authors will assume an air combat strategy of the enemy, so that an unstable environment model can be converted into a stable environment model. An air combat strategy obtained using a stable environmental model is therefore of little significance, as the obtained policy may only be effective against the assumed air combat strategy of the enemy and, therefore, it may be invalid or not optimal for other enemy air combat strategies. At the same time, UCAV 1-vs.-1 WVR air combat has high real-time and high confrontation characteristics, so, it is critical to accurately predict the maneuvers and intentions of the other party's UCAV. In the vast majority of articles, the author only uses the current UCAV motion state as the state space for reinforcement learning, which makes it difficult for a trained UCAV air combat strategy to learn predicted target maneuvering and intentional air combat decisions. This means that an air combat strategy will not consist of the intelligent behavior necessary to be in the dominant position in advance. In order to solve the above mentioned problems, we propose a UCAV 1-vs.-1 WVR air combat strategy generation method that is based on a multi-agent reinforcement learning method with the inclusion of a target maneuver prediction in this article. The main novelties of this paper are summarized below:  
1. We formulate the UCAV 1-vs.-1 WVR air combat as a two-player zero-sum Markov game (ZSMG). 2. We propose a UCAV 1-vs.-1 WVR air combat strategy generation algorithm based on multi-agent deep deterministic policy gradient (MADDPG). 3. The future maneuvering states of the target UCAV are introduced into the state space. 4. Introducing potential-based reward shaping method to improve the efficiency of maneuver strategy generation algorithm of UCAV.