## 1 Introduction  
With the development of unmanned combat air vehicles (UCAVs), the role of UCAVs is becoming increasingly significant in the field of combat [1]. UCAVs have high efficiency and can perform large overload maneuvers that are otherwise difficult to achieve with manned combat aircrafts. Therefore, it can be predicted that UCAVs will become the main protagonists of future air combats. Most UCAVs are too small to be equipped with medium-range air-to-air missiles (MRAMs). Therefore, within visual range (WVR) air combats are likely to become the main air combat scene of UCAVs [2]. The UCAV air combat scenario studied in this article is a 1-vs.-1 WVR UCAV air combat scenario using guns. Existing UCAVs can only complete simple military tasks, such as autonomously or semi-autonomously detecting, tracking, and striking ground targets. An example of a UCAV is the RQ-4 \"Global Hawk\" strategic unmanned reconnaissance aircraft. Although military UCAVs can already perform reconnaissance and ground attack missions, most of these missions rely on the manual decision-making of UCAV ground stations, which operate in the man-in-the-loop control mode. Furthermore, using UCAV ground stations to control UCAVs renders them vulnerable to bad weather conditions and electromagnetic interference, both of which can be problematic in real-time air combat situations. Therefore, it is crucial for UCAVs to be able to make control decisions automatically based on the air combat situations it faces; UCAV autonomous air combat is therefore an important research topic in the field of UAV intelligence. Owing to the rapidly changing battlefield environments, making autonomous maneuver decisions during air combats is challenging. For example, if one UCAV tracks another UCAV that is steady, and then, if tracked UCAV suddenly slows down, the tracking UCAV may overshoot; this will lead to a sudden change in the air combat situation. Therefore, autonomous maneuver decisions need to be based on mathematical optimization, artificial intelligence, and other technical methods in order to ensure the automatic generation of maneuver strategies in various air combat situations [3]. Since the early 1960s, a lot of research has been conducted on autonomous air combat, and some key research results have been achieved. At present, several methods have been proposed to enable autonomous maneuver decision-making during air combats, and these methods can be roughly divided into the following three categories: basic fighter maneuvers (BFM)-based methods using library and game theory, optimization-based methods, and artificial intelligence-based methods. First, typical research based on the basic fighter maneuvers (BFMs) using library and game theory are introduced. In [4], the first systematic study was conducted and a summary of the establishment of BFM expert systems was presented. The design of the maneuver library, control applications, and maneuver recognition based on the BFM expert system were proposed, and various problems that were encountered during maneuver decision-making based on the action library were elaborated in [5; 6]. Based on a combination of the BFM library, target prediction, and impact point calculations, an autonomous aerial combat framework for two-on-two engagements was proposed in [7]. Air combat maneuvering methods that are based on the optimization theory include the dynamic programming algorithm [8], intelligence optimization algorithm [9], statistical theory [10] and so on. In [8], approximate dynamic programming (ADP), a real-time autonomous one-to-one air combat method was studied, and the results were tested in real-time indoor autonomous vehicle test environments (RAVEN). ADP differs from classical dynamic programming in that it constructs a continuous function to approximate future returns. ADP does not need to perform future reward calculations for each discrete state, therefore, its real-time performance is reliable.