# Reinforcement Learning-Based Anti-Jamming in Networked UAV Radar Systems
## 1 Introduction  
A radar countermeasure is an important part of an electronic countermeasure [1]. An effective radar countermeasures can deal with the jamming of various electronic equipment and signal modes [2]. At present, the mainstream countermeasure methods are as follows: 1. Game theory: establish a network model of multiple agents and use game theory to model the adjustment of policies between jammer and radar as Stackelberg game. Through analyzing and solving the game equilibrium solution (such as the Nash equilibrium), we can obtain the optimal transmission power and other policies for users to realize anti-jamming [3; 4; 5]. 2. Physical layer security: this is a supplement to traditional cryptography-based methods. Users usually generate noise signals to confuse potential eavesdroppers, which significantly improves the quality and reliability of secure communication between legitimate terminals [6; 7]. 3. Machine learning: intelligent countermeasures represented by reinforcement learning have been widely used in recent years. Reinforcement learning realizes the policy iteration through the interaction between agent and environment and finally generates a reasonable policy set [8; 9]. With the progress of electronic jamming, the traditional jammer with single function and fixed working mode is promoted to intelligent jamming of multi-function, multi-parameter and variable working mode [10]. Therefore, the intelligent radar system becomes the focus of anti-jamming research [11]. Because the traditional radar system often uses passive methods such as a waveform library to identify the jamming form, this results in poor real-time policy [12]. When it deals with newforms of jamming, the utility of this approach is limited. Therefore, an intelligent radar system needs to generate real-time policies under the condition of flexible jamming forms. Radar countermeasure of an intelligent unmanned aerial vehicle (UAV) swarm is an effective form of intelligent radar countermeasure. Unlike the mission planning of general multi-radar systems, swarm intelligence UAV technology pays more attention to the individual UAV with a simple structure and limited energy. Through internal interaction and swarm control, the detection capability of a single UAV in a swarm is greatly promoted, and it is more flexible than traditional radar. A UAV has the characteristics of good concealment, flexible motion and rapid networking. An intelligent UAV cannot only perform common tasks such as surveillance of unknown areas, protection of key targets, territorial investigation and so on, and more flexible operations such as radar networking can be used to achieve radar countermeasures by airborne small radar [13]. Radar networking based on a UAV swarm greatly improves the ability of radar information acquisition by properly distributing radar stations with different systems, different frequency bands, different working modes, and linking them into networks. However, the following design considerations should be taken into account in intelligent networked UAV radar systems:  
1. Intelligent UAVs may be energy-limited. The energy modules on most UAVs may pose a major constraint on their motion ability. Meanwhile, when the UAV carries a radar payload to perform detection tasks, its motion and adaptation of the radar parameters may introduce considerable energy consumption. Therefore, a UAV may need to strike a beneficial trade-off between the motion energy and energy consumed by the radar. 2. The antenna performance of smart UAVs may be limited. Due to the limited volume and load capacity of the UAV, the number of array elements may be limited, resulting in poor spatial resolution. However, the beam width adopted by radar in certain tasks may demand high spatial resolutions, e.g., when measuring angle by beam scanning method. Therefore, the networked UAV radar system should exploit the distributed UAV cooperation to overcome the antenna constraint of a single UAV. 3. The topology of the networked UAV radar system may be complex.