# Reinforcement Learning-Based Anti-Jamming in Networked UAV Radar Systems
## 2 Related Works  
UAVs have been used widely in the anti-jamming field. A UAV is designed in [14] to avoid the jamming of the aerial jammer to the channel by differential game method. This method could generate the optimal policy to avoid jamming. A UAV was used in [15] to send onboard units information, which improved the communication performance of vehicular ad-hoc networks in the presence of intelligent jammer. As for swarm UAV, [PERSON] et al. proposed a countermeasure model based on airborne cognitive radio and used an improved energy detection method to detect jamming [16]. [PERSON] et al. presented a method of using a software-defined network (SDN) to counter software-defined radio (SDR) [17]. A new type of network using a dual-controller generation policy was proposed. [PERSON] et al. designed a multi-UAV countermeasure system and used Q-Learning algorithm to generate radar transmitting frequency [18]. [PERSON] et al. reduced the resource allocation problem in electronic countermeasures to an optimal allocation problem, and proposed a resource allocation algorithm based on particle swarm optimization theory [19]. At present, there is some research on radar reinforcement learning, mainly in the field of deep reinforcement learning. An intelligent radar countermeasure method was studied in [20] under the condition where the number of working modes of multi-functional radar was unknown. The jamming receiver recognized the working state of the radar by processing the information, and then generated a new effective jamming pattern. [PERSON] et al. combined reinforcement learning with cognitive radar in [21], so that transmitting and receiving could adapt to an unknown dynamic environment. It explained how to realize spectrum allocation of automotive radar in reinforcement learning-based cognitive radar system. [PERSON] et al. simulated an electronic attacker as a UAV and a defender as an observatory in [22], where the UAV needed to detect targets. A deep deterministic policy gradient (DDPG) algorithm based on variational Bayesian estimation was designed to train the motion policy of the UAV. [PERSON] et al. proved that some basic reinforcement learning algorithms could be successfully applied to dynamic detection of multiple-input-multiple-output (MIMO) radar with unknown environment and unknown number of targets [23]. Deep learning was used in [24] to discuss channel selection when wideband radar signals coexist with narrowband communication signals in the communication systems. Some progress has also been made in the research of the combination of jamming and physical layer security. For example, [PERSON] et al. reviewed the latest work of cooperative relaying and jamming in detail [6]. These technologies are used to protect wireless transmission from eavesdropping nodes. They explained that using cooperative security can improve the physical layer security, and proposed some methods and conclusions to improve the wireless communication performance based on physical layer security. For example, relay positioning can be further explored, multi-cellular design can be further studied, etc. This paper is of great significance to the design a security scheme in the 5G communication network. At the same time, [PERSON] et al. summarized the jamming policies in the physical layer of wireless communication and classified the jamming policies in [25], which provided a theoretical basis for the design of jamming policy in a wireless communication system. From the literature, it may be concluded that frequency-domain anti-jamming techniques have been widely adopted in the field of anti-jamming of UAV radar networking, but the degrees of freedom exhibited in motion and antenna arrays of networked-UAV radar systems are not fully exploited. Secondly, the radar anti-jamming techniques proposed in the literature generally adopted SIR or SNR as the reward in reinforcement learning, while it may not fully capture the information acquisition ability of the radar.