# Reinforcement Learning-Based Anti-Jamming in Networked UAV Radar Systems
## 5 Simulation Results
### Collaborative Target Ranging  
In this scenario, the radar aggregates the ranging measurements of multiple scattering points for target localization. Range measurement is carried out by using two radar UAV swarms. Two radar UAV swarms are separated from each other, and the target position is determined by the intersection area of beams form by the two swarms. In this scenario, because the distance spanning from the target to the radar is significantly larger than the horizontal size of the target, all scattering points of the target are in the same range profile and may be treated as a whole. Therefore, the radar UAV swarms tend to form wider beams to cover more scattering points, which results in improved SIRs and ranging accuracy. In the default policy of this group of experiments, the radar transmitting frequency is maintained at 9.4 GHz. The forwarding frequency is always 10.6 GHz. The width of both mainlobe is 50\({}^{\circ}\). The relay position is always at [6; 18]. The scanning mode of beam pointing is from right to left, and then from left to right. In order to display the results more vividly, we give the schematic diagram of time sequence (Figure 8), frequency chart (Figure 9) and normalized information chart (Figure 10). In the schematic diagram of the time sequence, the dot represents the position of each agent and the line represents the mainlobe range of each agent's 3 dB antenna pattern. Purple represents the communication UAV. Brown represents the radar UAV swarm (the distance between the UAV swarm is very small, so the whole swarm is replaced by a point). Red represents the jammer 1. Blue represents the target scattering point distribution. Pink represents the jammer 2. The legends of the schematic diagram of the time sequence are shown in Figure 11. In order to show the results more clearly, we exclude the base station shown in Figure 7. In order to illustrate the effectiveness of this discrete modelling provided by Q-Learning, we will give the results of normalized information quantity after enlarging the scale of the problem. It should be noted that the maximum and minimum information quantity of each experiment remain unchanged while increasing the scale of the problem. Take the scenario of collaborative target ranging for example, after taking the natural logarithm of each group's information  
Figure 7: Schematic diagram of gridded modeling. quantity, the maximum value is 6.2107, and the minimum value is \(-8.2248\), so it is still meaningful to compare the normalized information quantity. Figure 8: Schematic diagram of time sequence. Figure 8 illustrates the policy of UAV swarm radar beam synthesis and the communication UAV motion policy generated. According to the theory in Section 4, the ranging information is only related to SIR, so the beams tend to cover as many scattering echoes as possible. Simulations show that the beam pointing and mainlobe width fits the target area well. As the beam of jammer 2 scans from right to left, the communication UAV makes effective move to avoid jamming, while restricted its motion range to preserve energy as guided by the double greedy algorithm. Figure 9 illustrates the frequency policies adopted by the radar UAV swarms to avoid jamming. In order to better illustrate the effectiveness of DGQL, we introduce the information of the stochastic policy and the default policy as benchmarks. Stochastic policy refers to the random selection of all radar policies at each time. Default policy refers to a policy choice when the radar UAV swarm does not have any prior information about the target. A common default policy is that the radar keeps the beam scanning back and forth in space, keeps the transmission frequency and mainlobe width unchanged, and does not change the position of the communication UAV (always in the center). Figure 10 shows that DGQL generates significantly more information than stochastic policies and default policies.