# Reinforcement Learning-Based Anti-Jamming in Networked UAV Radar Systems
## 4 Algorithm Design
### Information Representation Reward Design  
The calculation of information quantity is based on SIR, which is related to the system topology and parameter setting. The SIR \(\eta\) received by the communication UAV is:  
\[\eta=\frac{[(\sum\limits_{i=1}^{5}P_{R}G_{Ri}^{2}\sigma_{i}-L_{R1})F_{R}-L_{R2 }]+[(\sum\limits_{j=1}^{5}P_{r}G_{rj}^{2}\sigma_{j}-L_{r1})F_{r}-L_{r2}]}{(P_{ j1}G_{j1}-L_{j1})+(P_{j2}G_{j2}-L_{j2})} \tag{9}\]  
Under different networking modes, the mapping relationship between SIR \(\eta\) and information quantity has different forms. In many cases, the information quantity obtained by the radar is not only  
Figure 5: Simulation result of 2-D beam synthesis with low-bit coding. (**a**) Coding scheme (**b**) 2-D radiation pattern. determined by SIR. Detailed theory will be introduced by the radar information representation method as follows. Target detection can be regarded as a process of information acquisition. The amount of information directly affects the accuracy of target parameter estimation. In statistical signal processing theory, Cramer-Rao lower bound (CRLB) is the most commonly used parameter to measure the accuracy of parameter estimation. CRLB is related to Fisher information in data determined by unknown parameters. It is assumed that the relationship between radar measurements \(\mathbf{x}\) and the target parameter vector \(\mathbf{\theta}\) is as follows:  
\[\mathbf{x}=h(\mathbf{\theta})+\mathbf{n} \tag{10}\]  
where \(h\) and \(\mathbf{n}\) denote the measurement function and the measurement noise, respectively. Then its Fisher information matrix \(\mathbf{I}(\mathbf{\theta})\) is defined as:  
\[\left[\mathbf{I}(\mathbf{\theta})\right]_{ij}=-E\bigg{[}\frac{\partial^{2}\ln p(y\big{|} \mathbf{\theta})}{\partial x_{i}\partial x_{j}}\bigg{]} \tag{11}\]  
where \(E\) represents the mathematical expectation of a random variable. For radar detection, since the LFM signal occupies a much wider bandwidth than information bandwidth, it can also obtain a large processing gain. Thus, LFM signal is the most commonly used signal form in radar detection. Taking LFM signal as an example, this paper analyses its information acquisition ability in \"range-Doppler\" measurement. LFM baseband signal has two main parameters: pulse width \(T\) and frequency modulation rate \(K\). The time-domain signal can be written as follows:  
\[s(t)=\frac{1}{T}\text{rec}\bigg{(}\frac{2t}{T}\bigg{)}\exp\bigg{(}j2\pi Kt^{2} \bigg{)} \tag{12}\]  
where rect is a rectangular window function defined as:  
\[\text{rect}\bigg{(}\frac{t}{T_{p}}\bigg{)}= \begin{cases}1&-T_{p}/2\leq t\leq T _{p}/2\\ 0&t<-T_{p}/2\quad\text{or}\quad t>T_{p}/2\end{cases}  \tag{13}\]  
In general, ranging and measuring Doppler shifts are performed simultaneously. Therefore, we consider the Fisher information under the \"range-Doppler\" measurement first. According to [29; 30], assuming that the delay of the echo signal is \(\tau\) and the Doppler shift is \(v\), then the ambiguity function of the signal is represented by \(A(\tau,v)\). The Fisher information matrix of the \"range-Doppler\" estimation is:  
\[I=\eta\left[ \begin{array}{c}-\frac{\partial^{2}A(\tau,v)}{\partial\tau^{2}} \bigg{|}_{\tau=0,v=0}&-\frac{\partial^{2}A(\tau,v)}{\partial\tau\partial v} \bigg{|}_{\tau=0,v=0}\\ -\frac{\partial^{2}A(\tau,v)}{\partial\tau\partial\tau}\bigg{|}_{\tau=0,v=0}& -\frac{\partial^{2}A(\tau,v)}{\partial b^{2}}\bigg{|}_{\tau=0,v=0}\end{array} \right] \tag{14}\]  
If the SIR of the echo is assumed to be \(\eta\), \(\mathbf{\theta}=\left[T,K\right]^{T}\), then the Fisher information matrix is:  
\[\mathbf{I}(\mathbf{\theta})=\eta\left[ \begin{array}{cc}\frac{1}{2T^{2}}+8\pi^{2}K^{ 2}T^{2}&2\pi KT^{2}\\ 2\pi KT^{2}&\frac{T^{2}}{2}\end{array} \right] \tag{15}\]  
Finding the determinant of the Fisher information matrix, we can get the total amount of information generated by the LFM waveform for \"range-Doppler\" measurement.