# A Dyna-Q-Based Solution for UAV Networks Against Smart Jamming Attacks
## 5 Simulation Results
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Dyna-Q** & **Q-Learning** & **WolLF-PHC** & **Improved Dyna-Q** \\ \hline
1 & 2.89319 & 2.71596 & 2.89776 & 3.12693 \\
2 & 2.92946 & 2.68807 & 2.87904 & 3.04405 \\
3 & 2.85423 & 2.62658 & 2.86797 & 2.98408 \\
4 & 2.98676 & 2.67052 & 2.90120 & 2.98828 \\
5 & 2.86721 & 2.75111 & 2.94359 & 3.06887 \\
6 & 2.99363 & 2.75530 & 2.90655 & 3.03335 \\
7 & 2.99477 & 2.65255 & 2.84658 & 3.07880 \\
8 & 2.86605 & 2.75300 & 2.91342 & 3.03412 \\
9 & 2.98484 & 2.75873 & 2.89929 & 3.07651 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Average SINR values of the UAV network over nine rounds of smart jamming attack. Figure 7: Average performance of the SDN-based learning algorithms using power allocation and path replanning strategy against smart attacks with \(P_{T}=P_{J}=0.4\), \(C_{m}=0.8\), \(C_{p}=0.2\), and \(C_{h}=0.4\) under the condition that jammers change their jamming strategy, including changing jamming channels and changing the positions of jammers, nine times at equal intervals over 300 time slots. (**a**) shows the average utility of the UAV network and (**b**) shows the average SINR of the UAV network. Figure 8: Average performance of the SDN-based communication scheme with \(C_{h}\in[0.1,0.8]\), \(C_{m}\in[0.1,0.8]\), and \(C_{p}\in[0,1,5]\), respectively. It can be seen from the simulation results that each of the three cost parameters has a particular pattern. The pattern of the frequency hopping cost \(C_{h}\) is that when the cost increases, the other three algorithms decrease rapidly, except for the slow decline of the improved Dyna-Q. This is because when the frequency jump cost increases, path replanning becomes the main strategy to avoid jamming. At this time, the SDN-based improved Dyna-Q can use the whole network situation information to better avoid jamming. The characteristic of UAV path replanning cost \(C_{m}\) is that when it increases, the UAV will use path planning less to avoid jamming. At this time, the problem gradually turns into the traditional problem of using a power allocation method to avoid jamming. In this case, the advantages of improved Dyna-Q will gradually diminish. As unit transmission cost \(C_{p}\) grows, any form of communication cost in the network will increase, because any type of network defense strategy relies on packet transmission. Consistent with expectations, as unit transmission cost \(C_{p}\) grows, the utility declines in roughly the same proportion, regardless of the algorithm used. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \(C_{m}\) & **Dyna-Q** & **Q-Learning** & **WoLF-PHC** & **Improved Dyna-Q** \\ \hline
0.1 & 1.13170 & 0.97344 & 1.07771 & 1.15310 \\
0.2 & 1.04668 & 1.03824 & 1.04958 & 1.08062 \\
0.3 & 0.96030 & 0.89309 & 0.91283 & 1.04212 \\
0.4 & 0.95543 & 0.88420 & 0.89263 & 0.96616 \\
0.5 & 0.84655 & 0.82759 & 0.86404 & 0.91704 \\
0.6 & 0.83745 & 0.70395 & 0.76227 & 0.88063 \\
0.7 & 0.70258 & 0.72456 & 0.71178 & 0.82772 \\
0.8 & 0.72501 & 0.67328 & 0.74244 & 0.76028 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Average utility values with different UAV path replanning costs. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \(C_{p}\) & **Dyna-Q** & **Q-Learning** & **WoLF-PHC** & **Improved Dyna-Q** \\ \hline
0 & 1.08415 & 0.78196 & 1.04347 & 1.30321 \\
0.1 & 1.05738 & 0.77221 & 1.00889 & 1.23439 \\
0.2 & 1.03222 & 0.76924 & 0.92380 & 1.21956 \\
0.3 & 0.98544 & 0.71159 & 0