# Energy Saving Strategy of UAV in MEC Based on Deep Reinforcement Learning
## 6 Simulation Results
### The Effects of Different Parameters  
In this section, we study the effects of different parameters on the performance of the DOTO algorithm, including the training interval, experience replay pool and training sample.  
In Table 1, we show the performance of DOTO under different training intervals \(\delta\). The DOTO algorithm converges faster with shorter training intervals, and thus more frequent policy updates. However, it is not necessary to update the policy frequently. The results in Table 1 also prove this point. Nonetheless, the training interval also should not be too large. This will cause the algorithm not to converge. Since the DOTO algorithm no longer converges when the training interval \(\delta\geq 40\), the data results are meaningless and are thus not displayed. After comprehensively considering the algorithm results and convergence performance, this paper sets the training interval \(\delta=10\).  
In Tables 2 and 3, we show the performance of DOTO under different experience replay pool sizes and training sample sizes. First, when the training sample size is 64, 128 and 256, we use different experience replay pool sizes to observe the performance of the algorithm. Then, after selecting a suitable size of the experience replay pool, we find a suitable training sample size according to the size of the experience replay pool. It can be seen from Table 2 that when the experience pool is small, the convergence of the DOTO algorithm fluctuates greatly. When the experience pool is larger, the convergence is slower. After comprehensively considering the algorithm results and convergence performance, we finally set the experience replay pool size to 1024.  
Figure 7: The corresponding calculation time and residual energy when adaptive \(K\) adopts different update intervals \(\Delta\). Here, we set \(N=8\).  
As shown in Table 3, when the training samples are too small, since DOTO cannot effectively utilize all the data in the experience pool, the convergence performance of the algorithm is reduced. When the number of samples is too large, old training data are often used, resulting in poor convergence performance. After comprehensively considering the algorithm results and convergence performance, we finally set the training sample size to 128.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Training Interval**} & \multirow{2}{*}{
\begin{tabular}{c} **Ratio of Residual Energy to** \\ **Optimal Decision** \\ \end{tabular} } & \multirow{2}{*}{**Convergence Performance**} \\ \hline
2 & 0.9985 & convergence fluctuates greatly \\ \hline
5 & 0.9990 & convergence fluctuates greatly \\ \hline
10 & 0.9991 & convergence \\ \hline
20 & 0.9990 & convergence \\ \hline
40 & & non-convergence \\ \hline
80 & & non-convergence \\ \hline \hline \end{tabular}
\end{table}
Table 1: The effect of training interval \(\delta\) on algorithm performance. Here, we set \(N=8\).  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Experience Replay**} & \multirow{2}{*}{\begin{tabular}{c} **Training** \\ **Sample** \\ \end{tabular} } & \
\begin{tabular}{c} **Ratio of Residual** \\ **Energy to Optimal** \\ **Decision** \\ \end{tabular} &
\begin{tabular}{c} **Convergence** \\ **Performance** \\ \end{tabular} \\ \hline
64 & 64 & 0.9987 &
\begin{tabular}{c} convergence \\ fluctuates greatly \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table}
Table 2: The effect of experience replay pool size on algorithm performance. Here, we set \(N=8\).