# PPO-Based Joint Optimization for UAV-Assisted Edge Computing Networks
## 5 Simulation Results  
In this work, we simulate and validate the designed UAV-assisted edge computing network. We consider the presence of \(K=9\) UEs and \(M=3\) ECSs in a square cell of length 400 m. The UAV hovers over the cell and is at an altitude between 40 m and 200 m. The carrier frequency is set to \(f_{c}=1000\) MHz, the bandwidth is 1 MHz, and the noise power spectral density is \(-174\) dBm\(/\)Hz. The air-to-ground channel is set with reference to the commonly used environment, i.e., \(a=9.61\), \(b=0.16\), \(\rho_{\text{Los}}=1\), and \(\rho_{\text{NLos}}=2\). The UEs and ECSs are generated using a hybrid process of uniform randomization plus Gaussian randomization. This is because, if the users were completely uniformly distributed in the region, it would make the planar location deployment of the UAVs not optimized. The PPO-related parameter settings are shown in Table 1.  
We plot the change in reward values during training of the PPO-based joint optimization algorithm in Figure 2. The vertical coordinate is the cumulative reward value per 200 time steps, and the horizontal coordinate is the time step. It can be seen that all three curves increase gradually with the increase in time steps and stabilize at three million time steps. The above changes can effectively illustrate that the proposed PPO-based joint optimization algorithm has good convergence properties. Meanwhile, the trend of the reward increasing with time steps also verifies the effectiveness of the proposed algorithm in UAV-assisted edge computing networks. The 3 curves in the figure correspond to varying transmission power levels, where training processes 1, 2, and 3 correspond to transmit power levels of 1.5 W, 1 W, and 0.5 W. As the figure illustrates, the reward value initially increases with the increase in power, implying that increasing power can achieve better system performance to some extent. Figure 3 plots the average delay versus transmit power for the UAV-assisted edge computing network, where PBJO stands for the proposed PPO-based joint optimization algorithm. Baseline 1 is the greedy scheme, in which the UEs always choose the ECS closest to the UAV, and the UAV hovers in the center of the cell. Baseline 2 is the stochastic scheme, in which the UEs are randomly matched with the ECSs, and the UAV randomly chooses the hovering location. The figure shows that, as transmit power increases, the average delay typically decreases. This is because higher transmit power facilitates increased communication rates, which ultimately reduces transmission delays. Notably, the effect of transmit power on the average delay in Baseline 1 is minimal. In this scheme, all UEs select the ECS closest to the UAV for matching. This results in an average delay that is \begin{table}
\begin{tabular}{c c} \hline \hline
**Simulation Parameter** & **Value** \\ \hline Steps per epoch & 200 \\ Gamma & 0.99 \\ Batch size & 64 \\ Clip range & 0.2 \\ Learning rate & 0.0003 \\ Number of environments & 8 \\ Steps per update & 2048 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Parameter settings. Figure 2: Training curve. dominated by waiting time. However, it has been observed that the proposed scheme consistently achieves a lower average delay, which implies a superior edge computing service experience. For instance, at a transmission power of 1 Watt, the PBJO scheme achieves an average delay of approximately 0.88 s, which is 0.12 and 0.3 s lower than Baseline 1 and Baseline 2, respectively. Although the waiting delay in Baseline 1 is on the high side, its average delay is still lower than that of Baseline 2, which shows the necessity of optimizing the three hovering positions of the UAV. Figure 4 plots the variation in the average delay of the UAV-assisted edge computing network versus the computational ability of the ECS. The average latency consistently decreases with increased computational power, aligning with our expectations.