# PPO-Based Joint Optimization for UAV-Assisted Edge Computing Networks
## 2 Related Work
### Motivation and Contribution  
In the context of next-generation mobile communication scenarios, an increasing number of UEs and ECSs are being connected to the network. At the same time, users have higher demands for their service experience. However, current research on edge computing lacks consideration of the decision-making of multiple UEs and ECSs that cannot meet the increasing demands of wireless network traffic. Additionally, existing works on UAVs in edge computing lack comprehensiveness, and further research is needed on the joint deployment of UAVs and edge computing decision-making. Furthermore, the emergence of deep reinforcement learning provides a new research direction for UAV-assisted edge computing. Motivated by the above observations, to improve the flexibility and timeliness of UAV-aided edge computing, this work designs a UAV-assisted edge computing network and further proposes an optimization problem for joint UAV deployment and edge computing decision-making. Additionally, in order to address the mixed variable optimization that exists in current UAV-assisted mobile edge computing networks, a deep reinforcement learning algorithm based on Proximal Policy Optimization (PPO) is proposed to solve the optimization problem. Specifically, our contributions are as follows:  
* This work designs a UAV-assisted edge computing network in which UAVs act as relay nodes to forward edge computing tasks when the wireless channel between UEs and ECSs is degraded. The devised network can proficiently tackle the issues of restricted wireless links and poor flexibility present in stationary ECS deployment settings, consequently encouraging the employment and evolution of mobile edge computing via UAVs.
* To meet the requirements of service experience, this work aims to minimize the average delay and designs an optimization problem for joint three-dimensional UAV deployment and scheduling decision-making of UEs and ECSs in edge computing. The model addresses the connection between decisions regarding the deployment and unloading of UAVs, effectively mitigating potential network performance issues arising from the interaction of multiple variables.
* This work proposes a joint optimization algorithm based on PPO within the framework of Deep Reinforcement Learning (DRL) to solve the complex optimization problem in the UAV-assisted edge computing network. The method proposed has the ability to handle situations in which discrete and continuous variables are present and is in line with UAV-aided edge computing networks. This is unlike traditional algorithms or existing DRL algorithms.
* Experimental results validate the effectiveness of the proposed PPO-based joint optimization algorithm. Compared to baseline algorithms, the PPO-based joint optimization algorithm achieves a lower average delay and better meets the requirements of user service experience.  
The rest of the paper is organized as follows. In Section 3, the considered system model and joint optimization problem are established. In Sections 4 and 5, the PPO-based joint optimization algorithm is designed, and the performance is verified by numerical results, respectively. Finally, Section 6 concludes the paper.