# Deep Reinforcement Learning for Energy-Efficient 6G V2X Networks
## 2 Related Work
### DRL-Based Approaches to Minimize Latency  
DRL-based algorithms are used to solve the resource allocation problem and minimize latency. [PERSON] et al. [39] presented a novel ratio-based offloading approach in a two-tier V2X network that covers both vertical and horizontal offloading. They used metaheuristic simulated annealing (SA) and TD3 algorithms to minimize system latency by optimizing offload ratios. [PERSON] et al. [31] proposed a scheme in which each vehicle can select an appropriate RSU to offload its application. The computed application is delivered to the vehicle after being computed with the cooperation of the RSUs. A joint optimization problem is formulated to minimize latency and maximize reliability by determining the amount of uploading, computing, and result downloading. They transformed the non-convex weighted sum problem into an MDP and used the MADDPG algorithm to minimize the termination time and the probable delivery failure penalty. [PERSON] et al. [35] formulated a base station (BS) deployment optimization problem to maximize the communication coverage ratio of vehicular networks under the constraints of moving velocity, energy consumption, and communication coverage radius. They modeled the prolonged decision-making process with a dual-layered decision loop in a multi-UAV vehicular network scenario. To solve the optimization problem, they developed a dense multi-agent reinforcement learning (DMARL) algorithm based on a dual-layer nested decision-making structure, decentralized deployment, and centralized training. [PERSON] et al. [36] formulated a resource allocation problem as an optimization problem to minimize the utility of the system under the constraint of ultra-reliable low-latency communications (URLLC). They considered a heterogeneous VEC with various communication technologies and multiple types of tasks. They modeled the URLLC constraint using the stochastic network calculus (SNC) and then introduced a Lyapunov-guided DRL approach to convert and solve the optimization problem. Their proposed method requires retraining the model as the types of tasks continue to grow. DQN and advantage actor-critic (A2C)-based algorithms are used to determine the offloading decision and solve the resource allocation problem. [PERSON] et al. [32] proposed a collaborative Internet of vehicles (IoV) architecture with integrated software-defined networking (SDN) based on cloud-edge clusters. They introduced a dynamic pricing model to deal with the complexity of the resource management problem and formulated a mathematical model with the objective of minimizing computation latency and maximizing the profits of edge service providers (ESPs). A clustering algorithm is used to determine offloading decisions and reduce the action space, while the DDQN algorithm is applied to achieve the optimal strategy for allocating computational resources. [PERSON] et al. [33] introduced a joint BS assignment and resource allocation (JBSRA) framework for mobile V2X users considering co-existence schemes based on the flexible duty cycle (DC) mechanism for unlicensed bands. They formulated an optimization problem to maximize the average throughput of the V2X network while satisfying the throughput of WiFi users and used the DDPG and DQN algorithms to solve the resource allocation problem. [PERSON] et al. [34] investigated a simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS)-assisted V2X network and formulated an optimization problem to maximize the achievable data rate for V2I users while guaranteeing latency and the reliability of V2V pairs. The MINLP is decomposed into two subproblems and solved using DDQN with an attention mechanism and implementing a standard optimization-based approach. [PERSON] et al. [37] formulated a joint task offloading and resource management problem to maximize computation efficiency in a MEC-enabled heterogeneous network.