# Deep Reinforcement Learning for Energy-Efficient 6G V2X Networks
## 2 Related Work
### DRL-Based Approaches to Jointly Minimize Energy Consumption and Latency  
DRL-based algorithms are used to minimize the weighted sum of energy consumption and latency under various constraints. [PERSON] et al. [21] modeled a computation task offloading problem as a partially observable MDP to minimize energy consumption and latency in vehicular edge computing (VEC) networks. They introduced a subtask priority scheduling method based on a directed acyclic graph (DAG) structure to guarantee the priority order of subtasks. They also proposed a DRL-based optimized distributed computation offloading (ODCO) scheme to optimize resource allocation. They did not consider a complex real-world scenario for vehicular networks and focused only on minimizing the average latency and energy consumption of vehicles. [PERSON] et al. [24] formulated an optimization problem to minimize energy consumption and task execution latency in a UAV-assisted mobile edge computing (MEC) system considering the device offloading priority. They used DRL for dynamic trajectory planning supported by matching theory to solve scheduling and computing resource allocation.  
PPO and multi-agent PPO (MAPPO) algorithms are used to dynamically allocate resources. [PERSON] et al. [22] proposed a semantic-aware multitask offloading network with the objective of maximizing the performance of the task execution of the semantic offloadingsystem, leading to reduced task execution time and energy consumption. A MAPPO algorithm is used to perform resource allocation in a distributed manner with low computational complexity. [PERSON] et al. [26] introduced a new UAV-assisted task offloading framework in the MEC system and developed task dependency and priority models. They formulated an optimization problem to jointly minimize system latency and the energy consumption of computational tasks. They combined asynchronous federated learning (FL) and the PPO algorithm to solve the minimization problem. The PPO algorithm is used to determine dynamic offloading decisions, while an asynchronous federated learning mechanism is used to train models that ensure data privacy.  
DRL-based deep Q-learning network (DQN) algorithms are employed to solve the resource allocation. [PERSON] et al. [20] introduced a scheme for heterogeneous multiserver computation offloading (HMSCO) assisted by UAVs in the MEC network. They formulated a minimization problem to reduce the system cost, which consists of a weighted sum of latency and energy consumption under the constraints of reliability requirements, maximum tolerable latency, computational resource limits, minimum communication rate, transmission power range, and interference threshold. They decomposed the minimization problem into two subproblems and solved it using game theory and a multi-agent enhanced dueling double deep Q-network (ED3 QN) with centralized training and distributed execution.  
[PERSON] et al. [23] designed a task offloading strategy to minimize task execution latency and energy consumption in a device-edge-cloud architecture. They first introduced a serial particle swarm optimization (SPSO) algorithm to find an optimal connected MEC node for each task. Then they designed a prioritized experience replay-based double deep Q-learning network (PERDDQN) to jointly optimize offloading ratio and resource allocation in the task offloading process. [PERSON] et al. [25] formulated an optimization problem to minimize latency and energy costs considering the appropriate time scales in the vehicle network enabled by VEC. They proposed a DQN-based multi-time scale MDP algorithm to jointly solve the service placement, network selection, and offloading problem.