# Deep Reinforcement Learning for Energy-Efficient 6G V2X Networks
## 2 Related Work
### DRL-Based Approaches to Minimize Energy Consumption  
Several studies proposed DRL-based algorithms to solve the resource allocation problem of minimizing the energy consumption. [PERSON] et al. [13] proposed a joint optimization problem to minimize the cost of age-of-information (AoI), energy consumption, and rental price for computation offloading in unmanned aerial vehicle (UAV)-assisted vehicular edge computing networks. They developed a solution using a DRL-based joint trajectory control and offloading allocation algorithm (DRL-TCOA) to solve the proposed computation of \begin{table}
\begin{tabular}{p{56.9 pt} p{113.8 pt} p{113.8 pt}} \hline
**Lit.** & **Objectives** & **Constraints** & **Solutions** \\ \hline \multirow{2}{*}{[11]} & & Resources & Optimal ILP and heuristic algorithms \\ \cline{2-3}  & & Latency and energy & DRL-TCOA \\ \hline \multirow{2}{*}{[14]} & & Energy and resources & PPO-based \\ \cline{2-3}  & & Energy & Convex optimization, DDQN, and DDPG \\ \cline{2-3}  & & Latency and resources & SCA and MADDPG \\ \hline \multirow{2}{*}{[17]} & & Latency and resources & Double-iteration joint optimization \\ \cline{2-3}  & & Transmission rate & DDPG-based \\ \hline \multirow{2}{*}{[19]} & & Latency & TD3 \\ \hline \multirow{2}{*}{[20]} & & Latency and resources & Game theory and ED3 QN \\ \hline \multirow{2}{*}{[21]} & & Resources & DAG-based and DRL-based ODCO \\ \hline \multirow{2}{*}{[22]} & & Latency and resources & MAPPO \\ \hline \multirow{2}{*}{[23]} & & Latency and resources & SPSO and PERDDQN \\ \cline{2-3}  & & Latency and resources & DRL and matching theory \\ \hline \multirow{2}{*}{[25]} & & Latency, energy, and resources & DQN-based \\ \hline \multirow{2}{*}{[26]} & & Latency and energy & PPO and asynchronous FL \\ \hline \multirow{2}{*}{[27]} & & Latency and resources & Distributed collaborative computing \\ \hline \multirow{2}{*}{[28]} & & Resources & GA and PSO \\ \hline \multirow{2}{*}{[29]} & & Energy & Decentralized and probability-based \\ \cline{2-3}  & & Latency & Lagrange multipliers \\ \hline \multirow{2}{*}{[31]} & & Resources & MADDPG \\ \hline \multirow{2}{*}{[32]} & & Latency and resources & Clustering algorithm and DDQN \\ \cline{2-3}  & & Throughput & DDPG and DQN \\ \hline \multirow{2}{*}{[34]} & & Latency and reliability & DDQN \\ \hline \multirow{2}{*}{[35]} & & Energy and resources & DMARL \\ \hline \multirow{2}{*}{[36]} & & URLLC & SNC and Lyapunov-guided DRL \\ \hline \multirow{2}{*}{[37]} & & Energy and resources & A2C-based \\ \hline \multirow{2}{*}{[38]} & & Latency and energy & Lyapunov optimization and JCORA \\ \hline \multirow{2}{*}{[39]} & & Latency & SA and TD3 \\ \hline \multirow{2}{*}{Ours} & & Latency and resources & Optimal ILP and PPO \\ \cline{2-3}  & & & \\ \end{tabular}
\end{table} 
Table 1: Comparison of research publications. floading problem. [PERSON] et al. [14] introduced energy harvesting (EH) into a vehicular ad hoc network (VANET) and suggested a decentralized multi-agent DRL-based resource allocation algorithm to maximize energy efficiency. They composed power splitting and DRL to efficiently divide the harvested energy. They formulated the non-convex optimization problem as a Markov decision process (MDP) and used a PPO-based algorithm to solve it. [PERSON] et al. [17] designed a distributed edge computing-based task offloading scheme in a vehicular network. They proposed a joint optimization problem for the offloading strategy and resource allocation. They developed a double-iteration joint optimization algorithm to minimize energy consumption by optimizing the offloading strategy and bandwidth allocation. Deep deterministic policy gradient (DDPG) and twin delayed DDPG (TD3) algorithms are employed to solve the resource scheduling problem and reduce energy consumption. [PERSON] et al. [15] studied an intelligent reflecting surface (IRS)-enabled UAV edge computing to improve energy efficiency and formulated a joint optimization problem of IRS phase shift, UAV trajectory, and power allocation.