## 2 Related Works  
Several optimization methods have been studied to improve system energy efficiency and QoS in vehicular networks [7]. In ref. [8], a context based scheduling scheme was proposed that uses the geographical location of vehicles and the interference level of shared resources. The distribution of interference is obtained through resource allocation and the reduction of packet collisions ensures stable management of resources. The cluster method classifies vehicles into different groups to facilitate dissemination of messages to other vehicles, as a typical method of vehicular communication. In ref. [9], a low-cost cluster-based communication method, which includes data estimation, was developed to improve the QoS of driver assistance with crash warning application. The intra-cluster communication method was applied for data estimation to reduce the additional cost of inter-cluster communication. It is difficult to ensure QoS requirements of the various existing algorithms that exploit traditional optimization methods. Machine learning has shown promise in addressing decision-making problems involving uncertain network communications [10]. The method finds optimal or close to optimal solutions in the uncertain and inherently non-stationary environment of vehicular networks. In [11], a reinforcement learning scheme was developed to fulfill the diverse requirements of different entities with stringent QoS requirements. This algorithm can minimize the overhead of resource provisioning for vehicular clouds. In [12], an online reinforcement learning based user association algorithm for network balancing was proposed to cope with dynamic changes using historical-based reinforcement learning. In [13], a two-level cluster-based routing method was developed for efficient data dissemination by adjusting two cluster methods, level-1 cluster heads and level-2 cluster heads. The level-1 cluster heads were selected by applying fuzzy logic to relative velocity, k-connectivity and link reliability factors. The level-2 cluster heads were selected by Q-learning to reduce communication overhead. In vehicular networks with multiple sensing components and realistic channel gain, high mobility generates large-scale continuous state space, which makes Q-learning inefficient. DRL, which uses deep neural networks (DNN) to represent the Q-table, can be modified to solve resource allocation problem in vehicular networks such that the continuous state is a direct input to the DNN [14]. In [15], the deep deterministic policy gradient (DDPG) based resource allocation scheme was proposed to maximize the sum rate of V2I communications, where the resource allocation problem is formulated as a decentralized discrete-time and finite-state Markov decision process (MDP) with incomplete network information. DDPG is capable of handling continuous high-dimensional action spaces while determining the optimal resource allocation strategy. In [16], a decentralized resource allocation mechanism based on DRL was proposed to maximize the achievable data rate and minimize the interference of V2I communications. DRL can be modified to both unicast and broadcast scenarios with a V2V vehicle as an agent. The agent decides on the optimal sub-band and transmission power levels. In [17], a DRL based decentralized mode selection and resource allocation algorithm was proposed to maximize the sum capacity while meeting the latency and reliability requirements. The mode selection and resource allocation problems were formulated as a Markov decision process to select the optimal transmission mode, sub-band and transmission power level. To address the limitations of local DRL models, a two-timescale federated DRL and graph theory based vehicle clustering method was developed. High mobility in vehicular networks results in frequent handover and channel state changes. The aforementioned studies consider neither realistic vehicle mobility nor changing channel states.