## 2 Related Work  
There have been more studies on the allocation of wireless resources for vehicular communication. The existing spectrum resource sharing schemes are mainly divided into two categories: traditional methods [12; 13; 14] and machine learning methods [15; 16; 17]. In conventional methods, under different constraints such as transmit power, quality of service, mathematical programming is used to optimize design goals of interest, such as maximizing spectrum efficiency or minimizing interference. Under normal circumstances, resource allocation problems are usually non-convex or mixed-integer nonlinear programming. Due to its NP-hardness, it is difficult to solve using effective methods. In order to deal with such problems, heuristic methods are usually used or sub-optimal solutions are pursued. In addition, the solution is highly dependent on the model. If you consider different objective functions or add new constraints, you need to develop a new solution [18]. Therefore, these many reasons have promoted the exploration of new resource allocation schemes. In recent years, machine learning has become a potential implementation technology for future wireless communications. Reinforcement learning (RL) in machine learning is widely used in wireless communication resource allocation to solve the challenges in traditional optimization methods. In a high-speed moving vehicular network, RL can perceive changes through interaction with the environment and make corresponding decisions. In addition, the goal problems and constraints that are difficult to optimize can be solved by designing corresponding training rewards [19]. Q-learning is a typical RL algorithm, which avoids the difficulty of acquiring dynamics by adopting the iterative update method. However, since Q-learning stores the state in the form of a table, it is not suitable for application in scenarios with too large a state space. By combining deep neural network (DNN) with Q-learning, deep Q-learning (DQN) is designed to handle larger state spaces. In [20], the problem of vehicular network spectrum sharing based on multi-agent RL was studied, resource sharing was modeled as a multi-agent RL problem, and then the fingerprint-based DQN method was used to solve it. In [21], the spectrum allocation problem in the vehicular network was studied, and the graph neural network (GNN) and DQN learning were used to optimize the sum capacity of the vehicular network. In [22], the problem of shared resource allocation between V2I and V2V links is studied. Under the deep reinforcement learning (DRL) framework, the existing resource allocation methods can be divided into modes 3 and 4. In the mode 3 scheme [23; 24; 25], the BS acts as a central controller to allocate resources for each user. In [23], a joint DRL algorithm was proposed, which can effectively deal with the discrete-continuous mixed action space in wireless communication systems and solve the power and sub-channel allocation problems of non-orthogonal multiple access systems. In [24], a resource allocation method based on graphs and DRL is proposed, in which channel resources are centrally allocated by BS, and the vehicle user uses DRL for distributed power control. In [25], the joint channel allocation and power allocation problem was transformed into an optimization problem, and a DRL framework based on an attention neural network was proposed to solve the problem. In the above-mentioned schemes, the BS needs to know the CSI of all users, that is, V2V user's communication channel gain, the interference link information, and the user's transmit power to determine the spectrum sharing decision. Feeding this information back to the BS will result in more significant overhead, especially when there are many users. This limits the application of the centralized solution in practice. In the mode 4 resource scheme [26; 27; 28], the users independently select the transmission spectrum without the participation of the BS, which reduces the signaling overhead of the BS.