# Cooperative Transmission Mechanism Based on Revenue Learning for Vehicular Networks
## 4 Collaborative Transmission Mechanism Based on Q-Learning for Vehicular Networks
### Multi-Point Collaborative Transmission with Revenue Learning  
Multi-point cooperative transmission is a communication operation mechanism that uses other neighboring terminals to send signals to another terminal to improve the signal to noise ratio of the target terminal. It was introduced earlier and will not be repeated here. Under the joint anti-interference modulation, some vehicular network users will be idle at some moments. Obviously, if these idle vehicular network users can be used to cooperate with the surrounding vehicular network users for communication, the overall performance of the vehicular network system will be further improved. However, since the vehicular network of the multi-point collaborative transmission mechanism needs to cooperate with the joint interference coordination mechanism, we use idle vehicular network users in the process of scheduling with the traditional arithmetic based on the utility function optimization, resulting in too many variables and a cumbersome process of calculation. Moreover, in the large-scale network, this will lead to computational resources and cost too much. Therefore, the method of revenue self-learning [16] is adopted in this paper to conduct scheduling among vehicular network users; that is, the vehicular network users can choose the optimal relay point for cooperation service through the revenue learning mechanism. This paper proposes a reinforcement learning algorithm of Q-learning, which is combined with the vehicular network of the joint anti-interference coordination algorithm. The vehicular network can continuously carry out self-learning and obtain profits to schedule cooperative multi-point transmission services and make full use of idle base stations to improve the overall performance of the system. When using the reinforcement learning algorithm of Q-learning, we should fully understand what the states, actions, and rewards are, and fully understand the \(Q\) matrix and payoff matrix of the learning scheduling strategy. Among them, the state represents the current connection status between each vehicular network user. The action represents the vehicular network user who will be taken to the next step to connect the user which has temporarily stopped communication by the vehicular network. That is, they cooperate to serve a vehicular network user. The reward is about the communication rate of the system communication. The \(Q\) matrix is constructed by learning the scheduling strategy. The vehicular network user at a certain time slot may be communicating with the base station or not communicating temporarily. They can take actions such as connecting with the cell base station, stopping the communication and not cooperating to serve other users, cooperating to serve other users 1, cooperating to serve other users 2, etc. The state is the permutation and combination of the connection running in the current cell cluster. For each base station, its \(Q\) matrix is a matrix of size (number of users in the cell cluster +1) * Number of users in the cell cluster). For a cell cluster, the \(Q\) matrix of a base station is shown in Table 1:  
In the joint interference coordination mechanism, the optimization objective is the logarithm sum of the effective rates of each user. So, we use the logarithm sum of the effective rates of all users in the cell cluster as the value of the payoff matrix to keep consistent with the algorithm in the joint interference coordination mechanism. Considering the practical problems in statistical calculation, we use the logarithm sum of the cumulative average rate of each user in the cell cluster to construct the payoff matrix and \(Q\) matrix; that is, we use \(\sum_{i=1}^{m}ln_{i}^{\textbf{f}^{\textbf{*}}_{i}\textbf{0}}(R_{i})\) as the feedback value of the payoff matrix, where \(R_{i}\) is the communication rate of user \(i\) in the cell cluster. The specific algorithm calculation process is shown in Algorithm 5.