# Toward Collaborative Intelligence in IoV Systems: Recent Advances and Open Issues
## 5 Collaborative Learning Technologies
### Reinforcement Learning
However, in order to achieve a trust evaluation based on the collaboration of a large portion of vehicles, an efficient incentive mechanism is required. Therefore, the authors propose a RL-based incentive mechanism to involve the participation of more vehicles. They use computer simulations to show that the mechanism can achieve a higher detection ratio than the conventional approach. [PERSON] et al. [163] discuss the user association problem for small cell dense VNs, where a vehicle could possibly connect to multiple small cell base stations. The mobility of vehicles result in handovers between different base stations, which makes the user association policy--specifically, which base station to connect with at a particular time--very important. In Ref. [163], the authors propose a signal-quality-aware user association algorithm based on Q-learning, a form of RL. The algorithm is able to make a tradeoff between the number of handovers and system performance while guaranteeing the QoS requirements. [PERSON] et al. [164] discuss the radio resource allocation at base stations in VNs. They define a Q-learning model where the state is the uplink and downlink data rate against aviate channel resource. The action is the ratio between the resource allocated for uplink and downlink data transmission. The reward is defined based on multiple network status parameters. By using the Q-learning algorithm, Ref. [164] is able to consider the future reward for a resource allocation policy, which can provide a solution with long-term efficiency. [PERSON] et al. [165] propose a RSU-assisted routing protocol for VANETs based on Q-learning. They use a three-phase routing algorithm. In the first phase, a packet is forwarded from the source node to a nearby RSU. In the second phase, the packet is forwarded between RSUs. In the last phase, the packet is delivered to the destination from a RSU to the destination. Hello messages are used to exchange information among neighbors and update Q-table. [PERSON] et al. [166] consider a scenario in which UAVs act as relay nodes to improve the coverage of cellular infrastructure. A reinforcement-learning-based approach is proposed to optimize the trajectories of multiple UAVs while minimizing the number of deployed UAVs and energy consumption. The learning is conducted at the central unit that interacts with other network entities to update the model. Numerical results are used to demonstrate the advantage of the RL-based approach over existing baselines. [PERSON] et al. [167] discuss the communication gateway selection problem in a multi-access vehicular environment where multiple types of radio access technologies, including Wi-Fi and cellular communication, coexist. They propose a Q-learning-based approach to support each vehicle to select the best gateway nodes among a cellular base station and multiple RSUs where RSUs are connected with vehicles through a Wi-Fi interface. In the learning model, the state is defined by the distribution of vehicles, and the action is the selection of the gateway. While each vehicle is a learning agent, the real learning process is conducted a centralized server, and the software defined network (SDN) technology is used to achieve centralized control of the entire vehicular network. There are some studies which discuss the use of reinforcement learning in improving vehicle path planning. [PERSON] et al. [168] propose a Q-learning algorithm to minimize the delay in transportation. In the Q-learning algorithm, the action is the driving direction, and Q-value represents the probability of satisfying the deadline constraint in transportation. They use both artificial transportation networks and real road networks in Beijing, Munich, and Singapore to evaluate the efficiency of the proposed algorithm. [PERSON] et al. [169] propose a RL-based scheme for route planning for hybrid electric vehicles for the purpose of finding the route with minimum energy consumption. The RL model is defined as follows.