# Toward Collaborative Intelligence in IoV Systems: Recent Advances and Open Issues
## 5 Collaborative Learning Technologies
### Reinforcement Learning  
In order to achieve a more accurate understanding of the complex vehicular environments, a vehicle has to utilize the information received from other vehicles in the vicinity. However, the vehicle has to ensure that whether the messages received from other vehicles are trustworthy or not. If the messages include fake or misleading information, the perception accuracy degrades drastically, possibly resulting in a fatal error. Therefore, the trust management in vehicular environments is an important consideration for achieving a good decision. [PERSON] et al. [157] propose a context-aware trust management model in VANETs. The context includes internal information, specifically the information collected from the local sensor (the sensor devices installed on the current vehicle), and external information received from other vehicles. A Q-learning-based approach is proposed to set the weights of the internal and external information in making the final decision. The information entropy theory is used in the trust calculation. Simulation results show that the model can achieve a higher precision for trust evaluation with reasonable commotional and communication overhead. Multi-agent reinforcement learning (MARL) is widely used in vehicular IoT to enable an efficient system in a distributed manner. However, the coordination among vehicles in a dynamic environment is particularly challenging. [PERSON] et al. [158] formulate the coordination problem by using a dynamic coordination graph. They propose coordination-graph-based MARL approaches to optimize the joint maneuvers of multiple vehicles. Experimental results are presented to show the performance of the proposed approaches in the decision making surrounding following or overtaking in highway situations to enable high-level strategic control of vehicles. [PERSON] et al. [159] consider the decentralized control of multiple robots, such as UAVs, for patrolling in partially observable environments with unknown prior knowledge. For example, for a post-disaster scenario, while an UAV can patrol continuously over the disaster area and collect information by coordinating with other UAVs, the decentralized control of multiple UAVs is challenging due to the environment, which is unknown and only partially observable for each UAV. The authors formulate the problem as a Bayes-adaptive transition-decoupled partially observable Markov decision process, and propose a Bayesian RL algorithm to improve the coordination and planning process of multiple robots. The experimental outcomes indicate that the proposed algorithm performs better than the existing methods. It is important to consider stability and efficiency in the design of a RL algorithm, especially for IoV applications, which require ultra-high reliability. [PERSON] et al. [160] propose a deterministic promotion RL algorithm approach that improves policy evaluation in critique and exploration in action. The action exploration in the proposed algorithm is conducted by a normalization-based evaluation approach in a way that can increase efficiency and decrease the dependencies among the explored actions. They use both computer simulations and real-world experiments to show the advantage of the proposed algorithm in a longitudinal velocity control scenario for vehicular environments. In Ref. [161], [PERSON] and [PERSON] discuss the jamming problem in vehicular networks. They investigate anti-jamming V2V communications based on joint control of power and channel selection. The problem is formulated as a multi-armed bandit problem, and a RL-based approach is tailored to solve the problem. Extensive simulations are conducted to show the performance of the approach in terms of multiple performance metrics, including power strategy, channel selection, and throughput. [PERSON] et al. [162] discuss the importance of trust evaluation in intrusion detection systems. They argue that the trust evaluation of vehicles should be conducted not only by RSUs, but also by the vehicles in the vicinity.