## 2 IoV Distributed Intelligence
### Overview of Distributed Learning Methods
Devices that cannot connect directly to the central node can interact with adjacent vehicles. In this paradigm, each device can be connected to its nearest vehicle. This learning method is also trained iteratively. First, each device sends its trained local FL model to its connected devices or to the central node. The central node then produces the global FL model and sends it to the corresponding devices. Finally, each device changes its local FL model depending on the FL parameters received from other devices or the BS. In FL, each device may train its local FL model using gradient descent (GD) techniques, while the BS aggregates the local FL models. In CFL, however, each device must both aggregate the local FL models received from other devices and train its own local FL model. With the presence of high-quality computation hardware such as multi-core Central Processing Units (CPUs), Graphical Processing Units (GPUs), and Tensor Processing Units (TPUs), vehicular nodes can themselves train the learning models without the need for parameter servers. In some cases, with reduced mobility, and through V2X technology, vehicular nodes can collaborate to solve learning tasks. Such a CFL approach can be highly efficient in terms of training. Without the presence of a third party in the learning process, this can also further strengthen vehicular data security and improve model convergence and efficiency, making it applicable to IoV scenarios with diverse and dynamic data sources [32]. **Federated Distillation (FD):** FD leverages outputs from models rather than the parameters in FL. Since the output dimensions are significantly smaller than the model sizes, it is much more communication efficient [29]. For example, each device in a classification task performs local iterations while saving the average model output for each class. These local average outputs, which aggregate and average the local average output among agents in each class, are sent to the central node regularly. Each device downloads the results that constitute the resultant global average. Finally, each agent runs local iterations with their loss function in addition to a regularizer that measures the difference between its prediction output of a training sample and the global average output for the given class of the sample, which is called knowledge distillation (KD), to translate the downloaded global knowledge into local models. FD can also be beneficial in IoV to improve model performance in a network of vehicles [33]. **Group alternate direction method of multipliers (GADMM):** The FL central node may not be able to communicate with remote edge devices. It can also be vulnerable to failure or act as a single point of attack. To this aim, GADMM intends to provide distributed learning without a central entity by using the alternate direction method of multipliers (ADMM) technique and interacting exclusively with surrounding devices by splitting the devices into head and tail groups. Only two devices from the tail/head group are selected and create a chain, with each device from the head or tail group exchanging variables. With GADMM, only half of the agents compete for the restricted bandwidth during each communication cycle. Furthermore, by restricting communication to two nearby agents, the communication energy may be greatly reduced [34]. In IoV, GADMM can be used for collaborative decision making, traffic flow optimization, or other distributed tasks, such as energy-efficient resource allocation [35]. **Split Learning (SL):** SL is a technique that allows resource-limited wireless devices to train complex models such as Deep Neural Networks (DNN). During the DNN training process, the model can be split vertically or horizontally, allowing multiple nodes to train a portion of the model with limited data samples and training latency. SL combined with different forms of DL can be useful in dynamic vehicular settings for producing reliable complex learning models. Moreover, since SL does not exchange raw data, data privacy is somewhat maintained [30].