# Overview of Distributed Machine Learning Techniques for 6G Networks
## 6 Future Directions
### Collaborative MARL with Wireless Information Sharing  
In the past, various MARL techniques have been proposed for implementation in different wireless networking environments. They are mainly based upon the different natures of the training processes. In general, MARL-based solutions involve multiple agents with a common environment trying to optimize a common reward signal. The training policies adapted by the individual agents can impact the other agents' surrounding environments, and as result can dictate their policies. In most wireless environments, agents can have a limited knowledge of the surrounding environments (i.e., partial observability), which makes the MARL problem non-stationary. Additionally, in the case of fully centralized training methods, exponential growths of action and observation spaces can become unbearable in most resource-constrained wireless networking scenarios, i.e., vehicular networks. Therefore, fully centralized training-based MARL solutions have shown limited performances, and other training policies are required. In recent times, other training approaches have been proposed wherein agents can learn effectively. One such approach can be found in [79], where the authors proposed a value-decomposition network-based cooperative MARL architecture. A deep neural network-based backpropagation approach is used to  
Figure 8: Computation offloading for FL. Reprinted/adapted with permission from Ref. [78]. 2021, Attribution 4.0 International (CC BY 4.0).  
Figure 7: The hierarchical FL platform in a Multi-EC VN architecture. Reprinted/adapted with permission from Ref. [78]. 2021, Attribution 4.0 International (CC BY 4.0).  
decompose a common value function into a set of agent-based value functions. However, the complexity of the considered decomposition network, limited uses of state information during training, and applicability towards a reduced class of centralized value functions are bottlenecks. Additionally, in [80], another value-function-based approach, is considered by estimating a joint action-value function from a set of local observation-based action individual agents values through neural networks. The issues of complexity, scalability, etc., can limit the uses of this MARL-based method for solving the given problem.  
In the following Figure 9, we propose a collaborative MARL-based solution based upon the data-sharing capabilities of wireless nodes. A dynamic vehicular scenario is considered where a set of moving vehicles, i.e., \(m=1,2,\cdots M\), are employing the MARL for performing the task \(T\). Task \(T\) can be a generic task, such as computation offloading, path planning, and collision avoidance. By employing the V2V and V2I communication technologies, vehicles can share the important environmental parameters, which can be used to reduce issues such as partial observability. MARL agents can have proper command over the nearby environment through such data sharing approaches and can be able to perform the training process with proper knowledge. This way, a centralized training method can be employed by avoiding the challenges introduced by the partial observability and unintended complexity of action and/or observation spaces.