# Overview of Distributed Machine Learning Techniques for 6G Networks
## 4 Distributed Machine Learning Algorithms
### Federated Learning
There are two basic approaches to accomplishing this: the first is to improve parallelism, which requires one to use more devices that operate independently, and the second is to boost the computation of each device, or to have each perform more complex operations. The heterogeneity of the network is another significant challenge in FL. Due to the variability of the hardware, connectivity, and available power in a federated system, the computing, communication, and data storage capabilities may change. Furthermore, the limits imposed by the network's and system's scale often result in just a small percentage of all available devices being active at any given moment. As a result, the proposed FL systems must account for low participation, be able to accept high heterogeneity, and be sufficiently robust in the case that active devices are turned off during an iteration owing to energy or connectivity constraints. Federated Averaging (FedAvg) Algorithm  
In terms of optimization, a large number of recent successful deep learning applications rely almost entirely on multiple variants of the stochastic gradient descent (SGD) method. As a result, the SGD method is considered as the starting point for federate optimization as well. At first, in might be used intuitively by calculating the gradient for the amount of samples to be processed and then updating the model in a single step for each communication cycle. This method is fast in terms of computing, but it requires a lot of training cycles to have a good model. In a federate environment, the cost of involving multiple devices is not particularly high; therefore, it is convenient to use a synchronous SGD method as the basis for optimization of a large set of samples (batch) before going to update the model. For each cycle, a section \(C\) of devices is chosen, and the gradient is calculated using all of the data available to them. As a result, \(C\) governs the amount of data that are processed, and \(C=1\) denotes the total amount of data handled. This basic algorithm is called FederatedSDG [26]. With \(C=1\) and a fixed learning rate \(\eta\), a common implementation of this algorithm has each device compute \(g_{k}=\
abla F_{k}(\omega_{t})\) and save the average gradient in the local data for the current model \(\omega_{t}\); and the central server aggregates these gradients and applies the update:  
\[\omega_{t+1}\leftarrow\omega_{t}-\eta\sum_{k=1}^{K}\frac{n_{k}}{n}g_{k}\quad since \quad\sum_{k=1}^{K}\frac{n_{k}}{n}g_{k}=\
abla f(\omega_{t})\]  
Each device, using its own local data, conducts a gradient descent step in the current model, and the server takes the average value of the resulting models. Increasing each device's computational load can be done by iterating its local update before doing the averaging calculation, having implemented the algorithm this way. This approach is called FederatedAveraging, or FedAvg [26; 28]. Three parameters determine the amount of computing: \(C\), the fraction of devices that conduct the computation in each cycle; \(E\), the number of times the training is completed locally every cycle; and \(B\), the size of the data partition to be processed (batch) locally before executing the update. Figure 3 shows the FL framework with FedAvg process. The main FL process elements, i.e., local FL device processing, FL communication, and the FedAvg process at a centralized FL server, are shown.