# Overview of Distributed Machine Learning Techniques for 6G Networks
## 4 Distributed Machine Learning Algorithms
### Federated Learning  
Google proposed the concept of FL in 2016 [24; 25]. Given the vast amount of data now available on the network, the goal is to construct ML models based on datasets dispersed across a large number of devices while avoiding dispersion of data. The goal of this strategy is to provide users with the benefits of having a huge volume of data available without having to store it centrally [26]. While defining a number \(N\) of data owners as \(F_{1},F_{2},...,F_{N}\) and having the goal of training a ML algorithm on their respective data \(D_{1},D_{2},...,D_{N}\), a conventional method is to group all the data together and use a set \(D=D_{1}\cup D_{2}\cup...\cup D_{N}\) in order to train a \(M_{SUM}\) model. In a FL system, data owners collaborate to train an \(M_{FED}\) model, in which each owner of \(F_{i}\) data does not need to expose their \(D_{i}\) data to other users [27]. The main characteristics of ideal scenarios for FL applications are:  
* Real-world data training on mobile devices has an advantage over training data obtained via proxy and storage in data centers. * As these data are massive and sensitive to privacy, it is better to avoid storing them in a data center for the sole purpose of training on a certain model. * In supervised tasks, data labeling can be done directly by the user. Many applications involving intelligent implementations in the mobile environment, such as picture classification, language models, and speech recognition, reflect the characteristics described above. In fact, even when communicating _anonymized_ information to the data center, the user is still at risk, whereas in FL, the information communicated is strictly necessary to enhance a particular model. The goal of FL is typically to minimize an objective function such as:  
\[\min_{\omega\in\mathbb{R}^{d}}F(\omega)\quad\text{where}\quad f(\omega)\triangleq \sum_{i=1}^{m}p_{k}F_{i}(\omega)\]  
where \(m\) is the number of devices \(\omega\) is the model parameter vector, \(p_{k}>0\), and \(\sum_{i=1}^{m}p_{k}F_{i}(\omega)=1\). \(F_{k}\) is the objective function of the \(k\)-th device. The objective function is often defined as the empirical risk to local data; e.g.,  
\[F(\omega)=\frac{1}{n_{k}}\sum_{jk=1}^{n_{k}}f_{jk}(\omega;x_{jk};y_{jk})\]  
where \(n_{k}\) is the number of examples locally available. Instead, \(p_{k}\) specifies the relative impact that each device has, and is defined as  
\[p_{k}=\frac{1}{n}\quad\text{or}\quad p_{k}=\frac{n_{k}}{n}\]  
where \(n=\sum_{k}n_{k}\) is the total amount of examples [28]. FL optimization approaches must deal with the following issues: since the data in use for training are typically dependent on the user's use of the mobile device, the variables usually are not independent and identically distributed (non-IID). As a result, any given information set of a user will not be representative of the population distribution. Some users will use services or applications much more than others, so there will be extremely variable amounts of data available per user. The number of devices (client) involved in the optimization process will far exceed the number of examples available for each of them, and as mobile devices typically result in slow or expensive connections, communication is limited. Communication costs are modest in data center optimization, whereas computing costs are the most significant. In federate optimization, on the other hand, the key expenses to consider are those related to communication. In fact, since each dataset on a device is relatively small in comparison to the overall dataset, and because of the high-performance processors found in today's smartphones, computing becomes almost irrelevant. As a result, the goal is to increase computational capacity while decreasing the number of communication steps required to train the model.