# A Systematic Literature Review on Distributed Machine Learning in Edge Computing
## 4 Answering the RQs
### RQ2--Techniques and Strategies
#### 4.2.1 Federated Learning  
One of the most well-known and commonly implemented approaches in the EI research field is Federated Learning (FL). FL is based on the concept of Distributed Selective Stochastic Gradient Descent (DSSGD), introduced by [PERSON] [76] in 2015, and initially related to privacy-preserving in deep learning. This approach allows each part of the system to keep its local model private while iteratively updating it by integrating gradients \begin{table}
\begin{tabular}{c c} \hline \hline  & **References** & **Works That Tackle the Challenges** \\ \hline CH1 & [10, 15, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] & [14, 28, 31, 32, 33, 34, 35, 36, 37, 38, 49, 52, 56, 65, 66, 67] \\ CH2 & [10, 24, 28, 29, 46, 66] & [16, 21, 23, 33, 34, 36, 39, 43, 46, 56, 68, 69] \\ CH4 & [10, 24, 27, 36, 44, 70] & [8, 9, 10, 24, 44, 51, 71, 72, 73] \\ CH5 & [10, 27] & – \\ CH6 & [10, 24, 44, 74, 75] & [10, 38, 70] \\ \hline \hline \end{tabular}
\end{table} 
Table 6: References to the challenges of Edge Intelligence. of others through a parameter server. According to [PERSON] et al. [72], DSSGD exploits the fact that SGD can be parallelized and executed asynchronously. As reported by [PERSON] et al. [77], the use of FL enables to train ML models on private client data through the iterative communications of parameters between the server and clients. The whole process begins with the initialization of a random global model in the server. Then, iteratively, the server sends the parameters to random clients, which must update the model with their own data and upload the new values to the server that averages the updated ones and replaces the global model with the averaged one. This process is repeated until it achieves the desired performance. The great advantage of FL that made this strategy so attractive to the EI community is that edge nodes exchange and aggregate their local ML models, thereby preserving data privacy, while at the same time avoiding extra computation, and reducing communication overhead when ML model sizes are sufficiently smaller than data sizes [26]. There are different implementations/versions of federated learning trying to overcome possible weak points. The FedCS [39], for example, is an FL protocol that focuses on client selection. In cases of heterogeneous clients, clients with more data, compared to others, will require a longer time to update models unless they have better/higher computational resources. In this protocol, clients notify the server of their resource information. Using this information, the Mobile Edge Computing (MEC) operator determines which clients should be chosen to complete the subsequent steps within a certain deadline. [PERSON] and [PERSON] [71] present a different approach, where both server and clients are edge nodes, but belong to distinct hierarchies. Then, the so-called worker nodes (clients in FL) train independently and send the weights and biases to the master node (server in FL). The master node, therefore, sends these updates to the other nodes. The architecture of Fog-embedded Privacy-Preserving Deep Learning (FPPDL) [72] presents an extra layer to the default FL. The data are collected by the end nodes (devices) that forward the transformed data to a nearby fog node. The intermediate nodes are responsible for the client function of the classic FL, computing model gradients based on the data received. In conventional federated learning, the parameters of the entire Deep Neural Network (DNN) structure are updated at the same time, contributing to a huge communication overhead. The parameters of the shallow layers help the system to learn general features of the content access. On the other hand, a large number of parameters are generated at the deep layers to learn specific features related to specific content features and context information of the end nodes. In [PERSON] and [PERSON] [78], as a consequence, the parameters of the shallow layers could be updated more frequently in contrast with those of the deep layers in an asynchronous fashion.