## 3 Insights into Federated Learning
### FL Architectures and Algorithms  
The architectures of FL are usually divided into three types: centralized, decentralized, and hierarchical. Figure 1 presents the usual topology FL applied in these studies. Figure 1: Centralized FL architecture [43; 56; 57]. As shown in Figure 1, a centralized FL system consists of a central server connecting with several clients in a star network. In the training round, some selected clients take part in the simultaneous training of a model using their local data, batches, epochs, and learning rate. Then, updated clients are then sent to the central server by the participated clients. The server aggregates a global model, and the global update is forwarded to the clients for the next training cycle. The training process is ended after several rounds. The server is regarded as the essential element of the network for coordinating the distribution of client updates to global aggregation while maintaining the confidentiality of the clients' training data. It aggregates a global model by averaging all the participated model updates. It is called Federated Averaging or FedAvg for short, which was first introduced by [PERSON], et al. in 2016 [PERSON] (2016). The models used in the works listed in Table 2 are usually deep learning ones: Long Short-Term Memory (LSTM) ([PERSON] et al., 2017; [PERSON] et al., 2018), Convolutional Recurrent Neural Networks (CRNN) ([PERSON] et al., 2018; [PERSON] et al., 2018; [PERSON] et al., 2018), and Generative Adversarial Nets (GAN) ([PERSON] et al., 2018). The pseudo-code of the whole training process is given in Algorithm 1 [PERSON] (2016). ```
ServerSite: \(\triangleright\) Run on Server  initialize model parameters \(W_{0}\) for each training round t do  random select clients for each client k in parallel do \(W_{t+1}^{k}\leftarrow\textbf{ClientSite}(k,W_{t})\)\(\triangleright\) Client update endfor \(W_{t+1}\leftarrow\sum_{k=1}^{K}\frac{n_{k}}{n}W_{t+1}^{k}\)\(\triangleright\) Global update endfor ClientSite(\(k,W_{t}\)): \(\triangleright\) Run on Clients  Fit local model with it dataset, batch size, epochs, and  learning rate to get new model update \(W_{t+1}^{k}\) Return (\(W_{t+1}^{k}\)) to server
```  
**Algorithm 1** FedAvg centralized  
Centralized FL architecture faces the main drawback when the number of clients is enormous ([PERSON] et al., 2018). This drawback is called a bottleneck because the vast communications link to the server leads to overload. There is another architecture without a server called a decentralized system, shown in Figure 2.  
Figure 2: Decentralized FL ([PERSON] et al., 2018; [PERSON] et al., 2018; [PERSON] et al., 2018). Decentralized FL is a network architecture with no server to manage the training process. As seen in Figure 2, all clients participated in a peer-to-peer (P2P) topology to carry out a training model. The clients also carry out local training in each communication round using their data set. Then, to reach an agreement on the global update, each client executes a model aggregation utilizing the model updates received from nearby clients through P2P communication. Decentralized FL may be made to entirely or partially rely on how it is used (see FL graph in [62]). The whole training process can be written in pseudocode as in (Algorithm 2).