# Towards Mobile Federated Learning with Unreliable Participants and Selective Aggregation
## 2 Background and Related Work
Typically, a training dataset contains a set of \(n\) data feature vectors \(x=\{x_{1},\ldots,x_{n}\}\) and a set of corresponding data labels. In the case of unsupervised learning, there is no data label. \(y=\{y_{1},\ldots,y_{n}\}\). Moreover, \(y_{j}=f(x_{j};\mathbf{w})\) denotes the predicted result from the model \(\mathbf{w}\) updated/trained by data vector \(x_{j}\)[13]. Most of the recent successful deep learning [14] applications use variants of SGD for optimization. In fact, many advances can be understood as an adaptation of the structure of the model (and hence the loss function) to facilitate optimization using simple gradient-based methods. Therefore, it is only natural that federated optimization algorithms are built based on SGD. After training and updating the local model comes an essential part of the FL process: global model aggregation. There are several proposed solutions to this issue. Experiments by [PERSON] et al. [15] show that large-batch synchronous state-of-the-art SGD optimization outperforms asynchronous approaches, in a data center application. To apply this approach in a federated context, a C-fraction of clients/parties is selected at each round, and the gradient of the loss over all the data held by these parties is computed. Thus, \(C\) controls the _global_ batch size, with \(C=1\) corresponding to full-batch (nonstochastic) gradient descent. While the batch size selection mechanism is different than selecting a batch by choosing individual examples uniformly at random, the batch gradients \(g\) computed by FedSGD still satisfies \(\mathbb{E}[g]=\bigtriangledown f(w)\). The baseline algorithm is FederatedSGD (FedSGD) [14]. A typical implementation of FedSGD with \(C=1\) and a fixed learning rate \(\eta\) has each client \(k\) compute \(g_{k}=\bigtriangledown F_{k}(w_{t})\), the average gradient on its local data at the current model \(w_{t}\), and the central server aggregates these gradients and applies the update \(w_{t+1}\gets w_{t}-\eta\sum_{k=1}^{K}\frac{n_{k}}{n}g_{k}\), since \(\sum_{k=1}^{K}\frac{n_{k}}{n}g_{k}=\bigtriangledown f(w_{t})\)[16]. An equivalent update is given by \(\forall k\), \(w_{t+1}^{k}\gets w_{t}-\eta g_{k}\) and then \(w_{t+1}\leftarrow\sum_{k<1}^{K}\frac{n_{k}}{n}w_{t+1}^{k}\). That is, each client locally performs a gradient descent step on the current model using its local data, and the server takes a weighted average of the resulting models. Within the algorithm, more computation steps  
Figure 3: Federated Learning Architecture. can be added to each client by iterating the local update \(w^{k}\gets w^{k}-\eta\bigtriangledown F_{k}(w^{k})\) multiple times before the averaging step [14]. This approach is called FederatedAveraging (FedAvg). The number of computational steps is controlled by three key parameters: \(C\), the fraction of clients performing computations in each round; \(E\), the number of training passes each client makes over its local dataset in each round; and \(B\), the local batch size used for client updates. Selecting \(B=\infty\) and \(E=1\) is equivalent to FedSGD. In FedAvg, for a client with \(n_{k}\) local examples, the number of local updates per round is given by \(u_{k}=E\frac{n_{k}}{B}\)[14]. Algorithm 1 is a complete pseudocode of the FedAvg procedure. ```
Server executes:  initialize \(w_{0}\) for each round \(t=1,2,\ldots\)do \(m\gets max(C\cdot K,1)\) \(S_{t}\leftarrow\) (random set of \(m\) clients) for each client \(k\in S_{t}\) in parallel do \(w^{k}_{t+1}\leftarrow\) ClientUpdate\((k,w_{t})\) \(w_{t+1}\leftarrow\sum_{k=1}^{K}\frac{n_{k}}{n}w^{k}_{t+1}\) ClientUpdate\((k,w)\)://Run on client \(k\) \(B\leftarrow\) split \(P_{k}\) into batches of size \(B\) for each local epoch \(i\) from \(1\) to \(E\)do for batch \(b\in B\)do \(w\gets w-\eta\bigtriangledown l(w;b)\)  return \(w\) to server
```  
**Algorithm 1**FederatedAveraging (FedAvg). The \(K\) clients are indexed by \(k\); \(B\) is the local minibatch size, \(E\) is the number of local epochs, and \(\eta\) is the learning rate.