SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 1 EvoFed: Federated Learning with Evolutionary Strategies
1: Input: Learning rates  $\eta$ ,  $\alpha$ , population size  $N$ , noise std  $\sigma$ , seed  $s$   
2: Initialize: server and  $M$  clients with seed  $s$  and identical parameters  $\theta_0$  and  $\theta_{0,j}$  respectively  
3: for each communication round  $t = 0,1,\dots,T - 1$  in parallel do  
4: for each client  $j$  in parallel do  
5:  $\theta_{t,j}^{\prime} = \theta_{t,j} - \eta \nabla_{\theta}L(\theta_{t,j}^{\prime})$  //Backpropagation Update  
6: Sample  $\epsilon_t^i\sim \mathcal{N}(0,\sigma I)$  //Sample perturbations  
7:  $\theta_{t,j}^{i} = \theta_{t,j} + \epsilon_{t}^{i}$  for  $i = 1,\dots,N$  //Initialize population  
8:  $\mathbf{f}_{t,j}^{i} = -||\theta_{t,j}^{i} - \theta_{t,j}^{\prime}||_{2}^{2}$  for  $i = 1,\dots,N$  // Compute fitness vector using  $L_{2}$  calculation  
9: end for  
10:  $\mathbf{F}_t^i = \frac{1}{\sum_j^M b_j}\sum_j^M b_j\mathbf{f}_{t,j}^i$  for  $i = 1,\dots,N$  // Server averages fitness vectors  
11:  $\theta_{t + 1} = \theta_t + \frac{\alpha}{N\sigma}\sum_{i = 1}^N\mathbf{F}_t^i\cdot \epsilon_t^i$  // Server updates model using the aggregated fitness  
12: Broadcast  $\{\mathbf{F}_t^1,\mathbf{F}_t^2,\ldots ,\mathbf{F}_t^N\}$   
13: for each client  $j$  in parallel do  
14:  $\theta_{t + 1,j} = \theta_{t,j} + \frac{\alpha}{N\sigma}\sum_{i = 1}^{N}\mathbf{F}_t^i\cdot \epsilon_t^i$  // Client updates model using the aggregated fitness  
15: end for  
16: end for
4.4 Broadcasting Fitness
Once the aggregation is complete, the server broadcasts  $\mathbf{F}_t$  to all client nodes, maintaining synchronicity across the network. This only involves the transmission of the fitness vector, again reducing communication overhead. The aggregated fitness vector can be used by the local clients (as well as by the server, if needed) to update the local models.
4.5 Client-side Update
When the global fitness vector  $\mathbf{F}_t$  is received from the server, the clients can update their local model following Eq. 5. This strategy involves the addition of a weighted sum of noise vectors to the current model parameters, where the weights are aggregated fitness values:
$$
\theta_ {t + 1} = \theta_ {t} + \frac {\alpha}{N \sigma} \sum_ {i = 1} ^ {N} \mathbf {F} _ {t} ^ {i} \cdot \epsilon_ {t} ^ {i} \tag {9}
$$
where  $\epsilon_t^i$  is the  $i$ -th noise vector from the population at time  $t$ .
Notably, the right side of Eq. 9 can be shown equivalent to the average of all locally updated models akin to FedAvg, i.e., it is straightforward to show that
$$
\theta_ {t + 1} = \frac {1}{\sum_ {j} ^ {M} b _ {j}} \sum_ {j = 1} ^ {M} b _ {j} \theta_ {t + 1, j} \tag {10}
$$
where  $\theta_{t + 1,j}$  is the updated model of client  $j$  at time  $t + 1$  defined as
$$
\theta_ {t + 1, j} = \theta_ {t} + \frac {\alpha}{N \sigma} \sum_ {i = 1} ^ {N} f _ {t, j} ^ {i} \cdot \epsilon_ {t} ^ {i}. \tag {11}
$$
4.6 Convergence Analysis
Theorem 1. Suppose that  $L_{j}(\theta)$  is the  $\beta$ -smooth function, i.e.,  $\| \nabla L_{j}(u) - \nabla L_{j}(v)\| \leq \beta \| u - v\|$  for any  $u, v$ , and also suppose that the variance of the stochastic gradient of  $D_{j}$  is bounded, i.e.,  $\mathbb{E}\| \nabla L_{j}(\theta) - \widetilde{\nabla} L_{j}(\theta)\|^{2} \leq B^{2}$  for all  $j$ . When perturbation  $\epsilon^i$  is sampled, a conditioned mirrored sampling is applied such that  $\frac{1}{N}\sum_{i = 1}^{N}\epsilon^{i} = 0$ ,  $\frac{1}{N}\sum_{i = 1}^{N}\left(\epsilon^{i}\right)^{2} \leq G^{2}$ ,  $\frac{1}{N}\sum_{i = 1}^{N}\left(\epsilon^{i}\right)^{3} = 0$ . Given a decreasing learning rate  $\eta_t < \frac{1}{4\alpha\beta}$ , EvoFed converges in the sense of
$$
\frac {1}{H _ {T}} \sum_ {t = 0} ^ {T - 1} \eta_ {t} \mathbb {E} \left[ \| \nabla L (\theta_ {t}) \| ^ {2} \right] \leq \frac {\mathbb {E} \left[ L (\theta_ {0}) \right] - L ^ {*}}{\alpha G ^ {2} H _ {T}} + 4 \alpha \beta B ^ {2} \left(\frac {1}{H _ {T}} \sum_ {t = 0} ^ {T - 1} \eta_ {t} ^ {2}\right)
$$
Figure 7: Privacy enhancement through Fully Homomorphic Encryption and Trusted Execution Environments. where  $H_{T} = \sum_{t=0}^{T-1} \eta_{t}$ , and  $L^{*}$  represents the minimum value of  $L(\theta)$ .
Given a decreasing learning rate (e.g.,  $\eta_t = \frac{\eta_0}{1 + t}$ ), it can be seen that  $H_T = \sum_{t=0}^{T-1} \eta_t \to \infty$  as  $T$  increases, while  $\sum_{t=0}^{T-1} \eta_t^2 < \infty$ . Consequently, the upper bound stated in Theorem 1 approaches 0 as  $T$  grows, ensuring convergence towards a stationary point. The detailed discussions and the proof can be found in Supplementary Materials.
4.7 EvoFed and Privacy Enhancements
Unlike other FL methods, EvoFed operates on fitness measures. Encryption on smaller fitness vectors requires a lower overhead compared to encryption on large model parameter vectors. Fig. 7 shows Fully Homomorphic Encryption (FHE) [48-50] that allows aggregation of encrypted data on the server while keeping individual client data confidential. For the decryption of the aggregated fitness vector, EvoFed can leverage third-party Trusted Execution Environments (TEEEs), such as Intel SGX [51], providing a secure space for sensitive computations.
4.8 Partial Client Participation
When FL has to operate with a large client pool, a typical practice is to select only a subset of clients in each global round. In EvoFed, this means that the newly joined clients in a given round can either download the latest model or else the last  $k$  fitness vectors from the server, where  $k$  is the time gap from the last participation. In the latter case, the model updates from Eq. (9) can be modified to accommodate  $k$ -step updates:
$$
\theta_ {t} = \theta_ {t - k} + \frac {\alpha}{N \sigma} \sum_ {l = 1} ^ {k} \sum_ {i = 1} ^ {N} \mathbf {F} _ {t - l} ^ {i} \cdot \epsilon_ {t - l} ^ {i}. \tag {12}
$$
Note that even in the former case where the latest model is downloaded by the newly joined clients, the level of security is not compromised as far as the client-to-server messages are concerned.
Our algorithm's effectiveness is assessed on three image classification datasets: FMNIST [52], MNIST [53], and CIFAR-10 [54]. Both MNIST and FMNIST contain 60,000 training samples and 10,000 test samples, whereas CIFAR-10 is composed of 50,000 training samples and 10,000 test samples. We employ a CNN model having 11k parameters for the MNIST and FMNIST datasets and a more substantial model with 2.3M parameters for CIFAR-10. A grid search has been conducted to identify the optimal performance hyperparameters for each baseline, as outlined in the results section. We take into account both global accuracy and communication costs to ascertain hyperparameters that maximize accuracy while minimizing communication overhead.
Data Distribution. We distribute the training set of each dataset among clients for model training, and the performance of the final global model is evaluated using the original test set. Our experimental setup contains  $M = 5$  clients with non-IID data distribution (assigning two classes to each client).
Implementation Details. Our EvoFed framework is built using JAX [55], which facilitates extensive parallelization and, in particular, consistent random number generation across a large number of nodes. We have implemented our framework on the Evosax [56] library, a convenient tool for the ES algorithm. EvoFed is configured with a population size of 128 and a mini-batch size of 256 for MNIST / FMNIST and 64 for CIFAR-10. We perform ten local epochs (performing ten BP steps before fitness calculation) and train over 1,000 global rounds.
Baselines. We compare the performance of the proposed EvoFed with BP, FedAvg, ES, FedAvg with quantization (Fed-quant), and FedAvg with Sparsification (Fed-sparse). In each scenario, we push
(a) MNIST Accuracy
(d) MNIST Comm. cost
(b) FMNIST Accuracy
(e) FMNIST Comm. cost
(c) CIFAR-10 Accuracy
(f) CIFAR-10 Comm. cost
Figure 8: Performance comparison of EvoFed and baseline methods on MNIST, FMNIST, and CIFAR-10 datasets. The top row displays the accuracy achieved by each method on the respective datasets, while the bottom row illustrates the communication cost associated with each method.
for maximum compression, stopping right before the model starts to show performance degradation relative to FedAvg with no compression. BP provides the upper-performance baseline, while ES serves as a reference emphasizing the significance of PBGE.
6 Results and Discussions
In this section, we discuss the experimental results in detail and provide further insights into the performance of EvoFed. The accuracy of EvoFed, compared with multiple baseline methods and different datasets, is shown in Fig. 8 (a), (b), and (c). Efficiently encoding and exchanging gradient information, EvoFed enhances the effectiveness of the ES algorithm across all tasks, delivering results comparable to FedAvg. Also, EvoFed achieves superior accuracy at an equivalent compression rate compared to sparsification. This suggests that utilizing a shared population of samples can reduce the information necessary for gradient compression, thereby enhancing the efficiency of the process. Fig. 8 (d), (e), and (f) shows the performance of each method as a function of communication load for all three datasets. It can be seen that EvoFed tends to utilize significantly less communication resources as compared to other high-accuracy techniques.
Table 1 summarizes the performance of different schemes on MNIST, FMNIST, and CIFAR-10 datasets, focusing on communication cost and accuracy. EvoFed achieves significantly lower communication costs compared to FedAvg while maintaining competitive accuracy levels. In the MNIST dataset, EvoFed achieves an accuracy of  $97.62\%$  with a mere  $9.2\mathrm{MB}$  of communication load, while FedAvg achieves  $98.09\%$  accuracy at a considerably high communication cost of  $73.7\mathrm{MB}$ . The effective compression achieved is an impressive  $98.8\%$  which indicates that the gradient vector is condensed into just  $1.2\%$  of the fitness vector that is communicated between clients and the server. Similarly, for the FMNIST dataset, EvoFed achieves an accuracy of  $84.72\%$  with only  $7.78\mathrm{MB}$  of communication, whereas FedAvg's accuracy is  $85.53\%$  with a communication cost of  $40.99\mathrm{MB}$ . The efficiency of EvoFed becomes even more apparent in the CIFAR-10 dataset where the model compression is over  $99.7\%$ . EvoFed achieves an accuracy of  $54.12\%$  with a low communication cost of only  $0.023\mathrm{GB}$ , surpassing FedAvg, which performs  $50.22\%$  at a communication cost of  $2.134\mathrm{GB}$ . The simpler ES method actually gives better performance as well as higher communication efficiency than EvoFed for MNIST but its accuracy for other data sets is highly limited.
Additional Experiments. Fig. 9 illustrates the performance of EvoFed with varying numbers of samples  $N$  within the population and a varying number of clients  $M$ . The model's performance is
Methods	MNIST	FMNIST	CIFAR-10
Comm. Cost (MB)	Max Acc.	Comm. Cost (MB)	Max Acc.	Comm. Cost (GB)	Max Acc.
90% Acc.	Max Acc.	70% Acc.	Max Acc.	45% Acc.	Max Acc.
ES	0.1	4.6	98.30%	0.48	4.85	76.86%	-	0.021	37.47%
FedAvg	4.2	73.7	98.09%	0.87	40.99	85.53%	0.266	2.134	50.22%
Fed-quant	1.7	41.8	98.15%	0.94	37.98	83.23%	0.086	0.800	49.78%
Fed-sparse	-	1.0	67.85%	2.78	17.13	73.17%	0.129	0.671	47.36%
EvoFed (our)	0.8	9.2	97.62%	0.75	7.78	84.72%	0.004	0.023	54.12%
Table 1: Performance of different schemes presented in tabular form, corresponding to Fig. 8.
significantly influenced by the population size, which directly affects the algorithm's exploration capability and compression rate. We observe a performance improvement as we increase the population size, although at the cost of increased computation and memory usage. Nonetheless, this improvement is not linear and plateaus once sufficient samples are generated to explore the parameter space. For instance, in the case of the FMNIST dataset and a model with 11K parameters, any performance enhancement beyond a generated sample size of 128 is marginal.
Fig. 9(b) showcases EvoFed's performance trend as the number of clients increases. A minor performance decline is observable for some larger values of M relative to  $\mathrm{M} = 5$ . This decline could potentially be attributed to the limited data available per client and the increased variance associated with local training. Nonetheless, we believe that carefully tuning the hyperparameters based on data distribution could assist in achieving more robust performance.
(a) Varying number of population from 8 to 4096
(b) Varying number of clients from 5 to 100
Figure 9: Effect of population size (left) and number of clients (right) on EvoFed
Supplementary Materials provide further details regarding hyperparameters, model architectures, and experiments. We also include a detailed ablation study with different evolutionary strategies, the population size, and the number of partitions, as well as detailed communication and computational complexity analyses.
EvoFed, the novel paradigm presented herein, unites FL and ES to offer an efficient decentralized machine learning strategy. By exchanging compact fitness values instead of extensive model parameters, EvoFed significantly curtails communication costs. Its performance parallels FedAvg while demonstrating an impressive equivalent model compression of over  $98.8\%$  on FMNIST and  $99.7\%$  on CIFAR-10 in representative experimental settings. Consequently, EvoFed represents a substantial stride forward in FL, successfully tackling the critical issue of high communication overhead.
A special acknowledgment goes to Do-yeon Kim for the valuable discussions on the convergence analysis presented in Section 4. This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (No. NRF-2019R1I1A2A02061135), and by IITP funds from MSIT of Korea (No. 2020-0-00626).
[1] Jakub Konečný, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492, 2016.
[55] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http://github.com/google/jax.
[56] Robert Tjarko Lange. evosax: Jax-based evolution strategies. arXiv preprint arXiv:2212.04180, 2022.