Title: EvoFed: Leveraging Evolutionary Strategies for Communication-Efficient Federated Learning
Venue: NeurIPS 2023
Accepted: yes
Keywords: evolutionary strategies

federated learning

gradient compression

distributed learning
Abstract: Federated Learning (FL) is a decentralized machine learning paradigm that enables collaborative model training across dispersed nodes without having to force individual nodes to share data.
However, its broad adoption is hindered by the high communication costs of transmitting a large number of model parameters. 
This paper presents EvoFed, a novel approach that integrates Evolutionary Strategies (ES) with FL to address these challenges.
EvoFed employs a concept of `fitness-based information sharing’, deviating significantly from the conventional model-based FL. 
Rather than exchanging the actual updated model parameters, each node transmits a distance-based similarity measure between the locally updated model and each member of the noise-perturbed model population. Each node, as well as the server, generates an identical population set of perturbed models in a completely synchronized fashion using the same random seeds. 
With properly chosen noise variance and population size, perturbed models can be combined to closely reflect the actual model updated using the local dataset, allowing the transmitted similarity measures (or fitness values) to carry nearly the complete information about the model parameters.
As the population size is typically much smaller than the number of model parameters, the savings in communication load is large. The server aggregates these fitness values and is able to update the global model. This global fitness vector is then disseminated back to the nodes, each of which applies the same update to be synchronized to the global model. Our analysis shows that EvoFed converges, and our experimental results validate that at the cost of increased local processing loads, EvoFed achieves performance comparable to FedAvg while reducing overall communication requirements drastically in various practical settings.

EvoFed: Leveraging Evolutionary Strategies for Communication-Efficient Federated Learning

Abstract

Federated Learning (FL) is a decentralized machine learning paradigm that enables collaborative model training across dispersed nodes without having to force individual nodes to share data. However, its broad adoption is hindered by the high communication costs of transmitting a large number of model parameters. This paper presents EvoFed, a novel approach that integrates Evolutionary Strategies (ES) with FL to address these challenges. EvoFed employs a concept of 'fitness-based information sharing', deviating significantly from the conventional model-based FL. Rather than exchanging the actual updated model parameters, each node transmits a distance-based similarity measure between the locally updated model and each member of the noise-perturbed model population. Each node, as well as the server, generates an identical population set of perturbed models in a completely synchronized fashion using the same random seeds. With properly chosen noise variance and population size, perturbed models can be combined to closely reflect the actual model updated using the local dataset, allowing the transmitted similarity measures (or fitness values) to carry nearly the complete information about the model parameters. As the population size is typically much smaller than the number of model parameters, the savings in communication load is large. The server aggregates these fitness values and is able to update the global model. This global fitness vector is then disseminated back to the nodes, each of which applies the same update to be synchronized to the global model. Our analysis shows that EvoFed converges, and our experimental results validate that at the cost of increased local processing loads, EvoFed achieves performance comparable to FedAvg while reducing overall communication requirements drastically in various practical settings.

1 Introduction

Federated Learning (FL) provides a decentralized machine learning framework that enables model training across many devices, known as clients, without needing to collect and process sensitive client data on the centralized server [1]. The typical FL process begins with each client downloading an identical initialized model from a central server, performing model updates with local data, and then uploading the updated local model for the next communication round. Subsequently, the server combines the uploaded models to refine the global model, typically using a technique like FedAvg [2]. This iterative cycle repeats for a fixed number of rounds, ensuring collaborative model improvement across clients.

Although FL provides notable benefits, such as a certain level of privacy preservation and the utilization of diverse data sources, one of the major challenges associated with FL is the significant

communication overhead involved in transmitting model updates between clients and the server, especially when dealing with models that have a large number of parameters.

Various strategies have been developed to mitigate the communication burden in FL. These techniques can be broadly classified into three categories: i) Compressing updates: sparsification [1], structured updates [3], and quantization [4, 5] reduce the size of transmitted model updates, ii) Local computation: performing multiple local epochs at the clients [6] lessens the frequency of communication with the server, and iii) Advanced aggregation methods and client selection: MOCHA [7] enhances the efficacy of update aggregation and [8, 9] reduce communication by only selecting a subset of clients to participate in each training round.

Existing FL techniques primarily rely on transmitting gradient signals or model updates, which are computed through backpropagation (BP). On the other hand, Evolutionary Strategies (ES) [1012] update model parameters by utilizing fitness values obtained from evaluating a population of models. This approach eliminates the need for a gradient signal, as depicted in Fig. 1. Recent advances in neuroevolution have shown promise in supervised settings and competitive performance with reinforcement learning in control tasks [1315, 11, 16-18]. In this context, ES offers a distinct advantage compared to traditional BP methods. This paradigm shift opens up the possibility of searching for novel solutions beyond the gradient-

𝜃𝜃

𝑡𝑡

𝜃𝜃

𝑡𝑡

1

𝜃𝜃

𝑡𝑡

6

𝜃𝜃

𝑡𝑡

5

𝜃𝜃

𝑡𝑡

4

𝜃𝜃

𝑡𝑡

3

𝜃𝜃

𝑡𝑡

2

Ask

Generate generation

𝜃𝜃

𝑡𝑡

𝜃𝜃

𝑡𝑡

1

𝜃𝜃

𝑡𝑡

6

𝜃𝜃

𝑡𝑡

5

𝜃𝜃

𝑡𝑡

4

𝜃𝜃

𝑡𝑡

3

𝜃𝜃

𝑡𝑡

2

Evaluate

Call fitness

𝜃𝜃

𝑡𝑡

𝜃𝜃

𝑡𝑡+1

Tell

Update generation

𝑖𝑖

𝜃𝜃

𝜃𝜃

𝑡𝑡

(

𝜃𝜃

𝑡𝑡

)

𝑡𝑡

→ 𝜃𝜃𝑡𝑡+1

based methods. However, it is critical to note that ES, despite its potential, still lags behind gradientbased methods in certain problems like supervised learning. Fig. 2(a) reveals the performance gap between ES and BP. This emphasizes the need for a balanced approach that would leverage the strengths of both ES and gradient-based methods to achieve optimal results.

Our proposed method operates on the premise of incorporating high-quality gradient signals into the evaluation process of ES. The process can be formalized as follows:

Given a base model, denoted by θ , we instantiate a population P comprised of N model samples. Each individual sample, θ i , is derived by adding random perturbations to θ . Unlike traditional ES, where the fitness of θ i corresponds to its performance on the task, we instead assess the fitness of each sample θ i by measuring its similarity to θ ′ , the model parameters updated through gradient descent steps. This operation effectively exploits the gradient signal to construct a fitness vector, a process we term Population-Based Gradient Encoding (PBGE). Fig. 2(b) shows the results of a representative experimental setting where PBGE is able to follow BP closely on the FMNIST dataset while maintaining an effective compression rate of over 98.8%. In particular, this method significantly outperforms sparsification strategies at equivalent compression rates.

In the context of FL, the gradient signal can be encoded into a fitness vector for the population of models and communicated to the server. For global synchronization, a well-established approach like FedAvg would involve reconstruction and aggregation of the clients' models and sharing of the aggregated model with the clients. However, by utilizing shared random seeds, we can ensure uniformity in the generated populations between clients and the server. This allows us

0

200

400

600

800

1000

Epochs

0

20

40

60

80

100

Test accuracy(%)

FMNIST

BP

ES(98.8%)

(a) Estimation with ES

0

200

400

600

800

1000

Epochs

0

20

40

60

80

100

Test accuracy(%)

FMNIST

Sparse(98.8%)

BP

PBGE(98.8%)

(b) The proposed method

to transmit and aggregate only the small fitness vectors without reconstructing the models, reinforcing communication efficiency. In addition, there is an advantage in implementing extra privacy measures. When encryption is desired, the required overhead would be much smaller with the fitness vectors than with the model parameter vectors because of the size difference. A brief overview of EvoFed is shown in Fig. 3 (the detailed methodology is provided in Section 4).

Generate perturbations

& Update local model

①

Seed

Get fitness scores

& Upload

②

Fitness

Broadcast aggregated

fitness & Update model

③

Aggregated

Fitness

Initial model

Perturbed model

Locally updated model (by backpropagation)

Globally updated model (by fitness)

Contributions. To summarize, our main contribution is to introduce a novel concept of PopulationBased Gradient Encoding (PBGE), which allows an accurate representation of the large local gradient vector using a relatively small fitness vector, significantly reducing the communication burden as well as the encryption overhead in FL environments. We also propose and verify the EvoFed framework that integrates PBGE into federated learning based on the exchange and aggregation of the fitness vectors between clients and the server.

Our approach achieves similar performance to FedAvg on both FMNIST and CIFAR-10 datasets. The advantage is that our scheme achieves very high compression, exceeding 98.8% on FMNIST and 99.7% on CIFAR-10 in key practical settings. Significantly, our EvoFed model also outperforms traditional compression methods, offering superior results at similar compression rates. The price we pay is the overhead of generating a population of perturbed models and computing similarity measures. This overhead could be substantial in the form of increased local processing time depending on the size of the population, but the client computational load can be traded with local memory space by employing parallel processing. In addition, as discussed later, reduce population size without affecting performance.

2 Related Work

Federated Learning. Several techniques have been proposed to alleviate the communication overhead in FL, such as model compression [1, 19, 20], distillation techniques [21-25], and client update subsampling [26, 27]. However, these approaches often come with trade-offs, including increased convergence time, lower accuracy, or additional computational requirements. More importantly, they still require the exchange of model parameters, which requires substantial communication bandwidth. Furthermore, [28] shows that the gradient sparsity of all participants has a negative impact on global convergence and communication complexity in FL.

Evolutionary Strategies. ES are black-box optimization algorithms inspired by biological evolution [29]. ES algorithms iteratively refine a population of solutions based on fitness evaluations. Natural Evolution Strategies (NES) is a specific variant of ES [30, 12, 31-35]. Within the NES framework, the distribution p ψ ( θ ) , parameterized by ψ , is adopted to represent the population and maximize the average objective value E θ ∼ p ψ [ f ( θ )] via stochastic gradient ascent, where f ( θ ) is the fitness function. NES algorithms leverage a score function estimator as in [10]. Our method follows the guidelines provided by [36], where the parameter distribution p ψ is a factored Gaussian. Accordingly, we can represent E θ ∼ p ψ [ f ( θ )] using the mean parameter vector θ , such that E θ ∼ p ψ [ f ( θ )] = E ϵ ∼N (0 ,I ) [ f ( θ + σϵ )] . Given a differentiable function estimator, the well-known conversion procedure allows optimization over θ to be rewritten as (see, for example, [37, 11, 38])

The strategy of sharing seeds for random number generators to synchronize parallel processes and maintain consistency is a well-established practice in areas ranging from parallel simulations to cryptographic protocols [39]. The appeal of shared random seeds stems from their ability to offer deterministic randomness, ensuring that multiple entities, operating independently, can produce synchronized and identical random outputs. Within the realm of ES, this concept was effectively harnessed by [36] to address the challenge of scalability. We also utilize the shared random seeds to

𝜃𝑡

5

𝜃𝑡

3

𝜃𝑡

6

𝜃𝑡

1

𝜃𝑡

2

𝜃𝑡

4

𝜃𝑡

𝑓(𝜃𝑡

𝑖

)  = 𝐿(𝜃

𝑡

𝑖

)

𝜃𝑡

5

𝜃𝑡

3

𝜃𝑡

6

𝜃𝑡

1

𝜃𝑡

2

𝜃𝑡

4

𝜃𝑡

𝜃𝑡

′

𝛁𝜽𝐄𝝐[𝒇 𝜽𝒕

𝒊

]

𝑓(𝜃𝑡

𝑖

) = -||𝜃 𝑡

𝑖 -𝜃𝑡

′

||

2

𝛁𝜽𝑳(𝜽𝒕)

𝛁𝜽𝐄𝝐[𝒇 𝜽𝒕

𝒊

]

𝜃𝑡

′

= 𝜃𝑡 -𝜂∇𝜃𝐿(𝜃𝑡)

Contour of fitness

𝑓(𝜃 𝑡

𝑖

)  = 𝐿(𝜃

𝑖

)

Contour of fitness

𝑓(𝜃𝑡

𝑖

) = -||𝜃 𝑡

𝑖 -𝜃𝑡

′

||

2

2

𝑡

2

scale down the communication load in a distributed setting. However, the key difference is that the work of [36] distributes the perturbations into multiple workers dealing with a single global objective, whereas our method generates identical perturbations across all clients, with each having a distinct objective. An important consequence is that with our method, each client only needs to communicate N fitness values to update the model, instead of MN values in [36] with M denoting the number of nodes, enabling scalability regardless of the number of clients.

Federated Learning and Evolutionary Strategies. Several studies have explored the optimization of FL using ES. For instance, [40] introduced an evolutionary approach for network architecture search (NAS) in real-time FL, which minimizes local payload while optimizing model performance. Sparse evolutionary training (SET) [41] substitutes fully connected layers in neural networks with sparse layers to decrease the number of model parameters. Furthermore, [42] presented the SET algorithm that optimizes neural networks in FL via a bi-objective method to maximize accuracy performance while minimizing communication overhead. Additionally, [43] introduces the MOEA/D framework [44] to the environment of FL and FLEA [45] utilized Evolutionary Algorithms in FL setup at the client-level to evolve models.

While these studies have made significant contributions, the present work establishes a unique way of using ES as a method to reduce communication overhead in FL by transmitting fitness values instead of model parameters. In particular, the new gradient-driven fitness function essentially separates our work from the traditional utilization of ES for compression.

3 Population-Based Gradient Encoding

This work focuses on the process of encoding gradient information through an identical population of models generated at both ends of some network link. Population distribution is a zero mean isotropic multivariate Gaussian with fixed covariance σ 2 I . We also adopt 'mirrored sampling' [46, 47] for variance reduction where the Gaussian noise vector ϵ is instantiated with pairs of perturbations ϵ , -ϵ .

Given the reference point θ ′ = θ -η ∇ L ( θ ) where η is the learning rate in the BP-based gradient update and ∇ L ( θ ) represents the gradient derived from the data, we define the fitness function f ( θ ) to measure the similarity between the model parameters θ and θ ′ : f ( θ ) = -|| θ -θ ′ || 2 2 . This choice ensures that the gradient of the expectation, ∇ θ E ϵ ∼N (0 ,I ) [ f ( θ + σϵ )] , aligns with the actual gradient ∇ L ( θ ) , effectively encoding the gradient information in the fitness values:

where the first equality simply follows from the assumption that ϵ is zero-mean. The visual representation of this process is illustrated in Fig. 4. Eq. 2 gives θ ′ = θ -1 2 ∇ θ E ϵ [ -|| ( θ + σϵ ) -θ ′ || 2 2 ] , and comparing with the BP operation θ ′ = θ -η ∇ L ( θ ) , we have

Now also utilizing Eq. 1, we write

𝜃𝜃

𝑡𝑡

,

𝑗𝑗

𝑖𝑖

𝜃𝜃

𝑡𝑡

,

𝑗𝑗

𝑖𝑖

𝑛𝑛

𝜃𝜃

𝑡𝑡

,

𝑗𝑗

′

𝑓𝑓

(

𝜃𝜃

𝑡𝑡

,

𝑗𝑗

𝑖𝑖

𝑛𝑛

)

𝜃𝜃

𝑡𝑡+1

,

𝑗𝑗

𝑖𝑖

Upload

f

(

𝜃𝜃

𝑡𝑡

,

𝑗𝑗

)

Download

𝐹𝐹

𝑡𝑡

𝜃𝜃

𝑡𝑡

,

𝑗𝑗

𝑖𝑖

𝑛𝑛

𝐹𝐹

𝑡𝑡

[

𝑛𝑛

]

𝜃𝜃

𝑡𝑡+1

,

𝑗𝑗

𝑖𝑖

𝑛𝑛

f

𝜃𝜃

𝑡𝑡

𝑛𝑛

𝜃𝜃

𝑡𝑡

,

𝑗𝑗

𝑖𝑖

𝑛𝑛

f

𝜃𝜃

𝑡𝑡

,

𝑗𝑗

|

𝑗𝑗

= 1

f

𝜃𝜃

𝑡𝑡

,

𝑗𝑗

|

𝑗𝑗

= 2

f

𝜃𝜃

𝑡𝑡

,

𝑗𝑗

|

𝑗𝑗

=

𝑀𝑀

Client

𝒋𝒋

Server

f

𝜃𝜃

𝑡𝑡

,

𝑗𝑗

𝑛𝑛

𝑗𝑗

= 1,2, . . ,

𝑀𝑀

→ 𝐹𝐹𝑡𝑡

[

𝑛𝑛

]

①

②

Evaluate

③

④

Aggregate

⑤

⑥

Update

⑦

Reconstruct

𝜃𝜃

𝑡𝑡

,

𝑗𝑗

𝑖𝑖

𝑛𝑛

: model parameter of

𝑛𝑛

𝑡𝑡𝑡

partition

f

𝜃𝜃

𝑛𝑛

: fitness of

𝑛𝑛

𝑡𝑡𝑡

partition

𝑡𝑡

,

𝑗𝑗

[ ⋅ ] : Indexing operation, 𝐼𝐼 𝑛𝑛 : indices of 𝑛𝑛 𝑡𝑡𝑡 partition where f is the specific distance-based fitness function defined above. Approximating the expectation by sampling, we further write

which shows how the gradient update information can be encoded using the fitness values f ( θ + σϵ i ) corresponding to the perturbations ϵ i . Consequently, this removes the need to evaluate each perturbation with the dataset, as is done in existing ES, and the entire update process is encoded with low-cost distance measurement.

Finally, the update on θ itself based on the fitness values is naturally given by

allowing a remote model update based on the transmitted f ( θ + σϵ i ) values with the shared knowledge of the ϵ i values.

The implemented algorithm consistently executes three steps: (i) Compute the target θ ′ through gradient descent, (ii) Implement perturbations to the model parameters and assess the perturbed parameters by computing their Euclidean distance to θ ′ , and (iii) Utilize the assessment results and encode the gradient with the fitness measures.

Partitioning. In the context of PBGE, the fitness function indicates the model distance, not the model performance on client data. This feature enables a unique partition-based approach for handling a large number of model parameters. Here, the parameters, flattened into a vector θ , are split into K partitions: θ [1] , θ [2] , ..., θ [ K ] . Each partition is then effectively encoded individually using PBGE, which is advantageous when working with large models, as storing a large population of those may be burdensome for clients with limited memory. Partitioning allows us to compute K fitness values per each perturbation, providing K times more reference points to encode the gradient information given a population size. Hence, even with a small population size (requiring less memory), we achieve robust performance as validated by the empirical results provided in Supplementary Materials. Essentially, partitioning provides a tradeoff of memory with communication as more fitness values now need to be communicated per each perturbation. This partitioning process is visualized in Fig. 5. Notably, this partitioning technique can be used with any model architecture, regardless of its specific design, providing a practical and efficient means of compressing large models.

4 EvoFed

EvoFed operates on the principle that the evolutionary update step depends only on the fitness values given the perturbation samples. In the FL context, we can leverage this characteristic to devise an accurate yet communication-efficient strategy for model updates. This section provides a detailed exposition of our methodology, breaking it down into stages for clarity. This iterative process, as outlined in Algorithm 1, aims to gradually converge the model parameters at all nodes to an optimal solution while minimizing data transmission during each update. An overall view of our proposed methodology is depicted in Fig. 3.

GradientSignal,

Contour of local loss

𝜃𝜃

𝑡𝑡

𝜃𝜃

𝑡𝑡

,

𝑗𝑗

′

𝑓𝑓

𝑡𝑡

,

𝑗𝑗

1

=

-𝑑𝑑

(

𝜃𝜃

𝑡𝑡

1

,

𝜃𝜃

𝑡𝑡

,

𝑗𝑗

′

)

Upload

𝐟𝐟

𝑡𝑡

,

𝑗𝑗

Download

𝐹𝐹

𝑡𝑡

𝜃𝜃

𝑡𝑡

5

𝜃𝜃

𝑡𝑡

3

𝜃𝜃

𝑡𝑡

6

𝜃𝜃

𝑡𝑡

1

𝜃𝜃

𝑡𝑡

2

𝜃𝜃

𝑡𝑡

9

𝜃𝜃

𝑡𝑡

8

𝜃𝜃

𝑡𝑡

7

𝜃𝜃

𝑡𝑡

4

Model Update

𝜃𝜃

𝑡𝑡+1

𝜃𝜃

𝑡𝑡

5

𝜃𝜃

𝑡𝑡

3

𝜃𝜃

𝑡𝑡

6

𝜃𝜃

𝑡𝑡

1

𝜃𝜃

𝑡𝑡

2

𝜃𝜃

𝑡𝑡

9

𝜃𝜃

𝑡𝑡

8

𝜃𝜃

𝑡𝑡

7

𝜃𝜃

𝑡𝑡

4

𝜃𝜃

𝑡𝑡

𝜃𝜃

𝑡𝑡+1

=

𝜃𝜃

𝑡𝑡

+

𝛼𝛼

9⋅𝜎𝜎

Σ𝑖𝑖=1

9

𝐹𝐹

𝑡𝑡

𝑖𝑖

⋅ 𝜎𝜎

𝑡𝑡

𝑖𝑖

Server

𝐟𝐟

𝑡𝑡

,

𝑗𝑗

𝑗𝑗=1

𝑀𝑀

→𝐹𝐹𝑡𝑡

Aggregate

L

j

(

𝜃𝜃

𝑡𝑡

)

Contour of aggregated fitness

𝐹𝐹

𝑡𝑡

4.1 Initial Setup

The initialization of the server and clients in EvoFed begins with the same baseline model, denoted by θ 0 . A key assumption is that the server and all clients share the same seed (e.g., via broadcasting) for identical random population generation. This approach ensures consistent baseline models and populations across all nodes.

In this step, a population of candidate solutions (in this case, models) is generated by adding random perturbations to the current best solution (the baseline model). The generation of an i -th member of the model population can be formulated as follows:

Here, N (0 , σI ) represents the perturbations sampled from a multivariate normal distribution with zero mean and a shared covariance matrix σI . We also denote the population at each node at any given time t as P t = { θ 1 t , θ 2 t , ..., θ N t } , where θ i t represents the i -th model in the population generated by adding the i -th perturbation to the baseline model parameters, and N is the size of the population.

4.2 Local Updates and Fitness Evaluation

Each client node begins by executing BP on its local dataset, using the baseline model, θ t , resulting in an updated model, θ ′ t . Following this, the fitness of each member θ i t in the local population, P t , is evaluated. This evaluation is done by measuring the similarity between θ ′ t and θ i t . The L2 norm or Euclidean distance serves as the measure of this similarity. The fitness of θ i t is represented as f ( θ i t ) :

The process of local update and fitness evaluation is illustrated in Fig. 6. The fitness values are the only information that needs to be communicated among nodes. The fitness vectors are significantly smaller in size compared to the model parameter vectors, which helps reduce the communication overhead. Hence, each client sends a fitness vector, f t = { f ( θ 1 t ) , f ( θ 2 t ) , ..., f ( θ N t ) } corresponding to all population members, to the server.

4.3 Server-side Aggregation and Update

The server's responsibility is to aggregate the fitness values reported by all client nodes, forming a global fitness vector F t comprising N elements, with F i t representing the fitness value of the i th member of the population. Each client's contribution to F t is weighted by their respective batch size b j , giving a larger influence to clients with potentially more accurate local updates.

The global fitness vector F t is computed as follows:

where M is the total number of clients and f t,j is the fitness value vector from client j at time t . After the aggregation, the server broadcasts F t so that each client updates the baseline model θ t .

4.4 Broadcasting Fitness

Once the aggregation is complete, the server broadcasts F t to all client nodes, maintaining synchronicity across the network. This only involves the transmission of the fitness vector, again reducing communication overhead. The aggregated fitness vector can be used by the local clients (as well as by the server, if needed) to update the local models.

4.5 Client-side Update

When the global fitness vector F t is received from the server, the clients can update their local model following Eq. 5. This strategy involves the addition of a weighted sum of noise vectors to the current model parameters, where the weights are aggregated fitness values:

where ϵ i t is the i -th noise vector from the population at time t .

Notably, the right side of Eq. 9 can be shown equivalent to the average of all locally updated models akin to FedAvg, i.e., it is straightforward to show that

where θ t +1 ,j is the updated model of client j at time t +1 defined as

4.6 Convergence Analysis

Theorem 1. Suppose that L j ( θ ) is the β -smooth function, i.e., ∥∇ L j ( u ) -∇ L j ( v ) ∥ ≤ β ∥ u -v ∥ for any u, v , and also suppose that the variance of the stochastic gradient of D j is bounded, i.e., E ∥∇ L j ( θ ) -˜ ∇ L j ( θ ) ∥ 2 ≤ B 2 for all j . When perturbation ϵ i is sampled, a conditioned mirrored sampling is applied such that 1 N ∑ N i =1 ϵ i = 0 , 1 N ∑ N i =1 ( ϵ i ) 2 ≤ G 2 , 1 N ∑ N i =1 ( ϵ i ) 3 = 0 . Given a decreasing learning rate η t < 1 4 αβ , EvoFed converges in the sense of

Upload

E(f

j

)

Upload

E(

𝐹𝐹

)

Download

𝐹𝐹

Download

𝐹𝐹

Encryption

𝐟𝐟𝑗𝑗 →

E(f

j

)

Client

Aggregation

E(

𝐟𝐟

𝑗𝑗

)

𝑗𝑗=1

𝑀𝑀

→

E(

𝐹𝐹

)

Server

Decryption

E

𝐹𝐹

→ 𝐹𝐹

TEEs

where H T = ∑ T -1 t =0 η t , and L ∗ represents the minimum value of L ( θ ) .

Given a decreasing learning rate (e.g., η t = η 0 1+ t ), it can be seen that H T = ∑ T -1 t =0 η t → ∞ as T increases, while ∑ T -1 t =0 η 2 t < ∞ . Consequently, the upper bound stated in Theorem 1 approaches 0 as T grows, ensuring convergence towards a stationary point. The detailed discussions and the proof can be found in Supplementary Materials.

4.7 EvoFed and Privacy Enhancements

Unlike other FL methods, EvoFed operates on fitness measures. Encryption on smaller fitness vectors requires a lower overhead compared to encryption on large model parameter vectors. Fig. 7 shows Fully Homomorphic Encryption (FHE) [48-50] that allows aggregation of encrypted data on the server while keeping individual client data confidential. For the decryption of the aggregated fitness vector, EvoFed can leverage third-party Trusted Execution Environments (TEEs), such as Intel SGX [51], providing a secure space for sensitive computations.

4.8 Partial Client Participation

When FL has to operate with a large client pool, a typical practice is to select only a subset of clients in each global round. In EvoFed, this means that the newly joined clients in a given round can either download the latest model or else the last k fitness vectors from the server, where k is the time gap from the last participation. In the latter case, the model updates from Eq. (9) can be modified to accommodate k -step updates:

Note that even in the former case where the latest model is downloaded by the newly joined clients, the level of security is not compromised as far as the client-to-server messages are concerned.

5 Experiments

Our algorithm's effectiveness is assessed on three image classification datasets: FMNIST [52], MNIST [53], and CIFAR-10 [54]. Both MNIST and FMNIST contain 60,000 training samples and 10,000 test samples, whereas CIFAR-10 is composed of 50,000 training samples and 10,000 test samples. We employ a CNN model having 11k parameters for the MNIST and FMNIST datasets and a more substantial model with 2.3M parameters for CIFAR-10. A grid search has been conducted to identify the optimal performance hyperparameters for each baseline, as outlined in the results section. We take into account both global accuracy and communication costs to ascertain hyperparameters that maximize accuracy while minimizing communication overhead.

Data Distribution. We distribute the training set of each dataset among clients for model training, and the performance of the final global model is evaluated using the original test set. Our experimental setup contains M = 5 clients with non-IID data distribution (assigning two classes to each client).

Implementation Details. Our EvoFed framework is built using JAX [55], which facilitates extensive parallelization and, in particular, consistent random number generation across a large number of nodes. We have implemented our framework on the Evosax [56] library, a convenient tool for the ES algorithm. EvoFed is configured with a population size of 128 and a mini-batch size of 256 for MNIST / FMNIST and 64 for CIFAR-10. We perform ten local epochs (performing ten BP steps before fitness calculation) and train over 1,000 global rounds.

Baselines. We compare the performance of the proposed EvoFed with BP, FedAvg, ES, FedAvg with quantization (Fed-quant), and FedAvg with Sparsification (Fed-sparse). In each scenario, we push

0

200

400

600

800

1000

Rounds

0

20

40

60

80

100

Global Accuracy(%)

MNIST

BP

ES(98.8%)

FedAvg

Fed-quant(75%)

Fed-sparse(98.8%)

EvoFed(98.8%)

0

200

400

600

800

1000

Rounds

0

20

40

60

80

100

Global Accuracy(%)

FMNIST

BP

ES(98.8%)

FedAvg

Fed-quant(75%)

Fed-sparse(98.8%)

EvoFed(98.8%)

0

200

400

600

800

1000

Epochs

0

10

20

30

40

50

60

70

Test accuracy(%)

CIFAR-10

BP

ES(99.9%)

FedAvg

Fed-quant(75%)

Fed-sparse(98.8%)

EvoFed(98.8%)

(a) MNIST Accuracy

(b) FMNIST Accuracy

(c) CIFAR-10 Accuracy

0

20

40

60

80

100

Global Accuracy(%)

10 -2

10 0

Communication(Log Scale)

MNIST

ES(98.8%)

FedAvg

Fed-quant(75%)

Fed-sparse(98.8%)

EvoFed(98.8%)

0

20

40

60

80

Global Accuracy(%)

10 -2

10 0

Communication(Log Scale)

FMNIST

ES(98.8%)

FedAvg

Fed-quant(75%)

Fed-sparse(98.8%)

EvoFed(98.8%)

0

10

20

30

40

50

Global Accuracy(%)

10 -4

10 -2

10 0

Communication(Log Scale)

CIFAR-10

ES(99.9%)

FedAvg

Fed-quant(75%)

Fed-sparse(98.8%)

EvoFed(98.8%)

(d) MNIST Comm. cost

(e) FMNIST Comm. cost

(f) CIFAR-10 Comm. cost

for maximum compression, stopping right before the model starts to show performance degradation relative to FedAvg with no compression. BP provides the upper-performance baseline, while ES serves as a reference emphasizing the significance of PBGE.

6 Results and Discussions

In this section, we discuss the experimental results in detail and provide further insights into the performance of EvoFed. The accuracy of EvoFed, compared with multiple baseline methods and different datasets, is shown in Fig. 8 (a), (b), and (c). Efficiently encoding and exchanging gradient information, EvoFed enhances the effectiveness of the ES algorithm across all tasks, delivering results comparable to FedAvg. Also, EvoFed achieves superior accuracy at an equivalent compression rate compared to sparsification. This suggests that utilizing a shared population of samples can reduce the information necessary for gradient compression, thereby enhancing the efficiency of the process. Fig. 8 (d), (e), and (f) shows the performance of each method as a function of communication load for all three datasets. It can be seen that EvoFed tends to utilize significantly less communication resources as compared to other high-accuracy techniques.

Table 1 summarizes the performance of different schemes on MNIST, FMNIST, and CIFAR-10 datasets, focusing on communication cost and accuracy. EvoFed achieves significantly lower communication costs compared to FedAvg while maintaining competitive accuracy levels. In the MNIST dataset, EvoFed achieves an accuracy of 97.62% with a mere 9.2 MB of communication load, while FedAvg achieves 98.09% accuracy at a considerably high communication cost of 73.7 MB. The effective compression achieved is an impressive 98.8% which indicates that the gradient vector is condensed into just 1.2% of the fitness vector that is communicated between clients and the server. Similarly, for the FMNIST dataset, EvoFed achieves an accuracy of 84.72% with only 7.78 MB of communication, whereas FedAvg's accuracy is 85.53% with a communication cost of 40.99 MB. The efficiency of EvoFed becomes even more apparent in the CIFAR-10 dataset where the model compression is over 99.7%. EvoFed achieves an accuracy of 54.12% with a low communication cost of only 0.023 GB, surpassing FedAvg, which performs 50.22% at a communication cost of 2.134 GB. The simpler ES method actually gives better performance as well as higher communication efficiency than EvoFed for MNIST but its accuracy for other data sets is highly limited.

Additional Experiments . Fig. 9 illustrates the performance of EvoFed with varying numbers of samples N within the population and a varying number of clients M . The model's performance is

significantly influenced by the population size, which directly affects the algorithm's exploration capability and compression rate. We observe a performance improvement as we increase the population size, although at the cost of increased computation and memory usage. Nonetheless, this improvement is not linear and plateaus once sufficient samples are generated to explore the parameter space. For instance, in the case of the FMNIST dataset and a model with 11K parameters, any performance enhancement beyond a generated sample size of 128 is marginal.

Fig. 9(b) showcases EvoFed's performance trend as the number of clients increases. A minor performance decline is observable for some larger values of M relative to M=5. This decline could potentially be attributed to the limited data available per client and the increased variance associated with local training. Nonetheless, we believe that carefully tuning the hyperparameters based on data distribution could assist in achieving more robust performance.

0

200

400

600

800

1000

Rounds

0

20

40

60

80

100

Accuracy (%)

Effect of Population Size on EvoFed

N=4096

N=2048

N=512

N=256

N=128

N=64

N=32

N=16

N=12

N=8

0

200

400

600

800

1000

Rounds

0

20

40

60

80

100

Accuracy (%)

Changing number of Clients

M=100

M=50

M=10

M=5

(a) Varying number of population from 8 to 4096

(b) Varying number of clients from 5 to 100

Supplementary Materials provide further details regarding hyperparameters, model architectures, and experiments. We also include a detailed ablation study with different evolutionary strategies, the population size, and the number of partitions, as well as detailed communication and computational complexity analyses.

7 Conclusion

EvoFed, the novel paradigm presented herein, unites FL and ES to offer an efficient decentralized machine learning strategy. By exchanging compact fitness values instead of extensive model parameters, EvoFed significantly curtails communication costs. Its performance parallels FedAvg while demonstrating an impressive equivalent model compression of over 98.8% on FMNIST and 99.7% on CIFAR-10 in representative experimental settings. Consequently, EvoFed represents a substantial stride forward in FL, successfully tackling the critical issue of high communication overhead.

Acknowledgments

A special acknowledgment goes to Do-yeon Kim for the valuable discussions on the convergence analysis presented in Section 4. This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (No. NRF-2019R1I1A2A02061135), and by IITP funds from MSIT of Korea (No. 2020-0-00626).

References

Jakub Koneˇ cn` y, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492, 2016.

HBrendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. Communication-efficient learning of deep networks from decentralized data. arXiv preprint arXiv:1602.05629, 2016.

Jianyu Wang, Ananda Theertha Yu, and Gregory Wornell. Federated learning with matched averaging. arXiv preprint arXiv:2002.06440, 2020.

Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. Qsgd: Communication-efficient sgd via gradient quantization and encoding. In Advances in Neural Information Processing Systems, pages 1709-1720, 2017.

Yuzhu Mao, Zihao Zhao, Guangfeng Yan, Yang Liu, Tian Lan, Linqi Song, and Wenbo Ding. Communication-efficient federated learning with adaptive quantization. ACM Transactions on Intelligent Systems and Technology (TIST), 13(4):1-26, 2022.

Sebastian U Stich. Local sgd converges fast and communicates little. arXiv preprint arXiv:1805.09767, 2018.

Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar. Federated multi-task learning. In Advances in Neural Information Processing Systems, pages 4424-4434, 2017.

Felix Sattler, Simon Wiedemann, Klaus-Robert Müller, and Wojciech Samek. Robust and communication-efficient federated learning from non-iid data. In IEEE transactions on neural networks and learning systems, 2019.

Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Fair resource allocation in federated learning. arXiv preprint arXiv:1905.10497, 2020.

Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Reinforcement learning, pages 5-32, 1992.

Frank Sehnke, Christian Osendorfer, Thomas Rückstieß, Alex Graves, Jan Peters, and Jürgen Schmidhuber. Parameter-exploring policy gradients. Neural Networks, 23(4):551-559, 2010.

Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jan Peters, and Jürgen Schmidhuber. Natural evolution strategies. The Journal of Machine Learning Research, 15(1):949-980, 2014.

Xin Yao. Evolving artificial neural networks. Proceedings of the IEEE, 87(9):1423-1447, 1999.

Joel Lehman and Risto Miikkulainen. Neuroevolution. Scholarpedia, 8(6):30977, 2013.

Sebastian Risi and Julian Togelius. Neuroevolution in games: State of the art and open challenges. CoRR, abs/1410.7326, 2014.

Sebastian Risi and Julian Togelius. Neuroevolution in games: State of the art and open challenges. IEEE Transactions on Computational Intelligence and AI in Games, 9(1):25-41, 2015.

Daniel Golovin, John Karro, Greg Kochanski, Chansoo Lee, Xingyou Song, and Qiuyi Zhang. Gradientless descent: High-dimensional zeroth-order optimization. arXiv preprint arXiv:1911.06317, 2019.

Robert Tjarko Lange, Tom Schaul, Yutian Chen, Chris Lu, Tom Zahavy, Valentin Dalibard, and Sebastian Flennerhag. Discovering attention-based genetic algorithms via meta-black-box optimization. arXiv preprint arXiv:2304.03995, 2023.

Felix Sattler, Simon Wiedemann, Klaus-Robert Müller, and Wojciech Samek. Sparse binary compression: Towards distributed deep learning with minimal communication. In 2019 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2019.

Jinjin Xu, Wenli Du, Yaochu Jin, Wangli He, and Ran Cheng. Ternary compression for communication-efficient federated learning. IEEE Transactions on Neural Networks and Learning Systems, 2020.

Eunjeong Jeong, Seungeun Oh, Hyesung Kim, Jihong Park, Mehdi Bennis, and Seong-Lyun Kim. Communication-efficient on-device machine learning: Federated distillation and augmentation under non-iid private data. arXiv preprint arXiv:1811.11479, 2018.

Sohei Itahara, Takayuki Nishio, Yusuke Koda, Masahiro Morikura, and Koji Yamamoto. Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-iid private data. arXiv preprint arXiv:2008.06180, 2020.

Felix Sattler, Arturo Marban, Roman Rischke, and Wojciech Samek. Communication-efficient federated distillation. arXiv preprint arXiv:2012.00632, 2020.

Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. Ensemble distillation for robust model fusion in federated learning. Advances in Neural Information Processing Systems, 33: 2351-2363, 2020.

Chuhan Wu, Fangzhao Wu, Lingjuan Lyu, Yongfeng Huang, and Xing Xie. Communicationefficient federated learning via knowledge distillation. Nature communications, 13(1):1-8, 2022.

Mingzhe Chen, Nir Shlezinger, H Vincent Poor, Yonina C Eldar, and Shuguang Cui. Communication-efficient federated learning. Proceedings of the National Academy of Sciences, 118(17):e2024789118, 2021.

Takayuki Nishio and Ryo Yonetani. Client selection for federated learning with heterogeneous resources in mobile edge. In ICC 2019-2019 IEEE international conference on communications (ICC), pages 1-7. IEEE, 2019.

Canh T Dinh, Nguyen H Tran, Minh NH Nguyen, Choong Seon Hong, Wei Bao, Albert Y Zomaya, and Vincent Gramoli. Federated learning over wireless networks: Convergence analysis and resource allocation. IEEE/ACM Transactions on Networking, 29(1):398-409, 2020.

Ingo Rechenberg. Evolutionsstrategie : Optimierung technischer systeme nach prinzipien der biologischen evolution. 1973.

Daan Wierstra, Tom Schaul, Jan Peters, and Jürgen Schmidhuber. Fitness expectation maximization. In Parallel Problem Solving from Nature-PPSN X: 10th International Conference, Dortmund, Germany, September 13-17, 2008. Proceedings 10, pages 337-346. Springer, 2008.

Sun Yi, Daan Wierstra, Tom Schaul, and Jürgen Schmidhuber. Stochastic search using the natural gradient. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 1161-1168, 2009.

Yi Sun, Daan Wierstra, Tom Schaul, and Jürgen Schmidhuber. Efficient natural evolution strategies. In Proceedings of the 11th Annual conference on Genetic and evolutionary computation, pages 539-546, 2009.

Tobias Glasmachers, Tom Schaul, and Jürgen Schmidhuber. A natural evolution strategy for multi-objective optimization. In Parallel Problem Solving from Nature, PPSN XI: 11th International Conference, Kraków, Poland, September 11-15, 2010, Proceedings, Part I 11, pages 627-636. Springer, 2010.

Tobias Glasmachers, Tom Schaul, Sun Yi, Daan Wierstra, and Jürgen Schmidhuber. Exponential natural evolution strategies. In Proceedings of the 12th annual conference on Genetic and evolutionary computation, pages 393-400, 2010.

Tom Schaul, Tobias Glasmachers, and Jürgen Schmidhuber. High dimensions and heavy tails for natural evolution strategies. In Proceedings of the 13th annual conference on Genetic and evolutionary computation, pages 845-852, 2011.

Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864, 2017.

James C Spall. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE transactions on automatic control, 37(3):332-341, 1992.

Yurii Nesterov and Vladimir Spokoiny. Random gradient-free minimization of convex functions. Foundations of Computational Mathematics, 17:527-566, 2017.

W. Diffie and M. Hellman. New directions in cryptography. IEEE Transactions on Information Theory, 22(6):644-654, 1976. doi: 10.1109/TIT.1976.1055638.

Hangyu Zhu and Yaochu Jin. Real-time federated evolutionary neural architecture search. IEEE Transactions on Evolutionary Computation, 26(2):364-378, 2021.

Decebal Constantin Mocanu, Elena Mocanu, Peter Stone, Phuong H Nguyen, Madeleine Gibescu, and Antonio Liotta. Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science. Nature communications, 9(1):1-12, 2018.

Hangyu Zhu and Yaochu Jin. Multi-objective evolutionary federated learning. IEEE transactions on neural networks and learning systems, 31(4):1310-1322, 2019.

Zheng-yi Chai, Chuan-dong Yang, and Ya-lun Li. Communication efficiency optimization in federated learning based on multi-objective evolutionary algorithm. Evolutionary Intelligence, pages 1-12, 2022.

Qingfu Zhang and Hui Li. Moea/d: A multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on evolutionary computation, 11(6):712-731, 2007.

Ivanoe De Falco, Antonio Della Cioppa, Tomas Koutny, Martin Ubl, Michal Krcma, Umberto Scafuri, and Ernesto Tarantino. A federated learning-inspired evolutionary algorithm: Application to glucose prediction. Sensors, 23(6):2957, 2023.

John Geweke. Antithetic acceleration of monte carlo integration in bayesian inference. Journal of Econometrics, 38(1-2):73-89, 1988.

Dimo Brockhoff, Anne Auger, Nikolaus Hansen, Dirk V Arnold, and Tim Hohm. Mirrored sampling and sequential selection for evolution strategies. In Parallel Problem Solving from Nature, PPSN XI: 11th International Conference, Kraków, Poland, September 11-15, 2010, Proceedings, Part I 11, pages 11-21. Springer, 2010.

Masahiro Yagisawa. Fully homomorphic encryption without bootstrapping. Cryptology ePrint Archive, 2015.

Hao Chen, Kim Laine, and Rachel Player. Simple encrypted arithmetic library-seal v2. 1. In Financial Cryptography and Data Security: FC 2017 International Workshops, WAHC, BITCOIN, VOTING, WTSC, and TA, Sliema, Malta, April 7, 2017, Revised Selected Papers 21, pages 3-18. Springer, 2017.

Junfeng Fan and Frederik Vercauteren. Somewhat practical fully homomorphic encryption. Cryptology ePrint Archive, 2012.

Ittai Anati, Shay Gueron, Simon Johnson, and Vincent Scarlata. Innovative technology for cpu based attestation and sealing. In Proceedings of the 2nd international workshop on hardware and architectural support for security and privacy, volume 13. ACM New York, NY, USA, 2013.

Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.

James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http://github.com/google/jax .

Robert Tjarko Lange. evosax: Jax-based evolution strategies. arXiv preprint arXiv:2212.04180, 2022.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper solves the huge communication burden problem in federated learning (FL). This paper proposes an evolutionary strategy (ES) based way to address the communication burden. Specifically, the ES only needs to transmit the fitness values among clients and server, and the fitness values could reflect the model update direction. The proposed EvoFed could compress the model more than 98.8% on FMNIST and 99.7% on CIFAR-10, nearly reducing all communication.\n\nSTRENGTHS:\n1. The proposed EvoFed is novel and interesting. Instead of transmitting models or model updates, the authors sample several model points with noise perturbation and then calculate fitness values which could reflect the gradient directions. Transmitting fitness values could save lots of communication cost.\n2. The paper is sounded and theoretically supported. EvoFed is motivated by the population-based gradient estimation, and Equation 4 could derive the plausibility of utilizing the distance function as the fitness value function. Additionally, this paper also provide the convergence analysis to show the learning speed of population-based ES. According to Theorem 1, if the population size M is large enough, the second term of RHS could be sufficiently small.\n3. Privacy enhancement via TEEs are considered.\n4. The supp. provides abundant experimental studies including the population sizes and the partition ways.\n\nWEAKNESSES:\n1. The paper only studies small datasets such as FMNIST and CIFAR-10. These two benchmarks are easy to obtain good performances. More complex datasets such as CIFAR-100 or benchmarks in FedML should be investigated. On complex datasets, the reviewer thinks that a population size of 32 or 128 could not lead to qualified accuracy compared with FedAvg.\n2. The reviewer sees that the local update epoch is one in section 4.2. However, updating local models up to more epochs should be studied and reported. If the local epoch becomes larger, the reviewer thinks that the population size should be larger.\n\nQUESTIONS:\n1. Some important experimental studies should be moved to the main body, e.g., the ablation study of population size, which is a significant factor in EvoFed.\n\nLIMITATIONS:\nThe authors provide limitations and social impacts.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to utilize a variant of evolution strategy in federated learning framework in order to reduce its communication cost while maintaining the accuracy. The proposed approach, EvoFed, is mainly based on so called OpenAI-ES, a variant of evolution strategy intended to be used for policy optimization on decentralized environment. Differently from OpenAI-ES, which communicates the fitness of each candidate solution (cumulative reward in RL setting), EvoFed lets clients perform stochastic gradient descent locally with their local dataset and the fitness of candidate solutions are assigned based on the distance to the locally-updated parameter value. In this way, the gradient information is utilized and it is expected to improve the efficiency of the training, which is one of the most important disadvantages of evolution strategies. The authors performed a theoretical analysis of the proposed algorithm as well as empirical comparison with some baseline approaches on MNIST, FMNIST, and CIFAR10.\n\nSTRENGTHS:\n* a novel and promising application of evolution strategies to federated learning framework to reduce the communication cost.\n* a novel fitness evaluation scheme for evolution strategies so that they can utilize stochastic gradient\n* empirical results showing performance comparative to FedAvg, a standard federated learning approach, with reduced communication cost.\n\nWEAKNESSES:\n* limited empirical evaluation: both the image size and the network size are relatively small. \n* theoretical bounds provided in the paper doesn’t probably reflect the reality.\n* limited technical novelty\n\nQUESTIONS:\n1. (related to the 1st weakness pointed out above) the experiments are done on small image classification tasks. Because it was not clearly stated in the introduction what kind of tasks are considered as target tasks of this work, I couldn’t confirm whether these experiments reflects the target of FL. Moreover, the network size is relatively small. For MNIST and FMNIST, the number of parameters of a used CNN is only 11k. For CIFAR-10, it is increased to 2.3M, but the accuracy of the model trained by the standard back propagation was only about 70 %. It is interesting to see whether the proposed approach is still effective for greater models, where the communication cost is more important.\n\n2. (related to the 2nd and 3rd weaknesses pointed out above) Related to Theorem 1, there are several questions. \n2.1. It seems the statement in Theorem 1 contains some mistakes. In delta_M, i appears but delta_M is used as if it is independent of i. Is the summation or something is missing? Moreover, theta in delta_M is not defined. Is it theta_t? If so, is the summation or something is missing? Because of these points, it was not clear what is the point of the theorem.\n2.2. Because of the above point, I am not sure if I understand it correctly. But it looks like that it doesn’t really implies the convergence as delta_M is irrelevant to T. Having a large M implies more communication cost. The intension of this Theorem should be more clearly explained in the main text.\n2.3. The standard deviation is supposed to be proportional to the square root of T to derive the bound. If T is large, it means that the candidate solutions are far from the current center of distribution. In this situation, I coudn’t understand why it makes sense to define the fitness of the candidate solutions by the distance from the updated solution. This point should also be explained.\n2.4. What is eta? The learning rate for SGD to get theta’? Please explain it.\n2.5. Immediately after Theorem 1, the authors say “the appropriate value of alpha is 0.5 by the aforementioned reasoning.” However, I couldn’t get which part is refereed to.\n\n3. The fitness is sometimes defined by the Euclidean norm, and sometimes its square. From the derivation I think it should be squared. I found this inconsistency in Eq (3) and Eq (6) and Algorithm 1. Please check this point carefully.\n\n4. In Eq (7) b_j is introduced, but it is not used later in the paper. It seems that they are implicitly assumed to be equal. Please state it.\n\n5. Figure 8.d, e, f are visible only for high accuracy part. I suggest to show these graph in log-scale so that the differences are visible also for the low accuracy part.\n\nLIMITATIONS:\nNo limitation is explained in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper offers a solution to a problem in federated learning: it's expensive to pass large model parameters back and forth from the server to the local/client nodes who train with their own data. The solution depends on the same random number generator and seed in every local node plus complete synchronization of model parameter updates. Rather than pass the model parms, distance measures (aka fitness scores) between  a population of perturbed models and a locally trained model are transmitted to the server, then aggregated and passed back to the clients who update the model parms locally.  This local (or server) model parameter update is the crux of the algorithm. The sum of all clients' distances for one perturbation model is tweaked according to an ES covariance matrix of perturbations  and added to the existing parameter with normalization and two  factors which are the learning rate parameter and inverse of the noise vector. This is repeated for every perturbation model in the local population.  This is called fitness-based information sharing and the method is called \"Population-Based Gradient Estimation'.  The population of perturbations within an Evolutionary Strategy must be carefully chosen in terms of variance (sigma) and size (M).  Because every local node references the same perturbed models and  the distances reflect the local data at each node, there is a claim of nearly full information encoding while the transmission cost of the distances is much smaller that then number of model parms.  The depth of the model update claim comes from showing that the summed differences of each client perturbation model  *and*, summed over M perturbation models,  approximates a gradient parallel to the gradient trained by back propagation.  The ES handles the accurate addition of noise relative to the perturbations and their variance. A privacy enhancement through Fully Homomorphic Encryption and the clients being TEEs/Trusted Execution Environments is also provided.\n\nSTRENGTHS:\nThis is a form of adaptive compression which is a great way to solve a communications overload problem. Most approaches transmit the updates to the model or gradient signals.  EvoFEd outperforms traditional compression methods. The L2 norm distance calculation is simple and so is the server-side batchsize-weighted aggregation and update. The client-side update of the model references noise vectors from the populations.  The integration with fully homomorphic encryption is good.\n\nWEAKNESSES:\nIt's not entirely clear from the paper in what types of fitness landscape this will work. And, it's not clear why a parallel gradient which is then perturbed is going to lead to *optimal* convergence. \n A Natural Evolutionary Strategy is used to generate perturbations of the model's parms and the newly perturbed models are then used as benchmarks to measure a distance that reveals the gradient of one model that is trained with back propagation. The stochastic gradient ascent of NES is discarded. More information on the factored Gaussian parameter distribution that motivates its selection would be helpful.  An explanation of why the NES was chosen over other ES would improve clarity.  \nIt's also not clear how to set any of the parameters in the NES, ie population (M) and variance.\n\nThis method requires more synchronization than a federated learning model in general.\n\nQUESTIONS:\nIs mirrored sampling important?\nWhat defines convergence?\nWhat role does batch size play in the algorithm's efficiency? Are clients expected to draw different batches after each update from the server?  \nHow are parameters chosen?\n\nLIMITATIONS:\nno limitation in this scope", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new method called EvoFed, which combines Evolutionary Strategies (ES) with Federated Learning to address the challenge of high communication costs in the traditional FL setting. Instead of sending the actual model parameters, each device sends a measure of how similar its updated model is to a set of perturbed models. These perturbed models are generated in the same way on every device and server. By carefully choosing the variance and size of the perturbed models, the similarity measures can carry almost all the information about the model parameters. Since the perturbed models are smaller in size, the communication load is reduced. The authors show results in a deep CNN network trained on MNIST, FMNIST and CIFAR-10, demonstrating that EvoFed performs as well as FedAvg and compresses the model by at least 98%.\n\nSTRENGTHS:\n1. The idea of using evolutionary schemes for model-sharing in FL is compelling and seems original.\n2. The work could have high impact (if evaluated thoroughly) and pave way for large-scale adoption of FL on edge applications.\n\nWEAKNESSES:\n1. The paper is difficult to follow at places. For instance, authors need to clarify their intuition for model partitioning in Section 3. How are the number of partitions chosen? Is this model-agnostic? If not, some empirical results or insights can help the reader reproduce the findings in their works.\n2. The paper has some hard to follow sentences and uses abbreviations that are not expanded prior (e.g., BP). A thorough proof-reading of the manuscript would be helpful.\n3. The evaluations are limited. The paper introduces a new paradigm for federated learning using evolutionary schemes which reduces communication costs significantly. In order to understand the true impact of this idea, it needs to be juxtaposed against some state-of-the-art pruning techniques for FL (e.g., Lin, Rongmei, Yonghui Xiao, Tien-Ju Yang, Ding Zhao, Li Xiong, Giovanni Motta, and Françoise Beaufays. \"Federated pruning: improving neural network efficiency with federated learning.\" arXiv preprint arXiv:2209.06359 (2022)).\n\nQUESTIONS:\nPlease see comments above in weakness.\n\nLIMITATIONS:\nThe paper could benefit from more discussion on the limitations of the proposed method. There is no obvious negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-09"}