Decentralized Learning with Random Walks and Communication-Efficient Adaptive Optimization
Abstract
We tackle the problem of federated learning (FL) in a peer-to-peer fashion without a central server.
While prior work mainly considered gossip-style protocols for learning, our solution is based on random walks.
This allows to communicate only to a single peer at a time, thereby reducing the total communication and enabling asynchronous execution.
To improve convergence and reduce the need for extensive tuning, we consider an adaptive optimization method – Adam.
Two extensions reduce its communication costs: state compression and multiple local updates on each client.
We theoretically analyze the convergence behaviour of the proposed algorithm and its modifications in the non-convex setting.
We show that our method can achieve performance comparable to centralized FL without communication overhead.
Empirical results are reported on a variety of tasks (vision, text), neural network architectures and large-scale federations (up to ∼ 342 k clients).
Introduction
Federated learning is typically set up with a central server coordinating the training process: selecting clients to participate in each round, sending the current model weights to the clients, and aggregating the received updates.
However, in some situations, FL with a central server is impractical due to, for example, costs, privacy concerns, or when peer-to-peer communication ( e g. , using Bluetooth) is preferred.
This paper tackles the problem of peer-to-peer learning in which users are connected via direct communication links into a potentially sparse communication graph.
They perform local updates to the model similarly to FL, but unlike in FL, the decision about what to do next and where to send the model is taken locally (by random selection among clients’ neighbors in the graph).
One example of the basis for the communication graph is routing times between users; this way, less time can be spent on model transfer and more on training in total.
Existing work has primarily focused on datacenter scenarios and gossip-style protocols for learning [11, 1], [12], [21, 22, 8] where messages are exchanged with all peers of a given client.
These methods have been shown to work successfully in data center settings to speed up training [1] and to address unreliable or expensive communication [22].
On the other hand, there is very limited literature on cross-device decentralized FL, and the scale of experiments is impractically small (< 100 clients)
We focus on a different style of algorithms – random walks – which allow communicating only to a single peer at a time.
They reduce total communication and enable asynchronous execution.
While such approaches have been considered before [2, 24, 3, 4], experimental results in FL are similarly limited.
We, therefore, aim to close this research gap with our paper.
Along with general learning, we consider the problem of adaptive optimization in the decentralized setting with random walks.
While adaptive optimization is straightforward to solve in centralized scenarios by using, for example, Adam [7] on the server side, naively applying it to the random walk setting leads to challenges.
The main issue is that the optimizer state (primarily, moments, each of which is equal in size to the model) needs to be transmitted along with the model, which can double or even triple communication costs.
We thus propose two strategies to mitigate this effect.
Firstly, we propose the use of compression for the optimizer’s state.
Secondly, we propose performing multiple local updates on each client before communicating to a neighbor.
We prove the convergence of such random walks to a vicinity of a stationary point in the non-convex setting and empirically show that they allow for similar communication vs. accuracy tradeoffs to traditional centralized FL. In summary, we make the following contributions in this work:
• we introduce Adam [7], a popular adaptive optimization method, in the decentralized setting with random walks.
This allows us to improve performance upon traditional random walk SGD-type of approaches, especially in complex and sparse gradient tasks; • we introduce two communication-efficient variants of Adam for this specific setting.
More specifically, we introduce a novel moment quantization method in the log domain and adopt the multiple local updates paradigm, popularized by [15]; • we provide a theoretical analysis of the convergence behavior of this novel random-walk Adam with our proposed modifications.
We show that it has similar rates to what traditional adaptive methods [25] obtain in the centralized setting; • we perform extensive experiments on various datasets and architectures, with federation sizes ranging from 100 to ∼ 342k clients, showing that our method achieves similar performance to the traditional centralized FedAvg.
This is in contrast to prior works, which mostly consider data center scenarios, synthetic data, or a small number of participants ( i.e. , < 100 ).  Related Work Decentralized learning is typically approached from one of two angles: gossip averaging or random walks. Gossip-style algorithms are rather extensively studied in the literature.
[11] showed that gossip averaging between computational nodes leads to speedup by eliminating the bottleneck on a central coordinating node (and coincidentally, a central point of failure).
[1] presented the stochastic gradient push (SGP) algorithm, which enables asymmetric communication topologies (in contrast to [11]) and therefore circumvents the need for deadlock-avoidance and synchronization.
Reliability and convergence speed were further improved by the RelaySGD method by [22] using spanning trees over a communication network to relay model updates between nodes (without decaying their magnitude).
In addition to practical advantages over SGP and quasi-global momentum [12],
RelaySGD boasts theoretical independence of data heterogeneity (non-iid split between nodes).
Since a straightforward implementation of gossip protocols leads to a significant increase in communication, even on sparse graphs, several approaches to compress the transferred updates have been proposed [8, 21].
For example, Choco-SGD and Choco-Gossip by [8], which converge linearly and allow arbitrary compression operators for updates, and PowerGossip [21], which directly compresses the model differences between neighboring nodes using low-rank linear approximation.
Importantly, all these works, as well as dedicated FL gossip papers [9, 17, 6], only consider a small number of nodes.
For scaled up problems, performance may drop noticeably (see [22, Appendix E.2]).
Algorithms based on random walks received less attention in this context, perhaps due to a potentially high number of communication rounds required for convergence.
However, these methods are worth studying since the overall communication may be considerably lower.
[2] and [3] presented uniform and weighted random walk SGD algorithms, proved their convergence, and demonstrated a favorable comparison with gossip averaging, but only on synthetic data.
[24] proposed inexact stochastic parallel random walk alternating direction method of multipliers (ISPW-ADMM), which utilizes multiple parallel random walks speed up learning, and showed gains over gossiping; however, they only considered 10 devices and it is unclear what kind of data were used.
For non-convex problems, [20] studied Markov chain gradient descent, its convergence and mixing rates, while [14] put forth another ADMM-based random walk algorithm in a decentralized optimization setting.
Again, largescale practical experiments are missing from these works.
Therefore, there is a need to compare gossip averaging and centralized FedAvg on realistic ML datasets and with sizeable federations of devices.
Furthermore, adaptive optimization is often preferable to simple SGD due to faster convergence and ease of learning rate tuning.
Hence, in contrast to prior work, we study random walks in combination with Adam.
To limit the communication overhead from transmitting the optimizer state between clients, we consider compression for moments, similar to [18] and multiple local updates on each node, similar to FedAvg [15].
For moments compression, we look at research on memory-efficient adaptive optimizers.
[18] proposed to store per-row and per-column sums of moving averages instead of full matrices for the second moment, using these sums to approximately reconstruct the full moment, and removing the first moment.
This approach could be extended to tensors for higherdimensional convolutional layers, and other factorization and low-rank approximation methods (random projections, power iteration, etc.) could be considered to improve the accuracy of the approximation.
Addressing the problem from a different angle, [5] partitioned tensors into separate blocks and quantized them independently (which can be parallelized for efficiency) and achieved almost full-precision performance for 8 -bit-per-parameter moments.
Finally, count-sketch methods can also be applied [19], reducing memory requirements to logarithmic and having a proven rate of convergence similar to Adam with β 1 = 0 .
Decentralized Federated Learning via a Random Walk
Assume that we have a pool of S clients where each client s ∈ S has access to a local dataset D s of
N s datapoints.
In federated learning (FL), we are interested in learning a vector of model parameters w according to the following
[Equation]
where p ( s ) =
N s /N and f ( w , ξ ) is the loss function computed with parameters w on a data sample ξ .
It is easy to see that this corresponds to sampling from the global data distribution (as if data of all clients are pulled together and sampled uniformly at random) to optimize f ( w ) .
Traditional FL approaches, such as FedAvg [15], optimize this objective by employing a central coordinator, i.e. , the server and exchanging messages in the form of model updates between the clients and the server.
In this work, we forego the notion of a central coordinator and instead design an algorithm where the clients can learn a model by communicating amongst themselves in a peer-to-peer fashion.
3.1 Decentralized federated learning
Consider a set of clients S connected by a set of communication links E into a graph G : ( S, E ) , called the communication graph .
For the rest of this work, we assume that G is connected, nonbipartite and undirected.
Clients can be personal devices, such as smartphones, IoT devices, and so on, as well as organizations ( e.g. , hospitals, banks, etc.).
The connections could be based on physical networks, virtual networks, routing times, trust, or other concepts.
Each client s , therefore, has a set of neighbors N ( s ) :
= { v ∈ S : ( s, v ) ∈ E } defined by the connections available to this client.
Client s receives a message M from neighbor h w ( t ) ← M w ( t +1) ← w ( t ) − η ∇ w f ( w ( t ) , D ( t ) s )
j ← random neighbor from N ( s ) return Send model w ( t +1) to neighbor j
Gradient-based learning by a random walk
To learn the parameters w in a peer-to-peer fashion, we can consider a random walk procedure.
At each step t of the training process, a client s receives the current model from one of its neighbors and updates this model with a gradient obtained using (a batch of) its private dataset.
Subsequently, the client samples one of its neighbors j ∈ N ( s ) , based on a predefined distribution over N ( s ) , and forwards the updated model along with any other relevant optimization parameters.
Algorithm 1 summarizes the above in pseudocode.
Each client can run additional routines to evaluate the model, and if the target metrics have been achieved, stop the training and propagate the final model across the communication graph such that every client has the most up-to-date copy.
This could be based on aggregate validation metrics across clients, or some collaborative decision-making process, like majority voting.
Designing the transition probability In order to optimize Eq.
1, we have to construct a random walk procedure that has the correct stationary distribution over the clients, i.e. , p ( s ) =
N s /N .
This will guarantee that, asymptotically, we are optimizing the global loss function.
Let A ∈ { 0 , 1 } S × S be an adjacency matrix that denotes the connectivity according to G ; if node i is connected to j , then A ij =
A ji = 1 and zero otherwise.
Now consider P to be a simple S × S transition matrix between the S nodes, such that the transition from a node j to a node i is given as P ij = p
( i | j )
= 1 / |N
( j ) | , ∀ i ∈ N ( j ) and zero otherwise.
|N ( · ) | returns the amount of neighbors of a specific node ( i.e. , its degree).
In this case, we can define D as a diagonal matrix with the degrees of each of the nodes in the diagonal.
Therefore, this simple strategy where each client samples a specific neighbor uniformly at random can be encoded as P = AD − 1 .
This transition probability requires only local information and is therefore appropriate for the decentralized setting.
Unfortunately, this simple strategy does not generally have the desired stationary distribution.
We can, however, impose a specific stationary distribution via a Metropolis-Hastings adjustment [3] of this simple proposal distribution:
[Equation]
since the normalizing constant of the desired stationary distribution is the same in each node.
This new transition probability guarantees that the stationary distribution is p ( s ) and still requires only local information sharing ( i.e. , data counts, and the number of neighbors), which can be done at the beginning of training.
Structure of the communication graph For a random walk procedure to be effective, we need a communication graph that facilitates fast “mixing." In other words, we need a communication graph G that allows the random walk to converge quickly to its stationary distribution. The main graph structure we assume in this work is that of “small-world" graphs [23]. Such graphs arise naturally in the context of social networks, wikis, and the internet.
The main property of small-world graphs is that, while they are sparsely connected, the transition between any two pairs of nodes can be done in a small number of steps.
Intuitively, they can be understood as ring graphs where some random connections have been replaced or added, with the new connections acting as “shortcuts" that allow for fast traversal.
Adaptive optimization methods, such as Adam [7], use exponential moving averages of gradients (first moment) and squared gradients (second moment) to correct for stochasticity in the gradient direction and adaptively increase or decrease the effective learning rate in response to the gradient variance across iterations.
Each moment is equivalent in size to the gradient vector and needs to be stored between optimization steps.
In centralized settings, the moments can be stored on the server and do not need to be transferred to the clients.
In a decentralized setting, however, each client needs to have access to these moments, and thus, they need to be transmitted along with other updates.
4.1 Reducing communication costs
Transmitting the moments of the optimizer on each step of the random walk increases communication costs considerably.
In this section, we propose two techniques that can effectively tackle this problem.
Quantization Our first strategy for reducing the communication cost is the compression of the state of Adam.
While it generally is possible to compress both moments, we choose to drop the first moment ( i.e. , set β 1 = 0 ), analogously to [18].
Not only does it help to save communication budget, but it also simplifies the convergence proof because compression of this moment would introduce bias in the update direction.
Compression of the second moment can be done, for example, through scalar quantization or factorisation [18, 21].
Client s receives a message M from neighbor h w ( t ) , comp ( m 2 ) , t ← M w 1 ← w ( t ) m 2 ← decomp ( comp ( m 2 )) for k ← 1 , . .
, K − 1 do g sk ← ∇ w f ( w k , D s ) m 2 ← β 2 m 2 + (1 − β 2 ) g 2 sk ˆ m 2 ← m 2 / (1 − β tK + k) w k +1 ← w k − η g sk / ( √ ˆ m 2 + ϵ ) end for w ( t +1) ← w K j ← random neighbor according to ˆ P : s return Send w ( t +1) , comp ( m 2 ) , t + 1 to neighbor j for a client s according to P : s j ← N ( s )
C = min (cid:16) N j |N ( s ) | N s |N ( j ) | , 1 (cid:17) , u ∼ U [0 , 1] if u < C then next_node ← j else next_node ← s end if return next_node In this work, we introduce a simple, effective, and novel scalar quantization procedure for the second moment: Quantization in the log domain. More specifically, our procedure is motivated by the observation that most of the values of the second moment are quite small and that uniform quantization in the log domain provides more resolution for these lower values.
Let v be the second moment: we use a single bit to denote whether v is non-zero and then use b − 1 bits to do uniform quantization on log v for the non-zero values on v .
We choose a uniform quantization strategy that represents the minimum and maximum values of the non-zero values of log v exactly; thus, there is no clipping.
Another strategy that can reduce the communication cost is to do multiple local steps on each client, popularized by FedAvg [15] in the server-orchestrated setting.
While simple in nature, to the best of our knowledge, such a strategy has not been considered before in the random-walk type of optimization literature.
We thus introduce the notion of multiple local updates in the random-walk type of optimization settings.
More specifically, each client updates the model
K times on their local dataset before continuing the random walk.
As a result, for a total of M model updates, we only need to (roughly) communicate M/K times, thus cutting down the overall communication cost by a factor of K .
Algorithm 2 presents the client-side logic of Adam with compressed moments and multiple local updates, whereas Algorithm 3 in presents the strategy for selecting the next node in the random walk.
4.2 Convergence analysis
In this section, we analyze the convergence behavior of decentralized-FL using random walks with no-momentum Adam and our proposed modifications in the non-convex setting.
More specifically, we investigate the convergence rate in three settings: (1) random walk with Adam, (2) random walk with Adam and second-moment quantization, and (3) random walk with Adam, second-moment quantization, and multiple local updates.
All the proofs are provided in Appendix A.
One of the main challenges for the theoretical analysis of this setting is in considering the bias in the gradient distribution that stems from performing a random walk to sample the gradients. In a similar fashion to [20], we provide the following lemma that characterizes this bias.
Lemma 4.1.
Let A ∈ { 0 , 1 } S × S be an adjacency matrix of a connected, non-bipartite and undirected graph with S nodes, and P a transition matrix (not necessarily symmetric) between the nodes that respects the connectivity of A .
Let the marginal distribution of the random walk over the nodes at timestep t be
[Equation]
where π 0 is the initial distribution over the nodes.
Let λ 1 , . .
, λ S be the eigenvalues of P .
Provided that 1 = λ 1 > λ 2 ≥ · · · ≥ λ S > − 1 we have that
[Equation]
where π ∗ is the stationary distribution of the random walk, | f | ∞ ≤ G and λ = max( λ 2 , | λ N | ) .
Beside this lemma, we also make the following standard assumptions in the non-convex optimization and federated learning literature [16].
Assumption 1 (Lipschitz gradient) .
Each local loss function f s is L -smooth ∀ s ∈ S , i.e. , ∥∇ f s ( x ) − ∇ f s ( y )
∥
≤ L ∥ x − y ∥ , ∀ x , y ∈ R D .
The global loss function f = (cid:80) s N s N f s is also L -smooth.
Assumption 2 (Bounded gradient norm) .
The stochastic gradient norm is upper bounded by a constant G
[Equation]
Assumption 3 (Bounded gradient variance) .
For any dimension i , the gradient variance at a client s is upper bounded by σ 2 li , ∀ s , and the global gradient variance by σ 2 gi
E s
[Equation]
Furthermore, we define σ 2 = (cid:80) i σ 2 i and σ 2 l = (cid:80) i σ 2 li .
Based on the above, we can prove the following theorem about the convergence of random-walk optimization with Adam and no momentum.
Theorem 4.1.
Let w ∈ R D be the model parameter vector that is optimized for T random walk steps.
Under assumptions 1, 2,
and
if the learning rate η and ϵ are chosen such that η ≤ ϵ 2 L and 1 − β 2 ≤ ϵ 2 16 G 2 then we have that the updates of random walk Adam without momentum satisfy
[Equation]
where w a is a randomly chosen iterate from w 1 , . .
. , w T .
Therefore, we can see that the asymptotic bound we obtain is similar to the traditional Adam [25] with an additional error term stemming from decentralization which, however, decreases with the number of iterations.
We now consider the case of performing b -bit scalar quantization to the second moment of Adam.
To the best of our knowledge, while a similar setting has been considered in practice by [5], it has not been theoretically analyzed before.
In order to analyze its convergence, we need one more assumption for the quantization procedure.
Assumption 4 (Bounded quantization noise) .
Let v be a variable to be quantized and Q ( · ) the quantization operation.
We assume that there is a constant q ≥ 0 such that the quantization noise is bounded as
[Equation]
Notice that our specific log-uniform quantization strategy conforms to this assumption since the upper bound on the quantization noise is given by half the step size of the uniform quantizer ∆ .
For any given dimension i we thus have | Q ( v i ) − v i | ≤ | v i exp (cid:0) ∆2 (cid:1) − v i | = | exp (cid:0) ∆2 (cid:1) − 1 || v i | .
In this case, we can prove the following theorem.
Theorem 4.2.
Let w ∈ R D be the model parameter vector that is optimized for T random walk steps.
Under assumptions 1, 2, 3, 4 and if the learning rate η and ϵ are chosen such that η ≤ ϵ 2 L and 1 − β 2 ≤ ϵ 2 16 G 2 then we have that the updates of random walk Adam without momentum and with second moment quantization satisfy
[Equation]
where w a is a randomly chosen iterate from w 1 , . .
. , w T .
For the second setting, we consider the case where each client does a fixed number of updates K before deciding whether to send the model to one of its neighbors.
While such a case has been analyzed theoretically before in gossip type of approaches [10], it has not been analyzed for random walks.
In this specific case, we require one more assumption, which we can use to prove Theorem 4.3.
Assumption 5 (Bounded difference between local and global gradients) .
For any given point w ∈ R D there is a constant ζ 2 that upper bounds the squared L 2 distance between the gradient on any given client s and the global gradient:
[Equation]
Theorem 4.3.
Let w ∈ R D be the model parameter vector that is optimized for T random walk steps and let K be the number of updates a client does before continuing the random walk.
Under assumptions 1, 2, 3, 4, {+
+} and if the learning rate η and ϵ are chosen such that η ≤ ϵ 4 L and 1 − β 2 ≤ ϵ 2 64 G 2 then we have that the updates of random walk Adam without momentum and second moment quantization satisfy
[Equation]
where w a is a randomly chosen iterate from w 1 , . .
. , w KT .
The bound is intuitive. We can see that we are essentially trading total variance σ 2 for local gradient variance σ 2 l and bias ζ .
As a result, this strategy will be useful whenever the data are close to i.i.d.
across the clients; in this case, ζ 2 will be small, and since σ 2 l ≤ σ 2 , it will be beneficial to do multiple local steps K to reduce the number of communication rounds T required.
Evaluation
We consider several established centralized federated learning benchmarks (vision and text) to evaluate the performance of our decentralized random walk optimization procedures with a variety of models.
Vision tasks include training ResNet-20 on non-i.i.d.
splits of CIFAR-10, CIFAR-100, with 100 and 500 clients respectively, as described in [16]; LeNet-5 model on the naturally non-i.i.d.
split of FEMNIST (partitioned by writer IDs into ∼ 3.6k clients)
Language tasks consist of next word prediction using an LSTM model on a non-i.i.d.
split of the Shakespeare dataset into 660 clients [15], and the StackOverflow tag prediction with a logistic regression model [16] and ∼ 342 k clients.
For the majority of experiments, we used a “small-world" graph with an average degree of five, implemented using the Watts–Strogatz model [23], with β = 0 .
Unless mentioned otherwise, we use 4 -bit per-tensor quantization for the second moment, and we consider a separate quantization grid for the weights of each layer.
Empirically, we found it beneficial to employ stochastic (in place of deterministic) rounding in log-space for quantization.
In the case of multiple local steps, we only quantize the second moment whenever it is transmitted and not in-between the local optimization steps.
More details about the experimental setup can be found in the Appendix.
Comparison against centralized FL First, we evaluate the performance of random walk learning against a centralized baseline (FedAvg with Adam on the server [16] and one epoch of updates on each client).
Figure 2 depicts validation accuracy as a function of total communication costs (both upload and download) for FedAvg and different versions of random walk optimization.
As the
StackOverflow task consists of large validation and test sets, we plot the accuracy on the first 10% of validation examples.
The final numbers (including evaluation on the full test set) are provided in Table 1.
We observe that even a simple RW-SGD performs well on vision tasks, provided that the learning rate is chosen appropriately.
Its accuracy on CIFAR-10 and FEMNIST is on par with or better than FedAvg, while communication is lower (in case of FEMNIST, by 32% ).
RW-Adam provides competitive performance even without hyperparameter tuning, thanks to learning rate adaptation via [Figure] [Table] the second moment.
Our quantization procedure brings communication costs down to the level of
RW-SGD without impact on accuracy.
When gradients are sparse, like in the case of StackOverflow
[Table]
or the task is more complex (CIFAR100)
[Table]
we see that adaptive optimization is crucial to obtain a better model.
On CIFAR-100, RW-Adam provides a 7% boost in validation accuracy compared to RW-SGD and gets closer to FedAvg.
Even more noticeably, on StackOverflow, RW-Adam maintains near-FedAvg performance, while RW-SGD only reaches less than a half of centralized performance within a comparable communication budget.
Finally [Table] we observe that Adam with the default hyperparameters shows robust performance in all the experiments involving a neural network.
At the same time, for SGD, we find it necessary to tune parameters on a per-task basis.
For instance, vision tasks run with a learning rate of 0 .
1 , while 1 . 0 is used for Shakespeare.
Comparison with gossiping To compare gossiping with random walks, we run both algorithms on CIFAR-10 and the Shakespeare dataset.
In both cases, we use the same small-world setup as before.
The choice of datasets is partially dictated by the limitations of gossiping for large-scale federations since the other tasks have a significantly larger number of clients.
In Table 2, we can observe that the raw gossip averaging communication far exceeds both FedAvg and random walks.
For instance, it requires more than 2 TB on CIFAR-10 to reach over 60% accuracy, which is significantly lower than what is achieved by other methods.
Even, hypothetically, applying
PowerGossip compression [21] to the updates (which would further reduce validation accuracy), the overall communication remains high ( ∼ 41 GB), and RW-QAdam reaches better performance.
Similarly, for Shakespeare: over 400 GB of raw communication is necessary for 40% accuracy, and [Table] even when accounting for possible compression, gossiping performance is considerably worse than
RW-QAdam with less communication.
5.2 Additional studies
Impact of the communication graph To empirically measure the dependence of our method on the communication graph topology, we conduct a study on CIFAR-10 with 100 clients.
We fix
RW-QAdam with 5 bits for the second moment and consider three specific communication graphs: (a) a fully connected graph, (b) a small-world graph with an average degree of 5 ( i.e. , as in all other experiments) and (c) a ring graph with a degree of 2 .
Figure 3a shows the convergence speed of random walk training.
We see that the small world and the fully connected graph have similar convergence (indicating good “mixing"), whereas the ring topology is lagging behind. [Figure] Impact of multiple local updates We argue for doing multiple local updates to improve performance while reducing communication costs. Indeed, our theory suggests that if the data are close to i.i.d.
across the nodes, multiple local updates can be beneficial.
Empirically, we observe that
K = 1 performed better on many (artificially or naturally) non-i.i.d.
tasks, except Shakespeare.
In
Figure 3b, we show that on the Shakespeare dataset, there is a big improvement in the communication vs. accuracy tradeoff when we move from K = 1 to K = 10 local updates on each client.
To further test our theory, we set up an i.i.d.
CIFAR-10 task with 100 clients.
Figure 3c depicts the communication vs accuracy tradeoffs of our quantized state Adam with K = 1 and K = 10 local updates.
We can similarly see the benefits of multiple local steps as our theory predicted.
As a reference, non-federated training reaches a test set accuracy of 78 .
9% in this setting, which is close to the performance of our random walks.
Conclusion
In this paper, we proposed a random-walk-based learning algorithm with compressed-state adaptive optimization to solve the problem of federated learning without a central server in a peer-to-peer fashion.
Since the communication graph of participating devices can be formed based on proximity or connection speeds, this approach has the potential to minimize the overall time spent on communication.
Despite the increase in the number of rounds necessary for convergence, the total communication cost of our approach is on par or better than that of federated averaging because each round involves communication between only two clients (as opposed to typically 10–100 in
FedAvg).
Our evaluation showed that compressed adaptive optimization on top of a random walk enables performance comparable with centralized FL solutions, even on sparsely connected graphs.
One potential disadvantage of random walk learning is that client updates are sequential, while in
FedAvg or gossiping, they can be done in parallel.
However, this drawback can be compensated by parallelizing hyperparameter search and running multiple random walks with different configurations.
Since there is no dependency between them, no synchronization is needed in this process, allowing to bypass stragglers and effectively gauge the performance of multiple models.
We leave a detailed methodology and evaluation of this approach for future work.
Another important aspect for future study is combining our approach with formal privacy notions.
For example, [4] have proposed network differential privacy and showed that it is well suited for simple random walks; continuing this line of research for adaptive and communication-efficient random walk optimizers is an interesting future direction.
Conceptually, a decentralized FL approach removes reliance on a central server, the independence and trustworthiness of which cannot be taken for granted in many parts of the world.
Beyond moving the control over the data to the user through FL, our work takes a step towards putting users in control over the entire learning process itself.
The immediate societal consequence of such an approach is more resilience to central interference but also less accountability and a higher potential of misuse by individuals.
Developing robust mechanisms against diverse forms of attacks will be crucial to establishing decentralized FL in the future.
Appendix A Convergence proof in the case of a random walk A.1 Proof of Lemma 4.
Let A be an undirected adjacency matrix that denotes the connectivity between N nodes; if node i is connected to j , then A ij =
A ji = 1 and zero otherwise.
Now also let P be the N × N transition matrix between the N nodes, where we have that a transition from a node j to a node i is given as P ij = p
( i | j )
= 1 /d ( j ) ∀ i ∈ N ( j ) and zero otherwise, where d ( · ) is a function that returns the amount of neighbors of a specific node ( i.e. , the degree).
In this case, we have that P = AD −where D is a diagonal matrix having the degrees of each of the nodes in the diagonal.
Having access to the above, we can compute the marginal distribution of being in a specific node i at time t as π t ( i ) =
(cid:80) j π t − 1 ( j ) p
( i | j ) .
This can be compactly written as
[Equation]
11) where π 0 is the initial distribution over the nodes ( e.g. , a one-hot vector when we deterministically start the random walk from a specific location).
The stationary distribution of the walk will be π ∗ and it will satisfy the relation π ∗ =
AD − 1 π ∗ .
For the purposes of this work, we are interested in bounding the divergence of π t from the stationary distribution π ∗ , as in our case π ∗ corresponds to sampling the global gradient distribution.
To do that, we will follow the analysis described at [34].
Let C
= D − 1 / 2 AD − 1 / 2 be the normalized connected and non-bipartite adjacency matrix which will have that its eigenvalues will be
=
λ 1 > λ 2 ≥ · · · ≥ λ N > − 1 and its corresponding orthonormal eigenvectors will be b 1 , . . .
, b N .
As AD − 1 is similar to D [Equation]
=
C , we will have that AD − 1 has the same eigenvalues as C .
Furthermore, we will have that its eigenvectors will be
[Equation]
thus the eigenvector for the i ’th eigenvalue of AD − 1 will be ˆ b i
=
D 1 / 2 b i .
Let the matrix of eigenvectors be denoted as ˆ B
= D 1 / 2 B , where B is the set of eigenvectors for C .
Now, since the set of eigenvectors constitute a basis, we can express the initial distribution π 0 under this basis, π 0 = (cid:80) Ni =1 c i ˆ b i , where c
i =
ˆ b Ti π 0 .
We will then have that
[Equation]
Now by taking the limit of t → ∞ we can see that the stationary distribution of the above is
[Equation]
as λ 1 = 1 and is the only eigenvalue that doesn’t decay with t .
Therefore, we have that the stationary distribution will be associated with the first eigenvector of AD − 1 .
We can now return to our original question; finding a bound between the distribution of the chain at timestep t and the stationary distribution.
We will consider the total variation distance, i.e. ,
[Equation]
(15) =2 (cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
N (cid:88) i =
[Equation]
(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
[Equation]
(cid:13)(cid:13)(cid:13)(cid:13)(cid:13) [Equation] (cid:88)
[Equation]
=
[Equation]
(cid:13)(cid:13)(cid:13)(cid:13)(cid:13) (16) = (cid:13) (cid:13)(cid:13)(cid:13)(cid:13)
N (cid:88) i =
c
[Equation]
i (cid:13) (cid:13) (cid:13) (cid:13)(cid:13)(17) ≤ √ N
[Equation] (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) N (cid:88) i = [Equation]
(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
[Equation]
We also have that
[Equation] [Equation] due to the ˆ b i forming an orthonormal basis.
By taking λ = max( λ 2 , | λ N | ) , we can now bound the previous in the following way
[Equation] [Equation]
where the last inequality is due to
[Equation]
[Equation]
Therefore, we have that
[Equation]
(22) and thus
[Equation]
Now, from the alternative representation of the total variation distance, we have that for f in a space of functions F with an image in R that takes as inputs samples from distributions p, q ,
TV ( p, q ) = 12 sup f ∈ F, | f |≤ 1 ( E p [ f ] − E
q
[ f ]) [29].
This can be generalized to arbitrary functions f bounded by G ; in this case, we have that TV ( p, q ) = 12 G sup f ∈ F, | f |≤ G ( E p [ f ] − E q
[ f ]) .
Therefore, applying this result to our distributions over selecting the nodes, we have that
[Equation]
and, thus, for any f ∈ F with | f | ≤ G we have that
[Equation]
We can see that this bound has a sublinear dependence on the number of nodes N .
Furthermore, as the total-variation distance is symmetric, we further have that
[Equation]
Therefore, both of these facts can be combined to yield
[Equation]
which constitutes Lemma 4.1.
To analyze the transition matrix arising from the Metropolis-Hastings adjustment of the simple proposal, ˆ P , we can work with the similar matrix
[Equation]
(28) here R is a diagonal matrix that has the desired stationary distribution, π ∗ , in its diagonal.
From [31] we know that the matrix S is symmetric (thus has only real eigenvalues) and satisfies the property that 1 = λ 1 > λ 2 · · · ≥ λ N > − 1 .
Now let the eigenvectors of S be
s i , we can show that the left eigenvectors of ˆ P are s Ti R 1 / 2 and the right eigenvectors are R − 1 / 2 s i .
From [31] we also know that the left and right eigenvectors of ˆ P associated with λ 1 = 1 are π ∗ and 1 respectively, therefore c 1 ˆ b 1 = π ∗ .
As a result, the same analysis and bounds apply.
A.2 Convergence analysis in the non-convex case
We will base this proof on the proof technique of [25] for the case of Adam with only the second moment; the main idea is to use the previous bounds in order to translate the expectation over the
Markov chain to an expectation over the stationary gradient distribution.
With w t we will denote the vector of optimized parameters at timestep t , with v t the vector of the second moments per parameter at timestep t and with g t the stochastic gradient obtained at timestep t .
we will We start with the update rule of Adam (without the first moment) for a single dimension
i
[Equation]
where both ϵ and the second moment v are non-negative.
We then assume an L-Lipschitz loss function and proceed by:
[Equation]
We then take an expectation over the distribution of the chain at timestep t , π t , along with the data samples ξ given that we are at w t
[Equation]
and we then apply Lemma 4.1 to express the expectations in the r.h.s.
to ones over the stationary distribution at timestep t , π ∗ .
For that, we need an upper bound on the functions, i.e. , | f | ∞ ≤ C , that we take the expectation over.
In this specific case, we have that
[Equation]
[Equation]
and thus
E π t ,
[Equation]
[Equation] + Lη [Equation] D
[Equation]
(cid:34)
[Equation]
(cid:0)
[Equation]
We then proceed in a similar manner to [25]
[Equation]
Given that ∇ f ( w t ) =
E π ∗ ,ξ [ g t ] , we can continue with
[Equation]
w
[Equation]
From [25] we have that the T 1 t term can be upper bounded by
[Equation]
Therefore, taking this into account and with the additional assumption that each gradient coordinate is upper bounded by G
[Equation]
we have that
[Equation]
[Equation]
Furthermore, due to the positivity of √ v t and ϵ we also have that
[Equation] [Equation]
Therefore
[Equation]
we have that the the T 2 t term can be upper bounded by [Equation]
[Equation]
since v t,i =
β 2 v t − 1 ,i +
[Equation]
1 − β [Equation] g 2 t ≥ β 2 v t − 1 ,i .
Putting everything together, we have that ≤
[Equation]
[Equation]
Now we have one more assumption, namely that the gradient variance of each dimension is bounded by σ 2 i .
In this way, we have that V [ g t,i ] =
E [ g 2 t,i ] − E [ g t,i ] 2 ≤ σ 2 i → E [ g 2 t,i ] ≤ σ
i + E [ g t,i ] 2 .
Therefore [Equation] we have that
[Equation]
There are some further assumptions taken from [25]
[Equation]
namely that η t , β 2 , ϵ are chosen such that
[Equation]
Using these, we have that
[Equation] [Equation]
and due to 0 ≤ v t [Equation] i ≤ G 2 and defining σ 2 = (cid:80) Di =1 σ 2 i we have that [Equation]
[Equation]
Therefore,
[Equation] [Equation]
Now by setting η t = η and taking a telescoping sum, i.e. , taking the expectation over π t for all t we have that (cid:88)
[Equation]
[Equation]
Rearranging the inequality and assuming that f ( w ∗ ) ≤ f ( w t ) ∀ t ( i.e. , w ∗ is the optimum) we have that
[Equation]
Furthermore, since λ < 1 , we have that the (cid:80) t λ t forms a geometric series, and therefore it is upper bounded by
[Equation]
Therefore, the above can be simplified to
[Equation]
Therefore, we have that
[Equation]
where w a is randomly chosen iterate from w 1 , . .
. , w T .
This ends up being a result similar to the one at [25], except with an additional term that depends sublinearly on the number of nodes ( N ) and decays with the number of iterations T .
A.3 Adding compression
To use adaptive optimizers effectively in our setting, it show in the main text that it is useful to quantize the second moment.
We will have to take this into account in the proof.
We begin the proof in a similar manner; the updates when the second moment is quantized are given as
[Equation]
where
[Equation]
r t − 1 ,i is the quantization noise on the second moment, i.e. , r t − 1 ,i = Q ( v t − 1 ,i ) − v t − 1 ,i , with Q ( v ) representing the quantized second moment.
We then assume a L-Lipschitz loss function and proceed by:
[Equation]
We then take an expectation over the distribution of the chain at timestep t , π t , and the data-samples ξ given that we are at w t along with an expectation over the quantization noise of round t − 1 , r t − 1 .
[Equation]
and we then apply Lemma 4.1 to express the expectations in the r [Equation] h [Equation]
s. to ones over the stationary distribution at timestep t , π ∗ .
[Equation]
and then proceed in a similar manner as before.
[Equation]
and since E π ∗
[Equation]
r t − 1 ,ξ [ g t ] =
E π ∗ ,ξ [ g t ] = ∇ f ( w t ) (as the quantization noise on timestep t − 1 does not affect the gradient at timestep t once we condition on w t ),
[Equation]
70) ≤ f ( [Equation]
[Equation]
ϵ
[Equation]

[Equation]
As we have quantization of the second moments
[Equation]
the bound from [25] doesn’t apply.
However, we can construct the following upper bound:
[Equation]
With our assumptions
[Equation]
we can show that
[Equation]
since the minimum value of v t − 1 ,i + r t − 1 ,i ≥
[Equation]
i.e. [Equation] the quantized second moment cannot be negative.
Thus, using these upper bounds we have that
[Equation]
[Equation]
and due to our assumptions on the quantization noise, i [Equation] e., Assumption 4 we have that
[Equation]
We thus end up with
[Equation]
Therefore [Equation] taking the bound on T 1 t into account and with the additional assumption that each gradient coordinate is upper bounded by G
[Equation]
we have that
[Equation]
Let us now take a look at the second term T 2 t .
For that term we have that
[Equation]
We now turn to bounding T 3 t .
We have that
[Equation]
For the first term, we have that
[Equation]
For the second term [Equation] we have that
[Equation]
Thus, by putting everything together [Equation] we have that
[Equation]
Therefore
[Equation]
we have that T 2 t can be bounded as follows [Equation] Thus
[Equation]
putting everything together [Equation] we have that
[Equation]
We adopt some further assumptions from [25] [Equation] namely that η t , β 2 , ϵ are chosen such that
[Equation]
Using these [Equation] along with the fact that σ 2 = (cid:80) i σ 2 i and v ≥ 0 we have that
[Equation]
105) and due to our quantization procedure that preserves the maximum value of the second moment
[Equation]
we have that v t − 1 ,i ≤ G 2 and therefore
[Equation]
Therefore, we have that
[Equation]
Now by setting η t = η and taking a telescoping sum we have that
[Equation]
Rearranging the inequality, doing the cancellations and noting that f ( w ∗ ) ≤ E π T , r T − 1 ,ξ [ f ( w T +1 )] we have that
[Equation]
with q = max t q t .
Thus, we end up with something that has similar asymptotic convergence as before, albeit with an additional error term that depends on the quantization procedure of the second moment, i.e. ,
[Equation]
where w a is randomly chosen iterate from w 1 , . .
. , w T .
A.4 The case of multiple local updates
It can be beneficial to do multiple local updates at each client, in order to decrease the overall communication cost to reach a specific target accuracy.
We will thus extend the convergence proof in order to consider such an approach.
Remember that the updates when the second moment is quantized are given as
[Equation]
where
[Equation]
We will now further assume that each client performs K local updates before it decides whether to send the model to a neighbor.
Similarly as before, assume an L-Lipschitz loss function and proceed by:
[Equation]
We will then take an expectation over the randomness at round t , given that we are at w t .
We will distinguish two cases; if t (mod K ) is zero, then we know that the client will use the randomness of the random walk in order to determine the next node, and if t (mod K ) > 0 , we know that the model will continue staying on this particular client for an update.
In the first case, we established the following upper bound
[Equation]
For the second case, we have to do some more work.
We first take the expectation over the randomness of timestep t .
This specific randomness involves only the data samples of a client s , i.e. , ξ s and the quantization noise r .
Therefore, we have that
[Equation]
[Equation]
We then work towards bounding T 1 .
We have that
[Equation]
[Equation]
where ∇ f s ( w t ) denotes the gradient computed on the full dataset of client s
, i.e. , E ξ s [ g ] .
From previous derivations and assumptions, we have that
[Equation]
thus [Equation] we can continue the upper bound as
[Equation]
where the last step is due to Young’s inequality applied to G ∥∇ f s ( w t ) − ∇ f ( w t ) ∥ 2 .
By then using two of our assumptions, namely, the bounded (local) variance and bounded difference between the local and global gradients, we have that
[Equation]
By reusing previous derivations
[Equation]
we can also show that
[Equation]
Thus [Equation] by putting everything together,
[Equation]
We will now bound T 3 .
[Equation]
Therefore
[Equation]
we have that
[Equation]
In order to maintain an upper bound, we have to impose some further assumptions, namely that
[Equation]
In this way, we have that
[Equation]
which constitutes our final bound for the case of a client locally updating the model without doing the random walk.
We now assume η t = η, q = max t q t and take a telescoping sum while considering both of these cases, where E t corresponds to an expectation over all randomness at round t .
KT (cid:88) t =
[Equation]
t ( mod K )= ∥∇ f ( w t )
∥+ ηT (cid:18) G √ 1 − βϵ+ Lη ϵ(cid:19) σ+ ηT (cid:18)
G √ 1 − β 2 +
LGη 2 ϵ √ 1 − β(cid:19)
Dq + η (cid:18)
Lη 2(1 − β 2 ) +
G √ 1 − β(cid:19)
D √ N T (cid:88) i
= λ
i − η 4 (cid:0) √ β
2 G + ϵ (cid:1) (cid:88) t ( mod K ) ̸
= ∥∇ f ( w t )
∥+ 2 T ( K − 1) (cid:18) ηG √ 1 − βϵ+ Lηϵ(cid:19)
(cid:18) σ 2 l 2 + ζ(cid:19) + T ( K − 1) (cid:18) η √ 1 − β 2 +
Lη2 ϵ √ 1 − β(cid:19)
DGq + T ( K − 1) η √ D 2 ϵ (cid:0) G 2 + ζ 2 (cid:1) (138) ≤ − η 4 (cid:0) √ β 2 G + ϵ (cid:1)
T (cid:88) t = ∥∇ f ( w t )
∥+ ηT (cid:18) G √ 1 − βϵ+ Lη ϵ(cid:19) σ 2 + T ( K − 1) η √ D 2 ϵ (cid:0) G 2 + ζ 2 (cid:1) + 2 T ( K − 1) (cid:18) ηG √ 1 − βϵ+ Lηϵ(cid:19)
(cid:18) σ 2 l 2 + ζ(cid:19) + ηKT (cid:18) 1 √ 1 −
[Equation]
Therefore
[Equation]
by simplifying and re-arranging we have that
[Equation]
We thus end up with
[Equation]
where w a is randomly chosen iterate from w 1 , . .
. , w KT .
B Experimental details
The experiments in this paper were performed on workstations, each of which is equipped with a single Nvidia RTX 2080Ti GPU, as well as on machines equipped with Nvidia V100 and A100 GPUs.
All experiments where performed on single GPUs only.
B.1 Models and datasets
FEMNIST FEMNIST is based on the extended MNIST (EMNIST) dataset [27] and is adapted to the federated setting.
Its images are MNIST-like and consist of 63 different classes of handwritten letters and digits.
In the federated setting, we consider all data-point written by an individual person to constitute an individual client’s dataset.
Any non-i.i.d.
ness therefore stems from the different writing-styles, as well as from the varying size of the clients’ datasets.
Additionally, the size of the individual clients’ datasets differ significantly.
We experiment with the dataset that is originally published by [26] and use their provided code to generate the dataset.
We remark that the statistics reported by the original creators do not align with the result of the provided code when we run it.
In Table 3, we report the dataset’s statistic as we compute them.
We use the standard LeNetconvolutional network [28] in all FEMNIST experiments. [Table]
Shakespeare We follow [26] in the creation of the Shakespeare dataset.
Each unique character’s spoken lines across all of Shakespear’s plays constitutes the dataset for an individual client in the federated setup.
This dataset is naturally non-i.i.d due to the 660 different characters’ identities and corresponding spoken lines.
Our model architecture consists of the 2 -layer LSTM model proposed in [26].
The task is formulated as next-character-prediction task over an alphabet of 80 characters.
The cross-entropy term is computed only on the next character prediction based on an encoding of a 80 character long sentences.
For the statistics in Table 3, we report each pair of 80 characters-long sentences plus next character as a single sample.
Please note that the differences between our statistics and what is reported by [26] has been raised in a github issue 2 .
StackOverflow
The StackOverflow dataset [32] is a collection of questions and answers scraped from the StackOverflow website.
Each user of that website who posted there within a specific time-frame is a client and their posts constitute its dataset.
In this work, we compare on the tag prediction task described in [16] using a simple logistic regression model.
Each posting is tagged with one or multiple of 500 possible tags, such that a client performs one-vs-all classification across these 500 tags.
Each posting is represented as a normalized bag-of-words across the 10 , 000 most frequent words in the training set.
Since the validation-set is very large, we report learning curves by sampling in the main paper were created by selecting the first 10% of data-points.
For the results in
Table 1, the final model was evaluated on the test set.
We do 30k steps in the case of random walk optimization and 1.5k steps for FedAvg.
CIFAR10/CIFAR
For the CIFAR10 dataset, we split the 45 k training examples acrossclients in a non-i.i.d.
way according to the label-skew setting presented in [16] with α = 1 . 0 .
For
CIFAR100, the training set is split across 500 clients in a non-i.i.d.
way following the PAM method in [16].
We keep α = 1 . 0 , β = 10 . 0 .
Experiments on both datasets are performed with the standard
ResNet-20 architecture for CIFAR data-sets, albeit with BatchNorm replaced by GroupNorm withgroups.
Gossip Averaging For gossip averaging experiments, we use CIFAR10 and Shakespeare.
The data split for both tasks is equivalent to the one described above, and so are the models.
We also use the same small-world communication graph with the average degree of 5 and β = 0 .
Training is done similarly to prior work on gossiping (e.g., [21]): using gradient descent with momentum (0.9) and weight decay ( 10 − 4 ); learning rate is 0.1.
In CIFAR10 experiment, all clients are updated in each round, but in Shakespeare, due to a significant slow-down caused by the larger federation and model, only 10 users get updated in every round.
The latter, however, doesn’t appear to impact the accuracycommunication trade-off, and thus our conclusions.
Finally, the PowerGossip communication savings are accounted by applying the compression ratio achieved in [21] for CIFAR10 (as the model is the same, ResNet-20), and by computing the ratio directly, layer-by-layer, following the PowerGossip method with 1 iteration for Shakespeare.
B.2 Hyperparameters
For all of the tasks involving a neural network ( i.e. , all tasks we considered, except StackOverflow) for RW-Adam we used the default Adam hyperparameters (without momentum), i.e. , a learning rate of 1 e − 3 , β 2 = 0 .
999 , β 1 = 0 and ϵ = 1 e − 7 .
For the quantized version of Adam, we used eitheror 5 bits per parameter, determined by comparing validation performance against the unquantized [Figure] [Table] version of Adam.
For RW-SGD we tuned the learning rate on a per-experiment basis via a validation set.
The range we considered was [1 e − 3 , 1] .
We found that a learning rate of 1 e − 1 worked well on all of the vision tasks [Table] whereas a learning rate of 1 worked well for Shakespeare.
For StackOverflow, we found that we need much larger learning rates; for RW-Adam and RW-QAdam we used a learning rate of 1 e − 1 whereas for SGD we used a learning rate of 1 e 5 .
The number of local steps in the case of random-walk optimization was picked from { 1 , 3 , 5 , 10 } , again determined on a validation set.
We used a batch-size of 128 for CIFAR10/100, Shakespeare, FEMNIST, and a batch-size of 100 for
StackOverflow.
For FedAvg, we used the hyperparameters provided at [16] for StackOverflow, whereas for CIFAR 10 / CIFAR100 / FEMNIST, we used SGD locally with a learning rate of 0 .
05 , 1 local epoch and
Adam with the default hyperparameters at the server (with momentum), i.e. , a learning rate of 1 e − 3 , β 1 = 0 .
9 , β 2 = 0 .
999 and ϵ = 1 e − 7 .
For Shakespeare we found it beneficial to use locally SGD with a learning rate of 1 . 0 , 1 local epoch and similarly SGD with a learning rate of 1 . 0 at the server.
For CIFAR10/CIFAR100/FEMNIST/
StackOverflow we sampled 10 clients per round, whereas we sampled 66 clients per round for Shakespeare.
C Additional results
In this section we provide an estimate of the variability of our results according to different random seeds on CIFAR10, CIFAR100, FEMNIST and Shakespeare; we used 4 random seeds for CIFAR 10, Shakespeare and 3 random seeds for FEMNIST and CIFAR100.
In Figure 4 and Table 4 we provide the average learning curves and the average test-set accuracy along with its standard error respectively.
We can see that, overall, our results have similar variability to the ones from FedAvg and do not change our conclusions; we can get similar or better accuracy to FedAvg with comparable communication costs.
Note that for CIFAR100, the performance of FedAvg is lower than what we reported in the main text; this is due to choosing a point in training for FedAvg that has similar communication costs as our RW-QAdam.