Title: Hierarchical Federated Learning with Multi-Timescale Gradient Correction
Venue: NeurIPS 2024
Accepted: yes
Keywords: Federated learning

gradient correction

hierarchical
Abstract: While traditional federated learning (FL) typically focuses on a star topology where clients are directly connected to a central server, real-world distributed systems often exhibit hierarchical architectures. Hierarchical FL (HFL) has emerged as a promising solution to bridge this gap, leveraging aggregation points at multiple levels of the system. However, existing algorithms for HFL encounter challenges in dealing with multi-timescale model drift, i.e., model drift occurring across hierarchical levels of data heterogeneity. In this paper, we propose a multi-timescale gradient correction (MTGC) methodology to resolve this issue. Our key idea is to introduce distinct control variables to (i) correct the client gradient towards the group gradient, i.e., to reduce client model drift caused by local updates based on individual datasets, and (ii) correct the group gradient towards the global gradient, i.e., to reduce group model drift caused by FL over clients within the group. We analytically characterize the convergence behavior of MTGC under general non-convex settings, overcoming challenges associated with couplings between correction terms. We show that our convergence bound is immune to the extent of data heterogeneity, confirming the stability of the proposed algorithm against multi-level non-i.i.d. data. Through extensive experiments on various datasets and models, we validate the effectiveness of MTGC in diverse HFL settings. The code for this project is available at https://github.com/wenzhifang/MTGC.

Hierarchical Federated Learning with Multi-Timescale Gradient Correction

Abstract

While traditional federated learning (FL) typically focuses on a star topology where clients are directly connected to a central server, real-world distributed systems often exhibit hierarchical architectures. Hierarchical FL (HFL) has emerged as a promising solution to bridge this gap, leveraging aggregation points at multiple levels of the system. However, existing algorithms for HFL encounter challenges in dealing with multi-timescale model drift , i.e., model drift occurring across hierarchical levels of data heterogeneity. In this paper, we propose a multi-timescale gradient correction (MTGC) methodology to resolve this issue. Our key idea is to introduce distinct control variables to (i) correct the client gradient towards the group gradient, i.e., to reduce client model drift caused by local updates based on individual datasets, and (ii) correct the group gradient towards the global gradient, i.e., to reduce group model drift caused by FL over clients within the group. We analytically characterize the convergence behavior of MTGC under general non-convex settings, overcoming challenges associated with couplings between correction terms. We show that our convergence bound is immune to the extent of data heterogeneity, confirming the stability of the proposed algorithm against multi-level non-i.i.d. data. Through extensive experiments on various datasets and models, we validate the effectiveness of MTGC in diverse HFL settings. The code for this project is available at https://github.com/wenzhifang/MTGC.

1 Introduction

In the past several years, federated learning (FL) has emerged as a prevalent approach for distributed training [17, 22, 11, 52, 24]. Conventional FL has typically considered a star topology training architecture, where clients directly communicate with a central server for model synchronization [35, 26]. Scaling this architecture to large numbers of clients becomes problematic, however, given the heterogeneity in FL resource availability and dataset statistics manifesting over large geographies [14, 17, 10]. In practice, such communication networks are often comprised of a hierarchical architecture from clients to the main server, as observed in edge/fog computing [25, 38] and softwaredefined networks (SDN) [20], where devices are supported by intermediate edge servers that are in turn connected to the cloud.

To bridge this gap, researchers have proposed hierarchical federated learning (HFL) which integrates group aggregations into FL frameworks [30, 5, 47, 15]. In HFL (see Fig. 1), clients are segmented into multiple groups, and the training within each group is coordinated by a group aggregator node (e.g., an edge server coordinating a cell). Meanwhile, the central server orchestrates the training globally by periodically aggregating models across all client groups, facilitated by the group aggregators.

Christopher G. Brinton

Fundamental challenges. One of the key objectives in FL is to reduce communication overhead while maintaining model performance. Research in conventional FL has established how the global aggregation period, i.e., the number of local iterations during two consecutive communications between clients and the server, impacts FL performance according to the degree of non-i.i.d. (nonindependent or non-identically distributed) across client datasets: when local datasets are more heterogeneous, longer aggregation periods cause client models to drift further apart. In HFL, the situation becomes more complex, and is not yet well studied. There are multiple levels of aggregations within/across client groups, and the frequency of these aggregations diminishes further up the hierarchy (since the communication costs become progressively more expensive). As a result, model drift occurs across multiple levels of non-i.i.d., at different timescales . In the canonical two-level case from Fig. 1, we have (i) intra-group non-i.i.d., similar to conventional FL, and (ii) inter-group non-i.i.d., arising from data heterogeneity across different groups. This introduces (i) client model drift caused by local updates on individual datasets, usually at a shorter timescale, as well as (ii) group model drift caused by FL over clients within the group, usually at a longer timescale.

In conventional star-topology FL, algorithms like ProxSkip [36], SCAFFOLD [18], and FedDyn [1] have shown promise for correcting client model drift through local regularization and gradient tracking/correction. However, these approaches are not easily extendable to the HFL scenario due to its multi-timescale communication architecture . Specifically, when integrating these methods into HFL, control variables introduced to handle data heterogeneity, such as gradient tracking or dynamic regularization, need to be carefully injected at each level of the hierarchy, taking into account their coupled effects in taming non-i.i.d. Convergence analysis elucidating the impact of different updating frequencies for such control variables remains an unsolved challenge. Existing works on HFL have also not aimed to directly correct for multi-timescale model drift. This can be seen by the fact that the convergence bounds in existing HFL methods [30, 5, 47, 13] become worse as the extent of non-i.i.d. in the system increases (e.g., gradient divergence between hierarchy levels in [47]). Some works have proposed adaptive control of the aggregation period in HFL [13, 31], but they require frequent model aggregations to prevent excessive drift. We thus pose the following question:

How can we tame multi-timescale model drift in non-i.i.d. hierarchical federated learning to provably enhance model convergence performance while not introducing frequent model aggregations?

1.1 Contributions

In this paper, we propose multi-timescale gradient correction (MTGC), a methodology which can effectively address multi-level model drift over the topology of HFL with a theoretical guarantee. As depicted in Fig. 1, our key idea is to introduce coupled gradient correction terms - client-group correction and group-global correction - to (i) correct the client gradient towards the group gradient, i.e., to reduce client model drift caused by local updates based on their individual datasets, and (ii) correct the group gradient towards the global gradient, i.e., to reduce group model drift caused by

Client

…

…

Group aggregator

Central server

(Global aggregator)

Group-global

correction

Client-group

correction

Global aggregation - performed after every      group aggregations

Group aggregation - performed after every       local iterations

H

E

…

Client

model drift

{1, 2}

{3, 4}

Labels : {1, 2, 3, 4}

{4, 5}

{6, 7}

Labels : {4, 5, 6, 7}

Labels at

each client

Client

model drift

Group model drift

Group

model drift

FL across clients within the group, respectively. MTGC thus assists each client model to evolve towards improvements in global performance during HFL. We propose a strategy for updating these gradient correction terms after every group aggregation and global aggregation, respectively, and analyze the convergence behavior of MTGC. Due to the coupling of correction terms and their updates being performed at different timescales, additional challenges arise for theoretical analysis compared to prior work. We thoroughly investigate this problem and make the following contributions:

We develop the multi-timescale gradient correction (MTGC) algorithm for taming leveled model drift in HFL. MTGC incorporates coupled control variables for correcting client gradients and group gradients, effectively tackling model biases arising from various levels of non-i.i.d. data at different timescales. The estimation and update procedures for these control variables rely solely on the model updates, ensuring that no significant additional communication overhead is introduced.

We characterize the convergence rate for MTGC under the non-convex setup. This rate is immune to the extent of intra and inter-group data heterogeneity, confirming the stability of our approach against multi-level non-i.i.d. statistics. Our theoretical result also demonstrates that MTGC achieves

linear speedup in the number of local iterations, group aggregations, and clients. Also, we show that the convergence rate of MTGC recovers that of SCAFFOLD, i.e., the non-hierarchical case, when the number of groups and group aggregation period reduces to one.

We conduct extensive experiments using various datasets and models across different parameter settings, which demonstrate the superiority of MTGC in diverse non-i.i.d. HFL environments.

1.2 Related Works

Algorithms for conventional FL. The seminal work [35] developed the FedAvg algorithm, incorporating multiple local updates into distributed SGD [58] to relieve communication bottlenecks within conventional star-topology FL. However, FedAvg convergence analysis makes assumptions such as bounded gradients [28, 56, 41] or bounded gradient dissimilarity [45, 12], showing it is not resistant to non-i.i.d. data. To tackle this issue, numerous techniques have been proposed in the literature, including incorporating static/dynamic regularizers [27, 1, 57, 16], adaptive control variables [29, 8, 51, 50], and/or gradient tracking methods [18, 32]. Despite these efforts, existing FL algorithms are not easily extendable to HFL due to the timescale mismatch of multi-level model aggregations induced by the hierarchical system topology. Optimizing these algorithms and ensuring their theoretical convergence in the presence of hierarchical model drift remains unsolved. Our paper addresses these issues through a principled multi-timescale gradient correction method.

Hierarchical FL. The authors of [5, 30, 13, 49, 55, 47] explored a new FL branch, HFL, tailored for hierarchical systems consisting of a central server, group aggregators, and clients. To tackle the issue of limited communication resources, the authors of [5] developed a FedAvg-like algorithm called hierarchical FedAvg tailored to HFL, and analyzed its convergence behavior. However, their algorithm is built upon an assumption of i.i.d. data. Another work [47] investigated the convergence behavior of hierarchical FedAvg under the non-i.i.d. setup. However, the convergence bound becomes worse as the extent of data heterogeneity increases, making the algorithm vulnerable to non-i.i.d. data characteristics. In [34], ProxSkip-HUB is introduced, but requires clients to compute full batch gradients and upload them to group aggregators after every iteration, which is impractical especially when training large-scale models. Overall, there is still a lack of an algorithm that fully addresses the unique challenge of HFL, i.e., the multi-timescale model drift problem, with theoretical guarantees. We fill this gap by introducing multi-timescale gradient correction and providing theoretical insights.

Gradient tracking/correction. Both gradient tracking and gradient correction aim to fix the local updating directions of clients to mitigate the impact of model drift caused by data heterogeneity. The gradient tracking concept was originally proposed and analyzed in [9] and then extended to consider various factors like time-vary graphs and asynchronous updates [37, 40, 42, 54]. Subsequently, SCAFFOLD [18] applied gradient tracking in FL to mitigate the impact of data heterogeneity across clients, ensuring convergence and stability in non-i.i.d. settings. More recently, in [32, 2], the authors demonstrate the effectiveness of gradient tracking in fully decentralized FL, where clients conduct model aggregations through local client-to-client communications. In [6, 43], gradient tracking is further studied in a semi-decentralized FL setup. Compared to all prior research, our work is the earliest attempt to design an algorithm specifically tailored to multi-timescale model drift in HFL and its training process with periodic local/global aggregations. This presents new challenges in our algorithm design and convergence analysis due to the coupling of our correction terms through their updates at different timescales. In Section 5, we empirically validate the effectiveness of our approach over the prior gradient correction method.

2 Background and Motivation

2.1 Problem Setup: Hierarchical FL

We consider the hierarchical system depicted in Fig. 1. The central server is connected to N group aggregators, each linked to the clients within its region, defined as a group. Each group j ∈ { 1 , 2 , . . . , N } consists of a set of n j non-overlapping clients, denoted C j , resulting in a total of ∑ N j =1 n j clients within the system. Each client i has its own local data distribution D i . The goal of HFL is to construct an optimal global model x ∗ considering the data distributions of all clients in the system. The role of each group aggregator j involves coordinating the training for the n j clients within its region, while the central server orchestrates the training across all N groups through

Client 4

x

∗

1

x

∗

2

x

∗

3

x

∗

4

¯

x

∗

1

¯

x

∗

2

x

∗

Initial

model

Client 1

(a) No gradient correction

(b) Only client-group gradient correction

(c) Multi-timescale gradient correction (Ours)

Client 2

Client 3

Group 1

optimum

Global

optimum

Group-global

correction

Client-group correction term

Corrected gradient at group 2

Corrected gradient at group 1

Client-group correction term

Corrected gradient

Group-global correction term

Local gradient at each client

Initial

model

Initial

model

x

∗

1

x

∗

2

x

∗

3

x

∗

4

Group 2

optimum

x

∗

1

x

∗

2

x

∗

3

x

∗

4

: Optimal point of each client

z

1

z

2

z

1

y

1

z

4

y

2

Client-group

correction

z

3

z

4

Client1

2

3

4

interaction with the group aggregators. We can formally state the HFL learning objective as follows:

Here, f : R d → R denotes the global loss function, f j : R d → R is the loss specific to group j , and F i : R d → R represents the local loss for client i . In addition, ξ i is the data point sampled from distribution D i . Note that our analysis can be easily extended to a weighted average form of (1) by incorporating positive coefficients for each f j ( x ) or F i ( x ) . For simplicity, these coefficients are assumed to be included in F i ( x ) as in previous works [18, 48].

2.2 Limitation of Existing Works

In HFL algorithms, group aggregations are conducted after every H local client updates, while global aggregations are performed after every E group aggregations, introducing different timescales. Moreover, different forms of data heterogeneity exist in HFL: (i) intra-group non-i.i.d., due to data heterogeneity across different clients i ∈ C j , and (ii) inter-group non-i.i.d., arising from data heterogeneity across different groups C 1 , ..., C N . These lead to client model drift and group model drift , respectively. The model drifts induced by multi-level data heterogeneity at different timescales hinder hierarchical FedAvg from converging. In Fig. 2(a), we see that during local training, each local model gradually converges towards the optimal point of its respective client's objective function. Hence, to guarantee theoretical convergence, existing HFL works either assume an i.i.d. setup [5] or rely on a bounded gradient dissimilarity assumption similar to the following [47, 15]:

The first inequality is employed to limit the group drift, i.e., the deviation of group gradients from the global gradient, while the second one bounds the client drift, i.e., the divergence of client gradients from their group gradient. As a result, the convergence bounds of algorithms in these works become worse as data heterogeneity increases (i.e., as δ 1 or δ 2 increase) [19]. Our approach, developed next, does not require these assumptions and remains stable regardless of the extent of data heterogeneity.

3 Algorithm

3.1 Intuition: Gradient Correction in Hierarchical FL

̸

When relying on multiple local SGD iterations as in hierarchical FedAvg, the model update process is not stable even when the model has reached the optimal x ∗ satisfying ∇ f ( x ∗ ) = 0 . Specifically, with γ as the learning rate, we have x ∗ = x ∗ -γ ∇ F i ( x ∗ ) , as the global optimum x ∗ may not necessarily be optimal for each client's local loss due to data heterogeneity, i.e., ∇ F i ( x ∗ ) = 0 [39]. Correcting the client gradient ∇ F i ( x ∗ ) to the global gradient ∇ f ( x ∗ ) is thus necessary to stabilize the process. 1

̸

Motivation and idea. In HFL, however, due to multi-level aggregations occurring at different timescales, it is infeasible to directly correct the client gradient to the global gradient. In particular,

clients are not able to communicate with the central server directly, and there are multiple group aggregation steps before the group aggregators communicate with the main server. Our idea is thus to inject two gradient correction terms: client-group correction and group-global correction. Specifically, the desired iteration to obtain the updated model x new at the optimal point x ∗ can be written as

where ∇ f j ( x ∗ ) -∇ F i ( x ∗ ) and ∇ f ( x ∗ ) -∇ f j ( x ∗ ) represent client-group and group-global correction terms, respectively. Since ∇ f ( x ∗ ) = 0 , the two correction terms will enable the model to remain at the optimal point. Given this intuition, the ideal local iteration at client i ∈ C j can be written as

where t , e , and h represent global communication rounds, group communication rounds, and client local iterations, respectively, ¯ x t,e j,h = 1 n j ∑ i ∈C j x t,e i,h is the averaged model within group j , and ¯ x t,e h = 1 N ∑ N j =1 1 n j ∑ i ∈C j x t,e i,h is the averaged model across the system. Based on (4), we expect to bring each client model closer to the global optima during local updates, as illustrated in Fig. 2(c).

Challenge encountered in HFL. However, it is important to note that the update process in (4) still cannot be directly used in HFL. This is because client-group communication and group-global communication do not occur at every iteration of HFL training; instead, they happen at different timescales, and clients are not able to obtain the current group information ∇ f j (¯ x t,e i,h ) and global information ∇ f (¯ x t,e i,h ) at every local iteration. We next propose a strategy that mimics the gradient correction described above while ensuring theoretical convergence.

3.2 Multi-Timescale Gradient Correction (MTGC)

Tackling multi-timescale model drifts. To approximate (4) during HFL training, we introduce two control variables z and y that track/approximate ∇ f j -∇ F i and ∇ f -∇ f j , respectively. The variables z and y are then employed to correct the local gradients to prevent model drifts. The challenge here is to keep updating z and y appropriately in the multi-timescale communication scenario, given that communications between the clients and group aggregator, and between the group aggregators and global aggregator, are not always feasible. We propose a strategy to update z after every H local iterations, i.e., whenever each client is able to communicate with the group aggregator, allowing the group information to be updated and shared among the clients within the same group. Similarly, we propose a strategy to update y after every E group aggregations, i.e.,

whenever the group aggregators are able to communicate with the global aggregator, enabling the global information to be refreshed and shared across all clients in the system. We name our strategy multi-timescale gradient correction (MTGC) due to the updates of z and y occurring in different timescales, to tackle the issue of multi-level model drift coupled across the hierarchy in HFL.

In Fig. 2(c), we illustrate MTGC during client-side model updates. In particular, at each local iteration h of group round e of global round t , each client i ∈ C j updates its local model as follows:

(i) Client-group correction term. In (5), z t,e i is responsible for correcting the gradient of client i ∈ C j towards the gradient of group j at the e -th group aggregation of global round t . After every group aggregation e at global round t , this term is updated at each client i as follows:

(ii) Group-global correction term. y t j in (5) aims to correct the gradient of group j towards the global gradient. At the end of global round t , this term is updated at group aggregator j as follows:

Key remarks. The updating policies for z t,e i and y t j follow similar patterns to the ideal corrections outlined in (4). Here, we observe that ∑ i ∈C j z t,e i = 0 , ∀ j and ∑ N j =1 y t j = 0 , indicating that the correction terms do not have an impact on the per-iteration model averages. Instead, the introduction of z t,e i and y t j eliminates model drifts of clients and groups, respectively, during local iterations. Intuitively, as the iteration approaches the global optimal point, we expect z t,e i →∇ f j ( x ∗ ) -∇ F i ( x ∗ ) and y t j →∇ f ( x ∗ ) -∇ f j ( x ∗ ) so that the update in (5) stabilizes at the global optimal point. We also see that z t,e i and y t j are coupled (5), i.e., the update of one of the terms affects x which in turn affects the other one, raising challenges for theoretical analysis. In Section 4, we will guarantee convergence of MTGC in general non-convex settings without relying on bounded data heterogeneity assumptions.

MTGC algorithm. The overall procedure of our training strategy is summarized in Algorithm 1, where we rewrite the updates of z t,e i and y t j in a different but equivalent manner to facilitate practical implementation of MTGC. Compared to hierarchical FedAvg, which does not consider any correction terms, we see that no additional communication is required for MTGC within each group round e . Additional communication is introduced only after E group aggregations for initializing z t, 0 i (Line 4) 2 and broadcasting y t +1 j (obtained in Line 14) to the clients in C j . We will see in Section 5 that these marginal additional costs lead to significant performance enhancements for HFL settings.

Generalization to arbitrary number of levels. The proposed MTGC algorithm can be extended to an HFL system architecture with an arbitrary number of levels. Further discussions and experimental results for a three-level case are provided in Appendix E.

3.3 Connection with SCAFFOLD

When the number of groups reduces to N = 1 with E = 1 , we have y t j = 0 (no group-global correction), and thus MTGC reduces to SCAFFOLD [18]. In SCAFFOLD, at each round t , clients perform local updates according to x t i,h +1 = x t i,h -γ ( ∇ F i ( x t i,h , ξ t i,h ) -c t i + c t ) , h = 0 , 1 , . . . , H -1 , where c t +1 i = c t i -c t + 1 Hγ ( ¯ x t -x t i,H ) , and the server aggregates local models and controlling variables as ¯ x t +1 = 1 N ∑ N i =1 x t i,H and c t +1 = 1 N ∑ N i =1 c t +1 i . We can show that c t i -c t in SCAFFOLD plays the same role as z t,e i in MTGC. However, the additional term y t j introduced in MTGC for the multi-level setting makes the convergence guarantee more challenging, as y t j is coupled with z t,e i and both are updated at different time scales. These aspects will be thoroughly examined next.

4 Convergence Analysis

In this section, we establish a convergence guarantee for the proposed MTGC algorithm. Our theoretical analysis relies on the following standard assumptions commonly used in the literature on stochastic optimization and FL under non-convex settings [18, 44, 4].

Assumption 1. Each local loss function F i is differentiable and L -smooth, i.e., there exists a positive constant L such that for any x and y , ∥∇ F i ( x ) -∇ F i ( y ) ∥ ≤ L ∥ y -x ∥ , ∀ i .

Assumption 2. The stochastic gradient ∇ F i ( x , ξ i ) is an unbiased estimate of the true gradient, i.e., E ξ i ∼D i [ ∇ F i ( x , ξ i )] = ∇ F i ( x ) , ∀ x and the variance of the stochastic gradient ∇ F i ( x , ξ i ) is uniformly bounded as E ξ i ∼D i ∥∇ F i ( x , ξ i ) -∇ F i ( x ) ∥ 2 ≤ σ 2 , ∀ x .

Note that (i) global aggregation, (ii) the update of upper-level correction variable y and local aggregation, and (iii) the update of lower-level correction variable z are performed at different timescales in MTGC. If we directly consider {∇ f (¯ x t ) } as in SCAFFOLD, it is difficult to capture the effects of group aggregation and correction variable z . Moreover, it is hard to establish a tight connection between ∇ f (¯ x t ) and x t,e i,h , ∀ i, h, τ since there is a large lag between x t,e i,h and ¯ x t . To tackle this, we introduce a new metric, which is the gradient ∇ f (ˆ x t,e ) at virtual sequence { ˆ x t,e = 1 N ∑ N j =1 ¯ x t,e j } , to characterize the convergence of MTGC.

We next state our main theoretical results. All the proofs are provided in Appendix F:

Theorem 4.1. Suppose Assumptions 1 and 2 hold and the learning rate satisfies γ ≤ 1 40 EHL . Then the iterates { ˆ x t,e } obtained by the MTGC algorithm satisfy

There are two key steps in our proof. The first is the characterization of the evolution of ∥ ∥ z t,e i + ∇ F i ( ¯ x t,e j ) -∇ f j ( ¯ x t,e j )∥ ∥ 2 and ∥ ∥ y t j + ∇ f j ( ˆ x t,e ) -∇ f ( ˆ x t,e )∥ ∥ 2 . By bounding these values that capture the error between each control variable and the ideal correction, we are able to establish a connection between the local updating direction and the global gradient without relying on the bounded gradient dissimilarity assumption, laying the foundation for the whole proof. The second is that we extracted a recursive relationship for the accumulation of group-level and client-level model drifts, and designed a novel Lyapunov function to mitigate the interplay impact between these drifts. Further details are provided in the appendix.

Applying an appropriate learning rate γ to Algorithm 1 yields the following corollary:

Corollary 4.1. Under the assumptions of Theorem 4.1, let F 0 = f (¯ x 0 ) -f ∗ . Then there exists a learning rate γ ≤ 1 40 EHL such that the iterates { ˆ x t,e } satisfy

̸

Discussions. Corollary 4.1 provides the convergence upper bound of the MTGC algorithm. It shows that the error approaches zero as T →∞ . If σ = 0 , the upper bound is dominated by the first term in the right-hand side of (9), which characterizes the speed of convergence of MTGC to a stationary point in the stochastic case. This reveals MTGC achieves linear speedup in the number of group aggregations E and local updates H . In other words, we can attain the same level of performance with less global communication rounds, i.e., a smaller value of T , by increasing the number of local iterations, i.e., H , and group aggregations, i.e., E . When considering the special case n j ′ = n, ∀ j ′ ∈ { 1 , 2 , . . . , N } with uniform client numbers, the rate becomes O ( √ F 0 Lσ 2 / √ NnTEH ) . This implies that MTGC attains linear speedup in the number of clients as well.

Moreover, we also see that our convergence rate recovers the results of SCAFFOLD when the number of groups reduces to N = 1 and the number of group aggregations reduces to E = 1 (see Appendix G for more discussions). We also highlight that, different from prior works on HFL where the convergence bound becomes worse as the extent of data heterogeneity increases, our bound is stable against multi-level non-i.i.d. data due to the multi-timescale gradient correction approach.

5 Experimental Results

5.1 Setup

Dataset, model, hyperparameters, and compute setting. In our experiments, we consider four widely used datasets: EMNIST-Letters (EMNIST-L) [7], Fashion-MNIST [53], CIFAR-10 [23], and

0

20

40

Global Rounds

84

86

88

90

92

Testing Accuracy (%)

EMNIST-L

0

25

50

75

100

Global Rounds

84

85

86

87

88

89

90

Testing Accuracy (%)

Fashion-MNIST

0

25

50

75

100

Global Rounds

60

65

70

75

80

85

Testing Accuracy (%)

CIFAR-10

0

25

50

75

100

Global Rounds

30

40

50

Testing Accuracy (%)

CIFAR-100

MTGC

SCAFFOLD

FedProx

HFedAvg

FedDyn

0

25

50

75

100

Global Rounds

86

88

90

92

Group i.i.d.

& client non-i.i.d.

Testing Accuracy (%)

EMNIST-L

0

25

50

75

100

Global Rounds

86

88

90

Testing Accuracy (%)

Fashion-MNIST

0

25

50

75

100

Global Rounds

65

70

75

80

85

Testing Accuracy (%)

CIFAR-10

0

25

50

75

100

Global Rounds

30

35

40

45

50

55

Testing Accuracy (%)

CIFAR-100

0

25

50

75

100

Global Rounds

86

88

90

92

Group non-i.i.d.

& client i.i.d.

Testing Accuracy (%)

0

25

50

75

100

Global Rounds

85

86

87

88

89

90

Testing Accuracy (%)

0

25

50

75

100

Global Rounds

65

70

75

80

85

Testing Accuracy (%)

0

25

50

75

100

Global Rounds

30

35

40

45

50

55

Testing Accuracy (%)

0

25

50

75

100

Global Rounds

86

88

90

92

Group non-i.i.d.

& client non-i.i.d.

Testing Accuracy (%)

0

25

50

75

100

Global Rounds

85

86

87

88

89

90

Testing Accuracy (%)

0

25

50

75

100

Global Rounds

65

70

75

80

85

Testing Accuracy (%)

0

25

50

75

100

Global Rounds

30

35

40

45

50

55

Testing Accuracy (%)

MTGC

Local Correction

Group Correction

HFedAvg

CIFAR-100 [23]. The former two are processed through a multi-layer perceptron (MLP) model, featuring two hidden layers, each comprising 200 neurons, and ending with a softmax layer. For the CIFAR-10 classification task, we employ a convolutional neural network (CNN) following the architecture outlined in seminal work [35]. For CIFAR-100, we adopt a ResNet-18 model with batch normalization layers substituted by group normalization layers. Across all algorithms considered, we maintain a consistent learning rate η = 0 . 1 and batch size 50 . We conduct the experiments based on a cluster of 3 NVIDIA A100 GPUs with 40 GB memory. Our code is based on the framework of [1].

FL data distribution. We set the total number of clients as 100 , evenly distributed over N = 10 groups. We also study the effect of N in Appendix B. We consider three different data distribution settings: (i) group i.i.d. & client non-i.i.d., (ii) group non-i.i.d. & client i.i.d., and (iii) group non-i.i.d. &client non-i.i.d. scenarios. First, in the group i.i.d. & client non-i.i.d. case, the training dataset is initially divided uniformly and randomly into N segments corresponding to N groups. Subsequently, each segment is further divided into 100 /N partitions for the clients using a Dirichlet distribution [1]. Second, in the group non-i.i.d. & client i.i.d. case, the dataset is first segmented into N partitions for the groups using a Dirichlet distribution, followed by a uniform random distribution of each segment to 100 /N clients. Finally, when both groups and clients are non-i.i.d., the dataset is split into N segments for the groups using a Dirichlet distribution, and then, each group's segment is distributed among 100 /N clients through a Dirichlet distribution. The Dirichlet parameter is set to 0 . 1 .

5.2 Results and Discussion

Comparison with conventional FL algorithms. For comparison, we first apply the well-known FL methods, FedProx [27], SCAFFOLD [18], and FedDyn [1], to HFL, by running their training algorithms within each group of the hierarchical system. We also consider HFedAvg [47] as a baseline. Fig. 3 compares MTGC with these baselines in the group non-i.i.d. & client non-i.i.d. case. We observe that MTGC outperforms all the considered conventional algorithms, achieving the highest testing accuracy, especially for the complicated CIFAR-100 dataset. FedDyn achieves the lowest performance, demonstrating significant variance and instability. The significant performance gap between MTGC and FedDyn, in particular, can be attributed to the hierarchical setup disrupting the special structure of FedDyn. This result reveals that some algorithms designed for the conventional star-topology FL may be non-trivial to be extended to hierarchical setups. The overall results confirm the effectiveness of our approach that effectively tackles the multi-timescale drift problem in HFL.

Comparison with gradient correction baselines. In Fig. 4, we compare MTGC with the gradient correction baselines. Specifically, we apply local correction ( z t,e i ) to HFedAvg, and group correction ( y t j ) to HFedAvg. These baselines can be viewed as schemes applying SCAFFOLD [18] within each group and across groups, respectively. We also report the results of the original HFedAvg to see the effects of gradient correction clearly. We make the following key observations. First, the testing accuracy achieved by HFedAvg decreases as the extent of data heterogeneity increases, e.g., from the first or second row to the third row in Fig. 4. This shows that data heterogeneity hinders the convergence of HFedAvg. Second, with the assistance of local or group correction, the algorithm attains a higher accuracy. In the case of group i.i.d. & client non-i.i.d., HFedAvg augmented with client local correction performs better than the variant with group correction. Conversely, in the scenario where groups are non-i.i.d. and clients are i.i.d., the opposite holds. This can be explained by the dominance of data heterogeneity in each case. In the former scenario, because the heterogeneity is primarily at the client-level, local client correction becomes more beneficial. On the other hand, in the latter scenario, where the heterogeneity shifts to the group level, group correction becomes more advantageous. Finally, we see that MTGC consistently outperforms baselines under all settings, where the performance gains brought by the multi-timescale gradient correction become more significant when it comes to the group non-i.i.d.& client non-i.i.d. case.

Speedup in H and E . In Table 5.1, we investigate the effects H and E , which determine the periods of group aggregation and global aggregation in HFL. We report the number of global rounds required to attain the desired testing accuracy of 80% for CIFAR-10 under different settings. We have the following observations: As E or H increases, the required number of global rounds of MTGC for achieving the desired accuracy decreases. This demonstrates the speedup of the proposed algorithm in the number of local iterations and group aggregations, which fits well with our theory discussed in Section 4. In addition, the speedup achieved by MTGC compared to HFedAvg gets more significant

as E or H increases. For instance, in the group i.i.d. & client non-i.i.d. case, MTGC attains 3 . 3 × speedup when E = 10 , H = 20 , which increases to 4 . 7 × when E = 10 , H = 40 . This reveals that MTGC utilizes local iterations better compared with the baselines.

Impact of data heterogeneity. Consistent with the results in Fig. 4, we see from Table 5.1 that the required number of global rounds of HFedAvg increases as data heterogeneity increases, while MTGC is more stable against non-i.i.d. data. The gain of MTGC over HFedAvg becomes evident as data heterogeneity increases, confirming the effectiveness of our multi-timescale gradient correction approach for addressing the unique challenges of HFL.

Further experiments. Additional experimental results including the impacts of hierarchical system parameters and the performance in 3 -level HFL are provided in Appendices B and E.

6 Conclusion and Limitation

We have proposed MTGC, a multi-timescale gradient correction approach for HFL. Embedded with control variables updated in different timescales, MTGC effectively corrects gradient biases and alleviates both client model drift and group model drift in hierarchical setups. We established the convergence bound of MTGC in the non-convex setup and showed its stability against multi-level data heterogeneity. Finally, we confirmed the advantage of our MTGC through extensive experiments in different non-i.i.d. HFL settings. A limitation of our work is that despite providing experiments for HFL systems with more than two levels (in Appendix E), our convergence analysis focused on the two-level case, which provides an interesting future direction of investigation.

Acknowledgments

This work was supported by the National Science Foundation (NSF) under grants CNS-2146171 and CPS-2313109, and by the Air Force Office of Scientific Research (AFOSR) under grant FA9550-241-0083.

References

Acar, D.A.E., Zhao, Y., Matas, R., Mattina, M., Whatmough, P., Saligrama, V.: Federated learning based on dynamic regularization. In: International Conference on Learning Representations (2020)

Alghunaim, S.A.: Local exact-diffusion for decentralized optimization and learning. IEEE Transactions on Automatic Control (2024)

Bao, W., Wang, H., Wu, J., He, J.: Optimizing the collaboration structure in cross-silo federated learning. In: International Conference on Machine Learning. pp. 1718-1736. PMLR (2023)

Bottou, L., Curtis, F.E., Nocedal, J.: Optimization methods for large-scale machine learning (2018)

Castiglia, T., Das, A., Patterson, S.: Multi-level local SGD: Distributed SGD for heterogeneous hierarchical networks. In: International Conference on Learning Representations (2020)

Chen, E., Wang, S., Brinton, C.G.: Taming subnet-drift in D2D-enabled fog learning: A hierarchical gradient tracking approach. arXiv preprint arXiv:2312.04728 (2023)

Cohen, G., Afshar, S., Tapson, J., van Schaik, A.: EMNIST: an extension of mnist to handwritten letters (2017)

Condat, L., Agarsk` y, I., Malinovsky, G., Richtárik, P.: TAMUNA: Doubly accelerated federated learning with local training, compression, and partial participation. arXiv preprint arXiv:2302.09832 (2023)

Di Lorenzo, P., Scutari, G.: Next: In-network nonconvex optimization. IEEE Transactions on Signal and Information Processing over Networks 2 (2), 120-136 (2016)

Fang, W., Han, D.J., Brinton, C.G.: Submodel partitioning in hierarchical federated learning: Algorithm design and convergence analysis. In: ICC 2024-IEEE International Conference on Communications. pp. 268-273. IEEE (2024)

Fang, W., Yu, Z., Jiang, Y., Shi, Y ., Jones, C.N., Zhou, Y .: Communication-efficient stochastic zeroth-order optimization for federated learning. IEEE Transactions on Signal Processing 70 , 5058-5073 (2022)

Haddadpour, F., Mahdavi, M.: On the convergence of local descent methods in federated learning. arXiv preprint arXiv:1910.14425 (2019)

Hosseinalipour, S., Azam, S.S., Brinton, C.G., Michelusi, N., Aggarwal, V., Love, D.J., Dai, H.: Multi-stage hybrid federated learning over large-scale D2D-enabled fog networks. IEEE/ACM transactions on networking 30 (4), 1569-1584 (2022)

Hosseinalipour, S., Brinton, C.G., Aggarwal, V., Dai, H., Chiang, M.: From federated to fog learning: Distributed machine learning over heterogeneous wireless networks. IEEE Communications Magazine 58 (12), 41-47 (2020)

Jiang, X., Zhu, H.: On the convergence of hierarchical federated learning with partial worker participation. In: The 40th Conference on Uncertainty in Artificial Intelligence (2024)

Jiang, X., Rodomanov, A., Stich, S.U.: Federated optimization with doubly regularized drift correction. In: Forty-first International Conference on Machine Learning (2024)

Kairouz, P., McMahan, H.B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A.N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., et al.: Advances and open problems in federated learning. Foundations and trends® in machine learning 14 (1-2), 1-210 (2021)

Karimireddy, S.P., Kale, S., Mohri, M., Reddi, S., Stich, S., Suresh, A.T.: SCAFFOLD: Stochastic controlled averaging for federated learning. In: International conference on machine learning. pp. 5132-5143. PMLR (2020)

Khaled, A., Mishchenko, K., Richtárik, P.: Tighter theory for local SGD on identical and heterogeneous data. In: International Conference on Artificial Intelligence and Statistics. pp. 4519-4529. PMLR (2020)

Kim, H., Feamster, N.: Improving network management with software defined networking. IEEE Communications Magazine 51 (2), 114-119 (2013)

Koloskova, A., Loizou, N., Boreiri, S., Jaggi, M., Stich, S.: A unified theory of decentralized SGD with changing topology and local updates. In: International Conference on Machine Learning. pp. 5381-5393. PMLR (2020)

Konecn` y, J., McMahan, H.B., Yu, F.X., Richtárik, P., Suresh, A.T., Bacon, D.: Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492 8 (2016)

Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images. Toronto, ON, Canada (2009)

Lan, G., Liu, X.Y., Zhang, Y., Wang, X.: Communication-efficient federated learning for resource-constrained edge devices. IEEE Transactions on Machine Learning in Communications and Networking (2023)

Li, H., Ota, K., Dong, M.: Learning iot in edge: Deep learning for the internet of things with edge computing. IEEE network 32 (1), 96-101 (2018)

Li, T., Sahu, A.K., Talwalkar, A., Smith, V.: Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine 37 (3), 50-60 (2020)

Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A., Smith, V.: Federated optimization in heterogeneous networks. In: Proceedings of Machine learning and systems. vol. 2, pp. 429-450 (2020)

Li, X., Huang, K., Yang, W., Wang, S., Zhang, Z.: On the convergence of FedAvg on Non-IID data. In: International Conference on Learning Representations (2019)

Liang, X., Shen, S., Liu, J., Pan, Z., Chen, E., Cheng, Y.: Variance reduced local SGD with lower communication complexity. arXiv preprint arXiv:1912.12844 (2019)

Liu, L., Zhang, J., Song, S., Letaief, K.B.: Client-edge-cloud hierarchical federated learning. In: IEEE International Conference on Communications. pp. 1-6 (2020)

Liu, L., Zhang, J., Song, S., Letaief, K.B.: Hierarchical federated learning with quantization: Convergence analysis and system design. IEEE Transactions on Wireless Communications 22 (1), 2-18 (2022)

Liu, Y., Lin, T., Koloskova, A., Stich, S.U.: Decentralized gradient tracking with local steps. Optimization Methods and Software pp. 1-28 (2024)

Ma, J., Long, G., Zhou, T., Jiang, J., Zhang, C.: On the convergence of clustered federated learning. arXiv preprint arXiv:2202.06187 (2022)

Malinovsky, G., Yi, K., Richtárik, P.: Variance reduced ProxSkip: Algorithm, theory and application to federated learning. In: Advances in Neural Information Processing Systems. vol. 35, pp. 15176-15189 (2022)

McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial intelligence and statistics. pp. 1273-1282. PMLR (2017)

Mishchenko, K., Malinovsky, G., Stich, S., Richtárik, P.: ProxSkip: Yes! local gradient steps provably lead to communication acceleration! finally! In: International Conference on Machine Learning. pp. 15750-15769. PMLR (2022)

Nedic, A., Olshevsky, A., Shi, W.: Achieving geometric convergence for distributed optimization over time-varying graphs. SIAM Journal on Optimization 27 (4), 2597-2633 (2017)

Nguyen, V.D., Chatzinotas, S., Ottersten, B., Duong, T.Q.: Fedfog: Network-aware optimization of federated learning over wireless fog-cloud systems. IEEE Transactions on Wireless Communications 21 (10), 8581-8599 (2022)

Pathak, R., Wainwright, M.J.: Fedsplit: An algorithmic framework for fast federated optimization. In: Advances in Neural Information Processing Systems. vol. 33, pp. 7057-7066 (2020)

Scutari, G., Sun, Y .: Distributed nonconvex constrained optimization over time-varying digraphs. Mathematical Programming 176 , 497-544 (2019)

Stich, S.U.: Local SGD converges fast and communicates little. In: International Conference on Learning Representations (2019)

Tian, Y., Sun, Y., Scutari, G.: Achieving linear convergence in distributed asynchronous multiagent optimization. IEEE Transactions on Automatic Control 65 (12), 5264-5279 (2020)

Wang, H., Chi, Y.: Communication-efficient federated optimization over semi-decentralized networks. In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 13241-13245. IEEE (2024)

Wang, J., Charles, Z., Xu, Z., Joshi, G., McMahan, H.B., y Arcas, B.A., Al-Shedivat, M., Andrew, G., Avestimehr, S., Daly, K., Data, D., Diggavi, S., Eichner, H., Gadhikar, A., Garrett, Z., Girgis, A.M., Hanzely, F., Hard, A., He, C., Horvath, S., Huo, Z., Ingerman, A., Jaggi, M., Javidi, T., Kairouz, P., Kale, S., Karimireddy, S.P., Konecny, J., Koyejo, S., Li, T., Liu, L., Mohri, M., Qi, H., Reddi, S.J., Richtarik, P., Singhal, K., Smith, V., Soltanolkotabi, M., Song, W., Suresh, A.T., Stich, S.U., Talwalkar, A., Wang, H., Woodworth, B., Wu, S., Yu, F.X., Yuan, H., Zaheer, M., Zhang, M., Zhang, T., Zheng, C., Zhu, C., Zhu, W.: A field guide to federated optimization (2021)

Wang, J., Liu, Q., Liang, H., Joshi, G., Poor, H.V .: Tackling the objective inconsistency problem in heterogeneous federated optimization. In: Advances in Neural Information Processing Systems. vol. 33, pp. 7611-7623 (2020)

Wang, J., Liu, Q., Liang, H., Joshi, G., Poor, H.V.: A novel framework for the analysis and design of heterogeneous federated learning. IEEE Transactions on Signal Processing 69 , 5234-5249 (2021)

Wang, J., Wang, S., Chen, R.R., Ji, M.: Demystifying why local aggregation helps: Convergence analysis of hierarchical SGD. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 36, pp. 8548-8556 (2022)

Wang, S., Ji, M.: A lightweight method for tackling unknown participation statistics in federated averaging. In: International Conference on Learning Representations (2024)

Wang, Z., Xu, H., Liu, J., Huang, H., Qiao, C., Zhao, Y.: Resource-efficient federated learning with hierarchical aggregation in edge computing. In: IEEE Conference on Computer Communications. pp. 1-10 (2021)

Wu, F., Guo, S., Qu, Z., He, S., Liu, Z., Gao, J.: Anchor sampling for federated learning with partial client participation. In: International Conference on Machine Learning. pp. 37379-37416. PMLR (2023)

Wu, F., Guo, S., Wang, H., Zhang, H., Qu, Z., Zhang, J., Liu, Z.: From deterioration to acceleration: A calibration approach to rehabilitating step asynchronism in federated optimization. IEEE Transactions on Parallel and Distributed Systems 34 (5), 1548-1559 (2023)

Wu, F., Li, Z., Li, Y., Ding, B., Gao, J.: Fedbiot: Llm local fine-tuning in federated learning without full model. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. pp. 3345-3355 (2024)

Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 (2017)

Xu, J., Tian, Y ., Sun, Y ., Scutari, G.: Distributed algorithms for composite optimization: Unified framework and convergence analysis. IEEE Transactions on Signal Processing 69 , 3555-3570 (2021)

Yang, H.: H-FL: A hierarchical communication-efficient and privacy-protected architecture for federated learning. arXiv preprint arXiv:2106.00275 (2021)

Yu, H., Yang, S., Zhu, S.: Parallel restarted SGD with faster convergence and less communication: Demystifying why model averaging works for deep learning. In: Proceedings of the AAAI conference on artificial intelligence. vol. 33, pp. 5693-5700 (2019)

Zhang, X., Hong, M., Dhople, S., Yin, W., Liu, Y.: FedPD: A federated learning framework with adaptivity to Non-IID data. IEEE Transactions on Signal Processing 69 , 6055-6070 (2021)

Zinkevich, M., Weimer, M., Li, L., Smola, A.: Parallelized stochastic gradient descent. In: Advances in Neural Information Processing Systems. vol. 23 (2010)

Appendix

A Connection Between HFL and Cluster FL

Our work focuses on HFL, employing a multi-layered structure consisting of local nodes, local aggregators, and a central server. Both clustered FL and HFL aim to improve FL learning efficiency by leveraging structured client groupings. The difference between them lies in the grouping criteria. HFL focuses on collaborative training over a given network topology, where clients are generally grouped based on their geographical location or network connection status, and aims to build a single global model under this setting. CFL groups clients to optimize model training, with different global models constructed depending on the group. [33] demonstrates how dynamic clustering based on data distributions can enhance model performance. [3] explores alleviating negative transfer from collaboration by clustering clients into non-overlapping coalitions based on their distribution distances and data quantities.

B Additional Experiments on CIFAR-10

0

25

50

75

100

Global Communication Rounds

0

.

60

0

.

65

0

.

70

0

.

75

0

.

80

0

.

85

Testing Accuracy

N

= 4

, n

j

= 25

HGC

Local Correction

Cell Correction

HFedAvg

0

25

50

75

100

Global Communication Rounds

0

.

60

0

.

65

0

.

70

0

.

75

0

.

80

0

.

85

Testing Accuracy

N

= 10

, n

j

= 10

HGC

Local Correction

Cell Correction

HFedAvg

0

25

50

75

100

Global Communication Rounds

0

.

60

0

.

65

0

.

70

0

.

75

0

.

80

0

.

85

Testing Accuracy

N

= 25

, n

j

= 4

HGC

Local Correction

Cell Correction

HFedAvg

0

25

50

75

100

Global Communication Rounds

0

.

60

0

.

65

0

.

70

0

.

75

0

.

80

0

.

85

Testing Accuracy

N

= 50

, n

j

= 2

HGC

Local Correction

Cell Correction

HFedAvg

The impact of system parameters. Fig. 5 shows how the performance of MTGC changes with different numbers of groups and clients in each group. From this figure, we observe that as the number of clients in each group, i.e., n j increases, client correction becomes more important. On the other hand, as the number of groups increases, the algorithm with group correction performs better than that with client correction.

0

20

40

60

80

100

Global Communication Rounds

0

.

60

0

.

65

0

.

70

0

.

75

0

.

80

0

.

85

Testing Accuracy

Cell i.i.d.

& client non-i.i.d.

E

= 10

, H

= 20

E

= 10

, H

= 30

E

= 10

, H

= 40

H

= 20

, E

= 20

H

= 20

, E

= 30

0

20

40

60

80

100

Global Communication Rounds

0

.

60

0

.

65

0

.

70

0

.

75

0

.

80

0

.

85

Testing Accuracy

Cell non-i.i.d.

& client i.i.d.

E

= 10

, H

= 20

E

= 10

, H

= 30

E

= 10

, H

= 40

H

= 20

, E

= 20

H

= 20

, E

= 30

0

20

40

60

80

100

Global Communication Rounds

0

.

60

0

.

65

0

.

70

0

.

75

0

.

80

0

.

85

Testing Accuracy

Both Cell non-i.i.d.

& client non-i.i.d.

E

= 10

, H

= 20

E

= 10

, H

= 30

E

= 10

, H

= 40

H

= 20

, E

= 20

H

= 20

, E

= 30

The impact of local iteration and group aggregation: Fig. 6 depicts the performance of MTGC under a different number of local iterations, i.e., H , and a different number of group aggregations, i.e., E . It's clear that MTGC achieves speedup in the number of local iterations and group aggregations.

Communication cost comparison. Compared to HFedAvg, MTGC requires initializing the correction variables at the start of each global round, which adds additionaly communication overhead. Specifically, for every E steps of group aggregation, MTGC incurs an additional communication cost equivalent to one transmission of the model parameters. In other words, the per-aggregation communication complexity of MTGC is E +1 E times that of HFedAvg. To show this impact, we have added experiments comparing the communication cost and testing accuracy at the client side. This

0

20000

40000

60000

80000

100000

120000

140000

Communication Cost (Mbs)

50

55

60

65

70

75

80

85

Testing Accuracy

MTGC

SCAFFOLD

HFedAvg

FedProx

(a) Testing accuracy versus communication cost

MTGC

SCAFFOLD

FedProx

HFedAvg

0

200

400

600

800

1000

Running Time (minutes)

75% Accuracy

100 Rounds

(b) Running time comparison

experiment was conducted on CIFAR-10 dataset with E = 30 and H = 20 under both client and group non-i.i.d. setup. The model and other parameters are the same as in the original manuscript. The results are shown in Fig. 7a. The results demonstrate that MTGC achieves higher testing accuracy for a given communication cost, highlighting the efficiency and effectiveness of our approach.

Running time comparison. We compared the computation time of our MTGC algorithm with the baselines. Using NVIDIA A100 GPUs with 40 GB memory, we conducted experiments on the CIFAR-10 dataset with E = 30 and H = 20 under both client and group non-i.i.d. setup. The model and other parameters are the same as in the original manuscript. We report the required time for attaining a preset accuracy of 75% and for running 100 global rounds in Fig. 7b of the attached pdf. The speedup in the convergence makes up for the introduced computation cost per iteration due to the extra operation induced by the correction variables. Actually, the computation cost incurred by the correction variable is relatively small compared to computing gradients in a neural network using backpropagation.

C Experiments on Distribution Shift Datasets

10

20

30

40

50

Global Communication Rounds

50

55

60

65

70

75

80

Testing Accuracy (%)

MTGC

SCAFFOLD

HFedAvg

FedProx

(a) Performance comparisons under label shift on Fashion-MNIST

10

20

30

40

50

Global Communication Rounds

60

62

64

66

68

70

72

74

76

Testing Accuracy (%)

MTGC

SCAFFOLD

HFedAvg

FedProx

(b) Performance comparisons under feature shift on Fashion-MNIST

To further show the robustness, we studied the performance of MTGC under another two different non-i.i.d. scenarios: label shift and feature shift, as referenced in [3, 33]. These experiments were performed using the Fashion-MNIST dataset.

For label shift [3, 33], we randomly assign 3 classes out of 10 classes to each group with a relatively balanced number of instances per class, and then assign 2 classes to each client. As discussed in [3], label shift adds more heterogeneity to this system. According to the results shown in Fig. 8a, it is clear that the proposed algorithm is more robust against data heterogeneity. Specifically, there is less oscillation in MTGC compared with HFedAvg and the attained accuracy of MTGC in the given communication round is higher than all baselines.

5

10

15

20

25

30

Global Communication Rounds

36

38

40

42

44

46

Testing Accuracy (%)

MTGC

SCAFFOLD

HFedAvg

FedProx

(a) Performance comparison on the Shakespeare dataset based on LSTM model

10

20

30

40

50

Global Communication Rounds

40

45

50

55

60

65

70

Testing Accuracy (%)

MTGC

SCAFFOLD

HFedAvg

FedProx

(b) Performance comparison under CINIC-10 dataset

For feature shift [3], we first partition data following the group non-i.i.d. & client non-i.i.d. case as in our original manuscript, and then let clients at different groups rotate images for different angles. Concretely, for the clients at the i -th group, the angle is -50 + 10 × i . Note that this rotation is only applied to the training set. The feature shift increases the diversity between the training set and the testing set, which thus adds difficulty to this classification task. In Fig. 8b, we see that MTGC attains the best performance among these baselines.

D Additional Experiments on CINIC-10 and Shakespear Datasets

We conducted additional experiments on the larger Shakespeare and CINIC-10 datasets. For the Shakespeare dataset , we randomly pick 100 characters (people) in Shakespeare's plays. We let each client have 1,500 samples, where each sample is a sequence of 80 characters (words). Considering that there are 100 clients in the system, there are 150,000 train samples in total. This means that the number of samples is 3 times that of CIFAR-10 (or CIFAR-100), which has 50,000 train samples. The performance comparison is presented in Fig. 9a, where we use the LSTM model, the same as [1], and set the learning rate 0 . 5 , H = 75 , and E = 30 . It is seen that MTGC consistently outperforms the baseline methods in larger datasets.

The CINIC-10 dataset contains 90,000 training images, 90,000 validation images, and 90,000 test images, significantly larger than CIFAR-10 and CIFAR-100 with 60,000 images. It includes images from both CIFAR-10 and ImageNet, enhancing diversity. We believe that the larger size and diversity of CINIC-10 further confirm the validity of our experiments. The model and hyperparameters used for the CINIC-10 dataset are the same as those of the CIFAR-10 task shown in the original manuscript. As illustrated in Fig. 9b, MTGC maintains its superior performance on the CINIC-10 dataset, consistent with its performance on other tasks.

E Extension to the HFL System with Arbitrary Number of Levels

We extend MTGC for the HFL system with M levels in this subsection. For presentation ease, we adopt different notations than those used in the main text. Specifically, we denote the number of total iterations at clients as r . The aggregation periods for level m are denoted as P m . This means that the m -th level aggregator aggregates the model from the clients within its coverage after every P m local iterations. The global server is treated as the first level aggregator. Note that P m > P m +1 and P m +1 | P m , ∀ m = 1 , . . . , M -1 . We denote the model maintained at the nodes connected to the m -th level aggregator ( k 1 , k 2 , . . . , k m -1 ) as x r k 1 ,...,k m , where k m ∈ { 1 , . . . , N m } . The gradient correction term between nodes ( k 1 , k 2 , . . . , k m -1 ) and ( k 1 , k 2 , . . . , k m ) is denoted as ν r k 1 ,k 2 ,...,k m . The overall procedures are summarized in Algorithm 2.

(

k

1

, k

2

, k

3

)

(

k

1

, k

2

)

(

k

1

)

1

st

layer aggregator

2

nd

layer aggregator

3

rd

layer aggregator

(1

,

1

,

1)

(1

,

1

,

2)

(1

,

2

,

1)

(1

,

2

,

2)

(2

,

1

,

1)

(2

,

1

,

2)

(2

,

2

,

1)

(2

,

2

,

2)

(1

,

1)

(1

,

2)

(2

,

1)

(2

,

2)

(2)

(1)

ν

2

ν

2

,

2

2

nd

level correction

ν

2

,

2

,

2

3

rd

level correction

1

st

level correction

0

25

50

75

100

Global Rounds

78

80

82

84

86

88

90

Testing Accuracy (%)

EMNIST-L

0

25

50

75

100

Global Rounds

80

82

84

86

88

90

Testing Accuracy (%)

Fashion-MNIST

0

25

50

75

100

Global Rounds

50

55

60

65

70

75

80

85

Testing Accuracy (%)

CIFAR-10

0

25

50

75

100

Global Rounds

25

30

35

40

45

50

Testing Accuracy (%)

CIFAR-100

We numerically validate the performance of MTGC by conducting experiments in the three-level case as shown in Fig. 10. The results are shown in Fig. 11. The total number of clients is set to be 100 while N 1 = 4 , N 2 = 5 , N 3 = 5 . Additionally, the aggregation periods are set to be P 1 = 500 , P 2 = 100 , P 3 = 10 . The data is non-i.i.d. distributed across each level.

F Proof of the Main Results

F.1 Preliminaries

Before proceeding to the proof of the main theorem. We introduce some basic inequalities in this subsection that will be frequently used in our proof.

Lemma F.1.1. For any set of K vectors { p k } K k =1 , ∥ ∥ ∥ 1 K ∑ K k =1 p k ∥ ∥ ∥ 2 ≤ 1 K ∑ K k =1 ∥ p k ∥ 2 ,

Lemma F.1.2. For any two vectors p , q ∈ R d , ∥ p + q ∥ 2 ≤ (1 + α ) ∥ p ∥ 2 + ( 1 + 1 α ) ∥ q ∥ 2 .

Lemma F.1.3. Suppose a sequence of random vectors { p k } K k =1 satisfy E [ p k ] = 0 , ∀ k. Then,

Lemma F.1.4. [46, Lemma 2] Suppose a sequence of random vectors { p k } K k =1 satisfy E [ p k | p k -1 , p k -2 ,. . . , p 1 ] = 0 , ∀ k. Then,

Lemma F.1.5. [21, Lemma 17] For any a 0 ≥ 0 , b ≥ 0 , c ≥ 0 , d > 0 , there exist a constant η ≤ 1 d such that

F.2 Proofs of Theorem 4.1 and Corollary 4.1

For the convenience of presentation, we introduce the following notations

where Z t,e j and Y t,e j characterize the biases between client-group correction term z t,e i and ∇ F i ( ¯ x t,e j ) -∇ f j ( ¯ x t,e j ) and between group-global correction term y t j and ∇ f j (ˆ x t,e ) -∇ f (ˆ x t,e ) , respectively, D t and Q t denote the group model drift and client model drift, respectively, and Θ t,e j represents model progress for group j .

To prove the convergence of MTGC, we start with characterizing the evolution of the global loss, i.e., f ( x ) through the following lemma.

Lemma F.2.1. Suppose that Assumptions 1 and 2 hold and γ ≤ 1 2 HL , then the iterates generated by Algorithm 1 satisfy

Lemma (F.2.1) implies that we need further to study the evolution of Q t and D t . In particular, we establish upper bounds for Q t and D t in Lemmas F.2.2 and F.2.3, respectively.

Lemma F.2.2. Suppose that Assumptions 1 and 2 hold and γ ≤ 1 8 HL , then the client model drift Q t , defined in (11) , can be bounded as

Lemma F.2.3. Suppose that Assumptions 1 and 2 hold and γ ≤ 1 10 EHL . Then the group model drift D t , defined in (11) , can be bounded as

The results shown in Lemmas F.2.2 and F.2.3 suggest that Z t,e j and Y t,e j are crucial for understanding the dynamics of MTGC. Hence, we derive upper bounds for Z t,e j and Y t,e j , which are presented in Lemmas F.2.4 and F.2.5, respectively.

Lemma F.2.4. Suppose that Assumptions 1 and 2 hold. Then the bias between client-group correction term z t,e i and ∇ F i ( ¯ x t,e j ) -∇ f j ( ¯ x t,e j ) , i.e., Z t,e j can be bounded as

Lemma F.2.5. Suppose that Assumptions 1 and 2 hold. Then the bias between group-global correction term y t j and ∇ f j (ˆ x t,e ) -∇ f (ˆ x t,e ) , i.e., Y t,e j defined in (11) , t ≥ 1 , can be bounded as

Additionally, when t = 0 , Y 0 ,e j can be bounded as

In addition, the upper bound of Θ t,e j is presented in the following lemma.

Lemma F.2.6. Suppose that Assumptions 1 and 2 hold. Then the group model progress at the ( t, e ) -th round, i.e., Θ t,e j , defined in (11) , can be bounded as

Recalling Lemma (F.2.1), we can see that what we actually need is the evolution of Q t + D t . With Lemmas F.2.2-F.2.6, we can characterize this evolution which is formalized in Lemma F.2.7.

Lemma F.2.7. Suppose that Assumptions 1 and 2 hold and γ ≤ 1 33 EHL , the model deviation Γ t = Q t + D t satisfies

The proofs of Lemmas F.2.1-F.2.7 are provided in Appendix F.3. With these lemmas, we are ready to prove Theorem 4.1.

F.2.1 Proof of Theorem 4.1

where Γ t = Q t + D t . Adding 2 γL 2 H Γ t on both sides of the above inequality and utilizing Lemma F.2.7, we have

For notation ease, we denote Φ t +1 = E f (¯ x t +1 ) -f ∗ + γL 2 H Γ t , Φ t ≥ 0 , ∀ t ≥ 0 . As long as γ ≤ E 8 HL in Theorem 4.1, 2304 γ 5 H 5 L 4 ≤ 36 γ 3 E 2 H 3 L 2 , we thus have

When γ ≤ 1 40 EHL , we have γH 2 -360 γ 3 E 2 H 3 L 2 ≥ γH 4 . Telescoping the above inequality from t = 1 to T -1 , we have

According to Lemma F.2.1, when t = 0 ,

it follows that

Combining (23) with (21) and plugging the upper bound of Γ 0 , established in Lemma F.2.7, into the inequality, we have

The setup of γ presented in Theorem 4.1 is enough to guarantee γH 2 -180 γ 3 E 2 H 3 L 2 -2 γL 2 H × ( 648 γ 4 H 4 L 2 +42 γ 2 E 2 H 2 ) ≥ γH 4 . Therefore, combining the last terms and taking some basic algebra operation, we have

The above bound can be further simplified to

This completes the proof of Theorem 4.1.

F.2.2 Proof of Corollary 4.1

Rewriting the bound in 24 as

and recalling Lemma F.1.5, one can claim that there exists a learning rate ( γEH ) ≤ 1 d such that

Given that we need γ ≤ 1 40 EHL for Theorem 4.1, we can set d = 40 L . We thus can find a step size in the range of ( γEH ) ≤ 1 40 L , i.e., γ ≤ 1 40 EHL such that

This completes the proof of Corollary 4.1.

F.3 Proofs of Lemmas F.2.1-F.2.7

F.3.1 Proof of Lemma F.2.1

Under the framework of MTGC, the virtual global model obeys the following iteration:

As ∑ i ∈C j z t,e i = 0 and ∑ N j =1 y t j = 0 , the the virtual global model iteration reduces to

With Assumption 1, we have

Plugging (28) into (29), we have

Utilizing E [ 1 N ∑ N j =1 1 n j ∑ i ∈C j ∑ H -1 h =0 ∇ F i ( x t,e i,h , ξ t,e i,h ) -1 N ∑ N j =1 1 n j ∑ i ∈C j ∑ H -1 h =0 ∇ F i ( x t,e i,h ) ] = 0 , we rewrite T 1 as follows

where the last inequality comes from Assumption 1.

On the other hand, we can bound T 2 as follows

where second inequality comes from Lemma F.1.3 and the last inequality follows Assumption 2 and Lemma F.1.4.

Plugging the derived upper bounds of T 1 and T 2 into 30 and utilized γ ≤ 1 2 HL , we obtain

Telescoping the above inequality from e = 0 to H -1 gives rise to Lemma F.2.1.

F.3.2 Proofs of Lemmas F.2.2 and F.2.6

Part I (Lemma F.2.2): Let q t,e j,h = 1 n j ∑ i ∈C j E ∥ ∥ ∥ ¯ x t,e j -x t,e i,h ∥ ∥ ∥ 2 , q t,e j, 0 = 0 . For 0 ≤ h ≤ H -2 , we have

where the first inequality comes from Lemma F.1.2 and Assumption 2 and the second inequality follows Lemma F.1.1. Let ρ 1 = ( 1 + 1 H -1 +4 γ 2 HL 2 ) . As γ ≤ 1 8 HL , we have ρ h 1 ≤ ρ H -1 1 ≤ ( 1 + 1 H 1 + 1 16( H 1) ) H -1 ≤ e 17 16 0 < 3 and ∑ H -1 h =0 ρ h 1 ≤ 3 H , where e 0 denotes Euler's number. We

thus have

where the last inequality follows Assumption 1.

Plugging the derived upper bound of n j q t,e j,h +1 into Q t = ∑ E -1 e =0 1 N ∑ N j =1 ( 1 H ∑ H -1 h =0 q t,e j,h ) gives

where the second inequality holds due to Lemmas F.1.1 and F.1.3, the second third inequality comes from Lemmas F.1.4 and Assumption 2, and the last inequality follows Assumption 1. Plugging this upper bound into ∑ E -1 e =0 1 N ∑ N j =1 Θ t,e j gives rise to Lemma F.2.6.

F.3.3 Proof of Lemma F.2.3

where we utilize 1 N ∑ N j =1 y t j = 0 and 1 n j ∑ i ∈C j z t,e i = 0 . Next, we bound E ∥ ∥ ¯ x t,e +1 j -ˆ x t,e +1 ∥ ∥ 2 as follows

where the first inequality comes from Lemma F.1.2. We thus have

where the first equality comes from Lemmas F.1.1 and the second inequality follows Lemmas F.1.3 and F.1.4. Additionally, we bound T 3 as

where the second inequality follows Lemma F.1.1 and the last inequality follows Assumption 1. Plugging the derived upper bound of T 3 into (31) gives rise to

. Let ρ 2 = 1 + 1 E -1 + 8 γ 2 EH 2 L 2 . As γ ≤ 1 10 EHL , we have ρ 2 ≤ ρ E -1 2 ≤ ( 1 + 1 E -1 + 1 12( E -1) ) E -1 ≤ e 13 12 0 < 3 , where e 0 denotes Euler's number. Therefore, we have N

Hence, for D t = ∑ E -1 e =0 1 N ∑ N j =1 E e t ∥ ∥ ¯ x t,e j -ˆ x t,e ∥ ∥ , we have

This completes the proof of Lemma F.2.3.

F.3.4 Proof of Lemma F.2.4

where the first inequality follows Lemma F.1.1 and the second inequality holds due to Assumption 2. In addition, when e ≥ 0 , z t,e i obeys the following iteration

. As ∑ i ∈C j z t,e i = 0 , we thus have z t,e +1 i = 1 H H -1 ∑ h =0   1 n j ∑ i ∈C j ∇ F i ( x t,e i,h , ξ t,e i,h ) -∇ F i ( x t,e i,h , ξ t,e i,h )   . To establish an upper bound for ∑ E -1 e =0 1 N ∑ N j =1 Z t,e +1 j , we start with bounding Z t,e j as follows 2

where the first inequality comes from Lemma F.1.1, the third inequality follows Lemma F.1.4 and Assumption 2, and the fourth inequality follows Assumption 1. Combining this bound with (34), we thus obtain Lemma F.2.4, i.e.,

F.3.5 Proof of Lemma F.2.5

Part I ( t ≥ 1 ): As ∑ i ∈C j z t,e i = 0 , the updating rule of y t +1 j can be simplified as

We next bound Y t +1 ,e j as follows

where the first inequality holds due to Lemma F.1.1, the second inequality follows Lemmas F.1.1 and F.1.3, the third inequality follows Lemmas F.1.1 and F.1.4 and Assumption 2, the fourth inequality follows Lemma F.1.1, and the final one comes from Assumption 1.

Given the above inequality, it follows that

Additionally,

which implies that we need further to bound E ∥ ∥ ˆ x t,τ -¯ x t +1 ∥ ∥ 2 and E ∥ ∥ ¯ x t +1 -ˆ x t +1 ,e ∥ ∥ 2 .

Under the framework of MTGC, the virtual global model obeys the following iteration:

As ∑ i ∈C j z t +1 ,τ i = 0 and ∑ N j =1 y t +1 j = 0 , the the virtual global model iteration reduces to

For E ∥ ∥ ¯ x t +1 -ˆ x t +1 ,e ∥ ∥ 2 , we have

where the second equality holds due to Lemma F.1.3, the second inequality follows Lemma F.1.4 and Assumption 2, the third inequality comes from Lemma F.1.1, the last inequality follows Lemma F.1.1 and Assumption 1.

Similarly, we can bound E ∥ ∥ ˆ x t,τ -¯ x t +1 ∥ ∥ 2 as

Given (41) and (42), it follows that

Combining (39) and (40), we can obtain

Plugging (43) into (44), we have

Part II ( t = 0 ): On the other hand, when t = 0 , we have

Similar to (41), we have

Combining the above two inequalities gives rise to

To this end, we complete the proof of Lemma F.2.5.

F.3.6 Proof of Lemma F.2.7

Part I ( t ≥ 1 ): Replacing ∑ E -1 e =0 1 N ∑ N j =1 Θ t,e j in (15) with an upper bound established in Lemma F.2.6 (i.e., (18)), and then plugging it into (13), we have

Summing (45) with the inequality established in Lemma F.2.3 and utilizing the upper bound of ∑ E -1 e =0 1 N ∑ N j =1 Y t,e j established in Lemma F.2.5, we have

Reorganizing the above inequality gives rise to

When γ ≤ 1 7 EHL , we have 48 γ 2 L 2 E 2 H 2 ≤ 1 and

As γ ≤ 1 33 EHL , we have 1 -( 60 γ 2 H 2 L 2 +768 γ 4 H 4 L 4 +36 γ 2 E 2 H 2 L 2 ) ≥ 2 3 and 3840 γ 4 H 4 L 4 + 120 γ 2 H 2 L 2 +120 γ 2 E 2 H 2 L 2 ≤ 1 3 . Hence, (46) can be simplified as

where

Further, we establish an upper bound for ϕ ( γ, σ 2 ) as follows,

where the last inequality holds due to 1 ¯ n = 1 N ∑ N j =1 1 n j ≤ 1 and γ ≤ √ ¯ nE 12 HL ≤ 1 33 EHL .

Part II ( t = 0 ): Summing (45) with the inequality established in Lemma F.2.3 and utilizing the upper bound of ∑ E -1 e =0 1 N ∑ N j =1 Y t,e j established in Lemma F.2.5, we have

Reorganizing the above inequality gives rise to

Following the same derivation as in Part I, we obtain where

√

To this end, we complete the proof of Lemma F.2.7.

G Recovering SCAFFOLD's Results

By comparing the communication complexity T required to achieve a ϵ -stationary solution, we can see that our result recovers that of SCAFFOLD when N = 1 and E = 1 . Specifically, for our MTGC algorithm, to achieve an ϵ error bound, according to Corollary 4.1, we can find a T to satisfy

Equivalently, T ≥ Lσ 2 F 0 ˜ NEHϵ 2 , T ≥ Lσ F 0 ( ϵ ) 3 2 , and T ≥ L F 0 ϵ . In other words, the MTGC algorithm will have an expected error smaller than ϵ if T satisfies

According to Theorem II of [18], to achieve the ϵ error bound, the number of global communication rounds SCAFFOLD needs to take can be expressed as

where we have converted the notation from [18] to our notation.

We see that the dominating term of the MTGC, O ( Lσ 2 F 0 ˜ NEHϵ 2 ) , recovers that of SCAFFOLD when N = 1 (i.e., ˜ N = n j ) and E = 1 , which corresponds to the case of a single group with a single (global) aggregator.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction clearly state the claims made, including the contributions made in the paper. Additionally, the claims made in the abstract and introduction are supported by the theoretical analysis (see Sec. 4) and experiments (see Sec. 5).

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitation are included in Sec. 6.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All the theorems, formulas, and proofs in the paper are numbered and crossreferenced. All assumptions are clearly stated in Sec. 4, and referenced in the statement of our theorem, i.e., Theorem 4.1. The proofs are provided in the supplemental material. Theorems and Lemmas that our proof relies upon are properly referenced.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The paper discloses all necessary information to reproduce the main experimental results in Sec. 5.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: All the datasets used in this work are open-sourced while the information about the models and parameters are clearly reported in Sec. 5. For the convenience of reproducibility, the code is attached to our submission.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The details about the setting of our experiments are reported in Sec. 5.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We report the standard deviations which are computed based on 3 random trials (see Sec. 5).

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide the information of the computer resources used for this work including the GPU type and memory information.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our work adheres to the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: There is no societal impact of the work performed.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We credit the framework that our work is based on in Sec. 5.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: Our paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper proposes a multi-timescale gradient correction (MTGC) methodology to deal with multi-timescale model drift. It introduces a distinct control variables to correct the client gradient towards the group gradient, and correct the group gradient towards the global gradient. Then, the stability of the proposed algorithm against multi-level non-i.i.d. data is shown empirically.

STRENGTHS:
1. The idea of correcting gradients is interesting 
2. The proof seems correct.

WEAKNESSES:
The model is tested on small datasets.

QUESTIONS:
1. What are the practical applications of HFL?
2. Are there any techniques implemented for preserving the privacy of the model?
3. The model has been tested on small datasets; how does it perform on larger datasets?
4. The ablation study is missing. How does the model perform when only one of the corrections is used?
5. How about the stability of the proposed method?

LIMITATIONS:
Yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-06-27

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel algorithm called Multi-Timescale Gradient Correction (MTGC) for Hierarchical Federated Learning (HFL). The authors address the challenge of multi-timescale model drift in HFL systems, where data heterogeneity exists both within client groups and across groups. MTGC introduces coupled gradient correction terms to mitigate client model drift and group model drift. The authors provide theoretical convergence analysis for non-convex settings and demonstrate MTGC's effectiveness through experiments on various datasets and models.\n\nSTRENGTHS:\n- The proposed MTGC algorithm is simple and easy to implement, introducing client-group and group-global correction terms. The paper clearly defines the problem it aims to solve. The motivation is clear and well-articulated, highlighting the gap in existing HFL algorithms.\n- The inclusion of theoretical analysis adds depth to the paper, providing a solid foundation for the proposed approach. The theoretical results show that MTGC achieves linear speedup in the number of local iterations, group aggregations, and clients. \n- The convergence bound of the proposed algorithm is immune to the extent of data heterogeneity, which is a significant strength.\n\nWEAKNESSES:\n- There is a lack of comparisons and discussions on clustered federated learning [1,2], which share similar context to some extent and could provide valuable background for the readers.\n\n\n- The experiments primarily focus on image classification tasks. Including other types of tasks (e.g., natural language processing) could strengthen the generalizability of the results. There is a lack of experiments on various types of distribution shifts such as domain shift, label shift etc. Exploring different non-i.i.d. scenarios could provide more insights into the algorithm's robustness.\n\n- While the paper analyzes model drift theoretically, there is a lack of empirical and theoretical analysis on how model drift affects generalization performance, especially in relation to the number of hierarchical communication levels. The study also does not clearly demonstrate in which scenarios and to what extent increasing the number of hierarchical communication levels benefits federated learning performance.\n\n- The paper could also be strengthened by providing more insights into the practical implications of the theoretical results.\n\n[1] An Efficient Framework for Clustered Federated Learning. NeurIPS 2020.\n\n[2] Optimizing the Collaboration Structure in Cross-Silo Federated Learning. ICML 2023\n\nQUESTIONS:\nPlease refer to the weaknesses mentioned above.\n\nLIMITATIONS:\nThe limitations of the study are discussed, and no potential negative societal impacts of the work have been identified.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-15"}

comments: SUMMARY:
This paper presents a method to address multi-timescale model drift in hierarchical federated learning. Specifically, it introduces two control variables to correct intra-group client drift and group model drift. The paper establishes the convergence bound in a non-convex setup and demonstrates its stability against multi-level data heterogeneity. Overall, the paper is well-written and easy to follow.

STRENGTHS:
1. The writing is clear and easy to follow.
2. I have initially checked the proofs and did not find any issues so far.

WEAKNESSES:
1. It would be beneficial to include comparisons of additional computational and communication costs of MTGC in the experimental section.

QUESTIONS:
1. In Algorithm 1, consider using different color blocks to mark $x$, $z$, and $y$. This could help in better understanding the algorithm's process.

2. How about the runtime of the algorithm? Would it be possible to include comparisons of runtime with the baseline in the experiments?

3. In the experiments, does Eq.5 require the addition of regularization coefficients for correction terms $z$ and $y$?

4. Could you include some measurements of the additional communication costs in the experiments?

LIMITATIONS:
N.A.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-16

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the usage of the gradient correction scheme to hierarchical federated learning. Specifically, the authors propose and analyze the multi-timescale gradient correction MTGC algorithm which is a direct generalization of SCAFFOLD to the framework where local clients aggregate their models on group server and group servers aggregate their models to a global server. The authors introduce control variables that correct i) client model drift and ii) group model drift that arises due to data heterogeneity. Theoretical convergence results are presented in the non-convex regime and experimental results showcase the ability of MTGC to address data heterogeneity both from clients and from group-servers.\n\nSTRENGTHS:\n- The paper places itself correctly in the existing literature, is well-structured and easy to follow.\n- The problem of hierarchical FL is relevant and fairly interesting for the ML community.\n- The proposed algorithm is natural and easy to understand.\n- Both theoretical and numerical results are provided.\n\nWEAKNESSES:\n- The main weakness of this work is its lack of novelty and technical contribution. The proposed MTGC is a straightforward extension of the well-known SCAFFOLD algorithm to a two-level hierarchical FL. Although the authors mention that the coupling of the two error correction variables is introducing new challenges I fail to see how this is the case. Indeed, the analysis of the correction variables appears to be largely the same as in SCAFFOLD and the authors to not make clear in the main body of their work what specific new challenges they faced and how they were circumvented. To summarize, although the paper is well written I do not see enough contribution to justify its acceptance to a top tier conference.\n\n- Minor typo: on line 110 in \"However, their is\" should be \"However, their algorithm is\".\n\nQUESTIONS:\nSee weaknesses section.\n\nLIMITATIONS:\nThe authors adequately addressed the limitations  of their work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-29"}