Title: SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learning
Venue: NeurIPS 2023
Accepted: yes
Keywords: Federated bilevel optimization

federated hypergradient

communication efficiency

system-level heterogeneity

linear speedup
Abstract: Federated bilevel optimization (FBO) has shown great potential recently in machine learning and edge computing due to the emerging nested optimization structure in meta-learning, fine-tuning, hyperparameter tuning, etc. However, existing FBO algorithms often involve complicated computations and require multiple sub-loops per iteration, each of which contains a number of communication rounds. In this paper, we propose a simple and flexible FBO framework named SimFBO, which is easy to implement without sub-loops, and includes a generalized server-side aggregation and update for improving communication efficiency. We further propose System-level heterogeneity robust FBO (ShroFBO) as a variant of SimFBO with stronger resilience to heterogeneous local computation. We show that SimFBO and ShroFBO provably achieve a linear convergence speedup with partial client participation and client sampling without replacement, as well as improved sample and communication complexities. Experiments demonstrate the effectiveness of the proposed methods over existing FBO algorithms.

SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learning

Abstract

Federated bilevel optimization (FBO) has shown great potential recently in machine learning and edge computing due to the emerging nested optimization structure in meta-learning, fine-tuning, hyperparameter tuning, etc. However, existing FBO algorithms often involve complicated computations and require multiple sub-loops per iteration, each of which contains a number of communication rounds. In this paper, we propose a simple and flexible FBO framework named SimFBO, which is easy to implement without sub-loops, and includes a generalized serverside aggregation and update for improving communication efficiency. We further propose System-level heterogeneity robust FBO (ShroFBO) as a variant of SimFBO with stronger resilience to heterogeneous local computation. We show that SimFBO and ShroFBO provably achieve a linear convergence speedup with partial client participation and client sampling without replacement, as well as improved sample and communication complexities. Experiments demonstrate the effectiveness of the proposed methods over existing FBO algorithms.

1 Introduction

Recent years have witnessed significant progress in a variety of emerging areas including metalearning and fine-tuning [11, 52], automated hyperparameter optimization [13, 10], reinforcement learning [31, 21], fair batch selection in machine learning [54], adversarial learning [76, 40], AIaware communication networks [27], fairness-aware federated learning [75], etc. These problems share a common nested optimization structure, and have inspired intensive study on the theory and algorithmic development of bilevel optimization. Prior efforts have been taken mainly on the single-machine scenario. However, in modern machine learning applications, data privacy has emerged as a critical concern in centralized training, and the data often exhibit an inherently distributed nature [70]. This highlights the importance of recent research and attention on federated bilevel optimization, and has inspired many emerging applications including but not limited to federated meta-learning [9], hyperparameter tuning for federated learning [25], resource allocation over communication networks [27] and graph-aided federated learning [71], adversarial robustness on edge computing [46], etc. In general, the federated bilevel optimization problem takes the following mathematical formulation.

AID-basedFHE

ITD-basedFHE

SimFBO

LocallyUpdateonxandAggregate

LocallyUpdateonxandAggregate

GeneralizedServer

Updateony，v,x

CalculateHypergradientEstimate

CalculateHypergradientEstimate

TCommunication

Rounds

Rounds

OneCommunication

Round

FEDIHGP:

ComputeHessian-vectorproducts

Ateachrounds

Updateony

Simultaneously

T;local

StartatQthrounds

Update ony，v,x

rounds

2N

Communication

FEDINN:

Updateony

ComputeHessian-vectorproducts

Rounds

where n is the total number of clients, the outer- and inner-functions f i ( x, y ) and g i ( x, y ) for each client i take the expectation forms w.r.t. the random variables ξ i and ζ i , and are jointly continuously differentiable. However, efficiently solving the federated problem in eq. (1) suffers from several main challenges posed by the federated hypergradient (i.e., ∇ Φ( x ) ) computation that contains the second-order global Hessian-inverse matrix, the lower- and upper-level data and system-level heterogeneity, and the nested optimization structure. To address these issues, [25, 65, 24, 22] proposed approximate implicit differentiation (AID)-based federated bilevel algorithms, which applied the idea of non-federated AID-based estimate in [15] to the federated setting, and involve two sub-loops for estimating the global lower-level solution y ∗ ( x ) and the Hessian-inverse-vector product, respectively. [69] then proposed AggITD by leveraging the idea of iterative differentiation, which improved the communication efficiency of AID-based approaches by synthesizing the lower-level optimization and the hypergradient computation into the same communication sub-loop. However, some limitations still remain in these approaches.

First, the sub-loops, each with a large number of communication rounds, often compute products of a series of matrix-vector products, and hence can complicate the implementation and increase the communication cost.

Second, the practical client sampling without replacement has not been studied in these methods due to challenges posed by the nested structure of AID- and ITD-based federated hypergradient estimators.

Third, as observed in the single-level federated learning [67], in the presence of heterogeneous system capabilities such as diverse computing power and storage, clients can take a variable number of local updates or use different local optimizers, which may make these FBO algorithms converge to the stationary point of a different objective.

1.1 Our Contributions

In this paper, we propose a communication-efficient federated bilevel method named SimFBO, which is simple to implement without sub-loops, flexible with a generalized server-side update, and resilient to system-level heterogeneity. Our specific contributions are summarized below.

A simple and flexible implementation. As illustrated in Figure 1, differently from AID- and ITD-based approaches that contain multiple sub-loops of communication rounds at each iteration, our proposed SimFBO is simpler with a single communication round per iteration, in which three variables y, x and v are updated simultaneously for optimizing the lower- and upper-level problems, and approximating the Hessian-inverse-vector product. SimFBO also includes a generalized serverside update on x, y, v , which accommodates the client sampling without replacement, and allows for a flexible aggregation to improve the communication efficiency.

Resilient server-side updates to system-level heterogeneity. In the presence of heterogeneous local computation, we show that the naive server-side aggregation can lead to the convergence to a stationary point of a different objective. To this end, we propose System-level heterogeneity robust FBO (ShroFBO) building on a normalized version of the generalized server-side update with correction, which provably converges to a stationary point of the original objective.

Convergence analysis and improved complexity. As shown in Table 1, our SimFBO and ShroFBO both achieve a sample complexity (i.e., the number of samples needed to reach an ϵ -accurate stationary point) of O ( ϵ -2 P -1 ) , which matches the best result obtained by FedMBO [24] but under a more practical client sampling without replacement. Moreover, SimFBO and ShroFBO both achieve the best communication complexity (i.e., the number of communication rounds to reach an ϵ -accurate stationary point) of O ( ϵ -1 ) , which improves those of other methods by an order of ϵ -1 / 2 . Technically, we develop novel analysis in characterizing the client drifts by the three variables, bounding the per-iteration progress in the global y and v updates, and proving the smoothness and bounded variance in local v updates via induction, which may be of independent interest.

Superior performance in practice. In the experiments, the proposed SimFBO method significantly improves over existing strong federated bilevel baselines such as AggITD, FedNest and LFedNest in both the i.i.d. and non-i.i.d. settings. We also validate the better performance of ShroFBO in the presence of heterogeneous local computation due to the resilient server-side updates.

2 SimFBO: A Simple and Flexible Framework

2.1 Preliminary: Federated Hypergradient Computation

The biggest challenge in FBO is to compute the federated hypergradient ∇ Φ( x ) due to the implicit and complex dependence of y ∗ ( x ) on x . Under suitable assumptions and using the implicit function theorem in [18], it has been shown that the ∇ Φ( x ) takes the form of

which poses several computational challenges in the federated setting. First, the second term at the right-hand side contains three global components in a nonlinear manner, and hence the direct aggregation of local hypergradients given by

is a biased estimation of ∇ Φ( x ) due to the client drift. Second, it is infeasible to compute, store and communicate the second-order Hessian-inverse and Jacobian matrices due to the limited computing and communication resource. Although various AID- and ITD-based approaches have been proposed to address these challenges, they still suffer from several limitations (as we point out in the introduction) such as complicated implementation, high communication cost, lack of client sampling without replacement, and vulnerability to the system-level heterogeneity. To this end, we propose a simple, flexible and communication-efficient FBO framework named SimFBO in this section.

2.2 Federated Hypergradient Surrogate

To estimate the federated hypergradient efficiently, we use the surrogate ¯ ∇ F ( x, y, v ) = ∇ x F ( x, y ) -∇ 2 xy G ( x, y ) v , where v ∈ R d y is an auxiliary vector. Then, it suffices to find y and v as efficient

estimates of the solutions to the global lower-level problem and the global linear system (LS) ∇ 2 yy G ( x, y ) v = ∇ y F ( x, y ) that is equivalent to solving following quadratic programming.

where R i ( x, y, v ) can be regarded as the loss function of client i for solving this global LS problem. Based on this surrogate, we next describe the proposed SimFBO framework.

2.3 Simple Local and Server-side Aggregations and Updates

Simple local update. Differently from FedNest [65] and AggITD [69] that perform the lower-level optimization, the federated hypergradient estimation and the upper-level update alternatively in different communication sub-loops, our SimFBO conducts the simple updates on all these three procedures simultaneously in each communication round. In specific, each communication round t first selects a subset C ( t ) of participating clients without replacement. Then, each active client i ∈ C ( t ) updates three variables y, v, x at k th local iteration simultaneously as

where η y , η v , η x correspond to the local stepsizes, a ( t,k ) i is a client-specific coefficient to increase the flexibility of the framework, ζ ( t,k ) i , ψ ( t,k ) i , ξ ( t,k ) i are independent samples, and the local hypergradient estimate takes the form of ¯ ∇ f i ( x, y, v ; ξ ) = ∇ x f i ( x, y ; ξ ) -∇ 2 xy g i ( x, y ; ξ ) v i . The variables y, v and x in eq. (3), which optimize the lower-level problem, the LS problem and the upper-level problem, are updated with totally τ ( t ) i local steps. Note that the updates in eq. (3) also allow for parallel computation on x, v and y locally.

Local and server-side aggregation. After completing all local updates, the next step is to aggregate such local information on both the client and server sides. As shown in eq. (4), each participating client i ∈ C ( t ) aggregates all the local gradients, and then communicate the aggregations q ( t ) y,i , q ( t ) v,i and q ( t ) x,i to the server. Then, on the server side, such local information is further aggregated to be q ( t ) y , q ( t ) v and q ( t ) x , which will be used for a subsequent generalized server-side update.

where ˜ p i := n | C ( t ) | p i is the effective weight of client i ∈ C ( t ) among all participating clients such that E ( ∑ i ∈ C ( t ) ˜ p i ) = 1 . Note that in eq. (4), the local aggregation q ( t ) y,i (similarly for v and x ) can be regarded as a linear combination of all local stochastic gradients, and hence covers a variety of local optimizers such as stochastic gradient descent, momentum-based gradient, variance reduction by choosing different coefficients a ( t,k ) i for i ∈ C ( t ) . This substantially enhances the flexibility of the proposed framework.

Algorithm 1 SimFBO and ShroFBO

Server-side updates. Based on the aggregated gradients q ( t ) y , q ( t ) v and q ( t ) x , we then perform serverlevel gradient-based updates on variables x, v and y simultaneously as

where γ y , γ v and γ x are server-side updating stepsizes for y , v , x and P r ( v ) := min { 1 , r ∥ v ∥ } v is a simple projection on a bounded ball with a radius of r . There are a few remarks about the updates in eq. (5). First, in contrast to existing FBO algorithms such as [65, 69], our introduced server-side updates leverage not only the client-side stepsizes η y , η v , η x , but also the server-side stepsizes γ y , γ v and γ x . This generalized two-learning-rate paradigm can provide more algorithmic and theoretical flexibility, and provides improved communication efficiency in practice and in theory. Second, the projection P r ( · ) serves as an important step to ensure the boundedness of variable v ( t ) , and hence guarantee the smoothness of the global LS problem and the boundedness of the estimation variance in v and x updates, both of which are crucial and necessary in the final convergence analysis. Note that we do not impose such projection on the local v ( t,k ) i variables because we can prove via induction that they are bounded given the boundedness of v ( t ) (see Proposition 1).

2.4 Resilient Server-side Updates against System-level Heterogeneity

Limitations under system-level heterogeneity. When clients have heterogeneous computing and storing capabilities (e.g., computer server v.s. phone in edge computing), an unequal number of local updates are often performed such that the global solution can be biased toward those of the clients with much more local steps or stronger optimizers. As observed in [67], this heterogeneity can deviate the iterates to minimize a different objective function. To explain this mismatch phenomenon, inspired by [57], we rewrite the server-side update on x (similarly for v and y ) in eq. (4) as

where a ( t ) i = [ a ( t, 0) i , ..., a ( t,τ ( t ) i -1) i ] T collects all local coefficients of client i , and h ( t ) x,i normalizes the aggregated gradient q ( t ) x,i by 1 / ∥ a ( t ) i ∥ 1 such that ∥ h ( t ) x,i ∥ does not grow with the increasing of τ ( t ) i .

Although such normalization can help to mitigate the system-level heterogeneity, the effective weight w i can deviate from the true weight p i of the original objective in eq. (1), and the iterates converge to the stationary point of a different objective that replaces all p i by w i in eq. (1) (see Theorem 1).

System-level heterogeneity robust FBO (ShroFBO). To address this convergence issue, we then propose a new method named ShroFBO with stronger resilience to such heterogeneity. Motivated by the normalized reformulation in eq. (6), ShroFBO adopts a different server-side aggregation as

where ˜ p i := n | C ( t ) | p i and h ( t ) y,i , h ( t ) v,i , h ( t ) x,i are the normalized local aggregations defined in eq. (6). Accordingly, the server-side updates become

Differently from eq. (6), we select the client weights to be ˜ p i to enforce the correct convergence to the stationary point of the original objective in eq. (1), as shown in Theorem 2 later.

3 Main Result

3.1 Assumptions and Definitions

We make the following standard definitions and assumptions for the outer- and inner-level objective functions, as also adopted in stochastic bilevel optimization [26, 21, 30] and in federated bilevel optimization [65, 69, 24].

Definition 1. A mapping F is L -Lipschitz continuous if for ∀ z, z ′ , ∥ F ( z ) -F ( z ′ ) ∥ ≤ L ∥ z -z ′ ∥ .

Since the overall objective Φ( x ) is nonconvex, the goal is expected to find an ϵ -accurate stationary point defined as follows.

Definition 2. We say z is an ϵ -accurate stationary point of the objective function Φ( x ) if E ∥∇ Φ( z ) ∥ 2 ≤ ϵ , where z is the output of an algorithm.

Assumption 1. For any x ∈ R d x , y ∈ R d y and i ∈ { 1 , 2 , ..., n } , f i ( x, y ) and g i ( x, y ) are twice continuously differentiable, and g i ( x, y ) is µ g -strongly convex w.r.t. y .

The following assumption imposes the Lipschitz continuity conditions on the upper- and lower-level objective functions and their derivatives.

Assumption 2. Function f i ( x, y ) is L f -Lipschitz continuous; the gradients ∇ f i ( x, y ) and ∇ g i ( x, y ) are L 1 -Lipschitz continuous; the second-order derivatives ∇ 2 f i ( x, y ) and ∇ 2 g i ( x, y ) are L 2 -Lipschitz continuous; and the third-order derivatives ∇ 3 g i ( x, y ) is L 3 -Lipschitz continuous for some constants L f , L 1 , L 2 , L 3 > 0 .

The Lipschitz continuity of the third-order derivative is necessary here to ensure the smoothness of v ∗ ( x ) , which guarantees the descent in the iterations of LS function (see Lemma 10), under our more challenging simultaneous and single-loop updating structure. Next, we assume the bounded variance conditions on the gradients and second-order derivatives.

Assumption 3. There exist constants σ 2 f , σ 2 g , σ 2 gg such that E [ ∥∇ f i ( x, y ) -∇ f i ( x, y ; ξ ) ∥ 2 ] ≤ σ 2 f , E [ ∥∇ g i ( x, y ) -∇ g i ( x, y ; ζ ) ∥ 2 ] ≤ σ 2 g and E [ ∥∇ 2 g i ( x, y ) -∇ 2 g i ( x, y ; ζ ) ∥ 2 ] ≤ σ 2 gg .

Assumption 4. For any x ∈ R d x , y ∈ R d y , there exist constants β gh ≥ 1 and σ gh ≥ 0 such that

We have β gh = 1 , and σ gh = 0 when all g i 's are identical.

This assumption of global heterogeneity uses β gh and σ gh to measure the dissimilarity of ∇ y g i ( x, y ) for all i .

3.2 Convergence and Complexity Analysis

It can be seen from eq. (2) that the boundedness of v is necessary to guarantee the smoothness (w.r.t. x, y ) and bounded variance in solving the local and global LS problems. Projecting the global v ( t ) vector and the local v ( t,k ) i , k ≥ 1 vectors onto a bounded set can be a feasible solution, but in this case, the local aggregation q ( t ) v,i is no longer a linear combination of local gradients. This can complicate the implementation and analysis, and degrade the flexibility of the framework. Fortunately, we show via induction that the projection of the server-side vector v ( t ) on a bounded set suffices to guarantee the boundedness of local vectors v t,k i .

Proposition 1 (Boundedness of Local v ) . Under Assumptions 1 and 2, for each iteration t , client i , and local iteration k = 1 , 2 , ..., τ ( t ) i , we have r i := ∥ v ( t,k ) i ∥ ≤ ( 1 + α max α min ) r, where the radius r = L f µ g and α min , α max are chosen such that α min ≤ a ( t,k ) i ≤ α max .

Next, we show an important proposition in characterizing the per-iteration progress of the global v ( t ) updates in approximating the solution of a reweighted global LS problem. Let ∆ ( t ) v = E ∥ v ( t ) -˜ v ∗ ( x ( t ) ) ∥ 2 denote the approximation error, where ˜ v ∗ be the minimizer of ∑ n i =1 w i R i ( x, ˜ y ∗ , · ) .

Proposition 2. Under the Assumption 1, 2 and 3, the iterates v ( t ) in solving the global LS problem generated by Algorithm 1 satisfy

Similarly, we can provide a per-iteration process of y ( t ) in approximating the solution ˜ y ∗ of the reweighted lower-level global function ∑ n i =1 w i g i ( x, · ) . Note that such characterizations do not exist in previous studies in single-level or minimax federated optimization with a single objective (e.g., [57]) because our analysis needs to handle three different lower-level, LS and upper-level objectives. As shown in Proposition 2, the bound involves the client drift term E ∥ v ( t ) -v ( t,k ) i ∥ 2 (similarly for y, x ), so the next step is to characterize this important quantity.

Proposition 3. Under Assumption 1 and 2, the local iterates client drift of v ( t,k ) i is bounded as

for all t ∈ { 0 , 1 , ..., T -1 } , k ∈ { 0 , 1 , ..., τ i -1 } and i ∈ { 1 , 2 , ..., n } . We define ¯ τ := ∑ n i =1 τ i /n and σ 2 M 1 := α 2 max ( σ 2 f + r 2 max σ 2 gg ) + α max ( L 2 f + r 2 max L 2 1 ) .

It can be seen from Proposition 3 that the bound on the client drift of the local updates on v is proportional to η v and ∥ a ( t ) i ∥ 1 . Since α min ≤ a ( t,k ) i ≤ α max , ∥ a ( t ) i ∥ 1 is proportional to the number τ ( t ) i of local steps. Thus, this client drift is controllable by choosing τ ( t ) i and the local stepsizes η v properly. Then, combining the results in the above Proposition 1, 2, 3, and under a proper Lyapunov function, we obtain the following theorem. Let P = | C ( t ) | be the number of sampled clients.

Theorem 1. Define ˜ Φ( x ) = ˜ F ( x, ˜ y ∗ ) as the objective function by replacing p i in eq. (1) with w i . Suppose Assumptions 1, 2 and 3 are satisfied. The iterates by SimFBO in Algorithm 1 satisfy

where γ x , γ y , γ v , η x , η y , η v are set in eq. (38) and M 1 , M 2 , M 3 are defined by eq. (40) in appendix. For the full client participation (i.e., P = n ), the sample complexity is ¯ τT = O ( n -1 ϵ -2 ) , and the number of communication rounds is T = O ( ϵ -1 ) . For partial client participation, the sample complexity is ¯ τT = O ( P -1 ϵ -2 ) , and the number of communication rounds is T = O ( P -1 ϵ -2 ) .

First, when set ¯ τ = O (1) , Theorem 1 shows that SimFBO converges to a stationary point of an objective function ˜ Φ( x ) with a rate of O ( 1 √ PT + 1 T ) , which, to the best of our knowledge, is the first linear speedup result under partial client participation without replacement. Note that without systemlevel heterogeneity, i.e., ∥ a ( t ) 1 ∥ = ... = ∥ a ( t ) n ∥ , w i = p i ∥ a ( t ) i ∥ 1 ∑ n j =1 p j ∥ a ( t ) j ∥ 1 = p i , and hence SimFBO converges to the stationary point of the original objective in eq. (1). However, in the presence of system-level heterogeneity, SimFBO may converge to the stationary point of a different objective. Second, when nearly full clients participate, the partial participation error is approximately zero. Then we can see that setting local update round ¯ τ to its upper-bound results in the best performance.

Theorem 2. Define Φ( x ) = F ( x, y ∗ ) as eq. (1) . Suppose Assumptions 1, 2 and 3 are satisfied. The iterates generated by ShroFBO in Algorithm 1 satisfy

by setting the same server-side and local stepsizes and M 1 , M 2 and M 3 as in Theorem 1. For full client participation, the sample complexity is ¯ τT = O ( n -1 ϵ -2 ) , and the number of communication rounds is T = O ( ϵ -1 ) . For partial client participation, the sample complexity is ¯ τT = O ( P -1 ϵ -2 ) , and the number of communication rounds is T = O ( P -1 ϵ -2 ) .

In Theorem 2, we show that even under the system-level heterogeneity, ShroFBO can converge to the original objective function with the same convergence rate as SimFBO. This justifies the design principle of robust server-side updates.

4 Related Work

Bilevel optimization. Bilevel optimization, first introduced by [3], has been studied for decades. A class of constraint-based bilevel methods was then proposed [20, 16, 59, 61], whose idea is to replace the lower-level problem by the optimality conditions. Gradient-based bilevel algorithms have attracted considerable attention due to the effectiveness in machine learning. Among them, AID-based approaches [8, 51, 38, 1] leveraged the implicit derivation of the hypergradient, which was then approximated via solving a linear system. ITD-based approaches [45, 12, 11, 56, 17] approximated the hypergradient based on automatic differentiation via the forward or backward mode. Agroup of stochastic bilevel approaches has been developed and analyzed recently based on Neumann series [5, 26, 1], recursive momentum [72, 23, 19] and variance reduction [72, 7], etc. For the lowerlevel problem with multiple solutions, several approaches were proposed based on the upper- and lower-level gradient aggregation [55, 43, 34], barrier types of regularization [41, 39], penalty-based formulations [58], primal-dual technique [62], and dynamic system-based methods [42].

Federated (bilevel) learning. Federated Learning was proposed to enable collaborative model training across multiple clients without compromising the confidentiality of individual data [32, 60, 49]. As one of the earliest methods of federated learning [47], FedAvg has inspired an increasing number of approaches to deal with different limitations such as slower convergence, high communication cost and undesired client drift by leveraging the techniques including proximal regularization [37], periodic variance reduction [48, 28], proximal splitting [50], adaptive gradients [53]. Theoretically, the convergence of FedAvg and its variants has been analyzed in various settings with the homogeneous [63, 68, 64, 2] or heterogeneous datasets [37, 66, 48, 29]. [67] analyzed the impact of the

system-level heterogeneity such as heterogeneous local computing on the convergence. [57] further extended the analysis and the methods to the minimax problem setting.

Federated bilevel optimization has not been explored well except for a few attempts recently. For example, [14, 35] proposed momentum-based bilevel algorithms, and analyzed their convergence in the setting with homogeneous datasets. In the setting with non-i.i.d. datasets, [65] and [24] proposed FedNest and FedMBO via AID-based federated hypergradient estimation, and [69] proposed an ITD-based aggregated approach named Agg-ITD. Momentum-based techniques have been also used by [22, 36] to improve the sample complexity. Moreover, there are some studies that focus on other distributed scenarios, including decentralized bilevel optimization [6, 73, 44], asynchronous bilevel optimization over directed network [74], and distributed bilevel network utility maximization [27].

MNIST-MLP on i.i.d. data

MNIST-MLP on non-i.i.d. data

CIFAR-10-CNN on i.i.d.data

100

100

95

S6

70

90

90

60

Trian accuracy

Trian accuracy

Trian accuracy

85

85

50

80

80

40

75

LFedNest

75

LFedNest

30

70

FedNest

70

FedNest

FBO-AggITD

FBO-AggITD

20

FBO-AggITD

65

SimFBO

65

SimFBO

SimFBO

10

60

0

200

400

600

800

1

60

0

200

400

600

800

1(

0

200

400

600

008

1000

1200

1400

1600

Communicationrounds

Communicationrounds

Communicationrounds

MNIST-MLP on i.i.d. data

MNIST-MLP on non-i.i.d. data

CIFAR-10-CNN on i.i.d. data

100

100

70

95

95

60

Testaccuracy

90

Testaccuracy

90

Test accuracy

50

85

85

40

80

80

75

LFedNest

75

LFedNest

70

FedNest

70

FedNest

20

65

FBO-AggITD

SimFBO

65

FBO-AggITD

SimFBO

10

FBO-AggITD

SimFBO

60

0

200

400

600

800

1000

60

0

200

400

600

800

1000

0

0

200

400

600

800

1000

1200

1400

1600

Communicationrounds

Communication rounds

Communicationrounds

5 Experiments

In this section, we perform two hyper-representation experiments to compare the performance of our proposed SimFBO algorithm with FBO-AggITD [69], FedNest [69], and LFedNest [65], and validate the better performance of ShroFBO in the presence of heterogeneous local computation. We test the performance on MNIST and CIFAR datasets with MLP and CNN backbones. We follow the same experimental setup and problem formulation as in [65, 69]. The details of all experimental specifications can be found in Appendix A.1.

Comparison to existing methods. The comparison results are presented in Figure 2 and Figure 3. It can be seen that across different datasets and backbones, our proposed SimFBO consistently converges much faster than other comparison methods, while achieving a much higher training and test accuracy. We do not plot the curves of FedNest and LFedNest on CIFAR and CNN, because they are hard to converge under various hyperparameter configurations using their source codes.

Performance under heterogeneous local computation. We now test the performance in the setting where a total of 10 clients perform a variable number of local steps. This is to simulate the scenario where clients have heterogeneous computing capabilities and hence can perform an uneven number of local updates. In this experiment, we choose the number τ i of the client i 's local steps from the set { 1 , ..., 10 } uniform at random. As shown in Figure 4, the proposed ShroFBO method performs the best due to the better resilience to such client heterogeneity. We also compare the convergence rate of our proposed SimFBO, FedNest and FBO-AggITD w.r.t. running time. The results are provided in Figure 5. All the settings for different algorithms are the same as in Appendix A.2. It can be seen that the proposed SimFBO still converges fastest with a higher test accuracy in terms of running time.

Communicationrounds

Toy example on non-i.i.d. data

Toy example on non-i.i.d. data

100

90

LFedNest

2.5

FedNest

80

FBO-AggITD

Testaccuracy

70

2.0

SimFBO

Test loss

ShroFBO

1.5

LFedNest

30

FBO-AggITD

FedNest

1.0

20

SimFBO

10

ShroFBO

0.5

20

40

60

80

100

20

40

60

80

100

Communicationrounds

101

MNIST-MLP time comparison on i.i.d. data

MNIST-MLP time comparison on non-i.i.d. data

100

100

95

95

90

06

Test accuracy

Testaccuracy

80-

75

5

FedNest

70

FedNest

FBO-AggITD

FBO-AggITD

65

65

SimFBO

SimFBO

60

60

0

720144021602880360043205040576064807200

0

720144021602880360043205040576064807200

Time (s)

Time (s)

6 Conclusion

In this paper, we propose a simple and communication-efficient federated bilevel algorithm named SimFBO and its variant ShroFBO with better resilience to the system-level heterogeneity. We show that both SimFBO and ShroFBO allow for more practical client sampling without replacement, and achieve better sample and communication complexities. Experiments demonstrate the great promise of the proposed methods. We anticipate that the proposed algorithms and the developed analysis can be applied to other distributed settings such as decentralized or asynchronous bilevel optimization, and the algorithms may be useful in applications such as hyperparameter tuning or fine-tuning in federated learning or AI-aware edge computing.

References

M. Arbel and J. Mairal. Amortized implicit differentiation for stochastic bilevel optimization. arXiv preprint arXiv:2111.14580 , 2021.

D. Basu, D. Data, C. Karakus, and S. Diggavi. Qsparse-local-sgd: Distributed sgd with quantization, sparsification and local computations. Advances in Neural Information Processing Systems , 32, 2019.

J. Bracken and J. T. McGill. Mathematical programs with optimization problems in the constraints. Operations research , 21(1):37-44, 1973.

T. Chen, Y. Sun, and W. Yin. Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems. Advances in Neural Information Processing Systems , 34:25294-25307, 2021.

T. Chen, Y. Sun, and W. Yin. A single-timescale stochastic bilevel optimization method. arXiv preprint arXiv:2102.04671 , 2021.

X. Chen, M. Huang, and S. Ma. Decentralized bilevel optimization. arXiv preprint arXiv:2206.05670 , 2022.

M. Dagréou, P. Ablin, S. Vaiter, and T. Moreau. A framework for bilevel optimization that enables stochastic and global variance reduction algorithms. arXiv preprint arXiv:2201.13409 , 2022.

J. Domke. Generic methods for optimization-based modeling. In Artificial Intelligence and Statistics , pages 318-326. PMLR, 2012.

A. Fallah, A. Mokhtari, and A. Ozdaglar. Personalized federated learning: A meta-learning approach. arXiv preprint arXiv:2002.07948 , 2020.

M. Feurer and F. Hutter. Hyperparameter optimization. Automated machine learning: Methods, systems, challenges , pages 3-33, 2019.

C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning , pages 1126-1135. PMLR, 2017.

L. Franceschi, M. Donini, P. Frasconi, and M. Pontil. Forward and reverse gradient-based hyperparameter optimization. In International Conference on Machine Learning , pages 11651173. PMLR, 2017.

L. Franceschi, P. Frasconi, S. Salzo, R. Grazzi, and M. Pontil. Bilevel programming for hyperparameter optimization and meta-learning. In International Conference on Machine Learning , pages 1568-1577. PMLR, 2018.

H. Gao. On the convergence of momentum-based algorithms for federated stochastic bilevel optimization problems. arXiv preprint arXiv:2204.13299 , 2022.

S. Ghadimi and M. Wang. Approximation methods for bilevel programming. arXiv preprint arXiv:1802.02246 , 2018.

S. Gould, B. Fernando, A. Cherian, P. Anderson, R. S. Cruz, and E. Guo. On differentiating parameterized argmin and argmax problems with application to bi-level optimization. arXiv preprint arXiv:1607.05447 , 2016.

R. Grazzi, L. Franceschi, M. Pontil, and S. Salzo. On the iteration complexity of hypergradient computation. In International Conference on Machine Learning , pages 3748-3758. PMLR, 2020.

A. Griewank and A. Walther. Evaluating derivatives: principles and techniques of algorithmic differentiation . SIAM, 2008.

Z. Guo, Q. Hu, L. Zhang, and T. Yang. Randomized stochastic variance-reduced methods for multi-task stochastic bilevel optimization. arXiv preprint arXiv:2105.02266 , 2021.

P. Hansen, B. Jaumard, and G. Savard. New branch-and-bound rules for linear bilevel programming. SIAM Journal on scientific and Statistical Computing , 13(5):1194-1217, 1992.

M. Hong, H.-T. Wai, Z. Wang, and Z. Yang. A two-timescale framework for bilevel optimization: Complexity analysis and application to actor-critic. arXiv preprint arXiv:2007.05170 , 2020.

F. Huang. Fast adaptive federated bilevel optimization. arXiv preprint arXiv:2211.01122 , 2022.

F. Huang and H. Huang. Biadam: Fast adaptive bilevel optimization methods. arXiv preprint arXiv:2106.11396 , 2021.

M. Huang, D. Zhang, and K. Ji. Achieving linear speedup in non-iid federated bilevel learning. arXiv preprint arXiv:2302.05412 , 2023.

Y. Huang, Q. Lin, N. Street, and S. Baek. Federated learning on adaptively weighted nodes by bilevel optimization. arXiv preprint arXiv:2207.10751 , 2022.

K. Ji, J. Yang, and Y . Liang. Bilevel optimization: Convergence analysis and enhanced design. In International conference on machine learning , pages 4882-4892. PMLR, 2021.

K. Ji and L. Ying. Network utility maximization with unknown utility functions: A distributed, data-driven bilevel optimization approach. arXiv preprint arXiv:2301.01801 , 2023.

S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International Conference on Machine Learning , pages 5132-5143. PMLR, 2020.

A. Khaled, K. Mishchenko, and P. Richtárik. First analysis of local gd on heterogeneous data. arXiv preprint arXiv:1909.04715 , 2019.

P. Khanduri, S. Zeng, M. Hong, H.-T. Wai, Z. Wang, and Z. Yang. A near-optimal algorithm for stochastic bilevel optimization via double-momentum. Advances in neural information processing systems , 34:30271-30283, 2021.

V. Konda and J. Tsitsiklis. Actor-critic algorithms. Advances in neural information processing systems , 12, 1999.

J. Koneˇ cn` y, B. McMahan, and D. Ramage. Federated optimization: Distributed optimization beyond the datacenter. arXiv preprint arXiv:1511.03575 , 2015.

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):2278-2324, 1998.

J. Li, B. Gu, and H. Huang. Improved bilevel model: Fast and optimal algorithm with theoretical guarantee. arXiv preprint arXiv:2009.00690 , 2020.

J. Li, F. Huang, and H. Huang. Local stochastic bilevel optimization with momentum-based variance reduction. arXiv preprint arXiv:2205.01608 , 2022.

J. Li, F. Huang, and H. Huang. Communication-efficient federated bilevel optimization with local and global lower level problems. arXiv preprint arXiv:2302.06701 , 2023.

T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems , 2:429-450, 2020.

R. Liao, Y. Xiong, E. Fetaya, L. Zhang, K. Yoon, X. Pitkow, R. Urtasun, and R. Zemel. Reviving and improving recurrent back-propagation. In International Conference on Machine Learning , pages 3082-3091. PMLR, 2018.

B. Liu, M. Ye, S. Wright, P. Stone, and Q. Liu. Bome! bilevel optimization made easy: A simple first-order approach. Advances in Neural Information Processing Systems , 35:17248-17262, 2022.

R. Liu, J. Gao, J. Zhang, D. Meng, and Z. Lin. Investigating bi-level optimization for learning and vision from a unified perspective: A survey and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(12):10045-10067, 2021.

R. Liu, X. Liu, X. Yuan, S. Zeng, and J. Zhang. A value-function-based interior-point method for non-convex bi-level optimization. In International Conference on Machine Learning , pages 6882-6892. PMLR, 2021.

R. Liu, Y. Liu, S. Zeng, and J. Zhang. Towards gradient-based bilevel optimization with nonconvex followers and beyond. Advances in Neural Information Processing Systems , 34:86628675, 2021.

R. Liu, P. Mu, X. Yuan, S. Zeng, and J. Zhang. A generic first-order algorithmic framework for bi-level programming beyond lower-level singleton. In International Conference on Machine Learning , pages 6305-6315. PMLR, 2020.

S. Lu, X. Cui, M. S. Squillante, B. Kingsbury, and L. Horesh. Decentralized bilevel optimization for personalized client learning. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 5543-5547. IEEE, 2022.

D. Maclaurin, D. Duvenaud, and R. Adams. Gradient-based hyperparameter optimization through reversible learning. In International conference on machine learning , pages 2113-2122. PMLR, 2015.

P. Manoharan, R. Walia, C. Iwendi, T. A. Ahanger, S. Suganthi, M. Kamruzzaman, S. Bourouis, W. Alhakami, and M. Hamdi. Svm-based generative adverserial networks for federated learning and edge computing attack model and outpoising. Expert Systems , page e13072, 2022.

B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics , pages 1273-1282. PMLR, 2017.

A. Mitra, R. Jaafar, G. J. Pappas, and H. Hassani. Linear convergence in federated learning: Tackling client heterogeneity and sparse gradients. Advances in Neural Information Processing Systems , 34:14606-14619, 2021.

M. Mohri, G. Sivek, and A. T. Suresh. Agnostic federated learning. In International Conference on Machine Learning , pages 4615-4625. PMLR, 2019.

R. Pathak and M. J. Wainwright. Fedsplit: An algorithmic framework for fast federated optimization. Advances in neural information processing systems , 33:7057-7066, 2020.

F. Pedregosa. Hyperparameter optimization with approximate gradient. In International conference on machine learning , pages 737-746. PMLR, 2016.

A. Rajeswaran, C. Finn, S. M. Kakade, and S. Levine. Meta-learning with implicit gradients. Advances in neural information processing systems , 32, 2019.

S. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Koneˇ cn` y, S. Kumar, and H. B. McMahan. Adaptive federated optimization. arXiv preprint arXiv:2003.00295 , 2020.

Y. Roh, K. Lee, S. E. Whang, and C. Suh. Fairbatch: Batch selection for model fairness. In International Conference on Learning Representations , 2021.

S. Sabach and S. Shtern. A first order method for solving convex bilevel optimization problems. SIAM Journal on Optimization , 27(2):640-660, 2017.

A. Shaban, C.-A. Cheng, N. Hatch, and B. Boots. Truncated back-propagation for bilevel optimization. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 1723-1732. PMLR, 2019.

P. Sharma, R. Panda, and G. Joshi. Federated minimax optimization with client heterogeneity. arXiv preprint arXiv:2302.04249 , 2023.

H. Shen and T. Chen. On penalty-based bilevel gradient descent method. arXiv preprint arXiv:2302.05185 , 2023.

C. Shi, J. Lu, and G. Zhang. An extended kuhn-tucker approach for linear bilevel programming. Applied Mathematics and Computation , 162(1):51-63, 2005.

R. Shokri and V. Shmatikov. Privacy-preserving deep learning. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security , pages 1310-1321, 2015.

A. Sinha, P. Malo, and K. Deb. A review on bilevel optimization: From classical to evolutionary approaches and applications. IEEE Transactions on Evolutionary Computation , 22(2):276-295, 2017.

D. Sow, K. Ji, Z. Guan, and Y. Liang. A constrained optimization approach to bilevel optimization with multiple inner minima. arXiv preprint arXiv:2203.01123 , 2022.

S. U. Stich. Local sgd converges fast and communicates little. arXiv preprint arXiv:1805.09767 , 2018.

S. U. Stich and S. P. Karimireddy. The error-feedback framework: Better rates for sgd with delayed gradients and compressed updates. The Journal of Machine Learning Research , 21(1):9613-9648, 2020.

D. A. Tarzanagh, M. Li, C. Thrampoulidis, and S. Oymak. Fednest: Federated bilevel, minimax, and compositional optimization. In International Conference on Machine Learning , pages 21146-21179. PMLR, 2022.

J. Wang and G. Joshi. Cooperative sgd: A unified framework for the design and analysis of local-update sgd algorithms. The Journal of Machine Learning Research , 22(1):9709-9758, 2021.

J. Wang, Q. Liu, H. Liang, G. Joshi, and H. V. Poor. Tackling the objective inconsistency problem in heterogeneous federated optimization. Advances in neural information processing systems , 33:7611-7623, 2020.

Z. Wang, K. Balasubramanian, S. Ma, and M. Razaviyayn. Zeroth-order algorithms for nonconvex minimax problems with improved complexities. arXiv preprint arXiv:2001.07819 , 2020.

P. Xiao and K. Ji. Communication-efficient federated hypergradient computation via aggregated iterative differentiation. arXiv preprint arXiv:2302.04969 , 2023.

E. P. Xing, Q. Ho, P. Xie, and D. Wei. Strategies and principles of distributed machine learning on big data. Engineering , 2(2):179-195, 2016.

P. Xing, S. Lu, L. Wu, and H. Yu. Big-fed: Bilevel optimization enhanced graph-aided federated learning. IEEE Transactions on Big Data , 2022.

J. Yang, K. Ji, and Y. Liang. Provably faster algorithms for bilevel optimization. Advances in Neural Information Processing Systems , 34:13670-13682, 2021.

S. Yang, X. Zhang, and M. Wang. Decentralized gossip-based stochastic bilevel optimization over communication networks. arXiv preprint arXiv:2206.10870 , 2022.

F. Yousefian. Bilevel distributed optimization in directed networks. In 2021 American Control Conference (ACC) , pages 2230-2235. IEEE, 2021.

Y. Zeng, H. Chen, and K. Lee. Improving fairness via federated learning. arXiv preprint arXiv:2110.15545 , 2021.

Y. Zhang, G. Zhang, P. Khanduri, M. Hong, S. Chang, and S. Liu. Revisiting and advancing fast adversarial training through the lens of bi-level optimization. In International Conference on Machine Learning , pages 26693-26712. PMLR, 2022.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on bilevel optimization in a federated learning environment. Bilevel optimization has various applications in federated learning (FL) and few recent works proposed versions of bilevel optimization schemes for FL. A challenging step in bilevel optimization is the computation of the \"hypergradient\", and the existing schemes are able to obtain an estimate of the hypergradient, albeit a biased estimate, with a substantial communication overhead (multiple rounds of communication per server-side update). The paper reformulates the hypergradient computation as a least squares problem that can provide an unbiased estimate of the hypergradient, while requiring only a single round of communication for every server-side update. Based on this estimate, the paper presents a federated bilevel optimization algorithm, SimFBO, and a variant robust to system-level heterogeneity, ShroFBO. The theoretical analyses demonstrate that the proposed algorithms can converge with a sample complexity comparable to that of existing federated bilevel schemes, while demonstrating a significant improvement in the communication overhead. Preliminary empirical results highlight the significant communication efficiency of the proposed schemes compared to the existing baselines.\n\nSTRENGTHS:\n**Well-motivated intuitive presentation.**\nThe authors do an excellent job in presenting this paper. The main problem and the challenges with the existing solutions are clearly discussed, and it is easy to see why the existing schemes are not very practical. The main idea of the use of the global least-squares reformulation, while obvious in hindsight, is very well presented and motivated, making it easy for the reader to follow along and realize how the existing challenges are mitigated. The theoretical results are clearly and intuitively presented with proper discussion highlighting the main parts of the analysis, and the main steps (with appropriate pointers to the supplement).\n\n\n**Critical reformulation removing both bias and communication overhead.**\nA key strength of this paper is a simple (yet of significant practical impact) reformulation of the hypergradient estimation using a standard quadratic program. A key property that the authors leverage is the fact that the global quadratic objective can be decomposed into per-client quadratic objectives, which is not true of the global hypergradient (which cannot be decomposed into per-client hypergradients). This simple yet powerful insight is then utilized to obtain an estimate which, upon proper solution of the global least-squares problem, is unbiased, and can be efficiently updated along side the upper and lower level variables in the bilevel problem. While this global least-squares reformulation does facilitate an intuitive communication-efficient algorithm, the paper also performs a thorough theoretical analysis, demonstrating how the inaccuracy in the hypergradient estimate plays into the convergence. The overall algorithm makes the solution of federated bilevel problems significantly more practical.\n\n\n**Generality of the proposed algorithmic framework.**\nThe authors do a great job at highlighting the generality of the algorithm framework. First, the general client and server aggregation (in equation (4)) allows us to cover various different client side optimizers, and the analysis is able to provide a convergence guarantee with such generalized aggregations. Second, the proposed framework incorporates system-level heterogeneity, allowing for different clients to perform different levels of client-side updates, and making the server-side aggregation robust to such heterogeneity. This robustness is demonstrated empirically, and the robust version of the algorithm is analysed theoretically.\n\nWEAKNESSES:\n**Increased hyperparameter space.**\nThe proposed framework utilizes various hyperparameters:\n- The client-side learning rates $\\eta_{y/v/x}$ and iterations $\\tau_i^{(t)}$ (for each server side update $t \\in [T]$ and each client $i \\in [n]$)\n- The server-side learning rates $\\gamma_{y/v/x}$\n- Potentially the choice of the client-side coefficients $a_i^{(t)} = \\left[a_i^{(t,0)}, \\ldots, a_i^{\\left(t, \\tau_i^{(t)}-1 \\right)} \\right]^\\top$ (for each client $i \\in [n]$), which might be potentially tied to $\\alpha_\\min, \\alpha_\\max$.\n- The hypergradient projection radius $r$\n\nAs per the theoretical analyses, it can be seen that the best convergence rate of any execution will critically depend on an appropriate setting of these problem-dependent hyperparameters. Since these hyperparameters often depend on quantities that cannot be efficiently estimated (such as Lipschitz constant), the practical bilevel implementations usually utilize some form of hyperparameter search. Hyperparameter optimization is known to be a hard unsolved problem in FL because of the overall communication overhead. This makes it hard to see how the proposed federated bilevel framework can live up to its practical potential -- one can view this proposed federated bilevel framework as having shifted the communication overhead from the model training stage to the hyperparameter optimization stage, without reducing the overall communication necessary for good training convergence (which involves trying various hyperparameters and training with them).\n\nQUESTIONS:\n- In lines 87-88, it seems that $\\tilde{v}^*(t)$ in the definition of $\\Delta_v^{(t)}$ depends on the iteration $t$. However, line 88 claims that $\\tilde{v}^*$ is the minimizer of $\\sum_{i=1}^n w_i R_i(x, y^*, \\cdot)$, which implies that $\\tilde{v}^*(t)$ is not dependent on $t$. Can this be clarified? Usually in the bilevel analysis, we are tracking quantities such as $|| y^*(x^{(t)}) - y^{(t+1)} ||$ for dependent variables, such as $y$ and $v$. So $\\tilde{v}^*(t)$ can also be the minimizer of $\\sum_{i=1}^n w_i R_i(x^{(t)}, y^*(x^{(t)}), \\cdot)$ or even $\\sum_{i=1}^n w_i R_i(x^{(t)}, y^{(t+1)}, \\cdot)$.\n\n- What is $\\bar{\\tau}$ in Theorem 1? It seems to be introduced in the theorem statement, but I am not unable to find (in the main paper) what this $\\bar{\\tau}$ is supposed to signify. Is it some aggregate of the $\\{ \\tau_i^{(t)} \\}$ across all $i \\in [n]$ and $t \\in [T]$?\n\nMinor:\n- The legend in Figure 2 (right) seems off since it has no SimFBO.\n\nLIMITATIONS:\nI did not find any discussion on limitations in the main paper. However, I do not anticipate any potential negative societal impact of this work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work consider the federated bilevel optimization problem. Compared to existing methods, the authors develop a new and simple method named SimFBO without subloops and requiring much fewer communication rounds at each iteration. In the setting with system-level heterogeneity like diverse local steps, they further propose a more robust version named ShroFBO that is shown with a correct convergence under such heterogeneity. Convergence analysis is developed for both methods. Empirical results have shown the great performance of the proposed methods.\n\nSTRENGTHS:\n1. Federated bilevel optimization is a relatively new but challenging problem even when the lower-level objective is strongly convex. This work provides a reasonable and quite effective framework to this problem. \n\n2. The proposed algorithms are novel, easy to understand and theoretically grounded. Providing a simple solution with strong empirical and theoretical performance is strong. \n\n3. Technical derivations seem to be nontrivial. For example, it is interesting to deal with the challenges by client drift, boundness of local iterates etc. It seems to fill the gap of communication complexity of $1/\\epsilon$ in federated bilevel problems. This is good.\n\nWEAKNESSES:\nThe proposed methods require second-order derivatives, which may cause some scalability issue. Is it possible to develop fully first-order methods given the current SimFBO framework.\n\nQUESTIONS:\nAnother question is: In the right plot of Fig. 2, is the legend correct? It missed SimFBO. \n\nOverall, this is a strong work with a promising solution to federated bilevel optimization, so I suggest the acceptance.\n\nLIMITATIONS:\nA limitation of their work is the requirement of the lower-level strong convexity in Assumption 1. It would be interesting to explore if this condition can be relaxed or eliminated, taking into account recent advancements in the field, such as those presented in [1, 2].\n\n[1] B. Liu et al. “Bome! bilevel optimization made easy: A simple first-order approach.” NeurIPS 2022.\n\n[2] R. Liu et al. “Averaged Method of Multipliers for Bi-Level Optimization without Lower-Level Strong Convexity.” ICML 2023.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies the federated bilevel optimization, where the lower- and upper-level objectives are defined over all clients. Since the lower-level solution is the minimizer of the average of all client objectives (i.e., in a global manner), the main computational challenge is to compute the global hypergradient with second-order derivatives. This work provides a simple and effective approach named SimFBO to this problem. It contains simultaneous updates on all three variables at each client, a generalized server-level update. It has no subloops. A variant of SimFBO called as ShroFBO is also proposed to deal with the heterogeneous client computation. \nTheoretical convergence is provided for these two approaches.  Experiments seem to show the proposed methods are much better than other baselines.\n\nSTRENGTHS:\n1. The work is well motivated and easy to follow. Finding a simple but effective solution to a complicated problem in federated setting is a interesting and important topic. \n\n2. The algorithm design is new, i.e., simultaneous local updates, aggregation, and generalized server-side operation. The simple structure will be useful in practice. But such simple structure seems to be not easy to prove in the federated setting. Specifically, it is usually challenging to guarantee the boundedness of $v$ variable. The authors show this via induction that as long as the server-side $v$ is bounded, the local $v$ is bounded as well. This seems to be new.\n\n3. Nonasymptotic convergence is provided and seems to be significant. It allows for client sampling without replacement, linear speedup, achieves the $1/\\epsilon$ communication complexity. In contrast, existing studies require clients to be selected with replacement due to the complex hypergradient construction. This work achieves the linear speedup without replacement, and as well as optimal communication complexity. These are new results in federated/distributed optimization.\n\nWEAKNESSES:\nFor the experimental part, can the authors explain why FedNest and AggITD perform poorly over CNNs? Is it due to a non-proper hyperparameter tuning or something else? More details should be provided.\n\nQUESTIONS:\n1. Does the $1/\\epsilon$ communication complexity hold for both partial and full client participation? Per my reading, it only holds for the full participation case, right? I think the authors should make this clearer. \n\n2. From the experiments, it is not clear to me how many local steps are used for the proposed methods? Also it would be great to investigate the impact of number of local steps on the final convergence rate and test accuracy. \n\n3. Is it possible to incorporate the variance reduction or momentum-based estimators to improve the sample complexity to $O(\\epsilon^{-3/2})$ under the current simple framework? It would be great to include some discussions on this.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the bilevel optimization problem in the federated learning context. The authors propose a novel gradient-based algorithm that effectively updates the arguments in both the inner and outer optimization problems simultaneously. Additionally, the paper extends this algorithm to handle data heterogeneity, making it more relevant for real-world scenarios. The theoretical analysis on the convergence rate of the proposed approach is also provided. While the content itself is interesting, the manuscript requires substantial proofreading and improvements before it is ready for peer review.\n\nSTRENGTHS:\n- The paper demonstrates a comprehensive treatment of the bilevel optimization problem in federated learning. Notably, the proposed method addresses data heterogeneity, contributing to the algorithm's practical applicability.\n\n- The inclusion of a comparison with existing methods in Table 1 is valuable.\n\nWEAKNESSES:\n- There are still rooms to improve the presentation of the paper.\n\n    - The paper suffers from unclear or inconsistent mathematical notations. E.g.\n\n        - In defining the objective functions, a bracket should be used otherwise the meaning is not correct. Eg. in (1), \n        $$\\min \\{\\Phi(x) = F(x.y^*(x))\\}$$\n        The objective functions defined elsewhere in the manuscript has the same issue.\n\n        - In (3), the notations $\\zeta_i$ and $\\xi_i$ are not defined. Similarly in (4). It is important to highlight they corresponds to local mini-batch dataset, as they make a difference in the proof of Lemmas in the appendix.\n        \n        - In assumption 1, it is better to use $\\mu_g$ for the $\\mu$-strongly convex constant as it is used elsewhere.\n                \n        - The notation $\\delta_t'$ in Propostiation 2 is not defined. Though the reviewer find the definition in the appendix, the notation should be defined when it is firstly used. Similarly for other notations such as $\\bar\\tau$ in Theorem 1.\n\n        - In Appendix $C$, Lemma $2$ is w.r.t. the function $G$ not $g$.\n\n        \n    - There are numerous typos and missing details in the appendix. E.g.\n\n        - In Proof of Lemma 3, the statement \"step 6 of algorithm (1)\" does not match the algorithm description.\n\n        - The proof of Lemma 4 missed some key steps and hence is not clear.\n        \n        - In the proof of Lemma 8 in L569. The first equality should have expectation on the LHS and the second equality should be $\\leq$.\n        \n    - Presentation\n\n        - The paper could benefit from providing concrete examples of bilevel optimization after (1) to underscore the problem's importance.\n\n        - The main text should reference where readers can find proofs for propositions.\n\n        - The description of the experimental setting is vague; a clearer explanation of why the studied problem is a bilevel optimization problem is needed.\n\n\n- The novelty in the proposed method is limited.\n    - The key method in Section 2.3 is a distributed version of the centralized gradient-based bilevel optimization method.\n\n    - The method for hetergoeneity is a simple generalization of FedNova.\n\n- Since the reviewer thinks the proposed method is a simple extension of existing algorithms under federated learning setting, the convergence rate analysis of the proposed algorithm is therefore a key contribution of the paper. There are several issues in technical details.\n    \n    - In Lemma 5, what is the definition of $\\nabla R_i(x,y,v)$? Is it only the partial derivative w.r.t. $v$? Otherwise the gradient is not well defined since $g_i$ and $f_i$ are only assumed to be twice differentiable.\n    \n    - Due to the unclear definition of the notations and many tiny typos in the manuscript, the proof is not very easy to follow. For example, Lemma 10 is a key lemma for the proof of theorem 1, but it is not clear why the first step (24) holds since $\\delta_t$ is not defined. Therefore, it is hard for the reviewer to check whether the proof is correct or not. \n\n    - In the proof of the theorem 1, it is not clear to the reviewer how the projection of $v$ is reflected in the proof of the theorem.\n     \n    \n    - The proof of Theorem 2, it does not make sense to the reviewer that \"taking $w_i=p_i$\" in L662.\n\n\n- It would greatly improve the quality of the paper if the summary of the results in Table 1 can be empirically demonstrated in the experiment.\n\n\nReference:\n\n[1] Gradient-based Bi-level Optimization for Deep Learning: A Survey\n\nQUESTIONS:\nSee the questions in the above section.\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-20"}