Title: Communication-Efficient Federated Bilevel Optimization with Global and Local Lower Level Problems
Venue: NeurIPS 2023
Accepted: yes
Keywords: Federated Learning

Bilevel Optimization
Abstract: Bilevel Optimization has witnessed notable progress recently with new emerging efficient algorithms. However, its application in the Federated Learning setting remains relatively underexplored, and the impact of Federated Learning's inherent challenges on the convergence of bilevel algorithms remain obscure.
In this work, we investigate Federated Bilevel Optimization problems and propose a communication-efficient algorithm, named FedBiOAcc. The algorithm leverages an efficient estimation of the hyper-gradient in the distributed setting and utilizes the momentum-based variance-reduction acceleration. Remarkably, FedBiOAcc achieves a communication complexity $O(\epsilon^{-1})$, a sample complexity $O(\epsilon^{-1.5})$ and the linear speed up with respect to the number of clients. We also analyze a special case of the Federated Bilevel Optimization problems, where lower level problems are locally managed by clients. We prove that FedBiOAcc-Local, a modified version of FedBiOAcc, converges at the same rate for this type of problems. Finally, we validate the proposed algorithms through two real-world tasks: Federated Data-cleaning and Federated Hyper-representation Learning. Empirical results show superior performance of our algorithms.

Communication-Efficient Federated Bilevel Optimization with Global and Local Lower Level Problems

Abstract

Bilevel Optimization has witnessed notable progress recently with new emerging efficient algorithms. However, its application in the Federated Learning setting remains relatively underexplored, and the impact of Federated Learning's inherent challenges on the convergence of bilevel algorithms remain obscure. In this work, we investigate Federated Bilevel Optimization problems and propose a communication-efficient algorithm, named FedBiOAcc. The algorithm leverages an efficient estimation of the hyper-gradient in the distributed setting and utilizes the momentum-based variance-reduction acceleration. Remarkably, FedBiOAcc achieves a communication complexity O ( ϵ -1 ) , a sample complexity O ( ϵ -1 . 5 ) and the linear speed up with respect to the number of clients. We also analyze a special case of the Federated Bilevel Optimization problems, where lower level problems are locally managed by clients. We prove that FedBiOAcc-Local, a modified version of FedBiOAcc, converges at the same rate for this type of problems. Finally, we validate the proposed algorithms through two real-world tasks: Federated Datacleaning and Federated Hyper-representation Learning. Empirical results show superior performance of our algorithms.

1 Introduction

Bilevel optimization [51, 48] has increasingly drawn attention due to its wide-ranging applications in numerous machine learning tasks, including hyper-parameter optimization [42], meta-learning [61] and neural architecture search [36]. A bilevel optimization problem involves an upper problem and a lower problem, wherein the upper problem is a function of the minimizer of the lower problem. Recently, great progress has been made to solve this type of problems, particularly through the development of efficient single-loop algorithms that rely on diverse gradient approximation techniques [23]. However, the majority of existing bilevel optimization research concentrates on standard, non-distributed settings, and how to solve the bilevel optimization problems under distributed settings have received much less attention. Federated learning (FL) [40] is a recently promising distributed learning paradigm. In FL, a set of clients jointly solve a machine learning task under the coordination of a central server. To protect user privacy and mitigate communication overhead, clients perform multiple steps of local update before communicating with the server. A variety of algorithms [50, 59, 16, 26, 1] have been proposed to accelerate this training process. However, most of these algorithms primarily address standard single-level optimization problems. In this work, we study the bilevel optimization problems in the Federated Learning setting and investigate the

following research question: Is it possible to develop communication-efficient federated algorithms tailored for bilevel optimization problems that also ensure a rapid convergence rate?

More specifically, a general Federated Bilevel Optimization problem has the following form:

A federated bilevel optimization problem consists of an upper and a lower level problem, the upper problem f ( x, y ) := 1 M ∑ M m =1 f ( m ) ( x, y ) relies on the solution y x of the lower problem, and g ( x, y ) := 1 M ∑ M m =1 g ( m ) ( x, y ) . Meanwhile, both the upper and the lower level problems are federated: In Eq.(1), we have M clients, and each client has a local upper problem f ( m ) ( x, y ) and a lower level problem g ( m ) ( x, y ) . Compared to single-level federated optimization problems, the estimation of the hyper-gradient in federated bilevel optimization problems is much more challenging. In Eq.(1), the hyper-gradient is not linear w.r.t the local hyper-gradients of clients, whereas the gradient of a single-level Federated Optimization problem is the average of local gradients. Consequently, directly applying the vanilla local-sgd method [40] to federated bilevel problems results in a large bias. In the literature [49, 33, 21, 53], researchers evaluate the hyper-gradient through multiple rounds of client-server communication, however, this approach leads to high communication overhead. In contrast, we view the hyper-gradient estimation as solving a quadratic federated problem and solving it with the local-sgd method. More specifically, we formulate the solution of the federated bilevel optimization as three intertwined federated problems: the upper problem, the lower problem and the quadratic problem for the hyper-gradient estimation. Then we address the three problems using alternating gradient descent steps, furthermore, to manage the noise of the stochastic gradient and obtain the fast convergence rate, we employ a momentum-based variance reduction technique.

Beyond the standard federated bilevel optimization problem as defined in Eq. 1, another variant of Federated Bilevel Optimization problem, which entails locally managed lower-level problems, is also frequently utilized in practical applications. For this type of problem, we can get an unbiased estimate of the global hyper-gradient using local hyper-gradient, thus we can solve it with a local-SGD like algorithm, named FedBiOAcc-Local. However, it is challenging to analyze the convergence of the algorithm. In particular, we need to bound the intertwined client drift error, which is intrinsic to FL and the bilevel-related errors e.g. the lower level solution bias. In fact, we prove that the FedBiOAcc-Local algorithm attains the same fast rate as FedBiO algorithm.

Finally, we highlight the main contributions of our paper as follows:

We propose FedBiOAcc to solve Federated Bilevel Optimization problems, the algorithm evaluates the hypergradient of federated bilevel optimization problems efficiently and achieves optimal convergence rate through momentum-based variance reduction. FedBiOAcc has sample complexity of O ( ϵ -1 . 5 ) , communication complexity of O ( ϵ -1 ) and achieves linear speed-up w.r.t the number of clients.

We study Federated Bilevel Optimization problem with local lower level problem for the first time, where we show the convergence of a modified version of FedBiOAcc, named FedBiOAcc-Local for this type of problems.

We validate the efficacy of the proposed FedBiOAcc algorithm through two real-world tasks: Federated Data Cleaning and Federated Hyper-representation Learning.

Notations ∇ denotes full gradient, ∇ x denotes partial derivative for variable x, higher order derivatives follow similar rules. [ K ] represents the sequence of integers from 1 to K , ¯ x represents average of the sequence of variables { x ( m ) } M m =1 . ¯ t s represents the global communication timestamp s .

2 Related Works

Bilevel optimization dates back to at least the 1960s when [51] proposed a regularization method, and then followed by many research works [10, 48, 55, 45], while in machine learning community, similar ideas in the name of implicit differentiation were also used in Hyper-parameter Optimization [30, 3, 2, 8]. Early algorithms for Bilevel Optimization solved the accurate solution of the lower problem for each upper variable. Recently, researchers developed algorithms that solve the lower problem with a fixed number of steps, and use the 'back-propagation through time' technique to compute the hyper-gradient [9, 39, 11, 43, 47]. Very Recently, it witnessed a surge of interest in using implicit differentiation to derive single loop algorithms [14, 17, 23, 28, 4, 56, 19, 32, 7, 20, 18]. In particular, [32, 7] proposes a way to iteratively evaluate the hyper-gradients to save computation. In this work, we view the hyper-gradient estimation of Federated Bilevel Optimization as solving a quadratic federated optimization problem and use a similar iterative evaluation rule as [32, 7] in local update.

The bilevel optimization problem is also considered in the more general settings. For example, bilevel optimization with multiple lower tasks is considered in [15], furthermore, [5, 57, 38, 13] studies the bilevel optimization problem in the decentralized setting, [25] studies the bilevel optimization problem in the asynchronous setting. In contrast, we study bilevel optimization problems under Federated Learning [40] setting. Federated learning is a promising privacy-preserving learning paradigm for distributed data. Compared to traditional data-center distributed learning, Federated Learning poses new challenges including data heterogeneity, privacy concerns, high communication cost, and unfairness. To deal with these challenges, various methods [26, 35, 46, 60, 41, 34] are proposed. However, bilevel optimization problems are less investigated in the federated learning setting. [54] considered the distributed bilevel formulation, but it needs to communicate the Hessian matrix for every iteration, which is computationally infeasible. More recently, FedNest [49] has been proposed to tackle the general federated nest problems, including federated bilevel problems. However, this method evaluates the full hyper-gradient at every iteration; this leads to high communication overhead; furthermore, FedNest also uses SVRG to accelerate the training. Similar works that evaluate the hyper-gradient with multiple rounds of client-server communication are [33, 21, 53, 58]. Finally, there is a concurrent work [12] that investigates the possibility of local gradients on Federated Bilevel Optimization, however, it only considers the homogeneous case, this setting is quite constrained and much simpler than the more general heterogeneous case we considered. Furthermore, [12] only considers the case where both the upper and the lower problem are federated, and omit the equally important case where the lower level problem is not federated.

3 Federated Bilevel Optimization

3.1 Some Mild Assumptions

Note that the formulation of Eq.(1) is very general, and we consider the stochastic heterogeneous case in this work. More specifically, we assume:

̸

where D ( m ) f and D ( m ) g are some probability distributions. Furthermore, we assume the local objectives could be potentially different: f ( m ) ( x, y ) = f ( k ) ( x, y ) or g ( m ) ( x, y ) = g ( k ) ( x, y ) for m = k, m, k ∈ [ M ] . Furthermore, we assume the following assumptions in our subsequent discussion:

̸

̸

Assumption 3.1. Function f ( m ) ( x, y ) is possibly non-convex and g ( m ) ( x, y ) is µ -strongly convex w.r.t y for any given x .

Assumption 3.2. Function f ( m ) ( x, y ) is L -smooth and has C f -bounded gradient;

Assumption 3.3. Function g ( m ) ( x, y ) is L -smooth, and ∇ xy g ( m ) ( x, y ) and ∇ y 2 g ( m ) ( x, y ) are Lipschitz continuous with constants L xy and L y 2 respectively;

Algorithm 1 Accelerated Federated Bilevel Optimization ( FedBiOAcc )

1: Input: Constants c ω , c ν , c u , γ , η , τ , r ; learning rate schedule { α t } , t ∈ [ T ] , initial state ( x 1 , y 1 , u 1 ); 2: Initialization: Set y ( m ) 1 = y 1 , x ( m ) 1 = x 1 , u ( m ) 1 = u 1 , ω ( m ) 1 = ∇ y g ( m ) ( x 1 , y 1 , B y ) , ν ( m ) 1 = ∇ x f ( m ) ( x 1 , y 1 ; B f, 1 ) - ∇ xy g ( m ) ( x 1 , y 1 ; B g, 1 ) u 1 and q 1 = ∇ y 2 g ( m ) ( x ( m ) 1 , y ( m ) 1 ; B g, 2 ) u 1 -∇ y f ( m ) ( x ( m ) 1 , y ( m ) 1 ; B f, 2 ) for m ∈ [ M ] 3: for t = 1 to T do 4: ˆ y ( m ) t +1 = y ( m ) t -γα t ω ( m ) t , ˆ x ( m ) t +1 = x ( m ) t -ηα t ν ( m ) t , ˆ u ( m ) t +1 = P r ( u ( m ) t -τα t q ( m ) t ) 5: if t mod I = 0 then 6: y ( m ) t +1 = 1 M ∑ M j =1 ˆ y ( j ) t +1 ; x ( m ) t +1 = 1 M ∑ M j =1 ˆ x ( j ) t +1 , u ( m ) t +1 = 1 M ∑ M j =1 ˆ u ( j ) t +1 7: else 8: y ( m ) t +1 = ˆ y ( m ) t +1 , x ( m ) t +1 = ˆ x ( m ) t +1 , u ( m ) t +1 = ˆ u ( m ) t +1 9: end if 10: Get ˆ ω ( m ) t +1 , ˆ ν ( m ) t +1 and ˆ q ( m ) t +1 following Eq. (7) 11: if t mod I = 0 then 12: ω ( m ) t +1 = 1 M ∑ M j =1 ˆ ω ( j ) t +1 , ν ( m ) t +1 = 1 M ∑ M j =1 ˆ ν ( j ) t +1 , q ( m ) t +1 = 1 M ∑ M j =1 ˆ q ( j ) t +1 , 13: else 14: ω ( m ) t +1 = ˆ ω ( m ) t +1 , ν ( m ) t +1 = ˆ ν ( m ) t +1 , q ( m ) t +1 = ˆ q ( m ) t +1 15: end if 16: end for

Assumption 3.4. We have unbiased stochastic first-order and second-order gradient oracle with bounded variance.

As stated in The assumption 3.1, we study the non-convex-strongly-convex bilevel optimization problems, this class of problems is widely studied in the non-distributed bilevel literature [22, 14]. Furthermore, Assumption 3.2 and Assumption 3.3 are also standard assumptions made in the nondistributed bilevel literature. Assumption 3.4 is widely used in the study of stochastic optimization problems. For Assumption 3.5, gradient difference is widely used in single level Federated Learning literature as a measure of client heterogeneity [28, 52]. Please refer to the full version of Assumptions in Appendix.

3.2 The FedBiOAcc Algorithm

A major difficulty in solving a Federated Bilevel Optimization problem Eq. (1) is evaluating the hyper-gradient ∇ h ( x ) . For the function class (non-convex-strongly-convex) we consider, the explicit form of hypergradient h ( x ) exists as ∇ h ( x ) = Φ( x, y x ) , where Φ( x, y ) is denoted as:

Based on Assumption 3.1 ∼ 3.3, we can verify Φ( x, y x ) is the hyper-gradient [14]. But since the clients only have access to their local data, for ∀ m ∈ [ M ] , the client evaluates:

̸

It is straightforward to verify that Φ ( m ) ( x, y ) is not an unbiased estimate of the full hyper-gradient, i.e. Φ( x, y x ) = 1 M ∑ M m =1 Φ ( m ) ( x, y x ) . To address this difficulty, we can view the Hyper-gradient computation as the process of solving a federated optimization problem .

In fact, Evaluating Eq. (2) is equivalent to the following two steps: first, we solve the quadratic federated optimization problem l ( u ) :

Suppose that we denote the solution of the above problem as u ∗ , then we have the following linear operation to get the hypergradient:

Compared to the formulation Eq. (2), Eq. (4) and Eq. (5) are more suitable for the distributed setting. In fact, both Eq. (4) and Eq. (5) have a linear structure. Eq. (4) is a (single-level) quadratic federated optimization problem, and we could solve Eq. (4) through local-sgd [40], suppose that each client maintains a variable u ( m ) t , and performs the following update:

where ∇ l ( m ) ( u ( m ) t ; B ) is client m 's the stochastic gradient of Eq. (4), and ( x ( m ) t , y ( m ) t ) denotes the upper and lower variable state at the timestamp t , the P r ( · ) denotes the projection to a bounded ball of radiusr . Note that Clients perform multiple local updates of u ( m ) t before averaging. As for Eq. (5), each client evaluates ∇ h ( m ) ( x ) locally: ∇ h ( m ) ( x ) = ∇ x f ( m ) ( x, y x ) -∇ xy g ( m ) ( x, y x ) u ∗ and the server averages ∇ h ( m ) ( x ) to get ∇ h ( x ) . In summary, the linear structure of Eq. (4) and Eq. (5) makes it suitable for local updates, therefore, reduce the communication cost.

More specifically, we perform alternative update of upper level variable x ( m ) t , the lower level variable y ( m ) t and hyper-gradient computation variable u ( m ) t . For example, for each client m ∈ [ M ] , we perform the following local updates:

Every I steps, the server averages clients' local states, this resembles the local-sgd method for single level federated optimization problems. Note that in the update of the upper variable x ( m ) t , we use u ( m ) t as an estimation of u ∗ in Eq. (5). An algorithm follows Eq. (6) is shown in Algorithm 2 of Appendix and we refer to it as FedBiO.

Comparison with FedNest. The update rule of Eq. 6 is very different from that of FedNest [49] and its follow-ups [21, 53]. In FedNest, a sub-routine named FedIHGP is used to evaluate Eq. (2) at every global epoch. This involves multiple rounds of client-server communication and leads to higher communication overhead. In contrast, Eq. (6) formulates the hyper-gradient estimation as an quadratic federated optimization problem, and then solves three intertwined federated problems through alternative updates of x , y and u .

Note that Eq. 6 updates the related variables through vanilla gradient descent steps. In the nonfederated setting, gradient-based methods such as stocBiO [23] requires large-batch size ( O ( ϵ -1 ) ) to reach an ϵ -stationary point, and we also analyze Algorithm 2 in Appendix to show the same dependence. To control the noise and remove the dependence over large batch size, we apply the momentum-based variance-reduction technique STORM [6]. In fact, Eq. (6) solves three intertwined optimization problems: the bilevel problem h ( x ) , the lower level problem g ( x, y ) and the hypergradient computation problem Eq (4). So we control the noise in the process of solving each of the three problems. More specifically, we have ω ( m ) t , ν ( m ) t and q ( m ) t to be the momentum estimator for x ( m ) t , y ( m ) t and u ( m ) t respectively, and we update them following the rule of STORM [6]:

where c ω , c ν and c u are constants, α t is the learning rate. Then we update the x ( m ) t , y ( m ) t and u ( m ) t as follows:

where γ , η , τ are constants and α t is the learning rate. The FedBiOAcc algorithm following Eq. (8) is summarized in Algorithm 1. As shown in line 6 and 12 of Algorithm 1, Every I iterations, we average both variables and the momentum.

3.3 Convergence Analysis

In this section, we study the convergence property for the FedBiOAcc algorithm. For any t ∈ [ T ] , we define the following virtual sequence:

we denote the average of the momentum similarly as ¯ ω t , ¯ ν t and ¯ q t . Then we consider the following Lyapunov function G t :

where y ¯ x t denotes the solution of the lower level problem g (¯ x t , · ) , u ¯ x t = [ ∇ y 2 g (¯ x, y ¯ x )] -1 ∇ y f (¯ x, y ¯ x ) denotes the solution of Eq (4) at state ¯ x t . Besides, γ, η, τ are learning rates and L, ˜ L are constants. Note that the first three terms of G t : h (¯ x t ) , ∥ ¯ y t -y ¯ x t ∥ 2 , ∥ ¯ u t -u ¯ x t ∥ 2 measures the errors of three federated problems: the upper level problem, the lower level problem and the hyper-gradient estimation. Then the last three terms measure the estimation error of the momentum variables: ¯ ω t , ¯ ν t and ¯ q t . The convergence proof primarily concentrates on bounding these errors, please see Lemma C.2 - C.6 in the Appendix for more details. Meanwhile, as in the single level federated optimization problems, local updates lead to client-drift error. More specifically, we need to bound ∥ x ( m ) t -¯ x t ∥ 2 , ∥ y ( m ) t -¯ y t ∥ 2 and ∥ u ( m ) t -¯ u t ∥ 2 , please see Lemma C.7 - C.11 for more details. Finally, we have the following convergence theorem:

Theorem 3.6. Suppose in Algorithm 1, we choose learning rate α t = δ ( u + t ) 1 / 3 , t ∈ [ T ] , for some constant δ and u , and let c ν , c ω , c u choose some value, η , γ and τ , r be some small values decided by the Lipschitz constants of h ( x ) , we choose the minibatch size to be b x = b y = b and the first batch to be b 1 = O ( Ib ) , then we have:

To reach an ϵ -stationary point, we need T = O ( κ 8 ( bM ) -1 ϵ -1 . 5 ) , I = O ( κ 5 / 3 ( bM ) -1 ϵ -0 . 5 ) .

As stated in the Theorem, to reach an ϵ -stationary point, we need T = O ( κ 8 ( bM ) -1 ϵ -1 . 5 ) , then the sample complexity for each client is Gc ( f, ϵ ) = O ( M -1 κ 8 ϵ -1 . 5 ) , Gc ( g, ϵ ) = O ( M -1 κ 8 ϵ -1 . 5 ) , Jv ( g, ϵ ) = O ( M -1 κ 8 ϵ -1 . 5 ) , Hv ( g, ϵ ) = O ( M -1 κ 8 ϵ -1 . 5 ) . So FedBiOAcc achieves the linear speed up w.r.t. to the number of clients M . Next, suppose we choose I = O ( κ 5 / 3 ( bM ) -1 ϵ -0 . 5 ) , then the number of communication round E = O ( κ 19 / 3 ϵ -1 ) . This matches the optimal communication complexity of the single level optimization problems as in the STEM [27]. Furthermore, compared to FedNest and its variants, FedBiOAcc has improved both the communication complexity and the iteration complexity. As for LocalBSCVR [12], FedBiOAcc obtains same rate, but incorporates the heterogeneous case. Note that it is much more challenging to analyze the heterogeneous case. In fact, if we assume homogeneous clients, we have local hyper-gradient (Eq. (3)) equals the global hyper-gradient (Eq. (2)), then we do not need to use the quadratic federated optimization problem view in Section 3.2, while the theoretical analysis is also simplified significantly.

4 Federated Bilevel Optimization with Local Lower Level Problems

In this section, we consider an alternative formulation of the Federated Bilevel Optimization problems as follows:

Same as Eq. (1), Eq. (10) has a federated upper level problem, however, Eq. (10) has a unique lower level problem for each client, which is different from Eq. (1). In fact, federated bilevel optimization problem Eq (10) can be viewed as a special type of standard federated learning problems. If we denote h ( m ) ( x ) = f ( m ) ( x, y ( m ) x ) , then Eq. (10) can be written as min x ∈ R p h ( x ) := 1 M ∑ M m =1 h ( m ) ( x ) .

But due to the bilevel structure of h ( m ) ( x ) , Eq. (10) is more challenging than the standard Federated Learning problems.

Hyper-gradient Estimation. Assume Assumption 3.1 ∼ Assumption 3.3 hold, then the hyper-gradient is Φ( x, y x ) = 1 M ∑ M m =1 Φ ( m ) ( x, y x ) , where Φ ( m ) ( x, y ) is defined in Eq. (3), in other words, the local hyper-gradient Φ ( m ) ( x, y ) is an unbiased estimate of the full hyper-gradient. This fact makes it possible to solve Eq. (10) with local-sgd like methods. More specifically, we solve the local bilevel problem h ( m ) ( x ) multiple steps on each client and then the server averages the local states from clients. Please refer to Algorithm 3 and the variance-reduction acceleration Algorithm 4 in the Appendix. For ease of reference, we name them FedBiO-Local and FedBiOAcc-Local, respectively.

Several challenges exist in analyzing FedBiO-Local and FedBiOAcc-Local. First, Eq. (3) involves Hessian inverse, so we only evaluate it approximately through the Neumann series [37] as:

where ξ x = { ξ j ( j = 1 , . . . , Q ) , ξ f , ξ g } , and we assume its elements are mutually independent. Φ ( m ) ( x, y ; ξ x ) is a biased estimate of Φ ( m ) ( x, y ) , but with bounded bias and variance (Please see Proposition D.2 for more details.) Furthermore, to reduce the computation cost, each client solves the local lower level problem approximately and we update the upper and lower level variable alternatively. The idea of alternative update is widely used in the non-distributed bilevel optimization [23, 56]. However, in the federated setting, client variables drift away when performing multiple local steps. As a result, the variable drift error and the bias caused by inexact solution of the lower level problem intertwined with each other. For example, in the local update, clients optimize the lower level variable y ( m ) towards the minimizer y ( m ) x ( m ) , but after the communication step, x ( m ) is smoothed among clients, as a result, the target of y ( m ) t changes which causes a huge bias.

In the appendix, we show the FedBiOAcc-Local algorithm achieves the same optimal convergence rate as FedBiOAcc, which has iteration complexity O ( ϵ -1 . 5 ) and communication complexity O ( ϵ -1 ) . However, since the lower level problem in Eq. (10) is unique for each client, FedBiOAcc-Local does not have the property of linear speed-up w.r.t the number of clients as FedBiOAcc does.

5 Numerical Experiments

In this section, we assess the performance of the proposed FedBiOAcc algorithm through two federated bilevel tasks: Federated Data Cleaning and Federated Hyper-representation Learning. The Federated Data Cleaning task involves global lower level problems, while the Hyper-representation Learning task involves local lower level problems. The implementation is carried out using PyTorch, and the Federated Learning environment is simulated using the PyTorch.Distributed package. Our experiments were conducted on servers equipped with an AMD EPYC 7763 64-core CPU and 8 NVIDIA V100 GPUs.

FedAvg

FedAvg

EIEIS:

FedAvg

2.0

FedNest

CommFedBiO

2.0

CommFedBiO

FedNest

2.0

CommFedBiO

FedNest

2.2

Validation Loss

FedMBO

AggITD

Validation Loss

FedMBO

AggITD

SS07

FedMBO

AggITD

SS07

1.5

Local-BSGVR

FedBiO

FedBiOAcC

1.5

Local-BSGVR

FedBiO

FedBiOAcc

Validation L

1.5

Local-BSGVR

FedBiO

FedBiOAcc

Validation

1.9

FedAvg

CommFedBiO

1.0

1.0

1.0

1.6

FedNest

FedMBO

AggITD

0.5

0.5

0.5

1.3

Local-BSGVR

FedBiO

FedBiOAcc

0.0

100

000

400

0.0

0

00T

200

000

400

0.0

0

100

200

300

400

1.0

0

100

200

00E

400

Communication Rounds

Communication Rounds

Communication Rounds

Communication Rounds

1 clients

5 clients

5 clients

1 clients

1 clients

5 clients

1 clients

5 clients

2.0

10 clients

2.0

10 clients

0

10 clients

2.2

10 clients

Validation Loss

Validation Loss

Validation Loss

Validation Loss

1.5

1.5

1.5

1.9

1.0

1.0

1.0

1.6

0.5

用

0.5

0.5

1.3

0.0

0

100

200

000

400

0.0

0

00T

200

000

400

0.0

0

100

200

300

400

1.0

0

100

200

0OE

400

Communication Rounds

Communication Rounds

Communication Rounds

Communication Rounds

5.1 Federated Data Cleaning

In this section, we consider the Federated Data Cleaning task. In this task, we are given a noisy training dataset whose labels are corrupted by noise and a clean validation set. Then we aim to find weights for training samples such that a model that is learned over the weighted training set performs well on the validation set. This is a federated bilevel problem when the noisy training set is distributed over multiple clients. The formulation of the task is included in Appendix B.1. This task is a specialization of Eq. (1).

Dataset and Baselines. We create 10 clients and construct datasets based on MNIST [31]. For the training set, each client randomly samples 4500 images (no overlap among clients) from 10 classes and then randomly uniformly perturb the labels of ρ ( 0 ≤ ρ ≤ 1 ) percent samples. For the validation set, each client randomly selects 50 clean images from a different class. In other words, the m th client only has validation samples from the m th class. This single-class validation setting introduces a high level of heterogeneity, such that individual clients are unable to conduct local cleaning due to they only have clean samples from one class. In our experiments, we test our FedBiOAcc algorithm, including the FedBiO algorithm (Algorithm 2 in Appendix) which does not use variance reduction; additionally, we also consider some baseline methods: a baseline that directly performs FedAvg [40] on the noisy dataset, this helps to verify the usefulness of data cleaning; Local-BSGVR [12], FedNest [49], CommFedBiO [33], AggITD [53] and FedMBO [21]. Note that Local-BSGVR is designed for the homogeneous setting, and the last four baselines all need multiple rounds of client-server communication to evaluate the hyper-gradient at each global epoch. We perform grid search to find the best hyper-parameters for each method and report the best results. Specific choices are included in Appendix B.1.

In figure 1, we compare the performance of different methods at various noise levels ρ . Note that the larger the ρ value, the more noisy the training data are. The noise level can be illustrated by the performance of the FedAvg algorithm, which learns over the noisy data directly. As shown in the figure, FedAvg learns almost nothing when ρ = 0 . 95 . Next, our algorithms are robust under various heterogeneity levels. When the noise level in the training set increases as the value of ρ increases, learning relies more on the signal from the heterogeneous validation set, and our algorithms consistently outperform other baselines. Finally, in figure 2, we vary the number of clients sampled per epoch, and the experimental results show that our FedBiOAcc converges faster with more clients in the training per epoch; in figure 3, we vary the number of local steps under different noisy levels. Interestingly, the algorithm benefit more from the local training under larger noise.

400

1 = 2

1 = 2

1 = 2

IIE:

1 = 2

1 = 5

1 = 5

1 = 5

1 = 5

2.0

1 = 10

2.0

1 = 10

2.0

1 = 10

2.2

1 = 10

Validation Loss

1 = 20

Validation Loss

1 = 20

Validation Loss

1 = 20

SS07

1 = 20

1.5

1.5

1.5

Validation

1.9

1.0

1.0

1.0

1.6

0.5

0.5

0.5

1.3

0.0

100

200

000

400

0.0

0

00T

200

000

400

0.0

0

100

300

400

1.0

0

100

200

300

Communication Rounds

Communication Rounds

Communication Rounds

Communication Rounds

Validation Loss

0.9

1.2

DistBiOAcc

FedBiO_Local

FedBiOAcc_Local

DistBiO

Validation Loss

0.9

FedBiO_Local

FedBiOAcc_Local

DistBiOAcC

DistBiO

Validation Loss

2.7

2.2

DistBiOAcc

FedBiO_Local

DistBiO

FedBiOAcc_Local

Validation Loss

1.9

2.5

FedBio_Local

FedBiOAcc_Local

DistBiOAcc

DistBiO

0.6

0.6

1.7

1.3

0.3

0.3

1.2

0.7

0.0

0

Communication Rounds

100

200

300

400

0.0

0

Communication Rounds

100

200

300

00

0.7

0

Communication Rounds

100

200

300

400

0.1

0

Communication Rounds

100

200

00E

400

2.8

DistBiOAcc

DistBiO

2.4

DistBiO

DistBiOAcc

*DistBiO

DistBiOAcc

DistBiOAcc

DistBiO

Validation Loss

2.4

FedBiO_Local

FedBiOAcc_Local

Validation Loss

2.0

FedBiO_Local

FedBiOAcc_Local

Validation Loss

3.7

FedBiO_Local

FedBiOAcc_Local

Validation Loss

3.2

FedBiO_Local

FedBiOAcc_Local

2.0

1.6

3.4

3.1

山

2.9

2.6

1.6

1.2

2.8

1.2

0

250

500

750

1000

0.8

0

250

500

750

1000

2.5

0

250

500

750

1000

2.0

0

250

500

750

1000

Communication Rounds

Communication Rounds

Communication Rounds

Communication Rounds

5.2 Federated Hyper-Representation Learning

In the Hyper-representation learning task, we learn a hyper-representation of the data such that a linear classifier can be learned quickly with a small number of data samples. A mathematical formulation of the task is included in Appendix B.2. Note that this task is an instantiation of Eq. (10), due to the fact that each client has its own tasks, and thus only the upper level problem is federated. We consider the Omniglot [29] and MiniImageNet [44] data sets. As in the non-distributed setting, we perform N -wayK -shot classification.

In this experiment, we compare FedBiOAcc-Local (Algorithm 4 in the Appendix) with three baselines FedBiO-Local (Algorithm 3 in the Appendix), DistBiO and DistBiOAcc. Note that DistBiO and DistBiOAcc are the distributed version of FedBiO-Local and FedBiOAcc-Local, respectively. In the experiments, we implement DistBiO and DistBiOAcc by setting the local steps as 1 for FedBiO-Local and FedBiOAcc-Local. We perform grid search for the hyper-parameter selection for both methods and choose the best ones, the specific choices of hyper-parameters are deferred to Appendix B.2. The results are summarized in Figure 4 (full results are included in Figure 5 and Figure 6 of Appendix. As shown by the results, FedBiOAcc converges faster than the baselines on both datasets and on all four types of classification tasks, which demonstrates the effectiveness of variance reduction and multiple steps of local training.

6 Conclusion

In this paper, we study the Federated Bilevel Optimization problems and introduce FedBiOAcc. In particular, FedBiOAcc evaluates the hyper-gradient by solving a federated quadratic problem, and mitigates the noise through momentum-based variance reduction technique. We provide a rigorous convergence analysis for our proposed method and show that FedBiOAcc has the optimal iteration complexity O ( ϵ -1 . 5 ) and communication complexity O ( ϵ -1 ) , and it also achieves linear speed-up w.r.t the number of clients. Besides, we study a type of novel Federated Bilevel Optimization problems with local lower level problems. We modify FedBiO for this type of problems and propose

FedBiOAcc-Local. FedBiOAcc-Local achieves the same optimal convergence rate as FedBiOAcc. Finally, we validate our algorithms with real-world tasks.

References

A. K. R. Bayoumi, K. Mishchenko, and P. Richtarik. Tighter theory for local sgd on identical and heterogeneous data. In International Conference on Artificial Intelligence and Statistics , pages 4519-4529, 2020.

Y. Bengio. Gradient-based optimization of hyperparameters. Neural computation , 12(8):18891900, 2000.

D. Chen and M. T. Hagan. Optimal use of regularization and cross-validation in neural network modeling. In IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No. 99CH36339) , volume 2, pages 1275-1280. IEEE, 1999.

T. Chen, Y. Sun, and W. Yin. A single-timescale stochastic bilevel optimization method. arXiv preprint arXiv:2102.04671 , 2021.

X. Chen, M. Huang, and S. Ma. Decentralized bilevel optimization. arXiv preprint arXiv:2206.05670 , 2022.

A. Cutkosky and F. Orabona. Momentum-based variance reduction in non-convex sgd. In Advances in Neural Information Processing Systems , pages 15236-15245, 2019.

M. Dagréou, P. Ablin, S. Vaiter, and T. Moreau. A framework for bilevel optimization that enables stochastic and global variance reduction algorithms. arXiv preprint arXiv:2201.13409 , 2022.

C. B. Do, C.-S. Foo, and A. Y. Ng. Efficient multiple hyperparameter learning for log-linear models. In NIPS , volume 2007, pages 377-384. Citeseer, 2007.

J. Domke. Generic methods for optimization-based modeling. In Artificial Intelligence and Statistics , pages 318-326. PMLR, 2012.

M. C. Ferris and O. L. Mangasarian. Finite perturbation of convex programs. Applied Mathematics and Optimization , 23(1):263-273, 1991.

L. Franceschi, M. Donini, P. Frasconi, and M. Pontil. Forward and reverse gradient-based hyperparameter optimization. In Proceedings of the 34th International Conference on Machine Learning-Volume 70 , pages 1165-1173. JMLR. org, 2017.

H. Gao. On the convergence of momentum-based algorithms for federated stochastic bilevel optimization problems. arXiv preprint arXiv:2204.13299 , 2022.

H. Gao, B. Gu, and M. T. Thai. On the convergence of distributed stochastic bilevel optimization algorithms over a network. In International Conference on Artificial Intelligence and Statistics , pages 9238-9281. PMLR, 2023.

S. Ghadimi and M. Wang. Approximation methods for bilevel programming. arXiv preprint arXiv:1802.02246 , 2018.

Z. Guo, Q. Hu, L. Zhang, and T. Yang. Randomized stochastic variance-reduced methods for multi-task stochastic bilevel optimization. arXiv preprint arXiv:2105.02266 , 2021.

F. Haddadpour and M. Mahdavi. On the convergence of local descent methods in federated learning. arXiv preprint arXiv:1910.14425 , 2019.

M. Hong, H.-T. Wai, Z. Wang, and Z. Yang. A two-timescale framework for bilevel optimization: Complexity analysis and application to actor-critic. arXiv preprint arXiv:2007.05170 , 2020.

F. Huang. On momentum-based gradient methods for bilevel optimization with nonconvex lower-level. arXiv preprint arXiv:2303.03944 , 2023.

F. Huang and H. Huang. Biadam: Fast adaptive bilevel optimization methods. arXiv preprint arXiv:2106.11396 , 2021.

F. Huang, J. Li, S. Gao, and H. Huang. Enhanced bilevel optimization via bregman distance. Advances in Neural Information Processing Systems , 35:28928-28939, 2022.

M. Huang, D. Zhang, and K. Ji. Achieving linear speedup in non-iid federated bilevel learning. arXiv preprint arXiv:2302.05412 , 2023.

K. Ji and Y. Liang. Lower bounds and accelerated algorithms for bilevel optimization. arXiv preprint arXiv:2102.03926 , 2021.

K. Ji, J. Yang, and Y . Liang. Provably faster algorithms for bilevel optimization and applications to meta-learning. arXiv preprint arXiv:2010.07962 , 2020.

K. Ji, J. Yang, and Y . Liang. Bilevel optimization: Convergence analysis and enhanced design. In International conference on machine learning , pages 4882-4892. PMLR, 2021.

Y. Jiao, K. Yang, T. Wu, D. Song, and C. Jian. Asynchronous distributed bilevel optimization. arXiv preprint arXiv:2212.10048 , 2022.

S. P. Karimireddy, S. Kale, M. Mohri, S. J. Reddi, S. U. Stich, and A. T. Suresh. Scaffold: Stochastic controlled averaging for on-device federated learning. arXiv preprint arXiv:1910.06378 , 2019.

P. Khanduri, P. Sharma, H. Yang, M. Hong, J. Liu, K. Rajawat, and P. Varshney. Stem: A stochastic two-sided momentum algorithm achieving near-optimal sample and communication complexities for federated learning. Advances in Neural Information Processing Systems , 34:6050-6061, 2021.

P. Khanduri, S. Zeng, M. Hong, H.-T. Wai, Z. Wang, and Z. Yang. A near-optimal algorithm for stochastic bilevel optimization via double-momentum. arXiv preprint arXiv:2102.07367 , 2021.

B. Lake, R. Salakhutdinov, J. Gross, and J. Tenenbaum. One shot learning of simple visual concepts. In Proceedings of the annual meeting of the cognitive science society , volume 33, 2011.

J. Larsen, L. K. Hansen, C. Svarer, and M. Ohlsson. Design and regularization of neural networks: the optimal use of a validation set. In Neural Networks for Signal Processing VI. Proceedings of the 1996 IEEE Signal Processing Society Workshop , pages 62-71. IEEE, 1996.

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):2278-2324, 1998.

J. Li, B. Gu, and H. Huang. A fully single loop algorithm for bilevel optimization without hessian inverse. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 7426-7434, 2022.

J. Li, J. Pei, and H. Huang. Communication-efficient robust federated learning with noisy labels. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 914-924, 2022.

T. Li, S. Hu, A. Beirami, and V. Smith. Ditto: Fair and robust federated learning through personalization. In International Conference on Machine Learning , pages 6357-6368. PMLR, 2021.

X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang. On the convergence of fedavg on non-iid data. arXiv preprint arXiv:1907.02189 , 2019.

H. Liu, K. Simonyan, and Y. Yang. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055 , 2018.

J. Lorraine, P. Vicol, and D. Duvenaud. Optimizing millions of hyperparameters by implicit differentiation. In International Conference on Artificial Intelligence and Statistics , pages 1540-1552. PMLR, 2020.

S. Lu, S. Zeng, X. Cui, M. Squillante, L. Horesh, B. Kingsbury, J. Liu, and M. Hong. A stochastic linearized augmented lagrangian method for decentralized bilevel optimization. Advances in Neural Information Processing Systems , 35:30638-30650, 2022.

D. Maclaurin, D. Duvenaud, and R. Adams. Gradient-based hyperparameter optimization through reversible learning. In International Conference on Machine Learning , pages 21132122, 2015.

B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics , pages 1273-1282. PMLR, 2017.

M. Mohri, G. Sivek, and A. T. Suresh. Agnostic federated learning. arXiv preprint arXiv:1902.00146 , 2019.

T. Okuno, A. Takeda, and A. Kawana. Hyperparameter learning via bilevel nonsmooth optimization. arXiv preprint arXiv:1806.01520 , 2018.

F. Pedregosa. Hyperparameter optimization with approximate gradient. arXiv preprint arXiv:1602.02355 , 2016.

S. Ravi and H. Larochelle. Optimization as a model for few-shot learning. In International conference on learning representations , 2017.

S. Sabach and S. Shtern. A first order method for solving convex bilevel optimization problems. SIAM Journal on Optimization , 27(2):640-660, 2017.

A. K. Sahu, T. Li, M. Sanjabi, M. Zaheer, A. Talwalkar, and V. Smith. On the convergence of federated optimization in heterogeneous networks. arXiv preprint arXiv:1812.06127 , 3, 2018.

A. Shaban, C.-A. Cheng, N. Hatch, and B. Boots. Truncated back-propagation for bilevel optimization. arXiv preprint arXiv:1810.10667 , 2018.

M. Solodov. An explicit descent method for bilevel convex optimization. Journal of Convex Analysis , 14(2):227, 2007.

D. A. Tarzanagh, M. Li, C. Thrampoulidis, and S. Oymak. Fednest: Federated bilevel, minimax, and compositional optimization. arXiv preprint arXiv:2205.02215 , 2022.

S. Wang, T. Tuor, T. Salonidis, K. K. Leung, C. Makaya, T. He, and K. Chan. Adaptive federated learning in resource constrained edge computing systems. IEEE Journal on Selected Areas in Communications , 37(6):1205-1221, 2019.

R. A. Willoughby. Solutions of ill-posed problems (an tikhonov and vy arsenin). SIAM Review , 21(2):266, 1979.

B. Woodworth. The minimax complexity of distributed optimization. arXiv preprint arXiv:2109.00534 , 2021.

P. Xiao and K. Ji. Communication-efficient federated hypergradient computation via aggregated iterative differentiation. arXiv preprint arXiv:2302.04969 , 2023.

P. Xing, S. Lu, L. Wu, and H. Yu. Big-fed: Bilevel optimization enhanced graph-aided federated learning.

I. Yamada, M. Yukawa, and M. Yamagishi. Minimizing the moreau envelope of nonsmooth convex functions over the fixed point set of certain quasi-nonexpansive mappings. In Fixed-Point Algorithms for Inverse Problems in Science and Engineering , pages 345-390. Springer, 2011.

J. Yang, K. Ji, and Y . Liang. Provably faster algorithms for bilevel optimization. arXiv preprint arXiv:2106.04692 , 2021.

S. Yang, X. Zhang, and M. Wang. Decentralized gossip-based stochastic bilevel optimization over communication networks. arXiv preprint arXiv:2206.10870 , 2022.

Y. Yang, P. Xiao, and K. Ji. Simfbo: Towards simple, flexible and communication-efficient federated bilevel learning. arXiv preprint arXiv:2305.19442 , 2023.

H. Yu, S. Yang, and S. Zhu. Parallel restarted sgd with faster convergence and less communication: Demystifying why model averaging works for deep learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, pages 5693-5700, 2019.

Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra. Federated learning with non-iid data. arXiv preprint arXiv:1806.00582 , 2018.

L. Zintgraf, K. Shiarli, V. Kurin, K. Hofmann, and S. Whiteson. Fast context adaptation via meta-learning. In International Conference on Machine Learning , pages 7693-7702. PMLR, 2019.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studied the federated bilevel optimization problems, in which the upper level losses are non convex and lover level losses are strongly convex. Two algorithms, named as FedBiOAcc and FedBiOAcc-Local,  were proposed for whether the lower level problems requires to be consensus or not. The authors provided both communication can computation complexity bounds for the proposed algorithms. Two real-world tasks are adopted to validate the theoretical results.\n\nSTRENGTHS:\nThe authors studied the bilevel optimization problems under federated learning framework and provided sufficient research motivation and literature review. They proposed recovering the hyper-gradient via solving a federated optimization problem, which is novel.\n\nWEAKNESSES:\nThere have been many existing works on federated/distributed bilevel optimization. The contribution of this paper might not be significant. Also, the numerical comparison needs to be more fair.\n\nQUESTIONS:\n* It is a good idea to estimate the hyper-gradient via solving the federated quadric problem. \n   * However, during the updating iteration, both $x_t$ and $y_t$ will be changing, and thus $l$. It leaves me concern that whether $u_t$ would converge. \n   * Also how would the initialization of $u_t$ affect the algorithm convergence? \n   * in line 161, would $u^*$ depend on $t$? \n   * BTW, would you remind me the meaning of subscript of $2$ in $\\nabla l^{(m)}$ in line 149? It looks like the definition is missing.\n\n* The paper leveraged the STORM type of variance reduction to accelerate the convergence. However, in the original STORM [1], as well as its variant SUSTAIN [2], the initial batch size is a fixed number. Why does it need $O(M^{-1}\\epsilon^{-0.5})$ initial batch size?\n\n* The numerical studies looks somehow unfair to me.\n  *  In terms of communication, most of the other algorithms only transmit two vectors $x_t$ and $y_t$ but FedBiOAcc needs three vectors $x_t$, $y_t$ and $u_t$. Instead of looking at communication rounds, would the communicated bytes be a more fair metric to measure the communication loads? \n  * The authors also didn't report the computation cost. Rather than the simple convergence rate (which omit lots of key factors), the computation time / number of gradient/Hessian computation would be a better metric?\n  * BTW, what is the FedBio? It doesn't have any description. What's the different between FedBio and FedBioAcc?\n\nSome minor questions\n* Why doesn't the paper also include local-upper-shared-lower level problems?\n* Would the results or algorithms be extended to the case with strongly convex / convex upper level problem?\n\n[1] Momentum-based variance reduction in non-convex sgd\n[2] A Near-Optimal Algorithm for Stochastic Bilevel Optimization via Double-Momentum\n\nLIMITATIONS:\nThe authors didn't mention any limitation and potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an algorithm named FedBiOAcc that solves the federated bilevel optimization problem. In particular, the hyper-gradient of the bilevel optimization problem is evaluated by solving a reformulated quadratic federated optimization problem. The upper-level problem, lower-level problem and the hyper-gradient estimation are optimized in an alternative manner. A rigorous convergence analysis of the algorithm is provided. Also, the paper novelly introduces a new problem of bilevel optimization with local lower-level problems and proves the same convergence rate for a FedBiOAcc-Local algorithm that adapts to this setting.\n\nSTRENGTHS:\n1. Reformulation of the general federated bilevel optimization into an easier-to-solve federated quadratic optimization problem is interesting and sound.\n2. The convergence analysis of both algorithms in the heterogeneous case is complex and sound.\n3. The paper introduces a new problem setting with local lower-level problems, which could be even more challenging to solve. The algorithm and analysis are adapted to this new setting.\n4. The formulations of the data cleaning task and the hyper-representation learning task into federated bilevel optimization problems are reasonable.\n\nWEAKNESSES:\n1. The motivation of the proposed problem can be better instantiated with actual needs or examples in real-life.\n\nQUESTIONS:\n1. While bilevel federated optimization is technically challenging, what is the motivation to solve bilevel problems in a federated manner? There might need more concrete motivations and a clear, practical application or usage. Is there a need for distributed computing resources in such problems? Is there a requirement to ensure data privacy for bilevel optimization practical problems?\n2. The convergence analysis of the paper claims linear convergence speed-ups with the number of clients $M$. Does the convergence always become faster with the number of clients? For example, it is possible that the clients are heterogeneous. Also, does the dataset size of each client affect the convergence? It is hard to believe that given a fixed total number of samples, distributing them among more clients makes learning easier/faster.\n3. [Minor] Some notations used in the paper were not explicitly explained before the first appearance. For example, the $\\tau$ and $\\mathcal{B}$ in Line 149. Also, what do the subscripts mean for $\\mathcal{B}$?\n\nLIMITATIONS:\nI do not find any particular limitations in the proposed method.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the problem of bilevel optimization in a federated learning or FL environment. Bilevel optimization has various FL applications, and a few recent works proposed versions of bilevel optimization schemes for FL. A challenging step in bilevel optimization is the computation of the \"hypergradient\", and the existing schemes are able to obtain an estimate of the hypergradient, albeit a biased estimate, with a substantial communication overhead (multiple rounds of communication per server-side update). The paper reformulates the hypergradient computation as a quadratic federated optimization problem that can provide an unbiased estimate of the hypergradient, while requiring only a single round of communication for every server-side update. Based on this insight, the paper proposes a federated bilevel optimization algorithm that updates the two variables in the bilevel problem, and the hypergradient estimate alternately on each client, with an intermittent averaging step across all clients. To improve the convergence with a constant batch size, the paper utilizes momentum based variance reduction techniques for all the three variables. The theoretical analysis establishes convergence with an improved iteration complexity compared to existing federated bilevel schemes, with a significantly improved communication complexity. Next, the paper studies an alternate form of a federated bilevel problem where all clients have local lower level variables, with applications in personalized FL. This setup allows for the local hypergradient computation without any need for communication across clients. The paper presents a modification of the previous algorithm for this problem and establishes convergence. The empirical evaluation considers two applications and compares the proposed scheme against various baselines, highlighting the improved communication complexity of the proposed scheme across various problem settings.\n\nSTRENGTHS:\n**Critical federated hypergradient estimation as a federated optimization.**\nA key strength of this paper is a simple (yet of significant practical impact) reformulation of the hypergradient estimation using a standard quadratic program. A key property that the authors leverage is the fact that the global quadratic objective can be decomposed into per-client quadratic objectives, which is not true of the global hypergradient (which cannot be decomposed into per-client hypergradients). This simple yet powerful insight is then utilized to obtain an estimate which, upon proper solution of the global least-squares problem, is unbiased, and can be efficiently updated along side the upper and lower level variables in the bilevel problem. While this global least-squares reformulation does facilitate an intuitive communication-efficient algorithm, the paper also proposes a momentum-based variance reduction scheme, and establishes a very favorable convergence rate both in terms of iterations (and thus sample complexity) and communication. The overall algorithm makes the solution of federated bilevel problems significantly more practical.\n\n**Favorable convergence rate.**\nAlong with the novel federated hypergradient estimation, this paper is also able to get a favorable convergence rate across various dimensions. First, the results are able to provide convergence with a constant batch size, and establish a sample complexity of $O(\\epsilon^{-1.5})$, which is very strong for bilevel optimization. Second, the analysis establishes a $(1/M)$-dependence on the sample complexity, indicating that having more parties in the FL setup allows for smaller per-client sample complexity. Finally, as I discussed earlier, the analysis establishes a $O(\\epsilon^{-1})$ communication complexity which matches the rate of the best single-level federated optimization algorithm.\n\n\n**Bilevel personalized FL.**\nThe paper explicitly studies a form of the federated bilevel optimization where each client has its own lower level problem, and I believe this general problem would encompass various personalized FL problems and applications. While the problem is easier in terms of the computation of the hypergradient, it is still an interesting problem, and the paper establishes convergence again with favorable iteration and communication complexities.\n\n\n**Positioning against existing literature.**\nIn addition to the section on related works, the authors do a great job at discussing their proposed scheme in comparison to existing literature during the algorithm development. For example, after presenting the main idea behind the communication efficient hypergradient computation, the paper positions this work in comparison to FedNest.\n\n**Intuitive presentation of the theoretical results.**\nIn addition to establishing favorable theoretical guarantees, the authors also present the theoretical results in an intuitive manner. First, they clearly present the Lyapunov function, highlighting the different terms in it corresponding to the different variable iterates. Then the relevant parts of the Appendix are referenced which makes it easy to verify the results while reading. After the main theorem is presented, the authors also do a good job of highlighting what it implies, and how it compares to other federated algorithms.\n\nWEAKNESSES:\n**Increased hyperparameter space.**\nThe proposed framework utilizes various hyperparameters such as the initial learning rates $\\gamma, \\eta, \\tau$, the learning rate schedule $\\lbrace \\alpha_t, t \\in [T] \\rbrace$, the momentum parameters $c_{\\omega / \\nu / u}$, the aggregation frequency $I$.\nAs per the theoretical analyses, it can be seen that the best convergence rate of any execution will critically depend on an appropriate setting of these problem-dependent hyperparameters. Since these hyperparameters often depend on quantities that cannot be efficiently estimated (such as Lipschitz constant), the practical bilevel implementations usually utilize some form of hyperparameter search. Hyperparameter optimization is known to be a hard unsolved problem in FL because of the overall communication overhead. This makes it hard to see how the proposed federated bilevel framework can live up to its practical potential -- one can view this proposed federated bilevel framework as having shifted the communication overhead from the model training stage to the hyperparameter optimization stage, without reducing the overall communication necessary for good training convergence (which involves trying various hyperparameters and training with them).\n\nQUESTIONS:\n- Line 2 in algorithm 1: Should it be $q_1^{(m)}$ since it is computed using $g^{(m)}$ and $f^{(m)}$?\n- Line 176-177: The ordering of the momentum terms and the corresponding variables in the \"... respectively\" needs to be updated.\n- In theorem 1, the results provide precise dependence on $\\kappa$, but I am unable to find the definition of $\\kappa$ in the theorem statement or anywhere else in the main paper. Can you please clarify what this $\\kappa$ is supposed to signify?\n\nLIMITATIONS:\nI did not find any discussion on limitations in the main paper or the supplement. However, I do not anticipate any potential negative societal impact of this work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
this paper proposed new algorithms for federated bilevel optimization. The authors apply the idea of [1] to the federated setting, which views the hypergradient computation as solving a quadratic subproblem. Combining with the STORM algorithm, the author proposed fedbioacc with a convergence guarantee based on this idea and then extend the results to a simpler case with local lower-level problems. numerical experiments on data cleaning and hyper representation are reported.

[1] Dagréou et al. "A framework for bilevel optimization that enables stochastic and global variance reduction algorithms"

STRENGTHS:
1. the proposed algorithms are pure single-loop algorithms compared with other global federated bilevel algorithms.
2. the authors provide convergence guarantees for both proposed algorithms.

WEAKNESSES:
1. One of my primary concerns is that single-loop algorithms in bilevel optimization perform worse than double-loop algorithms because of the hyper gradient estimation error introduced by the inexactness of the lower-level solution. this is different from the minimax problem because the hypergradient in bilevel optimization involves Jacobian and hessian inverse computation, thus being more complex. In practice, people are using a double-loop algorithm to reduce this hyper gradient estimation error.

2. the contribution of this paper looks trivial to me, the main ideas come from STORM and [Dagréou et al]. the proposed algorithm is very complex and thus difficult to implement and tune in practice.

QUESTIONS:
see weakness

LIMITATIONS:
see weakness
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed a communication efficient federated bilevel algorithm in both global and local lower-level problem setting that can achieve the best theoretical communication complexity. Extensive numerical experiments are provided to test the effectiveness of the proposed algorithm.\n\nSTRENGTHS:\nThey provide convergence analysis for the proposed algorithm. The numerical experiments are abundant and solid to verify the effectiveness of the proposed algorithm. Also, the federated bilevel with local lower-level problem is intriguing.\n\nWEAKNESSES:\n1.\tThe analysis technique for bilevel method that treats the Hessian-vector product as a solution of a quadratic problem was also utilized in [R1]-[R5]. I’m afraid this would alleviate some novelty of this paper. Also, some relevant literature on fully single loop bilevel algorithms like [R1]-[R2] and communication efficient federated bilevel methods [R4]-[R5] are missing. Especially, [R4] allows for any heterogeneity without Assumption 3.5 and [R5] achieves the same communication complexity as this work, could you elaborate the pros and cons of your work and [R4]-[R5]?  \n\n\n[R1] Junyi Li, Bin Gu, and Heng Huang. A fully single loop algorithm for bilevel optimization without hessian inverse. In Proceedings of the AAAI Conference on Artificial Intelligence, 2022.\n\n[R2] Michael Arbel and Julien Mairal. Amortized implicit differentiation for stochastic bilevel optimization. arXiv\npreprint arXiv:2111.14580, 2021.\n\n[R3] Mathieu Dagreou, Pierre Ablin, Samuel Vaiter, and Thomas Moreau. A framework for bilevel optimization that enables stochastic and global variance reduction algorithms. In Proceedings of the Advances in Neural Information Processing Systems, 2022.\n\n[R4] Quan Xiao, Han Shen, Wotao Yin, and Tianyi Chen. \"Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimization.\" arXiv preprint arXiv:2211.07096, 2022.\n\n[R5] Yifan Yang, Peiyao Xiao, and Kaiyi Ji. \"SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learning.\" arXiv preprint arXiv:2305.19442, 2023.\n\n2.\tThe role of heterogeneity is not expressly delineated in Theorem 3.6. It would be beneficial to see a how heterogeneity influences the convergence rate.\n\nQUESTIONS:\nThe three-level optimization perspective appears to be applicable to the local lower-level bilevel problem as well. Is there a specific reason for selecting the Neumann series approximation for the local lower-level problem, apart from the warm start strategy?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}