Title: First-Order Minimax Bilevel Optimization
Venue: NeurIPS 2024
Accepted: yes
Keywords: Bilevel Optimization

Machine Learning

Meta Learning
Abstract: Multi-block minimax bilevel optimization has been studied recently due to its great potential in multi-task learning, robust machine learning, and few-shot learning. However, due to the complex three-level optimization structure, existing algorithms often suffer from issues such as high computing costs due to the second-order model derivatives or high memory consumption in storing all blocks' parameters. In this paper, we tackle these challenges by proposing two novel fully first-order algorithms named FOSL and MemCS. FOSL features a fully single-loop structure by updating all three variables simultaneously, and MemCS is a memory-efficient double-loop algorithm with cold-start initialization. We provide a comprehensive convergence analysis for both algorithms under full and partial block participation, and show that their sample complexities match or outperform those of the same type of methods in standard bilevel optimization. We evaluate our methods in two applications: the recently proposed multi-task deep AUC maximization and a novel rank-based robust meta-learning. Our methods consistently improve over existing methods with better performance over various datasets.

First-Order Minimax Bilevel Optimization

Abstract

Multi-block minimax bilevel optimization has been studied recently due to its great potential in multi-task learning, robust machine learning, and few-shot learning. However, due to the complex three-level optimization structure, existing algorithms often suffer from issues such as high computing costs due to the second-order model derivatives or high memory consumption in storing all blocks' parameters. In this paper, we tackle these challenges by proposing two novel fully first-order algorithms named FOSL and MemCS. FOSL features a fully single-loop structure by updating all three variables simultaneously, and MemCS is a memory-efficient double-loop algorithm with cold-start initialization. We provide a comprehensive convergence analysis for both algorithms under full and partial block participation, and show that their sample complexities match or outperform those of the same type of methods in standard bilevel optimization. We evaluate our methods in two applications: the recently proposed multi-task deep AUC maximization and a novel rank-based robust meta-learning. Our methods consistently improve over existing methods with better performance over various datasets.

1 Introduction

In this paper, we study a general multi-block minimax bilevel optimization problem given by

where the upper- and lower-level function f i and g i for block i take the expectation form w.r.t. the random variables ξ , ζ , and are jointly continuously differentiable, z ∗ = ( z ∗ 1 ( x ) , ..., z ∗ n ( x ) ) ∈ R d z × n contains all lower-level optimal solutions, and n is the number of blocks. The above problem has various applications in machine learning, including deep AUC maximization [24, 23], meta-learning [16, 3], hyperparameter optimization [17], and robust learning [17]. This paper focuses on the setting with a nonconvex-strongly-concave minimax upper-level problem and a strongly-convex lower-level problem.

To date, barring a few works on optimizing special cases of this problem [17, 24], the solution algorithm to its general form has not been well studied. The primary obstacle lies in the significant computational cost per iteration, arising from the inherent structure of multi-block minimax bilevel optimization. To address this challenge, [17] considered a special case where y is the simplex variable, and introduced a single-loop gradient descent-ascent algorithm, based on the two-timescale bilevel

framework in [22]. [24] proposed a single-loop matrix-vector-based algorithm for a special case of our problem, where each upper-level function f i is evaluated only at the i th coordinate of y . However, these methods require computing the expensive second-order derivatives (i.e., the Hessian matrix or Hessian-vector product) per iteration, and the more efficient first-order approaches have not been explored yet. In this paper, we propose two efficient first-order minimax bilevel algorithms and further apply them to two novel ML applications. Our contributions are summarized as follows.

By converting the original minimax bilevel problem into a simple minimax problem, we first propose a fully first-order single-loop algorithm named FOSL, which is easy to implement by updating x, y and z simultaneously, and is computationally efficient without the calculation of any second-order Hessian or Jacobian matrices. We provide a convergence analysis for FOSL under a practical block sampling without replacement setting and show that its sample complexity matches the best-known result of the same type of methods in standard bilevel optimization. Technically, we characterize the gap between the reformulated and original problems and need to deal with the interplay among four variables in the error analysis.

In the settings where the number of blocks is substantial (e.g., in few-shot meta-learning), it becomes impractical to store all block-specific parameters to perform the single-loop optimization. To this end, we also propose a memory-efficient method named MemCS via a cold-start initialization, which randomly initializes a new weight for each sampled block, without saving it for the next iteration. We then analyze the convergence of MemCS under the partial-block and full-block participation, and show that it can achieve a better sample complexity than the same type of methods in standard bilevel optimization.

We further apply our approaches to two ML applications: deep AUC maximization and robust meta-learning. The first application pertains to the established field of AUC Maximization, while the second explores a novel application known as Rank-based Robust MetaLearning. We show the effectiveness of our methods over a variety of datasets including CIFAR100, CelebA, CheXpert, OGBG-MolPCBA, Mini-ImageNet and Tiered-ImageNet.

2 Related Work

(Minimax) bilevel optimization. Bilevel optimization, introduced in [2], has been extensively studied, with constraint-based methods [13, 20, 50, 51] and gradient-based methods [1, 45, 14, 49, 57] emerging as two predominant types of approaches. The constraint-based methods (e.g., [34, 38, 30, 35, 54]) reformulated the lower-level problem as a value-function-based constraint, and solved it via different constrained optimization algorithms. More recently, [16, 23] studied the minimax bilevel optimization problem and proposed single-loop algorithms with applications to robust machine learning and deep AUC maximization. In this paper, we propose two efficient, fully first-order algorithms with solid performance guarantees. In recent years, there has been a growing interest in gradient-based methods due to their efficiency in solving machine-learning problems. Within this category, Iterative Differentiation (ITD) based methods [6, 7, 14, 39, 49, 27] and Approximate Implicit Differentiation (AID) based methods [1, 5, 33, 45, 41, 14, 27, 22] are two important classes distinguished by their approaches to approximating hypergradients.

Deep AUC maximization (DAM). DAMmethods are aimed to mitigate the impact of imbalanced data in binary classification by directly maximizing the area under the ROC curve (AUC), a performance metric less affected by imbalanced data. As the AUC is difficult to optimize directly, research on DAM primarily focuses on devising effective optimization methods for its continuous surrogates [21, 4, 46, 8]. [37] proposed to reformulate the deep AUC maximization problem as a minimax optimization problem, providing the foundation for stochastic DAM algorithms developed in recent years [59, 60, 18, 24]. Among them, the most relevant work [24] formulated the DAM problem as a multi-block minimax optimization problem. In this work, we will use this form of DAM to demonstrate the effectiveness of our algorithm. A more comprehensive overview of DAM methods can be found in the survey [56].

Robust meta-learning. Meta-learning provides effective solutions to multi-task learning in few-shot learning settings. In meta-learning, one trains a meta-model that can be quickly turned into a model that adapts to new tasks with only a few updates. Meta-learning algorithms in real-world applications must be robust to handle corrupted or low-quality data such as outliers. The majority of robust metalearning methods encompass filtering [55], re-weighting [48, 28, 31, 36], and re-labeling[43, 52, 61]

on the sample level. Moreover, some other works focus on improving task-level robustness [28, 58]. In this work, we show that robust meta-learning can be formulated as a minimax bilevel optimization problem and solved with the proposed algorithm.

3 Algorithms

3.1 Reformulation as a Minimax Problem

Motivated by [34, 38, 30] in single-machine bilevel optimization, we reformulate the lower-level problem as a value-function-based constraint and aim to solve the following equivalent problem:

where z ∗ i := arg min z g i ( x, z ) . Inspired by [30], we form a Lagrangian L i with Lagrangian multiplier λ ≥ 0 to approximate the original problem for each block i in eq. (2), as

where v i is used to approximate the lower-level solution z ∗ i ( x ) of the i th block. Then, we turn to solve the following surrogate minimax problem:

where z = ( z 1 , ..., z n ) ∈ R d z × n , v = ( v 1 , ..., v n ) ∈ R d z × n and the standard basis vector e i has only one non-zero element of 1 at the i th coordinate. We show later in Section 4.2 that the gap between the gradients ∇ F ( x, y ∗ ( x ) , z ∗ ( x )) and ∇L ( x, y ∗ ( x ) , z ∗ λ ( x ) , z ∗ ( x )) of the original and surrogate problems can be effectively bounded by O (1 /λ ) , where y ∗ ( x ) denotes the maximize of outer-objective F ( x, · , z ∗ ( x )) and each vector z ∗ λ,i ( x ) in z ∗ λ ( x ) := ( z ∗ λ, 1 ( x ) , ..., z ∗ λ,n ( x )) denotes the minimizer of the Lagrangian function L i ( x, y ∗ ( x ) , · , v ) (where z ∗ λ,i ( x ) has not reliance on v ). This validates the effectiveness of the Lagrangian approximation for λ sufficiently large. Next, we propose two efficient algorithms, namely FOSL and MemCS, to solve the surrogate problem in eq. (3).

3.2 FOSL: Fully First-Order Single-Loop Method

As shown in Algorithm 1, we first sample a subset I t ⊂ S := { 1 , ..., n } of blocks without replacement. Noting that z i and v i are both block-specific variables, we then apply a stochastic ascent and descent step to update v i and z i for all block i ∈ I t as

where the gradient of L i w.r.t. z has no dependence on v . Since the solutions w.r.t. variables x and y depend on all blocks, we use the average of stochastic gradient estimators from the selected blocks in I t to update y and x as

Note that our algorithm takes a simpler fully single-loop structure via updating { v i,t , z i,t } i ∈ I t , x t and y t simultaneously at each iteration. Hence, it can also benefit from the hardware parallelism. In addition, different from the methods in [17, 24] that need to compute the higher order Hessian- or Jacobian-vector products, our method only uses the first-order gradients.

3.3 MemCS: Memory-Efficient Cold-Start Method

Note that in the single-loop optimization of Algorithm 1, all block-specific parameters v i,t and z i,t of blocks in I t need to be stored for the updates at iteration t +1 . However, in some ML applications,

such as few-shot meta-learning, the number of blocks/tasks is often large, and hence Algorithm 1 can suffer from significant memory consumption. To address this challenge, we propose a memoryefficient method named MemCS in Algorithm 2. Differently from the single-loop updates in FOSL, MemCS contains a sub-loop of K steps of gradient descent 1 in updating the block-specific variables v k i,t and z k i,t for k = 0 , ..., K -1 with random initialization v 0 i,t and z 0 i,t . After obtaining the outputs v K i,t , z K i,t of this sub-loop, the remaining step is to update y t and x t via gradient ascent and descent similarly as in FOSL.

4 Main Results

4.1 Assumptions

Definition 4.1. A mapping f is L -Lipschitz continuous if ∥ f ( x 1 ) -f ( x 2 ) ∥ ≤ L ∥ x 1 -x 2 ∥ , for any x 1 , x 2 . We say f is L -smooth if ∇ f is L -Lipschitz continuous.

Since the overall objective is nonconvex w.r.t. x , we aim to find an ϵ -accurate stationary point.

Definition 4.2. We call ¯ x as an ϵ -accurate stationary point of the objective function Φ( x ) if E ∥∇ Φ(¯ x ) ∥ 2 ≤ ϵ , where ϵ ∈ (0 , 1] and ¯ x is the output of an algorithm.

We use the following assumptions in the subsequent description. Note that these assumptions are widely adopted in existing studies [24, 17].

Assumption 4.3. For any x ∈ R d x , y ∈ R d y , z ∈ R d z and i ∈ { 1 , 2 , ..., n } , f i ( x, y ) and g i ( x, y ) are twice continuously differentiable, f i ( x, y, z ) is µ f -strongly concave w.r.t. y and g i ( x, z ) is µ g -strongly convex w.r.t. z .

The following assumption imposes the Lipschitz continuity on the upper- and lower-level functions and their derivatives.

Assumption 4.4. For any x ∈ R d x , z ∈ R d z and i ∈ { 1 , 2 , ..., n } , f i ( x, y, z ) is L f, 0 -Lipschitz continuous w.r.t. x , g i ( x, z ) is L g, 0 -Lipschitz continuous w.r.t. x , f i ( x, y, z ) is L f, 1 -smooth and g i ( x, y ) is L g, 1 -smooth. In addition, the second-order derivatives ∇ 2 f i ( x, y, z ) and ∇ 2 g i ( x, y ) are L f, 2 - and L g, 2 -Lipschitz continuous.

Next, we make a bounded variance assumption for the gradients in the stochastic setting.

Assumption 4.5. There exist constants σ f and σ g such that the variances E ∥∇ f i ( x, y, z ) -∇ f i ( x, y, z ; ξ ) ∥ 2 ≤ σ 2 f and E ∥∇ g i ( x, z ) -∇ g i ( x, z ; ζ ) ∥ 2 ≤ σ 2 g .

The following assumption on block heterogeneity measures the similarities of the upper-level gradients ∇ y f i ( x, z ) for all i . This has not been discussed in previous works, but it is necessary for our approach as we explore a more general outer-maximization solution y ∗ ( x ) for F , rather than for single f i .

Assumption 4.6. For any x ∈ R d x , z ∈ R d z , there exist constants β th ≥ 1 and σ th ≥ 0 such that

We have β th = 1 and σ th = 0 when all g i 's are identical.

4.2 Convergence analysis

For simplicity, we fix the number of involved blocks | I t | = P for all t . Let y ∗ ( x ) be the maximizer of F function w.r.t. y . Then, the overall objective of the original problem in eq. (1) w.r.t. x is given by

where z ∗ ( x ) is the lower-level minimizer and y ∗ ( x ) is the maximizer of F ( x, · , z ∗ ( x )) . In addition, recall that the objective function of the surrogate problem in eq. (3) w.r.t. x is given by L ∗ ( x ) := L ( x, y ∗ ( x ) , z ∗ λ ( x ) , z ∗ ( x ) ) . We next characterize the difference between the gradients of the original and the surrogate problems.

Proposition 4.7. Under Assumptions 4.3, 4.4, the gap between ∇ Φ( x ) and ∇L ∗ ( x ) can be bounded as

Due to the limit of space, the proof of Proposition 4.7 can be found in Lemma D.5 in the appendix. For a properly large λ , Proposition 4.7 guarantees that the stationary points of the original and surrogate problems are close to each other. However, too large λ can explode the gradient estimation variance, resulting in a much slower convergence rate. This trade-off makes the selection of λ important, as shown in our theorems later.

We next give an upper bound on the gradient norm E ∥∇L ∗ ( x t ) ∥ 2 of the surrogate problem. Denote h t x := ∇ x L i ( x t , y t , z t , v t ; ξ t x,i ) and its expectation as ˜ h t x .

Proposition 4.8. Under Assumptions 4.3, 4.4, 4.5, the consecutive iterates of Algorithm 1 satisfy:

for all t ∈ { 0 , 1 , ..., T -1 } , where L λ, 1 and L ∗ , 1 are given in Lemma D.1 and Lemma D.6 in the appendix respectively.

The proof of Proposition 4.8 can be found in Lemma E.1 in the appendix. The same result can be obtained for Algorithm 2 by replacing v i,t and z i,t with v K i,t and z K i,t . This proposition shows that the convergence rate of our algorithm relies on how fast the iterates y t , z i,t and v i,t converge to their optimal solutions at each iteration t . We next characterize the distance of y t to its optimal solution.

Proposition 4.9. Under Assumptions 4.3, 4.4, 4.5, the iterates of y t generated according to Algorithm 1 satisfy

The proof of Proposition 4.9 refers to Lemma E.3 in the appendix. It can be seen that with properly small stepsizes η x and η y , there exists a descent of the optimal distance E ∥ y t -y ∗ ( x t ) ∥ 2 , which is critical for the final convergence analysis. Similar results are obtained for v i,t and z i,t . Combining the above propositions and the auxiliary lemmas in the appendix, we get the following result for Algorithm 1.

Theorem 4.10 (Convergence of FOSL) . Suppose Assumptions 4.3, 4.4, 4.5, 4.6 are satisfied. Set parameters η x = O ( T -5 7 ) , η y = O ( T -4 7 ) , η z = O ( T -5 7 ) , η v = O ( T -4 7 ) and λ = O ( T 1 7 ) . Then, we have

where C gap is defined in Lemma D.5, C 2 , C 3 are defined in eq. (39) in the appendix, and Ψ t := L ∗ ( x t ) + K y E ∥ y t -y ∗ ( x t ) ∥ 2 + K z 1 n ∑ n i =1 E ∥ z i,t -z ∗ λ,i ( x t ) ∥ 2 + K v 1 n ∑ n i =1 E ∥ v i,t -z ∗ i ( x t ) ∥ 2 .

Next, we characterize the sample complexity for FOSL.

Corollary 4.11. Under the same setting of Theorem 4.10, our algorithm finds an ϵ -accurate stationary solution after T = O ( ϵ -7 2 ) interactions. The total sample complexity for all involved blocks is PT = O ( Pϵ -7 2 ) .

Compared with existing works [17, 24], our algorithm is free from second-order derivative computations. In addition, the sample complexity of our algorithm matches the best result [30] of the same type of methods in standard single-block bilevel optimization.

Next, we analyze the convergence for Algorithm 2 under the partial- and full-block participation.

Theorem 4.12 (Convergence of MemCS) . Suppose Assumptions 4.3, 4.4, 4.5, 4.6 are satisfied. Assume there exists some B > 0 such that ∥ z ∗ i ( x t ) ∥ ≤ B for any x t , i = 1 , ..., N . For the partial-block participation, by setting parameters η x = O ( P 1 5 T -2 3 ) , η y = O ( P -1 5 T -1 2 ) , η z = O ( P -1 10 T -1 6 ) , η v = O (1) , λ = O ( P 1 10 T 1 6 ) and taking ϵ sub = O ( P -2 5 T -2 3 ) , we have

where L λ, 1 := 3 λL g, 1 , C gap is defined by Lemma D.5 in the appendix and Ψ t := L ∗ ( x t )+ K y E ∥ y t -y ∗ ( x t ) ∥ 2 . For the full-block participation, by setting η x = O (1) , η y = O (1) , η z = O ( T -1 2 ) , η v = O (1) , λ = O ( T 1 2 ) and taking ϵ sub = O ( T -2 ) , we have

Note that the assumption ∥ z ∗ i ( x t ) ∥ ≤ B can be removed when the domain of x is a closed convex set and projected gradient descent is used to update x [12, 15]. We next characterize the sample complexity for MemCS.

Corollary 4.13. Under the same setting of Theorem 4.12,

For partial-block participation, our algorithm finds an ϵ -accurate stationary solution of Φ( x ) after T = O ( P -3 5 ϵ -3 ) outer iterations and K = O (log 1 ϵ ) inner iterations. The total sample complexity for all involved blocks is PKT = ˜ O ( P 2 5 ϵ -3 ) .

For full-block participation, our algorithm finds an ϵ -accurate stationary solution of Φ( x ) after T = O ( ϵ -1 ) outer iterations and K = O (log 1 ϵ ) inner iterations. The total sample complexity for all involved blocks is nKT = ˜ O ( nϵ -1 ) .

Note that the per-block sample complexity of our MemCS algorithm takes an order of ϵ -1 , which improves that of the same-type F 2 SA [30] by an order of ϵ -0 . 5 , based on a refined analysis on the smoothness of the overall objective function. Corollary 4.13 also shows that MemCS achieves a linear convergence speedup w.r.t. the number P of blocks. As far as we know, this is the first linear speedup result in multi-block minimax bilevel optimization.

5 Applications and Experiments

In this section, we conduct extensive experiments in two applications: deep AUC maximization and rank-based robust meta-learning. More experimental results such as time and space comparison are provided in Appendix B.

5.1 Deep AUC Maximization

5.1.1 Formulation

Deep AUC Maximization (DAM) addresses machine learning challenges presented by imbalanced datasets. In particular, the AUC (Area Under the ROC Curve) measures the likelihood that a positive sample's prediction score will be higher than that of a negative sample. As outlined by [24], the DAM issue is structured as a multi-block minimax bilevel optimization problem:

where g j ( u j , w j ) := 1 2 ∥ ∥ u j -( w j -˜ η ∇ L AVG ( w j ) )∥ ∥ 2 , L AVG ( w j ) := 1 n ∑ n i =1 ℓ ( w j ; x i , y i ) , ℓ denotes the task loss (e.g., the cross-entropy loss in binary classification tasks), and Φ j denotes the sample-level AUC loss function. The detailed formulation can be found in Appendix A.1. With the method in Section 3, we reformulate this problem as a single-level minimax optimization problem:

where L ( w , u , a, b, α, v ) := ∑ m j =1 Φ j ( u j , a j , b j , α j ) + λ ( g j ( u j , w j ) -g j ( v j , w j ) ) is the Lagrangian function of AUC loss function, and v j is the approximate optimal solution of g j .

5.1.2 Results

Settings. Following the work [24], we assess our methodology using four datasets, namely CIFAR100 [29], CelebA [40], CheXpert [26] and OGBG-MolPCBA [25], whose details are provided in Appendix B.1. We evaluate the effectiveness of our algorithm by comparing it with direct optimization on multi-block minimax AUC loss (mAUC) and compositional training on mAUC loss (mAUC-CT). The test AUC scores of mAUC and mAUC-CT for different datasets in Table 1 are derived from the original paper. Detailed configuration of experiments can be found in Appendix B.2.

Results. The results of deep AUC maximization on different datasets are shown in Table 1. The results indicate that our FOSL outperforms the mAUC method in terms of test AUC scores on all datasets and achieves comparative or better performance than mAUC-CT on various datasets. We proceed to visualize the AUC loss during the initial stages of training for all methods on CelebA, as depicted in Figure 1a and 1b. The figures illustrate that, in the initial stage, our method and mAUC-CT [24] exhibit a faster loss drop than mAUC, and our method achieves the fastest overall convergence rate. Furthermore, our approach exhibits a smaller fluctuation compared to other baseline methods.

CIFAR100

CelebA

CheXpert

OGBG-MolPCBA

mAUC[24]

0.9044 (0.0015)

0.9062 (0.0042)

0.8084 (0.1455)

0.7793 (0.0028)

mAUC-CT[24]

0.9272 (0.0014)

0.9192 (0.0004)

0.8198 (0.1495)

0.8406 (0.0044)

FOSL(ours)

0.9540 (0.0009)

0.9267 (0.0018)

0.8166 (0.0051)

0.8516 (0.0014)

(a)

(b)

(c)

(d)

10

30

CIFAR100

CelebA

mAUC-CT

mAUC

0.95

Train AUC Loss (103)

FOSL

mAUC

Train AUC Loss (103)

mAUC-CT

FOSL

06°0

0.90

Test AUC Score

0.85

Test AUC Score

0.85

0.80

A=0.1

A=10

0.80

A=0.1

A=10

0.75

0.70

A=2

A=4

A=6

A=30

A=50

0.75

A=2

入=6

入=4

A=50

A=30

A=8

A=8

口

25

50

Training Iterations

75

100

125

150

200

100

Training Time

150

200

250

300

350

400

0.65

0

200

400

Epochs

009

800

1000

0.70

0

5

10

Epochs

15

20

25

Impact of λ . To evaluate the impact of the hyper-parameter λ on training with FOSL algorithm, we conduct an ablation study on the CIFAR100 and the CelebA datasets. The test AUC scores along with training are depicted in Figure 1c and 1d. As shown in Figure 1c, our method sustains robust performance across a wide range of choices for λ . For example, training within a λ range of [2, 8] shows that the speed of convergence and the final test performance are not sensitive to the change of λ . A similar observation also holds for the CelebA dataset as shown in Figure 1d.

5.2 Robust Meta-learning with Rank-based Loss

5.2.1 Formulation

Our objective is to devise a robust meta-learning approach wherein, during each iteration, tasks with large loss values are filtered out, and the meta-model is updated with the remaining tasks. This approach effectively reduces the impact of tasks with noisy samples (noisy tasks), because deep learning models tend to acquire clean and simple patterns in their initial training stages [19], such that noisy samples/tasks often have large loss values. Further justification can be found in Figure 2.

We first define g [ i ] as the i th largest element of the set G = { g 1 , g 2 , ..., g n } , such that g [ n ] ≤ g [ n -1] ≤ ... ≤ g [1] . Denote the task-specific loss as g i ( ϕ, w i ) , where ϕ is the parameter of the meta-model and w i is the task-specific parameter. The proposed formulation is then given by:

where g [ i ] ( ϕ, w ∗ [ i ] ( ϕ ) ) is the i th largest task-specific loss given w ∗ := [ w ∗ i ( ϕ ) , ..., w ∗ n ( ϕ ) ] T , and w ∗ [ i ] ( ϕ ) is the corresponding optimal task-specific parameter.

With the Lemma 1 in [42], by introducing an auxiliary variable γ , we can reformulate the problem as:

Details about the derivation of the above formulation can be found in Appendix A.2. This formulation poses a non-convex optimization challenge for the primal variable ϕ , making it challenging to address using conventional optimization methods. Nevertheless, our proposed algorithm enables efficient resolution of this problem by reformulating the problem into a singlelevel minimax optimization problem as: min ϕ, w max γ, v L ( ϕ, w , γ, v ) , where L ( ϕ, w , γ, v ) := 1 n ∑ n i =1 f i ( ϕ, w i , γ )+ λ ( g i ( ϕ, w i ) -g i ( ϕ, v i ) ) is the Lagrangian function of the rank based loss function, v is an approximate optimal task-specific parameter of the lower-level problem.

(a)

(b)

(c)

(d)

0.12

Rand

0.12

Flip

1.0

Rand

0.8

Flip

0.10

noisy

clean

0.11

0.10

noisy

clean

0.8

MAML

MemCS

0.7

MAML

MemCS

bujuje

SSOI

0.06

0.08

SSO|

6ujue

0.09

0.08

0.07

Ratio

0.4

0.6

Ratio

0.6

0.5

0.06

0.4

0.04

0.05

0.04

0.2

0.3

0.02

0

200

400

Training iteration

600

800

1000

1200

0.03

0

200

400

Training iteration

600

008

1000

1200

0.0

0

300

Training iteration

600

900

1200

0.2

0

300

Training iteration

600

900

1200

5.2.2 Results

Settings. We perform meta-learning experiments on few-shot learning tasks, focusing on the capability of rapid adaptation to new tasks with limited samples. Adhering to standard few-shot learning configurations, we carry out 5-ways 5-shot learning experiments on Mini-ImageNet [53] (referred to as Mini in Table 2) and Tiered-ImageNet [47] (referred to as Tiered in Table 2), where each task involves a 5-class classification task, with five samples per class used as training data. Since our robust meta-learning formulation is built on that of MAML [6], we compare our method with MAMLon both clean dataset and noisy dataset to evaluate the effectiveness and robustness of our algorithm. In the noisy setting, we adopt a standard noisy training scheme in meta-learning, where the labels in a noisy task are randomly flipped. Specifically, we employ two label flipping strategies: Flip, where in each iteration, a certain portion of tasks (60% in Table 2) are designated as noisy, and each sample within the task is assigned to one of all labels with equal probability; and Rand, where a random noisy ratio is assigned to each task in every iteration, determining the proportion of samples to be flipped by randomly assigning another label to them. Detailed configuration of experiments can be found in Appendix B.2.

Results. Table 2 displays the test accuracies. These results show that in the presence of noisy tasks, both MAML and our MemCS method undergo a decline in performance across both datasets, yet our approach manages to sustain a reasonable performance. To further evaluate the resilience of our MemCS method against MAML, we executed additional experiments with escalating noise levels in the Flip scenario, with these findings detailed in Table 3. The data clearly show a performance decrease for both methodologies as the noise ratio intensifies. Nonetheless, our approach exhibits a notably more gradual decline in performance as the noise ratio escalates, especially at higher noise levels, signifying superior robustness compared to MAML.

Discussion. To show the effectiveness of our approach in facilitating robust learning, we have visualized the average losses for both clean and noisy tasks separately within the MAML training framework, as demonstrated in Figure 2. The graphical representation uncovers a consistent pattern where the losses associated with noisy tasks consistently exceed those related to clean tasks throughout the training period. This pattern underscores our approach's capacity to lessen the detrimental effects of noisy tasks. Further, we examine the noisy tasks in the update mechanism at five distinct intervals during the training phase, illustrated in Figure 2. The findings show that our methodology successfully deters the influence of noisy tasks on the meta-model's updates across both Rand and Flip scenarios, maintaining this protective measure throughout the training duration.

6 Conclusion

In this paper, we propose two fully first-order algorithms designed to address the challenges posed by multi-block minimax bilevel optimization problems: a fully single-loop algorithm, FOSL, and a

memory-efficient double-loop algorithm with cold-start initialization, MemCS. We show that our methods can achieve comparative and even better per-block sample complexities than other methods with the same type in standard bilevel optimization. The experimental results indicate that our methods consistently demonstrate superior performance and robustness in applications on deep AUC maximization and robust meta-learning.

References

Michael Arbel and Julien Mairal. Amortized implicit differentiation for stochastic bilevel optimization. arXiv preprint arXiv:2111.14580 , 2021.

Jerome Bracken and James T McGill. Mathematical programs with optimization problems in the constraints. Operations Research , 21(1):37-44, 1973.

Liam Collins, Aryan Mokhtari, and Sanjay Shakkottai. Task-robust model-agnostic metalearning. Advances in Neural Information Processing Systems , 33:18860-18871, 2020.

Corinna Cortes and Mehryar Mohri. Auc optimization vs. error rate minimization. Advances in Neural Information Processing Systems , 16, 2003.

Justin Domke. Generic methods for optimization-based modeling. In Artificial Intelligence and Statistics , pages 318-326. PMLR, 2012.

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning , pages 1126-1135. PMLR, 2017.

Luca Franceschi, Michele Donini, Paolo Frasconi, and Massimiliano Pontil. Forward and reverse gradient-based hyperparameter optimization. In International Conference on Machine Learning , pages 1165-1173. PMLR, 2017.

Wei Gao, Rong Jin, Shenghuo Zhu, and Zhi-Hua Zhou. One-pass auc optimization. In International Conference on Machine Learning , pages 906-914. PMLR, 2013.

Wei Gao and Zhi-Hua Zhou. On the consistency of auc pairwise optimization. arXiv preprint arXiv:1208.0645 , 2012.

Camille Garcin, Maximilien Servajean, Alexis Joly, and Joseph Salmon. Stochastic smoothing of the top-k calibrated hinge loss for deep imbalanced classification. In International Conference on Machine Learning , pages 7208-7222. PMLR, 2022.

Guillaume Garrigos and Robert M Gower. Handbook of convergence theorems for (stochastic) gradient methods. arXiv preprint arXiv:2301.11235 , 2023.

Saeed Ghadimi and Mengdi Wang. Approximation methods for bilevel programming. arXiv preprint arXiv:1802.02246 , 2018.

Stephen Gould, Basura Fernando, Anoop Cherian, Peter Anderson, Rodrigo Santa Cruz, and Edison Guo. On differentiating parameterized argmin and argmax problems with application to bi-level optimization. arXiv preprint arXiv:1607.05447 , 2016.

Riccardo Grazzi, Luca Franceschi, Massimiliano Pontil, and Saverio Salzo. On the iteration complexity of hypergradient computation. In International Conference on Machine Learning , pages 3748-3758. PMLR, 2020.

Riccardo Grazzi, Massimiliano Pontil, and Saverio Salzo. Bilevel optimization with a lowerlevel contraction: Optimal sample complexity without warm-start. Journal of Machine Learning Research , 24(167):1-37, 2023.

Alex Gu, Songtao Lu, Parikshit Ram, and Lily Weng. Nonconvex min-max bilevel optimization for task robust meta learning. In International Conference on Machine Learning , 2021.

Alex Gu, Songtao Lu, Parikshit Ram, and Lily Weng. Min-max multi-objective bilevel optimization with applications in robust machine learning. In International Conference on Learning Representations , 2023.

Zhishuai Guo, Mingrui Liu, Zhuoning Yuan, Li Shen, Wei Liu, and Tianbao Yang. Communication-efficient distributed stochastic auc maximization with deep neural networks. In International Conference on Machine Learning , pages 3864-3874. PMLR, 2020.

Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in Neural Information Processing Systems , 31, 2018.

Pierre Hansen, Brigitte Jaumard, and Gilles Savard. New branch-and-bound rules for linear bilevel programming. SIAM Journal on scientific and Statistical Computing , 13(5):1194-1217, 1992.

Ralf Herbrich. Large margin rank boundaries for ordinal regression. Advances in Large Margin Classifiers , pages 115-132, 2000.

Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and application to actorcritic. SIAM Journal on Optimization , 33(1):147-180, 2023.

Quanqi Hu, Bokun Wang, and Tianbao Yang. A stochastic momentum method for min-max bilevel optimization. 2021.

Quanqi Hu, Yongjian Zhong, and Tianbao Yang. Multi-block min-max bilevel optimization with applications in multi-task deep auc maximization. Advances in Neural Information Processing Systems , 35:29552-29565, 2022.

Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems , 33:22118-22133, 2020.

Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI conference on artificial intelligence , volume 33, pages 590-597, 2019.

Kaiyi Ji, Junjie Yang, and Yingbin Liang. Bilevel optimization: Convergence analysis and enhanced design. In International Conference on Machine Learning , pages 4882-4892. PMLR, 2021.

Krishnateja Killamsetty, Changbin Li, Chen Zhao, Feng Chen, and Rishabh Iyer. A nested bi-level optimization framework for robust few shot learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 7176-7184, 2022.

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.

Jeongyeol Kwon, Dohyun Kwon, Stephen Wright, and Robert D Nowak. A fully first-order method for stochastic bilevel optimization. In International Conference on Machine Learning , pages 18083-18113. PMLR, 2023.

Kuang-Huei Lee, Xiaodong He, Lei Zhang, and Linjun Yang. Cleannet: Transfer learning for scalable image classifier training with label noise. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 5447-5456, 2018.

Lihua Lei, Cheng Ju, Jianbo Chen, and Michael I Jordan. Non-convex finite-sum optimization via scsg methods. Advances in Neural Information Processing Systems , 30, 2017.

Renjie Liao, Yuwen Xiong, Ethan Fetaya, Lisa Zhang, KiJung Yoon, Xaq Pitkow, Raquel Urtasun, and Richard Zemel. Reviving and improving recurrent back-propagation. In International Conference on Machine Learning , pages 3082-3091. PMLR, 2018.

Gui-Hua Lin, Mengwei Xu, and Jane J Ye. On solving simple bilevel programs with a nonconvex lower level program. Mathematical Programming , 144(1-2):277-305, 2014.

Bo Liu, Mao Ye, Stephen Wright, Peter Stone, and Qiang Liu. Bome! bilevel optimization made easy: A simple first-order approach. Advances in Neural Information Processing Systems , 35:17248-17262, 2022.

Chenghao Liu, Zhihao Wang, Doyen Sahoo, Yuan Fang, Kun Zhang, and Steven CH Hoi. Adaptive task sampling for meta-learning. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XVIII 16 , pages 752-769. Springer, 2020.

Mingrui Liu, Zhuoning Yuan, Yiming Ying, and Tianbao Yang. Stochastic auc maximization with deep neural networks. arXiv preprint arXiv:1908.10831 , 2019.

Risheng Liu, Xuan Liu, Xiaoming Yuan, Shangzhi Zeng, and Jin Zhang. A value-function-based interior-point method for non-convex bi-level optimization. In International Conference on Machine Learning , pages 6882-6892. PMLR, 2021.

Risheng Liu, Yaohua Liu, Shangzhi Zeng, and Jin Zhang. Towards gradient-based bilevel optimization with non-convex followers and beyond. Advances in Neural Information Processing Systems , 34:8662-8675, 2021.

Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV) , December 2015.

Jonathan Lorraine, Paul Vicol, and David Duvenaud. Optimizing millions of hyperparameters by implicit differentiation. In International Conference on Artificial Intelligence and Statistics , pages 1540-1552. PMLR, 2020.

Siwei Lyu, Yanbo Fan, Yiming Ying, and Bao-Gang Hu. Average top-k aggregate loss for supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(1):7686, 2020.

Soufiane Mallem, Abul Hasnat, and Amir Nakib. Efficient meta label correction based on meta learning and bi-level optimization. Engineering Applications of Artificial Intelligence , 117:105517, 2023.

Yurii Nesterov and Boris T Polyak. Cubic regularization of newton method and its global performance. Mathematical Programming , 108(1):177-205, 2006.

Fabian Pedregosa. Hyperparameter optimization with approximate gradient. In International Conference on Machine Learning , pages 737-746. PMLR, 2016.

Alain Rakotomamonjy. Optimizing area under roc curve with svms. In ROCAI , pages 71-80, 2004.

Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenenbaum, Hugo Larochelle, and Richard S Zemel. Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv:1803.00676 , 2018.

Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. In International Conference on Machine Learning , pages 4334-4343. PMLR, 2018.

Amirreza Shaban, Ching-An Cheng, Nathan Hatch, and Byron Boots. Truncated backpropagation for bilevel optimization. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 1723-1732. PMLR, 2019.

Chenggen Shi, Jie Lu, and Guangquan Zhang. An extended kuhn-tucker approach for linear bilevel programming. Applied Mathematics and Computation , 162(1):51-63, 2005.

Ankur Sinha, Pekka Malo, and Kalyanmoy Deb. A review on bilevel optimization: From classical to evolutionary approaches and applications. IEEE Transactions on Evolutionary Computation , 22(2):276-295, 2017.

Haoliang Sun, Chenhui Guo, Qi Wei, Zhongyi Han, and Yilong Yin. Learning to rectify for robust learning with noisy labels. Pattern Recognition , 124:108467, 2022.

Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Advances in neural information processing systems , 29, 2016.

Xiaoyu Wang, Rui Pan, Renjie Pi, and Tong Zhang. Effective bilevel optimization via minimax reformulation. arXiv preprint arXiv:2305.13153 , 2023.

Zhen Wang, Guosheng Hu, and Qinghua Hu. Training noise-robust deep neural networks via meta-learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4524-4533, 2020.

Tianbao Yang and Yiming Ying. Auc maximization in the era of big data and ai: A survey. ACM Computing Surveys , 55(8):1-37, 2022.

Yifan Yang, Peiyao Xiao, and Kaiyi Ji. Achieving O ( ϵ -1 . 5 ) complexity in Hessian/Jacobian-free stochastic bilevel optimization. arXiv preprint arXiv:2312.03807 , 2023.

Huaxiu Yao, Yu Wang, Ying Wei, Peilin Zhao, Mehrdad Mahdavi, Defu Lian, and Chelsea Finn. Meta-learning with an adaptive task scheduler. Advances in Neural Information Processing Systems , 34:7497-7509, 2021.

Zhuoning Yuan, Zhishuai Guo, Nitesh Chawla, and Tianbao Yang. Compositional training for end-to-end deep auc maximization. In International Conference on Learning Representations , 2021.

Zhuoning Yuan, Yan Yan, Milan Sonka, and Tianbao Yang. Large-scale robust deep auc maximization: A new surrogate loss and empirical studies on medical image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 3040-3049, 2021.

Guoqing Zheng, Ahmed Hassan Awadallah, and Susan Dumais. Meta label correction for noisy label learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 11053-11061, 2021.

A Specifications of Applications

In this section, we provide a detailed introduction to the formulation utilized in Section 5.

A.1 Deep AUC Maximization (DAM)

For a classifier model f ( w ) , we have the AUC score function as

where Pr ( X ) denote probability of an event X being true. One of the surrogate loss (AUC square loss [9]) is given by:

where n + and n -are the numbers of positive examples and negative examples, respectively, and c is the margin parameter. One can transfer this problem into an equivalent minimax optimization problem according to Proposition 1 in [37] by:

where

where a, b are margin parameters, p = n + /n . This formulation decomposed the individual examples, which is more favorable in stochastic scenarios. [59] proposed a compositional training algorithm for this problem. The compositional objective function is formulated as:

where L AVG = 1 n ∑ n i =1 ℓ ( w ; x i , y i ) , ℓ denotes task loss, e.g. cross-entropy in classification tasks. Consider a multi-block problem with m tasks. This problem can be reformulated as a multi-block minimax bi-level optimization problem [24]:

where g j ( u j , w j ) := 1 2 ∥ ∥ u j -( w j -˜ η ∇ L AVG ( w j ) )∥ ∥ 2 .

A.2 Robust Meta-learning

Consider the formulation of Robust Meta-learning in Section 5.2:

where g [ n ] ( ϕ, w ∗ n ( ϕ ) ) ≤ g [ n -1] ( ϕ, w ∗ n -1 ( ϕ ) ) ≤ ... ≤ g [1] ( ϕ, w ∗ 1 ( ϕ ) ) denotes task-specific losses. Wedefine g ∗ i ( ϕ ) := g i ( ϕ, w ∗ i ( ϕ ) ) for simplicity in later formulations. The summation of the bottom-k losses is equivalent to the sum of all task losses subtracted by the sum of the top-(n-k) losses:

With the Lemma 1 in [42], we have:

Now we can convert the original upper-level problem to:

The problem of robust meta-learning is then formulated as:

where ϕ is the parameter of meta-model, and w = [ w i , ..., w n ] T is the vector of task specific parameters.

Inspired by [10], we introduce a smoothed version of the upper-level loss function by incorporating Gaussian noise into the indicator function for alignment with the assumption of our MemCS algorithm:

where ϵ > 0 is the smoothing parameter, and Z ∼ N (0 , 1) is standard normal random variable.

B Implementation Details and Extra Experimental Results

B.1 Datasets Description

Deep AUC Maximization. We assess our methodology using four datasets. The first dataset, CIFAR100 [29], serves primarily for classification endeavors. Within the context of the multi-block deep AUC maximization challenge, we treat each of the 100 categories as an individual block, with samples belonging to a specific category considered positive for that block. This dataset comprises 60 , 000 images, each measuring 32 × 32 pixels, divided into 50 , 000 training and 10 , 000 testing images. The CelebA [40] dataset encompasses 202,599 facial images, each annotated with a diverse set of attributes from 40 different features. The CheXpert [26] dataset includes 224 , 316 chest radiograph images from 65 , 240 patients, marked for 14 distinct observations. Adhering to the methodology proposed in [24], we employ a simplified version of CheXpert with a reduced image resolution and omit the Fracture observation due to insufficient positive samples. Lastly, the OGBGMolPCBA [25] dataset is employed to predict molecular properties, representing each molecule as a graph with atoms as nodes and chemical bonds as edges, featuring 437 , 929 such graphs annotated across 128 properties.

Robust Meta-learning. Our experiments are conducted over two popular datasets for few-shot learning: Mini-ImageNet [53] and Tiered-ImageNet [47]. Both datasets are subsets of the ILSVRC12 dataset. Mini-ImageNet comprises 100 classes, each containing 600 images with dimensions of 84 × 84 pixels. The 100 classes are distributed among training, validation, and testing sets with a ratio of 64:16:20, respectively. In contrast, Tiered-ImageNet is a more extensive and challenging dataset, featuring 779,165 images annotated across 608 classes. These classes are further organized into 34 categories, with 20 categories designated for training, 6 for validation, and 8 for testing.

B.2 Implementation Details

Deep AUC Maximization. For the CIFAR100 and the CelebA datasets, we use the ResNet18 architecture. For the large-scale CheXpert dataset, we opt for the DenseNet121 model pre-trained on ImageNet. When dealing with the OGBG-MolPCBA graph dataset, the Graph Isomorphism Network (GIN) is used for training. All experimental runs are performed using a single NVIDIA RTX 6000 GPU. Regarding hyperparameters, we set the total training epoch to 2000 for the CIFAR100 and 100 for the OGBG-MolPCBA datasets, adjust it to 40 for CelebA, and reduce it to 6 for CheXpert. The learning rate for the optimal approximator v is uniformly set to η v = 0 . 1 across all experiments, with η w = η u = η v /λ to maintain gradient magnitude consistency between u and v . This consistency is vital due to the influence of the λ parameter in the Lagrangian function on the update process for u .

Learning rate decay is applied to CelebA starting at the 30th epoch and to CheXpert at the 4th epoch, whereas no decay strategy is applied for CIFAR100 and OGBG-MolPCBA.

Robust Meta-learning. We adopt the Adam optimizer to update the meta-model in the context of MAML. For the hyper-parameters of MAML, we set the learning rate of meta-model update as η meta = 0 . 02 , and set the learning rate of fast adaptation as η adapt = 0 . 02 , with an adaptation step of 15. To be consistent with the DAM experiments, we configure the learning rate of the optimal approximator as η v = η adapt = 0 . 02 , and the learning rate of w and meta-model parameter ϕ as η ϕ = η w = η v /λ in the implementation. In practice, setting λ to 3 results in favorable performance. All experiments are conducted on a single NVIDIA RTX 6000 GPU using a widely used lightweight model featuring 4 convolutional layers (CNN4).

B.3 Extra Results on Deep AUC Maximization

This section presents the visualization of statistics throughout the training process and compares the computational costs with our method and AUC-CT [24]. To better grasp the training dynamics, we charted the test AUC scores across various training epochs for both the CIFAR100 and CelebA datasets, as shown in Figure A1. The findings demonstrate that our method not only exhibits enhanced generalization capabilities but also greater stability.

Moreover, to assess efficiency across varying dataset scales, we examined the average iteration times and total training time of our FOSL algorithm and AUC-CT [24] on different-sized datasets ( 32 × 32 in CIFAR100 vs. 224 × 224 in CheXpert) as detailed in Table A1. Note that the training time largely depends on the implementation and hyperparameters, such as the number of sampled tasks and batch sizes per task, suggesting that computational costs can be adjusted by modifying these hyperparameters according to the dataset. The result in Table A1 shows the ability to control training costs for datasets of various scales, which is indicated by the small gap between the total training time on CelebA and CheXpert datasets. Additionally, our method demonstrated a faster training pace compared to AUC-CT [24] under the same training settings. Note that the implementation of AUC-CT [24] avoided the calculation of second-order matrices so that the computational cost is more controllable with an increased dataset scale.

B.4 Comparison between FOSL and MemCS

In this section, we compare our two proposed methods within the same experimental setting, i.e. robust meta-learning. To make it compatible with our FOSL algorithm, we slightly adjusted our training setting so that the number of training tasks in the dataset is known (20000 tasks) while maintaining the same testing procedures as those used with the MemCS algorithm. The results, including test accuracy, memory cost, and computational cost, are detailed in Table A2. The result shows that using FOSL in a robust meta-learning setting can introduce a greatly increased memory cost, which is especially significant for a small model. However, the single-loop nature of FOSL can drastically shorten the average iteration time during training. This makes the FOSL algorithm a potentially advantageous choice in scenarios involving larger base models and fewer blocks.

C Notations

The original problem we solve here is

Moreover, we define z ∗ λ,i ( x ) = arg min z L i ( x, y ∗ ( x ) , z, v ) and y ∗ ( x ) = arg max y F ( x, y, z ∗ ( x ) ) . For the convenience of proof, we also define

For the notational convenience of FOSL, we define the estimators of client set I t as

Since we sample tasks without replacement and our estimators are unbiased, we have the expectations of estimators as

We also define the optimal Lagrangian estimator of x and its gradients as

D Proofs of Preliminary Lemmas

D.1 Some basic properties

Lemma D.1. Under Assumptions 4.3, 4.4, for ∀ λ ≥ 2 L f, 1 µ g , both L i ( x, y, z, v ) and L ( x, y, z, v ) are ( λµ g 2 ) -strongly convex in z and L λ, 1 -smooth in ( x, y, z ) , where L λ, 1 := 3 λL g, 1 .

Proof. Since λ ≥ 2 L f, 1 µ g ≥ 2 L f, 1 L g, 1 , we have that

Then the proof is complete.

Lemma D.2. Under Assumptions 4.3, 4.4, for λ ≥ max { 2 L f, 1 µ g , (1 + L g, 1 µ g ) L 2 f, 1 3 µ f L g, 1 } , we have ∥∇ z ∗ i ( x ) ∥ ≤ L g, 1 µ g , ∥∇ z ∗ λ,i ( x ) ∥ ≤ 12 L g, 1 µ g and ∥∇ y ∗ ( x ) ∥ ≤ ( 1 + L g, 1 µ g ) L f, 1 µ f .

Proof. Recall that we define z ∗ i ( x ) := arg min z g i ( x, z ) and z ∗ λ,i ( x ) := arg min z L i ( x, y ∗ ( x ) , z, v ) . Then we have ∇ z g i ( x, z ∗ i ( x ) ) = 0 and ∇ z L i ( x, y ∗ ( x ) , z ∗ λ,i ( x ) , v ) = 0 . Via implicit function theorem, we obtain

To measure that Lipschitz continuity of z ∗ i ( x ) and z ∗ λ,i ( x ) w.r.t. x, we take spectral norm of ∇ z ∗ i ( x ) and ∇ z ∗ λ,i ( x ) as

where (a) uses Assumption 4.4. Similarly, for y ∗ ( x ) , we have ∇ y F ( x, y ∗ ( x ) , z ∗ ( x ) ) = 0 . Via implicit function theorem, we have

which indicates

where (a) uses Assumption 4.3 and Assumption 4.4. Back to the second equation in eq. (8), with ∥∇ y ∗ ( x t ) ∥ ≤ (1 + L g, 1 µ g ) L f, 1 µ f , we have

where (a) uses Lemma D.1 and (b) uses λ ≥ (1 + L g, 1 µ g ) L 2 f, 1 3 µ f L g, 1 . Then the proof is complete.

Lemma D.3. Under Assumptions 4.3, 4.4, the optimal solutions z ∗ i ( x ) , z ∗ λ,i ( x ) and y ∗ i ( x ) are L ∗ ,z -, L ∗ ,z λ - and L ∗ ,y -smooth respectively, where we define L ∗ ,z , L ∗ ,z λ and L ∗ ,y as

for any i ∈ { 1 , ..., n } , where we assume λ ≥ { 2 L f, 1 /µ g , (1 + L g, 1 µ g ) L 2 f, 1 3 µ f L g, 1 , (1 + L g, 1 µ g ) L f, 1 L f, 2 3 µ f L g, 1 , L f, 1 L ∗ ,y 6 L g, 1 ( 1 + (1 + L g, 1 µ g ) L f, 1 µ f + 12 L g, 1 µ g ) -1 , ( (1 + L g, 1 µ g ) L f, 1 µ f +1 ) L f, 1 L g, 1 } .

Proof. Since z ∗ i ( x ) = arg min z g i ( x, z ) , we have ∇ z g i ( x, z ∗ ( x ) ) = 0 , which indicates that

For any x 1 , x 2 ∈ R d x , we have

where (a) uses Assumption 4.3, 4.4 and ( A -1 -B -1 ) = A -1 ( B -A ) B -1 ; (b) follows from Lemma D.2. Next, plug x = x 1 and x = x 2 into eq. (9) and differentiate these two equations, then we get

and by using eq. (9), we get

By combining eq. (11), eq. (12) and taking norm, we have

where (a) uses Assumption 4.4 and Lemma D.2; (b) follows from Assumption 4.4, Lemma D.2, and eq. (10). Similarly to eq. (10), from eq. (7), if we simplify the notation as

then we have

For the first term in eq. (13), via Lemma D.1, we have

where (a) uses Lemma D.2; (b) follows from Lemmas D.2, D.3 and λ ≥ (1 + L g, 1 µ g ) L f, 1 L f, 2 3 µ f L g, 1 ; (c) uses λ ≥ L f, 1 L ∗ ,y 6 L g, 1 [ 1 + (1 + L g, 1 µ g ) L f, 1 µ f + 12 L g, 1 µ g ] -1 . By using Lemma D.1, we have ∥ B -1 1 ∥ ≤ 2 λµ g , ∥ B -1 2 ∥ ≤ 2 λµ g , ∥ B 1 -B 2 ∥ ≤ 3 λL g, 1 ∥ x 1 -∥ and

where (a) uses Assumption 4.4 and Lemma D.2; (b) uses λ ≥ ( (1 + L g, 1 µ g ) L f, 1 µ f +1 ) L f, 1 L g, 1 . We also have

where (a) uses Lemma D.2. Combining eq. (14), eq. (15), eq. (16) with the results in Lemma D.1, we have

Thus, the proof is complete.

D.2 Gap of Lower-level Optimal Points

Lemma D.4. Under Assumptions 4.3, 4.4, for any given x and , the gap between the optimal solutions of the lower-level problem z ∗ i ( x ) and the surrogate minimax problem z ∗ λ,i ( x ) can be bounded as

for any i ∈ { 1 , ..., n } , where we assume λ ≥ max { 2 L f, 1 µ g , (1 + L g, 1 µ g ) L 2 f, 1 3 µ f L g, 1 } .

Proof. For each block, we can have that

where (a) uses Assumption 4.3; (b) follows from the definition of z ∗ i ( x ) and z ∗ λ,i ( x ) ; (c) uses Assumption 4.4. For the second part, since ∇ z g i ( x, z ∗ i ( x ) ) = 0 , ∇ z L i ( x, y ∗ ( x ) , z ∗ λ,i ( x ) ) = 0 , we have

which indicates that

Then the gap can be displayed as

where (a) can be satisfied because

where (a.1) uses Assumption 4.3 and Lemma D.1. Then, the proof is complete.

Lemma D.5. Under Assumptions 4.3, 4.4, the gap between ∇ Φ( x ) and H ∗ ( x ) can be bounded as

∥ ∥ ∇ Φ( x ) -H ∗ ( x ) ∥ ∥ 2 ≤ C gap λ 2 , where C gap := 3 ( 1 + L 2 f, 1 µ 2 g ) L 2 f, 1 ( L f, 0 µ g ) 2 + ( 1 + L 2 g, 1 µ 2 g ) 3 L 2 g, 1 2 ( L f, 0 µ g ) 4 and we assume λ ≥ max { 2 L f, 1 µ g , (1 + L g, 1 µ g ) L 2 f, 1 3 µ f L g, 1 } .

Proof. By the definitions of ∇ F ( x, y ∗ ( x ) , z ∗ ( x ) ) and H ∗ ( x ) , we have

For any i ∈ { 1 , ..., n } , we have

where (a) follows from Assumption 4.3, 4.4 and eq. (7); (b) follows from Assumption 4.3, 4.4 and Lemma 1 in [44]; (c) uses Lemma D.4. The proof is finished by substituting eq. (18) into eq. (17).

Lemma D.6. Under Assumptions 4.3, 4.4, the gradient of Lagrangian function with optimal solutions H ∗ ( x ) is L ∗ , 1 -Lipschitz continuous in x , where we define L ∗ , 1 := ( 1+ 12 L g, 1 µ g + ( 1+ L g, 1 µ g ) L f, 1 µ f ) L f, 1 + ( 1+ L g, 1 µ g ) L f, 0 L g, 2 µ g + L g, 2 [ 1 µ g ( L f, 0 L g, 2 µ g + L f, 1 ( 1+(1+ L g, 1 µ g ) L f, 1 µ f ) ) + 6 L g, 1 µ 2 g ( 1+ ( 1+ L g, 1 µ g ) )( L f, 1 + L f, 0 L g, 2 µ g )] and we assume λ ≥ { 2 L f, 1 /µ g , (1+ L g, 1 µ g ) L 2 f, 1 3 µ f L g, 1 , (1+ L g, 1 µ g ) L f, 1 L f, 2 3 µ f L g, 1 , L f, 1 L ∗ ,y 6 L g, 1 ( 1+(1+ L g, 1 µ g ) L f, 1 µ f + 12 L g, 1 µ g ) -1 , ( (1 + L g, 1 µ g ) L f, 1 µ f +1 ) L f, 1 L g, 1 } .

Proof. Recall that in eq. (6),

Then we have

By taking norm, we have

where (a) uses Assumption 4.4 and Lemma D.2; (b) follows from Lemma D.4 Then, the proof is complete.

E Proofs of Theorem 4.10 and and Corollary 4.11

E.1 Descent in Objective Function

Lemma E.1. Under Assumptions 4.3, 4.4, 4.5 and Lemma D.6, for L ∗ , 1 -smooth L ∗ ( x ) , the consecutive iterates of Algorithm 1 satisfy:

for all t ∈ { 0 , 1 , ..., T -1 } , where we assume λ ≥ 2 L f, 1 µ g .

Proof. Recall the definitions of L ∗ ( x ) and H ∗ ( x ) in eq. (6). By using the smoothness of L ∗ ( x t ) in Lemma D.6, we have that

where (a) follows from

and (a.1) uses Jensen inequality; (a.2) follows from Assumption 4.4 and Lemma D.1. Then, the proof is complete.

E.2 Bounds of Estimators

Lemma E.2. Under Assumptions 4.3, 4.4, 4.5, 4.6, the estimators of v i , z i , y and x can be bounded as

Proof. By using the definition of z ∗ i ( x t ) , we can have that

Similarly, we have

Next, for the estimator of x , we have

where (a) uses unbiased estimation in Assumption 4.5. For the first part of eq. (20), since tasks are selected without replacement, we have

where (a) uses the unbiased estimation assumption in Assumption 4.5. For the second part of eq. (20), we have

where (a) used the Lemma A.1 in [32]; (b) uses Assumption 4.4. By combining eq. (22) with eq. (21), the fourth inequality is proved. Last, for the estimator of y , we have

where (a) uses Assumption 4.5; (b) follows from the Lemma A.1 in [32]; (c) uses Assumption 4.5, 4.6; (d) follows from definition y ∗ ( x ) = arg max y 1 n ∑ n i f i ( x, y, z ∗ i ( x ) ) ; (e) uses Assumption 4.4 and (f) uses β th ≥ 1 . Then, the proof is complete.

E.3 Descent in Approximation Errors

Lemma E.3. Under Assumptions 4.3, 4.4, 4.5, 4.6, there exists δ v, 1 , δ z, 1 , δ y, 1 such that the iterates of v i,t , z i,t and y t in Algorithm 1 satisfy

for all t ∈ { 0 , ..., T -1 } , where we define δ v := δ v, 1 + 3 η 2 x L ∗ ,z 2 ( L 2 f, 0 + 2 λ 2 L 2 g, 0 ) , δ z := δ v, 1 + 3 η 2 x L ∗ ,z λ 2 ( L 2 f, 0 +2 λ 2 L 2 g, 0 ) , δ y := δ y, 1 + 3 η 2 x L ∗ ,y 2 ( L 2 f, 0 +2 λ 2 L 2 g, 0 ) and we assume λ ≥ { 2 L f, 1 /µ g , (1+ L g, 1 µ g ) L 2 f, 1 3 µ f L g, 1 , (1+ L g, 1 µ g ) L f, 1 L f, 2 3 µ f L g, 1 , L f, 1 L ∗ ,y 6 L g, 1 ( 1+(1+ L g, 1 µ g ) L f, 1 µ f + 12 L g, 1 µ g ) -1 , ( (1+ L g, 1 µ g ) L f, 1 µ f +1 ) L f, 1 L g, 1 } , η v ≤ µ g 2 L 2 g, 1 , η z ≤ µ g 32 L 2 g, 1 λ .

Proof. For the iterations of the lower-level problem, we have

For the first term of eq. (23), we have

where (a) follows from Assumption 4.3; (b) uses Lemma E.2; (c) results from η v ≤ µ g 2 L 2 g, 1 . For the second term of eq. (23), we have

For the last term of eq. (23), we have

where (a) use Lemma D.3 and Lemma 1 in [44]; (b) defines δ v := δ v, 1 + 3 η 2 x L ∗ ,z 2 ( L 2 f, 0 +2 λ 2 L 2 g, 0 ) . By plugging eq. (24), eq. (25), eq. (26) into eq. (23), we have

Then we get

Thus, the first inequality in the lemma is proved. Similarly, we have

We can bound the first term in eq. (27) similarly with eq. (24) as

where (a) uses Lemma E.2 and λ ≥ max { L f, 1 L g, 1 , σ f σ g } ; (b) uses η z λ ≤ µ g 32 L 2 g, 1 ; we bound the second term in eq. (27) as

and we bound the last term in eq. (27) similarly with eq. (26) as

where δ z := δ z, 1 + 3 η 2 x L ∗ ,z λ 2 ( L 2 f, 0 +2 λ 2 L 2 g, 0 ) . By plugging eq. (28), eq. (29), eq. (30) into eq. (27), we have

After telescoping, we obtain

Then the second inequality in the lemma is proved. Last, for y t and y ∗ ( x t ) , we have

We can bound the first term in eq. (31) as

where (a) uses the definition of y ∗ ( x t ) and eq. (5); (b) uses strong concavity of f i in y ; (c) follows from definition of y ∗ ( x t ) and Assumption 4.4; (d) uses Lemma E.2. We can bound the second term in eq. (31) as

where (a) follows from Lemma D.2. Also, we can get the bound of the last term as

where (a) uses Lemma D.2 and Lemma D.3; (b) use Lemma D.3 and Lemma 1 in [44]; (c) follows from Assumption 4.4; (d) defines δ y = δ y, 1 + 3 η 2 x L ∗ ,y 2 ( L 2 f, 0 + 2 λ 2 L 2 g, 0 ) . By plugging eq. (32), eq. (33), eq. (34) into eq. (34), we get

Then the last inequality is proved. Thus, the proof is complete.

E.4 Descent in the Lyapunov Function and Proof of Theorem 4.10

We define the Lyapunov function as

where the coefficients are given by

For convenience, we define the following constants:

We also constrain the conditions as below:

Plugging Lemma E.1, Lemma E.3 into eq. (35) and using eq. (36), we have the descent in the Lyapunov function as

Proof. To simplify the problem, we assume | I t | = P for t = 0 , ..., T -1 . By taking summation of eq. (39) from t = 0 to T -1 , we get

By using Lemma D.5 and eq. (40), we have

where (a) uses eq. (40); (b) takes η x = O ( T -5 7 ) , η y = O ( T -2 7 ) , η z = O ( T -5 7 ) , η v = O ( T -4 7 ) , λ = O ( T 1 7 ) , which satisfies eq. (38). Thus, Theorem 4.10 is proved.

E.5 Proof of Corollary 4.11

Proof. From eq. (41), to achieve ϵ -accurate stationary point of the objective function Φ( x ) in definition 4.2, we let 1 T ∑ T -1 t =0 E ∥∇ Φ( x t ) ∥ 2 = O ( T -2 7 ) ≤ ϵ . As a result, we can see that the epochs number we need is T = O ( ϵ -7 2 ) . The total sample complexity is PT = O ( Pϵ -7 2 ) . Then, we finish the proof.

F Proofs of Theorem 4.12 and Corollary 4.13

F.1 Descent in Objective Function

Lemma F.1. Under Assumptions 4.3, 4.4, 4.5 and Lemma D.6, for L ∗ , 1 -smooth L ∗ ( x ) , the consecutive iterates of Algorithm 2 satisfy:

Proof. Recall the definitions of L ∗ ( x ) and H ∗ ( x ) in eq. (6). By using the smoothness of L ∗ ( x t ) in Lemma D.6, we have that

where (a) uses

and (a.1) uses Jensen inequality; (a.2) follows from Lemma D.1. Then, the proof is complete.

F.2 Bounds of Estimators

Lemma F.2. Under Assumptions 4.3, 4.4, 4.5, we have the bounds of the estimators of y t and x t as

for any i ∈ { 1 , ..., n } and t ∈ { 0 , 1 , ..., T -1 } .

Proof. For the estimator of y , we have

where (a) follows from the Lemma A.1 in [32]; (b) uses Assumption 4.5, 4.6; (c) follows from definition y ∗ ( x ) = arg max y 1 n ∑ n i f i ( x, y, z ∗ i ( x ) ) ; (d) uses Assumption 4.4 and (e) uses β th ≥ 1 . Next, for the estimator of y , we have

Then, the proof is complete.

F.3 Bounds of Sub-loop Errors

Lemma F.3. Under Assumptions 4.3, 4.4, for ∀ δ ∈ R + , we assume that ∥ z ∗ i ( x t ) ∥ ≤ B for some B < ∞ . Then we have

Proof. From Lemma D.1, we have that L i is λµ g 2 -strongly convex in z and we have that L i is 2 λL g, 1 -Lipschitz continue in z when λ ≥ L f, 1 µ g ; also, from Assumption 4.3, 4.4, we have that g i is µ g -strongly convex in v and L g, 1 -smooth in v . According to Theorem 3.6 in [11], by taking 0 < η v t < 1 2 L g, 1 and 0 < η z t < 1 4 λL g, 1 , we have

To make sure E ∥ v K i,t -z ∗ i ( x t ) ∥ 2 ≤ ϵ sub and E ∥ z K i,t -z ∗ λ,i ( x t ) ∥ 2 ≤ ϵ sub for some ϵ sub ≥ 0 , we let

and

where (a) uses Lemma D.4; (b) take λ ≥ 2 L f, 1 µ g . Both can be achieved by taking

Then the proof is complete.

Lemma F.4. Under the Assumptions 4.3, 4.4, 4.5, the iterates of y t updates according to Algorithm 2 satisfy

for any t ∈ { 0 , ..., T -1 } , where we assume λ ≥ { 2 L f, 1 /µ g , (1 + L g, 1 µ g ) L 2 f, 1 3 µ f L g, 1 , (1 + L g, 1 µ g ) L f, 1 L f, 2 3 µ f L g, 1 , L f, 1 L ∗ ,y 6 L g, 1 ( 1 + (1 + L g, 1 µ g ) L f, 1 µ f + 12 L g, 1 µ g ) -1 , ( (1 + L g, 1 µ g ) L f, 1 µ f +1 ) L f, 1 L g, 1 } .

Proof. For y and y ∗ ( x ) , we have

We can bound the first term in eq. (43) as

where (a) uses the definition of y ∗ ( x t ) and eq. (5); (b) uses strong concavity of f i in y ; (c) follos from definition of y ∗ ( x t ) and Assumption 4.4; (d) uses Lemma F.2. We can bound the second term in eq. (43) as

where (a) follows from Lemma D.2. Also, we can get the bound of the last term as

where (a) uses Lemma D.2, D.3 and Lemma 1 in [44]; (b) follows from Assumption 4.4; (c) defines δ y = δ y, 1 + 3 η 2 x L ∗ ,y 2 ( L 2 f, 0 +2 λ 2 L 2 g, 0 ) . By plugging eq. (44), eq. (45), eq. (46) into eq. (43), we get

Then the proof is complete.

F.4 Descent in the Lyapunov Function and Proof of Theorem 4.12

We define the Lyapunov function as

where the coefficient is given by K y = 3 η x L 2 f, 1 η y µ f . We also constrain the conditions as below:

Plugging Lemma F.1, Lemma E.3 into eq. (47), we have the descent in the Lyapunov function as Ψ t +1 -Ψ t

where (a) uses Lemma F.2; (b) uses eq. (48) and takes

F.5 Proof of Theorem 4.12

Proof. For partial block participation, we take the summation of eq. (49) from t = 0 to T -1 . Then we have

For partial block participation, By using Lemma D.5, we have

where (a) follows from eq. (50); (b) follows from 1 ≤ P < n ; (c) takes η x = O ( P 1 5 T -2 3 ) , η y = O ( P -1 5 T -1 2 ) , λ = O ( P 1 10 T 1 6 ) , ϵ sub = O ( P -2 5 T -2 3 ) .

Next, for full block participation ( n = P ) , we have the descent in the Lyapunov function for full block participation as

Taking summation of eq. (52) from t = 0 to T -1 , we have

By using Lemma D.5 and eq. (53), we have

By taking η x = O (1) , η y = O (1) , λ = O ( T 1 2 ) , ϵ sub = O ( T -2 ) . Then Theorem 4.12 is proved.

F.6 Proof of Corollary 4.13

Proof. For tasks participate in updates partially, by eq. (51), we can find the ϵ -stationary point in definition 4.2 once we take T = O ( P -3 5 ϵ -3 ) . Note that we set the error of sub-loop as ϵ sub = O ( P -2 5 T -2 3 ) = O ( ϵ 2 ) . According to Lemma F.3, once we take η v = O (1) and η z = O ( P -1 10 T -1 6 ) , we have the iteration number of sub-loop as K = O ( log( 1 ϵ ) ) . Thus, we have the total sample complexity PKT = O ( P 2 5 ϵ -3 log( 1 ϵ )) = ˜ O ( P 2 5 ϵ -3 ) .

Similarly, by eq. (54), we can find the ϵ -stationary point in definition 4.2 once we take T = O ( ϵ -1 ) . Note that we set the error of sub-loop as ϵ sub = O ( T -2 ) = O ( ϵ 2 ) . Once we take η v = O (1) and η z = O ( T -1 2 ) , we have the iteration number of sub-loop as K = O ( log( 1 ϵ ) ) . Thus, we have the total sample complexity nKT = O ( nϵ -1 log( 1 ϵ )) = ˜ O ( nϵ -1 ) .

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS paper checklist" ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction in this paper accurately reflect the paper's contributions and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitation of previous works is discussed in the Introduction part.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide complete assumption in the main text and full proof in the appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide detailed setting of the experiments in the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We provide the code as the supplementary material.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Data and experimental settings are provided in the experiment part in the main text.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Error bars are provided in the experiments in Table 1.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The computational resources are described in the implementation detail part in the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The authors have reviewed the Code of Ethics and make sure the research conforms with it.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This paper focuses on the theoretical analysis of multi-block minimax bilevel optimization problem, which does not have a social impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The datasets used in this paper contain widely used real-world data; This research does not use pre-trained LLM or other generative models.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We have introduced all datasets with proper reference in the experiments part.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes two novel algorithms, FOSL and MemCS, for multi-block minimax bilevel optimization problems, avoiding the high complexity of computing the second-order gradient. Their theoretical analysis is quite solid and their experiments show proposed algorithms have superior performance and robustness in applications.\n\nOverall, the proposed algorithms are novel and well-justified with practical applications. However, further elaboration is needed regarding their practicality and efficiency.\n\nSTRENGTHS:\nThe paper is well-written and well-structured, with strong motivation and clear logic.\n\nThe authors provide a reformulated version of minimax bilevel optimization problems and demonstrate the gap between the reformulated and original problems. The convergence analysis is also solid and thorough. \n\nThey also validate the experimental results of FOSL and MemCS in Deep AUC Maximization and meta-learning cases respectively.\n\nWEAKNESSES:\nSince the paper primarily claims the efficiency of the proposed first-order algorithms, the authors should discuss the scale of problems these algorithms are suitable for. For example, second-order methods may fail when dealing with a large number of variables. In contrast, first-order algorithms generally handle larger-scale problems better, but the paper lacks experimental comparisons in this regard. \n\nBesides, the gradient calculation procedure is not mentioned in the paper. I believe it is quite important for incorporating optimization problems in neural networks.\n\nThere is also a typo in the title of algorithm 2: Cold-star\n\nQUESTIONS:\n- Since the theoretical proof is based on various assumptions, are there any restrictions on FOSL and MemCS? For example, do they only handle convex optimization problems? Including the previously mentioned issue of the optimization problem scale, I think the applicable scope of the two algorithms should be discussed in detail.\n\n- Could the authors include a comparison of the runtime differences between FOSL, MemCS, and second-order methods? This would better illustrate the effectiveness of the proposed methods.\n\n- In the experimental results, FOSL and MemCS perform better. Is this because these two algorithms have a better gradient estimator? The authors should provide an explanation for this.\n\n- Additionally, I would like to know how the gradient of the optimization problem is calculated. Is it from implicit differentiation, unrolling methods, or are there any direct analytical solutions of the gradient?\n\nIf my concerns are well addressed, I would be happy to raise my score.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-06-23"}

comments: SUMMARY:
This work proposes two novel fully first-order algorithms, named FOSL and MemCS, for multi-block minimax bilevel optimization problems. Specifically, the authors reformulate the lower-level problem as a value-function-based constraint and transform the minimax bilevel optimization into a surrogate minimax problem. FOSL and MemCS are proposed to solve the surrogate minimax problem by alternately updating the parameters through SGD. Theoretical analysis of the convergence is conducted and extensive experiments on deep AUC maximization and rank-based robust meta-learning show the effectiveness of the proposed method.

STRENGTHS:
The proposed method is sound and efficient.

WEAKNESSES:
1. I have concerns about the soundness of reformulating the minimax bilevel optimization as a minimax problem in Eq. 2. I hope the authors can conduct more analysis or provide some references that utilize the same technique.

2. I have a question about Proposition 4.9. Will it guarantee that $E|| y_{t+1}-y^*(x_{t+1}) || - E|| y_{t}-y^*(x_{t}) || \le 0$?

QUESTIONS:
Please see the weakness.

LIMITATIONS:
No potential negative societal impact.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-09

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies the multi-block minimax bilevel optimization problem. To address the high computation costs and high memory consumption issues of existing algorithms, this work proposes two fully first-order algorithms, i.e., FOSL and MemCS. Specifically, the authors convert the original minimax bilevel problem into a simple minimax problem and propose FOSL (first-order single-loop algorithm) to solve the optimization problem efficiently. Then, they introduce MemCS with cold-start initialization, which avoids storing the weights of blocks and reduces memory consumption. Moreover, the authors also provide convergence analysis for the proposed algorithms. Experimental results on the benchmark datasets show that the proposed method can achieve better or comparable performance than the baseline.\n\nSTRENGTHS:\n1. The authors convert the original minimax bilevel problem into a simple minimax problem and solve it by first-order single-loop algorithm.\n2. The authors provide a comprehensive convergence analysis on the proposed method.\n3. The proposed method is easy to converge and outperforms the baselines on CIFAR-100, CelebA and OGBG-MolPCBA.\n\nWEAKNESSES:\n1. AUC-CT avoids the calculation of second-order matrices and shows comparable efficiency to the proposed first-order algorithm, which reduces the contribution of this work to improving algorithm efficiency. It would be better to provide more discussions.\n2. It would be better to provide the memory consumption of baselines for better comparisons.\n3. The proposed method only achieves comparable performance to mAUC-CT on CheXpert. It would be better to provide more analysis.\n\nQUESTIONS:\nMy main concern is about the contribution of this work to computational efficiency (especially compared to AUC-CT).\n\nLIMITATIONS:\nI cannot find the discussion about the limitations of this work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

comments: SUMMARY:
The paper introduces FOSL and MemCS that are two new first-order algorithms for multi-block minimax bilevel optimization, demonstrating superior sample complexities and robust performance in empirical evaluations on deep AUC maximization and robust meta-learning applications.

STRENGTHS:
The paper introduces FOSL, a novel fully first-order single-loop algorithm for minimax bilevel optimization, simplifying the problem structure and achieving competitive sample complexity without second-order computations. MemCS is proposed as a memory-efficient method with cold-start initialization, addressing challenges in scenarios with numerous blocks and demonstrating improved sample complexity compared to traditional methods.

WEAKNESSES:
1. The assumptions look quite strong, e.g., Assumption 5.3 requires strong convexity, Assumption 5.4 requires several boundnesses on the derivatives and high-order derivatives, which usually are not satisfied in practice.  
2. It appears that the memory costs have not been compared.

QUESTIONS:
See weakness.

LIMITATIONS:
Not discussed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-13