Paper: An Inexact Conditional Gradient Method for Constrained Bilevel Optimization (ICLR-2024)

Reviewer comment (Theory):
Theorems 1 and 2 give the convergence rate when the hyper-objective is convex and nonconvex. However, it is mentioned in the abstract and the conclusion that the convergence guarantees are achieved under the setting that the upper level objective is convex and nonconvex. The convexity of the hyper-objective and the upper level objective is not equivalent. Perhaps I have omitted something. How to make the connection between the function for the hyper-objective and the function for the upper level objective to derive the optimal convergence guarantee when the upper level objective is convex and nonconvex?

Author response:
**Q2 Theorems 1 and 2 give the convergence rate when $\ell$ is convex and nonconvex. However, it is mentioned in the abstract and the conclusion that the convergence guarantees are achieved under the setting that $f$ is convex and nonconvex.**

**A2** We apologize for the confusion. Indeed, the convexity assumption should be on the hyper-objective function $\ell(x)$, not on the upper-level function $f(x)$. We have revised the abstract and the conclusion sections to clarify this point.

Reviewer comment (Novelty):
My major concern is with the novelty and the contributions of the paper. First of all, the setting considered and the underlying ideas presented in the paper have already appeared in an earlier paper (Akhtar et al., 2022) where authors have proposed SBFW. In SBFW, the authors consider a constrained stochastic bilevel problem and develop conditional gradient algorithms for solving the problem. Secondly, the authors of (Akhtar et al., 2022) have considered stochastic problems which are certainly more challenging than the deterministic setting considered in this work. I believe although the authors have shown dependence on condition number and removed the boundedness assumption on the partial gradient of \(f\) w.r.t. \(y\) (by using compactness of the constraint set) the major contributions may not be enough. In addition, the approximation of the product of Hessian inverse and partial gradient by solving a quadratic problem is quite popular in bilevel optimization and has already appeared in the AmIGO algorithm proposed in (Arbel et al ICLR 2022) and other papers.

Author response:
**A1**  While our IBCG method shares some similarities with the existing methods, such as SBFW and AmIGO, we believe that our paper makes a notable contribution to the study of constrained bilevel optimization. Specifically, our method offers the best-known convergence rate for the considered setting within the class of projection-free algorithms, as shown in Table 1 of the paper. Moreover, to achieve such a result, we also need to address several specific technical challenges, which we highlight below. 

- Although some of the techniques used in our analysis may have appeared in prior work, our unique contribution lies in the way we combine and customize these techniques to develop a projection-free method for solving a constrained bilevel optimization problem with a strongly convex lower-level objective function. By tracking lower-level optimal solution trajectories, utilizing a nested approximation involving Hessian inverse information, and employing a projection-free approach, we have proposed a more efficient and streamlined algorithm that performs well in the numerical experiments.

- Specifically, one key distinction of our IBCG method from SBFW and AmIGO is the way we approximate the Hessian matrix inversion in the hyper-gradient in (5). In our approach, we obtain $\tilde{\mathbf{v}}(x_k)$ by performing one step of gradient descent on the quadratic function in (8). This leads to a single-loop algorithm that only requires two matrix-vector products per iteration. On the other hand, SBFW constructs a biased stochastic estimate of the Hessian inverse matrix (as shown in (12) in [R1]) and requires $O(\kappa_g)$ matrix-vector products per iteration. Moreover, AmIGO obtains $\tilde{\mathbf{v}}(x_k)$ by running $T = O(\kappa_g)$ steps of gradient descent (see Theorem 1 in [R2]), which also incurs $O(\kappa_g)$ matrix-vector products per iteration.
As a result of this simpler algorithmic scheme, our analysis is fundamentally different from those in prior work. For instance, Lemma 6 in our paper is new in the context of projection-free algorithms, where we upper bound the error of approximating $\textbf{v}({\bf x}\_k)$ by ${\bf w}\_{k+1}$.

Due to the space limit in our initial submission, we have focused on tackling the major issues. For the camera-ready version, benefiting from an extra page, we intend to add a related remark on specific technical challenges.

[R1] Akhtar, Zeeshan, et al. ”Projection-free stochastic bi-level optimization.” IEEE Transactions on
Signal Processing 70 (2022): 6332-6347.

[R2] Arbel, Michael, and Julien Mairal. "Amortized implicit differentiation for stochastic bilevel optimization." arXiv preprint arXiv:2111.14580 (2021).

Reviewer comment (Experiments):
Other than matrix completion and toy problems it would be interesting to see the performance of IBCG on large-scale machine learning problems. Will the algorithm be able to compete with stochastic algorithms in that case since the considered algorithm only works for the deterministic setting?

Author response:
**A2**  In our experiments, we have found that our proposed algorithm is competitive with stochastic algorithms such as SBFW and TTSA in solving small to medium-scale problems. This may be attributed to the fact that the deterministic nature of our method allows it to explore the search space more systematically and thus converge to high-quality solutions in fewer iterations. On the other hand, we acknowledge that stochastic algorithms have an inherent advantage in handling large-scale problems due to subsampling, which can significantly reduce the computational cost and improve scalability. We would like to explore it in future work.

Reviewer comment (Writing):
Define \(\Omega_1\) and \(\Omega_2\) in equation (2).

Author response:
**A3** Following your suggestion, we have added the definition of $\Omega_1$ and $\Omega_2$ after equation (2).

Reviewer comment (Writing):
The definition of training and test data before equation (3) is not clear.

Author response:
**A4** Thanks for catching the typo! We have corrected it in the revised paper.

Reviewer comment (Writing):
What is \(\omega\) in the discussion before Section 3.1.

Author response:
**A5** Thanks for the question. Here, $\omega$ is the exponent of matrix multiplication in complexity theory, and the current best value is $\omega \approx 2.37$. We have clarified this point in the revised paper.

Reviewer comment (Writing):
Please make sure that \(\mathcal{O}\) and  \(\tilde{\mathcal{O}}\) are used consistently throughout the paper.

Author response:
**A6**  We will check the paper to ensure consistency.

Reviewer comment (Theory):
It is good to analyze the convergence in both nonconvex and convex bilevel setting, but it is unclear that when the composite function $l(x)$ can be convex. The only example I come up with is when $f(x,y^*(x))$ is jointly convex and $y^*(x)$ is linear in $x$, which means lower-level objective is quadratic in $y$. It would be better to explain some sufficient condition for $l(x)$ being convex. Also, as the convexity is required for $l(x)$, there are possibly some over-claims in the contribution and conclusion section saying that 'when upper-level objective $f$ is convex, ...'.

Author response:
**Q1  Example when $\ell(x)$ is convex, and over-claims in the contribution.**

**A1**  Thank you for raising this point! The reviewer is correct; we have made revisions to the contribution and conclusion sections to clarify that the convexity assumption is on the hyper-objective function $\ell(\mathbf{x})$, instead of the upper-level function $f(\mathbf{x})$.

Moreover, as the reviewer has mentioned, one sufficient condition for $\ell(\mathbf{x})$ being convex is when $f$ is jointly convex and $y^*(\mathbf{x})$ is linear in $\mathbf{x}$. As another example, we can consider a min-max optimization problem $\min_{\mathbf{x} \in \mathcal{X}} \max_{\mathbf{y} \in \mathbb{R}^m} f(\mathbf{x},\mathbf{y})$, where $f$ is convex in $\bf x$ and strongly-concave in $\bf y$. Note that this can also be reformulated as a bilevel optimization problem by letting $g({\bf x},{\bf y})=-f({\bf x},{\bf y})$, and the hyper-objective $\ell(\mathbf{x}) = \max_{\mathbf{y}} f(\mathbf{x},\mathbf{y})$ is indeed convex. On the other hand, as also noted by [R1], it seems that there are no general sufficient conditions to establish the convexity of $\ell$, and thus it needs to be verified on a case-by-case basis.  Thank you for raising this excellent point and we have added this discussion to the paper. 

[R1] Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. "A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and application to actor-critic." SIAM Journal on Optimization, 2023.

Reviewer comment (Theory):
Is it possible to extend the convergence analysis of IBCG to tackle the stochastic bilevel problem like SBFW?

Author response:
**Q2 Is it possible to extend the convergence analysis of IBCG to tackle the stochastic bilevel problem like SBFW?**

**A2** Thanks for raising this point! Extending the IBCG method to handle stochastic bilevel problems is indeed an interesting avenue for future work. We think the key challenge lies in controlling the approximation errors of $\\|y_k - y^*(x_k)\\|$ and $\\|w_{k+1}-v(x_k)\\|$, since the contraction properties in (22) and (24) would not hold due to stochastic noises. To tackle these issues, some of the techniques used in SBFW might be useful in this regard. As it is beyond the scope of this paper, we leave this for future work.

[R1] Akhtar, Zeeshan, et al. "Projection-free stochastic bi-level optimization." IEEE Transactions on Signal Processing 70 (2022): 6332-6347.

Reviewer comment (Experiments):
The experimental validation could be more complete. In particular, the method is applied only to low rank penalty, the only case (up to my knowledge) where Frank Wolfe shows a real benefit. What are some other possible applications of the method, where it would outperform accelerated proximal gradient? In other words, what are common sets such that FW's linear minimization operator can be computed efficiently, but projection cannot?

Author response:
**A1** Thanks for raising this point! Regarding first part of your comment, indeed, other than matrix completion problems, the application of our proposed method includes Model-Agnostic Meta-Learning (MAML) [R1] and Kernel Matrix Learning (KML) [R2], to name a few. In MAML, the goal is to develop a flexible model that can effectively adapt to multiple training sets in order to optimize performance for individual tasks -- see section 2.1 in our paper. It is worth noting that saddle point problems can be viewed as a special case of bilevel optimization problems by letting $g({\bf x},{\bf y})=-f({\bf x},{\bf y})$. Therefore, many examples including KML fit into our setting. In this example, the upper-level constraint set is $\mathcal K\triangleq \lbrace {\bf K}\succeq 0\mid \mbox{trace}({\bf K})=r \rbrace$ for some $r>0$. In fact, the projection onto set $\mathcal K$ requires the eigenvalue decomposition of an $n\times n$ matrix, thus incurring an $\mathcal O(n^3)$ complexity per iteration while the Linear Minimization Oracle (LMO) over $\mathcal K$ has a per-iteration complexity $\tilde{\mathcal O}(N_K/\sqrt{\delta})$ where $N_K$ denotes the number of non-zero entries and $\delta$ is the desired accuracy, as elaborated in Table 1 of [R3]. 

Regarding the second part of your comment, a particularly compelling illustration of the benefits of projection-free techniques is found in the spectrahedron case. Here, rather than needing to calculate a complete singular value decomposition for projections, one can simply compute the principal eigenvector for the LMO, such as through the Lanczos algorithm. There are some sets where the FW algorithm's LMO can be computed efficiently, but projection cannot, or is much harder, including $\ell_p$-norm ball, the nuclear norm ball (also called spectrahedron), matrix trace norm ball,  matrix max-norm ball, Birkhoff polytope and some other examples that can be found in Table 1 of [R3] . Due to the page limit, We have addressed the major issues raised by the reviewers in the revised manuscript. However, we plan to provide a more detailed discussion on this topic in the camera-ready version where we have an extra page.

[R1] Finn, Chelsea, Pieter Abbeel, and Sergey Levine. ”Model-agnostic meta-learning for fast adaptation
of deep networks.” ICML, 2017.

[R2] Lanckriet, Gert RG, et al. "Learning the kernel matrix with semidefinite programming." Journal of Machine learning research 5.Jan (2004): 27-72.

[R3] Braun, G., Carderera, A., Combettes, C. W., Hassani, H., Karbasi, A., Mokhtari, A., and Pokutta, S. (2022). Conditional gradient methods. arXiv preprint arXiv:2211.14103.

Reviewer comment (Experiments):
Some tricks exist when the upper optimization is constrained (eg, parametrizing the positive regularization hyperparameter as $\lambda = \exp(\mu)$ as in Pedregosa (2016), which may also improve numerical stability). How does the proposed method compare? when should one use a method or the other?

Author response:
**A3** 
Thanks for your question. The problem considered in Pedregosa (2016) indeed presents an intriguing formulation; however, it is tailored to a specific subclass of problems and does not encompass the broader spectrum of constrained bilevel optimization problems that our research addresses. In particular, in Pedregosa (2016), the following hyper-parameter optimization problem is considered:

$\min_{\lambda \in \mathcal{D}} \text{loss}(\mathcal{S}_{test}, X(\lambda))$

$\hbox {s.t} \quad X(\lambda) \in  arg\min_x \text{loss}(\mathcal{S}_{train}, x)+e^{\lambda}\||x\||^2$

In this framework, if the domain $\mathcal{D}$ is non-compact, the lower-level objective function may lack strong convexity, resulting in multiple solutions. This characteristic is crucial since bilevel optimization problems, where the lower level is merely convex, are generally intractable. The paper assumes a compact $\mathcal{D}$ with a bounded $\lambda \geq \epsilon$, ensuring strong convexity, however, without such compactness assumption convergence to an optimal solution cannot be guaranteed, potentially impacting the algorithm's stability and performance. Moreover, we would like to note that our proposed method can be implemented to the mentioned problem but due to the lack of sophisticated constraints at the upper level, we may find projection-based algorithms perform better.

Reviewer comment (Theory):
Given the practical focus of the paper, it seems far-fetched to not consider, above Section 3.1, that the cost of matrix matrix multiplication is $m^3$. Algorithms with lower exponents have huge constant terms and are not beneficial for the kind of $m$ that are dealt with in ML.

Author response:
**A4** That is an excellent point. As the reviewer has correctly pointed out, in most ML applications, the cost of matrix multiplication is  $\mathcal O(m^3)$, and the methods that achieve lower complexity of the form $\mathcal{O}(m^{\omega})$ where $ 2.37 \leq \omega < 3$ often ignore the large constant terms in their computational complexity. We will highlight this in the revised manuscript.