SlowGuess/NeurIPS_2023_OCR huggingface dataset:
A Generalized Alternating Method for Bilevel Optimization under the Polyak-Łojasiewicz Condition
Quan Xiao
Troy, NY, USA
xiaoq5@rpi.edu
Songtao Lu
Yorktown Heights, NY, USA
songtao@ibm.com
Tianyi Chen
chentianyi19@gmail.com
Bilevel optimization has recently regained interest owing to its applications in emerging machine learning fields such as hyperparameter optimization, meta-learning, and reinforcement learning. Recent results have shown that simple alternating (implicit) gradient-based algorithms can match the convergence rate of single-level gradient descent (GD) when addressing bilevel problems with a strongly convex lower-level objective. However, it remains unclear whether this result can be generalized to bilevel problems beyond this basic setting. In this paper, we first introduce a stationary metric for the considered bilevel problems, which generalizes the existing metric, for a nonconvex lower-level objective that satisfies the Polyak-Lojasiewicz (PL) condition. We then propose a Generalized ALternating mEthod for bilevel opTimization (GALET) tailored to BLO with convex PL LL problem and establish that GALET achieves an  $\epsilon$ -stationary point for the considered problem within  $\tilde{\mathcal{O}}(\epsilon^{-1})$  iterations, which matches the iteration complexity of GD for single-level smooth nonconvex problems.
Bilevel optimization (BLO) is a hierarchical optimization framework that aims to minimize the upper-level (UL) objective, which depends on the optimal solutions of the lower-level (LL) problem. Since its introduction in the 1970s [5], BLO has been extensively studied in operations research, mathematics, engineering, and economics communities [14], and has found applications in image processing [12] and wireless communications [8]. Recently, BLO has regained interests as a unified framework of modern machine-learning applications, including hyperparameter optimization [49, 21, 22, 54], meta-learning [20], representation learning [3], reinforcement learning [62, 59], continual learning [55, 4], adversarial learning [73] and neural architecture search [39]; see [41].
In this paper, we consider BLO in the following form
$$
\min  _ {x \in \mathbb {R} ^ {d _ {x}}, y \in S (x)} f (x, y) \quad \text {s . t .} S (x) \triangleq \underset {y \in \mathbb {R} ^ {d _ {y}}} {\arg \min } g (x, y) \tag {1}
$$
where both the UL objective  $f(x,y)$  and LL objective  $g(x,y)$  are differentiable, and the LL solution set  $S(x)$  is not necessarily a singleton. For ease of notation, we denote the optimal function value of the LL objective as  $g^{*}(x) \coloneqq \min_{y} g(x,y)$  and call it value function.
Although BLO is powerful for modeling hierarchical decision-making processes, solving generic BLO problems is known to be NP-hard due to their nested structure [31]. As a result, the majority of recent works in optimization theory of BLO algorithms are centered on nonconvex UL problems with strongly convex LL problems (nonconvex-strongly-convex), which permit the development of efficient algorithms; see e.g. [24, 27, 32, 9, 35, 10, 34, 67, 66]. The strong convexity assumption for LL ensures the uniqueness of the minimizer and a simple loss landscape of the LL problem, but it
GALET	V-PBGD	BOME	MGBiO	BGS	IAPTT-GM	IGFM
g(x,y)	PL + C	PL	PL	SC	Morse-Bott	Regular	PL + C
Non-singleton S(x)	✓	✓	✓	X	✓	✓	✓
Provable CQ	✓	Relaxed	X	/	/	Relaxed	/
Complexity	O(ε-1)	O(ε-1.5)	O(ε-4)	O(ε-1)	X	X	poly(ε-1)
Table 1: Comparison of the proposed method GALET with the existing BLO for non-strongly-convex LL problem (V-PBGD [57], BOME [37], MGBiO [28], BGS [2], IAPTT-GM [43], IGFM [7]). The notation  $\tilde{\mathcal{O}}$  omits the dependency on  $\log (\epsilon^{-1})$  terms and  $\mathrm{poly}(\epsilon^{-1})$  hides the dependency worse than  $\mathcal{O}(\epsilon^{-4})$ . 'C', 'SC' and 'Regular' stand for convex, strongly convex and Assumption 3.1 in [43], respectively. PL, Lipschitz Hessian and the assumption that eigenvalue bounded away from 0 in MGBiO imply SC. 'Relaxed' means that they solve a relaxed problem without CQ-valid issue and 'f' means that CQ is not needed as it is based on the implicit function theorem.
excludes many intriguing applications of BLO. In the context of machine learning, the LL objective might represent the training loss of a neural network, which can be non-strongly-convex [65].
To measure the efficiency of solving nonconvex BLO problems, it is essential to define its stationarity, which involves identifying a necessary condition for the optimality of BLO. In cases where the LL problem exhibits strongly convexity, the solution set  $S(x)$  becomes a singleton, leading to a natural definition of stationarity as the stationary point of the overall objective  $f(x, S(x))$ , i.e.,  $\nabla f(x, S(x)) = 0$ . However, for BLO problems with non-strongly-convex LL problems,  $S(x)$  may have multiple solutions, rendering  $\nabla f(x, S(x))$  ill-posed. This motivates an intriguing question:
Q1: What is a good metric of stationarity for BLO with nonconvex LL problems?
To address this question, we focus on the setting of BLO with the LL objective that satisfies the PL condition (nonconvex-PL). The PL condition not only encompasses the strongly convexity condition [24, 27, 32, 9, 35, 10, 34, 67, 66] and the Morse-Bott condition [2], but also covers many applications such as overparameterized neural networks [38], learning LQR models [19], and phase retrieval [61].
By reformulating the LL problem by its equivalent conditions, one can convert the BLO problem to a constrained optimization problem. Then with certain constraint qualification (CQ) conditions, a natural definition of the stationarity of the constrained optimization problem is the Karush-Kuhn-Tucker (KKT) point [11]. For example, constant rank CQ (CRCQ) was assumed in [37], and linear independence CQ (LICQ) was assumed or implied in [44, 28]. However, it is possible that none of these conditions hold for nonconvex-PL BLO (Section 2.2). In Section 2, we study different CQs on two constrained reformulations of (1) and then identify the best combination. Based on the right CQ on the right constrained reformulation, we prove the inherent CQ and propose a new notion of stationarity for the nonconvex-PL BLO, which strictly extends the existing measures in nonconvex-strongly-convex BLO [24, 27, 32, 9] and nonconvex-nonconvex BLO [2, 37, 28] without relaxing the problem [36, 57]. We emphasize the importance of defining new metric in Section 2.3.
Given a stationary metric, while  $\epsilon$ -stationary point can be found efficiently in  $\mathcal{O}(\epsilon^{-1})$  iterations for nonconvex and smooth single-level problem [6], existing works on the BLO with non-strongly-convex LL problem either lack complexity guarantee [2, 43, 36, 42], or occur slower rate [57, 37, 58, 7]. Moreover, most existing algorithms update the UL variable  $x$  after obtaining the LL parameter  $y$  sufficiently close to the optimal set  $S(x)$  by running GD from scratch, which is computationally expensive [37, 57]. In contrast, the most efficient algorithm for nonconvex-strongly-convex BLO updates  $x$  and  $y$  in an alternating manner, meaning that  $x$  is updated after a constant number of  $y$  updates from their previous values [9, 32]. Then another question arises:
Q2: Can alternating methods achieve the  $\mathcal{O}(\epsilon^{-1})$  complexity for non-strongly-convex BLO?
Addressing this question is far from trivial. First, we need to characterize the drifting error of  $S(x)$  induced by the alternating strategy, which involves the change in the LL solution sets between two consecutive UL iterations. However, we need to generalize the analysis in nonconvex-strongly-convex BLO [9, 32, 24, 27] because  $S(x)$  is not a singleton. Moreover, we need to select an appropriate Lyapunov function to characterize the UL descent, as the nature candidate  $f(x, S(x))$  is ill-posed without a unique LL minimizer. Finally, since the Lyapunov function we choose for UL contains both  $x$  and  $y$ , it is crucial to account for the drifting error of  $y$  as well.
By exploiting the smoothness of the value function  $g^{*}(x)$  and with the proper design of the Lyapunov function, we demonstrate the  $\tilde{\mathcal{O}} (\epsilon^{-1})$  iteration complexity of our algorithm, which is optimal in terms
of  $\epsilon$ . This result not only generalizes the convergence analysis in nonconvex-strongly-convex BLO [24, 27, 32, 9, 35, 10, 34, 67, 66] to the broader problem class, but also improves the complexity of existing works on nonconvex-non-strongly-convex BLO, specifically  $\tilde{\mathcal{O}} (\epsilon^{-1.5})$  in [57] and  $\tilde{\mathcal{O}} (\epsilon^{-4})$  in [37]; see Table 1. We present our algorithm in Section 3 and analyze its convergence in Section 4, followed by the simulations and conclusions in Section 5.
1.1 Related works
Nonconvex-strongly-convex BLO. The interest in developing efficient gradient-based methods and their nonasymptotic analysis for nonconvex-strongly-convex BLO has been invigorated by recent works [24, 32, 27, 9]. Based on the different UL gradient approximation techniques they use, these algorithms can be categorized into iterative differentiation and approximate implicit differentiation-based approaches. The iterative differentiation-based methods relax the LL problem by a dynamical system and use the automatic differentiation to approximate the UL gradient [21, 22, 25], while the approximate implicit differentiation-based methods utilize the implicit function theory and approximate the UL gradient by the Hessian inverse (e.g. Neumann series [9, 24, 27]; kernel-based methods [26]) or Hessian-vector production approximation methods (e.g. conjugate gradient descent [32, 54], gradient descent [35, 1]). Recent advances include variance reduction and momentum based methods [34, 67, 13]; warm-started BLO algorithms [1, 35]; distributed BLO approaches [63, 46, 68]; and algorithms solving BLO with constraints [64, 66]. Nevertheless, none of these attempts tackle the BLO beyond the strongly convex LL problem.
Nonconvex-nonconvex BLO. While nonconvex-strongly-convex BLO has been extensively studied in the literature, efficient algorithms for BLO with nonconvex LL problem remain under-explored. Among them, Liu et al. [43] developed a BLO method with initialization auxiliary and truncation of pessimistic trajectory; and Arbel and Mairal [2] generalized the implicit function theorem to a class of nonconvex LL functions and introduced a heuristic algorithm. However, these works primarily focus on analyzing the asymptotic performance of their algorithms, without providing finite-time convergence guarantees. Recently, Liu et al. [37] proposed a first-order method and established the first nonasymptotic analysis for non-strongly-convex BLO. Nonetheless, the assumptions such as CRCQ and bounded  $|f|, |g|$  are relatively restrictive. Huang [28] has proposed a momentum-based BLO algorithm, but the assumptions imply strongly convexity. Another research direction has addressed the nonconvex BLO problem by relaxation, such as adding regularization in the LL [42, 50], or replacing the LL optimal solution set with its  $\epsilon$ -optimal solutions [36, 57]. Although this relaxation strategy overcomes the CQ-invalid issues, it introduces errors in the original bilevel problem [7]. To the best of our knowledge, none of these BLO algorithms handling multiple LL solutions can achieve the optimal iteration complexity in terms of  $\epsilon$ .
Nonconvex-convex BLO. Another line of research focuses on the BLO with convex LL problem, which can be traced back to [48, 18]. Convex LL problems pose additional challenges of multiple LL solutions which hinder from using implicit-based approaches for nonconvex-strongly-convex BLO. To tackle multiple LL solutions, an aggregation approach was proposed in [56, 40]; a primal-dual algorithm was considered in [58]; a difference-of-convex constrained reformulated problem was explored in [72, 23]; an averaged multiplier method was proposed in [45]. Recently, Chen et al. [7] has pointed out that the objective of the non-strongly-convex LL problem can be discontinuous and proposed a zeroth-order smoothing-based method; Lu and Mei [47] have solved it by penalized min-max optimization. However, none of these attempts achieve the iteration complexity of  $\tilde{\mathcal{O}} (\epsilon^{-1})$ . Moreover, although some works adopted KKT related concept as stationary measure, they did not find the inherent CQ condition, so the necessity of their measure to the optimality of BLO is unknown [45, 47]. In this sense, our work is complementary to them. The comparison with closely related works is summarized in Table 1.
Notations. For any given matrix  $A \in \mathbb{R}^{d \times d}$ , we list the singular values of  $A$  in the increasing order as  $0 \leq \sigma_1(A) \leq \sigma_2(A) \leq \dots \leq \sigma_d(A)$  and denote the smallest positive singular value of  $A$  as  $\sigma_{\min}^+(A)$ . We also denote  $A^{-1}, A^\dagger, A^{1/2}$  and  $A^{-1/2}$  as the inverse of  $A$ , the Moore-Penrose inverse of  $A$  [29], the square root of  $A$  and the square root of the inverse of  $A$ , respectively.  $\operatorname{Ker}(A) = \{x: Ax = 0\}$ ,  $\operatorname{Ran}(A) = \{Ax\}$  denotes the null space and range space of  $A$ .
2 Stationarity Metric of Nonconvex-PL BLO
We will first introduce the equivalent constraint-reformulation of the nonconvex-PL BLO and then introduce our stationarity metric, followed by a section highlighting the importance of our results.
2.1 Equivalent constraint-reformulation of BLO
By viewing the LL problem as a constraint to the UL problem, BLO can be reformulated as a single-level nonlinear constrained optimization problem. Based on different equivalent characteristics of the LL problem, two major reformulations are commonly used in the literature [16]. The first approach is called value function-based reformulation, that is
$$
\min  _ {x \in \mathbb {R} ^ {d _ {x}}, y \in \mathbb {R} ^ {d _ {y}}} f (x, y) \quad \text {s . t .} g (x, y) - g ^ {*} (x) = 0. \tag {2}
$$
Clearly,  $g(x,y) - g^{*}(x) = 0$  is equivalent to  $y \in S(x)$  so that (2) is equivalent to (1).
On the other hand, recall the definition of PL function below, which is not necessarily strongly convex or even convex [33].
Definition 1 (PL condition). The function  $g(x, \cdot)$  satisfies the PL condition if there exists  $\mu_g > 0$  such that for any given  $x$ , it holds that  $\| \nabla_y g(x, y) \|^2 \geq 2 \mu_g (g(x, y) - g^*(x))$ ,  $\forall y$ .
According to Definition 1, for PL functions,  $\nabla_y g(x,y) = 0$  implies  $g(x,y) = g^{*}(x)$ . Therefore, the second approach replaces the LL problem with its stationary condition, that is
$$
\min  _ {x \in \mathbb {R} ^ {d _ {x}}, y \in \mathbb {R} ^ {d _ {y}}} f (x, y) \quad \text {s . t .} \nabla_ {y} g (x, y) = 0. \tag {3}
$$
We call (3) the gradient-based reformulation. The formal equivalence of (2) and (3) with (1) is established in Theorem 3 in Appendix.
For constrained optimization, the commonly used metric of quantifying the stationarity of the solutions are the KKT conditions. However, the local (resp. global) minimizers do not necessarily satisfy the KKT conditions [11]. To ensure the KKT conditions hold at local (resp. global) minimizer, one needs to assume CQ conditions, e.g., the Slater condition, LICQ, Mangasarian-Fromovitz constraint qualification (MFCQ), and the CRCQ [30]. Nevertheless, Ye and Zhu [70] have shown that none of these standard CQs are valid to the reformulation (2) for all types of BLO.
2.2 Stationarity metric
We will next establish the necessary condition for nonconvex-PL BLO via the calmness condition [11, Definition 6.4.1], which is weaker than the Slater condition, LICQ, MFCQ and CRCQ.
Definition 2 (Calmness). Let  $(x^{*},y^{*})$  be the global minimizer of the constrained problem
$$
\min  _ {x, y} f (x, y) \quad \text {s . t .} h (x, y) = 0. \tag {4}
$$
where  $h:\mathbb{R}^{d_x + d_y}\to \mathbb{R}^d$  and  $d\geq 1$  .If there exist positive  $\epsilon$  and  $M$  such that for any  $q\in \mathbb{R}^d$  with  $\| q\| \leq \epsilon$  and any  $\| (x',y') - (x^{*},y^{*})\| \leq \epsilon$  which satisfies  $h(x^{\prime},y^{\prime}) + q = 0$  , one has
$$
f \left(x ^ {\prime}, y ^ {\prime}\right) - f \left(x ^ {*}, y ^ {*}\right) + M \| q \| \geq 0 \tag {5}
$$
then the problem (4) is said to be calm with  $M$ .
The calmness of a problem quantifies the sensitivity of the objective to the constraints. Specifically, the calmness conditions of reformulations (2) and (3) are defined by setting  $h(x,y) = g(x,y) - g^{*}(x)$  and  $h(x,y) = g(x,y) + g^{*}(x)$ .
The calmness condition is the weakest CQ which can be implied by Slater condition, LICQ, MFCQ and CRCQ [11]. However, as we will show, even the calmness condition does not hold for the nonconvex-PL BLO when employing the value function-based reformulation (2).
Example 1. Considering  $x \in \mathbb{R}, y = [y_1, y_2]^\top \in \mathbb{R}^2$ , and the BLO problem as
$$
\min  _ {x, y} f (x, y) = x ^ {2} + y _ {1} - \sin (y _ {2}) \quad \text {s . t .} y \in \underset {y} {\arg \min } g (x, y) = \frac {1}{2} (x + y _ {1} - \sin (y _ {2})) ^ {2}. \tag {6}
$$
Optimality gap v.s. iterations
Stationary score v.s. iterations
Figure 1: Example 1 under different initialization. The solid and dashed lines represent two initialization in both plots. Top: the distance to the global optimal set measured by (46) v.s. iteration. Bottom: the stationary score of (2) and (3) v.s. iteration.
$y) = \nabla_{y}g(x,y)$  , respectively.
Lemma 1. Considering the BLO problem in Example 1, the LL objective satisfies the PL condition and the global minimizers of it are within the set
$$
\left\{\left(\bar {x}, \bar {y}\right) \mid \bar {x} = 0. 5, \bar {y} = \left[ \bar {y} _ {1}, \bar {y} _ {2} \right] ^ {\top}, 0. 5 + \bar {y} _ {1} - \sin \left(\bar {y} _ {2}\right) = 0 \right\} \tag {7}
$$
but the calmness condition of reformulation (2) is not satisfied under all of the global minimizers.
Figure 1 illustrates this fact by showing that the KKT score on the value function-based reformulation does not approach 0 as the distance to the optimal set decreases. This observation implies that the KKT conditions associated with the value function-based reformulation (2) do not constitute a set of necessary conditions for global minimizers for the nonconvex-PL BLO problems. As a result, the KKT point of (2) is unable to serve as the stationary point for the nonconvex-PL BLO problems.
Therefore, we adopt the gradient-based reformulation (3). Unfortunately, it is still possible that the standard CQs do not hold for some of the nonconvex-PL BLO problems associated with the gradient-based reformulation (3). To see this, let  $(x^{*},y^{*})$  be the global minimizer of (3). By denoting the matrix concatenating the Hessian and Jacobian as  $[\nabla_{yy}^2 g(x^*,y^*),\nabla_{yx}^2 g(x^*,y^*)]$ , the generic CQ conditions are instantiated in the gradient-based reformulation (3) by
- LICQ, MFCQ: The rows of  $[\nabla_{yy}^2 g(x^*, y^*), \nabla_{yx}^2 g(x^*, y^*)]$  are linearly independent; and,
- CRCQ:  $\exists$  neighborhood of  $(x^{*},y^{*})$  such that  $[\nabla_{yy}^{2}g(x,y),\nabla_{yx}^{2}g(x,y)]$  is of constant rank.
If  $g(x,y)$  is strongly convex over  $y$ , then  $\nabla_{yy}^2 g(x,y)$  is of full rank for any  $x$  and  $y$ , which ensures the LICQ, MFCQ and CRCQ of the gradient-based reformulation (3). However, if  $g(x,y)$  merely satisfies the PL condition like Example 1, none of the standard CQs hold, which is established next.
Lemma 2. Example 1 violates Slater condition, LICQ, MFCQ and CRCQ conditions of (3).
Remarkably, the following lemma demonstrates that the gradient-based reformulation of the nonconvex-PL BLO inherits the calmness condition.
Lemma 3 (Calmness of nonconvex-PL BLO). If  $g(x, \cdot)$  satisfies the PL condition and is smooth, and  $f(x, \cdot)$  is Lipschitz continuous, then (3) is calm at its global minimizer  $(x^{*}, y^{*})$ .
The Lipschitz smoothness of  $g(x,y)$  and Lipschitz continuity of  $f(x,y)$  over  $y$  are standard in BLO [9, 27, 24, 32, 35, 34, 1, 13, 37]. In this sense, nonconvex-PL BLO associated with the gradient-based reformulation (3) is naturally calm so that the KKT conditions are necessary conditions for its optimality [11]. A summary and illustration of our theory is shown in Figure 2.
To benefit the algorithm design (Section 3.2), we establish the necessary conditions of the optimality for nonconvex-PL BLO by slightly modifying the KKT conditions of (3) in the next theorem.
Theorem 1 (Necessary condition in nonconvex-PL BLO). If  $g(x, \cdot)$  satisfies the PL condition and is smooth, and  $f(x, \cdot)$  is Lipschitz continuous, then  $\exists w^* \neq 0$  such that
$$
\mathcal {R} _ {x} \left(x ^ {*}, y ^ {*}, w ^ {*}\right) := \left\| \nabla_ {x} f \left(x ^ {*}, y ^ {*}\right) + \nabla_ {x y} ^ {2} g \left(x ^ {*}, y ^ {*}\right) w ^ {*} \right\| ^ {2} = 0 \tag {8a}
$$
$$
\mathcal {R} _ {w} \left(x ^ {*}, y ^ {*}, w ^ {*}\right) := \left\| \nabla_ {y y} ^ {2} g \left(x ^ {*}, y ^ {*}\right) \left(\nabla_ {y} f \left(x ^ {*}, y ^ {*}\right) + \nabla_ {y y} ^ {2} g \left(x ^ {*}, y ^ {*}\right) w ^ {*}\right) \right\| ^ {2} = 0 \tag {8b}
$$
$$
\mathcal {R} _ {y} \left(x ^ {*}, y ^ {*}\right) := g \left(x ^ {*}, y ^ {*}\right) - g ^ {*} \left(x ^ {*}\right) = 0 \tag {8c}
$$
hold at the global minimizer  $(x^{*},y^{*})$  of (1).
This necessary condition is tight in the sense that it is a generalization of stationary measures in the existing literature for BLO with LL problem exhibiting strongly convexity [9, 27, 24, 32], PL with CRCQ [37], invertible Hessian and singleton solution [28] and Morse-Bott functions [2]. Thanks to the inherent calmness of PL BLO, our result eliminates the CRCQ condition in [37]. We next show the connections of our results with other works.
Nonconvex-strongly-convex BLO or PL with invertible Hessian and singleton solution. As  $S(x)$  is singleton and  $\nabla_{yy}g(x,y)$  is always non-singular, the solution to (8b) is uniquely given by
$$
w ^ {*} = - \left(\nabla_ {y y} ^ {2} g \left(x ^ {*}, S \left(x ^ {*}\right)\right)\right) ^ {- 1} \nabla_ {y} f \left(x ^ {*}, S \left(x ^ {*}\right)\right). \tag {9}
$$
Therefore, the necessary condition in (8) is equivalent to
$$
\nabla f (x ^ {*}, S (x ^ {*})) = \nabla_ {x} f (x ^ {*}, S (x ^ {*})) - \nabla_ {x y} ^ {2} g (x ^ {*}, S (x ^ {*})) \left(\nabla_ {y y} ^ {2} g (x ^ {*}, S (x ^ {*}))\right) ^ {- 1} \nabla_ {y} f (x ^ {*}, S (x ^ {*})) = 0
$$
where the first equality is obtained by the implicit function theorem. Therefore, (8) recovers the necessary condition  $\nabla f(x^{*},S(x^{*})) = 0$  for nonconvex-strongly-convex BLO.
Nonconvex-Morse-Bott BLO. Morse-Bott functions are special cases of PL functions [2]. In this case,  $\nabla_{yy}g(x,y)$  can be singular so that (8b) may have infinite many solutions, which are given by
$$
\mathcal {W} ^ {*} = - \left(\nabla_ {y y} ^ {2} g (x ^ {*}, y ^ {*})\right) ^ {\dagger} \nabla_ {y} f (x ^ {*}, y ^ {*}) + \mathrm {K e r} (\nabla_ {y y} ^ {2} g (x ^ {*}, y ^ {*})).
$$
According to [2, Proposition 6], for any  $y^{*} \in S(x^{*})$ ,  $\nabla_{xy}^{2}g(x^{*},y^{*}) \subset \mathrm{Ran}(\nabla_{yy}^{2}g(x^{*},y^{*}))$ , which is orthogonal to  $\mathrm{Ker}(\nabla_{yy}^{2}g(x^{*},y^{*}))$ . As a result, although the solution to (8b) is not unique, all of possible solutions yield the unique left hand side value of (8a). i.e.  $\forall w^{*} \in \mathcal{W}^{*}$ ,
$$
\nabla_ {x y} ^ {2} g (x ^ {*}, y ^ {*}) w ^ {*} = - \nabla_ {x y} ^ {2} g (x ^ {*}, y ^ {*}) \left(\nabla_ {y y} ^ {2} g (x ^ {*}, y ^ {*})\right) ^ {\dagger} \nabla_ {y} f (x ^ {*}, y ^ {*}).
$$
Plugging into (8a), we arrive at
$$
\nabla_ {x} f (x ^ {*}, y ^ {*}) \underbrace {- \nabla_ {x y} ^ {2} g (x ^ {*} , y ^ {*}) \left(\nabla_ {y y} ^ {2} g (x ^ {*} , y ^ {*})\right) ^ {\dagger}} _ {:= \phi (x ^ {*}, y ^ {*})} \nabla_ {y} f (x ^ {*}, y ^ {*}) = 0 \quad \text {a n d} \quad \nabla_ {y} g (x ^ {*}, y ^ {*}) = 0
$$
where  $\phi (x^{*},y^{*})$  is the same as the degenerated implicit differentiation in [2].