SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} = \| \nabla_ {x} f (x ^ {k}, y ^ {k + 1}) + \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k + 1}) w ^ {k + 1} - \nabla_ {x} f (x ^ {k}, y ^ {k + 1}) - \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k + 1}) w ^ {\dagger} (x ^ {k}, y ^ {k + 1}) \| \\ = \| \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k + 1}) \left(w ^ {k + 1} - w ^ {\dagger} \left(x ^ {k}, y ^ {k + 1}\right)\right) \| \\ \end{array}
$$
$$
\begin{array}{l} \leq \| \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k + 1}) \| \| w ^ {k + 1} - w ^ {\dagger} (x ^ {k}, y ^ {k + 1}) \| \\ \leq \ell_ {g, 1} \| w ^ {k + 1} - w ^ {\dagger} \left(x ^ {k}, y ^ {k + 1}\right) \| = \ell_ {g, 1} b _ {k}. \\ \end{array}
$$
Rearranging terms yields the conclusion.
Lemma 13. Under Assumption 1-2, we have
$$
F \left(x ^ {k}, y ^ {k + 1}; w ^ {\dagger} \left(x ^ {k}, y ^ {k + 1}\right)\right) - F \left(x ^ {k}, y ^ {k + 1}; w ^ {\dagger} \left(x ^ {k}, y ^ {k}\right)\right) \leq \frac {2 \beta N L _ {w} \ell_ {g , 1} ^ {2}}{\mu_ {g}} \mathcal {R} _ {y} \left(x ^ {k}, y ^ {k}\right) \tag {36}
$$
Proof. Letting  $y_{i}^{*} = \mathrm{Proj}_{S(x^{k})}(y^{k,i})$ , we first bound  $\| y^{k + 1} - y^k\| ^2$  by
$$
\begin{array}{l} \left\| y ^ {k + 1} - y ^ {k} \right\| ^ {2} = \left\| y ^ {k, N} - y ^ {k, N - 1} + y ^ {k, N - 1} - y ^ {k, N - 2} + \dots - y ^ {k, 0} \right\| ^ {2} \\ \leq N \sum_ {i = 0} ^ {N - 1} \| y ^ {k, i + 1} - y ^ {k, i} \| ^ {2} = \beta^ {2} N \sum_ {i = 0} ^ {N - 1} \| \nabla_ {y} g (x ^ {k}, y ^ {k, i}) \| ^ {2} \\ \stackrel {(a)} {=} \beta^ {2} N \sum_ {i = 0} ^ {N - 1} \| \nabla_ {y} g (x ^ {k}, y ^ {k, i}) - \nabla_ {y} g (x ^ {k}, y _ {i} ^ {*}) \| ^ {2} \\ \leq \beta^ {2} N \ell_ {g, 1} ^ {2} \sum_ {i = 0} ^ {N - 1} \| y ^ {k, i} - y _ {i} ^ {*} \| ^ {2} = \beta^ {2} N \ell_ {g, 1} ^ {2} \sum_ {i = 0} ^ {N - 1} d (y ^ {k, i}, S (x ^ {k})) ^ {2} \\ \stackrel {(b)} {\leq} \frac {2 \beta^ {2} N \ell_ {g , 1} ^ {2}}{\mu_ {g}} \sum_ {i = 0} ^ {N - 1} \left(g \left(x ^ {k}, y ^ {k, i}\right) - g ^ {*} \left(x ^ {k}\right)\right) \\ \stackrel {(c)} {\leq} \frac {2 \beta^ {2} N \ell_ {g , 1} ^ {2}}{\mu_ {g}} \sum_ {i = 0} ^ {N - 1} (1 - \beta \mu_ {g}) ^ {i} \left(g \left(x ^ {k}, y ^ {k}\right) - g ^ {*} \left(x ^ {k}\right)\right) \\ \leq \frac {2 \beta^ {2} N ^ {2} \ell_ {g , 1} ^ {2}}{\mu_ {g}} \left(g \left(x ^ {k}, y ^ {k}\right) - g ^ {*} \left(x ^ {k}\right)\right) = \frac {2 \beta^ {2} N ^ {2} \ell_ {g , 1} ^ {2}}{\mu_ {g}} \mathcal {R} _ {y} \left(x ^ {k}, y ^ {k}\right). \tag {37} \\ \end{array}
$$
where (a) comes from  $\nabla_y g(x^k, y_i^*) = 0$ , (b) is due to quadratic growth condition in Lemma 7, (c) is derived from the contraction in (17a).
Then, according to the definition of  $F(x,y;w) = f(x,y) + w^{\top}\nabla_{y}g(x,y)$ , it holds that
$$
\begin{array}{l} F \left(x ^ {k}, y ^ {k + 1}; w ^ {\dagger} \left(x ^ {k}, y ^ {k + 1}\right)\right) - F \left(x ^ {k}, y ^ {k + 1}; w ^ {\dagger} \left(x ^ {k}, y ^ {k}\right)\right) \\ = f \left(x ^ {k}, y ^ {k + 1}\right) + \left\langle \nabla_ {y} g \left(x ^ {k}, y ^ {k + 1}\right), w ^ {\dagger} \left(x ^ {k}, y ^ {k + 1}\right) \right\rangle - f \left(x ^ {k}, y ^ {k + 1}\right) - \left\langle \nabla_ {y} g \left(x ^ {k}, y ^ {k + 1}\right), w ^ {\dagger} \left(x ^ {k}, y ^ {k}\right) \right\rangle \\ = \langle \nabla_ {y} g (x ^ {k}, y ^ {k + 1}), w ^ {\dagger} (x ^ {k}, y ^ {k + 1}) - w ^ {\dagger} (x ^ {k}, y ^ {k}) \rangle \\ \leq \frac {\beta N L _ {w}}{2} \| \nabla_ {y} g (x ^ {k}, y ^ {k + 1}) \| ^ {2} + \frac {L _ {w}}{2 \beta N} \| y ^ {k + 1} - y ^ {k} \| ^ {2} \\ \stackrel {(a)} {\leq} \frac {2 \beta N L _ {w} \ell_ {g , 1} ^ {2}}{\mu_ {g}} \mathcal {R} _ {y} (x ^ {k}, y ^ {k}) \\ \end{array}
$$
where (a) is resulting from (34) and (37).
Lemma 14. Under Assumption 1-2, we have
$$
F \left(x ^ {k}, y ^ {k + 1}; w ^ {\dagger} \left(x ^ {k}, y ^ {k}\right)\right) - F \left(x ^ {k}, y ^ {k}; w ^ {\dagger} \left(x ^ {k}, y ^ {k}\right)\right) \leq \frac {2 \beta \ell_ {f , 0} \ell_ {g , 2} + L _ {F} \ell_ {g , 1} ^ {2} \beta^ {2}}{\mu_ {g}} \mathcal {R} _ {y} \left(x ^ {k}, y ^ {k}\right) \tag {38}
$$
Proof. We can expand  $F(x, y; w)$  with respect to  $y$  by
$$
\begin{array}{l} F \left(x ^ {k}, y ^ {k + 1}; w ^ {\dagger} \left(x ^ {k}, y ^ {k}\right)\right) \\ \leq F (x ^ {k}, y ^ {k}; w ^ {\dagger} (x ^ {k}, y ^ {k})) + \langle \nabla_ {y} f (x ^ {k}, y ^ {k}) + \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) w ^ {\dagger} (x ^ {k}, y ^ {k}), y ^ {k + 1} - y ^ {k} \rangle + \frac {L _ {F}}{2} \| y ^ {k + 1} - y ^ {k} \| ^ {2} \\ \end{array}
$$
$$
\begin{array}{l} = F (x ^ {k}, y ^ {k}; w ^ {\dagger} (x ^ {k}, y ^ {k})) - \beta \langle \nabla_ {y} f (x ^ {k}, y ^ {k}) + \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) w ^ {\dagger} (x ^ {k}, y ^ {k}), \nabla_ {y} g (x ^ {k}, y ^ {k}) \rangle + \frac {L _ {F}}{2} \beta^ {2} \| \nabla_ {y} g (x ^ {k}, y ^ {k}) \| ^ {2} \\ \stackrel {(a)} {\leq} F (x ^ {k}, y ^ {k}; w ^ {\dagger} (x ^ {k}, y ^ {k})) + \frac {L _ {F} \ell_ {g , 1} ^ {2} \beta^ {2}}{\mu_ {g}} \mathcal {R} _ {y} (x ^ {k}, y ^ {k}) \underbrace {- \beta \langle \nabla_ {y} f (x ^ {k} , y ^ {k}) + \nabla_ {y y} ^ {2} g (x ^ {k} , y ^ {k}) w ^ {\dagger} (x ^ {k} , y ^ {k}) , \nabla_ {y} g (x ^ {k} , y ^ {k}) \rangle} _ {J _ {2}} \\ \end{array}
$$
where (a) results from (37). Letting  $y^{*} = \arg \min_{y\in S(x^{k})}\| y - y^{k}\|$ , the bound of  $J_{2}$  is derived by
$$
J _ {2} \stackrel {(a)} {=} - \beta \langle \nabla_ {y} f (x ^ {k}, y ^ {k}) + \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) w ^ {\dagger} (x ^ {k}, y ^ {k}), \nabla_ {y} g (x ^ {k}, y ^ {k}) - \nabla_ {y} g (x ^ {k}, y ^ {*}) \rangle
$$
$$
\begin{array}{l} \stackrel {(b)} {=} - \beta \langle \nabla_ {y} f (x ^ {k}, y ^ {k}) + \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) w ^ {\dagger} (x ^ {k}, y ^ {k}), \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {\prime}) (y ^ {k} - y ^ {*}) \rangle \\ = - \beta \left\langle \nabla_ {y} f \left(x ^ {k}, y ^ {k}\right) + \nabla_ {y y} ^ {2} g \left(x ^ {k}, y ^ {k}\right) w ^ {\dagger} \left(x ^ {k}, y ^ {k}\right), \nabla_ {y y} ^ {2} g \left(x ^ {k}, y ^ {k}\right) \left(y ^ {k} - y ^ {*}\right) \right\rangle \\ + \beta \langle \nabla_ {y} f (x ^ {k}, y ^ {k}) + \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) w ^ {\dagger} (x ^ {k}, y ^ {k}), (\nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) - \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {\prime})) (y ^ {k} - y ^ {*}) \rangle \\ \leq - \beta \langle \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) (\nabla_ {y} f (x ^ {k}, y ^ {k}) + \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) w ^ {\dagger} (x ^ {k}, y ^ {k})) , y ^ {k} - y ^ {*} \rangle \\ + \beta \| \nabla_ {y} f (x ^ {k}, y ^ {k}) + \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) w ^ {\dagger} (x ^ {k}, y ^ {k}) \| \| \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) - \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {\prime}) \| \| y ^ {k} - y ^ {*} \| \\ \end{array}
$$
$$
\begin{array}{l} \stackrel {(c)} {\leq} - \beta \langle \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) (\nabla_ {y} f (x ^ {k}, y ^ {k}) + \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) w ^ {\dagger} (x ^ {k}, y ^ {k}))  , y ^ {k} - y ^ {*} \rangle + \beta \ell_ {f, 0} \ell_ {g, 2} \| y ^ {k} - y ^ {*} \| ^ {2} \\ \stackrel {(d)} {\leq} \beta \ell_ {f, 0} \ell_ {g, 2} \| y ^ {k} - y ^ {*} \| ^ {2} = \beta \ell_ {f, 0} \ell_ {g, 2} d (y ^ {k}, S (x ^ {k})) ^ {2} \\ \leq \frac {2 \beta \ell_ {f , 0} \ell_ {g , 2}}{\mu_ {g}} \left(g \left(x ^ {k}, y ^ {k}\right) - g ^ {*} \left(x ^ {k}\right)\right) = \frac {2 \beta \ell_ {f , 0} \ell_ {g , 2}}{\mu_ {g}} \mathcal {R} _ {y} \left(x ^ {k}, y ^ {k}\right) \tag {39} \\ \end{array}
$$
where (a) is due to  $\nabla_y g(x^k, y^*) = 0$ , (b) is due to the mean value theorem and there exists  $t \in [0,1]$  such that  $y' = ty^k + (1 - t)y^*$ , (c) is due to  $\|I - AA^\dagger\| \leq 1$  and
$$
\begin{array}{l} \| \nabla_ {y} f (x ^ {k}, y ^ {k}) + \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) w ^ {\dagger} (x ^ {k}, y ^ {k}) \| = \| (I - \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) (\nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k})) ^ {\dagger} \nabla_ {y} f (x ^ {k}, y ^ {k}) \| \\ \leq \left\| \nabla_ {y} f \left(x ^ {k}, y ^ {k}\right) \right\| \leq \ell_ {f, 0} \\ \end{array}
$$
and (d) comes from  $w^\dagger(x, y) = -\left(\nabla_{yy}^2 g(x^k, y^k)\right)^\dagger \nabla_y f(x, y)$  such that
$$
\begin{array}{l} \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) \left(\nabla_ {y} f (x ^ {k}, y ^ {k}) + \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) w ^ {\dagger} (x ^ {k}, y ^ {k})\right) \\ = \nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}) \underbrace {\left(I - \nabla_ {y y} ^ {2} g (x ^ {k} , y ^ {k}) \left(\nabla_ {y y} ^ {2} g (x ^ {k} , y ^ {k})\right) ^ {\dagger}\right) \nabla_ {y} f (x ^ {k} , y ^ {k})} _ {\in \mathrm {K e r} (\nabla_ {y y} ^ {2} g (x ^ {k}, y ^ {k}))} = 0. \\ \end{array}
$$
Here,  $I - \nabla_{yy}^2 g(x^k,y^k)\left(\nabla_{yy}^2 g(x^k,y^k)\right)^\dagger$  is the orthogonal projector onto  $\mathrm{Ker}(\nabla_{yy}^2 g(x^k,y^k))$ .
Finally, according to (31), Lemma 6 can be proved in the following way.
Proof. Adding (33), (35), (36) and (38), we get
$$
\begin{array}{l} F \left(x ^ {k + 1}, y ^ {k + 1}; w ^ {\dagger} \left(x ^ {k + 1}, y ^ {k + 1}\right)\right) - F \left(x ^ {k}, y ^ {k}; w ^ {\dagger} \left(x ^ {k}, y ^ {k}\right)\right) \\ \leq - \left(\frac {\alpha}{2} - \frac {L _ {w} ^ {2}}{2 \eta_ {2}} \frac {\alpha^ {2}}{\beta} - \frac {L _ {F}}{2} \alpha^ {2} - \ell_ {g, 1} L _ {w} \alpha^ {2}\right) \| d _ {x} ^ {k} \| ^ {2} + \frac {\alpha \ell_ {g , 1} ^ {2}}{2} b _ {k} ^ {2} \\ + \frac {\beta \eta_ {2} \ell_ {g , 1} ^ {2} + 2 \beta N L _ {w} \ell_ {g , 1} ^ {2} + 2 \beta \ell_ {f , 0} \ell_ {g , 2} + L _ {F} \ell_ {g , 1} ^ {2} \beta^ {2}}{\mu_ {g}} \mathcal {R} _ {y} \left(x ^ {k}, y ^ {k}\right). \tag {40} \\ \end{array}
$$
Then by choosing  $\alpha \leq \min \left\{\frac{1}{4L_F + 8L_w\ell_{g,1}},\frac{\beta\eta_2}{2L_w^2}\right\}$ , we yield the conclusion.
F Proof of Overall Descent of Lyapunov Function
Proof. According to (17a), (17b) and (29), we know
$$
\mathcal {R} _ {y} \left(x ^ {k + 1}, y ^ {k + 1}\right) - \mathcal {R} _ {y} \left(x ^ {k}, y ^ {k}\right) \leq \left(- \beta \mu_ {g} + \frac {\alpha \eta_ {1} \ell_ {g , 1}}{\mu_ {g}}\right) \mathcal {R} _ {y} \left(x ^ {k}, y ^ {k}\right) + \left(\frac {\alpha \ell_ {g , 1}}{2 \eta_ {1}} + \frac {\ell_ {g , 1} + L _ {g}}{2} \alpha^ {2}\right) \| d _ {x} ^ {k} \| ^ {2} \tag {41}
$$
Then since  $\mathbb{V}^k = F(x^k,y^k;w^\dagger (x^k,y^k)) + c\mathcal{R}_y(x^k,y^k)$ , then adding (40) and (41) yields
$$
\begin{array}{l} \mathbb {V} ^ {k + 1} - \mathbb {V} ^ {k} \leq - \left(\frac {\alpha}{2} - \frac {L _ {w} ^ {2}}{2 \eta_ {2}} \frac {\alpha^ {2}}{\beta} - \frac {L _ {F}}{2} \alpha^ {2} - \ell_ {g, 1} L _ {w} \alpha^ {2} - \frac {c \alpha \ell_ {g , 1}}{2 \eta_ {1}} - \frac {c (\ell_ {g , 1} + L _ {g})}{2} \alpha^ {2}\right) \| d _ {x} ^ {k} \| ^ {2} + \frac {\alpha \ell_ {g , 1} ^ {2}}{2} b _ {k} ^ {2} \\ - \left(c \beta \mu_ {g} - \frac {c \alpha \eta_ {1} \ell_ {g , 1} + \beta \eta_ {2} \ell_ {g , 1} ^ {2} + 2 \beta N L _ {w} \ell_ {g , 1} ^ {2} + 2 \beta \ell_ {f , 0} \ell_ {g , 2} + L _ {F} \ell_ {g , 1} ^ {2} \beta^ {2}}{\mu_ {g}}\right) \mathcal {R} _ {y} (x ^ {k}, y ^ {k}) \\ \stackrel {(a)} {=} - \left(\frac {\alpha}{4} - \frac {L _ {w} ^ {2}}{2 \eta_ {2}} \frac {\alpha^ {2}}{\beta} - \frac {L _ {F}}{2} \alpha^ {2} - \ell_ {g, 1} L _ {w} \alpha^ {2} - \frac {c (\ell_ {g , 1} + L _ {g})}{2} \alpha^ {2}\right) \| d _ {x} ^ {k} \| ^ {2} + \frac {\alpha \ell_ {g , 1} ^ {2}}{2} b _ {k} ^ {2} \\ - \left(c \beta \mu_ {g} - \frac {2 c ^ {2} \alpha \ell_ {g , 1} ^ {2} + \beta \eta_ {2} \ell_ {g , 1} ^ {2} + 2 \beta N L _ {w} \ell_ {g , 1} ^ {2} + 2 \beta \ell_ {f , 0} \ell_ {g , 2} + L _ {F} \ell_ {g , 1} ^ {2} \beta^ {2}}{\mu_ {g}}\right) \mathcal {R} _ {y} (x ^ {k}, y ^ {k}) \\ \stackrel {(b)} {=} - \left(\frac {\alpha}{4} - \frac {L _ {w} ^ {2}}{2 \eta_ {2}} \frac {\alpha^ {2}}{\beta} - \frac {L _ {F}}{2} \alpha^ {2} - \ell_ {g, 1} L _ {w} \alpha^ {2} - \frac {c \left(\ell_ {g , 1} + L _ {g}\right)}{2} \alpha^ {2}\right) \| d _ {x} ^ {k} \| ^ {2} + \frac {\alpha \ell_ {g , 1} ^ {2}}{2} b _ {k} ^ {2} \\ - \left(\frac {c \beta \mu_ {g}}{2} - \frac {2 c ^ {2} \alpha \ell_ {g , 1} ^ {2} + \beta \eta_ {2} \ell_ {g , 1} ^ {2} + L _ {F} \ell_ {g , 1} ^ {2} \beta^ {2}}{\mu_ {g}}\right) \mathcal {R} _ {y} (x ^ {k}, y ^ {k}) \\ \stackrel {(c)} {\leq} - \left(\frac {\alpha}{4} - \frac {L _ {w} ^ {2} \mu_ {g} ^ {2}}{1 6 \eta_ {2} \ell_ {g , 1} ^ {2}} \alpha - \frac {L _ {F}}{2} \alpha^ {2} - \ell_ {g, 1} L _ {w} \alpha^ {2} - \frac {c (\ell_ {g , 1} + L _ {g})}{2} \alpha^ {2}\right) \| d _ {x} ^ {k} \| ^ {2} + \frac {\alpha \ell_ {g , 1} ^ {2}}{2} b _ {k} ^ {2} \\ - \left(\frac {c \beta \mu_ {g}}{4} - \frac {\beta \eta_ {2} \ell_ {g , 1} ^ {2} + L _ {F} \ell_ {g , 1} ^ {2} \beta^ {2}}{\mu_ {g}}\right) \mathcal {R} _ {y} \left(x ^ {k}, y ^ {k}\right) \\ \stackrel {(d)} {\leq} - \left(\frac {\alpha}{8} - \frac {L _ {F}}{2} \alpha^ {2} - \ell_ {g, 1} L _ {w} \alpha^ {2} - \frac {c \left(\ell_ {g , 1} + L _ {g}\right)}{2} \alpha^ {2}\right) \| d _ {x} ^ {k} \| ^ {2} + \frac {\alpha \ell_ {g , 1} ^ {2}}{2} b _ {k} ^ {2} \\ - \left(\frac {c \beta \mu_ {g}}{8} - \frac {L _ {F} \ell_ {g , 1} ^ {2} \beta^ {2}}{\mu_ {g}}\right) \mathcal {R} _ {y} (x ^ {k}, y ^ {k}) \\ \stackrel {(e)} {\leq} - \frac {\alpha}{1 6} \| d _ {x} ^ {k} \| ^ {2} - \frac {c \beta \mu_ {g}}{1 6} \mathcal {R} _ {y} \left(x ^ {k}, y ^ {k}\right) + \frac {\alpha \ell_ {g , 1} ^ {2}}{2} b _ {k} ^ {2}. \tag {42} \\ \end{array}
$$
where (a) comes from setting  $\eta_{1} = 2c\ell_{g,1}$ , (b) is achieved by  $c \geq \max \left\{\frac{8\ell_{g,2}\ell_{f,0}}{\mu_g^2}, \frac{8NL_w\ell_{g,1}^2}{\mu_g^2}\right\}$ , (c) is gained by letting  $\frac{c\alpha}{\beta} \leq \frac{\mu_g^2}{8\ell_{g,1}^2}$ , (d) is achieved by
$$
\eta_ {2} = \frac {L _ {w} ^ {2} \mu_ {g} ^ {2}}{2 c \ell_ {g , 1}}, \quad c \geq 2 L _ {w} \sqrt {\ell_ {g , 1}}
$$
and (e) is earned by
$$
\alpha \leq \frac {1}{8 \left(L _ {F} + 2 \ell_ {g , 1} L _ {w} + c \left(L _ {g} + \ell_ {g , 1}\right)\right)}, \quad \beta \leq \frac {c \mu_ {g} ^ {2}}{1 6 L _ {F} \ell_ {g , 1} ^ {2}}.
$$
In a word, a sufficient condition for (42) is
$$
\begin{array}{l} c = \max  \left\{\frac {8 \ell_ {g , 2} \ell_ {f , 0}}{\mu_ {g} ^ {2}}, \frac {8 N L _ {w} \ell_ {g , 1} ^ {2}}{\mu_ {g} ^ {2}}, 2 L _ {w} \sqrt {\ell_ {g , 1}} \right\}, \qquad \eta_ {1} = 2 c \ell_ {g, 1}, \qquad \eta_ {2} = \frac {L _ {w} ^ {2} \mu_ {g} ^ {2}}{2 c \ell_ {g , 1}} \\ \beta \leq \min  \left\{\frac {1}{\ell_ {g , 1}}, \frac {c \mu_ {g} ^ {2}}{1 6 L _ {F} \ell_ {g , 1} ^ {2}}, \frac {c \ell_ {g , 1} ^ {2}}{\mu_ {g} ^ {2} (L _ {F} + 2 \ell_ {g , 1} L _ {w} + c (L _ {g} + \ell_ {g , 1}))} \right\}, \quad \alpha \leq \frac {\mu_ {g} ^ {2}}{8 c \ell_ {g , 1} ^ {2}} \beta \\ \end{array}
$$
Then rearranging the terms and telescoping yield
$$
\frac {1}{K} \sum_ {k = 0} ^ {K - 1} \| d _ {x} ^ {k} \| ^ {2} \leq \frac {8 \ell_ {g , 1} ^ {2}}{K} \sum_ {k = 0} ^ {K - 1} b _ {k} ^ {2} + \frac {1 6 \mathbb {V} ^ {0}}{\alpha K} \stackrel {(a)} {\leq} 1 6 (1 - \rho \sigma_ {g} ^ {2}) ^ {T} \frac {\ell_ {f , 0} \ell_ {g , 1} ^ {2}}{\mu_ {g}} + \frac {1 6 \mathbb {V} ^ {0}}{\alpha K}
$$
$$
\frac {1}{K} \sum_ {k = 0} ^ {K - 1} \mathcal {R} _ {y} (x ^ {k}, y ^ {k}) \leq \frac {8 \ell_ {g , 1} ^ {2} \alpha}{c \beta \mu_ {g} K} \sum_ {k = 0} ^ {K - 1} b _ {k} ^ {2} + \frac {1 6 \mathbb {V} ^ {0}}{c \beta \mu_ {g} K} \stackrel {(a)} {\leq} 1 6 (1 - \rho \sigma_ {g} ^ {2}) ^ {T} \frac {\alpha \ell_ {f , 0} \ell_ {g , 1} ^ {2}}{c \beta \mu_ {g} ^ {2}} + \frac {1 6 \mathbb {V} ^ {0}}{c \beta \mu_ {g} K}
$$
where (a) comes from Lemma 5. Then by choosing  $T = \log(K)$ , we know that
$$
\frac {1}{K} \sum_ {k = 0} ^ {K - 1} \| d _ {x} ^ {k} \| ^ {2} \leq \mathcal {O} \left(\frac {1}{K}\right), \quad \frac {1}{K} \sum_ {k = 0} ^ {K - 1} \mathcal {R} _ {y} \left(x ^ {k}, y ^ {k}\right) \leq \mathcal {O} \left(\frac {1}{K}\right). \tag {43}
$$
Moreover, since  $\nabla_w\mathcal{L}(x^k,y^k,w)$  is  $\ell_{g,1}^{2}$ -Lipschitz smooth over  $w$  according to Proposition 2 and
$$
\nabla_ {w} \mathcal {L} \left(x ^ {k}, y ^ {k}, w ^ {\dagger} \left(x ^ {k}, y ^ {k}\right)\right) = 0
$$
then it holds that
$$
\begin{array}{l} \| \nabla_ {w} \mathcal {L} (x ^ {k}, y ^ {k}, w ^ {k}) \| ^ {2} = \| \nabla_ {w} \mathcal {L} (x ^ {k}, y ^ {k}, w ^ {k}) - \nabla_ {w} \mathcal {L} (x ^ {k}, y ^ {k}, w ^ {\dagger} (x ^ {k}, y ^ {k})) \| ^ {2} \\ \leq \ell_ {g, 1} ^ {2} \| w ^ {k} - w ^ {\dagger} (x ^ {k}, y ^ {k}) \| ^ {2} \\ \leq 2 \ell_ {g, 1} ^ {2} \| w ^ {k} - w ^ {\dagger} (x ^ {k - 1}, y ^ {k}) \| ^ {2} + 2 \ell_ {g, 1} ^ {2} \| w ^ {\dagger} (x ^ {k - 1}, y ^ {k}) - w ^ {\dagger} (x ^ {k}, y ^ {k}) \| ^ {2} \\ \leq 2 \ell_ {g, 1} ^ {2} b _ {k - 1} ^ {2} + 2 \ell_ {g, 1} ^ {2} L _ {w} ^ {2} \alpha^ {2} \| d _ {x} ^ {k - 1} \| ^ {2}. \\ \end{array}
$$
Therefore, averaging the left-hand-side and plugging in the gradient of  $\mathcal{L}$  yield,
$$
\begin{array}{l} \frac {1}{K} \sum_ {k = 0} ^ {K - 1} \mathcal {R} _ {w} \left(x ^ {k}, y ^ {k}, w ^ {k}\right) = \frac {1}{K} \sum_ {k = 0} ^ {K - 1} \| \nabla_ {w} \mathcal {L} \left(x ^ {k}, y ^ {k}, w ^ {k}\right) \| ^ {2} \\ \leq \mathcal {O} \left(\frac {1}{K}\right) + \frac {2 \ell_ {g , 1} ^ {2} L _ {w} ^ {2} \alpha^ {2}}{K} \sum_ {k = 0} ^ {K - 2} \| d _ {x} ^ {k} \| ^ {2} + \frac {1}{K} \| \nabla_ {w} \mathcal {L} (x ^ {0}, y ^ {0}, w ^ {0}) \| ^ {2} \\ \leq \mathcal {O} \left(\frac {1}{K}\right). \tag {44} \\ \end{array}
$$
where the last inequality is due to (43) and the boundedness of  $\| \nabla_w\mathcal{L}(x^0,y^0,w^0)\| ^2$ .
Similarly, we can bound
$$
\begin{array}{l} \| \nabla_ {x} f (x ^ {k}, y ^ {k}) + \nabla_ {x y} ^ {2} g (x ^ {k}, y ^ {k}) w ^ {k} \| ^ {2} \\ \leq 2 \| d _ {x} ^ {k - 1} \| ^ {2} + 2 \| \nabla_ {x} f (x ^ {k}, y ^ {k}) + \nabla_ {x y} ^ {2} g (x ^ {k}, y ^ {k}) w ^ {k} - d _ {x} ^ {k - 1} \| ^ {2} \\ \leq 2 \| d _ {x} ^ {k - 1} \| ^ {2} + 4 \| \nabla_ {x} f (x ^ {k}, y ^ {k}) - \nabla_ {x} f (x ^ {k - 1}, y ^ {k}) \| ^ {2} + 4 \| \nabla_ {x y} ^ {2} g (x ^ {k}, y ^ {k}) - \nabla_ {x y} ^ {2} g (x ^ {k - 1}, y ^ {k}) \| ^ {2} \| w ^ {k} \| ^ {2} \\ \leq 2 \| d _ {x} ^ {k - 1} \| ^ {2} + 4 \ell_ {f, 1} ^ {2} \alpha^ {2} \| d _ {x} ^ {k - 1} \| ^ {2} + 4 \ell_ {g, 2} ^ {2} \alpha^ {2} \| d _ {x} ^ {k - 1} \| ^ {2} \| w ^ {k} \| ^ {2} \\ \end{array}
$$
$$
\begin{array}{l} \stackrel {(a)} {\leq} 2 \| d _ {x} ^ {k - 1} \| ^ {2} + 4 \ell_ {f, 1} ^ {2} \alpha^ {2} \| d _ {x} ^ {k - 1} \| ^ {2} + 4 \ell_ {g, 2} ^ {2} \alpha^ {2} \| d _ {x} ^ {k - 1} \| ^ {2} \left(\frac {2 \ell_ {f , 0} ^ {2}}{\sigma_ {g} ^ {2}} + \mathcal {O} \left(\frac {1}{K}\right)\right) \\ \leq \left(2 + 4 \ell_ {f, 1} ^ {2} \alpha^ {2} + 4 \ell_ {g, 2} ^ {2} \alpha^ {2} \left(\frac {2 \ell_ {f , 0} ^ {2}}{\sigma_ {g} ^ {2}} + \mathcal {O} \left(\frac {1}{K}\right)\right)\right) \| d _ {x} ^ {k - 1} \| ^ {2} \\ \end{array}
$$
where (a) comes from  $\| w^{k}\|^{2}\leq 2\| w^{\dagger}(x^{k - 1},y^{k})\|^{2} + 2b_{k - 1}^{2}\leq \frac{2\ell_{f,0}^{2}}{\sigma_{g}^{2}} +\mathcal{O}(1 / K)$ . Therefore, using (43), we know
$$
\frac {1}{K} \sum_ {k = 0} ^ {K - 1} \mathcal {R} _ {x} \left(x ^ {k}, y ^ {k}, w ^ {k}\right) = \frac {1}{K} \sum_ {k = 0} ^ {K - 1} \left\| \nabla_ {x} f \left(x ^ {k}, y ^ {k}\right) + \nabla_ {x y} g \left(x ^ {k}, y ^ {k}\right) w ^ {k} \right\| ^ {2} \leq \mathcal {O} \left(\frac {1}{K}\right). \tag {45}
$$
G Additional Experiments and Details
Synthetic experiments. We test our algorithm GALET on Example 1 with different initialization. To see whether our stationary metric is a necessary condition of the optimality of BLO, we need to
define a value directly reflects whether the algorithm achieves the global optimality or not. By the explicit form of the solution set in (7), the global optimality gap can be measured by
$$
\left\| x - 0. 5 \right\| ^ {2} + \left\| 0. 5 + y _ {1} - \sin (y _ {2}) \right\| ^ {2} \tag {46}
$$
where the first term corresponds the distance to the global optimal  $x^{*} = 0.5$  and the second term represents LL optimality. GALET achieves the global minimizers in (7) regardless of the initialization; see the top plot in Figure 1. The bottom plot in Figure 1 shows the stationary score of reformulation (2) and (3). If the KKT conditions of (2) hold, then it is equivalent to say the following
$$
\nabla_ {x} f (x, y) = 0, \quad \nabla_ {y} f (x, y) = 0, \quad g (x, y) - g ^ {*} (x) = 0. \tag {47}
$$
whose proof is attached below.
Proof. First, the KKT conditions of (2) can be written as there exists  $\sigma^{*}$  such that
$$
\nabla_ {x} f \left(x ^ {*}, y ^ {*}\right) + \sigma^ {*} \left(\nabla_ {x} g \left(x ^ {*}, y ^ {*}\right) - \nabla g ^ {*} \left(x ^ {*}\right)\right) = 0 \tag {48a}
$$
$$
\nabla_ {y} f \left(x ^ {*}, y ^ {*}\right) + \sigma^ {*} \nabla_ {y} g \left(x ^ {*}, y ^ {*}\right) = 0 \tag {48b}
$$
$$
g \left(x ^ {*}, y ^ {*}\right) - g ^ {*} \left(x ^ {*}\right) = 0 \tag {48c}
$$
(48c) implies  $y^{*} \in S(x^{*})$ . For PL function  $g(x,y)$ , we know that if  $y^{*} \in S(x^{*})$ , then  $\nabla_y g(x^*,y^*) = 0$  and according to Lemma 8, we have  $\nabla_x g(x^*,y^*) - \nabla_x g^* (x^*) = 0$ . Therefore, (48a) and (48b) are reduced to
$$
\nabla_ {x} f (x ^ {*}, y ^ {*}) = 0, \quad \nabla_ {y} f (x ^ {*}, y ^ {*}) = 0
$$
Therefore, we can use the stationary score defined by  $\| \nabla_x f(x,y)\|^2 +\| \nabla_x f(x,y)\|^2 +(g(x,y) - g^* (x))$  to measure whether the point achieves the KKT point of (2). For reformulation (3), our stationary measure is simply defined as the summation of the residual in (8). For both initializations, the stationary score of the value function-based reformulation (2) does not reach 0, whereas our measure based on the gradient-based reformulation (3) does. As the optimality gap of our algorithm converges to 0, this implies the KKT condition for the value function-based reformulation (2) does not serve as a necessary condition for optimality. In contrast, our stationary metric does, thereby validating our theoretical findings in Section 2.
We also test the sensitivity of the parameters of our method and show the trajectories starting from the same initialization in Figure 4. We conduct experiments on different parameter and the search grid is: UL stepsize  $\alpha \in \{0.1, 0.3, 0.5, 0.7, 0.9\}$ ; shadow implicit gradient loops and stepsizes  $T \in \{1, 5, 50\}$ ,  $\gamma \in \{0.1, 0.5\}$  and LL stepsizes and loops  $N \in \{1, 5\}$ ,  $\beta \in \{0.1, 0.5, 1, 3\}$ . The default choice of parameter is  $\alpha = 0.3$ ,  $K = 30$ ,  $\beta = 1$ ,  $N = 1$ ,  $\gamma = 0.1$ ,  $T = 1$ . We observe that our algorithm is least sensitive to the shadow implicit gradient level parameters since all of the combinations of  $T$  and  $\gamma$  lead to almost the same trajectory. For the LL parameters  $\beta$  and  $N$ , our algorithm is also robust as long as the learning rate  $\beta$  is neither set too large ( $\beta = 3$ , divergence) nor too small ( $\beta = 0.1$ , slow convergence). The UL stepsize plays an important role on the convergence of GALET as different choices of  $\alpha$  lead to distinct paths of GALET. The choices of  $\alpha = 0.1, 0.3, 0.5$  all lead to the convergence of GALET, while a slightly larger value ( $\alpha = 0.5$ ) causes GALET to prioritize optimizing the UL variable  $x$  first.
Real-data experiments. We compare our method with the existing methods on the data hyper-cleaning task using the MNIST and the FashionMNIST dataset [21]. Data hyper-cleaning is to train a classifier in a corrupted setting where each label of training data is replaced by a random class number with a corruption rate  $p_c$  that can generalize well to the unseen clean data. Let  $x$  be a vector being trained to label the noisy data and  $y$  be the model weight and bias, the objective function is given by
$$
\min  _ {x, y \in S (x)} f \left(x, y ^ {*} (x)\right) \triangleq \frac {1}{| \mathcal {D} _ {\text {v a l}} |} \sum_ {\left(u _ {i}, v _ {i}\right) \in \mathcal {D} _ {\text {v a l}}} \operatorname {C E} \left(y ^ {*} (x); u _ {i}, v _ {i}\right)
$$
$$
\text{s.t.} y \in S (x) = \arg \min  _ {y} g (x, y) \triangleq \frac {1}{| \mathcal {D} _ {\mathrm {t r}} |} \sum_ {(u _ {i}, v _ {i}) \in \mathcal {D} _ {\mathrm {t r}}} [ \sigma (x) ] _ {i} \mathrm {C E} (y; u _ {i}, v _ {i})
$$