SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\left\| w ^ {r} - x ^ {*} \left(w ^ {r}, z ^ {r}\right) \right\| ^ {2} + \left\| z ^ {r} - y ^ {*} \left(w ^ {r}, z ^ {r}\right) \right\| ^ {2} \tag {166}
$$
$$
\begin{array}{l} \leq 4 \left(\left(\frac {1 + \eta (p - L)}{\eta (p - L)}\right) ^ {2} \| x ^ {r + 1} - x ^ {r} \| ^ {2} + \left(\frac {1 + \alpha (p - L)}{\alpha (p - L)}\right) ^ {2} \| y ^ {r + 1} - y ^ {r} \| ^ {2}\right) \\ + \frac {2}{\beta^ {2}} \left(\| w ^ {r + 1} - w ^ {r} \| ^ {2} + \| z ^ {r + 1} - z ^ {r} \| ^ {2}\right) \\ + 8 \left\| y ^ {*} \left(w ^ {r}, z ^ {r}; \lambda_ {+} ^ {r + 1} \left(w ^ {r}, z ^ {r}\right)\right) - \bar {y} ^ {*} \left(w ^ {r}, z ^ {r}\right) \right\| ^ {2} + 8 \left\| x ^ {*} \left(w ^ {r}, z ^ {r}; \lambda_ {+} ^ {r + 1} \left(w ^ {r}, z ^ {r}\right)\right) - \bar {x} ^ {*} \left(w ^ {r}, z ^ {r}\right) \right\| ^ {2} \\ + 1 6 \sigma_ {2} ^ {2} \tau^ {2} \left\| g \left(x ^ {*} \left(w ^ {r}, z ^ {r}; \lambda^ {r + 1}\right), y ^ {*} \left(w ^ {r}, z ^ {r}; \lambda^ {r + 1}\right)\right) - g \left(x ^ {*} \left(w ^ {r}, z ^ {r}; \lambda^ {r + 1}\right)\right) - \delta \right\| ^ {2} \tag {167} \\ \end{array}
$$
$$
\begin{array}{l} \leq 8 \left(4 \eta \left(\frac {1 + \eta (p - L)}{\eta (p - L)}\right) ^ {2} + 2 \alpha \left(\frac {1 + \alpha (p - L)}{\alpha (p - L)}\right) ^ {2} + \frac {8}{p \beta}\right) 3 ^ {2} \cdot 6 3 \cdot 3 6 ^ {2} p ^ {2} \zeta^ {2} \bar {\sigma} _ {\mathrm {w}} ^ {2} \\ + \left(8 \bar {\sigma} _ {\mathrm {w}} \tau + 1 6 \sigma_ {2} ^ {2} \tau^ {2} \cdot \left(3 \cdot 6 3 \cdot 3 6 p \zeta \bar {\sigma} _ {\mathrm {w}}\right)\right) \left(3 \cdot 6 3 \cdot 3 6 p \zeta \bar {\sigma} _ {\mathrm {w}}\right). \tag {168} \\ \end{array}
$$
Therefore, once
$$
\begin{array}{l} 8 \left(4 \eta \left(\frac {1 + \eta (p - L)}{\eta (p - L)}\right) ^ {2} + 2 \alpha \left(\frac {1 + \alpha (p - L)}{\alpha (p - L)}\right) ^ {2} + \frac {8}{p \beta}\right) 3 ^ {2} \cdot 6 3 \cdot 3 6 ^ {2} p ^ {2} \zeta^ {2} \bar {\sigma} _ {\mathrm {w}} ^ {2} \\ + \left(8 \bar {\sigma} _ {\mathrm {w}} \tau + 1 6 \sigma_ {2} ^ {2} \tau^ {2} \cdot \left(3 \cdot 6 3 \cdot 3 6 p \zeta \bar {\sigma} _ {\mathrm {w}}\right)\right) \left(3 \cdot 6 3 \cdot 3 6 p \zeta \bar {\sigma} _ {\mathrm {w}}\right) <   \varrho \sim \mathcal {O} (1), \tag {169} \\ \end{array}
$$
i.e.,  $\zeta$  is small, then, Lemma 8 holds automatically. We can simply choose
$$
\zeta \leq \sqrt {\frac {\varrho}{2}} \max  \left\{\frac {1}{\zeta_ {1}}, \frac {1}{\zeta_ {2}} \right\} \tag {170}
$$
$$
\zeta_ {1} = 3 \cdot 3 6 p \bar {\sigma} _ {\mathrm {w}} \sqrt {8 \left(4 \eta \left(\frac {1 + \eta (p - L)}{\eta (p - L)}\right) ^ {2} + 2 \alpha \left(\frac {1 + \alpha (p - L)}{\alpha (p - L)}\right) ^ {2} + \frac {8}{p \beta}\right) + 6 3 \cdot 1 6 \sigma_ {2} ^ {2} \tau^ {2}}, \tag {171a}
$$
$$
\zeta_ {2} = 8 \cdot 3 \cdot 6 3 \cdot 3 6 p \tau \bar {\sigma} _ {\mathrm {w}} ^ {2}. \tag {171b}
$$
Second, if  $36p\zeta \big(\| y^{*}(w^{r},z^{r};\lambda_{+}^{r + 1}(w^{r},z^{r})) - \bar{y}^{*}(w^{r},z^{r})\|^{2} + \| x^{*}(w^{r},z^{r};\lambda_{+}^{r + 1}(w^{r},z^{r})) - \bar{x}^{*}(w^{r},z^{r})\|^{2}\big)\leq \vartheta \tau$  , then we have the following results.
$$
\begin{array}{l} \left\| w ^ {r} - x ^ {*} \left(w ^ {r}, z ^ {r}\right) \right\| ^ {2} + \left\| z ^ {r} - y ^ {*} \left(w ^ {r}, z ^ {r}\right) \right\| ^ {2} \\ \leq 1 2 \left(\left(\frac {1 + \eta (p - L)}{\eta (p - L)}\right) ^ {2} \cdot 8 \eta + \left(\frac {1 + \alpha (p - L)}{\alpha (p - L)}\right) ^ {2} \cdot 4 \alpha\right) \vartheta \tau + \frac {4 8 \vartheta \tau}{\beta p} + \frac {2 \vartheta \tau}{9 \zeta p} + 6 3 \cdot 4 8 \sigma_ {2} ^ {2} \vartheta \tau^ {2}. \tag {172} \\ \end{array}
$$
It is obvious that when  $\vartheta$  is small, then the RHS of (172) is less than constant  $\varrho \sim \mathcal{O}(1)$ , i.e., when
$$
\vartheta \leq \frac {\varrho}{\tau \zeta_ {3}} \tag {173}
$$
$$
\zeta_ {3} = 1 2 \left(\left(\frac {1 + \eta (p - L)}{\eta (p - L)}\right) ^ {2} \cdot 8 \eta + \left(\frac {1 + \alpha (p - L)}{\alpha (p - L)}\right) ^ {2} \cdot 4 \alpha\right) + \frac {4 8}{\beta p} + \frac {2}{9 \zeta p} + 6 3 \cdot 4 8 \sigma_ {2} ^ {2} \tau , \tag {174}
$$
then, Lemma 8 holds automatically.
Given the strong error bound, substituting (103) into (141) yields
$$
\begin{array}{l} \mathcal {Q} ^ {r + 1} - \mathcal {Q} ^ {r} \\ \leq - \frac {1}{8 \eta} \| x ^ {r + 1} - x ^ {r} \| ^ {2} - \frac {1}{4 \alpha} \| y ^ {r + 1} - y ^ {r} \| ^ {2} - \frac {p}{4 \beta} \| z ^ {r + 1} - z ^ {r} \| ^ {2} - \frac {p}{4 \beta} \| w ^ {r + 1} - w ^ {r} \| ^ {2} \\ - \tau \left(\frac {1}{6 3} - 1 2 p \zeta (3 \sigma_ {\mathrm {s t r o n g}} ^ {2} + 4 \sigma_ {2} ^ {2}) \tau\right) | g (x ^ {*} (v ^ {r}; \lambda^ {r + 1})), y ^ {*} (v ^ {r}; \lambda^ {r + 1})) - g ^ {*} (x ^ {*} (v ^ {r}; \lambda^ {r + 1})) - \delta | ^ {2} \\ + \left(\mu_ {g} L _ {g} ^ {2} + 2 \ell_ {g} ^ {2}\right) \frac {D _ {\mathcal {S}}}{r ^ {2}} - \frac {1}{4 \tau} \| \lambda^ {r + 1} - \lambda^ {r} \| ^ {2}. \tag {175} \\ \end{array}
$$
Following (152), we also require
$$
\frac {1}{6 3} - 1 2 p \zeta \left(3 \sigma_ {\text {s t r o n g}} ^ {2} + 4 \sigma_ {2} ^ {2}\right) \tau) > \frac {1}{1 2 6} \tag {176}
$$
$$
\tau <   \frac {1}{1 2 6 \cdot 1 2 p \zeta \left(3 \sigma_ {\text {s t r o n g}} ^ {2} + 4 \sigma_ {2} ^ {2}\right)} <   \frac {1}{1 2 6 \cdot 4 8 p \zeta \max  \left\{\sigma_ {\text {s t r o n g}} ^ {2} , \sigma_ {2} ^ {2} \right\}}. \tag {177}
$$
When the conditions shown in (146), (148), (150), (152), and (177) are satisfied, we can obtain
$$
\begin{array}{l} \mathcal {Q} ^ {r + 1} - \mathcal {Q} ^ {r} \\ \leq - \frac {1}{8 \eta} \| x ^ {r + 1} - x ^ {r} \| ^ {2} - \frac {1}{4 \alpha} \| y ^ {r + 1} - y ^ {r} \| ^ {2} - \frac {p}{4 \beta} \| z ^ {r + 1} - z ^ {r} \| ^ {2} - \frac {p}{4 \beta} \| w ^ {r + 1} - w ^ {r} \| ^ {2} \\ - \frac {\tau}{1 2 6} \left| \right. g \left(x ^ {*} \left(v ^ {r}; \lambda^ {r + 1}\right)\right), y ^ {*} \left(v ^ {r}; \lambda^ {r + 1}\right)\left. \right) - g ^ {*} \left(x ^ {*} \left(v ^ {r}; \lambda^ {r + 1}\right)\right) - \delta \left. \right| ^ {2} \\ + \left(\mu_ {g} L _ {g} ^ {2} + 2 \ell_ {g} ^ {2}\right) \frac {D _ {\mathcal {S}}}{r ^ {2}} - \frac {1}{4 \tau} \| \lambda^ {r + 1} - \lambda^ {r} \| ^ {2}. \tag {178} \\ \end{array}
$$
B.6 Equivalence of KKT Points between the Saddle Point Problem and Constrained Optimization
Lemma 10. Consider the following two problems:
$$
\left(\mathrm {P} 1\right): \quad \min  _ {x \in \mathcal {X}, y, g (x, y) - g ^ {*} (x) \leq \delta} f (x, y) + \frac {p}{2} \| y - z \| ^ {2} + \frac {p}{2} \| x - w \| ^ {2}, \tag {179a}
$$
$$
\left(\mathrm {P} 2\right): \quad \min  _ {x \in \mathcal {X}, y} \max  _ {0 \leq \lambda \leq \Lambda} f (x, y) + \lambda \left(g (x, y) - g ^ {*} (x) - \delta\right) + \frac {p}{2} \| y - z \| ^ {2} + \frac {p}{2} \| x - w \| ^ {2}. \tag {179b}
$$
Let  $x^{*},y^{*},w^{*},z^{*},\lambda^{*}$  denote the KKT solutions of P2. If  $\lambda^{*}\neq \Lambda$  , then the KKT solutions of P2 are also the KKT solutions of P1.
Proof. It is easy to check that the primal stationary conditions of P1 and P2 are the same. Regarding the stationarity of dual variable  $\lambda$ , we can apply the saddle point theorem. Note that problem P2 is strongly convex w.r.t.  $y$  and concave w.r.t.  $y$  and both feasible sets of  $x, y$  are compact. Then, there exists a saddle point such that
$$
\lambda^ {*} \left(g \left(x ^ {*}, y ^ {*}\right) - g ^ {*} \left(x ^ {*}\right) - \delta\right) \geq \lambda \left(g \left(x ^ {*}, y ^ {*}\right) - g ^ {*} \left(x ^ {*}\right) - \delta\right), \tag {180}
$$
which gives  $\lambda^{*}$  in following three cases
$$
\lambda^ {*} = \left\{ \begin{array}{l l} \Lambda & \text {i f} g \left(x ^ {*}, y ^ {*}\right) - g ^ {*} \left(x ^ {*}\right) - \delta > 0, \\ \lambda & \text {i f} g \left(x ^ {*}, y ^ {*}\right) - g ^ {*} \left(x ^ {*}\right) - \delta = 0, \\ 0 & \text {i f} g \left(x ^ {*}, y ^ {*}\right) - g ^ {*} \left(x ^ {*}\right) - \delta <   0. \end{array} \right. \tag {181}
$$
If  $\lambda^{*}\neq \Lambda$  , we can obtain
$$
\lambda^ {*} \geq 0, \quad \lambda^ {*} (g \left(x ^ {*}, y ^ {*}\right) - g ^ {*} \left(x ^ {*}\right) - \delta) = 0, \tag {182}
$$
which are the KKT conditions of P1.
Given this result, we can know that  $P(w,z)$  defined in (10b) becomes
$$
P (w, z) = \min  _ {x \in \mathcal {X}, y, g (x, y) - g ^ {*} (x) \leq \delta} f (x, y) + \frac {p}{2} \| x - w \| ^ {2} + \frac {p}{2} \| y - z \| ^ {2}, \tag {183}
$$
if  $0 \leq \lambda < \Lambda$ . Therefore, when the iterates converge, if the dual variable is strictly less than  $\Lambda$ , the converged point is the primal-dual (KKT) solution of the original problem.
C Boundedness of Dual Variable, LL Variables, and Potential Function
C.1 Boundedness of  $\mathcal{Q}^r$
From (13), we know that
$$
\begin{array}{l} \mathcal {Q} ^ {r} \left(x ^ {r}, y ^ {r}, w ^ {r}, z ^ {r}; \lambda^ {r}\right) \\ = K \left(x ^ {r}, y ^ {r}, w ^ {r}, z ^ {r}; \lambda^ {r}\right) - 2 D \left(w ^ {r}, z ^ {r}; \lambda^ {r}\right) + 2 P \left(w ^ {r}, z ^ {r}\right) (184) \\ { } ^ { p } ( w ^ { r } , z ^ { r } ) + K ( x ^ { r } , y ^ { r } , w ^ { r } , z ^ { r } ; \lambda ^ { r } ) - D ( w ^ { r } , z ^ { r } ; \lambda ^ { r } ) + ( P ( w ^ { r } , z ^ { r } ) - D ( w ^ { r } , z ^ { r } ; \lambda ^ { r } ) ) (185) \\ \end{array}
$$
$$
\stackrel {(a)} {\geq} P \left(w ^ {r}, z ^ {r}\right) \geq \underline {{f}} \tag {186}
$$
where  $(a)$  holds due to 1)  $K(x^{r},y^{r},w^{r},z^{r};\lambda^{r}) - D(w^{r},z^{r};\lambda^{r})\geq 0$  based on the definition of  $D(w^{r},z^{r};\lambda^{r})$  and 2) note that  $P(w,z) = \min_{x\in \mathcal{X},y}\max_{0\leq \lambda \leq \Lambda}f(x,y) + \lambda (g(x,y) - g^{*}(x) - \delta) + \frac{p}{2}\| x - w\| ^2 +\frac{p}{2}\| y - z\| ^2$  and  $P(w^{r},z^{r}) - D(w^{r},z^{r};\lambda^{r})\geq 0$  , which is true because the minimax equality theorem [49, 50] holds when  $K(x,y,w,z;\lambda)$  is strongly convex in  $x,y$  and linear (concave) in  $\lambda$
C.2 Boundedness of Dual Variable
Lemma 11.
Under A1-A5, suppose that the sequence  $\{x^r, y^r, w^r, z^r, u^r, \lambda^r, \forall r\}$  is generated by SLM. Assume that  $\lambda^r \sim \mathcal{O}(1/\sqrt{\delta}), y^r \sim \mathcal{O}(1)$  are bounded. When  $0 < \delta < 1$ , choose  $p = \Theta(\lambda^{r+1})$ ,  $\tau = \mathcal{O}(1)$ ,  $\alpha = \mathcal{O}(1/\lambda^{r+1})$ ,  $\zeta, \beta = \Theta(\delta^{1.5})$  such that  $p > L$ ,  $\alpha \leq 1/L_K$ ,  $\beta \leq 1$ , then,  $\lambda^{r+1}$  will either be upper bounded by  $\Lambda$  for at most  $\mathcal{O}(1/\delta^{2.5})$  steps or there exists a constant  $c$  such that
$$
\lambda^ {r + 1} \leq \frac {c \ell_ {f}}{\sqrt {\delta}} \sim \mathcal {O} \left(\frac {1}{\sqrt {\delta}}\right), \quad \Lambda > \frac {c \ell_ {f}}{\sqrt {\delta}}. \tag {187}
$$
In addition, when  $\delta > 1$ , choose  $\beta = \mathcal{O}(1)$ , then,  $\lambda^{r + 1} = \mathcal{O}(1)$ .
Proof. From (7b), we have
$$
y ^ {r + 1} = y ^ {r} - \alpha \left(\nabla_ {y} f \left(x ^ {r}, y ^ {r}\right) + \lambda^ {r + 1} \nabla_ {y} g \left(x ^ {r}, y ^ {r}\right) + p \left(y ^ {r} - z ^ {r}\right)\right). \tag {188}
$$
Applying the triangle inequality gives
$$
\left\| \lambda^ {r + 1} \nabla_ {y} g \left(x ^ {r}, y ^ {r}\right) \right\| \leq \frac {1}{\alpha} \| y ^ {r + 1} - y ^ {r} \| + \| \nabla_ {y} f \left(x ^ {r}, y ^ {r}\right) \| + p \| y ^ {r} - z ^ {r} \|. \tag {189}
$$
If  $\lambda^{r + 1} < \lambda^r$ , then  $\lambda^{r + 1}$  is bounded immediately. Otherwise, note that
$$
\| \lambda^ {r + 1} \nabla_ {y} g (x ^ {r}, y ^ {r}) \|
$$
$$
\geq \lambda^ {r + 1} \sqrt {2 \mu_ {g} \left(g \left(x ^ {r} , y ^ {r}\right) - g \left(x ^ {r} , y ^ {*} \left(x ^ {r}\right)\right)\right)} \tag {190}
$$
$$
= \lambda^ {r + 1} \sqrt {2 \mu_ {g} \left(g \left(x ^ {r} , y ^ {r}\right) - g \left(x ^ {r} , u ^ {r}\right) + g \left(x ^ {r} , u ^ {r}\right) - g \left(x ^ {r} , y ^ {*} \left(x ^ {r}\right)\right)\right)} \tag {191}
$$
$$
\stackrel {(b)} {=} \lambda^ {r + 1} \sqrt {\frac {2 \mu_ {g}}{\tau} \left(\lambda^ {r + 1} - \lambda^ {r}\right) + \delta + g \left(x ^ {r} , u ^ {r}\right) - g \left(x ^ {r} , y ^ {*} \left(x ^ {r}\right)\right)} \tag {192}
$$
where  $(a)$  holds due to the PL condition and nonnegativity of  $\lambda$ , in  $(b)$  we use the rule of the dual variable update.
Then, we can obtain
$$
\lambda^ {r + 1} \leq \frac {\frac {1}{\alpha} \| y ^ {r + 1} - y ^ {r} \| + \| \nabla_ {y} f \left(x ^ {r} , y ^ {r}\right) \| + p \| y ^ {r} - z ^ {r} \|}{\sqrt {\frac {2 \mu_ {g}}{\tau} \left(\lambda^ {r + 1} - \lambda^ {r}\right) + \delta + g \left(x ^ {r} , u ^ {r}\right) - g \left(x ^ {r} , y ^ {*} \left(x ^ {r}\right)\right)}} \tag {193}
$$
$$
\stackrel {(a)} {\leq} \frac {\frac {1}{\alpha} \| y ^ {r + 1} - y ^ {r} \| + \| \nabla_ {y} f \left(x ^ {r} , y ^ {r}\right) \| + p \| y ^ {r} - z ^ {r} \|}{\sqrt {\frac {2 \mu_ {g}}{\tau} \left(\lambda^ {r + 1} - \lambda^ {r}\right) + \delta}} \tag {194}
$$
where  $(a)$  follows from the definition  $g^{*}(x^{r})$ , i.e.,  $g(x^{r},u^{r})\geq g(x^{r},y^{*}(x^{r}))$ . As  $\lambda^{r + 1} > \lambda^r$ , we can further have
$$
\lambda^ {r + 1} \leq \frac {\frac {1}{\alpha} \| y ^ {r + 1} - y ^ {r} \| + \| \nabla_ {y} f (x ^ {r} , y ^ {r}) \| + p \| y ^ {r} - z ^ {r} \|}{\sqrt {\delta}}. \tag {195}
$$
Note that the LHS of (153) decreases on the order of  $(\mathcal{Q}^1 - \underline{f}) / r$ , with the lower bound of this quantity being  $\vartheta \tau$ . Therefore, according to (153) and (155), we can know that the iterates will be either in Case 1 for a certain number of iterations (i.e., at most  $\mathcal{O}((\mathcal{Q}^1 - \underline{f}) / (\vartheta \tau))$  steps) or in Case 2. In Case 1, the projection operation (i.e.,  $\mathcal{P}_{+}[\cdot]$ ) can always ensure that  $\lambda^{r + 1}$  is bounded by  $\Lambda$  so that we can choose  $p > L$ . In the following, it can be seen that the dual variable is automatically bounded, implying that the projection of the iterates onto the upper bound will be always inactive when the algorithm is in Case 2 (i.e.,  $\lambda^r < \Lambda$ ).
In Case 2, when  $0 < \delta < 1$  and the following conditions are met
$$
\eta , \alpha = \mathcal {O} \left(\frac {1}{\lambda^ {r}}\right), p = \Theta \left(\lambda^ {r}\right), \beta = \mathcal {O} \left(\delta^ {1. 5}\right), \tau = \Theta (1), \tag {196}
$$
and assuming that  $\lambda^r = \mathcal{O}(1 / \sqrt{\delta})$ , we can have  $\bar{\sigma}_{\mathrm{w}} = \mathcal{O}(1)$  due to (157),  $\vartheta = \mathcal{O}(\delta^2)$  due to (153),  $\mathcal{Q}^1 = \mathcal{O}(1 / \sqrt{\delta})$ , and
$$
\left\| y ^ {r} - z ^ {r} \right\| ^ {2} \stackrel {(1 9 8)} {=} \mathcal {O} \left(\left\| y ^ {r + 1} - y ^ {r} \right\| ^ {2} + \left\| z ^ {r + 1} - z ^ {r} \right\| ^ {2} / \beta^ {2}\right) \stackrel {(1 6 1)} {\leq} \mathcal {O} (\delta), \tag {197a}
$$
$$
\left\| y ^ {r + 1} - y ^ {r} \right\| ^ {2} \stackrel {(1 6 1)} {\leq} \mathcal {O} (\delta^ {2}) \tag {197b}
$$
$$
\left\| y ^ {r} - y ^ {r + 1} + y ^ {r + 1} - z ^ {r} \right\| \leq \left\| y ^ {r + 1} - y ^ {r} \right\| + \frac {1}{\beta} \left\| z ^ {r + 1} - z ^ {r} \right\|. \tag {198}
$$
Then, (195) can be written as
$$
\lambda^ {r + 1} \leq \frac {\lambda^ {r + 1} \| y ^ {r + 1} - y ^ {r} \|}{\alpha_ {0} \sqrt {\delta}} + \frac {\ell_ {f}}{\sqrt {\delta}} + \frac {p _ {0} \lambda^ {r + 1} \| y ^ {r} - z ^ {r} \|}{\sqrt {\delta}} \tag {199}
$$
$$
p = p _ {0} \lambda^ {r + 1}, \quad \alpha = \frac {\alpha_ {0}}{\lambda^ {r + 1}}. \tag {200}
$$
By utilizing the step-sizes as specified in (200) and substituting (197) back to (195), it can be easily checked that the first term of (199) is  $\mathcal{O}(\delta^{1/2}\lambda^{r+1}/\alpha_0)$ , and the last term is  $\mathcal{O}(p_0\lambda^{r+1})$ . From (161), it can be seen that when  $\zeta$  is small (or equivalently  $\beta_0$  is small, where  $\beta \stackrel{(150)}{=} \beta_0\delta^{1.5}$ ), there must exist a constant  $c$  such that
$$
\lambda^ {r + 1} \leq \frac {c \ell_ {f}}{\sqrt {\delta}} \sim \mathcal {O} \left(\frac {1}{\sqrt {\delta}}\right). \tag {201}
$$
When  $\delta > 1$ , it is easy to check that when  $\beta, \tau = \mathcal{O}(1), p = \Omega(\lambda^{r+1})$ , we have  $\lambda^{r+1} = \mathcal{O}(1)$ .
Given this result and Lemma 10, we can set the tunable variable  $\Lambda$  as  $\Theta(1/\sqrt{\delta})$ . After running the algorithm for  $T$  steps, if the last dual variable is  $\Lambda$ , i.e., the projection of the last iterate on  $\Lambda$  is active, then, we need to increase  $\Lambda$  and regenerate the sequence.
C.3 Boundedness of  $y^r$ ,  $\bar{y}^*(w^r, z^r)$ , and  $y^*(w^r, z^r; \lambda^{r+1})$
Lemma 12.
Under A1-A5 and the local regularity condition, suppose that the sequence  $\{x^r, y^r, w^r, z^r, u^r, \lambda^r, \forall r\}$  is generated by SLM. Assume that  $y^r, \bar{y}^*(w^r, z^r)$  are bounded. Then, we have  $\bar{y}^*(w^{r+1}, z^{r+1}), y^*(w^r, z^r; \lambda^{r+1}), y^{r+1}$  are also bounded.
Proof. We prove these results by induction. First, we assume that  $y^r, \bar{y}^*(w^r, z^r)$  are bounded, which gives the (gradient) Lipschitz continuity of  $g(x, \cdot)$  at these points.
Let  $\Delta R^{r} \triangleq \mu_{g}(L_{g}^{2} + \ell_{g}^{2})D_{\mathcal{S}} / r^{2}, r \geq 1$ . Recall the bounded set level set assumption that let
$$
\psi (x, y) = f (x, y), x \in \mathcal {X}, y \in \{y | g (x, y) - g ^ {*} (x) \leq \delta \}. \tag {202}
$$
Under the assumption that  $y^r, \bar{y}^*(w^r, z^r)$  are bounded, we can have the descent of the potential function up to  $\Delta R^r$  that is a decreasing sequence. By the fact that  $\psi(\bar{x}^*(w^{r+1}, z^{r+1}), \bar{y}^*(w^{r+1}, z^{r+1})) \leq P(w^{r+1}, z^{r+1})$ , for any  $(x^1, y^1, w^1, z^1; \lambda^1)$ , there exists a constant  $R$  such that
$$
\{\bar {y} ^ {*} \left(w ^ {r + 1}, z ^ {r + 1}\right) | P \left(w ^ {r + 1}, z ^ {r + 1}\right) \leq \mathcal {Q} ^ {r + 1} + \Delta R ^ {1} \} \subseteq \mathcal {B} \left(R \left(x ^ {1}, y ^ {1}, z ^ {1}; \lambda^ {1}\right)\right) \tag {203}
$$
which gives that  $\bar{y}^{*}(w^{r + 1},z^{r + 1})$  is bounded.
Applying (14) shown in Lemma 2 gives
$$
\left\| y ^ {*} \left(x ^ {r}, z ^ {r}; \lambda_ {+} ^ {r + 1} \left(w ^ {r}, z ^ {r}\right)\right) - \bar {y} ^ {*} \left(w ^ {r}, z ^ {r}\right) \right\|
$$
$$
\stackrel {(1 4)} {\leq} \sqrt {\sigma_ {\text {w e a k}} \left\| \lambda^ {r + 1} - \lambda_ {+} ^ {r + 1} \left(w ^ {r} , z ^ {r}\right) \right\| \left\| \lambda \left(w ^ {r} , z ^ {r}\right) - \lambda_ {+} ^ {r + 1} \left(w ^ {r} , z ^ {r}\right) \right\|} \tag {204}
$$
$$
\stackrel {(1 5 7)} {\leq} \mathcal {O} \left(\sqrt {\bar {\sigma} _ {\mathrm {w}} \tau \| g \left(x ^ {*} \left(w ^ {r} , z ^ {r} ; \lambda^ {r + 1}\right) , y ^ {*} \left(w ^ {r} , z ^ {r} ; \lambda^ {r + 1}\right)\right) - g \left(x ^ {*} \left(w ^ {r} , z ^ {r} ; \lambda^ {r + 1}\right)\right) - \delta \|}\right) \tag {205}
$$
$$
\stackrel {(a)} {\leq} \mathcal {O} (\sqrt {\delta}) \tag {206}
$$
where in  $(a)$  we choose  $\tau = \mathcal{O}(1)$  when  $\delta > 1$ , so we have  $\lambda, \bar{\sigma}_{\mathrm{w}} = \mathcal{O}(1)$ , and we choose step-sizes according to (196) and also apply (158) when  $0 < \delta \leq 1$ . So, we can have that  $y^{*}(w^{r}, z^{r}; \lambda_{+}^{r + 1}(w^{r}, z^{r})) \sim \mathcal{O}(\sqrt{\delta})$  is bounded. Also, it can be checked that
$$
\left\| y ^ {*} (w ^ {r}, z ^ {r}; \lambda^ {r + 1}) - y ^ {*} (x ^ {r}, z ^ {r}; \lambda_ {+} ^ {r + 1} (w ^ {r}, z ^ {r})) \right\|
$$
$$
\stackrel {(2 8 a)} {\leq} \frac {p + L}{p - L} \| \lambda^ {r + 1} - \lambda_ {+} ^ {r + 1} \left(w ^ {r}, z ^ {r}\right) \| \tag {207}
$$
$$
\stackrel {(a)} {\leq} \frac {p + L}{p - L} \tau \| g \left(x ^ {*} \left(v ^ {r}; \lambda^ {r + 1}\right), y ^ {*} \left(v ^ {r}; \lambda^ {r + 1}\right)\right) - g ^ {*} \left(x ^ {*} \left(v ^ {r}; \lambda^ {r + 1}\right)\right) - \delta) \|
$$
$$
\stackrel {(1 5 5)} {\leq} \mathcal {O} (\tau \delta) \sim \mathcal {O} (1) \tag {208}
$$
where  $(a)$  holds due to the nonexpansiveness of the nonnegative projection operator. Therefore,  $y^{*}(w^{r},z^{r};\lambda^{r + 1})$  is bounded.
Note that  $K(x, \cdot, z; \lambda^{r+1})$  is strongly convex with modulus  $p - L$  and gradient Lipschitz continuous with parameter  $p + L$ . From [51, Theorem 3.5.], we have
$$
\left\| y ^ {r + 1} - y ^ {*} \left(w ^ {r}, z ^ {r}; \lambda^ {r + 1}\right) \right\| \leq \left(1 - \frac {p - L}{p + L}\right) \left\| y ^ {r} - y ^ {*} \left(w ^ {r}, z ^ {r}; \lambda^ {r + 1}\right) \right\|, \tag {209}
$$
which directly gives the boundedness of  $y^{r + 1}$ .
D Theoretical Convergence Results
Proof. KKT Conditions.
For  $x, y, w, z, \lambda$ , we define  $\mathcal{F}$  as a map such that  $\mathcal{F}(x, y, w, z, u, \lambda) = (x^{+}, y^{+}, w^{+}, z^{+}, u^{+}, \lambda^{+})$  where  $(x^{+}, y^{+}, w^{+}, z^{+}, u^{+}, \lambda^{+})$  is the next iteration of Algorithm 1. It can be easily checked that the map  $\mathcal{F}$  is continuous and if  $x, y, w, z, u, \lambda$  is a fixed point of  $\mathcal{F}$ , i.e.,  $\mathcal{F}(x, y, w, z, u, \lambda) = (x, y, w, z, u, \lambda)$ , then  $(x, y, \lambda)$  is a pair of primal-dual stationary solutions of problem (1). Suppose that
$$
\left(x ^ {r}, y ^ {r}, w ^ {r}, z ^ {r}, u ^ {r}, \lambda^ {r}\right)\rightarrow (\bar {x}, \bar {y}, \bar {w}, \bar {z}, \bar {u}, \bar {\lambda}) \text {a l o n g a s u b s e q u e n c e} r \in \mathcal {T}. \tag {210}
$$
Notice the lower boundedness of  $\mathcal{Q}^r$  shown in (186) and Lemma 9, we can get
$$
\left\| x ^ {r + 1} - x ^ {r} \right\|\rightarrow 0, \quad \left\| y ^ {r + 1} - y ^ {r} \right\|\rightarrow 0, \quad \left\| z ^ {r + 1} - z ^ {r} \right\|\rightarrow 0 \tag {211}
$$
$$
\left\| g \left(x ^ {*} \left(w ^ {r}, z ^ {r}; \lambda^ {r + 1}\right), y ^ {*} \left(w ^ {r}, z ^ {r}; \lambda^ {r + 1}\right)\right) - g ^ {*} \left(x ^ {*} \left(w ^ {r}, z ^ {r}; \lambda^ {r + 1}\right)\right) - \delta \right\|\rightarrow 0, | g \left(x ^ {r}, u ^ {r}\right) - g ^ {*} \left(x ^ {r}\right) | \rightarrow 0,
$$
which gives  $\| x^{r + 1} - w^r\| \to 0,\| y^{r + 1} - z^r\| \to 0,\| \lambda^{r + 1} - \lambda^r\| \to 0$  , and further implies
$$
\left\|\left(x ^ {r + 1}, y ^ {r + 1}, w ^ {r + 1}, z ^ {r + 1}, u ^ {r + 1}, \lambda^ {r + 1}\right) - \left(x ^ {r}, y ^ {r}, w ^ {r}, z ^ {r}, u ^ {r}, \lambda^ {r}\right)\right\|\rightarrow 0. \tag {212}
$$
$$
\begin{array}{l} \| \mathcal {F} (\bar {x}, \bar {y}, \bar {w}, \bar {z}, \bar {u}, \bar {\lambda}) - (\bar {x}, \bar {y}, \bar {w}, \bar {z}, \bar {u}, \bar {\lambda}) \| \\ \stackrel {(a)} {=} \lim  _ {t \rightarrow \infty , t \in \mathcal {T}} \| \left(x ^ {r}, y ^ {r}, w ^ {r}, z ^ {r}, u ^ {r}, \lambda^ {r}\right) - \mathcal {F} \left(x ^ {r}, y ^ {r}, w ^ {r}, z ^ {r}, u ^ {r}, \lambda^ {r}\right) \| (213) \\ \stackrel {(b)} {=} \lim  _ {t \rightarrow \infty , t \in \mathcal {T}} \left\|\left(x ^ {r + 1}, y ^ {r + 1}, w ^ {r + 1}, z ^ {r + 1}, u ^ {r + 1}, \lambda^ {r + 1}\right) - \mathcal {F} \left(x ^ {r}, y ^ {r}, w ^ {r}, z ^ {r}, u ^ {r}, \lambda^ {r}\right)\right\| (214) \\ = 0 \\ \end{array}
$$
where  $(a)$  holds due to the continuity of  $\mathcal{F}$  and  $(b)$  follows from (212). Hence,  $(\bar{x},\bar{y},\bar{\lambda})$  satisfies
$$
\text {s t a t i o n a r i t y}: \quad \left\| \begin{array}{c} \frac {1}{\eta} (\bar {x} - \mathcal {P} _ {\mathcal {X}} (\bar {x} - \eta \nabla_ {x} \mathcal {L} (\bar {x}, \bar {y}; \bar {\lambda}))) \\ \nabla_ {y} \mathcal {L} (\bar {x}, \bar {y}; \bar {\lambda}) \end{array} \right\| = 0, \quad \bar {x} \in \mathcal {X}, \forall \eta > 0, \tag {215a}
$$
$$
\text {f e a s i b i l i t y}: \quad g (\bar {x}, \bar {y}) - g ^ {*} (\bar {x}) - \delta \leq 0, \bar {\lambda} \geq 0, \tag {215b}
$$
$$
\text {c o m p l e m e n t a r y} \quad \text {s l a c k n e s s}: \quad (g (\bar {x}, \bar {y}) - g ^ {*} (\bar {x}) - \delta) \bar {\lambda} = 0, \tag {215c}
$$
i.e., every limit point  $(\bar{x},\bar{y},\bar{\lambda})$  is a primal-dual stationary solution (KKT point) of problem (1).
Stationarity of Primal Variables. Recall