SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} + \frac {L _ {1} ^ {2} G _ {2}}{\mu_ {g} ^ {2}} \| x _ {t - 1} - x _ {t} \| ^ {2} + \left(1 - \frac {\alpha \mu_ {g}}{2}\right) \epsilon_ {t - 1} ^ {2} \\ = \left(1 - \frac {\alpha \mu_ {g}}{2}\right) \delta_ {t - 1} + G _ {2} \| y _ {t - 1} ^ {*} (x _ {t - 1}) - y _ {t} ^ {*} (x _ {t - 1}) \| ^ {2} + \frac {L _ {1} ^ {2} G _ {2}}{\mu_ {g} ^ {2}} \| x _ {t - 1} - x _ {t} \| ^ {2} \\ \leq \left(1 - \frac {\alpha \mu_ {g}}{2}\right) ^ {t - 1} \delta_ {1} + G _ {2} \sum_ {j = 0} ^ {t - 2} \left(1 - \frac {\alpha \mu_ {g}}{2}\right) ^ {j} \| y _ {t - 1 - j} ^ {*} (x _ {t - 1 - j}) - y _ {t - j} ^ {*} (x _ {t - 1 - j}) \| ^ {2} \\ + \frac {2 L _ {1} ^ {2} G _ {2}}{\mu_ {g} ^ {2}} \sum_ {j = 0} ^ {t - 2} \left(1 - \frac {\alpha \mu_ {g}}{2}\right) ^ {j} \| x _ {t - 1 - j} - x _ {t - j} \| ^ {2} \tag {10} \\ \end{array}
$$
where  $G_{2} = 2G_{1}(1 + \frac{2}{\alpha\mu_{g}})(1 - \alpha \mu_{g})$  and  $\delta_1 = \left(1 + c_2 + \frac{L_2^2(\rho\mu_g + DL_1^2 + DL_1\mu_g)^2}{L_1^2\mu_g^4}\right)\| y_2 - y_1^* (x_1)\|^2 +$ $\epsilon_1^2$
By substituting Equation (10) back into Equation (9), we can obtain that
$$
\begin{array}{l} \left\| \nabla f _ {t} \left(x _ {t}, y _ {t} ^ {*} \left(x _ {t}\right)\right) - \widehat {\nabla} f _ {t} \left(x _ {t}, y _ {t + 1}\right) \right\| ^ {2} \\ \leq 3 L _ {1} ^ {2} \left\{\left(1 - \frac {\alpha \mu_ {g}}{2}\right) ^ {t - 1} \delta_ {1} + G _ {2} \sum_ {j = 0} ^ {t - 2} \left(1 - \frac {\alpha \mu_ {g}}{2}\right) ^ {j} \| y _ {t - 1 - j} ^ {*} (x _ {t - 1 - j}) - y _ {t - j} ^ {*} (x _ {t - 1 - j}) \| ^ {2} \right. \\ \left. + \frac {2 L _ {1} ^ {2} G _ {2}}{\mu_ {g} ^ {2}} \sum_ {j = 0} ^ {t - 2} \left(1 - \frac {\alpha \mu_ {g}}{2}\right) ^ {j} \| x _ {t - 1 - j} - x _ {t - j} \| ^ {2} \right\}. \\ \end{array}
$$
H Proof of Theorem 5.7
Based on Lemma 2.2 in [14], we can have the following lemma to characterize the smoothness of the function  $f_{t}(x,y_{t}^{*}(x))$  w.r.t.  $x$  for any  $t\in [1,T]$ .
Lemma H.1. Suppose that Assumption 5.1 and Assumption 5.2 hold. Then for any  $t \in [1, T]$ , the function  $f_t(x, y_t^*)(x)$  is  $L_f$ -smooth, i.e., for any  $x$  and  $x'$ ,
$$
\left\| \nabla f _ {t} \left(x, y _ {t} ^ {*} (x)\right) - \nabla f _ {t} \left(x ^ {\prime}, y _ {t} ^ {*} \left(x ^ {\prime}\right)\right) \right\| \leq L _ {f} \| x - x ^ {\prime} \|,
$$
where the constant  $L_{f}$  is given by
$$
L _ {f} = L _ {1} + \frac {2 L _ {1} ^ {2} + L _ {0} ^ {2} L _ {2}}{\mu_ {g}} + \frac {L _ {1} ^ {3} + 2 L _ {0} L _ {1} L _ {2}}{\mu_ {g} ^ {2}} + \frac {L _ {0} L _ {1} ^ {2} L _ {2}}{\mu_ {g} ^ {3}}.
$$
For any  $t \in [1, T]$  we first define
$$
F _ {t, \eta} \left(x _ {t + 1}, y _ {t} ^ {*} \left(x _ {t + 1}\right)\right) = \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} f _ {t - i} \left(x _ {t + 1 - i}, y _ {t - i} ^ {*} \left(x _ {t + 1 - i}\right)\right).
$$
Based on Assumption 5.2, we can show the smoothness of  $F_{t,\eta}(x,y_t^* (x))$  w.r.t.  $x$ .
Lemma H.2. Suppose Assumption 5.2 holds. Then the following holds for function  $F_{t,\eta}(x_{t+1}, y_t^*(x_{t+1}))$ :
$$
F _ {t, \eta} (x _ {t + 1}, y _ {t} ^ {*} (x _ {t + 1})) - F _ {t, \eta} (x _ {t}, y _ {t} ^ {*} (x _ {t})) \leq \langle \nabla F _ {t, \eta} (x _ {t}, y _ {t} ^ {*} (x _ {t})), x _ {t + 1} - x _ {t} \rangle + \frac {L _ {f}}{2} \| x _ {t + 1} - x _ {t} \| ^ {2}.
$$
Proof. For any  $x$  and  $x'$ , we can know that
$$
\begin{array}{l} F _ {t, \eta} \left(x _ {t + 1}, y _ {t} ^ {*} \left(x _ {t + 1}\right)\right) - F _ {t, \eta} \left(x _ {t}, y _ {t} ^ {*} \left(x _ {t}\right)\right) \\ = \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} f _ {t - i} \left(x _ {t + 1 - i}, y _ {t - i} ^ {*} \left(x _ {t + 1 - i}\right)\right) - \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} f _ {t - i} \left(x _ {t - i}, y _ {t - i} ^ {*} \left(x _ {t - i}\right)\right) \\ = \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \left[ f _ {t - i} \left(x _ {t + 1 - i}, y _ {t - i} ^ {*} \left(x _ {t + 1 - i}\right)\right) - f _ {t - i} \left(x _ {t - i}, y _ {t - i} ^ {*} \left(x _ {t - i}\right)\right) \right] \\ \end{array}
$$
$$
\begin{array}{l} \stackrel {(a)} {\leq} \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \left[ \langle \nabla f _ {t - i} (x _ {t - i}, y _ {t - i} ^ {*} (x _ {t - i}), x _ {t + 1} - x _ {t} \rangle + \frac {L _ {f}}{2} \| x _ {t + 1} - x _ {t} \| ^ {2} \right] \\ = \left\langle \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \nabla f _ {t - i} \left(x _ {t - i}, y _ {t - i} ^ {*} \left(x _ {t - i}\right), x _ {t + 1} - x _ {t} \right\rangle + \frac {L _ {f}}{2} \| x _ {t + 1} - x _ {t} \| ^ {2} \right. \\ = \left\langle \nabla F _ {t, \eta} \left(x _ {t}, y _ {t} ^ {*} \left(x _ {t}\right)\right), x _ {t + 1} - x _ {t} \right\rangle + \frac {L _ {f}}{2} \left\| x _ {t + 1} - x _ {t} \right\| ^ {2} \\ \end{array}
$$
where (a) holds because of the smoothness of function  $f_{t}(x,y_{t}^{*}(x))$  w.r.t.  $x$ .
Next, based on Lemma H.2, we can have that
$$
\begin{array}{l} F _ {t, \eta} \left(x _ {t + 1}, y _ {t} ^ {*} \left(x _ {t + 1}\right)\right) - F _ {t, \eta} \left(x _ {t}, y _ {t} ^ {*} \left(x _ {t}\right)\right) \\ \leq \langle \nabla F _ {t, \eta} (x _ {t}, y _ {t} ^ {*} (x _ {t})), x _ {t + 1} - x _ {t} \rangle + \frac {L _ {f}}{2} \| x _ {t + 1} - x _ {t} \| ^ {2} \\ \leq - \beta \langle \nabla F _ {t, \eta} (x _ {t}, y _ {t} ^ {*} (x _ {t})), \widehat {\nabla} F _ {t, \eta} (x _ {t}, y _ {t + 1}) \rangle + \frac {\beta^ {2} L _ {f}}{2} \| \widehat {\nabla} F _ {t, \eta} (x _ {t}, y _ {t + 1}) \| ^ {2} \\ \leq - \beta \| \nabla F _ {t, \eta} \left(x _ {t}, y _ {t} ^ {*} \left(x _ {t}\right)\right) \| ^ {2} - \beta \langle \nabla F _ {t, \eta} \left(x _ {t}, y _ {t} ^ {*} \left(x _ {t}\right)\right), \widehat {\nabla} F _ {t, \eta} \left(x _ {t}, y _ {t + 1}\right) - \nabla F _ {t, \eta} \left(x _ {t}, y _ {t} ^ {*} \left(x _ {t}\right)\right) \rangle \\ + \frac {\beta^ {2} L _ {f}}{2} \| \widehat {\nabla} F _ {t, \eta} (x _ {t}, y _ {t + 1}) \| ^ {2} \\ \leq - \left(\frac {\beta}{2} - \beta^ {2} L _ {f}\right) \| \nabla F _ {t, \eta} \left(x _ {t}, y _ {t} ^ {*} \left(x _ {t}\right)\right) \| ^ {2} \\ + \left(\frac {\beta}{2} + \beta^ {2} L _ {f}\right) \| \nabla F _ {t, \eta} \left(x _ {t}, y _ {t} ^ {*} \left(x _ {t}\right)\right) - \widehat {\nabla} F _ {t, \eta} \left(x _ {t}, y _ {t + 1}\right) \| ^ {2} \\ \end{array}
$$
$$
\begin{array}{l} \left(\frac {\beta}{2} - \beta^ {2} L _ {f}\right) \sum_ {t = 1} ^ {T} \| \nabla F _ {t, \eta} \left(x _ {t}, y _ {t} ^ {*} \left(x _ {t}\right)\right) \| ^ {2} \leq \underbrace {\sum_ {t = 1} ^ {T} \left[ F _ {t , \eta} \left(x _ {t} , y _ {t} ^ {*} \left(x _ {t}\right)\right) - F _ {t , \eta} \left(x _ {t + 1} , y _ {t} ^ {*} \left(x _ {t + 1}\right)\right) \right]} _ {(a)} \\ + \left(\frac {\beta}{2} + \beta^ {2} L _ {f}\right) \underbrace {\sum_ {t = 1} ^ {T} \left\| \nabla F _ {t , \eta} \left(x _ {t} , y _ {t} ^ {*} \left(x _ {t}\right)\right) - \widehat {\nabla} F _ {t , \eta} \left(x _ {t} , y _ {t + 1}\right) \right\| ^ {2}} _ {(b)}. \tag {11} \\ \end{array}
$$
(1) We first have the following lemma to bound the term (a) from above:
Lemma H.3. The following inequality holds:
$$
\sum_ {t = 1} ^ {T} \left[ F _ {t, \eta} \left(x _ {t}, y _ {t} ^ {*} \left(x _ {t}\right)\right) - F _ {t, \eta} \left(x _ {t + 1}, y _ {t} ^ {*} \left(x _ {t + 1}\right)\right) \right] \leq \frac {2 M T}{W} + V _ {1, T}.
$$
Proof. First, it is clear that
$$
\begin{array}{l} F _ {t, \eta} \left(x _ {t}, y _ {t} ^ {*} \left(x _ {t}\right)\right) - F _ {t, \eta} \left(x _ {t + 1}, y _ {t} ^ {*} \left(x _ {t + 1}\right)\right) \\ = \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} f _ {t - i} \left(x _ {t - i}, y _ {t - i} ^ {*} \left(x _ {t - i}\right)\right) - \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} f _ {t - i} \left(x _ {t + 1 - i}, y _ {t - i} ^ {*} \left(x _ {t + 1 - i}\right)\right) \\ = \underbrace {\frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \left[ f _ {t - i} \left(x _ {t - i} , y _ {t - i} ^ {*} \left(x _ {t - i}\right)\right) - f _ {t + 1 - i} \left(x _ {t + 1 - i} , y _ {t + 1 - i} ^ {*} \left(x _ {t + 1 - i}\right)\right) \right]} _ {(a. 1)} \\ \end{array}
$$
$$
+ \underbrace {\frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \left[ f _ {t + 1 - i} \left(x _ {t + 1 - i} , y _ {t + 1 - i} ^ {*} \left(x _ {t + 1 - i}\right)\right) - f _ {t - i} \left(x _ {t + 1 - i} , y _ {t - i} ^ {*} \left(x _ {t + 1 - i}\right)\right) \right]} _ {(a. 2)}. \tag {12}
$$
For the term (a.1), we can obtain that
$$
\begin{array}{l} \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \left[ f _ {t - i} \left(x _ {t - i}, y _ {t - i} ^ {*} \left(x _ {t - i}\right)\right) - f _ {t + 1 - i} \left(x _ {t + 1 - i}, y _ {t + 1 - i} ^ {*} \left(x _ {t + 1 - i}\right)\right) \right] \\ = \frac {1}{W} [ f _ {t} (x _ {t}, y _ {t} ^ {*} (x _ {t})) + \eta f _ {t - 1} (x _ {t - 1}, y _ {t - 1} ^ {*} (x _ {t - 1})) + \ldots + \eta^ {K - 1} f _ {t + 1 - K} (x _ {t + 1 - K}, y _ {t + 1 - K} ^ {*} (x _ {t + 1 - K})) \\ \left. - f _ {t + 1} \left(x _ {t + 1}, y _ {t + 1} ^ {*} \left(x _ {t + 1}\right) - \eta f _ {t} \left(x _ {t}, y _ {t} ^ {*} \left(x _ {t}\right)\right) - \dots - \eta^ {K - 1} f _ {t + 2 - K} \left(x _ {t + 2 - K}, y _ {t + 2 - K} ^ {*} \left(x _ {t + 2 - K}\right)\right)\right) \right] \\ = \frac {1}{W} \left[ \eta^ {K - 1} f _ {t + 1 - K} \left(x _ {t + 1 - K}, y _ {t + 1 - K} ^ {*} \left(x _ {t + 1 - K}\right)\right) - f _ {t + 1} \left(x _ {t + 1}, y _ {t + 1} ^ {*} \left(x _ {t + 1}\right)\right) \right] \\ + \frac {1}{W} \sum_ {i = 1} ^ {K - 1} \left(\eta^ {i - 1} - \eta^ {i}\right) f _ {t + 1 - i} \left(x _ {t + 1 - i}, y _ {t + 1 - i} ^ {*} \left(x _ {t + 1 - i}\right)\right) \\ \leq \frac {M (1 + \eta^ {K - 1})}{W} + \frac {M}{W} \sum_ {i = 1} ^ {K - 1} \left(\eta^ {i - 1} - \eta^ {i}\right) \\ = \frac {2 M}{W} \tag {13} \\ \end{array}
$$
where the inequality holds because of Assumption 5.3.
For the term (a.2), we can have that
$$
\begin{array}{l} \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \left[ f _ {t + 1 - i} \left(x _ {t + 1 - i}, y _ {t + 1 - i} ^ {*} \left(x _ {t + 1 - i}\right)\right) - f _ {t - i} \left(x _ {t + 1 - i}, y _ {t - i} ^ {*} \left(x _ {t + 1 - i}\right)\right) \right] \\ \leq \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \sup  _ {x} \left[ f _ {t + 1 - i} \left(x, y _ {t + 1 - i} ^ {*} (x)\right) - f _ {t - i} \left(x, y _ {t - i} ^ {*} (x)\right) \right]. \tag {14} \\ \end{array}
$$
By substituting Equation (13) and Equation (14) back to Equation (12), Lemma H.3 can be proved:
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \left[ F _ {t, \eta} \left(x _ {t}, y _ {t} ^ {*} \left(x _ {t}\right)\right) - F _ {t, \eta} \left(x _ {t + 1}, y _ {t} ^ {*} \left(x _ {t + 1}\right)\right) \right] \\ \leq \sum_ {t = 1} ^ {T} \left[ \frac {2 M}{W} + \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \sup _ {x} \left[ f _ {t + 1 - i} \left(x, y _ {t + 1 - i} ^ {*} (x)\right) - f _ {t - i} \left(x, y _ {t - i} ^ {*} (x)\right) \right] \right] \\ \leq \frac {2 M T}{W} + V _ {1, T}. \\ \end{array}
$$
(2) Next, for the term (b) which captures the window-averaged hypergradient estimation error, it follows that
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \left\| \nabla F _ {t, \eta} (x _ {t}, y _ {t} ^ {*} (x _ {t})) - \widehat {\nabla} F _ {t, \eta} (x _ {t}, y _ {t + 1}) \right\| ^ {2} \\ = \sum_ {t = 1} ^ {T} \left\| \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \left[ \nabla f _ {t - i} \left(x _ {t - i}, y _ {t - i} ^ {*} \left(x _ {t - i}\right)\right) - \widehat {\nabla} f _ {t - i} \left(x _ {t - i}, y _ {t + 1 - i}\right) \right] \right\| ^ {2} \\ \end{array}
$$
$$
\begin{array}{l} = \sum_ {t = 1} ^ {T} \left[ \sum_ {i = 0} ^ {K - 1} \frac {\eta^ {i}}{W} \sum_ {j = 0} ^ {K - 1} \frac {\eta^ {j}}{W} \langle \nabla f _ {t - i} (x _ {t - i}, y _ {t - i} ^ {*} (x _ {t - i})) - \widehat {\nabla} f _ {t - i} (x _ {t - i}, y _ {t + 1 - i}), \right. \\ \left. \left. \nabla f _ {t - j} \left(x _ {t - j}, y _ {t - j} ^ {*} \left(x _ {t - j}\right)\right) - \widehat {\nabla} f _ {t - j} \left(x _ {t - j}, y _ {t + 1 - j}\right) \right\rangle \right] \\ \leq \sum_ {t = 1} ^ {T} \left[ \sum_ {i = 0} ^ {K - 1} \frac {\eta^ {i}}{W} \sum_ {j = 0} ^ {K - 1} \frac {\eta^ {j}}{W} \left[ \frac {1}{2} \left\| \nabla f _ {t - i} \left(x _ {t - i}, y _ {t - i} ^ {*} \left(x _ {t - i}\right)\right) - \widehat {\nabla} f _ {t - i} \left(x _ {t - i}, y _ {t + 1 - i}\right) \right\| ^ {2} \right. \right. \\ \left. + \frac {1}{2} \left\| \nabla f _ {t - j} (x _ {t - j}, y _ {t - j} ^ {*} (x _ {t - j})) - \widehat {\nabla} f _ {t - j} (x _ {t - j}, y _ {t + 1 - j}) \right\| ^ {2} \right] \\ = \sum_ {t = 1} ^ {T} \left[ \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \left\| \nabla f _ {t - i} \left(x _ {t - i}, y _ {t - i} ^ {*} \left(x _ {t - i}\right)\right) - \widehat {\nabla} f _ {t - i} \left(x _ {t - i}, y _ {t + 1 - i}\right) \right\| ^ {2} \right], \tag {15} \\ \end{array}
$$
which boils down to characterize the hypergradient estimation error  $\| \nabla f_t(x_t, y_t^*(x_t)) - \widehat{\nabla} f_t(x_t, y_{t+1})\|^2$  on the outer level objective function at each round.
By leveraging Theorem 5.6, it is clear that
$$
\begin{array}{l} \sum_ {t = 2} ^ {T} \left\| \nabla F _ {t, \eta} (x _ {t}, y _ {t} ^ {*} (x _ {t})) - \widehat {\nabla} F _ {t, \eta} (x _ {t}, y _ {t + 1}) \right\| ^ {2} \\ \leq \sum_ {t = 2} ^ {T} \left[ \frac {1}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \| \nabla f _ {t - i} \left(x _ {t - i}, y _ {t - i} ^ {*} \left(x _ {t - i}\right)\right) - \widehat {\nabla} f _ {t - i} \left(x _ {t - i}, y _ {t + 1 - i}\right) \| ^ {2} \right] \\ = \sum_ {t = 2} ^ {T} \left[ \frac {3 L _ {1} ^ {2}}{W} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \delta_ {t - i} \right] \\ \leq \frac {3 L _ {1} ^ {2}}{W} \sum_ {t = 2} ^ {T} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \left\{\underbrace {\left(1 - \frac {\alpha \mu_ {g}}{2}\right) ^ {t - 1 - i} \delta_ {1}} _ {(b. 1)} \right. \\ + \underbrace {G _ {2} \sum_ {j = 0} ^ {t - i - 2} \left(1 - \frac {\alpha \mu_ {g}}{2}\right) ^ {j} \| y _ {t - 1 - i - j} ^ {*} \left(x _ {t - 1 - i - j}\right) - y _ {t - i - j} ^ {*} \left(x _ {t - 1 - i - j}\right) \| ^ {2}} _ {(b. 2)} \\ \left. + \underbrace {\frac {2 L _ {1} ^ {2} G _ {2}}{\mu_ {g} ^ {2}} \sum_ {j = 0} ^ {t - 2 - i} \left(1 - \frac {\alpha \mu_ {g}}{2}\right) ^ {j} \| x _ {t - 1 - i - j} - x _ {t - i - j} \| ^ {2}} _ {(b. 3)} \right\}. \tag {16} \\ \end{array}
$$
Let  $\gamma = \frac{\alpha}{2}$ . For the first term (b.1), it can be seen that for  $1 - \frac{\alpha\mu_g}{2} < \eta < 1$
$$
\begin{array}{l} \sum_ {t = 2} ^ {T} \sum_ {i = 0} ^ {K - 1} \eta^ {i} (1 - \gamma \mu_ {g}) ^ {t - 1 - i} \delta_ {1} = \delta_ {1} \sum_ {t = 2} ^ {T} \sum_ {i = 0} ^ {K - 1} \left[ \left(\frac {1 - \gamma \mu_ {g}}{\eta}\right) ^ {t - 1 - i} \eta^ {t - 1} \right] \\ \leq \delta_ {1} \sum_ {t = 2} ^ {T} \eta^ {t - 1} \frac {\frac {1 - \gamma \mu_ {g}}{\eta}}{1 - \frac {1 - \gamma \mu_ {g}}{\eta}} \\ = \frac {\delta_ {1} (1 - \gamma \mu_ {g})}{\eta - 1 + \gamma \mu_ {g}} \frac {\eta (1 - \eta^ {T - 1})}{1 - \eta} \\ \leq \frac {\delta_ {1} \eta (1 - \gamma \mu_ {g})}{(1 - \eta) (\eta - 1 + \gamma \mu_ {g})}. \tag {17} \\ \end{array}
$$
For the second term (b.2), we have
$$
\begin{array}{l} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \sum_ {j = 0} ^ {t - 2 - i} (1 - \gamma \mu_ {g}) ^ {j} \| y _ {t - 1 - i - j} ^ {*} (x _ {t - 1 - i - j}) - y _ {t - i - j} ^ {*} (x _ {t - 1 - i - j}) \| ^ {2} \\ = \sum_ {j = 0} ^ {t - 2} (1 - \gamma \mu_ {g}) ^ {j} \| y _ {t - 1 - j} ^ {*} (x _ {t - 1 - j}) - y _ {t - j} ^ {*} (x _ {t - 1 - j}) \| ^ {2} \\ + \sum_ {j = 0} ^ {t - 3} \eta (1 - \gamma \mu_ {g}) ^ {j} \| y _ {t - 2 - j} ^ {*} (x _ {t - 2 - j}) - y _ {t - 1 - j} ^ {*} (x _ {t - 2 - j}) \| ^ {2} \\ + \dots \\ + \sum_ {j = 0} ^ {t - 1 - K} \eta^ {K - 1} (1 - \gamma \mu_ {g}) ^ {j} \| y _ {t - K - j} ^ {*} (x _ {t - K - j}) - y _ {t + 1 - K - j} ^ {*} (x _ {t - K - j}) \| ^ {2} \\ = \| y _ {t - 1} ^ {*} (x _ {t - 1}) - y _ {t} ^ {*} (x _ {t - 1}) \| ^ {2} + \left[ (1 - \gamma \mu_ {g}) + \eta \right] \| y _ {t - 2} ^ {*} (x _ {t - 2}) - y _ {t - 1} ^ {*} (x _ {t - 2}) \| ^ {2} \\ + \left[ (1 - \gamma \mu_ {g}) ^ {2} + \eta (1 - \gamma \mu_ {g}) + \eta^ {2} \right] \| y _ {t - 3} ^ {*} (x _ {t - 3}) - y _ {t - 2} ^ {*} (x _ {t - 3}) \| ^ {2} \\ + \dots \\ + \left[ (1 - \gamma \mu_ {g}) ^ {K - 2} + (1 - \gamma \mu_ {g}) ^ {K - 3} \eta + \ldots + \eta^ {K - 2} \right] \| y _ {t + 1 - K} ^ {*} (x _ {t + 1 - K}) - y _ {t + 2 - K} ^ {*} (x _ {t + 1 - K}) \| ^ {2} \\ + \sum_ {j = 0} ^ {t - 1 - K} \left\{\left[ (1 - \gamma \mu_ {g}) ^ {j + K - 1} + \eta (1 - \gamma \mu_ {g}) ^ {j + K - 2} + \dots + \eta^ {K - 1} (1 - \gamma \mu_ {g}) ^ {j} \right] \right. \\ \left. \cdot \left\| y _ {t - K - j} ^ {*} \left(x _ {t - K - j}\right) - y _ {t + 1 - K - j} ^ {*} \left(x _ {t - K - j}\right) \right\| ^ {2} \right\} \\ = \| y _ {t - 1} ^ {*} (x _ {t - 1}) - y _ {t} ^ {*} (x _ {t - 1}) \| ^ {2} + \frac {1 - (\frac {1 - \gamma \mu_ {g}}{\eta}) ^ {2}}{1 - \frac {1 - \gamma \mu_ {g}}{\eta}} \eta \| y _ {t - 2} ^ {*} (x _ {t - 2}) - y _ {t - 1} ^ {*} (x _ {t - 2}) \| ^ {2} \\ + \frac {1 - (\frac {1 - \gamma \mu_ {g}}{\eta}) ^ {3}}{1 - \frac {1 - \gamma \mu_ {g}}{\eta}} \eta^ {2} \| y _ {t - 3} ^ {*} (x _ {t - 3}) - y _ {t - 2} ^ {*} (x _ {t - 3}) \| ^ {2} + \dots \\ + \frac {1 - \left(\frac {1 - \gamma \mu_ {g}}{\eta}\right) ^ {K - 1}}{1 - \frac {1 - \gamma \mu_ {g}}{\eta}} \eta^ {K - 2} \| y _ {t + 1 - K} ^ {*} (x _ {t + 1 - K}) - y _ {t + 2 - K} ^ {*} (x _ {t + 1 - K}) \| ^ {2} \\ + \frac {1 - \left(\frac {1 - \gamma \mu_ {g}}{\eta}\right) ^ {K}}{1 - \frac {1 - \gamma \mu_ {g}}{\eta}} \eta^ {K - 1} \sum_ {j = 0} ^ {t - 1 - K} (1 - \gamma \mu_ {g}) ^ {j} \| y _ {t - K - j} ^ {*} (x _ {t - K - j}) - y _ {t + 1 - K - j} ^ {*} (x _ {t - K - j}) \| ^ {2} \\ \leq \frac {1}{1 - \frac {1 - \gamma \mu_ {g}}{\eta}} \left[ \| y _ {t - 1} ^ {*} (x _ {t - 1}) - y _ {t} ^ {*} (x _ {t - 1}) \| ^ {2} + \eta \| y _ {t - 2} ^ {*} (x _ {t - 2}) - y _ {t - 1} ^ {*} (x _ {t - 2}) \| ^ {2} \right. \\ + \dots + \eta^ {K - 2} \left\| y _ {t + 1 - K} ^ {*} \left(x _ {t + 1 - K}\right) - y _ {t + 2 - K} ^ {*} \left(x _ {t + 1 - K}\right) \right\| ^ {2} \\ \left. + \sum_ {j = 0} ^ {t - 1 - K} \eta^ {K - 1 + j} \| y _ {t - K - j} ^ {*} (x _ {t - K - j}) - y _ {t + 1 - K - j} ^ {*} (x _ {t - K - j}) \| ^ {2} \right], \\ \end{array}
$$
$$
\begin{array}{l} \sum_ {t = 2} ^ {T} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \sum_ {j = 0} ^ {t - 2 - i} (1 - \gamma \mu_ {g}) ^ {j} \| y _ {t - 1 - i - j} ^ {*} (x _ {t - 1 - i - j}) - y _ {t - i - j} ^ {*} (x _ {t - 1 - i - j}) \| ^ {2} \\ \leq \frac {\eta}{\eta - 1 + \gamma \mu_ {g}} \sum_ {t = 2} ^ {T} \left[ \| y _ {t - 1} ^ {*} (x _ {t - 1}) - y _ {t} ^ {*} (x _ {t - 1}) \| ^ {2} + \eta \| y _ {t - 2} ^ {*} (x _ {t - 2}) - y _ {t - 1} ^ {*} (x _ {t - 2}) \| ^ {2} \right. \\ \end{array}
$$
$$
\begin{array}{l} + \dots + \eta^ {K - 2} \left\| y _ {t + 1 - K} ^ {*} \left(x _ {t + 1 - K}\right) - y _ {t + 2 - K} ^ {*} \left(x _ {t + 1 - K}\right) \right\| ^ {2} \\ + \sum_ {j = 0} ^ {t - 1 - K} \eta^ {K - 1 + j} \| y _ {t - K - j} ^ {*} (x _ {t - K - j}) - y _ {t + 1 - K - j} ^ {*} (x _ {t - K - j}) \| ^ {2} \Bigg ] \\ \leq \frac {\eta}{(1 - \eta) (\eta - 1 + \gamma \mu_ {g})} \sum_ {t = 2} ^ {T} \sup _ {x} \| y _ {t - 1} ^ {*} (x) - y _ {t} ^ {*} (x) \| ^ {2} \\ = \frac {\eta}{(1 - \eta) (\eta - 1 + \gamma \mu_ {g})} H _ {2, T} \tag {18} \\ \end{array}
$$
where  $H_{2,T} = \sum_{t = 2}^{T}\sup_{x}\| y_{t - 1}^{*}(x) - y_{t}^{*}(x)\|^{2}$ .
Besides, we know that
$$
\begin{array}{l} \left\| x _ {t - 1} - x _ {t} \right\| ^ {2} \\ = \beta^ {2} \| \widehat {\nabla} F _ {t - 1} \left(x _ {t - 1}, y _ {t}\right) \| ^ {2} \\ \leq 2 \beta^ {2} \| \nabla F _ {t - 1} \left(x _ {t - 1}, y _ {t - 1} ^ {*} \left(x _ {t - 1}\right)\right) - \widehat {\nabla} F _ {t - 1} \left(x _ {t - 1}, y _ {t}\right) \| ^ {2} + 2 \beta^ {2} \| \nabla F _ {t - 1} \left(x _ {t - 1}, y _ {t - 1} ^ {*} \left(x _ {t - 1}\right)\right) \| ^ {2}. \\ \end{array}
$$
Following the same analysis for the second term (b.2), the following result can be obtained that for the third term (b.3):
$$
\begin{array}{l} \sum_ {t = 2} ^ {T} \sum_ {i = 0} ^ {K - 1} \eta^ {i} \sum_ {j = 0} ^ {t - 2 - i} (1 - \gamma \mu_ {g}) ^ {j} \| x _ {t - 1 - i - j} - x _ {t - i - j} \| ^ {2} \\ \leq \frac {\eta}{\eta - 1 + \gamma \mu_ {g}} \sum_ {t = 2} ^ {T} \left[ \| x _ {t - 1} - x _ {t} \| ^ {2} + \eta \| x _ {t - 2} - x _ {t - 1} \| ^ {2} \right. \\ + \dots + \eta^ {K - 2} \| x _ {t + 1 - K} - x _ {t + 2 - K} \| ^ {2} \\ \left. + \sum_ {j = 0} ^ {t - 1 - K} \eta^ {K - 1 + j} \| x _ {t - K - j} - x _ {t + 1 - K - j} \| ^ {2} \right] \\ \leq \frac {2 \beta^ {2} \eta}{\eta - 1 + \gamma \mu_ {g}} \sum_ {t = 2} ^ {T} \left[ \| \nabla F _ {t - 1} \left(x _ {t - 1}, y _ {t - 1} ^ {*} \left(x _ {t - 1}\right)\right) - \widehat {\nabla} F _ {t - 1} \left(x _ {t - 1}, y _ {t}\right) \| ^ {2} \right. \\ + \left\| \nabla F _ {t - 1} \left(x _ {t - 1}, y _ {t - 1} ^ {*}\right) \left(x _ {t - 1}\right)\right) \| ^ {2} \\ + \eta \Big (\left\| \nabla F _ {t - 2} \left(x _ {t - 2}, y _ {t - 2} ^ {*} \left(x _ {t - 2}\right)\right) - \widehat {\nabla} F _ {t - 2} \left(x _ {t - 2}, y _ {t - 1}\right) \right\| ^ {2} + \left\| \nabla F _ {t - 2} \left(x _ {t - 2}, y _ {t - 2} ^ {*} \left(x _ {t - 2}\right)\right) \right\| ^ {2} \Big) \\ + \dots \\ + \eta^ {K - 2} \left(\| \nabla F _ {t + 1 - K} (x _ {t + 1 - K}, y _ {t + 1 - K} ^ {*} (x _ {t + 1 - K})) - \widehat {\nabla} F _ {t + 1 - K} (x _ {t + 1 - K}, y _ {t + 2 - K}) \| ^ {2} \right. \\ \left. + \left\| \nabla F _ {t + 1 - K} \left(x _ {t + 1 - K}, y _ {t + 1 - K} ^ {*} \left(x _ {t + 1 - K}\right)\right) \right\| ^ {2}\right) \\ + \sum_ {j = 0} ^ {t - 1 - K} \eta^ {K - 1 + j} \left(\| \nabla F _ {t - K - j} \left(x _ {t - K - j}, y _ {t - K - j} ^ {*} \left(x _ {t - K - j}\right)\right) - \widehat {\nabla} F _ {t - K - j} \left(x _ {t - K - j}, y _ {t + 1 - K - j}\right) \| ^ {2} \right. \\ \left. \left. + \left\| \nabla F _ {t - K - j} \left(x _ {t - K - j}, y _ {t - K - j} ^ {*} \left(x _ {t - K - j}\right)\right) \right\| ^ {2}\right) \right] \\ \leq \frac {2 \beta^ {2} \eta}{\eta - 1 + \gamma \mu_ {g}} \frac {1 - \eta^ {T - 1}}{1 - \eta} \sum_ {t = 2} ^ {T} \| \nabla F _ {t - 1} (x _ {t - 1}, y _ {t - 1} ^ {*} (x _ {t - 1})) - \widehat {\nabla} F _ {t - 1} (x _ {t - 1}, y _ {t}) \| ^ {2} \\ + \frac {2 \beta^ {2} \eta}{\eta - 1 + \gamma \mu_ {g}} \frac {1 - \eta^ {T - 1}}{1 - \eta} \sum_ {t = 2} ^ {T} \| \nabla F _ {t - 1} (x _ {t - 1}, y _ {t - 1} ^ {*} (x _ {t - 1})) \| ^ {2} \\ \end{array}
$$
$$
\begin{array}{l} \leq \frac {2 \beta^ {2} \eta}{(1 - \eta) (\eta - 1 + \gamma \mu_ {g})} \sum_ {t = 2} ^ {T} \| \nabla F _ {t - 1} (x _ {t - 1}, y _ {t - 1} ^ {*} (x _ {t - 1})) - \widehat {\nabla} F _ {t - 1} (x _ {t - 1}, y _ {t}) \| ^ {2} \\ + \frac {2 \beta^ {2} \eta}{(1 - \eta) (\eta - 1 + \gamma \mu_ {g})} \sum_ {t = 2} ^ {T} \| \nabla F _ {t - 1} \left(x _ {t - 1}, y _ {t - 1} ^ {*} \left(x _ {t - 1}\right)\right) \| ^ {2}. \tag {19} \\ \end{array}
$$
Therefore, by substituting Equation (17), Equation (18) and Equation (19) into Equation (16), we can have that