SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} U ^ {k + 1} - U ^ {k} + \left(k ^ {2 \gamma - 1} (k + \gamma) + \frac {1}{4} \gamma (1 - \gamma) k ^ {2 \gamma - 2}\right) \langle \tilde {\mathbf {A}} x ^ {k + 1} - \tilde {\mathbf {A}} x ^ {k}, x ^ {k + 1} - x ^ {k} \rangle \\ = \underbrace {\left((k + 1) ^ {2 \gamma} - k ^ {2 \gamma - 2} (k + \gamma) ^ {2}\right)} _ {= q _ {k}} \| \tilde {\mathbf {A}} x ^ {k + 1} \| ^ {2} \\ + \underbrace {\left(\gamma (k + 1) \left(k + \frac {3}{4}\right) ^ {2 \gamma - 2} - \gamma k ^ {2 \gamma - 2} (k + \gamma) - \left(1 + \frac {k + \gamma}{k}\right) \frac {1}{4} \gamma (\gamma - 1) k ^ {2 \gamma - 2}\right)} _ {= s _ {k, 1}} \langle \tilde {\mathbb {A}} x ^ {k + 1}, x ^ {k + 1} - x ^ {0} \rangle \\ + \underbrace {\left(\gamma k ^ {2 \gamma - 1} - \gamma k \left(k - \frac {1}{4}\right) ^ {2 \gamma - 2} - \left(1 + \frac {k}{k + \gamma}\right) \frac {1}{4} \gamma (\gamma - 1) k ^ {2 \gamma - 2}\right)} _ {= s _ {k, 0}} \langle \tilde {\mathbf {A}} x ^ {k}, x ^ {k} - x ^ {0} \rangle \\ - \frac {1}{4} \gamma^ {2} (\gamma - 1) k ^ {2 \gamma - 3} \left\| x ^ {k + 1} - x ^ {0} \right\| ^ {2} - \frac {1}{4} \gamma^ {2} (\gamma - 1) k ^ {2 \gamma - 3} \frac {1}{1 + \frac {\gamma}{k}} \left\| x ^ {k} - x ^ {0} \right\| ^ {2} \\ + \frac {1}{4} \gamma (\gamma - 1) \left(k ^ {2 \gamma - 2} - (k - 1) ^ {2 \gamma - 2}\right) \left\| x ^ {k} - x ^ {0} \right\| ^ {2} \\ \end{array}
$$
$$
\dot {U} _ {4} (t) + 2 t ^ {2 \gamma} \left\langle \frac {d}{d t} \tilde {\mathbf {A}} (X (t)), \dot {X} (t) \right\rangle d s = - 2 \gamma (\gamma - 1) t ^ {2 \gamma - 3} \left\| X (t) - X _ {0} \right\| ^ {2}
$$
which can be obtained by differentiating the conservation law for  $E_4$ . Thus we may expect the order of the matching terms are equal, and the terms do not occur in the continuous version do not bother our desired conclusion. We check our expectation is true.
Terms  $\left\| x^{k + 1} - x^0\right\|^2$  and  $\left\| x^k -x^0\right\|^2$  correspond to  $\| X - X_0\| ^2$ . The coefficients for  $\left\| x^{k + 1} - x^0\right\|^2$  and  $\left\| x^k -x^0\right\|^2$  are clearly  $\mathcal{O}\left(k^{2\gamma -3}\right)$ , which equals the order of continuous counterpart. Since  $\gamma < 1$ , we know these terms are summable.
Next we observe  $q_{k} \leq 0$ . Observe
$$
\begin{array}{l} q _ {k} \leq 0 \iff \frac {(k + 1) ^ {2 \gamma}}{k ^ {2 \gamma}} \leq \frac {(k + \gamma) ^ {2}}{k ^ {2}} \\ \Longleftrightarrow \left(1 + \frac {1}{k}\right) ^ {2 \gamma} \leq \left(1 + \frac {\gamma}{k}\right) ^ {2} \Longleftrightarrow \left(1 + \frac {1}{k}\right) ^ {\gamma} \leq 1 + \frac {\gamma}{k}. \\ \end{array}
$$
To check the last inequality is true, consider  $f(x) = x^{\gamma}$ . Since this function is concave for  $0 < \gamma < 1$ , we see
$$
\left(1 + \frac {1}{k}\right) ^ {\gamma} = f \left(1 + \frac {1}{k}\right) \leq f (1) + \frac {1}{k} f ^ {\prime} (1) = 1 + \frac {\gamma}{k}.
$$
Finally we focus on  $s_{k,0}$  and  $s_{k,1}$ . As cross terms don't appear in continuous version, we may expect these terms are 'small', or in mathematical words, summable. From Cauchy-Schwarz inequality we know
$$
\left\langle \tilde {\mathbf {A}} x ^ {k + 1}, x ^ {k + 1} - x ^ {0} \right\rangle \leq \| \tilde {\mathbf {A}} x ^ {k + 1} \| \| x ^ {k + 1} - x ^ {0} \|
$$
$$
\left\langle \tilde {\mathbb {A}} x ^ {k}, x ^ {k} - x ^ {0} \right\rangle \leq \left\| \tilde {\mathbb {A}} x ^ {k} \right\| \left\| x ^ {k} - x ^ {0} \right\|.
$$
Since  $\left\| \tilde{\mathbf{A}} x^{k + 1} \right\|$  and  $\left\| \tilde{\mathbf{A}} x^k \right\|$  are bounded from the proof for case (i), we know two innerproduct terms are bounded. Thus if we show  $\sum_{k = 1}^{\infty} |s_{k,0}|, \sum_{k = 1}^{\infty} |s_{k,1}| < \infty$ , we can conclude  $U^k$  is bounded.
Considering Newton expansion, we see
$$
\begin{array}{l} \gamma (k + 1) \left(k + \frac {3}{4}\right) ^ {2 \gamma - 2} - \gamma k ^ {2 \gamma - 2} (k + \gamma) \\ = \gamma k ^ {2 \gamma - 1} + \gamma \left(1 + \frac {3}{2} (\gamma - 1)\right) k ^ {2 \gamma - 2} + \mathcal {O} \left(k ^ {2 \gamma - 3}\right) - \gamma k ^ {2 \gamma - 1} - \gamma^ {2} k ^ {2 \gamma - 2} \\ = \frac {1}{2} \gamma (\gamma - 1) k ^ {2 \gamma - 2} + \mathcal {O} \left(k ^ {2 \gamma - 3}\right) \\ \end{array}
$$
$$
s _ {k, 1} = \frac {1}{2} \gamma (\gamma - 1) k ^ {2 \gamma - 2} + \mathcal {O} (k ^ {2 \gamma - 3}) - (2 + \frac {\gamma}{k}) \frac {1}{4} \gamma (\gamma - 1) k ^ {2 \gamma - 2} = \mathcal {O} (k ^ {2 \gamma - 3}).
$$
With similar argument
$$
\begin{array}{l} s _ {k, 0} = \gamma k ^ {2 \gamma - 1} - \gamma k \left(k - \frac {1}{4}\right) ^ {2 \gamma - 2} - \left(1 + \frac {k}{k + \gamma}\right) \frac {1}{4} \gamma (\gamma - 1) k ^ {2 \gamma - 2} \\ = \gamma k ^ {2 \gamma - 1} - \gamma k \left(k ^ {2 \gamma - 2} - (2 \gamma - 2) \frac {1}{4} k ^ {2 \gamma - 3} + \mathcal {O} \left(k ^ {2 \gamma - 4}\right)\right) - \left(2 - \frac {\gamma}{k + \gamma}\right) \frac {1}{4} \gamma (\gamma - 1) k ^ {2 \gamma - 2} \\ = \frac {1}{2} \gamma (\gamma - 1) k ^ {2 \gamma - 2} + \mathcal {O} (k ^ {2 \gamma - 3}) - \left(\frac {1}{2} \gamma (\gamma - 1) k ^ {2 \gamma - 2} + \mathcal {O} (k ^ {2 \gamma - 3})\right) = \mathcal {O} (k ^ {2 \gamma - 3}) \\ \end{array}
$$
$$
\begin{array}{l} U ^ {k + 1} \leq U ^ {k} + s _ {k, 1} \left\| \tilde {\mathbf {A}} x ^ {k + 1} \right\| \left\| x ^ {k + 1} - x ^ {0} \right\| + s _ {k, 0} \left\| \tilde {\mathbf {A}} x ^ {k} \right\| \left\| x ^ {k} - x ^ {0} \right\| \\ - c _ {k + 1} \frac {\gamma}{k} \left\| x ^ {k + 1} - x ^ {0} \right\| ^ {2} - c _ {k + 1} \frac {\gamma}{k + \gamma} \left\| x ^ {k} - x ^ {0} \right\| ^ {2} + (c _ {k + 1} - c _ {k}) \left\| x ^ {k} - x ^ {0} \right\| ^ {2} \\ \leq \sum_ {m = 1} ^ {\infty} \left(| s _ {m, 1} | \left\| \tilde {\mathbf {A}} x ^ {k + 1} \right\| \left\| x ^ {k + 1} - x ^ {0} \right\| + | s _ {m, 0} | \left\| \tilde {\mathbf {A}} x ^ {k + 1} \right\| \left\| x ^ {k + 1} - x ^ {0} \right\|\right) \\ + \sum_ {m = 1} ^ {\infty} \left(\left| c _ {m + 1} \frac {\gamma}{m} \right| \left\| x ^ {k + 1} - x ^ {0} \right\| ^ {2} + \left(\left| c _ {m + 1} \frac {\gamma}{m + \gamma} \right| + | c _ {k + 1} - c _ {k} |\right) \left\| x ^ {k} - x ^ {0} \right\| ^ {2}\right) = M _ {1}. \\ \end{array}
$$
$$
\begin{array}{l} M _ {1} + \frac {1}{4} \gamma (1 - \gamma) k ^ {2 \gamma - 2} \left\| x ^ {k} - x ^ {0} \right\| ^ {2} \geq k ^ {2 \gamma} \| \tilde {\mathbf {A}} x ^ {k} \| ^ {2} + \gamma k \left(k - \frac {1}{4}\right) ^ {2 \gamma - 2} \left\langle \tilde {\mathbf {A}} x ^ {k}, x ^ {k} - x ^ {0} \right\rangle \\ \geq k ^ {2 \gamma} \| \tilde {\mathbf {A}} x ^ {k} \| ^ {2} + \gamma k \left(k - \frac {1}{4}\right) ^ {2 \gamma - 2} \langle \tilde {\mathbf {A}} x ^ {k}, x ^ {\star} - x ^ {0} \rangle \\ = k ^ {2 \gamma} \| \tilde {\mathbf {A}} x ^ {k} \| ^ {2} + \left\langle k ^ {\gamma} \tilde {\mathbf {A}} x ^ {k}, \gamma k ^ {1 - \gamma} \left(k - \frac {1}{4}\right) ^ {2 \gamma - 2} \left(x ^ {\star} - x ^ {0}\right) \right\rangle \\ \geq k ^ {2 \gamma} \| \tilde {\mathbf {A}} x ^ {k} \| ^ {2} - \frac {1}{2} k ^ {2 \gamma} \| \tilde {\mathbf {A}} x ^ {k} \| ^ {2} - \frac {\gamma^ {2}}{2} \left(k ^ {1 - \gamma} \left(k - \frac {1}{4}\right) ^ {2 \gamma - 2}\right) ^ {2} \left\| x ^ {\star} - x ^ {0} \right\| ^ {2}. \\ \end{array}
$$
Since  $k^{1 - \gamma}\left(k - \frac{1}{4}\right)^{2\gamma - 2} = \mathcal{O}\left(k^{\gamma - 1}\right)$  and  $\gamma < 1$ , this terms goes to zero as  $k \to \infty$  thus there is some  $M_2 > 0$  such that  $\frac{1}{2}\left(\gamma k^{1 - \gamma}\left(k - \frac{1}{4}\right)^{2\gamma - 2}\right)^2\left\|x^k - x^\star\right\|^2 \leq M_2$  for all  $k \geq 1$ . Reorganizing terms, we obtain the desired result
$$
\left\| \tilde {\mathbf {A}} x ^ {k} \right\| ^ {2} \leq \frac {2}{k ^ {2 \gamma}} \left(M _ {1} + M _ {2} + \gamma (1 - \gamma) k ^ {2 \gamma - 2} \left\| x ^ {\star} - x ^ {0} \right\| ^ {2}\right) = \mathcal {O} \left(\frac {1}{k ^ {2 \gamma}}\right).
$$
H Proof of convergence analysis for strongly monotone  $\mathbb{A}$
H.1 Proof of Proposition 6.1
We take dilated coordinate  $W(t) = C(t)(X(t) - X_0)$  as did in the proof of Proposition 3.2. Recall from (30), the second order version of the ODE was written as  $0 = \dot{W} - \beta(t)\dot{W} + C(t)\frac{d}{dt}\tilde{\mathbf{A}}(X(t))$ . Now for we multiply  $R(t)^2$  in the ODE and obtain
$$
0 = R (t) ^ {2} \left(\ddot {W} - \beta (t) \dot {W} + C (t) \frac {d}{d t} \tilde {\mathbb {A}} (X)\right).
$$
Now taking inner product with  $\tilde{W}$  and integrating we have
$$
\begin{array}{l} E_{1}\equiv \frac{R(t)^{2}}{2}\left\| \dot{W} (t)\right\|^{2} - \underbrace{\int_{t_{0}}^{t}R(s)^{2}\left(\frac{\dot{R}(s)}{R(s)}\left\| \dot{W} (s)\right\|^{2}\right)ds}_{*} - \int_{t_{0}}^{t}\beta (s)R(s)^{2}\left\| \dot{W} (s)\right\|^{2}ds \\ + \int_ {t _ {0}} ^ {t} C (s) R (s) ^ {2} \left\langle \frac {d}{d s} \tilde {\mathbb {A}} (X (s)), \dot {W} (s) \right\rangle d s. \\ \end{array}
$$
Note the second term which is obtained from integration by parts, would not appear if  $R(t) = 1$  as in Proposition 3.2. This is key term to exploit the condition  $\mathbb{A}$  is strongly monotone.
Now again with  $\dot{W}(t) = C(t)\left(\dot{X}(t) + \beta(s)(X(t) - X_0)\right)$ , we rewrite the last term as
$$
\begin{array}{l} \int_ {t _ {0}} ^ {t} C (s) R (s) ^ {2} \left\langle \frac {d}{d s} \tilde {\mathbf {A}} (X (s)), \dot {W} (s) \right\rangle d s \\ = \int_ {t _ {0}} ^ {t} C (s) ^ {2} R (s) ^ {2} \left\langle \frac {d}{d s} \tilde {\mathbf {A}} (X (s)), \dot {X} (s) \right\rangle d s + \int_ {t _ {0}} ^ {t} \left\langle \frac {d}{d s} \tilde {\mathbf {A}} (X (s)), C (s) R (s) ^ {2} \beta (s) W (s) \right\rangle d s. \\ \end{array}
$$
Taking integration by parts to the second term we have
$$
\begin{array}{l} \int_ {t _ {0}} ^ {t} \left\langle \frac {d}{d s} \tilde {\mathbb {A}} (X (s)), C (s) R (s) ^ {2} \beta (s) W (s) \right\rangle d s - \left[ \left\langle \tilde {\mathbb {A}} (X (W (s), s)), C (s) R (s) ^ {2} \beta (s) W (s) \right\rangle \right] _ {t _ {0}} ^ {t} \\ = - \int_ {t _ {0}} ^ {t} \left\langle \tilde {\mathbf {A}} (X (W, t)), \left(\beta (s) ^ {2} + 2 \frac {\dot {R} (s)}{R (s)} \beta (s) + \dot {\beta} (s)\right) C (s) R (s) ^ {2} W (s) + C (s) R (s) ^ {2} \beta (s) \dot {W} (s) \right\rangle d s \\ = \int_ {t _ {0}} ^ {t} \beta (s) R (s) ^ {2} \left\| \dot {W} (s) \right\| ^ {2} d s + \int_ {t _ {0}} ^ {t} \left(\beta (s) ^ {2} + \underbrace {2 \frac {\dot {R} (s)}{R (s)} \beta (s)} _ {*} + \dot {\beta} (s)\right) R (s) ^ {2} \left\langle \dot {W} (s), W (s) \right\rangle d s. \\ \end{array}
$$
The fact  $C(t)\tilde{\mathbf{A}}(X(W,t)) = -\dot{W}(t)$  is applied to the second equality. Note the fundamental theorem of calculus is valid since  $C(s)R(s)^2\beta(s)W(s)$  is differentiable,  $\tilde{\mathbf{A}}(X(W(s),t))$  is Lipschitz continuous in  $[t_0,t]$  by Lemma B.4, and so their inner product is absolutely continuous in  $[t_0,t]$ .
Now consider the second integrand except the term marked with \*. From integration by parts we have
$$
\begin{array}{l} \left. \int_ {t _ {0}} ^ {t} (\beta (s) ^ {2} + \dot {\beta} (s)) R (s) ^ {2} \left\langle \dot {W} (s), W (s) \right\rangle d s - \left[ (\beta (s) ^ {2} + \dot {\beta} (s)) R (s) ^ {2} \frac {1}{2} \| W (s) \| ^ {2} \right] _ {t _ {0}} ^ {t} \right. \\ = -\frac{1}{2}\int_{t_{0}}^{t}\left(\left(2\beta (s)\dot{\beta} (s) + \ddot{\beta} (s)\right)R(s)^{2} + \left(\underbrace{\beta(s)^{2}_{*} + \dot{\beta}(s)}_{*}\right)2R(s)\dot{R} (s)\right)\| W(s)\|^{2}  ds. \\ \end{array}
$$
The integrand except * marked term can be rewritten as
$$
\begin{array}{l} \frac {1}{2} \int_ {t _ {0}} ^ {t} \left(2 \beta (s) \dot {\beta} (s) R (s) + \ddot {\beta} (s) R (s) + 2 \dot {\beta} (s) \dot {R} (s)\right) R (s) \| W (s) \| ^ {2} d s \\ = \frac {1}{2} \int_ {t _ {0}} ^ {t} C (s) ^ {2} \left(2 \beta (s) \dot {\beta} (s) R (s) + \ddot {\beta} (s) R (s) + 2 \dot {\beta} (s) \dot {R} (s)\right) R (s) \| X (s) - X _ {0} \| ^ {2} d s \\ = \frac {1}{2} \int_ {t _ {0}} ^ {t} \frac {d}{d s} \left(C (s) ^ {2} R (s) ^ {2} \dot {\beta} (s)\right) \| X (s) - X _ {0} \| ^ {2} d s. \\ \end{array}
$$
Now collecting the terms marked with  $*$ , we have
$$
\begin{array}{l} - \int_ {t _ {0}} ^ {t} R (s) ^ {2} \left(\frac {\dot {R} (s)}{R (s)} \left\| \dot {W} (s) \right\| ^ {2}\right) d s + \int_ {t _ {0}} ^ {t} 2 \frac {\dot {R} (s)}{R (s)} \beta (s) R (s) ^ {2} \left\langle \dot {W} (s), W (s) \right\rangle d s - \int_ {t _ {0}} ^ {t} \beta (s) ^ {2} R (s) \dot {R} (s) \| W (s) \| ^ {2} d s \\ = - \int_ {t _ {0}} ^ {t} R (s) ^ {2} \frac {\dot {R} (s)}{R (s)} \left\| \dot {W} (s) - \beta (s) W (s) \right\| ^ {2} d s = - \int_ {t _ {0}} ^ {t} R (s) ^ {2} C (s) ^ {2} \frac {\dot {R} (s)}{R (s)} \left\| \dot {X} (s) \right\| ^ {2} d s. \\ \end{array}
$$
Collecting all results we have
$$
\begin{array}{l} E _ {1} \equiv \frac {R (t) ^ {2}}{2} \left\| \dot {W} (t) \right\| ^ {2} + \left[ \left\langle \tilde {\mathbf {A}} (X (W, s)), C (s) R (s) ^ {2} \beta (s) W (s) \right\rangle \right] _ {t _ {0}} ^ {t} + \left[ \left(\beta (s) ^ {2} + \dot {\beta} (s)\right) R (s) ^ {2} \frac {1}{2} \| W (s) \| ^ {2} \right] _ {t _ {0}} ^ {t} \\ + \int_ {t _ {0}} ^ {t} C (s) ^ {2} R (s) ^ {2} \left\langle \frac {d}{d s} \tilde {\mathbf {A}} (X (s)), \dot {X} (s) \right\rangle d s - \int_ {t _ {0}} ^ {t} R (s) ^ {2} C (s) ^ {2} \frac {\dot {R} (s)}{R (s)} \left\| \dot {X} (s) \right\| ^ {2} d s \\ + \frac {1}{2} \int_ {t _ {0}} ^ {t} \frac {d}{d s} \left(C (s) ^ {2} R (s) ^ {2} \dot {\beta} (s)\right) \| X (s) - X _ {0} \| ^ {2} d s \\ = \frac {C (t) ^ {2} R (t) ^ {2}}{2} \left(\| \mathbb {A} (X (t)) \| ^ {2} + 2 \beta (t) \langle \mathbb {A} (X (t)), X (t) - X _ {0} \rangle + \left(\beta (t) ^ {2} + \dot {\beta} (t)\right) \| X (t) - X _ {0} \| ^ {2}\right) \\ + \int_ {t _ {0}} ^ {t} C (s) ^ {2} R (s) ^ {2} \left(\left\langle \frac {d}{d s} \tilde {\mathbb {A}} (X (s)), \dot {X} (s) \right\rangle - \frac {\dot {R} (s)}{R (s)} \left\| \dot {X} (s) \right\| ^ {2}\right) d s - \int_ {t _ {0}} ^ {t} \frac {d}{d s} \left(\frac {C (s) ^ {2} R (s) ^ {2} \dot {\beta} (s)}{2}\right) \| X (s) - X _ {0} \| ^ {2} d s \\ - \underbrace {\frac {C (t _ {0}) ^ {2} R (t _ {0}) ^ {2}}{2} \left(2 \beta (t _ {0}) \langle \mathbb {A} (X (t _ {0})) , X (t _ {0}) - X _ {0} \rangle + \left(\beta (t _ {0}) ^ {2} + \dot {\beta} (t _ {0})\right) \| X (t _ {0}) - X _ {0} \| ^ {2}\right)} _ {\text {c o n s t a n t}}. \\ \end{array}
$$
Renaming  $E = E_{1}$  - constant, we get the desired result.
H.2 Proof of Theorem 6.2
H.2.1 Proof of the inequality  $V(t) \leq V(0)$
The basic structure of the proof is same as Appendix E.3. We do not repeat the whole proof here, instead we check the steps done in Appendix E.3 are also valid for the setup in Theorem 6.2.
(i)  $V$  is nonincreasing for Lipschitz continuous  $\mu$ -strongly monotone  $\tilde{\mathbb{A}}$ .
Recall  $V$  in Theorem 6.2 was defined as
$$
V (t) = \frac {\left(e ^ {\mu t} - e ^ {- \mu t}\right) ^ {2}}{2} \left\| \tilde {\mathbf {A}} (X (t)) \right\| ^ {2} + 2 \mu \left(1 - e ^ {- 2 \mu t}\right) \left\langle \tilde {\mathbf {A}} (X (t)), X (t) - X _ {0} \right\rangle - 2 \mu^ {2} \left(1 - e ^ {- 2 \mu t}\right) \| X (t) - X _ {0} \| ^ {2}. \tag {47}
$$
We first check following equality is true.
$$
V (t) = \frac {C (t) ^ {2} R (t) ^ {2}}{2} \left(\left\| \tilde {\mathbf {A}} (X (t)) \right\| ^ {2} + 2 \beta (t) \left\langle \tilde {\mathbf {A}} (X (t)), X (t) - X _ {0} \right\rangle + \left(\beta (t) ^ {2} + \dot {\beta} (t)\right) \| X (t) - X _ {0} \| ^ {2}\right). \tag {48}
$$
Recall we're considering (10),  $\beta(t) = \frac{2\mu}{e^{2\mu t} - 1}$ . As  $\frac{d}{dt}\log \left(1 - e^{-2\mu t}\right) = \frac{2\mu e^{-2\mu t}}{1 - e^{-2\mu t}} = \frac{2\mu}{e^{2\mu t} - 1} = \beta(t)$ , we have
$$
C (t) = e ^ {\int_ {\infty} ^ {t} \frac {2 \mu}{e ^ {2 \mu s} - 1} d s} = e ^ {\log \left(1 - e ^ {- 2 \mu t}\right)} = 1 - e ^ {- 2 \mu t}.
$$
As  $\dot{\beta} (t) = -\frac{4\mu^2e^{2\mu t}}{(e^{2\mu t} - 1)^2}$  and  $R(t) = e^{\mu t}$
$$
\frac {C (t) ^ {2} R (t) ^ {2}}{2} = \frac {1}{2} \left(1 - e ^ {- 2 \mu t}\right) ^ {2} e ^ {2 \mu t} = \frac {\left(e ^ {\mu t} - e ^ {- \mu t}\right) ^ {2}}{2}
$$
$$
C (t) ^ {2} R (t) ^ {2} \beta (t) = e ^ {- 2 \mu t} \left(e ^ {2 \mu t} - 1\right) ^ {2} \frac {2 \mu}{e ^ {2 \mu t} - 1} = 2 \mu \left(1 - e ^ {- 2 \mu t}\right)
$$
$$
\frac {C (t) ^ {2} R (t) ^ {2}}{2} \left(\beta (t) ^ {2} + \dot {\beta} (t)\right) = \frac {(e ^ {\mu t} - e ^ {- \mu t}) ^ {2}}{2} \left(\left(\frac {2 \mu}{e ^ {2 \mu t} - 1}\right) ^ {2} - \frac {4 \mu^ {2} e ^ {2 \mu t}}{(e ^ {2 \mu t} - 1) ^ {2}}\right) = 2 \mu^ {2} \left(e ^ {- 2 \mu t} - 1\right).
$$
This proves the desired equality. Now we show
$$
V (t) = E - \int_ {t _ {0}} ^ {t} C (s) ^ {2} R (s) ^ {2} \left(\left\langle \frac {d}{d s} \tilde {\mathbf {A}} (X (s)), \dot {X} (s) \right\rangle - \frac {\dot {R} (s)}{R (s)} \| \dot {X} (s) \| ^ {2}\right) d s, \tag {49}
$$
where  $E$  is from Proposition 6.1 which was defined as
$$
\begin{array}{l} E = \frac {C (t) ^ {2} R (t) ^ {2}}{2} \left(\left\| \tilde {\mathbf {A}} (X (t)) \right\| ^ {2} + 2 \beta (t) \left\langle \tilde {\mathbf {A}} (X (t)), X (t) - X _ {0} \right\rangle + \left(\beta (t) ^ {2} + \dot {\beta} (t)\right) \| X (t) - X _ {0} \| ^ {2}\right) \\ + \int_ {t _ {0}} ^ {t} C (s) ^ {2} R (s) ^ {2} \left(\left\langle \frac {d}{d s} \tilde {\mathbf {A}} (X (s)), \dot {X} (s) \right\rangle - \frac {\dot {R} (s)}{R (s)} \left\| \dot {X} (s) \right\| ^ {2}\right) d s - \int_ {t _ {0}} ^ {t} \frac {d}{d s} \left(\frac {C (s) ^ {2} R (s) ^ {2} \dot {\beta} (s)}{2}\right) \| X (t) - X _ {0} \| ^ {2} d s. \\ \end{array}
$$
From (48) and the definition of  $E$ , it is enough to show  $\frac{d}{ds}\left(\frac{C(s)^2R(s)^2\dot{\beta}(s)}{2}\right) = 0$ . Since
$$
\frac {C ^ {2} (t) R ^ {2} (t) \dot {\beta} (t)}{2} = \left(1 - e ^ {- 2 \mu t}\right) ^ {2} e ^ {2 \mu t} \left(- \frac {4 \mu^ {2} e ^ {2 \mu t}}{(e ^ {2 \mu t} - 1) ^ {2}}\right) = - 4 \mu^ {2},
$$
we see  $\frac{d}{ds}\left(\frac{C(s)^2R(s)^2\dot{\beta}(s)}{2}\right) = 0.$
Now since  $\tilde{\mathbb{A}}$  is Lipschitz continuous, we know  $E$  is constant from Proposition 6.1. Therefore from (49), for  $t > 0$ ,  $|h| < t$  we have
$$
V (t + h) - V (t) = \int_ {t} ^ {t + h} C (s) ^ {2} R (s) ^ {2} \left(\left\langle \frac {d}{d s} \tilde {\mathbf {A}} (X (s)), \dot {X} (s) \right\rangle - \frac {\dot {R} (s)}{R (s)} \left\| \dot {X} (s) \right\| ^ {2}\right) d s.
$$
As  $\frac{\dot{R}(s)}{R(s)} = \mu$  and  $\tilde{\mathbb{A}}$  is  $\mu$ -strongly monotone, from (2) we see
$$
\left\langle \frac {d}{d s} \tilde {\mathbb {A}} (X (s)), \dot {X} (s) \right\rangle - \frac {\dot {R} (s)}{R (s)} \left\| \dot {X} (s) \right\| ^ {2} \geq 0.
$$
Therefore  $V(t + h) - V(t) \geq 0$  for  $h > 0$ , we get the desired result.
(ii) Calculation of  $V(0)$  for Lipschitz continuous monotone  $\tilde{\mathbb{A}}$  Plugging  $t = 0$  to (47) we immediately obtain  $V(0) = 0$ .
(iii)  $V(t)\leq 0$  holds for all  $t\in [0,\infty)$  and general maximal  $\mu$  -strongly monotone  $\mathbb{A}$  We check the arguments in Appendix E.3.3 and Appendix E.3.4 are also valid here.
Define  $S$  as defined in Lemma E.1. Take  $t \in S$ , let  $T > t$ . As checked in Appendix C, the arguments used in the proof Proposition B.11 is also valid for the case  $\beta(t) = \frac{2\mu}{e^{2\mu t} - 1}$ . This fact provides the required sequence  $\{X_{\lambda_n}\}_{n \in \mathbb{N}}$ , where  $X_{\lambda_n}$  converges to  $X$  uniformly on  $[0, T]$ , and  $\dot{X}_{\lambda_n}$  converges weakly to  $\dot{X}$  in  $L^2([0, T], \mathbb{R}^n)$ . As in Appendix E.3.3, denote  $V_\lambda$  as  $V$  for the solution with  $\mathbb{A}_\lambda$ . Then from (i) we know  $V_{\lambda_n}$  is nonincreasing for all  $n \in \mathbb{N}$ , we have  $\lim \sup_{n \to \infty} V_{\lambda_n}(t) \leq \lim \sup_{n \to \infty} V_{\lambda_n}(0) = 0$ .
Moreover, we can check the extension of  $\tilde{\mathbb{A}}$  defined in Lemma E.1 is also valid. From (21) and (22), we know  $\left\| \dot{X}_{\lambda}(t)\right\| \leq \frac{e^{\mu T}}{\sqrt{2}}\| m(\mathbb{A}(X_0))\|$  and  $\| \mathbb{A}_{\lambda}(X_{\lambda}(t))\| \leq \sqrt{e^{2\mu T} + 1}\| m(\mathbb{A}(X_0))\|$  for all  $\lambda >0,t\in [0,T]$ . Therefore we can prove Corollary E.2 for the case  $\beta (t) = \frac{2\mu}{e^{2\mu t} - 1}$  with the same proof, replacing  $M_{\mathbb{A}}(T)$  by  $\sqrt{e^{2\mu T} + 1}\| m(\mathbb{A}(X_0))\|$  and  $M_{\mathrm{dot}}(T)$  by  $\frac{e^{\mu T}}{\sqrt{2}}\| m(\mathbb{A}(X_0))\|$ . Thus  $\tilde{\mathbf{A}} (X(t))$  is well-defined for  $t = 0$ , plugging  $t = 0$  to (47) we obtain  $V(0) = 0$ .
$$
\limsup_{n\to \infty}V_{\lambda_{n}}(t)\leq \limsup_{n\to \infty}V_{\lambda_{n}}(0) = 0 = V(0),
$$
it remains to show  $V(t) \leq \lim \sup_{n \to \infty} V_{\lambda_n}(t)$ . Observe, as equality  $\dot{X}(t) = -\tilde{\mathbf{A}}(X(t)) - \beta(t)(X(t) - X_0)$  holds since  $t \in S$ , from (48) we have
$$
V (t) = \frac {C (t) ^ {2} R (t) ^ {2}}{2} \left(\left\| \dot {X} (t) \right\| ^ {2} + \dot {\beta} (t) \| X (t) - X _ {0} \| ^ {2}\right).
$$
Therefore if is suffices to check Lemma E.4 is valid here with some  $U_{\lambda_n}$ . For some  $a > 0$ , Define  $U_{\lambda_n} : [0, \infty) \to \mathbb{R}$  as
$$
U _ {\lambda_ {n}} (t) = \left\| \dot {X} _ {\lambda_ {n}} (t) \right\| ^ {2} \underbrace {- \dot {\beta} (t) \| X _ {\lambda_ {n}} (t) - X _ {0} \| ^ {2} + \int_ {a} ^ {t} \left(\ddot {\beta} (s) - \frac {2 \dot {\beta} (s) ^ {2}}{\beta (s)}\right) \| X _ {\lambda_ {n}} (s) - X _ {0} \| ^ {2} d s} _ {f _ {n} (t)}.
$$
We proceed similar argument with Lemma B.5. Differentiating  $\dot{X}_{\lambda_n}(t) = -\mathbb{A}_{\lambda_n}(X(t)) - \beta (t)(X_{\lambda_n}(t) - X_0)$ , we have for almost all  $t > 0$
$$
\ddot {X} _ {\lambda_ {n}} (t) = - \frac {d}{d t} \mathbb {A} _ {\lambda_ {n}} (X (t)) - \dot {\beta} (t) (X _ {\lambda_ {n}} (t) - X _ {0}) - \beta (t) \dot {X} _ {\lambda_ {n}} (t).
$$
$$
\begin{array}{l} \dot {U} _ {\lambda_ {n}} (t) \\ = 2 \left\langle \dot {X} _ {\lambda_ {n}} (t), \ddot {X} _ {\lambda_ {n}} (t) \right\rangle - \ddot {\beta} (t) \| X _ {\lambda_ {n}} (t) - X _ {0} \| ^ {2} - 2 \dot {\beta} (t) \left\langle \dot {X} _ {\lambda_ {n}} (t), X _ {\lambda_ {n}} (t) - X _ {0} \right\rangle + \left(\ddot {\beta} (t) - \frac {2 \dot {\beta} (t) ^ {2}}{\beta (t)}\right) \| X _ {\lambda_ {n}} (t) - X _ {0} \| ^ {2} \\ = 2 \left\langle \dot {X} _ {\lambda_ {n}} (t), - \frac {d}{d t} \mathbb {A} _ {\lambda_ {n}} (X (t)) - \dot {\beta} (t) (X _ {\lambda_ {n}} (t) - X _ {0}) - \beta (t) \dot {X} (t) \right\rangle \\ - 2 \dot {\beta} (t) \left\langle \dot {X} _ {\lambda_ {n}} (t), X _ {\lambda_ {n}} (t) - X _ {0} \right\rangle - \frac {2 \dot {\beta} (t) ^ {2}}{\beta (t)} \| X _ {\lambda_ {n}} (t) - X _ {0} \| ^ {2} \\ = - 2 \left\langle \dot {X} _ {\lambda_ {n}} (t), \frac {d}{d t} \mathbb {A} _ {\lambda_ {n}} (X (t)) \right\rangle - 2 \beta (t) \left\| \dot {X} _ {\lambda_ {n}} (t) + \frac {\dot {\beta} (t)}{\beta (t)} \left(X _ {\lambda_ {n}} (t) - X _ {0}\right) \right\| ^ {2} \leq 0. \\ \end{array}
$$
Therefore  $U_{\lambda_n}$  is nonincreasing, we can prove  $\left\| \dot{X}(t) \right\| \leq \lim \sup_{n \to \infty} \left\| \dot{X}_{\lambda_n}(t) \right\|^2$  with same argument in Lemma E.4. Therefore we have  $V(t) \leq V(0) = 0$  for  $t \in S$ . Extending the result to  $t \in [0, \infty)$  can be done with the same argument done in Appendix E.3.4.
H.2.2 Proof for convergence rate
$$
V (t) = \frac {(e ^ {\mu t} - e ^ {- \mu t}) ^ {2}}{2} \left\| \tilde {\mathbf {A}} (X (t)) \right\| ^ {2} + 2 \mu (1 - e ^ {- 2 \mu t}) \left\langle \tilde {\mathbf {A}} (X (t)), X (t) - X _ {0} \right\rangle - 2 \mu^ {2} \left(1 - e ^ {- 2 \mu t}\right) \| X (t) - X _ {0} \| ^ {2}.
$$