SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathbf {T} = \sum_ {i = 0} ^ {N - 1} \delta_ {i} \mathbf {A} _ {i} + \sum_ {i = 0} ^ {N - 1} \lambda_ {i} \mathbf {B} _ {i} + \frac {\delta_ {N}}{2} \mathbf {e} _ {N} \mathbf {e} _ {N} ^ {\intercal} - \frac {1}{2} \mathbf {e} _ {N} \mathbf {e} _ {N} ^ {\intercal}.
$$
If  $[X = 0$  and  $T\succeq 0]$  is false, we can choose  $\mathbf{F}$  and  $\mathbf{G}$  that makes the value of Lagrangian to be  $-\infty$ . Thus the convex dual problem of (49) is
$$
\underset {\delta , \lambda , \tau} {\text {m a x i m i z e}} \underset {\mathbf {F}, \mathbf {G}} {\text {m i n i m i z e}} \mathcal {L} (\delta , \lambda , \tau , \mathbf {F}, \mathbf {G}) = \left\{ \begin{array}{l l} \underset {\delta , \lambda , \tau} {\text {m a x i m i z e}} & - \frac {\tau L}{2} R ^ {2} \\ \text {s u b j e c t t o} & \mathbf {X} = 0, \mathbf {T} \succeq 0 \\ & \delta_ {i} \geq 0, \quad \lambda_ {i} \geq 0, \quad \tau \geq 0 \end{array} . \right. \tag {50}
$$
For the constraint  $\mathbf{X} = 0$ ,  $\lambda_{i} = \delta_{i} - \delta_{i-1}$  for  $1 \leq i \leq N-1$ ,  $\tau = \delta_{N}$  and  $-\delta_{0} + \lambda_{0} + \delta_{N} = 0$ . By substituting  $v_{i+1} = \delta_{i}$  for  $0 \leq i \leq N-1$  and  $v_{0} = \delta_{N}$ , (50) becomes
$$
\underset {v _ {0}, \dots , v _ {N}} {\text {m a x i m i z e}} \underset {\mathbf {F}, \mathbf {G}} {\text {m i n i m i z e}} \mathcal {L} (\delta , \lambda , \tau , \mathbf {F}, \mathbf {G}) = \left\{ \begin{array}{l l} \underset {\delta , \lambda , \tau} {\text {m a x i m i z e}} & - \frac {v _ {0} L}{2} R ^ {2} \\ \text {s u b j e c t t o} & \mathbf {T} \succeq 0 \\ & 0 \leq v _ {0} \leq \dots \leq v _ {N} \end{array} . \right. \tag {51}
$$
Therefore if strong duality holds,  $\mathcal{P}_2(\mathcal{L}_{\mathrm{G}},H,R)$  becomes
$$
\underset {v _ {0}, \dots , v _ {N}} {\text {m i n i m i z e}} \frac {v _ {0} L}{2} R ^ {2} \tag {52}
$$
$$
\text {s u b j e c t . t o .} \mathrm {T} \succeq 0, 0 \leq v _ {0} \leq \dots \leq v _ {N}.
$$
We can apply a similar argument for  $\mathcal{P}_1(\mathcal{L}_{\mathrm{F}},H,R)$ . To begin with, define  $\{\mathbf{f}_i\}_{i = -1}^N$  as a unit vector of length  $N + 2$  which  $(i + 2)$ -component is 1. Additionally, define  $\{\mathbf{x}_i\}_{i = 0}^N$ ,  $\{\mathbf{C}_i\}_{i = 0}^{N - 1}$  and  $\{\mathbf{D}_i\}_{i = 0}^N$  as follows:
$$
\mathbf {x} _ {0} := \mathbf {f} _ {- 1}, \quad \mathbf {x} _ {i + 1} := \mathbf {x} _ {i} - \frac {1}{L} \sum_ {j = 0} ^ {i} h _ {i + 1, j} \mathbf {f} _ {j} \quad i = 0, \ldots , N - 1,
$$
$$
\mathbf {C} _ {i} := \frac {1}{2} \mathbf {f} _ {i + 1} \left(\mathbf {x} _ {i} - \mathbf {x} _ {i + 1}\right) ^ {\intercal} + \frac {1}{2} \left(\mathbf {x} _ {i} - \mathbf {x} _ {i + 1}\right) \mathbf {f} _ {i + 1} ^ {\intercal} + \frac {1}{2 L} \left(\mathbf {f} _ {i} - \mathbf {f} _ {i + 1}\right) \left(\mathbf {f} _ {i} - \mathbf {f} _ {i + 1}\right) ^ {\intercal}, \tag {53}
$$
$$
\mathbf {D} _ {i} := - \frac {1}{2} \mathbf {f} _ {i} \mathbf {x} _ {i} ^ {\intercal} - \frac {1}{2} \mathbf {x} _ {i} \mathbf {f} _ {i} ^ {\intercal} + \frac {1}{2 L} \mathbf {f} _ {i} \mathbf {f} _ {i} ^ {\intercal}.
$$
Then, if the strong duality holds, the problem
$$
\underset {f} {\text {m a x i m i z e}} f (x _ {N}) - f _ {\star}
$$
subject.to.  $[x_0,\ldots ,x_N$  are generated by FSFO with the matrix  $H]$  (54)
$$
[ \forall l \in \mathcal {L} _ {\mathrm {F}}, f \text {s a t i s f i e s} l ]
$$
$$
\underset {u _ {0}, \dots , u _ {N}} {\text {m a x i m i z e}} \frac {L}{2 u _ {N}} R ^ {2} \tag {55}
$$
$$
\text {s u b j e c t . t o .} \mathbf {S} \succeq 0, 0 \leq u _ {0} \leq \dots \leq u _ {N},
$$
$$
\mathbf {S} = \frac {L}{2} \mathbf {f} _ {- 1} \mathbf {f} _ {- 1} ^ {\intercal} + \sum_ {i = 0} ^ {N - 1} u _ {i} \mathbf {C} _ {i} + \sum_ {i = 0} ^ {N} (u _ {i} - u _ {i - 1}) \mathbf {D} _ {i}.
$$
By using Schur's Complement,
$$
\begin{array}{c c c} \mathbf {S} \succeq 0 & \Leftrightarrow & \mathbf {S} ^ {\prime} \succeq 0 \end{array}
$$
$$
\mathbf {S} ^ {\prime} = \sum_ {i = 0} ^ {N - 1} u _ {i} \mathbf {C} _ {i} + \sum_ {i = 0} ^ {N} (u _ {i} - u _ {i - 1}) \mathbf {D} _ {i} - \frac {1}{2 L} \left(\sum_ {i = 0} ^ {N} (u _ {i} - u _ {i - 1}) \mathbf {f} _ {i}\right) \left(\sum_ {i = 0} ^ {N} (u _ {i} - u _ {i - 1}) \mathbf {f} _ {i}\right) ^ {\intercal}.
$$
Hence (55) is equivalent to
$$
\underset {u _ {0}, \dots , u _ {N}} {\text {m a x i m i z e}} \frac {L}{2 u _ {N}} R ^ {2} \tag {56}
$$
$$
\text {s u b j e c t . t o .} \mathbf {S} ^ {\prime} \succeq 0, 0 \leq u _ {0} \leq \dots \leq u _ {N}.
$$
Now we will prove the following proposition.
Proposition 2. Consider a matrix  $H = \{h_{i,j}\}_{0\leq j < i\leq N}$  and  $H^{A}$ . If  $h_{i + 1,i}\neq 0$  for  $0\leq i\leq N - 1$  and treating the solution of infeasible maximize problem as 0, the optimal values of  $\mathcal{P}_1(\mathcal{L}_{\mathrm{F}},H,R)$  and  $\mathcal{P}_2(\mathcal{L}_{\mathrm{G}},H^A,R)$  are same.
Proof. To simplify the analysis, we can consider  $\{\mathbf{f}_i\}_{i=0}^N$  as a length  $N+1$  unit vector, as all terms with  $\mathbf{f}_{-1}$  can be eliminated by using  $\mathbf{S}'$  instead of  $\mathbf{S}$ . With this simplification, both  $\mathbf{S}'$  and  $\mathbf{T}$  belong to  $\mathbb{S}^{N+1}$ .
Next, let  $0 < u_0 \leq u_1 \leq \dots \leq u_N$  and  $v_i = \frac{1}{u_{N - i}}$  for  $0 \leq i \leq N$ , noting that  $0 < v_0 \leq \dots \leq v_N$ . It is important to observe that  $\mathbf{S}'$  and  $\mathbf{T}$  can be expressed in terms of  $S(H,u)$  and  $\mathcal{T}(H^A,v)$ , respectively. Furthermore, in the proof of Theorem 1 (A), we proved that  $S(H,u) = \mathcal{M}(u)^{\top}\mathcal{T}(H^{A},v)\mathcal{M}(u)$  for some invertible  $\mathcal{M}(u)$ . Thus,
$$
\begin{array}{c c c} \mathcal {S} (H, u) \succeq 0 & \Leftrightarrow & \mathcal {T} (H ^ {A}, v) \succeq 0. \end{array}
$$
$$
\left(a _ {0}, \dots , a _ {N}\right) \in \left\{\left(u _ {0}, \dots , u _ {N}\right) \mid \mathbf {S} ^ {\prime} \succeq 0, 0 <   u _ {0} \leq \dots \leq u _ {N} \right\} \tag {57}
$$
$$
\left(\frac {1}{a _ {N}}, \dots , \frac {1}{a _ {0}}\right) \in \left\{\left(v _ {0}, \dots , v _ {N}\right) \mid \mathbf {T} \succeq 0, 0 <   v _ {0} \leq \dots \leq v _ {N} \right\}. \tag {58}
$$
For the next step, we claim that the optimal value of (56) and
$$
\underset {u _ {0}, \dots , u _ {N}} {\text {m a x i m i z e}} \frac {L}{2 u _ {N}} R ^ {2} \tag {59}
$$
$$
\text {s u b j e c t . t o .} \mathbf {S} ^ {\prime} \succeq 0, 0 <   u _ {0} \leq \dots \leq u _ {N}
$$
are same, i.e., consider the case when all  $u_{i}$  are positive is enough. To prove that, assume there exists  $0 = u_{0} = \dots = u_{k}, 0 < u_{k + 1} \leq \dots \leq u_{N}$  and  $H$  that satisfy  $\mathbf{S}' \succeq 0$ . Next, observe that the  $\mathbf{f}_k\mathbf{f}_k^\intercal$  component of  $\mathbf{S}'$  is 0 but  $\mathbf{f}_{k + 1}\mathbf{f}_k^\intercal$  component of  $\mathbf{S}'$  is  $u_{k + 1}h_{k + 1,k} \neq 0$ , which makes  $\mathbf{S}' \succeq 0$  impossible.
When the optimal value of (52) is 0, it implies  $\{(v_0,\dots ,v_N)\mid \mathbf{T}\succeq 0,0 < v_0\leq \dots \leq v_N\}$  is an empty set. Therefore,  $\{(u_0,\ldots ,u_N)\mid \mathbf{S}'\succeq 0,0 < u_0\leq \dots \leq u_N\}$  is also empty set. Since (59) and (56) have the same optimal value, the optimal value of (56) is 0.
If the optimal value of (52) is positive, the optimal values of (56) and (52) are the same since (57) and (58) are equivalent.
Proof of Theorem 4.  $H_{\mathrm{OGM}}$  is the solution of (46) since (OGM) is  $\mathcal{A}^{\star}$ -optimal with respect to  $\mathcal{L}_{\mathrm{F}}$  and (P1). Additionally, if
$$
(4 6) = \underset {H, h _ {i + 1, i} \neq 0} {\text {m a x i m i z e}} \mathcal {P} _ {1} \left(\mathcal {L} _ {\mathrm {F}}, H, R\right) \tag {60}
$$
holds,  $H_{\mathrm{OGM}}$  is the solution to the following problem due to the strong duality.
$$
\underset {H, h _ {i + 1, i} \neq 0} {\text {m a x i m i z e}} \underset {u _ {0}, \dots , u _ {N}} {\text {m a x i m i z e}} \frac {L}{2 u _ {N}} R ^ {2} \tag {61}
$$
$$
\text {s u b j e c t . t o .} \mathbf {S} \succeq 0, 0 \leq u _ {0} \leq \dots \leq u _ {N}.
$$
Applying Proposition 2 and using the fact  $H_{\mathrm{OGM - G}} = H_{\mathrm{OGM}}^A$  provides that  $H_{\mathrm{OGM - G}}$  is the solution of
$$
\underset {H, h _ {i + 1, i} \neq 0} {\text {m a x i m i z e}} \underset {v _ {0}, \dots , v _ {N}} {\text {m i n i m i z e}} \frac {v _ {0} L}{2} R ^ {2} \tag {62}
$$
$$
(4 7) = \underset {H, h _ {i + 1, i} \neq 0} {\text {m a x i m i z e}} \mathcal {P} _ {1} \left(\mathcal {L} _ {\mathrm {F}}, H, R\right) \tag {63}
$$
holds, the optimal solution of (47) is the  $H_{\mathrm{OGM - G}}$  , which proves the  $\mathcal{A}^{\star}$  -optimality of (OGM-G) with respect to  $\mathcal{L}_{\mathrm{G}}$  and (P2). The proof of (60) and (63) uses the continuity argument with  $H$  and please refer [44, Claim 4].
Remark of Proof of  $\mathcal{A}^{\star}$ -optimality of (OGM-G). We proved that (OGM-G) is  $\mathcal{A}^{\star}$ -optimal if we use the subset of cocoercivity inequalities. Therefore, it is still open whether (OGM-G) is optimal or not among the  $L$ -smooth convex function's gradient minimization method.
Proof of  $\mathcal{A}^{\star}$ -optimality of  $(\mathrm{OBL - G_b})$ . We provide the proof that the  $H$  matrix of  $(\mathrm{OBL - G_b})$  is the solution of
$$
\underset {H \in \mathbb {R} _ {L} ^ {N \times N}} {\text {m i n i m i z e}} \mathcal {P} _ {2} \left(\mathcal {L} _ {\mathrm {G} ^ {\prime}}, H, R\right) \tag {64}
$$
To begin with, bring to mind the  $\mathcal{A}^{\star}$ -optimality of  $(\mathrm{OBL - F_b})$ :  $(\mathrm{OBL - F_b})$  is  $\mathcal{A}^{\star}$ -optimal with respect to the  $[\mathcal{L}_{\mathrm{F'}}$ ,  $P1]$ , i.e., the  $H$  matrix of  $(\mathrm{OBL - F_b})$  is the solution of
$$
\underset {H \in \mathbb {R} _ {L} ^ {N \times N}} {\text {m i n i m i z e}} \mathcal {P} _ {1} \left(\mathcal {L} _ {\mathrm {F} ^ {\prime}}, H, R\right). \tag {65}
$$
To prove the  $\mathcal{A}^{\star}$ -optimality of  $(\mathrm{OBL - G_b})$ , we use the  $\mathcal{A}^{\star}$ -optimality of  $(\mathrm{OBL - F_b})$ .
Under the assumption of strong duality, we could change  $\mathcal{P}_1(\mathcal{L}_{\mathrm{F}^{\prime}},H,R)$  into the following SDP:
$$
\underset {u _ {0}, \dots , u _ {N}} {\text {m a x i m i z e}} \frac {L}{2 u _ {N}} R ^ {2} \tag {66}
$$
$$
\text {s u b j e c t . t o .} \mathbf {S} _ {1} \succeq 0, 0 \leq u _ {0} \leq \dots \leq u _ {N}
$$
$$
\mathbf {S} _ {1} = \frac {L}{2} \mathbf {f} _ {- 1} \mathbf {f} _ {- 1} ^ {\intercal} + \sum_ {i = 0} ^ {N - 1} u _ {i} \mathbf {C} _ {i} + \sum_ {i = 0} ^ {N} (u _ {i} - u _ {i - 1}) \left(\mathbf {D} _ {i} - \frac {1}{2 L} \mathbf {f} _ {i} \mathbf {f} _ {i} ^ {\intercal}\right).
$$
Here we used the same notation with (53). Each  $\frac{1}{2L}\mathbf{f}_i\mathbf{f}_i^\top$  term is subtracted from the original S since we consider the inequality  $\llbracket x_i,x_\star \rrbracket -\frac{1}{2L}\| \nabla f(x_i)\| ^2$  instead of  $\llbracket x_i,x_\star \rrbracket$  for  $\mathcal{L}_{\mathrm{F^{\prime}}}$ . Moreover, (66) is equivalent to
$$
\underset {u _ {0}, \dots , u _ {N}} {\text {m a x i m i z e}} \frac {L}{2 u _ {N}} R ^ {2} \tag {67}
$$
$$
\text {s u b j e c t . t o .} \mathbf {S} _ {1} ^ {\prime} \succeq 0, 0 \leq u _ {0} \leq \dots \leq u _ {N}
$$
$$
\mathbf {S} _ {1} ^ {\prime} = \sum_ {i = 0} ^ {N - 1} u _ {i} \mathbf {C} _ {i} + \sum_ {i = 0} ^ {N} (u _ {i} - u _ {i - 1}) \left(\mathbf {D} _ {i} - \frac {1}{2 L} \mathbf {f} _ {i} \mathbf {f} _ {i} ^ {\intercal}\right) - \frac {1}{2 L} \left(\sum_ {i = 0} ^ {N} (u _ {i} - u _ {i - 1}) \mathbf {f} _ {i}\right) \left(\sum_ {i = 0} ^ {N} (u _ {i} - u _ {i - 1}) \mathbf {f} _ {i}\right) ^ {\intercal}.
$$
Similarly, under the assumption of strong duality,  $\mathcal{P}_2(\mathcal{L}_{\mathrm{G}}', H, R)$  is equivalent to
$$
\underset {v _ {0}, \dots , v _ {N}} {\text {m i n i m i z e}} \frac {v _ {0} L}{2} R ^ {2} \tag {68}
$$
$$
\text {s u b j e c t . t o .} \mathbf {T} _ {1} \succeq 0, 0 \leq v _ {0} \leq \dots \leq v _ {N}
$$
$$
\mathbf {T} _ {1} = \sum_ {i = 0} ^ {N - 1} v _ {i + 1} \mathbf {A} _ {i} + \sum_ {i = 0} ^ {N - 1} (v _ {i + 1} - v _ {i}) \left(\mathbf {B} _ {i} - \frac {1}{2 L} (\mathbf {e} _ {i} - \mathbf {e} _ {N}) (\mathbf {e} _ {i} - \mathbf {e} _ {N}) ^ {\intercal}\right) - \frac {1}{2} \mathbf {e} _ {N} \mathbf {e} _ {N} ^ {\intercal}.
$$
Proposition 3. Consider a matrix  $H = \{h_{k,i}\}_{0\leq i < k\leq N}$  and  $H^{A}$ . If  $h_{i + 1,i}\neq 0$  for  $0\leq i\leq N - 1$  and treating the solution of infeasible maximize problem as 0, the optimal values of  $\mathcal{P}_1(\mathcal{L}_{\mathrm{F}},H,R)$  and  $\mathcal{P}_2(\mathcal{L}_{\mathrm{G}},H^A,R)$  are same.
Proof. The proof structure is the same as the proof of Proposition 2. First consider  $\{\mathbf{f}_i\}_{i=0}^N$  as length  $N + 1$ , which gives  $\mathbf{S}_1'$ ,  $\mathbf{T}_1 \in \mathbb{S}^{N+1}$ . Furthermore, in the proof of Appendix A, we proved that
$$
\mathbf {S} _ {1} ^ {\prime} = \mathcal {M} (u) ^ {\intercal} \mathbf {T} _ {1} \mathcal {M} (u).
$$
$$
\mathbf {S} _ {1} ^ {\prime} \succeq 0 \quad \Leftrightarrow \quad \mathbf {T} \succeq 0.
$$
The other steps are the same as the proof of Proposition 2.
Proof of Theorem 5.  $H_{\mathrm{OGM}}$  is the solution of (46) since (OGM) is  $\mathcal{A}^{\star}$ -optimal with respect to  $\mathcal{L}_{\mathrm{F}}$  and (P1). Additionally, if
$$
(4 6) = \underset {H, h _ {i + 1, i} \neq 0} {\text {m a x i m i z e}} \mathcal {P} _ {1} \left(\mathcal {L} _ {\mathrm {F}}, H, R\right) \tag {69}
$$
$$
\underset {H, h _ {i + 1, i} \neq 0} {\text {m a x i m i z e}} \underset {u _ {0}, \dots , u _ {N}} {\text {m a x i m i z e}} \frac {L}{2 u _ {N}} R ^ {2} \tag {70}
$$
$$
\underset {H, h _ {i + 1, i} \neq 0} {\text {m a x i m i z e}} \underset {v _ {0}, \dots , v _ {N}} {\text {m i n i m i z e}} \frac {v _ {0} L}{2} R ^ {2} \tag {71}
$$
$$
\text {s u b j e c t . t o .} \mathbf {T} \succeq 0, 0 \leq v _ {0} \leq \dots \leq v _ {N}.
$$
$$
(4 7) = \underset {H, h _ {i + 1, i} \neq 0} {\text {m a x i m i z e}} \mathcal {P} _ {1} \left(\mathcal {L} _ {\mathrm {F}}, H, R\right) \tag {72}
$$
holds, the optimal solution of (47) is the  $H_{\mathrm{OGM - G}}$  , which proves the  $\mathcal{A}^{\star}$  -optimality of (OGM-G) with respect to  $\mathcal{L}_{\mathrm{G}}$  and (P2). The proof of (69) and (72) uses the continuity argument with  $H$  and please refer [44, Claim 4].
C Omitted parts in Section 3
C.1 Omitted calculations in Section 3.1
Strictly speaking, (7) is not a differential equation but rather a diffeo-integral equation. However, if  $H$  is separable, i.e., when  $H(t,s) = e^{\beta(s) - \gamma(t)}$  for some  $\beta, \gamma: [0,T) \to \mathbb{R}$ , then (7) can be reformulated as an ODE.
Assume the process (7) is well-defined. We can alternatively write (7) as
$$
X (0) = x _ {0}, \quad \dot {X} (t) = - e ^ {- \gamma (t)} \int_ {0} ^ {t} e ^ {\beta (s)} \nabla f (X (s)) d s. \tag {73}
$$
By multiplying  $e^{\gamma (t)}$  each side and differentiating, we obtain
$$
\ddot {X} (t) + \dot {\gamma} (t) \dot {X} (t) + e ^ {\beta (t) - \gamma (t)} \nabla f (X (t)) = 0.
$$
The H-dual of (73) is
$$
Y (0) = x _ {0}, \quad \dot {Y} (t) = - e ^ {\beta (T - t)} \int_ {0} ^ {t} e ^ {- \gamma (T - s)} \nabla f (Y (s)) d s.
$$
Under the well-definedness, by multiplying  $e^{-\beta (T - t)}$  each side and differentiating, we obtain
$$
\ddot {Y} (t) + \dot {\beta} (T - t) \dot {Y} (t) + e ^ {\beta (T - t) - \gamma (T - t)} \nabla f (Y (t)) = 0.
$$
When  $\beta(s) = \gamma(t) = r\log t$ , two ODEs become
$$
\ddot {X} (t) + \frac {r}{t} \dot {X} (t) + \nabla f (X (t)) = 0 \tag {74}
$$
$$
\ddot {Y} (t) + \frac {r}{T - t} \dot {X} (t) + \nabla f (X (t)) = 0, \tag {75}
$$
which are introduced in Section 3.2.
For the calculations of energy functions of (74) and (75), refer [51, Section 3.1, Section 4.2]. They proved that
$$
\begin{array}{l} \left. (r - 1) \| X (0) - x _ {\star} \| ^ {2} = T ^ {2} \left(f (X (T)) - x _ {\star}\right) + \frac {1}{2} \left\| T \dot {X} (T) + 2 (X (T) - x _ {\star}) \right\| ^ {2} \right. \\ + (r - 3) \| X (T) - x _ {\star} \| ^ {2} + \int_ {0} ^ {T} (r - 3) s \left\| \dot {X} (s) \right\| ^ {2} d s - \int_ {0} ^ {T} 2 s [ X (s), x _ {\star} ] d s \\ \end{array}
$$
holds for (74) and
$$
\begin{array}{l} \frac {1}{T ^ {2}} \left(f (Y (0)) - f (Y (T))\right) + \frac {- r + 3}{T ^ {4}} \| Y (0) - Y (T) \| ^ {2} \\ = \frac {1}{4 (r - 1)} \| \nabla f (Y (T)) \| ^ {2} + \int_ {0} ^ {T} \frac {r - 3}{(T - s) ^ {5}} \left\| (T - s) \dot {Y} (s) + 2 (Y (s) - Y (T)) \right\| ^ {2} d s - \int_ {0} ^ {T} \frac {2}{(T - s) ^ {3}} [ Y (s), Y (T) ] d s \\ \end{array}
$$
holds for (75). After arranging terms, we obtain the results in Section 3.2.
To begin with, we give a formal version of Theorem 2. Consider two following conditions.
$$
u (T) \left(f (X (T)) - f ^ {\star}\right) \leq \mathcal {U} (T), \quad (\forall X (0), x ^ {\star}, \left\{\nabla f (X (s)) \right\} _ {s \in [ 0, T ]} \in \mathcal {A} _ {1}). \tag {C3'}
$$
$$
\frac {1}{2} \| \nabla f (Y (T)) \| ^ {2} \leq \mathcal {V} (T), \quad (\forall Y (0), \{\nabla f (Y (s)) \} _ {s \in [ 0, T ]} \in \mathcal {A} _ {2}). \tag {C4'}
$$
where  $\mathcal{A}_1$  and  $\mathcal{A}_2$  are family of vectors which makes Fubini's Theorem can be applied and will be defined later in this section.
In [51], they considered the ODE which gradient term is  $2\nabla f(Y(t))$  instead of  $\nabla f(Y(t))$ .
Theorem 6 (Formal version of Theorem 2). Assume the C-FSFOMs (7) with  $H$  and  $H^A$  are well-defined in the sense that solutions to the diffeo-integral equations exist. Consider differentiable functions  $u, v \colon (0,T) \to \mathbb{R}$  that  $v(t) = \frac{1}{u(T - t)}$  for  $t \in [0,T]$  and
(i)  $\lim_{s\to 0}u(t) = 0$
(ii)  $\lim_{s\to -T}v(s)(f(Y(s)) - f(Y(T)) + \langle \nabla f(Y(T)),Y(T) - Y(s)\rangle) = 0.$
(iii)  $f$  is  $L$ -smooth and convex.
Then the following holds.
$$
\left[ \left(\mathrm {C} 3 ^ {\prime}\right) \text {i s s a t i s f i e d w i t h} u (\cdot) \text {a n d} H \right] \Leftrightarrow \left[ \left(\mathrm {C} 4 ^ {\prime}\right) \text {i s s a t i s f i e d w i t h} v (\cdot) \text {a n d} H ^ {A} \right].
$$
Calculations of energy functions via transformation Frist of all, we calculate  $\mathcal{U}(T) - u(T)(f(X(T)) - f^{\star})$ .
$$
\begin{array}{l} \mathcal {U} (T) - u (T) \left(f (X (T)) - f ^ {\star}\right) \\ = \frac {1}{2} \| X (0) - x ^ {\star} \| ^ {2} + \int_ {0} ^ {T} u ^ {\prime} (s) \left(f (X (s)) - f ^ {\star} + \langle \nabla f (X (s)), x ^ {\star} - X (s) \rangle\right) d s - u (T) \left(f (X (T)) - f ^ {\star}\right) d s \\ = \frac {1}{2} \| X (0) - x ^ {\star} \| ^ {2} + \int_ {0} ^ {T} u ^ {\prime} (s) \left\langle \nabla f (X (s)), x ^ {\star} - X (s) \right\rangle d s - \int_ {0} ^ {T} u (s) \left\langle \nabla f (X (s)), \dot {X} (s) \right\rangle d s \\ = \frac {1}{2} \| X (0) - x ^ {\star} \| ^ {2} + \int_ {0} ^ {T} u ^ {\prime} (s) \langle \nabla f (X (s)), x ^ {\star} - X (0) \rangle d s \\ + \int_ {0} ^ {T} u ^ {\prime} (s) \left\langle \nabla f (X (s)), X (0) - X (s) \right\rangle d s - \int_ {0} ^ {T} u (s) \left\langle \nabla f (X (s)), \dot {X} (s) \right\rangle d s \\ = \frac {1}{2} \left\| X (0) - x ^ {\star} - \int_ {0} ^ {T} u ^ {\prime} (s) \nabla f (X (s)) d s \right\| ^ {2} - \frac {1}{2} \left\| \int_ {0} ^ {T} u ^ {\prime} (s) \nabla f (X (s)) d s \right\| ^ {2} \\ + \int_ {0} ^ {T} u ^ {\prime} (s) \left\langle \nabla f (X (s)), X (0) - X (s) \right\rangle d s - \int_ {0} ^ {T} u (s) \left\langle \nabla f (X (s)), \dot {X} (s) \right\rangle d s. \\ \end{array}
$$
We used parts of integration and  $u(0) \colon = \lim_{s \to 0} u(s) = 0$ . Since  $X(0) - x^{\star}$  can have any value, (C3) is equivalent to
$$
\begin{array}{l} - \frac {1}{2} \left\| \int_ {0} ^ {T} u ^ {\prime} (s) \nabla f (X (s)) d s \right\| ^ {2} \\ + \int_ {0} ^ {T} u ^ {\prime} (s) \left\langle \nabla f (X (s)), X (0) - X (s) \right\rangle d s - \int_ {0} ^ {T} u (s) \left\langle \nabla f (X (s)), \dot {X} (s) \right\rangle d s \geq 0 \tag {76} \\ \end{array}
$$
holds for any  $\{\nabla f(X(s))\}_{s\in [0,T]}$
Now define a transformation  $\{f_s\in \mathbb{R}^d\}_{s\in [0,T]}\to \{g_s\in \mathbb{R}^d\}_{s\in [0,T]}$  as
$$
g _ {s} \colon = u (s) f _ {s} + \int_ {s} ^ {T} u ^ {\prime} (z) f _ {z} d z, \quad s \in [ 0, T ] \tag {77}
$$
and  $\{g_s\in \mathbb{R}^d\}_{s\in [0,T]}\to \{f_s\in \mathbb{R}^d\}_{s\in [0,T]}$
$$
f _ {s} \colon = \frac {1}{u (T)} g _ {0} + \frac {1}{u (s)} \left(g _ {s} - g _ {0}\right) - \int_ {s} ^ {T} \frac {u ^ {\prime} (b)}{u (b) ^ {2}} \left(g _ {b} - g _ {0}\right) d b, \quad s \in [ 0, T ]. \tag {78}
$$
One can show that the above two transformations are in the inverse relationship. Next, we calculate  $\mathcal{U}(T)$ . Define  $f_s \colon = \nabla f(X(s))$ . Under the transformation (77), we can find a simple expression of
$$
\begin{array}{l} (7 6) = - \frac {1}{2} \| g _ {0} \| ^ {2} - \int_ {0} ^ {T} u ^ {\prime} (s) \left\langle f _ {s}, X (s) - X (0) \right\rangle d s - \int_ {0} ^ {T} u (s) \left\langle f _ {s}, \dot {X} (s) \right\rangle d s \\ = - \frac {1}{2} \| g _ {0} \| ^ {2} - \int_ {0} ^ {T} u ^ {\prime} (s) \left\langle f _ {s}, \int_ {0} ^ {s} \dot {X} (a) d a \right\rangle d s + \int_ {0} ^ {T} u (s) \left\langle f _ {s}, \int_ {0} ^ {s} \dot {X} (a) d a \right\rangle d s \\ = - \frac {1}{2} \| g _ {0} \| ^ {2} - \int_ {0} ^ {T} u ^ {\prime} (s) \left\langle f _ {s}, \int_ {0} ^ {s} \dot {X} (a) d a \right\rangle d s + \int_ {0} ^ {T} \int_ {0} ^ {s} H (s, a) \left\langle u (s) f _ {s}, f _ {a} \right\rangle d a d s \tag {79} \\ \stackrel {(\circ)} {=} - \frac {1}{2} \| g _ {0} \| ^ {2} - \int_ {0} ^ {T} \left\langle \dot {X} (a), \int_ {a} ^ {T} u ^ {\prime} (s) f _ {s} d s \right\rangle d a + \int_ {0} ^ {T} \int_ {0} ^ {s} H (s, a) \left\langle u (s) f _ {s}, f _ {a} \right\rangle d a d s \\ = - \frac {1}{2} \| g _ {0} \| ^ {2} + \int_ {0} ^ {T} \int_ {0} ^ {s} H (s, a) \left\langle f _ {a}, g _ {s} \right\rangle d a d s. \\ \end{array}
$$
We used Fubini's Theorem at  $(\circ)$ . Next we calculate  $\mathcal{V}(T)$ .
$$
\begin{array}{l} \mathcal {V} (T) = \frac {1}{u (T)} \left(f (Y (0)) - f (Y (T))\right) + \int_ {0} ^ {T} \frac {d}{d s} \frac {1}{u (T - s)} \left(f (Y (s)) - f (Y (T)) + \langle \nabla f (Y (s)), Y (T) - Y (s) \rangle\right) d s \\ = \frac {1}{u (T)} \left(f (Y (0)) - f (Y (T))\right) + \int_ {0} ^ {T} \frac {u ^ {\prime} (T - s)}{u (T - s) ^ {2}} \langle Y (T) - Y (s), \nabla f (Y (s)) - \nabla f (Y (T)) \rangle d s \\ + \int_ {0} ^ {T} \frac {d}{d s} \frac {1}{u (T - s)} \left(f (Y (s)) - f (Y (T)) + \langle \nabla f (Y (T)), Y (T) - Y (s) \rangle\right) d s \\ \stackrel {(\circ)} {=} \frac {1}{u (T)} \left\langle \nabla f (Y (T)), Y (0) - Y (T) \right\rangle + \int_ {0} ^ {T} \frac {u ^ {\prime} (T - s)}{u (T - s) ^ {2}} \left\langle Y (T) - Y (s), \nabla f (Y (s)) - \nabla f (Y (T)) \right\rangle d s \\ - \int_ {0} ^ {T} \frac {1}{u (T - s)} \left\langle \dot {Y} (s), \nabla f (Y (s)) - \nabla f (Y (T)) \right\rangle d s. \\ \end{array}
$$
$(\circ)$  comes from parts of integration and the assumption
$$
\lim _ {s \to T} v (s) \left(f (Y (s)) - f (Y (T)) + \langle \nabla f (X (T)), Y (T) - Y (s) \rangle\right) = 0.
$$
To clarify,  $u'(T - s) = u'(z)|_{z = T - s}$ . Now briefly write  $g_s \colon = \nabla f(Y(T - s))$ . Then