SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Convergence Analysis of ODE Models for Accelerated First-Order Methods via Positive Semidefinite Kernels
Jungbin Kim
kjb2952@snu.ac.kr
Insoon Yang*
insoonyang@snu.ac.kr
We propose a novel methodology that systematically analyzes ordinary differential equation (ODE) models for first-order optimization methods by converting the task of proving convergence rates into verifying the positive semidefiniteness of specific Hilbert-Schmidt integral operators. Our approach is based on the performance estimation problems (PEP) introduced by Drori and Teboulle [8]. Unlike previous works on PEP, which rely on finite-dimensional linear algebra, we use tools from functional analysis. Using the proposed method, we establish convergence rates of various accelerated gradient flow models, some of which are new. As an immediate consequence of our framework, we show a correspondence between minimizing function values and minimizing gradient norms.
We consider the following convex optimization problem:
$$
\min  _ {x \in \mathbb {R} ^ {d}} f (x), \tag {1}
$$
where  $f: \mathbb{R}^d \to \mathbb{R}$  is a continuously differentiable ( $\mu$ -strongly) convex function. We assume that a minimizer  $x^{*}$  exists. First-order methods, for example, gradient descent and Nesterov's accelerated gradient method, are popular in solving this problem due to their low cost per iteration and dimension-free oracle complexities. These methods can be analyzed by examining their limiting ODEs. For instance, the gradient descent  $x_{k+1} = x_k - s\nabla f(x_k)$  corresponds to the gradient flow  $\dot{X}(t) = -\nabla f(X(t))$ . Building upon this idea, Su et al. [36] derived the limiting ODE for Nesterov's accelerated gradient method (AGM) [27] and analyzed its convergence rate, offering valuable insights into momentum-based algorithms. A common approach to establishing convergence rates of continuous-time ODE models involves using Lyapunov functions [23].
In this paper, we propose a generic framework that builds upon the performance estimation problem (PEP) presented in [8] for analyzing the convergence rates of ODE models for various accelerated first-order methods, including Nesterov's AGM. The proposed method is designed from the Lagrangian dual of a relaxed version of continuous-time PEP. Consequently, our framework transforms the task of proving convergence rate into verifying the positive semidefiniteness of a specific integral kernel. Moreover, our framework can also ensure the tightness of the resulting guarantee, meaning that the obtained convergence guarantee is optimal among all possible convergence rates that can be derived from weighted integrals of the inequalities employed in convergence proofs. Using the proposed framework, we confirm the convergence rates of existing ODE models and uncover those of new accelerated ODE models. In traditional convergence analysis of ODE models, it can be challenging to design appropriate Lyapunov functions. However, in our framework, we only need to verify
There is a line of work focused on systematically finding Lyapunov functions, often with the assistance of computers. See Section 1.1 for details.
the positive semidefiniteness of a specific integral kernel. This approach circumvents the need for Lyapunov function design, making our framework more straightforward for analyzing convergence rates.
In the discrete-time setting, the PEP framework has been extensively studied for its ability to systematically obtain tight convergence guarantees [42] and facilitate the design of new optimization methods [17, 18, 40]. However, analyzing PEP is typically regarded as challenging to comprehend due to the involvement of large matrices with complex expressions. In contrast, our framework utilizes integral kernels, which serve as a continuous-time counterpart to matrices. The computational process within our approach yields simpler outcomes. Consequently, our continuous-time PEP framework has the potential to offer valuable insights into the analysis of the discrete-time PEP, similar to how ODE models have helped designing and analyzing discrete-time methods in the literature [20, 47]. By bridging the gap between the continuous and discrete settings, our methodology enhances the understanding of the PEP framework.
Continuous-time models for first-order methods. The investigation into the continuous-time limit of accelerated first-order methods began with the study of AGM ODE [36, 2, 3]. Since then, subsequent researches have explored various aspects of the ODE models. These include generalizations within the mirror descent setup [20], a broader family of dynamics derived using Lagrangian mechanics [47, 48, 19], high-resolution ODE models [34, 33], and continuized methods [9]. Systematic methodologies for finding Lyapunov functions were developed, including deriving them from Hamilton's equations [6] or dilated coordinate systems [37]. For obtaining accelerated discrete-time algorithms, several studies have applied discretization schemes, such as symplectic [4] and Runge-Kutta [49] schemes, to discretize accelerated ODE models. [32] showed that applying multi-step integration schemes to the gradient flow also yields accelerated algorithms. A particularly relevant study is [19], as they present the dynamics in the form of (4) using the  $H$ -kernel, which plays a crucial role in our analysis.
Performance estimation problems. The idea of using performance estimation problems to analyze the convergence rate of optimization methods was first introduced by [8]. This concept was further refined by employing a convex interpolation argument in [42] and was applied to a wide range of settings in [5, 43, 44, 12, 11, 7, 16]. The idea of performance estimation has been used to construct Lyapunov functions in [41, 39, 25, 45]. In particular, [25] analyzes continuous-time ODE models. However, their methodology differs from ours, as they employed semidefinite programs of finite dimension. Another closely related approach is based on integral quadratic constraints (IQC) from control theory [24], which were used to analyze the convergence rate of first-order methods in [21]. The IQC framework has been further studied in [14, 15, 10, 22, 31]. One practical application of PEP and IQC is the design of novel algorithms by optimizing the convergence guarantees. Some notable examples include OGM [17], TMM [46], ITEM [40], and OGM-G [18].
2 Preliminaries and notations
In this section, we review some basic notions from functional analysis that will be used throughout the paper. For a more detailed treatment, we refer the reader to the textbooks [29, 30, 28].
Function spaces. We denote the set of continuous functions from  $[0,T]$  to  $\mathbb{R}^d$  by  $C([0,T];\mathbb{R}^d)$  and the set of continuously differentiable functions from  $[0,T]$  to  $\mathbb{R}^d$  by  $C^1 ([0,T];\mathbb{R}^d)$ . We define the space  $L^2 ([0,T];\mathbb{R}^d)$  as the set of all measurable functions  $f:[0,T]\to \mathbb{R}^d$  that satisfy  $\int_0^T\| f(x)\|_{\mathbb{R}^d}^2 dx < \infty$ . Then,  $L^2 ([0,T];\mathbb{R}^d)$  is a Hilbert space, equipped with an inner product and a norm defined by  $\langle f,g\rangle_{L^2 ([0,T];\mathbb{R}^d)} = \int_0^T\langle f(t),g(t)\rangle_{\mathbb{R}^d}dt$  and  $\| f\|_{L^2 ([0,T];\mathbb{R}^d)} = \sqrt{\langle f,f\rangle_{L^2([0,T];\mathbb{R}^d)}}$ .
Integral operators. An integral operator is a linear operator that maps a function  $f$  to another function  $Kf$  given by
$$
(K f) (t) = \int_ {0} ^ {T} k (t, \tau) f (\tau) d \tau , \tag {2}
$$
where  $k:[0,T]^2 \to \mathbb{R}$  is the associated integral kernel. Intuitively, an integral kernel can be seen as a continuous-time version of a matrix. A Hilbert-Schmidt kernel is an integral kernel  $k$  that is square integrable, i.e.,  $k \in L^2([0,T]^2; \mathbb{R})$ . When  $k$  is a Hilbert-Schmidt kernel, the associated integral operator  $K$  is a well-defined operator on  $L^2([0,T]; \mathbb{R}^d)$ , called a Hilbert-Schmidt integral operator. If a Hilbert-Schmidt kernel  $k$  is symmetric, i.e.,  $k(t,\tau) = k(\tau,t)$  for all  $t,\tau \in [0,T]$ , then the associated operator is also symmetric in the sense that  $\langle Kf,g\rangle = \langle f,Kg\rangle$  for all  $f,g \in L^2([0,T];\mathbb{R}^d)$ . Throughout this paper, we will use the term 'kernel' to refer to a Hilbert-Schmidt kernel.
Positive semidefinite kernels. A symmetric operator  $K$  on a Hilbert space is said to be positive semidefinite and denoted by  $K \succeq 0$  if  $\langle Kf, f \rangle \geq 0$  for all  $f$ . When a symmetric kernel  $k$  is associated with a positive semidefinite operator  $K$ , i.e.,  $\int_0^T \int_0^T k(t, \tau) f(t) f(\tau) \, dt \, d\tau \geq 0$  for all  $f \in L^2([0, T]; \mathbb{R})$ , we say that the kernel  $k$  is (integrally) positive semidefinite and denote it by  $k \succeq 0$ . For continuous kernels, positive semidefiniteness of  $k$  is equivalent to the following condition:  $\sum_{i=1}^{n} \sum_{j=1}^{n} c_i c_j k(t_i, t_j) \geq 0$  for any  $t_1, \ldots, t_n \in [0, T]$  and  $c_1, \ldots, c_n \in \mathbb{R}$ , given  $n \in \mathbb{N}$ .
Proposition 1. We summarize some basic properties of continuous positive semidefinite kernels:
(a) For  $\alpha \in C([0,T];\mathbb{R})$  , the kernel  $k(t,\tau) = \alpha (t)\alpha (\tau)$  is positive semidefinite.
(b) For  $k_{1}, k_{2} \succeq 0$ , their product  $k(t, \tau) = k_{1}(t, \tau)k_{2}(t, \tau)$  is positive semidefinite.
(c) For  $k \succeq 0$ , its anti-transpose  $(t, \tau) \mapsto k(T - \tau, T - t)$  is also positive semidefinite.
(d) If  $\alpha \in C^{1}([0,T];\mathbb{R}_{\geq 0})$  is an increasing function on  $[0,T]$ , then the symmetric kernel  $k$  defined as  $k(t,\tau) = \alpha (\tau)$  for  $t\geq \tau$  and  $k(t,\tau) = \alpha (t)$  for  $t\leq \tau$  is positive semidefinite.3
(e) For  $k \succeq 0$ , we have  $k(t, t) \geq 0$  for all  $t \in [0, T]$ .
3 Continuous PEP for minimizing objective function value
In this section, drawing inspiration from its discrete-time counterpart [8, 42], we propose a novel framework for analyzing the convergence rate of ODE models for first-order methods, called the continuous-time performance estimation problem (Continuous PEP). To illustrate this framework, we use the accelerated gradient flow as an example. Detailed steps can be found in Appendix C. Su et al. [36] derived the limiting ODE of Nesterov's AGM [27] as follows:
$$
\ddot {X} + \frac {3}{t} \dot {X} + \nabla f (X) = 0, \tag {AGMODE}
$$
with initial conditions  $X(0) = x_0$  and  $\dot{X}(0) = 0$ . Suppose we want to establish a convergence guarantee of AGM ODE in the form of
$$
f (X (T)) - f \left(x ^ {*}\right) \leq \rho \| x _ {0} - x ^ {*} \| ^ {2}. \tag {3}
$$
Here, we observe that the constant  $\rho$  can be seen as an upper bound of the performance of AGM ODE for the criterion  $(f(X(T)) - f(x^{*})) / \| x_{0} - x^{*}\|^{2}$ . To formalize this idea, we introduce the following optimization problem, which seeks to find the worst-case performance of the given ODE model:
$$
\max  _ { \begin{array}{l} f \in \mathcal {F} _ {0} (\mathbb {R} ^ {d}; \mathbb {R}) \\ X \in C ^ {1} ([ 0, T ]; \mathbb {R} ^ {d}) \end{array} } \frac {f (X (T)) - f (x ^ {*})}{\| x _ {0} - x ^ {*} \| ^ {2}} \tag {Exact PEP}
$$
subject to  $X$  is a solution to AGM ODE with  $X(0) = x_0$ ,  $\dot{X}(0) = 0$
$x^{*}$  is a minimizer of  $f$
where  $\mathcal{F}_{\mu}(\mathbb{R}^d;\mathbb{R})$  denotes the set of continuously differentiable  $\mu$ -strongly convex functions on  $\mathbb{R}^d$ . This problem is useful to analyze the convergence properties of ODE models because the optimal value  $\mathrm{val}(\mathrm{Exact~PEP})$  of Exact PEP directly provides the guarantee (3) with  $\rho = \mathrm{val}(\mathrm{Exact~PEP})$  regardless of any particular choice of  $f$ .
Proof sketch: The kernel  $k(t,\tau)$  can be expressed as a weighted integral of positive semidefinite kernels as  $k(t,\tau) = \alpha (0)\mathbf{1}_{[0,T]}(t)\mathbf{1}_{[0,T]}(\tau) + \int_0^T\dot{\alpha} (s)\mathbf{1}_{[s,T]}(t)\mathbf{1}_{[s,T]}(\tau)ds.$
3.1 Relaxation of PEP
Exact PEP is challenging to solve due to the presence of an unknown function  $f$  as an optimization variable. To address this difficulty, we relax the constraint  $f \in \mathcal{F}_0(\mathbb{R}^d; \mathbb{R})$  with a set of inequalities that are satisfied by  $f \in \mathcal{F}_0(\mathbb{R}^d; \mathbb{R})$ . Before that, we first note that AGM ODE can be expressed as the following continuous-time dynamical system (see [19]):
$$
\dot {X} (t) = - \int_ {0} ^ {t} H (t, \tau) \nabla f (X (\tau)) d \tau \tag {4}
$$
by setting  $H(t,\tau) = \tau^3 /t^3$ . Here,  $H(t,\tau)$  is called the  $H$ -kernel. We introduce two functions,  $\varphi :[0,T]\to \mathbb{R}$  and  $\gamma :[0,T]\to \mathbb{R}^d$ , defined as follows:
$$
\varphi (t) = \frac {1}{\| x _ {0} - x ^ {*} \| ^ {2}} \left(f (X (t)) - f \left(x ^ {*}\right)\right), \quad \gamma (t) = \frac {1}{\| x _ {0} - x ^ {*} \|} \nabla f (X (t)).
$$
Using the chain rule and the convexity of  $f$ , we can derive the following equality and inequality:
$$
0 = \dot {\varphi} (t) + \left\langle \gamma (t), \int_ {0} ^ {t} H (t, \tau) \gamma (\tau) d \tau \right\rangle , \tag {5}
$$
$$
0 \geq \varphi (t) + \left\langle \gamma (t), v + \int_ {0} ^ {t} \int_ {\tau} ^ {t} H (s, \tau) \gamma (\tau) d s d \tau \right\rangle ,
$$
where  $v = (x^{*} - x_{0}) / \| x_{0} - x^{*}\|$ . We can now relax Exact PEP by replacing its constraints with the equality and inequality above, resulting in the following problem:
$$
\begin{array}{l l} \max  _ {\varphi , \gamma , v} & \varphi (T) \\ \text {s u b j e c t t o} & (5) \text {h o l d s f o r a l l} t \in (0, T). \end{array} \tag {Relaxed PEP}
$$
Since any feasible solution to Exact PEP can be transformed into a feasible solution to Relaxed PEP, we have  $\mathrm{val}(\text{Relaxed PEP}) \geq \mathrm{val}(\text{Exact PEP})$ . Therefore, the convergence guarantee (3) holds with  $\rho = \mathrm{val}(\text{Relaxed PEP})$  when using the proposed relaxation.
3.2 Lagrangian dual of relaxed PEP
To obtain an upper bound on val(Relaxed PEP), we use Lagrangian duality. We introduce two Lagrange multipliers  $\lambda_1 \in C^1([0,T];\mathbb{R})$  and  $\lambda_2 \in C([0,T];\mathbb{R}_{\geq 0})$ , where we imposed certain regularity conditions, such as continuity and differentiability, to ensure that the dual problem is well-defined. We then define the Lagrangian function as
$$
\begin{array}{l} \mathcal {L} (\varphi , \gamma , v; \lambda_ {1}, \lambda_ {2}) = \varphi (T) - \int_ {0} ^ {T} \lambda_ {1} (t) \left(\dot {\varphi} (t) + \left\langle \gamma (t), \int_ {0} ^ {t} H (t, \tau) \gamma (\tau) d \tau \right\rangle\right) d t \\ \left. - \int_ {0} ^ {T} \lambda_ {2} (t) \left(\varphi (t) + \left\langle \gamma (t), v + \int_ {0} ^ {t} \int_ {\tau} ^ {t} H (s, \tau) \gamma (\tau) d s d \tau \right\rangle\right) d t. \right. \\ \end{array}
$$
When expressed in terms of the inner products in function spaces, we have
$$
\begin{array}{l} \mathcal {L} (\varphi , \gamma , v; \lambda_ {1}, \lambda_ {2}) = \varphi (T) - \left\langle \lambda_ {1}, \dot {\varphi} \right\rangle_ {L ^ {2} ([ 0, T ]; \mathbb {R})} - \left\langle \lambda_ {2}, \varphi \right\rangle_ {L ^ {2} ([ 0, T ]; \mathbb {R})} \\ - \frac {1}{2} \left\langle K \gamma , \gamma \right\rangle_ {L ^ {2} ([ 0, T ]; \mathbb {R} ^ {d})} - \left\langle \lambda_ {2} (t) v, \gamma (t) \right\rangle_ {L ^ {2} ([ 0, T ]; \mathbb {R} ^ {d})}, \tag {6} \\ \end{array}
$$
where  $K$  is the Hilbert-Schmidt integral operator with the symmetric kernel  $k$  defined by
$$
k (t, \tau) = \lambda_ {1} (t) H (t, \tau) + \lambda_ {2} (t) \int_ {\tau} ^ {t} H (s, \tau) d s, \quad t \geq \tau .
$$
The dual function is defined as  $\mathrm{Dual}(\lambda_1,\lambda_2) = \sup_{\varphi ,\gamma ,v}\mathcal{L}(\varphi ,\gamma ,v;\lambda_1,\lambda_2)$ . By weak duality, we have  $\mathrm{val}(\mathrm{Relaxed~PEP})\leq \mathrm{Dual}(\lambda_1,\lambda_2)$  for any feasible dual solution  $(\lambda_1,\lambda_2)$ . After performing some computations, we obtain the following expression for the dual objective function (see Appendix C):
$$
\operatorname {D u a l} \left(\lambda_ {1}, \lambda_ {2}\right) = \left\{ \begin{array}{l l} \inf  _ {\nu \in (0, \infty)} \left\{\nu : S _ {\lambda_ {1}, \lambda_ {2}, \nu} \succeq 0 \right\} & \text {i f} \lambda_ {1} (0) = 0, \lambda_ {1} (T) = 1 \dot {\lambda} _ {1} (t) = \lambda_ {2} (t) \\ \infty & \text {o t h e r w i s e ,} \end{array} \right. \tag {7}
$$
where  $S_{\lambda_1,\lambda_2,\nu}$  is a symmetric kernel on  $[0,T]^2$  given by
$$
S _ {\lambda_ {1}, \lambda_ {2}, \nu} (t, \tau) = \nu \left(\lambda_ {1} (t) H (t, \tau) + \lambda_ {2} (t) \int_ {\tau} ^ {t} H (s, \tau) d s\right) - \frac {1}{2} \lambda_ {2} (t) \lambda_ {2} (\tau), \quad t \geq \tau . \tag {8}
$$
We refer to  $S_{\lambda_1,\lambda_2,\nu}$  as the PEP kernel. In Appendix H.2, we show that (8) can be viewed as the continuous-time limit of the discrete-time PEP kernel presented in [8].
To describe our framework, given  $\nu_{\mathrm{feas}} \in (0,\infty)$ , suppose that the PEP kernel  $S_{\lambda_1,\lambda_2,\nu_{\mathrm{feas}}}$  is positive semidefinite with appropriate multiplier functions  $\lambda_1$  and  $\lambda_2$ . Then,  $\nu_{\mathrm{feas}}$  is a feasible solution of the minimization problem in (7), and thus  $\mathrm{Dual}(\lambda_1,\lambda_2) \leq \nu_{\mathrm{feas}}$ . On the other hand, by weak duality,  $\mathrm{val}(\mathrm{Relaxed~PEP}) \leq \inf \mathrm{Dual}(\lambda_1,\lambda_2)$ . Therefore, we conclude that  $\mathrm{val}(\mathrm{Exact~PEP}) \leq \mathrm{val}(\mathrm{Relaxed~PEP}) \leq \mathrm{Dual}(\lambda_1,\lambda_2) \leq \nu_{\mathrm{feas}}$ , which implies that the convergence guarantee (3) automatically holds with  $\rho = \nu_{\mathrm{feas}}$ .
Using this approach, we can recover the known convergence guarantee for AGM ODE in [36].
Proposition 2. AGM ODE achieves the convergence rate (3) with  $\rho = 2 / T^2$ .
Proof. By choosing the multiplier functions  $\lambda_1(t) = t^2 / T^2$  and  $\lambda_2(t) = 2t / T^2$ , we can compute the PEP kernel (8) as  $S_{\lambda_1, \lambda_2, \nu}(t, \tau) = (\nu - \frac{2}{T^2}) \frac{t\tau}{T^2}$ . Since the kernel  $(t, \tau) \mapsto t\tau$  is nonzero and positive semidefinite, we have  $S_{\lambda_1, \lambda_2, \nu} \succeq 0$  if and only if  $\nu \geq 2 / T^2$ . Thus, we obtain  $\mathrm{Dual}(\lambda_1, \lambda_2) = 2 / T^2$ , which establishes the convergence guarantee (3) with  $\rho = 2 / T^2$ .
Remark 1. Furthermore, this convergence guarantee is optimal among all possible guarantees obtained through the weighted integral of (5). The optimality of this rate follows from the fact that  $(\lambda_1,\lambda_2)$  is the optimal solution to the dual problem  $\min_{\lambda_1,\lambda_2}\mathrm{Dual}(\lambda_1,\lambda_2)$ . See Appendix H.1 for details.
3.3 Applying continuous PEP to various accelerated gradient flows
Note that the proposed method is not dependent on the choice of the H-kernel  $H(t,\tau)$ . Thus, it can be applied to arbitrary dynamics represented in the form of (4). Furthermore, while we have focused on the non-strongly convex case  $(\mu = 0)$  so far, the following paragraph demonstrates that our method can handle strongly convex objective functions by using a reparametrization technique.
Reparametrization from  $\mathcal{F}_{\mu}(\mathbb{R}^d;\mathbb{R})$  to  $\mathcal{F}_0(\mathbb{R}^d;\mathbb{R})$ . Consider a  $\mu$ -strongly convex objective function  $f$ . Since the proposed method is tailored for non-strongly convex objective functions, we choose to work with the convex function  $\hat{f}(x) \coloneqq f(x) - \frac{\mu}{2}\|x - x_0\|^2$  rather than working directly with  $f$ . Accordingly, we consider the following alternative formulation for the dynamical system (4), which involves  $\nabla \hat{f}$  instead of  $\nabla f$ :
$$
\dot {X} (t) = - \int_ {0} ^ {t} H ^ {F} (t, \tau) \nabla \hat {f} (X (\tau)) d \tau . \tag {9}
$$
The following theorem offers a general result that can be used to establish convergence guarantees for dynamical systems of the form (9).
Theorem 1. Let  $\nu >0$  and  $\lambda^F\in C^1 ([0,T];\mathbb{R}_{\geq 0})$  such that  $0\leq \lambda^{F}(0) < 1$ ,  $\lambda^F (T) = 1$ , and  $\dot{\lambda}^F (t)\geq 0$  for all  $t\in (0,T)$ . Then, any solution to the integro-differential equation (9) satisfies
$$
\tilde {f} (X (T)) - \tilde {f} (x ^ {*}) \leq \lambda^ {F} (0) \left(\tilde {f} (x _ {0}) - \tilde {f} (x ^ {*})\right) + \nu \| x _ {0} - x ^ {*} \| ^ {2},
$$
where  $\tilde{f}(x) \coloneqq f(x) - \frac{\mu}{2} \| x - x^{*} \|^2$ , if the following PEP kernel is positive semidefinite:
$$
S ^ {F} (t, \tau) = \nu \left(\lambda^ {F} (t) H ^ {F} (t, \tau) + \dot {\lambda} ^ {F} (t) \int_ {\tau} ^ {t} H ^ {F} (s, \tau) d s\right) - 2 \alpha^ {F} (t) \alpha^ {F} (\tau), \quad t \geq \tau , \tag {10}
$$
where  $\alpha^F (t) = \frac{1}{2}\frac{d}{dt}\{\lambda^F (t)(1 - \mu \int_0^t\int_0^s H^F (s,\tau)d\tau ds)\} .$
The proof of Theorem 1 is done by finding a dual feasible point to the PEP and can be found in Appendix C. Below, we establish convergence rates for various ODE models using Theorem 1.
4 In discrete PEP literature, a similar reparametrization technique was employed in [40, 13].
The equivalent representations (4) and (9) are in a one-to-one correspondence. See Appendix C.1 for details.
(a) Contour plot.
(b) Surface plot.
Figure 1: Visualization of the PEP kernel (11) for AGM-SC ODE.
AGM-SC ODE. We consider the following dynamical system modeling Nesterov's AGM for strongly convex case [26, Equation 2.2.22] (see [48, Equation 7]):
$$
\ddot {X} + 2 \sqrt {\mu} \dot {X} + \nabla f (X) = 0. \tag {AGM-SC ODE}
$$
This ODE model can be written as (9) with  $H^{F}(t,\tau) = (1 + \sqrt{\mu}\tau -\sqrt{\mu} t)e^{\sqrt{\mu} (\tau -t)}$  (see Appendix F.1). To use Theorem 1, we choose the multiplier function as  $\lambda^{F}(t) = e^{\sqrt{\mu}(t - T)}$ . The PEP kernel (10) can be computed as (see Appendix G.1)
$$
S ^ {F} (t, \tau) = \nu e ^ {\sqrt {\mu} (\tau - T)} - \frac {\mu}{2} e ^ {- 2 \sqrt {\mu} T}, \quad t \geq \tau . \tag {11}
$$
When  $\nu = \frac{\mu}{2} e^{-\sqrt{\mu} T}$ , this kernel is written as  $S^{F}(t,\tau) = \frac{\mu}{2} e^{-2\sqrt{\mu} T}(e^{\sqrt{\mu}\tau} - 1)$ , and is visualized in Figure 1. It is positive semidefinite since the function  $\tau \mapsto e^{\sqrt{\mu}\tau} - 1$  is a nonnegative increasing function (see Proposition 1 (d)). It follows from Theorem 1 that AGM-SC ODE achieves the following convergence guarantee:
$$
\tilde {f} (X (T)) - \tilde {f} (x ^ {*}) \leq e ^ {- \sqrt {\mu} T} \left(\tilde {f} (x _ {0}) - \tilde {f} (x ^ {*}) + \frac {\mu}{2} \| x _ {0} - x ^ {*} \| ^ {2}\right), \tag {12}
$$
which is consistent with the well-known  $O(e^{-\sqrt{\mu} T})$  convergence rate of AGM-SC ODE.
Unified AGM ODE. Using a unified Bregman Lagrangian framework, [19] obtained the following ODE that unifies AGM ODE and AGM-SC ODE:
$$
\ddot {X} + \frac {\sqrt {\mu}}{2} \left(\tanh  _ {t} + 3 \coth_ {t}\right) \dot {X} + \nabla f (X) = 0, \quad \text {(U n i f i e d A G M O D E)}
$$
where  $\tanh_t$  and  $\coth_t$  denote the corresponding hyperbolic functions with the argument  $\frac{\sqrt{\mu}}{2} t$ . This ODE model can be written as (9) with  $H^F(t, \tau) = (1 + \coth_t^2 (\log(\mathrm{sech}_t^2) - \log(\mathrm{sech}_\tau^2))) \frac{\sinh_\tau \cosh_\tau}{\sinh_t \cosh_t}$  (see Appendix F.2). We select the multiplier function as  $\lambda^F(t) = \sinh_t^2 / \sinh_T^2$ . With this choice, the PEP kernel (10) can be expressed as follows (see Appendix G.2):
$$
S ^ {F} (t, \tau) = \left(\nu - \frac {\mu}{2} \operatorname {c s c h} _ {T} ^ {2}\right) \frac {\tanh  _ {t} \tanh  _ {\tau}}{\sinh_ {T} ^ {2}} + \nu \frac {\tanh  _ {t} \tanh  _ {\tau} \sinh_ {\tau} ^ {2}}{\sinh_ {T} ^ {2}}, \quad t \geq \tau . \tag {13}
$$