# Convex optimization via inertial algorithms with vanishing Tikhonov regularization: fast convergence to the minimum norm solution

Hedy Attouch, Szilárd Csaba László

► To cite this version:

Hedy Attouch, Szilárd Csaba László. Convex optimization via inertial algorithms with vanishing Tikhonov regularization: fast convergence to the minimum norm solution. Mathematical Methods of Operations Research, 2024, 99, pp.307-347. 10.1007/s00186-024-00867-y. hal-05077575

HAL Id: hal-05077575

https://hal.science/hal-05077575v1

Submitted on 21 May 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
ORIGINAL ARTICLE

<figure><img src="image_2.png" /></figure>

Check for
updates

Convex optimization via inertial algorithms with vanishing
Tikhonov regularization: fast convergence to the minimum
norm solution

Hedy Attouch<fnref n="1" /> · Szilárd Csaba László<fnref n="2" />

<figure><img src="image_3.png" /></figure>

Received: 13 March 2024 / Revised: 20 April 2024 / Accepted: 3 May 2024 /
Published online: 27 June 2024

© The Author(s) 2024

Abstract

In a Hilbertian framework, for the minimization of a general convex differentiable function *f*, we introduce new inertial dynamics and algorithms that generate trajectories and iterates that converge fastly towards the minimizer of *f* with minimum norm. Our study is based on the non-autonomous version of the Polyak heavy ball method, which, at time *t*, is associated with the strongly convex function obtained by adding to *f* a Tikhonov regularization term with vanishing coefficient *ε*(*t*). In this dynamic, the damping coefficient is proportional to the square root of the Tikhonov regularization parameter *ε*(*t*). By adjusting the speed of convergence of *ε*(*t*) towards zero, we will obtain both rapid convergence towards the infimal value of *f*, and the strong convergence of the trajectories towards the element of minimum norm of the set of minimizers of *f*. In particular, we obtain an improved version of the dynamic of Su-Boyd-Candès for the accelerated gradient method of Nesterov. This study naturally leads to corresponding first-order algorithms obtained by temporal discretization. In the case of a proper lower semicontinuous and convex function *f*, we study the proximal algorithms in detail, and show that they benefit from similar properties.

**Keywords** Accelerated gradient methods · Convex optimization · Damped inertial dynamics · Minimum norm solution · Nesterov accelerated gradient method · Tikhonov approximation

S.C. László: This work was supported by a grant of the Ministry of Research, Innovation and Digitization, CNCS-UEFISCDI, project number PN-III-P1-1.1-TE-2021-0138, within PNCDI III.

<figure><img src="image_4.png" /></figure>

Szilárd Csaba László
szilard.laszlo@math.utcluj.ro

Hedy Attouch
hedy.attouch@umontpellier.fr

<footnote n="1">1 IMAG, Univ. Montpellier, CNRS, Montpellier, France</footnote>

<footnote n="2">² Department of Mathematics, Technical University of Cluj-Napoca, Str. Memorandumului nr. 28, 400114 Cluj-Napoca, Romania</footnote>
# 1 Introduction

Throughout the paper, $\mathcal{H}$ is a real Hilbert space which is endowed with the scalar product $\langle \cdot, \cdot \rangle$, with $\|x\|^2 = \langle x, x \rangle$ for $x \in \mathcal{H}$. We consider the convex minimization problem

$$ \min \{f(x) : x \in \mathcal{H}\}, \qquad (1) $$

where $f: \mathcal{H} \to \mathbb{R}$ is a convex continuously differentiable function whose solution set $S = \underset{\mu}{\operatorname{argmin}} f$ is nonempty. We aim at finding by rapid methods the element of minimum norm of $S$. As an original aspect of our approach, we start from the Polyak heavy ball with friction dynamic for strongly convex functions, and then adapt it to treat the case of general convex functions. Recall that a function $f: \mathcal{H} \to \mathbb{R}$ is said to be $\mu$-strongly convex for some $\mu > 0$ if $f - \frac{\mu}{2} \| \cdot \|^2$ is convex. In this setting, we have the exponential convergence result:

**Theorem 1** Suppose that $f: \mathcal{H} \to \mathbb{R}$ is a function of class $C^1$ which is $\mu$-strongly convex for some $\mu > 0$. Let $x(\cdot) : [t_0, +\infty[ \to \mathcal{H}$ be a solution trajectory of

$$ \ddot{x}(t) + 2\sqrt{\mu}\dot{x}(t) + \nabla f(x(t)) = 0. \qquad (2) $$

Then, the following property holds: $f(x(t)) - \min_{\mathcal{H}} f = O(e^{-\sqrt{\mu t}})$ as $t \to +\infty$.

Let us see how to take advantage of this fast convergence result, and how to adapt it to the case of a general convex differentiable function $f: \mathcal{H} \to \mathbb{R}$. The main idea is linked to Tikhonov's method of regularization. It consists in considering the corresponding non-autonomous dynamic which at time $t$ is governed by the gradient of the strongly convex function $f_t: \mathcal{H} \to \mathbb{R}$

$$ f_t(x) := f(x) + \frac{\varepsilon(t)}{2} \|x\|^2. $$

Then replacing $f$ by $f_t$ in (2), and noticing that $f_t$ is $\varepsilon(t)$-strongly convex, we obtain the dynamic

$$ (\text{TRIGS}) \quad \ddot{x}(t) + \delta\sqrt{\varepsilon(t)}\dot{x}(t) + \nabla f(x(t)) + \varepsilon(t)x(t) = 0, $$

with $\delta = 2$. (TRIGS) stands shortly for Tikhonov regularization of inertial gradient systems. In order not to asymptotically modify the equilibria, we suppose that $\varepsilon(t) \to 0$ as $t \to +\infty$. This condition implies that (TRIGS) falls within the framework of the inertial gradient systems with asymptotically vanishing damping. The importance of this class of inertial dynamics has been highlighted by several recent studies (Apidopoulos et al. 2018; Attouch et al. 2023; Attouch and Cabot 2017; Attouch et al.
2018; Attouch and Peypouquet 2016; Chambolle and Dossal 2015; Su et al. 2016), which make the link with the accelerated gradient method of Nesterov Nesterov (1983, 2004).

# 1.1 Historical facts and related results

In relation to optimization algorithms, a rich literature has been devoted to the coupling of dynamic gradient systems with Tikhonov regularization.

## 1.1.1 First-order gradient dynamics

For first-order gradient systems and subdifferential inclusions, the asymptotic hierarchical minimization property which results from the introduction of a vanishing viscosity term in the dynamic (in our context the Tikhonov approximation (Tikhonov 1963; Tikhonov and Arsenin 1977)) has been highlighted in a series of papers (Alvarez and Cabot 2006; Attouch 1996; Attouch and Cominetti 1996; Attouch and Czarnecki 2010; Baillon and Cominetti 2001; Cominetti et al. 2008; Hirstoaga 2006). In parallel way, there is a vast literature on convex descent algorithms involving Tikhonov and more general penalty, regularization terms. The historical evolution can be traced back to Fiacco and McCormick (1968), and the interpretation of interior point methods with the help of a vanishing logarithmic barrier. Some more specific references for the coupling of Prox and Tikhonov can be found in Cominetti (1997). The time discretization of the first-order gradient systems and subdifferential inclusions involving multiscale (in time) features provides a natural link between the continuous and discrete dynamics. The resulting algorithms combine proximal based methods (for example forward-backward algorithms), with the viscosity of penalization methods, see (Attouch et al. 2011a,b; Bot and Csetnek 2014; Cabot 2004, 2005; Hirstoaga 2006).

## 1.1.2 Second order gradient dynamics

First studies concerning the coupling of damped inertial dynamics with Tikhonov approximation concerned the heavy ball with friction system of Polyak (1987), where the damping coefficient $γ > 0$ is fixed. In Attouch and Czarnecki (2002) Attouch-Czarnecki considered the system

$$ \ddot{x}(t) + \gamma \dot{x}(t) + \nabla f(x(t)) + \varepsilon(t)x(t) = 0. \quad (3) $$

In the slow parametrization case $\int_{0}^{+\infty} \varepsilon(t)dt = +\infty$, they proved that any solution $x(\cdot)$ of (3) converges strongly to the minimum norm element of argmin $f$, see also Jendoubi and May (2010). A parallel study has been developed for PDE's, see Alvarez and Attouch (2001) for damped hyperbolic equations with non-isolated equilibria, and Alvarez and Cabot (2006) for semilinear PDE's. The system (3) is a special case of the general dynamic model

$$ \ddot{x}(t) + \gamma \dot{x}(t) + \nabla f(x(t)) + \varepsilon(t) \nabla g(x(t)) = 0 \quad (4) $$
which involves two functions $f$ and $g$ intervening with different time scale. When $\varepsilon(\cdot)$ tends to zero moderately slowly, it was shown in Attouch and Czarnecki (2017) that the trajectories of (4) converge asymptotically to equilibria that are solutions of the following hierarchical problem: they minimize the function $g$ on the set of minimizers of $f$. When $\mathcal{H} = \mathcal{H}_1 \times \mathcal{H}_2$ is a product space, defining for $x = (x_1, x_2)$, $f(x_1, x_2) := f_1(x_1) + f_2(x_2)$ and $g(x_1, x_2) := \|A_1x_1 - A_2x_2\|^2$, where the $A_i$, $i \in \{1, 2\}$ are linear operators, (4) provides (weakly) coupled inertial systems. The continuous and discrete-time versions of these systems have a natural connection to the best response dynamics for potential games (Attouch and Czarnecki 2010), domain decomposition for PDE's (Attouch et al. 2016), optimal transport (Attouch et al. 2010), coupled wave equations (Haraux and Jendoubi 2016).

In the quest for a faster convergence, the following system

$$ (\text{AVD})_{\alpha, \varepsilon} \quad \ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t) + \nabla f(x(t)) + \varepsilon(t)x(t) = 0, \quad (5) $$

has been studied by Attouch et al. (2018). It is a Tikhonov regularization of the dynamic

$$ (\text{AVD})_{\alpha} \quad \ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t) + \nabla f(x(t)) = 0, \quad (6) $$

which was introduced by Su et al. (2016). When $\alpha = 3$, $(\text{AVD})_{\alpha}$ can be viewed as a continuous version of the accelerated gradient method of Nesterov. It has been the subject of many recent studies which have given an in-depth understanding of the Nesterov acceleration method, see Apidopoulos et al. (2018); Attouch and Cabot (2017); Attouch et al. (2018); Su et al. (2016). The results obtained in Attouch et al. (2018) concerning (5) will serve as a basis for comparison.

## 1.2 Model results

To illustrate our results, let us consider the case $\varepsilon(t) = \frac{c}{t}$ where $r$ is positive parameter satisfying $0 < r \le 2$. The case $r = 2$ is of particular interest, it is related to the continuous version of the accelerated gradient method of Nesterov, with optimal convergence rate for general convex differentiable function $f$.

### 1.2.1 Case $r = 2$

Let us consider the (TRIGS) dynamic

$$ \ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t) + \nabla f(x(t)) + \frac{c}{t^2}x(t) = 0, \quad (7) $$

where the parameter $\alpha \ge 3$ plays a crucial role. As a consequence of Theorems 13 and 14 we have

**Theorem 2** Let $x : [t_0, +\infty[ \to \mathcal{H}$ be a solution of (7). We then have the following results:

<figure><img src="image_5.png" /></figure>
i) If $\alpha = 3$, then $f(x(t)) - \min_{\mathcal{H}} f = O\left(\frac{\ln t}{t^2}\right)$ as $t \to +\infty$.

ii) If $\alpha > 3$, then $f(x(t)) - \min_{\mathcal{H}} f = O\left(\frac{1}{t^2}\right)$ as $t \to +\infty$. Further, the trajectory $x$ is bounded, $\|\dot{x}(t)\| = O\left(\frac{1}{t}\right)$ as $t \to +\infty$, and there is strong convergence to the minimum norm solution:

$$ \liminf_{t \to +\infty} \|x(t) - x^*\| = 0. $$

### 1.2.2 Case $r < 2$

As a consequence of Theorems 11 and 16, we have:

**Theorem 3** Take $\varepsilon(t) = 1/t^r$, $\frac{2}{3} < r < 2$. Let $x : [t_0, +\infty[ \to \mathcal{H}$ be a global solution trajectory of

$$ \ddot{x}(t) + \frac{\delta}{t^{\frac{r}{2}}} \dot{x}(t) + \nabla f(x(t)) + \frac{1}{t^r} x(t) = 0. $$

Then, we have fast convergence the values, and strong convergence to the minimum norm solution:

$$ f(x(t)) - \min_{\mathcal{H}} f = \mathcal{O}\left(\frac{1}{t^{\frac{3r}{2}-1}}\right) \quad \text{and} \quad \liminf_{t \to +\infty} \|x(t) - x^*\| = 0. $$

These results are completed by showing that, if there exists $T \ge t_0$, such that the trajectory $\{x(t) : t \ge T\}$ stays either in the open ball $B(0, \|x^*\|)$ or in its complement, then $x(t)$ converges strongly to $x^*$ as $t \to +\infty$. Corresponding results for the associated proximal algorithms, obtained by temporal discretization, are obtained in Sect. 5.

A remarkable property of the above results is that the rate of convergence of values is comparable to the Nesterov accelerated gradient method. In addition, we have a strong convergence property to the minimum norm solution, with comparable numerical complexity. These results represent an important advance compared to previous works by producing new dynamics for which we have both rapid convergence of values and strong convergence towards the solution of minimum norm. Let us stress the fact that in our approach the fast convergence of the values and the strong convergence towards the solution of minimum norm are obtained for the same dynamic, whereas in the previous works (Attouch et al. 2018; Attouch and Czarnecki 2002), they are obtained for different dynamics obtained for different settings of the parameters. It is clear that the results extend naturally to obtaining strong convergence towards the solution closest to a desired state $x_d$. It suffices to replace in Tikhonov's approximation $\|x\|^2$ by $\|x - x_d\|^2$. This is important for inverse problems.

**Remark 4** We emphasize that in Attouch et al. (2022) the authors also considered the case $\varepsilon(t) = 1/t^r$, $0 < r < 2$ and showed the improved rates $f(x(t)) - \min_{\mathcal{H}} f =$
$O(\frac{1}{t^r})$ as $t \to +\infty$, $\|\dot{x}(t)\| = O(\frac{1}{\frac{t^r+2}{t^r}})$ as $t \to +\infty$ and the improved strong convergence result $\lim_{t \to \infty} \|x(t) - x^*\| = 0$. More precisely, by denoting $x_t$ the unique minimizer of the strongly convex function $f(x) + \frac{1}{2t^r}\|x\|^2$, it is shown in Attouch et al. (2022) that $\|x(t) - x_t\| = O(\frac{1}{\frac{2^r+2}{t^r}})$ as $t \to +\infty$ and this implies that $\lim_{t \to \infty} \|x(t) - x^*\| = 0$. The above results were further extended and improved in László (2023), where the rate $f(x(t)) - \min_{\mathcal{H}} f = O(\frac{1}{\frac{2^r+2}{t^r}})$ as $t \to +\infty$ has been obtained. Similar strong convergence results and fast rates were obtained in Attouch et al. (2023) and László (2024) for a system with explicit hessian driven damping and a system with implicit hessian driven damping, respectively.

Nevertheless, whether the strong convergence result $\lim_{t \to \infty} \|x(t) - x^*\| = 0$ can be obtained in case $r = 2$ is still a challenging open question.

## 1.3 Contents

In section 2, we show existence and uniqueness of a global solution for the Cauchy problem associated with (TRIGS). Then, based on Lyapunov analysis, we obtain convergence rates of the values which are valid for a general $\varepsilon(\cdot)$. Section 3 is devoted to an in-depth analysis in the critical case $\varepsilon(t) = c/t^2$. Section 4 is devoted to the study of the strong convergence property of the trajectories towards the minimum norm solution, in the case of a general $\varepsilon(\cdot)$. Then in Sect. 5 we obtain similar results for the associated proximal algorithms, obtained by temporal discretization.

## 2 Convergence analysis for general $\varepsilon(t)$

We are going to analyze via Lyapunov analysis the convergence properties as $t \to +\infty$ of the solution trajectories of the inertial dynamic (TRIGS) that we recall below

$$ \ddot{x}(t) + \delta\sqrt{\varepsilon(t)}\dot{x}(t) + \nabla f(x(t)) + \varepsilon(t)x(t) = 0. \quad (8) $$

Throughout the paper, we assume that $t_0$ is the origin of time, $\delta$ is a positive parameter, and

(H1) $f: \mathcal{H} \to \mathbb{R}$ is convex and differentiable, $\nabla f$ is Lipschitz continuous on bounded sets.

(H2) $S := \operatorname{argmin} f \neq \emptyset$. We denote by $x^*$ the element of minimum norm of $S$.

(H3) $\varepsilon: [t_0, +\infty[ \to \mathbb{R}^+$ is a nonincreasing function, of class $C^1$, such that $\lim_{t \to \infty} \varepsilon(t) = 0$.

### 2.1 Existence and uniqueness for the Cauchy problem

Let us first show that the Cauchy problem for (TRIGS) is well posed.
**Theorem 5** Given $(x_0, v_0) \in \mathcal{H} \times \mathcal{H}$, there exists a unique global classical solution $x: [t_0, +\infty[ \to \mathcal{H}$ of the Cauchy problem

$$ \begin{cases} \ddot{x}(t) + \delta\sqrt{\varepsilon(t)}\dot{x}(t) + \nabla f(x(t)) + \varepsilon(t)x(t) = 0 \\ x(t_0) = x_0, \quad \dot{x}(t_0) = v_0. \end{cases} \qquad (9) $$

*Proof* The proof relies on the combination of the Cauchy-Lipschitz theorem with energy estimates. First consider the Hamiltonian formulation of (9) as the first order system

$$ \begin{cases} \dot{x}(t) - y(t) = 0 \\ \dot{y}(t) + \delta\sqrt{\varepsilon(t)}y(t) + \nabla f(x(t)) + \varepsilon(t)x(t) = 0 \\ x(t_0) = x_0, \quad y(t_0) = v_0. \end{cases} \qquad (10) $$

According to the hypothesis $(H_1)$, $(H_2)$, $(H_3)$, and by applying the Cauchy-Lipschitz theorem in the locally Lipschitz case, we obtain the existence and uniqueness of a local solution. Then, in order to pass from a local solution to a global solution, we rely on the energy estimate obtained by taking the scalar product of (TRIGS) with $\dot{x}(t)$. It gives

$$ \frac{d}{dt} \left( \frac{1}{2} \|\dot{x}(t)\|^2 + f(x(t)) + \frac{1}{2} \varepsilon(t) \|x(t)\|^2 \right) + \delta \sqrt{\varepsilon(t)} \|\dot{x}(t)\|^2 - \frac{1}{2} \dot{\varepsilon}(t) \|x(t)\|^2 = 0. $$

From $(H_3)$, $\varepsilon(\cdot)$ is non-increasing. Therefore, the energy function $t \mapsto W(t)$ is decreasing where

$$ W(t) := \frac{1}{2} \|\dot{x}(t)\|^2 + f(x(t)) + \frac{1}{2} \varepsilon(t) \|x(t)\|^2. $$

The end of the proof follows a standard argument. Take a maximal solution defined on an interval $[t_0, T]$. If $T$ is infinite, the proof is over. Otherwise, if $T$ is finite, according to the above energy estimate, we have that $\|\dot{x}(t)\|$ remains bounded, just like $\|x(t)\|$ and $\|\ddot{x}(t)\|$ (use (TRIGS)). Therefore, the limit of $x(t)$ and $\dot{x}(t)$ exists when $t \to T$. Applying the local existence result at $T$ with the initial conditions thus obtained gives a contradiction to the maximality of the solution. □

## 2.2 General case

The control of the decay of $\varepsilon(t)$ to zero as $t \to +\infty$ will play a key role in the Lyapunov analysis of (TRIGS). Precisely, we will use the following condition.

**Definition 1** Given $\delta > 0$, we say that $t \mapsto \varepsilon(t)$ satisfies the controlled decay property $(\text{CD})_K$, if it is a nonincreasing function which satisfies: there exists $t_1 \ge t_0$ such that
for all $t \ge t_1$,

$$ \left( \frac{1}{\sqrt{\varepsilon(t)}} \right)' \le \min(2K - \delta, \delta - K), $$

where $K$ is a parameter such that $\frac{\delta}{2} < K < \delta$ for $0 < \delta \le 2$, and $\frac{\delta + \sqrt{\delta^2 - 4}}{2} < K < \delta$ for $\delta > 2$.

**Theorem 6** Let $x : [t_0, +\infty[ \to \mathcal{H}$ be a solution trajectory of (TRIGS). Let $\delta$ be a positive parameter. Suppose that $\varepsilon(\cdot)$ satisfies the condition (CD)$_K$ for some $K > 0$. Then, we have the following rate of convergence of values: for all $t \ge t_1$

$$ f(x(t)) - \min_{\mathcal{H}} f \le \frac{K \|x^*\|^2}{2} \frac{1}{\mathfrak{M}(t)} \int_{t_1}^{t} \varepsilon^{\frac{3}{2}}(s) \mathfrak{M}(s) ds + \frac{C}{\mathfrak{M}(t)}, \quad (11) $$

where

$$ \mathfrak{M}(t) = \exp \left( \int_{t_1}^{t} \mu(s) ds \right), \quad \mu(t) = - \frac{\dot{\varepsilon}(t)}{2\varepsilon(t)} + (\delta - K)\sqrt{\varepsilon(t)} $$

and

$$ C = (f(x(t_1)) - f(x^*)) + \frac{\varepsilon(t_1)}{2} \|x(t_1)\|^2 + \frac{1}{2} \|K \sqrt{\varepsilon(t_1)}(x(t_1) - x^*) + \dot{x}(t_1)\|^2. $$

**Proof Lyapunov analysis.** Set $f^* := f(x^*) = \min_{\mathcal{H}} f$. The energy function $\mathcal{E}: [t_0, +\infty[ \to \mathbb{R}_+$,

$$ \mathcal{E}(t) := (f(x(t)) - f^*) + \frac{\varepsilon(t)}{2} \|x(t)\|^2 + \frac{1}{2} \|c(t)(x(t) - x^*) + \dot{x}(t)\|^2, \quad (12) $$

will be the basis for our Lyapunov analysis. The function $c : [t_0, +\infty[ \to \mathbb{R}$ will be defined later, appropriately. Let us differentiate $\mathcal{E}(\cdot)$. By using the derivation chain rule, we get

$$ \begin{aligned} \dot{\mathcal{E}}(t) ={}& \langle \nabla f(x(t)), \dot{x}(t) \rangle + \frac{\dot{\varepsilon}(t)}{2} \|x(t)\|^2 + \varepsilon(t) \langle \dot{x}(t), x(t) \rangle \\ & + \langle c'(t)(x(t) - x^*) + c(t)\dot{x}(t) + \ddot{x}(t), c(t)(x(t) - x^*) + \dot{x}(t) \rangle. \end{aligned} \quad (13) $$

According to the constitutive Eq. (8), we have

$$ \ddot{x}(t) = -\varepsilon(t)x(t) - \delta\sqrt{\varepsilon(t)}\dot{x}(t) - \nabla f(x(t)). \quad (14) $$

Therefore,

$$ \langle c'(t)(x(t) - x^*) + c(t)\dot{x}(t) + \ddot{x}(t), c(t)(x(t) - x^*) + \dot{x}(t) \rangle $$

<figure><img src="image_6.png" /></figure>
$$
\begin{align}
&= \langle c'(t)(x(t) - x^*) + (c(t) - \delta\sqrt{\varepsilon(t)})\dot{x}(t) - (\varepsilon(t)x(t) \nonumber \\
&\quad + \nabla f(x(t))), c(t)(x(t) - x^*) + \dot{x}(t) \rangle \nonumber \\
&= c'(t)c(t)\|x(t) - x^*\|^2 + (c'(t) + c^2(t) - \delta c(t)\sqrt{\varepsilon(t)})\langle \dot{x}(t), x(t) - x^* \rangle \nonumber \\
&\quad + (c(t) - \delta\sqrt{\varepsilon(t)})\|\dot{x}(t)\|^2 \nonumber \\
&\quad - \varepsilon(t)\langle x(t), \dot{x}(t) \rangle - \langle \nabla f(x(t)), \dot{x}(t) \rangle - c(t)\langle \varepsilon(t)x(t) + \nabla f(x(t)), x(t) - x^* \rangle. \tag{15}
\end{align}
$$

By combining (13) with (15), we get

$$
\begin{equation}
\begin{aligned}
\dot{\varepsilon}(t) ={}& \frac{\dot{\varepsilon}(t)}{2} \|x(t)\|^2 + c'(t)c(t) \|x(t) - x^*\|^2 \\
& + (c'(t) + c^2(t) - \delta c(t)\sqrt{\varepsilon(t)}) \langle \dot{x}(t), x(t) - x^* \rangle \\
& + (c(t) - \delta\sqrt{\varepsilon(t)}) \|\dot{x}(t)\|^2 - c(t) \langle \varepsilon(t)x(t) + \nabla f(x(t)), x(t) - x^* \rangle.
\end{aligned}
\tag{16}
\end{equation}
$$

Consider the function

$$
f_t : \mathcal{H} \to \mathbb{R}, \quad f_t(x) = f(x) + \frac{\varepsilon(t)}{2} \|x\|^2.
$$

According to the strong convexity property of $f_t$, we have

$$
f_t(y) - f_t(x) \geq \langle \nabla f_t(x), y - x \rangle + \frac{\epsilon(t)}{2} \|x - y\|^2, \text{ for all } x, y \in \mathcal{H}.
$$

Take $y = x^*$ and $x = x(t)$ in the above inequality. We get

$$
f(x^*) + \frac{\varepsilon(t)}{2} \|x^*\|^2 - f(x(t)) - \frac{\varepsilon(t)}{2} \|x(t)\|^2 \geq \\
- \langle \nabla f(x(t)) + \varepsilon(t)(x(t), x(t) - x^*) + \frac{\varepsilon(t)}{2} \|x(t) - x^*\|^2.
$$

Consequently,

$$
\begin{equation}
\begin{aligned}
& -\langle \nabla f(x(t)) + \varepsilon(t)x(t), x(t) - x^* \rangle \le -(f(x(t)) - f(x^*)) \\
& + \frac{\varepsilon(t)}{2} \|x^*\|^2 - \frac{\varepsilon(t)}{2} \|x(t)\|^2 - \frac{\varepsilon(t)}{2} \|x(t) - x^*\|^2.
\end{aligned}
\tag{17}
\end{equation}
$$

By multiplying (17) with $c(t)$ and injecting in (16) we get

$$
\begin{align}
\dot{\varepsilon}(t) \le {}& -c(t)(f(x(t)) - f^*) + \left(\frac{\dot{\varepsilon}(t)}{2} - c(t)\frac{\varepsilon(t)}{2}\right) \|x(t)\|^2 \nonumber \\
& + \left(c'(t)c(t) - c(t)\frac{\varepsilon(t)}{2}\right) \|x(t) - x^*\|^2 + (c(t) - \delta\sqrt{\varepsilon(t)}) \|\dot{x}(t)\|^2 \nonumber \\
& + (c'(t) + c^2(t) - \delta c(t)\sqrt{\varepsilon(t)}) \langle \dot{x}(t), x(t) - x^* \rangle + c(t) \frac{\varepsilon(t)}{2} \|x^*\|^2. \tag{18}
\end{align}
$$
On the other hand, for a positive function $\mu(t)$ we have

$$ \begin{aligned} \mu(t)\mathcal{E}(t) = \mu(t)(f(x(t)) - f^*) + \mu(t)\frac{\varepsilon(t)}{2}\|x(t)\|^2 \\ + \frac{1}{2}\mu(t)c^2(t)\|x(t) - x^*\|^2 + \frac{1}{2}\mu(t)\|\dot{x}(t)\|^2 \\ + \mu(t)c(t)\langle\dot{x}(t), x(t) - x^*\rangle. \end{aligned} \quad (19) $$

By adding (18) and (19) we get

$$ \begin{aligned} \dot{\mathcal{E}}(t) + \mu(t)\mathcal{E}(t) \le (\mu(t) - c(t))(f(x(t)) - f^*) + \left(\frac{\dot{\varepsilon}(t)}{2} - c(t)\frac{\varepsilon(t)}{2} + \mu(t)\frac{\varepsilon(t)}{2}\right)\|x(t)\|^2 \\ + \left(c'(t)c(t) - c(t)\frac{\varepsilon(t)}{2} + \frac{1}{2}\mu(t)c^2(t)\right)\|x(t) - x^*\|^2 \\ + \left(c(t) - \delta\sqrt{\varepsilon(t)} + \frac{1}{2}\mu(t)\right)\|\dot{x}(t)\|^2 \\ + \left(c'(t) + c^2(t) - \delta c(t)\sqrt{\varepsilon(t)} + \mu(t)c(t)\right)\langle\dot{x}(t), x(t) - x^*\rangle \\ + c(t)\frac{\varepsilon(t)}{2}\|x^*\|^2. \end{aligned} \quad (20) $$

Since we have no control on the sign of $\langle\dot{x}(t), x(t) - x^*\rangle$, we take the coefficient in front of this term equal to zero, that is

$$ c'(t) + c^2(t) - \delta c(t)\sqrt{\varepsilon(t)} + \mu(t)c(t) = 0. \quad (21) $$

Take $c(t) = K\sqrt{\varepsilon(t)}$. Indeed, it is here that the choice of $c$, and of the corresponding parameter $K$, come into play. The relation (21) can be equivalently written

$$ \mu(t) = -\frac{\dot{\varepsilon}(t)}{2\varepsilon(t)} + (\delta - K)\sqrt{\varepsilon(t)}. $$

According to this choice for $\mu(t)$ and $c(t)$, the inequality (20) becomes

$$ \begin{aligned} \dot{\mathcal{E}}(t) + \mu(t)\mathcal{E}(t) \le \frac{1}{2\varepsilon(t)}\left(-\dot{\varepsilon}(t) + 2(\delta - 2K)\varepsilon(t)^{\frac{3}{2}}\right)(f(x(t)) - f^*) \\ + \frac{1}{4}\left(\dot{\varepsilon}(t) + 2(\delta - 2K)\varepsilon(t)^{\frac{3}{2}}\right)\|x(t)\|^2 \\ + \frac{K}{4}\left(K\dot{\varepsilon}(t) + 2\varepsilon(t)^{\frac{3}{2}}(-K^2 + \delta K - 1)\right)\|x(t) - x^*\|^2 \\ + \frac{1}{4\varepsilon(t)}\left(-\dot{\varepsilon}(t) + 2(K - \delta)\varepsilon(t)^{\frac{3}{2}}\right)\|\dot{x}(t)\|^2 + \frac{K\|x^*\|^2}{2}\varepsilon^{\frac{3}{2}}(t). \end{aligned} \quad (22) $$

Let us show that the condition $(\text{CD})_K$ provide the nonpositive sign for the coefficients in front of the terms of the right side of (22). Recall that, according to the hypotheses
(CD)$_K$, for all $t \ge t_1$ we have the properties a) and b):

$$
\begin{align*}
a) \left( \frac{1}{\sqrt{\varepsilon(t)}} \right)' &\le M_1(K) = \min(2K - \delta, \delta - K) = \begin{cases} 2K - \delta & \text{if } K \le \frac{2}{3}\delta \\ \delta - K, & \text{if } \frac{2}{3}\delta \le K, \end{cases} \\
b) \left( \frac{1}{\sqrt{\varepsilon(t)}} \right)' &\ge 0.
\end{align*}
$$

Without ambiguity we write briefly $M_1$ for $M_1(K)$. Note that $b)$ just expresses that $\varepsilon(\cdot)$ is non increasing. According to the hypotheses $(CD)_K$, we claim that for all $t \ge t_1$

$$
\left\{
\begin{array}{l}
i) \left(\frac{1}{\sqrt{\varepsilon(t)}}\right)' \le 2K - \delta \\
ii) \left(\frac{1}{\sqrt{\varepsilon(t)}}\right)' \ge \frac{\delta K - K^2 - 1}{K} \\
iii) \left(\frac{1}{\sqrt{\varepsilon(t)}}\right)' \le \delta - K.
\end{array}
\right.
\qquad (23)
$$

Let us justify these inequalities (23).

i) is a consequence of $\left(\frac{1}{\sqrt{\varepsilon(t)}}\right)' \le M_1$ and $M_1 \le 2K - \delta$.

ii) is a consequence of $\left(\frac{1}{\sqrt{\varepsilon(t)}}\right)' \ge 0$ and $\delta K - K^2 - 1 \le 0$. Precisely, when $\delta \le 2$ we have $\delta K - K^2 - 1 \le 2K - K^2 - 1 \le 0$. When $\delta > 2$, we have $\delta K - K^2 - 1 \le 0$ because $K \ge \frac{\delta+\sqrt{\delta^2-4}}{2}$.

iii) is a consequence of $\left(\frac{1}{\sqrt{\varepsilon(t)}}\right)' \le M_1$ and $M_1 \le \delta - K$.

The inequalities (23) can be equivalently written as follows: for all $t \ge t_1$

$$
\left\{
\begin{array}{l}
i) -\dot{\epsilon}(t) + 2(\delta - 2K)\epsilon(t)^{\frac{3}{2}} \le 0 \\
ii) K\dot{\epsilon}(t) + 2(\delta K - K^2 - 1)\epsilon(t)^{\frac{3}{2}} \le 0 \\
iii) -\dot{\epsilon}(t) + 2(K - \delta)\epsilon(t)^{\frac{3}{2}} \le 0.
\end{array}
\right.
\qquad (24)
$$

The inequalities (24) give that the coefficients entering the right side of (22) are nonpositive:

• i) gives that the coefficient of $f(x(t)) - f^*$ is nonpositive.

* Since $\dot{\epsilon}(t) \le 0$ we have $\dot{\epsilon}(t) + 2(\delta - 2K)\epsilon(t)^{\frac{3}{2}} \le -\dot{\epsilon}(t) + 2(\delta - 2K)\epsilon(t)^{\frac{3}{2}}$.
Therefore, by i) we have that the coefficient of $\|x(t)\|^2$ in (22) is nonpositive.

• ii) gives that the coefficient of $\|x(t) - x^*\|^2$ is nonpositive.

• iii) gives that the coefficient of $\|\dot{x}(t)\|^2$ is nonpositive.

Let us return to (22). Using (24) and the above results, we obtain

$$
\dot{\mathcal{E}}(t) + \mu(t) \mathcal{E}(t) \le \frac{K \|x^*\|^2}{2} \varepsilon^{\frac{3}{2}}(t), \quad \text{for all } t \ge t_1. \qquad (25)
$$
By multiplying (25) with $\mathfrak{M}(t) = \exp\left(\int_{t_1}^t \mu(s)ds\right)$ we obtain

$$ \frac{d}{dt} (\mathfrak{M}(t)\mathcal{E}(t)) \leq \frac{K \|x^*\|^2}{2} \varepsilon^{\frac{3}{2}}(t)\mathfrak{M}(t). \quad (26) $$

By integrating (26) on $[t_1, t]$ we get

$$ \mathcal{E}(t) \leq \frac{K \|x^*\|^2 \int_{t_1}^{t} \varepsilon^{\frac{3}{2}}(s) \mathfrak{M}(s) ds}{2 \mathfrak{M}(t)} + \frac{\mathfrak{M}(t_1) \mathcal{E}(t_1)}{\mathfrak{M}(t)}. \quad (27) $$

By definition of $\mathcal{E}(t)$ we deduce that

$$ f(x(t)) - \min_{\mathcal{H}} f \leq \frac{K \|x^*\|^2 \int_{t_1}^{t} \varepsilon^{\frac{3}{2}}(s) \mathfrak{M}(s) ds}{2 \mathfrak{M}(t)} + \frac{\mathcal{E}(t_1)}{\mathfrak{M}(t)}, \quad (28) $$

for all $t \geq t_1$, and this gives the convergence rate of the values. $\square$

**Remark 7** By integrating the relation $0 \leq \left(\frac{1}{\sqrt{\varepsilon(t)}}\right)' \leq M_1$ on an interval $[t_1, t]$, we get

$$ \frac{1}{\sqrt{\varepsilon(t_1)}} \leq \frac{1}{\sqrt{\varepsilon(t)}} \leq M_1 t + \frac{1}{\sqrt{\varepsilon(t_1)}} - M_1 t_1. $$

Therefore, denoting $C_1 = \frac{1}{\sqrt{\varepsilon(t_1)}} - M_1 t_1$, and $C_2 = \varepsilon(t_1)$ we have

$$ \frac{1}{(M_1 t + C_1)^2} \leq \varepsilon(t) \leq C_2. \quad (29) $$

This shows that the Lyapunov analysis developed previously only provides information in the case where $\varepsilon(t)$ is greater than or equal to $C/t^2$. Since the damping coefficient $\gamma(t) = \delta\sqrt{\varepsilon(t)}$, this means that $\gamma(t)$ must be greater than or equal to $C/t$. This is in accordance with the theory of inertial gradient systems with time-dependent viscosity coefficient, which states that the asymptotic optimization property is valid provided that the integral on $[t_0, +\infty[$ of $\gamma(t)$ is infinite, see Attouch and Cabot (2017).

As a consequence of Theorem 6 we have the following result.

**Corollary 8** Under the hypothesis of Theorem 6 we have

$$ \lim_{t \to +\infty} \mathfrak{M}(t) = +\infty. \quad (30) $$

Suppose moreover that $\varepsilon^{\frac{3}{2}}(\cdot) \in L^1(t_0, +\infty)$. Then

$$ \lim_{t \to +\infty} f(x(t)) = \min_{\mathcal{H}} f. \quad (31) $$
**Proof** By definition of $\mu(t)$, since $\varepsilon(\cdot)$ is nonincreasing and $\delta \ge K$, we have that $\mu(t)$ is nonnegative for all $t \ge t_1$. Therefore, $t \mapsto \mathfrak{M}(t)$ is a nondecreasing function. Let us write equivalently $\mu(t) = \frac{d}{dt} \ln \frac{1}{\sqrt{\varepsilon(t)}} + (\delta - K)\sqrt{\varepsilon(t)}$, and integrate on $[t_1, t]$. We obtain

$$ \mathfrak{M}(t) = \exp\left(\int_{t_1}^{t} \mu(s)ds\right) = \frac{C}{\sqrt{\varepsilon(t)}} \exp\left(\int_{t_1}^{t} (\delta - K)\sqrt{\varepsilon(s)}ds\right). $$

Since $\delta - K \ge 0$, we deduce that $\mathfrak{M}(t) \ge \frac{C}{\sqrt{\varepsilon(t)}}$. Since $\lim_{t \to \infty} \varepsilon(t) = 0$, we get

$$ \lim_{t \to +\infty} \mathfrak{M}(t) = +\infty. $$

Moreover, if we suppose that $\varepsilon^{\frac{3}{2}}(\cdot) \in L^1(t_0, +\infty)$, then by Attouch et al. (2018, Lemma A.3) we obtain

$$ \lim_{t \to +\infty} \frac{\int_{t_1}^{t} \varepsilon^{\frac{3}{2}}(s) \mathfrak{M}(s) ds}{\mathfrak{M}(t)} = 0. $$

Combining these properties with the convergence rate (11) of Theorem 6, we obtain (31). □

## 2.3 Particular cases

Since $\varepsilon(t) \to 0$ as $t \to +\infty$, (TRIGS) falls within the setting of the inertial dynamics with an asymptotic vanishing damping coefficient $\gamma(t)$. Here, $\gamma(t) = \delta\sqrt{\varepsilon(t)}$. We know with (Cabot et al. 2009) that for such systems, the optimization property is satisfied asymptotically if $\int_{t_0}^{+\infty} \gamma(t)dt = +\infty$ (i.e. $\gamma(t)$ does no tend too rapidly towards zero). By taking $\varepsilon(t) = \frac{c}{tp}$, it is easy to verify that the condition (CD)$_K$ is satisfied if $p \le 2$, that is $\sqrt{\varepsilon(t)} = \frac{c}{tp}$, with $p \le 1$, which is in accordance with the above property. Let us particularize Theorem 6 to situations where the integrals can be computed (at least estimated).

### 2.3.1 $\varepsilon(t)$ of order $1/t^2$

Take

$$ \varepsilon(t) = \frac{1}{(Mt + C)^2}, \quad M < M_1(K), \quad C \le C_1. $$

Then, $(\frac{1}{\sqrt{\varepsilon(t)}})' \le M_1(K)$ for all $t \ge t_0$ and the condition (CD)$_K$ is satisfied. Moreover,

$$ \mu(t) = \frac{M + \delta - K}{Mt + C}, \quad \mathfrak{M}(t) = \left( \frac{Mt + C}{Mt_0 + C} \right)^{\frac{M+\delta-K}{M}}. $$
Therefore, (11) becomes

$$ \mathcal{E}(t) \le \frac{K \|x^*\|^2}{2} \int_{t_0}^{t} \frac{(Ms + C)^{-\frac{2M+\delta-K}{M}}}{(Mt + C)^{\frac{M+\delta-K}{M}}} ds + \frac{(Mt_0 + C)^{\frac{M+\delta-K}{M}} \mathcal{E}(t_0)}{(Mt + C)^{\frac{M+\delta-K}{M}}}. \quad (32) $$

Consequently, we have

$$ \mathcal{E}(t) \le \frac{K \|x^*\|^2}{2(-M + \delta - K)} \frac{1}{(Mt + C)^2} + \frac{-\frac{K \|x^*\|^2}{2(-M + \delta - K)} (Mt_0 + C)^{-\frac{M+\delta-K}{M}} + (Mt_0 + C)^{\frac{M+\delta-K}{M}} \mathcal{E}(t_0)}{(Mt + C)^{\frac{M+\delta-K}{M}}}. $$

By assumption we have $M < M_1 \le \delta - K$. Therefore $\frac{M+\delta-K}{M} > 2$ and $-M + \delta - K > 0$. It follows that when $Mt + C \ge 1$

$$ \mathcal{E}(t) \le \frac{C'}{(Mt + C)^2}, \quad \text{with} \quad C' = \frac{K \|x^*\|^2}{2(-M + \delta - K)} + (Mt_0 + C)^{\frac{M+\delta-K}{M}} \mathcal{E}(t_0). $$

Observe that $\delta\sqrt{\mathcal{E}(t)} = \frac{\delta}{M} \frac{1}{t + \frac{C}{M}} = \frac{\alpha}{t + \beta}$, where we set $\alpha = \frac{\delta}{M}$ and $\beta = \frac{C}{M}$. Since $M < M_1 \le \frac{1}{3}\delta$ we get $\alpha \in ]3, +\infty[$. Indeed, we can get any $\alpha > 3$. Note also that by translating the time scale the result in the general case $\beta \ge 0$ results from its obtaining for a particular case $\beta = 0$. According to the fact that we can take for $\delta$ any positive number, we obtain

**Theorem 9** Take $\alpha \in ]3, +\infty[$, $c > 0$. Let $x : [t_0, +\infty[ \to \mathcal{H}$ be a solution trajectory of

$$ \ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t) + \nabla f(x(t)) + \frac{c}{t^2}x(t) = 0. $$

Then, the following convergence rate of the values is satisfied: as $t \to +\infty$

$$ f(x(t)) - \min_{\mathcal{H}} f = O\left(\frac{1}{t^2}\right). $$

**Remark 10** It is a natural question to compare our dynamic ($c > 0$) with the (Su et al. 2016) ($c = 0$), which was introduced as a continuous version of the Nesterov accelerated gradient method. We obtain the optimal convergence rate of values with an additional Tikhonov regularization term, which is a remarkable property. In fact, in the next sections we will prove that the Tikhonov term induces strong convergence of the trajectory to the minimum norm solution.

<figure><img src="image_7.png" /></figure>
### 2.3.2 $\epsilon(t)$ of order $1/t^r$, $\frac{2}{3} < r < 2$

Take $\epsilon(t) = 1/t^r$, $r < 2$. Then

$$ 
\begin{aligned}
\mu(t) &= -\frac{1}{2}\frac{\dot{\epsilon}(t)}{\epsilon(t)} + (\delta - K)\sqrt{\epsilon(t)} \\
&= \frac{r}{2t} + \frac{\delta - K}{t^{\frac{r}{2}}}.
\end{aligned}
$$

Therefore

$$ \mathfrak{M}(t) = \exp \int_{t_0}^{t} \left( \frac{r}{2s} + \frac{\delta - K}{s^{\frac{r}{2}}} \right) ds = C t^{\frac{r}{2}} \exp \left( \frac{2(\delta - K)}{2 - r} t^{1 - \frac{r}{2}} \right). $$

Set

$$ m(t) := t^{\frac{r}{2}} \exp \left( \frac{2(\delta - K)}{2 - r} t^{1 - \frac{r}{2}} \right). $$

According to (28) we have that for some $C_1 > 0$

$$ f(x(t)) - \min_{\mathcal{H}} f \le \frac{C_1}{m(t)} \int_{t_0}^{t} \frac{m(s)}{s^{\frac{3r}{2}}} ds + \frac{C_1}{m(t)}. \quad (33) $$

Note that according to $r < 2$, $m(t)$ is an increasing function which has an exponential growth as $t \to +\infty$. Accordingly, by the mean value theorem we have the following majorization.

$$ \frac{1}{m(t)} \int_{t_0}^{t} \frac{m(s)}{s^{\frac{3r}{2}}} ds \le \frac{m(t)}{m(t)} \int_{t_0}^{t} \frac{1}{s^{\frac{3r}{2}}} ds = \mathcal{O}\left(\frac{1}{t^{\frac{3r}{2}-1}}\right). \quad (34) $$

Let us summarize these results in the following statement.

**Theorem 11** Take $\epsilon(t) = 1/t^r$, $\frac{2}{3} < r < 2$, $\delta > 0$. Let $x : [t_0, +\infty[ \to \mathcal{H}$ be a global trajectory of

$$ \ddot{x}(t) + \frac{\delta}{t^{\frac{r}{2}}} \dot{x}(t) + \nabla f(x(t)) + \frac{1}{t^r} x(t) = 0. $$

Then, the following convergence rate of the values is satisfied: as $t \to +\infty$

$$ f(x(t)) - \min_{\mathcal{H}} f = \mathcal{O}\left(\frac{1}{t^{\frac{3r}{2}-1}}\right). $$

**Remark 12** When $r \to 2$ the exponent $\frac{3r}{2} - 1$ tends to 2. So there is a continuous transition in the convergence rate. As in Remark 10 the additional Tikhonov regularization term is expected to have a regularization effect (even better than in the case
$r = 2$). In addition, the above analysis makes appear another critical value, namely $r = \frac{2}{3}$.

### 3 In-depth analysis in the critical case $\varepsilon(t) = c/t^2$

Let us refine our analysis in the case where the Tikhonov regularization coefficient and the damping coefficient are respectively of order $1/t^2$ and $1/t$. Our analysis will now take into account the coefficients $\alpha$ and $c$ in front of these terms. So the Cauchy problem for (TRIGS) is written

$$ \begin{cases} \ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t) + \nabla f(x(t)) + \frac{c}{t^2}x(t) = 0 \\ x(t_0) = x_0, \quad \dot{x}(t_0) = v_0, \end{cases} \qquad (35) $$

where $t_0 > 0$, $c > 0$, $(x_0, v_0) \in \mathcal{H} \times \mathcal{H}$, and $\alpha \ge 3$. The starting time $t_0$ is taken strictly greater than zero to take into account the fact that the functions $\frac{c}{t^2}$ and $\frac{\alpha}{t}$ have singularities at 0. This is not a limitation of the generality of the proposed approach, since we will focus on the asymptotic behaviour of the generated trajectories.

#### 3.1 Convergence rate of the values

**Theorem 13** Let $t_0 > 0$ and, for some initial data $x_0, v_0 \in \mathcal{H}$, let $x : [t_0, +\infty[ \to \mathcal{H}$ be the unique global solution of (35). Then, the following results hold.

i) If $\alpha = 3$, then $f(x(t)) - \min_{\mathcal{H}} f = O\left(\frac{\ln t}{t^2}\right)$ as $t \to +\infty$.

ii) If $\alpha > 3$, then $f(x(t)) - \min_{\mathcal{H}} f = O\left(\frac{1}{t^2}\right)$ as $t \to +\infty$. Further, the trajectory $x$ is bounded and $\|\dot{x}(t)\| = O\left(\frac{1}{t}\right)$ as $t \to +\infty$.

**Proof** The analysis is parallel to that of Theorem 6. Set $f^* := f(x^*) = \min_{\mathcal{H}} f$. Let $b : [t_0, +\infty[ \to \mathbb{R}$, $b(t) = \frac{K}{t}$ where $K > 0$ will be defined later. Let us introduce $\mathcal{E} : [t_0, +\infty[ \to \mathbb{R}$,

$$ \mathcal{E}(t) := (f(x(t)) - f^*) + \frac{c}{2t^2} \|x(t)\|^2 + \frac{1}{2} \|b(t)(x(t) - x^*) + \dot{x}(t)\|^2, \quad (36) $$

that will serve as a Lypaunov function. Then,

$$ \begin{aligned} \dot{\mathcal{E}}(t) = & \langle \nabla f(x(t)), \dot{x}(t) \rangle - \frac{c}{t^3} \|x(t)\|^2 + \frac{c}{t^2} \langle \dot{x}(t), x(t) \rangle \\ & + \langle b'(t)(x(t) - x^*) + b(t)\dot{x}(t) + \ddot{x}(t), b(t)(x(t) - x^*) + \dot{x}(t) \rangle. \end{aligned} \quad (37) $$

According to the dynamic system (35), we have

$$ \ddot{x}(t) = -\frac{c}{t^2}x(t) - \frac{\alpha}{t}\dot{x}(t) - \nabla f(x(t)). \quad (38) $$

<figure><img src="image_8.png" /></figure>
Therefore,

$$
\begin{aligned}
\langle b'(t)(x(t) - x^*) + b(t)\dot{x}(t) + \ddot{x}(t), & b(t)(x(t) - x^*) + \dot{x}(t) \rangle = \\
& \left\langle -\frac{K}{t^3}(x(t) - x^*) + \frac{K-\alpha}{t}\dot{x}(t) - \left(\frac{c}{t^2}x(t) + \nabla f(x(t))\right), \frac{K}{t}(x(t) - x^*) + \dot{x}(t) \right\rangle = \\
& -\frac{K^2}{t^3}\|x(t) - x^*\|^2 + \frac{K^2 - \alpha K - K}{t^2}\langle \dot{x}(t), x(t) - x^* \rangle + \frac{K-\alpha}{t}\|\dot{x}(t)\|^2 \\
& -\frac{c}{t^2}\langle x(t), \dot{x}(t) \rangle - \langle \nabla f(x(t)), \dot{x}(t) \rangle - \frac{K}{t}\left\langle \frac{c}{t^2}x(t) + \nabla f(x(t)), x(t) - x^* \right\rangle.
\end{aligned}
\quad (39)
$$

Combining (37) and (39), we get

$$
\begin{aligned}
\dot{\mathcal{E}}(t) = & -\frac{c}{t^3} \|x(t)\|^2 - \frac{K^2}{t^3} \|x(t) - x^*\|^2 \\
& + \frac{K^2 - \alpha K - K}{t^2} \langle \dot{x}(t), x(t) - x^* \rangle + \frac{K - \alpha}{t} \|\dot{x}(t)\|^2 \\
& - \frac{K}{t} \left\langle \frac{c}{t^2}x(t) + \nabla f(x(t)), x(t) - x^* \right\rangle.
\end{aligned}
\quad (40)
$$

Consider the strongly convex function

$$
f_t : \mathcal{H} \to \mathbb{R}, \quad f_t(x) = f(x) + \frac{c}{2t^2} \|x\|^2.
$$

From the gradient inequality we have

$$
f_t(y) - f_t(x) \geq \langle \nabla f_t(x), y - x \rangle + \frac{c}{2t^2} \|x - y\|^2, \text{ for all } x, y \in \mathcal{H}.
$$

Take $y = x^*$ and $x = x(t)$ in the above inequality. We obtain

$$
f^* + \frac{c}{2t^2} \|x^*\|^2 - f(x(t)) - \frac{c}{2t^2} \|x(t)\|^2 \geq \\
- \left\langle \nabla f(x(t)) + \frac{c}{t^2} x(t), x(t) - x^* \right\rangle + \frac{c}{2t^2} \|x(t) - x^*\|^2.
$$

Consequently,

$$
\begin{aligned}
& -\left\langle \frac{c}{t^2}x(t) + \nabla f(x(t)), x(t) - x^* \right\rangle \\
\le & -(f(x(t)) - f^*) - \frac{c}{2t^2}\|x(t)\|^2 - \frac{c}{2t^2}\|x(t) - x^*\|^2 \\
& + \frac{c}{2t^2}\|x^*\|^2.
\end{aligned}
\qquad (41)
$$

By multiplying (41) with $\frac{K}{t}$, and injecting in (40), we obtain

$$
\dot{\mathcal{E}}(t) \leq -\frac{K}{t}(f(x(t)) - f^*) - \left(\frac{c}{t^3} + \frac{Kc}{2t^3}\right) \|x(t)\|^2 - \left(\frac{K^2}{t^3} + \frac{Kc}{2t^3}\right) \|x(t) - x^*\|^2
$$
$$ + \frac{K^2 - \alpha K - K}{t^2} \langle \dot{x}(t), x(t) - x^* \rangle + \frac{K - \alpha}{t} \|\dot{x}(t)\|^2 + \frac{cK}{2t^3} \|x^*\|^2. \quad (42) $$

On the other hand, by multiplying the function $\mathcal{E}(t)$ by $\mu(t) = \frac{\alpha - K + 1}{t}$, we obtain

$$ \begin{aligned} \mu(t)\mathcal{E}(t) = & \frac{\alpha - K + 1}{t} (f(x(t)) - f^*) + \frac{(\alpha - K + 1)c}{2t^3} \|x(t)\|^2 \\ & + \frac{(\alpha - K + 1)K^2}{2t^3} \|x(t) - x^*\|^2 \\ & + \frac{\alpha - K + 1}{2t} \|\dot{x}(t)\|^2 + \frac{(\alpha - K + 1)K}{t^2} \langle \dot{x}(t), x(t) - x^* \rangle. \end{aligned} \quad (43) $$

By adding (42) and (43), we get

$$ \begin{aligned} \dot{\mathcal{E}}(t) + \mu(t)\mathcal{E}(t) \le & \frac{\alpha - 2K + 1}{t} (f(x(t)) - f^*) + \frac{(\alpha - 2K - 1)c}{2t^3} \|x(t)\|^2 \\ & + \frac{(\alpha - K - 1)K^2 - Kc}{2t^3} \|x(t) - x^*\|^2 \\ & + \frac{K - \alpha + 1}{2t} \|\dot{x}(t)\|^2 + \frac{cK}{2t^3} \|x^*\|^2. \end{aligned} \quad (44) $$

**The case** $\alpha > 3$. Take $\frac{\alpha+1}{2} < K < \alpha - 1$. Since $\alpha > 3$, such $K$ exists. This implies that $\alpha - 2K + 1 < 0$, hence $\alpha - 2K - 1 < 0$, and $K - \alpha + 1 < 0$. In addition, since $c > 0$ there exists $K \in [\frac{\alpha+1}{2}, \alpha - 1]$ such that

$$ (\alpha - K - 1)K^2 - Kc \le 0. \quad (45) $$

Indeed, (45) can be deduced from the fact that the continuous function $\varphi(K) = (\alpha - K - 1)K$ is decreasing on the interval $[\frac{\alpha+1}{2}, \alpha - 1]$ and $\varphi(\alpha - 1) = 0$. Therefore, for every $c > 0$ there exists $K \in [\frac{\alpha+1}{2}, \alpha - 1]$ such that $c \ge \varphi(K)$. So take $K \in [\frac{\alpha+1}{2}, \alpha - 1]$ such that (45) holds. Then, by collecting the previous results, (44) yields

$$ \dot{\mathcal{E}}(t) + \mu(t)\mathcal{E}(t) \le \frac{cK}{2t^3} \|x^*\|^2. \quad (46) $$

Taking into account that $\mu(t) = \frac{\alpha - K + 1}{t}$, by multiplying (46) with $t^{\alpha - K + 1}$ we get

$$ \frac{d}{dt} (t^{\alpha - K + 1} \mathcal{E}(t)) \le \frac{cK}{2} \|x^*\|^2 t^{\alpha - K - 2}. \quad (47) $$

By integrating (47) on $[t_0, t]$, we get

$$ \mathcal{E}(t) \le \frac{cK \|x^*\|^2}{2(\alpha - K - 1) t^2} - \frac{cK \|x^*\|^2}{2(\alpha - K - 1)} \frac{t_0^{\alpha - K - 1}}{t^{\alpha - K + 1}} + \frac{t_0^{\alpha - K + 1} \mathcal{E}(t_0)}{t^{\alpha - K + 1}}. \quad (48) $$
Since $\alpha - K + 1 > 2$, we obtain

$$ \mathcal{E}(t) = O\left(\frac{1}{t^2}\right) \text{ as } t \to +\infty. \qquad (49) $$

By definition of $\mathcal{E}(t)$ we immediately deduce that

$$ f(x(t)) - \min_{\mathcal{H}} f = O\left(\frac{1}{t^2}\right) \text{ as } t \to +\infty, \qquad (50) $$

and further, that the trajectory $x(\cdot)$ is bounded and

$$ \|\dot{x}(t)\| = O\left(\frac{1}{t}\right) \text{ as } t \to +\infty. $$

**The case** $\alpha = 3$. Take $K = 2$. With the previous notations, we have now $\mu(t) = \frac{2}{t}$ and (44) gives

$$ \dot{\mathcal{E}}(t) + \frac{2}{t}\mathcal{E}(t) \le -\frac{c}{t^3}\|x(t)\|^2 - \frac{c}{t^3}\|x(t) - x^*\|^2 + \frac{c}{t^3}\|x^*\|^2 \le \frac{c}{t^3}\|x^*\|^2. \qquad (51) $$

After multiplication of (51) by $t^2$ we get

$$ \frac{d}{dt}(t^2 \mathcal{E}(t)) \le \frac{c}{t} \|x^*\|^2. \qquad (52) $$

By integrating (52) on $[t_0, t]$ we get

$$ \mathcal{E}(t) \le c \|x^*\|^2 \frac{\ln t}{t^2} - c \|x^*\|^2 \frac{\ln t_0}{t^2} + \frac{t_0^2 \mathcal{E}(t_0)}{t^2}. \qquad (53) $$

Consequently, we have

$$ \mathcal{E}(t) = O\left(\frac{\ln t}{t^2}\right) \text{ as } t \to +\infty. \qquad (54) $$

By definition of $\mathcal{E}(t)$ we immediately deduce that

$$ f(x(t)) - \min f = O\left(\frac{\ln t}{t^2}\right) \text{ as } t \to +\infty. \qquad (55) $$

which gives the claim. □
## 3.2 Strong convergence

**Theorem 14** Let $t_0 > 0$ and, for some starting points $x_0$, $v_0 \in \mathcal{H}$, let $x : [t_0, +\infty[ \to \mathcal{H}$ be the unique global solution of (35). Let $x^*$ be the element of minimal norm of $S = \text{argmin } f$, that is $x^* = \text{proj}_S 0$. Then, for all $\alpha > 3$ we have that

$$ \liminf_{t \to +\infty} \|x(t) - x^*\| = 0. $$

Further, if there exists $T \ge t_0$, such that the trajectory $\{x(t) : t \ge T\}$ stays either in the open ball $B(0, \|x^*\|)$ or in its complement, then $x(t)$ converges strongly to $x^*$ as $t \to +\infty$.

**Proof** The proof combines energetic and geometric arguments, as it was initiated in Attouch and Czarnecki (2002). We successively consider the three following configurations of the trajectory.

I. Assume that there exists $T \ge t_0$ such that $\|x(t)\| \ge \|x^*\|$ for all $t \ge T$. Let us denote $f_t(x) := f(x) + \frac{c}{2t^2}\|x\|^2$ and let $x_t := \text{argmin } f_t(x)$. Let us recall some classical properties of the Tikhonov approximation:

$$ \forall t > 0 \quad \|x_t\| \le \|x^*\|, \quad \text{and} \quad \lim_{t \to +\infty} \|x_t - x^*\| = 0. \qquad (56) $$

Using the gradient inequality for the strongly convex function $f_t$, we have

$$ f_t(x(t)) - f_t(x_t) \ge \frac{c}{2t^2} \|x(t) - x_t\|^2. $$

On the other hand

$$ f_t(x_t) - f_t(x^*) = f(x_t) - f^* + \frac{c}{2t^2}(\|x_t\|^2 - \|x^*\|^2) \ge \frac{c}{2t^2}(\|x_t\|^2 - \|x^*\|^2). $$

By adding the last two inequalities we get

$$ f_t(x(t)) - f_t(x^*) \ge \frac{c}{2t^2}(\|x(t) - x_t\|^2 + \|x_t\|^2 - \|x^*\|^2), \qquad (57) $$

Therefore, according to (56), to obtain the strong convergence of the trajectory $x(t)$ to $x^*$, it is enough to show that $f_t(x(t)) - f_t(x^*) = o(\frac{1}{t^2})$, as $t \to +\infty$. For $K > 0$, consider now the energy functional

$$ \begin{aligned} E(t) &= f_t(x(t)) - f_t(x^*) + \frac{1}{2} \left\| \frac{K}{t} (x(t) - x^*) + \dot{x}(t) \right\|^2 \\ &= (f(x(t)) - f(x^*)) + \frac{c}{2t^2} (\|x(t)\|^2 - \|x^*\|^2) + \frac{1}{2} \left\| \frac{K}{t} (x(t) - x^*) + \dot{x}(t) \right\|^2. \end{aligned} \qquad (58) $$

<figure><img src="image_9.png" /></figure>
Then,

$$ 
\begin{aligned}
\dot{E}(t) = \langle \nabla f_t(x(t)), \dot{x}(t) \rangle - \frac{c}{2t^3} (\|x(t)\|^2 - \|x^*\|^2) \\
+ \left\langle -\frac{K}{t^2}(x(t) - x^*) + \frac{K}{t}\dot{x}(t) + \ddot{x}(t), \frac{K}{t}(x(t) - x^*) + \dot{x}(t) \right\rangle.
\end{aligned}
\quad (59) $$

Let us examine the different terms of (59). According to the constitutive Eq. (35) we have

$$ 
\begin{aligned}
\left\langle -\frac{K}{t^2}(x(t) - x^*) + \frac{K}{t}\dot{x}(t) + \ddot{x}(t), \frac{K}{t}(x(t) - x^*) + \dot{x}(t) \right\rangle = \\
\left\langle -\frac{K}{t^2}(x(t) - x^*) + \frac{K - \alpha}{t}\dot{x}(t) - \left(\frac{c}{t^2}x(t) + \nabla f(x(t))\right), \frac{K}{t}(x(t) - x^*) + \dot{x}(t) \right\rangle = \\
-\frac{K^2}{t^3}\|x(t) - x^*\|^2 + \frac{K^2 - \alpha K - K}{t^2}\langle\dot{x}(t), x(t) - x^*\rangle + \frac{K - \alpha}{t}\|\dot{x}(t)\|^2 \\
-\frac{c}{t^2}\langle x(t), \dot{x}(t) \rangle - \langle \nabla f(x(t)), \dot{x}(t) \rangle - \frac{K}{t}\left\langle \frac{c}{t^2}x(t) + \nabla f(x(t)), x(t) - x^* \right\rangle.
\end{aligned}
\quad (60) $$

Further, from (41) we get

$$ 
\begin{aligned}
& -\frac{K}{t}\left\langle \frac{c}{t^2}x(t) + \nabla f(x(t)), x(t) - x^* \right\rangle \\
& \leq -\frac{K}{t}(f(x(t)) - f^*) - \frac{cK}{2t^3}\|x(t)\|^2 - \frac{cK}{2t^3}\|x(t) - x^*\|^2 + \frac{cK}{2t^3}\|x^*\|^2 \\
& = -\frac{K}{t}(f_t(x(t)) - f_t(x^*)) - \frac{cK}{2t^3}\|x(t) - x^*\|^2.
\end{aligned}
\quad (61) $$

Injecting (60) and (61) in (59) we get

$$ 
\begin{aligned}
\dot{E}(t) \leq & -\frac{K}{t}(f_t(x(t)) - f_t(x^*)) - \frac{c}{t^3}(\|x(t)\|^2 - \|x^*\|^2) - \frac{2K^2 + cK}{2t^3}\|x(t) - x^*\|^2 \\
& + \frac{K^2 - \alpha K - K}{t^2}\langle \dot{x}(t), x(t) - x^* \rangle + \frac{K - \alpha}{t}\|\dot{x}(t)\|^2.
\end{aligned}
\quad (62) $$

Consider now the function $\mu(t) = \frac{\alpha+1-K}{t}$. Then,

$$ 
\begin{aligned}
\mu(t)E(t) = & \frac{\alpha + 1 - K}{t}(f_t(x(t)) - f_t(x^*)) + \frac{K^2(\alpha + 1 - K)}{2t^3}\|x(t) - x^*\|^2 \\
& + \frac{K(\alpha + 1 - K)}{t^2}\langle \dot{x}(t), x(t) - x^* \rangle + \frac{\alpha + 1 - K}{2t}\|\dot{x}(t)\|^2.
\end{aligned}
\quad (63) $$

Consequently, (62) and (63) yield

$$ 
\begin{aligned}
\dot{E}(t) + \mu(t)E(t) \leq & \frac{\alpha + 1 - 2K}{t}(f_t(x(t)) - f_t(x^*)) - \frac{c}{t^3}(\|x(t)\|^2 - \|x^*\|^2) \\
& + \frac{K^2(\alpha - 1 - K) - cK}{2t^3}\|x(t) - x^*\|^2 + \frac{K - \alpha + 1}{2t}\|\dot{x}(t)\|^2
\end{aligned}
$$
$$ = \frac{\alpha + 1 - 2K}{t} (f(x(t)) - f(x^*)) + (\alpha - 1 - 2K) \frac{c}{2t^3} (\|x(t)\|^2 - \|x^*\|^2) \\ + \frac{K^2(\alpha - 1 - K) - cK}{2t^3} \|x(t) - x^*\|^2 + \frac{K - \alpha + 1}{2t} \|\dot{x}(t)\|^2. \quad (64) $$

Assume that $\frac{\alpha+1}{2} < K < \alpha - 1$. Since $\alpha > 3$ such $K$ exists. As in the proof of Theorem 13 we deduce that $\alpha - 2K + 1 < 0$, $K - \alpha + 1 < 0$ and since $c > 0$ there exists $K \in (\frac{\alpha+1}{2}, \alpha - 1)$ such that

$$ (\alpha - K - 1)K^2 - Kc \le 0. \quad (65) $$

So take $K \in (\frac{\alpha+1}{2}, \alpha - 1)$ such that (65) holds. Then, (64) leads to

$$ \dot{E}(t) + \frac{\alpha + 1 - K}{t} E(t) \le (\alpha - 1 - 2K) \frac{c}{2t^3} (\|x(t)\|^2 - \|x^*\|^2). \quad (66) $$

Let us integrate the differential inequality (66). After multiplication by $t^{\alpha+1-K}$ we get

$$ \frac{d}{dt} t^{\alpha+1-K} E(t) \le \frac{c}{2}(\alpha - 1 - 2K)t^{\alpha-2-K} (\|x(t)\|^2 - \|x^*\|^2) $$

and integrating the latter on $[T, t]$, $t > T$ we obtain

$$ E(t) \le \frac{c}{2}(\alpha - 1 - 2K) \frac{\int_{T}^{t} s^{\alpha-2-K} (\|x(s)\|^2 - \|x^*\|^2) ds}{t^{\alpha+1-K}} + \frac{T^{\alpha+1-K} E(T)}{t^{\alpha+1-K}}. \quad (67) $$

In one hand, from the definition of $E(t)$ we have

$$ f_t(x(t)) - f_t(x^*) \le E(t). $$

Therefore,

$$ \begin{aligned} f_t(x(t)) - f_t(x^*) \le \frac{c}{2}(\alpha - 1 - 2K) \\ \frac{\int_{T}^{t} s^{\alpha-2-K} (\|x(s)\|^2 - \|x^*\|^2) ds}{t^{\alpha+1-K}} + \frac{T^{\alpha+1-K} E(T)}{t^{\alpha+1-K}}. \end{aligned} $$

On the other hand (57) gives

$$ f_t(x(t)) - f_t(x^*) \ge \frac{c}{2t^2} (\|x(t) - x_t\|^2 + \|x_t\|^2 - \|x^*\|^2). $$

Consequently,

$$ (\alpha - 1 - 2K) \frac{\int_{T}^{t} s^{\alpha-2-K} (\|x(s)\|^2 - \|x^*\|^2) ds}{t^{\alpha-1-K}} + \frac{2T^{\alpha+1-K} E(T)}{cT^{\alpha-1-K}} $$

<figure><img src="image_10.png" /></figure>
$$ \geq \|x(t) - x_t\|^2 + \|x_t\|^2 - \|x^*\|^2. \qquad (68) $$

By assumption $\|x(t)\| \geq \|x^*\|$ for all $t \geq T$ and $\alpha - 1 - 2K < 0$. Hence, for all $t > T$, (68) leads to

$$ \frac{2T^{\alpha+1-K} E(T)}{ct^{\alpha-1-K}} \geq \|x(t) - x_t\|^2 + \|x_t\|^2 - \|x^*\|^2. \qquad (69) $$

Now, by taking the limit $t \to +\infty$ and using that $x_t \to x^*$, $t \to +\infty$ we get

$$ \lim_{t \to +\infty} \|x(t) - x_t\| \leq 0 $$

and hence

$$ \lim_{t \to +\infty} x(t) = x^*. $$

## II. Assume now that there exists $T \geq t_0$ such that $\|x(t)\| < \|x^*\|$ for all $t \geq T$. According to Theorem 13, we have that

$$ \lim_{t \to +\infty} f(x(t)) = \min_{\mathcal{H}} f. $$

Let $\bar{x} \in \mathcal{H}$ be a weak sequential cluster point of the trajectory $x$, which exists since, by Theorem 13, the trajectory is bounded. So, there exists a sequence $(t_n)_{n \in \mathbb{N}} \subseteq [T, +\infty)$ such that $t_n \to +\infty$ and $x(t_n)$ converges weakly to $\bar{x}$ as $n \to +\infty$. Since $f$ is weakly lower semicontinuous, we deduce that

$$ f(\bar{x}) \leq \liminf_{n \to +\infty} f(x(t_n)) = \min_{\mathcal{H}} f, $$

hence $\bar{x} \in \operatorname{argmin} f$. Now, since the norm is weakly lower semicontinuous, and since $\|x(t)\| < \|x^*\|$ for all $t \geq T$, we have

$$ \|\bar{x}\| \leq \liminf_{n \to +\infty} \|x(t_n)\| \leq \|x^*\|. $$

Combining $\bar{x} \in \operatorname{argmin} f$ with the definition of $x^*$, this implies that $\bar{x} = x^*$. This shows that the trajectory $x(\cdot)$ converges weakly to $x^*$. So

$$ \|x^*\| \leq \liminf_{t \to +\infty} \|x(t)\| \leq \limsup_{t \to +\infty} \|x(t)\| \leq \|x^*\|, $$

hence we have

$$ \lim_{t \to +\infty} \|x(t)\| = \|x^*\|. $$
Combining this property with $x(t) \to x^*$ as $t \to +\infty$, we obtain the strong convergence, that is

$$ \lim_{t \to +\infty} x(t) = x^*. $$

**III.** We suppose that for every $T \ge t_0$ there exists $t \ge T$ such that $\|x^*\| > \|x(t)\|$ and also there exists $s \ge T$ such that $\|x^*\| \le \|x(s)\|$. From the continuity of $x$, we deduce that there exists a sequence $(t_n)_{n \in \mathbb{N}} \subseteq [t_0, +\infty)$ such that $t_n \to +\infty$ as $n \to +\infty$ and, for all $n \in \mathbb{N}$ we have

$$ \|x(t_n)\| = \|x^*\|. $$

Consider $\bar{x} \in \mathcal{H}$ a weak sequential cluster point of $(x(t_n))_{n \in \mathbb{N}}$. We deduce as in case **II** that $\bar{x} = x^*$. Hence, $x^*$ is the only weak sequential cluster point of $x(t_n)$ and consequently the sequence $x(t_n)$ converges weakly to $x^*$. Obviously $\|x(t_n)\| \to \|x^*\|$ as $n \to +\infty$. So, it follows that $x(t_n) \to x^*$, $n \to +\infty$, that is $\|x(t_n) - x^*\| \to 0$ as $n \to +\infty$. This leads to $\liminf_{t \to +\infty} \|x(t) - x^*\| = 0$.

□

**4 Strong convergence-general case**

We are going to analyze via Lyapunov analysis the strong convergence properties as $t \to +\infty$ of the solution trajectories of the inertial dynamic (TRIGS) that we recall below

$$ \ddot{x}(t) + \delta\sqrt{\varepsilon(t)}\dot{x}(t) + \nabla f(x(t)) + \varepsilon(t)x(t) = 0. $$

**Theorem 15** Let consider the dynamic system (TRIGS) where we assume that $\varepsilon(\cdot)$ satisfies the condition (CD)$_K$ for some $K > 0$, $\int_{t_0}^{+\infty} \varepsilon^{\frac{3}{2}}(t)dt < +\infty$ and $\lim_{t \to +\infty} \frac{1}{\sqrt{\varepsilon(t)}\exp\left(\int_{t_0}^t (\delta - K)\sqrt{\varepsilon(s)}ds\right)} = 0$.

Then, for any global solution trajectory $x : [t_0, +\infty[ \to \mathcal{H}$ of (TRIGS),

$$ \liminf_{t \to +\infty} \|x(t) - x^*\| = 0, $$

where $x^*$ is the element of minimal norm of argmin $f$, that is $x^* = \text{proj}_{\text{argmin } f} 0$.

Further, if there exists $T \ge t_0$, such that the trajectory $\{x(t) : t \ge T\}$ stays either in the open ball $B(0, \|x^*\|)$ or in its complement, then $x(t)$ converges strongly to $x^*$ as $t \to +\infty$.

**Proof** The proof is parallel to that of Theorem 14. We analyze the behavior of the trajectory $x(\cdot)$ depending on its position with respect to the ball $B(0, \|x^*\|)$.

<figure><img src="image_11.png" /></figure>
I. Assume that $\|x(t)\| \ge \|x^*\|$ for all $t \ge T$. Let us denote $f_t(x) = f(x) + \frac{\varepsilon(t)}{2}\|x\|^2$, and consider the energy functional $E: [t_1, +\infty[ \to \mathbb{R}$ defined by

$$E(t) := f_t(x(t)) - f_t(x^*) + \frac{1}{2}\|c(t)(x(t) - x^*) + \dot{x}(t)\|^2,$$

where $c(t) = K\sqrt{\varepsilon(t)}$. Note that $E(t) = \mathcal{E}(t) - \frac{\varepsilon(t)}{2}\|x^*\|^2$, where $\mathcal{E}(t)$ was defined in the proof of Theorem 6. Hence, reasoning as in the proof of Theorem 6, see (25) (and keeping the term containing $\|x(t)\|^2$ in the right hand side of (22)), we get for all $t \ge t_1$ that

$$\dot{E}(t) + \mu(t)E(t) \le \left(\frac{\dot{\varepsilon}(t)}{2} - c(t)\frac{\varepsilon(t)}{2} + \mu(t)\frac{\varepsilon(t)}{2}\right) (\|x(t)\|^2 - \|x^*\|^2), \quad (70)$$

where $\mu(t) = -\frac{\dot{\varepsilon}(t)}{2\varepsilon(t)} + (\delta - K)\sqrt{\varepsilon(t)}$. An elementary computation gives $\frac{\dot{\varepsilon}(t)}{2} - c(t)\frac{\varepsilon(t)}{2} + \mu(t)\frac{\varepsilon(t)}{2} \le 0$, because of $\varepsilon(\cdot)$ decreasing and $K \ge \frac{\delta}{2}$. Since $\|x(t)\| \ge \|x^*\|$ for all $t \ge T$, (70) yields

$$\dot{E}(t) + \mu(t)E(t) \le 0, \text{ for all } t \ge T_1 = \max\{T, t_1\}. \quad (71)$$

Set

$$\mathfrak{M}(t) = \exp\left(\int_{T_1}^{t} \mu(s)ds\right) = \exp\left(\int_{T_1}^{t} -\frac{\dot{\varepsilon}(s)}{2\varepsilon(s)} + (\delta - K)\sqrt{\varepsilon(s)}ds\right).$$

Therefore, we have with $C = \sqrt{\varepsilon(T_1)}$

$$\mathfrak{M}(t) = C \frac{1}{\sqrt{\varepsilon(t)}} \exp \left( \int_{T_1}^{t} (\delta - K) \sqrt{\varepsilon(s)} ds \right).$$

Multiplying (71) with $\mathfrak{M}(t)$ and integrating on an interval $[T_1, t]$, we get for all $t \ge T_1$ that

$$\mathfrak{M}(t)E(t) \le \mathfrak{M}(T_1)E(T_1) = C'$$

Consequently, there exists $C'_1 > 0$ such that for all $t \ge T_1$ one has

$$E(t) \le \frac{C'_1 \sqrt{\varepsilon(t)}}{\exp\left(\int_{T_1}^{t} (\delta - K) \sqrt{\varepsilon(s)} ds\right)}.$$

Further, $f_t(x(t)) - f_t(x^*) \le E(t)$, for all $t \ge t_1$. Therefore,

$$f_t(x(t)) - f_t(x^*) \le \frac{C'_1 \sqrt{\varepsilon(t)}}{\exp\left(\int_{T_1}^{t} (\delta - K) \sqrt{\varepsilon(s)} ds\right)}, \text{ for all } t \ge T_1. \quad (72)$$
For fixed $t$ let us denote $x_{\varepsilon(t)} = \operatorname{argmin} f_t(x)$. Obviously $\|x_{\varepsilon(t)}\| \le \|x^*\|$.
Using the gradient inequality for the strongly convex function $f_t$ we have

$$f_t(x) - f_t(x_{\varepsilon(t)}) \ge \frac{\varepsilon(t)}{2} \|x - x_{\varepsilon(t)}\|^2 \text{ for all } x \in \mathcal{H} \text{ and } t \ge t_0.$$

On the other hand

$$\begin{aligned} f_t(x_{\varepsilon(t)}) - f_t(x^*) &= f(x_{\varepsilon(t)}) - f^* + \frac{\varepsilon(t)}{2} (\|x_{\varepsilon(t)}\|^2 - \|x^*\|^2) \\ &\ge \frac{\varepsilon(t)}{2} (\|x_{\varepsilon(t)}\|^2 - \|x^*\|^2). \end{aligned}$$

Now, by adding the last two inequalities we get

$$f_t(x) - f_t(x^*) \ge \frac{\varepsilon(t)}{2} (\|x - x_{\varepsilon(t)}\|^2 + \|x_{\varepsilon(t)}\|^2 - \|x^*\|^2) \text{ for all } x \in \mathcal{H} \text{ and } t \ge t_0. \quad (73)$$

Hence, (72) and (73) lead to

$$\begin{aligned} & \|x(t) - x_{\varepsilon(t)}\|^2 + \|x_{\varepsilon(t)}\|^2 - \|x^*\|^2 \\ & \le \frac{C_2'}{\sqrt{\varepsilon(t)} \exp\left(\int_{T_1}^t (\delta - K)\sqrt{\varepsilon(s)}ds\right)}, \quad \text{for all } t \ge T_1. \end{aligned} \quad (74)$$

Now, by taking the limit as $t \to +\infty$, and using that $x_{\varepsilon(t)} \to x^*$ as $t \to +\infty$ and the assumption in the hypotheses of the theorem we get $\lim_{t \to +\infty} \|x(t) - x_{\varepsilon(t)}\| \le 0$, and hence $\lim_{t \to +\infty} x(t) = x^*$.

II. Assume now, that $\|x(t)\| < \|x^*\|$ for all $t \ge T$. By Corollary 8 we get that $f(x(t)) \to \min f$ as $t \to +\infty$. Now, we take $\bar{x} \in \mathcal{H}$ a weak sequential cluster point of the trajectory $x$, which exists since the trajectory is bounded. This means that there exists a sequence $(t_n)_{n \in \mathbb{N}} \subseteq [T, +\infty)$ such that $t_n \to +\infty$ and $x(t_n)$ converges weakly to $\bar{x}$ as $n \to +\infty$. We know that $f$ is weakly lower semicontinuous, so one has

$$f(\bar{x}) \le \liminf_{n \to +\infty} f(x(t_n)) = \min f,$$

hence $\bar{x} \in \operatorname{argmin} f$. Now, since the norm is weakly lower semicontinuous one has that

$$\|\bar{x}\| \le \liminf_{n \to +\infty} \|x(t_n)\| \le \|x^*\|$$

which, from the definition of $x^*$, implies that $\bar{x} = x^*$. This shows that the trajectory $x(\cdot)$ converges weakly to $x^*$. So

$$\|x^*\| \le \liminf_{t \to +\infty} \|x(t)\| \le \limsup_{t \to +\infty} \|x(t)\| \le \|x^*\|,$$
hence we have

$$ \lim_{t \to +\infty} \|x(t)\| = \|x^*\|. $$

From the previous relation and the fact that $x(t) \to x^*$ as $t \to +\infty$, we obtain the strong convergence, that is

$$ \lim_{t \to +\infty} x(t) = x^*. $$

**III.** We suppose that for every $T \ge t_0$ there exists $t \ge T$ such that $\|x^*\| > \|x(t)\|$ and also there exists $s \ge T$ such that $\|x^*\| \le \|x(s)\|$. From the continuity of $x$, we deduce that there exists a sequence $(t_n)_{n \in \mathbb{N}} \subseteq [t_0, +\infty)$ such that $t_n \to +\infty$ as $n \to +\infty$ and, for all $n \in \mathbb{N}$ we have

$$ \|x(t_n)\| = \|x^*\|. $$

Consider $\bar{x} \in \mathcal{H}$ a weak sequential cluster point of $(x(t_n))_{n \in \mathbb{N}}$. We deduce as at case **II** that $\bar{x} = x^*$. Hence, $x^*$ is the only weak sequential cluster point of $x(t_n)$ and consequently the sequence $x(t_n)$ converges weakly to $x^*$.

Obviously $\|x(t_n)\| \to \|x^*\|$ as $n \to +\infty$. So, it follows that $x(t_n) \to x^*$, $n \to +\infty$, that is $\|x(t_n) - x^*\| \to 0$ as $n \to +\infty$. This leads to $\liminf_{t \to +\infty} \|x(t) - x^*\| = 0$.

□

**4.1 The case $\varepsilon(t)$ is of order $1/t^r$, $\frac{2}{3} < r < 2$**

Take $\varepsilon(t) = 1/t^r$, $\frac{2}{3} < r < 2$. Then, $\int_{t_0}^{+\infty} \varepsilon^{\frac{3}{2}}(t)dt = \int_{t_0}^{+\infty} \frac{1}{t^{\frac{3}{2}r}}dt < +\infty$, $(\frac{1}{\sqrt{\varepsilon(t)}})' = \frac{r}{2}t^{\frac{r}{2}-1}$ and

$$ \lim_{t \to +\infty} \frac{1}{\sqrt{\varepsilon(t)} \exp \left( \int_{t_0}^t (\delta - K) \sqrt{\varepsilon(s)} ds \right)} = \lim_{t \to +\infty} \frac{Ct^{\frac{r}{2}}}{\exp \left( \frac{2(\delta - K)}{2-r} t^{1-\frac{r}{2}} \right)} = 0. $$

Therefore, Theorem 15 can be applied. Let us summarize these results in the following statement.

**Theorem 16** Take $\varepsilon(t) = 1/t^r$, $\frac{2}{3} < r < 2$. Let $x : [t_0, +\infty[ \to \mathcal{H}$ be a global solution trajectory of

$$ \ddot{x}(t) + \frac{\delta}{t^{\frac{r}{2}}} \dot{x}(t) + \nabla f(x(t)) + \frac{1}{t^r} x(t) = 0. $$

Then, $\liminf_{t \to +\infty} \|x(t) - x^*\| = 0$.
Further, if there exists $T \ge t_0$, such that the trajectory $\{x(t) : t \ge T\}$ stays either in the open ball $B(0, \|x^*\|)$ or in its complement, then $x(t)$ converges strongly to $x^*$ as $t \to +\infty$.

## 5 Fast inertial algorithms with Tikhonov regularization

On the basis of the convergence properties of continuous dynamic (TRIGS), one would expect to obtain similar results for the algorithms resulting from its temporal discretization. To illustrate this, we will do a detailed study of the associated proximal algorithms, obtained by implicit discretization. A full study of the associated first-order algorithms would be beyond the scope of this article, and will be the subject of further study. So, for $k \ge 1$, consider the discrete dynamic

$$ (x_{k+1} - 2x_k + x_{k-1}) + \frac{\alpha}{k}(x_k - x_{k-1}) + \nabla f(x_{k+1}) + \frac{c}{k^2}\xi_k = 0, \quad (75) $$

with time step size equal to one. We take $\xi_k = x_k$, which gives

$$ (\text{IPATRE}) \quad \begin{cases} y_k = x_k + \alpha_k (x_k - x_{k-1}) \\ x_{k+1} = \text{prox}_f \left( y_k - \frac{c}{k^2} x_k \right), \end{cases} $$

where (IPATRE) stands for Inertial Proximal Algorithm with Tikhonov Regularization. According to (75) we have

$$ x_{k+1} = \alpha_k (x_k - x_{k-1}) - \nabla f(x_{k+1}) + \left(1 - \frac{c}{k^2}\right) x_k. \quad (76) $$

### 5.1 Convergence of values

We have the following result.

**Theorem 17** Let $(x_k)$ be a sequence generated by (IPATRE). Assume that $\alpha > 3$. Then for all $s \in [\frac{1}{2}, 1]$ the following hold:

(i) $f(x_k) - \min_{\mathcal{H}} f = o(k^{-2s})$, $\|x_k - x_{k-1}\| = o(k^{-s})$ and $\|\nabla f(x_k)\| = o(k^{-s})$ as $k \to +\infty$.

(ii) $\sum_{k=1}^{+\infty} k^{2s-1} (f(x_k) - \min_{\mathcal{H}} f) < +\infty$, $\sum_{k=1}^{+\infty} k^{2s-1} \|x_k - x_{k-1}\|^2 < +\infty$, $\sum_{k=1}^{+\infty} k^{2s} \|\nabla f(x_k)\|^2 < +\infty$.

**Proof** Given $x^* \in \operatorname{argmin}_f f$, set $f^* = f(x^*) = \min_{\mathcal{H}} f$. For $k \ge 2$, consider the discrete energy

$$ E_k := \|a_{k-1}(x_{k-1} - x^*) + b_{k-1}(x_k - x_{k-1} + \nabla f(x_k))\|^2 + d_{k-1}\|x_{k-1}\|^2, \quad (77) $$

<figure><img src="image_12.png" /></figure>
where $a_k = ak^{r-1}$, $2 < a < \alpha - 1$ and $b_k = k^r$, $r \in ]0, 1]$. The sequence ($d_k$) will be defined later. Set shortly $c_k := \frac{c}{k^2}$. Let us develop $E_k$.

$$ 
\begin{aligned}
E_k ={}& a_{k-1}^2 \|x_{k-1} - x^*\|^2 + b_{k-1}^2 \|x_k - x_{k-1}\|^2 + b_{k-1}^2 \|\nabla f(x_k)\|^2 \\
& + 2a_{k-1}b_{k-1} \langle x_k - x_{k-1}, x_{k-1} - x^* \rangle \\
& + 2a_{k-1}b_{k-1} \langle \nabla f(x_k), x_{k-1} - x^* \rangle \\
& + 2b_{k-1}^2 \langle \nabla f(x_k), x_k - x_{k-1} \rangle + d_{k-1} \|x_{k-1}\|^2.
\end{aligned} 
\quad (78) $$

Further

$$ 
\begin{aligned}
& 2a_{k-1}b_{k-1} \langle x_k - x_{k-1}, x_{k-1} - x^* \rangle \\
& = a_{k-1}b_{k-1} (\|x_k - x^*\|^2 - \|x_k - x_{k-1}\|^2 - \|x_{k-1} - x^*\|^2) \\
& 2a_{k-1}b_{k-1} \langle \nabla f(x_k), x_{k-1} - x^* \rangle = 2a_{k-1}b_{k-1} \langle \nabla f(x_k), x_k - x^* \rangle \\
& - 2a_{k-1}b_{k-1} \langle \nabla f(x_k), x_k - x_{k-1} \rangle.
\end{aligned} 
$$

Consequently, (78) becomes

$$ 
\begin{aligned}
E_k ={}& a_{k-1}b_{k-1} \|x_k - x^*\|^2 + (a_{k-1}^2 - a_{k-1}b_{k-1}) \|x_{k-1} - x^*\|^2 \\
& + (b_{k-1}^2 - a_{k-1}b_{k-1}) \|x_k - x_{k-1}\|^2 \\
& + b_{k-1}^2 \|\nabla f(x_k)\|^2 + 2a_{k-1}b_{k-1} \langle \nabla f(x_k), x_k - x^* \rangle \\
& + (2b_{k-1}^2 - 2a_{k-1}b_{k-1}) \langle \nabla f(x_k), x_k - x_{k-1} \rangle \\
& + d_{k-1} \|x_{k-1}\|^2.
\end{aligned} 
\quad (79) $$

Let us proceed similarly with $E_{k+1}$. Let us first observe that from (77) we have

$$ E_{k+1} = \|a_k(x_k - x^*) + b_k(\alpha_k(x_k - x_{k-1}) - c_k x_k)\|^2 + d_k \|x_k\|^2. $$

Therefore, after development we get

$$ 
\begin{aligned}
E_{k+1} ={}& a_k^2 \|x_k - x^*\|^2 + \alpha_k^2 b_k^2 \|x_k - x_{k-1}\|^2 + b_k^2 c_k^2 \|x_k\|^2 + 2\alpha_k a_k b_k \langle x_k - x_{k-1}, x_k - x^* \rangle \\
& - 2\alpha_k b_k^2 c_k \langle x_k - x_{k-1}, x_k \rangle - 2a_k b_k c_k \langle x_k, x_k - x^* \rangle + d_k \|x_k\|^2.
\end{aligned} 
\quad (80) $$

Further,

$$ 
\begin{aligned}
2\alpha_k a_k b_k \langle x_k - x_{k-1}, x_k - x^* \rangle ={}& -\alpha_k a_k b_k (\|x_{k-1} - x^*\| - \|x_k - x_{k-1}\|^2 - \|x_k - x^*\|^2) \\
& -2\alpha_k b_k^2 c_k \langle x_k - x_{k-1}, x_k \rangle = \alpha_k b_k^2 c_k (\|x_{k-1}\|^2 - \|x_k - x_{k-1}\|^2 - \|x_k\|^2) \\
& -2a_k b_k c_k \langle x_k, x_k - x^* \rangle = a_k b_k c_k (\|x^*\|^2 - \|x_k - x^*\|^2 - \|x_k\|^2).
\end{aligned} 
$$

Therefore, (80) yields

$$ 
\begin{aligned}
E_{k+1} ={}& (a_k^2 + \alpha_k a_k b_k - a_k b_k c_k) \|x_k - x^*\|^2 - \alpha_k a_k b_k \|x_{k-1} - x^*\|^2 \\
& + (\alpha_k^2 b_k^2 + \alpha_k a_k b_k - \alpha_k b_k^2 c_k) \|x_k - x_{k-1}\|^2 + (b_k^2 c_k^2 + d_k - \alpha_k b_k^2 c_k - a_k b_k c_k) \|x_k\|^2
\end{aligned} 
$$
$$+ \alpha_k b_k^2 c_k \|x_{k-1}\|^2 + a_k b_k c_k \|x^*\|^2. \quad (81)$$

By combining (79) and (81), we obtain

$$\begin{aligned} E_{k+1} - E_k ={}& (a_k^2 + \alpha_k a_k b_k - a_k b_k c_k - a_{k-1} b_{k-1}) \|x_k - x^*\|^2 \\ & + (-\alpha_k a_k b_k - a_{k-1}^2 + a_{k-1} b_{k-1}) \|x_{k-1} - x^*\|^2 \\ & + (\alpha_k^2 b_k^2 + \alpha_k a_k b_k - \alpha_k b_k^2 c_k - b_{k-1}^2 + a_{k-1} b_{k-1}) \|x_k - x_{k-1}\|^2 \\ & + (b_k^2 c_k^2 + d_k - \alpha_k b_k^2 c_k - a_k b_k c_k) \|x_k\|^2 \\ & + (\alpha_k b_k^2 c_k - d_{k-1}) \|x_{k-1}\|^2 - b_{k-1}^2 \|\nabla f(x_k)\|^2 \\ & + 2a_{k-1} b_{k-1} \langle \nabla f(x_k), x^* - x_k \rangle \\ & + (2b_{k-1}^2 - 2a_{k-1} b_{k-1}) \langle \nabla f(x_k), x_{k-1} - x_k \rangle + a_k b_k c_k \|x^*\|^2. \end{aligned} \quad (82)$$

By convexity of $f$, we have

$$\langle \nabla f(x_k), x^* - x_k \rangle \le f^* - f(x_k) \text{ and } \langle \nabla f(x_k), x_{k-1} - x_k \rangle \le f(x_{k-1}) - f(x_k).$$

According to the form of $(a_k)$ and $(b_k)$, there exists $k_0 \ge 2$ such that $b_k \ge a_k$ for all $k \ge k_0$. Consequently, $2b_{k-1}^2 - 2a_{k-1}b_{k-1} \ge 0$ which, according to the above convexity inequalities, gives

$$\begin{aligned} & 2a_{k-1}b_{k-1} \langle \nabla f(x_k), x^* - x_k \rangle + (2b_{k-1}^2 - 2a_{k-1}b_{k-1}) \langle \nabla f(x_k), x_{k-1} - x_k \rangle \\ & \le 2a_{k-1}b_{k-1}(f^* - f(x_k)) + (2b_{k-1}^2 - 2a_{k-1}b_{k-1}) [f(x_{k-1}) - f(x_k)] \\ & = -2a_{k-1}b_{k-1}(f(x_k) - f^*) + (2b_{k-1}^2 - 2a_{k-1}b_{k-1}) [(f(x_{k-1}) - f^*) - (f(x_k) - f^*)] \\ & = (2b_{k-1}^2 - 2a_{k-1}b_{k-1})(f(x_{k-1}) - f^*) - 2b_{k-1}^2(f(x_k) - f^*) \\ & = (2b_{k-1}^2 - 2a_{k-1}b_{k-1})(f(x_{k-1}) - f^*) \\ & \quad - (2b_k^2 - 2a_k b_k) + (2b_{k-1}^2 - 2b_k^2 + 2a_k b_k) (f(x_k) - f^*). \end{aligned} \quad (83)$$

Set $\mu_k := 2b_k^2 - 2a_k b_k$ and observe that $\mu_k \ge 0$ for all $k \ge k_0$, and $\mu_k \sim Ck^{2r}$ (we use $C$ as a generic positive constant). Let us also introduce $m_k := 2b_{k-1}^2 - 2b_k^2 + 2a_k b_k$, and observe that $m_k \ge 0$ for all $k \ge k_0$. Equivalently, let us show that for all $\frac{1}{2} \le r \le 1$ one has $b_k^2 - a_k b_k \le b_{k-1}^2$ for all $k \ge 1$. Equivalently $k^{2r} - ak^{2r-1} - (k-1)^{2r} \le 0$. By convexity of the function $x \mapsto x^{2r}$, the subgradient inequality gives

$$(x-1)^{2r} \ge x^{2r} - 2rx^{2r-1} \ge x^{2r} - ax^{2r-1},$$

where the second inequality comes from $2r < a$. Replacing $x$ with $k$ gives the claim. In addition $m_k \sim Ck^{2r-1}$. Combining (82) and (83), we obtain that for all $k \ge k_0$

$$\begin{aligned} E_{k+1} - E_k + \mu_k(f(x_k) - f^*) - \mu_{k-1}(f(x_{k-1}) - f^*) + m_k(f(x_k) - f^*) \\ \le (a_k^2 + \alpha_k a_k b_k - a_k b_k c_k - a_{k-1} b_{k-1}) \|x_k - x^*\|^2 \\ + (-\alpha_k a_k b_k - a_{k-1}^2 + a_{k-1} b_{k-1}) \|x_{k-1} - x^*\|^2 \end{aligned}$$

<figure><img src="image_13.png" /></figure>
$$ 
\begin{aligned}
&+ (\alpha_k^2 b_k^2 + \alpha_k a_k b_k - \alpha_k b_k^2 c_k - b_{k-1}^2 + a_{k-1} b_{k-1}) \|x_k - x_{k-1}\|^2 \\
&+ (b_k^2 c_k^2 + d_k - \alpha_k b_k^2 c_k - a_k b_k c_k) \|x_k\|^2 + (\alpha_k b_k^2 c_k - d_{k-1}) \|x_{k-1}\|^2 - b_{k-1}^2 \|\nabla f(x_k)\|^2 \\
&+ a_k b_k c_k \|x^*\|^2.
\end{aligned} 
\quad (84) $$

Let us now analyze the right hand side of (84).

i) Write the coefficient of $\|x_k - x^*\|^2$ so as to show a term similar to the coefficient of $\|x_{k-1} - x^*\|^2$. This will prepare the summation of these quantities. This gives

$$ 
\begin{aligned}
a_k^2 + \alpha_k a_k b_k - a_k b_k c_k - a_{k-1} b_{k-1} = (\alpha_{k+1} a_{k+1} b_{k+1} + a_k^2 - a_k b_k) \\
+ (\alpha_k a_k b_k - a_k b_k c_k - a_{k-1} b_{k-1} - \alpha_{k+1} a_{k+1} b_{k+1} + a_k b_k).
\end{aligned} 
\quad (85) $$

a) By definition, $\alpha_{k+1}a_{k+1}b_{k+1} + a_k^2 - a_k b_k = a(k+1)^{2r-1} - \alpha a(k+1)^{2r-2} + a^2 k^{2r-2} - ak^{2r-1}$. Proceeding as before, let us show that $a(x+1)^{2r-1} - \alpha a(x+1)^{2r-2} + a^2 x^{2r-2} - ax^{2r-1} \le 0$ for $x$ large enough. By taking $\frac{1}{2} \le r \le 1$, by convexity of the function $x \mapsto -x^{2r-1}$, the subgradient inequality gives $(2r-1)x^{2r-2} \ge (x+1)^{2r-1} - x^{2r-1}$. Therefore,

$$ 
\begin{aligned}
& a(x+1)^{2r-1} - a x^{2r-1} - \alpha a(x+1)^{2r-2} + a^2 x^{2r-2} \\
&\le a(2r-1)x^{2r-2} - \alpha a(x+1)^{2r-2} + a^2 x^{2r-2}.
\end{aligned} 
$$

But $a(2r-1)x^{2r-2} + a^2 x^{2r-2} \le \alpha a(x+1)^{2r-2}$ since $2r+a \le \alpha+1$ and the claim follows. Therefore, there exists $k_1 \ge k_0$ such that for all $\frac{1}{2} \le r \le 1$ we have

$$ \alpha_{k+1} a_{k+1} b_{k+1} + a_k^2 - a_k b_k \le 0, \text{ for all } k \ge k_1. \quad (86) $$

Set $v_k := -\alpha_{k+1}a_{k+1}b_{k+1} - a_k^2 + a_k b_k$. According to (86), $v_k \ge 0$ for all $k \ge k_1$, and $v_k \sim CK^{2r-2}$.

b) Consider now the second term in the right hand side of (85):

$$ 
\begin{aligned}
& a_k a_k b_k - a_k b_k c_k - a_{k-1} b_{k-1} - \alpha_{k+1} a_{k+1} b_{k+1} + a_k b_k \\
&= 2ak^{2r-1} - \alpha a k^{2r-2} - a c k^{2r-3} - a(k-1)^{2r-1} \\
&\quad -a(k+1)^{2r-1} + \alpha a(k+1)^{2r-2}.
\end{aligned} 
$$

Let us show that for all $\frac{1}{2} \le r \le 1$

$$ 
\begin{aligned}
\phi(x, r) = 2ax^{2r-1} - \alpha a x^{2r-2} - a c x^{2r-3} - a(x-1)^{2r-1} \\
- a(x+1)^{2r-1} + \alpha a(x+1)^{2r-2} \le 0
\end{aligned} 
$$

for $x$ large enough. By convexity of the function $x \mapsto x^{2r-1} - (x-1)^{2r-1}$ (one can easily verify that its second order derivative is nonnegative), the subgradient inequality gives $(x+1)^{2r-1} - 2x^{2r-1} + (x-1)^{2r-1} \ge (2r-1)(x^{2r-2} - (x-1)^{2r-2})$.
Therefore

$$
\begin{aligned}
\phi(x, r) &= -a[(x+1)^{2r-1} - 2x^{2r-1} + (x-1)^{2r-1}] \\
&\quad -\alpha ax^{2r-2} - acx^{2r-3} + \alpha a(k+1)^{2r-2} \\
&\le -a[(2r-1)(x^{2r-2} - (x-1)^{2r-2})] \\
&\quad -\alpha ax^{2r-2} - acx^{2r-3} + \alpha a(k+1)^{2r-2} \\
&= a(2r-1)(x-1)^{2r-2} - a(\alpha + 2r-1)x^{2r-2} \\
&\quad - acx^{2r-3} + \alpha a(x+1)^{2r-2}.
\end{aligned}
$$

Similarly, by convexity of the function $x \mapsto (x-1)^{2r-2} - x^{2r-2}$, the subgradient inequality gives $2x^{2r-2} - (x+1)^{2r-2} - (x-1)^{2r-2} \ge (2r-2)((x-1)^{2r-3} - x^{2r-3})$. Therefore, $a\alpha(x+1)^{2r-2} - a\alpha x^{2r-2} \le a\alpha(x^{2r-2} - (x-1)^{2r-2}) - a\alpha(2r-2)((x-1)^{2r-3} - x^{2r-3})$. Consequently,

$$
\phi(x, r) \le a(2r - 1 - \alpha)((x - 1)^{2r - 2} - x^{2r - 2}) - a\alpha(2r - 2)((x - 1)^{2r - 3} - x^{2r - 3}) - acx^{2r - 3}.
$$

Finally, by convexity of the function $x \mapsto x^{2r-2}$, the subgradient inequality gives $(x-1)^{2r-2} - x^{2r-2} \ge -(2r-2)x^{2r-3}$. Taking into account that $a(2r-1-\alpha) \le 0$ we get

$$
\phi(x, r) \leq -a(2r - 1 - 2\alpha)(2r - 2)x^{2r - 3} - a\alpha(2r - 2)(x - 1)^{2r - 3} - acx^{2r - 3}.
$$

Since $\frac{2\alpha+1-2r}{\alpha} > 1$ we obtain that $\phi(x, r) \le 0$ for $x > 1$.

Consequently, there exists $k_2 \ge k_1$ such that for all $\frac{1}{2} \le r \le 1$

$$
\alpha_k a_k b_k - a_k b_k c_k - a_{k-1} b_{k-1} - \alpha_{k+1} a_{k+1} b_{k+1} + a_k b_k \le 0, \text{ for all } k \ge k_2. \tag{87}
$$

Set $n_k := -\alpha_k a_k b_k + a_k b_k c_k + a_{k-1} b_{k-1} + \alpha_{k+1} a_{k+1} b_{k+1} - a_k b_k$. So $n_k \ge 0$ for all $k \ge k_2$ and $n_k \sim Ck^{2r-3}$.

ii) Let us now examine the coefficient of $\|x_k - x_{k-1}\|^2$. By definition we have

$$
\begin{aligned}
& \alpha_k^2 b_k^2 + \alpha_k a_k b_k - \alpha_k b_k^2 c_k - b_{k-1}^2 + a_{k-1} b_{k-1} \\
& = k^{2r} - (k-1)^{2r} + (-2\alpha + a)k^{2r-1} + a(k-1)^{2r-1} \\
& \quad + (\alpha^2 - \alpha a - c)k^{2r-2} + \alpha c k^{2r-3}.
\end{aligned}
$$

Let us show that for all $\frac{1}{2} \le r \le 1$

$$
\phi(x, r) = x^{2r} - (x - 1)^{2r} + (-2\alpha + a)x^{2r-1} + a(x - 1)^{2r-1} \\
+ (\alpha^2 - \alpha a - c)x^{2r-2} + \alpha c x^{2r-3} \le 0,
$$

<figure><img src="image_14.png" /></figure>
if $x$ is large enough. By convexity of the function $x \mapsto x^{2r} - ax^{2r-1}$, the subgradient inequality gives $((x-1)^{2r} - a(x-1)^{2r-1}) - (x^{2r} - ax^{2r-1}) \ge -(2rx^{2r-1} - a(2r-1)x^{2r-2})$. Therefore, taking into account that $r - \alpha + a \le 1 - \alpha + a \le 0$, we obtain

$$ \phi(x, r) \le 2(r - \alpha + a)x^{2r-1} - a(2r-1)x^{2r-2} + (\alpha^2 - \alpha a - c)x^{2r-2} + \alpha c x^{2r-3} \le 0, $$

for $x$ large enough. Consequently, there exist $k_3 \ge k_2$ such that for all $\frac{1}{2} \le r \le 1$

$$ \alpha_k^2 b_k^2 + \alpha_k a_k b_k - \alpha_k b_k^2 c_k - b_{k-1}^2 + a_{k-1} b_{k-1} \le 0, \text{ for all } k \ge k_3. \quad (88) $$

Set $\eta_k := -\alpha_k^2 b_k^2 - \alpha_k a_k b_k + \alpha_k b_k^2 c_k + b_{k-1}^2 - a_{k-1} b_{k-1}$. So $\eta_k \ge 0$ for all $k \ge k_3$ and $\eta_k \sim Ck^{2r-1}$.

iii) The coefficient of $\|x_{k-1}\|^2$ is $\alpha_k b_k^2 c_k - d_{k-1}$. We proceed in a similar way as in $i$), and write the coefficient of $\|x_k\|^2$ as

$$ b_k^2 c_k^2 + d_k - \alpha_k b_k^2 c_k - a_k b_k c_k = (-\alpha_{k+1} b_{k+1}^2 c_{k+1} + d_k) + (b_k^2 c_k^2 + \alpha_{k+1} b_{k+1}^2 c_{k+1} - \alpha_k b_k^2 c_k - a_k b_k c_k). $$

We have

$$ b_k^2 c_k^2 + \alpha_{k+1} b_{k+1}^2 c_{k+1} - \alpha_k b_k^2 c_k - a_k b_k c_k = c^2 k^{2r-4} + c(k+1)^{2r-2} - \alpha c(k+1)^{2r-3} - ck^{2r-2} + \alpha c k^{2r-3} - ack^{2r-3}. $$

Let us show that for all $\frac{1}{2} \le r \le 1$

$$ \phi(x, r) = c(x+1)^{2r-2} - \alpha c(x+1)^{2r-3} - cx^{2r-2} + \alpha c x^{2r-3} - acx^{2r-3} + c^2 x^{2r-4} \le 0 $$

for $x$ large enough. Since for $x$ large enough, the function $x \mapsto x^{2r-2} - \alpha x^{2r-3}$ is convex, the subgradient inequality gives

$$ (x^{2r-2} - \alpha x^{2r-3}) - ((x+1)^{2r-2} - \alpha(x+1)^{2r-3}) \ge -((2r-2)(x+1)^{2r-3} - \alpha(2r-3)(x+1)^{2r-4}). $$

Therefore, by taking into account that $r \le 1$, we obtain

$$ \phi(x, r) \le (2r-2)c(x+1)^{2r-3} - \alpha(2r-3)c(x+1)^{2r-4} - acx^{2r-3} + c^2 x^{2r-4} \le 0 $$

for $x$ large enough. Consequently, there exists $k_4 \ge k_3$ such that for all $\frac{1}{2} \le r \le 1$ we have

$$ b_k^2 c_k^2 + \alpha_{k+1} b_{k+1}^2 c_{k+1} - \alpha_k b_k^2 c_k - a_k b_k c_k \le 0 \text{ for all } k \ge k_4. \quad (89) $$

Let us denote $\sigma_k := \alpha_{k+1} b_{k+1}^2 c_{k+1} - d_k$ and $s_k := -b_k^2 c_k^2 - \alpha_{k+1} b_{k+1}^2 c_{k+1} + \alpha_k b_k^2 c_k + a_k b_k c_k$ and observe that $s_k \ge 0$ for all $k \ge k_4$ and $s_k \sim Ck^{2r-3}$.
Combining (84), (86), (87), (88) and (89) we obtain that for all $k \ge k_4$ and $r \in [\frac{1}{2}, 1]$ it holds

$$ 
\begin{aligned}
E_{k+1} - E_k + \mu_k(f(x_k) - f^*) - \mu_{k-1}(f(x_{k-1}) - f^*) + m_k(f(x_k) - f^*) \\
+ \nu_k \|x_k - x^*\|^2 - \nu_{k-1} \|x_{k-1} - x^*\|^2 + n_k \|x_k - x^*\|^2 \\
+ \sigma_k \|x_k\|^2 - \sigma_{k-1} \|x_{k-1}\|^2 + s_k \|x_k\|^2 \\
+ \eta_k \|x_k - x_{k-1}\|^2 + b_{k-1}^2 \|\nabla f(x_k)\|^2 \le a_k b_k c_k \|x^*\|^2.
\end{aligned} 
\quad (90) $$

Finally, take $d_{k-1} = \frac{1}{2}\alpha_k b_k^2 c_k$. Then, $\sigma_k = \frac{1}{2}\alpha_{k+1}b_{k+1}^2 c_{k+1} \sim Ck^{2r-2}$, $\sigma_k \ge 0$ for all $k \ge k_5 = \max(\alpha - 1, k_4)$. Further, $\mu_k, m_k, \nu_k, n_k, s_k$ and $\eta_k$ are nonnegative for all $k \ge k_5$ and $r \in [\frac{1}{2}, 1]$.

Assume now that $\frac{1}{2} \le r < 1$. According to $\sum_{k \ge k_5} a_k b_k c_k \|x^*\|^2 = ac \|x^*\|^2$ and $\sum_{k \ge k_5} k^{2r-3} = C < +\infty$, by summing up (90) from $k = k_5$ to $k = n > k_5$, we obtain that there exists $C_1 > 0$ such that

$$ 
\begin{aligned}
E_{n+1} &\le C_1, \\
\mu_n(f(x_n) - f^*) &\le C_1, \text{ hence } f(x_n) - f^* = \mathcal{O}(n^{-2r}), \\
\sum_{k \ge k_5} m_k(f(x_k) - f^*) &\le C_1, \text{ hence } \sum_{k \ge 1} k^{2r-1}(f(x_k) - f^*) < +\infty, \\
\nu_k \|x_k - x^*\|^2 &\le C_1, \text{ hence } \|x_n - x^*\| = \mathcal{O}(n^{1-r}), \\
\sum_{k \ge k_5} n_k \|x_k - x^*\|^2 &\le C_1, \text{ hence } \sum_{k \ge 1} k^{2r-3} \|x_k - x^*\|^2 < +\infty, \\
\sigma_k \|x_k\|^2 &\le C_1, \text{ hence } \|x_n\| = \mathcal{O}(n^{1-r}), \\
\sum_{k \ge k_5} s_k \|x_k\|^2 &\le C_1, \text{ hence } \sum_{k \ge 1} k^{2r-3} \|x_k\|^2 < +\infty, \\
\sum_{k \ge k_5} \eta_k \|x_k - x_{k-1}\|^2 &\le C_1, \text{ hence } \sum_{k \ge 1} k^{2r-1} \|x_k - x_{k-1}\|^2 < +\infty \\
\sum_{k \ge k_5} b_{k-1}^2 \|\nabla f(x_k)\|^2 &\le C_1, \text{ hence } \sum_{k \ge 1} k^{2r} \|\nabla f(x_k)\|^2 < +\infty.
\end{aligned}
$$

Since $\sum_{k \ge 1} k^{2r} \|\nabla f(x_k)\|^2 < +\infty$, we have $\|\nabla f(x_n)\| = o(n^{-r})$. Combining this property with $E_{n+1} \le C_1$ yields $\sup_{n \ge 1} \|an^{r-1}(x_n - x^*) + n^r(x_{n+1} - x_n)\| + \frac{c}{2}(1 - \frac{\alpha}{n})n^{2r-2}\|x_{n-1}\|^2 < +\infty$.

Let us show now, that $f(x_n) - f^* = o(n^{-2r})$ and $\|x_n - x_{n-1}\| = o(n^{-r})$. From (90) we get

$$ 
\begin{aligned}
&\sum_{k \ge 1} [(E_{k+1} + \mu_k(f(x_k) - f^*) + \nu_k \|x_k - x^*\|^2) \\
&\quad - (E_k + \mu_{k-1}(f(x_{k-1}) - f^*) + \nu_{k-1} \|x_{k-1} - x^*\|^2)]_+ < +\infty.
\end{aligned}
$$
Therefore, the following limit exists

$$ \lim_{k \to +\infty} (\|ak^{r-1}(x_k - x^*) + k^r(x_{k+1} - x_k)\|^2 \\ +d_k \|x_k\|^2 + \mu_k(f(x_k) - f^*) + \nu_k \|x_k - x^*\|^2). $$

Note that $d_k \sim Ck^{2r-2}$, $\mu_k \sim Ck^{2r}$ and $\nu_k \sim Ck^{2r-2}$. Further, we have $\sum_{k \ge 1} k^{2r-3} \|x_k - x^*\|^2 < +\infty$, $\sum_{k \ge 1} k^{2r-1} \|x_k - x_{k-1}\|^2 < +\infty$, $\sum_{k \ge 1} k^{2r-1} (f(x_k) - f^*) < +\infty$ and $\sum_{k \ge 1} k^{2r-3} \|x_k\|^2 < +\infty$, hence

$$ \sum_{k \ge 1} \frac{1}{k} (\|ak^{r-1}(x_k - x^*) + k^r(x_{k+1} - x_k)\|^2 + d_k \|x_k\|^2 \\ +\mu_k(f(x_k) - f^*) + \nu_k \|x_k - x^*\|^2) < +\infty. $$

Since $\sum_{k \ge 1} \frac{1}{k} = +\infty$ we get

$$ \lim_{k \to +\infty} (\|ak^{r-1}(x_k - x^*) + k^r(x_{k+1} - x_k)\|^2 + d_k \|x_k\|^2 \\ +\mu_k(f(x_k) - f^*) + \nu_k \|x_k - x^*\|^2) = 0 $$

and the claim follows. □

**Remark 18** The convergence rate of the values is $f(x_k) - \min_{\mathcal{H}} f = o(k^{-2s})$ for any $0 < s < 1$. Practically it is as good as the rate $f(x(t)) - \min_{\mathcal{H}} f = O(\frac{1}{t^2})$ obtained for the continuous dynamic.

## 5.2 Strong convergence to the minimum norm solution

**Theorem 19** Take $\alpha > 3$. Let $(x_k)$ be a sequence generated by (IPATRE). Let $x^*$ be the minimum norm element of argmin $f$. Then, $\lim_{k \to +\infty} \|x_k - x^*\| = 0$. Further, $(x_k)$ converges strongly to $x^*$ whenever $(x_k)$ is in the interior of the ball $B(0, \|x^*\|)$ for $k$ large enough, or $(x_k)$ is in the complement of the ball $B(0, \|x^*\|)$ for $k$ large enough.

**Proof Case I.** Assume that there exists $k_0 \in \mathbb{N}$ such that $\|x_k\| \ge \|x^*\|$ for all $k \ge k_0$. Set $c_k = \frac{c}{k^2}$, and define $f_{c_k}(x) := f(x) + \frac{c}{2k^2} \|x\|^2$. Consider the energy function defined in (77) with $r = 1$, that is $a_k = a$ and $b_k = k^2$, where we assume that $\max(2, \alpha - 2) < a < \alpha - 1$. Then,

$$ E_k = \|a(x_{k-1} - x^*) + (k-1)^2(x_k - x_{k-1} + \nabla f(x_k))\|^2 + d_{k-1} \|x_{k-1}\|^2, $$

where the sequence $(d_k)$ will be defined later. Next, we introduce another energy functional

$$ \mathcal{E}_k = \frac{1}{2} c_{k-1} (\|x_{k-1}\|^2 - \|x^*\|^2) + \|a(x_{k-1} - x^*) + (k-1)^2 (x_k - x_{k-1} + \nabla f(x_k))\|^2 $$
$$+d_{k-1}\|x_{k-1}\|^2. \qquad (91)$$

Note that $\mathcal{E}_k = \frac{1}{2}c_{k-1}(\|x_{k-1}\|^2 - \|x^*\|^2) + E_k$. Then,

$$\mathcal{E}_{k+1} - \mathcal{E}_k = \frac{1}{2}c_k(\|x_k\|^2 - \|x^*\|^2) - \frac{1}{2}c_{k-1}(\|x_{k-1}\|^2 - \|x^*\|^2) + E_{k+1} - E_k. \qquad (92)$$

According to (90), there exists $k_1 \ge k_0$ such that for all $k \ge k_1$

$$\begin{aligned} \mathcal{E}_{k+1} - \mathcal{E}_k + \mu_k(f(x_k) - f^*) - \mu_{k-1}(f(x_{k-1}) - f^*) + m_k(f(x_k) - f^*) \\ + v_k\|x_k - x^*\|^2 - v_{k-1}\|x_{k-1} - x^*\|^2 + n_k\|x_k - x^*\|^2 \\ + \eta_k\|x_k - x_{k-1}\|^2 + b_{k-1}^2\|\nabla f(x_k)\|^2 \le -\sigma_k\|x_k\|^2 + \sigma_{k-1}\|x_{k-1}\|^2 - s_k\|x_k\|^2 \\ + \frac{1}{2}c_k(\|x_k\|^2 - \|x^*\|^2) - \frac{1}{2}c_{k-1}(\|x_{k-1}\|^2 - \|x^*\|^2) + a_k b_k c_k \|x^*\|^2. \end{aligned} \qquad (93)$$

Adding $\frac{1}{2}(\mu_k + m_k)c_k(\|x_k\|^2 - \|x^*\|^2) - \frac{1}{2}\mu_{k-1}c_{k-1}(\|x_{k-1}\|^2 - \|x^*\|^2)$ to both side of (93) we get

$$\begin{aligned} \mathcal{E}_{k+1} - \mathcal{E}_k + \mu_k(f_{c_k}(x_k) - f_{c_k}(x^*)) - \mu_{k-1}(f_{c_{k-1}}(x_{k-1}) - f_{c_{k-1}}(x^*)) \\ + m_k(f_{c_k}(x_k) - f_{c_k}(x^*)) \\ + v_k\|x_k - x^*\|^2 - v_{k-1}\|x_{k-1} - x^*\|^2 + n_k\|x_k - x^*\|^2 \\ + \eta_k\|x_k - x_{k-1}\|^2 + b_{k-1}^2\|\nabla f(x_k)\|^2 \\ \le -\sigma_k\|x_k\|^2 + \sigma_{k-1}\|x_{k-1}\|^2 - s_k\|x_k\|^2 \\ + \frac{1}{2}(\mu_k + m_k + 1)c_k(\|x_k\|^2 - \|x^*\|^2) \\ - \frac{1}{2}(\mu_{k-1} + 1)c_{k-1}(\|x_{k-1}\|^2 - \|x^*\|^2) + a_k b_k c_k \|x^*\|^2. \end{aligned} \qquad (94)$$

The right hand side of (94) can be written as

$$\begin{aligned} & \left(\frac{1}{2}(\mu_k + m_k + 1)c_k - \sigma_k - s_k\right) (\|x_k\|^2 - \|x^*\|^2) \\ & + \left(-\frac{1}{2}(\mu_{k-1} + 1)c_{k-1} + \sigma_{k-1}\right) (\|x_{k-1}\|^2 - \|x^*\|^2) \\ & + (a_k b_k c_k - \sigma_k - s_k + \sigma_{k-1}) \|x^*\|^2. \end{aligned}$$

In this case we have $\mu_k = 2b_k^2 - 2a_k b_k = 2k^2 - 2ak$ and $m_k = 2b_{k-1}^2 - 2b_k^2 + 2a_k b_k = 2(a - 2)k + 2$. Further, $\sigma_k = \alpha_{k+1}b_{k+1}^2 c_{k+1} - d_k = c - \frac{\alpha c}{k+1} - d_k$ and $s_k = b_k^2 c_k - \alpha_{k+1} b_{k+1}^2 c_{k+1} + \alpha_k b_k^2 c_k + a_k b_k c_k = \frac{\alpha c}{k+1} + \frac{c(a-\alpha)}{k} + \frac{c^2}{k^2}$. Now, take $d_k = \frac{(a+2-\alpha)c}{2k} \ge 0$ and an easy computation gives that there exists $k_2 \ge k_1$ such that

<figure><img src="image_15.png" /></figure>
for all $k \ge k_2$ one has

$$
\begin{align*}
\frac{1}{2}(\mu_k + m_k + 1)c_k - \sigma_k - s_k &= -\frac{(a + 2 - \alpha)c}{2k} + \frac{2c^2 - 3c}{2k^2} \le 0, \\
-\frac{1}{2}(\mu_{k-1} + 1)c_{k-1} + \sigma_{k-1} &= \frac{c(a - 2 - \alpha)}{2(k-1)} + \frac{\alpha c}{k(k-1)} - \frac{c}{2(k-1)^2} \le 0 \\
a_k b_k c_k - \sigma_k - s_k + \sigma_{k-1} &= \frac{(a + 2 - \alpha)c}{2k} - \frac{(a + 2 - \alpha)c}{2(k-1)} - \frac{c^2}{k^2} \le 0.
\end{align*}
$$

Now, since by assumption $\|x_k\| \ge \|x^*\|$ for $k \ge k_0$, we get that the right hand side of (94) is nonpositive for all $k \ge k_2$. Hence, for all $k \ge k_2$ we have

$$
\begin{align}
\mathcal{E}_{k+1} - \mathcal{E}_k + \mu_k (f_{c_k}(x_k) - f_{c_k}(x^*)) - \mu_{k-1} (f_{c_{k-1}}(x_{k-1}) - f_{c_{k-1}}(x^*)) \nonumber \\
+ m_k (f_{c_k}(x_k) - f_{c_k}(x^*)) \nonumber \\
+ \nu_k \|x_k - x^*\|^2 - \nu_{k-1} \|x_{k-1} - x^*\|^2 + n_k \|x_k - x^*\|^2 \nonumber \\
+ \eta_k \|x_k - x_{k-1}\|^2 + b_{k-1}^2 \|\nabla f(x_k)\|^2 \le 0. \tag{95}
\end{align}
$$

Note that $\nu_k \sim C$. Therefore, from (95), similarly as in the proof of Theorem 17, we deduce that $\|x_k - x^*\|$ is bounded, and therefore $(x_k)$ is bounded. Further,

$$
\lim_{k \to +\infty} (\|a(x_k - x^*) + k(x_{k+1} - x_k)\|^2 + \mu_k (f_{c_k}(x_k) - f_{c_k}(x^*)) + \nu_k \|x_k - x^*\|^2) = 0,
$$

that is, $\lim_{k \to +\infty} \nu_k \|x_k - x^*\|^2 = 0$ and hence $\lim_{k \to +\infty} x_k = x^*$.

**Case II.** Assume that there exists $k_0 \in \mathbb{N}$ such that $\|x_k\| < \|x^*\|$ for all $k \ge k_0$. From there we get that $(x_k)$ is bounded. Now, take $\bar{x} \in \mathcal{H}$ a weak sequential cluster point of $(x_k)$, which exists since $(x_k)$ is bounded. This means that there exists a sequence $(k_n)_{n \in \mathbb{N}} \subseteq [k_0, +\infty) \cap \mathbb{N}$ such that $k_n \to +\infty$ and $x_{k_n}$ converges weakly to $\bar{x}$ as $n \to +\infty$. Since $f$ is weakly lower semicontinuous, according to Theorem 17 we have $f(\bar{x}) \le \liminf_{n \to +\infty} f(x_{k_n}) = \min f$, hence $\bar{x} \in \operatorname{argmin} f$. Since the norm is weakly lower semicontinuous, we deduce that

$$
\| \bar{x} \| \le \liminf_{n \to +\infty} \| x_{k_n} \| \le \| x^* \|.
$$

According to the definition of $x^*$, we get $\bar{x} = x^*$. Therefore $(x_k)$ converges weakly to $x^*$. So

$$
\|x^*\| \le \liminf_{k \to +\infty} \|x_k\| \le \limsup_{t \to +\infty} \|x_k\| \le \|x^*\|.
$$

Therefore, we have $\lim_{k \to +\infty} \|x_k\| = \|x^*\|$. From the previous relation and the fact that $x_k \to x^*$ as $k \to +\infty$, we obtain the strong convergence, that is $\lim_{k \to +\infty} x_k = x^*$.

**Case III.** Suppose that for every $k \ge k_0$ there exists $l \ge k$ such that $\|x^*\| > \|x_l\|$,
and suppose also there exists $m \ge k$ such that $\|x^*\| \le \|x_m\|$. So, let $k_1 \ge k_0$ and
$l_1 \ge k_1$ such that $\|x^*\| > \|x_{l_1}\|$. Let $k_2 > l_1$ and $l_2 \ge k_2$ such that $\|x^*\| > \|x_{l_2}\|$.
Continuing the process, we obtain ($x_{l_n}$), a subsequence of ($x_k$) with the property that $\|x_{l_n}\| < \|x^*\|$ for all $n \in \mathbb{N}$. By reasoning as in **Case II**, we obtain that $\lim_{n \to +\infty} x_{l_n} = x^*$. Consequently, $\liminf_{k \to +\infty} \|x_k - x^*\| = 0$. $\square$

## 5.3 Non-smooth case

Let us extend the results of the previous sections to the case of a proper lower semi-continuous and convex function $f: \mathcal{H} \to \mathbb{R} \cup \{+\infty\}$. We rely on the basic properties of the Moreau envelope $f_\lambda: \mathcal{H} \to \mathbb{R}$ ($\lambda$ is a positive real parameter), which is defined by

$$f_{\lambda}(x) = \min_{z \in \mathcal{H}} \left\{ f(z) + \frac{1}{2\lambda} \|z - x\|^2 \right\}, \quad \text{for any } x \in \mathcal{H}.$$

Recall that $f_\lambda$ is a convex differentiable function, whose gradient is $\lambda^{-1}$-Lipschitz continuous, and such that $\min_{\mathcal{H}} f = \min_{\mathcal{H}} f_\lambda$, $\arg\min_{\mathcal{H}} f_\lambda = \arg\min_{\mathcal{H}} f$. The interested reader may refer to Bauschke and Combettes (2011); Brézis (1972) for a comprehensive treatment of the Moreau envelope in a Hilbert setting. Since the set of minimizers is preserved by taking the Moreau envelope, the idea is to replace $f$ by $f_\lambda$ in the previous algorithm, and take advantage of the fact that $f_\lambda$ is continuously differentiable. Then, algorithm (IPATRE) applied to $f_\lambda$ now reads

$$(\text{IPATRE}) \quad \begin{cases} y_k = x_k + \alpha_k (x_k - x_{k-1}) \\ x_{k+1} = \operatorname{prox}_{f_\lambda} \left( y_k - \frac{c}{k^2} x_k \right). \end{cases}$$

By applying Theorems 17 and 19, we obtain fast convergence of the sequence ($x_k$) to the element of minimum norm of $f$. Thus, we just need to formulate these results in terms of $f$ and its proximal mapping. This is straightforward thanks to the following formulae from proximal calculus (Bauschke and Combettes 2011):

1. $f_\lambda(x) = f(\operatorname{prox}_{\lambda f}(x)) + \frac{1}{2\lambda} \|x - \operatorname{prox}_{\lambda f}(x)\|^2$.

2. $\nabla f_\lambda(x) = \frac{1}{\lambda} (x - \operatorname{prox}_{\lambda f}(x))$.

3. $\operatorname{prox}_{\theta f_\lambda}(x) = \frac{\lambda}{\lambda+\theta} x + \frac{\theta}{\lambda+\theta} \operatorname{prox}_{(\lambda+\theta)f}(x)$.

We obtain the following relaxed inertial proximal algorithm (NS stands for non-smooth):

$$(\text{IPATRE-NS}) \quad \begin{cases} y_k = x_k + (1 - \frac{\alpha}{k})(x_k - x_{k-1}) \\ x_{k+1} = \frac{\lambda}{1+\lambda} \left(y_k - \frac{c}{k^2} x_k\right) + \frac{1}{1+\lambda} \operatorname{prox}_{(\lambda+1)f} \left(y_k - \frac{c}{k^2} x_k\right). \end{cases}$$

**Theorem 20** Let $f: \mathcal{H} \to \mathbb{R} \cup \{+\infty\}$ be a convex, lower semicontinuous, proper function. Assume that $\alpha > 3$. Let ($x_k$) be a sequence generated by (IPATRE-NS). Then for all $s \in [\frac{1}{2}, 1[$, we have:

$$(i) \quad f(\operatorname{prox}_{\lambda f}(x_k)) - \min_{\mathcal{H}} f = o(k^{-2s}), \quad \|x_k - x_{k-1}\| = o(k^{-s}), \\ \qquad \|x_k - \operatorname{prox}_{\lambda f}(x_k)\| = o(k^{-s}) \quad \text{as } k \to +\infty.$$
$$ (ii) \quad \sum_{k=1}^{+\infty} k^{2s-1} \left( f(\operatorname{prox}_{\lambda, f}(x_k)) - \min_{\mathcal{H}} f \right) < +\infty, \quad \sum_{k=1}^{+\infty} k^{2s-1} \|x_k - x_{k-1}\|^2 < +\infty, \\ \sum_{k=1}^{+\infty} k^{2s} \|x_k - \operatorname{prox}_{\lambda, f}(x_k)\|^2 < +\infty. $$

$$ (iii) \liminf_{k \to +\infty} \|x_k - x^*\| = 0. \text{ Further, } (x_k) \text{ converges strongly to } x^* \text{ the element of minimum norm of argmin } f, \text{ if } (x_k) \text{ is in the interior of the ball } B(0, \|x^*\|) \text{ for } k \text{ large enough, or if } (x_k) \text{ is in the complement of the ball } B(0, \|x^*\|) \text{ for } k \text{ large enough.} $$

# 6 Conclusion, perspective

In the framework of convex optimization in general Hilbert spaces, we have introduced an inertial dynamic in which the damping coefficient and the Tikhonov regularization coefficient vanish as time tends to infinity. The judicious adjustment of these parameters makes it possible to obtain trajectories converging quickly (and strongly) towards the minimum norm solution. This seems to be the first time that these two properties have been obtained for the same dynamic. Indeed, the Nesterov accelerated gradient method and the hierarchical minimization attached to the Tikhonov regularization are fully effective within this dynamic. On the basis of Lyapunov's analysis, we have developed an in-depth mathematical study of the dynamic which is a valuable tool for the development of corresponding results for algorithms obtained by temporal discretization. We thus obtained similar results for the corresponding proximal algorithms. This study opens up a large field of promising research concerning first-order optimization algorithms. Many interesting questions such as the introduction of Hessian-driven damping to attenuate oscillations (Attouch et al. 2022, 2016; Boł et al. 2021), and the study of the impact of errors, perutrbations, deserve further study. These results also adapt well to the numerical analysis of inverse problems for which strong convergence and obtaining a solution close to a desired state are key properties.

**Funding** The research leading to these results received funding from Romanian Ministry of Research, Innovation and Digitization, CNCS-UEFISCDI, project number PN-III-P1-1.1-TE-2021-0138, within PNCDI III.

**Data availability** In this manuscript only the datasets generated by authors were analysed.

## Declarations

**Conflict of interest** The authors have no relevant financial or non-financial interests to disclose.

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
References

Alvarez F, Attouch H (2001) Convergence and asymptotic stabilization for some damped hyperbolic equations with non-isolated equilibria. ESAIM Control Optim Calc Var 6:539-552

Alvarez F, Cabot A (2006) Asymptotic selection of viscosity equilibria of semilinear evolution equations by the introduction of a slowly vanishing term. Discrete Contin Dyn Syst 15:921-938

Apidopoulos V, Aujol J-F, Dossal Ch (2018) The differential inclusion modeling the FISTA algorithm and optimality of convergence rate in the case $b \le 3$. SIAM J Optim 28(1):551-574

Attouch H (1996) Viscosity solutions of minimization problems. SIAM J Optim 6(3):769-806

Attouch H, Balhag A, Chbani Z, Riahi H (2022) Damped inertial dynamics with vanishing Tikhonov regularization: strong asymptotic convergence towards the minimum norm solution. J Differ Equ 311:29-58

Attouch H, Balhag A, Chbani Z, Riahi H (2023) Accelerated gradient methods combining Tikhonov regularization with geometric damping driven by the hessian. Appl Math Optim 88:29

Attouch H, Boł RI, Csetnek ER (2023) Fast optimization via inertial dynamics with closed-loop damping. J Eur Math Soc 25:1985-2056

Attouch H, Briceño-Arias LM, Combettes PL (2010) A parallel splitting method for coupled monotone inclusions. SIAM J Control Optim 48(5):3246-3270

Attouch H, Briceño-Arias LM, Combettes PL (2016) A strongly convergent primal-dual method for nonoverlapping domain decomposition. Numer Math 133(3):443-470

Attouch H, Cabot A (2017) Asymptotic stabilization of inertial gradient dynamics with time-dependent viscosity. J Differ Equ 263(9):5412-5458

Attouch H, Chbani Z, Fadili J, Riahi H (2022) First order optimization algorithms via inertial systems with Hessian driven damping. Math Progr 193:113-155

Attouch H, Chbani Z, Peypouquet J, Redont P (2018) Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity. Math Progr 168(1-2):123-175

Attouch H, Chbani Z, Riahi H (2018) Combining fast inertial dynamics for convex optimization with Tikhonov regularization. J Math Anal Appl 457:1065-1094

Attouch H, Cominetti R (1996) A dynamical approach to convex minimization coupling approximation with the steepest descent method. J Differ Equ 128(2):519-540

Attouch H, Czarnecki M-O (2002) Asymptotic control and stabilization of nonlinear oscillators with non-isolated equilibria. J Differ Equ 179:278-310

Attouch H, Czarnecki M-O (2010) Asymptotic behavior of coupled dynamical systems with multiscale aspects. J Differ Equ 248:1315-1344

Attouch H, Czarnecki M-O, Peypouquet J (2011) Prox-penalization and splitting methods for constrained variational problems. SIAM J Optim 21:149-173

Attouch H, Czarnecki M-O, Peypouquet J (2011) Coupling forward-backward with penalty schemes and parallel splitting for constrained variational inequalities. SIAM J Optim 21:1251-1274

Attouch H, Czarnecki M-O (2017) Asymptotic behavior of gradient-like dynamical systems involving inertia and multiscale aspects. J Differ Equ 262(3):2745-2770

Attouch H, Peypouquet J (2016) The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than $1/k^2$. SIAM J Optim 26(3):1824-1834

Attouch H, Peypouquet J, Redont P (2016) Fast convex minimization via inertial dynamics with Hessian driven damping. J Differ Equ 261(10):5734-5783

Baillon J-B, Cominetti R (2001) A convergence result for non-autonomous subgradient evolution equations and its application to the steepest descent exponential penalty trajectory in linear programming. J Funct Anal 187:263-273

Bauschke H, Combettes PL (2011) Convex analysis and monotone operator theory in Hilbert spaces. CMS Books in Mathematics, Springer

Boł RI, Csetnek ER (2014) Forward-Backward and Tseng's type penalty schemes for monotone inclusion problems. Set-Valued Var Anal 22:313-331

Boł RI, Csetnek ER, László SC (2021) Tikhonov regularization of a second order dynamical system with Hessian damping. Math Progr 189:151-186

Brézis H (1972) Opérateurs maximaux monotoles dans les espaces de Hilbert et équations d'évolution, Lecture Notes 5, North Holland

Cabot A (2004) Inertial gradient-like dynamical system controlled by a stabilizing term. J Optim Theory Appl 120:275-303
Cabot A (2005) Proximal point algorithm controlled by a slowly vanishing term: applications to hierarchical minimization. SIAM J Optim 15(2):555-572

Cabot A, Engler H, Gadat S (2009) On the long time behavior of second order differential equations with asymptotically small dissipation Trans. Am Math Soc 361:5983-6017

Chambolle A, Dossal Ch (2015) On the convergence of the iterates of Fista. J Optim Theory Appl 166:968-982

Cominetti R (1997) Coupling the proximal point algorithm with approximation methods. J Optim Theory Appl 95(3):581-600

Cominetti R, Peypouquet J, Sorin S (2008) Strong asymptotic convergence of evolution equations governed by maximal monotone operators with Tikhonov regularization. J Differ Equ 245:3753-3763

Fiacco A, McCormick G (1968) Nonlinear programming: sequential unconstrained minimization techniques. John Wiley and Sons, New York

Haraux A, Jendoubi MA (2016) A Liapunov function approach to the stabilization of second-order coupled systems arXiv preprint arXiv:1604.06547

Hirstoaga SA (2006) Approximation et résolution de problèmes d'équilibre, de point fixe et d'inclusion monotone. PhD thesis, Université Pierre et Marie Curie - Paris VI, HAL Id: tel-00137228

Jendoubi MA, May R (2010) On an asymptotically autonomous system with Tikhonov type regularizing term. Arch Math 95(4):389-399

László SC (2023) On the strong convergence of the trajectories of a Tikhonov regularized second order dynamical system with asymptotically vanishing damping. J Differ Equ 362:355-381

László SC (2024) Solving convex optimization problems via a second order dynamical system with implicit Hessian damping and Tikhonov regularization arXiv preprint arXiv:2401.02676

Nesterov Y (1983) A method of solving a convex programming problem with convergence rate $O(1/k^2)$. Soviet Math Dokl 27:372-376

Nesterov Y (2004) Introductory lectures on convex optimization: A basic course, vol 87. Applied Optimization. Kluwer Academic Publishers, Boston, MA

Polyak B (1987) Introduction to Optimization. Optimization Software-Inc, New York

Su W, Boyd S, Candès EJ (2016) A differential equation for modeling Nesterov's accelerated gradient method: theory and insights. J Mach Learn Res 17(153):1-43

Tikhonov AN (1963) Solution of incorrectly formulated problems and the regularization method. Sov Math 4:1035-1038

Tikhonov AN, Arsenin VY (1977) Solutions of Ill-Posed Problems. Winston, New York

**Publisher's Note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.