# Quasi-Nonexpansive Iterations on the Affine Hull of Orbits: From Mann's Mean Value Algorithm to Inertial Methods

Patrick L Combettes, Lilian E. Glaudin

► To cite this version:

Patrick L Combettes, Lilian E. Glaudin. Quasi-Nonexpansive Iterations on the Affine Hull of Orbits: From Mann's Mean Value Algorithm to Inertial Methods. SIAM Journal on Optimization, 2017, 27 (4), 10.1137/17M112806X. hal-01678600

HAL Id: hal-01678600

https://hal.sorbonne-universite.fr/hal-01678600

Submitted on 9 Jan 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
QUASI-NONEXPANSIVE ITERATIONS ON THE AFFINE HULL OF
ORBITS: FROM MANN'S MEAN VALUE ALGORITHM TO
INERTIAL METHODS*

PATRICK L. COMBETTES† AND LILIAN E. GLAUDIN‡

*Dedicated to the memory of Felipe Álvarez 1972–2017*

**Abstract.** Fixed point iterations play a central role in the design and the analysis of a large number of optimization algorithms. We study a new iterative scheme in which the update is obtained by applying a composition of quasi-nonexpansive operators to a point in the affine hull of the orbit generated up to the current iterate. This investigation unifies several algorithmic constructs, including Mann's mean value method, inertial methods, and multilayer memoryless methods. It also provides a framework for the development of new algorithms, such as those we propose for solving monotone inclusion and minimization problems.

**Key words.** averaged operator, fixed point iteration, forward-backward algorithm, inertial algorithm, mean value iterations, monotone operator splitting, nonsmooth minimization, Peaceman-Rachford algorithm, proximal algorithm

**AMS subject classifications.** Primary, 65J15, 47H09; Secondary, 47H05, 65K05, 90C25

**DOI.** 10.1137/17M112806X

**1. Introduction.** Algorithms arising in various branches of optimization can be efficiently modeled and analyzed as fixed point iterations in a real Hilbert space $\mathcal{H}$; see, e.g., [9, 10, 13, 16, 18, 19, 22, 26, 43]. Our paper unifies three important algorithmic fixed point frameworks that coexist in the literature: mean value methods, inertial methods, and multilayer memoryless methods.

Let $T: \mathcal{H} \to \mathcal{H}$ be an operator with fixed point set $\text{Fix } T$. In 1953, inspired by classical results on the summation of divergent series [11, 29, 44], Mann [34] proposed to extend the standard successive approximation scheme

$$ (1.1) \qquad x_0 \in \mathcal{H} \quad \text{and} \quad (\forall n \in \mathbb{N}) \quad x_{n+1} = Tx_n $$

to the mean value algorithm

$$ (1.2) \qquad x_0 \in \mathcal{H} \quad \text{and} \quad (\forall n \in \mathbb{N}) \quad x_{n+1} = T\bar{x}_n, \quad \text{where } \bar{x}_n \in \text{conv}(x_j)_{0 \le j \le n}. $$

In other words, the operator $T$ is not applied to the most current iterate as in the memoryless (single step) process (1.1), but to a point in the convex hull of the orbit $(x_j)_{0 \le j \le n}$ generated so far. His motivation was that, although the sequence generated by (1.1) may fail to converge to a fixed point of $T$, that generated by (1.2) can, under suitable conditions. This work was followed by interesting developments and analyses of such mean value iterations (e.g., [8, 12, 15, 28, 30, 32, 35, 37, 42]), especially in the case when $T$ is nonexpansive (1-Lipschitzian) or merely quasi-nonexpansive, that is (this notion was essentially introduced in [27]),

$$ (1.3) \qquad (\forall x \in \mathcal{H})(\forall y \in \text{Fix } T) \quad \|Tx - y\| \le \|x - y\|. $$

<footnote>*Received by the editors May 1, 2017; accepted for publication (in revised form) August 7, 2017; published electronically November 16, 2017.</footnote>

<footnote>http://www.siam.org/journals/siopt/27-4/M112806.html</footnote>

<footnote>†Corresponding author. North Carolina State University, Department of Mathematics, Raleigh, NC 27695-8205 (plc@math.ncsu.edu).</footnote>

<footnote>‡Sorbonne Universités – UPMC Univ. Paris 06, UMR 7598, Laboratoire Jacques-Louis Lions, F-75005, Paris, France (glaudin@ljll.math.upmc.fr).</footnote>
In [21], the asymptotic behavior of the mean value process

(1.4)

$x_0 \in \mathcal{H}$ and $(\forall n \in \mathbb{N}) \ x_{n+1} = \bar{x}_n + \lambda_n (T_n \bar{x}_n + e_n - \bar{x}_n)$, where $\bar{x}_n \in \text{conv}(x_j)_{0 \le j \le n}$,

was investigated under general conditions on the construction of the averaging process $(\bar{x}_n)_{n \in \mathbb{N}}$ and the assumptions that, for every $n \in \mathbb{N}$, $e_n \in \mathcal{H}$ models a possible error made in the computation of $T_n \bar{x}_n$, $\lambda_n \in ]0, 2[$, and $T_n: \mathcal{H} \to \mathcal{H}$ is firmly quasi-nonexpansive, i.e., $2T_n - \text{Id}$ is quasi-nonexpansive or, equivalently [10],

$$ (1.5) \qquad (\forall x \in \mathcal{H})(\forall y \in \text{Fix } T_n) \quad \langle y - T_n x \mid x - T_n x \rangle \le 0. $$

The idea of using the past of the orbit generated by an algorithm can also be found in the work of Polyak [39, 41], who drew inspiration from classical multistep methods in numerical analysis. His motivation was to improve the speed of convergence over memoryless methods. For instance, the classical gradient method [38] for minimizing a smooth convex function $f: \mathcal{H} \to \mathbb{R}$ is an explicit discretization of the continuous-time process $-\dot{x}(t) = \nabla f(x(t))$. Polyak [39] proposed to consider instead the process $-\ddot{x}(t) - \beta \dot{x}(t) = \nabla f(x(t))$, where $\beta \in ]0, +\infty[$, and studied the algorithm resulting from its explicit discretization. He observed that, from a mechanical viewpoint, the term $\ddot{x}(t)$ can be interpreted as an inertial component. More generally, for a proper lower semicontinuous convex function $f: \mathcal{H} \to ]-\infty, +\infty[$, Álvarez investigated in [1] an implicit discretization of the inertial differential inclusion $-\ddot{x}(t) - \beta \dot{x}(t) \in \partial f(x(t))$, namely,

$$ (1.6) \quad (\forall n \in \mathbb{N}) \quad x_{n+1} = \text{prox}_{\gamma_n f} \bar{x}_n + e_n, \quad \text{where} \quad \begin{cases} \bar{x}_n = (1 + \eta_n)x_n - \eta_n x_{n-1}, \\ \eta_n \in ]0, 1[, \\ \gamma_n \in ]0, +\infty[. \end{cases} $$

and where $\text{prox}_f$ is the proximity operator of $f$ [10, 36]. The inertial proximal point algorithm (1.6) has been extended in various directions; e.g., [3, 14, 17]; see also [5] for further motivation in the context of nonconvex minimization problems.

Working from a different perspective, a structured extension of (1.1) involving the composition of $m$ averaged nonexpansive operators was proposed in [19]. This $m$-layer algorithm is governed by the memoryless recursion

$$ (1.7) \quad (\forall n \in \mathbb{N}) \quad x_{n+1} = x_n + \lambda_n (T_{1,n} \cdots T_{m,n} x_n + e_n - x_n), \quad \text{where } \lambda_n \in ]0, 1]. $$

Recall that a nonexpansive operator $T: \mathcal{H} \to \mathcal{H}$ is averaged with constant $\alpha \in ]0, 1[$ if there exists a nonexpansive operator $R: \mathcal{H} \to \mathcal{H}$ such that $T = (1 - \alpha) \text{Id} + \alpha R$ [7, 10]. The multilayer iteration process (1.7) was shown in [19] to provide a synthetic analysis of various algorithms, in particular in the area of monotone operator splitting methods. It was extended in [25] to an overrelaxed method, i.e., one with parameters $(\lambda_n)_{n \in \mathbb{N}}$ possibly larger than 1.

In the literature, the asymptotic analysis of the above methods has been carried out independently because of their apparent lack of common structure. In the present paper, we exhibit a structure that unifies (1.1), (1.2), (1.4), (1.6), and (1.7) in a single algorithm of the form

$$ (1.8) \quad x_0 \in \mathcal{H} \quad \text{and} \quad (\forall n \in \mathbb{N}) \quad x_{n+1} = \bar{x}_n + \lambda_n (T_{1,n} \cdots T_{m,n} \bar{x}_n + e_n - \bar{x}_n), \\ \text{where } \bar{x}_n \in \text{aff}(x_j)_{0 \le j \le n} \quad \text{and} \quad \lambda_n \in ]0, +\infty[ $$

under the assumption that each operator $T_{i,n}$ is $\alpha_{i,n}$-averaged quasi-nonexpansive, i.e.,

$$ (1.9) \quad (\forall x \in \mathcal{H})(\forall y \in \text{Fix } T_{i,n}) $$

$$ 2(1 - \alpha_{i,n}) \langle y - T_{i,n}x \mid x - T_{i,n}x \rangle \le (2\alpha_{i,n} - 1)(\|x - y\|^2 - \|T_{i,n}x - y\|^2) $$
for some $\alpha_{i,n} \in ]0,1]$, which means that the operator $(1 - 1/\alpha_{i,n}) \text{Id} + (1/\alpha_{i,n}) T_{i,n}$ is quasi-nonexpansive. In words, at iteration $n$, a point $\bar{x}_n$ is picked in the affine hull of the orbit $(x_j)_{0 \le j \le n}$ generated so far, a composition of quasi-nonexpansive operators is applied to it, up to some error $e_n$, and the update $x_{n+1}$ is obtained via a relaxation with parameter $\lambda_n$. Note that (1.8)-(1.9) not only brings together mean value iterations, inertial methods, and the memoryless multilayer setting of [19, 25], but also provides a flexible framework to design new iterative methods.

The fixed point problem under consideration will be the following (note that we allow 1 as an averaging constant for added flexibility).

**PROBLEM 1.1.** Let $m$ be a strictly positive integer. For every $n \in \mathbb{N}$ and every $i \in \{1, \dots, m\}$, $\alpha_{i,n} \in ]0,1]$ and $T_{i,n}: \mathcal{H} \to \mathcal{H}$ is $\alpha_{i,n}$-averaged nonexpansive if $i < m$, and $\alpha_{m,n}$-averaged quasi-nonexpansive if $i = m$. In addition,

$$ (1.10) \qquad S = \bigcap_{n \in \mathbb{N}} \text{Fix } T_n \neq \emptyset, \quad \text{where } (\forall n \in \mathbb{N}) \quad T_n = T_{1,n} \cdots T_{m,n}, $$

and one of the following holds:

(a) For every $n \in \mathbb{N}$, $T_{m,n}$ is $\alpha_{m,n}$-averaged nonexpansive.

(b) $m > 1$, and, for every $n \in \mathbb{N}$, $\alpha_{m,n} < 1$ and $\bigcap_{i=1}^m \text{Fix } T_{i,n} \neq \emptyset$.

(c) $m = 1$.

The problem is to find a point in $S$.

To solve Problem 1.1, we are going to employ (1.8), which we now formulate more formally.

**Algorithm 1.2.**

Consider the setting of Problem 1.1. For every $n \in \mathbb{N}$, let $\phi_n$ be an averaging constant of $T_n$, let $\lambda_n \in ]0, 1/\phi_n]$ and for every $i \in \{1, \dots, m\}$, let $e_{i,n} \in \mathcal{H}$. Let $(\mu_{n,j})_{n \in \mathbb{N}, 0 \le j \le n}$ be a real array which satisfies the following:

(a) $\sup_{n \in \mathbb{N}} \sum_{j=0}^n |\mu_{n,j}| < +\infty$.

(b) $(\forall n \in \mathbb{N}) \sum_{j=0}^n \mu_{n,j} = 1$.

(c) $(\forall j \in \mathbb{N}) \lim_{n \to +\infty} \mu_{n,j} = 0$.

(d) There exists a sequence $(\chi_n)_{n \in \mathbb{N}} \in ]0, +\infty[$ such that $\inf_{n \in \mathbb{N}} \chi_n > 0$ and every sequence $(\xi_n)_{n \in \mathbb{N}} \in ]0, +\infty[$ that satisfies

$$ (1.11) \quad \left( \exists (\varepsilon_n)_{n \in \mathbb{N}} \in [0, +\infty[ \right) \quad \begin{cases} \sum_{n \in \mathbb{N}} \chi_n \varepsilon_n < +\infty, \\ (\forall n \in \mathbb{N}) \quad \xi_{n+1} \le \sum_{j=0}^n \mu_{n,j} \xi_j + \varepsilon_n \end{cases} $$

converges.

Let $x_0 \in \mathcal{H}$ and set
(1.12)

for $n = 0, 1, \dots$

$$ \left. \begin{aligned} \bar{x}_n &= \sum_{j=0}^{n} \mu_{n,j} x_j, \\ x_{n+1} &= \bar{x}_n + \lambda_n \left( T_{1,n} \left( T_{2,n} \left( \cdots T_{m-1,n} (T_{m,n} \bar{x}_n + e_{m,n}) + e_{m-1,n} \cdots \right) \right) + e_{2,n} \right) \\ & \qquad + e_{1,n} - \bar{x}_n. \end{aligned} \right\} $$
Remark 1.3. Here are some comments about the parameters appearing in Problem 1.1 and Algorithm 1.2.

(i) The composite operator $T_n$ of (1.10) is averaged quasi-nonexpansive with constant

$$ (1.13) \qquad \phi_n = \begin{cases} \left(1 + \left(\sum_{i=1}^{m} \frac{\alpha_{i,n}}{1-\alpha_{i,n}}\right)^{-1}\right)^{-1} & \text{if } \max_{1 \le i \le m} \alpha_{i,n} < 1, \\ 1 & \text{otherwise.} \end{cases} $$

The proof is given in [25, Proposition 2.5] for case (a) of Problem 1.1. It easily extends to case (b), while case (c) is trivial.

(ii) Examples of arrays $(\mu_{n,j})_{n \in \mathbb{N}, 0 \le j \le n}$ that satisfy conditions (a)-(d) in Algorithm 1.2 are provided in [21, section 2] in the case of mean value iterations, i.e., $\inf_{n \in \mathbb{N}} \min_{0 \le j \le n} \mu_{n,j} \ge 0$ with $\chi_n = 1$. An important instance with negative coefficients will be presented in Example 2.5.

(iii) The term $e_{i,n}$ in (1.12) models a possible numerical error in the implementation of the operator $T_{i,n}$.

The material is organized as follows. In section 2 we provide preliminary results. The main results on the convergence of the orbits of Algorithm 1.2 are presented in section 3. Section 4 is dedicated to new algorithms for fixed point computation, monotone operator splitting, and nonsmooth minimization based on the proposed framework.

**Notation.** $\mathcal{H}$ is a real Hilbert space with scalar product $\langle \cdot | \cdot \rangle$ and associated norm $\| \cdot \|$. We denote by $\mathrm{Id}$ the identity operator on $\mathcal{H}$; $\to$ and $\to$ denote, respectively, weak and strong convergence in $\mathcal{H}$. The positive and negative parts of $\xi \in \mathbb{R}$ are, respectively, $\xi^+ = \max\{0, \xi\}$ and $\xi^- = -\min\{0, \xi\}$. Finally, $\delta_{n,j}$ is the Kronecker delta: it takes on the value 1 if $n = j$, and 0 otherwise.

**2. Preliminary results.** In this section we establish some technical facts that will be used subsequently. We start with a Grönwall-type result.

**LEMMA 2.1.** Let $(\theta_n)_{n \in \mathbb{N}}$ and $(\varepsilon_n)_{n \in \mathbb{N}}$ be sequences in $[0, +\infty[$, and let $(\nu_n)_{n \in \mathbb{N}}$ be a sequence in $\mathbb{R}$ such that $(\forall n \in \mathbb{N}) \theta_{n+1} \le (1 + \nu_n)\theta_n + \varepsilon_n$. Then

$$ (2.1) \qquad (\forall n \in \mathbb{N}) \quad \theta_{n+1} \le \theta_0 \exp \left( \sum_{k=0}^{n} \nu_k \right) + \sum_{j=0}^{n-1} \varepsilon_j \exp \left( \sum_{k=j+1}^{n} \nu_k \right) + \varepsilon_n. $$

**Proof.** We have $(\forall n \in \mathbb{N}) 1 + \nu_n \le \exp(\nu_n)$. Therefore $\theta_1 \le \theta_0 \exp(\nu_0) + \varepsilon_0$ and

$$ \begin{align} (\forall n \in \mathbb{N} \setminus \{0\}) \quad \theta_{n+1} &\le \theta_n \exp(\nu_n) + \varepsilon_n \nonumber \\ &\le \theta_{n-1} \exp(\nu_n) \exp(\nu_{n-1}) + \varepsilon_{n-1} \exp(\nu_n) + \varepsilon_n \nonumber \\ &\le \theta_0 \prod_{k=0}^{n} \exp(\nu_k) + \sum_{j=0}^{n-1} \varepsilon_j \prod_{k=j+1}^{n} \exp(\nu_k) + \varepsilon_n \nonumber \\ (2.2) \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad = \theta_0 \exp \left( \sum_{k=0}^{n} \nu_k \right) + \sum_{j=0}^{n-1} \varepsilon_j \exp \left( \sum_{k=j+1}^{n} \nu_k \right) + \varepsilon_n, \end{align} $$

as claimed.

<figure><img src="image_2.png" /></figure>
LEMMA 2.2 (see [31, Theorem 43.5]). Let $(\xi_n)_{n \in \mathbb{N}}$ be a sequence in $\mathbb{R}$, let $\xi \in \mathbb{R}$, suppose that $(\mu_{n,j})_{n \in \mathbb{N}, 0 \le j \le n}$ is a real array that satisfies conditions (a)-(c) in Algorithm 1.2. Then $\xi_n \to \xi \Rightarrow \sum_{j=0}^n \mu_{n,j} \xi_j \to \xi$.

LEMMA 2.3. Let $(\beta_n)_{n \in \mathbb{N}}$, $(\gamma_n)_{n \in \mathbb{N}}$, $(\delta_n)_{n \in \mathbb{N}}$, $(\eta_n)_{n \in \mathbb{N}}$, and $(\lambda_n)_{n \in \mathbb{N}}$ be sequences in $[0, +\infty[$, let $(\phi_n)_{n \in \mathbb{N}}$ be a sequence in $[0, 1]$, let $(\vartheta, \sigma) \in ]0, +\infty]^2$, and let $\eta \in ]0, 1[$. Set $\beta_{-1} = \beta_0$ and

$$ (2.3) \qquad (\forall n \in \mathbb{N}) \quad \omega_n = \frac{1}{\phi_n} - \lambda_n, $$

and suppose that the following hold:

(a) $(\forall n \in \mathbb{N}) \quad \eta_n \le \eta_{n+1} \le \eta$.

(b) $(\forall n \in \mathbb{N}) \quad \gamma_n \le \eta(1 + \eta) + \eta\vartheta\omega_n$.

(c) $(\forall n \in \mathbb{N}) \quad \frac{\eta^2(1 + \eta) + \eta\sigma}{\vartheta} < \frac{1}{\phi_n} - \eta^2\omega_{n+1}$.

(d) $(\forall n \in \mathbb{N}) \quad 0 < \lambda_n \le \frac{\vartheta/\phi_n - \eta(\eta(1 + \eta) + \eta\vartheta\omega_{n+1} + \sigma)}{\vartheta(1 + \eta(1 + \eta) + \eta\vartheta\omega_{n+1} + \sigma)}$.

(e) $(\forall n \in \mathbb{N}) \quad \beta_{n+1} - \beta_n - \eta_n(\beta_n - \beta_{n-1}) \le \frac{(1/\phi_n - \lambda_n)(\eta_n/(\eta_n + \vartheta\lambda_n) - 1)}{\lambda_n} \delta_{n+1} + \gamma_n\delta_n$.

Then $\sum_{n \in \mathbb{N}} \delta_n < +\infty$.

*Proof.* We use arguments similar to those used in [3, 14]. It follows from (c) that $(\forall n \in \mathbb{N}) 0 < \vartheta/\phi_n - \eta^2\omega_{n+1}\vartheta - \eta^2(1 + \eta) - \eta\sigma$. This shows that $(\lambda_n)_{n \in \mathbb{N}}$ is well defined. Now set $(\forall n \in \mathbb{N}) \rho_n = 1/(\eta_n + \vartheta\lambda_n)$ and $\kappa_n = \beta_n - \eta_n\beta_{n-1} + \gamma_n\delta_n$. We derive from (a) and (e) that

$$ (2.4) \qquad (\forall n \in \mathbb{N}) \quad \begin{aligned} \kappa_{n+1} - \kappa_n &\le \beta_{n+1} - \eta_n\beta_n - \beta_n + \eta_n\beta_{n-1} + \gamma_{n+1}\delta_{n+1} - \gamma_n\delta_n \\ &\le \left( \frac{(1/\phi_n - \lambda_n)(\eta_n\rho_n - 1)}{\lambda_n} + \gamma_{n+1} \right) \delta_{n+1}. \end{aligned} $$

On the other hand, $(\forall n \in \mathbb{N}) \vartheta(1 + (\eta(1 + \eta) + \eta\vartheta\omega_{n+1} + \sigma)) > 0$. Consequently, (d) can be written as

$$ (2.5) \quad (\forall n \in \mathbb{N}) \quad \vartheta\lambda_n + \vartheta\lambda_n(\eta(1+\eta) + \eta\vartheta\omega_{n+1} + \sigma) \le \frac{\vartheta}{\phi_n} - \eta(\eta(1+\eta) + \eta\vartheta\omega_{n+1} + \sigma). $$

Using (a) and (b), and then (2.5), we get

$$ (2.6) \qquad (\forall n \in \mathbb{N}) \quad (\eta_n + \vartheta\lambda_n)(\gamma_{n+1} + \sigma) + \vartheta\lambda_n \le (\eta + \vartheta\lambda_n)(\eta(1+\eta) + \eta\vartheta\omega_{n+1} + \sigma) + \vartheta\lambda_n \le \frac{\vartheta}{\phi_n}. $$

However,

$$ (2.7) \qquad (\forall n \in \mathbb{N}) \quad \begin{aligned} & (\eta_n + \vartheta\lambda_n)(\gamma_{n+1} + \sigma) + \vartheta\lambda_n \le \frac{\vartheta}{\phi_n} \\ & \Leftrightarrow (\eta_n + \vartheta\lambda_n)(\gamma_{n+1} + \sigma) - (1/\phi_n - \lambda_n)\vartheta \le 0 \\ & \Leftrightarrow (1/\phi_n - \lambda_n)\left(\frac{-\vartheta}{\eta_n + \vartheta\lambda_n}\right) \le -(\gamma_{n+1} + \sigma) \\ & \Leftrightarrow \frac{(1/\phi_n - \lambda_n)(\eta_n\rho_n - 1)}{\lambda_n} + \gamma_{n+1} \le -\sigma. \end{aligned} $$
It therefore follows from (2.4) and (2.6) that

$$ (2.8) \qquad (\forall n \in \mathbb{N}) \quad \kappa_{n+1} - \kappa_n \le -\sigma\delta_{n+1}. $$

Thus, $(\kappa_n)_{n \in \mathbb{N}}$ is decreasing and

$$ (2.9) \qquad (\forall n \in \mathbb{N}) \quad \beta_n - \eta\beta_{n-1} = \kappa_n - \gamma_n\delta_n \le \kappa_n \le \kappa_0 $$

from which we infer that $(\forall n \in \mathbb{N}) \beta_n \le \kappa_0 + \eta\beta_{n-1}$. In turn,

$$ (2.10) \qquad (\forall n \in \mathbb{N} \setminus \{0\}) \quad \beta_n \le \eta^n \beta_0 + \kappa_0 \sum_{j=0}^{n-1} \eta^j \le \eta^n \beta_0 + \frac{\kappa_0}{1-\eta}. $$

Altogether, we derive from (2.8), (2.9), and (2.10) that

$$ (2.11) \qquad (\forall n \in \mathbb{N}) \quad \sigma \sum_{j=0}^{n} \delta_{j+1} \le \kappa_0 - \kappa_{n+1} \le \kappa_0 + \eta\beta_n \le \frac{\kappa_0}{1-\eta} + \eta^{n+1}\beta_0. $$

Hence, $\sum_{j \ge 1} \delta_j \le \kappa_0 / ((1-\eta)\sigma) < +\infty$, and the proof is complete.

LEMMA 2.4. Let $(\eta_n)_{n \in \mathbb{N}}$ be a sequence in $[0, 1[$. For every $n \in \mathbb{N}$, set

$$ (2.12) \qquad (\forall k \in \mathbb{N}) \quad \zeta_{k,n} = \begin{cases} 0 & \text{if } k \le n, \\ \displaystyle \sum_{j=n+1}^{k} (\eta_j - 1) & \text{if } k > n, \end{cases} $$

and $\chi_n = \sum_{k \ge n} \exp(\zeta_{k,n})$. Then the following hold:

(i) Let $\tau \in [2, +\infty[$ and suppose that $(\forall n \in \mathbb{N}) \eta_{n+1} = n/(n+1+\tau)$. Then $(\forall n \in \mathbb{N}) \chi_n \le (n+7)/2$.

(ii) Suppose that $(\exists \eta \in [0, 1]) (\forall n \in \mathbb{N}) \eta_n \le \eta$. Then $(\forall n \in \mathbb{N}) \chi_n \le e/(1-\eta)$.

Proof. (i): We have $(\forall n \in \mathbb{N})(\forall k \in \{n+1, n+2, \dots\}) \zeta_{k,n} = -(1+\tau) \sum_{j=n+1}^{k} 1/(j+\tau) \le -3 \sum_{j=n+1}^{k} 1/(j+2)$. Since $\xi \mapsto 1/(\xi+2)$ is decreasing on $[1, +\infty[$, it follows that

$$ (2.13) \qquad (\forall n \in \mathbb{N})(\forall k \in \{n+1, n+2, \dots\}) \quad \zeta_{k,n} \le -3 \int_{n+1}^{k+1} \frac{d\xi}{\xi+2} = \ln \frac{(n+3)^3}{(k+3)^3}. $$

Furthermore, since $\xi \mapsto 1/(\xi+3)^3$ is decreasing on $]-1, +\infty[$, (2.12) yields (2.14)

$$ (\forall n \in \mathbb{N}) \quad \chi_n \le \sum_{k \ge n} \frac{(n+3)^3}{(k+3)^3} \le (n+3)^3 \int_{n-1}^{+\infty} \frac{d\xi}{(\xi+3)^3} = \frac{(n+3)^3}{2(n+2)^2} \le \frac{n+7}{2}. $$

(ii): Note that

$$ (2.15) \qquad (\forall n \in \mathbb{N})(\forall k \in \{n+1, n+2, \dots\}) \quad \zeta_{k,n} = \sum_{j=n+1}^{k} (\eta_j - 1) \le \sum_{j=n+1}^{k} (\eta - 1) = (\eta - 1)(k-n). $$

Since $\xi \mapsto \exp((\eta-1)\xi)$ is decreasing on $]-1, +\infty[$, it follows that (2.16)

$$ (\forall n \in \mathbb{N}) \quad \chi_n \le \sum_{k \ge n} \exp((\eta-1)(k-n)) \le \int_{n-1}^{+\infty} \exp((\eta-1)(\xi-n)) d\xi = \frac{\exp(1-\eta)}{1-\eta}, $$

which proves the assertion.
The next example provides an instance of an array $(\mu_{n,j})_{n \in \mathbb{N}, 0 \le j \le n}$ satisfying the conditions of Algorithm 1.2 with negative entries. This example will be central to the study of the convergence of some inertial methods.

*Example 2.5.* Let $(\mu_{n,j})_{n \in \mathbb{N}, 0 \le j \le n}$ be a real array such that $\mu_{0,0} = 1$ and (2.17)

$$ (\forall n \in \mathbb{N}) \quad 1 \le \mu_{n,n} < 2 \quad \text{and} \quad (\forall j \in \{0, \dots, n\}) \quad \mu_{n,j} = \begin{cases} 1 - \mu_{n,n} & \text{if } j = n-1, \\ 0 & \text{if } j < n-1. \end{cases} $$

For every $n \in \mathbb{N}$, set

$$ (2.18) \qquad (\forall k \in \mathbb{N}) \quad \zeta_{k,n} = \begin{cases} 0 & \text{if } k \le n, \\ \displaystyle \sum_{j=n+1}^{k} (\mu_{j,j} - 2) & \text{if } k > n, \end{cases} $$

and suppose that $\chi_n = \sum_{k \ge n} \exp(\zeta_{k,n}) < +\infty$. Then $(\mu_{n,j})_{n \in \mathbb{N}, 0 \le j \le n}$ satisfies conditions (a)-(d) in Algorithm 1.2.

*Proof.*

(a): $(\forall n \in \mathbb{N}) \sum_{j=0}^{n} |\mu_{n,j}| = \mu_{n,n} + |1 - \mu_{n,n}| \le 3$.

(b): $(\forall n \in \mathbb{N}) \sum_{j=0}^{n} \mu_{n,j} = (1 - \mu_{n,n}) + \mu_{n,n} = 1$.

(c): Let $j \in \mathbb{N}$. Then $(\forall n \in \mathbb{N}) n > j + 1 \Rightarrow \mu_{n,j} = 0$. Hence, $\lim_{n \to +\infty} \mu_{n,j} = 0$.

(d): We have $(\forall n \in \mathbb{N}) \chi_n = \sum_{k \ge n} \exp(\zeta_{k,n}) \ge \exp(\zeta_{n,n}) = 1$. Now suppose that $(\xi_n)_{n \in \mathbb{N}}$ is a sequence in $[0, +\infty[$ such that there exists a sequence $(\varepsilon_n)_{n \in \mathbb{N}}$ in $[0, +\infty[$ that satisfies

$$ (2.19) \qquad \sum_{n \in \mathbb{N}} \chi_n \varepsilon_n < +\infty \quad \text{and} \quad (\forall n \in \mathbb{N}) \quad \xi_{n+1} \le \sum_{j=0}^{n} \mu_{n,j} \xi_j + \varepsilon_n. $$

Set $\theta_0 = 0$ and $(\forall n \in \mathbb{N}) \theta_{n+1} = [\xi_{n+1} - \xi_n]^+$ and $\nu_n = \mu_{n,n} - 2$. It results from (2.19), (2.17), and the inequalities $\xi_1 - \xi_0 \le (\mu_{0,0} - 1)\xi_0 + \varepsilon_0$ and

$$ (2.20) \qquad \begin{aligned} (\forall n \in \mathbb{N} \setminus \{0\}) \quad & \xi_{n+1} - \xi_n \le (\mu_{n,n} - 1)\xi_n + (1 - \mu_{n,n})\xi_{n-1} + \varepsilon_n \\ & = (\mu_{n,n} - 1)(\xi_n - \xi_{n-1}) + \varepsilon_n, \end{aligned} $$

that $(\forall n \in \mathbb{N}) \theta_{n+1} \le (\mu_{n,n} - 1)\theta_n + \varepsilon_n = (1 + \nu_n)\theta_n + \varepsilon_n$. Consequently, we derive from Lemma 2.1 and (2.18) that $(\forall n \in \mathbb{N}) \theta_{n+1} \le \sum_{k=0}^{n} \varepsilon_k \exp(\zeta_{n,k})$. Using [31, Theorem 141], this yields

$$ (2.21) \qquad \sum_{n \in \mathbb{N}} \theta_{n+1} \le \sum_{n \in \mathbb{N}} \sum_{k=0}^{n} \varepsilon_k \exp(\zeta_{n,k}) = \sum_{k \in \mathbb{N}} \varepsilon_k \sum_{n \ge k} \exp(\zeta_{n,k}) = \sum_{k \in \mathbb{N}} \varepsilon_k \chi_k. $$

Now set $(\forall n \in \mathbb{N}) \omega_n = \xi_n - \sum_{k=0}^{n} \theta_k$. Since $\sum_{k \in \mathbb{N}} \chi_k \varepsilon_k < +\infty$, we infer from (2.21) that $\sum_{n \in \mathbb{N}} \theta_n < +\infty$. Thus, since $\inf_{n \in \mathbb{N}} \xi_n \ge 0$, $(\omega_n)_{n \in \mathbb{N}}$ is bounded below and

$$ (2.22) \quad (\forall n \in \mathbb{N}) \quad \omega_{n+1} = \xi_{n+1} - \theta_{n+1} - \sum_{k=0}^{n} \theta_k \le \xi_{n+1} - \xi_{n+1} + \xi_n - \sum_{k=0}^{n} \theta_k = \omega_n. $$

Altogether, $(\omega_n)_{n \in \mathbb{N}}$ converges, and so does therefore $(\xi_n)_{n \in \mathbb{N}}$.

<figure><img src="image_3.png" /></figure>
**3. Asymptotic behavior of Algorithm 1.2.** The main result of the paper is the following theorem, which analyzes the asymptotic behavior of Algorithm 1.2.

THEOREM 3.1. *Consider the setting of Algorithm 1.2. For every $n \in \mathbb{N}$, define* (3.1)

$$ \vartheta_n = \lambda_n \sum_{i=1}^{m} \|e_{i,n}\| \quad \text{and} \quad (\forall i \in \{1, \dots, m\}) \quad T_{i+,n} = \begin{cases} T_{i+1,n} \cdots T_{m,n} & \text{if } i \neq m, \\ \text{Id} & \text{if } i = m, \end{cases} $$

and set

$$ (3.2) \qquad \nu_n: S \to [0, +\infty[ : x \mapsto \vartheta_n(2\|\bar{x}_n - x\| + \vartheta_n). $$

Then the following hold:

(i) Let $n \in \mathbb{N}$ and $x \in S$. Then $\|x_{n+1} - x\| \le \sum_{j=0}^{n} |\mu_{n,j}| \|x_j - x\| + \vartheta_n$.

(ii) Let $n \in \mathbb{N}$ and $x \in S$. Then

$$ \|x_{n+1} - x\|^2 \le \sum_{j=0}^{n} \mu_{n,j} \|x_j - x\|^2 - \frac{1}{2} \sum_{j=0}^{n} \sum_{k=0}^{n} \mu_{n,j} \mu_{n,k} \|x_j - x_k\|^2 \\ - \lambda_n (1/\phi_n - \lambda_n) \|T_n \bar{x}_n - \bar{x}_n\|^2 + \nu_n(x). $$

(iii) Let $n \in \mathbb{N}$ and $x \in S$. Then

$$ \|x_{n+1} - x\|^2 \\ \le \sum_{j=0}^{n} \mu_{n,j} \|x_j - x\|^2 - \frac{1}{2} \sum_{j=0}^{n} \sum_{k=0}^{n} \mu_{n,j} \mu_{n,k} \|x_j - x_k\|^2 \\ + \lambda_n (\lambda_n - 1) \|T_n \bar{x}_n - \bar{x}_n\|^2 \\ - \lambda_n \max_{1 \le i \le m} \left( \frac{1 - \alpha_{i,n}}{\alpha_{i,n}} \|(\text{Id} - T_{i,n}) T_{i+,n} \bar{x}_n - (\text{Id} - T_{i,n}) T_{i+,n} x\|^2 \right) + \nu_n(x). $$

Now assume that, in addition,

$$ (3.3) \qquad \sum_{n \in \mathbb{N}} \chi_n \sum_{j=0}^{n} \sum_{k=0}^{n} [\mu_{n,j} \mu_{n,k}]^{-} \|x_j - x_k\|^2 < +\infty \quad \text{and} \quad (\forall x \in S) \quad \sum_{n \in \mathbb{N}} \chi_n \nu_n(x) < +\infty. $$

Then the following hold:

(iv) Let $x \in S$. Then $(\|x_n - x\|)_{n \in \mathbb{N}}$ converges.

(v) $\lambda_n(1/\phi_n - \lambda_n)\|T_n \bar{x}_n - \bar{x}_n\|^2 \to 0$.

(vi) $\sum_{j=0}^{n} \sum_{k=0}^{n} [\mu_{n,j} \mu_{n,k}]^{+} \|x_j - x_k\|^2 \to 0$.

(vii) Suppose that

$$ (3.4) \qquad (\exists \varepsilon \in ]0, 1[)(\forall n \in \mathbb{N}) \quad \lambda_n \le (1 - \varepsilon)/\phi_n. $$

Then $x_{n+1} - \bar{x}_n \to 0$. In addition, if every weak sequential cluster point of $(\bar{x}_n)_{n \in \mathbb{N}}$ is in $S$, then there exists $x \in S$ such that $x_n \to x$.

(viii) Suppose that $(\bar{x}_n)_{n \in \mathbb{N}}$ has a strong cluster point $x$ in $S$ and that (3.4) holds. Then $x_n \to x$.

(ix) Let $x \in S$ and suppose that $(\exists \varepsilon \in ]0, 1[)(\forall n \in \mathbb{N}) \lambda_n \le \varepsilon + (1 - \varepsilon)/\phi_n$. Then

$$ \lambda_n \max_{1 \le i \le m} \frac{1 - \alpha_{i,n}}{\alpha_{i,n}} \|(\text{Id} - T_{i,n}) T_{i+,n} \bar{x}_n - (\text{Id} - T_{i,n}) T_{i+,n} x\|^2 \to 0. $$
Proof. Let $n \in \mathbb{N}$ and set

$$ (3.5) \quad e_n = T_{1,n} (T_{2,n} (\cdots T_{m-1,n} (T_{m,n} \bar{x}_n + e_{m,n}) + e_{m-1,n} \cdots) + e_{2,n}) + e_{1,n} - T_n \bar{x}_n. $$

If $m > 1$, using the nonexpansiveness of the operators $(T_{i,n})_{1 \le i \le m-1}$, we obtain
(3.6)

$$ \begin{aligned} \|e_n\| &\le \|e_{1,n}\| + \left\| T_{1,n} \left( T_{2,n} (\cdots T_{m-1,n} (T_{m,n} \bar{x}_n + e_{m,n}) + e_{m-1,n} \cdots) \right. \right. \\ & \qquad \left. \left. + e_{2,n} \right) - T_{1,n} \cdots T_{m,n} \bar{x}_n \right\| \\ &\le \|e_{1,n}\| + \left\| T_{2,n} \left( T_{3,n} (\cdots T_{m-1,n} (T_{m,n} \bar{x}_n + e_{m,n}) + e_{m-1,n} \cdots) + e_{3,n} \right) \right. \\ & \qquad \left. + e_{2,n} - T_{2,n} \cdots T_{m,n} \bar{x}_n \right\| \\ &\le \|e_{1,n}\| + \|e_{2,n}\| + \left\| T_{3,n} \left( T_{4,n} (\cdots T_{m-1,n} (T_{m,n} \bar{x}_n + e_{m,n}) + e_{m-1,n} \cdots) \right. \right. \\ & \qquad \left. \left. + e_{4,n} \right) + e_{3,n} - T_{3,n} \cdots T_{m,n} \bar{x}_n \right\| \\ &\vdots \\ &\le \sum_{i=1}^{m} \|e_{i,n}\|. \end{aligned} $$

Thus, we infer from (3.1) that

$$ (3.7) \qquad \lambda_n \|e_n\| \le \vartheta_n. $$

On the other hand, we derive from (1.12) and (3.5) that

$$ (3.8) \qquad x_{n+1} = \bar{x}_n + \lambda_n (T_n \bar{x}_n + e_n - \bar{x}_n). $$

Now set

$$ (3.9) \qquad R_n = \frac{1}{\phi_n} T_n - \frac{1 - \phi_n}{\phi_n} \mathrm{Id} \quad \text{and} \quad \eta_n = \lambda_n \phi_n. $$

Then $\eta_n \in ]0, 1]$, Fix $R_n = \text{Fix } T_n$, and $R_n$ is quasi-nonexpansive since $T_n$ is averaged quasi-nonexpansive with constant $\phi_n$ by Remark 1.3(i). Furthermore, (3.8) can be written as

$$ (3.10) \qquad x_{n+1} = \bar{x}_n + \eta_n (R_n \bar{x}_n - \bar{x}_n) + \lambda_n e_n. $$

Next, we define

$$ (3.11) \qquad z_n = \bar{x}_n + \lambda_n (T_n \bar{x}_n - \bar{x}_n) = \bar{x}_n + \eta_n (R_n \bar{x}_n - \bar{x}_n). $$

Let $x \in S$. Since $x \in \text{Fix } R_n$ and $R_n$ is quasi-nonexpansive, we have

$$ \begin{aligned} \|z_n - x\| &= \|(1 - \eta_n)(\bar{x}_n - x) + \eta_n(R_n \bar{x}_n - x)\| \\ &\le (1 - \eta_n)\|\bar{x}_n - x\| + \eta_n\|R_n \bar{x}_n - x\| \\ (3.12) \qquad &\le \|\bar{x}_n - x\|. \end{aligned} $$

Hence, (3.10) and (3.7) yield

$$ (3.13) \qquad \|x_{n+1} - x\| \le \|z_n - x\| + \lambda_n \|e_n\| \le \|z_n - x\| + \vartheta_n. $$
In turn, it follows from (3.12) and (3.2) that

$$ (3.14) \qquad \|x_{n+1} - x\|^2 \le \|z_n - x\|^2 + 2\vartheta_n \|z_n - x\| + \vartheta_n^2 \le \|z_n - x\|^2 + \nu_n(x). $$

In addition, [10, Lemma 2.14(ii)] yields

$$ (3.15) \qquad \|\bar{x}_n - x\|^2 = \left\| \sum_{j=0}^{n} \mu_{n,j}(x_j - x) \right\|^2 = \sum_{j=0}^{n} \mu_{n,j} \|x_j - x\|^2 - \frac{1}{2} \sum_{j=0}^{n} \sum_{k=0}^{n} \mu_{n,j} \mu_{n,k} \|x_j - x_k\|^2. $$

(i): By (3.13) and (3.12),

$$ (3.16) \quad (\forall n \in \mathbb{N})(\forall x \in S) \quad \|x_{n+1} - x\| \le \|\bar{x}_n - x\| + \vartheta_n \le \sum_{j=0}^{n} |\mu_{n,j}| \|x_j - x\| + \vartheta_n. $$

(ii): Let $n \in \mathbb{N}$ and $x \in S$. Since

$$ \begin{align} \|z_n - x\|^2 &= \|(1 - \eta_n)(\bar{x}_n - x) + \eta_n(R_n\bar{x}_n - x)\|^2 \nonumber \\ &= (1 - \eta_n)\|\bar{x}_n - x\|^2 + \eta_n\|R_n\bar{x}_n - x\|^2 - \eta_n(1 - \eta_n)\|R_n\bar{x}_n - \bar{x}_n\|^2 \tag{3.17} \\ &\le \|\bar{x}_n - x\|^2 - \eta_n(1 - \eta_n)\|R_n\bar{x}_n - \bar{x}_n\|^2, \nonumber \end{align} $$

we deduce from (3.14) and (3.9) that

$$ \begin{align} \|x_{n+1} - x\|^2 &\le \|z_n - x\|^2 + \nu_n(x) \nonumber \\ &\le \|\bar{x}_n - x\|^2 - \eta_n(1 - \eta_n)\|R_n\bar{x}_n - \bar{x}_n\|^2 + \nu_n(x) \tag{3.18} \\ &= \|\bar{x}_n - x\|^2 - \lambda_n(1/\phi_n - \lambda_n)\|T_n\bar{x}_n - \bar{x}_n\|^2 + \nu_n(x). \nonumber \end{align} $$

In view of (3.15), we obtain the announced inequality.

(iii): Let $n \in \mathbb{N}$ and $x \in S$. We derive from [10, Proposition 4.35] that

$$ (3.19) \qquad (\forall i \in \{1, \dots, m-1\}) \quad (\forall (u, v) \in \mathcal{H}^2) \qquad \|T_{i,n}u - T_{i,n}v\|^2 \le \|u-v\|^2 - \frac{1-\alpha_{i,n}}{\alpha_{i,n}} \|(\mathrm{Id} - T_{i,n})u - (\mathrm{Id} - T_{i,n})v\|^2. $$

If $m > 1$, using this inequality successively for $i = 1, \dots, m-1$ leads to

$$ \begin{align} \|T_n \bar{x}_n - x\|^2 &= \|T_{1,n} \cdots T_{m,n} \bar{x}_n - T_{1,n} \cdots T_{m,n} x\|^2 \nonumber \\ &\le \|T_{m,n} \bar{x}_n - T_{m,n} x\|^2 \nonumber \\ &\quad - \sum_{i=1}^{m-1} \frac{1-\alpha_{i,n}}{\alpha_{i,n}} \|(\mathrm{Id} - T_{i,n}) T_{i+1,n} \bar{x}_n - (\mathrm{Id} - T_{i,n}) T_{i+1,n} x\|^2 \nonumber \\ &\le \|T_{m,n} \bar{x}_n - T_{m,n} x\|^2 \tag{3.20} \\ &\quad - \max_{1 \le i \le m-1} \frac{1-\alpha_{i,n}}{\alpha_{i,n}} \|(\mathrm{Id} - T_{i,n}) T_{i+1,n} \bar{x}_n - (\mathrm{Id} - T_{i,n}) T_{i+1,n} x\|^2. \nonumber \end{align} $$

Note that, in cases (a) and (c) of Problem 1.1,

$$ (3.21) \qquad \|T_{m,n} \bar{x}_n - T_{m,n} x\|^2 \le \|\bar{x}_n - x\|^2 - \frac{1 - \alpha_{m,n}}{\alpha_{m,n}} \|(\mathrm{Id} - T_{m,n}) \bar{x}_n - (\mathrm{Id} - T_{m,n}) x\|^2. $$
This inequality remains valid in case (b) of Problem 1.1 since [10, Proposition 4.49(i)] implies that

$$ (3.22) \qquad \text{Fix} (T_{1,n} \cdots T_{m,n}) = \bigcap_{i=1}^{m} \text{Fix} T_{i,n} $$

and, therefore, that $x \in \text{Fix} T_{m,n}$. Altogether, we deduce from (3.20) and (3.21) that (3.23)

$$ \|T_n \bar{x}_n - x\|^2 \le \|\bar{x}_n - x\|^2 - \max_{1 \le i \le m} \frac{1 - \alpha_{i,n}}{\alpha_{i,n}} \|(\text{Id} - T_{i,n}) T_{i+,n} \bar{x}_n - (\text{Id} - T_{i,n}) T_{i+,n} x\|^2. $$

Hence, it follows from (3.11) that

$$ \begin{align} \|z_n - x\|^2 &= \|(1 - \lambda_n)(\bar{x}_n - x) + \lambda_n(T_n \bar{x}_n - x)\|^2 \nonumber \\ &= (1 - \lambda_n)\|\bar{x}_n - x\|^2 + \lambda_n\|T_n \bar{x}_n - x\|^2 + \lambda_n(\lambda_n - 1)\|T_n \bar{x}_n - \bar{x}_n\|^2 \nonumber \\ &\le \|\bar{x}_n - x\|^2 - \lambda_n \max_{1 \le i \le m} \frac{1 - \alpha_{i,n}}{\alpha_{i,n}} \|(\text{Id} - T_{i,n}) T_{i+,n} \bar{x}_n - (\text{Id} - T_{i,n}) T_{i+,n} x\|^2 \tag{3.24} \\ &\quad + \lambda_n(\lambda_n - 1)\|T_n \bar{x}_n - \bar{x}_n\|^2. \nonumber \end{align} $$

In view of (3.14) and (3.15), the inequality is established.

(iv): Let $x \in S$ and set

$$ (3.25) \quad (\forall n \in \mathbb{N}) \quad \begin{cases} \xi_n = \|x_n - x\|^2, \\ \varepsilon_n = \nu_n(x) + \frac{1}{2} \sum_{j=0}^{n} \sum_{k=0}^{n} [\mu_{n,j} \mu_{n,k}]^{-} \|x_j - x_k\|^2. \end{cases} $$

Since $\inf_{n \in \mathbb{N}} \lambda_n(1/\phi_n - \lambda_n) \ge 0$, (3.3) and (ii) imply that

$$ (3.26) \quad \sum_{n \in \mathbb{N}} \chi_n \varepsilon_n < +\infty \quad \text{and} \quad (\forall n \in \mathbb{N}) \quad \xi_{n+1} \le \sum_{j=0}^{n} \mu_{n,j} \xi_j + \varepsilon_n. $$

In turn, it follows from (3.3) and condition (d) in Algorithm 1.2 that $(\|x_n - x\|)_{n \in \mathbb{N}}$ converges.

(v)-(vi): Let $x \in S$. Then it follows from (iv) that $\rho = \lim_{n \to +\infty} \|x_n - x\|$ is well defined. Hence, Lemma 2.2 implies that $\sum_{j=0}^{n} \mu_{n,j} \|x_j - x\|^2 \to \rho^2$ and therefore that

$$ (3.27) \quad \sum_{j=0}^{n} \mu_{n,j} \|x_j - x\|^2 - \|x_{n+1} - x\|^2 \to 0. $$

Since $\inf_{n \in \mathbb{N}} \chi_n > 0$, (3.3) yields

$$ (3.28) \quad \nu_n(x) \to 0 \quad \text{and} \quad \sum_{j=0}^{n} \sum_{k=0}^{n} [\mu_{n,j} \mu_{n,k}]^{-} \|x_j - x_k\|^2 \to 0. $$

It follows from (ii), (3.27), and (3.28) that

$$ \begin{align} 0 &\le \lambda_n(1/\phi_n - \lambda_n)\|T_n \bar{x}_n - \bar{x}_n\|^2 + \frac{1}{2} \sum_{j=0}^{n} \sum_{k=0}^{n} [\mu_{n,j} \mu_{n,k}]^+ \|x_j - x_k\|^2 \nonumber \\ &\le \sum_{j=0}^{n} \mu_{n,j} \|x_j - x\|^2 - \|x_{n+1} - x\|^2 + \frac{1}{2} \sum_{j=0}^{n} \sum_{k=0}^{n} [\mu_{n,j} \mu_{n,k}]^- \|x_j - x_k\|^2 + \nu_n(x) \tag{3.29} \\ &\quad \to 0, \nonumber \end{align} $$

which gives the desired conclusions.
(vii): Set $\zeta = 1/\varepsilon - 1$. We deduce from (3.3) and (3.2) that $\sum_{n \in \mathbb{N}} \vartheta_n^2 < +\infty$. Hence, it follows from (3.8), (3.7), (3.4), and (v) that

$$ (3.30) \qquad \begin{aligned} \|x_{n+1} - \bar{x}_n\|^2 &\le 2 \left( \lambda_n^2 \|T_n \bar{x}_n - \bar{x}_n\|^2 + \lambda_n^2 \|e_n\|^2 \right) \\ &\le 2 \left( \frac{\lambda_n}{1/\phi_n - \lambda_n} \lambda_n (1/\phi_n - \lambda_n) \|T_n \bar{x}_n - \bar{x}_n\|^2 + \vartheta_n^2 \right) \\ &\le 2 \left( \zeta \lambda_n (1/\phi_n - \lambda_n) \|T_n \bar{x}_n - \bar{x}_n\|^2 + \vartheta_n^2 \right) \\ &\to 0. \end{aligned} $$

Therefore $x_{n+1} - \bar{x}_n \to 0$ and hence the weak sequential cluster points of $(x_n)_{n \in \mathbb{N}}$ lie in $S$. In view of (iv) and [10, Lemma 2.47], the claim is proved.

(viii): Since $x_{n+1} - \bar{x}_n \to 0$ by (3.30), $(x_n)_{n \in \mathbb{N}}$ has a strong cluster point $x \in S$. In view of (iv), $x_n \to x$.

(ix): Set $\zeta = 1/\varepsilon - 1$. Then, for every $n \in \mathbb{N}$, $\lambda_n \le 1/(1+\zeta) + \zeta/(\phi_n(1+\zeta))$ and hence $(1+\zeta)\lambda_n - 1 \le \zeta/\phi_n$, i.e., $\lambda_n - 1 \le \zeta(1/\phi_n - \lambda_n)$. We therefore derive from (iii), (3.27), (3.3), (v), and (3.28) that

$$ (3.31) \qquad \begin{aligned} 0 \le \lambda_n \max_{1 \le i \le m} \frac{1 - \alpha_{i,n}}{\alpha_{i,n}} \|(\mathrm{Id} - T_{i,n}) T_{i+,n} \bar{x}_n - (\mathrm{Id} - T_{i,n}) T_{i+,n} x\|^2 \\ \le \sum_{j=0}^n \mu_{n,j} \|x_j - x\|^2 - \|x_{n+1} - x\|^2 + \frac{1}{2} \sum_{j=0}^n \sum_{k=0}^n [\mu_{n,j} \mu_{n,k}]^{-} \|x_j - x_k\|^2 \\ + \lambda_n (\lambda_n - 1) \|T_n \bar{x}_n - \bar{x}_n\|^2 + \nu_n(x) \\ \le \sum_{j=0}^n \mu_{n,j} \|x_j - x\|^2 - \|x_{n+1} - x\|^2 + \frac{1}{2} \sum_{j=0}^n \sum_{k=0}^n [\mu_{n,j} \mu_{n,k}]^{-} \|x_j - x_k\|^2 \\ + \zeta \lambda_n (1/\phi_n - \lambda_n) \|T_n \bar{x}_n - \bar{x}_n\|^2 + \nu_n(x) \\ \to 0, \end{aligned} $$

which shows the assertion.

<figure><img src="image_4.png" /></figure>

Next, we present two corollaries that are instrumental in the analysis of two important special cases of our framework: mean value and inertial multilayer algorithms.

**COROLLARY 3.2.** Consider the setting of Algorithm 1.2 and define $(\vartheta_n)_{n \in \mathbb{N}}$ as (3.1). Assume that

$$ (3.32) \qquad \inf_{n \in \mathbb{N}} \min_{0 \le j \le n} \mu_{n,j} \ge 0 \quad \text{and} \quad \sum_{n \in \mathbb{N}} \chi_n \vartheta_n < +\infty. $$

Then the following hold:

(i) $\sum_{j=0}^n \sum_{k=0}^n \mu_{n,j} \mu_{n,k} \|x_j - x_k\|^2 \to 0$.

(ii) Let $x \in S$ and suppose that $(\exists \varepsilon \in ]0, 1[)(\forall n \in \mathbb{N}) \lambda_n \le \varepsilon + (1 - \varepsilon)/\phi_n$. Then

$$ \lambda_n \max_{1 \le i \le m} \frac{1 - \alpha_{i,n}}{\alpha_{i,n}} \|(\mathrm{Id} - T_{i,n}) T_{i+,n} \bar{x}_n - (\mathrm{Id} - T_{i,n}) T_{i+,n} x\|^2 \to 0. $$

(iii) Suppose that every weak sequential cluster point of $(\bar{x}_n)_{n \in \mathbb{N}}$ is in $S$ and that $(\exists \varepsilon \in ]0, 1[)(\forall n \in \mathbb{N}) \lambda_n \le (1 - \varepsilon)/\phi_n$. Then $x_{n+1} - \bar{x}_n \to 0$ and there exists $x \in S$ such that $x_n \to x$.

(iv) Suppose that every weak sequential cluster point of $(\bar{x}_n)_{n \in \mathbb{N}}$ is in $S$, that $\sup_{n \in \mathbb{N}} \phi_n < 1$, and that $(\exists \varepsilon \in ]0, 1[)(\forall n \in \mathbb{N}) \lambda_n \le \varepsilon + (1 - \varepsilon)/\phi_n$. Then $x_{n+1} - \bar{x}_n \to 0$ and there exists $x \in S$ such that $x_n \to x$.
(v) Suppose that every weak sequential cluster point of $(\bar{x}_n)_{n \in \mathbb{N}}$ is in $S$ and that $\inf_{n \in \mathbb{N}} \mu_{n,n} > 0$. Then $x_n - \bar{x}_n \to 0$ and there exists $x \in S$ such that $x_n \to x$.

*Proof.* We derive from Theorem 3.1(i) that $(\forall n \in \mathbb{N}) \|x_{n+1} - x\| \le \sum_{j=0}^{n} \mu_{n,j} \|x_j - x\| + \vartheta_n$. In turn, it follows from condition (d) in Algorithm 1.2 that $(\|x_n - x\|)_{n \in \mathbb{N}}$ converges. As a result, $(x_n)_{n \in \mathbb{N}}$ is bounded and (3.32) therefore implies (3.3).

(i)–(iii): These follow, respectively, from items (vi), (ix), and (vii) in Theorem 3.1.

(iv): Set $\delta = \varepsilon(1 - \sup_{n \in \mathbb{N}} \phi_n)$. Then $\delta \in ]0, \varepsilon[$ and $(\forall n \in \mathbb{N}) (\varepsilon - \delta)/\phi_n \ge \varepsilon$. Hence,

$$ (3.33) \qquad (\forall n \in \mathbb{N}) \quad \lambda_n \le \varepsilon + \frac{1 - \varepsilon}{\phi_n} = \varepsilon + \frac{1 - \delta}{\phi_n} - \frac{\varepsilon - \delta}{\phi_n} \le \frac{1 - \delta}{\phi_n}. $$

The claim therefore follows from (iii).

(v): Set

$$ (3.34) \qquad \theta = \frac{1}{\inf_{n \in \mathbb{N}} \mu_{n,n}} \quad \text{and} \quad (\forall n \in \mathbb{N})(\forall j \in \{0, \dots, n\}) \quad \gamma_{n,j} = \begin{cases} \frac{\mu_{n,n} + 1}{2} & \text{if } j = n, \\ \frac{\mu_{n,j}}{2} & \text{if } j < n. \end{cases} $$

Then, using Apollonius' identity, [10, Lemma 2.12(iv)], and (3.34), we obtain

$$ (3.35) \qquad \frac{1}{4} \|\bar{x}_n - x_n\|^2 $$

$$
\begin{align*}
&= \frac{1}{2} \left( \|\bar{x}_n - x\|^2 + \|x_n - x\|^2 \right) - \left\| \frac{\bar{x}_n + x_n}{2} - x \right\|^2 \\
&= \frac{1}{2} \left( \|\bar{x}_n - x\|^2 + \|x_n - x\|^2 \right) - \left\| \sum_{j=0}^{n} \gamma_{n,j} (x_j - x) \right\|^2 \\
&= \frac{1}{2} \left( \|\bar{x}_n - x\|^2 + \|x_n - x\|^2 \right) - \sum_{j=0}^{n} \gamma_{n,j} \|x_j - x\|^2 + \sum_{0 \le j < k \le n} \gamma_{n,j} \gamma_{n,k} \|x_j - x_k\|^2 \\
&\le \frac{1}{2} \left( \sum_{j=0}^{n} \mu_{n,j} \|x_j - x\|^2 + \|x_n - x\|^2 \right) - \sum_{j=0}^{n} \gamma_{n,j} \|x_j - x\|^2 \\
&\qquad + \frac{1}{4} \left( \sum_{0 \le j < k < n} \mu_{n,j} \mu_{n,k} \|x_j - x_k\|^2 + \sum_{j=0}^{n-1} \mu_{n,j} (\mu_{n,n} + 1) \|x_j - x_n\|^2 \right) \\
&\le \frac{1}{2} \left( \sum_{j=0}^{n} \mu_{n,j} \|x_j - x\|^2 + \|x_n - x\|^2 \right) - \sum_{j=0}^{n} \gamma_{n,j} \|x_j - x\|^2 \\
&\qquad + \frac{1}{4} \left( \sum_{0 \le j < k \le n} \mu_{n,j} \mu_{n,k} \|x_j - x_k\|^2 + \theta \sum_{j=0}^{n-1} \mu_{n,j} \mu_{n,n} \|x_j - x_n\|^2 \right) \\
&\le \frac{1}{2} \left( \sum_{j=0}^{n} \mu_{n,j} \|x_j - x\|^2 + \|x_n - x\|^2 \right) - \sum_{j=0}^{n} \gamma_{n,j} \|x_j - x\|^2 \\
&\qquad + \frac{1 + \theta}{4} \sum_{0 \le j < k \le n} \mu_{n,j} \mu_{n,k} \|x_j - x_k\|^2.
\end{align*}
$$
Next, let us set $\rho = \lim \|x_n - x\|^2$. Then it follows from Lemma 2.2 that $\sum_{j=0}^n \mu_{n,j} \|x_j - x\|^2 \to \rho$ and $\sum_{j=0}^n \gamma_{n,j} \|x_j - x\|^2 \to \rho$. On the other hand, (i) asserts that $\sum_{0 \le j < k \le n} \mu_{n,j} \mu_{n,k} \|x_j - x_k\|^2 \to 0$. Altogether, (3.35) yields $\|\bar{x}_n - x_n\| \to 0$. Thus, the weak sequential cluster points of $(x_n)_{n \in \mathbb{N}}$ belong to $S$, and the conclusion follows from the fact that $(\|x_n - x\|)_{n \in \mathbb{N}}$ converges and from [10, Lemma 2.47]. □

**COROLLARY 3.3.** *Consider the setting of Algorithm 1.2 with*

$$ (3.36) \qquad (\forall i \in \{1, \dots, m\}) (\forall n \in \mathbb{N}) \quad e_{i,n} = 0. $$

Set $x_{-1} = x_0$ and suppose that there exists a sequence $(\eta_n)_{n \in \mathbb{N}}$ in $[0, 1[$ such that $\eta_0 = 0$ and

$$ (3.37) \qquad (\forall n \in \mathbb{N})(\forall j \in \{0, \dots, n\}) \quad \mu_{n,j} = \begin{cases} 1 + \eta_n & \text{if } j = n, \\ -\eta_n & \text{if } j = n-1, \\ 0 & \text{if } j < n-1. \end{cases} $$

For every $n \in \mathbb{N}$, set

$$ (3.38) \qquad (\forall k \in \mathbb{N}) \quad \zeta_{k,n} = \begin{cases} 0 & \text{if } k \le n, \\ \displaystyle \sum_{j=n+1}^{k} (\eta_j - 1) & \text{if } k > n, \end{cases} $$

and assume that $\chi_n = \sum_{k \ge n} \exp(\zeta_{k,n})$. Suppose that one of the following is satisfied:

(a) $\sum_{n \in \mathbb{N}} \chi_n \eta_n \|x_n - x_{n-1}\|^2 < +\infty$.

(b) $\sum_{n \in \mathbb{N}} n \|x_n - x_{n-1}\|^2 < +\infty$ and there exists $\tau \in [2, +\infty[$ such that $(\forall n \in \mathbb{N} \setminus \{0\}) \eta_n = (n-1)/(n+\tau)$.

(c) $\sum_{n \in \mathbb{N}} \eta_n \|x_n - x_{n-1}\|^2 < +\infty$ and there exists $\eta \in [0, 1[$ such that $(\forall n \in \mathbb{N}) \eta_n \le \eta$.

(d) Set $(\forall n \in \mathbb{N}) \omega_n = 1/\phi_n - \lambda_n$. There exist $(\sigma, \vartheta) \in ]0, +\infty[$ such that

$$ (3.39) \qquad (\forall n \in \mathbb{N}) \quad \begin{cases} \eta_n \le \eta_{n+1} \le \eta, \\ \lambda_n \le \frac{\vartheta/\phi_n - \eta(\eta(1+\eta) + \eta\vartheta\omega_{n+1} + \sigma)}{\vartheta(1+\eta(1+\eta) + \eta\vartheta\omega_{n+1} + \sigma)}, \\ \frac{\eta^2(1+\eta) + \eta\sigma}{\vartheta} < \frac{1}{\phi_n} - \eta^2\omega_{n+1}. \end{cases} $$

Then the following hold:

(i) $\lambda_n(1/\phi_n - \lambda_n) \|T_n \bar{x}_n - \bar{x}_n\|^2 \to 0$.

(ii) Let $x \in S$ and suppose that $(\exists \varepsilon \in ]0, 1[)(\forall n \in \mathbb{N}) \lambda_n \le \varepsilon + (1-\varepsilon)/\phi_n$. Then

$$ \lambda_n \max_{1 \le i \le m} \frac{1 - \alpha_{i,n}}{\alpha_{i,n}} \|(\mathrm{Id} - T_{i,n}) T_{i+,n} \bar{x}_n - (\mathrm{Id} - T_{i,n}) T_{i+,n} x\|^2 \to 0. $$

(iii) $\bar{x}_n - x_n \to 0$.

(iv) Suppose that every weak sequential cluster point of $(\bar{x}_n)_{n \in \mathbb{N}}$ is in $S$. Then there exists $x \in S$ such that $x_n \rightharpoonup x$.

**Proof.** In view of Example 2.5, $(\mu_{n,j})_{n \in \mathbb{N}, 0 \le j \le n}$ satisfies conditions (a)–(d) in Algorithm 1.2.
(a): Set $\chi = \inf_{n \in \mathbb{N}} \chi_n$ and define $(\nu_n)_{n \in \mathbb{N}}$ as in (3.2). We have $\sup_{n \in \mathbb{N}} (1+\eta_n) \le 2$ and

(3.40)

$$ (\forall n \in \mathbb{N}) \quad \begin{cases} \chi_n \sum_{j=0}^{n} \sum_{k=0}^{n} [\mu_{n,j} \mu_{n,k}]^{-} \|x_j - x_k\|^2 = (1 + \eta_n) \chi_n \eta_n \|x_n - x_{n-1}\|^2, \\ (\forall x \in S) \quad \chi_n \nu_n(x) = 0. \end{cases} $$

Hence (3.3) holds, and (i) and (ii) follow from items (v) and (ix) in Theorem 3.1, respectively. Furthermore, (3.37) implies that

$$ (3.41) \quad \|\bar{x}_n - x_n\|^2 \le \eta_n^2 \|x_n - x_{n-1}\|^2 \le \eta_n \|x_n - x_{n-1}\|^2 \le \frac{\chi_n \eta_n}{\chi} \|x_n - x_{n-1}\|^2 \to 0. $$

Thus, (iii) holds. In turn, the weak sequential cluster points of $(x_n)_{n \in \mathbb{N}}$ belong to $S$ and (iv) therefore follows from Theorem 3.1(iv) and [10, Lemma 2.47].

(b) ⇒ (a): It follows from Lemma 2.4(i) that

$$ (3.42) \quad \sum_{n \in \mathbb{N}} \chi_n \eta_n \|x_n - x_{n-1}\|^2 \le \sum_{n \in \mathbb{N}} \frac{n+7}{2} \|x_n - x_{n-1}\|^2 < +\infty. $$

(c) ⇒ (a): Lemma 2.4(ii) asserts that $\sup_{n \in \mathbb{N}} \chi_n \le e/(1-\eta)$.

(d) ⇒ (c): Let $x \in S$. It follows from Theorem 3.1(ii) that

$$ (\forall n \in \mathbb{N}) \quad \|x_{n+1} - x\|^2 \le (1+\eta_n) \|x_n - x\|^2 - \eta_n \|x_{n-1} - x\|^2 + \eta_n (1+\eta_n) \|x_n - x_{n-1}\|^2 $$

$$ (3.43) \quad - \lambda_n \left( \frac{1}{\phi_n} - \lambda_n \right) \|T_n \bar{x}_n - \bar{x}_n\|^2. $$

Now set $\beta_{-1} = \|x_0 - x\|^2$ and

$$ (3.44) \quad (\forall n \in \mathbb{N}) \quad \beta_n = \|x_n - x\|^2, \quad \delta_n = \|x_n - x_{n-1}\|^2, \quad \text{and} \quad \rho_n = \frac{1}{\eta_n + \lambda_n \vartheta}. $$

Then

$$ (\forall n \in \mathbb{N}) \quad \begin{aligned} \|T_n \bar{x}_n - \bar{x}_n\|^2 &= \frac{1}{\lambda_n^2} \left\| (x_{n+1} - x_n) + \eta_n (x_{n-1} - x_n) \right\|^2 \\ &= \frac{1}{\lambda_n^2} \left( \delta_{n+1} + \eta_n^2 \delta_n + \eta_n \left( 2 \left\langle \sqrt{\rho_n} (x_{n+1} - x_n) \middle| \frac{x_{n-1} - x_n}{\sqrt{\rho_n}} \right\rangle \right) \right) \\ (3.45) \qquad \qquad \qquad \qquad \ge \frac{1}{\lambda_n^2} \left( \delta_{n+1} + \eta_n^2 \delta_n - \eta_n \left( \rho_n \delta_{n+1} + \frac{\delta_n}{\rho_n} \right) \right). \end{aligned} $$

Thus, we derive from (3.43) that

$$ (3.46) \quad (\forall n \in \mathbb{N}) \quad \beta_{n+1} - \beta_n - \eta_n (\beta_n - \beta_{n-1}) \le \frac{(1/\phi_n - \lambda_n)(\eta_n \rho_n - 1)}{\lambda_n} \delta_{n+1} + \gamma_n \delta_n, $$

where

$$ (3.47) \quad (\forall n \in \mathbb{N}) \quad \gamma_n = \eta_n(1 + \eta_n) + \eta_n \left( \frac{1}{\phi_n} - \lambda_n \right) \frac{1 - \rho_n \eta_n}{\rho_n \lambda_n} \ge 0. $$

However, it follows from (3.44) that $(\forall n \in \mathbb{N}) \vartheta = (1 - \rho_n \eta_n)/(\rho_n \lambda_n)$. Hence, (3.47) yields

$$ (3.48) \quad (\forall n \in \mathbb{N}) \quad \gamma_n = \eta_n(1 + \eta_n) + \eta_n \left( \frac{1}{\phi_n} - \lambda_n \right) \vartheta \le \eta(1 + \eta) + \eta \vartheta \omega_n. $$

Thus, by Lemma 2.3, $\sum_{n \in \mathbb{N}} \eta_n \delta_n \le \sum_{n \in \mathbb{N}} \delta_n < +\infty$ and we conclude that (c) is satisfied. □
**Remark 3.4.** In Corollary 3.3, no errors were allowed in the implementation of the operators. It is however possible to allow errors in multilayer inertial methods in certain scenarios. For instance, suppose that in Corollary 3.3 we make the additional assumptions that $\lambda_n \equiv 1$ and that $\bigcup_{n \in \mathbb{N}} \text{ran } T_{1,n}$ is bounded. At the same time, let us introduce errors of such that $(\forall i \in \{1, \dots, m\}) \sum_{n \in \mathbb{N}} \chi_n \|e_{i,n}\| < +\infty$. Note that (1.12) becomes (3.49)

$$ \text{for } n = 0, 1, \dots \\ \quad \left| \begin{aligned} \bar{x}_n &= (1 + \eta_n)x_n - \eta_n x_{n-1}, \\ x_{n+1} &= T_{1,n}(T_{2,n}(\dots T_{m-1,n}(T_{m,n}\bar{x}_n + e_{m,n}) + e_{m-1,n}\dots) + e_{2,n}) + e_{1,n}. \end{aligned} \right. $$

Hence, the assumptions imply that $(x_n)_{n \in \mathbb{N}}$ is bounded. In turn, $(\bar{x}_n)_{n \in \mathbb{N}}$ is bounded and it follows from (3.2) that $(\forall x \in S) \sum_{n \in \mathbb{N}} \chi_n \nu_n(x) < +\infty$. An inspection of the proof of Corollary 3.3 then reveals immediately that its conclusions under any of assumptions (a)–(c) remain true.

**4. Examples and applications.** In this section we exhibit various existing results as special cases of our framework. Our purpose is not to exploit it to its full capacity but rather to illustrate its potential on simple instances. We first recover the main result of [21] on algorithm (1.4).

**Example 4.1.** We consider the setting studied in [21]. Let $(T_n)_{n \in \mathbb{N}}$ be a sequence of firmly quasi-nonexpansive operators from $\mathcal{H}$ to $\mathcal{H}$ such that $S = \bigcap_{n \in \mathbb{N}} \text{Fix } T_n \neq \emptyset$. Then the problem of finding a point in $S$ is a special case of Problem 1.1(c), where we assume that $\alpha_{1,n} \equiv 1/2$. In addition, let $(e_n)_{n \in \mathbb{N}}$ be a sequence in $\mathcal{H}$ such that $\sum_{n \in \mathbb{N}} \|e_n\| < +\infty$, let $(\lambda_n)_{n \in \mathbb{N}}$ be a sequence in $[0, 2]$ such that $0 < \inf_{n \in \mathbb{N}} \lambda_n \le \sup_{n \in \mathbb{N}} \lambda_n < 2$, and let $(\mu_{n,j})_{n \in \mathbb{N}, 0 \le j \le n}$ be an array with entries in $[0, +\infty[$ which satisfies the following:

(a) $(\forall n \in \mathbb{N}) \sum_{j=0}^{n} \mu_{n,j} = 1$.

(b) $(\forall j \in \mathbb{N}) \lim_{n \to +\infty} \mu_{n,j} = 0$.

(c) Every sequence $(\xi_n)_{n \in \mathbb{N}}$ in $[0, +\infty[$ such that

$$ (4.1) \quad \left( \exists (\varepsilon_n)_{n \in \mathbb{N}} \in [0, +\infty[ \right) \begin{cases} \sum_{n \in \mathbb{N}} \varepsilon_n < +\infty, \\ (\forall n \in \mathbb{N}) \quad \xi_{n+1} \le \sum_{j=0}^{n} \mu_{n,j} \xi_j + \varepsilon_n, \end{cases} $$

converges.

Clearly, conditions (a)–(c) above imply that, in Algorithm 1.2, conditions (a)–(d) are satisfied. Now let $x_0 \in \mathcal{H}$, and define a sequence $(x_n)_{n \in \mathbb{N}}$ by

$$ (4.2) \quad (\forall n \in \mathbb{N}) \quad x_{n+1} = \bar{x}_n + \lambda_n (T_n \bar{x}_n + e_n - \bar{x}_n), \quad \text{where } \bar{x}_n = \sum_{j=0}^{n} \mu_{n,j} x_j, $$

which corresponds to a 1-layer instance of (1.12). This mean iteration process was seen in [21] to cover several classical mean iteration methods, as well as memoryless convex feasibility algorithms [18] (see also [13]). The result obtained in [21, Theorem 3.5(i)] on the weak convergence of $(x_n)_{n \in \mathbb{N}}$ to a point in $S$ corresponds to the special case of Corollary 3.2(iii) in which we further set $\chi_n \equiv 1$.

Next, we retrieve the main result of [25] on the convergence of an overrelaxed version of (1.7) and the special cases discussed there, in particular, those of [19].
**Example 4.2.** We consider the setting studied in [25], which corresponds to Problem 1.1(a). Given $x_0 \in \mathcal{H}$ and sequences $(e_{1,n})_{n \in \mathbb{N}}$, ..., $(e_{m,n})_{n \in \mathbb{N}}$ in $\mathcal{H}$ such that $\sum_{n \in \mathbb{N}} \lambda_n \sum_{i=1}^m \|e_{i,n}\| < +\infty$, construct a sequence $(x_n)_{n \in \mathbb{N}}$ via the m-layer recursion

$$ (4.3) \quad (\forall n \in \mathbb{N}) \quad x_{n+1} = x_n + \lambda_n (T_{1,n}(T_{2,n}(\cdots T_{m-1,n}(T_{m,n}x_n + e_{m,n}) + e_{m-1,n}\cdots) + e_{2,n}) + e_{1,n} - x_n), $$
where $0 < \lambda_n \le \varepsilon + (1-\varepsilon)/\phi_n$.

Note that (4.3) corresponds to the memoryless version of (1.12). The result on the weak convergence of $(x_n)_{n \in \mathbb{N}}$ obtained in [25, Theorem 3.5(iii)] corresponds to the special case of Corollary 3.2(iv) in which the following additional assumptions are made:

(a) $(\forall n \in \mathbb{N}) \chi_n = 1$ and $(\forall j \in \{0, \dots, n\}) \mu_{n,j} = \delta_{n,j}$.

(b) $(\exists \varepsilon \in ]0, 1[)(\forall n \in \mathbb{N}) \lambda_n \le (1 - \varepsilon)(1/\phi_n + \varepsilon)$.

Note that condition (a) above implies that, in Algorithm 1.2, conditions (a)-(c) trivially hold, while condition (d) follows from [10, Lemma 5.31]. We also observe that [25, Theorem 3.5(iii)] itself extends the results of [19, section 3], where the relaxation parameters $(\lambda_n)_{n \in \mathbb{N}}$ are confined to $]0, 1]$.

The next two examples feature mean value and inertial iterations in the case of a single quasi-nonexpansive operator. As is easily seen, the memoryless algorithm (1.1) can fail to produce a convergent sequence in this scenario.

**Example 4.3.** Let $T: \mathcal{H} \to \mathcal{H}$ be a quasi-nonexpansive operator such that $\text{Id} - T$ is demiclosed at 0 and $\text{Fix} T \ne \emptyset$, let $(\mu_{n,j})_{n \in \mathbb{N}, 0 \le j \le n}$ be an array in $[0, +\infty[$ that satisfies conditions (a)-(d) in Algorithm 1.2 with $\chi_n = 1$ and such that $\inf_{n \in \mathbb{N}} \mu_{n+1,n} \mu_{n+1,n+1} > 0$, and let $(e_n)_{n \in \mathbb{N}}$ be a sequence in $\mathcal{H}$ such that $\sum_{n \in \mathbb{N}} \|e_n\| < +\infty$. Let $x_0 \in \mathcal{H}$ and iterate

$$ (4.4) \quad (\forall n \in \mathbb{N}) \quad x_{n+1} = T\bar{x}_n + e_n, \quad \text{where} \quad \bar{x}_n = \sum_{j=0}^{n} \mu_{n,j}x_j. $$

Then $T\bar{x}_n - \bar{x}_n \to 0$ and there exists $x \in \text{Fix} T$ such that $x_n \to x$ and $\bar{x}_n \to x$.

*Proof.* We apply Corollary 3.2 in the setting of Problem 1.1(c) with $T_{1,n} \equiv T$, $\alpha_{1,n} \equiv 1$, $\phi_n \equiv 1$, and $\lambda_n \equiv 1$. First, note that (3.32) is satisfied. Furthermore, Corollary 3.2(v) entails that $\bar{x}_n - x_n \to 0$, while Corollary 3.2(i) yields $\mu_{n+1,n}\mu_{n+1,n+1} \|x_{n+1} - x_n\|^2 \to 0$ and hence $x_{n+1} - x_n \to 0$. Therefore $\bar{x}_n - T\bar{x}_n = (\bar{x}_n - x_n) + (x_n - x_{n+1}) + e_n \to 0$. Since $\text{Id} - T$ is demiclosed at 0, it follows that every weak sequential cluster point of $(\bar{x}_n)_{n \in \mathbb{N}}$ is in $\text{Fix} T$. In view of Corollary 3.2(v), the proof is complete. □

**Example 4.4.** Let $T: \mathcal{H} \to \mathcal{H}$ be a quasi-nonexpansive operator such that $\text{Id} - T$ is demiclosed at 0 and $\text{Fix} T \ne \emptyset$, and let $(\eta_n)_{n \in \mathbb{N}}$ be a sequence in $[0, 1[$ such that $\eta_0 = 0$, $\eta = \sup_{n \in \mathbb{N}} \eta_n < 1$, and $(\forall n \in \mathbb{N}) \eta_n \le \eta_{n+1}$. Let $(\sigma, \vartheta) \in ]0, +\infty]^2$ be such that $(\eta^2(1 + \eta) + \eta\sigma)/\vartheta < 1 - \eta^2$, and let $(\lambda_n)_{n \in \mathbb{N}}$ be a sequence in $]0, 1[$ such that $0 < \inf_{n \in \mathbb{N}} \lambda_n \le \sup_{n \in \mathbb{N}} \lambda_n \le (\vartheta - \eta(\eta(1 + \eta) + \eta\vartheta + \sigma))/(\vartheta(1 + \eta(1 + \eta) + \eta\vartheta + \sigma))$. Let $x_0 \in \mathcal{H}$, set $x_{-1} = x_0$, and iterate

$$ (4.5) \quad (\forall n \in \mathbb{N}) \quad x_{n+1} = \bar{x}_n + \lambda_n (T\bar{x}_n - \bar{x}_n), \quad \text{where} \quad \bar{x}_n = (1 + \eta_n)x_n - \eta_n x_{n-1}. $$

Then $T\bar{x}_n - \bar{x}_n \to 0$ and there exists $x \in \text{Fix} T$ such that $x_n \to x$. In the case when $T$ is nonexpansive, this result appears in [14, Theorem 5].
Proof. This is an instance of items (i) and (iv) in Corollary 3.3(d) and Problem 1.1(c) in which $T_{1,n} \equiv T$, $\alpha_{1,n} \equiv 1$, and $\phi_n \equiv 1$. Note that condition (d) in Corollary 3.3 is satisfied since $(\forall n \in \mathbb{N}) \omega_n = 1 - \lambda_n < 1$. □

Next, we consider applications to monotone operator splitting. Let us recall basic notions about a set-valued operator $A: \mathcal{H} \to 2^{\mathcal{H}}$ [10]. We denote by $\mathrm{ran} A = \{u \in \mathcal{H} \mid (\exists x \in \mathcal{H}) u \in Ax\}$ the range of $A$, by $\mathrm{dom} A = \{x \in \mathcal{H} \mid Ax \neq \emptyset\}$ the domain of $A$, by $\mathrm{zer} A = \{x \in \mathcal{H} \mid 0 \in Ax\}$ the set of zeros of $A$, by $\mathrm{gra} A = \{(x, u) \in \mathcal{H} \times \mathcal{H} \mid u \in Ax\}$ the graph of $A$, and by $A^{-1}$ the inverse of $A$, i.e., the operator with graph $\{(u, x) \in \mathcal{H} \times \mathcal{H} \mid u \in Ax\}$. The resolvent of $A$ is $J_A = (\mathrm{Id} + A)^{-1}$ and $s: \mathrm{dom} A \to \mathcal{H}$ is a selection of $A$ if $(\forall x \in \mathrm{dom} A) s(x) \in Ax$. Moreover, $A$ is monotone if

$$ (4.6) \qquad (\forall(x, u) \in \mathrm{gra} A)(\forall(y, v) \in \mathrm{gra} A) \quad \langle x - y \mid u - v \rangle \ge 0, $$

and maximally monotone if there exists no monotone operator $B: \mathcal{H} \to 2^{\mathcal{H}}$ such that $\mathrm{gra} A \subset \mathrm{gra} B \neq \mathrm{gra} A$. In this case, $J_A$ is a firmly nonexpansive operator defined everywhere on $\mathcal{H}$ and the reflector $R_A = 2J_A - \mathrm{Id}$ is nonexpansive. We denote by $\Gamma_0(\mathcal{H})$ the class of proper lower semicontinuous convex functions from $\mathcal{H}$ to $]-\infty, +\infty]$. Let $f \in \Gamma_0(\mathcal{H})$. For every $x \in \mathcal{H}$, $f + \|x - \cdot\|^2/2$ possesses a unique minimizer, which is denoted by $\mathrm{prox}_f x$. We have $\mathrm{prox}_f = J_{\partial f}$, where

$$ (4.7) \qquad \partial f: \mathcal{H} \to 2^{\mathcal{H}}: x \mapsto \{u \in \mathcal{H} \mid (\forall y \in \mathcal{H}) \langle y - x \mid u \rangle + f(x) \le f(y)\} $$

is the Moreau subdifferential of $f$. Our convergence results will rest on the following asymptotic principle.

LEMMA 4.5. Let $A$ and $B$ be maximally monotone operators from $\mathcal{H}$ to $2^{\mathcal{H}}$, let $(x_n, u_n)_{n \in \mathbb{N}}$ be a sequence in $\mathrm{gra} A$, let $(y_n, v_n)_{n \in \mathbb{N}}$ be a sequence in $\mathrm{gra} B$, let $x \in \mathcal{H}$, and let $v \in \mathcal{H}$. Suppose that $x_n \to x$, $v_n \to v$, $x_n - y_n \to 0$, and $u_n + v_n \to 0$. Then the following hold:

(i) $(x, -v) \in \mathrm{gra} A$ and $(x, v) \in \mathrm{gra} B$.

(ii) $0 \in Ax + Bx$ and $0 \in -A^{-1}(-v) + B^{-1}v$.

Proof. Apply [10, Proposition 26.5] with $\mathcal{K} = \mathcal{H}$ and $L = \mathrm{Id}$. □

<figure><img src="image_5.png" /></figure>

As discussed in [19], many splitting methods can be analyzed within the powerful framework of fixed point methods for averaged operators. The analysis provided in the present paper therefore makes it possible to develop new methods in this framework, for instance, mean value or inertial versions of standard splitting methods. We provide two such examples below. First, we consider the Peaceman-Rachford splitting method, which typically does not converge unless strong requirements are imposed on the underlying operators [20]. In the spirit of Mann's work [34], we show that mean iterations induce the convergence of this algorithm.

PROPOSITION 4.6. Let $A: \mathcal{H} \to 2^{\mathcal{H}}$ and $B: \mathcal{H} \to 2^{\mathcal{H}}$ be maximally monotone operators such that $\mathrm{zer}(A+B) \neq \emptyset$ and let $\gamma \in ]0, +\infty[$. Let $(a_n)_{n \in \mathbb{N}}$ and $(b_n)_{n \in \mathbb{N}}$ be sequences in $\mathcal{H}$ such that $\sum_{n \in \mathbb{N}} \|a_n\| < +\infty$ and $\sum_{n \in \mathbb{N}} \|b_n\| < +\infty$, let $x_0 \in \mathcal{H}$, and let $(\mu_{n,j})_{n \in \mathbb{N}, 0 \le j \le n}$ be an array in $]0, +\infty[$ that satisfies conditions (a)-(d) in
Algorithm 1.2 with $\chi_n \equiv 1$ and such that $\inf_{n \in \mathbb{N}} \mu_{n+1,n}\mu_{n+1,n+1} > 0$. **Iterate**

$$ (4.8) \qquad \left\{ \begin{array}{l} \text{for } n = 0, 1, \dots \\ \qquad \bar{x}_n = \sum_{j=0}^{n} \mu_{n,j} x_j, \\ \qquad y_n = J_{\gamma B} \bar{x}_n + b_n, \\ \qquad z_n = J_{\gamma A} (2y_n - \bar{x}_n) + a_n, \\ \qquad x_{n+1} = \bar{x}_n + 2(z_n - y_n). \end{array} \right. $$

Then there exists $x \in \text{Fix } R_{\gamma A}R_{\gamma B}$ such that $x_n \to x$ and $\bar{x}_n \to x$. Now set $y = J_{\gamma B}x$. Then $y \in \text{zer}(A+B)$, $z_n - y_n \to 0$, $y_n \to y$, and $z_n \to y$.

*Proof.* Set $T = R_{\gamma A}R_{\gamma B}$ and $(\forall n \in \mathbb{N}) e_n = 2a_n + R_{\gamma A}(R_{\gamma B}\bar{x}_n + 2b_n) - R_{\gamma A}(R_{\gamma B}\bar{x}_n)$. Then $T$ is nonexpansive, $\text{Id} - T$ is therefore demiclosed, and, since $\text{zer}(A+B) \neq \emptyset$, [10, Proposition 26.1(iii)(b)] yields $\text{Fix } T = \text{Fix } R_{\gamma A}R_{\gamma B} \neq \emptyset$. In addition, we derive from (4.8) that

$$ (4.9) \qquad (\forall n \in \mathbb{N}) \quad x_{n+1} = T\bar{x}_n + e_n, $$

where

$$ (4.10) \qquad \begin{aligned} \sum_{n \in \mathbb{N}} \|e_n\| &\le \sum_{n \in \mathbb{N}} (2\|a_n\| + \|R_{\gamma A}(R_{\gamma B}\bar{x}_n + 2b_n) - R_{\gamma A}(R_{\gamma B}\bar{x}_n)\|) \\ &\le \sum_{n \in \mathbb{N}} 2(\|a_n\| + \|b_n\|) \\ &< +\infty. \end{aligned} $$

Consequently, we deduce from Example 4.3 that $(x_n)_{n \in \mathbb{N}}$ and $(\bar{x}_n)_{n \in \mathbb{N}}$ converge weakly to a point $x \in \text{Fix } T = \text{Fix } R_{\gamma A}R_{\gamma B}$, and that $T\bar{x}_n - \bar{x}_n \to 0$. In addition, [10, Proposition 26.1(iii)(b)] asserts that $y \in \text{zer}(A+B)$. Next, we derive from (4.8) and (4.9) that $2(z_n - y_n) = x_{n+1} - \bar{x}_n = (T\bar{x}_n - \bar{x}_n) + e_n \to 0$. It remains to show that $y_n \to y$. Since $(\bar{x}_n)_{n \in \mathbb{N}}$ converges weakly, it is bounded. However, $(\forall n \in \mathbb{N}) \|y_n - y_0\| = \|J_{\gamma B}\bar{x}_n - J_{\gamma B}x_0 + b_n\| \le \|\bar{x}_n - x_0\| + \|b_n\|$. Therefore $(y_n)_{n \in \mathbb{N}}$ is bounded. Now let $z$ be a weak sequential cluster point of $(y_n)_{n \in \mathbb{N}}$, say $y_{k_n} \to z$. In view of [10, Lemma 2.46], it is enough to show that $z = y$. To this end, set $(\forall n \in \mathbb{N}) v_n = \gamma^{-1}(\bar{x}_n - y_n + b_n)$ and $w_n = \gamma^{-1}(2y_n - \bar{x}_n - z_n + a_n)$. Then $(\forall n \in \mathbb{N}) (z_n - a_n, w_n) \in \text{gra } A$ and $(y_n - b_n, v_n) \in \text{gra } B$. In addition, we have $\bar{x}_{k_n} \to x$, $z_{k_n} - a_{k_n} \to z$, $v_{k_n} \to \gamma^{-1}(x-z)$, $(z_{k_n} - a_{k_n}) - (y_{k_n} - b_{k_n}) \to 0$, and $v_{k_n} + w_{k_n} = \gamma^{-1}(y_{k_n} - z_{k_n} + a_{k_n} + b_{k_n}) \to 0$. Hence, we derive from Lemma 4.5(i) that $(z, \gamma^{-1}(x-z)) \in \text{gra } B$, i.e., $z = J_{\gamma B}x = y$. □

*Remark 4.7.* Let $f$ and $g$ be functions in $\Gamma_0(\mathcal{H})$, and specialize Proposition 4.6 to $A = \partial f$ and $B = \partial g$. Then $\text{zer}(A+B) = \text{Argmin}(f+g)$. Moreover, (4.8) becomes

$$ (4.11) \qquad \left\{ \begin{array}{l} \bar{x}_n = \sum_{j=0}^{n} \mu_{n,j} x_j, \\ y_n = \text{prox}_{\gamma g} \bar{x}_n + b_n, \\ z_n = \text{prox}_{\gamma f} (2y_n - \bar{x}_n) + a_n, \\ x_{n+1} = \bar{x}_n + 2(z_n - y_n), \end{array} \right. $$

and we conclude that there exists a point $y \in \text{Argmin}(f+g)$ such that $y_n \to y$ and $z_n \to y$.
We now propose a new forward-backward splitting framework which includes existing instances as special cases. The following notion will be needed to establish strong convergence properties (see [4, Proposition 2.4] for special cases).

**DEFINITION 4.8** (see [4, Definition 2.3]). An operator $A: \mathcal{H} \to 2^{\mathcal{H}}$ is demiregular at $x \in \text{dom } A$ if, for every sequence $(x_n, u_n)_{n \in \mathbb{N}}$ in $\text{gra } A$ and every $u \in \text{Ax}$ such that $x_n \to x$ and $u_n \to u$, we have $x_n \to x$.

**PROPOSITION 4.9.** Let $\beta \in ]0, +\infty[$, let $\varepsilon \in ]0, \min\{1/2, \beta\}$[, let $x_0 \in \mathcal{H}$, let $A: \mathcal{H} \to 2^{\mathcal{H}}$ be maximally monotone, and let $B: \mathcal{H} \to \mathcal{H}$ be $\beta$-cocoercive, i.e.,

$$ (4.12) \qquad (\forall x \in \mathcal{H})(\forall y \in \mathcal{H}) \quad \langle x - y \mid Bx - By \rangle / \beta \ge \|Bx - By\|^2. $$

Furthermore, let $(\gamma_n)_{n \in \mathbb{N}}$ be a sequence in $[\varepsilon, 2\beta/(1+\varepsilon)]$, let $(\mu_{n,j})_{n \in \mathbb{N}, 0 \le j \le n}$ be a real array that satisfies conditions (a)-(d) in Algorithm 1.2, and let $(a_n)_{n \in \mathbb{N}}$ and $(b_n)_{n \in \mathbb{N}}$ be sequences in $\mathcal{H}$ such that $\sum_{n \in \mathbb{N}} \chi_n \|a_n\| < +\infty$ and $\sum_{n \in \mathbb{N}} \chi_n \|b_n\| < +\infty$. Suppose that $\text{zer}(A+B) \ne \emptyset$ and that

$$ (4.13) \qquad (\forall n \in \mathbb{N}) \quad \lambda_n \in \left[ \varepsilon, 1 + (1-\varepsilon) \left( 1 - \frac{\gamma_n}{2\beta} \right) \right], $$

and set $(\forall n \in \mathbb{N}) \phi_n = 2/(4 - \gamma_n/\beta)$. For every $n \in \mathbb{N}$, iterate

$$ (4.14) \qquad x_{n+1} = \bar{x}_n + \lambda_n \left( J_{\gamma_n A} (\bar{x}_n - \gamma_n (B\bar{x}_n + b_n)) + a_n - \bar{x}_n \right), \quad \text{where} \quad \bar{x}_n = \sum_{j=0}^{n} \mu_{n,j} x_j. $$

Suppose that one of the following is satisfied:

(a) $\inf_{n \in \mathbb{N}} \min_{0 \le j \le n} \mu_{n,j} \ge 0$.

(b) $a_n \equiv b_n \equiv 0$, $(\mu_{n,j})_{n \in \mathbb{N}, 0 \le j \le n}$ satisfies (3.37), and one of conditions (a)-(d) in Corollary 3.3 is satisfied.

Then the following hold:

(i) $J_{\gamma_n A}(\bar{x}_n - \gamma_n B\bar{x}_n) - \bar{x}_n \to 0$.

(ii) Let $z \in \text{zer}(A+B)$. Then $B\bar{x}_n \to Bz$.

(iii) There exists $x \in \text{zer}(A+B)$ such that $x_n \to x$.

(iv) Suppose that $A$ or $B$ is demiregular at every point in $\text{zer}(A+B)$. Then there exists $x \in \text{zer}(A+B)$ such that $x_n \to x$.

**Proof.** We apply Corollary 3.2 in case (a) and from Corollary 3.3 in case (b). We first note that (4.14) is an instance of Algorithm 1.2 with $m=2$ and $(\forall n \in \mathbb{N})$ $T_{1,n} = J_{\gamma_n A}$, $T_{2,n} = \text{Id} - \gamma_n B$, $e_{1,n} = a_n$, and $e_{2,n} = -\gamma_n b_n$. Indeed, for every $n \in \mathbb{N}$, $T_{1,n}$ is $\alpha_{1,n}$-averaged with $\alpha_{1,n} = 1/2$ [10, Remark 4.34(iii) and Corollary 23.9], $T_{2,n}$ is $\alpha_{2,n}$-averaged with $\alpha_{2,n} = \gamma_n/(2\beta)$ [10, Proposition 4.39], and the averaging constant of $T_{1,n}T_{2,n}$ is therefore given by (1.13) as

$$ (4.15) \qquad \frac{\alpha_{1,n} + \alpha_{2,n} - 2\alpha_{1,n}\alpha_{2,n}}{1 - \alpha_{1,n}\alpha_{2,n}} = \frac{2}{4 - \gamma_n/\beta} = \phi_n. $$

On the other hand, we are in the setting of Problem 1.1(a) since [10, Proposition 26.1(iv)(a)] yields $(\forall n \in \mathbb{N}) \text{Fix}(T_{1,n}T_{2,n}) = \text{zer}(A+B) \ne \emptyset$. We also observe that, in view of (4.13),

$$ (4.16) \qquad (\forall n \in \mathbb{N}) \quad \varepsilon \le \lambda_n \le \varepsilon + \frac{1-\varepsilon}{\phi_n}, \quad \frac{1-\alpha_{1,n}}{\alpha_{1,n}} = 1, \quad \text{and} \quad \frac{1-\alpha_{2,n}}{\alpha_{2,n}} \ge \varepsilon $$
which, by (1.13), yields

$$ (4.17) \qquad \sup_{n \in \mathbb{N}} \phi_n \le \frac{1 + \varepsilon}{1 + 2\varepsilon} < 1. $$

In addition, it results from (4.15) that $(\forall n \in \mathbb{N}) \lambda_n \le 1/\phi_n + \varepsilon \le 2 - \gamma_n/(2\beta) + \varepsilon \le 2+\varepsilon$.
Therefore,

$$ (4.18) \qquad \begin{cases} \sum_{n \in \mathbb{N}} \chi_n \lambda_n \|e_{1,n}\| = (2 + \varepsilon) \sum_{n \in \mathbb{N}} \chi_n \|a_n\| < +\infty, \\ \sum_{n \in \mathbb{N}} \chi_n \lambda_n \|e_{2,n}\| \le 2\beta(2 + \varepsilon) \sum_{n \in \mathbb{N}} \chi_n \|b_n\| < +\infty, \end{cases} $$

which establishes (3.32) for case (a). Altogether, (4.16), Corollary 3.2(ii), and Corollary 3.3(ii) imply that, for every $z \in \text{zer}(A+B)$,

$$ (4.19) \qquad \begin{cases} (T_{1,n} - \text{Id})T_{2,n}\bar{x}_n + (T_{2,n} - \text{Id})z = (T_{1,n} - \text{Id})T_{2,n}\bar{x}_n - (T_{1,n} - \text{Id})T_{2,n}z \to 0, \\ (T_{2,n} - \text{Id})\bar{x}_n - (T_{2,n} - \text{Id})z \to 0. \end{cases} $$

Now set

$$ (4.20) \qquad (\forall n \in \mathbb{N}) \quad y_n = J_{\gamma_n A}(\bar{x}_n - \gamma_n B\bar{x}_n), \quad u_n = \frac{\bar{x}_n - y_n}{\gamma_n} - B\bar{x}_n, \quad \text{and} \quad v_n = B\bar{x}_n, $$

and note that

$$ (4.21) \qquad (\forall n \in \mathbb{N}) \quad u_n \in A y_n. $$

(i): Let $z \in \text{zer}(A+B)$. Then (4.19) yields $J_{\gamma_n A}(\bar{x}_n - \gamma_n B\bar{x}_n) - \bar{x}_n = (T_{1,n} - \text{Id})T_{2,n}\bar{x}_n + (T_{2,n} - \text{Id})z + (T_{2,n} - \text{Id})\bar{x}_n - (T_{2,n} - \text{Id})z \to 0$.

(ii): We derive from (4.19) that

$$ (4.22) \qquad \|B\bar{x}_n - Bz\| = \gamma_n^{-1} \|T_{2,n}\bar{x}_n - \bar{x}_n - T_{2,n}z + z\| \le \varepsilon^{-1} \|T_{2,n}\bar{x}_n - \bar{x}_n - T_{2,n}z + z\| \to 0. $$

(iii): Let $(k_n)_{n \in \mathbb{N}}$ be a strictly increasing sequence in $\mathbb{N}$ and let $y \in \mathcal{H}$ be such that $\bar{x}_{k_n} \to y$. In view of Corollary 3.2(iv) in case (a), and Corollary 3.3(iv) in case (b), it remains to show that $y \in \text{zer}(A+B)$. We derive from (i) that $y_n - \bar{x}_n \to 0$. Hence $y_{k_n} \to y$. Now let $z \in \text{zer}(A+B)$. Then (ii) implies that $B\bar{x}_n \to Bz$. Altogether, $y_{k_n} \to y$, $v_{k_n} \to Bz$, $y_{k_n} - \bar{x}_{k_n} \to 0$, $u_{k_n} + v_{k_n} \to 0$, and, for every $n \in \mathbb{N}$, $u_{k_n} \in A y_{k_n}$ and $v_{k_n} \in B\bar{x}_{k_n}$. It therefore follows from Lemma 4.5(ii) that $y \in \text{zer}(A+B)$.

(iv): By (iii), there exists $x \in \text{zer}(A+B)$ such that $x_n \to x$. In addition, we derive from Corollary 3.2(iv) or Corollary 3.3(iii) that

$$ (4.23) \qquad \bar{x}_n - x_{n+1} \to 0 \quad \text{or} \quad \bar{x}_n - x_n \to 0. $$

Hence it follows from (4.20) and (i) that $y_n \to x$, and, from (4.20) and (ii), that $u_n \to -Bx$. In turn, if $A$ is demiregular on zer $(A+B)$, we derive from (4.21) that $y_n \to x$. Since $y_n - \bar{x}_n \to 0$, (4.23) yields $x_n \to x$. Now suppose that $B$ is demiregular on zer $(A+B)$. Since $\bar{x}_n \to x$, (ii) implies that $\bar{x}_n \to x$ and it follows from (4.23) that $x_n \to x$. □

Remark 4.10. As noted in Remark 3.4, we can allow errors in inertial multi-layer methods and, in particular, in the inertial forward-backward algorithm. Thus,
suppose that, in Proposition 4.9, $\lambda_n \equiv 1$ and A has bounded domain. Then $\bigcup_{n \in \mathbb{N}} \text{ran } T_{1,n} = \bigcup_{n \in \mathbb{N}} \text{ran} (\text{Id} + \gamma_n A)^{-1} = \text{dom } A$ is bounded. Hence, it follows from Remark 3.4 that, if $\sum_{n \in \mathbb{N}} \chi_n \|a_n\| < +\infty$ and $\sum_{n \in \mathbb{N}} \chi_n \|b_n\| < +\infty$, the conclusions of Proposition 4.9(b) under any of assumptions (a)-(c) of Corollary 3.3 remain true for the inertial forward-backward algorithm

$$ (4.24) \qquad \left\{ \begin{array}{l} \text{for } n = 0, 1, \dots \\ \qquad \bar{x}_n = (1 + \eta_n)x_n - \eta_n x_{n-1}, \\ \qquad x_{n+1} = J_{\gamma_n A}(\bar{x}_n - \gamma_n(B\bar{x}_n + b_n)) + a_n. \end{array} \right. $$

Remark 4.11. Let $f \in \Gamma_0(\mathcal{H})$, let $g: \mathcal{H} \to \mathbb{R}$ be convex and differentiable with a $1/\beta$-Lipschitzian gradient, and suppose that $\text{Argmin}(f+g) \neq \emptyset$. Then $\nabla g$ is $\beta$-cocoercive [10, Corollary 18.17]. Upon setting $A = \partial f$ and $B = \nabla g$ in Proposition 4.9, we see that, for every $n \in \mathbb{N}$, (4.14) becomes (4.25)

$$ x_{n+1} = \bar{x}_n + \lambda_n \left( \text{prox}_{\gamma_n f} \left( \bar{x}_n - \gamma_n (\nabla g(\bar{x}_n) + b_n) \right) + a_n - \bar{x}_n \right), \quad \text{where} \quad \bar{x}_n = \sum_{j=0}^{n} \mu_{n,j} x_j, $$

and we conclude that there exists $x \in \text{Argmin}(f+g)$ such that $x_n \rightharpoonup x$ and $\nabla g(\bar{x}_n) \to \nabla g(x)$.

Remark 4.12. Various results on the convergence of the forward-backward splitting algorithm can be recovered from Proposition 4.9.

(i) Suppose that $(\forall n \in \mathbb{N}) \lambda_n \le 1$ and $(\forall j \in \{0, \dots, n\}) \mu_{n,j} = \delta_{n,j}$. Then conditions (a)-(d) in Algorithm 1.2 hold with $\chi_n \equiv 1$ by [10, Lemma 5.31]. In turn, Proposition 4.9(a)(iii) reduces to [19, Corollary 6.5]. In the context of Remark 4.11, Proposition 4.9(a)(iii) captures [24, Theorem 3.4(i)]. In this setting, under suitable conditions on the errors, it is shown in [23, Theorem 3(vi)] that $(f+g)(x_n) - \min(f+g)(\mathcal{H}) = o(1/n)$.

(ii) In the context of Remark 4.11, let $(\eta_n)_{n \in \mathbb{N}}$ be a sequence in $[0, 1]$ that satisfies condition (b) in Corollary 3.3, and set $x_{-1} = x_0$ and $\eta_0 = 0$. If $\gamma_n \equiv \gamma_0 \le \beta$ and $\lambda_n \equiv 1$, Proposition 4.9(b)(iii) covers the scheme (4.26)

$$ (\forall n \in \mathbb{N}) \quad x_{n+1} = \text{prox}_{\gamma_0 f} \left( x_n + \eta_n (x_n - x_{n-1}) - \gamma_0 \nabla g(x_n + \eta_n (x_n - x_{n-1})) \right) $$

studied in [17, Theorem 4.1], where it was established that

$$ (4.27) \qquad \sum_{n \in \mathbb{N}} n \|x_n - x_{n-1}\|^2 < +\infty. $$

In this case, it is shown in [6, Theorem 1] that $(f+g)(x_n) - \min(f+g)(\mathcal{H}) = o(1/n^2)$.

(iii) If $\lambda_n \equiv 1$, then Proposition 4.9(b)(iii) under hypothesis (c) of Corollary 3.3 establishes a statement made in [33, Theorem 1]. Let us note, however, that the proof of [33] is not convincing as the authors appear to use the weak continuity of some operators which are merely strongly continuous.

(iv) Suppose that $(\forall n \in \mathbb{N}) \lambda_n \le (1-\varepsilon)(2+\varepsilon-\gamma_n/(2\beta))$ and $(\forall j \in \{0, \dots, n\}) \mu_{n,j} = \delta_{n,j}$. Then items (a)(iii) and (a)(iv) of Proposition 4.9 capture, respectively, items (iii) and (iv)(a)-(b) of [25, Proposition 4.4]. In addition, in the context of Remark 4.11, Proposition 4.9(a)(iii) captures [25, Proposition 4.7(iii)].
(v) Proposition 4.9 also applies to the proximal point algorithm. Indeed, when $B=0$, it suffices to allow $\beta = +\infty$ and $\alpha_{2,n} \equiv 0$, to set $1/+\infty = 0$ and $1/0 = +\infty$, and to take $(\gamma_n)_{n \in \mathbb{N}}$ in $[\varepsilon, +\infty[$. In this setting, the proof remains valid and

(a) Items (i) and (iii) of Proposition 4.9(b) under hypothesis (c) of Corollary 3.3 capture the error-free case of [2, Theorem 3.1], while Theorem 3.1 covers its general case.

(b) Let $\eta \in ]0, 1/3[$, set $\sigma = (1-3\eta)/2$ and $\vartheta = 2/3$, and suppose that $\lambda_n \equiv 1$. Then Proposition 4.9(b) under hypothesis (d) of Corollary 3.3 yields [3, Proposition 2.1].

Next, we derive from Corollary 3.2 a mean value extension of Polyak's subgradient projection method [40] (likewise, an inertial version can be derived from Corollary 3.3).

*Example 4.13.* Let $C$ be a nonempty closed convex subset of $\mathcal{H}$ with projector $P_C$, let $f: \mathcal{H} \to \mathbb{R}$ be a continuous convex function such that $\text{Argmin}_C f \neq \emptyset$ and $\theta = \min f(C)$ is known. Suppose that one of the following holds:

(i) $f$ is bounded on every bounded subset of $\mathcal{H}$.

(ii) The conjugate $f^*$ of $f$ is supercoercive, i.e., $\lim_{\|u\| \to +\infty} f^*(u)/\|u\| = +\infty$.

(iii) $\mathcal{H}$ is finite dimensional.

Let $\eta \in ]0, 1[$, let $\varepsilon \in ]0, \eta/(2+\eta)[$, let $(\mu_{n,j})_{n \in \mathbb{N}, 0 \le j \le n}$ be an array in $[0, +\infty[$ that satisfies conditions (a)-(d) in Algorithm 1.2, let $(\xi_n)_{n \in \mathbb{N}}$ be in $[\eta, 2-\eta]$, let $(\lambda_n)_{n \in \mathbb{N}}$ be in $[\varepsilon, (1-\varepsilon)(2-\xi_n/2)]$, let $s$ be a selection of $\partial f$, and let $x_0 \in C$. Iterate (4.28)

for $n=0, 1, \dots$

$$ \left\{ \begin{array}{ll} \bar{x}_n = \sum_{j=0}^{n} \mu_{n,j} x_j, & \text{if } s(\bar{x}_n) \neq 0, \\ x_{n+1} = \begin{cases} \bar{x}_n + \lambda_n \left( P_C \left( \bar{x}_n + \xi_n \frac{\theta - f(\bar{x}_n)}{\|s(\bar{x}_n)\|^2} s(\bar{x}_n) \right) - \bar{x}_n \right) & \text{if } s(\bar{x}_n) = 0. \end{cases} & \end{array} \right. $$

Then there exists $x \in \text{Argmin}_C f$ such that $x_n \to x$.

*Proof.* Let $G$ be the subgradient projector onto $D = \{x \in \mathcal{H} \mid f(x) \le \theta\}$ associated with $s$, that is,

$$ (4.29) \qquad G: \mathcal{H} \to \mathcal{H}: x \mapsto \begin{cases} x + \frac{\theta - f(x)}{\|s(x)\|^2} s(x) & \text{if } f(x) > \theta, \\ x & \text{if } f(x) \le \theta. \end{cases} $$

Then $G$ is firmly quasi-nonexpansive [10, Proposition 29.41(iii)]. Now set $(\forall n \in \mathbb{N})$ $T_{1,n} = P_C$, $T_{2,n} = \text{Id} + \xi_n(G - \text{Id})$, $\alpha_{1,n} = 1/2$, and $\alpha_{2,n} = \xi_n/2$. Then, for every $n \in \mathcal{H}$, $T_{1,n}$ is an $\alpha_{1,n}$-averaged nonexpansive operator [10, Proposition 4.16], $T_{2,n}$ is an $\alpha_{2,n}$-averaged quasi-nonexpansive operator, [10, Proposition 4.49(i)] yields

$$ (4.30) \qquad \text{Fix } T_{1,n}T_{2,n} = \text{Fix } T_{1,n} \cap \text{Fix } T_{2,n} = C \cap D = \text{Argmin}_C f, $$

and Remark 1.3(i) asserts that $T_{1,n}T_{2,n}$ is an averaged quasi-nonexpansive operator with constant $\phi_n = 2/(4-\xi_n)$ and $\lambda_n \le (1-\varepsilon)/\phi_n$. Thus, the problem of minimizing $f$ over $C$ is a special case of Problem 1.1(b) with $m=2$, and (4.28) is a special case of Algorithm 1.2 with $e_{1,n} \equiv 0$ and $e_{2,n} \equiv 0$. Now let $(k_n)_{n \in \mathbb{N}}$ be a strictly increasing sequence in $\mathbb{N}$ and let $x \in \mathcal{H}$ be such that $\bar{x}_{k_n} \to x$. Then, by Corollary 3.2(iii), it
remains to show that $x \in C \cap D$. We derive from Corollary 3.2(ii) and (4.30) that
$T_{2,k_n} \bar{x}_{k_n} - P_C T_{2,k_n} \bar{x}_{k_n} \to 0$ and $\bar{x}_{k_n} - T_{2,k_n} \bar{x}_{k_n} \to 0$. Therefore $C \ni P_C T_{2,k_n} \bar{x}_{k_n} = (P_C T_{2,k_n} \bar{x}_{k_n} - T_{2,n} \bar{x}_{k_n}) + (T_{2,k_n} \bar{x}_{k_n} - \bar{x}_{k_n}) + \bar{x}_{k_n} \rightharpoonup x$ and, since $C$ is weakly closed, $x \in C$. On the other hand, $\|G\bar{x}_n - \bar{x}_n\| = \|T_{2,n}\bar{x}_n - \bar{x}_n\|/\xi_n \le \|T_{2,n}\bar{x}_n - \bar{x}_n\|/\eta \to 0$. Since (iii) ⇒ (ii) ⇔ (i) [10, Proposition 16.20] and (i) imply that Id $-G$ is demiclosed at 0 [10, Proposition 29.41(vii)], we conclude that $x \in \text{Fix } G = D$. □

Remark 4.14. Example 4.13 reverts to Polyak’s classical result [40, Theorem 1]
in the case when (∀n ∈ N) λn = 1 and (∀j ∈ {0, . . . , n}) µn,j = δn,j. The unrelaxed
pattern λn ≡ 1 is indeed achievable because (∀n ∈ N) λn ∈ [ε, (1 − ε)(2 − ξn/2)] and
(1 − ε)(2 − ξn/2) ≥ (1 − ε)(2 − (2 − η)/2) > (1 − η/(2 + η))(1 + η/2) = 1.

REFERENCES

[1] F. ALVAREZ, *On the minimizing property of a second order dissipative system in Hilbert spaces*, SIAM J. Control Optim., 38 (2000), pp. 1102–1119.

[2] F. ALVAREZ, *Weak convergence of a relaxed and inertial hybrid projection-proximal point algorithm for maximal monotone operators in Hilbert space*, SIAM J. Optim., 14 (2004), pp. 773–782.

[3] F. ALVAREZ AND H. ATTOUCH, *An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping*, Set-Valued Anal., 9 (2001), pp. 3–11.

[4] H. ATTOUCH, L. M. BRICEÑO-ARIAS, AND P. L. COMBETTES, *A parallel splitting method for coupled monotone inclusions*, SIAM J. Control Optim., 48 (2010), pp. 3246–3270.

[5] H. ATTOUCH, X. GOUDOU, AND P. REDONT, *The heavy ball with friction method*, I. The continuous dynamical system: Global exploration of the local minima of real-valued function by asymptotic analysis of a dissipative dynamical system, Commun. Contemp. Math., 2 (2000), pp. 1–34.

[6] H. ATTOUCH AND J. PEYPOUQUET, *The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than 1/k²*, SIAM J. Optim., 26 (2016), pp. 1824–1834.

[7] J. B. BAILLON, R. E. BRUCK, AND S. REICH, *On the asymptotic behavior of nonexpansive mappings and semigroups in Banach spaces*, Houston J. Math., 4 (1978), pp. 1–9.

[8] D. BARRO AND E. CANESTRELLI, *Combining stochastic programming and optimal control to decompose multistage stochastic optimization problems*, OR Spectrum, 38 (2016), pp. 711–742.

[9] H. H. BAUSCHKE AND J. M. BORWEIN, *On projection algorithms for solving convex feasibility problems*, SIAM Rev., 38 (1996), pp. 367–426.

[10] H. H. BAUSCHKE AND P. L. COMBETTES, *Convex Analysis and Monotone Operator Theory in Hilbert Spaces*, 2nd ed., Springer, New York, 2017.

[11] E. BOREL, *Leçons sur les Séries Divergentes*, Gauthier-Villars, Paris, 1901.

[12] D. BORWEIN AND J. BORWEIN, *Fixed point iterations for real functions*, J. Math. Anal. Appl., 157 (1991), pp. 112–126.

[13] J. M. BORWEIN, G. LI, AND M. K. TAM, *Convergence rate analysis for averaged fixed point iterations in common fixed point problems*, SIAM J. Optim., 27 (2017), pp. 1–33.

[14] R. I. BOȚ, E. R. CSETNEK, AND C. HENDRICH, *Inertial Douglas-Rachford splitting for monotone inclusion problems*, Appl. Math. Comput., 256 (2015), pp. 472–487.

[15] C. BREZINSKI AND J.-P. CHEHAB, *Nonlinear hybrid procedures and fixed point iterations*, Numer. Funct. Anal. Optim., 19 (1998), pp. 465–487.

[16] C. L. BYRNE, *Iterative Optimization in Inverse Problems*, CRC Press, Boca Raton, FL, 2014.

[17] A. CHAMBOLLE AND C. DOSSAL, *On the convergence of the iterates of the "Fast iterative shrinkage/thresholding algorithm,"* J. Optim. Theory Appl., 166 (2015), pp. 968–982.

[18] P. L. COMBETTES, *Quasi-Fejérian analysis of some optimization algorithms*, in Inherently Parallel Algorithms for Feasibility and Optimization, D. Butnariu, Y. Censor, and S. Reich, eds., Elsevier, New York, 2001, pp. 115–152.

[19] P. L. COMBETTES, *Solving monotone inclusions via compositions of nonexpansive averaged operators*, Optimization, 53 (2004), pp. 475–504.

[20] P. L. COMBETTES, *Iterative construction of the resolvent of a sum of maximal monotone operators*, J. Convex Anal., 16 (2009), pp. 727–748.

[21] P. L. COMBETTES AND T. PENNANEN, *Generalized Mann iterates for constructing fixed points in Hilbert spaces*, J. Math. Anal. Appl., 275 (2002), pp. 521–536.
[22] P. L. COMBETTES AND J.-C. PESQUET, *Stochastic quasi-Fejér block-coordinate fixed point iterations with random sweeping*, SIAM J. Optim., 25 (2015), pp. 1221-1248.

[23] P. L. COMBETTES, S. SALZO, AND S. VILLA, *Consistent learning by composite proximal thresholding*, Math. Program., to appear.

[24] P. L. COMBETTES AND V. R. WAJS, *Signal recovery by proximal forward-backward splitting*, Multiscale Model. Simul., 4 (2005), pp. 1168-1200.

[25] P. L. COMBETTES AND I. YAMADA, *Compositions and convex combinations of averaged nonexpansive operators*, J. Math. Anal. Appl., 425 (2015), pp. 55-70.

[26] D. DAVIS, *Convergence rate analysis of primal-dual splitting schemes*, SIAM J. Optim., 25 (2015), pp. 1912-1943.

[27] J. B. DIAZ AND F. T. METCALF, *On the structure of the set of subsequential limit points of successive approximations*, Bull. Amer. Math. Soc. (N.S), 73 (1967), pp. 516-519.

[28] W. G. DOTSON, *On the Mann iterative process*, Trans. Amer. Math. Soc., 149 (1970), pp. 65-73.

[29] L. FEJÉR, *Untersuchungen über Fouriersche Reihen*, Math. Ann., 58 (1903), pp. 51-69.

[30] C. W. GROETSCH, *A note on segmenting Mann iterates*, J. Math. Anal. Appl., 40 (1972), pp. 369-372.

[31] K. KNOPP, *Theory and Application of Infinite Series*, Blackie & Son, London, 1954.

[32] P. KÜGLER AND A. LEITÃO, *Mean value iterations for nonlinear elliptic Cauchy problems*, Numer. Math., 96 (2003), pp. 269-293.

[33] D. A. LORENZ AND T. POCK, *An inertial forward-backward algorithm for monotone inclusions*, J. Math. Imaging Vision, 51 (2015), pp. 311-325.

[34] W. R. MANN, *Mean value methods in iteration*, Proc. Amer. Math. Soc., 4 (1953), pp. 506-510.

[35] W. R. MANN, *Averaging to improve convergence of iterative processes*, in Fundamental Analysis Methods in Numerical Analysis, Lecture Notes in Math. 701, Springer, Berlin, 1979, pp. 169-179.

[36] J. J. MOREAU, *Fonctions convexes duales et points proximaux dans un espace Hilbertien*, C. R. Acad. Sci. Paris Sér. A, 255 (1962), pp. 2897-2899.

[37] C. OUTLAW AND C. W. GROETSCH, *Averaging iteration in a Banach space*, Bull. Amer. Math. Soc. (N.S.), 75 (1969), pp. 430-432.

[38] B. T. POLYAK, *Gradient methods for minimizing functionals*, USSR Comput. Math. Phys., 3 (1963), pp. 864-878.

[39] B. T. POLYAK, *Some methods of speeding up the convergence of iteration methods*, USSR Comput. Math. Math. Phys., 4 (1964), pp. 1-17.

[40] B. T. POLYAK, *Minimization of unsmooth functionals*, USSR Comput. Math. Math. Phys., 9 (1969), pp. 14-29.

[41] B. T. POLYAK, *Comparison of the speed of convergence of single-step and multistep algorithms of optimization when there are noises*, Engrg. Cybernet., 15 (1977), pp. 6-10.

[42] B. E. RHOADES, *Fixed point iterations using infinite matrices*, Trans. Amer. Math. Soc., 196 (1974), pp. 161-176.

[43] K. SLAVAKIS AND I. YAMADA, *The adaptive projected subgradient method constrained by families of quasi-nonexpansive mappings and its application to online learning*, SIAM J. Optim., 23 (2013), pp. 126-152.

[44] O. TOEPLITZ, *Über allgemeine lineare Mittelbildungen*, Prace Mat.-Fiz., 22 (1911), pp. 113-119.