# Proper Generalized Decomposition for Nonlinear Convex Problems in Tensor Banach Spaces

Antonio Falco, Anthony Nouy

► To cite this version:

Antonio Falco, Anthony Nouy. Proper Generalized Decomposition for Nonlinear Convex Problems in Tensor Banach Spaces. Numerische Mathematik, 2012, 121 (3), pp.503-530. 10.1007/s00211-011-0437-5. hal-00609108

HAL Id: hal-00609108

https://hal.science/hal-00609108

Submitted on 3 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Proper Generalized Decomposition for Nonlinear Convex Problems in Tensor Banach Spaces

Antonio Falcó · Anthony Nouy

the date of receipt and acceptance should be inserted later

**Abstract** Tensor-based methods are receiving a growing interest in scientific computing for the numerical solution of problems defined in high dimensional tensor product spaces. A family of methods called Proper Generalized Decompositions methods have been recently introduced for the a priori construction of tensor approximations of the solution of such problems. In this paper, we give a mathematical analysis of a family of progressive and updated Proper Generalized Decompositions for a particular class of problems associated with the minimization of a convex functional over a reflexive tensor Banach space.

**Mathematics Subject Classification (2000)** 65K10,49M29

**Keywords** Nonlinear Convex Problems, Tensor Banach spaces, Proper Generalized Decomposition, High-Dimensional Problems, Greedy Algorithms

## 1 Introduction

Tensor-based methods are receiving a growing interest in scientific computing for the numerical solution of problems defined in high dimensional tensor product spaces, such as partial differential equations arising from stochastic calculus (e.g. Fokker-Planck

This work is supported by the ANR (French National Research Agency, grants ANR-2010-COSI-006 and ANR-2010-BLAN-0904) and by the PRCEU-UCH30/10 grant of the Universidad CEU Cardenal Herrera.

<footnote>A. Falcó
Departamento de Ciencias, Físicas, Matemáticas y de la Computación,
Universidad CEU Cardenal Herrera,
San Bartolomé 55
46115 Alfara del Patriarca (Valencia), Spain.
E-mail: afalco@uch.ceu.es</footnote>

<footnote>A. Nouy
LUNAM Université, GeM, UMR CNRS 6183, Ecole Centrale Nantes, Université de Nantes
1 rue de la Noë, BP 92101,
44321 Nantes Cedex 3, France.
E-mail: anthony.nouy@ec-nantes.fr</footnote>
# Proper Generalized Decomposition for Nonlinear Convex Problems in Tensor Banach Spaces

Antonio Falcó · Anthony Nouy

the date of receipt and acceptance should be inserted later

**Abstract** Tensor-based methods are receiving a growing interest in scientific computing for the numerical solution of problems defined in high dimensional tensor product spaces. A family of methods called Proper Generalized Decompositions methods have been recently introduced for the a priori construction of tensor approximations of the solution of such problems. In this paper, we give a mathematical analysis of a family of progressive and updated Proper Generalized Decompositions for a particular class of problems associated with the minimization of a convex functional over a reflexive tensor Banach space.

**Mathematics Subject Classification (2000)** 65K10,49M29

**Keywords** Nonlinear Convex Problems, Tensor Banach spaces, Proper Generalized Decomposition, High-Dimensional Problems, Greedy Algorithms

## 1 Introduction

Tensor-based methods are receiving a growing interest in scientific computing for the numerical solution of problems defined in high dimensional tensor product spaces, such as partial differential equations arising from stochastic calculus (e.g. Fokker-Planck

This work is supported by the ANR (French National Research Agency, grants ANR-2010-COSI-006 and ANR-2010-BLAN-0904) and by the PRCEU-UCH30/10 grant of the Universidad CEU Cardenal Herrera.

<footnote>A. Falcó
Departamento de Ciencias, Físicas, Matemáticas y de la Computación,
Universidad CEU Cardenal Herrera,
San Bartolomé 55
46115 Alfara del Patriarca (Valencia), Spain.
E-mail: afalco@uch.ceu.es</footnote>

<footnote>A. Nouy
LUNAM Université, GeM, UMR CNRS 6183, Ecole Centrale Nantes, Université de Nantes
1 rue de la Noë, BP 92101,
44321 Nantes Cedex 3, France.
E-mail: anthony.nouy@ec-nantes.fr</footnote>
equations) or quantum mechanics (e.g. Schrödinger equation), stochastic parametric partial differential equations in uncertainty quantification with functional approaches, and many mechanical or physical models involving extra parameters (for parametric analyses),... For such problems, classical approximation methods based on the a priori selection of approximation bases suffer from the so called “curse of dimensionality” associated with the exponential (or factorial) increase in the dimension of approximation spaces. Tensor-based methods consist in approximating the solution $\mathbf{u} \in V$ of a problem, where $V$ is a tensor space generated by $d$ vector spaces $V_j$ (assume e.g. $V_j = \mathbb{R}^{n_j})^1$, using separated representations of the form

$$ \mathbf{u} \approx \mathbf{u}_m = \sum_{i=1}^{m} w_i^{(1)} \otimes \dots \otimes w_i^{(d)}, \quad w_i^{(j)} \in V_j \quad (1) $$

where $\otimes$ represents the Kronecker product. The functions $w_i^{(j)}$ are not a priori selected but are chosen in an optimal way regarding some properties of $\mathbf{u}$.

A first family of numerical methods based on classical constructions of tensor approximations [21,17,33] have been recently investigated for the solution of high-dimensional partial differential equations [18,3,20,26]. They are based on the systematic use of tensor approximations inside classical iterative solvers. Another family of methods, called Proper Generalized Decomposition (PGD) methods [25,9,31,32,15], have been introduced for the direct construction of representations of type (1). PGD methods introduce alternative definitions of tensor approximations, not based on natural best approximation problems, for the approximation to be computable without a priori information on the solution $\mathbf{u}$. The particular structure of approximation sets allows the interpretation of PGDs as generalizations of Proper Orthogonal Decomposition (or Singular Value Decomposition, or Karhunen-Loève Decomposition) for the a priori construction of a separated representation $\mathbf{u}_m$ of the solution. They can also be interpreted as a priori model reduction techniques in the sense that they provide a way for the a priori construction of optimal reduced bases for the representation of the solution. Several definitions of PGDs have been proposed. Basic PGDs are based on a progressive construction of the sequence $\mathbf{u}_m$, where at each step, an additional elementary tensor $\otimes_{k=1}^d w_m^{(k)}$ is added to the previously computed decomposition $\mathbf{u}_{m-1}$ [24,2,28]. Progressive definitions of PGDs can thus be considered as Greedy algorithms [35] for constructing separated representations [6,1]. A possible improvement of these progressive decompositions consists in introducing some updating steps in order to capture an approximation of the optimal decomposition, which would be obtained by defining the whole set of functions simultaneously (and not progressively). For many applications, these updating strategies allow recovering good convergence properties of separated representations [29,32,31].

In [6], convergence results are given for the progressive Proper Generalized Decomposition in the case of the high-dimensional Laplacian problem. In [15], convergence is proved in the more general setting of linear elliptic variational problems in tensor Hilbert spaces. The progressive PGD is interpreted as a generalized singular value decomposition with respect to the metric induced by the operator, which is not neces-

<footnote n="1">¹ More precisely, $V$ is the closure with respect to a norm $\| \cdot \|$ of the algebraic tensor space $V = \bigotimes_{j=1}^d V_j = \operatorname{span} \{ \otimes_{j=1}^d v^{(j)} : v^{(j)} \in V_j \text{ and } 1 \le j \le d \}$</footnote>
equations) or quantum mechanics (e.g. Schrödinger equation), stochastic parametric partial differential equations in uncertainty quantification with functional approaches, and many mechanical or physical models involving extra parameters (for parametric analyses),... For such problems, classical approximation methods based on the a priori selection of approximation bases suffer from the so called “curse of dimensionality” associated with the exponential (or factorial) increase in the dimension of approximation spaces. Tensor-based methods consist in approximating the solution $\mathbf{u} \in V$ of a problem, where $V$ is a tensor space generated by $d$ vector spaces $V_j$ (assume e.g. $V_j = \mathbb{R}^{n_j})^1$, using separated representations of the form

$$ \mathbf{u} \approx \mathbf{u}_m = \sum_{i=1}^{m} w_i^{(1)} \otimes \dots \otimes w_i^{(d)}, \quad w_i^{(j)} \in V_j \quad (1) $$

where $\otimes$ represents the Kronecker product. The functions $w_i^{(j)}$ are not a priori selected but are chosen in an optimal way regarding some properties of $\mathbf{u}$.

A first family of numerical methods based on classical constructions of tensor approximations [21,17,33] have been recently investigated for the solution of high-dimensional partial differential equations [18,3,20,26]. They are based on the systematic use of tensor approximations inside classical iterative solvers. Another family of methods, called Proper Generalized Decomposition (PGD) methods [25,9,31,32,15], have been introduced for the direct construction of representations of type (1). PGD methods introduce alternative definitions of tensor approximations, not based on natural best approximation problems, for the approximation to be computable without a priori information on the solution $\mathbf{u}$. The particular structure of approximation sets allows the interpretation of PGDs as generalizations of Proper Orthogonal Decomposition (or Singular Value Decomposition, or Karhunen-Loève Decomposition) for the a priori construction of a separated representation $\mathbf{u}_m$ of the solution. They can also be interpreted as a priori model reduction techniques in the sense that they provide a way for the a priori construction of optimal reduced bases for the representation of the solution. Several definitions of PGDs have been proposed. Basic PGDs are based on a progressive construction of the sequence $\mathbf{u}_m$, where at each step, an additional elementary tensor $\otimes_{k=1}^d w_m^{(k)}$ is added to the previously computed decomposition $\mathbf{u}_{m-1}$ [24,2,28]. Progressive definitions of PGDs can thus be considered as Greedy algorithms [35] for constructing separated representations [6,1]. A possible improvement of these progressive decompositions consists in introducing some updating steps in order to capture an approximation of the optimal decomposition, which would be obtained by defining the whole set of functions simultaneously (and not progressively). For many applications, these updating strategies allow recovering good convergence properties of separated representations [29,32,31].

In [6], convergence results are given for the progressive Proper Generalized Decomposition in the case of the high-dimensional Laplacian problem. In [15], convergence is proved in the more general setting of linear elliptic variational problems in tensor Hilbert spaces. The progressive PGD is interpreted as a generalized singular value decomposition with respect to the metric induced by the operator, which is not neces-

<footnote n="1">¹ More precisely, $V$ is the closure with respect to a norm $\| \cdot \|$ of the algebraic tensor space $V = \bigotimes_{j=1}^d V_j = \operatorname{span} \{ \otimes_{j=1}^d v^{(j)} : v^{(j)} \in V_j \text{ and } 1 \le j \le d \}$</footnote>
sarily a crossnorm on the tensor product space.

In this paper, we propose a theoretical analysis of progressive and updated Proper Generalized Decompositions for a class of problems associated with the minimization of an elliptic and differentiable functional $J$,

$$J(\mathbf{u}) = \min_{\mathbf{v} \in V} J(\mathbf{v}),$$

where $V$ is a reflexive tensor Banach space. In this context, progressive PGDs consist in defining a sequence of approximations $\mathbf{u}_m \in V$ defined by

$$\mathbf{u}_m = \mathbf{u}_{m-1} + \mathbf{z}_m, \quad \mathbf{z}_m \in S_1$$

where $S_1$ is a tensor subset with suitable properties (e.g. rank-one tensors subset, Tucker tensors subset, ...), and where $\mathbf{z}_m$ is an optimal correction in $S_1$ of $\mathbf{u}_{m-1}$, defined by

$$J(\mathbf{u}_{m-1} + \mathbf{z}_m) = \min_{\mathbf{z} \in S_1} J(\mathbf{u}_{m-1} + \mathbf{z})$$

Updated progressive PGDs consist in correcting successive approximations by using the information generated in the previous steps. At step $m$, after having computed an optimal correction $\mathbf{z}_m \in S_1$ of $\mathbf{u}_{m-1}$, a linear (or affine) subspace $U_m \subset V$ such that $\mathbf{u}_{m-1} + \mathbf{z}_m \in U_m$ is generated from the previously computed information, and the next approximation $\mathbf{u}_m$ is defined by

$$J(\mathbf{u}_m) = \min_{\mathbf{v} \in U_m} J(\mathbf{v}) \le J(\mathbf{u}_{m-1} + \mathbf{z}_m)$$

The outline of the paper is as follows. In section 2, we briefly recall some classical properties of tensor Banach spaces. In particular, we introduce some assumptions on the weak topology of the tensor Banach space in order for the (updated) progressive PGDs to be well defined (properties of subsets $S_1$). In section 3, we introduce a class of convex minimization problems on Banach spaces in an abstract setting. In section 4, we introduce and analyze the progressive PGD (with or without updates) and we provide some general convergence results. While working on this paper, the authors became aware of the work [7], which provides a convergence proof for the purely progressive PGD when working on tensor Hilbert spaces. The present paper can be seen as an extension of the results of [7] to the more general framework of tensor Banach spaces and to a larger family of PGDs, including updating strategies and a general selection of tensor subsets $S_1$. In section 5, we present some classical examples of applications of the present results: best approximation in $L^p$ tensor spaces (generalizing the multidimensional singular value decomposition to $L^p$ spaces), solution of $p$-Laplacian problem, and solution of elliptic variational problems (involving inequalities or equalities).

## 2 Tensor Banach spaces

We first consider the definition of the algebraic tensor space $a \otimes_{j=1}^d V_j$ generated from Banach spaces $V_j$ ($1 \le j \le d$) equipped with norms $\|\cdot\|_j$. As underlying field we choose
sarily a crossnorm on the tensor product space.

In this paper, we propose a theoretical analysis of progressive and updated Proper Generalized Decompositions for a class of problems associated with the minimization of an elliptic and differentiable functional $J$,

$$J(\mathbf{u}) = \min_{\mathbf{v} \in V} J(\mathbf{v}),$$

where $V$ is a reflexive tensor Banach space. In this context, progressive PGDs consist in defining a sequence of approximations $\mathbf{u}_m \in V$ defined by

$$\mathbf{u}_m = \mathbf{u}_{m-1} + \mathbf{z}_m, \quad \mathbf{z}_m \in S_1$$

where $S_1$ is a tensor subset with suitable properties (e.g. rank-one tensors subset, Tucker tensors subset, ...), and where $\mathbf{z}_m$ is an optimal correction in $S_1$ of $\mathbf{u}_{m-1}$, defined by

$$J(\mathbf{u}_{m-1} + \mathbf{z}_m) = \min_{\mathbf{z} \in S_1} J(\mathbf{u}_{m-1} + \mathbf{z})$$

Updated progressive PGDs consist in correcting successive approximations by using the information generated in the previous steps. At step $m$, after having computed an optimal correction $\mathbf{z}_m \in S_1$ of $\mathbf{u}_{m-1}$, a linear (or affine) subspace $U_m \subset V$ such that $\mathbf{u}_{m-1} + \mathbf{z}_m \in U_m$ is generated from the previously computed information, and the next approximation $\mathbf{u}_m$ is defined by

$$J(\mathbf{u}_m) = \min_{\mathbf{v} \in U_m} J(\mathbf{v}) \le J(\mathbf{u}_{m-1} + \mathbf{z}_m)$$

The outline of the paper is as follows. In section 2, we briefly recall some classical properties of tensor Banach spaces. In particular, we introduce some assumptions on the weak topology of the tensor Banach space in order for the (updated) progressive PGDs to be well defined (properties of subsets $S_1$). In section 3, we introduce a class of convex minimization problems on Banach spaces in an abstract setting. In section 4, we introduce and analyze the progressive PGD (with or without updates) and we provide some general convergence results. While working on this paper, the authors became aware of the work [7], which provides a convergence proof for the purely progressive PGD when working on tensor Hilbert spaces. The present paper can be seen as an extension of the results of [7] to the more general framework of tensor Banach spaces and to a larger family of PGDs, including updating strategies and a general selection of tensor subsets $S_1$. In section 5, we present some classical examples of applications of the present results: best approximation in $L^p$ tensor spaces (generalizing the multidimensional singular value decomposition to $L^p$ spaces), solution of $p$-Laplacian problem, and solution of elliptic variational problems (involving inequalities or equalities).

## 2 Tensor Banach spaces

We first consider the definition of the algebraic tensor space $a \otimes_{j=1}^d V_j$ generated from Banach spaces $V_j$ ($1 \le j \le d$) equipped with norms $\|\cdot\|_j$. As underlying field we choose
$\mathbb{R}$, but the results hold also for $\mathbb{C}$. The suffix 'a' in $a \otimes_{j=1}^d V_j$ refers to the 'algebraic' nature. By definition, all elements of

$$ \mathbf{V} := a \bigotimes_{j=1}^{d} V_j $$

are finite linear combinations of elementary tensors $\mathbf{v} = \bigotimes_{j=1}^d v^{(j)}$ ($v^{(j)} \in V_j$).

A typical representation format is the Tucker or tensor subspace format

$$ \mathbf{u} = \sum_{i \in \mathbf{I}} \mathbf{a}_i \bigotimes_{j=1}^{d} b_{ij}^{(j)}, \qquad (2) $$

where $\mathbf{I} = I_1 \times \dots \times I_d$ is a multi-index set with $I_j = \{1, \dots, r_j\}$, $r_j \le \dim(V_j)$, $b_{ij}^{(j)} \in V_j$ ($i_j \in I_j$) are linearly independent (usually orthonormal) vectors, and $\mathbf{a}_i \in \mathbb{R}$. Here, $i_j$ are the components of $\mathbf{i} = (i_1, \dots, i_d)$. The data size is determined by the numbers $r_j$ collected in the tuple $\mathbf{r} := (r_1, \dots, r_d)$. The set of all tensors representable by (2) with fixed $\mathbf{r}$ is

$$ \mathcal{T}_{\mathbf{r}}(\mathbf{V}) := \left\{ \mathbf{v} \in \mathbf{V} : \begin{array}{l} \text{there are subspaces } U_j \subset V_j \text{ such that} \\ \dim(U_j) = r_j \text{ and } \mathbf{v} \in \mathbf{U} := a \bigotimes_{j=1}^{d} U_j. \end{array} \right\} \quad (3) $$

To simplify the notations, the set of rank-one tensors (elementary tensors) will be denoted by

$$ \mathcal{R}_1(\mathbf{V}) := \mathcal{T}_{(1, \dots, 1)}(\mathbf{V}) = \{ \bigotimes_{k=1}^{d} w^{(k)} : w^{(k)} \in V_k \}. $$

By definition, we then have $\mathbf{V} = \operatorname{span} \mathcal{R}_1(\mathbf{V})$. We also introduce the set of rank-m tensors defined by

$$ \mathcal{R}_m(\mathbf{V}) := \left\{ \sum_{i=1}^{m} \mathbf{z}_i : \mathbf{z}_i \in \mathcal{R}_1(\mathbf{V}) \right\}. $$

We say that $\mathbf{V}_{\parallel \cdot \parallel}$ is a Banach tensor space if there exists an algebraic tensor space $\mathbf{V}$ and a norm $\| \cdot \|$ on $\mathbf{V}$ such that $\mathbf{V}_{\parallel \cdot \parallel}$ is the completion of $\mathbf{V}$ with respect to the norm $\| \cdot \|$, i.e.

$$ \mathbf{V}_{\parallel \cdot \parallel} := \| \cdot \| \bigotimes_{j=1}^{d} V_j = a \overline{\bigotimes_{j=1}^{d} V_j}^{\| \cdot \|}. $$

If $\mathbf{V}_{\parallel \cdot \parallel}$ is a Hilbert space, we say that $\mathbf{V}_{\parallel \cdot \parallel}$ is a Hilbert tensor space.

## 2.1 Topological properties of Tensor Banach spaces

Observe that $\operatorname{span} \mathcal{R}_1(\mathbf{V})$ is dense in $\mathbf{V}_{\parallel \cdot \parallel}$. Since $\mathcal{R}_1(\mathbf{V}) \subset \mathcal{T}_{\mathbf{r}}(\mathbf{V})$ for all $\mathbf{r} \ge (1, 1, \dots, 1)$, then $\operatorname{span} \mathcal{T}_{\mathbf{r}}(\mathbf{V})$ is also dense in $\mathbf{V}_{\parallel \cdot \parallel}$.

Any norm $\| \cdot \|$ on $a \otimes_{j=1}^d V_j$ satisfying

$$ \| \bigotimes_{j=1}^{d} v^{(j)} \| = \prod_{j=1}^{d} \| v^{(j)} \|_{j} \quad \text{for all } v^{(j)} \in V_j \ (1 \le j \le d) \quad (4) $$

is called a crossnorm.
$\mathbb{R}$, but the results hold also for $\mathbb{C}$. The suffix 'a' in $a \otimes_{j=1}^d V_j$ refers to the 'algebraic' nature. By definition, all elements of

$$ \mathbf{V} := a \bigotimes_{j=1}^{d} V_j $$

are finite linear combinations of elementary tensors $\mathbf{v} = \bigotimes_{j=1}^{d} v^{(j)}$ ($v^{(j)} \in V_j$). A typical representation format is the Tucker or tensor subspace format

$$ \mathbf{u} = \sum_{i \in \mathbf{I}} \mathbf{a}_i \bigotimes_{j=1}^{d} b_{ij}^{(j)}, \qquad (2) $$

where $\mathbf{I} = I_1 \times \dots \times I_d$ is a multi-index set with $I_j = \{1, \dots, r_j\}$, $r_j \le \dim(V_j)$, $b_{ij}^{(j)} \in V_j$ ($i_j \in I_j$) are linearly independent (usually orthonormal) vectors, and $\mathbf{a}_i \in \mathbb{R}$. Here, $i_j$ are the components of $\mathbf{i} = (i_1, \dots, i_d)$. The data size is determined by the numbers $r_j$ collected in the tuple $\mathbf{r} := (r_1, \dots, r_d)$. The set of all tensors representable by (2) with fixed $\mathbf{r}$ is

$$ \mathcal{T}_{\mathbf{r}}(\mathbf{V}) := \left\{ \mathbf{v} \in \mathbf{V} : \begin{array}{l} \text{there are subspaces } U_j \subset V_j \text{ such that} \\ \dim(U_j) = r_j \text{ and } \mathbf{v} \in \mathbf{U} := a \bigotimes_{j=1}^{d} U_j. \end{array} \right\} \qquad (3) $$

To simplify the notations, the set of rank-one tensors (elementary tensors) will be denoted by

$$ \mathcal{R}_1(\mathbf{V}) := \mathcal{T}_{(1,\dots,1)}(\mathbf{V}) = \{ \bigotimes_{k=1}^{d} w^{(k)} : w^{(k)} \in V_k \}. $$

By definition, we then have $\mathbf{V} = \operatorname{span} \mathcal{R}_1(\mathbf{V})$. We also introduce the set of rank-m tensors defined by

$$ \mathcal{R}_m(\mathbf{V}) := \left\{ \sum_{i=1}^{m} \mathbf{z}_i : \mathbf{z}_i \in \mathcal{R}_1(\mathbf{V}) \right\}. $$

We say that $\mathbf{V}_{\parallel \cdot \parallel}$ is a Banach tensor space if there exists an algebraic tensor space $\mathbf{V}$ and a norm $\| \cdot \|$ on $\mathbf{V}$ such that $\mathbf{V}_{\parallel \cdot \parallel}$ is the completion of $\mathbf{V}$ with respect to the norm $\| \cdot \|$, i.e.

$$ \mathbf{V}_{\parallel \cdot \parallel} := \| \cdot \| \bigotimes_{j=1}^{d} V_j = a \overline{\bigotimes_{j=1}^{d} V_j}^{\| \cdot \|}. $$

If $\mathbf{V}_{\parallel \cdot \parallel}$ is a Hilbert space, we say that $\mathbf{V}_{\parallel \cdot \parallel}$ is a Hilbert tensor space.

## 2.1 Topological properties of Tensor Banach spaces

Observe that $\operatorname{span} \mathcal{R}_1(\mathbf{V})$ is dense in $\mathbf{V}_{\parallel \cdot \parallel}$. Since $\mathcal{R}_1(\mathbf{V}) \subset \mathcal{T}_{\mathbf{r}}(\mathbf{V})$ for all $\mathbf{r} \ge (1, 1, \dots, 1)$, then $\operatorname{span} \mathcal{T}_{\mathbf{r}}(\mathbf{V})$ is also dense in $\mathbf{V}_{\parallel \cdot \parallel}$.

Any norm $\| \cdot \|$ on $a \otimes_{j=1}^d V_j$ satisfying

$$ \| \bigotimes_{j=1}^{d} v^{(j)} \| = \prod_{j=1}^{d} \| v^{(j)} \|_{j} \quad \text{for all } v^{(j)} \in V_j \ (1 \le j \le d) \qquad (4) $$

is called a crossnorm.
Remark 1 Eq. (4) implies the inequality $\|\otimes_{j=1}^d v^{(j)}\| \lesssim \prod_{j=1}^d \|v^{(j)}\|_j$ which is equivalent to the continuity of the tensor product mapping

$$ \otimes : \times_{j=1}^{d} (V_j, \|\cdot\|_j) \to \left( a \otimes_{j=1}^{d} V_j, \|\cdot\| \right), \qquad (5) $$

given by $\otimes((v^{(1)}, \dots, v^{(d)}) = \otimes_{j=1}^d v^{(j)}$, where $(X, \|\cdot\|)$ denotes a vector space $X$ equipped with norm $\|\cdot\|$.

As usual, the dual norm to $\|\cdot\|$ is denoted by $\|\cdot\|^*$. If $\|\cdot\|$ is a crossnorm and also $\|\cdot\|^*$ is a crossnorm on $a \otimes_{j=1}^d V_j^*$, i.e.

$$ \|\otimes_{j=1}^d \varphi^{(j)}\|^* = \prod_{j=1}^d \|\varphi^{(j)}\|_j^* \quad \text{for all } \varphi^{(j)} \in V_j^* \quad (1 \le j \le d), \qquad (6) $$

$\|\cdot\|$ is called a reasonable crossnorm. Now, we introduce the following norm.

**Definition 1** Let $V_j$ be Banach spaces with norms $\|\cdot\|_j$ for $1 \le j \le d$. Then for $v \in V = a \otimes_{j=1}^d V_j$, we define the norm $\|\cdot\|_{\vee}$ by

$$ \|v\|_{\vee} := \sup \left\{ \frac{|(\varphi^{(1)} \otimes \varphi^{(2)} \otimes \dots \otimes \varphi^{(d)})(v)|}{\prod_{j=1}^{d} \|\varphi^{(j)}\|_{j}^{*}} : 0 \ne \varphi^{(j)} \in V_{j}^{*}, 1 \le j \le d \right\}. \qquad (7) $$

The following proposition has been proved in [14].

**Proposition 1** Let $V_{\|\cdot\|}$ be a Banach tensor space with a norm satisfying $\|\cdot\| \gtrsim \|\cdot\|_{\vee}$ on $V$. Then the set $\mathcal{T}_r(V)$ is weakly closed.

## 2.2 Examples

### 2.2.1 The Bochner spaces

Our first example, the Bochner spaces, are a generalization of the concept of $L^p$-spaces to functions whose values lie in a Banach space which is not necessarily the space $\mathbb{R}$ or $\mathbb{C}$.

Let $X$ be a Banach space endowed with a norm $\|\cdot\|_X$. Let $I \subset \mathbb{R}^s$ and $\mu$ a finite measure on $I$ (e.g. a probability measure). Let us consider the Bochner space $L_{\mu}^p(I; X)$, with $1 \le p < \infty$, defined by

$$ L_{\mu}^{p}(I; X) = \{v : I \to X : \int_{I} \|v(x)\|_{X}^{p} d\mu(x) < \infty\}, $$

and endowed with the norm

$$ \|v\|_{\Delta_p} = \left( \int_I \|v(x)\|_X^p d\mu(x) \right)^{1/p} $$

We now introduce the tensor product space $V_{\|\cdot\|_{\Delta_p}} = X \otimes_{\|\cdot\|_{\Delta_p}} L_{\mu}^p(I)$. For $1 \le p < \infty$, the space $L_{\mu}^p(I; X)$ can be identified with $V_{\|\cdot\|_{\Delta_p}}$ (see Section 7, Chapter 1 in [10]). Moreover, the following proposition can be proved (see Proposition 7.1 in [10]):
Remark 1 Eq. (4) implies the inequality $\|\otimes_{j=1}^d v^{(j)}\| \lesssim \prod_{j=1}^d \|v^{(j)}\|_j$ which is equivalent to the continuity of the tensor product mapping

$$ \otimes : \times_{j=1}^{d} (V_j, \|\cdot\|_j) \to \left( a \otimes_{j=1}^{d} V_j, \|\cdot\| \right), \qquad (5) $$

given by $\otimes(v^{(1)}, \dots, v^{(d)}) = \otimes_{j=1}^d v^{(j)}$, where $(X, \|\cdot\|)$ denotes a vector space $X$ equipped with norm $\|\cdot\|$.

As usual, the dual norm to $\|\cdot\|$ is denoted by $\|\cdot\|^*$. If $\|\cdot\|$ is a crossnorm and also $\|\cdot\|^*$ is a crossnorm on $a \otimes_{j=1}^d V_j^*$, i.e.

$$ \|\otimes_{j=1}^d \varphi^{(j)}\|^* = \prod_{j=1}^d \|\varphi^{(j)}\|_j^* \quad \text{for all } \varphi^{(j)} \in V_j^* \quad (1 \le j \le d), \qquad (6) $$

$\|\cdot\|$ is called a reasonable crossnorm. Now, we introduce the following norm.

**Definition 1** Let $V_j$ be Banach spaces with norms $\|\cdot\|_j$ for $1 \le j \le d$. Then for $v \in V = a \otimes_{j=1}^d V_j$, we define the norm $\|\cdot\|_{\vee}$ by

$$ \|v\|_{\vee} := \sup \left\{ \frac{|(\varphi^{(1)} \otimes \varphi^{(2)} \otimes \dots \otimes \varphi^{(d)})(v)|}{\prod_{j=1}^{d} \|\varphi^{(j)}\|_{j}^{*}} : 0 \ne \varphi^{(j)} \in V_{j}^{*}, 1 \le j \le d \right\}. \qquad (7) $$

The following proposition has been proved in [14].

**Proposition 1** Let $V_{\|\cdot\|}$ be a Banach tensor space with a norm satisfying $\|\cdot\| \gtrsim \|\cdot\|_{\vee}$ on $V$. Then the set $\mathcal{T}_r(V)$ is weakly closed.

## 2.2 Examples

### 2.2.1 The Bochner spaces

Our first example, the Bochner spaces, are a generalization of the concept of $L^p$-spaces to functions whose values lie in a Banach space which is not necessarily the space $\mathbb{R}$ or $\mathbb{C}$.

Let $X$ be a Banach space endowed with a norm $\|\cdot\|_X$. Let $I \subset \mathbb{R}^s$ and $\mu$ a finite measure on $I$ (e.g. a probability measure). Let us consider the Bochner space $L_{\mu}^p(I; X)$, with $1 \le p < \infty$, defined by

$$ L_{\mu}^{p}(I; X) = \{v : I \to X : \int_{I} \|v(x)\|_{X}^{p} d\mu(x) < \infty\}, $$

and endowed with the norm

$$ \|v\|_{\Delta_p} = \left( \int_I \|v(x)\|_X^p d\mu(x) \right)^{1/p} $$

We now introduce the tensor product space $V_{\|\cdot\|_{\Delta_p}} = X \otimes_{\|\cdot\|_{\Delta_p}} L_{\mu}^p(I)$. For $1 \le p < \infty$, the space $L_{\mu}^p(I; X)$ can be identified with $V_{\|\cdot\|_{\Delta_p}}$ (see Section 7, Chapter 1 in [10]). Moreover, the following proposition can be proved (see Proposition 7.1 in [10]):
**Proposition 2** For $1 \le p < \infty$, the norm $\|\cdot\|_{\Delta_p}$ satisfies $\|\cdot\|_{\Delta_p} \gtrsim \|\cdot\|_{\bigvee}$ on $X \otimes_a L_{\mu}^p(I)$.

By Propositions 2 and 1, we then conclude:

**Corollary 1** For $1 \le p < \infty$, the set $\mathcal{T}_r(X \otimes_a L_{\mu}^p(I))$ is weakly closed in $L_{\mu}^p(I; X)$. In particular, for $K \subset \mathbb{R}^k$, we have that $\mathcal{T}_r(L_{\nu}^p(K) \otimes_a L_{\mu}^p(I))$ and $\mathcal{R}_1(L_{\nu}^p(K) \otimes_a L_{\mu}^p(I))$ are weakly closed sets in $L_{\nu \otimes \mu}^p(K \times I)$.

### 2.2.2 The Sobolev spaces

Let $\Omega = \Omega_1 \times \dots \times \Omega_d \subset \mathbb{R}^d$, with $\Omega_k \subset \mathbb{R}$. Let $\alpha \in \mathbb{N}^d$ denote a multi-index and $|\alpha| = \sum_{k=1}^d \alpha_k$. $D^\alpha(\mathbf{u}) = \partial_{x_1}^{\alpha_1} \dots \partial_{x_d}^{\alpha_d}(\mathbf{u})$ denotes a partial derivative of $\mathbf{u}(x_1, \dots, x_d)$ of order $|\alpha|$. For a fixed $1 \le p < \infty$, we introduce the Sobolev space

$$H^{m,p}(\Omega) = \{\mathbf{u} \in L^p(\Omega) : D^\alpha(\mathbf{u}) \in L^p(\Omega), 0 \le |\alpha| \le m\}$$

equipped with the norm

$$\|\mathbf{v}\|_{m,p} = \sum_{0 \le |\alpha| \le m} \|D^\alpha(\mathbf{v})\|_{L^p(\Omega)}$$

We let $V_k = H^{m,p}(\Omega_k)$, endowed with norms $\|\cdot\|_{m,p;k}$ defined by

$$\|w\|_{m,p;k} = \sum_{j=0}^{m} \|\partial_{x_k}^j(w)\|_{L^p(\Omega_k)}.$$

Then we have the following equality

$$H^{m,p}(\Omega) = \|\cdot\|_{m,p} \bigotimes_{j=1}^{d} H^{m,p}(\Omega_j).$$

A first result is the following.

**Proposition 3** For $1 < p < \infty$, $m \ge 0$ and $\Omega = \Omega_1 \times \dots \times \Omega_d$, the set

$$\mathcal{R}_1 \left( a \bigotimes_{j=1}^{d} H^{m,p}(\Omega_j) \right) = \left\{ \bigotimes_{k=1}^{d} w^{(k)} : w^{(k)} \in H^{m,p}(\Omega_k) \right\},$$

is weakly closed in $(H^{m,p}(\Omega), \|\cdot\|_{m,p})$.

To prove the above proposition we need the following two lemmas.

**Lemma 1** Assume $1 < p < \infty$ and $\Omega = \Omega_1 \times \dots \times \Omega_d$. Then the set $\mathcal{R}_1(a \bigotimes_{j=1}^d L^p(\Omega_j))$ is weakly closed in $L^p(\Omega)$.

*Proof* Let $\{\mathbf{v}_n\}_{n \in \mathbb{N}}$, with $\mathbf{v}_n = \bigotimes_{j=1}^d v_n^{(j)}$, be a sequence in $\mathcal{R}_1(a \bigotimes_{j=1}^d L^p(\Omega_j))$ that weakly converges to an element $\mathbf{v} \in L^p(\Omega)$. Then the sequence $\{\mathbf{v}_n\}_{n \in \mathbb{N}}$ is bounded in $L^p(\Omega)$, and also the sequences $\{v_n^{(j)}\}_{n \in \mathbb{N}} \subset L^p(\Omega_j)$ for each $j \in \{1, 2, \dots, d\}$. Then, for each $j \in \{1, 2, \dots, d\}$, we can extract a subsequence, namely $\{v_{n_k}^{(j)}\}_{k \in \mathbb{N}}$, that weakly converges to some $v^{(j)} \in L^p(\Omega_j)$. Since weak convergence in $L^p(\Omega_j)$ implies the convergence in distributional sense, that is, the subsequence $\{v_{n_k}^{(j)}\}_{k \in \mathbb{N}}$ converges to $v^{(j)}$ in $D'(\Omega_j)$. From Proposition 6.2.3 of [4], we have that $\{\bigotimes_{j=1}^d v_{n_k}^{(j)}\}_{k \in \mathbb{N}}$ converges to $\bigotimes_{j=1}^d v^{(j)}$ in $D'(\Omega)$. By uniqueness of the limit, we obtain the desired result. $\square$
**Proposition 2** For $1 \le p < \infty$, the norm $\|\cdot\|_{\Delta_p}$ satisfies $\|\cdot\|_{\Delta_p} \gtrsim \|\cdot\|_{\bigvee}$ on $X \otimes_a L_{\mu}^p(I)$.

By Propositions 2 and 1, we then conclude:

**Corollary 1** For $1 \le p < \infty$, the set $\mathcal{T}_r(X \otimes_a L_{\mu}^p(I))$ is weakly closed in $L_{\mu}^p(I; X)$. In particular, for $K \subset \mathbb{R}^k$, we have that $\mathcal{T}_r(L_{\nu}^p(K) \otimes_a L_{\mu}^p(I))$ and $\mathcal{R}_1(L_{\nu}^p(K) \otimes_a L_{\mu}^p(I))$ are weakly closed sets in $L_{\nu \otimes \mu}^p(K \times I)$.

### 2.2.2 The Sobolev spaces

Let $\Omega = \Omega_1 \times \dots \times \Omega_d \subset \mathbb{R}^d$, with $\Omega_k \subset \mathbb{R}$. Let $\alpha \in \mathbb{N}^d$ denote a multi-index and $|\alpha| = \sum_{k=1}^d \alpha_k$. $D^\alpha(\mathbf{u}) = \partial_{x_1}^{\alpha_1} \dots \partial_{x_d}^{\alpha_d}(\mathbf{u})$ denotes a partial derivative of $\mathbf{u}(x_1, \dots, x_d)$ of order $|\alpha|$. For a fixed $1 \le p < \infty$, we introduce the Sobolev space

$$H^{m,p}(\Omega) = \{\mathbf{u} \in L^p(\Omega) : D^\alpha(\mathbf{u}) \in L^p(\Omega), 0 \le |\alpha| \le m\}$$

equipped with the norm

$$\|\mathbf{v}\|_{m,p} = \sum_{0 \le |\alpha| \le m} \|D^\alpha(\mathbf{v})\|_{L^p(\Omega)}$$

We let $V_k = H^{m,p}(\Omega_k)$, endowed with norms $\|\cdot\|_{m,p;k}$ defined by

$$\|w\|_{m,p;k} = \sum_{j=0}^{m} \|\partial_{x_k}^j (w)\|_{L^p(\Omega_k)}.$$

Then we have the following equality

$$H^{m,p}(\Omega) = \|\cdot\|_{m,p} \bigotimes_{j=1}^{d} H^{m,p}(\Omega_j).$$

A first result is the following.

**Proposition 3** For $1 < p < \infty$, $m \ge 0$ and $\Omega = \Omega_1 \times \dots \times \Omega_d$, the set

$$\mathcal{R}_1 \left( a \bigotimes_{j=1}^{d} H^{m,p}(\Omega_j) \right) = \left\{ \bigotimes_{k=1}^{d} w^{(k)} : w^{(k)} \in H^{m,p}(\Omega_k) \right\},$$

is weakly closed in $(H^{m,p}(\Omega), \|\cdot\|_{m,p})$.

To prove the above proposition we need the following two lemmas.

**Lemma 1** Assume $1 < p < \infty$ and $\Omega = \Omega_1 \times \dots \times \Omega_d$. Then the set $\mathcal{R}_1(a \bigotimes_{j=1}^d L^p(\Omega_j))$ is weakly closed in $L^p(\Omega)$.

*Proof* Let $\{\mathbf{v}_n\}_{n \in \mathbb{N}}$, with $\mathbf{v}_n = \bigotimes_{j=1}^d v_n^{(j)}$, be a sequence in $\mathcal{R}_1(a \bigotimes_{j=1}^d L^p(\Omega_j))$ that weakly converges to an element $\mathbf{v} \in L^p(\Omega)$. Then the sequence $\{\mathbf{v}_n\}_{n \in \mathbb{N}}$ is bounded in $L^p(\Omega)$, and also the sequences $\{v_n^{(j)}\}_{n \in \mathbb{N}} \subset L^p(\Omega_j)$ for each $j \in \{1, 2, \dots, d\}$. Then, for each $j \in \{1, 2, \dots, d\}$, we can extract a subsequence, namely $\{v_{n_k}^{(j)}\}_{k \in \mathbb{N}}$, that weakly converges to some $v^{(j)} \in L^p(\Omega_j)$. Since weak convergence in $L^p(\Omega_j)$ implies the convergence in distributional sense, that is, the subsequence $\{v_{n_k}^{(j)}\}_{k \in \mathbb{N}}$ converges to $v^{(j)}$ in $D'(\Omega_j)$. From Proposition 6.2.3 of [4], we have that $\{\bigotimes_{j=1}^d v_{n_k}^{(j)}\}_{k \in \mathbb{N}}$ converges to $\bigotimes_{j=1}^d v^{(j)}$ in $D'(\Omega)$. By uniqueness of the limit, we obtain the desired result. □
**Lemma 2** Assume $1 < p < \infty$, $m \ge 1$ and $\Omega = \Omega_1 \times \dots \times \Omega_d$. For any measurable functions $w_k: \Omega_k \to \mathbb{R}$ such that $\otimes_{k=1}^d w_k \neq 0$, we have $\otimes_{k=1}^d w_k \in H^{m,p}(\Omega)$ if and only if $w_k \in H^{m,p}(\Omega_k)$ for all $k \in \{1 \dots d\}$.

*Proof* Suppose that $w_k \in H^{m,p}(\Omega_k)$ for all $k \in \{1 \dots d\}$. Since

$$ \begin{aligned} \| \otimes_{k=1}^d w_k \|_{m,p} &= \sum_{0 \le |\alpha| \le m} \prod_{k=1}^d \| \partial_{x_k}^{\alpha_k}(w_k) \|_{L^p(\Omega_k)} \\ &\le \sum_{\alpha \in \{0, \dots, m\}^d} \prod_{k=1}^d \| \partial_{x_k}^{\alpha_k}(w_k) \|_{L^p(\Omega_k)} \\ &= \prod_{k=1}^d \left( \sum_{j=0}^m \| \partial_{x_k}^j(w_k) \|_{L^p(\Omega_k)} \right) = \prod_{k=1}^d \| w_k \|_{m,p;k} \end{aligned} $$

we have $\otimes_{k=1}^d w_k \in H^{m,p}(\Omega)$.

Conversely, if $\otimes_{k=1}^d w_k \in H^{m,p}(\Omega)$, then

$$ \| \otimes_{k=1}^d w_k \|_{m,p} = \sum_{0 \le |\alpha| \le m} \| D^\alpha (\otimes_{k=1}^d w_k) \|_{L^p(\Omega)} < \infty $$

which implies that $\|D^\alpha(\otimes_{k=1}^d w_k)\|_{L^p(\Omega)} < \infty$ for all $\alpha$ such that $0 \le |\alpha| \le m$. Taking $\alpha = (0, \dots, 0)$, we obtain

$$ \| \otimes_{k=1}^d w_k \|_{L^p(\Omega)} = \prod_{k=1}^d \| w_k \|_{L^p(\Omega_k)} < \infty $$

and therefore $\|w_k\|_{L^p(\Omega_k)} < \infty$ for all $k$. Now, for $k \in \{1 \dots d\}$, taking $\alpha = (\dots, 0, j, 0, \dots)$ such that $\alpha_k = j$, with $1 \le j \le m$, and $\alpha_l = 0$ for $l \ne k$, we obtain

$$ \| D^\alpha (\otimes_{l=1}^d w_l) \|_{L^p(\Omega)} = \| \partial_{x_k}^j w_k \|_{L^p(\Omega_k)} \prod_{l \ne k} \| w_l \|_{L^p(\Omega_l)} $$

and then $\|\partial_{x_k}^j w_k\|_{p,\Omega_k} < \infty$ for all $j \in \{1, \dots, m\}$. Therefore $w_k \in H^{m,p}(\Omega_k)$ for all $k \in \{1 \dots d\}$. $\square$

*Proof of Proposition 3* For $m=0$ the proposition follows from Lemma 1. Now, assume $m \ge 1$, and let us consider a sequence

$$ \{\mathbf{z}_n\}_{n \in \mathbb{N}} \subset \mathcal{R}_1 \left( a \bigotimes_{j=1}^d H^{m,p}(\Omega_j) \right) $$

that weakly converges to an element $\mathbf{z} \in H^{m,p}(\Omega)$. Since

$$ \mathcal{R}_1 \left( a \bigotimes_{j=1}^d H^{m,p}(\Omega_j) \right) \subset \mathcal{R}_1 \left( a \bigotimes_{j=1}^d L^p(\Omega_j) \right), $$

we have $\mathbf{z} \in \mathcal{R}_1(a \bigotimes_{j=1}^d L^p(\Omega_j))$ because, from Lemma 1, the latter set is weakly closed in $L^p(\Omega)$. Therefore, there exist $w_k \in L^p(\Omega_k)$ such that $\mathbf{z} = \otimes_{k=1}^d w_k$. Since
**Lemma 2** Assume $1 < p < \infty$, $m \ge 1$ and $\Omega = \Omega_1 \times \dots \times \Omega_d$. For any measurable functions $w_k: \Omega_k \to \mathbb{R}$ such that $\otimes_{k=1}^d w_k \neq 0$, we have $\otimes_{k=1}^d w_k \in H^{m,p}(\Omega)$ if and only if $w_k \in H^{m,p}(\Omega_k)$ for all $k \in \{1 \dots d\}$.

*Proof* Suppose that $w_k \in H^{m,p}(\Omega_k)$ for all $k \in \{1 \dots d\}$. Since

$$ \begin{aligned} \| \otimes_{k=1}^d w_k \|_{m,p} &= \sum_{0 \le |\alpha| \le m} \prod_{k=1}^d \| \partial_{x_k}^{\alpha_k}(w_k) \|_{L^p(\Omega_k)} \\ &\le \sum_{\alpha \in \{0, \dots, m\}^d} \prod_{k=1}^d \| \partial_{x_k}^{\alpha_k}(w_k) \|_{L^p(\Omega_k)} \\ &= \prod_{k=1}^d \left( \sum_{j=0}^m \| \partial_{x_k}^j(w_k) \|_{L^p(\Omega_k)} \right) = \prod_{k=1}^d \| w_k \|_{m,p;k}, \end{aligned} $$

we have $\otimes_{k=1}^d w_k \in H^{m,p}(\Omega)$.

Conversely, if $\otimes_{k=1}^d w_k \in H^{m,p}(\Omega)$, then

$$ \| \otimes_{k=1}^d w_k \|_{m,p} = \sum_{0 \le |\alpha| \le m} \| D^\alpha (\otimes_{k=1}^d w_k) \|_{L^p(\Omega)} < \infty $$

which implies that $\|D^\alpha(\otimes_{k=1}^d w_k)\|_{L^p(\Omega)} < \infty$ for all $\alpha$ such that $0 \le |\alpha| \le m$. Taking $\alpha = (0, \dots, 0)$, we obtain

$$ \| \otimes_{k=1}^d w_k \|_{L^p(\Omega)} = \prod_{k=1}^d \| w_k \|_{L^p(\Omega_k)} < \infty $$

and therefore $\|w_k\|_{L^p(\Omega_k)} < \infty$ for all $k$. Now, for $k \in \{1 \dots d\}$, taking $\alpha = (\dots, 0, j, 0, \dots)$ such that $\alpha_k = j$, with $1 \le j \le m$, and $\alpha_l = 0$ for $l \ne k$, we obtain

$$ \| D^\alpha (\otimes_{l=1}^d w_l) \|_{L^p(\Omega)} = \| \partial_{x_k}^j w_k \|_{L^p(\Omega_k)} \prod_{l \ne k} \| w_l \|_{L^p(\Omega_l)} $$

and then $\|\partial_{x_k}^j w_k\|_{p,\Omega_k} < \infty$ for all $j \in \{1, \dots, m\}$. Therefore $w_k \in H^{m,p}(\Omega_k)$ for all $k \in \{1 \dots d\}$. $\square$

*Proof of Proposition 3* For $m=0$ the proposition follows from Lemma 1. Now, assume $m \ge 1$, and let us consider a sequence

$$ \{\mathbf{z}_n\}_{n \in \mathbb{N}} \subset \mathcal{R}_1 \left( a \bigotimes_{j=1}^d H^{m,p}(\Omega_j) \right) $$

that weakly converges to an element $\mathbf{z} \in H^{m,p}(\Omega)$. Since

$$ \mathcal{R}_1 \left( a \bigotimes_{j=1}^d H^{m,p}(\Omega_j) \right) \subset \mathcal{R}_1 \left( a \bigotimes_{j=1}^d L^p(\Omega_j) \right), $$

we have $\mathbf{z} \in \mathcal{R}_1(a \bigotimes_{j=1}^d L^p(\Omega_j))$ because, from Lemma 1, the latter set is weakly closed in $L^p(\Omega)$. Therefore, there exist $w_k \in L^p(\Omega_k)$ such that $\mathbf{z} = \otimes_{k=1}^d w_k$. Since
$\mathbf{z} \in H^{m,p}(\Omega)$, from Lemma 2, $w_k \in H^{m,p}(\Omega_k)$ for $1 \le k \le d$, and therefore $\mathbf{z} = \bigotimes_{k=1}^d w_k \in \mathbb{R}_1 (a \otimes_{j=1}^d H^{m,p}(\Omega_j))$. $\square$

From [14] it follows the following statement.

**Proposition 4** The set $\mathcal{T}_r (a \otimes_{j=1}^d H^{m,2}(\Omega_j))$ is weakly closed in $H^{m,2}(\Omega)$.

### 3 Optimization of functionals over Banach spaces

Let $V$ be a reflexive Banach space, endowed with a norm $\| \cdot \|$. We denote by $V^*$ the dual space of $V$ and we denote by $\langle \cdot, \cdot \rangle : V^* \times V \to \mathbb{R}$ the duality pairing. We consider the optimization problem

$$J(u) = \min_{v \in V} J(v) \qquad (\pi)$$

where $J: V \to \mathbb{R}$ is a given functional.

#### 3.1 Some useful results on minimization of functionals over Banach spaces

In the sequel, we will introduce approximations of $(\pi)$ by considering an optimization on subsets $M \subset V$, i.e.

$$\inf_{v \in M} J(v) \qquad (8)$$

We here recall classical theorems for the existence of a minimizer (see e.g. [13]).

We recall that a sequence $v_m \in V$ is weakly convergent if $\lim_{m \to \infty} \langle \varphi, v_m \rangle$ exists for all $\varphi \in V^*$. We say that $(v_m)_{m \in \mathbb{N}}$ converges weakly to $v \in V$ if $\lim_{m \to \infty} \langle \varphi, v_m \rangle = \langle \varphi, v \rangle$ for all $\varphi \in V^*$. In this case, we write $v_m \to v$.

**Definition 2** A subset $M \subset V$ is called weakly closed if $v_m \in M$ and $v_m \to v$ implies $v \in M$.

Note that 'weakly closed' is stronger than 'closed', i.e., $M$ weakly closed $\Rightarrow M$ closed.

**Definition 3** We say that a map $J: V \to \mathbb{R}$ is weakly sequentially lower semicontinuous (respectively, weakly sequentially continuous) in $M \subset V$ if for all $v \in M$ and for all $v_m \in M$ such that $v_m \to v$, it holds $J(v) \le \liminf_{m \to \infty} J(v_m)$ (respectively, $J(v) = \lim_{m \to \infty} J(v_m)$).

If $J': V \to V^*$ exists as Gateaux derivative, we say that $J'$ is strongly continuous when for any sequence $v_n \to v$ in $V$ it holds that $J'(v_n) \to J'(v)$ in $V^*$.

Recall that the convergence in norm implies the weak convergence. Thus, $J$ weakly sequentially (lower semi)continuous in $M \Rightarrow J$ (lower semi)continuous in $M$. It can be shown (see Proposition 41.8 and Corollary 41.9 in [37]) the following result.

**Proposition 5** Let $V$ be a reflexive Banach space and let $J: V \to \mathbb{R}$ be a functional, then the following statements hold.
$\mathbf{z} \in H^{m,p}(\Omega)$, from Lemma 2, $w_k \in H^{m,p}(\Omega_k)$ for $1 \le k \le d$, and therefore $\mathbf{z} = \bigotimes_{k=1}^{d} w_k \in \mathcal{R}_1 (a \bigotimes_{j=1}^{d} H^{m,p}(\Omega_j))$. $\square$

From [14] it follows the following statement.

**Proposition 4** The set $\mathcal{T}_r (a \bigotimes_{j=1}^{d} H^{m,2}(\Omega_j))$ is weakly closed in $H^{m,2}(\Omega)$.

### 3 Optimization of functionals over Banach spaces

Let $V$ be a reflexive Banach space, endowed with a norm $\| \cdot \|$. We denote by $V^*$ the dual space of $V$ and we denote by $\langle \cdot, \cdot \rangle : V^* \times V \to \mathbb{R}$ the duality pairing. We consider the optimization problem

$$J(u) = \min_{v \in V} J(v) \qquad (\pi)$$

where $J: V \to \mathbb{R}$ is a given functional.

#### 3.1 Some useful results on minimization of functionals over Banach spaces

In the sequel, we will introduce approximations of $(\pi)$ by considering an optimization on subsets $M \subset V$, i.e.

$$\inf_{v \in M} J(v) \qquad (8)$$

We here recall classical theorems for the existence of a minimizer (see e.g. [13]).

We recall that a sequence $v_m \in V$ is weakly convergent if $\lim_{m \to \infty} \langle \varphi, v_m \rangle$ exists for all $\varphi \in V^*$. We say that $(v_m)_{m \in \mathbb{N}}$ converges weakly to $v \in V$ if $\lim_{m \to \infty} \langle \varphi, v_m \rangle = \langle \varphi, v \rangle$ for all $\varphi \in V^*$. In this case, we write $v_m \to v$.

**Definition 2** A subset $M \subset V$ is called weakly closed if $v_m \in M$ and $v_m \to v$ implies $v \in M$.

Note that 'weakly closed' is stronger than 'closed', i.e., $M$ weakly closed $\Rightarrow M$ closed.

**Definition 3** We say that a map $J: V \to \mathbb{R}$ is weakly sequentially lower semicontinuous (respectively, weakly sequentially continuous) in $M \subset V$ if for all $v \in M$ and for all $v_m \in M$ such that $v_m \to v$, it holds $J(v) \le \liminf_{m \to \infty} J(v_m)$ (respectively, $J(v) = \lim_{m \to \infty} J(v_m)$).

If $J': V \to V^*$ exists as Gateaux derivative, we say that $J'$ is strongly continuous when for any sequence $v_n \to v$ in $V$ it holds that $J'(v_n) \to J'(v)$ in $V^*$.

Recall that the convergence in norm implies the weak convergence. Thus, $J$ weakly sequentially (lower semi)continuous in $M \Rightarrow J$ (lower semi)continuous in $M$. It can be shown (see Proposition 41.8 and Corollary 41.9 in [37]) the following result.

**Proposition 5** Let $V$ be a reflexive Banach space and let $J: V \to \mathbb{R}$ be a functional, then the following statements hold.
(a) If $J$ is a convex and lower semicontinuous functional, then $J$ is weakly sequentially lower semicontinuous.

(b) If $J' : V \to V^*$ exists on $V$ as Gateaux derivative and is strongly continuous (or compact), then $J$ is weakly sequentially continuous.

Finally, we have the following two useful theorems.

**Theorem 1** Assume $V$ is a reflexive Banach space, and assume $M \subset V$ is bounded and weakly closed. If $J : M \to \mathbb{R} \cup \{\infty\}$ is weakly sequentially lower semicontinuous, then problem (8) has a solution.

*Proof* Let $\alpha = \inf_{v \in A} J(v)$ and $\{v_n\}_{n \in \mathbb{N}}$ be a minimizing sequence. Since $A$ is bounded, $\{v_n\}_{n \in \mathbb{N}}$ is a bounded sequence in a reflexive Banach space and therefore, there exists a subsequence $\{v_{n_k}\}_{k \in \mathbb{N}}$ that converges weakly to an element $u \in V$. Since $A$ is weakly closed, $u \in A$ and since $J$ is weakly sequentially lower semicontinuous, $J(u) \le \liminf_{k \to \infty} J(v_{n_k}) = \alpha$. Therefore, $J(u) = \alpha$ and $u$ is solution of the minimization problem. $\square$

We now remove the assumption that $M$ is bounded by adding a coercivity condition on $J$.

**Theorem 2** Assume $V$ is a reflexive Banach space, and $M \subset V$ is weakly closed. If $J : M \to \mathbb{R} \cup \{\infty\}$ is weakly sequentially lower semicontinuous and coercive on $M$, then problem (8) has a solution.

*Proof* Pick an element $v_0 \in M$ such that $J(v_0) \ne \infty$ and define $M_0 = \{v \in M : J(v) \le J(v_0)\}$. Since $J$ is coercive, $M_0$ is bounded. Since $M$ is weakly closed and $J$ is weakly sequentially lower semicontinuous, $M_0$ is weakly closed. The initial problem is then equivalent to $\inf_{v \in M_0} J(v)$, which admits a solution from Theorem 1. $\square$

## 3.2 Convex optimization in Banach spaces

From now one, we will assume that the functional $J$ satisfies the following assumptions.

(A1) $J$ is Fréchet differentiable, with Fréchet differential $J' : V \to V^*$.

(A2) $J$ is elliptic, i.e. there exist $\alpha > 0$ and $s > 1$ such that for all $v, w \in V$;

$$ \langle J'(v) - J'(w), v - w \rangle \ge \alpha \|v - w\|^s \quad (9) $$

In the following, $s$ will be called the ellipticity exponent of $J$.

**Lemma 3** Under assumptions (A1)-(A2), we have

(a) For all $v, w \in V$,

$$ J(v) - J(w) \ge \langle J'(w), v - w \rangle + \frac{\alpha}{s} \|v - w\|^s. \quad (10) $$

(b) $J$ is strictly convex.

(c) $J$ is bounded from below and coercive, i.e. $\lim_{\|v\| \to \infty} J(v) = +\infty$.
(a) If $J$ is a convex and lower semicontinuous functional, then $J$ is weakly sequentially lower semicontinuous.

(b) If $J' : V \to V^*$ exists on $V$ as Gateaux derivative and is strongly continuous (or compact), then $J$ is weakly sequentially continuous.

Finally, we have the following two useful theorems.

**Theorem 1** Assume $V$ is a reflexive Banach space, and assume $M \subset V$ is bounded and weakly closed. If $J : M \to \mathbb{R} \cup \{\infty\}$ is weakly sequentially lower semicontinuous, then problem (8) has a solution.

*Proof* Let $\alpha = \inf_{v \in A} J(v)$ and $\{v_n\}_{n \in \mathbb{N}}$ be a minimizing sequence. Since $A$ is bounded, $\{v_n\}_{n \in \mathbb{N}}$ is a bounded sequence in a reflexive Banach space and therefore, there exists a subsequence $\{v_{n_k}\}_{k \in \mathbb{N}}$ that converges weakly to an element $u \in V$. Since $A$ is weakly closed, $u \in A$ and since $J$ is weakly sequentially lower semicontinuous, $J(u) \le \liminf_{k \to \infty} J(v_{n_k}) = \alpha$. Therefore, $J(u) = \alpha$ and $u$ is solution of the minimization problem. $\square$

We now remove the assumption that $M$ is bounded by adding a coercivity condition on $J$.

**Theorem 2** Assume $V$ is a reflexive Banach space, and $M \subset V$ is weakly closed. If $J : M \to \mathbb{R} \cup \{\infty\}$ is weakly sequentially lower semicontinuous and coercive on $M$, then problem (8) has a solution.

*Proof* Pick an element $v_0 \in M$ such that $J(v_0) \ne \infty$ and define $M_0 = \{v \in M : J(v) \le J(v_0)\}$. Since $J$ is coercive, $M_0$ is bounded. Since $M$ is weakly closed and $J$ is weakly sequentially lower semicontinuous, $M_0$ is weakly closed. The initial problem is then equivalent to $\inf_{v \in M_0} J(v)$, which admits a solution from Theorem 1. $\square$

## 3.2 Convex optimization in Banach spaces

From now one, we will assume that the functional $J$ satisfies the following assumptions.

(A1) $J$ is Fréchet differentiable, with Fréchet differential $J' : V \to V^*$.

(A2) $J$ is elliptic, i.e. there exist $\alpha > 0$ and $s > 1$ such that for all $v, w \in V$;

$$ \langle J'(v) - J'(w), v - w \rangle \ge \alpha \|v - w\|^s \quad (9) $$

In the following, $s$ will be called the ellipticity exponent of $J$.

**Lemma 3** Under assumptions (A1)-(A2), we have

(a) For all $v, w \in V$,

$$ J(v) - J(w) \ge \langle J'(w), v - w \rangle + \frac{\alpha}{s} \|v - w\|^s. \quad (10) $$

(b) $J$ is strictly convex.

(c) $J$ is bounded from below and coercive, i.e. $\lim_{\|v\| \to \infty} J(v) = +\infty$.
Proof (a) For all $v, w \in V$,

$$
\begin{align*}
J(v) - J(w) &= \int_0^1 \frac{d}{dt} J(w + t(v-w))dt = \int_0^1 \langle J'(w + t(v-w)), v-w \rangle dt \\
&= \langle J'(w), v-w \rangle + \int_0^1 \langle J'(w + t(v-w)) - J'(w), v-w \rangle dt \\
&\geq \langle J'(w), v-w \rangle + \int_0^1 \frac{\alpha}{t} \|t(v-w)\|^s dt \\
&= \langle J'(w), v-w \rangle + \frac{\alpha}{s} \|v-w\|^s
\end{align*}
$$

(b) From (a), we have for $v \neq w$,

$$
J(v) - J(w) > \langle J'(w), v - w \rangle
$$

(c) Still from (a), we have for all $v \in V$,

$$
J(v) \geq J(0) + \langle J'(0), v \rangle + \frac{\alpha}{s} \|v\|^s \geq J(0) - \|J'(0)\| \|v\| + \frac{\alpha}{s} \|v\|^s
$$

which gives the coercivity and the fact that $J$ is bounded from below.

<figure><img src="image_1.png" /></figure>

The above properties yield the following classical result.

**Theorem 3** Under assumptions (A1)-(A2), the problem $(\pi)$ admits a unique solution $u \in V$ which is equivalently characterized by

$$
\langle J'(u), v \rangle = 0 \quad \forall v \in V \tag{11}
$$

*Proof* We here only give a sketch of proof of this very classical result. $J$ is continuous and a fortiori lower semicontinuous. Since $J$ is convex and lower semicontinuous, it is also weakly sequentially lower semicontinuous (Proposition 5(a)). The existence of a solution then follows from Theorem 2. The uniqueness is given by the strict convexity of $J$, and the equivalence between $(\pi)$ and (11) classically follows from the differentiability of $J$. $\square$

**Lemma 4** Assume that $J$ satisfies (A1)-(A2). If $\{v_m\} \subset V$ is a sequence such that $J(v_m) \xrightarrow{m \to \infty} J(u)$, where $u$ is the solution of $(\pi)$, then $v_m \to u$, i.e.

$$
\|u - v_m\| \xrightarrow{m \to \infty} 0
$$

*Proof* By the ellipticity property (10) of $J$, we have

$$
J(v_m) - J(u) \geq \langle J'(u), v_m - u \rangle + \frac{\alpha}{s} \|u - v_m\|^s = \frac{\alpha}{s} \|u - v_m\|^s. \quad (12)
$$

Therefore,

$$
\frac{\alpha}{s} \|u - v_m\|^s \leq J(v_m) - J(u) \xrightarrow{m \to \infty} 0,
$$

which ends the proof. $\square$
Proof (a) For all $v, w \in V$,

$$
\begin{align*}
J(v) - J(w) &= \int_0^1 \frac{d}{dt} J(w + t(v-w)) dt = \int_0^1 \langle J'(w + t(v-w)), v-w \rangle dt \\
&= \langle J'(w), v-w \rangle + \int_0^1 \langle J'(w + t(v-w)), J'(w), v-w \rangle dt \\
&\geq \langle J'(w), v-w \rangle + \int_0^1 \frac{\alpha}{t} \|t(v-w)\|^s dt \\
&= \langle J'(w), v-w \rangle + \frac{\alpha}{s} \|v-w\|^s
\end{align*}
$$

(b) From (a), we have for $v \neq w$,

$$
J(v) - J(w) > \langle J'(w), v - w \rangle
$$

(c) Still from (a), we have for all $v \in V$,

$$
J(v) \geq J(0) + \langle J'(0), v \rangle + \frac{\alpha}{s} \|v\|^s \geq J(0) - \|J'(0)\| \|v\| + \frac{\alpha}{s} \|v\|^s
$$

which gives the coercivity and the fact that $J$ is bounded from below.

<figure><img src="image_2.png" /></figure>

The above properties yield the following classical result.

**Theorem 3** Under assumptions (A1)-(A2), the problem $(\pi)$ admits a unique solution $u \in V$ which is equivalently characterized by

$$
\langle J'(u), v \rangle = 0 \quad \forall v \in V \tag{11}
$$

*Proof* We here only give a sketch of proof of this very classical result. *J* is continuous and a fortiori lower semicontinuous. Since *J* is convex and lower semicontinuous, it is also weakly sequentially lower semicontinuous (Proposition 5(a)). The existence of a solution then follows from Theorem 2. The uniqueness is given by the strict convexity of *J*, and the equivalence between $(\pi)$ and (11) classically follows from the differentiability of *J*. □

**Lemma 4** Assume that $J$ satisfies (A1)-(A2). If $\{v_m\} \subset V$ is a sequence such that $J(v_m) \underset{m\to\infty}{\longrightarrow} J(u)$, where $u$ is the solution of $(\pi)$, then $v_m \to u$, i.e.

$$
\|u - v_m\| \xrightarrow{m \to \infty} 0
$$

*Proof* By the ellipticity property (10) of $J$, we have

$$
J(v_m) - J(u) \geq \langle J'(u), v_m - u \rangle + \frac{\alpha}{s} \|u - v_m\|^s = \frac{\alpha}{s} \|u - v_m\|^s. \quad (12)
$$

Therefore,

$$
\frac{\alpha}{s} \|u - v_m\|^s \leq J(v_m) - J(u) \xrightarrow{m \to \infty} 0,
$$

which ends the proof. □
# 4 Progressive Proper Generalized Decompositions in Tensor Banach Spaces

## 4.1 Definition of progressive Proper Generalized Decompositions

We now consider the minimization problem $(\pi)$ of functional $J$ on a reflexive tensor Banach space $V = V_{\parallel,\parallel}$. Assume that we have a functional $J: V_{\parallel,\parallel} \to \mathbb{R}$ satisfying (A1)-(A2) and a weakly closed subset $S_1$ in $V_{\parallel,\parallel}$ such that

(B1) $S_1 \subset V$, with $0 \in S_1$,

(B2) for each $v \in S_1$ we have $\lambda v \in S_1$ for all $\lambda \in \mathbb{R}$, and

(B3) $\text{span} S_1$ is dense in $V_{\parallel,\parallel}$.

By using the notation introduced in Section 2.2 we give the following examples.

**Example 1** Consider $V_{\parallel,\parallel} = L_{\mu}^{p}(I; X)$ and $S_1 = \mathcal{T}_r(X \otimes_a L_{\mu}^{p}(I))$.

**Example 2** Consider $V_{\parallel,\parallel} = H^{m,2}(\Omega)$ and $S_1 = \mathcal{T}_r(a \otimes_{j=1}^{d} H^{m,2}(\Omega_j))$.

**Example 3** Consider $V_{\parallel,\parallel} = H^{m,p}(\Omega)$ and $S_1 = \mathcal{R}_1(a \otimes_{j=1}^{d} H^{m,p}(\Omega_j))$.

The set $S_1$ can be used to characterize the solution of problem $(\pi)$ as shown by the following result.

**Lemma 5** Assume that $J$ satisfies (A1)-(A2) and let $u^* \in V_{\parallel,\parallel}$ satisfying

$$J(\mathbf{u}^*) = \min_{\mathbf{z} \in S_1} J(\mathbf{u}^* + \mathbf{z}). \qquad (13)$$

Then $u^*$ solves $(\pi)$.

*Proof* For all $\gamma \in \mathbb{R}_+$ and $z \in S_1$,

$$J(\mathbf{u}^* + \gamma \mathbf{z}) \geq J(\mathbf{u}^*)$$

and therefore

$$\langle J'(\mathbf{u}^*), \mathbf{z} \rangle = \lim_{\gamma \searrow 0} \frac{1}{\gamma} (J(\mathbf{u}^* + \gamma \mathbf{z}) - J(\mathbf{u}^*)) \geq 0$$

holds for all $z \in S_1$. From (B2), we have

$$\langle J'(\mathbf{u}^*), \mathbf{z} \rangle = 0 \quad \forall \mathbf{z} \in S_1,$$

From (B3), we then obtain

$$\langle J'(\mathbf{u}^*), \mathbf{v} \rangle = 0 \quad \forall \mathbf{v} \in V_{\parallel,\parallel},$$

and the lemma follows from Theorem 3. $\square$

In the following, we denote by $S_m$ the set

$$S_m = \left\{ \sum_{i=1}^{m} \mathbf{z}_i : \mathbf{z}_i \in S_1 \right\}$$

The next two lemmas will be useful to define a progressive Proper Generalized Decomposition.
# 4 Progressive Proper Generalized Decompositions in Tensor Banach Spaces

## 4.1 Definition of progressive Proper Generalized Decompositions

We now consider the minimization problem $(\pi)$ of functional $J$ on a reflexive tensor Banach space $V = V_{\parallel,\parallel}$. Assume that we have a functional $J: V_{\parallel,\parallel} \to \mathbb{R}$ satisfying (A1)-(A2) and a weakly closed subset $S_1$ in $V_{\parallel,\parallel}$ such that

(B1) $S_1 \subset V$, with $0 \in S_1$,

(B2) for each $v \in S_1$ we have $\lambda v \in S_1$ for all $\lambda \in \mathbb{R}$, and

(B3) $\text{span} S_1$ is dense in $V_{\parallel,\parallel}$.

By using the notation introduced in Section 2.2 we give the following examples.

**Example 1** Consider $V_{\parallel,\parallel} = L_{\mu}^{p}(I; X)$ and $S_1 = \mathcal{T}_r(X \otimes_a L_{\mu}^{p}(I))$.

**Example 2** Consider $V_{\parallel,\parallel} = H^{m,2}(\Omega)$ and $S_1 = \mathcal{T}_r(a \otimes_{j=1}^{d} H^{m,2}(\Omega_j))$.

**Example 3** Consider $V_{\parallel,\parallel} = H^{m,p}(\Omega)$ and $S_1 = \mathcal{R}_1(a \otimes_{j=1}^{d} H^{m,p}(\Omega_j))$.

The set $S_1$ can be used to characterize the solution of problem $(\pi)$ as shown by the following result.

**Lemma 5** Assume that $J$ satisfies (A1)-(A2) and let $u^* \in V_{\parallel,\parallel}$ satisfying

$$J(\mathbf{u}^*) = \min_{\mathbf{z} \in S_1} J(\mathbf{u}^* + \mathbf{z}). \qquad (13)$$

Then $u^*$ solves $(\pi)$.

*Proof* For all $\gamma \in \mathbb{R}_+$ and $z \in S_1$,

$$J(\mathbf{u}^* + \gamma \mathbf{z}) \geq J(\mathbf{u}^*)$$

and therefore

$$\langle J'(\mathbf{u}^*), \mathbf{z} \rangle = \lim_{\gamma \searrow 0} \frac{1}{\gamma} (J(\mathbf{u}^* + \gamma \mathbf{z}) - J(\mathbf{u}^*)) \geq 0$$

holds for all $z \in S_1$. From (B2), we have

$$\langle J'(\mathbf{u}^*), \mathbf{z} \rangle = 0 \quad \forall \mathbf{z} \in S_1,$$

From (B3), we then obtain

$$\langle J'(\mathbf{u}^*), \mathbf{v} \rangle = 0 \quad \forall \mathbf{v} \in V_{\parallel,\parallel},$$

and the lemma follows from Theorem 3. □

In the following, we denote by $S_m$ the set

$$S_m = \left\{ \sum_{i=1}^{m} \mathbf{z}_i : \mathbf{z}_i \in S_1 \right\}$$

The next two lemmas will be useful to define a progressive Proper Generalized Decomposition.
**Lemma 6** For each $\mathbf{v} \in \mathbf{V}_{\parallel \cdot \parallel}$, the set

$$ \mathbf{v} + S_1 = \{\mathbf{v} + \mathbf{w} : \mathbf{w} \in S_1\} $$

is weakly closed in $\mathbf{V}_{\parallel \cdot \parallel}$.

*Proof* Assume that $\mathbf{v} + \mathbf{w}_n \to \mathbf{w}$ for some $\{\mathbf{w}_n\}_{n \ge 1} \subset S_1$, then $\mathbf{w}_n \to \mathbf{w} - \mathbf{v}$ and since $S_1$ is weakly closed, $\mathbf{w} - \mathbf{v} \in S_1$. In consequence $\mathbf{w} \in \mathbf{v} + S_1$ and the lemma follows. □

**Lemma 7 (Existence of a $S_1$-minimizer)** Assume that $J : \mathbf{V}_{\parallel \cdot \parallel} \to \mathbb{R}$ satisfies (A1)-(A2). Then for any $\mathbf{v} \in \mathbf{V}_{\parallel \cdot \parallel}$, the following problem admits a solution:

$$ \min_{\mathbf{z} \in S_1} J(\mathbf{v} + \mathbf{z}) = \min_{\mathbf{w} \in \mathbf{v} + S_1} J(\mathbf{w}) $$

*Proof* Fréchet differentiability of $J$ implies that $J$ is continuous and since $J$ is convex, we have that $J$ is weakly sequentially lower semicontinuous by Proposition 5. Moreover, $J$ is coercive on $\mathbf{V}_{\parallel \cdot \parallel}$ by Lemma 3(c). By Lemma 6, $\mathbf{v} + S_1$ is a weakly closed subset in $\mathbf{V}_{\parallel \cdot \parallel}$. Then, the existence of a minimizer follows from Theorem 2. □

**Definition 4 (Progressive PGDs)** Assume that $J : \mathbf{V}_{\parallel \cdot \parallel} \to \mathbb{R}$ satisfies (A1)-(A2), we define a progressive Proper Generalized Decomposition $\{\mathbf{u}_m\}_{m \ge 1}$, over $S_1$, of $\mathbf{u} = \arg \min_{\mathbf{v} \in \mathbf{V}_{\parallel \cdot \parallel}} J(\mathbf{v})$ as follows. We let $\mathbf{u}_0 = \mathbf{0}$ and for $m \ge 1$, we construct $\mathbf{u}_m \in \mathbf{V}_{\parallel \cdot \parallel}$ from $\mathbf{u}_{m-1} \in \mathbf{V}_{\parallel \cdot \parallel}$ as we show below. We first find an element $\hat{\mathbf{z}}_m \in S_1 \subset \mathbf{V}$ such that

$$ J(\mathbf{u}_{m-1} + \hat{\mathbf{z}}_m) = \min_{\mathbf{z} \in S_1} J(\mathbf{u}_{m-1} + \mathbf{z}) \quad (*) $$

Then at each step $m$ and before to update $m$ to $m+1$, we can choose one of the following strategies denoted by $c, l$ and $r$, respectively:

(c) Let $\mathbf{z}_m = \hat{\mathbf{z}}_m$. Define $\mathbf{u}_m = \mathbf{u}_{m-1} + \mathbf{z}_m$, update $m$ to $m+1$ and goto (*).

(l) Let $\mathbf{z}_m = \hat{\mathbf{z}}_m$. Construct a closed subspace $\mathbf{U}(\mathbf{u}_{m-1} + \mathbf{z}_m)$ in $\mathbf{V}_{\parallel \cdot \parallel}$ such that $\mathbf{u}_{m-1} + \mathbf{z}_m \in \mathbf{U}(\mathbf{u}_{m-1} + \mathbf{z}_m)$. Then, define

$$ \mathbf{u}_m = \arg \min_{\mathbf{v} \in \mathbf{U}(\mathbf{u}_{m-1} + \mathbf{z}_m)} J(\mathbf{v}), $$

update $m$ to $m+1$ and go to (*).

(r) Construct a closed subspace $\mathbf{U}(\hat{\mathbf{z}}_m)$ in $\mathbf{V}_{\parallel \cdot \parallel}$ such that $\hat{\mathbf{z}}_m \in \mathbf{U}(\hat{\mathbf{z}}_m)$, and define

$$ \mathbf{z}_m = \arg \min_{\mathbf{z} \in \mathbf{U}(\hat{\mathbf{z}}_m)} J(\mathbf{u}_{m-1} + \mathbf{z}). $$

Then, define

$$ \mathbf{u}_m = \mathbf{u}_{m-1} + \mathbf{z}_m = \arg \min_{\mathbf{v} \in \mathbf{u}_{m-1} + \mathbf{U}(\hat{\mathbf{z}}_m)} J(\mathbf{v}), $$

update $m$ to $m+1$ and go to (*).

Strategies of type (l) and (r) are called *updates*. Observe that to each progressive Proper Generalized Decomposition $\{\mathbf{u}_m\}_{m \ge 1}$ of $\mathbf{u}$ we can assign a sequence of symbols (perhaps finite), that we will denote by

$$ \underline{\alpha}(\mathbf{u}) = \alpha_1 \alpha_2 \cdots \alpha_k \cdots $$
**Lemma 6** For each $\mathbf{v} \in \mathbf{V}_{\parallel \cdot \parallel}$, the set

$$ \mathbf{v} + S_1 = \{\mathbf{v} + \mathbf{w} : \mathbf{w} \in S_1\} $$

is weakly closed in $\mathbf{V}_{\parallel \cdot \parallel}$.

*Proof* Assume that $\mathbf{v} + \mathbf{w}_n \to \mathbf{w}$ for some $\{\mathbf{w}_n\}_{n \ge 1} \subset S_1$, then $\mathbf{w}_n \to \mathbf{w} - \mathbf{v}$ and since $S_1$ is weakly closed, $\mathbf{w} - \mathbf{v} \in S_1$. In consequence $\mathbf{w} \in \mathbf{v} + S_1$ and the lemma follows. □

**Lemma 7 (Existence of a $S_1$-minimizer)** Assume that $J : \mathbf{V}_{\parallel \cdot \parallel} \to \mathbb{R}$ satisfies (A1)-(A2). Then for any $\mathbf{v} \in \mathbf{V}_{\parallel \cdot \parallel}$, the following problem admits a solution:

$$ \min_{\mathbf{z} \in S_1} J(\mathbf{v} + \mathbf{z}) = \min_{\mathbf{w} \in \mathbf{v} + S_1} J(\mathbf{w}) $$

*Proof* Fréchet differentiability of $J$ implies that $J$ is continuous and since $J$ is convex, we have that $J$ is weakly sequentially lower semicontinuous by Proposition 5. Moreover, $J$ is coercive on $\mathbf{V}_{\parallel \cdot \parallel}$ by Lemma 3(c). By Lemma 6, $\mathbf{v} + S_1$ is a weakly closed subset in $\mathbf{V}_{\parallel \cdot \parallel}$. Then, the existence of a minimizer follows from Theorem 2. □

**Definition 4 (Progressive PGDs)** Assume that $J : \mathbf{V}_{\parallel \cdot \parallel} \to \mathbb{R}$ satisfies (A1)-(A2), we define a progressive Proper Generalized Decomposition $\{\mathbf{u}_m\}_{m \ge 1}$, over $S_1$, of $\mathbf{u} = \arg \min_{\mathbf{v} \in \mathbf{V}_{\parallel \cdot \parallel}} J(\mathbf{v})$ as follows. We let $\mathbf{u}_0 = \mathbf{0}$ and for $m \ge 1$, we construct $\mathbf{u}_m \in \mathbf{V}_{\parallel \cdot \parallel}$ from $\mathbf{u}_{m-1} \in \mathbf{V}_{\parallel \cdot \parallel}$ as we show below. We first find an element $\hat{\mathbf{z}}_m \in S_1 \subset \mathbf{V}$ such that

$$ J(\mathbf{u}_{m-1} + \hat{\mathbf{z}}_m) = \min_{\mathbf{z} \in S_1} J(\mathbf{u}_{m-1} + \mathbf{z}) \quad (*) $$

Then at each step $m$ and before to update $m$ to $m+1$, we can choose one of the following strategies denoted by $c, l$ and $r$, respectively:

(c) Let $\mathbf{z}_m = \hat{\mathbf{z}}_m$. Define $\mathbf{u}_m = \mathbf{u}_{m-1} + \mathbf{z}_m$, update $m$ to $m+1$ and goto (*).

(l) Let $\mathbf{z}_m = \hat{\mathbf{z}}_m$. Construct a closed subspace $\mathbf{U}(\mathbf{u}_{m-1} + \mathbf{z}_m)$ in $\mathbf{V}_{\parallel \cdot \parallel}$ such that $\mathbf{u}_{m-1} + \mathbf{z}_m \in \mathbf{U}(\mathbf{u}_{m-1} + \mathbf{z}_m)$. Then, define

$$ \mathbf{u}_m = \arg \min_{\mathbf{v} \in \mathbf{U}(\mathbf{u}_{m-1} + \mathbf{z}_m)} J(\mathbf{v}), $$

update $m$ to $m+1$ and go to (*).

(r) Construct a closed subspace $\mathbf{U}(\hat{\mathbf{z}}_m)$ in $\mathbf{V}_{\parallel \cdot \parallel}$ such that $\hat{\mathbf{z}}_m \in \mathbf{U}(\hat{\mathbf{z}}_m)$, and define

$$ \mathbf{z}_m = \arg \min_{\mathbf{z} \in \mathbf{U}(\hat{\mathbf{z}}_m)} J(\mathbf{u}_{m-1} + \mathbf{z}). $$

Then, define

$$ \mathbf{u}_m = \mathbf{u}_{m-1} + \mathbf{z}_m = \arg \min_{\mathbf{v} \in \mathbf{u}_{m-1} + \mathbf{U}(\hat{\mathbf{z}}_m)} J(\mathbf{v}), $$

update $m$ to $m+1$ and go to (*).

Strategies of type (l) and (r) are called *updates*. Observe that to each progressive Proper Generalized Decomposition $\{\mathbf{u}_m\}_{m \ge 1}$ of $\mathbf{u}$ we can assign a sequence of symbols (perhaps finite), that we will denote by

$$ \underline{\alpha}(\mathbf{u}) = \alpha_1 \alpha_2 \cdots \alpha_k \cdots $$
where $\alpha_k \in \{c, l, r\}$ for all $k = 1, 2, \dots$. That means that $\mathbf{u}_k$ was obtained without update if $\alpha_k = c$, or with an update strategy of type $l$ or $r$ if $\alpha_k = l$ or $\alpha_k = r$ respectively. In particular, the progressive PGD defined in [7] coincides with a PGD where $\alpha_k = c$ for all $k \ge 1$. Such a decomposition is called a *purely progressive PGD*, while a decomposition such that $\alpha_k = l$ or $\alpha_k = r$ for some $k$ is called an *updated progressive PGD*.

**Remark 2** The update $\alpha_m = l$ can be defined with several updates at each iteration. Letting $\mathbf{u}_m^{(0)} = \mathbf{u}_{m-1} + \hat{\mathbf{z}}_m$, we introduce a sequence $\{\mathbf{u}_m^{(p)}\}_{p=1}^{d_m} \subset \mathbf{V}_{\|\cdot\|}$ defined by

$$ \mathbf{u}_m^{(p+1)} = \arg \min_{\mathbf{v} \in \mathbf{U}(\mathbf{u}_m^{(p)})} J(\mathbf{v}) $$

with $\mathbf{U}(\mathbf{u}_m^{(p)})$ being a closed linear subspace of $\mathbf{V}_{\|\cdot\|}$ which contains $\mathbf{u}_m^{(p)}$. We finally let $\mathbf{u}_m = \mathbf{u}_m^{(d_m)}$.

In [14] it was introduced the following definition. For a given $\mathbf{v}$ in the algebraic tensor space $\mathbf{V}$, the minimal subspaces $U_{j,\min}(\mathbf{v}) \subset V_j$ are given by the intersection of all subspaces $U_j \subset V_j$ satisfying $\mathbf{v} \in a \otimes_{j=1}^d U_j$. It can be shown [14] that $a \otimes_{j=1}^d U_{j,\min}(\mathbf{v})$ is a finite dimensional subspace of $\mathbf{V}$.

**Example 4** (Illustrations of updates) For a given $\mathbf{v}_m \in \mathbf{V}_{\|\cdot\|}$ (e.g. $\mathbf{v}_m = \mathbf{u}_{m-1} + \mathbf{z}_m$ if $\alpha_m = l$ or $\mathbf{v}_m = \hat{\mathbf{z}}_m$ if $\alpha_m = r$) there are several possible choices for defining a linear subspace $\mathbf{U}(\mathbf{v}_m)$. Among others, we have

- $\mathbf{U}(\mathbf{v}_m) = a \otimes_{j=1}^d U_{j,\min}(\mathbf{v}_m)$. In the case of $\alpha_m = l$, all subspaces $\mathbf{U}(\mathbf{u}_{m-1} + \mathbf{z}_m)$ are finite dimensional and we have that $\mathbf{u}_m \in \mathbf{V}$ for all $m \ge 1$.

- Assume that $\mathbf{v}_m = \sum_{i=1}^m \alpha_i \mathbf{z}_i$ for some $\{\mathbf{z}_1, \dots, \mathbf{z}_m\} \subset \mathbf{V}_{\|\cdot\|}$, $\alpha_i \in \mathbb{R}$, $1 \le i \le m$. Then we can define

$$ \mathbf{U}(\mathbf{v}_m) = \operatorname{span} \{\mathbf{z}_1, \dots, \mathbf{z}_m\}. $$

In the context of Greedy algorithms for computing best approximations, an update of type $\alpha_m = r$ by using an orthonormal basis of $\mathbf{U}(\mathbf{v}_m)$ corresponds to an orthogonal Greedy algorithm.

- Assume $\mathbf{v}_m \in \mathbf{V}$. Fix $k \in \{1, 2, \dots, d\}$. By using $a \otimes_{j=1}^d V_j \cong V_k \otimes_a (a \otimes_{j \neq k} V_j)$, we can write $\mathbf{v}_m = \sum_{i=1}^m w_i^{(k)} \otimes (\otimes_{j \neq k} w_i^{(j)})$ for some elementary tensors $w_i^{(k)} \otimes (\otimes_{j \neq k} w_i^{(j)})$ for $i = 1, \dots, m$. Then we can define the linear subspace

$$ \mathbf{U}(\mathbf{v}_m) = \left\{ \sum_{i=1}^{m} v_i^{(k)} \otimes \left( \bigotimes_{j \neq k} w_i^{(j)} \right) : v_i^{(k)} \in V_k, 1 \le i \le m \right\}. $$

The minimization on $\mathbf{U}(\mathbf{v}_m)$ corresponds to an update of functions along dimension $k$ (functions in the Banach space $V_k$). Following the remark 2, several updates could be defined by choosing a sequence of updated dimensions.
where $\alpha_k \in \{c, l, r\}$ for all $k = 1, 2, \dots$. That means that $\mathbf{u}_k$ was obtained without update if $\alpha_k = c$, or with an update strategy of type $l$ or $r$ if $\alpha_k = l$ or $\alpha_k = r$ respectively. In particular, the progressive PGD defined in [7] coincides with a PGD where $\alpha_k = c$ for all $k \ge 1$. Such a decomposition is called a *purely progressive PGD*, while a decomposition such that $\alpha_k = l$ or $\alpha_k = r$ for some $k$ is called an *updated progressive PGD*.

**Remark 2** The update $\alpha_m = l$ can be defined with several updates at each iteration. Letting $\mathbf{u}_m^{(0)} = \mathbf{u}_{m-1} + \hat{\mathbf{z}}_m$, we introduce a sequence $\{\mathbf{u}_m^{(p)}\}_{p=1}^{d_m} \subset \mathbf{V}_{\|\cdot\|}$ defined by

$$ \mathbf{u}_m^{(p+1)} = \arg \min_{\mathbf{v} \in \mathbf{U}(\mathbf{u}_m^{(p)})} J(\mathbf{v}) $$

with $\mathbf{U}(\mathbf{u}_m^{(p)})$ being a closed linear subspace of $\mathbf{V}_{\|\cdot\|}$ which contains $\mathbf{u}_m^{(p)}$. We finally let $\mathbf{u}_m = \mathbf{u}_m^{(d_m)}$.

In [14] it was introduced the following definition. For a given $\mathbf{v}$ in the algebraic tensor space $\mathbf{V}$, the minimal subspaces $U_{j,\min}(\mathbf{v}) \subset V_j$ are given by the intersection of all subspaces $U_j \subset V_j$ satisfying $\mathbf{v} \in a \otimes_{j=1}^d U_j$. It can be shown [14] that $a \otimes_{j=1}^d U_{j,\min}(\mathbf{v})$ is a finite dimensional subspace of $\mathbf{V}$.

**Example 4 (Illustrations of updates)** For a given $\mathbf{v}_m \in \mathbf{V}_{\|\cdot\|}$ (e.g. $\mathbf{v}_m = \mathbf{u}_{m-1} + \mathbf{z}_m$ if $\alpha_m = l$ or $\mathbf{v}_m = \hat{\mathbf{z}}_m$ if $\alpha_m = r$) there are several possible choices for defining a linear subspace $\mathbf{U}(\mathbf{v}_m)$. Among others, we have

- $\mathbf{U}(\mathbf{v}_m) = a \otimes_{j=1}^d U_{j,\min}(\mathbf{v}_m)$. In the case of $\alpha_m = l$, all subspaces $\mathbf{U}(\mathbf{u}_{m-1} + \mathbf{z}_m)$ are finite dimensional and we have that $\mathbf{u}_m \in \mathbf{V}$ for all $m \ge 1$.

- Assume that $\mathbf{v}_m = \sum_{i=1}^m \alpha_i \mathbf{z}_i$ for some $\{\mathbf{z}_1, \dots, \mathbf{z}_m\} \subset \mathbf{V}_{\|\cdot\|}$, $\alpha_i \in \mathbb{R}$, $1 \le i \le m$. Then we can define

$$ \mathbf{U}(\mathbf{v}_m) = \operatorname{span} \{\mathbf{z}_1, \dots, \mathbf{z}_m\}. $$

In the context of Greedy algorithms for computing best approximations, an update of type $\alpha_m = r$ by using an orthonormal basis of $\mathbf{U}(\mathbf{v}_m)$ corresponds to an orthogonal Greedy algorithm.

- Assume $\mathbf{v}_m \in \mathbf{V}$. Fix $k \in \{1, 2, \dots, d\}$. By using $a \otimes_{j=1}^d V_j \cong V_k \otimes_a (a \otimes_{j \neq k} V_j)$, we can write $\mathbf{v}_m = \sum_{i=1}^m w_i^{(k)} \otimes (\otimes_{j \neq k} w_i^{(j)})$ for some elementary tensors $w_i^{(k)} \otimes (\otimes_{j \neq k} w_i^{(j)})$ for $i = 1, \dots, m$. Then we can define the linear subspace

$$ \mathbf{U}(\mathbf{v}_m) = \left\{ \sum_{i=1}^{m} v_i^{(k)} \otimes \left( \bigotimes_{j \neq k} w_i^{(j)} \right) : v_i^{(k)} \in V_k, 1 \le i \le m \right\}. $$

The minimization on $\mathbf{U}(\mathbf{v}_m)$ corresponds to an update of functions along dimension $k$ (functions in the Banach space $V_k$). Following the remark 2, several updates could be defined by choosing a sequence of updated dimensions.
## 4.2 On the convergence of the progressive PGDs

Now, we study the convergence of progressive PGDs. Recall that $\hat{z}_m \in S_1$ is a solution of

$$J(\mathbf{u}_{m-1} + \hat{\mathbf{z}}_m) = \min_{\mathbf{z} \in S_1} J(\mathbf{u}_{m-1} + \mathbf{z}),$$

For $\alpha_m = c$, we have $\mathbf{z}_m = \hat{\mathbf{z}}_m$ and $\mathbf{u}_m = \mathbf{u}_{m-1} + \mathbf{z}_m$, so that

$$J(\mathbf{u}_m) = J(\mathbf{u}_{m-1} + \mathbf{z}_m) = J(\mathbf{u}_{m-1} + \hat{\mathbf{z}}_m)$$

For $\alpha_m = l$, we have $\mathbf{z}_m = \hat{\mathbf{z}}_m$ and $\mathbf{u}_m$ is obtained by an update (or several updates) of $\mathbf{u}_{m-1} + \mathbf{z}_m$, so that

$$J(\mathbf{u}_m) \leq J(\mathbf{u}_{m-1} + \mathbf{z}_m) = J(\mathbf{u}_{m-1} + \hat{\mathbf{z}}_m)$$

Otherwise, for $\alpha_m = r$, we have $\mathbf{u}_m = \mathbf{u}_{m-1} + \mathbf{z}_m$ with $\mathbf{z}_m$ obtained by an update of $\hat{\mathbf{z}}_m$, such that

$$J(\mathbf{u}_m) = J(\mathbf{u}_{m-1} + \mathbf{z}_m) \leq J(\mathbf{u}_{m-1} + \hat{\mathbf{z}}_m)$$

We begin with the following Lemma.

**Lemma 8** Assume that $J$ satisfies (A1)-(A2). Then $\{J(\mathbf{u}_m)\}_{m \geq 1}$, where $\{\mathbf{u}_m\}_{m \geq 1}$ is a progressive Proper Generalized Decomposition, over $S_1$, of

$$\mathbf{u} = \arg \min_{\mathbf{v} \in \mathbf{V}_{\parallel, \parallel}} J(\mathbf{v}),$$

is a non increasing sequence:

$$J(\mathbf{u}_m) \leq J(\mathbf{u}_{m-1}) \text{ for all } m \geq 1.$$

Moreover, if $J(\mathbf{u}_m) = J(\mathbf{u}_{m-1})$, $\mathbf{u}_{m-1}$ is the solution of $(\pi)$.

*Proof* By definition, we have

$$J(\mathbf{u}_m) \leq J(\mathbf{u}_{m-1} + \mathbf{z}_m) \leq J(\mathbf{u}_{m-1} + \hat{\mathbf{z}}_m) \leq J(\mathbf{u}_{m-1} + \mathbf{z}) \quad \forall \mathbf{z} \in S_1$$

In particular, since $\mathbf{0} \in S_1$ by assumption (B1), we have $J(\mathbf{u}_m) \leq J(\mathbf{u}_{m-1})$. If $J(\mathbf{u}_m) = J(\mathbf{u}_{m-1})$, we have

$$J(\mathbf{u}_{m-1}) = \min_{\mathbf{z} \in S_1} J(\mathbf{u}_{m-1} + \mathbf{z})$$

and by Lemma 5, we have that $\mathbf{u}_{m-1}$ solves $(\pi)$. $\square$

**Remark 3** If $J(\mathbf{u}_m) = J(\mathbf{u}_{m-1})$ holds for some $m > 1$, that is $\mathbf{u}_{m-1}$ is the solution of $(\pi)$, then the updated PGD is described by a finite sequence of symbols $\underline{\alpha}(\mathbf{u}) = \alpha_1 \alpha_2 \cdots \alpha_{m-1}$, where $\alpha_k \in \{c, l, r\}$ for $1 \leq k \leq m-1$. Otherwise, $\{J(\mathbf{u}_m)\}_{m \in \mathbb{N}}$ is a strictly decreasing sequence of real numbers and $\underline{\alpha}(\mathbf{u}) \in \{c, l, r\}^\mathbb{N}$.

**Definition 5** Let $\alpha \in \{c, l, r\}$. Then $\alpha^\infty \in \{c, l, r\}^\mathbb{N}$ denotes the infinite sequence of symbols $\alpha \alpha \cdots \alpha \cdots$.

From now on, we will distinguish two convergence studies, one with a weak continuity assumption on functional $J$, the other one without weak continuity assumption on $J$ but with an additional Lipschitz continuity assumption on the differential $J'$.
## 4.2 On the convergence of the progressive PGDs

Now, we study the convergence of progressive PGDs. Recall that $\hat{z}_m \in S_1$ is a solution of

$$J(\mathbf{u}_{m-1} + \hat{\mathbf{z}}_m) = \min_{\mathbf{z} \in S_1} J(\mathbf{u}_{m-1} + \mathbf{z}),$$

For $\alpha_m = c$, we have $\mathbf{z}_m = \hat{\mathbf{z}}_m$ and $\mathbf{u}_m = \mathbf{u}_{m-1} + \mathbf{z}_m$, so that

$$J(\mathbf{u}_m) = J(\mathbf{u}_{m-1} + \mathbf{z}_m) = J(\mathbf{u}_{m-1} + \hat{\mathbf{z}}_m)$$

For $\alpha_m = l$, we have $\mathbf{z}_m = \hat{\mathbf{z}}_m$ and $\mathbf{u}_m$ is obtained by an update (or several updates) of $\mathbf{u}_{m-1} + \mathbf{z}_m$, so that

$$J(\mathbf{u}_m) \le J(\mathbf{u}_{m-1} + \mathbf{z}_m) = J(\mathbf{u}_{m-1} + \hat{\mathbf{z}}_m)$$

Otherwise, for $\alpha_m = r$, we have $\mathbf{u}_m = \mathbf{u}_{m-1} + \mathbf{z}_m$ with $\mathbf{z}_m$ obtained by an update of $\hat{\mathbf{z}}_m$, such that

$$J(\mathbf{u}_m) = J(\mathbf{u}_{m-1} + \mathbf{z}_m) \le J(\mathbf{u}_{m-1} + \hat{\mathbf{z}}_m)$$

We begin with the following Lemma.

**Lemma 8** Assume that $J$ satisfies (A1)-(A2). Then $\{J(\mathbf{u}_m)\}_{m \ge 1}$, where $\{\mathbf{u}_m\}_{m \ge 1}$ is a progressive Proper Generalized Decomposition, over $S_1$, of

$$\mathbf{u} = \arg \min_{\mathbf{v} \in \mathbf{V}_{\parallel \cdot \parallel}} J(\mathbf{v}),$$

is a non increasing sequence:

$$J(\mathbf{u}_m) \le J(\mathbf{u}_{m-1}) \text{ for all } m \ge 1.$$

Moreover, if $J(\mathbf{u}_m) = J(\mathbf{u}_{m-1})$, $\mathbf{u}_{m-1}$ is the solution of $(\pi)$.

*Proof* By definition, we have

$$J(\mathbf{u}_m) \le J(\mathbf{u}_{m-1} + \mathbf{z}_m) \le J(\mathbf{u}_{m-1} + \hat{\mathbf{z}}_m) \le J(\mathbf{u}_{m-1} + \mathbf{z}) \quad \forall \mathbf{z} \in S_1$$

In particular, since $0 \in S_1$ by assumption (B1), we have $J(\mathbf{u}_m) \le J(\mathbf{u}_{m-1})$. If $J(\mathbf{u}_m) = J(\mathbf{u}_{m-1})$, we have

$$J(\mathbf{u}_{m-1}) = \min_{\mathbf{z} \in S_1} J(\mathbf{u}_{m-1} + \mathbf{z})$$

and by Lemma 5, we have that $\mathbf{u}_{m-1}$ solves $(\pi)$. $\square$

**Remark 3** If $J(\mathbf{u}_m) = J(\mathbf{u}_{m-1})$ holds for some $m > 1$, that is $\mathbf{u}_{m-1}$ is the solution of $(\pi)$, then the updated PGD is described by a finite sequence of symbols $\underline{\alpha}(\mathbf{u}) = \alpha_1 \alpha_2 \cdots \alpha_{m-1}$, where $\alpha_k \in \{c, l, r\}$ for $1 \le k \le m-1$. Otherwise, $\{J(\mathbf{u}_m)\}_{m \in \mathbb{N}}$ is a strictly decreasing sequence of real numbers and $\underline{\alpha}(\mathbf{u}) \in \{c, l, r\}^\mathbb{N}$.

**Definition 5** Let $\alpha \in \{c, l, r\}$. Then $\alpha^\infty \in \{c, l, r\}^\mathbb{N}$ denotes the infinite sequence of symbols $\alpha \alpha \cdots \alpha \cdots$.

From now on, we will distinguish two convergence studies, one with a weak continuity assumption on functional $J$, the other one without weak continuity assumption on $J$ but with an additional Lipschitz continuity assumption on the differential $J'$.
### 4.2.1 A first approach for weakly sequentially continuous functionals

Here, we introduce the following assumption.

(A3) The map $J: V_{\parallel \cdot \parallel} \to \mathbb{R}$ is weakly sequentially continuous.

**Theorem 4** Assume that $J$ satisfies (A1)-(A3). Then every progressive Proper Generalized Decomposition $\{u_m\}_{m \ge 1}$, over $S_1$, of

$$u = \arg \min_{v \in V_{\parallel \cdot \parallel}} J(v)$$

converges in $V_{\parallel \cdot \parallel}$ to $u$, that is,

$$\lim_{m \to \infty} \|u - u_m\| = 0$$

*Proof* From Lemma 8, $\{J(u_m)\}$ is a non increasing sequence. If there exists $m$ such that $J(u_m) = J(u_{m-1})$, from Lemma 8, we have $u_m = u$, which ends the proof. Let us now suppose that $J(u_m) < J(u_{m-1})$ for all $m$. $J(u_m)$ is a strictly decreasing sequence which is bounded below by $J(u)$. Then, there exists

$$J^* = \lim_{m \to \infty} J(u_m) \ge J(u).$$

If $J^* = J(u)$, Lemma 4 allows to conclude that $u_m \to u$. Therefore, it remains to prove that $J^* = J(u)$. Since $J$ is coercive, the sequence $\{u_m\}_{m \in \mathbb{N}}$ is bounded in $V_{\parallel \cdot \parallel}$. Then, there exists a subsequence $\{u_{m_k}\}_{k \in \mathbb{N}}$ that weakly converges to some $u^* \in V$. Since $J$ is weakly sequentially continuous, we have

$$J^* = \lim_{k \to \infty} J(u_{m_k}) = J(u^*).$$

By definition of the PGD, we have for all $z \in S_1$,

$$J(u_{m_{(k+1)}}) \le J(u_{m_k+1}) \le J(u_{m_k} + z)$$

Taking the limit with $k$, and using the weak sequential continuity of $J$, we obtain

$$J(u^*) \le J(u^* + z) \quad \forall z \in S_1,$$

and by Lemma 5, we obtain $u^* = u$ and a fortiori $J(u^*) = J(u)$, that concludes the proof. □

### 4.2.2 A second approach for a class of functionals with Lipschitz continuous derivative on bounded sets

Now, assume that assumption (A3) is replaced by

(A3) $J' : V_{\parallel \cdot \parallel} \to V_{\parallel \cdot \parallel}^*$ is Lipschitz continuous on bounded sets, i.e. for $A$ a bounded set in $V_{\parallel \cdot \parallel}$, there exists a constant $C_A > 0$ such that

$$\|J'(v) - J'(w)\| \le C_A \|v - w\| \quad (14)$$

for all $v, w \in A$.

The next three lemmas will give some useful properties of the sequence $\{z_m\}_{m \ge 1}$.
### 4.2.1 A first approach for weakly sequentially continuous functionals

Here, we introduce the following assumption.

(A3) The map $J: V_{\parallel \cdot \parallel} \to \mathbb{R}$ is weakly sequentially continuous.

**Theorem 4** Assume that $J$ satisfies (A1)-(A3). Then every progressive Proper Generalized Decomposition $\{u_m\}_{m \ge 1}$, over $S_1$, of

$$u = \arg \min_{v \in V_{\parallel \cdot \parallel}} J(v)$$

converges in $V_{\parallel \cdot \parallel}$ to $u$, that is,

$$\lim_{m \to \infty} \|u - u_m\| = 0$$

*Proof* From Lemma 8, $\{J(u_m)\}$ is a non increasing sequence. If there exists $m$ such that $J(u_m) = J(u_{m-1})$, from Lemma 8, we have $u_m = u$, which ends the proof. Let us now suppose that $J(u_m) < J(u_{m-1})$ for all $m$. $J(u_m)$ is a strictly decreasing sequence which is bounded below by $J(u)$. Then, there exists

$$J^* = \lim_{m \to \infty} J(u_m) \ge J(u).$$

If $J^* = J(u)$, Lemma 4 allows to conclude that $u_m \to u$. Therefore, it remains to prove that $J^* = J(u)$. Since $J$ is coercive, the sequence $\{u_m\}_{m \in \mathbb{N}}$ is bounded in $V_{\parallel \cdot \parallel}$. Then, there exists a subsequence $\{u_{m_k}\}_{k \in \mathbb{N}}$ that weakly converges to some $u^* \in V$. Since $J$ is weakly sequentially continuous, we have

$$J^* = \lim_{k \to \infty} J(u_{m_k}) = J(u^*).$$

By definition of the PGD, we have for all $z \in S_1$,

$$J(u_{m_{(k+1)}}) \le J(u_{m_k+1}) \le J(u_{m_k} + z)$$

Taking the limit with $k$, and using the weak sequential continuity of $J$, we obtain

$$J(u^*) \le J(u^* + z) \quad \forall z \in S_1,$$

and by Lemma 5, we obtain $u^* = u$ and a fortiori $J(u^*) = J(u)$, that concludes the proof. □

### 4.2.2 A second approach for a class of functionals with Lipschitz continuous derivative on bounded sets

Now, assume that assumption (A3) is replaced by

(A3) $J' : V_{\parallel \cdot \parallel} \to V_{\parallel \cdot \parallel}^*$ is Lipschitz continuous on bounded sets, i.e. for $A$ a bounded set in $V_{\parallel \cdot \parallel}$, there exists a constant $C_A > 0$ such that

$$\|J'(v) - J'(w)\| \le C_A \|v - w\| \quad (14)$$

for all $v, w \in A$.

The next three lemmas will give some useful properties of the sequence $\{z_m\}_{m \ge 1}$.
**Lemma 9** Assume that J satisfies (A1)-(A2) and let {$\mathbf{u}_m$}$_{m \ge 1}$ be a progressive Proper Generalized Decomposition, over $S_1$, of

$$
\mathbf{u} = \arg \min_{\mathbf{v} \in \mathbf{V}_{\parallel, \parallel}} J(\mathbf{v}).
$$

Then

$$
\langle J'(\mathbf{u}_{m-1} + \mathbf{z}_m), \mathbf{z}_m \rangle = 0,
$$

for all $m \ge 1$.

Proof Let $\mathbf{z}_m = \lambda_m \mathbf{w}_m$, with $\lambda_m \in \mathbb{R}^+$ and $\|\mathbf{w}_m\| = 1$. In the cases $\alpha_m = c$ (purely progressive PGD) and $\alpha_m = l$, we have $\mathbf{z}_m = \hat{\mathbf{z}}_m \in \arg \min_{\mathbf{z} \in S_1} J(\mathbf{u}_{m-1} + \mathbf{z})$. From assumption (B2), we obtain

$$
J(\mathbf{u}_{m-1} + \lambda_m \mathbf{w}_m) \leq J(\mathbf{u}_{m-1} + \lambda \mathbf{w}_m)
$$

for all $\lambda \in \mathbb{R}$. This inequality is also true for $\alpha_m = r$ since $\mathbf{z}_m = \arg \min_{\mathbf{z} \in \mathcal{U}(\hat{\mathbf{z}}_m)} J(\mathbf{u}_{m-1} + \mathbf{z})$ and $\mathcal{U}(\hat{\mathbf{z}}_m)$ is a linear space. Taking $\lambda = \lambda_m \pm \gamma$, with $\gamma \in \mathbb{R}^+$, we obtain for all cases

$$
0 \le \frac{1}{\gamma} \left( J(\mathbf{u}_{m-1} + \lambda_m \mathbf{w}_m \pm \gamma \mathbf{w}_m) - J(\mathbf{u}_{m-1} + \lambda_m \mathbf{w}_m) \right).
$$

Taking the limit $\gamma \searrow 0$, we obtain $0 \le \pm \langle J'(\mathbf{u}_{m-1} + \lambda_m \mathbf{w}_m), \mathbf{w}_m \rangle$ and therefore

$$
\langle J'(\mathbf{u}_{m-1} + \lambda_m \mathbf{w}_m), \mathbf{w}_m \rangle = 0,
$$

which ends the proof. $\square$

**Lemma 10** Assume that J satisfies (A1)-(A2). Then the corrections {$\mathbf{z}_m$}$_{m \ge 1}$ of a progressive Proper Generalized Decomposition {$\mathbf{u}_m$}$_{m \ge 1}$, over $S_1$, of

$$
\mathbf{u} = \arg \min_{\mathbf{v} \in \mathbf{V}_{\parallel, \parallel}} J(\mathbf{v}),
$$

satisfy

$$
\sum_{m=1}^{\infty} \|\mathbf{z}_m\|^s < \infty, \text{ for some } s > 1, \qquad (15)
$$

and thus,

$$
\lim_{m \to \infty} \|\mathbf{z}_m\| = 0. \tag{16}
$$

*Proof* By the ellipticity property (9), we have

$$
J(\mathbf{u}_{m-1}) - J(\mathbf{u}_{m-1} + \mathbf{z}_m) \geq \langle -J'(\mathbf{u}_{m-1} + \mathbf{z}_m), \mathbf{z}_m \rangle + \frac{\alpha}{s} \|\mathbf{z}_m\|^s
$$

for some $s > 1$ and $\alpha > 0$. Using Lemma 9 and $J(\mathbf{u}_m) \le J(\mathbf{u}_{m-1} + \mathbf{z}_m)$, we then obtain

$$
J(\mathbf{u}_{m-1}) - J(\mathbf{u}_m) \geq \frac{\alpha}{s} \|\mathbf{z}_m\|^s \quad (17)
$$

Now, summing on $m$, and using $\lim_{m \to \infty} J(\mathbf{u}_m) = J^* < \infty$, we obtain

$$
\frac{\alpha}{s} \sum_{m=1}^{\infty} \|\mathbf{z}_m\|^s \le \sum_{m=1}^{\infty} (J(\mathbf{u}_{m-1}) - J(\mathbf{u}_m)) = J(0) - J^* < +\infty.
$$

which implies $\lim_{m \to \infty} \|\mathbf{z}_m\|^s = 0$. The continuity of the map $x \mapsto x^{1/s}$ at $x = 0$ proves (16). $\square$
**Lemma 9** Assume that J satisfies (A1)-(A2) and let {$\mathbf{u}_m$}$_{m \ge 1}$ be a progressive Proper Generalized Decomposition, over $S_1$, of

$$
\mathbf{u} = \arg \min_{\mathbf{v} \in \mathbf{V}_{\parallel, \parallel}} J(\mathbf{v}).
$$

Then

$$
\langle J'(\mathbf{u}_{m-1} + \mathbf{z}_m), \mathbf{z}_m \rangle = 0,
$$

for all $m \ge 1$.

Proof Let $\mathbf{z}_m = \lambda_m \mathbf{w}_m$, with $\lambda_m \in \mathbb{R}^+$ and $\|\mathbf{w}_m\| = 1$. In the cases $\alpha_m = c$ (purely progressive PGD) and $\alpha_m = l$, we have $\mathbf{z}_m = \hat{\mathbf{z}}_m \in \arg \min_{\mathbf{z} \in S_1} J(\mathbf{u}_{m-1} + \mathbf{z})$. From assumption (B2), we obtain

$$
J(\mathbf{u}_{m-1} + \lambda_m \mathbf{w}_m) \leq J(\mathbf{u}_{m-1} + \lambda \mathbf{w}_m)
$$

for all $\lambda \in \mathbb{R}$. This inequality is also true for $\alpha_m = r$ since $\mathbf{z}_m = \arg \min_{\mathbf{z} \in \mathcal{U}(\hat{\mathbf{z}}_m)} J(\mathbf{u}_{m-1} + \mathbf{z})$ and $\mathcal{U}(\hat{\mathbf{z}}_m)$ is a linear space. Taking $\lambda = \lambda_m \pm \gamma$, with $\gamma \in \mathbb{R}^+$, we obtain for all cases

$$
0 \le \frac{1}{\gamma} \left( J(\mathbf{u}_{m-1} + \lambda_m \mathbf{w}_m \pm \gamma \mathbf{w}_m) - J(\mathbf{u}_{m-1} + \lambda_m \mathbf{w}_m) \right).
$$

Taking the limit $\gamma \searrow 0$, we obtain $0 \le \pm \langle J'(\mathbf{u}_{m-1} + \lambda_m \mathbf{w}_m), \mathbf{w}_m \rangle$ and therefore

$$
\langle J'(\mathbf{u}_{m-1} + \lambda_m \mathbf{w}_m), \mathbf{w}_m \rangle = 0,
$$

which ends the proof. $\square$

**Lemma 10** Assume that J satisfies (A1)-(A2). Then the corrections {$\mathbf{z}_m$}$_{m \ge 1}$ of a progressive Proper Generalized Decomposition {$\mathbf{u}_m$}$_{m \ge 1}$, over $S_1$, of

$$
\mathbf{u} = \arg \min_{\mathbf{v} \in \mathbf{V}_{\parallel, \parallel}} J(\mathbf{v}),
$$

satisfy

$$
\sum_{m=1}^{\infty} \|\mathbf{z}_m\|^s < \infty, \text{ for some } s > 1, \qquad (15)
$$

and thus,

$$
\lim_{m \to \infty} \|\mathbf{z}_m\| = 0. \tag{16}
$$

*Proof* By the ellipticity property (9), we have

$$
J(\mathbf{u}_{m-1}) - J(\mathbf{u}_{m-1} + \mathbf{z}_m) \geq \langle -J'(\mathbf{u}_{m-1} + \mathbf{z}_m), \mathbf{z}_m \rangle + \frac{\alpha}{s} \|\mathbf{z}_m\|^s
$$

for some $s > 1$ and $\alpha > 0$. Using Lemma 9 and $J(\mathbf{u}_m) \le J(\mathbf{u}_{m-1} + \mathbf{z}_m)$, we then
obtain

$$
J(\mathbf{u}_{m-1}) - J(\mathbf{u}_m) \geq \frac{\alpha}{s} \|\mathbf{z}_m\|^s \quad (17)
$$

Now, summing on $m$, and using $\lim_{m \to \infty} J(\mathbf{u}_m) = J^* < \infty$, we obtain

$$
\frac{\alpha}{s} \sum_{m=1}^{\infty} \|\mathbf{z}_m\|^s \le \sum_{m=1}^{\infty} (J(\mathbf{u}_{m-1}) - J(\mathbf{u}_m)) = J(0) - J^* < +\infty.
$$

which implies $\lim_{m \to \infty} \|\mathbf{z}_m\|^s = 0$. The continuity of the map $x \mapsto x^{1/s}$ at $x = 0$
proves (16). $\square$
**Lemma 11** Assume that J satisfies (A1)-(A3). Then for every progressive Proper Generalized Decompositions {$\mathbf{u}_m$}$_{m\ge 1}$, over $S_1$, of $\mathbf{u} = \arg\min_{\mathbf{v}\in\mathbf{V}_{\parallel,\parallel}} J(\mathbf{v})$, there exists $C > 0$ such that for $m \ge 1$,

$$|\langle J'(\mathbf{u}_{m-1}), \mathbf{z} \rangle| \le C \|\mathbf{z}_m\| \|\mathbf{z}\|,$$

for all $\mathbf{z} \in S_1$.

*Proof* Since $J(\mathbf{u}_m)$ converges and since $J$ is coercive, {$\mathbf{u}_m$}$_{m\ge 1}$ is a bounded sequence. Since $\|\mathbf{z}_m\| \to 0$ as $m \to \infty$ (Lemma 10), {$\mathbf{z}_m$}$_{m\ge 1}$ is also a bounded sequence. Let $a > 0$ such that $\sup_m \|\mathbf{u}_m\| + \sup_m \|\mathbf{z}_m\| \le a$ and let $C_B$ be the Lipschitz continuity constant of $J'$ on the bounded set $B = \{\mathbf{v} \in \mathbf{V}_{\parallel,\parallel} : \|\mathbf{v}\| \le a\}$. Then

$$-\langle J'(\mathbf{u}_{m-1}), \mathbf{z} \rangle = \langle J'(\mathbf{u}_{m-1} + \mathbf{z}) - J'(\mathbf{u}_{m-1}), \mathbf{z} \rangle - \langle J'(\mathbf{u}_{m-1} + \mathbf{z}), \mathbf{z} \rangle \\ \le C_B \|\mathbf{z}\|^2 - \langle J'(\mathbf{u}_{m-1} + \mathbf{z}), \mathbf{z} \rangle$$

for all $\mathbf{z} \in A = \{\mathbf{z} \in S_1 : \|\mathbf{z}\| \le \sup_m \|\mathbf{z}_m\|\}$. By convexity of $J$ and since $J(\mathbf{u}_{m-1} + \mathbf{z}_m) \le J(\mathbf{u}_{m-1} + \mathbf{z})$ for all $\mathbf{z} \in S_1$, we have

$$\langle J'(\mathbf{u}_{m-1} + \mathbf{z}), \mathbf{z}_m - \mathbf{z} \rangle \le J(\mathbf{u}_{m-1} + \mathbf{z}_m) - J(\mathbf{u}_{m-1} + \mathbf{z}) \le 0$$

Therefore, for all $\mathbf{z} \in A$, we have

$$-\langle J'(\mathbf{u}_{m-1}), \mathbf{z} \rangle \le C_B \|\mathbf{z}\|^2 - \langle J'(\mathbf{u}_{m-1} + \mathbf{z}), \mathbf{z}_m \rangle \\ \le C_B \|\mathbf{z}\|^2 - \langle J'(\mathbf{u}_{m-1} + \mathbf{z}) - J'(\mathbf{u}_{m-1} + \mathbf{z}_m), \mathbf{z}_m \rangle \quad (\text{Lemma 9}) \\ \le C_B \|\mathbf{z}\|^2 + C_B \|\mathbf{z} - \mathbf{z}_m\| \|\mathbf{z}_m\| \quad (\text{Lemma 10}) \\ \le C_B \left( \|\mathbf{z}\|^2 + \|\mathbf{z}\| \|\mathbf{z}_m\| + \|\mathbf{z}_m\|^2 \right)$$

Let $\mathbf{z} = \mathbf{w}\|\mathbf{z}_m\| \in A$, with $\|\mathbf{w}\| = 1$. Then

$$|\langle J'(\mathbf{u}_{m-1}), \mathbf{w} \rangle| \le 3C_B \|\mathbf{z}_m\| \quad \forall \mathbf{w} \in \{\mathbf{w} \in S_1 : \|\mathbf{w}\| = 1\}$$

Taking $\mathbf{w} = \mathbf{z}/\|\mathbf{z}\|$, with $\mathbf{z} \in S_1$, and $C = 3C_B > 0$ we obtain

$$|\langle J'(\mathbf{u}_{m-1}), \mathbf{z} \rangle| \le C \|\mathbf{z}_m\| \|\mathbf{z}\| \quad \forall \mathbf{z} \in S_1$$

□

Since $\mathbf{V}_{\parallel,\parallel}$ is reflexive, we can identify $\mathbf{V}_{\parallel,\parallel}^{**}$ with $\mathbf{V}_{\parallel,\parallel}$ and the duality pairing $\langle \cdot, \cdot \rangle \mathbf{V}_{\parallel,\parallel}^{**}, \mathbf{V}_{\parallel,\parallel}^{*}$ with $\langle \cdot, \cdot \rangle \mathbf{V}_{\parallel,\parallel}^{*}, \mathbf{V}_{\parallel,\parallel}$ (i.e. weak and weak-* topologies coincide on $\mathbf{V}_{\parallel,\parallel}^{*}$).

**Lemma 12** Assume that J satisfies (A1)-(A3). Then for every progressive Proper Generalized Decomposition {$\mathbf{u}_m$}$_{m\ge 1}$, over $S_1$, of $\mathbf{u} = \arg\min_{\mathbf{v}\in\mathbf{V}_{\parallel,\parallel}} J(\mathbf{v})$, the sequence {$J'(\mathbf{u}_m)$}$_{m\in\mathbb{N}}$ weakly-* converges to $\mathbf{0}$ in $\mathbf{V}_{\parallel,\parallel}^{*}$, that is, $\lim_{m\to\infty} \langle J'(\mathbf{u}_m), \mathbf{z} \rangle = 0$ for all $\mathbf{z}$ in a dense subset of $\mathbf{V}_{\parallel,\parallel}^{*}$.
**Lemma 11** Assume that J satisfies (A1)-(A3). Then for every progressive Proper Generalized Decompositions {$\mathbf{u}_m$}$_{m\ge 1}$, over $S_1$, of $\mathbf{u} = \arg\min_{\mathbf{v}\in\mathbf{V}_{\parallel,\parallel}} J(\mathbf{v})$, there exists $C > 0$ such that for $m \ge 1$,

$$|\langle J'(\mathbf{u}_{m-1}), \mathbf{z} \rangle| \le C \|\mathbf{z}_m\| \|\mathbf{z}\|,$$

for all $\mathbf{z} \in S_1$.

*Proof* Since $J(\mathbf{u}_m)$ converges and since $J$ is coercive, {$\mathbf{u}_m$}$_{m\ge 1}$ is a bounded sequence. Since $\|\mathbf{z}_m\| \to 0$ as $m \to \infty$ (Lemma 10), {$\mathbf{z}_m$}$_{m\ge 1}$ is also a bounded sequence. Let $a > 0$ such that $\sup_m \|\mathbf{u}_m\| + \sup_m \|\mathbf{z}_m\| \le a$ and let $C_B$ be the Lipschitz continuity constant of $J'$ on the bounded set $B = \{\mathbf{v} \in \mathbf{V}_{\parallel,\parallel} : \|\mathbf{v}\| \le a\}$. Then

$$-\langle J'(\mathbf{u}_{m-1}), \mathbf{z} \rangle = \langle J'(\mathbf{u}_{m-1} + \mathbf{z}) - J'(\mathbf{u}_{m-1}), \mathbf{z} \rangle - \langle J'(\mathbf{u}_{m-1} + \mathbf{z}), \mathbf{z} \rangle \\ \le C_B \|\mathbf{z}\|^2 - \langle J'(\mathbf{u}_{m-1} + \mathbf{z}), \mathbf{z} \rangle$$

for all $\mathbf{z} \in A = \{\mathbf{z} \in S_1 : \|\mathbf{z}\| \le \sup_m \|\mathbf{z}_m\|\}$. By convexity of $J$ and since $J(\mathbf{u}_{m-1} + \mathbf{z}_m) \le J(\mathbf{u}_{m-1} + \mathbf{z})$ for all $\mathbf{z} \in S_1$, we have

$$\langle J'(\mathbf{u}_{m-1} + \mathbf{z}), \mathbf{z}_m - \mathbf{z} \rangle \le J(\mathbf{u}_{m-1} + \mathbf{z}_m) - J(\mathbf{u}_{m-1} + \mathbf{z}) \le 0$$

Therefore, for all $\mathbf{z} \in A$, we have

$$-\langle J'(\mathbf{u}_{m-1}), \mathbf{z} \rangle \le C_B \|\mathbf{z}\|^2 - \langle J'(\mathbf{u}_{m-1} + \mathbf{z}), \mathbf{z}_m \rangle \\ \le C_B \|\mathbf{z}\|^2 - \langle J'(\mathbf{u}_{m-1} + \mathbf{z}) - J'(\mathbf{u}_{m-1} + \mathbf{z}_m), \mathbf{z}_m \rangle \quad (\text{Lemma 9}) \\ \le C_B \|\mathbf{z}\|^2 + C_B \|\mathbf{z} - \mathbf{z}_m\| \|\mathbf{z}_m\| \quad (\text{Lemma 10}) \\ \le C_B \left( \|\mathbf{z}\|^2 + \|\mathbf{z}\| \|\mathbf{z}_m\| + \|\mathbf{z}_m\|^2 \right)$$

Let $\mathbf{z} = \mathbf{w} \|\mathbf{z}_m\| \in A$, with $\|\mathbf{w}\| = 1$. Then

$$|\langle J'(\mathbf{u}_{m-1}), \mathbf{w} \rangle| \le 3C_B \|\mathbf{z}_m\| \quad \forall \mathbf{w} \in \{\mathbf{w} \in S_1 : \|\mathbf{w}\| = 1\}$$

Taking $\mathbf{w} = \mathbf{z}/\|\mathbf{z}\|$, with $\mathbf{z} \in S_1$, and $C = 3C_B > 0$ we obtain

$$|\langle J'(\mathbf{u}_{m-1}), \mathbf{z} \rangle| \le C \|\mathbf{z}_m\| \|\mathbf{z}\| \quad \forall \mathbf{z} \in S_1$$

□

Since $\mathbf{V}_{\parallel,\parallel}$ is reflexive, we can identify $\mathbf{V}_{\parallel,\parallel}^{**}$ with $\mathbf{V}_{\parallel,\parallel}$ and the duality pairing $\langle \cdot, \cdot \rangle \mathbf{V}_{\parallel,\parallel}^{**}, \mathbf{V}_{\parallel,\parallel}^{*}$ with $\langle \cdot, \cdot \rangle \mathbf{V}_{\parallel,\parallel}^{*}, \mathbf{V}_{\parallel,\parallel}$ (i.e. weak and weak-* topologies coincide on $\mathbf{V}_{\parallel,\parallel}^{*}$).

**Lemma 12** Assume that J satisfies (A1)-(A3). Then for every progressive Proper Generalized Decomposition {$\mathbf{u}_m$}$_{m\ge 1}$, over $S_1$, of $\mathbf{u} = \arg\min_{\mathbf{v}\in\mathbf{V}_{\parallel,\parallel}} J(\mathbf{v})$, the sequence {$J'(\mathbf{u}_m)$}$_{m\in\mathbb{N}}$ weakly-* converges to 0 in $\mathbf{V}_{\parallel,\parallel}^{*}$, that is, $\lim_{m\to\infty} \langle J'(\mathbf{u}_m), \mathbf{z} \rangle = 0$ for all $\mathbf{z}$ in a dense subset of $\mathbf{V}_{\parallel,\parallel}^{*}$.
*Proof* The sequence {$\mathbf{u}_m$}$_{m \in \mathbb{N}}$ being bounded, and since $J'$ is Lipschitz continuous on bounded sets, we have that there exists a constant $C > 0$ such that

$$ \|J'(\mathbf{u}_m)\| = \|J'(\mathbf{u}_m) - J'(\mathbf{u})\| \le C\|\mathbf{u} - \mathbf{u}_m\| $$

That proves that {$J'(\mathbf{u}_m)$} $\subset \mathbf{V}_{\|\cdot\|}^*$ is a bounded sequence. Since $\mathbf{V}_{\|\cdot\|}^*$ is also re-flexive, from any subsequence of {$J'(\mathbf{u}_m)$}$_{m \in \mathbb{N}}$, we can extract a further subsequence {$J'(\mathbf{u}_{m_k})$}$_{k \in \mathbb{N}}$ that weakly-* converges to an element $\varphi \in \mathbf{V}_{\|\cdot\|}^*$. By using Lemma 11, we have for all $\mathbf{z} \in S_1$,

$$ |\langle J'(\mathbf{u}_{m_k}), \mathbf{z} \rangle| \le C \|\mathbf{z}_{m_k+1}\| \|\mathbf{z}\|. $$

Taking the limit with $k$, and using Lemma 10, we obtain

$$ \langle \varphi, \mathbf{z} \rangle = 0 \quad \forall \mathbf{z} \in S_1, $$

By using assumption (B3), we conclude that $\varphi = 0$. Since from any subsequence of the initial sequence {$J'(\mathbf{u}_m)$}$_{m \in \mathbb{N}}$ we can extract a further subsequence that weakly-* converges to the same limit 0, then the whole sequence converges to 0. $\square$

**Lemma 13** Assume that $J$ satisfies (A1)-(A3) and consider a progressive Proper Generalized Decomposition {$\mathbf{u}_m$}$_{m \ge 1}$, over $S_1$, of $\mathbf{u} = \arg \min_{\mathbf{v} \in \mathbf{V}_{\|\cdot\|}} J(\mathbf{v})$ such that for the ellipticity constant $s$ of $J$ and $\underline{\alpha}(\mathbf{u}) = \alpha_1 \cdots \alpha_m \cdots$, one of the two following conditions hold:

(a) $s > 1$ and there exists a subsequence {$\alpha_{m_k}$}$_{k \in \mathbb{N}}$ such that $\alpha_{m_k} = l$ for all $k \ge 1$.

(b) $1 < s \le 2$ and there exists $k \ge 1$ such that $\underline{\alpha}(\mathbf{u}) = \alpha_1 \cdots \alpha_{k-1} \alpha^\infty$ where $\alpha \in \{c, r\}$.

Then, there exists a subsequence {$\mathbf{u}_{m_k}$}$_{k \in \mathbb{N}}$ such that

$$ \langle J'(\mathbf{u}_{m_k}), \mathbf{u}_{m_k} \rangle \to 0. $$

*Proof* First, assume that condition (a) holds. Recall that if $\alpha_m = l$ for some $m \ge 1$, the $\mathbf{u}_m$ is obtained by the minimization of $J$ on the closed subspace $\mathbf{U}(\mathbf{u}_{m-1}+\mathbf{z}_m) \subset \mathbf{V}_{\|\cdot\|}$. The global minimum is attained and unique, and it is characterized by $\langle J'(\mathbf{u}_m), \mathbf{v} \rangle = 0$ for all $\mathbf{v} \in \mathbf{U}(\mathbf{u}_{m-1}+\mathbf{z}_m)$. Thus, under condition (a), there exists a subsequence such that $\langle J'(\mathbf{u}_{m_k}), \mathbf{u}_{m_k} \rangle = 0$ for all $k \ge 1$. Now, we consider that statement (b) holds. Without loss of generality we may assume that $\underline{\alpha}(\mathbf{u}) = \alpha^\infty$ where $\alpha \in \{c, r\}$. In both cases, $\mathbf{u}_m = \sum_{k=1}^m \mathbf{z}_k$. Thus, we have

$$ \begin{aligned} |\langle J'(\mathbf{u}_m), \mathbf{u}_m \rangle| & \le \sum_{k=1}^m |\langle J'(\mathbf{u}_m), \mathbf{z}_k \rangle| \\ & \le C \sum_{k=1}^m \|\mathbf{z}_{m+1}\| \|\mathbf{z}_k\| \quad (\text{By Lemma 11}). \end{aligned} $$

Let $s^* > 1$ be such that $1/s^* + 1/s = 1$. By Holder's inequality, we have

$$ \begin{aligned} |\langle J'(\mathbf{u}_m), \mathbf{u}_m \rangle| & \le C \|\mathbf{z}_{m+1}\| m^{1/s^*} \left( \sum_{k=1}^m \|\mathbf{z}_k\|^s \right)^{1/s} \\ & = C \left( m \|\mathbf{z}_{m+1}\|^{s^*} \right)^{1/s^*} \left( \sum_{k=1}^m \|\mathbf{z}_k\|^s \right)^{1/s} . \end{aligned} \quad (18) $$
Proof The sequence {$\mathbf{u}_m$}$_{m \in \mathbb{N}}$ being bounded, and since $J'$ is Lipschitz continuous on bounded sets, we have that there exists a constant $C > 0$ such that

$$ \|J'(\mathbf{u}_m)\| = \|J'(\mathbf{u}_m) - J'(\mathbf{u})\| \le C\|\mathbf{u} - \mathbf{u}_m\| $$

That proves that {$J'(\mathbf{u}_m)$} $\subset \mathbf{V}_{\|\cdot\|}^*$ is a bounded sequence. Since $\mathbf{V}_{\|\cdot\|}^*$ is also re-flexive, from any subsequence of {$J'(\mathbf{u}_m)$}$_{m \in \mathbb{N}}$, we can extract a further subsequence {$J'(\mathbf{u}_{m_k})$}$_{k \in \mathbb{N}}$ that weakly-* converges to an element $\varphi \in \mathbf{V}_{\|\cdot\|}^*$. By using Lemma 11, we have for all $\mathbf{z} \in S_1$,

$$ |\langle J'(\mathbf{u}_{m_k}), \mathbf{z} \rangle| \le C \|\mathbf{z}_{m_k+1}\| \|\mathbf{z}\|. $$

Taking the limit with $k$, and using Lemma 10, we obtain

$$ \langle \varphi, \mathbf{z} \rangle = 0 \quad \forall \mathbf{z} \in S_1, $$

By using assumption (B3), we conclude that $\varphi = 0$. Since from any subsequence of the initial sequence {$J'(\mathbf{u}_m)$}$_{m \in \mathbb{N}}$ we can extract a further subsequence that weakly-* converges to the same limit 0, then the whole sequence converges to 0. $\square$

**Lemma 13** Assume that $J$ satisfies (A1)-(A3) and consider a progressive Proper Generalized Decomposition {$\mathbf{u}_m$}$_{m \ge 1}$, over $S_1$, of $\mathbf{u} = \arg \min_{\mathbf{v} \in \mathbf{V}_{\|\cdot\|}} J(\mathbf{v})$ such that for the ellipticity constant $s$ of $J$ and $\underline{\alpha}(\mathbf{u}) = \alpha_1 \cdots \alpha_m \cdots$, one of the two following conditions hold:

(a) $s > 1$ and there exists a subsequence {$\alpha_{m_k}$}$_{k \in \mathbb{N}}$ such that $\alpha_{m_k} = l$ for all $k \ge 1$.

(b) $1 < s \le 2$ and there exists $k \ge 1$ such that $\underline{\alpha}(\mathbf{u}) = \alpha_1 \cdots \alpha_{k-1} \alpha^\infty$ where $\alpha \in \{c, r\}$.

Then, there exists a subsequence {$\mathbf{u}_{m_k}$}$_{k \in \mathbb{N}}$ such that

$$ \langle J'(\mathbf{u}_{m_k}), \mathbf{u}_{m_k} \rangle \to 0. $$

Proof First, assume that condition (a) holds. Recall that if $\alpha_m = l$ for some $m \ge 1$, the $\mathbf{u}_m$ is obtained by the minimization of $J$ on the closed subspace $\mathbf{U}(\mathbf{u}_{m-1}+\mathbf{z}_m) \subset \mathbf{V}_{\|\cdot\|}$. The global minimum is attained and unique, and it is characterized by $\langle J'(\mathbf{u}_m), \mathbf{v} \rangle = 0$ for all $\mathbf{v} \in \mathbf{U}(\mathbf{u}_{m-1}+\mathbf{z}_m)$. Thus, under condition (a), there exists a subsequence such that $\langle J'(\mathbf{u}_{m_k}), \mathbf{u}_{m_k} \rangle = 0$ for all $k \ge 1$. Now, we consider that statement (b) holds. Without loss of generality we may assume that $\underline{\alpha}(\mathbf{u}) = \alpha^\infty$ where $\alpha \in \{c, r\}$. In both cases, $\mathbf{u}_m = \sum_{k=1}^m \mathbf{z}_k$. Thus, we have

$$ \begin{aligned} |\langle J'(\mathbf{u}_m), \mathbf{u}_m \rangle| & \le \sum_{k=1}^m |\langle J'(\mathbf{u}_m), \mathbf{z}_k \rangle| \\ & \le C \sum_{k=1}^m \|\mathbf{z}_{m+1}\| \|\mathbf{z}_k\| \quad (\text{By Lemma 11}). \end{aligned} $$

Let $s^* > 1$ be such that $1/s^* + 1/s = 1$. By Holder's inequality, we have

$$ \begin{aligned} |\langle J'(\mathbf{u}_m), \mathbf{u}_m \rangle| & \le C \|\mathbf{z}_{m+1}\| m^{1/s^*} \left( \sum_{k=1}^m \|\mathbf{z}_k\|^s \right)^{1/s} \\ & = C \left( m \|\mathbf{z}_{m+1}\|^{s^*} \right)^{1/s^*} \left( \sum_{k=1}^m \|\mathbf{z}_k\|^s \right)^{1/s} . \end{aligned} \quad (18) $$
From Lemma 10, we have $\sum_{k=1}^{\infty} \|\mathbf{z}_k\|^s < \infty$. Then there exists a subsequence such that $m_k \|\mathbf{z}_{m_k+1}\|^s \to 0$. For $1 < s \le 2$, we have $s \le s^*$. Since $\lim_{k \to \infty} \|\mathbf{z}_k\| = 0$, we have $\|\mathbf{z}_k\|^{s^*} \le \|\mathbf{z}_k\|^s$ for $k$ large enough, and therefore we also have $m_k \|\mathbf{z}_{m_k+1}\|^{s^*} \to 0$, which from (18) ends the proof of the lemma. $\square$

**Theorem 5** Assume that $J$ satisfies (A1)-(A3) and consider a progressive Proper Generalized Decomposition $\{\mathbf{u}_m\}_{m \ge 1}$, over $S_1$, of $\mathbf{u} = \arg \min_{\mathbf{v} \in \mathbf{V}_{\|\cdot\|}} J(\mathbf{v})$ such that the ellipticity constant of $J$ and $\underline{\alpha}(\mathbf{u})$ satisfy one of the following conditions:

(a) $s > 1$ and there exists a subsequence $\{\alpha_{m_k}\}_{k \in \mathbb{N}}$ such that $\alpha_{m_k} = l$ for all $k \ge 1$.

(b) $1 < s \le 2$ and there exists $k \ge 1$ such that $\underline{\alpha}(\mathbf{u}) = \alpha_1 \cdots \alpha_{k-1} \alpha^\infty$ where $\alpha \in \{c, r\}$.

(c) $s > 1$ and $\underline{\alpha}(\mathbf{u})$ is finite.

Then $\{\mathbf{u}_m\}_{m \ge 1}$, converges in $\mathbf{V}_{\|\cdot\|}$ to $\mathbf{u}$, that is,

$$ \lim_{m \to \infty} \|\mathbf{u} - \mathbf{u}_m\| = 0. $$

*Proof* From Lemma 8, $\{J(\mathbf{u}_m)\}$ is a non increasing sequence. If (c) holds, there exists $m$ such that $J(\mathbf{u}_m) = J(\mathbf{u}_{m-1})$ and from Lemma 8, we have $\mathbf{u}_m = \mathbf{u}$, which ends the proof. Let us now suppose that $J(\mathbf{u}_m) < J(\mathbf{u}_{m-1})$ for all $m$. $J(\mathbf{u}_m)$ is a strictly decreasing sequence which is bounded below by $J(\mathbf{u})$. Then, there exists

$$ J^* = \lim_{m \to \infty} J(\mathbf{u}_m) \ge J(\mathbf{u}). $$

If $J^* = J(\mathbf{u})$, Lemma 4 allows to conclude that $\{\mathbf{u}_m\}$ strongly converges to $\mathbf{u}$. Therefore, it remains to prove that $J^* = J(\mathbf{u})$. By the convexity of $J$, we have

$$ J(\mathbf{u}_m) - J(\mathbf{u}) \le \langle J'(\mathbf{u}_m), \mathbf{u}_m - \mathbf{u} \rangle = \langle J'(\mathbf{u}_m), \mathbf{u}_m \rangle - \langle J'(\mathbf{u}_m), \mathbf{u} \rangle $$

By Lemmas 12 and 13, we have that there exists a subsequence $\{\mathbf{u}_{m_k}\}_{k \in \mathbb{N}}$ such that $\langle J'(\mathbf{u}_{m_k}), \mathbf{u}_{m_k} \rangle \to 0$ and $\langle J'(\mathbf{u}_{m_k}), \mathbf{u} \rangle \to 0$, and therefore

$$ J^* - J(\mathbf{u}) = \lim_{k \to \infty} J(\mathbf{u}_{m_k}) - J(\mathbf{u}) \le 0 $$

Since we already had $J^* \ge J(\mathbf{u})$, this yields $J^* = J(\mathbf{u})$, which ends the proof. $\square$

# 5 Examples

## 5.1 On the Singular Value Decomposition in $L^p$ spaces for $p \ge 2$

A Banach space $V$ is said to be smooth if for any linearly independent elements $x, y \in V$, the function $\phi(t) = \|x - ty\|$ is differentiable. A Banach space is said to be uniformly smooth if its modulus of smoothness

$$ \rho(\tau) = \sup_{\substack{x,y \in V \\ \|x\|=\|y\|=1}} \left\{ \frac{\|x + \tau y\| + \|x - \tau y\|}{2} - 1 \right\}, \quad \tau > 0, $$

satisfies the condition

$$ \lim_{\tau \to 0} \frac{\rho(\tau)}{\tau} = 0. $$
From Lemma 10, we have $\sum_{k=1}^{\infty} \|z_k\|^s < \infty$. Then there exists a subsequence such that $m_k \|z_{m_k+1}\|^s \to 0$. For $1 < s \le 2$, we have $s \le s^*$. Since $\lim_{k \to \infty} \|z_k\| = 0$, we have $\|z_k\|^{s^*} \le \|z_k\|^s$ for $k$ large enough, and therefore we also have $m_k \|z_{m_k+1}\|^{s^*} \to 0$, which from (18) ends the proof of the lemma. $\square$

**Theorem 5** Assume that $J$ satisfies (A1)-(A3) and consider a progressive Proper Generalized Decomposition $\{u_m\}_{m \ge 1}$, over $S_1$, of $u = \arg \min_{v \in V_{\|\cdot\|}} J(v)$ such that the ellipticity constant $s$ of $J$ and $\underline{\alpha}(u)$ satisfy one of the following conditions:

(a) $s > 1$ and there exists a subsequence $\{\alpha_{m_k}\}_{k \in \mathbb{N}}$ such that $\alpha_{m_k} = l$ for all $k \ge 1$.

(b) $1 < s \le 2$ and there exists $k \ge 1$ such that $\underline{\alpha}(u) = \alpha_1 \cdots \alpha_{k-1} \alpha^\infty$ where $\alpha \in \{c, r\}$.

(c) $s > 1$ and $\underline{\alpha}(u)$ is finite.

Then $\{u_m\}_{m \ge 1}$, converges in $V_{\|\cdot\|}$ to $u$, that is,

$$ \lim_{m \to \infty} \|u - u_m\| = 0. $$

*Proof* From Lemma 8, $\{J(u_m)\}$ is a non increasing sequence. If (c) holds, there exists $m$ such that $J(u_m) = J(u_{m-1})$ and from Lemma 8, we have $u_m = u$, which ends the proof. Let us now suppose that $J(u_m) < J(u_{m-1})$ for all $m$. $J(u_m)$ is a strictly decreasing sequence which is bounded below by $J(u)$. Then, there exists

$$ J^* = \lim_{m \to \infty} J(u_m) \ge J(u). $$

If $J^* = J(u)$, Lemma 4 allows to conclude that $\{u_m\}$ strongly converges to $u$. Therefore, it remains to prove that $J^* = J(u)$. By the convexity of $J$, we have

$$ J(u_m) - J(u) \le \langle J'(u_m), u_m - u \rangle = \langle J'(u_m), u_m \rangle - \langle J'(u_m), u \rangle $$

By Lemmas 12 and 13, we have that there exists a subsequence $\{u_{m_k}\}_{k \in \mathbb{N}}$ such that $\langle J'(u_{m_k}), u_{m_k} \rangle \to 0$ and $\langle J'(u_{m_k}), u \rangle \to 0$, and therefore

$$ J^* - J(u) = \lim_{k \to \infty} J(u_{m_k}) - J(u) \le 0 $$

Since we already had $J^* \ge J(u)$, this yields $J^* = J(u)$, which ends the proof. $\square$

# 5 Examples

## 5.1 On the Singular Value Decomposition in $L^p$ spaces for $p \ge 2$

A Banach space $V$ is said to be smooth if for any linearly independent elements $x, y \in V$, the function $\phi(t) = \|x - ty\|$ is differentiable. A Banach space is said to be uniformly smooth if its modulus of smoothness

$$ \rho(\tau) = \sup_{\substack{x,y \in V \\ \|x\| = \|y\| = 1}} \left\{ \frac{\|x + \tau y\| + \|x - \tau y\|}{2} - 1 \right\}, \quad \tau > 0, $$

satisfies the condition

$$ \lim_{\tau \to 0} \frac{\rho(\tau)}{\tau} = 0. $$
In uniformly smooth spaces, and only in such spaces, the norm is uniformly Fréchet differentiable. It can be shown that the $L^p$-spaces for $1 < p < \infty$ are uniformly smooth (see Corollary 6.12 in [27]).

Following section 2.2.1, we introduce the tensor product of Lebesgue spaces

$$L_{\mu}^{p}(I_{1} \times I_{2}) = L_{\mu_{1}}^{p}(I_{1}) \otimes_{\Delta_{p}} L_{\mu_{2}}^{p}(I_{2}) = L_{\mu_{1}}^{p}(I_{1}, L_{\mu_{2}}^{p}(I_{2})),$$

with $p \ge 2$, and $\mu = \mu_1 \otimes \mu_2$ a finite product measure. Recall that

$$\|\mathbf{v}\|_{\Delta_p} = \left( \int_{I_1 \times I_2} |\mathbf{v}(x)|^p d\mu(x) \right)^{1/p}$$

Let $\mathbf{u}$ be a given function in $L_{\mu}^{p}(I_{1} \times I_{2})$. We introduce the functional $J: L_{\mu}^{p}(I_{1} \times I_{2}) \to \mathbb{R}$ defined by

$$J(\mathbf{v}) = \frac{1}{p} \|\mathbf{v} - \mathbf{u}\|_{\Delta_p}^p.$$

Let $G: L_{\mu}^{p}(I_{1} \times I_{2}) \to \mathbb{R}$ be the functional given by the $p$-norm $G(\mathbf{v}) = \|\mathbf{v}\|_{\Delta_p}$. It is well known (see for example page 170 in [19]) that $G$ is Fréchet differentiable, with

$$G'(\mathbf{v}) = \mathbf{v} |\mathbf{v}|^{p-2} \|\mathbf{v}\|_{\Delta_p}^{1-p} \in L_{\mu}^{q}(I_{1} \times I_{2}),$$

with $q$ such that $1/q+1/p = 1$. We denote by $C^k$ the set of $k$-times Fréchet differentiable functionals from $L_{\mu}^{p}(I_{1} \times I_{2})$ to $\mathbb{R}$. Then, if $p$ is an even integer, $G \in C^{\infty}$. Otherwise, when $p$ is not an even integer, the following statements hold (see [5] and 13.13 in [23]):

(a) If $p$ is an integer, $G$ is $(p-1)$-times differentiable with Lipschitzian highest Fréchet derivative.

(b) Otherwise, $G$ is $[p]$-times Fréchet differentiable with highest derivative being Hölderian of order $p - [p]$.

(c) $G$ has no higher Hölder Fréchet differentiability properties.

As a consequence we obtain that $G \in C^2$ for all $p \ge 2$, and the functional $J$ is also Fréchet differentiable with Fréchet derivative given by $J'(\mathbf{v}) = G(\mathbf{v} - \mathbf{u})^{p-1}G'(\mathbf{v} - \mathbf{u})$, that is,

$$\langle J'(\mathbf{v}), \mathbf{w} \rangle = \int_{I_1 \times I_2} (\mathbf{v} - \mathbf{u}) |\mathbf{v} - \mathbf{u}|^{p-2} \mathbf{w} \, d\mu.$$

Thus, $J$ satisfies assumption (A1). It is well-known that if a functional $F: V \to W$, where $V$ and $W$ are Banach spaces, is Fréchet differentiable at $v \in V$, then it is also locally Lipschitz continuous at $v \in V$. Thus, if $p \ge 2$, we have that $J' \in C^1$, and as a consequence $J'$ satisfies (A3).

Finally, in order to prove the convergence of the (updated) progressive PGD for each $\mathbf{u} \in L_{\mu}^{p}(I_{1} \times I_{2})$ over $S_{1} = T_{(r_{1}, r_{2})}(L_{\mu_{1}}^{p}(I_{1}) \otimes_{a} L_{\mu_{2}}^{p}(I_{2}))$, where $(r_{1}, r_{2}) \in \mathbb{N}^{2}$, we have to verify that (A2) on $J$ is satisfied. Since there exists a constant $\alpha_{p} > 0$ such that for all $s, t \in \mathbb{R}$,

$$(|s|^{p-2}s - |t|^{p-2}t)(s-t) \ge \alpha_p |s-t|^p$$

(see for example (7.1) in [8]), then, for all $\mathbf{v}, \mathbf{w} \in L_{\mu}^{p}(I_{1} \times I_{2})$,

$$\langle J'(\mathbf{v}) - J'(\mathbf{w}), \mathbf{v} - \mathbf{w} \rangle \ge \alpha_p \|\mathbf{v} - \mathbf{w}\|^p,$$

which proves the ellipticity property of $J$, and assumption (A2) holds.

From Theorem 5, we conclude that the (updated) progressive Proper Generalized Decomposition converges
In uniformly smooth spaces, and only in such spaces, the norm is uniformly Fréchet differentiable. It can be shown that the $L^p$-spaces for $1 < p < \infty$ are uniformly smooth (see Corollary 6.12 in [27]).

Following section 2.2.1, we introduce the tensor product of Lebesgue spaces

$$L_{\mu}^{p}(I_{1} \times I_{2}) = L_{\mu_{1}}^{p}(I_{1}) \otimes_{\Delta_{p}} L_{\mu_{2}}^{p}(I_{2}) = L_{\mu_{1}}^{p}(I_{1}, L_{\mu_{2}}^{p}(I_{2})),$$

with $p \ge 2$, and $\mu = \mu_1 \otimes \mu_2$ a finite product measure. Recall that

$$\|\mathbf{v}\|_{\Delta_p} = \left( \int_{I_1 \times I_2} |\mathbf{v}(x)|^p d\mu(x) \right)^{1/p}$$

Let $\mathbf{u}$ be a given function in $L_{\mu}^{p}(I_{1} \times I_{2})$. We introduce the functional $J: L_{\mu}^{p}(I_{1} \times I_{2}) \to \mathbb{R}$ defined by

$$J(\mathbf{v}) = \frac{1}{p} \|\mathbf{v} - \mathbf{u}\|_{\Delta_p}^p.$$

Let $G: L_{\mu}^{p}(I_{1} \times I_{2}) \to \mathbb{R}$ be the functional given by the $p$-norm $G(\mathbf{v}) = \|\mathbf{v}\|_{\Delta_p}$. It is well known (see for example page 170 in [19]) that $G$ is Fréchet differentiable, with

$$G'(\mathbf{v}) = \mathbf{v} |\mathbf{v}|^{p-2} \|\mathbf{v}\|_{\Delta_p}^{1-p} \in L_{\mu}^{q}(I_{1} \times I_{2}),$$

with $q$ such that $1/q+1/p = 1$. We denote by $C^k$ the set of $k$-times Fréchet differentiable functionals from $L_{\mu}^{p}(I_{1} \times I_{2})$ to $\mathbb{R}$. Then, if $p$ is an even integer, $G \in C^{\infty}$. Otherwise, when $p$ is not an even integer, the following statements hold (see [5] and 13.13 in [23]):

(a) If $p$ is an integer, $G$ is $(p-1)$-times differentiable with Lipschitzian highest Fréchet derivative.

(b) Otherwise, $G$ is $[p]$-times Fréchet differentiable with highest derivative being Hölderian of order $p - [p]$.

(c) $G$ has no higher Hölder Fréchet differentiability properties.

As a consequence we obtain that $G \in C^2$ for all $p \ge 2$, and the functional $J$ is also Fréchet differentiable with Fréchet derivative given by $J'(\mathbf{v}) = G(\mathbf{v} - \mathbf{u})^{p-1}G'(\mathbf{v} - \mathbf{u})$, that is,

$$\langle J'(\mathbf{v}), \mathbf{w} \rangle = \int_{I_1 \times I_2} (\mathbf{v} - \mathbf{u}) |\mathbf{v} - \mathbf{u}|^{p-2} \mathbf{w} \, d\mu.$$

Thus, $J$ satisfies assumption (A1). It is well-known that if a functional $F: V \to W$, where $V$ and $W$ are Banach spaces, is Fréchet differentiable at $v \in V$, then it is also locally Lipschitz continuous at $v \in V$. Thus, if $p \ge 2$, we have that $J' \in C^1$, and as a consequence $J'$ satisfies (A3).

Finally, in order to prove the convergence of the (updated) progressive PGD for each $\mathbf{u} \in L_{\mu}^{p}(I_{1} \times I_{2})$ over $S_{1} = T_{(r_{1}, r_{2})}(L_{\mu_{1}}^{p}(I_{1}) \otimes_{a} L_{\mu_{2}}^{p}(I_{2}))$, where $(r_{1}, r_{2}) \in \mathbb{N}^{2}$, we have to verify that (A2) on $J$ is satisfied. Since there exists a constant $\alpha_{p} > 0$ such that for all $s, t \in \mathbb{R}$,

$$(|s|^{p-2}s - |t|^{p-2}t)(s-t) \ge \alpha_p |s-t|^p$$

(see for example (7.1) in [8]), then, for all $\mathbf{v}, \mathbf{w} \in L_{\mu}^{p}(I_{1} \times I_{2})$,

$$\langle J'(\mathbf{v}) - J'(\mathbf{w}), \mathbf{v} - \mathbf{w} \rangle \ge \alpha_p \|\mathbf{v} - \mathbf{w}\|^p,$$

which proves the ellipticity property of $J$, and assumption (A2) holds.

From Theorem 5, we conclude that the (updated) progressive Proper Generalized Decomposition converges
- for all $p \ge 2$ if conditions (a) or (c) of Theorem 5 hold.

- for $p = 2$ if condition (b) of Theorem 5 holds.

Let us detail the application of the progressive PGD over

$$S_1 = \mathcal{T}_{(r_1, r_2)}(L_{\mu_1}^p(I_1) \otimes_a L_{\mu_2}^p(I_2)).$$

We claim that in dimension $d = 2$, we can only consider the case $r_1 = r_2 = r$. The claim follows from the fact that (see [14]) for each $\mathbf{v} \in L_{\mu_1}^p(I_1) \otimes_a L_{\mu_2}^p(I_2)$, there exist two minimal subspaces $U_{j,\min}(\mathbf{v})$, $j = 1, 2$, with $\dim U_{1,\min}(\mathbf{v}) = \dim U_{2,\min}(\mathbf{v})$ and such that $\mathbf{v} \in U_{1,\min}(\mathbf{v}) \otimes_a U_{2,\min}(\mathbf{v})$. In consequence, for a fixed $r \in \mathbb{N}$ and for

$$\mathbf{u} \in L_{\mu}^{p}(I_{1} \times I_{2}) \setminus L_{\mu_{1}}^{p}(I_{1}) \otimes_{a} L_{\mu_{2}}^{p}(I_{2})$$

we let

$$\mathbf{u}_1 \in \arg \min_{\mathbf{z} \in \mathcal{T}_{(r,r)}(L_{\mu_1}^p(I_1) \otimes_a L_{\mu_2}^p(I_2))} J(\mathbf{z}).$$

Then there exist two bases $\{u_1^{(j)}, \dots, u_r^{(j)}\} \subset L_{\mu_j}^p(I_j)$ of $U_{j,\min}(\mathbf{v})$, for $j = 1, 2$, such that

$$\mathbf{u}_1 = \sum_{k=1}^{r} \sum_{l=1}^{r} \sigma_{k,l} u_k^{(1)} \otimes u_l^{(2)},$$

and $\mathbf{u} - \mathbf{u}_1 \notin L_{\mu_1}^p(I_1) \otimes_a L_{\mu_2}^p(I_2)$. Proceeding inductively we can write

$$\mathbf{u}_m = \sum_{k=1}^{mr} \sum_{l=1}^{mr} \sigma_{k,l} u_k^{(1)} \otimes u_l^{(2)}.$$

At step $m$, an example of update of type $\alpha_m = r$ would consist in updating the coefficients $\{\sigma_{k,l} : k, l \in \{(m-1)r+1, \dots, mr\}\}$. An example of update of type $\alpha_m = l$ would consist in updating the whole set of coefficients $\{\sigma_{k,l} : k, l \in \{1, \dots, mr\}\}$.

In the case $p = 2$ and when we take orthonormal bases, it corresponds to the classical SVD decomposition in the Hilbert space $L_{\mu}^2(I_1 \times I_2)$. In this case we have

$$\mathbf{u}_m = \sum_{j=1}^{mr} \sigma_j u_j^{(1)} \otimes u_j^{(2)}.$$

where $\sigma_j = |\langle \mathbf{u}, u_1^{(j)} \otimes u_2^{(j)} \rangle|$, for $1 \le j \le mr$.

In this sense, the progressive PGD can be interpreted as a SVD decomposition of a function $\mathbf{u}$ in a $L^p$-space where $p \ge 2$. Let us recall that for $p > 2$, an update strategy of type $(l)$ is required for applying Theorem 5 (at least for a subsequence of iterates).

The above results can be naturally extended to tensor product of Lebesgue spaces, $\mathcal{A} \otimes_{k=1}^d L_{\mu_k}^p(I_k) \|\cdot\|_{\Delta_p}$ with $d > 2$ and $S_1 = \mathcal{R}_1(\mathcal{A} \otimes_{k=1}^d L_{\mu_k}^p(I_k))$, leading to a generalization of multidimensional singular value decomposition introduced in [15] for the case of Hilbert tensor spaces.
- for all $p \ge 2$ if conditions (a) or (c) of Theorem 5 hold.

- for $p = 2$ if condition (b) of Theorem 5 holds.

Let us detail the application of the progressive PGD over

$$S_1 = \mathcal{T}_{(r_1, r_2)}(L_{\mu_1}^p(I_1) \otimes_a L_{\mu_2}^p(I_2)).$$

We claim that in dimension $d = 2$, we can only consider the case $r_1 = r_2 = r$. The claim follows from the fact that (see [14]) for each $\mathbf{v} \in L_{\mu_1}^p(I_1) \otimes_a L_{\mu_2}^p(I_2)$, there exist two minimal subspaces $U_{j,\min}(\mathbf{v})$, $j = 1, 2$, with $\dim U_{1,\min}(\mathbf{v}) = \dim U_{2,\min}(\mathbf{v})$ and such that $\mathbf{v} \in U_{1,\min}(\mathbf{v}) \otimes_a U_{2,\min}(\mathbf{v})$. In consequence, for a fixed $r \in \mathbb{N}$ and for

$$\mathbf{u} \in L_{\mu}^{p}(I_{1} \times I_{2}) \setminus L_{\mu_{1}}^{p}(I_{1}) \otimes_{a} L_{\mu_{2}}^{p}(I_{2})$$

we let

$$\mathbf{u}_1 \in \arg \min_{\mathbf{z} \in \mathcal{T}_{(r,r)}(L_{\mu_1}^p(I_1) \otimes_a L_{\mu_2}^p(I_2))} J(\mathbf{z}).$$

Then there exist two bases $\{u_1^{(j)}, \dots, u_r^{(j)}\} \subset L_{\mu_j}^p(I_j)$ of $U_{j,\min}(\mathbf{v})$, for $j = 1, 2$, such that

$$\mathbf{u}_1 = \sum_{k=1}^{r} \sum_{l=1}^{r} \sigma_{k,l} u_k^{(1)} \otimes u_l^{(2)},$$

and $\mathbf{u} - \mathbf{u}_1 \notin L_{\mu_1}^p(I_1) \otimes_a L_{\mu_2}^p(I_2)$. Proceeding inductively we can write

$$\mathbf{u}_m = \sum_{k=1}^{mr} \sum_{l=1}^{mr} \sigma_{k,l} u_k^{(1)} \otimes u_l^{(2)}.$$

At step $m$, an example of update of type $\alpha_m = r$ would consist in updating the coefficients $\{\sigma_{k,l} : k, l \in \{(m-1)r+1, \dots, mr\}\}$. An example of update of type $\alpha_m = l$ would consist in updating the whole set of coefficients $\{\sigma_{k,l} : k, l \in \{1, \dots, mr\}\}$.

In the case $p = 2$ and when we take orthonormal bases, it corresponds to the classical SVD decomposition in the Hilbert space $L_{\mu}^2(I_1 \times I_2)$. In this case we have

$$\mathbf{u}_m = \sum_{j=1}^{mr} \sigma_j u_j^{(1)} \otimes u_j^{(2)}.$$

where $\sigma_j = |\langle \mathbf{u}, u_1^{(j)} \otimes u_2^{(j)} \rangle|$, for $1 \le j \le mr$.

In this sense, the progressive PGD can be interpreted as a SVD decomposition of a function $\mathbf{u}$ in a $L^p$-space where $p \ge 2$. Let us recall that for $p > 2$, an update strategy of type $(l)$ is required for applying Theorem 5 (at least for a subsequence of iterates).

The above results can be naturally extended to tensor product of Lebesgue spaces, $\mathcal{A} \otimes_{k=1}^d L_{\mu_k}^p(I_k) \|\cdot\|_{\Delta_p}$ with $d > 2$ and $S_1 = \mathcal{R}_1(\mathcal{A} \otimes_{k=1}^d L_{\mu_k}^p(I_k))$, leading to a generalization of multidimensional singular value decomposition introduced in [15] for the case of Hilbert tensor spaces.
## 5.2 Nonlinear Laplacian

We here present an example taken from [8]. We refer to section 2.2.2 for the introduction to the properties of Sobolev spaces. Let $\Omega = \Omega_1 \times \dots \times \Omega_d$. Given some $p > 2$, we let $V_{\parallel \cdot \parallel} = H_0^{1,p}(\Omega)$, which is the closure of $C_c^\infty(\Omega)$ (functions in $C^\infty(\Omega)$ with compact support in $\Omega$) in $H^{1,p}(\Omega)$ with respect to the norm in $H^{1,p}(\Omega)$. We equip $H_0^{1,p}(\Omega)$ with the norm

$$ \|v\| = \left( \sum_{k=1}^{d} \| \partial_{x_k}(v) \|_{L^p(\Omega)} \right)^{1/p} $$

which is equivalent to the norm $\| \cdot \|_{1,p}$ on $H^{1,p}(\Omega)$ introduced in section 2.2.2. We then introduce the functional $J: V_{\parallel \cdot \parallel} \to \mathbb{R}$ defined by

$$ J(v) = \frac{1}{p} \|v\|^p - \langle f, v \rangle, $$

with $f \in V_{\parallel \cdot \parallel}^*$. Its Fréchet differential is

$$ J'(v) = A(v) - f $$

where

$$ A(v) = - \sum_{k=1}^{d} \frac{\partial}{\partial x_k} \left( \left| \frac{\partial v}{\partial x_k} \right|^{p-2} \frac{\partial v}{\partial x_k} \right) $$

$A$ is called the $p$-Laplacian. Assumptions (A1)-(A3) on the functional are satisfied (see [8]). Assumption (B3) on the set $\mathcal{R}_1(a \otimes_{j=1}^d H_0^{m,p}(\Omega_j))$ is also satisfied. Indeed, it can be easily proved from Proposition 3 that the set $\mathcal{R}_1(a \otimes_{j=1}^d H_0^{1,p}(\Omega_j))$ is weakly closed in $(H_0^{1,p}(\Omega), \| \cdot \|_{1,p})$. Since the norm $\| \cdot \|$ is equivalent to $\| \cdot \|_{1,p}$ on $H_0^{1,p}(\Omega)$, it is also weakly closed in $(H_0^{1,p}(\Omega), \| \cdot \|)$.

Then, from Theorem 5, the progressive PGD converges if there exists a subsequence of updates of type (l).

## 5.3 Linear elliptic variational problems on Hilbert spaces

Let $V_{\parallel \cdot \parallel} = \overline{V_1 \otimes_a \dots \otimes_a V_d}^{\parallel \cdot \parallel}$ be a tensor product of Hilbert spaces. We consider the following problem

$$ J(u) = \min_{v \in K} J(v), \quad J(v) = \frac{1}{2} a(v, v) - l(v) $$

where $K \subset V_{\parallel \cdot \parallel}$, $a: V_{\parallel \cdot \parallel} \times V_{\parallel \cdot \parallel} \to \mathbb{R}$ is a coercive continuous symmetric bilinear form,

$$ a(v, v) \geq \alpha \|v\|^2 \quad \forall v \in V_{\parallel \cdot \parallel}, $$

$$ a(v, w) \leq \beta \|v\| \|w\| \quad \forall v, w \in V_{\parallel \cdot \parallel}, $$

$l: V_{\parallel \cdot \parallel} \to \mathbb{R}$ is a continuous linear form,

$$ l(v) \leq \gamma \|v\| \quad \forall v \in V_{\parallel \cdot \parallel}. $$
## 5.2 Nonlinear Laplacian

We here present an example taken from [8]. We refer to section 2.2.2 for the introduction to the properties of Sobolev spaces. Let $\Omega = \Omega_1 \times \dots \times \Omega_d$. Given some $p > 2$, we let $V_{\parallel \cdot \parallel} = H_0^{1,p}(\Omega)$, which is the closure of $C_c^\infty(\Omega)$ (functions in $C^\infty(\Omega)$ with compact support in $\Omega$) in $H^{1,p}(\Omega)$ with respect to the norm in $H^{1,p}(\Omega)$. We equip $H_0^{1,p}(\Omega)$ with the norm

$$ \|v\| = \left( \sum_{k=1}^{d} \| \partial_{x_k}(v) \|_{L^p(\Omega)} \right)^{1/p} $$

which is equivalent to the norm $\| \cdot \|_{1,p}$ on $H^{1,p}(\Omega)$ introduced in section 2.2.2. We then introduce the functional $J: V_{\parallel \cdot \parallel} \to \mathbb{R}$ defined by

$$ J(v) = \frac{1}{p} \|v\|^p - \langle f, v \rangle, $$

with $f \in V_{\parallel \cdot \parallel}^*$. Its Fréchet differential is

$$ J'(v) = A(v) - f $$

where

$$ A(v) = - \sum_{k=1}^{d} \frac{\partial}{\partial x_k} \left( \left| \frac{\partial v}{\partial x_k} \right|^{p-2} \frac{\partial v}{\partial x_k} \right) $$

$A$ is called the $p$-Laplacian. Assumptions (A1)-(A3) on the functional are satisfied (see [8]). Assumption (B3) on the set $\mathcal{R}_1(a \otimes_{j=1}^d H_0^{m,p}(\Omega_j))$ is also satisfied. Indeed, it can be easily proved from Proposition 3 that the set $\mathcal{R}_1(a \otimes_{j=1}^d H_0^{1,p}(\Omega_j))$ is weakly closed in $(H_0^{1,p}(\Omega), \| \cdot \|_{1,p})$. Since the norm $\| \cdot \|$ is equivalent to $\| \cdot \|_{1,p}$ on $H_0^{1,p}(\Omega)$, it is also weakly closed in $(H_0^{1,p}(\Omega), \| \cdot \|)$.

Then, from Theorem 5, the progressive PGD converges if there exists a subsequence of updates of type (l).

## 5.3 Linear elliptic variational problems on Hilbert spaces

Let $V_{\parallel \cdot \parallel} = \overline{V_1 \otimes_a \dots \otimes_a V_d}^{\parallel \cdot \parallel}$ be a tensor product of Hilbert spaces. We consider the following problem

$$ J(u) = \min_{v \in K} J(v), \quad J(v) = \frac{1}{2} a(v, v) - \ell(v) $$

where $K \subset V_{\parallel \cdot \parallel}$, $a: V_{\parallel \cdot \parallel} \times V_{\parallel \cdot \parallel} \to \mathbb{R}$ is a coercive continuous symmetric bilinear form,

$$ a(v, v) \geq \alpha \|v\|^2 \quad \forall v \in V_{\parallel \cdot \parallel}, $$

$$ a(v, w) \leq \beta \|v\| \|w\| \quad \forall v, w \in V_{\parallel \cdot \parallel}, $$

$\ell: V_{\parallel \cdot \parallel} \to \mathbb{R}$ is a continuous linear form,

$$ \ell(v) \leq \gamma \|v\| \quad \forall v \in V_{\parallel \cdot \parallel}. $$
Case where $K$ is a closed and convex subset of $V_{\parallel \cdot \parallel}$. The solution $u$ is equivalently characterized by the variational inequality

$$a(\mathbf{u}, \mathbf{v} - \mathbf{u}) \geq \ell(\mathbf{v} - \mathbf{u}) \quad \forall \mathbf{v} \in K$$

In order to apply the results of the present paper, we have to recast the problem as an optimization problem in $V_{\parallel \cdot \parallel}$. We introduce a convex and Fréchet differentiable functional $j: V_{\parallel \cdot \parallel} \to \mathbb{R}$ with Fréchet differential $j': V_{\parallel \cdot \parallel} \to V_{\parallel \cdot \parallel}^*$, such that $j(v) = 0$ if $v \in K$ and $j(v) > 0$ if $v \notin K$. We further assume that $j'$ is Lipschitz on bounded sets. We let $j_{\epsilon}(v) = \epsilon^{-1}j(v)$, with $\epsilon > 0$, and introduce the following penalized problem

$$J_{\epsilon}(\mathbf{u}_{\epsilon}) = \min_{\mathbf{v} \in V_{\parallel \cdot \parallel}} J_{\epsilon}(\mathbf{v}), \quad J_{\epsilon}(\mathbf{v}) = J(\mathbf{v}) + j_{\epsilon}(\mathbf{v})$$

As $\epsilon \to 0$, $j_{\epsilon}$ tends to the indicator function of set $K$ and $\mathbf{u}_{\epsilon} \to \mathbf{u}$ (see e.g. [16]). Assumptions (A1)-(A2) are verified since $J_{\epsilon}$ is Fréchet differentiable with Fréchet differential $J'_{\epsilon}: V_{\parallel \cdot \parallel} \to V_{\parallel \cdot \parallel}^*$ defined by

$$\langle J'_{\epsilon}(\mathbf{v}), \mathbf{z} \rangle = a(\mathbf{v}, \mathbf{z}) - \ell(\mathbf{z}) + \langle j'_{\epsilon}(\mathbf{v}), \mathbf{z} \rangle,$$

and $J_{\epsilon}$ is elliptic since

$$\langle J'_{\epsilon}(\mathbf{v}) - J'_{\epsilon}(\mathbf{w}), \mathbf{v} - \mathbf{w} \rangle = a(\mathbf{v} - \mathbf{w}, \mathbf{v} - \mathbf{w}) + \langle j'_{\epsilon}(\mathbf{v}) - j'_{\epsilon}(\mathbf{w}), \mathbf{v} - \mathbf{w} \rangle \geq \alpha \|\mathbf{v} - \mathbf{w}\|^2$$

Assumption (A3) comes from the continuity of $a$ and $\ell$ and from the properties of $j'$.

Case where $K = V_{\parallel \cdot \parallel}$. If $K = V_{\parallel \cdot \parallel}$, we recover the classical case of linear elliptic variational problems on Hilbert spaces analyzed in [15]. In this case, the bilinear form $a$ defines a norm $\|\mathbf{v}\|_a = \sqrt{a(\mathbf{v}, \mathbf{v})}$ on $V_{\parallel \cdot \parallel}$, equivalent to the norm $\|\cdot\|$. The functional $J$ is here equal to

$$J(\mathbf{v}) = \frac{1}{2} \|\mathbf{u} - \mathbf{v}\|_a^2 - \frac{1}{2} \|\mathbf{u}\|_a^2$$

The progressive PGD can be interpreted as a generalized Eckart-Young decomposition (generalized singular value decomposition) with respect to this non usual metric, and defined progressively by

$$\|\mathbf{u} - \mathbf{u}_m\|_a^2 = \min_{\mathbf{z} \in S_1} \|\mathbf{u} - \mathbf{u}_{m-1} - \mathbf{z}\|_a^2$$

We have

$$J(\mathbf{u}_{m-1}) - J(\mathbf{u}_m) = \frac{1}{2} \|\mathbf{z}_m\|_a^2 := \frac{1}{2} \sigma_m^2$$

and

$$\|\mathbf{u} - \mathbf{u}_m\|_a^2 = \|\mathbf{u}\|_a^2 - \sum_{k=1}^{m} \sigma_k^2 \underset{m \to \infty}{\longrightarrow} 0$$

where $\sigma_m$ can be interpreted as the dominant singular value of $(\mathbf{u} - \mathbf{u}_{m-1}) \in V_{\parallel \cdot \parallel}$. The PGD method has been successfully applied to this class of problems in different contexts: separation of spatial coordinates for the solution of Poisson equation in high dimension [2, 6], separation of physical variables and random parameters for the solution of parameterized stochastic partial differential equations [28].
Case where K is a closed and convex subset of $V_{\parallel \cdot \parallel}$. The solution $\mathbf{u}$ is equivalently characterized by the variational inequality

$$a(\mathbf{u}, \mathbf{v} - \mathbf{u}) \geq \ell(\mathbf{v} - \mathbf{u}) \quad \forall \mathbf{v} \in K$$

In order to apply the results of the present paper, we have to recast the problem as an optimization problem in $V_{\parallel \cdot \parallel}$. We introduce a convex and Fréchet differentiable functional $j: V_{\parallel \cdot \parallel} \to \mathbb{R}$ with Fréchet differential $j': V_{\parallel \cdot \parallel} \to V_{\parallel \cdot \parallel}^*$, such that $j(\mathbf{v}) = 0$ if $\mathbf{v} \in K$ and $j(\mathbf{v}) > 0$ if $\mathbf{v} \notin K$. We further assume that $j'$ is Lipschitz on bounded sets. We let $j_{\epsilon}(\mathbf{v}) = \epsilon^{-1}j(\mathbf{v})$, with $\epsilon > 0$, and introduce the following penalized problem

$$J_{\epsilon}(\mathbf{u}_{\epsilon}) = \min_{\mathbf{v} \in V_{\parallel \cdot \parallel}} J_{\epsilon}(\mathbf{v}), \quad J_{\epsilon}(\mathbf{v}) = J(\mathbf{v}) + j_{\epsilon}(\mathbf{v})$$

As $\epsilon \to 0$, $j_{\epsilon}$ tends to the indicator function of set K and $\mathbf{u}_{\epsilon} \to \mathbf{u}$ (see e.g. [16]). Assumptions (A1)-(A2) are verified since $J_{\epsilon}$ is Fréchet differentiable with Fréchet differential $J'_{\epsilon}: V_{\parallel \cdot \parallel} \to V_{\parallel \cdot \parallel}^*$ defined by

$$\langle J'_{\epsilon}(\mathbf{v}), \mathbf{z} \rangle = a(\mathbf{v}, \mathbf{z}) - \ell(\mathbf{z}) + \langle j'_{\epsilon}(\mathbf{v}), \mathbf{z} \rangle,$$

and $J_{\epsilon}$ is elliptic since

$$\langle J'_{\epsilon}(\mathbf{v}) - J'_{\epsilon}(\mathbf{w}), \mathbf{v} - \mathbf{w} \rangle = a(\mathbf{v} - \mathbf{w}, \mathbf{v} - \mathbf{w}) + \langle j'_{\epsilon}(\mathbf{v}) - j'_{\epsilon}(\mathbf{w}), \mathbf{v} - \mathbf{w} \rangle \geq \alpha \|\mathbf{v} - \mathbf{w}\|^2$$

Assumption (A3) comes from the continuity of $a$ and $\ell$ and from the properties of $j'$.

Case where $K = V_{\parallel \cdot \parallel}$. If $K = V_{\parallel \cdot \parallel}$, we recover the classical case of linear elliptic variational problems on Hilbert spaces analyzed in [15]. In this case, the bilinear form $a$ defines a norm $\|\mathbf{v}\|_a = \sqrt{a(\mathbf{v}, \mathbf{v})}$ on $V_{\parallel \cdot \parallel}$, equivalent to the norm $\|\cdot\|$. The functional $J$ is here equal to

$$J(\mathbf{v}) = \frac{1}{2} \|\mathbf{u} - \mathbf{v}\|_a^2 - \frac{1}{2} \|\mathbf{u}\|_a^2$$

The progressive PGD can be interpreted as a generalized Eckart-Young decomposition (generalized singular value decomposition) with respect to this non usual metric, and defined progressively by

$$\|\mathbf{u} - \mathbf{u}_m\|_a^2 = \min_{\mathbf{z} \in S_1} \|\mathbf{u} - \mathbf{u}_{m-1} - \mathbf{z}\|_a^2$$

We have

$$J(\mathbf{u}_{m-1}) - J(\mathbf{u}_m) = \frac{1}{2} \|\mathbf{z}_m\|_a^2 := \frac{1}{2} \sigma_m^2$$

and

$$\|\mathbf{u} - \mathbf{u}_m\|_a^2 = \|\mathbf{u}\|_a^2 - \sum_{k=1}^{m} \sigma_k^2 \underset{m \to \infty}{\longrightarrow} 0$$

where $\sigma_m$ can be interpreted as the dominant singular value of $(\mathbf{u} - \mathbf{u}_{m-1}) \in V_{\parallel \cdot \parallel}$. The PGD method has been successfully applied to this class of problems in different contexts: separation of spatial coordinates for the solution of Poisson equation in high dimension [2, 6], separation of physical variables and random parameters for the solution of parameterized stochastic partial differential equations [28].
# 6 Conclusion

In this paper, we have considered the solution of a class of convex optimization problems in tensor Banach spaces with a family of methods called progressive Proper Generalized Decomposition (PGD) that consist in constructing a sequence of approximations by successively correcting approximations with optimal elements in a given subset of tensors. We have proved the convergence of a large class of PGD algorithms (including update strategies) under quite general assumptions on the convex functional and on the subset of tensors considered in the successive approximations. The resulting succession of approximations has been interpreted as a generalization of a multidimensional singular value decomposition (SVD). Some possible applications have been considered.

Further theoretical investigations are still necessary for a better understanding of the different variants of PGD methods and the introduction of more efficient algorithms for their construction (e.g. alternated direction algorithms, ...). The analysis of algorithms for the solution of successive approximation problems on tensor subsets is still an open problem. In the case of dimension $d = 2$, further analyses would be required in order to better characterize the PGD as a direct extension of SVD when considering more general norms.

# References

1. A. Ammar, F. Chinesta, and A. Falcó. On the convergence of a greedy rank-one update algorithm for a class of linear systems. *Archives of Computational Methods in Engineering*, 17(4):473-486, 2010.

2. A. Ammar, B. Mokdad, F. Chinesta, and R. Keunings. A new family of solvers for some classes of multidimensional partial differential equations encountered in kinetic theory modelling of complex fluids. *Journal of Non-Newtonian Fluid Mechanics*, 139(3):153-176, 2006.

3. J. Ballani and L. Grasedyck. A projection method to solve linear systems in tensor format. Technical Report 309, RWTH Aachen, 2010.

4. P. Blanchard and E. Brüning. Mathematical Methods in Physics: Distributions, Hilbert Space Operators, and Variational Methods. Birkhäuser, 2003.

5. R. Bonic and J. Frampton. Differentiable functions on certain Banach spaces, *Bull. Am. Math. Soc.*, 71(2):393-395, 1965.

6. C. Le Bris, T. Lelievre, and Y. Maday. Results and questions on a nonlinear approximation approach for solving high-dimensional partial differential equations. *Constructive Approximation*, 30(3):621-651, 2009.

7. E. Cances, V. Ehrlacher, and T. Lelievre. Convergence of a greedy algorithm for high-dimensional convex nonlinear problems. *arXiv*, arXiv(1004.0095v1 [math.FA]), April 2010.

8. C. Canuto and K. Urban. Adaptive optimization of convex functionals in banach spaces. *SIAM J. Numer. Anal.*, 42(5):2043-2075, 2005.

9. F. Chinesta, A. Ammar, and E. Cueto. Recent advances in the use of the Proper Generalized Decomposition for solving multidimensional models. *Archives of Computational Methods in Engineering*, 17(4):373-391, 2010.

10. A. Defant, and K. Floret. Tensor Norms and Operator ideals. North-Holland, 1993.

11. L. De Lathauwer, B. De Moor, and J. Vandewalle. A multilinear singular value decomposition. *SIAM J. Matrix Anal. Appl.*, 21(4):1253-1278, 2000.

12. V. de Silva and L.-H. Lim. Tensor rank and ill-posedness of the best low-rank approximation problem. *SIAM Journal of Matrix Analysis & Appl.*, 30(3):1084-1127, 2008.

13. I. Ekeland and R. Teman. *Convex Analysis and Variational Problems*. Classics in Applied Mathematics. SIAM, 1999.

14. A. Falcó and W. Hackbusch. On minimal subspaces in tensor representations *Preprint 70/2010* Max Planck Institute for Mathematics in the Sciences, 2010.

15. A. Falcó and A. Nouy. A Proper Generalized Decomposition for the solution of elliptic problems in abstract form by using a functional Eckart-Young approach. *J. Math. Anal. Appl.*, 376: 469-480, 2011.
# 6 Conclusion

In this paper, we have considered the solution of a class of convex optimization problems in tensor Banach spaces with a family of methods called progressive Proper Generalized Decomposition (PGD) that consist in constructing a sequence of approximations by successively correcting approximations with optimal elements in a given subset of tensors. We have proved the convergence of a large class of PGD algorithms (including update strategies) under quite general assumptions on the convex functional and on the subset of tensors considered in the successive approximations. The resulting succession of approximations has been interpreted as a generalization of a multidimensional singular value decomposition (SVD). Some possible applications have been considered.

Further theoretical investigations are still necessary for a better understanding of the different variants of PGD methods and the introduction of more efficient algorithms for their construction (e.g. alternated direction algorithms, ...). The analysis of algorithms for the solution of successive approximation problems on tensor subsets is still an open problem. In the case of dimension $d = 2$, further analyses would be required in order to better characterize the PGD as a direct extension of SVD when considering more general norms.

# References

1. A. Ammar, F. Chinesta, and A. Falcó. On the convergence of a greedy rank-one update algorithm for a class of linear systems. *Archives of Computational Methods in Engineering*, 17(4):473-486, 2010.

2. A. Ammar, B. Mokdad, F. Chinesta, and R. Keunings. A new family of solvers for some classes of multidimensional partial differential equations encountered in kinetic theory modelling of complex fluids. *Journal of Non-Newtonian Fluid Mechanics*, 139(3):153-176, 2006.

3. J. Ballani and L. Grasedyck. A projection method to solve linear systems in tensor format. Technical Report 309, RWTH Aachen, 2010.

4. P. Blanchard and E. Brüning. Mathematical Methods in Physics: Distributions, Hilbert Space Operators, and Variational Methods. Birkhäuser, 2003.

5. R. Bonic and J. Frampton. Differentiable functions on certain Banach spaces, *Bull. Am. Math. Soc.*, 71(2):393-395, 1965.

6. C. Le Bris, T. Lelievre, and Y. Maday. Results and questions on a nonlinear approximation approach for solving high-dimensional partial differential equations. *Constructive Approximation*, 30(3):621-651, 2009.

7. E. Cances, V. Ehrlacher, and T. Lelievre. Convergence of a greedy algorithm for high-dimensional convex nonlinear problems. *arXiv*, arXiv(1004.0095v1 [math.FA]), April 2010.

8. C. Canuto and K. Urban. Adaptive optimization of convex functionals in banach spaces. *SIAM J. Numer. Anal.*, 42(5):2043-2075, 2005.

9. F. Chinesta, A. Ammar, and E. Cueto. Recent advances in the use of the Proper Generalized Decomposition for solving multidimensional models. *Archives of Computational Methods in Engineering*, 17(4):373-391, 2010.

10. A. Defant, and K. Floret. Tensor Norms and Operator ideals. North-Holland, 1993.

11. L. De Lathauwer, B. De Moor, and J. Vandewalle. A multilinear singular value decomposition. *SIAM J. Matrix Anal. Appl.*, 21(4):1253-1278, 2000.

12. V. de Silva and L.-H. Lim. Tensor rank and ill-posedness of the best low-rank approximation problem. *SIAM Journal of Matrix Analysis & Appl.*, 30(3):1084-1127, 2008.

13. I. Ekeland and R. Teman. *Convex Analysis and Variational Problems*. Classics in Applied Mathematics. SIAM, 1999.

14. A. Falcó and W. Hackbusch. On minimal subspaces in tensor representations *Preprint 70/2010* Max Planck Institute for Mathematics in the Sciences, 2010.

15. A. Falcó and A. Nouy. A Proper Generalized Decomposition for the solution of elliptic problems in abstract form by using a functional Eckart-Young approach. *J. Math. Anal. Appl.*, 376: 469-480, 2011.
16. R. Glowinski. *Numerical Methods for Nonlinear Variational Problems*. Springer-Verlag, New York, 1984.

17. L. Grasedyck. *Hierarchical Singular Value Decomposition of Tensors*. *SIAM J. Matrix Anal. Appl.*, 31:2029-2054, 2010.

18. W. Hackbusch, N. Khoromskij, S. A. Sauter, and E. E. Tyrtyshnikov. Use of tensor formats in elliptic eigenvalue problems. Technical Report Research Report No.2010-78, Max Planck Institute for Mathematics in the Sciences, 2008.

19. R. Holmes. *Geometric Functional Analysis and its Applications*. Springer-Verlag, New York, 1975.

20. B.N. Khoromskij and C. Schwab. Tensor-structured galerkin approximation of parametric and stochastic elliptic pdes. Technical Report Research Report No. 2010-04, ETH, 2010.

21. T. G. Kolda and B. W. Bader. Tensor decompositions and applications. *SIAM Review*, 51(3):455-500, 2009.

22. T.G. Kolda. Orthogonal tensor decompositions. *SIAM J. Matrix Analysis & Applications*, 23(1):243-255, 2001.

23. A. Kriegl and P.W. Michor. *The Convenient Setting of Global Analysis* Mathematical Surveys and Monographs Volume 53, American Mathematical Society, 1997

24. P. Ladevèze. *Nonlinear Computational Structural Mechanics - New Approaches and Non-Incremental Methods of Calculation*. Springer Verlag, 1999.

25. P. Ladevèze, J.-C. Passieux and D. Nîlsson. The LATIN multiscale computational method and the Proper Generalized Decomposition *Computer Methods in Applied Mechanics and Engineering*, 199:1287-1296, 2010.

26. H. G. Matthies and E. Zander. Sparse representations in stochastic mechanics. In *Manolis Papadrakakis, George Stefanou, and Vissarion Papadopoulos, editors, Computational Methods in Stochastic Dynamics*, volume 22 of *Computational Methods in Applied Sciences*, pages 247-265. Springer Netherlands, 2011.

27. T. Morrison. *Functional Analysis. An Introduction to Banach Space Theory*. John Wiley and Sons, 2001

28. A. Nouy. A generalized spectral decomposition technique to solve a class of linear stochastic partial differential equations. *Computer Methods in Applied Mechanics and Engineering*, 196(45-48):4521-4537, 2007.

29. A. Nouy. Generalized spectral decomposition method for solving stochastic finite element equations: invariant subspace problem and dedicated algorithms. *Computer Methods in Applied Mechanics and Engineering*, 197:4718-4736, 2008.

30. A. Nouy. Recent developments in spectral stochastic methods for the numerical solution of stochastic partial differential equations *Archives of Computational Methods in Engineering*, 16:251-285, 2009.

31. A. Nouy. Proper Generalized Decompositions and separated representations for the numerical solution of high dimensional stochastic problems. *Archives of Computational Methods in Engineering*, 17:403-434, 2010.

32. A. Nouy. A priori model reduction through Proper Generalized Decomposition for solving time-dependent partial differential equations. *Computer Methods in Applied Mechanics and Engineering*, 199:1603-1626, 2010.

33. I. Oseledets and E. Tyrtyshnikov. TT-cross approximation for multidimensional arrays *Linear Algebra And Its Applications*, 2010, 432, 70-88

34. R. A. Ryan. *Introduction to tensor products of Banach spaces*. Springer, 2002.

35. V. Temlyakov. Greedy Approximation *Acta Numerica*, 17:235-409, 2008.

36. Andre Uschmajew. Well-posedness of convex maximization problems on stiefel manifolds and orthogonal tensor product approximations. *Numerische Mathematik*, 115:309-331 2010.

37. Eberhard Zeidler. *Nonlinear Functional Analysis and its Applications III. Variational Methods and Optimization*. Springer-Verlag, 1985.
16. R. Glowinski. *Numerical Methods for Nonlinear Variational Problems*. Springer-Verlag, New York, 1984.

17. L. Grasedyck. *Hierarchical Singular Value Decomposition of Tensors*. *SIAM J. Matrix Anal. Appl.*, 31:2029-2054, 2010.

18. W. Hackbusch, N. Khoromskij, S. A. Sauter, and E. E. Tyrtyshnikov. Use of tensor formats in elliptic eigenvalue problems. Technical Report Research Report No.2010-78, Max Planck Institute for Mathematics in the Sciences, 2008.

19. R. Holmes. *Geometric Functional Analysis and its Applications*. Springer-Verlag, New York, 1975.

20. B.N. Khoromskij and C. Schwab. Tensor-structured galerkin approximation of parametric and stochastic elliptic pdes. Technical Report Research Report No. 2010-04, ETH, 2010.

21. T. G. Kolda and B. W. Bader. Tensor decompositions and applications. *SIAM Review*, 51(3):455-500, 2009.

22. T.G. Kolda. Orthogonal tensor decompositions. *SIAM J. Matrix Analysis & Applications*, 23(1):243-255, 2001.

23. A. Kriegl and P.W. Michor. *The Convenient Setting of Global Analysis* Mathematical Surveys and Monographs Volume 53, American Mathematical Society, 1997.

24. P. Ladevèze. *Nonlinear Computational Structural Mechanics - New Approaches and Non-Incremental Methods of Calculation*. Springer Verlag, 1999.

25. P. Ladevèze, J.-C. Passieux and D. Nîlsson. The LATIN multiscale computational method and the Proper Generalized Decomposition *Computer Methods in Applied Mechanics and Engineering*, 199:1287-1296, 2010.

26. H. G. Matthies and E. Zander. Sparse representations in stochastic mechanics. In *Manolis Papadrakakis, George Stefanou, and Vissarion Papadopoulos, editors, Computational Methods in Stochastic Dynamics*, volume 22 of *Computational Methods in Applied Sciences*, pages 247-265. Springer Netherlands, 2011.

27. T. Morrison. *Functional Analysis. An Introduction to Banach Space Theory*. John Wiley and Sons, 2001

28. A. Nouy. A generalized spectral decomposition technique to solve a class of linear stochastic partial differential equations. *Computer Methods in Applied Mechanics and Engineering*, 196(45-48):4521-4537, 2007.

29. A. Nouy. Generalized spectral decomposition method for solving stochastic finite element equations: invariant subspace problem and dedicated algorithms. *Computer Methods in Applied Mechanics and Engineering*, 197:4718-4736, 2008.

30. A. Nouy. Recent developments in spectral stochastic methods for the numerical solution of stochastic partial differential equations *Archives of Computational Methods in Engineering*, 16:251-285, 2009.

31. A. Nouy. Proper Generalized Decompositions and separated representations for the numerical solution of high dimensional stochastic problems. *Archives of Computational Methods in Engineering*, 17:403-434, 2010.

32. A. Nouy. A priori model reduction through Proper Generalized Decomposition for solving time-dependent partial differential equations. *Computer Methods in Applied Mechanics and Engineering*, 199:1603-1626, 2010.

33. I. Oseledets and E. Tyrtyshnikov. TT-cross approximation for multidimensional arrays *Linear Algebra And Its Applications*, 2010, 432, 70-88

34. R. A. Ryan. *Introduction to tensor products of Banach spaces*. Springer, 2002.

35. V. Temlyakov. Greedy Approximation *Acta Numerica*, 17:235-409, 2008.

36. Andre Uschmajew. Well-posedness of convex maximization problems on stiefel manifolds and orthogonal tensor product approximations. *Numerische Mathematik*, 115:309-331 2010.

37. Eberhard Zeidler. *Nonlinear Functional Analysis and its Applications III. Variational Methods and Optimization*. Springer-Verlag, 1985.