# Tensor Decomposition Through Neural Architectures
## 1 Introduction
Such a separated representation constructor was called Proper Generalized Decomposition (PGD) and was used for separating (i) the space coordinates; (ii) the space and time; and (iii) the space, time, and problem parameters, as reported in [26; 27; 28; 29] and the numerous references therein. The separated representations considered in the above-mentioned PGD is a particular case of the tensor formats widely employed in the domain of multidimensional approximation and solution of pPDEs, with a variety of techniques, procedures, and applications [30; 31; 32; 33; 34]. Tensor formats are, above all, an efficient procedure of data compression, applicable as soon as data are expressed in a tensor form (matrix or hyper-matrix). Thus, the machine learning community has considered the use of tensor formats for alleviating the computational cost and the amount of data needed for training large neural networks, such as those involved in spatio-temporal learning (e.g., video sequences), large recurrent long-short-term-memory neural networks [35; 36; 37; 38], or, more generally, large neural networks assimilating big-size data [39; 40; 41; 42; 43]. This work does not aim at using tensor decomposition for alleviating the neural networks' training cost. It aims at using more or less conventional neural architectures for providing the tensor decomposition of the output data, when considering as input data the model features expressed by scalars, or also by a tensor decomposition. Thus, the present work addresses different scenarios widely encountered in science and engineering applications, such as the ones described below:  
1. There are cases in which problem solutions (or images) were compressed for easy transfer, manipulation, and storage, for example by performing an SVD. Of course, data could be decompressed in order to be assimilated by the usual available learning technologies. However, such a decompression should be, if possible, avoided, knowing that most of the time, data are first reduced (by using an auto-encoder, for instance) prior to constructing the regression at the latent space level. 2. Problems defined in high-dimensional spaces, for instance, those involving several parameters assumed as model extra-coordinates, and solved by using separated representations (PGD-like) or tensor formats, result in a finite sum decomposition, quite similar to an SVD or HOSVD (high-order singular value decomposition) of the resulting multi-parametric solution [44; 45]. However, in the present situation, reconstructing the discrete form of the multi-parametric solution cannot be envisaged without incurring in the so-called curse of dimensionality. If the problem involves \(D\) dimensions and the approximation in each of these dimensions makes use of \(N\) degrees of freedom, the number of the resulting degrees of freedom becomes \(N^{D}\), quickly requiring unattainable memory and computation cost. 3. When problems involve a number of conformation coordinates (e.g., Fokker-Planck, Schrodinger, or Boltzmann equations), the solution is intrinsically defined in high-dimensional spaces, where separated representations are the simplest way of addressing them. In all the aforementioned cases, the data have a tensor format because of the reasons stated, and one would like to manipulate that tensor format directly, when considering them in learning approaches. In what follows, we address four different problem typologies:  
1. The first type of problem concerns a time-dependent parametrized PDE, whose transient solution is expressed by its SVD, composed of space and time modes, scaled by the corresponding singular values. From different problem solutions related to different choices of the problem parameter, one would like to infer, for a given new parameter choice, the reduced basis (SVD modes) associated to that parameter choice, in order to proceed with a Garlekin projection into the reduced basis, that is, elaborating a learning procedure able to infer the parametric principal modes. 2.