# Tensor Decomposition Through Neural Architectures
## 2 Methodology  
We consider the general parametrized PDE expressed by  
\[\mathcal{L}(u)(x,t;\mu)=f(x,t), \tag{1}\]  
where \(\mathcal{L}\) refers to a linear or nonlinear operator acting on the unknown and searched field \(u(x,t)\), that, in turn, depends parametrically on the model parameters \(\mu\), i.e., \(u(x,t;\mu)\). The function \(f(x,t)\) is the right hand-side of the equation.  
One possibility for computing such a parametric solution \(u(x,t;\mu)\) consists of sampling the parametric space \(\{\mu_{1},\mu_{2},\ldots\}\) and solving the associate problem for each parameter in the sampling set, \(\mu_{i},\ \forall i\), resulting in the set of solutions \(\{u^{1}(x,t),u^{2}(x,t),\ldots\}\), where \(u^{i}(x,t)\equiv u(x,t;\mu_{i})\).  
The discrete form of those solutions consists of vectors \(\mathbf{U}_{n}^{i}\), where index \(\bullet_{n}\) refers to the time step, that can be grouped in matrix \(\mathbf{Q}^{i}=\{\mathbf{U}_{1}^{i},\mathbf{U}_{2}^{i},\ldots\}\) that defines the so-called snapshot matrix \(\mathbf{Q}^{i}\) related to the \(i\)-parameter choice.  
Matrix \(\mathbf{Q}^{i}\) can be decomposed by applying the SVD as  
\[\mathbf{Q}^{i}=\hat{\mathbb{V}}^{i}\mathbb{D}^{i}\mathbb{W}^{i^{T}}, \tag{2}\]  
where the components of the diagonal matrix \(\mathbb{D}^{i}\) inform on the significance of the associated space and time modes. The matrices \(\hat{\mathbb{V}}^{i}\) and \(\mathbb{W}^{i^{T}}\) are, respectively, the left hand-side and the right hand-side singular vectors of \(\mathbf{Q}^{i}\). It is usual practice to only consider the vectors in matrix \(\hat{\mathbb{V}}^{i}\) associated with the highest components of \(\mathbb{D}^{i}\), that correspond to truncate \(\hat{\mathbb{V}}^{i}\), represented by \(\mathbb{V}^{i}=\{\mathbf{V}_{1}^{i},\ldots,\mathbf{V}_{r}^{i}\}\), where \(r\) refers to the number of retained modes, a number assumed to be the same for all the chosen parameter combinations.  
After establishing the reduced bases related to the different parameter choices:  
\[\left\{\begin{array}{l}\mu_{1}\rightarrow\mathbb{V}^{1}\\ \mu_{2}\rightarrow\mathbb{V}^{2}\\ \vdots\end{array}\right., \tag{3}\]  
one is tempted to look for the regression which is able to provide, from any parameter choice \(\mu\), the reduced basis \(\mathbb{V}(\mu)\):  
\[\mathbb{V}=\mathcal{H}(\mu). \tag{4}\]  
If the vectors composing \(\mathbb{V}\) are noted by \(\mathbf{V}_{1},\ldots,\mathbf{V}_{r}\), its orthonormality should be ensured; that is, \(\mathbf{V}_{i}\cdot\mathbf{V}_{j}=\delta_{ij}\) (with \(\delta\) here being the Kroenecker delta).  
Thus, the training of the neural network \(\mathcal{H}(\mu)\) could consider the loss function  
\[\texttt{Loss}=\sum_{i}\Bigl{(}\mathbb{V}^{i}-\mathcal{H}(\mu^{i})\Bigr{)}^{2} +\lambda_{1}\Biggl{(}\sum_{k,l\
eq k}\lvert\mathbf{V}_{k}\cdot\mathbf{V}_{l} \rvert\Biggr{)}+\lambda_{2}\Biggl{|}\Biggl{(}\sum_{k}\lvert\mathbf{V}_{k}\cdot \mathbf{V}_{k}\rvert\Biggr{)}-r\Biggr{|}, \tag{5}\]  
with \(\lambda_{1}\) and \(\lambda_{2}\) being two penalty coefficients.  
A simpler loss, that involves a single penalty, and that provides almost the same accuracy in the numerical experiments described later, consists of:  
\[\mathsf{Loss}=\sum_{i}\left(\mathbb{V}^{i}-\mathcal{H}(\mu^{i})\right)^{2}+ \lambda\left|\left(\sum_{kJ}\left|\mathbf{V}_{k}\cdot\mathbf{V}_{l}\right| \right)-r\right|. \tag{6}\]