## 2 Background
### Optimization of the Proposed Objective Function  
The proposed objective function is solved through an alternating minimization scheme. More precisely, we solve for \(\mathcal{Z}\) and \(\mathbf{E}\) while fixing \(\{\mathbf{U}^{(m)}\}_{m=1}^{M}\), \(\mathbf{U}^{*}\) and \(\mathbf{w}\) through the alternating direction method of multipliers (ADMM). Then, the individual subspaces, \(\{\mathbf{U}^{(m)}\}_{m=1}^{M}\), and the common subspace, \(\mathbf{U}^{*}\), across all layers, along with \(\mathbf{w}\), are estimated efficiently using eigendecomposition and HOOI, respectively. To solve the proposed objective function described in Equation (12), an auxiliary variable, \(\mathcal{H}\), is introduced to allow variables separability. By introducing \(\mathcal{H}\), the objective function is reformulated as follows:  
\[ \begin{split}&\min_{\begin{subarray}{c}\mathbf{U}^{*}\mathbf{U}^{(m)},\mathcal{Z},\\ \mathbf{E},\mathbf{w},\mathcal{H}\end{subarray}}\|\mathcal{H}\|_{\oplus}+ \gamma\|\mathbf{E}\|_{2,1}-\lambda\sum_{m=1}^{M}\left[tr\bigg{(}\mathbf{U}^{( m)}{}^{\top}\mathbf{Z}_{N}^{(m)}\mathbf{U}^{(m)}\bigg{)}+w_{m}tr\bigg{(} \mathbf{U}^{*\top}\mathbf{U}^{(m)}{}\mathbf{U}^{(m)}{}^{\top}\mathbf{U}^{*} \bigg{)}\right]\\ &\text{s.t.}\quad\mathbf{A}^{(m)}=\mathbf{A}^{(m)}\mathbf{Z}^{(m )}+\mathbf{E}^{(m)},\mathbf{E}=[\mathbf{E}^{(1)};\mathbf{E}^{(2)};\ldots; \mathbf{E}^{(M)}],\mathbf{Z}^{(m)}=\mathbf{Z}^{(m)}{}^{\top}\\ &\mathbf{U}^{(m)}{}^{\top}\mathbf{U}^{(m)}=\mathbf{I},m=\{1, \ldots,M\},\mathcal{Z}=\mathcal{H},\mathcal{H}=\Phi(\mathcal{H}^{(1)}, \mathcal{H}^{(2)},\ldots,\mathcal{H}^{(M)}),\\ &\|\mathbf{w}\|_{2}=1,\mathbf{w}\geq 0,\mathbf{U}^{*\top} \mathbf{U}^{*}=\mathbf{I}.\end{split}  \tag{13}\]  
The augmented Lagrangian function can be then written as follows:  
\[ \begin{split}&\min_{\begin{subarray}{c}\mathbf{U}^{*}\mathbf{U}^{(m)},\mathcal{Z},\\ \mathbf{E},\mathbf{w},\mathcal{H}\end{subarray}}\|\mathcal{H}\|_{\oplus}+ \gamma\|\mathbf{E}\|_{2,1}-\lambda\sum_{m=1}^{M}\left[tr\bigg{(}\mathbf{U}^{( m)}{}^{\top}\mathbf{Z}_{N}^{(m)}\mathbf{U}^{(m)}\bigg{)}+w_{m}tr\bigg{(} \mathbf{U}^{*\top}\mathbf{U}^{(m)}{}^{\top}\mathbf{U}^{*}\bigg{)}\right]\\ &+\sum_{m=1}^{M}\bigg{(}\langle\mathbf{X}_{1}^{(m)},\mathbf{A}^{( m)}-\mathbf{A}^{(m)}\mathbf{Z}^{(m)}-\mathbf{E}^{(m)}\rangle+\frac{\beta_{1}}{2}\| \mathbf{A}^{(m)}-\mathbf{A}^{(m)}\mathbf{Z}^{(m)}-\mathbf{E}^{(m)}\|_{F}^{2} \bigg{)}\\ &+\langle\lambda_{2},\mathcal{Z}-\mathcal{H}\rangle+\frac{\beta_{2 }}{2}\|\mathcal{Z}-\mathcal{H}\|_{F}^{2}\\ &\text{s.t.}\quad\mathbf{E}=[\mathbf{E}^{(1)};\mathbf{E}^{(2)}; \ldots;\mathbf{E}^{(M)}],\mathbf{Z}^{(m)}=\mathbf{Z}^{(m)}{}^{\top},\mathbf{U}^ {(m)}{}^{\top}\mathbf{U}^{(m)}=\mathbf{I},m=\{1,\ldots,M\},\\ &\mathcal{H}=\Phi(\mathcal{H}^{(1)},\mathcal{H}^{(2)},\ldots, \mathcal{H}^{(M)}),\|\mathbf{w}\|_{2}=1,\mathbf{w}\geq 0,\mathbf{U}^{* \top}\mathbf{U}^{*}=\mathbf{I},\end{split}  \tag{14}\]where the matrix \(\mathbf{X}_{1}^{(m)}\) and tensor \(\mathcal{X}_{2}\) are the Lagrangian multipliers. \(\beta_{1}\) and \(\beta_{2}\) are penalty parameters. \(\langle\cdot\rangle\) is the trace inner product. To this end, Equation (14) can be solved by an alternating minimization scheme as follows:  
* **Updating**\(\mathcal{H}\): Fixing all the variables except \(\mathcal{H}\) and considering all the terms that includes the variable \(\mathcal{H}\) from Equation (14) gives: \[ \begin{split}\mathcal{H}_{k+1}&=\min_{\mathcal{H}} \|\mathcal{H}\|_{\otimes}+\langle\mathcal{X}_{2_{k}},\mathcal{H}_{k}- \mathcal{Z}_{k}\rangle+\frac{\beta_{2}}{2}\|\mathcal{H}_{k}-\mathcal{Z}_{k}\| _{F}^{2}\\ &=\min_{\mathcal{H}}\|\mathcal{H}\|_{\otimes}+\frac{\beta_{2}}{2} \|\mathcal{H}_{k}-\mathcal{P}_{k}\|_{F}^{2},\end{split} \] (15) where \(\mathcal{P}_{k}=\mathcal{Z}_{k}+\frac{\mathcal{X}_{2_{k}}}{\beta_{2}}\).