## 3 MTR Algorithm Based on Multi-Layer Sparse Structure
### Non-Linear Extensions Based on Kernel Tricks  
In order to improve the fitting ability of the multi-layer MTR model in the nonlinear case, the kernel technique is introduced to extend it. Based on the Representer Theorem [36], the linear learner can be extended to a nonlinear learner by kernelization in the Reproducing Kernel Hilbert Space (RKHS). Denoting the regression coefficient matrix \(\mathbf{W}\) as the combination of mappings of input samples in \(\mathcal{H}\), we have \(\mathbf{W}=\mathbf{A}\mathbf{X}^{\top}\), where \(\mathbf{A}\in\mathbb{R}^{Q\times N}\) is the coefficient matrix. Rewrite Equation (2) in trace form:  
\[\min_{\mathbf{W}\mathbf{S}}\frac{1}{N}tr\Big{(}(\mathbf{Y}-\mathbf{S}( \mathbf{W}\mathbf{X}+\mathbf{B}))^{\top}(\mathbf{Y}-\mathbf{S}(\mathbf{W} \mathbf{X}+\mathbf{B}))\Big{)}+\lambda tr\Big{(}\mathbf{W}^{\top}\mathbf{W} \Big{)}+\alpha tr\Big{(}\mathbf{S}^{\top}\mathbf{D}\mathbf{S}\Big{)}. \tag{3}\]  
where \(\alpha\) is the penalty parameter and \(\mathbf{D}\in\mathbb{R}^{Q\times Q}\) is the diagonal matrix, and  
\[(\mathbf{D})_{i,j}=\frac{1}{2\|\mathbf{s}_{i}\|_{2}}. \tag{4}\]  
in which \(\mathbf{s}_{i}\) is the \(i\)-th column of \(\mathbf{S}\). In order to avoid division by zero, we further regularize \((\mathbf{D})_{i,i}\) according to the following equation,  
\[(\mathbf{D})_{i,j}=\frac{1}{2\sqrt{\mathbf{s}_{i}^{\top}\mathbf{s}_{i}+\xi}}. \tag{5}\]  
where \(\xi>0\) is a small constant. Equation (6) is approximately equivalent to Equation (5) when \(\xi\to 0\). We map the input data into the high-dimensional Hilbert space \(\mathcal{H}\) by the nonlinear mapping function \(\phi(\cdot)\), and the associated kernel function is \(\kappa\big{(}\mathbf{x}_{i},\mathbf{x}_{j}\big{)}=\phi(\mathbf{x}_{i})^{\top} \phi(\mathbf{x}_{j})\). According to the Representer Theorem, the regression matrix \(\mathbf{W}\) can be represented by \(\mathbf{W}=\mathbf{A}\Phi(\mathbf{X})^{\top}\), where \(\Phi(\mathbf{X})=[\phi(\mathbf{x}_{1}),\phi(\mathbf{x}_{2}),\cdots,\phi( \mathbf{x}_{N})]\). Substituting it into Equation (4), we can obtain the following objective function:  
\[ \begin{split}\min_{\mathbf{A},\mathbf{S}}\frac{1}{N}tr\Big{(} \Big{(}\mathbf{Y}-\mathbf{S}\mathbf{A}\[PERSON](\mathbf{X})^{\top}\Phi(\mathbf{X} )\Big{)}^{\top}\Big{(}\mathbf{Y}-\mathbf{S}\mathbf{A}\[PERSON](\mathbf{X})^{\top }\[PERSON](\mathbf{X})\Big{)}\Big{)}\\ +\lambda tr\Big{(}\Big{(}\mathbf{A}\[PERSON](\mathbf{X})^{\top}\Big{)} ^{\top}\Big{(}\mathbf{A}\[PERSON](\mathbf{X})^{\top}\Big{)}\Big{)}+\alpha tr\Big{(} \mathbf{S}^{\top}\mathbf{D}\mathbf{S}\Big{)}\end{split}  \tag{6}\]  
Note that the bias \(\mathbf{b}\) is omitted, since it can be absorbed into \(\mathbf{W}\) by adding additional dimension into the input \(\mathbf{X}\). Define \(\mathbf{K}=\Phi(\mathbf{X})^{\top}\Phi(\mathbf{X})\) to be the kernel matrix in \(\mathcal{H}\). Then, Equation (6) can be simplified:  
\[\min_{\mathbf{A},\mathbf{S}}\frac{1}{N}tr\Big{(}(\mathbf{Y}-\mathbf{S}\mathbf{ A}\mathbf{K})^{\top}(\mathbf{Y}-\mathbf{S}\mathbf{A}\mathbf{K})\Big{)}+\lambda tr \Big{(}\mathbf{A}\mathbf{K}\mathbf{A}^{\top}\Big{)}+\alpha tr\Big{(}\mathbf{S} ^{\top}\mathbf{D}\mathbf{S}\Big{)} \tag{7}\]  
The above equation provides a learning framework for multi-target sparse kernel regression based on latent variables. The multi-layer multi-output structure formed by introducing the space of latent variables can decompose the original regression coefficient matrix into a new coefficient matrix \(\mathbf{A}\) and a structure matrix \(\mathbf{S}\), and the coefficient matrix \(\mathbf{A}\) is utilized to explore the correlation information between inputs and outputs, and the structure matrix \(\mathbf{S}\) is employed to represent the correlation among targets sparsely. Based on the framework of multi-target sparse kernel regression, the regularization constraints on the coefficient matrix can decouple the redundant correlations between different input features; and the regularization constraints on the structure matrix can reveal the correlation structure between targets.