# Consensus Guided Multi-View Unsupervised Feature Selection with Hybrid Regularization
## 3 Proposed Method
### Optimization  
The non-convex optimization formulation in (5) presents computational intractability when approached through conventional analytical methods. To transform it into a convex problem for further optimization, we introduce auxiliary variables to simplify the subsequent derivation process, making the objective function more intuitive and eliminating the need for complex matrix multiplication. Specifically, the positive semi-definite matrix \(M\) inherently enables spectral decomposition through its characteristic eigenvalue-eigenvector configuration:  
\[\mathbf{M}=\mathbf{E}\mathbf{\Lambda}\mathbf{E}^{\top}=(\mathbf{E}\mathbf{\Lambda}^{\frac{1}{2}})(\mathbf{ E}\mathbf{\Lambda}^{\frac{1}{2}})^{\top} \tag{6}\]where \(\mathbf{\Lambda}\in\mathbb{R}^{n\times n}\) is a diagonal matrix with \(n\) eigenvalues of \(\mathbf{M}\), \(\mathbf{E}\) is the corresponding eigenvector matrix. Comparing Equations (4) and (6), it can be seen that \(\mathbf{E}\mathbf{\Lambda}^{\frac{1}{2}}\) reflects the embedding of the samples in the feature subspace. By minimizing \(\left\|(\mathbf{X}^{v}\mathbf{W}^{v})(\mathbf{X}^{v}\mathbf{W}^{v})^{\top}-\mathbf{M}\right\|_{F}^{2}\) for the \(v\)-th view, the optimal solution can be obtained based on the Eckart-Young-Mirsky Theorem [30; 31] as follows:  
\[\mathbf{Y}^{*}=\mathbf{X}^{v}\mathbf{W}^{v*}=\mathbf{E}^{*}(\mathbf{\Lambda}^{*})^{\frac{1}{2}} \tag{7}\]  
Here, \(\mathbf{\Lambda}^{*}\in\mathbb{R}^{k\times k}\) is a diagonal matrix corresponding to the largest \(k\) eigenvalues of \(\mathbf{M}\), and \(\mathbf{E}^{*}\in\mathbb{R}^{n\times k}\) is a matrix composed of the corresponding \(k\) eigenvectors. According to [13; 32; 33], the upper bound of \(\left\|(\mathbf{X}^{v}\mathbf{W}^{v})(\mathbf{X}^{v}\mathbf{W}^{v})^{\top}-\mathbf{M}\right\|_{F}\) is determined by \(\mathbf{X}^{v}\mathbf{W}^{v}-\mathbf{Y}^{*}\). Thus, we can further derive the upper bound of \(\sum_{v=1}^{V}\left\|(\mathbf{X}^{v}\mathbf{W}^{v})(\mathbf{X}^{v}\mathbf{W}^{v})^{\top}-\mathbf{M }\right\|_{F}\) as follows:  
\[ \begin{split}&\sum_{v=1}^{V}\|(\mathbf{Y}^{*})(\mathbf{Y}^{*})^{\top}- \mathbf{M}\|_{F}\leq\sum_{v=1}^{V}\|(\mathbf{Y}^{v})(\mathbf{Y}^{v})^{\top}-\mathbf{M}\|_{F}\\ &\leq\sum_{v=1}^{V}2(\|\mathbf{Y}^{*}\|_{F}+\|\mathbf{\Psi}\|_{F})\|\mathbf{ \Psi}\|_{F}+\sum_{v=1}^{V}\|(\mathbf{Y}^{*})(\mathbf{Y}^{*})^{\top}-\mathbf{M}\|_{F}\\ & s.t.\ \mathbf{\Psi}=\mathbf{Y}^{v}-\mathbf{Y}^{*},\ \mathbf{Y}^{v}=\mathbf{X}^{v}\mathbf{W}^{v} \end{split}  \tag{8}\]  
Since \(\sum_{v=1}^{V}\|(\mathbf{Y}^{*})(\mathbf{Y}^{*})^{\top}-\mathbf{M}\|_{F}\) and \(\|\mathbf{Y}^{*}\|_{F}\) are constants, the upper bound of \(\sum_{v=1}^{V}\|(\mathbf{Y}^{v})(\mathbf{Y}^{v})^{\top}-\mathbf{M}\|_{F}\) is related to \(\mathbf{\Psi}\). As a result, problem (5) can be rewritten to minimize its upper bound as follows:  
\[ \begin{split}&\min_{\mathbf{W}^{v}}J=\sum_{v=1}^{V}\|\mathbf{X}^{v}\mathbf{W}^{ v}-\mathbf{Y}^{*}\|_{F}^{2}+\alpha\|\mathbf{W}^{v}\|_{2,1}+\sigma\|\mathbf{W}^{v}\|_{F}^{2} \\ &\text{s.t. }\mathbf{Y}^{*}=\mathbf{E}^{*}(\mathbf{\Lambda}^{*})^{\frac{1}{2}} \end{split}  \tag{9}\]  
The globally optimal solution can be derived by solving \(\frac{\partial J}{\partial\mathbf{W}^{v}}=0\), yielding the following result:  
\[\frac{\partial J}{\partial\mathbf{W}}=2(\mathbf{X}^{v})^{\top}\mathbf{X}^{v}\mathbf{W}^{v}-2( \mathbf{X}^{v})^{\top}\mathbf{Y}^{*}+2\alpha\mathbf{F}^{v}\mathbf{W}^{v}+2\sigma\mathbf{I}^{v}\mathbf{ W}^{v}=0 \tag{10}\]  
\[\mathbf{W}^{v}=\Big{(}(\mathbf{X}^{v})^{\top}\mathbf{X}^{v}+\alpha\mathbf{F}^{v}+\sigma\mathbf{I}^ {v}\Big{)}^{-1}(\mathbf{X}^{v})^{\top}\mathbf{Y}^{*} \tag{11}\]  
\[\mathbf{F}^{v}=diag(\frac{1}{2||\mathbf{w}_{1}^{v}||_{2}+\mu},\frac{1}{2||\mathbf{w}_{2}^{ v}||_{2}+\mu},\cdots,\frac{1}{2||\mathbf{w}_{d}^{v}||_{2}+\mu}) \tag{12}\]  
where \(\mathbf{I}^{v}\in\mathbb{R}^{n\times n}\) is the identity matrix, and \(\mu\) is a small positive constant that prevents division by zero.