# An Effective Multi-Label Feature Selection Model Towards Eliminating Noisy Features
## 4 Multi-Task Optimization for ELC  
Equation (1) describes an integer programming problem, which is NP-hard and complicated to solve. Moreover, the \(\ell_{2,0}\)-norm constraint in Equation (1) is non-smooth, which leads to a slow convergence rate. In this section, we devise an efficient framework to address this problem by using the sparse multi-task learning technology in the proximal alternating direction method (PADM) framework [40]. Suppose the spectral decomposition of the correlation matrix \(\mathbf{S}\) can be denoted as  
\[\mathbf{S}=\mathbf{\Phi}\mathbf{\Sigma}\mathbf{\Phi}^{T}=\mathbf{\Phi}diag( \sigma_{1},\ldots,\sigma_{l})\mathbf{\Phi}^{T},\sigma_{1}\geq\ldots\geq\sigma _{l},\]  
where \(\mathbf{\Phi}\) and \(\mathbf{\Sigma}\) are respectively the eigenvector and eigenvalue matrices of \(\mathbf{S}\). Then, Equation (1) can be reformulated as  
\[\min_{\mathbf{W},\mathbf{p}}\frac{1}{2}\left\|\mathbf{Y}^{T}\mathbf{X}diag( \mathbf{p})\mathbf{W}-\mathbf{\Gamma}^{*}\right\|_{F}^{2},\ s.t.\ \mathbf{W}\in\mathbb{R}^{d\times l},\left\|\mathbf{W}\right\|_{2,1}\leq t, \mathbf{p}\in\{0,1\}^{d},\mathbf{p}^{T}\mathbf{1}=k, \tag{2}\]  
where \(\mathbf{\Gamma}^{*}=n\mathbf{\Phi}\mathbf{\Sigma}^{1/2}\), \(t\) is a hyperparameter to constrain \(\left\|\mathbf{W}\right\|_{2,1}\) to a convex solution, \(\mathbf{p}\) is a feature indicator vector that reflects whether the corresponding features are selected or not (1 for selected and 0 for otherwise), and \(\mathbf{1}\) is the vector with all ones. On the basis of Equation (2), ELC is actually reformulated as a multivariate regression problem, which enables the multi-task learning technology [41]. This technology aims to learn a common set of features to tackle multiple relevant tasks and excels at various sparse learning formulations, including the optimization problem in Equation (1). Based on the multi-task learning technology, we then obtain the equivalent form of ELC as follows:  
\[\min_{\mathbf{W},\mathbf{p}}\frac{1}{2}\left\|\mathbf{\hat{A}}diag(\mathbf{p })\mathbf{W}-\mathbf{\Gamma}^{*}\right\|_{F}^{2}+\lambda\left\|\mathbf{W} \right\|_{2,1},\ s.t.\ \mathbf{p}\in\{0,1\}^{d},\mathbf{p}^{T}\mathbf{1}=k, \tag{3}\]  
where \(\mathbf{\hat{A}}=\mathbf{Y}^{T}\mathbf{X}\), and \(\lambda>0\) is the regularization parameter. Clearly, we can apply the augmented Lagrangian method to solve this problem. Then, Equation (3) is further reformulated as  
\[\min_{\mathbf{U},\mathbf{W},\mathbf{p}}\frac{1}{2}\left\|\mathbf{\hat{A}}diag (\mathbf{p})\mathbf{W}-\mathbf{\Gamma}^{*}\right\|_{F}^{2}+\lambda\left\| \mathbf{U}\right\|_{2,1},\ s.t.\ \mathbf{U}=\mathbf{W},\mathbf{p}\in\{0,1\}^{d}, \mathbf{p}^{T}\mathbf{1}=k. \tag{4}\]  
The Lagrangian function can be defined as  
\[\mathcal{L}(\mathbf{U},\mathbf{W},\mathbf{p},\mathbf{V})=\frac{1}{2}\left\| \mathbf{\hat{A}}diag(\mathbf{p})\mathbf{W}-\mathbf{\Gamma}^{*}\right\|_{F}^{ 2}+\frac{\beta}{2}\left\|\mathbf{W}-\mathbf{U}\right\|^{2}+\lambda\left\| \mathbf{U}\right\|_{2,1}-tr\left(\mathbf{V}^{T}(\mathbf{W}-\mathbf{U})\right), \tag{5}\]  
where \(\mathbf{V}=\left(\mathbf{v}_{1}^{T},\ldots,\mathbf{v}_{d}^{T}\right)^{T}\in \mathbb{R}^{d\times l}\) is the Lagrangian multiplier, and \(\beta>0\) is the penalty parameter. Equation (5) involves four variables, that is, the auxiliary variable \(\mathbf{U}\), the feature weight matrix \(\mathbf{W}\), the feature indicator vector \(\mathbf{p}\), and the Lagrangian multiplier \(\mathbf{V}\). Clearly, simultaneously optimizing four variables is impractical. Accordingly, \(\mathbf{V}\) is temporarily fixed for simplification in the following analysis.