# An Effective Multi-Label Feature Selection Model Towards Eliminating Noisy Features
## 3 The Methodology: ELC  
### Model Description  
In this paper, we use \(\{\mathbf{x}_{i},\mathbf{y}_{i}\}_{i=1}^{n}\) to denote the data set, where \(\mathbf{X}=[\mathbf{x}_{1};...;\mathbf{x}_{n}]\in\mathbb{R}^{n\times d}\) represents the instance matrix and instances are characterized by \(d\) features in the feature set \(\mathbf{F}=\{\mathbf{f}_{1},...,\mathbf{f}_{d}\}\). \(\mathbf{Y}=[\mathbf{y}_{1},...,\mathbf{y}_{l}]\in\{0,1\}^{n\times l}\) denotes the target label matrix, where \(y_{ij}=1\) represents a positive label and \(y_{ij}=0\) corresponds to a negative one.  
Then, we formulate the multi-label feature selection by embedding label correlation (ELC) as follows:  
\[\min_{\mathbf{W}}\frac{1}{2}\left\|\hat{\mathbf{Y}}^{T}\hat{\mathbf{Y}}- \mathbf{S}\right\|_{F}^{2},\,s.t.\,\hat{\mathbf{Y}}=\frac{1}{n}(\mathbf{X} \mathbf{W})^{T}\mathbf{Y},\mathbf{W}\in\{0,1\}^{d\times l},\left\|\mathbf{W} \right\|_{2,0}=k, \tag{1}\]  
where \(\mathbf{S}\in\mathbb{R}^{l\times l}\) represents the label correlation matrix calculated over the initial label matrix, and \(k\) is the number of selected features. \(\mathbf{W}\in\mathbb{R}^{d\times l}\) is the feature selection matrix, where \(w_{ij}\) indicates the importance (also known as weight) of the \(i\)-th feature to the \(j\)-th label.  
Equation (1) is actually the feature evaluation function of ELC, which is essentially a Frobenius-norm quadratic model. The matrix \(\mathbf{S}\) represents the label correlations extracted from the label space, and its each element describes a relation between two target labels. These correlations can be easily obtained by some quantitative measurements, including RBF kernel function, Pearson correlation coefficient, etc. \(\hat{\mathbf{Y}}^{T}\hat{\mathbf{Y}}\) represents the label correlations extracted from the reduced feature space. \(\hat{\mathbf{Y}}^{T}\hat{\mathbf{Y}}\) is differentiated from \(\mathbf{S}\) on account of the disturbance of noisy features. As described in Section 1, noisy features may distort the structure of the feature space and provide negative learning information. Considering this, ELC evaluates features based on their abilities of preserving label correlations in the feature space, that is, keeping feature-label space consistency. The features that can minimize the discrepancy between \(\hat{\mathbf{Y}}^{T}\hat{\mathbf{Y}}\) and \(\mathbf{S}\) will be highly scored by ELC. In this way, ELC can be expected to construct an optimal feature subspace with eliminating different kinds of noisy features.  
Under the constraint of the \(\ell_{2,0}\)-norm in Equation (1), only \(k\) row in \(\mathbf{W}\) is nonzero. This corresponds to the \(k\) selected features for \(l\) target labels, where 1 represents selected and 0 represents none. Note that \(k\) is most likely to be unequal to \(l\). That is, more than one feature may be selected responsible for discriminating the same label, or only one feature is discriminative for more than one label. In the former case, multiple features are unified to recognize one target, while one feature deals with multiple recognition sub-tasks in the latter case.