# An Effective Multi-Label Feature Selection Model Towards Eliminating Noisy Features
## 3 The Methodology: ELC
### Property Analysis
In addition to the above property with respect to maximally embedding label correlations, another important property of ELC is illustrated as follows:  
**Property 2**. : _Feature redundancy can be minimized by ELC._  
Proof. : \(tr(\mathcal{H}^{T}\mathcal{H})=\sum_{i,j=1}^{k}\left(\left(\mathbf{\hat{f}}_ {i}^{T}\mathbf{Y}\right)(\mathbf{\hat{f}}_{j}^{T}\mathbf{Y})^{T}\right)^{2}= \sum_{i,j=1}^{k}\sum_{c=1}^{l}\left(\left\langle\mathbf{\hat{f}}_{i},\mathbf{y }_{c}\right\rangle\langle\mathbf{\hat{f}}_{j},\mathbf{y}_{c}\right\rangle\right) ^{2}\)  
\(=\sum_{i,j=1}^{k}\sum_{c=1}^{l}n^{4}\sigma_{\mathbf{y}_{c}}^{4}\rho_{\mathbf{ \hat{f}}_{i},\mathbf{y}_{c}}^{2}\rho_{\mathbf{\hat{f}}_{j},\mathbf{y}_{c}}^{2}\), where \(\sigma_{\mathbf{y}_{c}}\) is the standard deviation of the label \(\mathbf{y}_{c}\), and \(\rho_{\mathbf{\hat{f}}_{i},\mathbf{y}_{c}}\) and \(\rho_{\mathbf{\hat{f}}_{j},\mathbf{y}_{c}}\) are the Pearson correlation coefficients of \(\mathbf{y}_{c}\) with the features \(\mathbf{\hat{f}}_{i}\) and \(\mathbf{\hat{f}}_{j}\), respectively. Then, we have \(\min_{\mathbf{W}}\left\|\hat{\mathbf{Y}}^{T}\hat{\mathbf{Y}}-\mathbf{S}\right\| _{F}^{2}\propto\min_{\mathbf{W}}\sum_{i,j=1}^{k}\sum_{c=1}^{l}n^{4}\sigma_{ \mathbf{y}_{c}}^{4}\rho_{\mathbf{\hat{f}}_{i},\mathbf{y}_{c}}^{2}\rho_{\mathbf{ \hat{f}}_{j},\mathbf{y}_{c}}^{2}\). Clearly, \(n\) and \(\sigma_{\mathbf{y}_{c}}\) are constant in the feature selection process. \(\sum_{c=1}^{l}\rho_{\mathbf{\hat{f}}_{i},\mathbf{y}_{c}}\rho_{\mathbf{\hat{f}} _{j},\mathbf{y}_{c}}\) can be regarded as the shared label dependency of the features \(\mathbf{\hat{f}}_{i}\) and \(\mathbf{\hat{f}}_{j}\), that is, the feature redundancy for recognizing the target \(\mathbf{y}_{c}\). Therefore, ELC can minimize feature redundancy in its feature selection process. Note that the term \(\sum_{c=1}^{l}\rho_{\mathbf{\hat{f}}_{i},\mathbf{y}_{c}}\rho_{\mathbf{\hat{f}} _{j},\mathbf{y}_{c}}\) in Property 2 is obtained by introducing the label correlation information. This is a completely novel estimation for the label-specific feature redundancy. The most majority of existing feature selection approaches (including the single-label and multi-label ones) adopt a univariate measurement criterion and merely the top-\(k\) features have opportunities to prevail. This strategy largely increases the redundant recognition information shared between features. For example, if we select the genes that are all discriminative for the diabetes type 1, we probably cannot give an accurate diagnosis since these features may be less aware of other types of diabetes. This is why we have to reduce recognition redundancy and enrich recognition information. Some approaches are able to reduce feature redundancy, while their focus is not the label-specific redundancy. For example, \(\sum_{i,j=1}^{k}\rho_{\mathbf{\hat{f}}_{i},\mathbf{\hat{f}}_{j}}\) is actually reduced in SPFS [20]. This term includes an additional information irrelevant to recognition, and correspondingly, it is inappropriate. In contrast, ELC removes label-specific feature redundancy and is more suitable for multi-label learning with eliminating noisy features. As discussed above, ELC processes two properties, i.e., maximally preserving label correlation information and minimizing label-specific feature redundancy. These characteristics account for the superior ability of ELC in eliminating noisy features and picking out remarkable ones.