# A Feature Selection Method for Multi-Label Text Based on Feature Importance
## 4 Experiment and Results
### Evaluation Metrics  
The results of the multi-label classification experiments were evaluated by the following five evaluation metrics, which are described in Formulas (20)-(24) [41].  
In order to describe these formulas conveniently, we define the following variables: \(S=\{(x_{i},Y_{i})|1\leq i\leq p\}\) denotes a multi-label test set, \(\gamma=\{L_{1},L_{2},\cdots,L_{q}\}\) denotes the category set, \(h(x_{i})\) denotes the multi-label classifier, \(f(x_{i},L_{s})\) denotes the prediction function and \(rank_{f}(x_{i},L_{s})\) denotes the ranking function. Where \(x_{i}\) is a sample, \(Y_{i}\) is the set of categories of \(x_{i}\), and \(Y_{i}\subseteq\gamma\), and \(p\) is the number of samples in \(S\). Also, if \(f(x_{i},L_{s})>f(x_{i},L_{t})\), then \(rank_{f}(x_{i},L_{s})<rank_{f}(x_{i},L_{t})\).  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**NO.** & **Data set** & 1S1 & _dim(S)_ & **LC(S)** & \(LCard(S)\) & \(LDen(S)\) \\ \hline
1 & Medical & 978 & 1449 & 45 & 1.2454 & 0.0277 \\
2 & Business & 11,214 & 438 & 30 & 1.5990 & 0.0533 \\
3 & Computers & 12,444 & 681 & 33 & 1.5072 & 0.0457 \\
4 & Entertainment & 12,730 & 640 & 21 & 1.4137 & 0.0673 \\
5 & Health & 9205 & 612 & 32 & 1.6441 & 0.0514 \\
6 & Social & 12,111 & 1047 & 39 & 1.2793 & 0.0328 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Data sets description.  
Average precision (\(AP\)), which evaluates the average fraction of categories ranked above a particular category \(L_{s}\). The higher the value is, the better the performance is.  
\[AP=\frac{1}{p}\sum\limits_{i=1}^{p}\frac{1}{|Y_{i}|}\sum\limits_{L\in Y_{i}} \frac{\left|\left\{L_{s}|rank_{f}(x_{i},L_{s})\leq rank_{f}(x_{i},L_{s}),L_{s} \in Y_{i}\right\}\right|}{rank_{f}(x_{i},L)} \tag{20}\]  
Hamming loss (\(HL\)), which evaluates how many times a sample-label pair is misclassified. The smaller the value is, the better the performance is.  
\[HL=\frac{1}{p}\sum\limits_{i=1}^{p}\frac{1}{q}\left|h(x_{i})\Delta Y_{i}\right| \tag{21}\]  
where \(\Delta\) denotes the symmetric difference between two sets.  
One error (\(OE\)), which evaluates how many times the top-ranked category is not in the set of proper categories of the sample. The smaller the value is, the better the performance is.  
\[OE=\frac{1}{p}\sum\limits_{i=1}^{p}\left[\arg\max\limits_{L\in Y_{i}}f(x_{i},L)\
otin Y_{i}\right] \tag{22}\]  
where for any predicate \(\pi\), \([\pi]\) equals 1 if \(\pi\) holds and 0 otherwise.  
Coverage (\(CV\)), which evaluates how many steps are needed, on average, to move down the category list in order to cover all the proper categories of the sample. The smaller the value is, the better the performance is.  
\[CV=\frac{1}{p}\sum\limits_{i=1}^{p}\max\limits_{L\in Y_{i}}rank_{f}(x_{i},L)-1 \tag{23}\]  
Ranking loss (\(RL\)), which evaluates the average fraction of category pairs that are not correctly ordered. The smaller the value is, the better the performance is.  
\[RL=\frac{1}{p}\sum\limits_{i=1}^{p}\frac{1}{|Y_{i}|\big{|}|}|(L_{s},L_{t})|f(x _{i},L_{s})\leq f(x_{i},L_{t}),(L_{s},L_{t})\in Y_{i}*\overline{Y_{i}}\}| \tag{24}\]  
where \(\overline{Y_{i}}\) denotes the complementary set of \(Y_{i}\) in \(\gamma\).