# The Influence of Feature Representation of Text on the Performance of Document Classification
## 4 Results
### AUROC Performance and Rankings  
Table 4 summarizes the macro-averaged AUROC performance of the document representation models. The results, especially the distribution of the top-ranked models in the table, suggest that classification models based on the GOW model were inferior to the ones built upon BOW and continuous-space models. Furthermore, the results provide a hint towards top-performing models within individual groups; in the bag-of-words group of models, the TFIDF variant showed superior performance; word2 vec-50 and doc2 vec-25 were top-ranked in the two groups of continuous-space models, while gow-avg was the top-ranked graph-of-words model. Furthermore, Figure 2 presents the average ranks of the four top-ranked models in each group. The average-rank diagram shows that bow-TFIDF and the doc2 vec-25 model variants lead to a comparable performance (average ranks of 1.57 and 1.71, respectively), which was quite different from the overall performance of the other two models. The critical distance (CD) analysis, depicted in the diagram, also shows that the differences between the bag-of-words and continuous-space model variants were not statistically significant; the graph-based model was the only model with a statistically significant inferior performance. Note that we have repeated this analysis for the ranking according to the other performance measures considered in the paper (accuracy, F1-score, precision and recall). The average rank diagrams in all four cases were almost identical to the one reported in Figure 2, hence we omit them. \begin{table}
\begin{tabular}{c c c c c c c c} \hline  & **Brown2** & **Brown4** & **Brown10** & **Brown15** & **Reuters8** & **WebKB** & **20 News** \\ \hline bow-TF & 0.99 & 0.94 & 0.91 & 0.88 & 0.99 & 0.91 & 0.93 \\ bow-TFIDF & 0.99 & 0.92 & 0.89 & 0.90 & 0.99 & 0.91 & 0.97 \\ \hline word2 vec-25 & 0.99 & 0.87 & 0.82 & 0.79 & 0.98 & 0.91 & 0.94 \\ word2 vec-50 & 0.99 & 0.89 & 0.83 & 0.81 & 0.98 & **0.92** & 0.95 \\ word2 vec-75 & 0.99 & 0.88 & 0.82 & 0.80 & 0.99 & **0.92** & 0.95 \\ word2 vec-100 & 0.99 & 0.89 & 0.82 & 0.82 & 0.99 & **0.92** & 0.95 \\ \hline doc2 vec-25 & 0.99 & 0.94 & 0.92 & 0.89 & 0.99 & 0.90 & 0.96 \\ doc2 vec-50 & 1.00 & 0.95 & 0.92 & 0.91 & 0.98 & 0.88 & 0.96 \\ doc2 vec-75 & 1.00 & 0.95 & 0.91 & 0.87 & 0.98 & 0.88 & 0.95 \\ doc2 vec-100 & 0.99 & 0.93 & 0.91 & 0.88 & 0.96 & 0.85 & 0.95 \\ doc2 vec-200 & 0.99 & 0.93 & 0.90 & 0.86 & 0.96 & 0.84 & 0.92 \\ doc2 vec-500 & 0.98 & 0.91 & 0.84 & 0.84 & 0.96 & 0.82 & 0.91 \\ doc2 vec-1000 & 0.99 & 0.90 & 0.85 & 0.83 & 0.97 & 0.84 & 0.92 \\ \hline gow-avg & 0.98 & 0.90 & 0.76 & 0.80 & 0.78 & 0.70 & 0.61 \\ gow-qrt & 0.96 & 0.87 & 0.78 & 0.81 & 0.79 & 0.70 & 0.62 \\ gow-hist & 0.95 & 0.88 & 0.76 & 0.81 & 0.79 & 0.71 & 0.62 \\ \hline \end{tabular}
\end{table} 
Table 4: Macro-averaged area under the receiver-operating curve (AUROC) of the 16 variants of the document representation models on the seven classification tasks. The top-performing models for each task are shown in bold. Table 5 summarizes the multi-objective rankings of the 16 document representation models according to the AUROC performance measure. The average ranks suggest that doc2 vec had a performance superior to all the other representation model variants. Again, this was only true for the variants of doc2 vec corresponding to low-dimensional feature vectors (50 and 75). The other models were ranked as follows: BOW variants closely followed the top-ranked doc2 vec variants, word2 vec variants followed them, while GOW variants were the worst performers. Just as in the case of macro-averaged AUROC, gov-avg was the top-ranked variant of the GOW representation model. The overall results of the multi-objective ranking closely resembled the results obtained using the macro-averaged AUROC. Figure 3 depicts the comparison of the four top-ranked variants in each of the four groups of models according to the multi-objective AUROC ranking.