Large-scale Multi-label Text Classification - Revisiting Neural Networks
Authors: Jinseok Nam, Jungi Kim, Iryna Gurevych, Johannes Fürnkranz
ICLR 2014

Abstract: Large-scale datasets with multi-labels are becoming readily available, and the demand for large-scale multi-label classification algorithm is also increasing. In this work, we propose to utilize a single-layer Neural Networks approach in large-scale multi-label text classification tasks with recently proposed learning techniques. We carried out experiments on six textual datasets with varying characteristics and size, and show that a simple Neural Networks model equipped with recent advanced techniques for Neural Networks components such as an activation layer, optimization, and generalization techniques performs as well as or even outperforms the previous state-of-the-art approaches on large-scale datasets with diverse characteristics.


Peer review:
[Review]:
The paper describes a series of experiments for multi-labeled text classification using neural networks. The classification model is a simple one hidden layer NN integrating rectifier units and dropout.  A comparison of this model with baselines (Binary relevance and a ranking NN) is performed on 6 multi-labeled datasets with different characteristics.
This is an experimental paper. The NN model is a classical MLP and the only new algorithmic contribution concerns the prediction of decision thresholds for binary decisions. On the other hand, the experimental comparison is extensive and allows the authors to examine the benefits of recent improvements for NNs on the task of text classification. The datasets characteristics are representative of different text classification problems (dataset and vocabulary size, label cardinality). Different loss functions are also used for the evaluation. The paper could then be useful for popularizing NNs for text classification.
I am not sure that the BP-MLL is a reference algorithm for learning to rank, and there are probably better candidates. On the other hand it is true that the simple binary framework (BR in the paper) is a strong baseline despite its simplicity, so that the results could be considered as significant. An extension of this work could be to consider large scale problems (both in the vocabulary size and in the number of categories, since several benchmarks are now available with a very large number of classes – see for example the LSHTC challenges). A deeper discussion on the respective complexity of the different approaches and of their behavior when the number of training examples varies (learning curves) would strengthen the paper.