L i GCN : Label-interpretable Graph Convolutional Networks for Multi-label Text Classification (2022)

Abstract: Multi-label text classification (MLTC) is an attractive and challenging task in natural language processing (NLP). Compared with single-label text classification, MLTC has a wider range of applications in practice. In this paper, we propose a label-interpretable graph convolutional network model to solve the MLTC problem by modeling tokens and labels as nodes in a heterogeneous graph. In this way, we are able to take into account multiple relationships including token-level relationships. Besides, the model allows better interpretability for predicted labels as the token-label edges are exposed. We evaluate our method on four real-world datasets and it achieves competitive scores against selected baseline methods. Specifically, this model achieves a gain of 0.14 on the F1 score in the small label set MLTC, and 0.07 in the large label set scenario.

## introduction in the real world, we have seen an explosion of information on the internet, such as tweets, microblogs, articles, blog posts, etc. a practical issue is to assign classification labels to those instances. such labels may be emotion tags for tweets and micro-blogs @xcite , or topic category tags for news, articles and blog posts @xcite . multi-label text classification (mltc) is the problem of assigning one or more labels to each instance. deep learning has been applied for mltc due to their strong representation capacity in nlp tasks. it has been shown that convolutional neural networks (cnns) @xcite achieve satisfying results for multi-label emotion classification @xcite . besides, many recurrent neural networks (rnns)-based models @xcite are also playing an important role @xcite . recent breakthrough of pre-trained models, i.e., bert @xcite and roberta @xcite text labels s1 我不知道类似这样的困惑到 底 还 要 持 续 多 久 。 (i don't know how long the confusion like this will last.) anxiety s2 nothing happened to make me sad but i almost burst into tears like 3 times today pessimism, sadness s3 ...the price of basf ag shares improved on thursday due to its better than expected half year results. at 0900 gmt basf was up 51 pfennigs at 42.75 marks... c15, c151, c152, ccat to tackle the mentioned challenges and investigate different perspectives, we propose labelinterpretable graph convolutional networks for mltc. we model each token and class label as nodes in a heterogeneous graph, considering various types of edges: token-token, token-label, and label-label. then we apply graph convolution to graph-level classification. as gcn works well in semi-supervised learning @xcite , we can then ease the impact of data imbalance. finally, since the token-label relationships are exposed in the graph, one can easily identify the triggering tokens to a specific class, providing a good interpretability for multi-label classification. the contributions of our work are as follows: (1) we transfer the mltc task to a link prediction task within a constructed graph to predict output labels. in this way, our model is able to provide token-level interpretation for classification. (2) to the best of our knowledge, this is the first work that considers token-label relationships within a manner of a graph neural network for mltc, allowing label interpretability. (3) we conduct extensive experiments on four representative datasets and achieve competitive results. we also demonstrate comprehensive analysis and ablation studies to show the effectiveness of our proposed model for label nodes and token-label edges. we release our code in https: //github.com/irenezihuili/ligcn . graph neural networks in nlp previous research has introduced gcn-based methods for nlp tasks by formulating them as graph-structured data tasks. a fundamental task is text classification. many works show that it is possible to utilize interrelations of documents or tokens to infer the labels @xcite . besides, some nlp tasks focus on learning relationships between nodes in a graph, such as concept prerequisites @xcite and leveraging dependency trees predicted by gcns for machine translation @xcite . recently, variations of gcn models have been investigated for general text classification tasks @xcite @xcite . limited efforts have been made to apply gcns for multi-label text classification. for example, emograph @xcite is a model that captures the dependencies among emotions through graph networks. ## method in this section, we first provide task definition and preliminary, then we introduce the proposed model for multi-label text classification. ## experimental results we evaluate on four public datasets, summarized in we report the following evaluation metrics: micro/macro f1, jaccard index we report micro-average and macro-average f1 scores as did by previous works @xcite if the label set is small. besides, we follow the jaccard index used by @xcite @xcite , as always referred as multi-label accuracy. the definition is given below: ## analysis ## conclusion in this work, we propose a label-interpretable graph model, ligcn, to solve the mltc problem as a link prediction task. our model is able to provide token-level explanation for the classification and therefore enjoys better label interpretability. experiments on four public datasets show that our model achieved competitive scores. in the future, we will experiment with more complex graph encoders, extend this idea to single-label and extreme multi-label classification tasks @xcite .