Inductive Topic Variational Graph Auto-Encoder for Text Classification (2021)

Abstract: Graph convolutional networks (GCNs) have been applied recently to text classification and produced an excellent performance. However, existing GCN-based methods do not assume an explicit latent semantic structure of documents, making learned representations less effective and difficult to interpret. They are also transductive in nature, thus cannot handle out-of-graph documents. To address these issues, we propose a novel model named inductive Topic Variational Graph Auto-Encoder (T-VGAE), which incorporates a topic model into variational graph-auto-encoder (VGAE) to capture the hidden semantic information between documents and words. T-VGAE inherits the interpretability of the topic model and the efficient information propagation mechanism of VGAE. It learns probabilistic representations of words and documents by jointly encoding and reconstructing the global word-level graph and bipartite graphs of documents, where each document is considered individually and decoupled from the global correlation graph so as to enable inductive learning. Our experiments on several benchmark datasets show that our method outperforms the existing competitive models on supervised and semi-supervised text classification, as well as unsupervised text representation learning. In addition, it has higher interpretability and is able to deal with unseen documents.

## introduction recently, graph convolutional networks (gcns) @xcite have been successfully applied to text classification tasks @xcite @xcite . in addition to the local information captured by cnn or rnn, gcns learn word and document representations by taking into account the global correlation information embedded in the corpuslevel graph, where words and documents are nodes connected by indexing or citation relations. however, the hidden semantic structures, such as latent topics in documents @xcite @xcite , is still ignored by most of these methods @xcite @xcite , which can improve the text representation and provide extra interpretability (in which the probabilistic generative process and topics make more sense to humans compared to neural networks, i.e. topics can be visually represented by top-10 or 20 most probable word clusters). although few studies such as @xcite have proposed incorporating a topic structure into gcns, the topics are extracted in advance from the set of documents, independently from the graph and information propagation among documents and words. we believe that the topics should be determined in accordance with the connections in the graph. for example, the fact that two words are connected provides extra information that these words are on a similar topic(s). moreover, existing gcn-based methods are limited by their transductive learning nature, i.e. a document can be classified only if it is already seen in the training phase @xcite @xcite . the lack of inductive learning ability for unseen documents is a critical issue in practical text classification applications, where we have to deal with new documents. it is ## experiment in this section, to evaluate the effectiveness of our proposed t-vgae, experiments are conducted on both supervised and semi-supervised text classification tasks, as well as unsupervised topic modeling tasks. ## conclusion in this paper, we proposed a novel deep latent variable model t-vgae via combining the topic model with vgae. it can learn more interpretable representations and leverage the latent topic semantic to improve the classification performance. t-vgae inherits advantages from the topic model and vgae: probabilistic interpretability and efficient label propagation mechanism. experimental results demonstrate the effectiveness of our method along with inductive learning. as future work, it would be interesting to explore better-suited prior distribution in the generative process. it is also possible to extend our model to other tasks, such as information recommendation and link prediction.