Few-Shot Text Classification with Edge-Labeling Graph Neural Network-Based Prototypical Network (2020)

Abstract: In this paper, we propose a new few-shot text classification method. Compared with supervised learning methods which require a large corpus of labeled documents, our method aims to make it possible to classify unlabeled text with few labeled data. To achieve this goal, we take advantage of advanced pre-trained language model to extract the semantic features of each document. Furthermore, we utilize an edge-labeling graph neural network to implicitly models the intra-cluster similarity and the inter-cluster dissimilarity of the documents. Finally, we take the results of the graph neural network as the input of a prototypical network to classify the unlabeled texts. We verify the effectiveness of our method on a sentiment analysis dataset and a relation classification dataset and achieve the state-of-the-art performance on both tasks.

## introduction in recent years, deep learning has achieved great success in many fields such as computer vision, speech recognition and natural language processing. however, to train a deep learning model usually requires a large scale of manually annotated data, which greatly limits the practicality and scalability of the model. in the text classification task, the problem of greed for large amounts of labeled data also exists. text classification is an important task in natural language processing field. over the years, many text classification methods have been proposed, varying from cnn based methods @xcite @xcite to rnn, including lstm and gru-based methods @xcite . but all the above supervised learning methods require a large corpus of labeled data, making these models hindered in practical application. in recent years, some scholars have contributed to solving the few-shot text classification problem. @xcite proposed a dataset fewrel and tested the performance of several typical few-shot learning models proposed in recent years on this dataset. @xcite proposed an adaptive metric learning model, which can automatically determine the best weighted combination of a set of metrics obtained from a meta-learning process for a newly arrived few-shot text classification task. @xcite proposed a prototypical network model @xcite based on a hybrid attention mechanism, which has a better performance for noisy data scenes. @xcite improved prototypical network by adopting hierarchical attention mechanism, which is applied in feature level, word level and instance level to enhance the expressive ability of semantic space. @xcite applied the dynamic routing algorithm in meta-learning and proposed an induction network, which achieves a better generalization ability on different few-shot text classification tasks. we believe that previous few-shot text classification methods generally have two flaws. first, all the methods mentioned above adopt a glove word embedding combined with cnn or rnn structure for text embedding instead of more advanced pre-trained language models proposed in recent years, such as elmo @xcite , gpt @xcite and bert @xcite . second, these methods focus more on the semantic features of the texts itself, ignoring the potential relationships this work is licensed under a creative commons attribution 4.0 international license. license details: http:// creativecommons.org/licenses/by/4.0/ . ## related works few-shot learning (fsl) aims to learn classifiers for new classes with only a few training examples per class. given a support set of labeled instances, the goal is to classify instances from query set. the seminal work on few-shot learning dates back to the early 2000s @xcite . more recent work can be divided into two types: similarity-based methods @xcite @xcite and optimization-based methods @xcite . researchers have also studied fsl in various nlp tasks @xcite @xcite @xcite . graph neural networks (gnn) were first proposed by @xcite and @xcite . li (2015) further extended it with gated recurrent units. generalized convolution-based propagation rules also have been directly applied to graphs @xcite @xcite , offering a balance between expressivity and sample complexity. in recent years, a few approaches @xcite @xcite further explored gnns for few-shot learning. ## conclusion this work addresses the problem of few-shot text classification. we take advantage of advanced pretrained language model bert to extract the semantic feature of the text. then we introduce edgelabeling graph neural network to further model the potential relationships between texts. finally we utilize a prototypical network to classify the query text. in the experiments, our method achieves the state-of-the-art performance on both arsc and fewrel datasets. for future work, we can consider generalizing our method to other few-shot nlp problems, and even to non-nlp tasks.