Distinct Label Representations for Few-Shot Text Classification (2021)

Abstract: Few-shot text classification aims to classify inputs whose label has only a few examples. Previous studies overlooked the semantic relevance between label representations. Therefore, they are easily confused by labels that are relevant. To address this problem, we propose a method that generates distinct label representations that embed information specific to each label. Our method is applicable to conventional few-shot classification models. Experimental results show that our method significantly improved the performance of few-shot text classification across models and datasets.

## introduction few-shot text classification @xcite @xcite has been actively studied aiming to classify texts whose labels have only a few examples. such infrequent labels are pervasive in datasets in practice, which are headaches for text classifiers because of the lack of training examples. @xcite showed that the conventional text classifiers are annoyed by the over-fitting problem when the distribution of labels is skewed in a dataset. few-shot classification has two approaches: metric-based and meta-learning based methods. the metric-based methods conduct classification based on distances estimated by a certain metric, e.g., cosine similarity @xcite , euclidean distance @xcite , convolutional neural networks @xcite , and graph neural networks @xcite . metric-based methods in natural language processing focus on representation generation that are suitable for few-shot classification using the attention mechanism with various granularity @xcite , local and global matching of representations @xcite , and word co-occurrence patterns in attention mechanisms @xcite . in contrast, meta-larning based methods learn to learn for achieving higher accuracy by learning parameter generation @xcite , learning rates and parameter updates @xcite , and parameter updates using gradients @xcite @xcite . all of these previous studies overlooked the effects of the semantic relevance between label representations, which confuses few-shot classifiers. as a result, the classifiers tend to fail distinguishing examples with semantically relevant labels. to address this problem, we propose a mechanism that compares label representations to derive distinctive representations. it learns semantic differences between labels and generates representations that embed information specific to each label. our method can be easily applied to existing few-shot classification models. we evaluated our method using the standard benchmarks of huffpost and fewrel @xcite . experimental results showed that our method significantly improved the performance of previous few-shot classifiers across models and datasets, and achieved the state-of-the-art accuracy. ## few-shot text classification this section describes the problem definition and a general form of conventional few-shot classifiers. ## proposed method figure 1 shows the overview of our method. it adds a mechanism for learning to generate distinctive label representations into conventional few-shot classification models by converting its training into multi-task learning. our method adds a difference extractor (section 3.1) and a loss function based on mutual information (section 3.2) to an arbitrary few-shot classification model. ## experiment we evaluated our method on different few-shot classification models using the standard benchmarks. ## conclusion and future work in this paper, we introduced a novel method shedding light on semantic relations between labels. our method improved the classification accuracy of representative few-shot classifiers on both huffpost and fewrel datasets, confirming the reasonable applicability of the proposed method. technically, our method can be applied to other classification problems that handle semantic labels, such as image and entity classifications. we will conduct evaluation to see its effects on various types of classifcations.