MGIMN : Multi-Grained Interactive Matching Network for Few-shot Text Classification (2022)

Abstract: Text classification struggles to generalize to unseen classes with very few labeled text instances per class. In such a few-shot learning (FSL) setting, metric-based meta-learning approaches have shown promising results. Previous studies mainly aim to derive a prototype representation for each class. However, they neglect that it is challenging-yet-unnecessary to construct a compact representation which expresses the entire meaning for each class. They also ignore the importance to capture the inter-dependency between query and the support set for few-shot text classification. To deal with these issues, we propose a meta-learning based method MGIMN which performs instance-wise comparison followed by aggregation to generate class-wise matching vectors instead of prototype learning. The key of instance-wise comparison is the interactive matching within the class-specific context and episode-specific context. Extensive experiments demonstrate that the proposed method significantly outperforms the existing SOTA approaches, under both the standard FSL and generalized FSL settings.

## introduction few-shot text classification has attracted considerable attention because of significant academic research value and practical application value @xcite @xcite @xcite . many efforts are devoted towards different goals like generalization to new classes @xcite @xcite , adaptation to new domains and tasks @xcite @xcite @xcite . low-cost generalization to new classes is critical to deal with the growing long-tailed categories, which is common for intention classification @xcite 2019 the "s nk , s n " and "s" represent the instance level, class level and episode level interaction respectively. while "a" in the purple circle denotes alignment between query and instances. the "'m" stands for matching operation. @xcite and relation classification @xcite . to prevent over-fitting to few-shot new classes and avoid retraining the model when the class space changes, metric-based meta learning has become the major framework with significant results @xcite . the core idea is that episode sampling is employed in meta-training phase to learn the relationship between query and candidate classes @xcite . a key challenge is inducing class-wise representations from support sets because nature language expressions are diverse @xcite and highly informative lexical features are unshared across episodes @xcite . under metric-based meta learning framework, many valuable backbone networks have emerged. @xcite presented prototypical network that computes prototype of each class using a simple mean pooling method. @xcite proposed hybrid attention mechanism to ease the negative effects of noisy support examples. @xcite proposed an induction network to induce better prototype representations. @xcite obtained each prototype vector by aggregating local matching and instance matching information. @xcite proposed a novel method to compute prototypes based on both lexical features and word occurrence patterns. all these previous works first obtain class-wise representations and then perform class-wise comparisons. however, it is challenging-yet-unnecessary to construct a compact representation which expresses the entire meaning for each class @xcite . in text matching research, compareaggregate methods which perform token-level comparisons followed by sentence-level aggregation has already been successful @xcite @xcite . besides backbone networks,there are still some work that can be further combined. @xcite utilized classlabel information for extracting more discriminative prototype representation. @xcite generated a large-scale meta tasks from unlabeled text in a self-supervised manner. in this paper we propose multi-grained interactive matching network, a backbone network for few-shot text classification. the core difference between us with previous efforts is that our work performs instance-level comparisons followed by class-wise aggregation. specifically, first, all text sequences including query and all support instances are encoded to contextual representations. second, as depicted in figure 1 , we design a novel multi-grained interactive matching mechanism to perform instance-wise comparisons which capture the inter-dependency between query and all support instances. third, class-wise aggregate layer obtains class-wise matching vector between query and each class. finally, a prediction layer predicts final results. in contrast to standard fsl setting, generalized fsl setting is a more challenging-yet-realistic setting where seen classes and new classes are coexistent @xcite . in such a setting, we analyze the relationship between inference speed and the number of classes, and verify the necessity of retrieval, which is ignored by previous studies. our contributions are listed as follows: ## method as illustrated in the figure 2 , mgimn consists of four modules: encoder layer, instance matching layer, class-wise aggregation layer and prediction layer. ## conclusion in this research investigation, we introduce the multi-grained interactive matching network(mgimn) for the text classification task with few-shot manner. meanwhile, we introduce a two-stage method retrieval-then-classify (rtc) to solve the inference performance in realistic scenery. experiment results illustrate that the presented method obtains the best result in all five different kinds of datasets with two evaluation metrics. moreover, rtc method obviously make the inference speed getting faster. we will make further investigations on the the task of domain adaptation problem by extending our proposed method.