Dynamic Memory Induction Networks for Few-Shot Text Classification (2020)

Abstract: This paper proposes Dynamic Memory Induction Networks (DMIN) for few-short text classification. The model develops a dynamic routing mechanism over static memory, enabling it to better adapt to unseen classes, a critical capability for few-short classification. The model also expands the induction process with supervised learning weights and query information to enhance the generalization ability of meta-learning. The proposed model brings forward the state-of-the-art performance significantly by 2 4% improvement on the miniRCV1 and ODIC datasets. Detailed analysis is further performed to show how the proposed network achieves the new performance.

## introduction few-shot text classification, which requires models to perform classification with a limited number of training instances, is important for many applications but yet remains to be a challenging task. early studies on few-shot learning @xcite employ data augmentation and regularization techniques to alleviate overfitting caused by data sparseness. more recent research leverages meta-learning @xcite @xcite to extract transferable knowledge among meta-tasks in meta episodes. a key challenge for few-shot text classification is inducing class-level representation from support sets @xcite , in which key information is often lost when switching between meta-tasks. recent solutions @xcite leverage a memory component to maintain models' learning experience, e.g., by finding from a supervised stage the content that is similar to the unseen classes, leading to the state-of-the-art performance. however, the memory weights are static during inference and the capability of the model is still limited when adapted to new classes. another prominent challenge is the instance-level diversity caused by various reasons @xcite , resulting in the difficulty of finding a fixed prototype for a class @xcite . recent research has shown that models can benefit from query-aware methods @xcite . in this paper we propose dynamic memory induction networks (dmin) to further tackle the above challenges. dmin utilizes dynamic routing @xcite to render more flexibility to memory-based few-shot learning @xcite in order to better adapt the support sets, by leveraging the routing component's capacity in automatically adjusting the coupling coefficients during and after training. based on that, we further develop induction models with query information to identify, among diverse instances in support sets, the sample vectors that are more relevant to the query. these two modules are jointly learned in dmin. the proposed model achieves new state-of-theart results on the minircv1 and odic datasets, improving the best performance by 2∼4% accuracy. we perform detailed analysis to further show how the proposed network achieves the improvement. ## related work few-shot learning has been studied in early work such as @xcite and more recent work @xcite @xcite @xcite @xcite @xcite . researchers have also investigated few-shot learning in various nlp tasks @xcite @xcite @xcite , including text classification @xcite @xcite @xcite . memory mechanism has shown to be very effective in many nlp tasks @xcite @xcite . in the fewshot learning scenario, researchers have applied memory networks to store the encoded contextual information in each meta episode @xcite @xcite . specifically @xcite and @xcite build a two-stage training procedure and regard the supervisely learned class representation as a memory component. ## conclusion we propose dynamic memory induction networks (dmin) for few-shot text classification, which builds on external working memory with dynamic routing, leveraging the latter to track previous learning experience and the former to adapt and generalize better to support sets and hence to unseen classes. the model achieves new state-of-the-art results on the minircv1 and odic datasets. since dynamic memory can be a learning mechanism more general than what we have used here for fewshot learning, we will investigate this type of models in other learning problems.