Venue: ACL 2020

# Dynamic Memory Induction Networks for Few-Shot Text Classification

 Ruiying Geng\({}^{1}\), Binhua Li\({}^{1}\), Yongbin Li\({}^{1}\), Jian Sun\({}^{1}\), Xiaodan Zhu\({}^{2}\)

\({}^{1}\) Alibaba Group, Beijing

\({}^{2}\) Ingenuity Labs Research Institute & ECE, Queen's University

{ruiying.gry,binhua.lbh,shuide.lyb,jian.sun}@alibaba-inc.com zhu2048@gmail.com

Corresponding author.

###### Abstract

This paper proposes Dynamic Memory Induction Networks (DMIN) for few-shot text classification. The model utilizes dynamic routing to provide more flexibility to memory-based few-shot learning in order to better adapt the support sets, which is a critical capacity of few-shot classification models. Based on that, we further develop induction models with query information, aiming to enhance the generalization ability of meta-learning. The proposed model achieves new state-of-the-art results on the miniRCV1 and ODIC dataset, improving the best performance (accuracy) by 2\(\sim\)4%. Detailed analysis is further performed to show the effectiveness of each component.

## 1 Introduction

Few-shot text classification, which requires models to perform classification with a limited number of training instances, is important for many applications but yet remains to be a challenging task. Early studies on few-shot learning Salamon and Bello (2017) employ data augmentation and regularization techniques to alleviate overfitting caused by data sparseness. More recent research leverages meta-learning Finn et al. (2017); Zhang et al. (2018); Sun et al. (2019) to extract transferable knowledge among meta-tasks in meta episodes.

A key challenge for few-shot text classification is inducing class-level representation from support sets Gao et al. (2019), in which key information is often lost when switching between meta-tasks. Recent solutions Gidaris and Komodakis (2018) leverage a memory component to maintain models' learning experience, e.g., by finding from a supervised stage the content that is similar to the unseen classes, leading to the state-of-the-art performance. However, the memory weights are static during inference and the capability of the model is still limited when adapted to new classes. Another prominent challenge is the instance-level diversity caused by various reasons Gao et al. (2019); Geng et al. (2019), resulting in the difficulty of finding a fixed prototype for a class Allen et al. (2019). Recent research has shown that models can benefit from query-aware methods Gao et al. (2019).

In this paper we propose Dynamic Memory Induction Networks (DMIN) to further tackle the above challenges. DMIN utilizes dynamic routing Sabour et al. (2017); Geng et al. (2019) to render more flexibility to memory-based few-shot learning Gidaris and Komodakis (2018) in order to better adapt the support sets, by leveraging the routing component's capacity in automatically adjusting the coupling coefficients during and after training. Based on that, we further develop induction models with query information to identify, among diverse instances in support sets, the sample vectors that are more relevant to the query. These two modules are jointly learned in DMIN.

The proposed model achieves new state-of-the-art results on the miniRCV1 and ODIC datasets, improving the best performance by 2\(\sim\)4% accuracy. We perform detailed analysis to further show how the proposed network achieves the improvement.

## 2 Related Work

Few-shot learning has been studied in early work such as Fe-Fei et al. (2003); Fei-Fei et al. (2006) and more recent work Ba et al. (2016); Santoro et al. (2016); Munkhdalai and Yu (2017); Ravi and Larochelle (2016); Mishra et al. (2017); Finn et al. (2017); Vinyals et al. (2016); Snell et al. (2017); Sung et al. (2018); Allen et al. (2019). Researchers have also investigated few-shot learning in various NLP tasks Dou et al. (2019); Wu et al. (2019); Gu et al. (2018); Chen et al. (2019); Obamuyide and Vlachos (

2019; Hu et al., 2019), including text classification Yu et al. (2018); Rios and Kavuluru (2018); Xu et al. (2019); Geng et al. (2019); Gao et al. (2019); Ye and Ling (2019).

Memory mechanism has shown to be very effective in many NLP tasks Tang et al. (2016); Das et al. (2017); Madotto et al. (2018). In the few-shot learning scenario, researchers have applied memory networks to store the encoded contextual information in each meta episode Santoro et al. (2016); Cai et al. (2018); Kaiser et al. (2017). Specifically Qi et al. (2018) and Gidaris and Komodakis (2018) build a two-stage training procedure and regard the supervisely learned class representation as a memory component.

## 3 Dynamic Memory Induction Network

### Overall Architecture

An overview of our Dynamic Memory Induction Networks (DMIN) is shown in Figure 1, which is built on the two-stage few-shot framework Gidaris and Komodakis (2018). In the supervised learning stage (upper, green subfigure), a subset of classes in training data are selected as the base sets, consisting of \(C_{base}\) number of base classes, which is used to finetune a pretrained sentence encoder and to train a classifier.

In the meta-learning stage (bottom, orange subfigure), we construct an "episode" to compute gradients and update our model in each training iteration. For a \(C\)-way \(K\)-shot problem, a training episode is formed by randomly selecting \(C\) classes from the training set and choosing \(K\) examples within each selected class to act as the support set \(S=\cup^{C}_{c=1}\{x_{c,s},y_{c,s}\}^{K}_{s=1}\). A subset of the remaining examples serve as the query set \(Q=\{x_{q},y_{q}\}^{L}_{q=1}\). Training on such episodes is conducted by feeding the support set \(S\) to the model and updating its parameters to minimize the loss in the query set \(Q\).

### Pre-trained Encoder

We expect that developing few-shot text classifier should benefit from the recent advance on pre-trained models Peters et al. (2018); Devlin et al. (2019); Radford et al. (2018). Unlike recent work Geng et al. (2019), we employ BERT-base Devlin et al. (2019) for sentence encoding , which has been used in recent few-shot learning models Bao et al. (2019); Soares et al. (2019). The model architecture of BERT Devlin et al. (2019) is a multi-layer bidirectional Transformer encoder based on the original Transformer model Vaswani et al. (2017). A special classification embedding ([CLS]) is inserted as the first token and a special token ([SEP]) is added as the final token. We use the \(d\)-dimensional hidden vector output from the \([CLS]\) as the representation \(e\) of a given text \(x\): \(e=E(x|\theta)\). The pre-trained BERT model provides a powerful context-dependent sentence representation and can be used for various target tasks, and it is suitable for the few-shot text classification task Bao et al. (2019); Soares et al. (2019).

We finetune the pre-trained BERT encoder in the supervised learning stage. For each input document \(x\), the encoder \(E(x|\theta)\) (with parameter \(\theta\)) will output a vector \(e\) of \(d\) dimension. \(W_{base}\) is a matrix that maintains a class-level vector for each base class, serving as a base memory for meta-learning. Both \(E(x|\theta)\) and \(W_{base}\) will be further tuned in the meta training procedure. We will show in our experiments that replacing previous models with pre-trained encoder outperforms the corresponding state-of-the-art models, and the proposed DMIN can further improve over that.

### Dynamic Memory Module

At the meta-learning stage, to induce class-level representations from given support sets, we develop a dynamic memory module (DMM) based on knowledge learned from the supervised learning stage through the memory matrix \(W_{base}\). Unlike static memory Gidaris and Komodakis (2018), DMM utilizes dynamic routing Sabour et al. (2017) to render more flexibility to the memory learned from base classes to better adapt support sets. The routing component can automatically adjust the coupling coefficients during and after training, which inherently suits for the need of few-shot learning.

Specifically, the instances in the support sets

Figure 1: An overview of Dynamic Memory Induction Network with a 3-way 2-shot example.

 

are first encoded by the BERT into sample vectors \(\{e_{c,s}\}^{K}_{s=1}\) and then fed to the following dynamic memory routing process.

Dynamic Memory Routing ProcessThe algorithm of the dynamic memory routing process, denoted as DMR, is presented in Algorithm 1.

Given a memory matrix \(M\) (here \(W_{base}\)) and sample vector \(q\in R^{d}\), the algorithm aims to adapt the sample vector based on memory \(M\) learned in the supervised learning stage.

\[q^{\prime}=DMR(M,q). \tag{1}\]

First, for each entry \(m_{i}\in M\), the standard matrix-transformation and squash operations in dynamic routing Sabour et al. (2017) are applied on the inputs:

\[\hat{m}_{ij} = squash(W_{j}m_{i}+b_{j}), \tag{2}\] \[\hat{q}_{j} = squash(W_{j}q+b_{j}), \tag{3}\]

where the transformation weights \(W_{j}\) and bias \(b_{j}\) are shared across the inputs to fit the few-shot learning scenario.

We then calculate the Pearson Correlation Coefficients (PCCs) Hunt (1986); Yang et al. (2019) between \(\hat{m}_{i}\) and \(\hat{q}_{j}\).

\[p_{ij} = tanh(PCCs(\hat{m}_{ij},\hat{q}_{j})), \tag{4}\] \[PCCs = \frac{Cov(x_{1},x_{2})}{\sigma_{x_{1}}\sigma_{x_{2}}}. \tag{5}\]

where the general formula of PCCs is given above for vectors \(x_{1}\) and \(x_{2}\). Since PCCs values are in the range of [-1, 1], they can be used to encourage or penalize the routing parameters.

```
0:\(r\), \(q\) and memory \(M=\left\{m_{1},m_{2},...,m_{n}\right\}\)
0:\(\textbf{v}=v_{1},v_{2},...,v_{l}\), \(q^{\prime}\)
1:for all\(m_{i},v_{j}\)do
2:\(\hat{m}_{ij}=squash(W_{j}m_{i}+b_{j})\)
3:\(\hat{q}_{j}=squush(W_{j}q+b_{j})\)
4:\(\alpha_{ij}=0\)
5:\(p_{ij}=tanh(PCCs(\hat{m}_{ij},\hat{q}_{j}))\)
6:endfor
7:for\(r\) iterations do
8:\(d_{i}=softmax\left(\alpha_{i}\right)\)
9:\(\hat{v}_{j}=\sum_{i=1}^{n}(d_{ij}+p_{ij})\hat{m}_{ij}\)
10:\(v_{j}=squash(\hat{v}_{j})\)
11: for all \(i,j\): \(\alpha_{ij}=\alpha_{i,j}+p_{ij}\hat{m}_{ij}v_{j}\)
12: for all \(j\): \(\hat{q}_{j}=\frac{\hat{q}_{j}+v_{j}}{2}\)
13: for all \(i,j\): \(p_{ij}=tanh(PCCs(\hat{m}_{ij},\hat{q}_{j}))\)
14:endfor
15:\(q^{\prime}=concat[\textbf{v}]\)
16:Return\(q^{\prime}\)
```

**Algorithm 1** Dynamic Memory Routing Process 



### Similarity Classifier

In the final classification stage, we then feed the novel class vector \(e_{c}\) and query vector \(e_{q}\) to the classifier discussed above in the supervised training stage and get the classification score. The standard setting for neural network classifiers is, after having extracted the feature vector \(e\in R^{d}\), to estimate the classification probability vector \(p\) by first computing the raw classification score \(s_{k}\) of each category \(k\in[1,K^{*}]\) using the dot-product operator \(s_{k}=e^{T}w_{k}^{*}\), and then applying softmax operator across all the \(K^{*}\) classification scores. However, this type of classifiers do not fit few-shot learning due to completely novel categories. In this work, we compute the raw classification scores using a cosine similarity operator:

\[s_{k}=\tau\cdot cos(e,w_{k}^{*})=\tau\cdot\bar{e}^{T}\overline{w}_{k}^{*}, \tag{11}\]

where \(\overline{e}=\frac{e}{\|e\|}\) and \(\overline{w}_{k}^{*}=\frac{w_{k}^{*}}{\|w_{k}^{*}\|}\) are \(l_{2}-\)normalized vectors, and \(\tau\) is a learnable scalar value. After the base classifier is trained, all the feature vectors that belong to the same class must be very closely matched with the single classification weight vector of that class. So the base classification weights \(W_{base}=\{w_{b}\}_{b=1}^{C_{base}}\) trained in the \(1\)st stage can be seen as the base classes' feature vectors.

In the few-shot classification scenario, we feed the query vector \(e_{q}\) and novel class vector \(e_{c}\) to the classifier and get the classification scores in a unified manner.

\[s_{q,c}=\tau\cdot cos(e_{q},e_{c})=\tau\cdot\overline{e}_{q}^{T}\overline{e}_ {c}. \tag{12}\]

### Objective Function

In the supervised learning stage, the training objective is to minimize the cross-entropy loss on \(C_{base}\) number of base classes given an input text \(x\) and its label \(\mathbf{y}\):

\[L_{1}(x,\mathbf{y},\mathbf{\hat{y}})=-\sum_{k=1}^{C_{base}}y_{k}log(\hat{y}_{k}), \tag{13}\]

where \(\mathbf{y}\) is one-hot representation of the ground truth label, and \(\mathbf{\hat{y}}\) is the predicted probabilities of base classes with \(\hat{y}_{k}=softmax(s_{k})\).

In the meta-training stage, for each meta episode, given the support set \(S\) and query set \(Q=\{x_{q},y_{q}\}_{q=1}^{L}\), the training objective is to minimize the cross-entropy loss on \(C\) novel classes.

\[L_{2}(S,Q)=-\frac{1}{C}\sum_{c=1}^{C}\frac{1}{L}\sum_{q=1}^{L}y_{q}log(\hat{ y}_{q}), \tag{14}\]

where \(\hat{y}_{q}=softmax(s_{q})\) is the predicted probabilities of \(C\) novel classes in this meta episode, with \(s_{q}=\{s_{q,c}\}_{c=1}^{C}\) from Equation 12. We feed the support set \(S\) to the model and update its parameters to minimize the loss in the query set \(Q\) in each meta episode.

## 4 Experiments

### Dataset and Evaluation Metrics

We evaluate our model on the miniRCV1 Jiang et al. (2018) and ODIC dataset Geng et al. (2019). Following previous work Snell et al. (2017); Geng et al. (2019), we use few-shot classification accuracy as the evaluation metric. We average over \(100\) and \(300\) randomly generated meta-episodes from the testing set in miniRCV1 and ODIC, respectively. We sample 10 test texts per class in each episode for evaluation in both the \(1\)-shot and \(5\)-shot scenarios.

### Implementation Details

We use Google pre-trained BERT-Base model as our text encoder, and fine-tune the model in the training procedure. The number of base classes \(C_{base}\) on ODIC and miniRCV1 is set to be 100 and 20, respectively. The number of DMR interaction is 3. We build episode-based meta-training models with \(C=[5,10]\) and \(K=[1,5]\) for comparison. In addition to using K sample texts as the support set, the query set has 10 query texts for each of the C sampled classes in every training episode. For example, there are \(10\times 5+5\times 5=75\) texts in one training episode for a 5-way 5-shot experiment.

### Results

We compare DMIN with various baselines and state-of-the-art models: BERT Devlin et al. (2019) finetune, ATAML Jiang et al. (2018), Rel. Net Sung et al. (2018), Ind. Net Geng et al. (2019), HATT Gao et al. (2019), and LwoF Gidaris and Komodakis (2018). Note that we re-implement them with the BERT sentence encoder for direct comparison.

Overall PerformanceThe accuracy and standard deviations of the models are shown in Table 1 and 2. We can see that DMIN consistently outperform all existing models and achieve new state-of-the-art results on both datasets. The differences between DMIN and all the other models are statistically significant under the one-tailed paired t-test at the 95% significance level

Note that LwoF builds a two-stage training procedure with a memory module learnt from the supervised learning and used in the meta-learning stage, but the memory mechanism is static after training, while DMIN uses dynamic memory routing to automatically adjust the coupling coefficients after training to generalize to novel classes, and outperform LwoF significantly. Note also that the performance of some of the baseline models (Rel. Net and Ind. Net) reported in Table 1 and 2 is higher than that in Geng et al. (2019) since we used BERT to replace BiLSTM-based encoders. The BERT encoder improves the baseline models by a powerful context meaning representation ability, and our model can further outperform these models with a dynamic memory routing method. Even with these stronger baselines, the proposed DMIN consistently outperforms them on both dataset.

Ablation StudyWe analyze the effect of different components of DMIN on ODIC in Table 3. Specifically, we remove DMM and QIM, and vary the number of DMR iterations. We see that the best performance is achieved with \(3\) iterations. The results show the effectiveness of both the dynamic memory module and the induction module with query information.

### Further Analysis

Figure 2 is the t-SNE visualization Maaten and Hinton (2008) for support sample vectors before and after DMM under a \(10\)-way \(5\)-shot setup on ODIC. We randomly select a support set with 50 texts (10 texts per class) from the ODIC testing set, and obtain the sample vectors before and after DMM, i.e., \(\left\{e_{c,s}\right\}_{c=1,...5,s=1...10}\) and \(\left\{e^{\prime}_{c,s}\right\}_{c=1,..5,s=1...10}\). We can see that the support vectors produced by the DMM are better separated, demonstrating the effectiveness of DMM in leveraging the supervised learning experience to encode semantic relationships between lower level instance features and higher level class features for few-shot text classification.

## 5 Conclusion

We propose Dynamic Memory Induction Networks (DMIN) for few-shot text classification, which builds on external working memory with dynamic routing, leveraging the latter to track previous learning experience and the former to adapt and generalize better to support sets and hence to unseen classes. The model achieves new state-of-the-art results on the miniRCV1 and ODIC datasets. Since dynamic memory can be a learning mechanism more general than what we have used here for few-shot learning, we will investigate this type of models in other learning problems.

#### Acknowledgments

The authors would like to thank the organizers of ACL-2020 and the reviewers for their helpful suggestions. The research of the last author is supported by the Natural Sciences and Engineering Research Council of Canada (NSERC).

\begin{table}
\begin{tabular}{l c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**5-way Acc.**} & \multicolumn{2}{c}{**10-way Acc.**} \\  & 1-shot & 5-shot & 1-shot & 5-shot \\ \hline BERT & 30.79\(\pm\)0.68 & 63.31\(\pm\)0.73 & 23.48\(\pm\)0.53 & 61.18\(\pm\)0.82 \\ ATAML & 45.05\(\pm\)0.14 & 27.79\(\pm\)0.27 & 39.48\(\pm\)0.23 & 61.74\(\pm\)0.36 \\ Rel. Net & 59.19\(\pm\)0.12 & 78.35\(\pm\)0.27 & 44.69\(\pm\)0.19 & 67.49\(\pm\)0.23 \\ Ind. Net & 60.97\(\pm\)0.16 & 80.91\(\pm\)0.19 & 46.15\(\pm\)0.26 & 69.42\(\pm\)0.34 \\ HATT & 60.40\(\pm\)0.17 & 79.46\(\pm\)0.32 & 47.09\(\pm\)0.28 & 68.58\(\pm\)0.37 \\ LowF & 63.35\(\pm\)0.26 & 78.83\(\pm\)0.38 & 48.61\(\pm\)0.21 & 69.57\(\pm\)0.35 \\ \hline DMIN & **65.72\(\pm\)**0.28 & **82.39\(\pm\)**0.24 & **49.54\(\pm\)**0.31 & **72.52\(\pm\)**0.25 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Comparison of accuracy (%) on miniRCV1 with standard deviations.

\begin{table}
\begin{tabular}{l c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**5-way Acc.**} & \multicolumn{2}{c}{**10-way Acc.**} \\  & 1-shot & 5-shot & 1-shot & 5-shot \\ \hline BERT & 38.06\(\pm\)0.27 & 64.24\(\pm\)0.36 & 29.24\(\pm\)0.19 & 64.53\(\pm\)0.35 \\ ATAML & 79.60\(\pm\)0.42 & 88.53\(\pm\)0.57 & 63.52\(\pm\)0.34 & 77.36\(\pm\)0.57 \\ Rel. Net & 79.41\(\pm\)0.42 & 87.93\(\pm\)0.31 & 64.36\(\pm\)0.58 & 78.62\(\pm\)0.54 \\ Ind. Net & 81.28\(\pm\)0.26 & 89.67\(\pm\)0.28 & 64.53\(\pm\)0.38 & 80.48\(\pm\)0.25 \\ HATT & 81.57\(\pm\)0.47 & 82.97\(\pm\)0.58 & 65.75\(\pm\)0.61 & 81.53\(\pm\)0.56 \\ LauFoF & 79.52\(\pm\)0.29 & 87.34\(\pm\)0.34 & 65.04\(\pm\)0.43 & 60.69\(\pm\)0.37 \\ \hline DMIN & **83.46\(\pm\)**0.36 & **91.75\(\pm\)**0.23 & **67.31\(\pm\)**0.25 & **82.84\(\pm\)**0.38 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Comparison of accuracy (%) on ODIC with standard deviations.

\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Model** & **Iteration** & **1 Shot** & **5 Shot** \\ \hline w/o DMM & 3 & 81.79 & 90.19 \\ w/o QMM & 3 & 82.37 & 90.57 \\ DMIN & 1 & 82.70 & 90.92 \\ DMN & 2 & 82.95 & 91.18 \\ DMIN & 3 & **83.46** & **91.75** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Ablation study of accuracy (%) on ODIC in a 5-way setup.

Figure 2: Effect of the Dynamic Memory Module in a 10-way 5-shot setup.

 



## References

* Allen et al. (2019) Kelsey Allen, Evan Shelhamer, Hanul Shin, and Joshua Tenenbaum. 2019. Infinite mixture prototypes for few-shot learning. In _Proceedings of the 36th International Conference on Machine Learning_, volume 97 of _Proceedings of Machine Learning Research_, pages 232-241, Long Beach, California, USA. PMLR.
* Ba et al. (2016) Jimmy Ba, Geoffrey E Hinton, Volodymyr Mnih, Joel Z Leibo, and Catalin Ionescu. 2016. Using fast weights to attend to the recent past. In _Advances in Neural Information Processing Systems_, pages 4331-4339.
* Bao et al. (2019) Yujia Bao, Menghua Wu, Shiyu Chang, and Regina Barzilay. 2019. Few-shot text classification with distributional signatures. _arXiv preprint arXiv:1908.06039_.
* Cai et al. (2018) Qi Cai, Yingwei Pan, Ting Yao, Chenggang Yan, and Tao Mei. 2018. Memory matching networks for one-shot image recognition. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_, pages 4080-4088.
* Chen et al. (2019) Mingyang Chen, Wen Zhang, Wei Zhang, Qiang Chen, and Huajun Chen. 2019. Meta relational learning for few-shot link prediction in knowledge graphs. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 4208-4217, Hong Kong, China. Association for Computational Linguistics.
* Das et al. (2017) Rajarshi Das, Manzil Zaheer, Siva Reddy, and Andrew McCallum. 2017. Question answering on knowledge bases and text using universal schema and memory networks. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 358-365, Vancouver, Canada. Association for Computational Linguistics.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 4171-4186.
* Dou et al. (2019) Zi-Yi Dou, Keyi Yu, and Antonios Anastasopoulos. 2019. Investigating meta-learning algorithms for low-resource natural language understanding tasks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 1192-1197.
* Fei et al. (2003) Li Fei-Fei et al. 2003. A bayesian approach to unsupervised one-shot learning of object categories. In _Computer Vision, 2003. Proceedings. Ninth IEEE International Conference on_, pages 1134-1141. IEEE.
* Fei-Fei et al. (2006) Li Fei-Fei, Rob Fergus, and Pietro Perona. 2006. One-shot learning of object categories. _IEEE transactions on pattern analysis and machine intelligence_, 28(4):594-611.
* Finn et al. (2017) Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In _Proceedings of the 34th International Conference on Machine Learning-Volume 70_, pages 1126-1135. JMLR. org.
* Gao et al. (2019) Tianyu Gao, Xu Han, Zhiyuan Liu, and Maosong Sun. 2019. Hybrid attention-based prototypical networks for noisy few-shot relation classification. In _Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence,(AAAI-19), New York, USA_.
* Geng et al. (2019) Ruying Geng, Binhua Li, Yongbin Li, Xiaodan Zhu, Ping Jian, and Jian Sun. 2019. Induction networks for few-shot text classification. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 3895-3904, Hong Kong, China. Association for Computational Linguistics.
* Gidaris and Komodakis (2018) Spryros Gidaris and Nikos Komodakis. 2018. Dynamic few-shot visual learning without forgetting. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_, pages 4367-4375.
* Gu et al. (2018) Jiatao Gu, Yong Wang, Yun Chen, Victor OK Li, and Kyunghyun Cho. 2018. Meta-learning for low-resource neural machine translation. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 3622-3631.
* Hu et al. (2019) Ziniu Hu, Ting Chen, Kai-Wei Chang, and Yizhou Sun. 2019. Few-shot representation learning for out-of-vocabulary words. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 4102-4112, Florence, Italy. Association for Computational Linguistics.
* Hunt (1986) Ronald J Hunt. 1986. Percent agreement, pearson's correlation, and kappa as measures of inter-examiner reliability. _Journal of Dental Research_, 65(2):128-130.
* Jiang et al. (2018) Xiang Jiang, Mohammad Havaei, Gabriel Chartrand, Hassan Chouaib, Thomas Vincent, Andrew Jesson, Nicolas Chapados, and Stan Matwin. 2018. Attentive task-agnostic meta-learning for few-shot text classification.
* Kaiser et al. (2017) Lukasz Kaiser, Ofir Nachum, Aurko Roy, and Samy Bengio. 2017. Learning to remember rare events. _arXiv preprint arXiv:1703.03129_.
* van der Maaten and Hinton (2008) Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. _Journal of machine learning research_, 9(Nov):2579-2605

Andrea Madotto, Chien-Sheng Wu, and Pascale Fung. 2018. Mem2seq: Effectively incorporating knowledge bases into end-to-end task-oriented dialog systems. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1468-1478.
* Mishra et al. (2017) Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. 2017. A simple neural attentive meta-learner. _arXiv preprint arXiv:1707.03141_.
* Munkhdalai and Yu (2017) Tsendsuren Munkhdalai and Hong Yu. 2017. Meta networks. _arXiv preprint arXiv:1703.00837_.
* Obamujide and Vlachos (2019)Abiola Obamujide and Andreas Vlachos. 2019. Model-agnostic meta-learning for relation classification with limited supervision. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 5873-5879, Florence, Italy. Association for Computational Linguistics.
* Peters et al. (2018) Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, pages 2227-2237, New Orleans, Louisiana. Association for Computational Linguistics.
* Qi et al. (2018) Hang Qi, Matthew Brown, and David G Lowe. 2018. Low-shot learning with imprinted weights. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_, pages 5822-5830.
* Radford et al. (2016) Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training.
* Ravi and Larochelle (2016) Sachin Ravi and Hugo Larochelle. 2016. Optimization as a model for few-shot learning.
* Rios and Kavuluru (2018) Anthony Rios and Ramakanth Kavuluru. 2018. Few-shot and zero-shot multi-label learning for structured label spaces. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 3132-3142.
* Sabour et al. (2017) Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. 2017. Dynamic routing between capsules. In _Advances in Neural Information Processing Systems_, pages 3856-3866.
* Salamon and Bello (2017) Justin Salamon and Juan Pablo Bello. 2017. Deep convolutional neural networks and data augmentation for environmental sound classification. _IEEE Signal Processing Letters_, 24(3):279-283.
* Santoro et al. (2016) Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. 2016. Metax-learning with memory-augmented neural networks. In _International conference on machine learning_, pages 1842-1850.
* Snell et al. (2017) Jake Snell, Kevin Swersky, and Richard Zemel. 2017. Prototypical networks for few-shot learning. In _Advances in Neural Information Processing Systems_, pages 4077-4087.
* Soares et al. (2019) Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, and Tom Kwiatkowski. 2019. Matching the blanks: Distributional similarity for relation learning. _arXiv preprint arXiv:1906.03158_.
* Sun et al. (2019) Qianru Sun, Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele. 2019. Meta-transfer learning for few-shot learning. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_, pages 403-412.
* Sung et al. (2018) Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. 2018. Learning to compare: Relation network for few-shot learning. In _In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_, pages 1199-1208.
* Tang et al. (2016) Duyu Tang, Bing Qin, and Ting Liu. 2016. Aspect level sentiment classification with deep memory network. In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_, pages 214-224.
* Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In _Advances in Neural Information Processing Systems_, pages 5998-6008.
* Vinyals et al. (2016) Oriol Vinyals, Charles Blundell, Tim Lillicrap, Daan Wierstra, et al. 2016. Matching networks for one shot learning. In _Advances in Neural Information Processing Systems_, pages 3630-3638.
* Wu et al. (2019) Jiawei Wu, Wenhan Xiong, and William Yang Wang. 2019. Learning to learn and predict: A meta-learning approach for multi-label classification. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 4345-4355.
* Xu et al. (2019) Hu Xu, Bing Liu, Lei Shu, and P Yu. 2019. Openworld learning and application to product classification. In _The World Wide Web Conference_, pages 3413-3419. ACM.
* Yang et al. (2019) Zhengxin Yang, Jinchao Zhang, Fandong Meng, Shuhao Gu, Yang Feng, and Jie Zhou. 2019. Enhancing context modeling with a query-guided capsule network for document-level translation. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 1527-1537.
* Ye and Ling (2019) Zhi-Xiu Ye and Zhen-Hua Ling. 2019. Multi-level matching and aggregation network for few-shot relation classification. In _Proceedings of the 57th Annual Meeting of the Association for Computational 

Linguistics_, pages 2872-2881, Florence, Italy. Association for Computational Linguistics.
* Yu et al. (2018) Mo Yu, Xiaoxiao Guo, Jinfeng Yi, Shiyu Chang, Saloni Potdar, Yu Cheng, Gerald Tesauro, Haoyu Wang, and Bowen Zhou. 2018. Diverse few-shot text classification with multiple metrics. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, pages 1206-1215.
* Zhang et al. (2018) Ruixiang Zhang, Tong Che, Zoubin Ghahramani, Yoshua Bengio, and Yangqiu Song. 2018. Metagan: An adversarial approach to few-shot learning. In _Advances in Neural Information Processing Systems_, pages 2365-2374.