Venue: NAACL 2022

# Meta Learning for Natural Language Processing: A Survey

 Hung-yi Lee

National Taiwan University

hungyilee@ntu.edu.tw &Shang-Wen Li

Amazon AI\({}^{*}\)shangwel@amazon.com &Ngoc Thang Vu

University of Stuttgart

thangvu@ims.uni-stuttgart.de

Work done while working at Amazon Inc. The current affiliation is Meta AI.

###### Abstract

Deep learning has been the mainstream technique in natural language processing (NLP) area. However, the techniques require many labeled data and are less generalizable across domains. Meta-learning is an arising field in machine learning studying approaches to learn better learning algorithms. Approaches aim at improving algorithms in various aspects, including data efficiency and generalizability. Efficacy of approaches has been shown in many NLP tasks, but there is no systematic survey of these approaches in NLP, which hinders more researchers from joining the field. Our goal with this survey paper is to offer researchers pointers to relevant meta-learning works in NLP and attract more attention from the NLP community to drive future innovation. This paper first introduces the general concepts of meta-learning and the common approaches. Then we summarize task construction settings and application of meta-learning for various NLP problems and review the development of meta-learning in NLP community.

## 1 Introduction

Recently, deep learning (DL) based natural language processing (NLP) has been one of the research mainstreams and yields significant performance improvement in many NLP problems. However, DL models are data-hungry. The downside limits such models' application to different domains, languages, countries, or styles because collecting in-genre data for model training are costly.

To address the challenges, meta-learning techniques are gaining attention. Meta-learning, or Learning to Learn, aims to learn better learning algorithms, including better parameter initialization Finn et al. (2017), optimization strategy Andrychowicz et al. (2016); Ravi and Larochelle (2017), network architecture Zoph and Le (2017); Zoph et al. (2018); Pham et al. (2018a), distance metrics Vinyals et al. (2016); Gao et al. (2019); Sung et al. (2018), and beyond Mishra et al. (2018). Meta-learning allows faster fine-tuning, converges to better performance, yields more generalizable models, and it achieves outstanding results for few-shot image classification Triantafillou et al. (2020). The benefits alleviate the dependency of learning algorithms on labels and make model development more scalable. Image processing is one of the machine learning areas with abundant applications and established most of the examples in the previous survey papers on meta-learning Hospedales et al. (2021); Huisman et al. (2021).

On the other hand, there are works showing benefits of meta-learning techniques in performance and data efficiency via applying meta-learning to NLP problems. Please refer to Tables 2 and 3 in the appendix for NLP applications improved by meta-learning. Tutorial Lee et al. (2021b) and Workshop Lee et al. (2021a) are organized at ACL 2021 to encourage exchange and collaboration among NLP researchers interested in these techniques. To facilitate more NLP researchers and practitioners benefiting from the advance of meta-learning and participating in the area, we provide a systematic survey of meta-learning to NLP problems in this paper. There is another survey paper on meta-learning in NLP Yin (2020). While Yin (2020) describes meta-learning methods in general, this paper focuses on the idea of making meta-learning successful when applied to NLP and provides a broader review of publications on NLP meta-learning. This paper is organized as below.

* A brief introduction of meta-learning backgrounds, general concepts, and algorithms in Section 2.
* Common settings for constructing meta-learning tasks in Section 3.



* Adaptation of general meta-learning approaches to NLP problems in Section 4.
* Meta-learning approaches for special topics, including knowledge distillation and life-long learning for NLP applications in Section 5.

Due to space constraints, we will not give too many detailed descriptions of general meta-learning techniques in this survey paper. For general concepts of meta-learning, we encourage readers to read the previous overview paper Yin et al. (2020); Hospedales et al. (2021); Huisman et al. (2021).

## 2 Background Knowledge for Meta Learning

The goal of machine learning (ML) is to find a function \(f_{\theta}(x)\) parametrized by model parameters \(\theta\) for inference from training data. For machine translation (MT), the input \(x\) is a sentence, while \(f_{\theta}(x)\) is the translation of \(x\); for automatic speech recognitoin (ASR), \(x\) is an utterance, while \(f_{\theta}(x)\) is the transcription; In DL, \(\theta\) are the network parameters, or weights and biases of a network. To learn \(\theta\), there is a loss function \(l(\theta;\mathcal{D})\), where \(\mathcal{D}\) is a set of paired examples for training,

\[\mathcal{D}=\{(x_{1},y_{1}),(x_{2},y_{2}),...,(x_{K},y_{K})\}, \tag{1}\]

where \(x_{k}\) is function input, \(y_{k}\) is the ground truth, and \(K\) is the number of examples in \(\mathcal{D}\). The loss function \(l(\theta;\mathcal{D})\) is defined as below:

\[l(\theta;\mathcal{D})=\sum_{k=1}^{K}d(f_{\theta}(x_{k}),y_{k}). \tag{2}\]

where \(d(f_{\theta}(x_{k}),y_{k})\) is the "distance" between the function output \(f_{\theta}(x_{k})\) and the ground truth \(y_{k}\). For classification problem, \(d(.,.)\) can be cross-entropy; for regression, it can be L1/L2 distance. The following optimization problem is solved to find the optimal parameter set \(\theta^{*}\) for inference via minimizing the loss function \(l(\theta;\mathcal{D})\).

\[\theta^{*}=\arg\min_{\theta}l(\theta;\mathcal{D}). \tag{3}\]

In meta-learning, what we want to learn is a learning algorithm. The learning algorithm can also be considered as a function, denoted as \(F_{\phi}(.)\). The input of \(F_{\phi}(.)\) is the training data, while the output of the function \(F_{\phi}(.)\) is the learned model parameters, or \(\theta^{*}\) in (3). The learning algorithm \(F_{\phi}(.)\) is parameterized by _meta-parameters_\(\phi\), which is what we want to learn in meta-learning. If \(F_{\phi}(.)\) represents gradient descent for deep network, \(\phi\) can be initial parameters, learning rate, network architecture, etc. Different meta-learning approaches focus on learning different components. For example, model-agnostic meta-learning (MAML) focuses on learning initial parameters Finn et al. (2017), which will be further descried in Section 4.1. Learning to Compare methods like Prototypical Network Snell et al. (2017) in Section 4.2 learn the latent representation of the inputs and their distance metrics for comparison. Network architecture search (NAS) in Section 4.3 learns the network architecture Zoph and Le (2017); Zoph et al. (2018); Pham et al. (2018).

To learn meta-parameters \(\phi\), _meta-training tasks_\(\mathcal{T}_{train}\) are required.

\[\mathcal{T}_{train}=\{\mathcal{T}_{1},\mathcal{T}_{2},...,\mathcal{T}_{N}\}, \tag{4}\]

where \(\mathcal{T}_{n}\) is a task, and \(N\) is the number of tasks in \(\mathcal{T}_{train}\). Usually, all the tasks belong to the same NLP problem; for example, all the \(\mathcal{T}_{n}\) are QA but from different corpora, but it is also possible that the tasks belong to various problems. Each task \(\mathcal{T}_{n}\) includes a _support set_\(\mathcal{S}_{n}\) and a _query set_\(\mathcal{Q}_{n}\). Both \(\mathcal{S}_{n}\) and \(\mathcal{Q}_{n}\) are paired examples as \(\mathcal{D}\) in (1). The support set plays the role of training data in typical ML, while the query set can be understood as the testing data in typical ML. However, to not confuse the reader, we use the terms support and query sets in the context of meta-learning instead of training and testing sets.

In meta-learning, there is a loss function \(L(\phi;\mathcal{T}_{train})\), which represents how "bad" a learning algorihtm paramereized by \(\phi\) is on \(\mathcal{T}_{train}\). \(L(\phi;\mathcal{T}_{train})\) is the performance over all the tasks in \(\mathcal{T}_{train}\),

\[L(\phi;\mathcal{T}_{train})=\sum_{n=1}^{N}l(\theta^{n};\mathcal{Q}_{n}). \tag{5}\]

The definition of the function \(l(.)\) above is the same as in (2). \(l(\theta^{n};\mathcal{Q}_{n})\) for each task \(\mathcal{T}_{n}\) is obtained as below. For each task \(\mathcal{T}_{n}\) in \(\mathcal{T}_{train}\), we use a support set \(\mathcal{S}_{n}\) to learn a model by the learning algorihtm \(F_{\phi}\). The learned model is denoted as \(\theta^{n}\), where \(\theta^{n}=F_{\phi}(\mathcal{S}_{n})\). This procedure is equivalent to typical ML training. We called this step _within-task training_. Then \(\theta^{n}\) is evaluated on \(\mathcal{Q}_{n}\) to obtain \(l(\theta^{n};\mathcal{Q}_{n})\) in (5). We called this step _within-task testing_. One execution of within-task training and 

followed by one execution of within-task testing is called an _episode_.

The optimization task below is solved to learn meta-parameters \(\phi\).

\[\phi^{*}=\arg\min_{\phi}L(\phi;\mathcal{T}_{train}). \tag{6}\]

If \(\phi\) is differentiable with respect to \(L(\phi;\mathcal{T}_{train})\), then we can use gradient descent to learn meta-parameters; if not, we can use reinforcement learning algorithm or evolutionary algorithm. Solving (6) is called _cross-task training_ in this paper, which usually involves running many episodes on meta-training tasks. To evaluate \(\phi^{*}\), we need _meta-testing tasks_\(\mathcal{T}_{test}\), tasks for evaluating algorithms parameterized by meta-parameters \(\phi^{*1}\). We do _cross-task testing_ on \(\mathcal{T}_{test}\), that is, running an episode on each meta-testing task to evaluate algorithms parameterized by meta-parameters \(\phi^{*}\).

In order to facilitate the reading of our paper, we summarize the most important terminologies and their meanings in Table 1 in the appendix.

## 3 Task Construction

In this section, we discuss different settings of constructing meta-training tasks \(\mathcal{T}_{train}\) and meta-testing tasks \(\mathcal{T}_{test}\).

### Cross-domain Transfer

A typical setting for constructing the tasks is based on domains Qian and Yu (2019); Yan et al. (2020); Li et al. (2020); Park et al. (2021); Chen et al. (2020); Huang et al. (2020); Dai et al. (2020); Wang et al. (2021); Dingliwal et al. (2021); Qian et al. (2021). In this setting, all the tasks, no matter belonging to \(\mathcal{T}_{train}\) or \(\mathcal{T}_{test}\), are the same NLP problems. In each task \(\mathcal{T}_{n}\), the support set \(\mathcal{S}_{n}\) and the query set \(\mathcal{Q}_{n}\) are from the same domain, while different tasks contain the examples from different domains. In each task, the model is trained on the support set of a domain (usually having a small size) and evaluated on the query set in the same domain, which can be considered as _domain adaptation_. From the meta-training tasks \(\mathcal{T}_{train}\), cross-task training finds meta-parameters \(\phi^{*}\) parameterizing the learning algorithm \(F_{\phi^{*}}\). With a sufficient number of tasks in \(\mathcal{T}_{train}\), cross-task training should find a suitable \(\phi^{*}\) for a wide range of domains, and thus also works well on the tasks in \(\mathcal{T}_{test}\) containing the domains unseen during cross-task training. Hence, meta-learning can be considered as one way to improve _domain adaptation_. If the support set in each task includes only a few examples, the meta-learning has to find the meta-parameters \(\phi^{*}\) that can learn from a small support set and generalize well to the query set in the same domain. Therefore, meta-learning is considered one way to achieve _few-shot learning_.

The cross-domain setting is widespread. We only provide a few examples in this subsection. In MT, each meta-training task includes the documents from a specific domain (e.g., news, laws, etc.), while each meta-testing task also contains documents from one domain but not covered by the meta-training tasks (e.g., medical records) Li et al. (2020). For another example, both meta-training and meta-testing tasks are DST. The meta-training tasks include hotel booking, flight ticket booking, etc., while the testing task is taxi booking Huang et al. (2020); Wang et al. (2021); Dingliwal et al. (2021). Domain has different meanings in different NLP problems. For example, in speech processing tasks, the domains can refer to accents Winata et al. (2020); Huang et al. (2021) or speakers Klejch et al. (2019); Wu et al. (2021); Huang et al. (2022).

### Cross-lingual Transfer

If we consider different languages as different domains, then the cross-lingual transfer can be regarded as a special case of cross-domain transfer. Suppose each task contains the examples of an NLP problem from one language, and different tasks are in different languages. In this case, cross-task training finds meta-parameters \(\phi^{*}\) from the languages in \(\mathcal{T}_{train}\), and cross-task testing evaluate the meta-parameters \(\phi^{*}\) on new langauges in \(\mathcal{T}_{test}\). This setting aims at finding the learning algorithm \(F_{\phi^{*}}(.)\) that works well on the NLP problem of any language given the support set of the language. Cross-language settings have been applied to NLI and QA in X-MAML Nooralahzadeh et al. (2020), documentation classification van der Heijden et al. (2021), dependency parsing Langedijk et al. (2021), MT Gu et al. (2018), and ASR Hsu et al. (2020); Winata et al. (2020); Chen et al. (2020); Xiao et al. (2021).

For the meta-learning methods aiming at learning the initial parameters like MAML (will be intro 

duced in Section 4.1), the network architecture used in all tasks must have the same network architecture. A unified network architecture across all tasks is not obvious in cross-lingual learning because the vocabularies in different tasks are different. Before multilingual pretrained models are available, unified word embeddings across languages are required. Gu et al. (2018) uses the universal lexical representation to overcome the input-output mismatch across different languages. Recently, by using multilingual pretrained models as encoders, such as M-BERT Devlin et al. (2019) or XLM-R Conneau et al. (2020), all languages can share the same network architecture Nooralahzadeh et al. (2020); van der Heijden et al. (2021).

### Cross-problem Training

Here the meta-training and meta-testing tasks can come from different problems. For example, the meta-training tasks include MT and NLI, while the meta-testing tasks include QA and DST. The cross-problem setting is not usual, but there are still some examples. In Bansal et al. (2020a), the meta-training tasks are the GLUE benchmark tasks Wang et al. (2018), while the meta-testing tasks are NLP problems, including entity typing, NLI, sentiment classification, and various other text classification tasks, not in the GLUE. All the meta-training and meta-testing tasks can be formulated as classification but with different classes. In Indurthi et al. (2020), the meta-training tasks are MT and ASR, while the meta-testing task is speech translation (ST). CrossFit is a benchmark corpus for this cross-problem setting Ye et al. (2021).

The intrinsic challenge in the cross-problem setting is that different NLP problems may need very different meta-parameters in learning algorithms, so it may be challenging to find unified meta-parameters on the meta-training tasks that can generalize to meta-testing tasks. In addition, the meta-learning algorithms learning initial parameters such as MAML require all the tasks to have a unified network architecture. If different problems need different network architecture, then the original MAML cannot be used in the cross-problem setting. LEOPARD Bansal et al. (2020a) and ProtoMAML van der Heijden et al. (2021) are the MAML variants that can be used in the classification tasks with different class numbers. Both approaches use the data of a class to generate the class-specific head, so only the parameters of the head parameter generation model are required. The head parameter generation model is shared across all classes, so the network architecture becomes class-number agnostic. On the other hand, recently, universal models for a wide range of NLP problems have been emgered Raffel et al. (2019); Chen et al. (2021); Ao et al. (2021). We believe the development of the universal models will intrigue the cross-problem setting in meta-learning.

### Domain Generalization

Traditional supervised learning assumes that the training and testing data have the same distribution. Domain shift refers to the problem that a model performs poorly when training data and testing data have very different statistics. Domain adaptation in Section 3.1 uses little domain-specific data to adapt the model2. On the other hand, _domain generalization_ techniques attempt to alleviate the domain mismatch issue by producing models that generalize well to novel testing domains.

Footnote 2: The domain-specific data are usually labelled, but unlabelled domain-specific data can be leveraged as well Kouw and Loog (2021), which is out of scope here.

Meta-learning can also be used to realize domain generalization by learning an algorithm that can train from one domain but evaluate on the other. To simulate the domain generalization scenario, a set of meta-training tasks are constructed by sampling data from different domains as the support and query sets. With the meta-training tasks above, cross-task training will find the meta-parameters \(\phi^{*}\) that work well on the scenario where the training (support) and testing (query) examples are from different domains. Fig. 1 shows how to construct tasks for domain generalization and compares the construction with the cross-domain transfer setting. The setting has been used to improve the domain generalization for semantic parsing Wang et al. (2017),

Figure 1: The task construction of cross-domain tranfer in Section 3.1 and domain generalization in Section 3.4. Different colors represents data from different domains.

 

2021) and language generalization3 for sentiment classification and relevance classification Li et al. (2020).

Footnote 3: if a language is considered as a domain

### Task Augmentation

In meta-learning, it is critical to have a large number of diverse tasks in the meta-training tasks \({\mathcal{T}_{train}}\) to find a set of meta-parameters \(\phi^{*}\) that can generalize well to the meta-testing tasks. However, considering the setting in the previous subsections, different tasks contain examples in various domains, language, or even NLP problems, so a large and diverse \({\mathcal{T}_{train}}\) are often not available. In typical ML, data augmentation comes in handy when data is lacking. In meta-learning, augmenting tasks is similarly understood as data augmentation in ML. Data augmentation becomes task augmentation because the "training examples" in meta-learning are a collection of tasks. Task augmentation approaches in meta-learning can be categorized into two main directions: a) Inventing more tasks (without human labeling efforts) to increase the number and diversity of the meta-training tasks \({\mathcal{T}_{train}}\). b) Splitting training data from one single dataset into homogenous partitions that allow applying meta-learning techniques and therefore improve the performance. NLP-specific methods have been proposed in both categories.

Inventing more tasksThe main question is how to construct a massive amount of tasks efficiently. There is already some general task augmentation approahces proposed for general meta-learning Yao et al. (2021); Ni et al. (2021); Rajendran et al. (2020); Yao et al. (2021). Here we only focus on NLP-specific approaches. Inspired from the self-supervised learning, Bansal et al. (2020) generates a large number of cloze tasks, which can be considered as multi-class classification tasks but obtained without labeling effort, to augment the meta-training tasks. Bansal et al. (2021) further explores the influence of unsupervised task distribution and creates task distributions that are inductive to better meta-training efficacy. The self-supervised generated tasks improve the performance on a wide range of different meta-testing tasks which are classification problems Bansal et al. (2020), and it even performs comparably with supervised meta-learning methods on FewRel 2.0 benchmark Gao et al. (2019) on 5-shot evaluation Bansal et al. (2021).

Generating tasks from a monolithic corpusMany tasks can be constructed with one monolithic corpus Huang et al. (2018); Guo et al. (2019); Wu et al. (2019); Jiang et al. (2019); Chien and Lieow (2019); Li et al. (2020); MacLaughlin et al. (2020); Wang et al. (2020); Pasunuru and Bansal (2020); Xu et al. (2021); Murty et al. (2021)). First, the training set of the corpus is split into support partition, \({\mathcal{D}_{s}}\), and query partition, \({\mathcal{D}_{q}}\). Two subsets of examples are sampled from \({\mathcal{D}_{s}}\) and \({\mathcal{D}_{q}}\) as the support set, \({\mathcal{S}}\), and query set, \({\mathcal{Q}}\), respectively. In each episode, model parameters \(\theta\) are updated with \({\mathcal{S}}\), and then the losses are computed with the updated model and \({\mathcal{Q}}\). The meta-parameters \(\phi\) are then updated based on the losses, as the meta-learning framework introduced in Section 2. The test set of the corpus is used to build \({\mathcal{T}_{test}}\) for evaluation. As compared to constructing \({\mathcal{T}_{train}}\) from multiple relevant corpora, which are often not available, building \({\mathcal{T}_{train}}\) with one corpus makes meta-learning methodology more applicable. Besides, results obtained from one corpus are more comparable with existing NLP studies. However, only using a single data stream makes the resulting models less generalizable to various attributes such as domains and languages.

How to sample the data points to form a task4 is the key in such category. In NAS research in Section 4.3, the support and query sets are usually randomly sampled. Learning to Compare in Section 4.2 splits the data points of different classes in different tasks based on some predefined criteria. There are some NLP-specific ways to construct the tasks. In Huang et al. (2018), a relevance function is designed to sample the support set \({\mathcal{S}}\) based on its relevance to the query set \({\mathcal{Q}}\). In Guo et al. (2019), a retrieval model is used to retrieve the support set \({\mathcal{S}}\) from the whole dataset. DReCa Murty et al. (2021) applies clustering on BERT representations to create tasks.

Footnote 4: If a corpus includes data from different domains, and we sample the data in the same domain to create a task, then the setting here becomes cross-domain in Section 3.1.

## 4 Meta-Learning for NLP Tasks

This section shows the most popular meta-learning methods for NLP and how they fit into NLP tasks. Due to space limitations, only the major trends are mentioned. Please refer to Table 2 and 3 in the appendix for a complete survey.

 



### Learning to Initialize

In typical DL, gradient descent is widely used to solve (3). Gradient descent starts from a set of initial parameters \(\theta^{0}\), and then the parameters \(\theta\) are updated iteratively according to the directions of the gradient. There is a series of meta-learning approaches targeting at learning the initial parameters \(\theta^{0}\). In these learn-to-init approaches, the meta-parameters \(\phi\) to be learned are the initial parameters \(\theta^{0}\) for gradient descent, or \(\phi=\theta^{0}\). MAML Finn et al. (2017) and its first-order approximation, FOMAML Finn et al. (2017), Reptile Nichol et al. (2018), etc., are the representative approaches of learn-to-init. We surveyed a large number of papers using MAML-based approaches to NLP applications in the last three years and summarized them in Table 4 in the appendix.

Learning to Initialize v.s. Self-supervised LearningThe learn-to-init approaches aim at learning a set of good initial parameters. On the other hand, self-supervised approaches like BERT also have the same target. There is a natural question: are they complementary? Based on the survey in Table 4 in the appendix, it is common to use the self-supervised models to "initialize" the meta-parameters \(\phi\) in learn-to-init approaches. To find the optimal \(\phi^{*}\) in (5), gradient descent is used as well, and thus the "initial parameters for initial parameters", or \(\phi^{0}\) is required. A self-supervised model usually serves the role of \(\phi^{0}\), and the learn-to-init approaches further update \(\phi^{0}\) to find \(\phi^{*}\).

Learn-to-init and self-supervised learning are complementary. The self-supervised objectives are different from the objective of the target NLP problem, so there is a "learning gap". On the other hand, learn-to-init approaches learn to achieve good performance on the query sets of the meta-training tasks, so it directly optimizes the objective of the NLP problems. The benefit of self-supervised learning is that it does not require labeled data, while labeling is still needed to prepare the examples in meta-training tasks.

Learning to Initialize v.s. Multi-task LearningMulti-task learning is another way to initialize model parameters, which usually serves as the baseline of learn-to-init in the literature. In multi-task learning, all the labelled data from the meta-training tasks is put together to train a model. That is, all the support sets \(\mathcal{S}_{n}\) and query sets \(\mathcal{Q}_{n}\) in the meta-training tasks \(\mathcal{T}_{train}\) are put together as a training set \(\mathcal{D}\), and the loss (3) is optimized to find a parameter \(\theta^{*}\). Then \(\theta^{*}\) is used as initial parameters for the meta-testing tasks.

Both multi-task learning and meta-learning leverage the examples in the meta-training tasks, but with different training criteria. Learn-to-init finds the initial parameters suitable to be updated by updating the model on the support sets and then evaluating it on the query sets. In contrast, multi-task learning does not consider that the initial parameters would be further updated at all during training. Therefore, in terms of performance, learn-to-init is usually shown to be better than multi-task learning Dou et al. (2019); Chen et al. (2020b). On the other hand, in terms of training speed, meta-learning, which optimizes (5), is more computationally intensive than multi-task learning optimizing (3).

Three-stage InitializationSince learn-to-init, multi-task, self-supervised learning all have their pros and cons, they can be integrated to draw on the strong points of each other. A common way to integrate the three approaches is "three-stage initialization" as below. a) First, initialize a model by self-supervised learning, which leverages unlabeled data. Its objective is usually not directly related to the target NLP problem. b) Then, multi-task learning is used to fine-tune the self-supervised model. The objective of multi-task learning is the target NLP problem but does not consider the update procedure in gradient descent. c) Finally, learn-to-init, which finds the initial parameters suitable for update, is used to fine-tune the multi-task model.

Learn-to-init is chosen to be the last stage because its training objective is closest to the target of looking for good initial parameters, but it is the most computationally intensive method, and thus it is only used to change the model a little bit. The three-stage initialization has been tested in several works Nooralahzadeh et al. (2020); Wu et al. (2021); van der Heijden et al. (2021); Langedijk et al. (2021), but it does not always improve the performance Wu et al. (2021); van der Heijden et al. (2021).

ChallengesLearn-to-init is an essential paradigm for few-shot learning and usually achieves outstanding results in the few-shot learning benchmarks of image classification Tri

antafillou et al. (2020). However, it has fallen short of yielding state-of-the-art results on NLP few-shot learning benchmarks Ye et al. (2021); Chen et al. (2022); Bragg et al. (2021). For example, on the cross-task few-shot learning benchmark, CrossFit, simple multi-task learning outperforms existing learn-to-init in many cases Ye et al. (2021). One possible reason is meta-learning methods are susceptible to hyper-parameters and even random seeds Antoniou et al. (2019). Hence, it is difficult to obtain decent performance without exhaustively tuning hyperparameters. The research about developing more stable learn-to-init methods may lead to more practical real-world applications for the approaches. There is a study about stabilizing the cross-task training of learn-to-init methods by reducing the variance of gradients for NLP Wang et al. (2021b).

### Learning to Compare

Learning to Compare methods are widely applied to NLP tasks. Among many others, we find applications of Learning to Compare methods in text classification Yu et al. (2018); Tan et al. (2019); Geng et al. (2019); Sun et al. (2019); Geng et al. (2020), sequence labeling Hou et al. (2020); Oguz and Vu (2021), semantic relation classification Ye and Ling (2019); Chen et al. (2019a); Gao et al. (2019a); Ren et al. (2020), knowledge completion Xiong et al. (2018); Wang et al. (2019b); Zhang et al. (2020); Sheng et al. (2020) and speech recognition Lux and Vu (2021) tasks.

Most of the proposed methods are based on Matching Network Vinyals et al. (2016), Prototypical Network Snell et al. (2017) and Relation Network Sung et al. (2018), and extend these architectures in two aspects: a) how to embed text input in a vector space with/without context information, and b) how to compute the distance/similarity/relation between two inputs in this space. Since these questions have had deep roots in the computation linguistics research for many years Schutze (1992); Manning and Schutze (1999), Learning to Compare methods is one of the most important methods among other meta-learning methods in the context of NLP despite their simplicity. Notably, to date, such family of methods is mainly applied to classification tasks.

### Neural Network Architecture Search

Neural network architecture search (NAS) is another common meta-learning technique applied to NLP including language modeling WikiText-103 Merity et al. (2017), PTB Mikolov et al. (2010)), NER CoNLL-2003 Sang and De Meulder (2003)), TC GLUE Wang et al. (2019a)), and MT WMT'14 Bojar et al. (2014)). As discussed in Section 3.5, these techniques are often trained/evaluated with a single, matched dataset, which is different from other meta-learning approaches.

Moreover, in contrast to conventional NAS methods that focus on learning the topology in an individual recurrent or convolutional cell, NAS methods have to be redesigned in order to make the search space suitable for NLP problems, where contextual information often plays an important role. Jiang et al. (2019) pioneers the application of NAS to NLP tasks beyond language modeling (NER in this case), and improves differentiable NAS by redesigning its search space for natural language processing. Li et al. (2020) extends the search space of NAS to cover more RNN architectures and allow the exploring of intra- and inter-token connection to increase the expressibility of searched networks. As the popularity of pre-trained language models (PLM) grows in NLP area, researchers also apply NAS to discover better topology for PLM such as BERT. Wang et al. (2020) introduces Hardware-Aware Transformers (HAT) to search Transformer architecture optimized for inference speed and memory footprint in different hardware platforms. NAS-BERT Xu et al. (2021) and AdaBERT Chen et al. (2020) explores task-agnostic and task-dependent network compression techniques with NAS respectively. EfficientBERT Dong et al. (2021) applies NAS to search for more efficient architecture of feed-forward network that is suitable for edge device deployment.

To show the efficacy of NAS, we summarize the performance of several state-of-the-art NAS approaches on GLUE benchmarks Wang et al. (2019a) in Table 5 in the appendix. These approaches are applied to BERT to discover architectures with smaller sizes, faster inference speed, and better model accuracy. For comparison, performance from original and manually compressed BERT models is also presented. The results show that the BERT architecture improved by NAS yields performance competitive to BERT (c.f., 82.3 from EfficientBERT vs 82.5 from BERT) and is 6.9x smaller and 4.4x faster. The searched architecture also outperforms man 

ually designed, parameter- and inference-efficient model (MobileBERT\({}_{\text{TINY}}\)) at similar size and speed. These results suggest the efficacy of NAS in discovering more efficient network architectures. As NLP researchers continue to design even larger PLMs while the need of deployment on edge devices grows, we expect there will be increasing investment in innovating NAS techniques to make PLM networks more compact and accelerate inference.

ChallengesThe main bottleneck for NAS being widely applied is the prohibitive requirement in computation resources for architecture search. Approaches such as Efficient Neural Architecture Search (ENAS, Pham et al. (2018b)) and Flexible and Expressive Neural Architecture Search (FENAS, Pasunuru and Bansal (2020)) are proposed to improve the search efficiency. As PLMs usually have bulky sizes and slow training speed, search efficiency is even more critical when applying NAS to PLM. Weight-sharing techniques are often applied to accelerate searching Wang et al. (2020a); Dong et al. (2021); Xu et al. (2021b).

### Meta-learning for Data Selection

Multi-linguality, multi-task, and multi-label see many impacts on NLP problems due to the diversity of human languages. To learn models with balanced performance over attributes (e.g., languages, tasks, labels), a common approach is to weight the training examples for data selection to learn models with balanced performance over the attributes, and it is a natural assumption that meta-learning techniques derive more generalizable weighting than manually tuned hyperparameters. For example, Wu et al. (2019) add another gradient update step wrapping the conventional classifier update for training meta-parameters that controls the weight when aggregating losses from different labels to update classifier's parameters. In addition to gradient update, meta-learned weights are also applied directly to training examples for data selection to address the issue of noisy labeling. Shu et al. (2019) propose a technique to jointly learn a classifier and a weighting function, where a conventional gradient update for the classifier and a meta-learning update for the weighting is performed alternatively. The function weights examples to mitigate model overfitting towards biased training data caused by corrupted labels or class imbalance. Zheng et al. (2021) apply a similar framework but extend the weighting with a label correction model. Both techniques show improvement over SOTA in text classification with biased training data.

Additionally, as the progress in the research of pre-training and transfer learning, there is a trend of leveraging datasets in multiple languages, domains, or tasks to jointly pre-train models to learn transferable knowledge. A meta-learned data selector can also help in this scenario by choosing examples that benefit model training and transferability. For instance, Wang et al. (2020b) investigate the common challenge of imbalanced training examples across languages in multilingual MT, which is conventionally addressed by tuning hyper-parameters manually to up-sample languages with less resources. The authors propose Differentiable Data Selection (DDS) to parameterize the sampling strategies. DDS is trained with episodes and REINFORCE algorithm to optimize parameters of sampler and MT models in an alternating way for the MT models to converge with better performance across languages. Pham et al. (2021) formulate data sampling for multilingual MT as a problem of back-translation to generate examples of parallel utterances from unlabeled corpora in target language. The back-translation is jointly trained with MT models to improve translation result through better distribution of training examples and data augmentation. Tarunesh et al. (2021) further study knowledge transferring across tasks and languages. The authors combine Reptile and DDS to meta-learn samplers with six different languages (en, hi, es, de, fr, and zh) and five different tasks (QA, NLI, paraphrase identification, POS tagging, and NER) and demonstrate competitive performance on XtreEME multilingual benchmark dataset Hu et al. (2020).

## 5 Meta-learning beyond Accuracy

In the previous sections, meta-learning is used to obtain better evaluation metrics for NLP applications. This section illustrates how meta-learning can improve NLP applications from more aspects beyond performance.

### Learn to Knowledge Distillation

Knowledge distillation method was proposed in Hinton et al. (2015). The main goal is to transfer knowledge from a so-called teacher model, e.g., a vast neural network trained with a lot of training data, to a more compact student model, e.g., a neu 

ral network with much less trainable parameters. The main weaknesses of this method are as follows: a) the number of teacher models is fixed to one that could limit the power of the transferring process; b) the teacher model is not optimized for the transferring process and c) the teacher model is not aware of the student model during the transferring process. Meta-learning methods can be applied to partially fix these issues. The high-level idea is to increase the number of teacher models and the number of student models and consider each pair of a teacher model and a student model as a task in the meta-learning framework. By doing so, we can train a meta teacher model that works better than a single teacher model (Pan et al., 2020), and we can optimize the transferring process and force the teacher model to be aware of the student model (Zhou et al., 2022).

### Learn to Life-long learning

This subsection discusses how to use meta-learning to improve _lifelong learning_ (LLL) (Chen and Liu, 2018). The real world is changing and evolving from time to time, and therefore machines naturally need to update and adapt to the new data they receive. However, when a trained deep neural network is adapted to a new dataset with a different distribution, it often loses the knowledge previously acquired and performs the previous seen data worse than before. This phenomenon is called _catastrophic forgetting_(McCloskey and Cohen, 1989). There is a wide range of LLL approaches aiming for solving catastrophic forgetting (Parisi et al., 2019). Among them, the following directions apply meta-learning: 5

Footnote 5: On the other hand, in meta-learning, usually, we assume stationary task distribution. Can we do meta-learning with evident distributional shift or when tasks arrive sequentially? There is also research along the direction (Finn et al., 2019; Yap et al., 2021), but out of the scope of this review paper.

Meta-learning for Regularization-based LLL methods

aim to consolidate essential parameters in a model when adapting models with new data (Kirkpatrick et al., 2017; Zenke et al., 2017; Schwarz et al., 2018; Aljundi et al., 2018; Ehret et al., 2021). Meta-learning targets "how to consolidate" and has some successful examples in NLP applications. KnowledgeEditor (De Cao et al., 2021) learns the parameter update strategies that can learn the new data and simultaneously retain the same predictions on the old data. KnowledgeEditor has been applied to fact-checking and QA. Editable Training (Sinitsin et al., 2020) employs learn-to-init approaches to find the set of initial parameters, ensuring that new knowledge can be learned after updates without harming the performance of old data. Editable Training empirically demonstrates the effectiveness on MT.

Meta-learning for Data-based LLL MethodsThe basic idea of data-based methods is to store a limited number of previously seen training examples in memory and then use them for empirical replay, that is, training on seen examples to recover knowledge learned (Sprechmann et al., 2018; de Masson d'Autume et al., 2019; Sun et al., 2019) or to derive optimization constraints (Lopez-Paz and Ranzato, 2017; Li and Hoiem, 2017; Saha and Roy, 2021). A hurdle for data-based approaches is the need to store an unrealistically large number of training examples in memory to achieve good performance. To achieve sample efficiency, Obamuyide and Vlachos (2019); Wang et al. (2020c); Wu et al. (2021a) uses meta-learning to learn a better adaptation algorithm that recovers the knowledge learned with a limited amount of previously seen data. Experiments on text classification and QA benchmarks validate the effectiveness of the framework, achieving state-of-the-art performance using only 1% of the memory size (Wang et al., 2020c).

## 6 Conclusion

This paper investigates how meta-learning is used in NLP applications. We review the task construction settings (Section 3), the commonly used methods including learning to initialize, learning to compare and neural architecture search (Section 4), and highlight research directions that go beyond improving performance (Section 5). We hope this paper will encourage more researchers in the NLP community to work on meta-learning.

## References

* J. Aljundi, F. Babiloni, M. Elhoseiny, M. Rohrbach, and T. Tuytelaars (2018)Memory aware synapses: Learning what (not) to forget. In ECCV, Cited by: SS1.
* M. Andrychowicz, M. Denil, S. Gomez, M. W. Hoffman, D. Pfau, T. Schaul, B. Shillingford, and N. de Freitas (2016)Learning to learn by gradient descent by gradient descent. In NIPS, Cited by: SS1.
* A. Antoniou, H. Edwards, and A. Storkey (2019)How to train your mall. In International Conference on Learning Representations, 

Junyi Ao, Rui Wang, Long Zhou, Shujie Liu, Shuo Ren, Yu Wu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, and Furu Wei. 2021. SpeechTS: Unified-modal encoder-decoder pre-training for spoken language processing.
* de Masson d'Autume et al. (2019) Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong, and Dani Yogatama. 2019. Episodic memory in lifelong language learning. In _NeurIPS_.
* Bansal et al. (2021) Trapit Bansal, Karthick Prasad Gunasekaran, Tong Wang, Tsendsuren Munkhdalai, and Andrew McCallum. 2021. Diverse distributions of self-supervised tasks for meta-learning in NLP. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_.
* Bansal et al. (2020a) Trapit Bansal, Rishikesh Jha, and Andrew McCallum. 2020a. Learning to few-shot learn across diverse natural language classification tasks. In _Proceedings of the 28th International Conference on Computational Linguistics_.
* Bansal et al. (2020b) Trapit Bansal, Rishikesh Jha, Tsendsuren Munkhdalai, and Andrew McCallum. 2020b. Self-supervised meta-learning for few-shot natural language classification tasks. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_.
* Barawa et al. (2019) Ahmed Barawa, Mojeed Abisiga, Ibrahim Gabadgesin, and Afeez Fakunle. 2019. Leveraging end-to-end speech recognition with neural architecture search. In _IJSER_.
* Bojar et al. (2014) Ondrej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Speccia, and Ales Tamchyna. 2014. Findings of the 2014 workshop on statistical machine translation. In _Proceedings of the Ninth Workshop on Statistical Machine Translation_.
* Bose et al. (2020) Avishek Joey Bose, Ankit Jain, Piero Molino, and William L. Hamilton. 2020. Meta-graph: Few shot link prediction via meta learning.
* Bragg et al. (2021) Jonathan Bragg, Arman Cohan, Kyle Lo, and Iz Beltagy. 2021. FLEX: Unifying evaluation for few-shot nlp. In _Advances in Neural Information Processing Systems_.
* Chen et al. (2020a) Daoyuan Chen, Yaliang Li, Minghui Qiu, Zhen Wang, Bofang Li, Bolin Ding, Hongbo Deng, Jun Huang, Wei Lin, and Jingren Zhou. 2020a. Adabert: Task-adaptive bert compression with differentiable neural architecture search. In _IJCAI_.
* Chen et al. (2019a) Mingyang Chen, Wen Zhang, Wei Zhang, Qiang Chen, and Huajun Chen. 2019a. Meta relational learning for few-shot link prediction in knowledge graphs. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_.
* Chen et al. (2020b) Xilun Chen, Asish Ghoshal, Yashar Mehdad, Luke Zettlemoyer, and Sonal Gupta. 2020b. Low-resource domain adaptation for compositional task-oriented semantic parsing. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_.
* Chen et al. (2022) Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis, and He He. 2022. Meta-learning via language model in-context tuning. _ACL_.
* Chen et al. (2020c) Yangbin Chen, Tom Ko, Lifeng Shang, Xiao Chen, Xin Jiang, and Qing Li. 2020c. An investigation of few-shot learning in spoken term classification. In _INTERSPEECH_.
* Chen et al. (2021) Yi-Chen Chen, Po-Han Chi, Shu wen Yang, Kai-Wei Chang, Jheng Bao Lin, Sung-Feng Huang, Da-Rong Liu, Chi-Liang Liu, Cheng-Kuang Lee, and Hung yi Lee. 2021. SpeechNet: A universal modularized model for speech processing tasks.
* Chen et al. (2020d) Yi-Chen Chen, Juli-Yang Hsu, Cheng-Kuang Lee, and Hung yi Lee. 2020d. DARTS-ASR: Differentiable architecture search for multilingual speech recognition and adaptation. In _INTERSPEECH_.
* Chen et al. (2019) Yutian Chen, Yannis Assael, Brendan Shillingford, David Budden, Scott Reed, Helga Zen, Quan Wang, Luis C Cobo, Andrew Trask, Ben Laurie, Caglar Gulcehre, Aaron van den Oord, Oriol Vinyals, and Nando de Freitas. 2019. Sample efficient adaptive text-to-speech. In _ICLR_.
* Chen and Liu (2018) Zhiyuan Chen and Bing Liu. 2018. Lifelong machine learning. _Synthesis Lectures on Artificial Intelligence and Machine Learning_, 12.
* Chen and Lieew (2019) Jen-Tzung Chien and Wei Xiang Lieew. 2019. Meta learning for hyperparameter optimization in dialogue system. In _INTERSPEECH_.
* Conkin et al. (2021) Henry Conkin, Bailin Wang, Kenny Smith, and Ivan Titov. 2021. Meta-learning to compositionally generalize. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_.
* Conneau et al. (2020) Alexis Conneau, Karltixy Khandelwal, Naman Goyal, Vishrav Chaudhry, Guillaume Wenzek, Francisco Guzman, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_.
* Dai et al. (2020) Yinpei Dai, Hangyu Li, Chengguang Tang, Yongbin Li, Jian Sun, and Xiaodan Zhu. 2020. Learning low-resource end-to-end goal-oriented dialog for fast and reliable system deployment. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_.
* De Cao et al. (2021) Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Editing factual knowledge in language models. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_.
* Dingliwal et al. (2021) Saket Dingliwal, Shuyang Gao, Sanchit Agarwal, Chien-Wei Lin, Tagyoung Chung, and Dilek Hakkani-Tur. 2021. Few shot dialogue state tracking using meta-learning. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_.
* Dong et al. (2021) Chenhe Dong, Guangrun Wang, Hang Xu, Jiefeng Peng, Xiaozhe Ren, and Xiaodan Liang. 2021. EfficientBERT: Progressively searching multilayer perceptron via warm-up knowledge distillation. In _EMNLP_ 

Zi-Yi Dou, Keyi Yu, and Antonios Anastasopoulos. 2019: Investigating meta-learning algorithms for low-resource natural language understanding tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Cited by: SS4.
* B. Ehret, C. Henning, M. Cervera, A. Meulemans, J. Von Oswald, and B. F. Grewe (2021)Continual learning in recurrent neural networks. In International Conference on Learning Representations, External Links: Link Cited by: SS3.1.
* R. Eloff, H. A. Engelbrecht, and H. Kamper (2019)Multimodal one-shot learning of speech and images. In ICASSP, Cited by: SS5.
* C. Finn, P. Abbeel, and S. Levine (2017)Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, Cited by: SS6.
* C. Finn, A. Rajeswaran, S. Kakade, and S. Levine (2019)Online meta-learning. In ICML, Cited by: SS1, SS2.

 

Mike Huisman, Jan N. van Rijn, and Aske Plaat (2021)A survey of deep meta-learning. In AI Review (AIRE) Journal, Cited by: SS4.3.
* S. Indurthi, H. Han, N. Kumar Lakumarapu, B. Lee, I. Chung, S. Kim, and C. Kim (2020)Data efficient direct speech-to-text translation with modality agnostic meta-learning. In ICASSP, Cited by: SS5.
* Y. Jiang, C. Hu, T. Xiao, C. Zhang, and J. Zhu (2019)Improved differentiable architecture search for language modeling and named entity recognition. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Cited by: SS6.
* Z. Ke, L. Shi, S. Sun, E. Meng, B. Wang, and X. Qiu (2021)Pre-training with meta learning for Chinese word segmentation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, External Links: Link Cited by: SS1, SS2.
* J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al. (2017)Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences114. Cited by: SS1, SS2.
* O. Klejch, J. Fainberg, and P. Bell (2018)Learning to adapt: a meta-learning approach for speaker adaptation. In INTERSPEECH, Cited by: SS1, SS2.
* O. Klejch, J. Fainberg, P. Bell, and S. Renals (2019)Speaker adaptive training using model agnostic meta-learning. In ASRU, Cited by: SS1, SS2.
* W. M. Kouw and M. Loog (2021)A review of domain adaptation without target labels. IEEE Transactions on Pattern Analysis and Machine Intelligence43 (3), pp. 766-785. Cited by: SS1, SS2.
* A. Langedijk, V. Dankers, P. Lippe, S. Bos, B. Cardenas Guevara, H. Yannakoudakis, and E. Shutova (2021)Meta-learning for fast cross-lingual adaptation in dependency parsing. In arXiv, Cited by: SS1, SS2.
* H. Lee, M. Mohtarami, S. Li, D. Jin, M. Korpusik, S. Dong, N. Thang Vu, and D. Hakkani-Tur (2021)Proceedings of the 1st workshop on meta learning and its applications to natural language processing. In Proceedings of the 1st Workshop on Meta Learning and Its Applications to Natural Language Processing, Cited by: SS1, SS2.
* H. Lee, N. Thang Vu, and S. Li (2021)Meta learning and its applications to natural language processing. ACL-IJCNLP 2021, pp. 15. Cited by: SS1, SS2.
* D. Li, Y. Yang, Y. Song, and T. M. Hospedales (2018)Learning to generalize: data-learning for domain generalization. In AAAI, Cited by: SS1, SS2.
* R. Li, X. Wang, and H. Yu (2020)MetaMT, a meta learning method leveraging multiple domain data for low resource machine translation. Proceedings of the AAAI Conference on Artificial Intelligence34 (05). Cited by: SS1, SS2.
* Y. Li, C. Hu, Y. Zhang, N. Xu, Y. Jiang, T. Xiao, J. Zhu, T. Liu, and C. Li (2020)Learning architectures from an extended search space for language modeling. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Cited by: SS1, SS2.
* Z. Li, M. Kumar, W. Headden, B. Yin, Y. Wei, Y. Zhang, and Q. Yang (2020)Learn to cross-lingual transfer with meta graph learning across heterogeneous languages. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: SS1, SS2.
* Z. Li and D. Hoiem (2017)Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence40, pp. . Cited by: SS1, SS2.
* D. Lopez-Paz and M. Ranzato (2017)Gradient episodic memory for continual learning. In NeurIPS, Cited by: SS1, SS2.
* F. Lux and N. Thang Vu (2021)Meta-learning for improving rare word recognition in end-to-end asr. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5974-5978. Cited by: SS1, SS2.
* Xin Lv, Y. Gu, X. Han, L. Hou, J. Li, and Z. Liu (2019)Adapting meta knowledge graph information for multi-hop reasoning over few-shot relations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Cited by: SS1, SS2.
* A. MacLaughlin, J. Dhamala, A. Kumar, S. Venkatapathy, R. Venkatesan, and R. Gupta (2020)Evaluating the effectiveness of efficient neural architecture search for sentence-pair tasks. In Proceedings of the First Workshop on Insights from Negative Results in NLP, Cited by: SS1, SS2.
* A. Madotto, Z. Lin, C. Wu, and P. Fung (2019)Personalizing dialogue agents via meta-learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Cited by: SS1, SS2.
* C. Manning and H. Schutze (1999)Foundations of statistical natural language processing, Cited by: SS1, SS2.
* H. Mazzawi, X. Gonzalvo, A. Kracun, P. Sridhar, N. Subrahmanya, I. Lopez Moreno, H. J. Park, and P. Violette (2019)Improving keyword spotting and language identification via neural architecture search at scale. In INTERSPEECH, Cited by: SS1, SS2.
* M. McCloskey and N. J. Cohen (1989)Catastrophic interference in connectionist networks: the sequential learning problem. In Psychology of Learning and Motivation, Vol. 24, Cited by: SS1, SS2.
* S. Merity, B. McCann, and R. Socher (2017)Revisiting activation regularization for language rnns. arXiv preprint arXiv:1708.01009. Cited by: SS1, SS2.
* M. M'hamdi, D. Soon Kim, F. Derroncourt, T. Bui, X. Ren, and J. May (2021)X-METRA-ADA: cross-lingual meta-transfer learning adaptation to natural language understanding and question answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Cited by: SS1, SS2.
* F. Mi, M. Huang, J. Zhang, and B. Faltings (2019)Meta-learning for low-resource natural language generation in task-oriented dialogue systems. In IJCAI, Cited by: SS1, SS2.
* T. Mikolov, M. Karaflat, L. Burget, J. Cernocky, and S. Khudanpur (2010)Recurrent neural network based language model. In INTERSPEECH, Cited by: SS1, SS2.

 

* Mishra et al. (2018) Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. 2018. A simple neural attentive meta-learner. In _ICLR_.
* Murty et al. (2021) Shikhar Murty, Tatsunori B. Hashimoto, and Christopher Manning. 2021. DReCa: A general task augmentation strategy for few-shot natural language inference. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_.
* Ni et al. (2021) Renkun Ni, Micah Goldblum, Amr Sharaf, Kezhi Kong, and Tom Goldstein. 2021. Data augmentation for meta-learning. In _Proceedings of the 38th International Conference on Machine Learning_, volume 139 of _Proceedings of Machine Learning Research_, pages 8152-8161. PMLR.
* Nichol et al. (2018) Alex Nichol, Joshua Achiam, and John Schulman. 2018. On first-order meta-learning algorithms.
* Nooralahzadeh et al. (2020) Farhad Nooralahzadeh, Giannis Bekoulis, Johannes Bjerva, and Isabelle Augenstein. 2020. Zero-shot cross-lingual transfer with meta learning. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_.
* Obamuyide and Vlachos (2019a)Abiola Obamuyide and Andreas Vlachos. 2019a. Meta-learning improves lifelong relation extraction. In _Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)_.
* Obamuyide and Vlachos (2019b) Abiola Obamuyide and Andreas Vlachos. 2019b. Model-agnostic meta-learning for relation classification with limited supervision. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_.
* Oguz and Vu (2021) Cennet Oguz and Ngoc Thang Vu. 2021. Few-shot learning for slot tagging with attentive relational network. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_.
* Pan et al. (2020) Haojie Pan, Chengyu Wang, Minghui Qiu, Yichang Zhang, Yaliang Li, and Jun Huang. 2020. Meta-KD: A meta knowledge distillation framework for language model compression across domains. _arXiv preprint arXiv:2012.01266_.
* Parisi et al. (2019) German I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. 2019. Continual lifelong learning with neural networks: A review. _Neural Networks_, 113:54-71.
* Park et al. (2021) Cheonobok Park, Yunwon Tae, TaeHee Kim, Soyoung Yang, Mohammad Azam Khan, Lucy Park, and Jaegul Choo. 2021. Unsupervised neural machine translation for low-resource domains via meta-learning. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_.
* Parami and Lee (2020) Archit Parami and Minwoo Lee. 2020. Few-shot keyword spotting with prototypical networks.
* Pasunuru and Bansal (2019) Ramakanth Pasunuru and Mohit Bansal. 2019. Continual and multi-task architecture search. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_.
* Pasunuru and Bansal (2020) Ramakanth Pasunuru and Mohit Bansal. 2020. FENAS: Flexible and expressive neural architecture search. In _Findings of the Association for Computational Linguistics: EMNLP 2020_.
* Pham et al. (2018a) Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. 2018a. Efficient neural architecture search via parameter sharing. In _ICML_.
* Pham et al. (2021) Hieu Pham, Melloy Guan, Barret Zoph, Quoc Le, and Jeff Dean. 2018b. Efficient neural architecture search via parameters sharing. In _ICML_.
* Pham et al. (2021) Hieu Pham, Xinyi Wang, Yiming Yang, and Graham Neubig. 2021. Meta back-translation. In _ICLR_.
* Qian et al. (2021) Kun Qian, Wei Wei, and Zhou Yu. 2021. A student-teacher architecture for dialog domain adaptation under the meta-learning setting. In _AAAI_.
* Qin and Yu (2019) Kun Qian and Zhou Yu. 2019. Domain adaptive dialog generation via meta learning. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_.
* Raffel et al. (2019) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. _arXiv preprint arXiv:1910.10683_.
* Rajendran et al. (2020) Janarthanan Rajendran, Alexander Irpan, and Eric Jang. 2020. Meta-learning requires meta-augmentation. In _NeurIPS_.
* Ravi and Larochelle (2017) Sachin Ravi and Hugo Larochelle. 2017. Optimization as a model for few-shot learning. In _ICLR_.
* Ren et al. (2020) Haopeng Ren, Yi Cai, Xiaofeng Chen, Guohua Wang, and Qing Li. 2020. A two-phase prototypical network model for incremental few-shot relation classification. In _Proceedings of the 28th International Conference on Computational Linguistics_.
* Saha and Roy (2021) Gobinda Saha and Kaushik Roy. 2021. Gradient projection memory for continual learning. _ICLR_.
* Sang and De Meulder (2003) Erik Tjong Kim Sang and Fin De Meulder. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In _HLT-NAACL_.
* Schutze (1992) H Schutze. 1992. Dimensions of meaning. In _Proceedings of the 1992 ACM/IEEE conference on Supercomputing_, pages 787-796.
* Schwarz et al. (2018) Jonathan Schwarz, Wojciech Czarnecki, Jelena Luketina, Agnieszka Grabska-Barwinska, Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. 2018. Progress & compress: A scalable framework for continual learning. In _ICML_.
* Serra et al. (2019) Joan Serra, Santiago Pascual, and Carlos Segura. 2019. Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversion. In _NeurIPS_.
* Sheng et al. (2020) Jiawei Sheng, Shu Guo, Zhenyu Chen, Juwei Yue, Lihong Wang, Tingwen Liu, and Hongbo Xu. 2020. Adaptive attentional network for few-shot knowledge graph completion. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_.
* Shu et al. (2019) Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. 2019. Meta-Weight-Net: Learning an explicit mapping for sample weighting. In _NeurlPS_.
* Sinitsin et al. (2020) Anton Sinitsin, Vesvool Polkhotnyuk, Dmitriy Pyrkin, Sergei Popov, and Artem Babenko. 2020. Editable neural networks. In _ICLR_ 

Jake Snell, Kevin Swersky, and Richard S. Zemel. 2017. Prototypical networks for few-shot learning. In _NIPS_.
* Sprechmann et al. (2018) Pablo Sprechmann, Siddhant M. Jayakumar, Jack W. Rae, Alexander Pritzel, Adria Puigdomenech Badia, Benigno Uria, Oriol Vinyals, Demis Hassabis, Razvan Pascanu, and Charles Blundell. 2018. Memory-based parameter adaptation. In _ICLR_.
* Sun et al. (2019a) Fan-Keng Sun, Cheng-Hao Ho, and Hung-Yi Lee. 2019a. LAMOL: LLanguage MOdeling for Lifelong Language Learning. In _ICLR_.
* Sun et al. (2018) Jingyuan Sun, Shaonan Wang, and Chenqing Zong. 2018. Memory, show the way: Memory based few shot word representation learning. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_.
* Sun et al. (2019b) Shengli Sun, Qingfeng Sun, Kevin Zhou, and Tengchao Lv. 2019b. Hierarchical attention prototypical networks for few-shot text classification. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_.
* Sun et al. (2020) Zhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, and Denny Zhou. 2020. MobileBERT: a compact task-agnostic BERT for resource-limited devices. In _ACL_.
* Sung et al. (2018) Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H.S. Torr, and Timothy M. Hospedales. 2018. Learning to compare: Relation network for few-shot learning. In _CVPR_.
* Suris et al. (2019) Didac Suris, Dave Epstein, Heng Ji, Shih-Fu Chang, and Carl Vondrick. 2019. Learning to learn words from narrated video. In _ECCV_.
* Tan et al. (2019) Ming Tan, Yang Yu, Haoyu Wang, Dakuo Wang, Saloni Potdar, Shiyu Chang, and Mo Yu. 2019. Out-of-Domain detection for low-resource text classification tasks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_.
* Tarunesh et al. (2021) Ishan Taruneh, Sushi Khysla, Vishwajeet Kumar, Ganesh Ramakrishnan, and Preeth Jyothi. 2021. Meta-learning for effective multi-task and multilingual modelling. In _EACL_.
* Triantiafillou et al. (2020) Eleni Triantiafillou, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Utku Evi, Kelvin Xu, Ross Goroshin, Carles Gelada, Kevin Swersky, Pierre-Antoine Manzagol, and Hugo Larochelle. 2020. Meta-Dataset: A dataset of datasets for learning to learn from few examples. In _International Conference on Learning Representations_.
* Vinyals et al. (2016) Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. 2016. Matching networks for one shot learning. In _NIPS_.
* Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In _Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_.
* Wang et al. (2019a) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel B Bowman. 2019a. Glue: A multi-task benchmark and analysis platform for natural language understanding. In _ICLR_.
* Wang et al. (2021) Balilin Wang, Mirella Lapata, and Ivan Titov. 2021a. Meta-learning for domain generalization in semantic parsing. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_.
* Wang et al. (2020a) Hanrui Wang, Zhanghao Wu, Zhiijian Liu, Han Cai, Ligeng Zhu, Chuang Gan, and Song Han. 2020a. HAT: Hardware transformers for efficient natural language processing. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_.
* Wang et al. (2021b) Lingxiao Wang, Kevin Huang, Tengyu Ma, Quanquan Gu, and Jing Huang. 2021b. Variance-reduced first-order meta-learning for natural language processing tasks. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_.
* Wang et al. (2020b) Xinyi Wang, Yulia Tsvetkov, and Graham Neubig. 2020b. Balancing training for multilingual neural machine translation. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_.
* Wang et al. (2019b) Zihao Wang, Kwuping Lai, Piji Li, Lidong Bing, and Wai Lam. 2019b. Taskling long-tailed relations and uncommon entities in knowledge graph completion. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_.
* Wang et al. (2020c) Zirui Wang, Sanket Vaibhav Mehta, Barnabas Poczos, and Jaime Carbonell. 2020c. Efficient meta lifelong-learning with limited memory. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_.
* Winata et al. (2020a) Genta Indra Winata, Samuel Cahyawjiaya, Zhaojiang Lin, Zihan Liu, Peng Xu, and Pascale Fung. 2020a. Meta-transfer learning for code-switched speech recognition. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_.
* Winata et al. (2020b) Genta Indra Winata, Samuel Cahyawjiaya, Zihan Liu, Zhaojiang Lin, Andrea Madotto, Peng Xu, and Pascale Fung. 2020b. Learning fast adaptation on cross-accented speech recognition. In _INTERSPEECH_.
* Wu et al. (2019) Jiawei Wu, Wenhan Xiong, and William Yang Wang. 2019. Learning to learn and predict: A meta-learning approach for multi-label classification. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_.
* Wu et al. (2020) Qianhui Wu, Zijia Lin, Guoxin Wang, Hui Chen, Borje F. Karlsson, Biqing Huang, and Chin-Yew Lin. 2020. Enhanced meta-learning for cross-lingual named entity recognition with minimal resources. In _AAAI_I.
* Wu et al. (2021a) Tongtong Wu, Xuekai Li, Yuan-Fang Li, Reza Haffari, Guilin Qi, Yujin Zhu, and Guoqiang Xu. 2021a. Curriculum-meta learning for order-robust continual relation extraction. In _AAAI_I.
* Wu et al. (2021b) Yuan-Kuei Wu, Kuan-Po Huang, Yu Tsao, and Hung-yi Lee. 2021b. One shot learning for speech separation. In _ICASSP_.

 

Mengzhou Xia, Guoqing Zheng, Subhabrata Mukherjee, Midal Shokohii, Graham Neubig, and Ahmed Hassan Awadalah. 2021. MetaXL: Meta representation transformation for low-resource cross-lingual learning. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_.
* Xiao et al. (2021) Yubei Xiao, Ke Gong, Pan Zhou, Guolin Zheng, Xiaodan Liang, and Liang Lin. 2021. Adversarial meta sampling for multilingual low-resource speech recognition. In _AAAI_.
* Xiong et al. (2018) Wenhan Xiong, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. 2018. One-shot relational learning for knowledge graphs. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_.
* Xu et al. (2021a) Guangyue Xu, Parisi Kordjamshidi, and Joyce Chai. 2021a. Zero-shot compositional concept learning. In _Proceedings of the First Workshop on Meta Learning and Its Applications to Natural Language Processing_.
* Xu et al. (2021b) Jin Xu, Xu Tan, Renqian Luo, Kaitao Song, Jian Li, Tao Qin, and Tie-Yan Liu. 2021b. Nas-bert: task-agnostic and adaptive-size bert compression with neural architecture search. In _Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining_.
* Xu et al. (2021c) Weijia Xu, Batool Haider, Jason Krone, and Saab Mansour. 2021c. Soft layer selection with meta-learning for zero-shot cross-lingual transfer. In _Proceedings of the First Workshop on Meta Learning and Its Applications to Natural Language Processing_.
* Yan et al. (2020) Ming Yan, Hao Zhang, Di Jin, and Joey Tianyi Zhou. 2020. Multi-source meta transfer for low resource multiple-choice question answering. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_.
* Yao et al. (2021a) Huaxiu Yao, Long-Kai Huang, Linjun Zhang, Ying Wei, Li Tian, James Zou, Junzhou Huang, and Zhenhui Li. 2021a. Improving generalization in meta-learning via task augmentation. In _Proceedings of the 38th International Conference on Machine Learning_, volume 139 of _Proceedings of Machine Learning Research_, pages 11887-11897. PMLR.
* Yao et al. (2021b) Huaxiu Yao, Linjun Zhang, and Chelsea Finn. 2021b. Metaclearning with fewer tasks through task interpolation.
* Yap et al. (2021) Pauching Yap, Hippolyt Ritter, and David Barber. 2021. Addressing catastrophic forgetting in few-shot problems. In _ICML_.
* Ye et al. (2021) Qinyuan Ye, Bill Yuchen Lin, and Xiang Ren. 2021. CrossFit: A few-shot learning challenge for cross-task generalization in NLP. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_.
* Ye and Ling (2019) Zhi-Xiu Ye and Zhen-Hua Ling. 2019. Multi-level matching and aggregation network for few-shot relation classification. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_.
* Yin (2020) Wenpeng Yin. 2020. Meta-learning for few-shot natural language processing: A survey. _arXiv preprint arXiv:2007.09604_.
* Yu et al. (2018) Mo Yu, Xiaoxiao Guo, Jinfeng Yi, Sihyu Chang, Saloni Potdar, Yu Cheng, Gerald Tesauro, Haeoya Wang, and Bowen Zhou. 2018. Diverse few-shot text classification with multiple metrics. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_.
* Zenke et al. (2017) Friedemann Zenke, Ben Poole, and Surya Ganguli. 2017. Continual learning through synaptic intelligence. In _ICML_.
* Zhang et al. (2020) Chuxu Zhang, Huaxiu Yao, Chao Huang, Meng Jiang, Zhenhui Li, and Nitesh V Chawla. 2020. Few-shot knowledge graph completion. In _AAAI_.
* Zheng et al. (2021) Guoqing Zheng, Ahmed H. Awadallah, and Susan Dumais. 2021. Meta label correction for noisy label learning. In _AAAI_.
* Zhou et al. (2022) Wangchunshu Zhou, Canwen Xu, and Julian McAuley. 2022. BERT learns to teach: Knowledge distillation with meta learning. _ACL_.
* Zoph and Le (2017) Barret Zoph and Quoc V. Le. 2017. Neural architecture search with reinforcement learning. In _ICLR_.
* Zoph et al. (2018) Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V. Le. 2018. Learning transferable architectures for scalable image recognition. In _CVPR_.

 



## Appendix A Appendx

\begin{table}
\begin{tabular}{|l|p{216.8pt}|} \hline Terminologies & Meaning \\ \hline \hline (NLP) Problem Model Parameter Meta-parameter Support Set Query Set Task Task Training Within-task Training Within-task Testing Episode Meta-training Tasks Meta-testing Tasks Cross-task Training Across-task Testing Cross-task Testing & a type of NLP problems like QA, POS, or MT parameters of models making inference for underlying problems parameters of learning algorithms (e.g., model init, optimizers) that are shared across tasks in set of training examples for updating model parameters \\ \hline \end{tabular}
\end{table}
Table 1: Terminologies and their meanings.





\begin{table}
\begin{tabular}{l l l l} \hline \hline Work & Method & Application & Task construction \\ \hline Dou et al. (2019) & INIT & TC & Cross-problem \\ Bansal et al. (2020a) & INIT & TC & Cross-problem \\ Holla et al. (2020) & INIT & TC & A task includes sentences containing the same word with different senses. \\ Zhou et al. (2022) & INIT & TC & Knowledge Distillation \\ Pan et al. (2020) & COM & TC & Knowledge Distillation \\ van der Heijden et al. (2021) & INIT & TC & Cross-lingual \\ Bansal et al. (2020b) & INIT & TC & Cross-problem (some tasks are generated in an self-supervised way) \\ Murty et al. (2021) & INIT & TC & Cross-problem \\ Wang et al. (2021b) & INIT & TC, DST & Cross-domain \\ Yu et al. (2018) & COM & TC & Cross-domain \\ Tan et al. (2019) & COM & TC & Cross-domain \\ Geng et al. (2019) & COM & TC & Cross-domain \\ Sun et al. (2019b) & COM & TC & The tasks are seperated by class labels. \\ Gong et al. (2020) & COM & TC & The tasks are seperated by class labels. \\ Li et al. (2020c) & COM & TC & Domain Generalization \\ Wu et al. (2019) & OPT & TC & Monolithic \\ Pasuuru and Bansal (2020) & NAS & TC & Monolithic \\ Pasunuru and Bansal (2019) & NAS & TC & Monolithic \\ Xu et al. (2021c) & OPT & TC & Domain Generalization \\ Zheng et al. (2021) & SEL & TC & Monolithic \\ Wu et al. (2020) & INIT & TAG & Cross-lingual \\ Xia et al. (2021) & INIT & TC, TAG & Cross-domain \\ Hou et al. (2020) & COM & TAG & Cross-domain \\ Oguz and Vu (2021) & COM & TAG & The tasks are seperated by class labels. \\ Li et al. (2020b) & NAS & DG & Monolithic \\ Jiang et al. (2019) & NAS & TAG & Monolithic \\ Obamuyide and Vlachos (2019b) & INIT & IE & Each task includes the examples for a relation. \\ Bose et al. (2020) & INIT & IE & Each task is a graph. \\ Lv et al. (2019) & INIT & IE & Each task includes the examples for a relation. \\ Chen et al. (2019a) & COM & IE & Each task includes the examples for a relation. \\ Gao et al. (2019a) & COM & IE & Each task includes the examples for a relation. \\ Ren et al. (2020) & COM & IE & Each task includes the examples for a relation. \\ Xiong et al. (2018) & COM & IE & Each task includes the examples for a relation. \\ Wang et al. (2019b) & INIT & IE & Each task includes the examples for a relation. \\ Zhang et al. (2020) & COM & IE & Each task includes the examples for a relation. \\ Sheng et al. (2020) & COM & IE & Each task includes the examples for a relation. \\ Hu et al. (2019) & INIT & WE & Each task includes the context of a word. \\ Sun et al. (2018) & COM & WE & Each task includes the context of a word. \\ M’handi et al. (2021) & INIT & QA, TAG & Cross-lingual, Domain Generalization \\ Nooralhazharch et al. (2020) & INIT & QA, TC & Cross-lingual \\ Yan et al. (2020) & INIT & QA & Cross-domain \\ Gu et al. (2018) & INIT & MT & Cross-lingual \\ Indurthi et al. (2020) & INIT & MT & Cross-problem \\ Li et al. (2020a) & INIT & MT & Cross-domain \\ Park et al. (2021) & INIT & MT & Cross-domain \\ Wang et al. (2020b) & SEL & MT & Monolithic \\ \hline \hline \end{tabular}
\end{table}
Table 2: An organization of works on meta-learning in NLP. The **Application** column lists the applications that are performed in corresponding papers. We use the following abbreviations. **QA**: Question Answering. **MT**: Machine Translation. **TC**: Text Classification (including Natural Language Inference). **IE**: Information Extraction (including Relation Classificaciton and Knowledge Graph Completion). **WE**: Word Ebedding **TAG**: Sequence Tagging. **PAR**: Parsing. **DST**: Dialogue State Tracking. **DG**: Dialgoue Generation (including Natural Language Generation). **MG**: Multimodal Grounding. **ASR**: Automatic Speech Recognition. **SS**: Source Separation. **KS**: Keyword Spotting. **VC**: Voice Cloning. **SED**: Sound Event Detection. The **Method** column lists the involving meta-learning methods. **INIT** is learning to initialize; **COM** is learning to compare; **NAS** is network architecture search; **OPT** is learning to optimize; **ALG** is learning the learning algorithm; **SEL** is learning to select data. **Task construction** column lists the way each work is built for training meta-parameters. Please refer to Section 3 for the description about task construction.





\begin{table}
\begin{tabular}{l l l l} \hline \hline Work & Method & Application & Task construction \\ \hline Pham et al. (2021) & SEL & MT & Monolithic \\ Guo et al. (2019) & INIT & PAR & Monolithic \\ Huang et al. (2018) & INIT & PAR & Monolithic \\ Langedijk et al. (2021) & INIT & PAR & Cross-lingual \\ Chen et al. (2020b) & INIT & PAR & Cross-domain \\ Wang et al. (2021a) & INIT & PAR & Domain Generalization \\ Qian and Yu (2019) & INIT & DG & Cross-domain \\ Madotto et al. (2019) & INIT & DG & Cross-domain (each domain is one type of persona) \\ Mi et al. (2019) & INIT & DG & Cross-domain \\ Huang et al. (2020a) & INIT & DST & Cross-domain \\ Dingliwal et al. (2021) & INIT & DST & Cross-domain \\ Huang et al. (2020b) & INIT & DST & Cross-domain \\ Dai et al. (2020) & INIT & DG & Cross-domain \\ Qian et al. (2021) & INIT & DG & Cross-domain \\ Chien and Lieow (2019) & OPT & DG & Monolithic \\ Hsu et al. (2020) & INIT & ASR & Cross-lingual \\ Klejch et al. (2019) & INIT & ASR & Cross-domain (each domain refers to a speaker) \\ Winata et al. (2020a) & INIT & ASR & Cross-lingual \\ Winata et al. (2020b) & INIT & ASR & Cross-domain (each domain refers to a accent) \\ Xiao et al. (2021) & INIT & ASR & Cross-lingual \\ Klejch et al. (2018) & OPT & ASR & Cross-domain (each domain refers to a speaker) \\ Chen et al. (2020d) & NAS & ASR & Cross-lingual \\ Baruwa et al. (2019) & NAS & ASR & Monolithic \\ Wu et al. (2021b) & INIT & SS & Cross-domain (each domain refers to a speaker) \\ Huang et al. (2021) & INIT & SS & Cross-domain(each domain refers to a accent) \\ Chen et al. (2020c) & INIT & KS & The tasks are separated by keyword sets. \\ Parnami and Lee (2020) & COM & KS & The tasks are separated by keyword sets. \\ Huh et al. (2021) & COM & KS & The tasks are separated by keyword sets. \\ Mazzawi et al. (2019) & NAS & KS & Monolithic \\ Lux and Vu (2021) & COM & KS & The tasks are separated by keyword sets. \\ Serra et al. (2019) & ALG & VC & Cross-domain (each domain refers to a speaker) \\ Chen et al. (2019b) & ALG & VC & Cross-domain (each domain refers to a speaker) \\ Huang et al. (2022) & INIT & VC & Cross-domain (each domain refers to a speaker) \\ Tarunesh et al. (2021) & INIT, SEL & QA, TC, TAG & Cross-lingual, Cross-problem \\ Eloff et al. (2019) & COM & MG & Monolithic \\ Suris et al. (2019) & ALG & MG & Each task contains multiple examples of text-image pairs. \\ Xu et al. (2021a) & COM & MG & Each task contains an image and a word set. \\ De Cao et al. (2021) & OPT & TC, QA & Life-long learning \\ Sinitsin et al. (2020) & INIT & MT & Life-long learning \\ Wang et al. (2020c) & INIT & TC, QA & Life-long learning \\ Wu et al. (2021a) & INIT & IE & Life-long learning \\ Obamuyide and Vlachos (2019) & INIT & IE & Life-long learning \\ \hline \hline \end{tabular}
\end{table}
Table 3: Continue of Table 2. Pham et al. (2021) learns a backtranslation model for data augmentation, so it is considered as SEL.





\begin{table}
\begin{tabular}{|l|p{142.3pt}|p{56.9pt}|} \hline Work & Method & How to Initialize the Initialization \\ \hline (Bansal et al., 2020a) & LEOPARD & BERT \\ (Li et al., 2020a) & MAML & Word Embedding \\ (Park et al., 2021) & MAML & XLM \\ (Gu et al., 2018) & FOMAML & Word Embedding \\ (Langedijk et al., 2021) & FOMAML & mBERT \\ (Chen et al., 2020b) & Reptile & BART \\ (Huang et al., 2020a) & MAML & BERT \\ (Wang et al., 2021b) & Propose a new method based on Reptile & Word Embedding \\ (Dingliwal et al., 2021) & Reptile & RoBERTa \\ (Qian and Yu, 2019) & MAML & Word Embedding \\ (Qian et al., 2021) & MAML & Word Embedding \\ (Madotto et al., 2019) & MAML & Word Embedding \\ (Dai et al., 2020) & MAML & - \\ (Hsu et al., 2020) & FOMAML & Multilingual ASR \\ (Xiao et al., 2021) & MAML/FOMAML/Reptile & - \\ (Winata et al., 2020b) & MAML & Pretrain by Supervised Learning \\ (Klejch et al., 2019) & FOMAML & - \\ Huang et al., 2021) & MAML/FOMAML & - \\ Indurthi et al., 2020) & FOMAML & - \\ Winata et al., 2020a) & FOMAML & - \\ Wu et al., 2021b) & MAML & Pretrain by Multi-task Learning \\ (Ke et al., 2021) & MAML & BERT \\ Xia et al., 2021) & MetaXL & mBERT/XLM-R \\ (Dou et al., 2019) & MAML/FOMAML/Reptile & BERT \\ (Obumayide and Vlachos, 2019b) & FOMAML & Word Embedding \\ Lv et al., 2019) & MAML & - \\ Holla et al., 2020) & FOMAML/Proto(FO)MAML & Word Embedding/ELMo/BERT \\ (Huang et al., 2020b) & MAML & Word Embedding \\ (Mi et al., 2019) & MAML & - \\ Wang et al., 2021a) & DG-MAML & BERT \\ (Conklin et al., 2021) & DG-MAML & - \\ (M’hamdi et al., 2021) & MAML & mBERT \\ (Nooralahzadeh et al., 2020) & MAML & BERT/mBERT/XLM-R \\ Garaci et al., 2021) & MAML & mBERT \\ van der Heijden et al., 2021) & FOMAML/Reptile/Proto(FO)MAML & XLM-R \\ (Bansal et al., 2020b) & LEOPARD & BERT \\ Murty et al., 2021) & FOMAML & BERT \\ Hua et al., 2020) & Repitic & - \\ Yan et al., 2020) & MAML & BERT/RoBERTa \\ Wang et al., 2019b) & Reptile & - \\ Bose et al., 2020) & Meta-Graph & - \\ \hline \end{tabular}
\end{table}
Table 4: Summary of learn-to-init variants. This table contains the following information. (1) **Method**: There are many variants in the learn-to-init family. The most representative one is MAML. Typical MAML Finn et al. (2017) has large computation intensity, so the first-order approximations like FOMAML Finn et al. (2017) and Reptile Nichol et al. (2018) are widely used. DG-MAML Li et al. (2018) is for domain generalization. Typical learn-to-init assumes that all the tasks use the same network architecture, but LEOPARD Bansal et al. (2020a) and Proto(FO)MAML Triantfillou et al. (2020) are proposed to overcome the limitation. (2) **How to Initialize the Initialization**: Learn-to-init approaches aim at learning the initial parameters. But where does the initialization of MAML come from? We found that using self-supervised pre-training as initialization is common. The table specifies the pre-trained models used to initialize the learn-to-init methods. ’-’ means the initial parameters are learned from random initialization or cannot tell based on the descriptions in the papers.

\begin{table}
\begin{tabular}{l|c l|c c c c c c} \hline Model & \#Params & Latency & MNLI & QQP & QNLI & SST-2 & MRPC & RTE & Avg \\ \hline BERT\({}_{BASE}\) (Google) & 108.9M & 362ms & **84.6** & **71.2** & 90.5 & **93.5** & **88.9** & 66.4 & **82.5** \\ MobileBERTT\({}_{\text{NNY}}\)Sun et al. (2020) & 15.1M & 96ms & 81.5 & 68.9 & 89.5 & 91.7 & 87.9 & 65.1 & 80.8 \\ \hline AdaBERT Chen et al. (2020a) & 6.4-9.5M & 12.4-28.5ms & 81.6 & 70.7 & 86.8 & 91.8 & 85.1 & 64.4 & 80.1 \\ EfficientBERT Dong et al. (2021) & 16M & 103ms & 83.0 & **71.2** & **90.6** & 92.3 & **88.9** & **67.8** & 82.3 \\ \hline \end{tabular}
\end{table}
Table 5: Performance of selected NAS approaches on the test set of GLUE benchmark.