Original: Meta Learning for Natural Language Processing: A Survey
Revised: Meta Learning for Natural Language Processing: A Survey

Original: Abstract
Revised: Abstract

Original: Deep learning has been the mainstream technique in natural language processing (NLP) area.
Revised: Deep learning has been the mainstream technique in natural language processing (NLP) area.

Original: However, the techniques require many labeled data and are less generalizable across domains.
Revised: However, the techniques require many labeled data and are less generalizable across domains.

Original: Meta-learning is an arising ﬁeld in machine learning studying approaches to learn better learning algorithms.
Revised: Meta-learning is an arising ﬁeld in machine learning studying approaches to learn better learning algorithms.

Original: Approaches aim at improving algorithms in various aspects, including data efﬁciency and generalizability.
Revised: Approaches aim at improving algorithms in various aspects, including data efﬁciency and generalizability.

Original: Efﬁcacy of approaches has been shown in many
Revised: Efﬁcacy of approaches has been shown in many

Original: NLP tasks, but there is no systematic survey of these approaches in NLP, which hinders more researchers from joining the ﬁeld.
Revised: NLP tasks, but there is no systematic survey of these approaches in NLP, which hinders more researchers from joining the ﬁeld.

Original: Our goal with this survey paper is to offer researchers pointers to relevant meta-learning works in NLP and attract more attention from the NLP community to drive future innovation.
Revised: Our goal with this survey paper is to offer researchers pointers to relevant meta-learning works in NLP and attract more attention from the NLP community to drive future innovation.

Original: This paper ﬁrst introduces the general concepts of meta-learning and the common approaches.
Revised: This paper ﬁrst introduces the general concepts of meta-learning and the common approaches.

Original: Then we summarize task construction settings and application of meta-learning for various
Revised: Then we summarize task construction settings and application of meta-learning for various

Original: NLP problems and review the development of meta-learning in NLP community.
Revised: NLP problems and review the development of meta-learning in NLP community.

Original:  Introduction
Revised:  Introduction

Original: Recently, deep learning (DL) based natural language processing (NLP) has been one of the research mainstreams and yields signiﬁcant performance improvement in many NLP problems.
Revised: Recently, deep learning (DL) based natural language processing (NLP) has been one of the research mainstreams and yields signiﬁcant performance improvement in many NLP problems.

Original: However, DL models are data-hungry.
Revised: However, DL models are data-hungry.

Original: The downside limits such models’ application to different domains, languages, countries, or styles because collecting in-genre data for model training are costly.
Revised: The downside limits such models’ application to different domains, languages, countries, or styles because collecting in-genre data for model training are costly.

Original: To address the challenges, meta-learning techniques are gaining attention.
Revised: To address the challenges, meta-learning techniques are gaining attention.

Original: Meta-learning, or

Original:  Learning to Learn, aims to learn better learning algorithms, including better parameter initialization (Finn et al., 2017), optimization strategy (Ravi and Larochelle, 2017; Andrychowicz et al., 2016), network architecture (Zoph and Le, 2017; Zoph et al., 2018; Pham et al., 2018a), distance metrics (Gao et al., 2019; Vinyals et al., 2016; Sung et al., 2018), and beyond (Mishra et al., 2018).
Revised: Meta-learning, or Learning to Learn, aims to learn better learning algorithms, including better parameter initialization (Finn et al., 2017), optimization strategy (Andrychowicz et al., 2016; Ravi and Larochelle, 2017), network architecture (Zoph and Le, 2017; Zoph et al., 2018; Pham et al., 2018a), distance metrics (Vinyals et al., 2016; Gao et al., 2019a;

Revised: Sung et al., 2018), and beyond (Mishra et al., 2018).

Original: Metalearning allows faster ﬁne-tuning, converges to better performance, yields more generalizable models, and  achieves outstanding results for few-shot learning in many applications.
Revised: Meta-learning allows faster ﬁne-tuning, converges to better performance, yields more generalizable models, and it achieves outstanding results for fewshot image classiﬁcaition (Triantaﬁllou et al., 2020).

Original: The beneﬁts alleviate the dependency of learning algorithms on labels and make model development more scalable.
Revised: The beneﬁts alleviate the dependency of learning algorithms on labels and make model development more scalable.

Original: Image processing is one of the machine learning areas with abundant applications and established most of the examples in the previous survey papers of meta-learning (Huisman et al., 2021; Hospedales et al., 2021).
Revised: Image processing is one of the machine learning areas with abundant applications and established most of the examples in the previous survey papers on meta-learning (Hospedales et al., 2021; Huisman et al., 2021).

Original: On the other hand, there are works showing beneﬁts of meta-learning techniques in performance and data efﬁciency via applying meta-learning to NLP problems.
Revised: On the other hand, there are works showing beneﬁts of meta-learning techniques in performance and data efﬁciency via applying meta-learning to NLP problems.

Original: Please refer to Tables2 and 3  in the appendix for NLP applications improved by meta-learning.
Revised: Please refer to Tables 3 and 4 in the appendix for NLP applications improved by meta-learning.

Original: Tutorial
Revised: Tutorial

Original: (Lee et al., 2021b) and workshop (Lee et al., 2021a) were organized at ACL 2021 to encourage communication and collaboration among NLP researcherswho are interested in the techniques.
Revised: (Lee et al., 2021b) and Workshop (Lee et al., 2021a) are organized at ACL 2021 to encourage exchange and collaboration among NLP researchers interested in these techniques.

Original: However, to our best knowledge, there is no survey paper for the applications of  meta-learning  to NLP.
Revised: There is a survey paper on metalearning in NLP (Yin, 2020). While Yin (2020) describes meta-learning methods in general, this paper focuses on the idea of making meta-learning successful when applied to NLP and provides a broader review of publications on NLP meta-learning. Due to space constraints, we will not give too many detailed descriptions of general meta-learning techniques in this survey paper. For general concepts of meta-learning, we encourage readers to read the previous overview paper (Yin, 2020; Hospedales et al., 2021; Huisman et al., 2021).

Original: To facilitate more NLP researchers and practitioners beneﬁting from the advance of meta-learning and participating in the area, we provide a systematic survey of meta-learning to NLP problems in this paper.
Revised: To facilitate more NLP researchers and practitioners beneﬁting from the advance of meta-learning and participating in the area, we provide a systematic survey of meta-learning to NLP problems in this paper.

Original: The survey paper is organized as below.
Revised: The survey paper is organized as below.

Original: a) A brief introduction of meta-learning backgrounds, general concepts, and algorithmsin Section 2.
Revised: • A brief introduction of meta-learning back- grounds, general concepts, and algorithms

Original: b)
Revised: in Section 2.

Original:  Common settings for constructing meta-learning tasks in Section 3.
Revised: • Common settings for constructing metalearning tasks in Section 3.

Original: c) Adaptation of general metalearning approaches to NLP problems in Section 4.
Revised: • Adaptation of general meta-learning approaches to NLP problems in Section 4.

Original: d) Meta-learning approaches for special topics including knowledge distillation and life-long learning for NLP applications in Section 5.
Revised: • Meta-learning approaches for special topics including knowledge distillation and life-long learning for NLP applications in Section 5.

Original: This survey paper intends to provide researchers with pointers to recent meta-learning breakthroughs in the NLP community, help them understand the techniques, and inspire more innovation.[Table]
Revised: This survey paper intends to provide researchers with pointers to recent meta-learning breakthroughs in the NLP community, help them understand the techniques, and inspire more innovation.

Original:  Background Knowledge for Meta Learning
Revised:  Background Knowledge for Meta Learning

Original: The goal of machine learning (ML) is to ﬁnd a function f θ ( x ) parametrized by model parameters θ for inference from training data.
Revised: The goal of machine learning (ML) is to ﬁnd a function f θ ( x ) parametrized by model parameters θ for inference from training data.

Original: For machine translation (MT), the input x is a sentnce, while f θ ( x ) is the translation of x ; for automatic speech recognitoin (ASR), x is an uttearnce, while f θ ( x ) is the transcription; In DL, θ are the network paramters, or weights and biases of a network.
Revised: For machine translation (MT), the input x is a sentence, while f θ ( x ) is the translation of x ; for automatic speech recognitoin (ASR), x is an utterance, while f θ ( x ) is the transcription; In DL, θ are the network parameters, or weights and biases of a network.

Original: To learn θ , there is a loss function l ( θ ; D ) , where D is a set of paired examples for training,
Revised: To learn θ , there is a loss function l ( θ ; D ) , where D is a set of paired examples for training,

Original: [Equation]
Revised: [Equation]

Original: where x k is function input,ˆ y k is the ground truth, and K is the number of examples in D .
Revised: where x k is function input, y k is the ground truth, and K is the number of examples in D .

Original: The loss function l ( θ ; D ) is deﬁned as below:
Revised: The loss function l ( θ ; D ) is deﬁned as below:

Original: [Equation]
Revised: [Equation]

Original: where d ( f θ ( x k ) ,ˆ y k ) is the “distance” between the function output f θ ( x k ) and the ground truthˆ y k .
Revised: where d ( f θ ( x k ) , y k ) is the “distance” between the function output f θ ( x k ) and the ground truth y k .

Original: For classiﬁcation problem, d ( ., . ) can be cross-entropy; for regression, it can be L1/L2 distance.
Revised: For classiﬁcation problem, d ( ., . ) can be cross-entropy; for regression, it can be L1/L2 distance.

Original: The following optimization problem is solved to ﬁnd the optimal parameter set θ ∗ for inference via minimizing the loss function l ( θ ; D ) .
Revised: The following optimization problem is solved to ﬁnd the optimal parameter set θ ∗ for inference via minimizing the loss function l ( θ ; D ) .

Original: [Equation]
Revised: [Equation]

Original: In meta-learning, what we want to learn is a learning algorithm.
Revised: In meta-learning, what we want to learn is a learning algorithm.

Original: The learning algorithm can also be considered as a function, denoted as F φ ( . ) .
Revised: The learning algorithm can also be considered as a function, denoted as F φ ( . ) .

Original: The input of F φ ( . ) is the training data, while the output of the function F φ ( . ) is the learned model parameters, or θ ∗ in (3).
Revised: The input of F φ ( . ) is the training data, while the output of the function F φ ( . ) is the learned model parameters, or θ ∗ in (3).

Original: The learning algorithm F φ ( . ) is parameterized by meta-parameters φ , which is what we want to learn in meta-learning.
Revised: The learning algorithm F φ ( . ) is parameterized by meta-parameters φ , which is what we want to learn in meta-learning.

Original: If F φ ( . ) represents gradient descent for deep network, φ can be initial parameters, learning rate, network architecture, etc.
Revised: If F φ ( . ) represents gradient descent for deep network, φ can be initial parameters, learning rate, network architecture, etc.

Original: Different meta-learning approaches focus on learning different components.
Revised: Different meta-learning approaches focus on learning different components.

Original: For example, model-agnostic meta-learning (MAML) focuses on learning initial parameters (Finn et al., 2017), which will be further descried in Section 4.1.
Revised: For example, model-agnostic meta-learning (MAML) focuses on learning initial parameters (Finn et al., 2017), which will be further descried in Section 4.1.

Original: Learning to
Revised: Learning to

Original: Compare methods like Prototypical Network (Snell et al., 2017) in Section 4.2 learn the latent representation of the inputs and their distance metrics for comparison.
Revised: Compare methods like Prototypical Network (Snell et al., 2017) in Section 4.2 learn the latent representation of the inputs and their distance metrics for comparison.

Original: Network architecture search (NAS) in
Revised: Network architecture search (NAS) in

Original: Section 4.3 learns the network architecture (Zoph and Le, 2017; Zoph et al., 2018; Pham et al., 2018a).
Revised: Section 4.3 learns the network architecture (Zoph and Le, 2017; Zoph et al., 2018; Pham et al., 2018a).

Original: To learn meta-parameters φ , meta-training tasks
Revised: To learn meta-parameters φ , meta-training tasks

Original: T train are required.
Revised: T train are required.

Original: [Equation]
Revised: [Equation]

Original: where T n is a task, and N is the number of tasks in T train .
Revised: where T n is a task, and N is the number of tasks in T train .

Original: Usually, all the tasks belong to the same
Revised: Usually, all the tasks belong to the same

Original: NLP problem; for example, all the T n are QA but from different corpora, but it is also possible that the tasks belong to various problems.
Revised: NLP problem; for example, all the T n are QA but from different corpora, but it is also possible that the tasks belong to various problems.

Original: Each task T n includes a support set S n and a query set Q n .
Revised: Each task T n includes a support set S n and a query set Q n .

Original: Both
Revised: Both

Original: S n and Q n are paired examples as D in (1).
Revised: S n and Q n are paired examples as D in (1).

Original: The support set plays the role of training data in typical
Revised: The support set plays the role of training data in typical

Original: ML, while the query set can be understood as the testing data in typical ML.
Revised: ML, while the query set can be understood as the testing data in typical ML.

Original: However, to not confuse the reader, we use the terms support and query sets in the context of meta-learning instead of training and testing sets.
Revised: However, to not confuse the reader, we use the terms support and query sets in the context of meta-learning instead of training and testing sets.

Original: In meta-learning, there is a loss function
Revised: In meta-learning, there is a loss function

Original: L ( φ ; T train ) , which represents how “bad” a learning algorihtm paramereized by φ is on T train .
Revised: L ( φ ; T train ) , which represents how “bad” a learning algorihtm paramereized by φ is on T train .

Original: L ( φ ; T train ) is the performance over all the tasks in
Revised: L ( φ ; T train ) is the performance over all the tasks in

Original: T train ,
Revised: T train ,

Original: [Equation]
Revised: [Equation]

Original: The deﬁnition of the function l ( . ) above is the same as in (2).
Revised: The deﬁnition of the function l ( . ) above is the same as in (2).

Original: l ( θ n ; Q n ) for each task T n is obtained as below.
Revised: l ( θ n ; Q n ) for each task T n is obtained as below.

Original: For each task T n in T train , we use a support set S n to learn a model by the learning algorihtm F φ .
Revised: For each task T n in T train , we use a support set S n to learn a model by the learning algorihtm F φ .

Original: The learned model is denoted as θ n , where θ n = F φ ( S n ) .
Revised: The learned model is denoted as θ n , where θ n = F φ ( S n ) .

Original: This procedure is equivalent to typical ML training.
Revised: This procedure is equivalent to typical ML training.

Original: We called this step withintask training .
Revised: We called this step withintask training .

Original: Then θ n is evaluated on Q n to obtaing l ( θ n ; Q n ) in (5).
Revised: Then θ n is evaluated on Q n to obtain l ( θ n ; Q n ) in (5).

Original: We called this step within-task testing .
Revised: We called this step within-task testing .

Original: One execution of within-task training and followed by one execution of within-task testing is called an episode .
Revised: One execution of within-task training and followed by one execution of within-task testing is called an episode .

Original: The optimization task below is solved to learn meta-parameteres φ .
Revised: The optimization task below is solved to learn meta-parameteres φ .

Original: [Equation]
Revised: [Equation]

Original: If φ is differentiable with respect to L ( φ ; T train ) , then we can use gradient descent to learn metaparameters; if not, we can use reinforcement learning algorithm or evolutionary algorithm.
Revised: If φ is differentiable with respect to L ( φ ; T train ) , then we can use gradient descent to learn metaparameters; if not, we can use reinforcement learning algorithm or evolutionary algorithm.

Original: Solving (6) is called across-task training in this paper, which usually involves running many episodes on metatraining tasks.
Revised: Solving (6) is called cross-task training in this paper, which usually involves running many episodes on metatraining tasks.

Original: To evaluate φ ∗ , we need meta-testing tasks T test , tasks for evaluating algorithms parameterized by meta-parameters φ ∗ 1 .
Revised: To evaluate φ ∗ , we need meta-testing tasks T test , tasks for evaluating algorithms parameterized by meta-parameters φ ∗ 1 .

Original: We do across-task testing on T test , that is, running an episode on each meta-testing task to evaluate algorithms parameterized by meta-parameters φ ∗ .
Revised: We do cross-task testing on T test , that is, running an episode on each meta-testing task to evaluate algorithms parameterized by meta-parameters φ ∗ .

Original: In order to faciliate the reading of our paper, we summarize the most important terminologies and their meanings in Table 1 .
Revised: In order to faciliate the reading of our paper, we summarize the most important terminologies and their meanings in Table 1 .

Original:  Task Construction
Revised:  Task Construction

Original: In this section, we discuss different settings of constructing meta-training tasks T train and metatesting tasks T test 
Revised: In this section, we discuss different settings of constructing meta-training tasks T train and metatesting tasks T test .

Original: . 3. Cross-domain Transfer
Revised: 3. Cross-domain Transfer

Original: A typical setting for constructing the tasks is based on domains (Yan et al., 2020;Li et al., 2020a;
Revised: A typical setting for constructing the tasks is based on domains (Qian and Yu, 2019; Yan et al., 2020;

Original:  Park et al., 2021; Chen et al.,  2020a; Hua; Wang et al., 2021b; Dingliwal et al., 2021; Qian and Yu, 2019;
Revised: Li et al., 2020a; Park et al., 2021; Chen et al., 2020b; Huang et al., 2020a; Dai et al., 2020; Wang et al., 2021b;

Original: Qian et al., 2021; Dai et al., 2020).
Revised: Dingliwal et al., 2021; Qian et al., 2021).

Original: In this setting, all the tasks, no matter belonging to T train or T test , are the same NLP problems.
Revised: In this setting, all the tasks, no matter belonging to T train or T test , are the same NLP problems.

Original: In each task T n , the support set S n and the query set Q n are from the same domain, while different tasks contain the examples from different domains.
Revised: In each task T n , the support set S n and the query set Q n are from the same domain, while different tasks contain the examples from different domains.

Original: In each task, the model is trained on the support set of a domain (usually having a small size) and evaluated on the query set in the same domain, which can be considered as domain adaptation .
Revised: In each task, the model is trained on the support set of a domain (usually having a small size) and evaluated on the query set in the same domain, which can be considered as domain adaptation .

Original: From the meta-training tasks T train , across-task training ﬁnds meta-parameters φ ∗ parameterizing the learning algorithm F φ ∗ .
Revised: From the meta-training tasks T train , cross-task training ﬁnds meta-parameters φ ∗ parameterizing the learning algorithm F φ ∗ .

Original: With a sufﬁcient number of tasks in T train , across-task training should ﬁnd a suitable φ ∗ for a wide range of domains, and thus also works well on the tasks in T test containing the domains unseen during acrosstask training.
Revised: With a sufﬁcient number of tasks in T train , cross-task training should ﬁnd a suitable φ ∗ for a wide range of domains, and thus also works well on the tasks in T test containing the domains unseen during cross-task training.

Original: Hence, meta-learning can be considered as one way to improve domain adaptation .
Revised: Hence, metalearning can be considered as one way to improve domain adaptation .

Original: If the support set in each task includes only a few examples, the meta-learning has to ﬁnd the metaparameters φ ∗ that can learn from a small support set and generalize well to the query set in the same domain.
Revised: If the support set in each task includes only a few examples, the meta-learning has to ﬁnd the meta-parameters φ ∗ that can learn from a small support set and generalize well to the query set in the same domain.

Original: Therefore, meta-learning is considered one way to achieve few-shot learning .
Revised: Therefore, meta-learning is considered one way to achieve few-shot learning .

Original: The cross-domain setting is widespread.
Revised: The cross-domain setting is widespread.

Original: We only provide a few examples in this subsection.
Revised: We only provide a few examples in this subsection.

Original: In MT, each meta-training task includes the documents from a speciﬁc domain (e.g., news, laws, etc.), while each meta-testing task also contains documents from one domain but not covered by the meta-training tasks (e.g., medical records) (Li et al., 2020a).
Revised: In MT, each meta-training task includes the documents from a speciﬁc domain (e.g., news, laws, etc.), while each meta-testing task also contains documents from one domain but not covered by the meta-training tasks (e.g., medical records) (Li et al., 2020a).

Original: For another example, both meta-training and meta-testing tasks are DST.
Revised: For another example, both meta-training and meta-testing tasks are DST.

Original: The meta-training tasks include hotel booking, ﬂight ticket booking, etc., while the testing task is taxi booking (Hua;
Revised: The meta-training tasks include hotel booking, ﬂight ticket booking, etc., while the testing task is taxi booking (Huang et al., 2020a;

Original: Wang et al., 2021b; Dingliwal et al., 2021).
Revised: Wang et al., 2021b; Dingliwal et al., 2021).

Original: Domain has different meanings in different NLP problems.
Revised: Domain has different meanings in different NLP problems.

Original: For example, in speech processing tasks, the domains can refer to accents (Winata et al., 2020b;
Revised: For example, in speech processing tasks, the domains can refer to accents (Winata et al., 2020b;

Original: Huang et al., 2021a) or speakers (Klejch et al., 2019; Wu et al., 2021b).
Revised: Huang et al., 2021) or speakers (Klejch et al., 2019; Wu et al., 2021b; Huang et al., 2022).

Original: 3. Cross-lingual Transfer
Revised: 3. Cross-lingual Transfer

Original: If we consider different languages as different domains, then the cross-lingual transfer can be regarded as a special case of cross-domain transfer.
Revised: If we consider different languages as different domains, then the cross-lingual transfer can be regarded as a special case of cross-domain transfer.

Original: Suppose each task contains the examples of an NLP problem from one language, and different tasks are in different languages.
Revised: Suppose each task contains the examples of an NLP problem from one language, and different tasks are in different languages.

Original: In this case, across-task training ﬁnds meta-parameters φ ∗ from the languages in T train , and across-task testing evaluate the meta-parameters φ ∗ on new langauges in T test .
Revised: In this case, cross-task training ﬁnds meta-parameters φ ∗ from the languages in T train , and cross-task testing evaluate the metaparameters φ ∗ on new langauges in T test .

Original: This setting aims at ﬁnding the learning algorithm F φ ∗ ( . ) that works well on the NLP problem of any language given the support set of the language.
Revised: This setting aims at ﬁnding the learning algorithm F φ ∗ ( . ) that works well on the NLP problem of any language given the support set of the language.

Original: Cross-language settings have been applied to NLI and QA in X-MAML (Nooralahzadeh et al., 2020), documentation classicaiotn (van der Heijden et al., 2021), dependency parsing (Langedijk et al., 2021),
Revised: Crosslanguage settings have been applied to NLI and QA in X-MAML (Nooralahzadeh et al., 2020), documentation classiﬁcation (van der Heijden et al., 2021), dependency parsing (Langedijk et al., 2021),

Original: MT (Gu et al., 2018), and ASR (Hsu et al., 2020;
Revised: MT (Gu et al., 2018), and ASR (Hsu et al., 2020;

Original: Xiao et al., 2021; Winata et al., 2020a; Chen et al., 2020c).
Revised: Winata et al., 2020a; Chen et al., 2020d; Xiao et al., 2021).

Original: For the meta-learning methods aiming at learning the initial parameters like MAML (will be introduced in Section 4.1), the network architecture used in all tasks must have the same network architecture.
Revised: For the meta-learning methods aiming at learning the initial parameters like MAML (will be introduced in Section 4.1), the network architecture used in all tasks must have the same network architecture.

Original: A uniﬁed network architecture across all tasks is not obvious in cross-lingual learning because the vocabularies in different tasks are different.
Revised: A uniﬁed network architecture across all tasks is not obvious in cross-lingual learning because the vocabularies in different tasks are different.

Original: Before multilingual pretrained models are available, uniﬁed word embeddings across languages are required.
Revised: Before multilingual pretrained models are available, uniﬁed word embeddings across languages are required.

Original: Gu et al.
Revised: Gu et al.

Original: (2018) uses the universal lexical representation to overcome the input-output mismatch across different languages.
Revised: (2018) uses the universal lexical repre- [Table] sentation to overcome the input-output mismatch across different languages.

Original: Recently, by using multilingual pretrained models as encoders, such as
Revised: Recently, by using multilingual pretrained models as encoders, such as

Original: M-BERT (Devlin et al., 2019) or XLM-R (Conneau et al., 2020), all languages can share the same network architecture (Nooralahzadeh et al., 2020; van der Heijden et al., 2021).
Revised: M-BERT (Devlin et al., 2019) or XLM-R (Conneau et al., 2020), all languages can share the same network architecture (Nooralahzadeh et al., 2020; van der Heijden et al., 2021).

Original: 3. Cross-problem Training
Revised: 3. Cross-problem Training

Original: Here the meta-training and meta-testing tasks can come from different problems.
Revised: Here the meta-training and meta-testing tasks can come from different problems.

Original: For example, the meta-training tasks include MT and NLI, while the meta-testing tasks include QA and DST.
Revised: For example, the meta-training tasks include MT and NLI, while the meta-testing tasks include QA and DST.

Original: The cross-problem setting is not usual, but there are still some examples.
Revised: The cross-problem setting is not usual, but there are still some examples.

Original: In (Bansal et al., 2020a), the metatraining tasks are the GLUE benchmark tasks, while the meta-testing tasks are NLP problems, including entity typing, NLI, sentiment classiﬁcation, and various other text classiﬁcation tasks, not in the GLUE.
Revised: In Bansal et al. (2020a) , the metatraining tasks are the GLUE benchmark tasks (Wang et al., 2018), while the meta-testing tasks are NLP problems, including entity typing, NLI, sentiment classiﬁcation, and various other text classiﬁcation tasks, not in the GLUE.

Original: All the meta-training and meta-testing tasks can be formulated as classiﬁcation but with different classes.
Revised: All the meta-training and meta-testing tasks can be formulated as classiﬁcation but with different classes.

Original: In (Indurthi et al., 2020), the meta-training tasks are MT and ASR, while the meta-testing task is speech translation (ST).
Revised: In Indurthi et al. (2020), the meta-training tasks are MT and ASR, while the meta-testing task is speech translation (ST).

Revised: CrossFit is a benchmark corpus for this crossproblem setting (Ye et al., 2021).

Original: The intrinsic challenge in the cross-problem setting is that different NLP problems may need very different meta-parameters in learning algorithms, so it may be challenging to ﬁnd uniﬁed metaparameters on the meta-training tasks that can generalize to meta-testing tasks.
Revised: The intrinsic challenge in the cross-problem setting is that different NLP problems may need very different meta-parameters in learning algorithms, so it may be challenging to ﬁnd uniﬁed metaparameters on the meta-training tasks that can generalize to meta-testing tasks.

Original: In addition, the metalearning algorithms learning initial parameters such as MAML require all the tasks to have a uniﬁed network architecture.
Revised: In addition, the metalearning algorithms learning initial parameters such as MAML require all the tasks to have a uniﬁed network architecture.

Original: If different problems need different network architecture, then the original
Revised: If different problems need different network architecture, then the original

Original: MAML cannot be used in the cross-problem setting.
Revised: MAML cannot be used in the cross-problem setting.

Original: LEOPARD (Bansal et al., 2020a) and ProtoMAML (van der Heijden et al., 2021) are the
Revised: LEOPARD (Bansal et al., 2020a) and ProtoMAML (van der Heijden et al., 2021) are the

Original: MAML variants that can be used in the classiﬁcation tasks with different class numbers.
Revised: MAML variants that can be used in the classiﬁcation tasks with different class numbers.

Original: Both approaches use the data of a class to generate the class-speciﬁc head, so only the parameters of the head parameter generation model are required.
Revised: Both approaches use the data of a class to generate the class-speciﬁc head, so only the parameters of the head parameter generation model are required.

Original: The head parameter generation model is shared across all classes, so the network architecture becomes class-number agnostic.
Revised: The head parameter generation model is shared across all classes, so the network architecture becomes class-number agnostic.

Original: On the other hand, recently, universal models for a wide range of NLP problems have been emgered (Raffel et al., 2019;
Revised: On the other hand, recently, universal models for a wide range of NLP problems have been emgered (Raffel et al., 2019;

Original: Chen et al., 2021; Ao et al., 2021).
Revised: Chen et al., 2021; Ao et al., 2021).

Original: We believe the development of the universal models will intrigue the cross-problem setting in meta-learning.
Revised: We believe the development of the universal models will intrigue the cross-problem setting in meta-learning.

Original: 3. Domain Generalization
Revised: 3. Domain Generalization

Original: Traditional supervised learning assumes that the training and testing data have the same distribution.
Revised: Traditional supervised learning assumes that the training and testing data have the same distribution.

Original: Domain shift refers to the problem that a model performs poorly when training data and testing data have very different statistics.
Revised: Domain shift refers to the problem that a model performs poorly when training data and testing data have very different statistics.

Original: To solve the issue, domain adaptation  uses little labeled data to adapt the model, which has been described in Section 3.1.
Revised: Domain adaptation in Section 3.1 uses little domain-speciﬁc data to adapt the model 2 .

Original: On the other hand, domain generalization techniques attempt to alleviate the domain mismatch issue by producing models which by design generalize well to novel testing domains.
Revised: On the other hand, domain generalization techniques attempt to alleviate the domain mismatch issue by producing models that generalize well to novel testing domains.

Original: Meta-learning can also be used to realize domain generalization by learning an algorithm that can train from one domain but evaluate on the other.
Revised: Meta-learning can also be used to realize domain generalization by learning an algorithm that can train from one domain but evaluate on the other.

Original: To simulate the domain generalization scenario, a set of meta-training tasks are constructed by sampling data from different domains as the support and query sets.
Revised: To simulate the domain generalization scenario, a set of meta-training tasks are constructed by sampling data from different domains as the support and query sets.

Original: With the meta-training tasks above, across-task training will ﬁnd the meta-parameters φ ∗ that work well on the scenario where the training (support) and testing (query) examples are from different domains.
Revised: With the meta-training tasks above, cross-task training will ﬁnd the meta-parameters φ ∗ that work well on the scenario where the training (support) and testing (query) examples are from different domains.

Original: Fig.
Revised: Fig.

Original: 1 shows how to construct tasks for domain generalization and compare the construction with the cross-domain transfer setting.
Revised: 1 shows how to construct tasks for domain generalization and compare the construction with the cross-domain transfer setting.

Original: The setting has been used to improve the domain generalization for semantic parsing (Wang et al., 2021a) and language generalization 2 for sentiment classiﬁcation and relevance classiﬁcation (Li et al., 2020c).
Revised: The setting has been used to improve the domain generalization for semantic parsing (Wang et al., 2021a) and language generalization 3 for sentiment classiﬁcation and relevance classiﬁcation (Li et al., 2020c).

Original: 3. Task Augmentation
Revised: 3. Task Augmentation

Original: In meta-learning, it is critical to have a large number of diverse tasks in the meta-training tasks T train to ﬁnd a set of meta-parameters φ ∗ that can generalize well to the meta-testing tasks.
Revised: In meta-learning, it is critical to have a large number of diverse tasks in the meta-training tasks T train to ﬁnd a set of meta-parameters φ ∗ that can generalize well to the meta-testing tasks.

Original: However, considering the setting in the previous subsections, different tasks contain examples in various domains, language, or even NLP problems, so a large and diverse T train are often not available.
Revised: However, considering the setting in the previous subsections, different tasks contain examples in various domains, language, or even NLP problems, so a large and diverse T train are often not available.

Original: In typical
Revised: In typical

Original: ML, data augmentation comes in handy when data is lacking.
Revised: ML, data augmentation comes in handy when data is lacking.

Original: In meta-learning, augmenting tasks is similarly understood as data augmentation in ML.
Revised: In meta-learning, augmenting tasks is similarly understood as data augmentation in ML.

Original: Data augmentation becomes task augmentation because the “training examples” in meta-learning are a collection of tasks.
Revised: Data augmentation becomes task augmentation because the “training examples” in meta-learning are a collection of tasks.

Original: Task augmentation approaches in meta-learning can be categorized into two main directions: a) Inventing more tasks (without human labeling efforts) to increase the number and diversity of the meta-training tasks T train .
Revised: Task augmentation approaches in meta-learning can be categorized into two main directions: a) Inventing more tasks (without human labeling efforts) to increase the number and diversity of the meta-training tasks T train .

Original: b) Splitting training data from one single dataset into homogenous partitions that allow applying meta-learning techniques and therefore improve the performance.
Revised: b) Splitting training data from one single dataset into homogenous partitions that allow applying meta-learning techniques and therefore improve the performance.

Original: NLP-speciﬁc methods have been proposed in both categories.
Revised: NLP-speciﬁc methods have been proposed in both categories.

Original: Inventing more tasks
Revised: Inventing more tasks

Original: The main question is how to construct a massive amount of tasks efﬁciently.
Revised: The main question is how to construct a massive amount of tasks efﬁciently.

Original: There is already some general task augmentation approahces proposed for general meta-learning (Yao et al., 2021a; Ni et al., 2021; Rajendran et al., 2020;
Revised: There is already some general task augmentation approahces proposed for general meta-learning (Yao et al., 2021a; Ni et al., 2021; Rajendran et al., 2020;

Original: Yao et al., 2021b).
Revised: Yao et al., 2021b).

Original: Here we only focus on
Revised: Here we only focus on

Original: NLP-speciﬁc approaches.
Revised: NLP-speciﬁc approaches.

Original: Inspired from the selfsupervised learning, Bansal et al.
Revised: Inspired from the self- supervised learning, Bansal et al.

Original: (2020b) generates a large number of cloze tasks, which can be considered as multi-class classiﬁcation tasks but obtained without labeling effort, to augment the meta-training tasks.
Revised: (2020b) generates a large number of cloze tasks, which can be considered as multi-class classiﬁcation tasks but obtained without labeling effort, to augment the meta-training tasks.

Original:  The self-supervised generated tasks improve the performance on the meta-testing tasks which are classiﬁcation problems.
Revised: Bansal et al. (2021) further explores the inﬂuence of unsupervised task distribution and creates task distributions that are inductive to better meta-training efﬁcacy. The self-supervised generated tasks improve the performance on a wide range of different meta-testing tasks which are classiﬁcation problems (Bansal et al., 2020b), and it even performs comparably with supervised metalearning methods on FewRel 2.0 benchmark (Gao et al., 2019b) on 5-shot evaluation (Bansal et al., 2021).

Original: Generating tasks from a monolithic corpus
Revised: Generating tasks from a monolithic corpus

Original: Many tasks can be constructed with one monolithic corpus (Huang et al.
Revised: Many tasks can be constructed with one monolithic corpus (Huang et al.

Original: ; Guo et al.
Revised: ; Guo et al.

Original: Murty et al.
Revised: ; Wu et al.

Original: ; Li et al.
Revised: ; Jiang et al.

Original: (2020b); Pasunuru and Bansal (2020);
Revised: (2019); Chien and Lieow (2019)

Original: Wang et al.
Revised: ; Li et al.

Original: (2020a)
Revised: (2020b)

Original: ; Jiang et al.

Original: ; MacLaughlin et al.
Revised: ; MacLaughlin et al.

Original: ; Wu et al.
Revised: Wang et al.

Original: Chien and Lieow (2019); Xu et al.
Revised: (2020a); Pasunuru and Bansal (2020); Xu et al.

Original: (2021a)).
Revised: (2021a)

Original:  First, the training set of the corpus is split into support partition, D s , and query partition, D q .
Revised: ; Murty et al. First, the training set of the corpus is split into support partition, D s , and query partition, D q .

Original: Two subsets of examples are sampled from D s and D q as the support set, S , and query set, Q , respectively.
Revised: Two subsets of examples are sampled from D s and D q as the support set, S , and query set, Q , respectively.

Original: In each episode, model parameters θ are updated with
Revised: In each episode, model parameters θ are updated with

Original: S , and then the losses are computed with the updated model and Q .
Revised: S , and then the losses are computed with the updated model and Q .

Original: The meta-parameters φ are then updated based on the losses, as the meta-learning framework introduced in Section 2.
Revised: The meta-parameters φ are then updated based on the losses, as the meta-learning framework introduced in Section 2.

Original: The test set of the corpus is used to build T test for evaluation.
Revised: The test set of the corpus is used to build T test for evaluation.

Original: As compared to constructing T train from multiple relevant corpora, which are often not available, building T train with one corpus makes meta-learning methodology more applicable.
Revised: As compared to constructing T train from multiple relevant corpora, which are often not available, building T train with one corpus makes meta-learning methodology more applicable.

Original: Besides, results obtained from one corpus are more comparable with existing NLP studies.
Revised: Besides, results obtained from one corpus are more comparable with existing NLP studies.

Original: However, only using a single data stream makes the resulting models less generalizable to various attributes such as domains and languages.
Revised: However, only using a single data stream makes the resulting models less generalizable to various attributes such as domains and languages.

Original: How to sample the data points to form a taskis the key in such category.
Revised: How to sample the data points to form a taskis the key in such category.

Original: In NAS research in
Revised: In NAS research in

Original: Section 4.3, the support and query sets are usually randomly sampled.
Revised: Section 4.3, the support and query sets are usually randomly sampled.

Original: Learning to Compare in Section 4.2 splits the data points of different classes in different tasks based on some predeﬁned criteria.
Revised: Learning to Compare in Section 4.2 splits the data points of different classes in different tasks based on some predeﬁned criteria.

Original: There are some NLP-speciﬁc ways to construct the tasks.
Revised: There are some NLP-speciﬁc ways to construct the tasks.

Original: In (Huang et al., 2018), a relevance function is designed to sample the support set S based on its relevance to the query set Q .
Revised: In Huang et al. (2018), a relevance function is designed to sample the support set S based on its relevance to the query set Q .

Original: In (Guo et al., 2019), a retrieval model is used to retrieve the support set
Revised: In Guo et al. (2019), a retrieval model is used to retrieve the support set

Original: S from the whole dataset.
Revised: S from the whole dataset.

Original: DReCa (Murty et al., 2021) applies clustering on BERT representations to create tasks.
Revised: DReCa (Murty et al., 2021) applies clustering on BERT representations to create tasks.

Original:  Meta-Learning for NLP Tasks
Revised:  Meta-Learning for NLP Tasks

Original: This section shows the most popular meta-learning methods for NLP and how they ﬁt into NLP tasks.
Revised: This section shows the most popular meta-learning methods for NLP and how they ﬁt into NLP tasks.

Original: Due to space limitations, only the major trends are mentioned.
Revised: Due to space limitations, only the major trends are mentioned.

Original: Please refer to Table2 and 3  in the appendix for a complete survey.
Revised: Please refer to Table 3 and 4 in the appendix for a complete survey.

Original: 4. Learning to Initialize
Revised: 4. Learning to Initialize

Original: In typical DL, gradient descent is widely used to solve (3).
Revised: In typical DL, gradient descent is widely used to solve (3).

Original: Gradient descent starts from a set of initial parameters θ 0 , and then the parameters θ are updated iteratively according to the directions of the gradient.
Revised: Gradient descent starts from a set of initial parameters θ 0 , and then the parameters θ are updated iteratively according to the directions of the gradient.

Original: There is a series of meta-learning approaches targeting at learning the initial parameters θ 0 .
Revised: There is a series of meta-learning approaches targeting at learning the initial parameters θ 0 .

Original: In these learn-to-init approaches, the metaparameters φ to be learned are the initial parameters θ 0 for gradient descent, or φ = θ 0 .
Revised: In these learn-to-init approaches, the metaparameters φ to be learned are the initial parameters θ 0 for gradient descent, or φ = θ 0 .

Original: MAML (Finn et al., 2017) and its ﬁrst-order approximation, FOMAML (Finn et al., 2017), Reptile (Nichol et al., 2018), etc., are the representative approaches of learn-to-init.
Revised: MAML (Finn et al., 2017) and its ﬁrst-order approximation, FOMAML (Finn et al., 2017), Reptile (Nichol et al., 2018), etc., are the representative approaches of learn-to-init.

Original: We surveyed a large number of papers using MAML-based approaches to NLP applications in the last three years and summarized them in Table 4 in the appendix.
Revised: We surveyed a large number of papers using MAML-based approaches to NLP applications in the last three years and summarized them in Table 5 in the appendix.

Original: Learning to Initialize v.s. Self-supervised Learning
Revised: Learning to Initialize v.s. Self-supervised Learning

Original: The learn-to-init approaches aim at learning a set of good initial parameters.
Revised: The learn-to-init approaches aim at learning a set of good initial parameters.

Original: On the other hand, self-supervised approaches like BERT also have the same target.
Revised: On the other hand, self-supervised approaches like BERT also have the same target.

Original: There is a natural question: are they complementary?
Revised: There is a natural question: are they complementary?

Original: Based on the survey, it is common to use the self-supervised models to “initialize” the meta-parameters φ in learn-to-init approaches.
Revised: Based on the survey in Table 5 in the appendix, it is common to use the self-supervised models to “initialize” the metaparameters φ in learn-to-init approaches.

Original: To ﬁnd the optimal φ ∗ in (5), gradient descent is used as well, and thus the “initial parameters for initial parameters”, or φ 0 is required.
Revised: To ﬁnd the optimal φ ∗ in (5), gradient descent is used as well, and thus the “initial parameters for initial parameters”, or φ 0 is required.

Original: A self-supervised model usually serves the role of φ 0 , and the learnto-init approaches further update φ 0 to ﬁnd φ ∗ .
Revised: A self-supervised model usually serves the role of φ 0 , and the learn-to-init approaches further update φ 0 to ﬁnd φ ∗ .

Original: Learn-to-init and self-supervised learning are complementary.
Revised: Learn-to-init and self-supervised learning are complementary.

Original: The self-supervised objectives are different from the objective of the target NLP problem, so there is a “learning gap”.
Revised: The self-supervised objectives are different from the objective of the target NLP problem, so there is a “learning gap”.

Original: On the other hand, learn-to-init approaches learn to achieve good performance on the query sets of the meta-training tasks, so it directly optimizes the objective of the
Revised: On the other hand, learn-to-init approaches learn to achieve good performance on the query sets of the meta-training tasks, so it directly optimizes the objective of the

Original: NLP problems.
Revised: NLP problems.

Original: The beneﬁt of self-supervised learning is that it does not require labeled data, while labeling is still needed to prepare the examples in meta-training tasks.
Revised: The beneﬁt of self-supervised learning is that it does not require labeled data, while labeling is still needed to prepare the examples in meta-training tasks.

Original: Learning to Initialize v.s. Multi-task Learning
Revised: Learning to Initialize v

Revised: s. Multi-task Learning

Original: Multi-task learning is another way to initialize model parameters, which usually serves as the baseline of learn-to-init in the literature.
Revised: Multi-task learning is another way to initialize model parameters, which usually serves as the baseline of learn-to-init in the literature.

Original: In multi-task learning,people put all the labeled data from the meta-training tasks  together to train a model.
Revised: In multi-task learning, all the labelled data from the meta-training tasks is put together to train a model.

Original: That is, all the support sets S n and query sets Q n in the meta-training tasks T train are put together as a training set D , and the loss (3) is optimized to ﬁnd a parameter θ ∗ .
Revised: That is, all the support sets S n and query sets Q n in the metatraining tasks T train are put together as a training set D , and the loss (3) is optimized to ﬁnd a parameter θ ∗ .

Original: Then θ ∗ is used as initial parameters for the meta-testing tasks.
Revised: Then θ ∗ is used as initial parameters for the meta-testing tasks.

Original: Both multi-task learning and meta-learning leverage the examples in the meta-training tasks, but with different training criteria.
Revised: Both multi-task learning and meta-learning leverage the examples in the meta-training tasks, but with different training criteria.

Original: Learn-to-init ﬁnds the initial parameters suitable to be updated by updating the model on the support sets and then evaluating it on the query sets.
Revised: Learn-to-init ﬁnds the initial parameters suitable to be updated by updating the model on the support sets and then evaluating it on the query sets.

Original: In contrast, multi-task learning does not consider that the initial parameters would be further updated at all during training.
Revised: In contrast, multi-task learning does not consider that the initial parameters would be further updated at all during training.

Original: Therefore, in terms of performance, learn-to-init is usually shown to be better than multi-task learning (Chen et al., 2020a; Dou et al., 2019).
Revised: Therefore, in terms of performance, learn-to-init is usually shown to be better than multi-task learning (Dou et al., 2019; Chen et al., 2020b).

Original: On the other hand, in terms of training speed, meta-learning, which optimizes (5), is more computationally intensive than multi-task learning optimizing (3).
Revised: On the other hand, in terms of training speed, meta-learning, which optimizes (5), is more computationally intensive than multi-task learning optimizing (3).

Original: Three-stage Initialization
Revised: Three-stage Initialization

Original: Since learn-to-init, multi-task, self-supervised learning all have their pros and cons, they can be integrated to draw on the strong points of each other.
Revised: Since learn-to-init, multi-task, self-supervised learning all have their pros and cons, they can be integrated to draw on the strong points of each other.

Original: A common way to integrate the three approaches is “three-stage initialization” as below.
Revised: A common way to integrate the three approaches is “three-stage initialization” as below.

Original: a) First, initialize a model by self-supervised learning, which leverages unlabeled data.
Revised: a) First, initialize a model by self-supervised learning, which leverages unlabeled data.

Original: Its objective is usually not directly related to the target NLP problem.
Revised: Its objective is usually not directly related to the target NLP problem.

Original: b) Then, multi-task learning is used to ﬁne-tune the self-supervised model.
Revised: b) Then, multi-task learning is used to ﬁne-tune the self-supervised model.

Original: The objective of multi-task learning is the target
Revised: The objective of multi-task learning is the target

Original: NLP problem but does not consider the update procedure in gradient descent.
Revised: NLP problem but does not consider the update procedure in gradient descent.

Original: c) Finally, learn-to-init, which ﬁnds the initial parameters suitable for update, is used to ﬁne-tune the multi-task model.
Revised: c) Finally, learn-to-init, which ﬁnds the initial parameters suitable for update, is used to ﬁne-tune the multi-task model.

Original: Learn-to-init is chosen to be the last stage because its training objective is closest to the target of looking for good initial parameters, but it is the most computationally intensive method, and thus it is only used to change the model a little bit.
Revised: Learn-to-init is chosen to be the last stage because its training objective is closest to the target of looking for good initial parameters, but it is the most computationally intensive method, and thus it is only used to change the model a little bit.

Original: The three-stage initialization has been tested in several works (Langedijk et al., 2021; Nooralahzadeh et al., 2020; Wu et al., 2021b; van der Heijden et al.,  2021), but it does not always improve the performance (Wu et al., 2021b; van der Heijden et al., 2021). 
Revised: The three-stage initialization has been tested in several works (Nooralahzadeh et al., 2020; Wu et al., 2021b; van der Heijden et al., 2021; Langedijk et al., 2021), but it does not always improve the performance (Wu et al., 2021b; van der Heijden et al., 2021). Challenges

Original: 4. Learning to Compare
Revised: Learn-to-init is an essential paradigm for few-shot learning and usually achieves outstanding results in the few-shot learning benchmarks of image classiﬁcation (Triantaﬁllou et al., 2020). However, it has fallen short of yielding state-ofthe-art results on NLP few-shot learning benchmarks (Ye et al., 2021; Chen et al., 2022; Bragg et al., 2021). For example, on the cross-task fewshot learning benchmark, CrossFit, simple multitask learning outperforms existing learn-to-init in many cases (Ye et al., 2021). One possible reason is meta-learning methods are susceptible to hyperparameters and even random seeds (Antoniou et al., 2019).

Revised: Hence, it is difﬁcult to obtain decent performance without exhaustively tuning hyperparameters.

Revised: The research about developing more stable learn-to-init methods may lead to more practical real-world applications for the approaches.

Revised: There is a study about stabilizing the cross-task training of learn-to-init methods by reducing the variance of gradients for NLP (Wang et al., 2021b).

Revised: 4. Learning to Compare

Original: Learning to Compare methods are widely applied to NLP tasks.
Revised: Learning to Compare methods are widely applied to NLP tasks.

Original: Among many others, we ﬁnd applications of Learning to Compare methods in text classiﬁcation (Yu et al., 2018; Tan et al., 2019; Geng et al., 2019; Sun et al., 2019b; Geng et al.), sequence labeling (Hou et al., 2020; Oguz and Vu), semantic relation classiﬁcation (Ye and Ling, 2019; Chen et al., 2019a; Gao et al., 2019;
Revised: Among many others, we ﬁnd applications of Learning to Compare methods in text classiﬁcation (Yu et al., 2018; Tan et al., 2019; Geng et al., 2019; Sun et al., 2019b; Geng et al., 2020), sequence labeling (Hou et al., 2020; Oguz and Vu, 2021), semantic relation classiﬁcation (Ye and Ling, 2019; Chen et al., 2019a; Gao et al., 2019a;

Original: Ren et al.), knowledege completion (Xiong et al.
Revised: Ren et al., 2020), knowledege completion (Xiong et al., 2018;

Original: ; Wang et al.; Zhang et al. ; Sheng et al.) and speech recognition (Lux and Vu, 2021) tasks.
Revised: Wang et al., 2019b; Zhang et al., 2020; Sheng et al., 2020) and speech recognition (Lux and Vu, 2021) tasks.

Original: Most of the proposed methods are based on
Revised: Most of the proposed methods are based on

Original: Matching Network (Vinyals et al., 2016), Prototypical Network (Snell et al., 2017) and Relation Network (Sung et al., 2018), and extend these architectures in two aspects: a) how to embed text input in a vector space with/without context information, and b) how to compute the distance/similarity/relation between two inputs in this space.
Revised: Matching Network (Vinyals et al., 2016), Prototypical Network (Snell et al., 2017) and Relation Network (Sung et al., 2018), and extend these architectures in two aspects: a) how to embed text input in a vector space with/without context information, and b) how to compute the distance/similarity/relation between two inputs in this space.

Original: Since these questions have had deep roots in the computation linguistics research for many years (Sch¨utze, 1992;
Revised: Since these questions have had deep roots in the computation linguistics research for many years (Sch¨utze, 1992;

Original: Manning and Schutze, 1999), Learning to Compare methods is one of the most important methods among other meta-learning methods in the context of NLP despite their simplicity.
Revised: Manning and Schutze, 1999), Learning to Compare methods is one of the most important methods among other meta-learning methods in the context of NLP despite their simplicity.

Revised: Notably, to date, such family of methods is mainly applied to classiﬁcation tasks.

Original: 4. Neural Network Architecture Search
Revised: 4. Neural Network Architecture Search

Original: Neural network architecture search (NAS) is another common meta-learning technique applied to NLP including language modeling (WikiText- (Merity et al., 2017),PTB (Mikolov et al., 2010)),
Revised: Neural network architecture search (NAS) is another common meta-learning technique applied to NLP including language modeling (WikiText103 (Merity et al., 2017),

Original:  NER (CoNLL-2003 (Sang and De Meulder, 2003)), TC (GLUE (Wang et al., 2019)), and MT (WMT’ (Bojar et al., 2014)).
Revised: PTB (Mikolov et al., 2010)), NER (CoNLL-2003 (Sang and De Meulder, 2003)), TC (GLUE (Wang et al., 2019a)), and MT (WMT’14 (Bojar et al., 2014)).

Original: As discussed in Section 3.5, these techniques are often trained/evaluated with a single, matched dataset, which is different from other meta-learning approaches.
Revised: As discussed in Section 3.5, these techniques are often trained/evaluated with a single, matched dataset, which is different from other meta-learning approaches.

Original: Moreover, in constrast to conventional NAS methods that focus on learning the topology in an individual recurrent or convolutional cell, NAS methods have to be redesigned in order to make the search space suitable for NLP problems, where contextual information often plays an important role.
Revised: Moreover, in contrast to conventional NAS methods that focus on learning the topology in an individual recurrent or convolutional cell, NAS methods have to be redesigned in order to make the search space suitable for NLP problems, where contextual information often plays an important role.

Original: Jiang et al.
Revised: Jiang et al.

Original: (2019) pioneers the application of NAS to
Revised: (2019) pioneers the application of NAS to

Original: NLP tasks beyond language modeling (NER in this case), and improves differentiable NAS by redesigning its search space for natural language processing.
Revised: NLP tasks beyond language modeling (NER in this case), and improves differentiable NAS by redesigning its search space for natural language processing.

Original: Li et al.
Revised: Li et al.

Original: (2020b) extends the search space of NAS to cover more RNN architectures and allow the exploring of intra- and inter-token connection to increase the expressibility of searched networks.
Revised: (2020b) extends the search space of NAS to cover more RNN architectures and allow the exploring of intra- and inter-token connection to increase the expressibility of searched networks.

Original: Pasunuru and Bansal (2020) proposes Flexible and Expressive

Original: Neural Architecture Search (FENAS) to expand the search space of Efﬁcient Neural Architecture Search (ENAS, Pham et al. (2018b)) with more functions (e.g., skip-based tanh, ReLU) and atomic-level operations (e.g., addition, subtraction, element-wise multiplication).

Original: As the popularity of Transformer architecture grows in NLP area, researchers also apply NAS to discover better topology for Transformer.
Revised: As the popularity of pre-trained language models (PLM) grows in NLP area, researchers also apply NAS to discover better topology for PLM such as BERT.

Original: Wang et al.
Revised: Wang et al.

Original: (2020a) introduces Hardware-Aware 
Revised: (2020a) introduces Hardware-Aware Transformers (HAT) to search Transformer architecture optimized for inference speed and memory footprint in different hardware platforms. NAS-BERT (Xu et al., 2021b) and AdaBERT (Chen et al., 2020a) explores task-agnostic and task-dependent network compression techniques with NAS respectively. EfﬁcientBERT (Dong et al., 2021) applies NAS to search for more efﬁcient architecture of feed-forward network that is suitable for edge device deployment. To show the efﬁcacy of NAS, we summarize the performance of several state-of-the-art NAS approaches on GLUE benchmarks (Wang et al., 2019a) in Table 2.

Original: Transformers (HAT) to search Transformer architecture optimized for inference speed and memory footprint in different hardware platforms.
Revised: These approaches are applied to BERT to discover architectures with smaller sizes, faster inference speed, and better model accuracy. For comparison, performance from original and manually compressed BERT models is also presented. The results show that the BERT architecture improved by NAS yields performance competitive to BERT (c.f., 82.3 from EfﬁcientBERT vs 82.

Original: As Transformer is heavy in computation, weight-sharing technique is adopted to make architecture search more efﬁcient.
Revised: from BERT) and is 6.9x smaller and 4.4x faster. The searched architecture also outperforms manually designed, parameter- and inference-efﬁcient model (MobileBERT TINY ) at similar size and speed. These results suggest the efﬁcacy of NAS in discovering more efﬁcient network architectures. As

Revised: NLP researchers continue to design even larger

Revised: PLMs while the need of deployment on edge devices grows, we expect there will be increasing investment in innovating NAS techniques to make

Revised: PLM networks more compact and accelerate inference.

Revised: Challenges

Revised: The main bottleneck for NAS being widely applied is the prohibitive requirement in computation resources for architecture search.

Revised: Approaches such as Efﬁcient Neural Architecture

Revised: Search (ENAS, Pham et al.

Revised: (2018b))

Revised: and Flexible and Expressive Neural Architecture Search (FENAS, Pasunuru and Bansal (2020)) are proposed to improve the search efﬁciency.

Revised: As PLMs usually have bulky sizes and slow training speed, search efﬁciency is even more critical when applying NAS to

Revised: PLM.

Revised: Weight-sharing techniques are often applied to accelerate searching (Wang et al., 2020a; Dong et al., 2021; Xu et al., 2021b).

Original: 4. Meta-learning for Data Selection
Revised: 4. Meta-learning for Data Selection

Original: Multi-linguality, multi-task, and multi-label see many impacts on NLP problems due to the diversity of human languages.
Revised: Multi-linguality, multi-task, and multi-label see many impacts on NLP problems due to the diversity of human languages.

Original: To learn models with balanced performance over attributes (e.g., languages, tasks, labels), a common approach is to weight the training examples for data selection to learn models with balanced performance over the attributes, and it is a natural assumption that meta-learning techniques derive more generalizable weighting than manually tuned hyperparameters.
Revised: To learn models with balanced performance over attributes (e.g., languages, tasks, labels), a common approach is to weight the training examples for data selection to learn models with balanced performance over the attributes, and it is a natural assumption that meta-learning techniques derive more generalizable weighting than manually tuned hyperparameters.

Original: For example, Wu et al.
Revised: For example, Wu et al.

Original: (2019) add another gradient update step wrapping the conventional classiﬁer update for training metaparameters that controls the weight when aggregating losses from different labels to update classiﬁer’s parameters.
Revised: (2019) add another gradient update step wrapping the conventional classiﬁer update for training meta-parameters that controls the weight when aggregating losses from different labels to update classiﬁer’s parameters.

Original: In addition to gradient update, metalearned weights are also applied directly to training examples for data selection to address the issue of noisy labeling.
Revised: In addition to gradient update, meta-learned weights are also applied directly to training examples for data selection to address the issue of noisy labeling.

Original: Shu et al.
Revised: Shu et al.

Original:  propose a technique to jointly learn a classiﬁer and a weighting function, where a conventional gradient update for the classiﬁer and a meta-learning update for the weighting is performed alternatively.
Revised: (2019) propose a technique to jointly learn a classiﬁer and a weighting function, where a conventional gradient update for the classiﬁer and a meta-learning update for the weighting is performed alternatively.

Original: The function weights examples to mitigate model overﬁtting towards biased training data caused by corrupted labels or class imbalance.
Revised: The function weights examples to mitigate model overﬁtting towards biased training data caused by corrupted labels or class imbalance.

Original: Zheng et al.
Revised: Zheng et al.

Original: (2021) apply a similar framework but extend the weighting with a label correction model.
Revised: (2021) apply a similar framework but extend the weighting with a label correction model.

Original: Both techniques show improvement over SOTA in text classiﬁcation with biased training data.
Revised: Both techniques show improvement over SOTA in text classiﬁcation with biased training data.

Original: Additionally, as the progress in the research of pre-training and transfer learning, there is a trend of leveraging datasets in multiple languages, domains, or tasks to jointly pre-train models to learn transferable knowledge.
Revised: Additionally, as the progress in the research of pre-training and transfer learning, there is a trend of leveraging datasets in multiple languages, domains, or tasks to jointly pre-train models to learn transferable knowledge.

Original: A meta-learned data selector can also help in this scenario by choosing examples that beneﬁt model training and transferability.
Revised: A meta-learned data selector can also help in this scenario by choosing examples that beneﬁt model training and transferability.

Original: For instance, Wang et al.
Revised: For instance, Wang et al.

Original: (2020b) investigate the common challenge of imbalanced training examples across languages in multilingual MT, which is conventionally addressed by tuning hyperparameters manually to up-sample languages with less resources.
Revised: (2020b) investigate the common challenge of imbalanced training examples across languages in multilingual MT, which is conventionally addressed by tuning hyperparameters manually to up-sample languages with less resources.

Original: The authors propose Differentiable Data Selection (DDS) to parameterize the sampling strategies.
Revised: The authors propose Differentiable Data Selection (DDS) to parameterize the sampling strategies.

Original: DDS is trained with episodes and REINFORCE algorithm to optimize parameters of sampler and MT models in an alternating way for the MT models to converge with better performance across languages.
Revised: DDS is trained with episodes and REINFORCE algorithm to optimize parameters of sampler and MT models in an alternating way for the MT models to converge with better performance across languages.

Original: Pham et al.
Revised: Pham et al.

Original: (2021) formulate data sampling for multilingual MT as a problem of back-translation to generate examples of parallel utterances from unlabeled corpora in target language.
Revised: (2021) formulate data sampling for multilingual MT as a problem of back-translation to generate examples of parallel utterances from unlabeled corpora in target language.

Original: The back-translation is jointly trained with MT models to improve translation result through better distribution of training examples and data augmentation.
Revised: The back-translation is jointly trained with MT models to improve translation result through better distribution of training examples and data augmentation.

Original: Tarunesh et al.
Revised: Tarunesh et al.

Original: (2021) further study knowledge transferring across tasks and languages.
Revised: (2021) further study knowledge transferring across tasks and languages.

Original: The authors combine Reptile and DDS to meta-learn samplers with six different languages (en, hi, es, de, fr, and zh) and ﬁve different tasks (QA, NLI, paraphrase identiﬁcation, POS tagging, and NER) and demonstrate competitive performance on XTREME multilingual benchmark dataset (Hu et al., 2020).
Revised: The authors combine Reptile and DDS to meta-learn samplers with six different languages (en, hi, es, de, fr, and zh) and ﬁve different tasks (QA, NLI, paraphrase identiﬁcation, POS tagging, and NER) and demonstrate competitive performance on XTREME multilingual benchmark dataset (Hu et al., 2020).

Original:  Meta-learning beyond Accuracy
Revised:  Meta-learning beyond Accuracy

Original: In the previous sections, meta-learning is used to obtain better evaluation metrics for NLP applications.
Revised: In the previous sections, meta-learning is used to obtain better evaluation metrics for NLP applications.

Original: This section illustrates how meta-learning can improve NLP applications from more aspects beyond performance.
Revised: This section illustrates how meta-learning can improve NLP applications from more aspects beyond performance.

Original: 5. Learn to Knowledge Distillation
Revised: 5. Learn to Knowledge Distillation

Original: Knowledge distillation method was proposed in (Hinton et al., 2015).
Revised: Knowledge distillation method was proposed in (Hinton et al., 2015).

Original: The main goal is to transfer knowledge from a so-called teacher model, e.g., a vast neural network trained with a lot of training data, to a more compact student model, e.g., a neural network with much less trainable parameters.
Revised: The main goal is to transfer knowledge from a so-called teacher model, e.g., a vast neural network trained with a lot of training data, to a more compact student model, e.g., a neural network with much less trainable parameters.

Original: The main weaknesses of this method are as follows: a) the number of teacher models is ﬁxed to one that could limit the power of the transferring process; b) the teacher model is not optimized for the transferring process and c) the teacher model is not aware of the student model during the transferring process.
Revised: The main weaknesses of this method are as follows: a) the number of teacher models is ﬁxed to one that could limit the power of the transferring process; b) the teacher model is not optimized for the transferring process and c) the teacher model is not aware of the student model during the transfer- [Table] ring process.

Original: Meta-learning methods can be applied to partially ﬁx these issues.
Revised: Meta-learning methods can be applied to partially ﬁx these issues.

Original: The high-level idea is to increase the number of teacher models and the number of student models and consider each pair of a teacher model and a student model as a task in the meta-learning framework.
Revised: The high-level idea is to increase the number of teacher models and the number of student models and consider each pair of a teacher model and a student model as a task in the meta-learning framework.

Original: By doing so, we can train a meta teacher model that works better than a single teacher model (Pan et al., 2020), and we can optimize the transferring process and force the teacher model to be aware of the student model (Zhou et al., 2021).
Revised: By doing so, we can train a meta teacher model that works better than a single teacher model (Pan et al., 2020), and we can optimize the transferring process and force the teacher model to be aware of the student model (Zhou et al., 2022).

Original: 5. Learn to Life-long learning
Revised: 5. Learn to Life-long learning

Original: This subsection discusses how to use meta-learning to improve lifelong learning (LLL) (Chen and Liu, 2018).
Revised: This subsection discusses how to use meta-learning to improve lifelong learning (LLL) (Chen and Liu, 2018).

Original: The real world is changing and evolving from time to time, and therefore machines naturally need to update and adapt to the new data they receive.
Revised: The real world is changing and evolving from time to time, and therefore machines naturally need to update and adapt to the new data they receive.

Original: However, when a trained deep neural network is adapted to a new dataset with a different distribution, it often loses the knowledge previously acquired and performs the previous seen data worse than before.
Revised: However, when a trained deep neural network is adapted to a new dataset with a different distribution, it often loses the knowledge previously acquired and performs the previous seen data worse than before.

Original: This phenomenon is called catastrophic forgetting (McCloskey and Cohen, 1989).
Revised: This phenomenon is called catastrophic forgetting (McCloskey and Cohen, 1989).

Original: There is a wide range of LLL approaches aiming for solving catastrophic forgetting (Parisi et al., 2019).
Revised: There is a wide range of LLL approaches aiming for solving catastrophic forgetting (Parisi et al., 2019).

Original: Among them, the following directions apply metalearning:
Revised: Among them, the following directions apply metalearning

Original: Meta-learning for Regularization-based LLL methods
Revised: :Meta-learning for Regularization-based LLL methods

Original: Regularization-based LLL methods aim to consolidate essential parameters in a model when adapting models with new data (Kirkpatrick et al., 2017;Schwarz et al., 2018; Zenke et al., 2017; 
Revised: Regularization-based LLL methods aim to consolidate essential parameters in a model when adapting models with new data (Kirkpatrick et al., 2017; Zenke et al., 2017; Schwarz et al., 2018;

Original: Aljundi et al., 2018; Ehret et al., 2021).
Revised: Aljundi et al., 2018; Ehret et al., 2021).

Original: Metalearning targets “how to consolidate” and has some successful examples in NLP applications.
Revised: Metalearning targets “how to consolidate” and has some successful examples in NLP applications.

Original: KnowledgeEditor (dec) learns the parameter update strategies that can learn the new data and simultaneously retain the same predictions on the old data.
Revised: KnowledgeEditor (De Cao et al., 2021) learns the parameter update strategies that can learn the new data and simultaneously retain the same predictions on the old data.

Original: KnowledgeEditor has been applied to fact-checking and QA.
Revised: KnowledgeEditor has been applied to factchecking and QA.

Original: Editable Training (Sinitsin et al.,2020) employs learn-to-init approaches to ﬁnd the set of initial parameters, ensuring that new knowledge can be learned after updates without harming the performance of old data.
Revised: Editable Training (Sinitsin et al., employs learn-to-init approaches to ﬁnd the set of initial parameters, ensuring that new knowledge can be learned after updates without harming the performance of old data.

Original: Editable Training empirically demonstrates the effectiveness on MT.
Revised: Editable Training empirically demonstrates the effectiveness on MT.

Original: Meta-learning for Data-based LLL Methods
Revised: Meta-learning for Data-based LLL Methods

Original: The basic idea of data-based methods is to store a limited number of previously seen training examples in memory and then use them for empirical replay, that is, training on seen examples to recover knowledge learned (de Masson d'Autume et al., 2019; Sun et al., 2019a; Sprechmann et al., 2018) or to derive optimization constraints (LopezPaz and Ranzato, 2017; Li and Hoiem, 2017; Saha and Roy, 2021).
Revised: The basic idea of data-based methods is to store a limited number of previously seen training examples in memory and then use them for empirical replay, that is, training on seen examples to recover knowledge learned (Sprechmann et al., 2018; de Masson d'Autume et al., 2019; Sun et al., 2019a) or to derive optimization constraints (Lopez-Paz and Ranzato, 2017; Li and Hoiem, 2017; Saha and Roy, 2021).

Original: A hurdle for data-based approaches is the need to store an unrealistically large number of training examples in memory to achieve good performance.
Revised: A hurdle for data-based approaches is the need to store an unrealistically large number of training examples in memory to achieve good performance.

Original: To achieve sample efﬁciency,  Wang et al.
Revised: To achieve sample efﬁciency, Obamuyide and Vlachos (2019a) ; Wang et al.

Original: (2020c)
Revised: (2020c);

Original: ; Wu et al.
Revised: Wu et al.

Original: (2021a); Obamuyide and Vlachos (2019) uses meta-learning to learn a better adaptation algorithm that recovers the knowledge learned with a limited amount of previously seen data.
Revised: (2021a) uses meta-learning to learn a better adaptation algorithm that recovers the knowledge learned with a limited amount of previously seen data.

Original: Experiments on text classiﬁcation and QA benchmarks validate the effectiveness of the framework, achieving state-of-the-art performance using only 1% of the memory size (Wang et al., 2020c).
Revised: Experiments on text classiﬁcation and QA benchmarks validate the effectiveness of the framework, achieving state-of-the-art performance using only 1% of the memory size (Wang et al., 2020c).

Original:  Conclusion
Revised:  Conclusion

Original: This paper investigates how meta-learning is used in
Revised: This paper investigates how meta-learning is used in

Original: NLP applications.
Revised: NLP applications.

Original: We review the task construction settings (Section 3), the commonly used methods including learning to initialize, learning to compare and neural architecture search
Revised: We review the task construction settings (Section 3), the commonly used methods including learning to initialize, learning to compare and neural architecture search

Original: (Section 4), and highlight research directions that go beyond improving performance (Section 5).
Revised: (Section 4), and highlight research directions that go beyond improving performance (Section 5).

Original: We hope this paper will encourage more researchers in the NLP community to work on meta-learning.
Revised: We hope this paper will encourage more researchers in the NLP community to work on meta-learning.

Original: A Appendx
Revised: A Appendx

Original: [Table] 
Revised: [Table] [Table] [Table]

Revised: Multilingual ASR

Original: [Table]
Revised: [Table]

Original:  [Table] 
Revised: MAML/ [Table] [Table] MAML

Revised: Pretrain

Revised: [Table]

Revised: Supervised Learning

Revised: [Table]

Revised: FOMAML

Revised: [Table]

Revised: MAML/

Revised: [Table]

Revised: FOMAML

Revised: [Table]

Revised: FOMAML

Revised: [Table]

Revised: MAML

Revised: [Table]

Revised: Learning

Revised: [Table]

Revised: BERT

Revised: [Table]

Revised: mBERT/

Revised: [Table]

Revised: [Table]

Revised: Word Embedding

Revised: [Table]

Revised: Word Embedding/ELMo/BERT

Revised: [Table]

Revised: Word Embedding

Revised: [Table]

Revised: MAML -

Revised: [Table]

Revised: DG

Revised: [Table]

Revised: BERT (

Revised: [Table]

Revised: DG

Revised: [Table]

Revised: MAML -

Revised: [Table]

Revised: MAML mBERT (Nooralahzadeh

Revised: [Table]

Revised: MAML

Revised: BERT/mBERT/XLM-R

Revised: [Table]

Revised: MAML mBERT (

Revised: [Table]

Revised: der Heijden

Revised: [Table]

Revised: FOMAML/Reptile/

Revised: [Table]

Revised: MAML

Revised: XLM-R (

Revised: [Table]

Revised: LEOPARD

Revised: BERT

Revised: [Table]

Revised: FOMAML

Revised: BERT

Revised: [Table]

Revised: [Table]

Revised: MAML

Revised: BERT/RoBERTa

Revised: [Table]

Revised: [Table]