Venue: AutoML 2022 Track Main

Reviews:

final_score: rating: 4: Marginally above the acceptance threshold (use sparsely)
review_content: summary_of_contributions: This paper propose to meta-learn adapters as a way to efficiently finetune/adapt large pretrained (language) models to tasks at hand. In particular, the authors follow the standard adapter settings as proposed in Houlsby et al. 2019, but encase a base adapter layer with Meta-Adapters - two-layer bottleneck MLPs - which are trained during the outer loop of an otherwise standard gradient-based MAML-like Meta-Learning optimization process. Through a set of ablations, the authors are able to show that when compared to the standard non-meta-learned adapter setting their meta-adapter approach is more sample-efficient, performing much in various few-shot classification settings. A large experimental evaluation on 17 downstream few-shot classification tasks against the adapter baseline, another adapter-reference and one meta-learning baseline shows Meta-Adapter comparing favourable when accounting for the number of parameters tuned.
overall_review: Positive aspects include:
* the idea of Meta-Learning through Meta-Adapters is, to the best of my knowledge, novel.
* within the adapter-domain of Meta-Learning, the proposed approach appears to perform quite favourably.
* The paper is well written and easy to follow.

For negative aspects, see concerns listed in Section 3 on the Technical Quality and Correctness.
review_summary: I am going with a rejection for now. While the method performs better than the adapter baselines, the overall approach is quite incremental, and it is unclear how it compares to and ties in with other similar approaches to this problem. This connects to the main issue, which is the quality of the experimental evaluation that lacks a more thorough comparison against other meaningful baselines.
review_title: null

final_score: rating: 4: Marginally above the acceptance threshold (use sparsely)
review_content: summary_of_contributions: The motivation of the paper:
1. pre-trained model is very useful, but fine tuning it with all parameters is impractical.
2. Adapter method is proposed to reduce the number of model parameters to be fine tuned by adding additional task-specific parameters and keeping frozeing pretrained model. However, this method is not useful for few shot learning tasks.

Contributions:

1. This paper proposed a new parameter efficient fine tuning approach based on pretrained model based on the Adapter method on the top of meta learning idea, in particular MAML, called: Meta-Adapter.
2. They add some new blocks to the pretrained model: meta-adapter block (\w), in addition to the original pretrained model layers (\theta), and adapter (\phi).
   1)  \phi will be meta-learned, \w will be learned along with \phi, \theta will be fixed during training and inference. 
   2) During inference, learned \w will be fixed, \phi will be fine tuned based on new tasks.
3. Experiments validate the effectiveness and efficiency of the proposed model.
overall_review: Positive:

1. This paper proposed a new parameter efficient fine tuning approach based on pretrained model based on the Adapter method on the top of meta learning idea, in particular MAML, called: Meta-Adapter.
2. The formulation is clear for autoML community.
3. Experiments validate the effectiveness and efficiency of the proposed model.

Negative:

Some points are not clear and should be clarified:
1. the similarities and difference between Meta-Apdater and MAML-Adapter.
    It seems that the only difference is
    - Meta-Adapter adds some new blocks to the pretrained model.
    - MAML-Adaper does not introduce new blocks.

   The similarity is to meta-learn the adapter parameters.

2. What is the specific setting for "transfer learning" in the experiment? what is source and target? These settings are important for readers.

3. Since the Meta-Adapter approach brings more parameters (meta-adapter blocks) to Adapter approach, why does Meta-Adapter have less parameters than Adapter (as shown in Fig.1)? I am confused with this.
review_summary: Pre-trained model is very useful, but fine tuning it with all parameters is impractical. Adapter method is proposed to reduce the number of model parameters to be fine tuned by adding additional task-specific parameters and keeping frozeing pretrained model. However, this method is not useful for few shot learning tasks.

Considering the limitation above, this paper proposed a new parameter efficient fine tuning approach based on pretrained model based on the Adapter method on the top of meta learning idea, in particular MAML, called: Meta-Adapter.

I made decision based on the positive and negative points listed above.
review_title: null

final_score: rating: 2: Reject, not good enough
review_content: summary_of_contributions: + this work generalizes the transfer-learning technique for pretrained models Adapter (Houlsby et al., 2019) to the meta-setting, by adding Meta-Adapters. In few-shot learning settings, these Meta-Adapters, which are meta-learned but not fine-tuned, serve the purpose of easing optimization.
+ competitive performance with state-of-the-art (sota) few-shot learning methods on NLP benchmarks
+ performant on NLP benchmarks while only requiring to fine-tune 0.6% of the parameters compared to a sota approach.
overall_review: \+ Meta-Adapter significantly reduces the number of parameters to be finetuned on top of a pre-trained model, while keeping performance mostly on par with SOTA.

major:  
\- The largest drawback is that the approach is only validated on natural language processing (NLP) benchmarks, leaving the question whether the same parameter-performance characteristic can be maintained on computer vision, regression or reinforcement learning benchmarks.

\- the downstream benefit of (meta-)training fewer parameters should be discussed more. For example compare the inference time training speedup (not the meta-training speedup, which seems to be marginal compared to the parameter reduction).

minor:  
\- no comparison with meta-baseline [3], a very similar technique where meta-parameters \omega are the same as \theta (in fact, a further trained version, also from a pretrained \theta).

\- no mention or comparison to ANIL/BOIL, which are very similar to Adapter and (parameter) improvements over MAML, but don't leverage pretrained models.

[3] https://arxiv.org/abs/2003.04390
review_summary: Meta-adapter is an extension of Adapter (Houlsby et al. 2019) to the meta-learning setting. Similar to Adapter, Meta-adapter enables to fine-tune a significantly lower number of parameters for task adaptation by introducing intermediate layer modules. However, it does so while maintaining performance in the few-shot setting, unlike Adapter, as shown in this paper. The main drawback of the work is that it is only evaluated on NLP tasks. To make general few-shot AutoML conclusions, the approach needs to be evaluated at least also on one (more) common meta-learning computer vision benchmark ((mini/tiered)-ImageNet, ...). Next to this, the writing/clarity should also be fixed, and the benefit of fewer parameters for few-shot learning better motivated. These things would make it possible to increase the score.
review_title: null

final_score: rating: 5: Accept, good paper
review_content: summary_of_contributions: The paper proposes the usage of meta-learning for parameter efficient fine-tuning using adapters within a pretrained transformer model. To enable the usage of adapters in the few-shot setting, additional meta-adapters are introduced that trained during meta training alongside with the adapter weights and the fine-tune learning rate. During meta testing, the meta-adapters are frozen as well and only the adapter weights are fine-tuned from the meta-learned initialization. A large evaluation on many different NLP tasks shows superior performance to other parameter efficient fine-tuning methods and even comparable performance to full fine-tuning, in particular for very low sample sizes.
overall_review: The proposed approach to combine meta learning with the parameter-efficient adapter method for fine-tuning that allows to use a single frozen backbone with task specific adapters is novel to my knowledge and provides an interesting extension of simply meta-training the adapters. The proposed approach is nicely presented and easy to follow. Furthermore, the paper provides a large experimental evaluation on many different NLP tasks and comparison with different baseline methods coming that have been proposed in the fine-tuning and meta-learning context. The following are my additional questions and recommendations:

1. How effective is meta-learning the fine-tuning learning? An ablation study would be interesting (e.g., what are the meta-learned learning rates vs. the learning rates found by grid search for the adapter-only networks).
2. What is the importance of meta-adapter placement around the regular adapter layers? Would it suffice to have a single meta-adapter either before or after the regular adapter?
3. For the ablation study that does meta-learning without meta-adapter (i.e., the MAML-Adapter case) it would be interesting to see the MAML-Adapter performance for uncased data and multiple adapter sizes as well as there seems the performance seems to be quite sensitive to the adapter size.
4. Table 2 and 3: The usage of meta-adapter in conjunction with adapter size in this table can be confusing: Does the adapter size relate to the bottleneck size of the meta-adapter blocks, the adapter blocks, or both in a meta-adapter network? If it refers only to the size of the meta-adapter blocks, how large are the adapter blocks in a meta-adapter network and what is the reason for the difference in fine-tuned params for meta-adapter and adapter networks in table 3?
5. What would be the effect of fine-tuning the meta-adapter block parameters as well? Could this lead to further improvements in settings for large k (e.g., k=16)?

Minor:

1. Line 223: How do the 340 evaluations arise given there 10 training sets, 3 possible values for k and 17 tasks?

review_summary: The paper presents in interesting idea combining meta-learning with a parameter efficient method for fine-tuning. It provides extensive experimental evaluation of the effectiveness of the approach.
review_title: null

Meta-review: metareview: I would like to thank the reviewers for constructively engaging with the submission!

`All reviewers` agree that there is value in approaches which efficiently repurpose pre-trained models to serve ever wider distributions of tasks, even when the basic principles underlying the approach are well known. While model "adapters" have long been used in transfer learning research, e.g. vision, NLP and RL, and meta-learning has been applied to a myriad of few-shot settings, it is also well understood that fast generalisation settings introduce additional considerations; for example, meta-learning with heterogeneous task distributions is known to be challenging, and tuning adapter-based methods to heterogeneous distributions of tasks particularly difficult, e.g. due to different capacity requirements for various tasks. Hence, incremental work is valuable and needed to understand such trade-offs in detail. I believe this is the conclusion of all but `Reviewer r3Zo`. I will revisit their arguments below.

In terms of impact of work on AutoML, I do agree with `all reviewers` that it may be limited or medium, but I would like to emphasize the value of the methodology followed in the paper, largely borrowed from meta-learning, for future AutoML works: a meta-objective is defined using a heterogeneous set of problems, including a left-out set of tasks which is used for final evaluations, and a difficult optimisation problem is solved in order to find appropriate learning rates and adaptation sub-spaces which are conducive to generalisation on a broad number of related problems. 

I really appreciated the work that `Reviewer r3Zo` put into finding ways to improve the paper, which I strongly advise authors to implement! However, I do not agree that demonstrating applicability of their approach to other domains is mandatory for acceptance; this has actually been done, e.g. Flennerhag et al. 2020, Rusu et al. 2018 in RL and few-shot visual processing respectively. 

As for my own recommendations for the authors, I suggest using the work of Baik et al. 2020 to further improve meta-learned hyper-parameters, such as learning rates. I believe that showing the value of meta-adapters vs. meta-learned hyper-parameters could improve the paper further, and would also be more in line with similar attempts in AutoML.

On balance, I would like to recommend acceptance, considering the improvements made by authors as per suggestions of reviewers.


### References:
* Flennerhag, Sebastian, Andrei A. Rusu, Razvan Pascanu, Francesco Visin, Hujun Yin, and Raia Hadsell. "Meta-Learning with Warped Gradient Descent." In International Conference on Learning Representations. 2019.
* Rusu, Andrei A., Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. "Meta-Learning with Latent Embedding Optimization." In International Conference on Learning Representations. 2018.
* Baik, Sungyong, Myungsub Choi, Janghoon Choi, Heewon Kim, and Kyoung Mu Lee. "Meta-learning with adaptive hyperparameters." Advances in Neural Information Processing Systems 33 (2020): 20755-20765.


confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept