Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Authors correctly point out the tradeoff between adaptation costs and performance on novel datasets. Motivated by this tradeoff, authors introduce a novel adaptation layer, based on squeeze-and-excitation, that performs task-based feature modulation and is meta-learned in combination with a per-task learned linear head. The proposed method performs well on a variety of large-scale and difficult few-shot adaptation benchmarks at a reasonable computation cost. 
strengths_and_weaknesses: STRENGTHS:

A well-motivated, well-reasoned and widely applicable approach. Results are convincing and the model clearly accomplishes what it sets out to accomplish, in the way that it claims to accomplish it. Paper is clearly written, if not entirely well-organized or -focused (see below). 

WEAKNESSES:

The paper suffers from a misplaced focus in its presentation. Many of the named concepts presented as novel are not, and the truly novel contribution is somewhat limited and discussed only briefly. 

1. CaSE is introduced as a novel layer but is identical to SE but for the context pooling (discussed below) and the final activation layer (an implementation detail discussed in supplementary). 

2. Related to above, the adaptive mode / inference mode (described as “fundamental” on pg4 line 142) of CaSE is identical to TADAM (TADAM: Task Dependent Adaptive Metric for Improved Few-Shot Learning, NeurIPS2018), and also just a common-sense approach to handling a support vs query set. 

3. I hesitate to call the UpperCaSE meta-training scheme particularly novel or coordinate-descent-based. While optimization does switch back and forth between CaSE and head parameters, the head parameters reset with every new batch. In actuality the CaSE parameters are being meta-learned, while head parameters are being set in the inner loop, and UpperCaSE is just a straightforward and common-sense approach to meta-training. This procedure (train linear head to convergence, propagate gradient into meta-learned layers) is also the exact same procedure already introduced by MetaOptNet (Meta-Learning with Differentiable Convex Optimization, CVPR2019).

In my eyes the true novelty of CaSE is in taking the FiLM/TADAM approach to task-conditioning, and replacing the redundant task encoder layers with the appropriate intermediate network activations. This crucial difference is discussed only briefly in related work (pg6 line 237-238). I also consider this a somewhat limited conceptual contribution, empirical results aside. 

In my eyes, this calls for a fairly substantial text revision, where the contribution is mainly a novel _approach_ to efficient adaptation rather than a “new adaptive block” (lines 7-8), and the conceptual linkages to TADAM/FiLM are explored and discussed rather than the similarities/differences relative to SE, which are much less relevant in this context. I recognize this could be a pretty idiosyncratic and overly specific take though – I’ll be curious to see what other reviewers think on the novelty/contribution. 

Less importantly, CaSE is conceptually quite similar to TADAM, and above proposed revisions aside, TADAM is at least worth a mention in related work, as a FiLM-derivative approach to few-shot learning. 

questions: Given CaSE’s conceptual similarity to FiLM and TADAM, which I consider to be far more appropriate baselines than SE, what exactly do the authors consider to be the true contribution and proper source of novelty here? 

Given the above answer, how can this position be strengthened and made clearer in the body text of the paper?

limitations: Limitations are discussed clearly and fairly. Societal impacts are addressed very briefly, though since the impacts match those of few-shot learning more broadly, this is sensible.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The work proposes a novel method for few-shot image classification. The method is named UpperCaSE and is based on adaptation of the Squeeze-and-Excitation block to learn task contextual parameters. Their approach is hybrid in terms of a combination between meta-learning and fine-tuning of the network. The proposed hybrid approach aims to bridge the gap between fine-tuning approaches which are more accurate, and the meta-learning approaches which have lower adaptation cost. Optimization protocol is based on Coordinate-Descent between the CaSE blocks in the network body (cross task parameters) and the task specific parameters of the head (last linear layer). The approach achieves new sota results for meta-learners on the VTAB+MD and ORBIT.
strengths_and_weaknesses: **Strengths**
- The paper is written in a clear and easy to follow manner.
- The method requires only a single forward pass over the context.
- The methods is simple and novel yet achieves very good results compared to other meta-learners.
- Experimentation study is comprehensive and also shows the downside on structured tasks.

**Weaknesses**
- I would have liked to see some ablation study and discussion on the CaSE block architecture choice. 
questions: I would have like to know how much the context pooling layer in the CaSE block is sensitive to the design choices. For example, have you tried to use several layers? adding a BN layer or other thing that either did not change the outcome or did not work at all.
limitations: I believe that the authors adequately addressed the limitations and potential negative societal impact of their work.
I appreciate for being honest about the lower results on structure datasets and the hypothesis that this case might require fine-tuning of the network's body.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper mainly focuses on few-shot learning. The authors advocate the importance of an efficient finetuning algorithm for FSL. To the end, they propose a new module based on SE block, which generates scaling parameters using task-specific information. Furthermore, they propose to leverage coordinate-descent in meta-training to solve the problems of instability, vanishing gradients and high memory consumption.
strengths_and_weaknesses: 1. The authors conduct extensive experiments to show the effectiveness of their method.
2. The proposed coordinate-descent for meta-learning is interesting and can be a potential plug-in solver for other meta-learning methods.

questions: 1. The main concern comes from the proposed CaSE. This module takes episode-level features as input and provides channel-wise scaling to the final features, which reminds me of MTL [1]. The MTL is also aimed to meta-train new modules, that produce scaling and shifting parameters, and use them directly in testing phase. I find it very similar to the proposed method. It would be better if the authors could explain the exact difference between two methods and the significance.
2. Beyond MTL, there are other works that can fulfill the goal of finetuning only few parameters on each episode, such as [2]. The authors should provide more discussion and comparison.
3. I wonder if the authors can provide results on smaller dataset like miniImageNet so that the method can be compared with more state-of-the-art methods.


[1] Sun Q, Liu Y, Chua T S, et al. Meta-transfer learning for few-shot learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 403-412.

[2] Li W H, Liu X, Bilen H. Cross-domain Few-shot Learning with Task-specific Adapters[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 7161-7170.
limitations: Detailed comments are in the questions. In summary, I find the proposed module similar to the existing method in FSL, which is the main problem of the method.
review_title: null

Meta-review: metareview: The reviewers consider the work technically solid, but were concerned about the contextualization of this work in the literature.  post-rebuttal, some of the reviewers concerns are resolved.
confidence: Certain
Decision: Accept