Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper presents a new framework to unify two streams of few-shot learning methods, the episodicc meta-learning and pre-train finetune-based few-shot learning. Besides, a new meta-dropout strategy is proposed to improve the generalization ability. Extensive results on several benchmark like PascalVOC, COCO, CUB and mini-ImageNet validate the proposed approach. 
main_review: Pros:
1. The idea to unify the episodic meta-learning and pre-train finetune-based few-shot learning is meaningful for the improvment of the few-shot learning performance.
2. The proposed algorithm is extensively validated on the popular benchmarks like PascalVOC, COCO, CUB and mini-ImageNet.


Cons:
1. The performance gain of meta-dropout is of large variance. For example, Based on Table 1, different splitting setting may have large variations. In split1, the gain of meta-dropout on Meta-R-CNN is around 10 for 1shot for split1 but only 3.3 for split2.  Also, how about the performance gain for meta-dropout on other approaches besides from Meta R-CNN and TFA. 
2. The comparison in the experiments miss several state-of-art paper results. For example, in Table 1 on the VOC2007 benchmark, there are a few published paper with better performance as listed. To name a few:
[R1] Wu, J.; Liu, S.; Huang, D.; and Wang, Y. 2020. Multi-Scale Positive Sample Refinement for Few-Shot Object Detection.In ECCV 2020
[R2] Li, B.; Yang, B.; Liu, C.; Liu, F.; Ji, R.; and Ye, Q. 2021. Beyond Max-Margin: Class Margin Equilibrium for Fewshot Object Detection. In CVPR 2021
3. The contribution of the paper is not significant enough to match the level of ICLR. 
summary_of_the_review: The paper targets at the interesting problem of few-shot learning and presents a unified framework and a meta-dropout method. Although the experiments are widely validated on several benchmarks, the main concern is the comparison of the experiments miss several State-of-the-art paper results. 

-----------------------------------------------------------------------------
Update after the rebuttal:
The rebuttal does not well address the concerns mentioned in the previous round of review. For example, the experiments can list the setup using a fixed selection of novel samples for a fair comparison with state-of-the-arts besides from the multiple rounds of results listed in the paper.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper addresses few-shot learning in a combination of episodic meta-learning-based and pre-train finetune-based approach. It proposes meta-dropout, that consists in using dropout on training the base classes during meta-training, but not when tuning on novel classes. It presents results for both classification and object detection tasks.
main_review: The paper is hard to read and the novelty and impact of the proposed contributions are not clear. Some questions that would help to clarify the formulation:
1)  The paper claims to have a novel benefit from combining pre-training with episodic training. In that direction, references on previous works combining pre-training and episodic training are missing. Just to mention a few combining pre-training and episodic training: https://arxiv.org/pdf/2003.04390.pdf
https://arxiv.org/abs/1903.03096.
2) Related works claim "Pre-train finetune-based approaches are basic yet ignored in few-shot learning due to the excellent
performance of episode-based methods". But this statement ignores recent literature showing superior performance of 
finetune-based approaches for few shot classification (not referenced in the paper) and also the findings of Wang et al. 2020  for object detection (referenced in the paper). Some references: 
https://openreview.net/forum?id=Hq8rqCDkETi
https://arxiv.org/abs/1904.04232
https://openreview.net/pdf?id=Q0hm0_G1mpH
3) It is not clear the improvement or novelty of the proposed approach when compared with existent approaches. As TFA seams to be the closer reference, a clear comparison with it is missing (Wang et al. 2020). The results presented are not comparable as they dont present base classes performance. TFA main property is to preserve base classes performance while at the same time it obtains good performance on novel classes. As table containing results only show novel classes performance, it is not clear if this property is maintained or if results shown on novel may be associated with a decrease in base classes performance.
4) There are no partial results on the element that compose the formulation that would help to clarify the importance of the individual contributions. The results on table 2 showing the  benefits of meta-dropout are inferior to TFA original performance. TFA original results are not presented in the table. 
5) The fact that AP is improved but not AP50 should be better justified as it can be seen as a proxy of localization precision performance.
6) The change in batch size alone adopted in the paper can induce some non intuitive side effects that make comparison with other results unclear.
7) It is not clear why results of VOC where show on split 1 only.
8) References and comparison with SOTA few shot detection approaches and results are missing.
9) Results on classification on top of CUB and Mini-Imagenet seems to benefit from a strong baseline. The evaluation of the proposal in a more diversified few shot dataset such as Meta-Dataset and/or VTAB would clarify the relevance of the proposal in respect to few shot classification SOTA results.
summary_of_the_review: The paper is hard to read and the novelty and impact of the proposed contributions are not clear. Results lack a clear comparison with the recent literature.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper aims to improve the generalization power of meta-learning. It claims to propose an effective strategy named meta-dropout, which is applied to the transferable knowledge generalized from base categories to novel categories. The proposed strategy aims to prevent neural units from co-adapting excessively in the meta-training stage (where somehow I did not get the idea of what it is).
main_review: I understand from the abstract and introduction that the paper aims to combine two different solutions of meta-learning and build a unified one that could potentially improve the task generalization ability of the meta-model. However, it is not easy for me to find any proof of new ideas to achieve that target in the method sections. More importantly, it is not new in terms of the "idea" of combining "the episodic meta-learning-based and pre-train finetune-based few-shot learning". For example in the CVPR19 paper "Meta-transfer learning for few-shot learning" or in the ICLR19 paper "Meta-Learning with Latent Embedding Optimization", they all use the idea of pre-trained models in solving few-shot tasks and train models with episodic tasks. Other detailed comments are in the following: It is not clear in Fig1 what is the target knowledge or concrete format of the knowledge to learn or transfer to testing tasks; It is not clear why using a dropout in the meta-learning process can be called meta-dropout. Any new design of its optimization in meta-level?

summary_of_the_review: This paper is not easy to follow and lacks novelty in the method.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose a unified framework for meta-learning based algorithm and pretrain finetune based algorithm. Drop out is applied for few-shot learning and achieve better generalization ability. Experiments on two tasks validate the effectiveness of the proposed algorithm.
main_review: Strengths: validate the proposed algorithm on both few-shot object detection and few-shot classification tasks.

Weakness: unclear contributions

The paper claims to have three contributions: 1. utilize the idea of meta-learning to integrate two very different streams of few-shot learning, i.e., the episodic meta-learning-based and pre-train ﬁnetune-based few-shot learning, and form a uniﬁed meta-learning framework. 2. propose a simple yet effective strategy, named meta-dropout, to improve the generalization power of meta-knowledge in our framework. 3. Experiments of baselines from different streams evaluate the effectiveness of our approach on the few-shot object detection and image classiﬁcation datasets, i.e., Pascal VOC, MS COCO, CUB, and mini-ImageNet.

However, after reading this paper, I felt the contribution claimed is not supported well and thus, the contribution of this paper is not very clearly stated. The detailed reasons are as follows:

In section 3, the authors formulated episode-based meta-learning and pretrain finetune based meta-learning. From section 3.1, it seems the difference only lies in if the backbone is trainable or not. I do not see any motivation of dividing into two categories and why a unified framework is a contribution here. Also, 3.2.2 and 3.2.3 states the unified framework, and i do not see any novelty for these formulations here.

Section 3.3 shows how to apply the dropout mechanism to improve the generalization of meta-learning models. I do not see any further improvements for drop out but only applying drop out to meta-learning models. Thus, i do not feel this is a strong contribution for this paper.

As for the experiment results, i appreciate the authors do experiment on two tasks. However, the baselines are too old (baselines in 2019). The datasets are also not large-scaled as tiered-Imagenet or Meta-dataset. Thus, it does not have sufficient proof on the state-of-the-art performance of the algorithm.


summary_of_the_review: The pros of this submission is to validate the algorithm on different few-shot tasks but the contribution is unclear.
review_title: null

Meta-review: This work proposes an approach to unify pre-training-based and meta-learning-based few-shot learning, inspired by dropout.

None of the reviewers support the acceptance of this work, despite the authors' detailed rebuttals, with the majority of reviewers confirming their preference for rejection following the author response. 

I unfortunately could not find a good reason to dissent from the reviewers majority opinion, and therefore also recommend rejection at this time.
Decision: Reject