Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a novel idea to improve few-shot learning by synergizing ideas from the transfer learning and meta-learning communities. Specifically, from transfer learning, it leverages the backbones pretrained on large image datasets and fine-tuned parameter-efficient adapters, while from meta-learning, it uses metric learning-based final layer classifiers trained with episodic protocols. The authors conducted experiments on several downstream datasets, illustrating that their proposed method, FIT, achieved superior accuracy at low-shot on benchmarking datasets. In addition, they demonstrated the method's parameter efficiency in distributed few-shot learning including model personalization and federated learning tasks.
strength_and_weaknesses: Strengths: The paper introduces a novel strategy to combine ideas from transfer learning and meta-learning communities for developing data-efficient and parameter-efficient learning systems. The authors motivated the proposed method well. The method achieved superior performance at low-shot on several benchmarking datasets. In addition, the authors further demonstrated FIT's parameter efficiency and superior accuracy in distributed low-shot personalization and federated learning applications. The evaluation is relatively convincing.

Weaknesses: The paper includes a range of quantitative evaluation results, but it would be better to also provide a more in-depth explanation of the experimental results and the authors' insights on why their proposed method works, and in which settings it would suffer.
clarity,_quality,_novelty_and_reproducibility: The paper is clearly written and easy to follow. The work is original, combining previously existing ideas from the transfer learning and meta-learning communities.

There are some minor typos in the paper, such as in the bottom part of the second page "... while using using ≈ 1% of the updateable parameters when compared to the leading transfer learning method BiT".

summary_of_the_review: This paper proposes a novel method combining existing ideas from transfer learning and meta-learning, which enables a parameter and data-efficient network architecture for low-shot learning. The authors conducted a range of experiments on standard datasets to illustrate the effectiveness of their proposed method. This strategy could be inspiring to the community and potentially generalize beyond imaging classification tasks.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes FiLM Transfer framework (FIT) for image classification tasks via incorporating the ideas from transfer learning and meta learning. The proposed method aims to address the learning with small data and the communication efficiency in distributed learning, in the context of the personalization and federated learning. The FIT method basically consists of a pretrained backbone with an automatically configured Naive Bayes final layer classifier. The outperformances are demonstrated through the experiments in few shot, personalization and federated learning tasks.


strength_and_weaknesses: Strengths:
1. The motivation is well-elaborated with regard to the limited data and communication efficiency. 
2. The writing is easy to follow. 

Weakness:
1. The design of the network architectures are basically based on the ideas from the transfer learning (frozen backbone) and meta learning (Naive Bayes final layer). Is the any difference of the design in FIT when compared with the original ideas in transfer learning and meta learning? or just simply using the two ideas without modification? Please clarify it.
2. There are two personalization concepts in this work. One is in the evaluation on the ORBIT dataset. Another one is in the federated learning. What is the difference and the relationships between these two? 
4. In the experiments of few shot federated learning, the defined upper bound and lower bound for reference are useful for performance analysis during the evaluation of different settings. However, the details of model evaluation/test are not provided, for the global and personal models, for example, the setup and partitions of the datasets used in global or local test which are important in federated learning.  
5. In the first sentence of the last paragraph in Page 8, the authors mentioned that, no suitable federated learning system for comparison. As the key idea of the proposed method in the scenarios of federated learning is based on partial model sharing/update, why the authors did not include the comparison of some partial model transmission-based federated learning (FL) methods or in personalized FL (PFL) scenarios, e.g., the sparisification based FL and PFL in which the communication efficiency and the performance of personal models are investigated. 
6. In FL settings used on Tab. A.7 and Fig. A.3, the performance of conventional models based FL methods should be provided, with regard to accuracy and communication efficiency. It will help to demonstrate the effectiveness. 
7. In the writing, more discussion should be provided around the proposed method in the federated settings, such as the advantages and the difference from the conventional partial model based PFL methods.


clarity,_quality,_novelty_and_reproducibility: The paper is easy to follow, but the originality and novelty are a bit incremental and limited to some extent. 
summary_of_the_review: The merits and concerns are presented in the  main review. The main concerns are about the novelty of the network design, as well as the experimental design in federated settings.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a parameter and data efficient network architecture and the corresponding training procedure for personalization and federated learning. The proposed architecture includes FiLM layers in the pretrained backbones and takes Naive Bayes classifier as the head layer. 
strength_and_weaknesses: The paper is well motivated as both personalization and federated learning require parameter efficient and training data efficient model. 
The proposed idea is succinct and to me it has potential to be generalized to applications other than CV.
The paper provides sufficient and convincing experiment results to show the superior performance of the proposed architecture. 

Some details need to clarify or fix:
1. During the episodic fine tuning, in each iteration, do you optimize \phi and e via gradient ascent for only one step, or optimize them by running the gradient ascent for multiple steps? 
2. In section 4.1, for the one-shot learning, how do you split the downstream dataset D as D_train and D_test?
3. In the second paragraph on page 8, it's not clear to me why Naive Bayes head is not transferred to the server. Do you assume that there are no shared classes among clients? 
4. There is a typo in the 4th paragraph on page 3, "b_d" should be "d_b"
clarity,_quality,_novelty_and_reproducibility: In general the paper is well written, the authors explained in detail why they propose these techniques. 
The experimental results are fully discussed. 
summary_of_the_review: Overall, it's a good paper, I'd suggest to accept this paper for publication.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose FiLM Transfer (FiT) which combines transfer learning and meta-learning. FiT aims to achieve parameter efficiency under a limited number of training samples without degrading accuracy. This potentially results in inexpensive model updates for personalization and communication-efficient federated learning. 
FiT backbone is a frozen pre-trained network that is augmented with parameter-efficient FiLM layers (for adapting to the new tasks), and a Naive Bayes classifier layer with fewer updateable parameters rather than the linear layer.
strength_and_weaknesses: **Strengths**
1) The proposed FiT has superior parameter efficiency and classification accuracy using orders of magnitude fewer updateable parameters and communication cost than BiT.
2) The paper is well-organized with detailed experiments and comparisons across various downstream tasks in few-shot learning.


**Weakness**
1) The paper combines ideas from FiLM (to learn from little data and generalize to more complex and/or substantially different data) layers and meta-learning-inspired episodic training protocol [Dumoulin'2021] to outperform transfer learning baseline BiT.


clarity,_quality,_novelty_and_reproducibility: The paper is well-written, with a comparison table wrt BiT that is much appreciated. The paper is a novel combination of existing works (FiLM, [Dumoulin'2021) for applications in personalized and FL-based image classification which outperforms BiT.
summary_of_the_review: The paper proposes a technique, FiT to yield parameter efficient models that outperform BiT by combining transfer learning with meta-learning.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper:    The paper proposed a parameter and data efficient architecture for low-shot transfer learning consisting of an automatically configured Naive Bayes classifier and FiLM layers that are used to adapt a fixed, pretrained backbone to a downstream dataset. 
strength_and_weaknesses:   Strength: The method proposed in this paper is simple, efficient and easy to implement. The adequate experimental results on few-shot,VTAB-1k benchmark, Personalization video dataset and Few-shot Federated Learning verify the efficiency of FiT.


Weaknesses: The motivation for using a Naive Bayes classifier is not clear, why it would be better than a traditional MLP classification layer.The design form of Feature-wise Linear Modulation (FiLM) layers is very simple and direct. Why is such design effective?

clarity,_quality,_novelty_and_reproducibility:   The quality of the paper is relatively high, but the methods section is not clear enough, it is slightly difficult to read, and it is  original.
summary_of_the_review:    This paper proposed Feature-wise Linear Modulation layers, which can be easily inserted into Resnet or Efficient networks, and only a few parameters are added. The use of an automatically configured Naive Bayes classifier to replace the traditional classification layer significantly improves the performance of the few shot and image/video classification.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes FiLM Transfer (FiT) which combines ideas from transfer learning and meta-learning for developing data-efficient and parameter-efficient learning. Empirical results shows that FiML achieves superior performance at low-shot on several benchmarking datasets. The results also confirm FIT's parameter efficiency and superior accuracy in distributed low-shot personalization and federated learning applications. The paper is well motivated and the works provides an extensive set of experiments supporting the efficiency and effectiveness of the proposed method. Thus, it is of value to the ICLR community.
justification_for_why_not_higher_score: While the problem is well-motivated and the experiments confirm the effectiveness of the proposed method, the proposed approach is mainly combination of existing ideas in transfer learning and meta-learning.
justification_for_why_not_lower_score: The proposed method is effective and of interest to the ICLR community.
Decision: Accept: poster