Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to address the complex problem of cross-modal semi-supervised few-shot learning with noisy data. A rather complex approach is proposed with combines Bayesian mixture mode, VAE, GAN, and prototypical learning. 
main_review: This paper provides enough details of their method, and shows performance improvement over some existing methods. However, I have major concerns about the problem formulation, the motivation of the method, and the experiments. 

This paper aims to address four problems at the same time: (1) few-shot learning, (2) multimodal FS learning, (3) semi-supervised FS learning, and (4) FS learning under data noise. It is not clear what are the unique challenges posed by each problem, and why this paper combines all the problems together to create a new and complex problem. There need to be more intuitive justifications on the importance of the problem studied. 

The method is also quite complex and not well-motivated. It is unclear what challenge doe each component address. It is also difficult for me to understand what value does the proposed mixture model offer that is specific to the problem studied.

The experiment setting is confusing to me. Could the author explain in more details about the (1) multimodal data, (2) semi-supervised setting, and (3) dataset noise. It seems to me that all the baseline results are reproduced on the new setting (please correct me if I'm wrong), because previous methods are not targeted to this problem. In such cases, the improvements over baselines are expected. The authors either need to adapt the baseline, or provide more justifications on why the baselines are strong enough. 
summary_of_the_review: This paper has limitations in the problem formulation, the motivation of the method, and the experiments. In my opinion, it needs major improvement.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper presents a cross-modal semi-supervised few-shot learning approach for image classification. The idea is to train variational auto-encoder (VAE) with both image and text data for learning a feature representation. Then features are extracted from a test sample and assigned to the class of the closest prototype train sample. In addition, a generative adversarial objective is employed for learning the latent code of the VAE. Since the proposed idea is meant for learning from noisy labels, there is an uncertainty prior in the image features as part of the infinite Gaussian mixture distribution. The approach is evaluated on standard few-shot learning benchmarks which are modified to account for the noise labels. The results are promising compared to the prior work. 
main_review: Paper strengths:

+ The proposed approach relies on existing methodologies that are combined in a new way. 

+ The problem is clearly described and the paper is well-written.

+ The approach delivers good performance in comparison to the related approaches.

Paper weaknesses: 

- The paper focuses on image-text multi-modal learning. In the introduction, this is can be only implicitly inferred after reading the introduction. A clear problem statement would be necessary.

- Novelty: It is not clear in which sense is the variational autoencoder (VAE) heterogeneous. The term is actually not defined in the context of the problem. Moreover, relying on text and images to train the VAE is not unusual. It should be clarified what is heterogeneous and why it is novel.

- Although the paper is well-written, it relies on several approaches which have to be described. The current structure of the paper does not help to understand the complete approach. For instance, the explanation of the uncertainty prior is not clear. Moreover, the method section lacks an overall description of the approach prior to describing the details. Overall, the method section could be improved to help the reader easier follow the paper's idea.

- Symmetric and asymmetric noise are not often encountered in real-world problems. It would be more interesting to see semantic label noise being considered. 


Improvements:

- Omniglot could be considered in the evaluations too. It is also a standard benchmark. 
summary_of_the_review: Overall, the paper addresses the problem of noisy labels next to cross-modal semi-supervised few-shot learning. This problem formulation makes the proposed work different from the existing approaches. Moreover, the proposed formulation could be considered novel regardless of combining existing approaches. The main issue of the paper is the difficulty to follow the proposed approach. One needs to read several times the paper for understanding the structure of the proposed method. This is a major point that needs improvement. Finally, considering semantic label noise would be the whole idea more plausible for real-world setups.

Post-rebuttal comment:

The rebuttal has helped to improve the paper. My concerns have been addressed.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper focuses on semi-supervised few-shot learning with multi-modality data. The authors introduce an uncertainty prior of an infinite Gaussian model, integrating multi-modality information from image and text data into a heterogenous variational autoencoder. Meanwhile, a new variational lower bound is derived for the inference of parameters. In addition, a GAN is developed for the data sparsity in few shot learning. Experimental results demonstrate the effectiveness of the proposed method to some extent.
main_review: # Strengths
1. The idea of introducing Bayesian deep learning to the semi-supervised few-shot learning problem sounds reasonable.
2. The provided experimental results show the effectiveness of the proposed method to some extent.

# Weaknesses
1. Why could the introduced uncertainty prior mitigate the impact of corrupted labels and outliers?  How to ensure the relationship between the uncertainty prior and the noisy data? It is suggested to give a more intuitive explanation. 
2. The authors claim that their robust variational lower bound can infer network parameters more accurately. However, it is hard to see the advantages of the robust variational lower bound intuitively from the paper.
3. It seems that there are multiple loss terms in the proposed objective function, such as Eq. (10). How does the author determine their respective weights during optimization? Do these weight parameters have much influence on the results?
4. It is suggested to give more implementation details, such as the network structure, experimental configuration, training and inference time.
5. The writing of this paper needs to be further polished. For example, "the parameters of infinite Gaussian mixture model" -> "the parameters of an infinite Gaussian mixture model"? "the performance gains shows" -> "the performance gains show"?

summary_of_the_review: Overall, this paper provides a new perspective on the cross-modal semi-supervised few-shot learning.
review_title: null

Meta-review: This paper aims to address the problem of cross-modal semi-supervised few-shot learning with noisy data, and proposed a robust cross-modal semi-supervised few-shot learning (RCFSL) based on Bayesian deep learning. The approach combines several existing techniques for tackling a new problem in a non-trivial approach. Empirical results demonstrate the effectiveness of the proposed method to some extent. 

While the proposed integrated complex approach seems to be novel in the proposed unique setting, there are some major concerns from the reviewers. One concern is about the lack of clear justification on technical contributions for the proposed methodology in the complex settings. In particular, it lacks of comprehensive ablation studies for analyzing and understanding the source of gains by the proposed complex method, and the baselines in the experiments also do not look strong enough. In addition, many aspects of the paper writing and presentation are not satisfied (e.g., the math formulation in Section 2 is densely presented making it difficult to follow). 

Overall, this is a borderline case, where the paper did contribute a new method for the interesting cross-modal semi-supervised few-shot learning task, but some major concerns on the weaknesses remain at its current form. Therefore, it cannot be recommended for acceptance. Nonetheless, I hope authors can improve the paper by fully addressing these issues and hope to see it accepted in the near future.
Decision: Reject