Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper describes a few-shot learning approach that takes into account the semantic relatedness of the ground truth labels rather than just using them as binary labels. In the few-shot learning setting, the paper describes a method to train the semantic feature learner (on labels) during the meta-training phase and then at meta-test time use the semantic feature learner to get the said features for the query images. This way both the support and query examples would have semantic features and empirically it has shown to work on many few-shot benchmarks on computer vision datasets.  
main_review: Strength:
The paper is clearly written and easy to follow. 
Empirically they show that using such semantic features clearly shows improvements over the visual features. 

Questions:
(i) The method (TASNet) is built over Protonet (Snell et al) method. Does it show similar performance gain with other popular architectures such as MetaOptNet, etc.?
(ii) Details on the architectures of visual feature learner (f) and semantic feature learner (g) are missing?

Weakness:
The motivation for the paper involves robust few-shot learning with different categories under similar background, interference. The results are only on the full test set. Results on a few episodes where such conditions exist would cement the robustness of the method.
summary_of_the_review: Overall the paper suggests a method that makes an improvement over an existing state-of-the-art, but the efficacy of their enhancements are not fully tested. In principal adding textual data that is semantically less ambiguous than image data helps.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In few-shot learning for image classification, visual features alone may represent multiple objects in an image. The label text provides additional information that could serve as useful inductive bias, indicating which object in an image the model should pay attention to. There are multiple ways to use such text information to help few-shot classification. The authors propose to build a model that predicts both the image classes and the text label (or its Glove embedding). In other words, an additional loss term is introduced to regulate the learning of visual features. The authors show that the additional "semantic features" (because Glove embedding is used) help improve few-shot image classification.
main_review: The paper proposed a reasonable model to use the text labels in image classification. The experimental results look convincing. With that said, the proposed solution looks overly complicated.  A simple baseline should be considered: a multi-task learning model with two outputs, one to predict the image class,  the other to fit the Glove embedding. Compared with this simple baseline, the extra complicity of the proposed model should be better motivated. Also it would be good to have an empirical comparison with the simple two-outputs baseline.


summary_of_the_review: The authors use additional information in the training labels to improve few-shot image classification. The solution is a combination of different existing tricks. Without proper ablation experiments, it is difficult to infer generalisable knowledge from the results.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a task-adaptive semantic feature learning mechanism to incorporate semantic features for both support and query samples. Two modality combination implementations, feature concatenation and feature fusion, are devised based on the semantic feature learner. Experimental results are provided on four benchmark datasets that demonstrate that the proposed mechanism outperforms existing methods.
main_review: Strengths:
+ A task-adaptive semantic feature learner is proposed to incorporate semantic features for both support and query samples.
+ Two modality combination implementations, feature concatenation and feature fusion, are proposed.
+ The visual and semantic features are learned separately, thus preserving the structural heterogeneity of different modalities.
+ Experimental results are provided on four benchmark datasets that demonstrate that the proposed approach outperforms existing approaches.

Weaknesses:
- A clear definition of N-way K-shot few-shot tasks is not provided.
- What does interference mean in the context of Figure 1?
- What is the definition of semantic features? Are they text-based features?
- How are visual-feature class centers and label-derived semantic features computed?
- What is the difference between D_b, D_v, and D_n datasets?
- Effort has to be put in Section 3 to improve its clarity and readability. It is highly recommended to clearly define the problem to be addressed and the approach to be outlined along with any notation in the beginning of the section.
- No discussion of the baselines and their settings is provided. Also, what is the meaning of dashes in Tables 1 and 2?
- It is not clear why jointly learning visual and semantic features will not improve performance?

Some minor comments follow:
- Abstract: "to incorporates" -> "to incorporate"
- pg. 1: Please define acronyms before they are first used, e.g., FSL
- pg. 2: "Few-shot classification methods exhibits" -> "Few-shot classification methods exhibit"
            "learning a discriminative representations" -> "learning discriminative representations"
- pg. 3: "the these methods is" -> "these methods is"
- Table 2 caption: "in Tabel1" -> "in Table 1"
- pg. 9: "except" -> "expect"
- pg. 6: "We computes" -> "We compute"
summary_of_the_review: Overall, I recommend the rejection of the paper. My major concern is that the main contribution of the paper, i.e., separately learning the visual and semantic features, is not very intuitive and well justified. Furthermore, the proposed modality combination approaches are well-known methods.

----After rebuttal----

I would like to thank the authors for their effort to answer my questions. Even though the technical novelty of the paper is limited, as also supported by the new experiments added, the responses to my questions are satisfying. Thus, I increase my score from 3 to 5.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a few-shot learning method that incorporates semantic features (i.e., class label embeddings) for both support and query samples.
Although some existing methods use the semantic features for support samples to improve the performance, no methods use them for query samples. 
To this end, the proposed method regresses the semantic features for the query samples from the support samples.
Experiments show the effectiveness of the proposed method.
main_review: Strengths
- Using additional semantic information such as class label embeddings is interesting in few-shot learning problems.

Weaknesses
- The proposed method seems incremental to me because the only difference from the existing method is that it predicts the semantic features for the query set from the support set and uses it for the few-shot classification.
- There is room for improvement in the clarity of the paper. For example, I didn't know what "semantic features" meant until I read section 3.2.
I recommend describing concrete examples of it at the early stage of this paper such as Introduction, which also clarifies the problem setting tackled in this paper.
In addition, the explanation of the proposed method (Section 3, 3-1, and 3-2) seems unnecessarily redundant. 
- The experiments are unconvincing. The effect of using pre-trained feature extractor $E_{\theta}$ is unclear. Did comparison methods use this pre-trained extractor in the experiments? Although the proposed method seems to work well in Tables 1 and 2, I don't know if the semantic features for query samples introduced in the proposed method are working because the description of the comparison methods is insufficient. Did you confirm that the effect of learning the semantics feature learner locally per each task? 

Minor Comments
- index $c_i$ is a bit confusing in eq. (2) since $c_i \in$ { $1,\dots, N$ } . ${\cal S} _c$ is undefined in eq. (3).


summary_of_the_review: I have some concerns described in Main Review. Therefore, I recommended "3: reject, not good enough".

----After rebuttal----

Thanks for the response. I still have some doubts about the technical novelty of the proposed method, but
given the other reviews and response, and the strength of the experimental results, I increase my score from 3 to 5.
review_title: null

Meta-review: This paper proposes a few-shot learning method that learns task-adaptive semantic features that can incorporate for both of the support and query sets. Two approaches for modality combination are developed. The additional experiments in the author response addressed some concerns of the reviewers. However, the technical novelty of the proposed method is high enough since the proposed method uses existing techniques.
Decision: Reject