Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes to use areas to model prototypes in FSL. The prototypes, named as big prototypes, are represented by hyper-spheres with dynamic sizes. Rather than point-based prototypes, the new area-based prototypes  in the embedding space can represent the class-level information with more expressivity.
main_review: Strengths:
1) The motivation of the paper is interesting. Using two sets of learnable parameters to represent the class prototype can reduce the influence of biased data.
2)The metrics of the proposed big prototypes are easy to calculate. Both the radius and hyper-spheres center participate in the backward propagation, which leads to the adjustable centers and radius corresponding to different categories.
3) Extensive experiments in both NLP and CV evaluate the effectiveness of the proposed method, especially under the setting of cross-domian FSL learning.


Weakness:
1) More analysis on the experimental details are recommended. I am interested in the feature distribution in the training process. It seems that only the points outside the hyperspheres affect the loss function. That means the proposed method can reduce the number of outliers of classes, and the learned features gather more closer in each class compared with the original prototypes method. It is better to provide some visualization of feature distribution.

2) The applicability of the method is somewhat limited. The result of 1-shot task on miniImageNet is missing. In 1-shot learning, the proposed model cannot estimate the radius of the class by a single image, so the model sill suffers from the point-biased limitation?


summary_of_the_review: The paper proposes "big prototypes" for few shot learning, which represents the prototypes by hyper-spheres with dynamic sizes. The new method is easy to implement and can reduce the influence of biased data. Extensive experiments in both NLP and CV demonstrate the effectiveness of the proposed method. More details or analysis are needed to improve the paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposed to represent prototypes by hyperspheres with dynamic sizes. 
A so-called big prototype is characterized by the center of the hypersphere and the radius of the sphere. 
Empirical results are conducted on few-shot named entity recognition (NER), few-shot relation extraction (RE) and few-shot image classifica- tion. 
 
main_review: The paper extends and mainly compares with prototypical network published in 2017. 
However, there are many works trying to improve prototypical network to model the prototypes more appropriately in literature. To name a few, see the following 

[1] Distribution Consistency Based Covariance Metric Networks for Few-Shot Learning, AAAI-19
[2] Variational Few-Shot Learning, ICCV-19
[3] A Two-Stage Approach to Few-Shot Learning for Image Recognition, TIP-19

As for ``Ren et al., 2018; Gao et al., 2019a; Allen et al., 2019; Pan et al., 2019; Ding et al., 2021a" mentioned in introduction, can you compare with them empirically? Instead of just saying they "cannot uncover the overall ground truth distribution". 
Why using hyperspheres  is better than using a distribution? Any theoretical analysis? Especially, please note that the hypersphere is still regular, while a distribution can be complicated. 
Without fully discuss and empirically compare with the above-mentioned works, I cannot judge whether the performance gain is meaningful. 

summary_of_the_review: The paper lacks  thorough literature review. To decide whether this solution indeed works, the authors should fully discuss and empirically compare with existing works. Just comparing with prototypical network is far from enough. 

==== After rebuttal

Thanks for the new empirical results. I raise the score because the empirical part is now more completed. 
However, it is still unknown why using hyperspheres can obtain better performance than others (especially those using distribution)  consistently. More insights and theoretical analysis are still required.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In few-shot learning, prototypes have been widely used to represent classes and then classification can be performed by computing distances to prototype representations of each class. This paper proposes to use hyperspheres to model prototypes in the feature space, instead of vector points, to enhance the expressivity of class-level information. The proposed hypersphere model only needs two sets of parameters (the center and the radius of hyperspheres), so it does not bring additional burden to the optimization calculation of the objective function. Extensive experiments in both NLP and CV are conducted to evaluate the effectiveness of the proposed model. The results shows that the proposed model significantly outperforms the baseline, and it performs well in cross-domain few-shot relation extraction.
main_review: Strengths:
- The proposed big prototype model is a simple and easy to understand extension of the prototype model, which brings more adaptability to the latter. The simplicity of the model makes it easy to generalize to tasks that require a prototype model.
- The experimental section has demonstrated the effectiveness of big prototypes in three tasks across NLP and CV, which are few-shot named recognition, few-shot relation extraction, and few-shot image classification. 
- The additional analysis of the change of hypersphere radius with sample density is helpful to understand the mechanism of big prototype model more intuitively.
- The writing is mostly clear and easy to follow.


Weaknesses：
- In the paper, the metric of two vectors is calculated by a L2 norm distance function. However, in some tasks using prototype models, cosine similarity metric considered more appropriate. So can the big prototype model be effectively applied in such a scenario? Is there anything that needs to be adjusted?
- The feature representation of all samples of a class often has a specific manifold distribution, not necessarily Gaussian distribution. What is the effect on the manifold distribution of feature representation before and after optimization with big prototype model? Could relevant visualization results or analysis be provided to show this impact?
summary_of_the_review: As the author stated in Section 4, the experimental goal is not to lead in all the leaderboards, but to verify the effectiveness of the model only with baseline. Therefore, from this point of view, the proposed model is indeed better than the baseline model for comparison.

For me, the method proposed in the article is simple and has aroused my thinking. At the same time, it also brings some more worthy questions. I expect the author’s answer. At present, I think the proposed model is effective and enlightening. Therefore, I tend to suggest acceptance. Of course, I am also happy to see different opinions from other reviewers, and further adjust the score according to the feedback of the author.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose to augment an existing prototype-based few-shot learning approach by augmenting each prototype with a radius. They detail varied numerical experiments showing the advantage provided by this addition.
main_review: # Strength:
- evaluation on a variety of tasks from different domains
- simple, intuitive, well-justified idea

# Weakness:
- slim contribution:  addition of one parameter to an existing network
- only tested out on a single prototype network from 2017, which hasn't been SOTA for a long time. No comparisons to more recent prototype baselines [1,2] or approaches such as meta-learning [3], or self-supervision [4]. This lack of comparison makes the claims much less substantial. Big prototypes improve on vanilla prototypes, but do they make Euclidean prototypes competitive wrt modern methods?
- While the writing is generally clear, there are many mistakes and at times clumsy formulations getting in the way of clarity and rigor
- The way the prototypes are initialized (3.3) is not very clear to me at all. What is the benefit of this approach instead of simply computing the average center and radius? And if it is a contribution, why isn't it evaluated?


# Suggestions:
The authors could turn the extreme simplicity of their approach into a strength if they adapted their method to a variety of prototype-based algorithms, including more recent and better-performing ones. Spheres exist in hyperbolic space, for example.

# Details:

**abstract:**
- dense vectors = prototype?
- the radius of the sphere is not constant if I understand it properly
- what is an atactic manifold?
- the authors are constantly the world "metric" to mean "distance in a metric space." This makes for some confusing sentences
introduction:
- what is the meaning of derivative in  "derivative prototype-based methods"?
- how are Big prototypes easier to "model"?

**Fig1:**
- the fact that the embeddings change as well makes the figures hard to understand.
- are the center and radius really learned separately and not end-to-end? This is not my understanding from the rest of the paper
Related work
- type of NMS
-  "find the true location of the prototypes of the entire dataset" I don't think there is such a thing as a true embedding for a prototype.

**3 Methods**
-  "One big prototype is represented by two terms" -> parameters
- why limit yourself to the case in which the data is a vector of dimension L? it could be an image, for example
- "We now introduce big prototypes, which are a series of hyperspheres " -> a set of hyperspheres 
- 3.3 is quite confusing to read. For example, z^j_n is never defined. 

Equation 5: \tilde{r} is indexed by n, and the sum at the denominator is not indexed. It makes it look like the \tilde{r} could be canceled out, when it is obviously not the case

**Algo1:**

Init: And why not just average over Dn?

**Table1**
 P,R,F never defined. "spaticularly"

[1] Gao et al.  Hybrid attention-based prototypical networks for noisy few-shot relation classification. AAAI, 2019.

[2] Mettes et al. Hyperspherical prototype networks. NeurIPS 2019

[3] Rusu et al.  Meta-learning with latent embedding optimization. ICLR 2019

[4] Gidaris et al. Dynamic few-shot visual learning without forgetting CVPR 2018




summary_of_the_review: A good idea with an interesting multi-domain evaluation. However, the lack of comparison with modern approaches and the use of a single backbone from 4 years ago makes the method not as convincing as it could be. Ultimately, this is not enough for ICLR.

I strongly encourage the authors to adapt their simple idea to a wider range of methods (including hyperbolic prototypes) and add more baselines (and also significantly polish the writing and rigor). There is a potentially very good paper to come from this.

Post discussion: the authors have addressed my concerns and have added many experiments to better substantiate their claim. In the end, a simple yet interesting idea.
review_title: null

Meta-review: Thanks for your submission to ICLR.

This paper presents an extension to prototypical networks based on using hyperspheres to represent the prototypes.  Strong empirical results are presented using this approach.

Overall, this is a very borderline paper and could go either way.  The idea itself it simple, though the results seem to be fairly strong.  I read through the paper myself and tend to think that it could use a bit more work before it's ready.  Some of the issues raised by the reviewers---particularly with respect to experiments and literature review---are worth nailing down.  Further, I think that the method could be explored in a more principled/theoretical way.  For instance, when reading this idea, the first thing that pops into my mind is that representing the prototype with a hypersphere is very similar to representing a distribution (e.g., a Gaussian) using a mean and covariance (in this case, a spherical covariance).  Indeed, if you take the KL divergence between two spherical Gaussians, you get something very similar to the expression used in the paper.  This is all to say that there may be other more general directions to take this idea, or other interpretations of what is going on.

Please do keep in mind the comments of the reviewers when preparing a future version of the manuscript.
Decision: Reject