Venue: ICLR 2023 TinyPapers

Reviews:

final_score: rating: High Potential (HP): a submission which meets the reviewing criteria and has potential to make an impact on the field
review_content: summary_of_contributions: A new feature learning technique for continual learning is proposed.
strengths_and_weaknesses: S1: Overall I think this paper is clear and easy to understand.

S2: Experimental results show that the proposed method is somehow effective.

W1: I would say the intuition behind the proposed t-mFV similarity is not very clear, I have to refer to the cited papers to understand more details on the method. I would recommend adding some figures to better illustrate the idea of the proposed similarity.

W2: It seems that the proposed method does not achieve the best results when the buffer size is larger, and given that the current similarity is just the t-vMF similarity from  (Kobayashi, 2021), I would be interested in how to further integrate the technique in the continual learning setting.
suggested_changes: C1: I would suggest to further clarify the intuition of the t-vMF similarity.

C2: I would suggest to bold only the best numbers in the comparison table, it is misleading to just bold the number from the proposed method.
review_title: null

final_score: rating: Great Start (GS): a submission which meets some of the reviewing criteria but has room for improvement
review_content: summary_of_contributions: This paper proposes a new method for improving the representation of data in continual learning by using a different similarity metric instead of cosine similarity in supervised contrastive loss. The authors suggest using a similarity function based on the von Mises-Fisher distribution, further extended with a student t-distribution (t-vMF) for representation learning. They validate their method on an image classification dataset and achieve promising results.
strengths_and_weaknesses: Strength:
- The paper proposes to use t-vMF as the similarity metric in supervised contrastive learning to improve representation learning in continual learning.
- The results of the experiments are promising and show that the proposed method outperforms some existing baselines.

Weakness:
1. Motivation for using a different t-vMF as a similarity metric. Cosine similarity has been proven effective in many areas. I wonder what is the motivation for replacing it? Do the authors find some specific weakness in the cosine similarity metric, and how does the proposed metric address this weakness?
2. Motivation for using it for continual learning. Why do authors specially choose the setting of continual learning to validate the proposed metric? Is the proposed metric specifically useful for continual learning, and if so, why? 

See "Suggested Changes" for other weaknesses and potential improvements.
suggested_changes: I understand this is a tiny paper with 2 pages limit, but the authors can further clarify and extend these in either the main text or appendix to make the paper clearer:
1. Clarification of motivations. Per weakness 1, the authors should give a direct intuition of why t-vMF is needed and what are the benefits of it compared to cosine similarity. Per weakness 2, the authors should explain why using continual learning for evaluation, because if we want to validate the effectiveness and generalization of the proposed metric, we usually start with the most common setting (e.g. the most basic image classification) and then potentially extend to other more challenging settings.
2. The experimental settings of class incremental learning and task incremental. E.g., how many stages are there in total? How many classes/tasks are there in each stage? What are exactly the tasks? ... 
3. More comparison experiments. As the proposed metric performs worse than Cha et al., 2021 when the buffer size is 500, the authors can consider adding more different buffer sizes to study how the proposed metric relates to the buffer size. E.g., does it work better when the buffer size is smaller? 
4. Experimental details. What are the detailed experimental settings, e.g. how do the authors tune the hyper-parameters as I think it can make a big influence, given the results of different methods are similar. Also, how sensitive is the proposed method to the hyper-parameter $k$?
review_title: null

Meta-review: metareview: This paper proposes to use a different similarity measure that is t-vMF for supervised contrastive learning in the continual learning setting, the experimental results are clear to show that the propsod method is effective, yet it is raised by both reviewers that the motivation of using t-vMF similarity is not clear, and also more experiment and implemental details are needed to make the paper reproducible.
Overall, this paper needs some revisions to be CCR.

confidence: 5: The area chair is absolutely certain
Decision: Invite to archive