Venue: ICLR 2023 TinyPapers

Reviews:

final_score: rating: Great Start (GS): a submission which meets some of the reviewing criteria but has room for improvement
review_content: summary_of_contributions: Authors have provided a way of Self-Supervised Continual Learning (SCL) for regularization based class incremental learning by using randomly transferred labels as opposed to traditional data augmentation techniques.
strengths_and_weaknesses: Strength:
1. Use of randomly transformed labels as compared to general data augmentation is interesting.
2. Author also have shown their method outperformed on split-CIFAR10 and split-SVHN datasets for 5 tasks.

Weakness
1. More information is required for reproducing the results.
2. Better if the paper can be submitted as full paper not as tiny papers, lots of important details will get missed.
suggested_changes: 1. More information is required for reproducing the results.
2. Better if the paper can be submitted as full paper not as tiny papers, along with more details.
3. Better to share codes, so that other researchers can take reproduce the results.
review_title: null

final_score: rating: Great Start (GS): a submission which meets some of the reviewing criteria but has room for improvement
review_content: summary_of_contributions: This work proposes a self-supervised learning (SSL) phase before each task to improve class-incremental learning (CIL). Particularly, the authors proposed the Random Label Augmentation (RLA) task that randomly assigned labels to the unlabeled data. Experiments showed encouraging results compared to some baselines.
strengths_and_weaknesses: Strengths

- Combining SSL and CIL is a promising and interesting research direction. A suitable pre-text task for CIL will likely to have good impacts to the community.
- The results are encouraging.

Weaknesses
1. My major concern of this work is the motivation of the RLA loss. Particularly, it is unclear to me how RLA can benefit CIL. Training with random labels during pre-text training may cause more harm because the model may need to unlearn the wrong label order during supervised learning. The authors are encouraged to explain the benefits of RLA more clearly.
2. Other concerns and suggestions
* The Self-supervised continual learning setting is unclear. Which data is used during pre-text training, is it the same data during down-stream training? 
* The pre-text training phase is essentially a supervised learning. It is unclear how catastrophic forgetting does not arise during this stage.
* Why pre-text training is designed as a multi-task learning instead of a simple single task learning?
* What are the advantages of the proposed RLA compared to contrastive learning (e.g. using SimCLR loss)?
* The claim that RLA has better complexities compared to other baselines may need to be checked more carefully. RLA induces more parameters for the multi-task classifier, which is more expensive compared to the classifier for the rotation pre-text.
* It would be better to implement the proposed framework on better backbones (such as ResNets) which have better representation capability.
* More recent and standard baselines should be included.
* The reference style should be revised.
* The writing can be further improved. 


suggested_changes: Please see **Weaknesses**.
review_title: null

Meta-review: metareview: This paper introduces a new method that integrates self-supervised learning (SSL) and continual learning (CIL), demonstrating promising results on the split-CIFAR10 and split-SVHN datasets. However, as other reviewers have noted, the paper lacks essential details required for reproducibility, making it challenging to replicate the study. Additionally, the paper does not adequately explain the benefits of Random Label Augmentation (RLA) for CIL, raising concerns that training with random labels during pre-text training may be counterproductive.


confidence: 5: The area chair is absolutely certain
Decision: Invite to revise