Venue: ICLR 2023 TinyPapers

Reviews:

final_score: rating: Great Start (GS): a submission which meets some of the reviewing criteria but has room for improvement
review_content: summary_of_contributions: Double descent (DD) is an interesting phenomenon in machine learning (or, in specific, overparameterized models), where the test loss increases and decreases as the model size grows. However, it was mostly analyzed in the context of supervised learning. The paper argues that double descent doesn't happen in self-supervised learning with an experiment. 
strengths_and_weaknesses: *Strengths*
- The findings are clearly communicated, and the paper appropriately discusses other relevant literature. 
- The topic of the paper is relevant to the ICLR community. 

*Weaknesses*
- Not much theoretical discussion on why DD might not occur in an unsupervised learning setup. 
- The argument is based on a single experiment, even though it is not straightforward how to vary the complexity of the model.
suggested_changes: - I believe the authors are tackling an interesting problem, and the research is essential. I have some suggestions for the experiments. Double descent can be observed in an overparametrized linear regression model, where the exact solution can be computed. Could the authors experiment on a linear autoencoder setup? Regarding how to vary the model's complexity, the authors can fix the dimension and vary the number of training examples to see if DD happens; this allows to separate regions where N >> D, N ~= D, and N << D. 
- I would be surprised if DD does not happen in the above setup since just optimizing the encoder makes the problem to be the same as linear regression (ignoring the interaction with decoders). However, including this will make the justification more convincing.
review_title: null

Decision: Invite to archive