Venue: ICLR 2023 TinyPapers

Reviews:

final_score: rating: High Potential (HP): a submission which meets the reviewing criteria and has potential to make an impact on the field
review_content: summary_of_contributions: The authors show that in multiclassification and unbalanced binary settings with noisy labels, Bayes decision rules are unidentifiable. 
strengths_and_weaknesses: The paper is well-written and assesses a meaningful problem. The authors motivate their contributions very clearly and concisely.  The proofs are well-supported and written which improves understandability and engagement with the concepts. 

Given that it's a tiny paper, I think the authors did an excellent job of putting the key ideas forward. 
Although most content is in the appendix, the authors provided key ideas and theorem statements in the main body which made it easier to follow. I believe the full paper will reduce the work posted in the appendix, but in general, I find the presentation excellent. 

The authors do answer fundamental questions, however, they don't provide a nice link to related work. The authors should improve their assessment of related work and highlight the gaps they novelly bridge in the literature. 
suggested_changes: The authors should improve their assessment of related work and highlight the gaps they novelly bridge in the literature. 

Not suggestions, questions:

- I am curious about cases where y is continuous and when data is a mix of categorical and non-categorical variable.

- Would identifiability be achieved if one maps the balanced multi-classification model to a balanced paired binary classification model?
review_title: null

final_score: rating: Clear, Correct, and Reproducible (CCR): a submission which meets the reviewing criteria
review_content: summary_of_contributions: This paper focuses on classification with noisy labels and the difficulty of dealing with the unknown distribution of label noise. The authors demonstrate that in most cases, the Bayes decision rule is unidentified when dealing with noisy labels, indicating that knowledge of the noise distribution is necessary. However, they also propose a simple method for learning the Bayes decision rule without prior knowledge of the noise distribution, as long as it can be identified.
strengths_and_weaknesses: Strengths: 

- The paper is well-written and presents a thorough investigation of the identifiability of the Bayes classifier in classification scenarios with noisy labels.
- The authors provide a clear explanation of the problem and their findings.

Weaknesses:

- The paper's strengths include its theoretical rigor and the development of a simple algorithm to learn the Bayes decision rule in certain cases. However, the paper's narrow focus on the Bayes classifier may limit its practical applicability, and it would benefit from more discussion of other approaches and methods for classification with noisy labels.
suggested_changes: Suggested changes: 

- The authors could expand their discussion to include other approaches and methods for classification with noisy labels beyond the Bayes classifier, as well as potential applications and real-world examples of the problem.
- Additionally, showing experimental results on a simple dataset would improve the paper. I would recommend two experiments: 1) demonstrating that the proposed method works well regardless of the label noise distribution in the balanced binary classification problem, and 2) showing the failure of the Bayes classifier in the imbalanced binary classification problem.
review_title: null

Meta-review: metareview: This is an interesting work that shows
- One can not easily relax the assumption that label noise distribution is known for training a noise-robust classifier.
- However, this assumption can be relaxed in a balanced binary classification setting but not in general settings where classification label space is huge. 

I am not sure how the researcher will use this finding but is worth noting. 
confidence: 5: The area chair is absolutely certain
Decision: Invite to present