Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The paper addresses the question of learning with noisy label
The idea is to consider an underlying causal graph explaining the observed noisy label. 
To do so the authors consider that there is an underlying generative process and that the observed noisy label is  generated both by the true label and a relevant representation of the image of interest
The paper the provides a variational inference method based on this generative process and on a relevant representation of the noisy image
This is combined with coteaching already introduced in [9]
Experiments are thereafter performed on several datasets and results (taking into account std) are quite comparable with the T-Revision approach and outperforms co-teaching when the noise is severe

main_review: * The idea of modeling noise generation through a generative process is original and new
* More precision should be given about the way some noise is added to the data. A reference is given but one may wonder the impact of the way the noise is added on the performance of the algorithm, especially when using variational inference. Can you add a discussion on the impact of the choice of the noise model? 
* Can you detail how noise impact precision and recall for each model?
* What about the choice of prior on Z and Y? 
* The claim is that the framework is general and that it can be combined with any method dealing with label noise. Can you explain the choice of co-teaching? What about combination of the framework with other label-noise learning methods?
limitations_and_societal_impact: No concern
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper describes an approach to utilize causal information in the form of a Structural Causal Model (SCM) to facilitate learning under instance-dependent label noise. The approach utilizes semi-supervised learning principles, relying on the causal origin of this effect due to the entanglement between the class conditional $P(Y|X)$ and the marginal $P(X)$ in the anti-causal direction.
main_review: The described approach utilizes the properties of anti-causal learning in a novel area by applying it to learning with noisy labels. It is applicable in the more complex case of instance-dependent label noise where most of the previous methods are not applicable, or require special assumptions. 
Clearly the present method also require some assumptions, but these are mainly causal, therefore more natural for a human expert to specify. According to the experiments the method achieve consistent improvement over the state of the art, and this improvement is increasing with increasing label-noise.

My main question is about the assumed causal model:
If the noisy labels are generated using crowdsourcing, and the worker only see the image itself, what mechanism provide the link between $Y$ and $\tilde{Y}$ ? What type of side-channel relay this mechanism?
If the human expert providing noisy labels can see the real object, the model is clearly valid, but this is not the typical case. 

What condition make the approximation that the clean label $Y$ is conditionally independent from the noisy label $\tilde{Y}$ given $X$ in the inference model acceptable, while the approach still benefits from assuming direct causal effect from $Y$ to $\tilde{Y}$? Or in other way, what would happen if we assume $X$ is enough to generate $\tilde{Y}$.

Algorithm 1 lists $\rho$ (the noise rate) as its input, where it is used? How realistic it is to know this value?

line 132 (minor error in text) but it is impossible [to tell] which class each cluster belongs to.

limitations_and_societal_impact: The author clearly addresses the assumptions made to make the noise transition model identifiable. These assumptions defining the limitations of the model. 
On the societal impact: Given this method can handle instance dependent label-noise, can it be applied in a situation where this instance dependence introduces a bias? For example if a protected attribute of an individual influences the noisy labels.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: In this paper, the authors handle the label noise problem by exploiting the causal information, which is a new perspective. The setting of instance-dependent label noise learning is more general and realistic in real life. The motivation of this paper is strong. The proposed method also has a good empirical performance and can be easily integrated into current methods.
main_review: The authors have proposed a novel method for label-noise learning. Different from most of the current works, instead of making assumptions on the transition relationship, the authors exploit the dependence relationship among variables when generating the noisy data.  In such a way, the important information for helping identify the transition relationship and clean labels can be revealed. To learn the useful information, a structural generative model is proposed and outperforms the state-of-the-art method. This paper is well organized. The novelty of this paper is high. To be best of my knowledge, this is the first paper to handle the instance-dependent label-noise problem from a causal perspective. But some of my major concerns might have to be addressed.
limitations_and_societal_impact: 1. If the noise is dependent on the instance, the proposed method works well, but will the proposed method introduce bias when the noise is independent of the instance? I think it is better to add some discussion on this point.
2. The experimental settings such as network structures and hyper-parameters of Figure 2 are not clearly introduced neither in the supplementary material or main paper. Please clarify it.
3. There might have a typo in either Table2 or Table 3. For IDN-40%, the accuracy of Mixup in Table 2 and Table 3 are exactly same, and the accuracy of Mixup in Table 3 is higher than the proposed method.
review_title: null

Meta-review: The paper considers the problem of label noise, but with the additional assumption that there is a causal relationship from the label to the instance (for example in image classification). By using a structural causal model, the authors propose a instance dependent noise model that shows good performance on benchmarks.

Three reviewers considered the paper, and all agree that the paper considers an important unsolved problem (instance dependent label noise), and provides a novel solution (via a structural causal model). The reviewers had several concerns which the authors satisfactorily addressed in their rebuttal. While one score was lower, the reviewer indicated in comments that they would support acceptance after the author rebuttal. A brief discussion post rebuttal among the reviewers resulted in all agreeing that the paper should be accepted. Therefore it is with great pleasure that I recommend the paper for NeurIPS.
Decision: Accept (Poster)