Instance-dependent Label-noise Learning under a Structural Causal Model
Authors: Yu Yao
Tongliang Liu
Mingming Gong
Bo Han
Gang Niu
Kun Zhang
Venue: NeurIPS 2021 Poster (2021)
Keywords: Label-noise Learning
Structural Causal Models
Casual Inference

Abstract: Label noise generally degenerates the performance of deep learning algorithms because deep neural networks easily overfit label errors. 
Let $X$ and $Y$ denote the instance and clean label, respectively. 
When $Y$ is a cause of $X$, according to which many datasets have been constructed, e.g., \textit{SVHN} and \textit{CIFAR}, the distributions of $P(X)$ and $P(Y|X)$ are generally entangled. This means that the unsupervised instances are helpful to learn the classifier and thus reduce the side effect of label noise. However, it remains elusive on how to exploit the causal information to handle the label-noise problem. We propose to model and make use of the causal process in order to correct the label-noise effect.
Empirically, the proposed method outperforms all state-of-the-art methods on both synthetic and real-world label-noise datasets.

Instance-Dependent Label-Noise Learning under Structural Causal Models
Yu Yao
Tongliang Liu't
Mingming Gong?
Bo Han Gang Niu
Kun Zhang
TML Lab, University of Sydney; University of Melbourne; Hong Kong Baptist University;
"RIKEN AIP; Carnegie Mellon University
Abstract
Label noise generally degenerates the performance of deep learning algorithms because deep neural networks easily overfit label errors. Let X and Y denote the instance and clean label, respectively. When Y is cause of X, according to which many datasets have been constructed, e.g., SVHN and CIFAR, the distributions of P(X) and PXY|X) are generally entangled. This means that the unsupervised instances are helpful learn the classifier and thus reduce the side effect of label noise. However, it remains elusive on how to exploit the causal information to handle the label-noise problem. We propose to model and make use of the causal process in order to correct the label-noise effect. Empirically, the proposed method outperforms all state-of-the-art methods on both synthetic and real-world label- noise datasets.
Introduction
Learning with noisy labels can be dated back to [1] and has recently drawn a lot of attention [15, 19, 11, 10, 28]. real life. large-scale datasets are likely to contain label noise. It is partly because that many cheap but imperfect data collection methods such as crowd-sourcing and web crawling are widely used to build large-scale datasets. Training with such data usually lead poor generalization abilities of deep neural networks because they can memorize noisy labels [2, 33).
To improve the generalization ability of training with noisy labels, one family of existing label-noise learning methods is to model how the label noise was generated [17, 19, 27, 34, 14). Specifically, these methods try reveal the transition relationship from clean labels noisy labels of instances, ie., the distribution P(Ý X),, where Y, Y and X the random variables for the noisy label, latent clean label, and instance, respectively. The advantage of modelling label noise is that given only the noisy data, when the transition relationship identifiable, classifiers can be learned converge to the optimal ones defined by the clean data, with theoretical guarantees. However, the transition relationship is not identifiable in general. To make it identifiable, various assumptions have been made on the transition relationship. For example, Natarajan et al. [17] assume that the transition relationship is instance independent, i.c., P(Y|Y, = P(Ý P(Y))): Xia et al. [30] assume that the P(Ý Y, X) is dependent on different parts of an instance. Cheng et al. [7] assume that the label noise rates are upper bounded. In practice, these assumptions may not be satisfied and are generally hard to be verified given noisy data alone.
Inspired by causal learning [20, 25, 21, 23), provide causal perspective of label-noise learning method named CausalNL. We exploit the causal information to help identifiability of the tran- sition matrix XY) other than making assumptions directly on the transition relationship.
Correspondence to Tongliang Liu Siggliinnnii@syyyyey...dd.uu
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Specifically, we assume that the data containing instance- dependent label noise is generated according the causal graph in Fig. 1. For example, for the Street View House Num- bers (SVHN) dataset [18], represents the image containing the digit; Y represents the clean label of the digit shown on the plate; Z represents the latent variable that captures the infor- mation affectiig the generation of the images, c.g., orientation, lighting, and font style. Here Y is naturally a cause of X and the causal generative process can be described in the following way. First, the house plate is generated according to the street number and attached to the front door. Then, the house plate is captured by a camera (installed in a Google street view car) to form X, taking into account of other factors such as illu- mination and viewpoint. Finally, the images containing house numbers are collected and relabeled to form the dataset. Let us denote the annotated label by the noisy label Y as the annotator may not be always reliable, especially when the dataset very large but the budget limited. During the annotation process, the noisy labels were generated according to both the images and the range of predefined digit numbers. Hence, both X and Y are causes of Y. Note that most existing image datasets are collected with the causal relationship that Y causes X. For example, see the widely used FashionMNIST and CIFAR. When we synthesize instance- seraanee-deeedennl label noise based on them, we will have the causal graph illustrated in Fig. 1. Note also that some datasets are generated with the causal relationship that X causes Y. Other than using domain knowledge, the different causal relationships can be verified by employing causal discovery [26, 25, 21, 37].
Z
X
Y
Figure I: A graphical causal model which reveals generative process of the data which contains instance- dependent label noise, where the shaded variables are observable and the unshaded variables are latent.
When the latent clean label Y is a cause of X, P(X) will generally contain some information about P(Y X) This is because, under such a generative process, the distributions of P(X) and P(Y]X) are entangled [22, 39]. To help estimate P(Y with P(X), we make use of the causal generative process to estimate PY) which directly benefits from P(X) by generative modeling. The modeling of P( in turn encourages the identifiability of the transition relationship and helps learn P( ()). For example, in Fig. 2(a), we have added mnstance-dependent label-noise with a rate 45% (i.e., IDLN-45%) to the MOON dataset and employed different methods [10, 36] to solve the label-noise learning problem. As illustrated in Fig. 2(b) and Fig. 2(c). previous methods fail to infer clean labels. In contrast, by constraining the conditional distribution of the instances, i.e., restricting the data of each class to be on a manifold by setting the dimension of the latent variable Z to be 1 1-dimensional, the label transition as well as the clean labels can be successfully recovered (by the proposed method), which showed in Fig. 2(d). It is worth noting that the idea of finding P(Y|X) by modeling PYY) and P(X[Y) instead has been exploited in the context of domain adaptation; for instance, it inspires target shift, (generalized) conditional shift and other settings for domain adaptation [38. 9]
Specifically, to make use of the causal graph to contribute to the identifiability of the transition matrix, we propose a causally inspired deep generative method, which models the causal structure with all the observable and latent variables, i.c., the instance X. the noisy label Y, the latent feature Z, and the latent clean label Y. The proposed generative model captures the variables' relationship indicated by the causal graph. Furthermore, built on the variational autoencoder (VAE) framework [12]. we build an inference network which could efficiently infer the latent variables Z and Y when maximising the marginal likelihood p(X, Y) on the given noisy data. In the decoder phase, the data will be reconstructed by exploiting the conditional distribution of instances P(X|Y, Z) and the transition relationship P( X), i.c.
po ()
P(Z = P(Y y)po, (XlY y. Z 2)po, (ÝlY y, Xyddyy
2.31
will be exploited, where 0: = (01, 02) are the parameters of the causal generative model (more details can be found in Section 3). Ay high level, according the equation, given the noisy data and the distributions of Z and Y, constraining Pe, (X|Y, Z) will also greatly reduce the uncertainty of
Po, (Y X) and thus contribute to the identifiability of the transition matrix. Note that adding constraint on po, ( Z) is natural; for example, images often have a low-dimensional manifold [3]. We can restrict P(Z) to fulfill the constraint on Pe, (X) Z). By exploiting the causal structure and
2

---

nosy treining data
cD co-teaching
mixup
CausaINL
148
1.30
2
140
1.30
635
0.5-
6.0
030
12
-22
4.01
-152
-.3 455 00
61 10 35
-10
45
0F
19 15 24
-5 -55
90
14 20
-10 -5 00 85
(a)
(b)
(c)
(d)
Figure 2: (a) An illustration of the MOON training dataset which contains 45% of instance-dependent label noise. Different instances have different noise rates which are randomly generated according to Xia et al. [30]. (b)-(d) The illustration of the classification performance of co-teaching, mixup, and our method, respectively.
the constraint on instances to better model label noise, the proposed method significantly outperforms the baselines. When the label noise rate is large, the superiority is evidenced by large gain in the classification performance.
The rest of the paper is organized as follows. In Section 2, we briefly review the background knowledge of label-noise learning and causality. Section 3. we formulate our method, named CausalNI,, and discuss how it helps learn clean classifier, followed by the implementation details. Experimental validations are provided in Section 4. Section 5 concludes the paper.
Noisy Labels and Causality
In this section, firstly, we introduce how to model label noise. Then, we introduce the structural causal model and discuss how to exploit the model to encourage the identifiability of the transition relationship and help learn the classifier.
Transition Relationship To build statistically consistent classifier that converges to the optimal
classifier defined on clean data by only employing noisy data, the transition relationship P(||Y, X) has to be identified. Given an instance, the conditional distribution can be written in C' C matrix which is called the transition matrix [19, 29, 30J. where C' represents the number of classes. Specifically, for each instance x, there is a transition matrix T(x). The ij-th entry of the transition
matrix is (x) P(Y Y = j, X = x),, which represents the probability that the instance with the clean label = j will have a noisy label =
The transition matrix has been widely studied to build statistically consistent classifiers, because the clean class posterior distribution P(Y(a)) = [P(Y 1|X x)..... P(Y = C = x)) can be inferred by using the transition matrix and the noisy class posterior p( = [P(Ý
1|X = z)..... P(Ý = C = z)] i.c., we have P(Y = ((Y) Specifically, the transition matrix has been used modify loss functions to build risk-consistent estimators; see, e.g.,
[8, 19, 35, 28), and has been used to correct hypotheses to build Ulassifier-consister algorithms; see, e.g.. [17, 24, 19]. Moreover, the state-of-the-and statically inconsistent algorithms [11, 10] also diagonal entries of the transition matrix to help select reliable examples used for training.
However, the distribution |Y X) is not generally identifiable [28]. To make it identifiable, one has to resort to additional assumptions. The most widely used assumption is that given clean label Y, the noisy label Y is conditionally independent of instance X, i.e., P( = P(ÝIY). Under such assumption, the transition relationship PY) be successfully identified with the anchor point assumption [15, 34, 14]. However, in the real-world scenarios, this assumption may be hard to satisfied. Although PY can be used to approximate P X). the approximation error can be large in many cases. As for the efforts to model the instance-dependen transition matrix directly, existing methods rely on rather strong assumptions, e.g.. the bounded noise rate assumption [7], the part-depentn deteddepnndnt label noise assumption [30], and the requirement of additional information about the transition matrix [4]. Although the assumptions help the methods achieve superior performance empirically, they are generally difficult verify fulfill, limiting their applications in practice.
Structural Causal Model Motivated by the limitation the current methods, we provide new causal perspective to learn the identifiable of instance-dependent label noise model. Here we briefly
3

---

reconszuciin
loss
22
9
co loss
9
co-teaching J6ss
Rop (x1x)
21
2
Pla(" (X..X
OR loss
Reconssruction loss
Figure A working flow of our method.
introduce some background knowledge of causality [25] used in this paper. A structural causal model (SCM) consists of set of variables connected by set of functions. It represents flow of information and reveals causal relationships among all the variables, providing fine-gras aie-ggained description of the data generation process. The causal structure encoded by SCMs can be represented as graphical casual model as shown in Fig. 1, where each node is variable and each edge is function involving noise. The SCM corresponding to the graph in Fig. I can be written as
(1) where ez, cy, ex and Eq are independent exogenous variables, and they sometimes also called error variables. For example, ex are an error variable for X, which is responsible for any difference between the actual value of X and the value predicted on the basis of Z and Y alone. Each equation specifies a distribution of a variable conditioned on its parents (which are empty set for root cause variables in the graph).
Z = EZ, Y ey:
EY: X = =((Z,,,x), Y = -(x(XX,,
By making use of the SCM, the benefit of the instances to learning the classifier can be clearly explained. Specifically, the instance X function of its label Y and latent feature Z, which means that the instance X is generated from Y and Z. Therefore X must contains information about its clean label Y and latent feature Z. That the reason that P(X) can help identify PYY|X) and also X However, since we do not have clean labels, it is hard to fully identify XXY from P(X) in the unsupervised setting. For example, on the MOON dataset shown Fig. 2, we can possibly discover the two clusters by enforcing the manifold constraint, but it is impossible to see which class each cluster belongs to. We show the following that we can make use of the property of P(X)) Y) to help model label noise, i.e., encourage the identifiability of the transition relationship, thereby learning better classifier.
Specifically, under the Markov condition [20], which intuitively means the independence of exogenous variables, the joint distribution P(Ý, X,Y, specified by the SCM can be factorized as follows.
P(X, Y, Y. Z) = P()))(2))(X)), 2)) This motivates us extend VAE [12] to perform inference our causal model to fit the noisy data in the next section. In the decoder phase, given the noisy data and the distributions of Z and Y, adding a constraint on PY Z) will reduce the uncertainty in the distribution P(Y Y. X)) In other words, modeling P( Z) will encourage the identifiability of the transition relationship and thus better model label noise. Since P(Ý) Y. X) functions as a bridge connect the noisy labels to clean labels, we accordingly can better learn PXY or the classifier by only using the noisy data.
(2)
There normally two ways add constraints the instances, i.e., assuming specific parametric generative model or introducing prior knowledge of the instances. In this paper, since we mainly study the image classification problem with noisy labels, we focus on the manifold property of images and apply the low-dimensional manifold constraint to the instances.
3 Causality Captured Instance-Dependent Label-Noise Learning
In this section, we propose a structural generative method which captures the causal relationship and utilizes P(X) to help identify the label-noise transition matrix, and therefore, our method leads to a better classifier that assigns more accurate labels.

---

3.1 Variational Inference under the Structural Causal Model
To model the generation process of noisy data and to approximate the distribution of the noisy data, our method is designed to follow the causal factorization (see Eq. 2). Specifically, our model contains two decoder networks which jointly model distribution pe (X, Y|Y, 2) and two encoder (inference) networks which jointly model the posterior distribution qe (Z, YJX). Here we discuss each component of our model in detail.
Let the two decoder networks model the distributions ps, (X Z) and Po. (Y]Y, respectively. Let O1 and O2 be learnable parameters of the distributions. Without loss of generality, we set p(Z) to standard normal distribution and p(Y) to uniform distribution. Then, modeling the joint distribution in Eq. 2 boils down to modeling the distribution (X, fI,, 2), which is decomposed as follows:
Po X, YIY, Z) po, (X 2) po, X).
To infer latent variables Z and Y with only observable variables X and Y, we design an inference network which model the variational distribution go(Z, Y|Ý, X). Specifically, let go (Z|Y, X) and qa, (YI², X) be the distributions parameterized by learnable parameters Φi and Φ2, and then the posterior distribution be decomposed as follows:
L (Z, YI X) = toa (Z 21YYX)u g) Uo (YJY, X),
(4)
where we do not include Y as a conditioning variable in log (2 X) because the causal graph implies Z 1 YiX,Y. One problem with this posterior form is that we cannot directly employ To. (YIY, X) to predict labels on the test data, on which Y is absent.
To reduce computational costs and to allow our method efficiently infer clean labels, we approximate qo. (Y| x by assuming that given the instance X, the clean label Y is conditionally independent from the noisy label Y. i.e. To (YTÝ, = qt, (Y This approximation is expected not to have very large approximation error because the images contain sufficient information to predict the clean
labels. Thus, we could simplify Eq. 4 as follows
( X|x = 2o (Z X) (Y(X).
(5)
such that our encoder networks model qdh (Z|Y, X) and qd, (YJX), respectively. This way, lo, (Y|X) can be used to infer clean labels efficiently. We also found that the encoder network modelling qs: (Y|X) can directly act as a regularizer, which helps identify pe, pe, Y. YY X). Moreover, be benefited from this, our method can serve as a general framework which can be easily integrated with the current discriminative label-noise methods [28, 16, 10], and we will showcase it by collaborating co-teaching [10] with our method.
Optimization of Parameters Because the marginal distribution po (X, Y) is usually intractable, to learn the set of parameters (0,,02, Φ1, db2) given only noisy data, we follow the variational inference framework [5] to maximize the negative evidence lower-bound ELBO(x, 9) of the marginal likeli- hood of each datapoint (I, ū) instead of maximizing the marginal likelihood itself. By ensembling our decoder and encoder networks, ELBO(r; (z. ÿ) is derived as follows:
ELBO(2, 5) = [(x.))) [log Po, (æly, =)] + Ey-ga, (F12) [log Po, (7l,,, x)]
- -kl(a, (X Eyrgo, (M]s) [kl(go x)))(
(6)
where kl(.) is the Kullback-Leibler divergence between two distributions. The derivation details left out in Appendix A. Our model learns the class-conditional distribution P(X by maximiz- ing the first expectation in ELBO, which is equivalent to minimizing the reconstruction loss [12]. By learning P(X), the inference network qob (Y|X) has to select a suitable parameter d² which samples the y and z minimize the reconstruction loss E. ((,,,))).( (2) [log Po, (alyy =)]. When the dimension of Z is chosen to be much smaller than that of X, to obtain a smaller reconstruction error, the decoder has to utilize the information provided by Y. and force the value of Y to be useful for
Thhoreticallly this approximation is not necessary and can be casily removed. Specifically, we could let an encoder model 4oz (Y)), X). Then (({)x)) is obtained as q( ({{]x)= Σ, 962 (Y i, i, ))ess (Y' i.xX). where gog (Y [X) is the noisy class- posterros distribution and can be learned by using noisy training data.
5

---

Algorithm 1 CausalNL
Input: A noisy sample S, Average noise rate p. Total epoch Tmaz, Batch size N
1: For T= TT=1..... man::
2: For mini-batch S = {ai jirj:000
{ūr Jim in S:
3: Feed S to encoders 9of and Opf to get clean label sets L1 and Lg, respectively;
4: Feed (S, I1) to encoder døz to get a representation set H1, feed (5, L2) to log to get H2;
5: Update dog and dog with co-teaching loss;
6: Feed (L1, H1) to decoder Poj to get reconstructed dataset $j, feed (L2, H2) to do get Sz:
7: Feed (Si, I1) to decoder Pog to get predicted noisy labels f,, feed (S2, L2) to Pog to get lzz
8: Update networks lof. log. tos and Pøg by calculating ELBO on
LT). update
networks L0g. 263. Por and Pog by calculating ELBO on (S, S2, L2);
Output: The inference network Got-
prediction. Furthermore, constrain the Y to be one-hot vector, and then Y could be cluster ID of which the manifold of the belongs.
So far, the latent variable Y can be inferred as a cluster ID instead of clean class ID. To further link the clusters to clean labels, a naive approach is to select some reliable examples and keep the cluster numbers to be consistent with the noisy labels on these examples. In such way, the latent representation and clean label Y can be effectively inferred, and therefore, it encourages the identifiability of the transition relationship pe, (Y),,X) To achieve this, instead of explicitly selecting the reliable example in advance, our method is trained end-to to-dto- i... reliable examples are selected dynamically during the update of parameters of our model by using the co-teaching technique [10]. The advantage of this approach is that the selection bias of the reliable example [6] can be greatly reduced. Intuitively, the accurately selected reliable examples can encourage the identifiability of Po, (Y X)) and pe, (X), Z), and the accurately estimated pe, (Y) and Po, (X Z) will encourage the network to select more reliable examples.
3.2 Practical Implementation
Our method is summarized in Algorithm and illustrated in Fig. Here we introduce the structure of our model and loss functions.
Model Structure Because we incorporate co-teaching in our model training, we need to add a copy of the decoder and encoders in our method. As the two branches share the same architectures, we first present the details of the first branch and then briefly introduce the second branch.
For the first branch, we need a set of encoders and decoders to model the distributions in Eq. and 5. Specifically, we have two encoder networks
Y1 do9 (X), Z1 4og (X,Y))
for Eq. 5 and two decoder networks
X1 = :p: ((112))) Yi = pog Yi)
for Fq. 3. The first encoder de: (X) takes an instance X as input dot (X) and output predicted clean label Y,. The second encoder dog (X, Y1) takes both the instance X and the generated label Yi as input and outputs a latent feature Z1. Then the generated Yi and Zı are passed through the decoder pay (Y1, Z1) which will generate a reconstructed image X1. Finally, the generated X1 and Yi are the input another decoder Pog (X1, Yi) which returns predicted noisy labels Ý, It is worth mentioning that the reparameterization trick [12] is used for sampling, so to allow backpropagation in dog X, Vi).
Similarly, the encoder and decoder networks in the second branch are defined as follows:
Yz = (X), Ze ~ Log (X, Y2),
X₂ = doş (Y2, Ez), Y22 = pog (X2, Y2).

---

During training, we let two encoders lot (X) and Gog (X) teach each other given every bini-batch.
Loss Functions We divide the loss functions into two parts. The first part is the negative ELBO in Eq. 6, and the second part is co-teaching loss. The detailed formulation is leaved in Appendix B.
For the negative ELBO, the first term (=(),)))n Po, (aly, 2)] is a reconstruction loss, and we use the f1 loss for reconstruction. The second term is -Eyras (M(a) [Log pe, (gly, æ)], which aims to learn noisy labels given inference y and ær and can be simply replaced by the cross-entropy loss on outputs of both decoders Pag (X1, Y1) and Dog (X2, X2) with the noisy labels contained in the training data. The additional two terms are two regularizers. To calculate (g (Y a)) llp(Y)) we assume
that the prior P(Y) is a uniform distribution. Then minimizing (g (Y a (l)))) is equivalent to maximizing the entropy of Go1 (x) for each instance x, i.e., Ey lss (y(ar) log 201 (y/ar) The benefit for having this term is that it could reduce the overfiting problem of the inference network. For Eyras, (Y]=) [Rd (go Zly, (Z))], we let p( be standard multivariate Gaussian distribu- tion. Empirically, go ( (Zly. gg 2) is modeled by the encoders dog (X) and lez (X) which are designed to be deterministic mappings, therefore, the expectation can be removed, and only the kl term kl(gs(lyy x)) is left. When p(Z) is Gaussian distribution, the kl term nicely has closed form solution [12], i.e., - Ej (1 + log((a,)2) - (1²)² - (a,) where J the dimension of latent representation 2, and ory and μ4 are the encoder outputs.
For the co-teaching loss, we follow the work of Han et al. [10]. Intuitively, two encoders dsg (X) and dsp(A) (X) feed all data forward and selects some data of possibly clean labels. Then, two networks communicate with each other to select possible clean data in this mini-batch and use them for training. Finally, each encoder backpropagates over the data selected by its peer network and updates itself by cross-entropy loss.
4
Experiments
In this section, we compare the classification accuracy of proposed method with popular label-noise learning algorithms [15, 19, 11, 10, 28, 36, 16] on both synthetic and real-world datasets.
4.1 Experimental Setup
Datasets We examine the efficacy of our approach on manually corrupted versions of four datasets, i.e., FashionMNIST [31], SVHN (18), CIFARIO, CIFAR [13], and one real-world noisy dataset,
i.e. Clothing/MM M [32]. FushionMNIST contains 60,000 training images and 10,000 test images with 10 classes; SVHN contains 73,257 training images and 26,032 test images with 10 classes. CIFARIO contains 50,000 training images and 10,000 test images. CIFARIO and CIFARIOO both contain 50,000 training images and 10,000 test images but the former has 10 classes of images, and the latter has 10 classes of images. The four datasets contain clean data. We add instance-dependent label noise to the training sets manually according Xia et al. [30]. Clothing IM has 1M images with real- world noisy labels and 10k images with clean labels for testing. For all the synthetic noisy datasets, the experiments are repeated 5 times.
Network structure and optimization For a fair comparison, all experiments are conducted on NVIDIA Tesla V100, and all methods are implemented by PyTorrch Dimension of the latent representation is set to 25 for all synthetic noisy datasets. For encoder networks d (X) and le (X), we use the same network structures with the baseline method. Specially, we use ResNet-18 network for FashionMNIST, a ResNet-34 network for SVHN and CIFARIO, a ResNet-50 network for CIFARI00 without pretraining. For ClothingIM. we ResNet-50 networks pre-trained on ImageNet. The aumaeetgmennniionn methods random crop and horizontal flip are used for our method. For Clothing M, we use ResNet-50 network pre-trained on ImageNet, and the clean training data is not used. The dimensionality of the latent representation is set to 100. Due to limited space, we leave the detailed structure of other decoders and encoders in Appendix C.
Baselines and measurements We compare the proposed method with the following state-of-the-an approaches: (i). CE, which trains the standard deep network with the cross-entropy loss on noisy datasets. (ii). Decoupling [16], which trains two networks on samples whose predictions from the two networks are different. (ii). MentorNet [11] and Co-teaching [10], which mainly handle noisy labels
7

---

Table 1: Means and standard deviations (percentage) of classification accuracy on FashionMNIST with different label noise levels.
IDN-20%
IDN-30%
IDN-40%
IDN-45%
IDN-50%
CE
88.54+0.32
88.38=0.42
84.22+0.35
69.72=10.72
52.32
Co-teaching
91.21#0.31
90.30=:0.42
89.10=0.29
86.78
63.22#1.556
Decoupling
90.70=#0.28
90.34=0..36
88.78:0.44
87.54
68.72
MentorNet
91.57++.22
90.52++0.4
88.144+0.76
85.12+00.76
61.62++.42
Mixup
88.68+0.37
88.02+00.77
85.47+0.55
79.57+0.75
66.02+2.58
Forward
90.05++0.43
88.65+00.43
86.27+0.48
73.35+1.03
58.23++3114
Reweight
90.27::0.27
89.58=00.77
87.0441..32
80.6910.89
14.13
T-Revision
91.58:10.31
90.1110.61
89.46:10:42
84.01=11.14
68.99
CausaINL
90.8411..1
90.68:10.77
90.01:10.45
88.75:0.81
78.19
Table 2: Means and standard deviations (percentage) of classification accuracy on SVHN with different label noise levels.
IDN-20%
IDN-30%
IDN-40%
IDN-45%
IDN-50%
CE
91.5110.45
91.2110.43
87.87
67.15:1.665
51.01
Co-teaching
93.93+0.31
92.06=00.31
91.93++081
89.33+0.71
67.62
Decoupling
90.02::0.25
91.59:0.22
88.27+0.42
84.57=0.89
65.14
MentorNet
94.08+0.11
92.73=10.77
90.41+0.49
87.45=0.77
61.23
Mixup
89.7330.37
90.02::0.35
85.47+0.63
82.41#0.62
68.95
Forward
91.89+0.31
91.59++0.23
89.33++0.53
80.15+1.91
62.53
Reweight
9..44++0.3
92.32+00.51
91.31++0.67
85.93++0.8
64.13
T-Revision
93.1440.53
93.51+00.4
92.65++0.76
88.54+1.58
64.51+3.42
CausalNL
94.06+00.23
93.86+0065
93.82-+0.64
93.19+0.93
Table 3: Means and standard deviations (percentage) classification accuracy on CIFARIO with different label noise levels.
IDN-20%
IDN-30%
IDN-40%
IDN-45%
IDN-50%
CE
70.26
69.1510.65
62.4551.86
51.72=#1.34
39.42:2.52
Co-teaching
80.9610.31
78.56:0.61
73.4110.78
71.60:0.79
45.9242.21
Decoupling
78.7140.15
75.17
61.7311034
58.61+11.73
50.4342.19
MentorNet
81.03++0.24
77.22+00.47
71.83+00.49
66.18:10.64
47.89+2.03
Mixup
73.17+0.34
70.02+00.31
61.56++0.71
56.45+0.67
48.95+22.58
Forward
7..64
69.75+0.556
60.2110.75
48.81+2.59
46.27+1.30
Reweight
76.23+10.25
70.12+0.72
62.58-10.46
51.54
45.46:12.56
T-Revision
16.17
70.36:L0.54
64.09-10.37
52.42
49.02
CausaINL
81.47:00.32
80.38
7..5
1..60
67.39
by training on instances with small loss values. (iv). Forward [19], Reweight [15], and T-Revision [28]. These approaches utilize dlassdde transition matrix to correct the loss function. For these baselines, we follow the experiments settings of the original papers. We report average test accuracy on over the last ten epochs each model on the clean test set. Higher classification accuracy means that the algorithm is more robust to the label noise.
4.2 Classification Accuracy Evaluation
Results on synthetic noisy datasets Tables 2, 3, and report classification accuracy on datasets of F-MNIST, SVHN, CIFAR-1O, and CIFARIO0, respectively. The synthetic experiments reveal that our method powerful handling instance-dependent label noise particularly in the situation of high noise rates. Specifically, for all datasets, the classification accuracy of our method

---

Table Means and standard deviations (percentage) of classification accuracy on CIFARIOO with different label noise levels.
IDN-20%
IDN-30%
IDN-40%
IDN-45%
IDN-50%
CE 30,42==0.44
24.15+0.78
21.45=00.70
15.23#1.32
14,42#2.21
Co-teaching
37.96=0.53
33.43+0.74
1..04
25.60=0.93
23.97
Decoupling
36.53=00449
30.93+1..8
27.85=0.91
23.81#1.11
19.59#2.12
MentorNet
38.91+0.54
34.23++0.73
31.89+1..19
2..53++1.33
24.15++2.31
Mixup
32.92-+0.76
29.76+00877
25.92+1.26
23.13+2.15
21.31+1.32
Forward
36.38+00992
33.17=00.73
26.75++0.93
21.93++1.29
19.27+2.11
Reweight
36.73=0.72
31.91110911
28.39:1.46
24.12
20.23
T-Revision
37.24:10.85
36.5410.799
17.23:1.113
25.53
22.54::1.95
CausaINL
41.47:10.43
40.98:10.22
34.02.10.95
33.34
32.129:12.23
Table 5: Classification accuracy on Clothing In the experiments, only noisy samples are exploited train and validate the deep model.
CE
Decoupling MentorNet
Co-teaching
Forward
Reweight
T-Revision
caualNL
68.88
54.53
56.79
60.15
69.91
70.40
70.97
72.24
decrease much slower than that of baseline methods. Additionally, the classification accuracies these datasets are improved by using CausaINL, which implies that our method should capture the underlying data generation process, and then Y should be a cause of X for all these datasets.
For noisy F-MNIST, SVHN and CIFAR- 10, in the easy case IDN-20% almost all methods work well. When the noise rate is 30%, the advantages of causalNL begin to show. We surpassed all methods obviously. When the noise rate raises, all the baselines are gradually defeated. Finally, in the hardest case, i.c., IDN-50% the superiority of causalNL widens the gap of performance. The classification accuracy of causalNL at least over 10% higher than the best baseline method. For noisy CIFAR- 100, none of the methods works well. However, causalNL still overtakes the other methods with clear gaps for all different levels of noise rate.
Results on the woald noisy dataset On the real-world noisy dataset Clothing M,, our method causalNL outperforms all the baselines, as shown in Table 5, The experimental results also show that the noise type in Clothing MMM M is more likely to be instance-dependent label noise, suggesting that the instance-independent assumption on the transition matrix sometimes can be strong.
Conclusion
In this paper, have investigated how use P(X) to help learn instance-dependent label noise. Specifically, previous assumptions are made on the transition matrix, and the assumptions are hard to be verified and might be violated on real-world datasets. From inspired by causal perspective, when Y is a cause of X, then P(X) should contain useful information to infer the clean label Y. We propose novel generative approach called causalNI. for lestance-depeneer label-noise learning. Our model makes use of the causal graph to contribute to the identifiability of the transition matrix, and therefore helps learn clean labels. In order to learn P(X). compared to the previous methods, our method contains more parameters. But experiments on both synthetic and real-world noisy datasets show that bit sacrifice on computational efficiency is worth i.c., the classification accuracy of Na significantly outperforms all the state-of-the-arr methods. Additionally, the results also indicates that in classification problems, Y can usually be considered cause of X, and suggests that the understanding and modeling of the data generation process can help leverage additional information that useful in solving advanced machine learning problems concerning the relationship between different modules of the data joint distribution. In our future work, we will study the theoretical properties of our method and establish identifiability results under certain assumptions on the data-generativ process.

---

Acknowledgments
TL was partially supported by Australian Research Council Projects DP-180103424, DE-190101473, and IC-190100031. GM was supported by Australian Research Council Project DE210101624, BH was supported by supported by the RGC Early Career Scheme No. 22200720 and NSFC Young Scientists Fund No. 62006202. GN was supported by JST AIP Acceleration Research Grant Number JPMJCR20U3, Japan. KZ was supported in part by the National Institutes of Health (NIH) under Contract 101HL159805, by the United States Air Force under Contract No. FA8650-17-C77 the NSF-Convergence Accelerator Track-D award #2134901, and by grant from Apple. The NIH or NSF is not responsible for the views reported in this article. The authors thank the reviewers and the meta-reviewer for their helpful and constructive comments on this work.
by
References
[1] Dana Angluin and Philip Laird. Learning from noisy examples. Machine Learning. 2(4): 343-370, 1988.
[2] Devansh Arpit, Stanislaw Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, al. A closer look memorization in deep networks. In International Conference Machine Learning, pages 233-242. PMLR, 2017.
[3] Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of machine learning research, 7(11), 2006.
[4] Antonin Berthon, Bo Han, Gang Niu, Tongliang Liu, and Masashi Sugiyama. Confidence scores make instance-dependent label-noise learning possible. In ICML, 2021.
[5] David M Blei, Alp Kucukelbir, and Jon McAuliffe. Variational inference: review for statisticians. Journal of the American statistical Association, 112(518):859-877 2017.
[6] Hao Cheng. Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, and Yang Liu. Learning with instance-dependent label noise: A sample sieve approach. In ICLR, 2021.
[7] Jiacheng Cheng, Tongliang Liu, Kotagiri Ramamohanarzo, and Dacheng Tao. Learning with bounded instance and label-dependent label noise. In ICML, 2020.
[8] Jacob Goldberger and Ehud Ben-Reuven Training deep neural- using noise adapta- tion layer. In ICLR, 2017.
[9] Mingming Gong, Kun Zhang, Tongliang Liu, Dacheng Tao, Clark Glymour, and Bernhard Schölkopf. Domain adaptation with conditional transferable components. In International conference on machine learning, pages
PMLR, 2016.
[10] Bo Han, Quamming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In NeurIPS, pages 8527-8537, 2018
[11] Lu Jiang, Zhengyuan Zhou, Thomas Leung. Li-Jia Li, and Li Fei-Fei. MentorNet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In ICML, pages 2309-2318, 2018
[12] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint 1311v11312.61 14, 2013.
[13] Alex Krizhevsky, Learning multiple layers of features from tiny images. Technical report, 2009.
[14] Xuefeng Li, Tongliang Liu, Bo Han, Gang Niu, and Masashi Sugiyama. Provably end-to-end label-noise learning without anchor points. In ICML, 2021.
[15] Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting. IEEE Transactions on pattern analysis and machine intelligence, 387)4447--61,, 2016.
10

---

[16] Eran Malach and Shai Shalev-Shwarte Decoupling" when to update" from" how to update". In NeurIPS, pages 960-970,20 2017.
[17] Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. Learning with noisy labels. In NeurIPS, pages 196-1204, 2013.
[18] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y.Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop Deep Learning and Unsupervised Feature Learning, 2011.
[19]
Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: A loss correction approach. In CVPR, pages 2017.
[20] Judea Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, New York, NY, USA, 2000. ISBN 0-521-77362-8.
[21] Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: founda- tions and learning algorithms. The MIT Press, 2017.
[22] Schölkopf. D Janzing, J Peters, Sgouritsa, K Zhang, and J Mooij. On causal and anticausal learning. In 29th International Conference on Machine Learning (ICML 2012), pages 1255- 1262. International Machine Learning Society, 2012.
[23] Bernhard Schölkopf. Causality for machine learning. arXiv preprint arXiv: 2019.
[24] Clayton Scott. A rate of convergence for mixture proportion estimation, with application to learning from noisy labels. In AISTATS, pages 838-846, 2015.
[25] Peter Spirtes and Kun Zhang. Causal discovery and inference: concepts and recent methodolog- ical advances. In Applied informatics, volume 3, pages 1-28. SpringerOpen, 2016.
[26] Peter Spirtes, Clark N Glymour, Richard Scheines, David Heckerman, Christopher Meek, Gregory Cooper, and Thomas Richardson. Causation, prediction, and search. MIT press, 2000.
[27] Songhua Wu, Xiaobo Xia, Tongliang Liu, Bo Han, Mingming Gong, Nannan Wang, Haifeng Liu, and Gang Niu. Class2simi: A noise reduction perspective learning with noisy labels. In International Conference on Machine Learning, pages 11285-11295. PMLR, 2021.
[28] Xiaobo Xia, Tongliang Liu, Nannan Wang, Bo Han, Chen Gong, Gang Niu, and Masashi Sugiyama. Are anchor points really indispensable in label-noise learning? In NeurlPS, pages 6835 6846, 2019.
[29] Xiaobo Xia, Tongliang Liu, Nannan Wang, Bo Han, Chen Gong, Gang Niu, and Masashi Sugiyama. Are anchor points really indispensable in label-noise learning? In NeurIPS, pages 6838-6849, 2019.
[30] Xiaobo Xia, Tongliang Liu, Bo Han, Nannan Wang. Mingming Gong, Haifeng Liu, Gang Niu, Dacheng Tao, and Masashi Sugiyama. Part-dependent label noise: Towards instance-dependent label noise. In NeurIPS, 2020.
[31] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv: 1tt.:1708.7747 2017.
[32] Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. Learning from massive noisy labeled data for image classification. In CVPR, pages 2691-2699, 2015.
[33] Quanming Yao. Hansi Yang, Bo Han, Gang Niu, and James T Kwok. Searching to exploit memorization effect in learning with noisy labels. In ICML, 2020.
[34] Yu Yao, Tongliang Liu, Bo Han, Mingming Gong, Jiankang Deng, Gang Niu, and Masashi Sugiyama. Dual Reducing estimation error for transition matrix label-noise learning. In NeurIPS, 2020.
11

---

[35] Xiyu Yu, Tongliang Liu, Mingming Gong, and Dacheng Tao. Learning with biased complemen tary labels. In ECCV, pages 68-83, 2018.
[36] Hongyi Zhang, Moustapha Cisse, Yann Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In ICLR, 2018.
[37] Kun Zhang and Aapo Hyvarinen. On the identifiability of the post-nonlinear causal model. arXiv preprint arXiv: 20......22229 2012.
[38] Kun Zhang, Mingming Gong, and Bernhard Schölkopf. Multi-source domain adaptation: A causal view. In Twenty-ninth AAAI conference on artificial intelligence, 2015.
[39] Kun Zhang, Jiji Zhang, and Bernhard Schölkopf. Distinguishing cause from effect based on exogeneity. arXiv preprint 156605,, 2015.
12


Reviews:
[{"review_id": "t3C_hb3UfL4", "reviewer": "Reviewer_38W1", "summary": "The paper addresses the question of learning with noisy label\nThe idea is to consider an underlying causal graph explaining the observed noisy label. \nTo do so the authors consider that there is an underlying generative process and that the observed noisy label is  generated both by the true label and a relevant representation of the image of interest\nThe paper the provides a variational inference method based on this generative process and on a relevant representation of the noisy image\nThis is combined with coteaching already introduced in [9]\nExperiments are thereafter performed on several datasets and results (taking into account std) are quite comparable with the T-Revision approach and outperforms co-teaching when the noise is severe\n", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "* The idea of modeling noise generation through a generative process is original and new\n* More precision should be given about the way some noise is added to the data. A reference is given but one may wonder the impact of the way the noise is added on the performance of the algorithm, especially when using variational inference. Can you add a discussion on the impact of the choice of the noise model? \n* Can you detail how noise impact precision and recall for each model?\n* What about the choice of prior on Z and Y? \n* The claim is that the framework is general and that it can be combined with any method dealing with label noise. Can you explain the choice of co-teaching? What about combination of the framework with other label-noise learning methods?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No concern", "review_text": "", "extra_scores": {"ethical_concerns": "No concern", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2h"}, "extra_text": {}}, {"review_id": "fcH6-RPMArF", "reviewer": "Reviewer_LYgh", "summary": "In this paper, the authors handle the label noise problem by exploiting the causal information, which is a new perspective. The setting of instance-dependent label noise learning is more general and realistic in real life. The motivation of this paper is strong. The proposed method also has a good empirical performance and can be easily integrated into current methods.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors have proposed a novel method for label-noise learning. Different from most of the current works, instead of making assumptions on the transition relationship, the authors exploit the dependence relationship among variables when generating the noisy data.  In such a way, the important information for helping identify the transition relationship and clean labels can be revealed. To learn the useful information, a structural generative model is proposed and outperforms the state-of-the-art method. This paper is well organized. The novelty of this paper is high. To be best of my knowledge, this is the first paper to handle the instance-dependent label-noise problem from a causal perspective. But some of my major concerns might have to be addressed.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1. If the noise is dependent on the instance, the proposed method works well, but will the proposed method introduce bias when the noise is independent of the instance? I think it is better to add some discussion on this point.\n2. The experimental settings such as network structures and hyper-parameters of Figure 2 are not clearly introduced neither in the supplementary material or main paper. Please clarify it.\n3. There might have a typo in either Table2 or Table 3. For IDN-40%, the accuracy of Mixup in Table 2 and Table 3 are exactly same, and the accuracy of Mixup in Table 3 is higher than the proposed method. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "_4J6BNAYMy", "reviewer": "Reviewer_yi5B", "summary": "The paper describes an approach to utilize causal information in the form of a Structural Causal Model (SCM) to facilitate learning under instance-dependent label noise. The approach utilizes semi-supervised learning principles, relying on the causal origin of this effect due to the entanglement between the class conditional $P(Y|X)$ and the marginal $P(X)$ in the anti-causal direction.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The described approach utilizes the properties of anti-causal learning in a novel area by applying it to learning with noisy labels. It is applicable in the more complex case of instance-dependent label noise where most of the previous methods are not applicable, or require special assumptions. \nClearly the present method also require some assumptions, but these are mainly causal, therefore more natural for a human expert to specify. According to the experiments the method achieve consistent improvement over the state of the art, and this improvement is increasing with increasing label-noise.\n\nMy main question is about the assumed causal model:\nIf the noisy labels are generated using crowdsourcing, and the worker only see the image itself, what mechanism provide the link between $Y$ and $\\tilde{Y}$ ? What type of side-channel relay this mechanism?\nIf the human expert providing noisy labels can see the real object, the model is clearly valid, but this is not the typical case. \n\nWhat condition make the approximation that the clean label $Y$ is conditionally independent from the noisy label $\\tilde{Y}$ given $X$ in the inference model acceptable, while the approach still benefits from assuming direct causal effect from $Y$ to $\\tilde{Y}$? Or in other way, what would happen if we assume $X$ is enough to generate $\\tilde{Y}$.\n\nAlgorithm 1 lists $\\rho$ (the noise rate) as its input, where it is used? How realistic it is to know this value?\n\nline 132 (minor error in text) but it is impossible [to tell] which class each cluster belongs to.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The author clearly addresses the assumptions made to make the noise transition model identifiable. These assumptions defining the limitations of the model. \nOn the societal impact: Given this method can handle instance dependent label-noise, can it be applied in a situation where this instance dependence introduces a bias? For example if a protected attribute of an individual influences the noisy labels.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper considers the problem of label noise, but with the additional assumption that there is a causal relationship from the label to the instance (for example in image classification). By using a structural causal model, the authors propose a instance dependent noise model that shows good performance on benchmarks.

Three reviewers considered the paper, and all agree that the paper considers an important unsolved problem (instance dependent label noise), and provides a novel solution (via a structural causal model). The reviewers had several concerns which the authors satisfactorily addressed in their rebuttal. While one score was lower, the reviewer indicated in comments that they would support acceptance after the author rebuttal. A brief discussion post rebuttal among the reviewers resulted in all agreeing that the paper should be accepted. Therefore it is with great pleasure that I recommend the paper for NeurIPS.
Decision: Accept (Poster)