Wasserstein Adversarial Regularization (WAR) on label noise

1 INTRODUCTION . Deep neural networks require large amount of accurately annotated training samples to achieve good generalization performances . Unfortunately , annotating large datasets is a challenging and costly task , which is practically impossible to do perfectly for every task at hand . It is then most likely that datasets will contain incorrectly labeled data , which induces noise in those datasets and can hamper learning . This problem is often referred to as learning with label noise or noisy labels . The probability of facing this problem increases when the dataset contains several fine grained classes that are difficult to distinguish ( Schroff et al. , 2011 ; Krause et al. , 2016 ; Dubey et al. , 2018 ) . As pointed out in ( Zhang et al. , 2017 ) , deep convolutional neural networks have huge memorization abilities and can learn very complex functions . That is why training with noisy data labels can lead to poor generalization ( Arpit et al. , 2017 ; Wang et al. , 2018 ; Choi et al. , 2018 ) . Hence in this paper we propose a method tackling the problem of overfitting on noisy labels , and this without access to a clean validation dataset . This problem has been considered in recent literature , mainly in three ways . First are data cleaning methods : ( Vahdat , 2017 ; Xiao et al. , 2015 ; Li et al. , 2017 ) learn relations between noisy and clean labels before estimating new labels for training . In ( Lee et al. , 2018 ) , few human verified labels were necessary to detect noisy labels and adapt learning . In ( Jiang et al. , 2018 ; Ren et al. , 2018 ) , the methods rely either on a curriculum or on meta-gradient updates to re-weight training sets and downweight samples with noisy labels . Second are Transition probability-based methods : ( Sukhbaatar & Fergus , 2014 ; Patrini et al. , 2017 ; Hendrycks et al. , 2018 ) estimate a probability for each label to be flipped to another class and use these estimations to build a noise transition matrix . In ( Sukhbaatar & Fergus , 2014 ) , the authors add an extra linear layer to the softmax in order to learn the noise transition matrix itself , while ( Hendrycks et al. , 2018 ) uses a small set of clean validation data to estimate it . ( Patrini et al. , 2017 ) proposes a forward/backward loss correction method , which exploits the noise transition matrix to correct the loss function itself . Third are regularization-based methods . In ( Reed et al. , 2015 ; Ma et al. , 2018 ) , the authors use a mixture between the noisy labels and network predictions . In ( Tanaka et al. , 2018 ) , the regularization is achieved by alternatively optimizing the network parameters and estimating the true labels while the authors of ( Han et al. , 2018 ; Yu et al. , 2019 ) propose peer networks feedbacking each other about predictions for the noisy labels . Song et al . ( 2019 ) proposes to replace noisy labels in the mini-batch by the consistent network predictions during training , while Chen et al . ( 2019 ) proposes noisy cross-validation to identify samples that have correct labels . In ( Wang et al. , 2019 ; Zhang & Sabuncu , 2018 ; Ghosh et al. , 2017 ) , robust loss functions are proposed to overcome limitations of cross entropy loss function . In contrast to those works , we propose to regularize predictions in areas of the feature space close to the decision boundary of conflicting classes , therefore mitigating the influence of noisy labels . To do so , we use the adversarial regularization ( AR ) framework ( Goodfellow et al. , 2015 ; Miyato et al. , 2018 ) on the noisy label problem . We use AR to reduce the discrepancy between the prediction of a true input sample and the one obtained by a near-by adversarial sample . To reduce this discrepancy , we use a loss based on the Wasserstein distance computed with respect to a ground cost encoding class similarities . This ground cost provides the flexibility to regularize with different strengths pairs of classes . This strength can depend on semantic relations , classes similarities , or prior knowledge ( e.g . on annotators ’ mistakes ) . This way , the classifier can discriminate non-similar objects robustly under the presence of noise and class overlap . We name our proposed method Wasserstein Adversarial Regularization ( WAR ) . WAR allows incorporating specific knowledge about the potential degree of mixing of classes through a ground cost that can be designed w.r.t . the problem at hand . Nevertheless , this knowledge might be unknown or difficult to craft . In this paper , we use distances between word embeddings of the class names to derive a semantic ground cost . Experiments in five datasets ( Fashon-MNIST , CIFAR10 , CIFAR100 and real life examples on clothing classification and a remote sensing semantic segmentation dataset ) under label noise conditions show that WAR outperforms the state-of-the-art in providing robust class predictions . 2 WASSERSTEIN ADVERSARIAL REGULARIZATION FOR LABEL NOISE . Given a set of labeled data { xi , yi } i=1 , ... N , we are interested in learning a classifier pθ defined by a set of parameters θ . Data xi are usually elements of Rd , while yi ∈ C are one-hot vectors encoding the belonging to one of C classes . The empirical risk minimization principle is used to learn pθ . Given a loss function L , the optimal set of parameters for the classifier is given by arg minθ ∑N i=1 L ( xi , yi , pθ ) . Without loss of generality , we will consider thatL is the cross-entropy loss : LCE ( xi , yi , pθ ) = − ∑ c y ( c ) i log pθ ( xi ) ( c ) . The considered label noise problem arises whenever some elements yi do not match the real class of xi . Several scenarii exist : in the symmetric label noise , labels can be flipped uniformly across all the classes , whereas in the asymmetric label noise , labels y in the training set can be flipped with higher probability toward specific classes . We note that the first scenario , while thoroughly studied in the literature , is highly improbable in real situations : for example , it is more likely that an annotator mislabels two breeds of dogs than a dog and a car . Hence , noise in labels provided by human annotators is not symmetric since annotators make mistakes depending on class similarities ( Misra et al. , 2016 ) . To prevent a classifier to overfit on noisy labels , we would like to regularize its decision function in areas where the local uniformity of training labels is broken . To achieve such desired local uniformity , robust optimization can be used . This amounts to enforce that predicted labels are uniform in a local neighborhood Ui of data point xi . This changes the total loss function in the following way : arg min θ N∑ i=1 max xui ∈Ui L ( xi , yi , pθ ) ( 1 ) 2.1 ADVERSARIAL REGULARIZATION . Because the robust optimization problem in equation 1 is hard to solve exactly , adversarial training ( Goodfellow et al. , 2015 ; Shaham et al. , 2015 ) was proposed as a possible surrogate . Instead of solving the max inner problem , it suggests to replace it by enforcing uniformity in the direction of maximum perturbation , called the adversarial direction . Mainly used for robustness against adversarial examples , adversarial training is however not adapted to our problem , since it can enforce uniformity around a false label . Following the same reasoning , we propose to cast the problem as a regularization term of the initial loss function . The total learning loss is then Ltot ( xi , yi , pθ ) = LCE ( xi , yi , pθ ) + βRAR ( xi , pθ ) , where RAR is a regularization term reading : RAR ( xi , pθ ) = D ( pθ ( xi + rai ) , pθ ( xi ) ) with r a i = argmax ri , ‖ri‖≤ε D ( pθ ( xi + ri ) , pθ ( xi ) ) . ( 2 ) Basically , it minimizes an isotropic divergence D between the probability output pθ ( xi + ra ) and pθ ( xi ) . A sound choice for D can be the Kullback-Leibler ( KL ) divergence . RAR can be seen as a ( negative ) measure of local smoothness , or also as a Local Lipschitz constant in the ε neighborhood of xi with respect to the metric D , hence a measure of complexity of the function . RAR promotes local uniformity in the predictions without using the potentially noisy label yi : therefore , it reduces the influence of noisy labels , since it is computed from the prediction pθ ( xi ) that can be correct when the true label is not . RAR shares strong similarities with the Virtual Adversarial Training ( VAT ) from Miyato et al . ( 2018 ) , at the notable exception that we do not consider a semi-supervised learning problem and that we regularize on the labeled training positions xi , where VAT is applied on unlabeled samples . It can be shown ( proof in the supplementary material A.1 ) that this regularization acts as a label smoothing technique : Lemma 1 Let D be the Kullback-Leibler divergence . Let = ββ+1 ∈ [ 0 , 1 [ . Let y a i = pθ ( xi + r a i ) be the predicted ( smooth ) adversarial label . LetH be the entropy . The regularized learning problem Ltot ( xi , yi , pθ ) is equivalent to : Ltot ( xi , yi , pθ ) ≡ LCE ( xi , ( 1− ) yi + yai , pθ ) − H ( yai ) . This leads to the following interpretation : instead of learning over the exact label or over a mix between the exact label and the network prediction , we learn over an interpolation with the adversarial label , while maximizing the entropy of the adversarial label ( i.e . blurring the boundaries of the classifier ) . Related developments can be found in adversarial label smoothing ( ALS ) ( Shafahi et al. , 2018 ; Goibert & Dohmatob , 2019 ) , which aims at providing robustness against adversarial attacks . Yet , one of the major limit of this approach is that the regularization is conducted with the same magnitude everywhere . As a consequence , a strong regularization can remove the label noise , but also hinder the ability of the classifier to separate similar classes where a complex boundary is needed . To overcome this issue , we propose to replace D by a geometry-aware divergence taking into account the specific relationships between the classes . 2.2 WASSERSTEIN ADVERSARIAL REGULARIZATION ( WAR ) . To make the divergence aware of specific relationships between classes , we replace the isotropic divergence D with a Wasserstein distance computed in the labels space . We name our proposed method Wasserstein Adversarial Regularization . Frogner et al . ( 2015 ) already used the Wasserstein distance as a loss in a learning system between the output of the model for multi-label learning . The interest of the Wasserstein distance is to take into consideration the geometry of the label space . We define the proposed regularization term RWAR as follows : RWAR ( xi ) = OT λ C ( pθ ( xi + r a i ) , pθ ( xi ) ) with r a i = argmax ri , ‖ri‖≤ε OTλC ( pθ ( xi + ri ) , pθ ( xi ) ) . ( 3 ) OTλC is an optimal transport ( OT ) distance ( Peyré & Cuturi , 2019 ) . The OT problem seeks an optimal coupling T ∗ ∈ U ( α , β ) = { T |T ≥ 0 , T1 = α , T > 1 = β } between two distributions α , β with respect to a ground cost C ∈ Rn1×n2 . U ( α , β ) is the space of joint probability distributions with marginals α and β. OT distances are classically expressed through the Wasserstein distance WC ( α , β ) = min T∈U ( α , β ) 〈T , C〉 , where 〈. , .〉 is the Frobenius product . Unfortunately , this distance is expensive to compute ( cubical complexity ) . In practice , we will use the solution of the sharp entropic variant of the optimal transport problem ( Luise et al. , 2018 ) : OTλC ( α , β ) = 〈T ∗λ , C〉 with T ∗λ = argmin T∈U ( α , β ) 〈T , C〉 − λH ( T ) where H denotes the entropy function and λ the regularization strength . Using this regularized version has several advantages : i ) it lowers the computational complexity to quadratic , ii ) it turns the problem into a strongly convex one , for which gradients can be computed efficiently and iii ) it allows to vectorize the computation of all Wasserstein distances in a batch , which is particularly appealing for training deep neural nets . Based on ( Genevay et al. , 2018 ) , we use the AutoDiff framework , which approximates the derivative of this regularization with a fixed number of iterations of the Sinkhorn algorithm . Choice of ground cost . The ground cost C reflects the geometry of the label space . It bridges the gap between AR and WAR . An uninformative 0-1 ground cost , i.e . 0 over the diagonal and 1 everywhere , would give the total variation ( TV ) loss ( Remark 2.26 in ( Peyré & Cuturi , 2019 ) ) , which could also be used as D in the AR framework . Below , we refer to this special case as RWAR.0-1 . To define a C matrix encoding class relations , multiple choices are possible . We could calculate the Wasserstein distance between classes , but this would be biased as we have noisy labels . We could set it manually ( Tuia et al. , 2011 ) , but this becomes unpractical when a large set of classes is present . In absence of prior information about the nature of the source of labelling errors , we decided to rely on semantic distances based on word embeddings such as word2vec ( Mikolov et al. , 2013 ) . Similarities between classes are then defined via Euclidean distances in the embedding space , as proposed in ( Frogner et al. , 2015 ) . Finally , as our method requires large values of the cost between similar classes , we apply the function e−m ( where m is the Euclidean distance between the two class names ) element-wise and set the diagonal of C to 0 . Function smoothness and ground metric . Now we discuss how the proposed regularization term regularizes the model pθ with a smoothness controlled by the ground metric C. It is not possible to extend the label smoothing in Lemma 1 to WAR because the OT distance does not admit a close form solution , but we can still show how RWAR promotes label smoothness . To this end , we look at the regularization term OTλC ( p̂θ ( x ) , pθ ( x+ r ) ) for a given sample x and a pre-computed r. We can prove ( see A.2 of the supplementary material ) the following lemma : Lemma 2 Minimizing RWAR with a symmetric cost C such that Ci , i = 0 , ∀i is equivalent to minimizing a weighted total variation ( TV ) norm between pθ ( x ) and pθ ( x+ r ) . cTV ( pθ ( x ) , pθ ( x+ r ) ) ≤ ∑ k ck|pθ ( x ) k − pθ ( x+ r ) k| ≤ OTλC ( pθ ( x ) , pθ ( x+ r ) ) ( 4 ) where ck = mini , i 6=k ck , i is the minimal off-diagonal cost for row k of C and c = mink ck is a global minimum out of the diagonal . By minimizing the proposed RWAR regularization with r belonging in a small ball around x , we actually minimize a local approximation of the Lipschitz constant of pθ . This has the effect of smoothing-out the model around x and makes it more robust to label noise . One can see the effect of the cost matrix in the center term of equation 4 , where the values in the ground metric correspond to a weighting of a total variation , hence controlling the effect of the regularization . Interestingly , the Wasserstein distance can be bounded both below ( c ) and above ( c̄ ) by Total Variation and weighted total variation similarly to the equation above . Finally , in practice we minimize the expectation of the OT loss , which means that we will penalize areas of high density similarly to a regularization with the Sobolev norm ( i.e . penalizing the expected norm of the model gradient ( Mroueh et al. , 2018 ) ) , while keeping a finer control of the class relations , since we use the ground loss C that promotes anisotropy . Illustration of the effect of RWAR We illustrate AR and WAR in a simple toy 3 classes classification problem with noise ( Figure 1 ) . Each column of the figure corresponds to a divergence function D. The top row illustrates the values on the simplex , while the bottom row shows the classification predictions when using D as adversarial regularization . From left to right , we compare the effect of training with the cross entropy alone ( CCE , no regularization ) , RWAR with λ = 0.1 and λ = 0.05 , as well as RAR with TV , KL and L2 divergences as D. For the classification problem , we generated two close classes ( in orange and red ) , as well as a third ( in black ) , which is far from the others . Then , we introduced noisy labels ( of the black class ) in the region of the red class . On this toy example , CCE overfits the noisy black labels , yet is able to distinguish the red and orange classes . The RAR regularizers , being class agnostic , correct for the noisy black labels in the bottom part , but smooth the complex decision function between the orange and red classes . On the contrary , RWAR uses a different cost per pair of classes , illustrated in the top left panel of Figure 1 : the smallest cost is set between the red and black classes , which has the effect of promoting adversarial examples in that direction . This cost / smoothing relation is due to the fact that our problem is a minimization of the OT loss : in other words , the higher the cost between the classes , the less the binary decision boundary will be smoothed . Finally , the effect of the global λ parameter can also be appreciated in the classification results : while using λ = 0.05 , the smoothing of the loss is decreased and the final decision boundary between the mixed classes keeps all its complexity .

Summary: This paper proposes a new regularization scheme inspired from (virtual) adversarial training to tackle the problem of learning with noisy labels. While based on the adversarial training (AR), it was found that AR does not directly transferable to deal with noisy labels. The author then proposed the Wasserstein version of AR replacing the KL with the Wasserstein distance and its approximate. This gives the proposed  Wasserstein Adversarial Regularization (WAR) which provide considerable robustness improvement on 5 datasets (both classification and segmentation). The correlation between WAR regularization and boundary smoothing is justified both theoretically and empirically with toy examples. The advantage of WAR regularization over existing methods is the flexibility to incorporate intra-class divergence, making it plausible against asymmetric label noise, which is more common in real-world datasets. The authors have done solid work in this paper. The experiment is complete, in terms of the scale, noise settings and comparison to existing works.