Bayesian Metric Learning for Robust Training of Deep Models under Noisy Labels
Authors: Toan Tran
Hieu Vu
Gustavo Carneiro
Hung Bui
Venue:  (2021)
Keywords: Noisy labels
Deep metric learning
Bayesian inference
Variational inference

Abstract: Label noise is a natural event of data collection and annotation and has been shown to have significant impact on the performance of deep learning models regarding accuracy reduction and sample complexity increase. This paper aims to develop a novel theoretically sound Bayesian deep metric learning that is robust against noisy labels. 
Our proposed approach is inspired by a linear Bayesian large margin nearest neighbor classification, and is a combination of Bayesian learning, triplet loss-based deep metric learning and variational inference frameworks. We theoretically show the robustness under label noise of our proposed method. The experimental results on benchmark data sets that contain both synthetic and realistic label noise show a considerable improvement in the classification accuracy of our method compared to the linear Bayesian metric learning and the point estimate deep metric learning.

Under review as a conference paper at ICLR 2021
BAYESIAN METRIC LEARNING FOR ROBUST TRAINING OF DEEP MODELS UNDER NOISY LABELS
Anonymous authors Paper under double-blind review
ABSTRACT
Label noise is a natural event of data collection and annotation and has been shown to have significant impact on the performance of deep learning models regarding accuracy reduction and sample complexity increase. This paper aims to develop a novel theoretically sound Bayesian deep metric learning that robust against noisy labels. Our proposed approach is inspired by linear Bayesian large margin nearest neighbor classification, and is a combination of Bayesian learning, triplet loss-based deep metric learning and variational inference frameworks. We the- oretically show the robustness under label noise of our proposed method. The experimental results benchmark data sets that contain both synthetic and real- istic label noise show a considerable improvement in the classification accuracy of our method compared to the linear Bayesian metric learning and the point estimate deep metric learning.
INTRODUCTION
Deep learning has been shown as a dominant learning framework in various domains of machine learning and computer vision. One of the major limitations of deep learning is that it often requires relatively clean data sets that do not contain label noise naturally caused by human labeling errors, measurement errors, subjective biases and other issues (Frénay et al. 2014 Ghosh al. |017|Al0 Algan & Ulusoy 2019). The performance of a machine learning method can be significantly affected by noisy labels both in terms of the reduction in the accuracy rate and the increase in sample complexity. Particularly for deep learning, a deep neural network (DNN) can generalize poorly when trained with noisy training sets which contain high proportion of noisy labels since a DNN can over-fit those noisy training data sets (Zhang et al.) 2016 Algaa & Ulusoy 2020). Developing deep learning methods that can perform well on noisy training data is essential since it can enable the use of deep models in many real-life applications.
There have been several approaches proposed to handle learning issues caused by label noise, for example: data cleaning (Angelova et al.] 2005 Chu et al.] 2016). label correction (Reed et al.) 2014). additional linear correction layers (Sukhbaatar et al. 22044), dimensionality-driver learn- ing (Ma et al. 2018), bootstrapping (Reed et al., curriculum learning-model based approach such as MentorNet (Jiang et al., (2018) or CoTeaching (Han et al.] 2018), loss correction (or noise- & Vasconcelos) 2009 Ghosh et al. 2017 Zhang & Sabuncu 2018 Thulasidasan et al. 2019 Ma et 2020), or a combination of the techniques above (Li et al. 2019). Relevant this paper an existing theoretically sound approach: Bayesian large margin nearest neighbor classification (BLMNN) (Wang & Tan 2018) that employs Bayesian inference to improve the robustness of a point estimation-
tolerant loss) (Masnadi-Shiree
2020 Nguyen
linear metric learning method. BLMNNN then introduces a method to approximate the posterior distribution of the under- lying distance parameter given the triplet data by using the stochastic variational inference. More importantly. BLMNN (Wang& & Tan 2018) also provides theoretical guarantee about the robust- ness of the method, which says that it can work with non-uniform label noise. Although BLMNN has been mathematically shown to be robust against label noise, it only focuses simple linear Mahalanobis distance that can not capture the nonlinear relationships of data points in deep metric learning (Lu et al. 2017).
In this paper, we introduce a Bayesian deep metric learning framework that is robust against noisy labels. Our proposed method (depicted in Fig. T1 is inspired by the BLMNN (Wang & Tan 2018

---

Under review as a conference paper at ICLR 2021
Triplet net
Feature Space
DNN(O)
P triplet loss)
Shared weights
triplet loss
DNN(0)
Shared weights
DNN(@)
Distribution for triplet net parameter 0
Figure An overview of our proposed Bayesian Deep Metric Learning method
deep metric learning (Hoffer Ailon 2015; Hu et al. 2015 Wang et al. 2017 Lu et al. 2017 Do et al. 2019), and Bayes by Backprop (Blundell al. 2015). Compared the BLMNN that only considers a linear metric learning, our framework can handle non-linear deep metric learning, which is useful for many real-life applications. Moreover, directly applying the variational Bayes learning (Wang & Tan 2018) in deep learning is challenging since requires sampling from a dis- tribution of the neural network parameters. Instead, we adapt the variational inference by Blundell et al. (Blundell et al. 2013), which allows to efficiently sample the parameters of Bayes neural networks by using
algorithm. We also theoretically show the robust- ness of our proposed method when working with label noise. The experimental results on several noisy data sets show that our novel proposed method can generalize better compared to the linear BLMNN (Wang & Tan 2018) and the point estimation-based deep metric learning (Hoffer & Ailon 2015
Lu et al. 2017). especially when the noise level increases.
It is important to emphasize that the motivation of our method is to produce better calibrated model that is more robust to noisy label training, and, as a result, less likely to overfit the training set than the linear BLMNN (Wang & Tan 2018) and the point estimation
deep metric learning (Hoffer & Ailon 2015 Lu et al. 2017). Therefore this is a paper that introduces a new theoretical frame- work to solve noisy label learning instead of presenting method that is competitive against the best approaches of the field (such as Mentornet (Jiang et al.) 2018) or Co-teaching (Han et al. 2018)) in large-scale datasets (e.g., webvision (Li al. 2017) and Clothing 1M (Xiao al. (0155). Fur- thermore, deep metric learning has been in fact considered in the Bayesian settings before (Ishfaq al.) 2018: Karaletsos et al., 20155), and recently, in (Lin al.] 2018), but
the context of
noisy labels. Consequently, our proposed framework can be used by other methods that can deal
with noisy label learning, but the extension of those methods using our proposed approach is out of the scope of this paper.
RELATED WORK
2.1 POINT ESTIMATION-BASER DISTANCE METRIC LEARNING
The goal of distance metric learning (or metric learning) is to learn a distance function to measure the similarity between training samples. Metric learning has been shown have great success in many visual applications such as face recognition, image classification, visual search, visual tracking, and person re-identification (Lu et al. 2017). In principle, a supervised metric learning method aims to leamn distance metric which pulls together samples from the same class while pushing away those from different classes. Based on the complexity of the distance, metric learning can be classified into two types: linear, focusing on linear distance (e.g.. Mahalanobis), which often suffers from the nonlinear relationship of data points (Lu et al. 2017); and non-linear, which nowadays is mostly based deep learning.
2

---

Under review as a conference paper at ICLR 2021
Deep metric learning (DML) is motivated by the fact that deep learning is an effective solution to a non-linear transformation of imput samples (Lu et al. 2017, Do et al.) 2019). The key idea of DML is to explicitly learn a set of hierarchical non-linear transformations to map input data points into a feature space that is used for comparing or matching these data points in a more effective manner. DML unifies feature learning and metric learning into joint learning framework. DMI. is shown to be more ad advantageous compared to traditional models, for example, with respect to classification performance since DML does not rely on the classification layer of trained model that often depends on the type of problems (Lu et al. 2017 1.7,DD Do et al. 2019). Relevant to this paper the triplet boss-based metric learning (Lu et al. / et al. (015). in which the original training data set is represented by set of independent triplets. formed by anchor sample, one sample of the same class and another sample from a different class. The training process is then performed by minimizing the triplet loss to simultaneously approximate the anchor to the sample with the same class and separate the anchor from the sample of different class.
2.2 ROBUST LINEAR DISTANCE METRIC LEARNING VIA BAYESIAN INFERENCE
One drawback of the point estimation DML. is that it is likely to over-fit the noisy labels (Wang & Tann 2014] 2018). Wann et al. (2016) introduces the Deep Stochastic Neighbor Compression (DSNC) method that aims to jointly learn a nonlinear transformation that preserves the neighbor- hood of the data, and a compressed version of the training data set (Ahmed et al.). DSNC is also robust against label noise. Motivated by the fact that Bayesian learning is good choice for bust learning (Zhu et al. 2014 Yang et al. 2012). Wang & Tan (2018) introduced the theoretically sound Bayesian large margin nearest neighbor classification (BLMNN) improve the robustness of the linear metric learning under the presence of label noise. The BI BNNNN framework represents the large margin nearest neighbor classification using a linear metric learning in the form of variational Bayes method that takes the prior distribution of the transformation matrix into account and estimate the posterior distribution via stochastic variational inference VS)... BLMNN provides mathematical definitions of the noisy label triplet (a type of non-uniform label noise) and the 3-robust algorithm against noisy label triplets. More importantly, a theoretically guarantee of the robustness of BLMNN is also provided in (Wang& & Tan 2018). Although BLMNN efficiently addresses the training issue caused by the noisy label, one key limitation of BLMNN is that based on a linear metric trans- formation that cannot capture the nonlinear relationships of data points in deep metric learning. However, the extension of BLMNN to the non-linear case is not straightforward due to the com- plexity of the posterior distribution estimation as well as the sampling process of high-dimensional parameters, and such extension is the main target of our paper.
2.3 BAYES BY BACKPROP
Bayesian neural networks (MacKay, 1995 Neal 2012 Gal & Ghahramani 2015) aim to estimate the posterior distribution of the network parameters given the training data. However, that inference framework is often intractable, especially when working with high dimensional parameters (Blun-) dell et al. 2015). Moreover, exactly calculating the posterior of the weights is challenging since it requires the integration that is known to be computationally expensive. Blundell et al. (Blundell Bayes by Backprop method for estimat-
ee al. 2015) introduced the iilippoppagationccopaaaiiiiiiiii ing the posterior distribution of the network parameter. That method is inspired by the variational free energy inference in which the exact posterior is approximated by a variational distribution by solving an optimization problem. In principle, Bayes by Backprop can directly work on the network parameter by minimizing compression cost function (Blundell et al. 2015).
Our proposed method is inspired by (Wang & Tan 2018). where we replace the linear Mahalanobis metric by a mon-linear deep metric. Our novel proposed method therefore leverages the good perfor- mance of the deep metric learning (Lu et al., 2017) and the robustness to label noise of a Bayesian framework (Yang et al. 2012, Wang & Tan 2018). We first represent triplet-based deep metric learning using Bayesian inference. By replacing the softmax loss by a triplet loss (and so learning a metric), we impose more strict constraints, where points of the same class are forced to collapse into a small region of the feature space instead of just belonging to regions within the class boundaries. We argue that this more strict constraints has the potential to introduce more robust feature spaces for classification under noisy label training. To approximate the posterior of the network parame- ters, we then employ the efficient variational framework (Blundell et al. 2015) that allows sample
3

---

Under review as a conference paper at ICLR 2021
from Bayesian neural network using the backpropagation framework. We theoretically show that our proposed method is robust against noisy triplet relying on a Bayesian inference framework.
METHODOLOGY
In this section, we explain our proposed method. Then, we theoretically show the robustness of that method under noisy label triplet.
3.1 DEEP METRIC LEARNING WITH TRIPLET LOSS
Let us denote the original data set by T {(x,, vr)), where x; IR is sample and y: Y C N is its corresponding label. Let S = ((i.j)((x Vr), (x). E T and yi = yj and D = ((x,))((x 9:), (x,, y2) E 7 and y; * yý} denote the set of pairs of the data points with the same labels and the set of pairs with different labels, then triplet is defined by
(i. where triplet-based deep metric learning framework (Hoffer & Ailon 2015 Do et al. 2019) aims to minimize the following triplet loss function for the triplet
(i,j) E S, and (i, E
Lu et al. 2017
2 (i,
cle (0) h(T + dz(x,, dl (xx,
(1)
where the deep metric is defined by
(x,, x,) = [l 0) - 0)
(2)
with ((-00 denoting the network function parameterized by 0, and h(a) max(0, a), a I
representing the hinge loss function (with T > 0 being a margin between (x,x,xx) and d (x, x:))
xx).
3.2 BAYESIAN DEEP METRIC LEARNING
Let us denote the training data set by {=k.k=1,..... which contains independent triplets
2 (.. and the network parameter by E R². According to the large margin principle (Wang
& Tan 2018), the likelihood function p(2 can be defined as follows:
p(2 = II
p(x,, Xj, X1, U/. Uj. 31
(
C
II cxp(-2 max(1 + (x,,)x,) (x,, 0)), (i.1E)
(3)
where C' is normalising constant. Without loss of generality, assume that the prior of the network parameter is a Gaussian with mean and covariance matrix Vo. i.e, p(0) =00)uo.. Vo). By using the Bayes' rule, the posterior distribution of the parameter 0 can be represented by
p(0) Z) p(Z)0) p(0).
(4)
Naively estimating the posterior in equation Fis often intractable, especially for the high dimensional parameter space of deep models. Hence, we employ the variational learning approach (Blundell et al. 2015 Blei et al. 2017) to approximate the posterior distribution in equation 4 In particular, the goal that variational method to estimate the variational parameter of a distribution for the network parameters, denoted by q(0) A) that minimizes the Kullback-Leiber (KL) divergence between the variational distribution and the true posterior, i.c.,
X arg min (1( ())2(()
= arg min KL (1( ())) (logg(()))))
(5)
The objective function in equation embodies a trade-of oofe-offfod between the complexity of the data and the simplicity of the prior. Blundell (Blundell et al. 2015) pointed out that directly evaluating that cost function is computationally expensive (Blundell et al. 2013). so they introduced an unbiased Monte-Carlo approximation of the exact cost function in equation 5 with the following function:
F(2,0) log (((0)))))))) (2) log(p((())) log lzgo()))(())
(6)
4

---

Under review as a conference paper at ICLR 2021
Algorithm Bayesian Deep Metric Learning (BDML)
Establish the triplet training set Z Initialize the variational parameter À = (H,,)
repeat
Sample ~ I))
Compute with equation] 7
Compute the likelihood p Z (0) with equation 3
Compute function h0, A) with equation 9
Update μ. o with equation g
until convergence
where (1()) (((l)) is the i-th Monte Carlo sample of the variational posterior distribution assumed to be diagonal Gaussian distribution, i.c., ~ N(,, o²1), where is the identity matrix, then sample of the weights 0 can be obtained by
0 ~ + log(1 + exp(p)) os,
where log(1 + exp p), the variational posterior parameters À (H.p). with μ R and p E R, denoting point-wise multiplication, and & ~ NNO,,), The variational parameters
(p,p) p) are updated by (Blundell et al. 2015):
+ μ - a--VVil 1h(0, A)
P p av-VVphle vph(0,)))
where a is the step size, and
h(0, A) = log (0 log (())p((0)))
(9)
The gradients in equation g can be estimated as follows (Blundell al. 2015):
A) = @h(0, A)
(0, X)
00 +
(10)
A))(B,A) = Oh(0, X)
Oh(0,))
+
(11)
00 1 + exp(-p)
3p
The whole training process of our proposed method is presented in Algorithm
3.3 ROBUSTNESS TO LABEL. NOISE
This subsection first introduces the defimition of a noisy label triplet and the robustness of a learning algorithm under that label noise. We then theoretically show the robustness of our proposed method against noisy labels.
Definition 1. A triplet 2 = ji, I) of data points is defined as a label noisy triplet (Wang & Tan X;,X; Wiethe set of neighbors (Weinberger & Saul] 2009) of x,, where N, R ||x xill d}. with 8 being sufficiently small; 2) Viy (1 ya) 1, where
2018
if:
{x
1, if y = y/
Vij
(0, otherwwise
and
3) xx) (x, x1) D Cs. with C2 > 0 denoting a threshold.
Definition 2. Let Z be arbitrary training set and 2' a label noisy triplet. Then learning algorithm A is 3-robust against label noise in the Bayesian inference sense if
log p( p() z') log pl (0) <
(12)
Lemma 1. Les Z be an arbitrary training set (in the sense that it may may not contain any noisy label triplet). Assuming that z' (1,j,l) is noisy label triplet with the corresponding threshold Ca in Definition [ and the normalising constant of the likelihood (), C is defined in equation such that C < exp (2(Ca + 1)). Then
log 2(2, =' log pl 2(210)] & 2(Ca log C.
(13)
5

---

Under review as a conference paper at ICLR 2021
Proof. Suppose that p(0) # 0. Given the i..d. assumption from equation 3 we have
p(2, = p()).)) p(z
(14)
Thus
log 2 = log ( + log p
(15)
Note that 0 < p(=100) 1, have log p( (0) S 0, and then
[log 2 log p( 2 [0) = log (=
(16)
Since z' [i) () is noisy label triplet (Definition
obtain
1 + dř (x,, d (x, x1) + Ca >
(17)
then, by combining with the corresponding likelihood in equation 3 we obtain
=lg'p(='(0)
log log(exp(2(1 Ca))) 2(C2 + 1) log C.
(18)
The resutl in equation 13 is obtained by integrating equation 18 into equation 16
Remark 1. Assuming that the prior p(0) is arbitrary, but fixed and that the noisy triplet z' and the original data Z are sampled from the same distribution, by using Bayes' rule we then get similar estimate for the posterior distribution, i.e.,
log p(0 2') log p(0 < 2(Ca + 1) loog C' 3.
(19)
Hence, according to the Definition and equation label triplet.
19
Algorithm Tis Bayesian robust against noisy
Theorem 1. Given Lemma 1 and supposing that we have N noisy label triplets 2i, 2/y that are conditionally independent given the parameter 0, then
loggp(]((
log 2(] S N,3.
(20)
Proof. By applying the result in Equation 19 of Lemma 1 we get
log p(0) (2, - log p(0) 8;
log p(0) < 1ogg(( Z. 21) < 8;
log p0 (0] x log p(
B.
(21)
Moreover, by using the triangle inequality we obtain equation 20
Given an arbitrary training set, the theoretical result in Lemma D indicates that our proposed method 3-robust against noisy label triplet z'. However, in many real-life applications, the noise level is often much larger, that is, we have to deal with a number of noisy triplets. The theoretical result in Theorem which is extension of equation 19 shows the robustness of our proposed novel method given particular noise level represented by N noisy triplets. Moreover, it would be interesting to clarify that although the definition of the robustness of metric learning algorithm had been dicussed, for example, in (Bellet & Habrard 2015 Huai et al. 2019). these papers do not take noisy labels into account.
EXPERIMENTS AND RESULTS
In this section, we quantitatively evaluate our proposed method based on the experiments conducted on several benchmark data sets that contain different types of label noise. In particular, our proposed method Bayesian deep metric learning (BDML) compared with BLMNN and the deterministic triplet-based deep metric learning (DML) with respect to classification performance
the data sets MNIST (LeCun et al. 1998) and CIFAR-10 10 (Krizhevsky et al. 2012) with synthetic label noise, and face retrieval results iments, we adopt a simple network architecture for deep learning models with two fully-connecte hidden layers of 51 512-dimension and Tanh activation function between intermediate layers. Our
UIS-Cele--I 1M (Guo et 2016) with realistic noisy labels. In all our exper-

---

Under review as a conference paper at ICLR 2021
Table I: Classification results for MNIST (mean=tstdev) after five runs for different rates of uniform label noise. The best result per column is in bold
Noise Rate
0%
10%
20%
30%
%
PCA
97.87 = 0.09
95.35 = 0.30
89.56 = 0.53
80.70 = 0.47
59.02 = 0.65
BLMNN
98.98 + 0.08
97.08 + 0.20
92.25 = 0.51
81.47 + 0.17
63.60 + 0.81
DML
98.66 + 0.07
97.76 + 0.20
95.34 + 0.35
91.40 + 0.52
71.86 + 0.47
BDML (ours)
98.79 + 0.09
97.60 + 0.18
95.25 = 0.31
91.59 + 0.36
73.60 + 0.6
proposed framework consists of the following steps: (i) extract features by an unsupervised method; (ii) apply PCA to project data points onto a smaller dimensional space: (iii) standardize features to standard Normal distribution, then use these features for comparing methods.
The values of these hyper-parameters were selected by using 10% of training data as a hold-out set for validation. Performance is calculated by 3-NN classifier, each experiment is repeated five times, and the comparison results with respect to the top-1 classification accuracy (mean-tstdev) as function of the percentage of noisy labels in the training set are reported.
4.1 IMAGE CLASSIFICATION WITH SYNTHETIC LABEL NOISE
4.1.1 SYMMETRIC NOISE
The MNIST data set (LeCun et al., 1998) contains 28 x 28 black and white handwritten digits in 10 classes, (with 50000 training and 10000 testing samples). The new training data set of MNIST is generated by randomly stratified tub-sampling 5000 training data points from the original training data (i.c., 500 samples for each class), and the standard test set is unchanged - this enables a fair comparison between different methods. The 300-dimensional feature vector for each data point is extracted by using the unsupervised representation learning model CSVDDNet (Wang & Tan 2018 2016). Uniform (or symmetric) label noise with corresponding level in the set {0, 10%, 20%, 30% is then injected into the new training data set by flipping the labels by the respective proportion of data points.
Hyper-parameters seltings: for the BDML model we run 50 epochs, set the learning rate to 0.0001 with decay rate 0.5 epochs 30, prior distribution for weights and bias are Gaussian distributions with mean O and standard deviation 1 for weights, 2 for bias; batch size is set to 1024; for the DML model, we run 50 epochs. learning rate is set to 0.00001 with decay rate 0.5 at epoch 30, batch size is set to 128. For the BLMNNN model, we follow the hyper-parameter setting suggested by authors Wang & Tan 2018)
Table 1 shows that in the case of clean data set, BLMNN, which employs linear transformation, produces slightly better classification performance than both DML and BDML. However, when the noise level increases, deep metric-based methods outperform linear model. In particular, in the case of 30% noise, our method outperforms BLMNN with large margin (7. 12%), and in the most extreme case with 50% of noise, BDML produces better results than both BLMNN and DML.
The CIFAR- R-10 data set (Krizhevsky et al. 2012) consists of 32 32 colour images (with 50000 training and 10000 testing samples, and 10 classes). The new training data set is generated by ran- domly sub-sampling 5000 data points from original training data (i.c., 500 samples for each class), while the standard test set remained. The O048-dimensional feature of each data point is extracted with the recently proposed Simple Framework for Contrastive Learning of Visual Representations (SimCLR) (Chen et al. 2020) for the feature extraction process.
Soper-parametery settings: for the BDML model, we run 50 epochs, set learning rate to 0.0001 with decay rate 0.5 at epoch 30, prior distribution for weights and bias are Gaussian distributions with mean 0 and standard deviation for weights, 2 for bias. For DML. model, we set learning rate to 0.00001 with decay rate 0.5 at epoch 30 and 50, batch size is set to 512. For BLMNN model, we use the prior mean 0.00001 and variance 0.001.
We report the performance of three models with respect to different noise levels in the set {0, 10%, 20%, 30%, 50%} in Table 2 It is clear that the use of a deep metric learning gives better classifi- cation results comparing with linear method, especially when the noise level increases. To be more specific, the gap increase from 2.67% at clean label to 13.66% at 50% label noise. More importantly, the results in that table consistently show that our proposed BDML method, which adopt Bayesian
7

---

0.73 Under review as a conference paper at ICLR 2021
Table 2: Classification results for CIFAR-10 (mean=isdde)) after five runs for different rates of uniform label noise. The best result per column in bold
Noise Rate
0%
10%
20%
30%
50%
PCA
76.64 0.41
72.98 0.29
67.83 + 0.54
60.91 +: 0.18
44.06
BLMNN
79.96 + 0.17
76.56 - 0.28
71.18 + 0.53
64.89 0.38
47.74 0.15
DML
81.81 + 0.30
80.21 + 0.29
0.34
74.05 0.24
59.76 0.64
BDML (ours) 82.63 + 0.12
81.01 1101=:00 0.27
78.81 0.25
75.17=0.33
61.40 0.46
Table 3: Classification results for CIFAR-10 (mean=stdev) after five runs for different class-
conditional label noises, noise rates are presented as a vector with length equals to the number of classes. The best result per row is in bold
Noise Rate
PCA
BLMNN
DML
BDML (ours)
[0.5 0.6 0.4 0.4
0.4
53.68 1 0.43
55.99 0.30
67.76 0.16
69.47 0.49
[0.3 0.5 0.2 0.5 0.4
0.1
0.5)
56.89 0.47
59.71 = 0.80
70.60 +: 0.38
72.32 0.37
[0.1 0.6 0.4 0.5
0.5
0.1 0.1 0.2]
60.49 + 0.36
62.63 + 0.29
71.43 = 0.50
72.89 0.47
inference, outperforms DML,, which adopt MLE for parameter estimation, across different noise levels.
4.1.2 CLASS-CONDITIONAL LABEL NOISE
In this experiment, we examine those methods above on 10, 000 sub-sampled data set being stratified from the original CIFAR- 10 data set (Krizhessky et al. 2012). That is, the new data subset remains the class distributions of the original training data set. The class-conditional label noise ranging from 10% to 60%, is randomly generated by flipping the labels with corresponding proportions of data samples. We keep the previous mentioned hyper- apeer-parameters settings for each model. The results reported in Table show that our proposed method BDML consistently outperforms DML and BLMNN with large margins across different noise vectors.
4.2 FACE RETRIEVAL ON MS-CELEB IM WITH REALISTIC NOISY LABELS
MS-Celeb-IM (Guo et al. 2016) is a large scale data set (with 10M images of 100, 000 celebri- ties) that contains realistic label noise. We adopt the same setting (Wang Tan 2018) to form the new training set for this experiment. In particular, we first randomly select subset of the orig- inal training with 100K images of 1200 persons. This subset then split with ratio 9:1 into a training set that contains approximately 20% of label noise and clean test set, respectively. The 4096-dimensionality feature vector each data point in this subset is extracted by using the VG- GFace (Parkhi et al. 2015) model. We run 80 epochs, for the BDML model learning rate is 0.0001 with decay rate 0.5 at epoch 25, 45, 55, 65, prior distribution for both weights and bias are Gaus- sian distributions with mean and standard deviation 0.5; batch size is set to 256. For the DML model, we set learning rate to 0.00001 with decay rate 0.5 at epoch 10, 20, 40, batch size is set to 128. For the BLMNN model, follow the hyper-parameter setting suggested by authors (Wang & Tan 2018). Table shows the comparison using the mean of average precision (mAP) between our proposed BDML and other competing methods.
consistently indicate the advantage of using deep metric in real world label noisy dataset. Both DML and BDML show their better retrieval performance compared the linear distance metric. Moreover, it is clear from Table 4 that our proposed method BDML is superior to the point estimation DML- this is evidence that the use of Bayesian inference more robust than the point estimation when working with label noise.
The results in Table 4
CONCLUSIONS
In this paper, we proposed a novel theoretically sound Bayesian deep metrie learning method to handle the issue of learning a deep model with noisy labels. To the best of our knowledge, this is the first work that introduces Bayesian inference in deep metric learning to improve the generalization capacity of two existing methods: robust Bayesian linear metric learning and point estimation deep metric learning. is also important to note that our main goal with this paper is the introduction of innovative framework that improve the performance of both baselines, including BLMNN
8

---

Under review as a conference paper at ICLR 2021
Table 4: Experiment results for MS-Celeb-IM (mAP %).
The best result per column is in bold
Noise Rate
0%
20%
PCA
86.21
82.09
BLMNN
89.16
85.11
DML
91.89
86.50
BDML (ours)
93.45
87.66
and DML, We are not aware of other SOTA methods in the scope of this paper. In the future, we plan to test our proposed method with more complicated base models (e.g.. ResNet (He et al. 2016)) instead of the simple neural network to obtain better experimental results. We also plan to extend our current work to make robust under simulated label noise (Wang & Tan (2018) which has been shown to be more challenging than the types of label noise mentioned in paper.
REFERENCES
Samsuddin Ahmed, Abol Basher, Abu Naser Rashid Reza, and Ho Yub Jung. A brief overview of deep metric learning methods.
Görkem Algan and Iikay Ulusoy. Image classification with deep learning in the presence of noisy labels: A survey, arXiv preprint arXiv:1912.05170, 2019.
Görkem Algan and ilkay Ulusoy. Label noise types and their effects deep learning. arXiv preprint arXiv:2003. 10471, 2020.
Anelia Angelova, Yaser Abu-Mostafam, and Pictro Perona. Pruning training sets for learning of ob- ject categories. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, volume 1, pp. 494-501. IEEE, 2005.
Aurélien Bellet and Amaury Habrard. Robustness and generalization for metric learning. Neuro- 2015.
computing,
David Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: review for statisti- cians. Journal of the American statistical Association, 112(518):55-88777 8777 2017.
Charles Blundell, Julien Cornebise, Koray Kavukeuoglu, and Daan Wierstra. Weight uncertainty in neural network. In International Conference Machine Learning, pp. 1613-1622, 2015.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709, 2020.
Xu Chu, Ihab Ilyas, Sanjay Krishnan, and Jiannan Wang. Data cleaning: Overview and emerging challenges. In Proceedings of the 2016 International Conference on Management of Data. pp. 2201-2208 2016.
Thanh-Tose Toan Do, Toan Tran, Ian Reid, Vijay Kumar, Tuan Hoang, and Gustavo Carneiro. A theoret- ically sound upper bound on the triplet loss for improving the efficiency of deep distance metric learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Benoît Frénay, Ata Kabán, et al. A comprehensive introduction to label noise. In ESANN, 2014.
Yarin Gal and Zoubin Ghahramani. Bayesian convolutional neural networks with bernoulli approx- imate variational inference. arXiv preprint arXiv:1506 10..018806.02588, 2015.
Aritra Ghosh, Himanshu Kumar, and PS Sastry. Robust loss functions under label noise for deep neural networks. In AAAI, pp. 1919-1925, 2017.
Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. MS-Celeb-ING 1M: A dataset and benchmark for large scale face recognition. In European Conference on Computer Vision. Springer, 2016.

---

Under review as a conference paper at ICLR 2021
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Advances in neural information processing systems, pp. 8527-8537, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Elad Hoffer and Nir Ailon. Deep metric learning using triplet network. In International Workshop on Similarity-Basse Based Pattern Recognition, pp. 94-92. Springer, 2015.
Junlin Hu, Jiwen Lu, and Yap-Peng Tan. Deep metrie learning for visual tracking. IEEE Transac- fions Circuits and Systems for Video Technology, 26(11): 26(11):2056-2066 2015.
Mengdi Huai, Hongfei Xue, Chenglin Miao, Liuyi Yao, Lu Su, Changyou Chen, and Aidong Zhang. Deep metric learning: The generalization analysis and an adaptive algorithm. In IJCAI, pp. 2535- 2541, 2019.
Haque Ishfaq, Assaf Hoogi, and Daniel Rubin. Tvae: metric learning. 2018.
Triplet-based variational autoencoder using
Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data- driven curriculum for very deep neural networks corrupted labels. In International Conference on Machine Learning, pp. 2304-2313, 2018.
Theofanis Karaletsos, Serge Belongie, and Gunnar Rätsch. Bayesian representation learning with oracle constraints. arXiv preprint arXiv: 1506.05011 2015.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. In Advances in neural information processing systems, pp. 1097-1105, 2012.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2322 1998.
Junnan Li, Richard Socher, and Steven CH Hoi. Dividemix: Learning with noisy labels as semi- supervised learning. arXiv preprint arXiv:2002.07394 2020.
Wen Li, Limin Wang, Wei Li, Eirikur Agustsson, and Luc Van Gool. Webvision database: Visual learning and understanding from web data. arXiv preprint arXiv: 028622 2017.
Xudong Lin, Yueqi Duan, Qiyuan Dong, Jiwen Lu, and Jie Zhou. Deep variational metrie learning. In Proceedings of the European Conference Computer Vision (ECCV), pp. 689-704, 2018.
Jiwen Lu, Junlin Hu, and Jie Zhou. Deep metric learning for visual understanding: An overview of recent advances. IEEE Signal Processing Magazine, 34(6):76-84, 2017.
Xingjun Ma, Yisen Wang, Michael E Houle, Shuo Zhou, Sarah M Erfani, Shu-Tao Xia, Sudan- thi Wijewickrema, and James Bailey. dimensionality-drivee learning with noisy labels. arXiv preprint 18..:iv:.00.... 02612, 2018
Xingjun Ma, Hanxun Huang, Yisen Wang, Simone Romano, Sarah Erfani, and James Bailey, Nor- malized loss functions for deep learning with noisy labels. arXiv preprint arXiv:2006.13555 2020.
David JC MacKay. Bayesian neural networks and density networks. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 354(1):73-85) 80, 1995.
Hamed Sosmadi-Shirard and Nuno Vasconcelos. On the design of loss functions for classification: theory, robustness outliers, and savageboost. In Advances in neural information processing systems, pp. 1049-1056, 2009.
Radford M Neal. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012.
10

---

Under review as a conference paper at ICLR 2021
Duc Tam Nguyen, Chaithanya Kumar Mummadi, Thi Phuong Nhung Ngo, Thi Hoai Phuong Nguyen, Laura Beggel, and Thomas Brox. Self: Learning to filter noisy labels with self- ensembling. arXiv preprint arXiv:1910.01842, 2019.
Omkar M Parkhi, Andrea Vedaldi, and Andrew Zisserman. Deep face recognition. 2015.
Scott Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich. Training deep neural networks on noisy labels with bootstrapping. arXiv preprint arXiv: 141.6554,, 2014.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings the IEEE conference computer vision and pattern recognition, pp. 315-823.22 2015.
2015.
Sainbayar Sukhbaatar, Joan Bruna. Manohar Paluri, Lubomir Bourdev, and Rob Fergus. Training convolutional networks with noisy labels. arXiv preprint arXiv:1406.2085 2014.
Sunil Thulasidasan, Tanmoy Bhattacharya, Jeff Bilmes, Gopinath Chennupati, and Jamal Mohd- Yusof. Combating label noise in deep learning using abstention. In International Conference on Machine Learning, pp. 0234-6243, 2019.
Dong Wang and Xiaoyang Tan. Robust distance metric learning in the presence of label noise. In AAAL, pp. 1321-1327, 2014.
Dong Wang and Xiaoyang Tan. Unsupervised feature learning with c-svddnet. Pattern Recognition, 60:473-485, 2016.
Dong Wang and Xiaoyang Tan. Robust distance metric learning via bayesian inference. IEEE Transactions on Image Processing, 2743):154--153, 2018.
Jian Wang, Feng Zhou, Shilei Wen, Xiao Liu, and Yuanqing Lin. Deep metric learning with angular loss. In Proceedings of the IEEE International Conference Computer Vision, pp. 2593-2601, 2017.
Wenlin Wang, Changyou Chen, Wenlin Chen, Piyush Rai, and Lawrence Carin. Deep metric learn- ing with data summarization. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 777-794. Springer, 2016.
Kilian Q Weinberger and Lawrence K Saul. Distance metric learning for large margin nearest neigh- bor classification. Journal of Machine Learning Research, 10(Feh):207-244, 2009.
Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. Learning from massive noisy labeled data for image classification. In Proceedings of the IEEE conference computer vision and patlern recognition, pp. 2691-2699, 2015.
Liu Yang, Rong Jin, and Rahul Sukthankar. Bayesian active distance metric learning. arXiv preprint 1rXiv:/206.10 5283, 2012.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03533 2016.
Zhilu Zhang and Mert R Sabuneu. Generalized eross entropy loss for training deep neural networks with noisy labels. arXiv preprint arXiv:/805.000 8805007836, 2018.
Jun Zhu, Ning Chen, and Eric P Xing. Bayesian inference with posterior regularization and appli- cations to infinite latent syms. The Journal of Machine Learning Research, 15(1):1799-188) 1847, 2014.
11


Reviews:
[{"review_id": "rEL8AqSt5Z", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a robust Bayesian deep metric learning framework against noise label inspired the BLMNN (Wang & Tan, 2018), deep metric learning (Hoffer & Ailon, 2015; Hu et al., 2015; Wang et al., 2017; Lu et al., 2017; Do et al., 2019), and Bayes by Backprop (Blundell et al., 2015). Directly applying the variational Bayes learning (Wang & Tan, 2018) in deep learning is challenging since it requires sampling from a distribution of the neural network parameters. Instead, this paper adapts the variational inference by Blundell et al. (Blundell et al., 2015), which allows to efficiently sample the parameters of a Bayes neural networks by using a backpropagation-compatible algorithm. The experimental results on several noisy data sets show that our novel proposed method can generalize better compared to the linear BLMNN (Wang & Tan, 2018) and the point estimation-based deep metric learning (Hoffer & Ailon,\n2015; Lu et al., 2017), especially when the noise level increases.\n\nPros:\n1.  This paper is well-organized and well-written.\n2.  Adapting the variational inference by Blundell et al. (Blundell et al., 2015) for Bayesian DML sounds good.\n3.  The theoretical analysis is completed (though meaningless)\n\nCons:\n1.\tThe novelty is limited (just using sliding window to instead growing window). It is more like a combination of variational inference (Blundell et al., 2015) and DML.\n2.\tThe results of Theorem 1 are meaningless. Some papers [1] Robustness and Generalization for Metric Learning. and [2] Deep Metric Learning: The Generalization Analysis and an Adaptive Algorithm. may help you to understand this point. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "oizq-foVD1X", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Briefing:\n\nThis paper proposes a new Bayesian metric learning method, robust to noisy labels. The paper introduces a variational formulation for incorporating the Bayesian framework to triplet-loss training, with supporting experiments.\n\nStrong points\n\nThe paper proposes fancy variational derivation for the Bayesian frameworks for triplet-loss training.\n\nWeak points\n\n(1) An essential reference [1] and other triplet ablations [2] seems to be missing.\n\n[1] Lin, Xudong, et al. \"Deep variational metric learning.\" Proceedings of the European Conference on Computer Vision (ECCV). 2018.\n\n[2] Duan, Yueqi, et al. \"Deep adversarial metric learning.\" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.\n\n\n(2) Connected to (1), Experiments do not seem to be enough for supporting the superiority of the proposed method.\n\n\nComments:\n\n(1) The author should clarify the difference between [1]. [1] also uses variational formulation for the triplet loss.\n\n(2) Experimental setup seems to be not enough. Along with [1], typical retrieval dataset such as CUB-200- 2011, Stanford Online Products dataset, Cars196 dataset should be concerned.\n\n(3) Along with [1] and [2], the author should compare recent triplet-loss-based embeddings with the mentioned typical retrieval datasets.\n\nNote:\n\nThe paper seems to miss the related works that must compare. The paper also does not seem to include typical retrieval tasks to compare with other metric learning methods, which has been standard experiments in this field. Unless a clear explanation for this issue are provided, the reviewer cannot avoid rejecting this paper. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "mCI7jEicWQf", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "# Summary\n\nThe authors present a method to employ Bayesian deep learning with a metric learning (triplet loss) objective. This is a significant extension to existing Bayesian deep learning approaches, which have focused on regression and classification approaches with vanilla L2 and cross entropy losses.\n\n# pros:\n\n- The proposed method appears to be first to soundly combine deep metric learning with a Bayesian approach (Bayes by Backprop) to estimate parameter uncertainties. \n\n- In an extensive evaluation, the method is shown to deal well with highly noisy labels (up to 50%) in classification tasks like MNIST and CIFAR-10. In high noise setups, the method is shown to outperform linear Bayesian metric learning, as well as deep point estimate metric learning.\n\n- The paper is clearly written\n\n# cons:\n\n- The method is not evaluated against the state of the art on the evaluated datasets, albeit the authors give a convincing reason for this, namely that the methodological novelty of the proposed approach warrants proof-of-concept results with a relatively simple neural network architecture\n\n# comments:\n\n- It would be interesting to hear in more detail what motivated the authors' choice of Blundell's approach to uncertainty estimation as compared to e.g. the Dropout- / Deep Gaussian Process-based works by Yarin Gal et al. \n\n- The abstract mentions \"simulated label noise\" as something the authors have considered, whereas the experiments mention \"synthetic\" and \"realistic label noise\", and conclusion refers to \"simulated label noise\" in the context of future work. Could the authors please clarify, and add a more thorough description of the respective types of noise?\n\n- I wasn't able to fully follow the theoretical derivation of the authors on how the approach of Blundell et al. (2015), proposed for vanilla regression problems, transfers to a metric learning setup. Hence I have low confidence in my review. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "jyWgStwaVS2", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces a Bayesian deep metric learning framework that is robust against noise labels. The proposed method is inspired by the BLMNN (Wang & Tan, 2018), deep metric learning (Hoffer & Ailon, 2015; Hu et al., 2015; Wang et al., 2017; Lu et al., 2017; Do et al., 2019), and Bayes by Backprop (Blundell et al., 2015).  Different from BLMNN that only considers a linear metric learning, the authors’ framework can handle non-linear deep metric learning, which is useful for many real-life applications. Moreover, directly applying the variational Bayes learning (Wang & Tan, 2018) in deep learning is challenging since it requires sampling from a distribution of the neural network parameters. Instead, The author adapt the variational inference by Blundell et al. (Blundell et al., 2015), which allows to efficiently sample the parameters of a Bayes neural networks by using a backpropagation-compatible algorithm. They also theoretically show the robustness of the proposed method when working with label noise. The experimental results on several noisy data sets show that their novel proposed method can generalize better compared to the linear BLMNN (Wang & Tan, 2018) and the point estimation-based deep metric learning (Hoffer & Ailon, 2015; Lu et al., 2017), especially when the noise level increases. In my opinion, the main novelty of this\n\n+ves: \n+ The idea of using the variational inference by Blundell et al. (Blundell et al., 2015) to derive Bayesian version of deep metric learning is interesting. \n\n+ Overall, the paper is well-written and well-organized. In particular, the Related Work section has a nice flow and puts the proposed method into context. Despite the method having limited novelty (sliding window instead of a growing window), the method has been well motivated in Sections 1 and 3. \n\n+ The theoretical analysis section is completed. And the results section is well structured. It's nice to see the advantages of the proposed method over other compared methods under different datasets with label noise\n \nConcerns: \n- The key concern about the paper is the contributions of this paper. In my opinion, the novelty of this paper is limited. The main contribution is to propose a Bayesian version of deep metric learning. In fact, the main strategies and motivation from (Blundell et al., 2015) and (Wang & Tan, 2018). I cannot find any original contributions such as sampling methods and novel loss function in this paper. In a word, this paper only combines variational inference by Blundell et al. (Blundell et al., 2015) with deep metric learning without any original techniques.\n \n- Lack of sufficient analysis to show that your method is robust to label noise. In modeling, the authors did not consider any label information, but the model is robust to label noise. This is questionable. In addition, the result of Theorem 1 is meaningless. \nI don’t want to waste my time explaining why your theoretical results are meaningless, but I recommend some papers as follows:\n[1] Deep Metric Learning: The Generalization Analysis and an Adaptive Algorithm\n[2] Robustness and Generalization for Metric Learning\n\n-According to my experience, the Bayesian version of a model should be much higher than the original model. But the author's experimental results are not satisfied. \n\nI can't find any contribution that can conquer me, thus, this paper should not be accepted by ICLR2021\n\n \n\n\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers have commented on the lack of novelty of the paper as it reads only as applying the variational inference framework of Blundell et al. (2015) to deep metric learning (R2 and R4). Furthermore, the paper has not properly positioned itself when compared to previous works on "Deep variational metric learning" and "Deep adversarial metric learning" (R1) and other previous literature that have studied robustness for metric learning. The argument on robustness to noisy labels needs to be expanded and better fleshed out in a future version of the paper. 
Decision: Reject