Original: B AYESIAN M ETRIC L EARNING FOR R OBUST T RAINING OF D EEP M ODELS UNDER N OISY L ABELS
Revised: B AYESIAN M ETRIC L EARNING FOR R OBUST T RAINING OF D EEP M ODELS UNDER N OISY L ABELS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Label noise is a natural event of data collection and annotation and has been shown to have signiﬁcant impact on the performance of deep learning models regarding accuracy reduction and sample complexity increase.
Revised: Label noise is a natural event of data collection and annotation and has been shown to have signiﬁcant impact on the performance of deep learning models regarding accuracy reduction and sample complexity increase.

Original: This paper aims to develop a novel theoretically sound Bayesian deep metric learning that is robust against noisy labels.
Revised: This paper aims to develop a novel theoretically sound Bayesian deep metric learning that is robust against noisy labels.

Original: Our proposed approach is inspired by a linear Bayesian large margin nearest neighbor classiﬁcation and is a combination of Bayesian learning, triplet loss-based deep metric learning and variational inference frameworks.
Revised: Our proposed approach is inspired by a linear Bayesian large margin nearest neighbor classiﬁcation and is a combination of Bayesian learning, triplet loss-based deep metric learning and variational inference frameworks.

Original: We theoretically show the robustness under label noise of our proposed method.
Revised: We theoretically show the robustness under label noise of our proposed method.

Original: The experimental results on benchmark data sets that contain both simulated and realistic label noise show a considerable improvement in the classiﬁcation accuracy of our method compared to the linear Bayesian metric learning and the point estimate deep metric learning.
Revised: The experimental results on benchmark data sets that contain both synthetic and realistic label noise show a considerable improvement in the classiﬁcation accuracy of our method compared to the linear Bayesian metric learning and the point estimate deep metric learning.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep learning has been shown as a dominant learning framework in various domains of machine learning and computer vision.
Revised: Deep learning has been shown as a dominant learning framework in various domains of machine learning and computer vision.

Original: One of the major limitations of deep learning is that it often requires relatively clean data sets that do not contain label noise naturally caused by human labeling errors, measurement errors, subjective biases and other issues (Fr ´ enay et al., 2014; Ghosh et al., 2017; Algan & Ulusoy, 2019).
Revised: One of the major limitations of deep learning is that it often requires relatively clean data sets that do not contain label noise naturally caused by human labeling errors, measurement errors, subjective biases and other issues (Fr ´ enay et al., 2014; Ghosh et al., 2017; Algan & Ulusoy, 2019).

Original: The performance of a machine learning method can be signiﬁcantly affected by noisy labels both in terms of the reduction in the accuracy rate and the increase in sample complexity.
Revised: The performance of a machine learning method can be signiﬁcantly affected by noisy labels both in terms of the reduction in the accuracy rate and the increase in sample complexity.

Original: Particularly for deep learning, a deep neural network (DNN) can generalize poorly when trained with noisy training sets which contain high proportion of noisy labels since a DNN can over-ﬁt those noisy training data sets (Zhang et al., 2016; Algan & Ulusoy, 2020).
Revised: Particularly for deep learning, a deep neural network (DNN) can generalize poorly when trained with noisy training sets which contain high proportion of noisy labels since a DNN can over-ﬁt those noisy training data sets (Zhang et al., 2016; Algan & Ulusoy, 2020).

Original: Developing deep learning methods that can perform well on noisy training data is essential since it can enable the use of deep models in many real-life applications.
Revised: Developing deep learning methods that can perform well on noisy training data is essential since it can enable the use of deep models in many real-life applications.

Original: There have been several approaches proposed to handle learning issues caused by label noise, for example: data cleaning (Angelova et al., 2005; Chu et al., 2016), label correction (Reed et al., 2014), additional linear correction layers (Sukhbaatar et al., 2014), dimensionality-driven learning (Ma et al., 2018), bootstrapping (Reed et al., 2014), curriculum learning-model based approach such as MentorNet (Jiang et al., 2018) or CoTeaching (Han et al., 2018), loss correction (or noisetolerant loss) (Masnadi-Shirazi & Vasconcelos, 2009; Ghosh et al., 2017; Zhang & Sabuncu, 2018;
Revised: There have been several approaches proposed to handle learning issues caused by label noise, for example: data cleaning (Angelova et al., 2005; Chu et al., 2016), label correction (Reed et al., 2014), additional linear correction layers (Sukhbaatar et al., 2014), dimensionality-driven learning (Ma et al., 2018), bootstrapping (Reed et al., 2014), curriculum learning-model based approach such as MentorNet (Jiang et al., 2018) or CoTeaching (Han et al., 2018), loss correction (or noisetolerant loss) (Masnadi-Shirazi & Vasconcelos, 2009; Ghosh et al., 2017; Zhang & Sabuncu, 2018;

Original: Thulasidasan et al., 2019; Ma et al., 2020), or a combination of the techniques above (Li et al., 2020; Nguyen et al., 2019).
Revised: Thulasidasan et al., 2019; Ma et al., 2020), or a combination of the techniques above (Li et al., 2020; Nguyen et al., 2019).

Original: Relevant to this paper is an existing theoretically sound approach:
Revised: Relevant to this paper is an existing theoretically sound approach:

Original: Bayesian large margin nearest neighbor classiﬁcation (BLMNN) (Wang & Tan, 2018) that employs
Revised: Bayesian large margin nearest neighbor classiﬁcation (BLMNN) (Wang & Tan, 2018) that employs

Original: Bayesian inference to improve the robustness of a point estimation-based linear metric learning method.
Revised: Bayesian inference to improve the robustness of a point estimation-based linear metric learning method.

Original: BLMNN then introduces a method to approximate the posterior distribution of the underlying distance parameter given the triplet data by using the stochastic variational inference.
Revised: BLMNN then introduces a method to approximate the posterior distribution of the underlying distance parameter given the triplet data by using the stochastic variational inference.

Original: More importantly, BLMNN (Wang & Tan, 2018) also provides a theoretical guarantee about the robustness of the method, which says that it can work with non-uniform label noise.
Revised: More importantly, BLMNN (Wang & Tan, 2018) also provides a theoretical guarantee about the robustness of the method, which says that it can work with non-uniform label noise.

Original: Although BLMNN has been mathematically shown to be robust against label noise, it only focuses on a simple linear
Revised: Although BLMNN has been mathematically shown to be robust against label noise, it only focuses on a simple linear

Original: Mahalanobis distance that can not capture the nonlinear relationships of data points in deep metric learning (Lu et al., 2017).
Revised: Mahalanobis distance that can not capture the nonlinear relationships of data points in deep metric learning (Lu et al., 2017).

Original: In this paper, we introduce a Bayesian deep metric learning framework that is robust against noisy labels.
Revised: In this paper, we introduce a Bayesian deep metric learning framework that is robust against noisy labels.

Original: Our proposed method (depicted in Fig. 1) is inspired by the BLMNN (Wang & Tan, 2018), deep metric learning (Hoffer & Ailon, 2015;
Revised: Our proposed method (depicted in Fig. 1) is inspired by the BLMNN (Wang & Tan, 2018), deep metric learning (Hoffer & Ailon, 2015;

Original: Hu et al., 2015; Wang et al., 2017; Lu et al., 2017;
Revised: Hu et al., 2015; Wang et al., 2017; Lu et al., 2017;

Original: Do et al., 2019), and Bayes by Backprop (Blundell et al., 2015).
Revised: Do et al., 2019), and Bayes by Backprop (Blundell et al., 2015).

Original: Compared to the BLMNN that only considers a linear metric learning, our framework can handle non-linear deep metric learning, which is useful for many real-life applications.
Revised: Compared to the BLMNN that only considers a linear metric learning, our framework can handle non-linear deep metric learning, which is useful for many real-life applications.

Original: Moreover, directly applying the variational Bayes learning (Wang & Tan, 2018) in deep learning is challenging since it requires sampling from a distribution of the neural network parameters.
Revised: Moreover, directly applying the variational Bayes learning (Wang & Tan, 2018) in deep learning is challenging since it requires sampling from a distribution of the neural network parameters.

Original: Instead, we adapt the variational inference by Blundell et al.
Revised: Instead, we adapt the variational inference by Blundell et al.

Original: (Blundell et al., 2015), which allows to efﬁciently sample the parameters of a Bayes neural networks by using a backpropagation-compatible algorithm.
Revised: (Blundell et al., 2015), which allows to efﬁciently sample the parameters of a Bayes neural networks by using a backpropagation-compatible algorithm.

Original: We also theoretically show the robustness of our proposed method when working with label noise.
Revised: We also theoretically show the robustness of our proposed method when working with label noise.

Original: The experimental results on several noisy data sets show that our novel proposed method can generalize better compared to the linear
Revised: The experimental results on several noisy data sets show that our novel proposed method can generalize better compared to the linear

Original: BLMNN (Wang & Tan, 2018) and the point estimation-based deep metric learning (Hoffer & Ailon, 2015; Lu et al., 2017), especially when the noise level increases.
Revised: BLMNN (Wang & Tan, 2018) and the point estimation-based deep metric learning (Hoffer & Ailon, 2015; Lu et al., 2017), especially when the noise level increases.

Original: It is important to emphasize that the motivation of our method is to produce a better calibrated model that is more robust to noisy label training, and, as a result, less likely to overﬁt the training set than the linear BLMNN (Wang & Tan, 2018) and the point estimation-based deep metric learning (Hoffer & Ailon, 2015; Lu et al., 2017).
Revised: It is important to emphasize that the motivation of our method is to produce a better calibrated model that is more robust to noisy label training, and, as a result, less likely to overﬁt the training set than the linear BLMNN (Wang & Tan, 2018) and the point estimation-based deep metric learning (Hoffer & Ailon, 2015; Lu et al., 2017).

Original: Therefore this is a paper that introduces a new theoretical framework to solve noisy label learning instead of presenting method that is competitive against the best approaches of the ﬁeld (such as Mentornet (Jiang et al., 2018) or Co-teaching (Han et al., 2018)) in large-scale datasets (e.g., webvision (Li et al., 2017) and Clothing 1M (Xiao et al., 2015)).
Revised: Therefore this is a paper that introduces a new theoretical framework to solve noisy label learning instead of presenting method that is competitive against the best approaches of the ﬁeld (such as Mentornet (Jiang et al., 2018) or Co-teaching (Han et al., 2018)) in large-scale datasets (e.g., webvision (Li et al., 2017) and Clothing 1M (Xiao et al., 2015)).

Original: Furthermore, deep metric learning has been in fact considered in the Bayesian settings before (Ishfaq et al., 2018; Karaletsos et al., 2015),  but not in the context of noisy labels.
Revised: Furthermore, deep metric learning has been in fact considered in the Bayesian settings before (Ishfaq et al., 2018; Karaletsos et al., 2015), and recently, in (Lin et al., 2018), but not in the context of noisy labels.

Original: Consequently, our proposed framework can be used by other methods that can deal with noisy label learning, but the extension of those methods using our proposed approach is out of the scope of this paper.
Revised: Consequently, our proposed framework can be used by other methods that can deal with noisy label learning, but the extension of those methods using our proposed approach is out of the scope of this paper.

Original:  R ELATED W ORK 2. P OINT E STIMATION -B ASED D ISTANCE M ETRIC L EARNING
Revised:  R ELATED W ORK 2. P OINT E STIMATION -B ASED D ISTANCE M ETRIC L EARNING

Original: The goal of distance metric learning (or metric learning) is to learn a distance function to measure the similarity between training samples.
Revised: The goal of distance metric learning (or metric learning) is to learn a distance function to measure the similarity between training samples.

Original: Metric learning has been shown to have great success in many visual applications such as face recognition, image classiﬁcation, visual search, visual tracking, and person re-identiﬁcation
Revised: Metric learning has been shown to have great success in many visual applications such as face recognition, image classiﬁcation, visual search, visual tracking, and person re-identiﬁcation

Original: (Lu et al., 2017).
Revised: (Lu et al., 2017).

Original: In principle, a supervised metric learning method aims to learn a distance metric which pulls together samples from the same class while pushing away those from different classes.
Revised: In principle, a supervised metric learning method aims to learn a distance metric which pulls together samples from the same class while pushing away those from different classes.

Original: Based on the complexity of the distance, metric learning can be classiﬁed into two types: linear, focusing on linear distance (e.g., Mahalanobis), which often suffers from the nonlinear relationship of data points (Lu et al., 2017); and non-linear, which nowadays is mostly based on deep learning.
Revised: Based on the complexity of the distance, metric learning can be classiﬁed into two types: linear, focusing on linear distance (e.g., Mahalanobis), which often suffers from the nonlinear relationship of data points (Lu et al., 2017); and non-linear, which nowadays is mostly based on deep learning.

Original: Deep metric learning (DML) is motivated by the fact that deep learning is an effective solution to a non-linear transformation of input samples (Lu et al., 2017; Do et al., 2019).
Revised: Deep metric learning (DML) is motivated by the fact that deep learning is an effective solution to a non-linear transformation of input samples (Lu et al., 2017; Do et al., 2019).

Original: The key idea of
Revised: The key idea of

Original: DML is to explicitly learn a set of hierarchical non-linear transformations to map input data points into a feature space that is used for comparing or matching these data points in a more effective manner.
Revised: DML is to explicitly learn a set of hierarchical non-linear transformations to map input data points into a feature space that is used for comparing or matching these data points in a more effective manner.

Original: DML uniﬁes feature learning and metric learning into a joint learning framework.
Revised: DML uniﬁes feature learning and metric learning into a joint learning framework.

Original: DML is shown to be more advantageous compared to traditional models, for example, with respect to classiﬁcation performance since DML does not rely on the classiﬁcation layer of a trained model that often depends on the type of problems (Lu et al., 2017; Do et al., 2019).
Revised: DML is shown to be more advantageous compared to traditional models, for example, with respect to classiﬁcation performance since DML does not rely on the classiﬁcation layer of a trained model that often depends on the type of problems (Lu et al., 2017; Do et al., 2019).

Original: Relevant to this paper is the triplet loss-based metric learning (Lu et al., 2017; Schroff et al., 2015), in which the original training data set is represented by a set of independent triplets, formed by an anchor sample, one sample of the same class and another sample from a different class.
Revised: Relevant to this paper is the triplet loss-based metric learning (Lu et al., 2017; Schroff et al., 2015), in which the original training data set is represented by a set of independent triplets, formed by an anchor sample, one sample of the same class and another sample from a different class.

Original: The training process is then performed by minimizing the triplet loss to simultaneously approximate the anchor to the sample with the same class and separate the anchor from the sample of different class.
Revised: The training process is then performed by minimizing the triplet loss to simultaneously approximate the anchor to the sample with the same class and separate the anchor from the sample of different class.

Original: 2. R OBUST L INEAR D ISTANCE M ETRIC L EARNING VIA B AYESIAN I NFERENCE
Revised: 2. R OBUST L INEAR D ISTANCE M ETRIC L EARNING VIA B AYESIAN I NFERENCE

Original: One drawback of the point estimation DML is that it is likely to over-ﬁt the noisy labels (Wang & Tan, 2014; 2018).
Revised: One drawback of the point estimation DML is that it is likely to over-ﬁt the noisy labels (Wang & Tan, 2014; 2018).

Original: Wang et al.
Revised: Wang et al.

Original: (2016) introduces the Deep Stochastic Neighbor Compression (DSNC) method that aims to jointly learn a nonlinear transformation that preserves the neighborhood of the data, and a compressed version of the training data set (Ahmed et al.).
Revised: (2016) introduces the Deep Stochastic Neighbor Compression (DSNC) method that aims to jointly learn a nonlinear transformation that preserves the neighborhood of the data, and a compressed version of the training data set (Ahmed et al.).

Original: DSNC is also robust against label noise.
Revised: DSNC is also robust against label noise.

Original: Motivated by the fact that Bayesian learning is a good choice for robust learning (Zhu et al., 2014; Yang et al., 2012), Wang & Tan (2018) introduced the theoretically sound Bayesian large margin nearest neighbor classiﬁcation (BLMNN) to improve the robustness of the linear metric learning under the presence of label noise.
Revised: Motivated by the fact that Bayesian learning is a good choice for robust learning (Zhu et al., 2014; Yang et al., 2012), Wang & Tan (2018) introduced the theoretically sound Bayesian large margin nearest neighbor classiﬁcation (BLMNN) to improve the robustness of the linear metric learning under the presence of label noise.

Original: The BLMNN framework represents the large margin nearest neighbor classiﬁcation using a linear metric learning in the form of a variational
Revised: The BLMNN framework represents the large margin nearest neighbor classiﬁcation using a linear metric learning in the form of a variational

Original: Bayes method that takes the prior distribution of the transformation matrix into account and estimate the posterior distribution via stochastic variational inference (SVI).
Revised: Bayes method that takes the prior distribution of the transformation matrix into account and estimate the posterior distribution via stochastic variational inference (SVI).

Original: BLMNN provides mathematical deﬁnitions of the noisy label triplet (a type of non-uniform label noise) and the β -robust algorithm against noisy label triplets.
Revised: BLMNN provides mathematical deﬁnitions of the noisy label triplet (a type of non-uniform label noise) and the β -robust algorithm against noisy label triplets.

Original: More importantly, a theoretically guarantee of the robustness of BLMNN is also provided in (Wang & Tan, 2018).
Revised: More importantly, a theoretically guarantee of the robustness of BLMNN is also provided in (Wang & Tan, 2018).

Original: Although BLMNN efﬁciently addresses the training issue caused by the noisy label, one key limitation of BLMNN is that it is based on a linear metric transformation that cannot capture the nonlinear relationships of data points in deep metric learning.
Revised: Although BLMNN efﬁciently addresses the training issue caused by the noisy label, one key limitation of BLMNN is that it is based on a linear metric transformation that cannot capture the nonlinear relationships of data points in deep metric learning.

Original: However, the extension of BLMNN to the non-linear case is not straightforward due to the complexity of the posterior distribution estimation as well as the sampling process of high-dimensional parameters, and such extension is the main target of our paper.
Revised: However, the extension of BLMNN to the non-linear case is not straightforward due to the complexity of the posterior distribution estimation as well as the sampling process of high-dimensional parameters, and such extension is the main target of our paper.

Original: 2. B AYES BY B ACKPROP
Revised: 2. B AYES BY B ACKPROP

Original: Bayesian neural networks (MacKay, 1995; Neal, 2012; Gal & Ghahramani, 2015) aim to estimate the posterior distribution of the network parameters given the training data.
Revised: Bayesian neural networks (MacKay, 1995; Neal, 2012; Gal & Ghahramani, 2015) aim to estimate the posterior distribution of the network parameters given the training data.

Original: However, that inference framework is often intractable, especially when working with high dimensional parameters (Blundell et al., 2015).
Revised: However, that inference framework is often intractable, especially when working with high dimensional parameters (Blundell et al., 2015).

Original: Moreover, exactly calculating the posterior of the weights is challenging since it requires the integration that is known to be computationally expensive.
Revised: Moreover, exactly calculating the posterior of the weights is challenging since it requires the integration that is known to be computationally expensive.

Original: Blundell et al.
Revised: Blundell et al.

Original: (Blundell et al., 2015) introduced the “backpropagation-compatible” Bayes by Backprop method for estimating the posterior distribution of the network parameter.
Revised: (Blundell et al., 2015) introduced the “backpropagation-compatible” Bayes by Backprop method for estimating the posterior distribution of the network parameter.

Original: That method is inspired by the variational free energy inference in which the exact posterior is approximated by a variational distribution by solving an optimization problem.
Revised: That method is inspired by the variational free energy inference in which the exact posterior is approximated by a variational distribution by solving an optimization problem.

Original: In principle, Bayes by Backprop can directly work on the network parameter by minimizing a compression cost function (Blundell et al., 2015).
Revised: In principle, Bayes by Backprop can directly work on the network parameter by minimizing a compression cost function (Blundell et al., 2015).

Original: Our proposed method is inspired by (Wang & Tan, 2018), where we replace the linear Mahalanobis metric by a non-linear deep metric.
Revised: Our proposed method is inspired by (Wang & Tan, 2018), where we replace the linear Mahalanobis metric by a non-linear deep metric.

Original: Our novel proposed method therefore leverages the good performance of the deep metric learning (Lu et al., 2017) and the robustness to label noise of a Bayesian framework (Yang et al., 2012; Wang & Tan, 2018).
Revised: Our novel proposed method therefore leverages the good performance of the deep metric learning (Lu et al., 2017) and the robustness to label noise of a Bayesian framework (Yang et al., 2012; Wang & Tan, 2018).

Original: We ﬁrst represent a triplet-based deep metric learning using Bayesian inference.
Revised: We ﬁrst represent a triplet-based deep metric learning using Bayesian inference.

Original: By replacing the softmax loss by a triplet loss (and so learning a metric), we impose more strict constraints, where points of the same class are forced to collapse into a small region of the feature space instead of just belonging to regions within the class boundaries.
Revised: By replacing the softmax loss by a triplet loss (and so learning a metric), we impose more strict constraints, where points of the same class are forced to collapse into a small region of the feature space instead of just belonging to regions within the class boundaries.

Original: We argue that this more strict constraints has the potential to introduce more robust feature spaces for classiﬁcation under noisy label training.
Revised: We argue that this more strict constraints has the potential to introduce more robust feature spaces for classiﬁcation under noisy label training.

Original: To approximate the posterior of the network parameters, we then employ the efﬁcient variational framework (Blundell et al., 2015) that allows to sample from a Bayesian neural network using the backpropagation framework.
Revised: To approximate the posterior of the network parameters, we then employ the efﬁcient variational framework (Blundell et al., 2015) that allows to sample from a Bayesian neural network using the backpropagation framework.

Original: We theoretically show that our proposed method is robust against noisy triplet relying on a Bayesian inference framework.
Revised: We theoretically show that our proposed method is robust against noisy triplet relying on a Bayesian inference framework.

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: In this section, we explain our proposed method.
Revised: In this section, we explain our proposed method.

Original: Then, we theoretically show the robustness of that method under noisy label triplet.
Revised: Then, we theoretically show the robustness of that method under noisy label triplet.

Original: 3. D EEP METRIC LEARNING WITH TRIPLET LOSS
Revised: 3. D EEP METRIC LEARNING WITH TRIPLET LOSS

Original: Let us denote the original data set by T = { ( x i , y i ) } Ni =1 , where x i ∈ R D is a sample and
Revised: Let us denote the original data set by T = { ( x i , y i ) } Ni =1 , where x i ∈ R D is a sample and

Original: y i ∈
Revised: y i ∈

Original: Y ⊂ N is its corresponding label.
Revised: Y ⊂ N is its corresponding label.

Original: Let S
Revised: Let S

Original: = { ( i, j ) | ( x i , y i ) , ( x j , y j ) ∈ T and y i =
Revised: = { ( i, j ) | ( x i , y i ) , ( x j , y j ) ∈ T and y i =

Original: y j } and D
Revised: y j } and D

Original: = { ( i, j ) | ( x i , y i ) , ( x j , y j ) ∈ T and y i (cid:54) =
Revised: = { ( i, j ) | ( x i , y i ) , ( x j , y j ) ∈ T and y i (cid:54) =

Original: y j } denote the set of pairs of the data points with the same labels and the set of pairs with different labels, then a triplet is deﬁned by z =
Revised: y j } denote the set of pairs of the data points with the same labels and the set of pairs with different labels, then a triplet is deﬁned by z =

Original: ( i, j, l ) , where ( i, j ) ∈ S , and ( i, l ) ∈ D .
Revised: ( i, j, l ) , where ( i, j ) ∈ S , and ( i, l ) ∈ D .

Original: A triplet-based deep metric learning framework (Hoffer & Ailon, 2015;
Revised: A triplet-based deep metric learning framework (Hoffer & Ailon, 2015;

Original: Lu et al., 2017; Do et al., 2019) aims to minimize the following triplet loss function for the triplet z
Revised: Lu et al., 2017; Do et al., 2019) aims to minimize the following triplet loss function for the triplet z

Original: = ( i, j, l ) ,
Revised: = ( i, j, l ) ,

Original: [Equation]
Revised: [Equation]

Original: i,j,l
Revised: i,j,l

Original: [Equation]
Revised: [Equation]

Original: tri
Revised: tri

Original: [Equation]
Revised: [Equation]

Original: where the deep metric is deﬁned by
Revised: where the deep metric is deﬁned by

Original: [Equation]
Revised: [Equation]

Original: with f ( · , θ ) denoting the network function parameterized by θ , and h ( a ) = max(0 , a ) , a ∈ R representing the hinge loss function (with τ > 0 being a margin between d f ( x i , x j ) and d f ( x i , x l ) ).
Revised: with f ( · , θ ) denoting the network function parameterized by θ , and h ( a ) = max(0 , a ) , a ∈ R representing the hinge loss function (with τ > 0 being a margin between d f ( x i , x j ) and d f ( x i , x l ) ).

Original: [Equation]
Revised: [Equation]

Original: Let us denote the training data set by Z = { z k } k =1 ,..., |Z| , which contains independent triplets z
Revised: Let us denote the training data set by Z = { z k } k =1 ,..., |Z| , which contains independent triplets z

Original: = ( i, j, l ) and the network parameter by θ ∈ R P .
Revised: = ( i, j, l ) and the network parameter by θ ∈ R P .

Original: According to the large margin principle (Wang & Tan, 2018), the likelihood function p ( Z| θ ) can be deﬁned as follows:
Revised: According to the large margin principle (Wang & Tan, 2018), the likelihood function p ( Z| θ ) can be deﬁned as follows:

Original: [Equation]
Revised: [Equation]

Original: where C is a normalising constant.
Revised: where C is a normalising constant.

Original: Without loss of generality, we assume that the prior of the network parameter is a Gaussian with mean µ 0 and covariance matrix V 0 , i.e, p ( θ ) =
Revised: Without loss of generality, we assume that the prior of the network parameter is a Gaussian with mean µ 0 and covariance matrix V 0 , i.e, p ( θ ) =

Original: N ( θ | µ 0 , V 0 ) .
Revised: N ( θ | µ 0 , V 0 ) .

Original: By using the Bayes’ rule, the posterior distribution of the parameter θ can be represented by
Revised: By using the Bayes’ rule, the posterior distribution of the parameter θ can be represented by

Original: [Equation]
Revised: [Equation]

Original: Naively estimating the posterior in equation 4 is often intractable, especially for the high dimensional parameter space of deep models.
Revised: Naively estimating the posterior in equation 4 is often intractable, especially for the high dimensional parameter space of deep models.

Original: Hence, we employ the variational learning approach (Blundell et al., 2015; Blei et al., 2017) to approximate the posterior distribution in equation 4.
Revised: Hence, we employ the variational learning approach (Blundell et al., 2015; Blei et al., 2017) to approximate the posterior distribution in equation 4.

Original: In particular, the goal of that variational method is to estimate the variational parameter λ of a distribution for the network parameters, denoted by q ( θ | λ ) that minimizes the Kullback-Leiber (KL) divergence between the variational distribution and the true posterior, i.e.,
Revised: In particular, the goal of that variational method is to estimate the variational parameter λ of a distribution for the network parameters, denoted by q ( θ | λ ) that minimizes the Kullback-Leiber (KL) divergence between the variational distribution and the true posterior, i.e.,

Original: [Equation]
Revised: [Equation]

Original: The objective function in equation 5 embodies a trade-off between the complexity of the data and the simplicity of the prior.
Revised: The objective function in equation 5 embodies a trade-off between the complexity of the data and the simplicity of the prior.

Original: Blundell et al.
Revised: Blundell et al.

Original: (Blundell et al., 2015) pointed out that directly evaluating that cost function is computationally expensive (Blundell et al., 2015), so they introduced an unbiased
Revised: (Blundell et al., 2015) pointed out that directly evaluating that cost function is computationally expensive (Blundell et al., 2015), so they introduced an unbiased

Original: Monte-Carlo approximation of the exact cost function in equation 5 with the following function:
Revised: Monte-Carlo approximation of the exact cost function in equation 5 with the following function:

Original: [Equation]
Revised: [Equation]

Original: Algorithm
Revised: Algorithm

Original: Establish the triplet training set Z
Revised: Establish the triplet training set Z

Original: Initialize the variational parameter λ = ( µ, ρ ) repeat
Revised: Initialize the variational parameter λ = ( µ, ρ ) repeat

Original: Sample ε ∼ N (0 , I )
Revised: Sample ε ∼ N (0 , I )

Original: Compute θ with equation
Revised: Compute θ with equation

Original: Compute the likelihood p ( Z| θ ) with equation
Revised: Compute the likelihood p ( Z| θ ) with equation

Original: Compute function h ( θ, λ ) with equation
Revised: Compute function h ( θ, λ ) with equation

Original: Update µ, ρ with equationuntil convergence where θ ( i ) ∼ q ( θ | λ ) is the i -th Monte Carlo sample of the variational posterior distribution assumed to be a diagonal Gaussian distribution, i.e., θ ∼ N ( µ, σ 2 I ) , where I is the identity matrix, then a sample of the weights θ can be obtained by
Revised: Update µ, ρ with equationuntil convergence where θ ( i ) ∼ q ( θ | λ ) is the i -th Monte Carlo sample of the variational posterior distribution assumed to be a diagonal Gaussian distribution, i.e., θ ∼ N ( µ, σ 2 I ) , where I is the identity matrix, then a sample of the weights θ can be obtained by

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where σ = log(1 + exp ρ ) , the variational posterior parameters are λ
Revised: where σ = log(1 + exp ρ ) , the variational posterior parameters are λ

Original: =
Revised: =

Original: ( µ, ρ ) , with µ ∈ R P and ρ ∈ R , ◦ denoting point-wise multiplication, and ε ∼ N (0 , I ) .
Revised: ( µ, ρ ) , with µ ∈ R P and ρ ∈ R , ◦ denoting point-wise multiplication, and ε ∼ N (0 , I ) .

Original: The variational parameters λ = ( µ, ρ )
Revised: The variational parameters λ = ( µ, ρ )

Original: are updated by (Blundell et al., 2015):
Revised: are updated by (Blundell et al., 2015):

Original: [Equation]
Revised: [Equation]

Original: 8) where α is the step size, and
Revised: 8) where α is the step size, and

Original: [Equation]
Revised: [Equation]

Original: The gradients in equation 8 can be estimated as follows (Blundell et al., 2015):
Revised: The gradients in equation 8 can be estimated as follows (Blundell et al., 2015):

Original: [Equation]
Revised: [Equation]

Original: The whole training process of our proposed method is presented in Algorithm 1.
Revised: The whole training process of our proposed method is presented in Algorithm 1.

Original: 3. R OBUSTNESS TO L ABEL N OISE
Revised: 3. R OBUSTNESS TO L ABEL N OISE

Original: This subsection ﬁrst introduces the deﬁnition of a noisy label triplet and the robustness of a learning algorithm under that label noise.
Revised: This subsection ﬁrst introduces the deﬁnition of a noisy label triplet and the robustness of a learning algorithm under that label noise.

Original: We then theoretically show the robustness of our proposed method against noisy labels.
Revised: We then theoretically show the robustness of our proposed method against noisy labels.

Original: Deﬁnition 1.
Revised: Deﬁnition 1.

Original: A triplet z =
Revised: A triplet z =

Original: ( i, j, l ) of data points is deﬁned as a label noisy triplet (Wang & Tan, 2018)
Revised: ( i, j, l ) of data points is deﬁned as a label noisy triplet (Wang & Tan, 2018)

Original: if: 1) x j , x l ∈ N i –the set of neighbors (Weinberger & Saul, 2009) of x i , where N i = { x ∈ R D : (cid:107) x − x i (cid:107) ≤ δ } , with δ > 0 being sufﬁciently small; 2) y ij (1 − y il ) = 1 , where y ij = (cid:26) 1 , if y i =
Revised: if: 1) x j , x l ∈ N i –the set of neighbors (Weinberger & Saul, 2009) of x i , where N i = { x ∈ R D : (cid:107) x − x i (cid:107) ≤ δ } , with δ > 0 being sufﬁciently small; 2) y ij (1 − y il ) = 1 , where y ij = (cid:26) 1 , if y i =

Original: y j 0 , otherwwise; and 3)
Revised: y j 0 , otherwwise; and 3)

Original: d 2 f ( x i , x j ) − d 2 f ( x i , x l ) ≥ C d
Revised: d 2 f ( x i , x j ) − d 2 f ( x i , x l ) ≥ C d

Original: , with C d > 0 denoting a threshold.
Revised: , with C d > 0 denoting a threshold.

Original: Deﬁnition 2.
Revised: Deﬁnition 2.

Original: Let Z be an arbitrary training set and z (cid:48) a label noisy triplet.
Revised: Let Z be an arbitrary training set and z (cid:48) a label noisy triplet.

Original: Then a learning algorithm
Revised: Then a learning algorithm

Original: A is β -robust against label noise in the Bayesian inference sense if
Revised: A is β -robust against label noise in the Bayesian inference sense if

Original: [Equation]
Revised: [Equation]

Original: Lemma 1.
Revised: Lemma 1.

Original: Let Z be an arbitrary training set (in the sense that it may or may not contain any noisy label triplet).
Revised: Let Z be an arbitrary training set (in the sense that it may or may not contain any noisy label triplet).

Original: Assuming that z (cid:48)
Revised: Assuming that z (cid:48)

Original: =
Revised: =

Original: ( i, j, l ) is a noisy label triplet with the corresponding threshold
Revised: ( i, j, l ) is a noisy label triplet with the corresponding threshold

Original: C d in Deﬁnition 1, and the normalising constant of the likelihood p ( Z| θ ) , C is deﬁned in equationsuch that C ≤ exp (2( C d + 1)) .
Revised: C d in Deﬁnition 1, and the normalising constant of the likelihood p ( Z| θ ) , C is deﬁned in equationsuch that C ≤ exp (2( C d + 1)) .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Suppose that p ( θ ) (cid:54) = 0 .
Revised: Suppose that p ( θ ) (cid:54) = 0 .

Original: Given the i.i.d.
Revised: Given the i.i.d.

Original: assumption from equation 3 we have
Revised: assumption from equation 3 we have

Original: [Equation]
Revised: [Equation]

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: Note that 0 < p ( z (cid:48) | θ ) ≤ 1 , we have log p ( z (cid:48) | θ ) ≤ 0 , and then
Revised: Note that 0 < p ( z (cid:48) | θ ) ≤ 1 , we have log p ( z (cid:48) | θ ) ≤ 0 , and then

Original: [Equation]
Revised: [Equation]

Original: Since z (cid:48)
Revised: Since z (cid:48)

Original: =
Revised: =

Original: ( i, j, l ) is a noisy label triplet (Deﬁnition 1), we obtain
Revised: ( i, j, l ) is a noisy label triplet (Deﬁnition 1), we obtain

Original: [Equation]
Revised: [Equation]

Original: 17) then, by combining with the corresponding likelihood in equation 3, we obtain
Revised: 17) then, by combining with the corresponding likelihood in equation 3, we obtain

Original: [Equation]
Revised: [Equation]

Original: The resutl in equation 13 is obtained by integrating equation 18 into equation 16.
Revised: The resutl in equation 13 is obtained by integrating equation 18 into equation 16.

Original: Remark 1.
Revised: Remark 1.

Original: Assuming that the prior p ( θ ) is arbitrary, but ﬁxed and that the noisy triplet z (cid:48) and the original data Z are sampled from the same distribution, by using Bayes’ rule we then get a similar estimate for the posterior distribution, i.e.,
Revised: Assuming that the prior p ( θ ) is arbitrary, but ﬁxed and that the noisy triplet z (cid:48) and the original data Z are sampled from the same distribution, by using Bayes’ rule we then get a similar estimate for the posterior distribution, i.e.,

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Hence, according to the Deﬁnition 2 and equation 19, Algorithm 1 is Bayesian robust against noisy label triplet.
Revised: Hence, according to the Deﬁnition 2 and equation 19, Algorithm 1 is Bayesian robust against noisy label triplet.

Original: Theorem 1.
Revised: Theorem 1.

Original: Given Lemma 1, and supposing that we have N noisy label triplets z (cid:48) 1 , . . .
Revised: Given Lemma 1, and supposing that we have N noisy label triplets z (cid:48) 1 , . . .

Original: , z
Revised: , z

Original: [Equation]
Revised: [Equation]

Original: N that are conditionally independent given the parameter θ , then
Revised: N that are conditionally independent given the parameter θ , then

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: By applying the result in Equation 19 of Lemma 1 we get
Revised: By applying the result in Equation 19 of Lemma 1 we get

Original: [Equation]
Revised: [Equation]

Original: Moreover, by using the triangle inequality we obtain equation 20.
Revised: Moreover, by using the triangle inequality we obtain equation 20.

Original: Given an arbitrary training set, the theoretical result in Lemma 1 indicates that our proposed method is β -robust against a noisy label triplet z (cid:48) .
Revised: Given an arbitrary training set, the theoretical result in Lemma 1 indicates that our proposed method is β -robust against a noisy label triplet z (cid:48) .

Original: However, in many real-life applications, the noise level is often much larger, that is, we have to deal with a number of noisy triplets.
Revised: However, in many real-life applications, the noise level is often much larger, that is, we have to deal with a number of noisy triplets.

Original: The theoretical result in Theorem 1, which is an extension of equation 19, shows the robustness of our proposed novel method given a particular noise level represented by N noisy triplets.
Revised: The theoretical result in Theorem 1, which is an extension of equation 19, shows the robustness of our proposed novel method given a particular noise level represented by N noisy triplets.

Revised: Moreover, it would be interesting to clarify that although the deﬁnition of the robustness of a metric learning algorithm had been dicussed, for example, in (Bellet & Habrard, 2015; Huai et al., 2019), these papers do not take noisy labels into account.

Original:  E XPERIMENTS AND R ESULTS
Revised:  E XPERIMENTS AND R ESULTS

Original: In this section, we quantitatively evaluate our proposed method based on the experiments conducted on several benchmark data sets that contain different types of label noise.
Revised: In this section, we quantitatively evaluate our proposed method based on the experiments conducted on several benchmark data sets that contain different types of label noise.

Original: In particular, our proposed method Bayesian deep metric learning (BDML) is compared with BLMNN and the deterministic triplet-based deep metric learning (DML) with respect to classiﬁcation performance on the data sets
Revised: In particular, our proposed method Bayesian deep metric learning (BDML) is compared with BLMNN and the deterministic triplet-based deep metric learning (DML) with respect to classiﬁcation performance on the data sets

Original: MNIST (LeCun et al., 1998) and CIFAR-10 (Krizhevsky et al., 2012) with synthetic label noise, and face retrieval results on MS-Celeb-1M (Guo et al., 2016) with realistic noisy labels.
Revised: MNIST (LeCun et al., 1998) and CIFAR-10 (Krizhevsky et al., 2012) with synthetic label noise, and face retrieval results on MS-Celeb-1M (Guo et al., 2016) with realistic noisy labels.

Original: In all our experiments, we adopt a simple network architecture for deep learning models with two fully-connected hidden layers of 512 -dimension and Tanh activation function between intermediate layers. 
Revised: In all our experiments, we adopt a simple network architecture for deep learning models with two fully-connected hidden layers of 512 -dimension and Tanh activation function between intermediate layers. Our

Original: Our proposed framework consists of the following steps: (i) extract features by an unsupervised method; (ii) apply PCA to project data points onto a smaller dimensional space; (iii) standardize features to standard Normal distribution, then use these features for comparing methods.
Revised: [Table] proposed framework consists of the following steps: (i) extract features by an unsupervised method; (ii) apply PCA to project data points onto a smaller dimensional space; (iii) standardize features to standard Normal distribution, then use these features for comparing methods.

Original: The values of these hyper-parameters were selected by using 10% of training data as a hold-out set for validation.
Revised: The values of these hyper-parameters were selected by using 10% of training data as a hold-out set for validation.

Original: Performance is calculated by 3-NN classiﬁer, each experiment is repeated ﬁve times, and the comparison results with respect to the top-1 classiﬁcation accuracy (mean ± stdev) as a function of the percentage of noisy labels in the training set are reported.[Table]
Revised: Performance is calculated by 3-NN classiﬁer, each experiment is repeated ﬁve times, and the comparison results with respect to the top-1 classiﬁcation accuracy (mean ± stdev) as a function of the percentage of noisy labels in the training set are reported.

Original: [Table] I MAGE C LASSIFICATION WITH S YNTHETIC L ABEL N OISE [Table] NOISE
Revised: 4. I MAGE C LASSIFICATION WITH S YNTHETIC L ABEL N OISE 4.1. S YMMETRIC NOISE

Original: The MNIST data set (LeCun et al., 1998) contains 28 × 28 black and white handwritten digits in 10 classes, (with 50000 training and 10000 testing samples).
Revised: The MNIST data set (LeCun et al., 1998) contains 28 × 28 black and white handwritten digits in 10 classes, (with 50000 training and 10000 testing samples).

Original: The new training data set of MNIST is generated by randomly stratiﬁed sub-sampling 5000 training data points from the original training data (i.e., 500 samples for each class), and the standard test set is unchanged – this enables a fair comparison between different methods.
Revised: The new training data set of MNIST is generated by randomly stratiﬁed sub-sampling 5000 training data points from the original training data (i.e., 500 samples for each class), and the standard test set is unchanged – this enables a fair comparison between different methods.

Original: The 300 -dimensional feature vector for each data point is extracted by using the unsupervised representation learning model CSVDDNet (Wang & Tan, 2018; 2016).
Revised: The 300 -dimensional feature vector for each data point is extracted by using the unsupervised representation learning model CSVDDNet (Wang & Tan, 2018; 2016).

Original: Uniform (or symmetric) label noise with corresponding level in the set { 0 , 10% , 20% , 30% } is then injected into the new training data set by ﬂipping the labels by the respective proportion of data points.
Revised: Uniform (or symmetric) label noise with corresponding level in the set { 0 , 10% , 20% , 30% } is then injected into the new training data set by ﬂipping the labels by the respective proportion of data points.

Original: Hyper-parameters settings: for the BDML model we run 50 epochs, set the learning rate to 0 .with decay rate 0 .
Revised: Hyper-parameters settings: for the BDML model we run 50 epochs, set the learning rate to 0 .with decay rate 0 .

Original: 5 at epochs 30 , prior distribution for weights and bias are Gaussian distributions with mean 0 and standard deviation 1 for weights, 2 for bias; batch size is set to 1024 ; for the
Revised: 5 at epochs 30 , prior distribution for weights and bias are Gaussian distributions with mean 0 and standard deviation 1 for weights, 2 for bias; batch size is set to 1024 ; for the

Original: DML model , we run 50 epochs, learning rate is set to 0 .
Revised: DML model , we run 50 epochs, learning rate is set to 0 .

Original: 00001 with decay rate 0 .
Revised: 00001 with decay rate 0 .

Original: 5 at epoch 30 , batch size is set to 128 .
Revised: 5 at epoch 30 , batch size is set to 128 .

Original: For the BLMNN model , we follow the hyper-parameter setting suggested by authors (Wang & Tan, 2018)
Revised: For the BLMNN model , we follow the hyper-parameter setting suggested by authors (Wang & Tan, 2018)

Original: Table 1 shows that in the case of clean data set, BLMNN, which employs a linear transformation, produces slightly better classiﬁcation performance than both DML and BDML.
Revised: Table 1 shows that in the case of clean data set, BLMNN, which employs a linear transformation, produces slightly better classiﬁcation performance than both DML and BDML.

Original: However, when the noise level increases, deep metric-based methods outperform linear model.
Revised: However, when the noise level increases, deep metric-based methods outperform linear model.

Original: In particular, in the case of 30% noise, our method outperforms BLMNN with a large margin ( 7 . 12% ), and in the most extreme case with 50% of noise, BDML produces better results than both BLMNN and DML.
Revised: In particular, in the case of 30% noise, our method outperforms BLMNN with a large margin ( 7 . 12% ), and in the most extreme case with 50% of noise, BDML produces better results than both BLMNN and DML.

Original: The CIFAR-10 data set (Krizhevsky et al., 2012) consists of 32 × 32 colour images (withtraining and 10000 testing samples, and 10 classes).
Revised: The CIFAR-10 data set (Krizhevsky et al., 2012) consists of 32 × 32 colour images (withtraining and 10000 testing samples, and 10 classes).

Original: The new training data set is generated by randomly sub-sampling 5000 data points from original training data (i.e., 500 samples for each class), while the standard test set remained.
Revised: The new training data set is generated by randomly sub-sampling 5000 data points from original training data (i.e., 500 samples for each class), while the standard test set remained.

Original: The 2048 -dimensional feature of each data point is extracted with the recently proposed Simple Framework for Contrastive Learning of Visual Representations (SimCLR) (Chen et al., 2020) for the feature extraction process.
Revised: The 2048 -dimensional feature of each data point is extracted with the recently proposed Simple Framework for Contrastive Learning of Visual Representations (SimCLR) (Chen et al., 2020) for the feature extraction process.

Original: Hyper-parameters settings: for the BDML model , we run 50 epochs, set learning rate to 0 .
Revised: Hyper-parameters settings: for the BDML model , we run 50 epochs, set learning rate to 0 .

Original: 0001 with decay rate 0 .
Revised: 0001 with decay rate 0 .

Original: 5 at epoch 30 , prior distribution for weights and bias are Gaussian distributions with mean 0 and standard deviation 1 for weights, 2 for bias.
Revised: 5 at epoch 30 , prior distribution for weights and bias are Gaussian distributions with mean 0 and standard deviation 1 for weights, 2 for bias.

Original: For DML model , we set learning rate to 0 .
Revised: For DML model , we set learning rate to 0 .

Original: 00001 with decay rate 0 .
Revised: 00001 with decay rate 0 .

Original: 5 at epoch 30 and 50 , batch size is set to 512 .
Revised: 5 at epoch 30 and 50 , batch size is set to 512 .

Original: For BLMNN model , we use the prior mean 0 .
Revised: For BLMNN model , we use the prior mean 0 .

Original: 00001 and variance 0 .
Revised: 00001 and variance 0 .

Original: We report the performance of three models with respect to different noise levels in the set { 0 , 10% , 20% , 30% , 50% } in Table 2.
Revised: We report the performance of three models with respect to different noise levels in the set { 0 , 10% , 20% , 30% , 50% } in Table 2.

Original: It is clear that the use of a deep metric learning gives better classiﬁcation results comparing with linear method, especially when the noise level increases.
Revised: It is clear that the use of a deep metric learning gives better classiﬁcation results comparing with linear method, especially when the noise level increases.

Original: To be more speciﬁc, the gap increase from 2 .
Revised: To be more speciﬁc, the gap increase from 2 .

Original: 67% at clean label to 13 .
Revised: 67% at clean label to 13 .

Original: 66% at 50% label noise.
Revised: 66% at 50% label noise.

Original: More importantly, the results in that table consistently show that our proposed BDML method, which adopt Bayesianinference, outperforms DML, which adopt MLE for parameter estimation, across different noise levels.
Revised: More importantly, the results in that table consistently show that our proposed BDML method, which adopt Bayesian

Original: [Table] [Table]
Revised: [Table] [Table]

Revised: inference, outperforms DML, which adopt MLE for parameter estimation, across different noise levels.

Original:  C LASS - CONDITIONAL L ABEL N OISE
Revised: 4.1. C LASS - CONDITIONAL L ABEL N OISE

Original: In this experiment [Table] we examine those methods above on 10 , 000 sub-sampled data set being stratiﬁed from the original CIFAR-10 data set (Krizhevsky et al., 2012).
Revised: In this experiment, we examine those methods above on 10 , 000 sub-sampled data set being stratiﬁed from the original CIFAR-10 data set (Krizhevsky et al., 2012).

Original: That is, the new data subset remains the class distributions of the original training data set.
Revised: That is, the new data subset remains the class distributions of the original training data set.

Original: The class-conditional label noise ranging from 10% to 60% , is randomly generated by ﬂipping the labels with corresponding proportions of data samples.
Revised: The class-conditional label noise ranging from 10% to 60% , is randomly generated by ﬂipping the labels with corresponding proportions of data samples.

Original: We keep the previous mentioned hyper-parameters settings for each model.
Revised: We keep the previous mentioned hyper-parameters settings for each model.

Original: The results reported in Table 3 show that our proposed method BDML consistently outperforms DML and BLMNN with large margins across different noise vectors.
Revised: The results reported in Table 3 show that our proposed method BDML consistently outperforms DML and BLMNN with large margins across different noise vectors.

Original: 4. F ACE R ETRIEVAL ON MS-C ELEB 1M WITH R EALISTIC N OISY L ABELS
Revised: 4. F ACE R ETRIEVAL ON MS-C ELEB 1M WITH R EALISTIC N OISY L ABELS

Original: MS-Celeb-1M (Guo et al., 2016) is a large scale data set (with 10 M images of 100 , 000 celebrities) that contains realistic label noise.
Revised: MS-Celeb-1M (Guo et al., 2016) is a large scale data set (with 10 M images of 100 , 000 celebrities) that contains realistic label noise.

Original: We adopt the same setting in (Wang & Tan, 2018) to form the new training set for this experiment.
Revised: We adopt the same setting in (Wang & Tan, 2018) to form the new training set for this experiment.

Original: In particular, we ﬁrst randomly select a subset of the original training with 100 K images of 1200 persons.
Revised: In particular, we ﬁrst randomly select a subset of the original training with 100 K images of 1200 persons.

Original: This subset is then split with ratio 9:1 into a training set that contains approximately 20% of label noise and a clean test set, respectively.
Revised: This subset is then split with ratio 9:1 into a training set that contains approximately 20% of label noise and a clean test set, respectively.

Original: The 4096 -dimensionality feature vector of each data point in this subset is extracted by using the VGGFace (Parkhi et al., 2015) model.
Revised: The 4096 -dimensionality feature vector of each data point in this subset is extracted by using the VGGFace (Parkhi et al., 2015) model.

Original: We run 80 epochs, for the BDML model learning rate is 0 .with decay rate 0 .
Revised: We run 80 epochs, for the BDML model learning rate is 0 .with decay rate 0 .

Original: 5 at epoch 25 , 45 , 55 , 65 , prior distribution for both weights and bias are Gaussian distributions with mean 0 and standard deviation 0 .
Revised: 5 at epoch 25 , 45 , 55 , 65 , prior distribution for both weights and bias are Gaussian distributions with mean 0 and standard deviation 0 .

Original: 5 ; batch size is set to 256 .
Revised: 5 ; batch size is set to 256 .

Original: For the DML model , we set learning rate to 0 .
Revised: For the DML model , we set learning rate to 0 .

Original: 00001 with decay rate 0 .
Revised: 00001 with decay rate 0 .

Original: 5 at epoch 10 , 20 , 40 , batch size is set to 128 .
Revised: 5 at epoch 10 , 20 , 40 , batch size is set to 128 .

Original: For the BLMNN model , we follow the hyper-parameter setting suggested by authors (Wang &
Revised: For the BLMNN model , we follow the hyper-parameter setting suggested by authors (Wang &

Original: Tan, 2018).
Revised: Tan, 2018).

Original: Table 4 shows the comparison using the mean of average precision (mAP) between our proposed BDML and other competing methods.
Revised: Table 4 shows the comparison using the mean of average precision (mAP) between our proposed BDML and other competing methods.

Original: The results in Table 4 consistently indicate the advantage of using deep metric in real world label noisy dataset.
Revised: The results in Table 4 consistently indicate the advantage of using deep metric in real world label noisy dataset.

Original: Both DML and BDML show their better retrieval performance compared to the linear distance metric.
Revised: Both DML and BDML show their better retrieval performance compared to the linear distance metric.

Original: Moreover, it is clear from Table 4 that our proposed method BDML is superior to the point estimation DML – this is evidence that the use of Bayesian inference is more robust than the point estimation when working with label noise.
Revised: Moreover, it is clear from Table 4 that our proposed method BDML is superior to the point estimation DML – this is evidence that the use of Bayesian inference is more robust than the point estimation when working with label noise.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: In this paper, we proposed a novel theoretically sound Bayesian deep metric learning method to handle the issue of learning a deep model with noisy labels.
Revised: In this paper, we proposed a novel theoretically sound Bayesian deep metric learning method to handle the issue of learning a deep model with noisy labels.

Original: To the best of our knowledge, this is the ﬁrst work that introduces Bayesian inference in deep metric learning to improve the generalization capacity of two existing methods: robust Bayesian linear metric learning and point estimation deep metric learning. 
Revised: To the best of our knowledge, this is the ﬁrst work that introduces Bayesian inference in deep metric learning to improve the generalization capacity of two existing methods: robust Bayesian linear metric learning and point estimation deep metric learning. It is also important to note that our main goal with this paper is the introduction of an innovative framework that can improve the performance of both baselines, including BLMNN [Table] and DML.

Revised: We are not aware of other SOTA methods in the scope of this paper.

Original: In the future, we plan to test our proposed method with more complicated base models (e.g., ResNet (He et al., 2016)) instead of the simple neural network to obtain better experimental results.
Revised: In the future, we plan to test our proposed method with more complicated base models (e g., ResNet (He et al., 2016)) instead of the simple neural network to obtain better experimental results.

Original: We also plan to extend our current work to make it robust under simulated label noise (Wang & Tan, 2018) which has been shown to be more challenging than the types of label noise mentioned in our paper.
Revised: We also plan to extend our current work to make it robust under simulated label noise (Wang & Tan, 2018) which has been shown to be more challenging than the types of label noise mentioned in our paper