Title: Bayesian Metric Learning for Uncertainty Quantification in Image Retrieval
Venue: NeurIPS 2023
Accepted: yes
Keywords: Laplace approximation

metric learning

uncertainty quantification

weight posterior

bayesian
Abstract: We propose a Bayesian encoder for metric learning. Rather than relying on neural amortization as done in prior works, we learn a distribution over the network weights with the Laplace Approximation. We first prove that the contrastive loss is a negative log-likelihood on the spherical space. We propose three methods that ensure a positive definite covariance matrix. Lastly, we present a novel decomposition of the Generalized Gauss-Newton approximation. Empirically, we show that our Laplacian Metric Learner (LAM) yields well-calibrated uncertainties, reliably detects out-of-distribution examples, and has state-of-the-art predictive performance.

Bayesian Metric Learning for Uncertainty Quantification in Image Retrieval

Abstract

We propose a Bayesian encoder for metric learning. Rather than relying on neural amortization as done in prior works, we learn a distribution over the network weights with the Laplace Approximation. We first prove that the contrastive loss is a negative log-likelihood on the spherical space. We propose three methods that ensure a positive definite covariance matrix. Lastly, we present a novel decomposition of the Generalized Gauss-Newton approximation. Empirically, we show that our Laplacian Metric Learner (LAM) yields well-calibrated uncertainties, reliably detects out-of-distribution examples, and has state-of-the-art predictive performance.

1 Introduction

Metric learning seeks data representations where similar observations are near and dissimilar ones are far. This elegantly allows for building retrieval systems with simple nearest-neighbor search. Such systems easily cope with a large number of classes, and new classes can organically be added without retraining. While these retrieval systems show impressive performance, they quickly, and with no raised alarms, deteriorate with out-of-distribution data [38]. In particular, in safety-critical applications, the lack of uncertainty estimation is a concern as retrieval errors may propagate unnoticed through the system, resulting in erroneous and possibly dangerous decisions.

We present the Laplacian Metric Learner (LAM) to estimate reliable uncertainties of image embeddings as demonstrated in Fig. 1. We learn a distribution over the network weights (weight posterior) from

0.0

0.5

1.0

Uncertainty

PFE

LAM

0.0

0.5

1.0

Uncertainty

0.0

0.5

1.0

Uncertainty

2

2

which we obtain a stochastic representation by embedding an image through sampled neural networks. This Bayesian formulation has multiple benefits, namely (1) robustness to out-of-distribution examples, (2) calibrated in-distribution uncertainties, and (3) a slight improvement in predictive performance.

More specifically, our method extends the Laplace Approximation [27] for metric learning. We present a probabilistic interpretation of the contrastive loss [15] which justifies that it can be interpreted as an unnormalized negative log-likelihood. We propose three solutions to ensure a positive semidefinite Hessian for the contrastive loss and present two approaches to compute the Generalized GaussNewton [12] approximation for ℓ 2 -normalized networks. Finally, we boost our method with the online training procedure from [29] to achieve state-of-the-art performance.

We are not the first to consider uncertainty quantification in image retrieval. Seminal works [38, 33] have addressed the lack of uncertainties in retrieval with amortized inference [14], where a neural network predicts a stochastic embedding. The issues with this approach are that (1) it requires strong assumptions on the distribution of the embedding, (2) the networks are often brittle and difficult to optimize, and (3) out-of-distribution detection relies on the network's capacity to extrapolate uncertainties. As neural networks extrapolate poorly [50], the resulting predicted uncertainties are unreliable for out-of-distribution data [10] and are thus, in practice, of limited value.

In contrast, our method does not assume any distribution on the stochastic embeddings, is simple to optimize, and does not rely on a neural network to extrapolate uncertainties. Instead, our weight posterior is derived from the curvature of the loss landscape and the uncertainties of the latent embeddings deduced (rather than learned) with sampling. Empirically, we show that this leads to reliable out-of-distribution performance and calibrated uncertainties in both controlled toy experiments and challenging real-world applications such as bird, face, and place recognition.

2 Related Work

Metric learning attempts to map data to an embedding space, where similar data are close together and dissimilar data are far apart. This is especially useful for retrieval tasks with many classes and few observations per class such as place recognition [46] and face recognition [37] or for tasks where classes are not well-defined, such as food tastes [48] or narratives in online discussions [4].

There exist many metric losses that optimize for a well-behaved embedding space. We refer to the excellent survey by [30] for an overview. We here focus on the contrastive loss [15]

which has shown state-of-the-art performance [30] and is one of the most commonly used metric losses. Here, f θ is a neural network parametrized by θ which maps from the observation space to the embedding space. The loss consists of two terms, one that attracts observations from the same class ( anchor x a and positive x p ), and one that repels observations from different classes ( anchor x a and negative x n ). The margin m ≥ 0 ensures that negatives are repelled sufficiently far. We will later present a probabilistic extension of the contrastive loss that allows us to learn stochastic, rather than deterministic, features in the embedding space.

Uncertainty in deep learning is studied across many domains to mitigate fatal accidents and allow for human intervention when neural networks make erroneous predictions. Current methods can be divided into methods that apply amortized optimization to train a neural network to predict the parameters of the output distribution, and methods that do not. The amortized methods, best known from the variational autoencoder (V AE) [19, 36], seem attractive at first as they can directly estimate the output distribution (without requiring sampling), but they suffer from mode collapse and are sensitive to out-of-distribution data due to the poor extrapolation capabilities of neural networks [31, 10]. 'Bayes by Backprop' [2] learns a distribution over parameters variationally but is often deemed too brittle for practical applications. Alternatives to amortized methods includes deep ensembles [23], stochastic weight averaging (SWAG) [28], Monte-Carlo dropout [13] and Laplace Approximation (LA) [24, 27] which all approximate the generally intractable weight posterior p ( θ |D ) of a neural network. We propose to extend LA to metric learning.

Laplace approximations (LA) can be applied for every loss function L that can be interpreted as an unnormalized log-posterior by performing a second-order Taylor expansion around a chosen weight

vector θ ∗ such that

where L ∗ is the loss evaluated in θ ∗ . Imposing the unnormalized log-posterior to be a second-order polynomial is equivalent to assuming the posterior to be Gaussian. If θ ∗ is a MAP estimate, the first-order term vanishes, and the second-order term can be interpreted as a precision matrix, the inverse of the covariance. Assuming θ ∗ is a MAP estimate, this second-order term is negative semidefinite for common (convex) supervised losses, such as the mean-squared error and cross-entropy. Recently, Daxberger et al. [8] demonstrated that post-hoc LA is scalable and produces well-behaved uncertainties for classification and regression. The Laplacian Autoencoder (LAE) [29] improves on the post-hoc LA with an online Monte Carlo EM training procedure to learn a well-behaved posterior. It demonstrates state-of-the-art uncertainty quantification for unsupervised representation learning.

Uncertainty in metric learning is not new [44], but the majority of recent methods apply amortized inference to predict distributions in the embedding space [47, 3, 5, 33, 38, 39, 41], making them sensitive to mode collapse and out-of-distribution data. Alternatives like deep ensembles [42] and MonteCarlo dropout [43] suffer from increased training time, poor empirical performance, and limited Bayesian interpretation [8]. We explore LA in metric learning and attain state-of-the-art performance.

z

f

θ

θ

(

µ

θ

, σ

2

θ

)

x

∼ N

3 Laplacian Metric Learning

To perform Bayesian retrieval, we estimate the weight posterior of the embedding network f θ such that we can sample data embeddings to propagate uncertainty through the decision process. The embedding network is parametrized by θ ∈ Θ and trained with the contrastive loss. The network maps an image x ∈ X := R HWC to an embedding z ∈ Z , which is restricted to be on a Z -dimensional unit sphere Z := S Z . This spherical normalization is often done in retrieval to obtain faster retrieval and a slight performance boost [1, 35]. Fig. 2 illustrate our Bayesian mapping from image to latent space.

Laplace Approximation (LA) recap. We apply LA to obtain the posterior over the weights θ . LA comes in two flavors: (1) The post-hoc LA is found by first training a standard deterministic network through gradient steps with the contrastive loss to find the maximum a posteriori (MAP) parameters θ ∗ . Since we are in a local optimum, the first-order term in the second-order Taylor expansion (Eq. 2) vanishes, and we can define the parameter distribution as

The advantage of post-hoc LA is that the training procedure does not change, and already trained neural networks can be made Bayesian. In practice, however, we empirically observe post-hoc LA to be unstable. We refer to the large standard deviation in Table 1, Table 2, and Table 3 over random seeds. The instability stems from curvature differences between the local minimas through stochastic optimization. (2) Online LA [29] improves on this instability by marginalizing the LA during training with Monte Carlo EM. This helps the training recover a solution θ ∗ where the Hessian reflects the loss landscape. Specifically, at each step t during training, we keep in memory a Gaussian distribution on the parameters q t ( θ ) = N ( θ | θ t , H -1 θ t ) . The parameters are updated through an expected gradient step

and a discounted Laplace update

where α describes an exponential moving average, similar to momentum-like training. The initialization follows the isotropic prior q 0 ( θ ) = N ( θ | 0 , σ 2 prior I ) .

<latexi sh

1_b

64="fLBAq

QNH

9

MK

X

c

0

>

V

S8

EJ3

W

O

ok

d

C/Y

2

7

uR

y

v

+p

j

z

w

G

Z

g

I

n

P

D

U

m

T

F

r

5

<latexi sh

1_b

64="Ef

HyR8WpQ

c

Y

w

Z9nIL

Mo

>A

C

X

VBNS

3

2q

v

FKUm

0

k

J

/

P

j

O

D

5g

d

u

r

T

G

+

z

7

<latexi sh

1_b

64="pd gwmCL

3O9y

BoN2

8E

+

Y

>A

7

c

V

S

J

Ur q/

Iv

H

WMF

K

T

QXD

P

j

k

u

0

Z

G

5

n

R

z

f

<latexi sh

1_b

64="jS9JgzD

Tq

+w

75

cQXR

E

o

>A

B

H

V

N

8

3

W

O

K

k

d

C/Y

2

M

0

v

p

G

Z

I

nf

P

y

U

m

F

r

The probabilistic contrastive likelihood. LA expects the loss function to be a log-posterior, that is log-likelihood plus log-prior. A simple choice of the log-prior is ∥ θ ∥ 2 2 (weight decay). Further, we find that the contrastive loss Eq. 1 has a probabilistic interpretation and is a negative log-likelihood on the spherical space. We define the attractive term P GLYPH<1>GLYPH<0> ( z | x, θ ) ∼ N S ( z | f θ ( x ) , κ ) and the repelling term P GLYPH<0>GLYPH<1> ( z | x, θ ) ∼ N S ( z | -f θ ( x ) , κ ) as Von Mises-Fisher distributions on the latent space spherical space, where the concentration parameter κ ≥ 0 . The product of these two likelihoods yields a contrastive likelihood, which is valid on the spherical space, and its negative log-likelihood is equivalent to the contrastive loss (proof and details in ?? ).

The intuition, helped by electrostatics, is that the log probability density of these V on Mises-Fisher distributions (i.e. spherical Gaussians) plays the role of a potential energy. The precision parameter κ controls the concentration of the potential energy and, consequently, the strength of the associated force. If κ = 0 the distribution is uniform, the potential is constant and the associated force is zero. On the spherical space, moreover, a repulsive term from some point is equivalent to an attractive term from the antipodal point, which in turn is mathematically equivalent (see ?? ) to a negative precision κ . This is the fundamental reason behind the non positive-definiteness of the hessian.

Hessian of the contrastive loss. Both post-hoc and online LA require the Hessian of the contrastive loss ∇ 2 θ L con ( θ ; D ) . The Hessian is commonly approximated with the Generalized Gauss-Newton (GGN) approximation [12, 8, 6, 11]. The GGN decomposes the loss into L = g ◦ f , where g is usually chosen as the loss function and f the model function, and only f is linearized [22].

However, in our case, this decomposition is non-trivial. Recall that the last layer of our network is an ℓ 2 normalization layer, which projects embeddings onto a hyper-sphere. This normalization layer can either be viewed as part of the model f (linearized normalization layer) or part of the loss g (non-linearized normalization layer). The former can be interpreted as using the Euclidean distance and the latter as using the Arccos distance for the contrastive loss (see ?? and ?? ). These two share the zero- and first-order terms for normalized embeddings but not the second-order derivatives due to the GGN linearization. The Euclidean interpretation leads to simpler derivatives and interpretations, and we will therefore use it for our derivations. We emphasize that the Arccos is theoretically a more accurate approximation, because the ℓ 2 -layer is not linearized, and we provide derivations in ?? .

The GGN matrix for contrastive loss with the Euclidean interpretation is given by

where J ij θ = ( J θ f θ ( x i ) ⊤ , J θ f θ ( x j ) ⊤ ) ⊤ , with J θ is the Jacobian wrt. the parameters and H p and H n are the Hessians of the contrastive loss wrt. the model output for positive and negative pairs. The first sum runs over positive pairs and the second sum runs over negative pairs within the margin . Negative pairs outside the margin do not contribute to the Hessian, and can therefore be ignored to reduce the computational load ( ?? ).

The eigenvalues of the Hessian wrt. to the output are (0 , 2) and ( -2 , 0) for the positive H p and negative H n terms, so we are not guaranteed to have a positive semidefinite Hessian, H θ . To avoid covariances with negative eigenvalues, we propose three solutions to ensure a positive semidefinite Hessian. Proofs are in ?? .

Ensuring positive definiteness of the covariance matrix . We do not want to be restricted in the choice of the prior except to have non-zero precision, so we must ensure that ∇ 2 θ L con ( θ ∗ ; D ) is positive semidefinite. Differently from the standard convex losses, this is not ensured by the GGN approximation [18]. Our main insight is that we can ensure a positive semidefinite Hessian H θ by only manipulating the Hessians H p and H n in Eq. 6.

1. Positive: The repelling term is ignored, such that only positive pairs contribute to the Hessian.

Fixed: The cross derivatives are ignored.

3. Full: Positive semidefiniteness is ensured with ReLU, max(0 , ∇ 2 θ L con ( θ ; D )) , on the Hessian of the loss wrt. the parameters.

The positive approximation is inspired by [38], which only uses positive pairs to train an uncertainty module. The gradient arrows in Fig. 3a illustrate that negative pairs are neglected when computing the Hessian of the contrastive loss. The fixed approximation considers one data point at a time, assuming the other one is fixed. Thus, given a pair of data points, this can be interpreted as first moving one data point, and then the second (rather than both at the same time). We formalize this in ?? . Fig. 3b illustrate this idea when all points except a are fixed. Lastly, we propose the full Hessian of the contrastive loss (Fig. 3c) and ensure positive semidefiniteness by computing the ReLU of the diagonal Hessian. This approximation can equivalently be interpreted as a projection into the space of psd matrixes. In practice, the Hessian scales quadratically in memory wrt. the number of parameters. To mitigate this, we approximate this Hessian by its diagonal and only apply the LA on the last layer [25, 9]. We experimentally find that the fixed approximation yields the best performance (Section 4).

Hard negative mining. Most pairs, namely the negatives outside the margin, do not contribute to the Hessian, so it is wasteful to compute their Hessian. Therefore, we use hard negative mining [30] to only compute the Hessian of pairs that have non-zero Hessian, i.e. the negative sample lie within the margin (illustrated with the dotted line in Fig. 3).

Von Mises-Fisher distribution. To obtain a single measure of uncertainty from our sampled image embeddings (red dots in Fig. 4), we fit a von Mises-Fisher distribu-

m

p

p

a

n

n

n

m

p

p

a

n

n

m

p

p

a

n

(

a

)

positives

(

b

)

fixed

(

c

)

full

tion. The von Mises-Fisher distribution describes a normal distribution where all probability mass lies on a Z -dimensional hyper-sphere. It is parametrized with a directional mean µ and a scalar concentration parameter κ , which can be interpreted as the inverse of an isotropic covariance κ = 1 /σ 2 , i.e. , small κ means high uncertainty and large κ means low uncertainty. There exist several methods to estimate κ . We opt for the simplest and most computationally efficient [40] (see ?? ).

4 Experiments

We benchmark our method against strong probabilistic retrieval models. Probabilistic Face Embeddings (PFE) [38] and Hedge Image Embedding (HIB) [33] perform amortized inference and thus estimate the mean and variance of latent observation. We also compare against MC Dropout [13] and Deep Ensemble [23], two approximate Bayesian methods, which have successfully been applied in image retrieval [43, 42].

We compare the models' predictive performance with the recall (recall@ k ) and mean average precision (mAP@ k ) among the k nearest neighbors [47, 30, 1]. We evaluate the models' abilities to interpolate and extrapolate uncertainties by measuring the Area Under the Sparsification Curve (AUSC), Expected Calibration Error (ECE) on in-distribution (ID) data, the Area Under Receiver Operator Curve (AUROC), and Area Under Precision-Recall Curve (AUPRC) on out-of-distribution (OOD) data. We provide more details on these metrics in ?? .

We extend StochMan [11] with the Hessian backpropagation for the contrastive loss, and the PyTorch [34] code is publicly available 2 . ?? details the experimental setup.

<latexi sh

1_b

64="Oy

3

FHP

Y/

fCwQd0En

NX

>A

B

c

V

S8

Ur

q

oj

W

vZR

D

+g

2

L

5u

m

K

G

z

p

J

T

9

M

I

k

7

<latexi sh

1_b

64="/XG

W

mcSnvQ

k

q7

P

VL

w

>A

B

H

N

8

EJ3Ur

9

0

R

o

y2

Z

I

Cj

z

M

Y

u

+O

d

gf

D

T

p

5

K

F

<latexi sh

1_b

64="AV9

JW

k

QK

r

w

Hyzc

>

B

NS8

E

3U

q/

L

0

PR

7

o

2

I

Cj

Z

X

M

Y

u

+O

d

gf nG

D

m

F

p

v

5

T

<latexi sh

1_b

64="

k

IgqfV

FGDOMmK

X

Tp/5

u

>A

B

H

c

NS8

EJ3Ur

9L

0

WPR

7

o

y2

Z

Q

Cj

z

Y

+

d

n

w

v

<latexi sh

1_b

64="F

S

RATZ

WwD

vBN80

C5UjP

>

H

c

V

EJ3

r

q/

9L

7

o

y2k

I

Q

zX

M

Y

u

+O

d

gf nG

p

mK

<latexi sh

1_b

64="F

S

RATZ

WwD

vBN80

C5UjP

>

H

c

V

EJ3

r

q/

9L

7

o

y2k

I

Q

zX

M

Y

u

+O

d

gf nG

p

mK

<latexi sh

1_b

64="/XG

W

mcSnvQ

k

q7

P

VL

w

>A

B

H

N

8

EJ3Ur

9

0

R

o

y2

Z

I

Cj

z

M

Y

u

+O

d

gf

D

T

p

5

K

F

<latexi sh

1_b

64="/XG

W

mcSnvQ

k

q7

P

VL

w

>A

B

H

N

8

EJ3Ur

9

0

R

o

y2

Z

I

Cj

z

M

Y

u

+O

d

gf

D

T

p

5

K

F

<latexi sh

1_b

64="AV9

JW

k

QK

r

w

Hyzc

>

B

NS8

E

3U

q/

L

0

PR

7

o

2

I

Cj

Z

X

M

Y

u

+O

d

gf nG

D

m

F

p

v

5

T

<latexi sh

1_b

64="yUjm

ZP2MH

uKd

z

/nWGo0

>A

c

VDJSgN

E

LR

Q

CvI

f

Y9O

k

T

p

F

q

8

r

5

7

3

X

w

<latexi sh

1_b

64="

k

IgqfV

FGDOMmK

X

Tp/5

u

>A

B

H

c

NS8

EJ3Ur

9L

0

WPR

7

o

y2

Z

Q

Cj

z

Y

+

d

n

w

v

<latexi sh

1_b

64="F

S

RATZ

WwD

vBN80

C5UjP

>

H

c

V

EJ3

r

q/

9L

7

o

y2k

I

Q

zX

M

Y

u

+O

d

gf nG

p

mK

<latexi sh

1_b

64="F

S

RATZ

WwD

vBN80

C5UjP

>

H

c

V

EJ3

r

q/

9L

7

o

y2k

I

Q

zX

M

Y

u

+O

d

gf nG

p

mK

<latexi sh

1_b

64="/XG

W

mcSnvQ

k

q7

P

VL

w

>A

B

H

N

8

EJ3Ur

9

0

R

o

y2

Z

I

Cj

z

M

Y

u

+O

d

gf

D

T

p

5

K

F

<latexi sh

1_b

64="/XG

W

mcSnvQ

k

q7

P

VL

w

>A

B

H

N

8

EJ3Ur

9

0

R

o

y2

Z

I

Cj

z

M

Y

u

+O

d

gf

D

T

p

5

K

F

<latexi sh

1_b

64="AV9

JW

k

QK

r

w

Hyzc

>

B

NS8

E

3U

q/

L

0

PR

7

o

2

I

Cj

Z

X

M

Y

u

+O

d

gf nG

D

m

F

p

v

5

T

<latexi sh

1_b

64="ADu dWU5K

jZH

Lv2

F0g

k

I

>

B9X

c

V

S

N

EOz

G

Mr

Y

J

y

o

8R

mQ

p

w

P

3

q

f

C

+

n

T

<latexi sh

1_b

64="

k

IgqfV

FGDOMmK

X

Tp/5

u

>A

B

H

c

NS8

EJ3Ur

9L

0

WPR

7

o

y2

Z

Q

Cj

z

Y

+

d

n

w

v

<latexi sh

1_b

64="F

S

RATZ

WwD

vBN80

C5UjP

>

H

c

V

EJ3

r

q/

9L

7

o

y2k

I

Q

zX

M

Y

u

+O

d

gf nG

p

mK

<latexi sh

1_b

64="F

S

RATZ

WwD

vBN80

C5UjP

>

H

c

V

EJ3

r

q/

9L

7

o

y2k

I

Q

zX

M

Y

u

+O

d

gf nG

p

mK

<latexi sh

1_b

64="/XG

W

mcSnvQ

k

q7

P

VL

w

>A

B

H

N

8

EJ3Ur

9

0

R

o

y2

Z

I

Cj

z

M

Y

u

+O

d

gf

D

T

p

5

K

F

0.0

0.5

1.0

AUROC

FashionMNIST

0.0

0.5

AUROC

CIFAR

0.0

0.5

AUROC

CUB200

0.0

0.5

AUROC

LFW

(a) Out-of-distribution detection.

0.0

0.5

mAP@5

FashionMNIST

0.00

0.25

0.50

mAP@5

CIFAR

0.0

0.2

0.4

mAP@5

CUB200

0.00

0.25

0.50

mAP@5

LFW

(b) Predictive performance.

Experimental Summary. We first summarize our experimental results. Across five datasets, three network architectures, and three different sizes of the latent space (ranging from 3 to 2048 ), we find that LAM has well-calibrated uncertainties, reliably detects OOD examples, and achieves state-of-theart predictive performance. Fig. 4a shows that the uncertainties from online LAM reliably identify OOD examples. Online LAM outperforms other Bayesian methods, such as post-hoc LAM and MC dropout, on this task, which in turn clearly improves upon amortized methods that rely on a neural network to extrapolate uncertainties. Fig. 4b shows that LAM consistently matches or outperforms existing image retrieval methods in terms of predictive performance. We find that the fixed Hessian approximation with the Arccos distance performs the best, especially on higher dimensional data.

Ablation: Positive definiteness covariance matrix. We experimentally study which method to ensure a positive semidefinite Hessian has the best performance measured in both predictive performance (mAP@ 5 ) and uncertainty quantification (AUROC, AUSC). We found that all methods perform similarly on simple datasets and low dimensional hyper-spheres, but the fixed approximation with Arccos distance performs better on more challenging datasets and higher dimensional hyper-spheres. We present results on one of these more challenging datasets, namely the LFW [17] face recognition dataset with the CUB200 [45] bird dataset as an OOD dataset. We use a ResNet50 [16] with a GeM pooling layer [35] and a 2048 dimensional embedding and diagonal, last-layer LA [8].

Table 1 shows the performance for post-hoc and online LA with fixed, positive, or full Hessian approximation using either Euclidean or Arccos distance. Across all metrics, the online LA with Arccos distance and the fixed Hessian approximation performs similarly or the best. We proceed to benchmark this method against sev-

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

0.0

0.2

0.4

0.6

0.8

1.0

PFE

LAM (posthoc)

LAM (online)

Best fit

0.0

0.5

(a) FashionMNIST

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.5

(b) CIFAR10

±

±

eral strong probabilistic baselines on closed-set retrieval and a more challenging open-set retrieval.

Closed-Set Retrieval. OOD capabilities are critical for identifying distributional shifts, outliers, and irregular user inputs, which might hinder the propagation of erroneous decisions in an automated system. We evaluate OOD performance on the commonly used benchmarks [31], where we use (1)

±

FashionMNIST [49] as ID and MNIST [26] as OOD, and (2) CIFAR10 [21] as ID and SVHN [32] as OOD. We use, respectively, a standard 2 - or 3 -layer relu convolutional network followed by a single linear layer on which we compute LA with a diagonal Hessian.

FashionMNIST (ID) vs MNIST ( OOD ). Table 2 shows that both PFE and posthoc LAM have a similar predictive performance to the deterministic model. This is not surprising, as both methods are initialized with the deterministic parameters, and then uncertainties are learned (PFE) or deduced (post-hoc LAM) with frozen weights. The awareness of uncertainties during training, grants the online LAM slightly higher predictive performance.

0.00

0.25

0.50

0.75

1.00

TPR

0.6

0.8

1.0

FPR

PFE

LAM (posthoc)

LAM (online)

(a) FashionMNIST

0.00

0.25

0.50

0.75

1.00

TPR

0.4

0.6

0.8

1.0

(b) CIFAR10

PFE uses amortized inference to predict variances. This works reasonably within distribution but does not work well for OOD detection. This is because a neural network is trusted to extrapolate far away from the data distribution. Table 2 shows that MC dropout, LAM (post-hoc), and LAM (online) assign high uncertainty to observations outside the training distribution. Fig. 5 shows that both post-hoc and online LAM are near perfectly calibrated, giving very low ECE measures (Table 2).

CIFAR10 (ID) vs SVHN ( OOD ) is a slightly harder setting. Table 2 yields similar conclusions as before; Bayesian approaches such as MC dropout, LAM (post-hoc), and LAM (online) better detect OOD examples than neural amortized methods such as PFE. Online LAM has a similar predictive performance to state-of-the-art while having better ID (lower ECE and higher AUSC) and OOD (higher AUROC and AUPRC) performance. Fig. 5 shows the calibration plot for CIFAR10, where online LAM has near-perfect calibration. The CIFAR10 ROC curves (Fig. 6) show that online LAM is better at distinguishing ID and OOD examples.

Open-Set Retrieval. A key advantage of metric learning methods is that they easily cope with a large number of classes and new classes can be added seamlessly. We therefore evaluate LAM's performance on challenging open-set retrieval, where none of the classes in the test set are available during training. We first test with CUB200 [45] as ID and CAR196 [20] as OOD similarly to Warburg et al. [47], and second, test with LFW [17] as ID and CUB200 as OOD. We use a ResNet50 [16] with a GeM pooling layer [35] and a 2048 dimensional embedding and diagonal, last-layer LA [8].

CUB200 (ID) vs CARS196 ( OOD ). The CUB-200-2011 dataset [45] has 200 bird species captured from different perspectives and in different environments. We follow the zero-shot train/test split [30]. In this zero-shot setting, the trained models have not seen any of the bird species in the test set, and the learned features must generalize well across species. Table 3 shows that LAM matches or surpasses the predictive performance of all other methods. LAM (post-hoc) achieves state-of-the-art

±

±

±

±

±

±

PFE

. . .

LAM (p)

. . .

LAM (o)

. . .

Lower uncertainty

Higher uncertainty

SIN

VO

HE

AG

PO

TON

predictive performance, while LAM (online) matches the predictive performance of the deterministic trained model while achieving state-of-the-art AUROC and AUPRC for OOD detection.

LFW (ID) vs CUB200 ( OOD ). Face recognition is another challenging metric learning task with many applications in security and surveillance. The goal is to retrieve images of the same person as in the query image. Table 3 shows that online LAM outperforms existing methods both in terms of predictive performance and uncertainty quantification. Fig. 7 shows that PFE assigns higher uncertainty to images from the ID dataset (faces) than those from the OOD dataset (birds).

In contrast, both online and post-hoc LAM better associate high variance to OOD examples, while PFE predicts high variance to ID examples. Furthermore, online LAM seems to assign the highest variance to images in which the background is complex and thus camouflages the birds.

Visual Place recognition is important for the longterm operation of autonomous robots [7], where the goal is to retrieve images taken within a radius of 25 meters from a query image. The high number of unique places and varying visual appearance of each location - including weather, dynamic, structural, viewpoint, seasonal, and day/night changes -makes visual place recognition a challenging metric learning

0

25

50

75

100

Filter Out Rate

0.6

0.7

0.8

mAP@1

PFE

MC Dropout

LAM Posthoc

LAM Online

PFE

. . .

LAM (p)

. . .

LAM (o)

. . .

Lower uncertainty

Higher uncertainty

problem. Reliable uncertainties and reliable out-of-distribution behavior are important to avoid incorrect loop-closure, which can deteriorate the autonomous robots' location estimate. We evaluate on MSLS [46], which is the largest and most diverse place recognition dataset currently available comprised of 1 . 6 M images from 30 cities spanning six continents. We use the standard train/test split, training on 24 cities and testing on six other cities. We use the same model as in open-set retrieval. Margin

Table 5 shows that online LAM yields state-of-the-art uncertainties for visual place recognition measured with AUSC, while matching the predictive performance of the alternative probabilistic and deterministic methods on both the MSLS validation and the challenge set. Fig. 8 shows the sparsification curves on the challenge set. Both online and post-hoc LAM have monotonically increasing sparsification curves, implying that when we remove the most uncertain observations, the predictive performance increase. This illustrates that LAM produces reliable uncertainties for this challenging open-set retrieval task. Fig. 9 shows the queries associated with the highest and lowest uncertainty. LAM predicts high uncertainty to images with are blurry, captured facing into the pavement, or contain mostly vegetation. These images do not have features that are descriptive of a specific place, making them hard to geographically locate.

Ablations. We conduct ablations on LFW for online and post-hoc LAM. Table 4 shows that the predictive performance does not depend on the margin, however, OOD performance decreases significantly for larger margins. We find that LAM is robust to the number of pairs sampled per batch. We find that a larger latent space results in better-calibrated uncertainties. Online LAM is rather robust to the choice of memory factor α . However, choosing a memory factor in the 0.0001-0.001 range gives both good AUROC and AUSC. This memory factor α can be interpreted as a momentum-like parameter that relates to the learning rate and indicates how fast we should update the current Hessian [29].

Lastly, we perform cold-posterior tempering of post-hoc LA by scaling the log-likelihood hessian with a factor β > 0 , such that the precision in Eq. 3 became β · ∇ 2 θ L con ( θ ∗ ; D ) + σ -2 prior I . We do not see any benefits from changing to other values than one.

Limitations. Similar to other Bayesian methods, LAM relies on n samples to obtain uncertainties. This makes inferences n times slower. Computing the Hessian at every step during online LAM also makes training time slower. To combat long training times and high memory usage, we use a last-layer LAM and thus only estimate and sample for a weight posterior of the last layer. The last-layer LAM training time is 3 hours for online LAM vs 2 . 3 hours for deterministic contrastive loss on LFW, and 30 minutes vs 15 minutes loss on CUB200 on an NVIDIA RTX A5000.

5 Conclusion

In this paper, we have introduced a Bayesian encoder for metric learning, the Laplacian Metric Learner (LAM), which uses the Laplace approximation. We prove that the contrastive loss is unnormalized negative log-likelihood on the spherical space, and develop three Hessian approximations, which ensures a positive definite covariance matrix. We propose a novel decomposition of the Generalized Gauss-Newton approximation that improves Hessian approximations of ℓ 2 -normalized networks. Empirically, we demonstrate that LAM consistently produces well-calibrated uncertainties, reliably detects out-of-distribution examples, and achieves state-of-the-art predictive performance on both closed-set and challenging open-set image retrieval tasks.

Acknowledgement. This work was supported by research grants (15334, 42062) from VILLUM FONDEN. This project received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement 757360). The work was partly funded by the Novo Nordisk Foundation through the Center for Basic Machine Learning Research in Life Science (NNF20OC0062606).

References

Relja Arandjelovi´ c, Petr Gronat, Akihiko Torii, Tomas Pajdla, and Josef Sivic. Netvlad: CNN architecture for weakly supervised place recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 5297-5307, 2016.

Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks, 2015.

Jie Chang, Zhonghao Lan, Changmao Cheng, and Yichen Wei. Data uncertainty learning in face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 5710-5719, 2020.

Peter Ebert Christensen, Frederik Warburg, Menglin Jia, and Serge Belongie. Searching for structure in unfalsifiable claims. arXiv preprint arXiv:2209.00495 , 2022.

Sanghyuk Chun, Seong Joon Oh, Rafael Sampaio De Rezende, Yannis Kalantidis, and Diane Larlus. Probabilistic embeddings for cross-modal retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 8415-8424, 2021.

Felix Dangel, Frederik Kunstner, and Philipp Hennig. BackPACK: Packing more into backprop. In International Conference on Learning Representations (ICLR) , 2020.

Andrew J. Davison, Ian D. Reid, Nicholas D. Molton, and Olivier Stasse. Monoslam: Real-time single camera slam. IEEE Transactions on Pattern Analysis and Machine Intelligence , 29(6): 1052-1067, 2007. doi: 10.1109/TPAMI.2007.1049.

Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. Laplace redux-effortless Bayesian deep learning. In NeurIPS , 2021.

John Denker and Yann LeCun. Transforming neural-net output levels to probability distributions. Advances in Neural Information Processing Systems , 1990.

Nicki S Detlefsen, Martin Jørgensen, and Søren Hauberg. Reliable training and estimation of variance networks. In Advances in Neural Information Processing Systems , 2019.

Nicki S. Detlefsen, Alison Pouplin, Cilie W. Feldager, Cong Geng, Dimitris Kalatzis, Helene Hauschultz, Miguel González Duque, Frederik Warburg, Marco Miani, and Søren Hauberg. StochMan, 2021. URL https://github.com/MachineLearningLifeScience/ stochman/ .

F Dan Foresee and Martin T Hagan. Gauss-Newton approximation to Bayesian learning. In Proceedings of International Conference on Neural Networks (ICNN'97) , volume 3, pages 1930-1935. IEEE, 1997.

Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning , pages 1050-1059. PMLR, 2016.

Samuel Gershman and Noah Goodman. Amortized inference in probabilistic reasoning. In Proceedings of the annual meeting of the cognitive science society , volume 36, 2014.

Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06) , volume 2, pages 1735-1742. IEEE, 2006.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016.

Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst, October 2007.

Alexander Immer, Maciej Korzepa, and Matthias Bauer. Improving predictions of Bayesian neural nets via local linearization. In International Conference on Artificial Intelligence and Statistics , pages 703-711. PMLR, 2021.

Diederik P Kingma and Max Welling. Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 , 2013.

J. Krause, M. Stark, J. Deng, and L. Fei-Fei. 3d object representations for fine-grained categorization. In 2013 IEEE International Conference on Computer Vision Workshops , pages 554-561, 2013.

Alex Krizhevsky. Learning multiple layers of features from tiny images, 2009.

Frederik Kunstner, Philipp Hennig, and Lukas Balles. Limitations of the empirical fisher approximation for natural gradient descent. Advances in neural information processing systems , 32, 2019.

Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in Neural Information Processing Systems , 30, 2017.

Pierre Simon Laplace. Mémoire sur la probabilité des causes par les événements. Mémoire de l'Académie Royale des Sciences , 1774.

Yann LeCun, John Denker, and Sara Solla. Optimal brain damage. Advances in Neural Information Processing Systems , 1989.

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):2278-2324, 1998.

David J. C. MacKay. A practical Bayesian framework for backpropagation networks. Neural Computation , 4(3):448-472, 1992.

Wesley J Maddox, Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson. A simple baseline for Bayesian uncertainty in deep learning. Advances in Neural Information Processing Systems , 32, 2019.

Marco Miani, Frederik Warburg, Pablo Moreno-Muñoz, Nicke Skafte Detlefsen, and Søren Hauberg. Laplacian autoencoders for learning stochastic representations. In Advances in Neural Information Processing Systems , 2022.

Kevin Musgrave, Serge Belongie, and Ser-Nam Lim. A metric learning reality check. In European Conference on Computer Vision , pages 681-699. Springer, 2020.

Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't know? arXiv preprint arXiv:1810.09136 , 2018.

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning, 2011.

Seong Joon Oh, Kevin Murphy, Jiyan Pan, Joseph Roth, Florian Schroff, and Andrew Gallagher. Modeling uncertainty with hedged instance embedding. arXiv preprint arXiv:1810.00319 , 2018.

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in PyTorch. In NIPS-W , 2017.

Filip Radenovi´ c, Giorgos Tolias, and Ondˇ rej Chum. Fine-tuning CNN image retrieval with no human annotation. IEEE transactions on pattern analysis and machine intelligence , 41(7): 1655-1668, 2018.

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning , pages 1278-1286. PMLR, 2014.

Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 815-823, 2015.

Yichun Shi and Anil K Jain. Probabilistic face embeddings. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 6902-6911, 2019.

Yale Song and Mohammad Soleymani. Polysemous visual-semantic embedding for crossmodal retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1979-1988, 2019.

Suvrit Sra. A short note on parameter approximation for von Mises-Fisher distributions: and a fast implementation of i s (x). Computational Statistics , 27(1):177-190, 2012.

Jennifer J Sun, Jiaping Zhao, Liang-Chieh Chen, Florian Schroff, Hartwig Adam, and Ting Liu. View-invariant probabilistic embedding for human pose. In European Conference on Computer Vision , pages 53-70. Springer, 2020.

Ahmed Taha, Yi-Ting Chen, Teruhisa Misu, Abhinav Shrivastava, and Larry Davis. Unsupervised data uncertainty learning in visual retrieval systems. arXiv preprint arXiv:1902.02586 , 2019.

Ahmed Taha, Yi-Ting Chen, Xitong Yang, Teruhisa Misu, and Larry Davis. Exploring uncertainty in conditional multi-modal retrieval systems. arXiv preprint arXiv:1901.07702 , 2019.

Luke Vilnis and Andrew McCallum. Word representations via Gaussian embedding. arXiv preprint arXiv:1412.6623 , 2014.

Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. CaltechUCSD birds 200, 2011.

Frederik Warburg, Søren Hauberg, Manuel Lopez-Antequera, Pau Gargallo, Yubin Kuang, and Javier Civera. Mapillary street-level sequences: A dataset for lifelong place recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2020.

Frederik Warburg, Martin Jørgensen, Javier Civera, and Søren Hauberg. Bayesian triplet loss: Uncertainty quantification in image retrieval. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 12158-12168, 2021.

Michael Wilber, Iljung S Kwak, David Kriegman, and Serge Belongie. Learning concept embeddings with combined human-machine expertise. In Proceedings of the IEEE International Conference on Computer Vision , pages 981-989, 2015.

Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms, 2017.

Keyulu Xu, Mozhi Zhang, Jingling Li, Simon S. Du, Ken ichi Kawarabayashi, and Stefanie Jegelka. How neural networks extrapolate: From feedforward to graph neural networks, 2021.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a Laplace approximation-based probabilistic retrieval approach (aka. Bayesian metric learning for image retrieval). The author provides a probabilistic view of the contrastive loss based on the von-Mises Fisher distribution and corrections for the Hessian positive definiteness. Extensive experimental evaluations are performed, demonstrating the advantages of the presented approach in calibrated uncertainties, out-of-distribution detection, close-set and open-set retrieval, and ablations on a few parameters of design choices.\n\nSTRENGTHS:\n[Originality] This paper presents the use of Laplace approximation instead of amortized inference for probabilistic retrieval. Although similar ideas have been proposed to improve the amortization gap of variational autoencoders in several prior works, its development in the context of metric learning is novel. This includes the introduction of a probabilistic view of the contrastive loss and the utilization of the Hessian approximation based on GCN.\n\nOn the other hand, previous Bayesian metric learning approaches have also been proposed, such as hedge instance embedding (HIE) and probabilistic face embeddings (PFE). In this regard, the novelty of this work lies in the contrastive learning-based loss and the learning of stochastic embeddings based on the Laplacian autoencoder (Miani, M. et al., 2022).\n\nMiani, M., Warburg, F., Moreno-Muñoz, P., Skafte, N., & Hauberg, S. (2022). Laplacian autoencoders for learning stochastic representations. *Advances in Neural Information Processing Systems*, *35*, 21059-21072.\n\n[Quality] Extensive empirical evaluations, including a careful ablation study, are conducted. The results demonstrate that the proposed approach outperforms HIE and PFE in the considered cases. \n\n[Clarity] The paper effectively uses figures to demonstrate ideas. However, I do find the need of reading with jumps between the main text and the supplemental material in order to grasp the concept. The organization can be improved. \n\n[Significance] This paper demonstrates uncertainty quantification based on Laplace approximation in the context of probabilistic retrieval. This is an important topic for improving the robustness and mitigating the silent failure of deep neural network systems. The main contribution of this paper appears to be the empirical study, which can be informative and useful for practitioners.\n\nWEAKNESSES:\nIt appears that this paper builds upon prior work, including the Laplacian autoencoder (Miani, M. et al., 2022), as well as several works on uncertainty in metric learning. Consequently, I am more concerned about the unique technical contributions of this work. In this regard, the necessity of the probabilistic view of the contrastive loss is not well presented and not well motivated.\n\n Another missing piece is how to perform test-time retrieval within the proposed framework and how its efficiency compares to the amortized approach. Given a query image, does the proposed approach learn a stochastic representation for it? Is the retrieval based on ranking the deterministic similarities between the query image and all other candidate images? If not, how is it performed? \n\nThe organization and clarity of this paper could be improved. If certain results are deemed important enough, they should be moved into the main text. The authors should focus on introducing the unique aspects of this work and provide a strong motivation for them, such as the Von Mises-Fisher distribution. The frequent references to the Appendix disrupt the logical flow of the paper.\n\nQUESTIONS:\nThe probabilistic view in the supplemental also seems a little bit weird to me since both argument and parameter are data-dependent. Would it be sufficient to just treat the probabilistic contrastive likelihood as a second-order differentiable loss, if they are mathematically equivalent anyway?  \n\nWhat are the main roadblocks that have been overcome in applying the Laplacian autoencoder (LAE) to probabilistic retrieval?\n\nAre there any other ways of approximating Hessian for contrastive loss, while still maintains scalability and positive definiteness? Is KFAC applicable, e.g., Ritter et al., 2018? \n\nRitter, H., Botev, A., & Barber, D. (2018, January). A scalable laplace approximation for neural networks. In 6th International Conference on Learning Representations, ICLR 2018-Conference Track Proceedings (Vol. 6). International Conference on Representation Learning.\n\nLIMITATIONS:\nThe authors discussed limitations about computation load which I think is reasonable.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors show that contrastive loss can be viewed as a likelihood after projection onto the spherical space. This consequently allows them to use the Laplace approximation to estimate the posterior over the parameters. To make the construction further amenable to estimation, the authors propose approaches to make the Hessian positive definite. Finally, the experimental results and ablations go into details about the design choices.\n\nSTRENGTHS:\n- The authors use a conceptually simple approach to constructing approximate posteriors on top of neural networks using the Laplace approximation.\n- The authors provide several analogies for various concepts introduced which makes the reader comfortable, and would be resourceful paper for the community.\n- Most design choices in the method are covered by ablations, which is much appreciated.\n\nWEAKNESSES:\n- The Laplace approximation hinges on an assumption of vanishing gradient which may not be true for modern large neural networks and large datasets used for representation learning. See Question 2.\n- The overall method involves a fair number of moving parts, and it would be good to reconcile them as a single algorithm or a list of bullet points for easy digestion for the reader.\n- The method can be computationally expensive, and the authors have to resort to construct stochasticity of the last-layer parameters only. Much of the post-hoc LA literature seems to be working with a similar setup, so I do not count this as a major weakness of this paper but rather the whole community.\n\n### Minor\n\n- Please use `\\citet` instead of `\\citep` for references directly referring to the paper, for instance in Line 164.\n- It would be great to have Figure 3 on the same page as the text on ensuring positive definiteness on Page 4.\n\nQUESTIONS:\n1. In Line 46, the authors claim that they do not assume any distribution on the stochastic embeddings. But the choice of using Laplacian approximation on the parameters to construct the posterior implicitly makes an assumption on the constructed embeddings. Could the authors clarify this?\n2. As stated in Eq. (2), the Laplace approximation comes from a second-order Taylor expansion, where the first-order gradient vanishes due to $\\theta^\\star$ being the optima. Does this does really happen in practice, and how numerically close are we to zero for instance w.r.t. the size of models/data?\n3. Could the authors confirm if my understanding of the overall approach is correct? (a) Using online LA to construct the posterior (b) Use samples from the posterior to construct stochastic embeddings (c) Use embedding samples to construct $\\kappa$ for the von Mises-Fisher distribution which is used as a single number to quantify uncertainty.\n\nLIMITATIONS:\nYes. See also weaknesses and questions.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-05"}

comments: SUMMARY:
They propose a Bayesian encoder for metric learning. They learn a distribution over the network weights with the Laplace approximation. They first prove that the contrastive loss is a negative log-likelihood on the spherical space. They propose three methods that ensure a positive definitive covariance matrix. They present a novel decomposition of the Generalized Gauss-Newton approximation. 
The empirical results leads previous methods on OOD examples.

STRENGTHS:
- Well organized paper and good writing. Clearly presenting the idea. Code released.
- The bayesian approach for uncertainty measurement in metric learning is interesting.
- The method show improved results, especially on OOD examples, and ablation study is comprehensive.

WEAKNESSES:
- First the experimental results only achieved limited improvement.
- As the paper claimed, The method is slow in computation as it is a bayesian method.
- The paper only focus on a classical contrastive loss which is a margin-based loss, how about other cases? for example, the proxy-based losses?
- Overall the performance improvement is not significant, and the computational efficiency is not competitive, compared with other uncertainty method.

QUESTIONS:
1. Could you provide a systematic analysis of the computational efficiency?
2. Proxy-anchor loss can be a good case to study with this uncertainty measurement.

LIMITATIONS:
yes, they have addressed the limitations of the paper.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-10