Approximations to the Fisher Information Metric of Deep Generative Models for Out-Of-Distribution Detection
Authors: Sam Dauncey
Christopher C. Holmes
Christopher Williams
Fabian Falck
Venue: Accepted by TMLR ()

Abstract: Likelihood-based deep generative models such as score-based diffusion models and variational
autoencoders are state-of-the-art machine learning models approximating high-dimensional
distributions of data such as images, text, or audio. One of many downstream tasks they
can be naturally applied to is out-of-distribution (OOD) detection. However, seminal work
by Nalisnick et al. which we reproduce showed that deep generative models consistently
infer higher log-likelihoods for OOD data than data they were trained on, marking an open
problem. In this work, we analyse using the gradient of a data point with respect to the
parameters of the deep generative model for OOD detection, based on the simple intuition
that OOD data should have larger gradient norms than training data. We formalise measuring
the size of the gradient as approximating the Fisher information metric. We show that the
Fisher information matrix (FIM) has large absolute diagonal values, motivating the use of
chi-square distributed, layer-wise gradient norms as features. We combine these features to
make a simple, model-agnostic and hyperparameter-free method for OOD detection which
estimates the joint density of the layer-wise gradient norms for a given data point. We
find that these layer-wise gradient norms are weakly correlated, rendering their combined
usage informative, and prove that the layer-wise gradient norms satisfy the principle of (data
representation) invariance. Our empirical results indicate that this method outperforms the
Typicality test for most deep generative models and image dataset pairings.

Published in Transactions on Machine Learning Research (10/2024)
Approximations to the Fisher Information Metric of Deep Generative Models for Out-Of-Distribatio Detection
Sam Dauncey*
s.dauncey@sms.ed. ed.
Deportment of Mathematics
University of Edinburyh
Chris Holmes
cholmes@staas.or.cco
Department of Statistics
University of Oxford The Alan Turing Institute
Christopher Williams
williams@stats.orr www.tssttttttttttatoooooo
Department of Statistics University of Oxford
Fabian Falck
fabicn.ffolckBsstttsoorccc
Department of Statistics University of Oxford The Alan Turing Institute
Work partially done during an internship at University of Oaford.
Reviewed on OpenReview: https: // openreview. net/ forum? id- Ecuutén
Abstract
Likelihood-based deep generative models such as score-based diffusion models and variational autoencoders are state-of-the-ar machine learning models approximating digh-dimensional distributions of data such as images, text, or audio. One of many downstream tasks they can be naturally applied is out-of-distribution (OOD) detection. However, seminal work by Nalisnick et. al. which we reproduce showed that deep generative models consistently infer higher logllikel for OOD data than data they were trained on, marking open problem. In this work, we analyse using the gradient of a data point with respect to the parameters of the deep generative model for OOD detection, based on the simple intuition that OOD data should have larger gradient norms than training data. We formalise measuring the size the gradient as approximating the Fisher information metric. We show that the Fisher information matrix (FIM) has large absolute diagonal values, motivating the use of chi-square distributed, layer-wise gradient norms features. We combine these features to make simple, model-agnostic and hyperparameter-free method for OOD detection which estimates the joint density of the layer-wise gradient norms for given data point. We find that these layer-wise gradient norms are weakly correlated, rendering their combined usage informative, and prove that the layer-wise gradient norms satisfy the principle of (data representation) invariance. Our empirical results indicate that this method outperforms the Typicality test for most deep generative models and image dataset pairings.
1 Introduction
Neural networks can be highly confident but incorrect when given inputs different to the distribution of data they were trained on (Szegedy et al., 2014; Nguyen al., 2015). While domain generalisation Zhou al. (2021) and domain adaptation (Garg et al., 2023; Ganin et al., 2016) methods tackle this problem by learning

---

Published in Transactions on Machine Learning Research (10/2024)
Figure 1: Gradients from certain layers are highly informalive for OOD delection. We select two highly informative neural net- work layers of a deep generative model with parameters 0,, 0, from Glow Kingma & Dhariwal (2018) model trained on CIFAR-10 and plot the LL-norm of the gradients fo, [<)](=))] for in-distribution (CIFAR-10) and aut-of-distribution (SVHN) samples æ. The gradient norm of these two layers al- lows to separate in- and out-of-distribution samples.
log
In-distribution out-of-distributiin
log fgf
machine learning systems which are robust or can actively adapt to domain shift, there may be scenarios where for specific data points this domain shift is too severe to draw reliable inferences. Identifying and possibly filtering out anomalies or oul-of-distribulion (OOD) inputs before deploying the model in the wild is a viable strategy in such cases, especially for safety-critical applications (Ulmer et al., 2020; Stilgoe, 2020; Baur et al., 2021).
Deep generative models such variational autoencoders Kingma & Welling (2014), normalising flows Papamakarios et al. (2021), autoregressive models Van den Oord et al. (2016); Salimans et al. (2017) and diffusion models Sohl-Dickstein et al. (2015b); Ho et al. (2020) are an important family of models in machine learning which allow us to generate high-quality samples from high-dimenssonal, multi-modal conditional or unconditional distributions in domains such as images, videos, text and speech. Many current state-of-the-art methods are probabilistic: they approximate the data log- ligliikl,hood, the likelihood of a data sample given the learned model parameters under the data distribution, or lover-bound thereof. This renders them a natural candidate for the task of OOD detection as they 'out-of-the-box' provide OOD metric Bishop (1994) (i.e. the approximated data likelihood) which they use the training objective. However, Nalisnick et al. (2019a); Choi el al. (2018) showed that many of the above mentioned classes of probabilistic deep generative models consistently infer higher liglliiel for data points drawn from OOD datasets (non-training data) than for in-distribution (training data) samples. In Fig. 2 we replicated their results with a Glow model, and show that score-based diffusion models are likewise affected by the phenomenon. This result very surprising given that generative models are trained to maximise the log-likelihood of the training data, and are able to generate high-fidelity, diverse samples from the training distribution. This marks an open problem for deep generative models, and renders the direct use of their estimated likelihood in out-of-distribution detection infeasible. It also questions what deep generative models learn during training, and how they generalise.
Related work tackled this problem from two angles: explaining why lig-likelihood estimates of these models fail to discriminate, Kirichenko et al. (2020); Zhang et al. (2021a); Le Lan & Dinh (2021); Caterini & Loaiza-Ganent (2022), and proposing likelihood-based OOD detection methods and adaptations of existing ones which may overcome these shortcomings Ren et al. (2019); Choi et al. (2018); Hendrycks et al. (2018); Liu et al. (2020); Havtorn et al. (2021); Nalisnick et al. (2019b). In $4) we will analyse the previous work on gradient-based based OOD detection, linking independent discoveries of other authors Nguyen et al. (2019); Kwon et al. (2020); Choi et al. (2021); Bergamin et al. (2022).
Motivation and Intuition. This paper presents an alternative approach for OOD delection which we motivate in the following. Consider the example of a (linear) regression model fitted on some (in-distribution) training data. If we now include an outlier in our training data and refit the model, the outlier will have lot of influence on our estimate of the model's parameters compared other in-distribution data points. One way to formalise this intuition using the hat-value: The hat-value is defined the derivative dg of the dg
2

---

Published in Transactions on Machine Learning Research (10/2024)
model's prediction 9 with respect to a given (dependent) data point y. It describes the leverage of single data point, which is used for fitting the model, on the model's prediction for that data point after fitting.
We extend this intuition of OOD to deep learning by considering the gradient of the log- log-likelihood of a deep generative model with respect to its parameters, also known as the score. If a neural network converged to a local (or global) minimum, we expect the gradient of the likelihood with respect to the model's parameters to be flat for training data points. Hence, the score is small in norm, and performing an optimiser step for a training data point would not change its parameters much. If after many epochs of training we were now to present the model an OOD data point which it has not seen before, hhat-value of a neural network' be steep: the norm of the score would be large, just like hat values are large (in absolute value) for OOD data. An optimizer step with an OOD data point would change the neural network's parameters a lot. It is this intuition which motivates us to theoretically analyse the use of the gradient for OOD detection. In preview of our analyses, in Fig. (and later more thoroughly in Fig. 4) we will preciscly observe that (layer- wisc) gradient norms are in general larger for OOD than for in-distribution data, which enables the use of gradients for OOD detection. Code to reproduce our experimental results is publicly available on GitHub
expect the gradient-resembeee resembling the
Our contributions are as follows: (a) We analyse the use of the gradient of a data point with respect to the parameters of a deep generative model for OOD detection, and formalise this as approximating the Fisher information metric, a natural way of measuring the size of the gradient. (b) We show that the Fisher information matrix (FIM) has large absolute diagonal values, motivating the use of layer-wise gradient norms which are chi-square distributed as possible approximation. Our theoretical results show that layer layer-wise gradients satisfy the principle of (data representation) invariance Le Lan & Dinh (2021), desirable property for OOD methods. We also find that these layer-wise gradient norms are weakly correlated, making their combined usage more informative. (c) We propose a first simple, model-agnostic and hyperparameter-fiee method which estimates the joint density of layer-wise gradient norms for a given data point. In our experiments, we find that this method outperforms the Typicality test for most doep generative models and image dataset pairings.
2 Current Methods for OOD detection
In this section, we define the OOD detection problem, describe the open problem of using deep generative models for OOD detection and how the input representation may explain this, and how a gradient- method can be a compelling approach which is invariant to the input representation.
2.1 OOD detection: Problem Formulation
Given training data æ1 TI... EN drawn from a distribution p over the input space X C R2, we define the problem of OOD detection as assigning an OOD score S(x) to each æ E X such that points with low OOD scores are semantically similar to points sampled from p. OOD detection is unsupervised if it is not given class label information at training time.
The specific problem we are interested in is leveraging recent advances deep generative models for unsupervised OOD detection. Here a deep generative model p is trained to approximate the distribution of some training data .N ~ p, and S is a statistic derived from p" (such as the model likelihood Nalisnick et al. (2019b), latent variable hierarchy Schirrmeister et al. (2020); Havtorn et al. (2021), or combinations thercof Morningstar et al. (2021)).
In order to evaluate an OOD detection method, one is required to select semantically dissimilar surrogate out-distributions (e.g. a different dataset) to test against, Previous work has sought to define OOD detection as a generic test against data sampled from any differing distribution Hendrycks & Gimpel (2017). Our additional requirement that the out-distribution is semantically dissimilar is motivated by recent theoretical work by Zhang et al. (2021a) showing that single-sample test against all out-distributions is impossible.
http::://gichubb.cm///aa////////
Modeee g- Knoxledge
3

---

Published in Transactions on Machine Learning Research (10/2024)
Slew
cm can
-1
-14
-31
-44
-08 -23 4.3 Negativv
Negestoo
Figure 2: Counter-inluilivee properties of likelihood-bused generalive models. Histogram of the negative log-likelihoods inferred from Diffusion Ho et al. (2020) model [Left] and Glow Kingma & Dhariwal (2018) model [right] trained on one of four image datasets (corresponding to the four subplots) and evaluated on the test. set of all four datasets, respectively. For diffusion models we use the negative log-likelihood from one step of the diffuuion process pd (zolar 11). For both models we scale the log- ligeliiiihooos by the dimensionality of the data, in this case 3 32 x 32. This Figure replicates the results in the seminal paper by Nalisnick et al. (2019a), noting that results for diffusion models novel. We find that the training dataset has counter-intuiively small impact the ordering of the datasets as ranked by log-likelihood.
2.2 Likelihood- based methodology for unsupervised OOD detection
Likelihood thresholding. Bishop (1994) proposed using the learned model's negative log likelihood an OOD score 5(x) = -ogog lllgoo(()) (x). In their seminal paper, Nalisnick et al. empirically demonstrated that this approach fails for wide variety of deep generative models (Nalisnick et al., 2019a). In particular they showed that certain image datasets such as SVHN are assigned systemically higher likelihoods than other image datasots such as CIFAR10, independent of the training distribution. We replicate this result (for a Glow model, lype of normalising flow, and for the first time denoising diffusion model) in Figure 2. In their follow up work Nalisnick et al. argue that, in the example of a standard Gaussian of large dimension D, samples close to the origin should be classified as OOD the Gaussian annulus result Blum et al. (2020) demonstrates that the vast majority of samples from have a distance of vD from the origin. Generalising this to generative models, they argue that samples with likelihoods much higher than likelihoods of in-distribution samples must be semantically atypical (Nalisnick et al., 2019b). They use this to motivate OOD scoring based on the likelihood being too high or too low, defining the typicality Cover & Thomas (1991) score ((x:) log p H. where IÅ is the average log-likelihood on some held-out training data.
Likelihood ratios. The likelihood assigned by deep generative models has been shown to strongly correlate with complexity metrics such the compression ratio achieved by simple image compression algorithms Serrà et al. (2020), and likelihoods from other generative models trained on highly diverse image distributions Schirrmciste et al. (2020), with the highest likelihoods being assigned to constant images. This is somewhat expected, as for discrete data the negative log likelihood is directly proportional to the number of bits needed to encode the data under arithmetic coding. To add to these findings, in Appendix A.3 we use a very simple complexity metric TV, the total variation achieved by considering the image as a vector [0, 17784, to show that the whole of MNIST is contained in a set of bounded complexity with volume (Lebesgue measure) 10-110 Thus model needs to only assign very low prior probability mass to this set for high likelihoods to be
4

---

Published in Transactions on Machine Learning Research (10/2024)
The log-likelihood heavily depends on data repre- sentation Le Lan Dinh (2021). Here we plot the first two samples of the CIFAR10 dataset and the difference in Bits Per Dimension (BPD) induced by changing from an RGB to an HSV colour model:
Figure 3:
log2 (x) - logg PHSV (x)
A RRGB-HSV BPD
3 x 32 x 32
In Appendix B.1, we provide experimental details and in replicate this for the first 20 samples, where we observe A RGB BCBB values ranging from 0.18 to 1.76 BPD
4AGB 0.43
ARGe VSVV 1.01
achieved, demonstrating rating the important connection between volume, complexity and model likelihoods which hence discuss in $2.3.
Ren et al. (2019) argue in favour of using likelihood ratio tests in order to factor out the influence of the "background likelihood", the model's bias towards assigning high likelihoods to images with low complexity. In practice, this requires modelling the background likelihood via corruption of training data Ren et al. (2019), out-of-the-box and neural compressors Serrå et al. (2020); Zhang et al. (2021b) or the levels of a model's latent variable heirarchy Schirrmeister al. (2020); Havtorn al. (2021), leading to restrictions for the data modalities models to which the method can be applied to. In general there is limited number of use cases whereby one can pre-specify the OOD distribution DOD they are interested in well enough that they can evaluate the likelihood of the data under this OOD distribution log (oD (which is necessary to evaluate the likelihood), without access to samples from this distribution at which point classification becomes a better-studied option.
2.3 Representation dependence of the likelihood
Le Lan & Dinh (2021) emphasise that the definition of likelihood requires choosing a method of assigning volumes to the input space X. Specifically, datapoints could be represented as belonging to some other input space T, linked via a smooth invertible coordinate transformation T X' - T. The model probability density for given datapoint E X, which denote (x), will thus differ from the probability density p4 (t) of the corresponding point t = T(x) by factor of the Jacobian determinant of T (Le Lan & Dinh, 2021):
(t) (x)
dx
The volume element 8T describes the change of volumes local to æ as passod through T. This term can grow or shrink exponentially with the dimensionality of the problem, making its effect counter- intuiivvly large. As an empirical example in the case of image distributions, in Fig. 3 and Appendix B.1 Fig 8 consider the case of change of color model TRGB-HS from Red-Green-BBue (RGB) to Aue-Saturation- Value (HSV) representation. We compute the induced change in bits per dimension as a scaled log-value of the volume element A BPD
STRCB 188V and report values for 20 non-cherry picked CIFAR-10 10 images ranging from 0.18 1.76. For comparison the average BPD reported in the seminal paper by Nalisnick et al. (2019a) was 3.46 on CIFAR10, compared to 2.39 on SVHN when evaluating with the same model. Hence, if we use the likelihood for OOD detection, whether classify sample OOD or not may flip for some samples merely by changing how the data is represented.
RGB-HSV
3x32x37 log
The
Motivated by the strong impact of the volune element, Le Lan & Dinh (2021) propose a principle of (representation) invariance; given a perfect model pd of the data distribution, the outcome of an unsupervised OOD detection method should not depend on how we represent the input space X. In theory likelihood ratios are innresentation-invaiaan (Le Lan & Dinh, 2021), however in practice the method used to generate the background distribution often re-introduces dependence on the representation. For example Ren et al.
5

---

Published in Transactions on Machine Learning Research (10/2024)
(2019) propose to generate the background distribution by re-sampling randomly chosen pixels as independent uniform distributions, re-introducing the notion of volume.
2.4 Invariance of the gradient under invertible transformations
To achieve representation invariant OOD score (Le Lan & Dinh, 2021), thus motivated to quotient out the effect of the volume element in Eq. (8). We now present first theoretical contribution, which shows that methods based on the gradient of the log-likelihood do precisely this.
Proposition 1. Let (x) and (t) be two probability density functions corresponding to the same model distribution od being represented on two different measure spaces X' and T. Suppose these representations encode the same information, i.e. there exists a smooth, invertible reparameterization T' X' + T such that for x E X and E T representing the same point we have T(x) = tt Then, the gradient vector Vo (oolog7) p") is invariant to the choice of representation, and particular, Vo(log = V p (x)).
Proof. See Appendix A.1. We prove analagous results for variational lower bounds (e.g. the ELBO of a VAE) in Appendix A.2.
Remark 1. Training a generative model pd with initialisation parameters 6g with ligelikelihood as the loss via gradient-descent produces training trajectories O, 0x which are representationninvaria
The interpretation of the above results is subtle. We would like to caution the reader by noting it does not mean that the inductive biases are discarded when the gradient is computed as inductive biases pertaining to distances between data points are frequently encoded in the parameter space. Further, remark may explain why the likelihood can still be used to train deep generative models and allow them to generate convincing samples when using gradient-based optimisation algorithm, even though the likelihood value itself appears uninformative for detecting if data is in-distribution.
3 Methodology
In this section, we develop patieemmattialll--pinpppppp method for gradient-based OOD detection.
3.1 Layer-wise gradients are highly informative and differ in size by orders of magnitudes
We are now interested in formulating a method which uses the intuitively plausible (see $1) and data representation-inaariin (sce $2.4) score Vol(x) follog (x) for OOD detcction. A naïve approach would be to measure the size of the score vector by computing the L norm I|Vol(=))13 of the gradient Nguyen et al. (2019). We can view this L norm as the directional derivative of the log- ligliikllooodd log in the direction of its own gradient Vo((z), which can be intuited as a measure of how much the model can learn about the given datapoint with one small gradient update.
In the following, we analyse this idea, demonstrating its limitations: We empirically find that the size of the norm of the score vector is dominated by specific neural network layers which the overall gradient norm cannot capture. In Fig. 4, we train deep generative models (here: Glow Kingma & Dhariwal (2018) and diffusion models Ho et al. (2020)) on training dataset (here: CelebA). We then draw a batch of items from different evaluation datasets and compute the squared layer-wise L²-norm of the gradients of the log-likelihood of deep generalive model with respect to the parameters 0j of the corresponding layer, i.e.
fo, (ZL.B) Ve, (Ei,, ((x))) The histrogrammes in the left two columns plot fe, (II.B) B) for each layer separately, the plots in the rightmost column shows their interaction in scatterplot.
Two points are worth noting: We observe that for given neural network layer (and different batches), the gradients are of a similar size, but across layers, the scale of the layer-wise gradient norms differs by orders of magnitudes. In particular, taking the norm over the entire score vector would overshadow the signal of layers with a smaller norm by those on much larger magnitudes. Second, note that the layer-wise gradient norms do not strongly correlate for randomly selected layers. In particular, one may find two layers with corresponding features So, (.): and fo, (#1:B) which allow us to separate training (in-distribution)
6

---

Published in Transactions on Machine Learning Research (10/2024)
out of distribution SVHN
distrittttin CelebA
out-of-istri GTSRa
out-o dddiributinn CIIAN-11
-3.0
ouf-ol-dstr bution imageNet32
Gited distributio
Model
-3.5
-1.0
-2.5
-2.
-15
-10
-0.5
f6
log log///0s
-17.5
-18.0
-18.5
Diftuson Model
11..0
21.5
-25
-13
bgg (a(i)
log fall) 4)
Figure 4: Layer-wise gradients of the log-likelihood (the score) are highly informative for OOD detection. Their size differs by orders of magnitudes between layers, and they are not strictly correlated, rendering layer-wise gradients (in contrast to the full gradient) discriminatory features for OOD detection. In each row, we randomly select two layers with parameters 0,, 0, from Glow Kingma & Dhariwal (2018) model [Top] Diffusion model Ho et al. (2020) [Bottom], which have 1353 and 276 layers, respectively. The models are trained CelebA, dataset that has proved challenging for OOD detection in previous work Nalisnick et al. (2019b). We then evaluate this model on batches 1:13 (B 5) drawn from the in-distribution and OOD test datasets and compute the squared wise -orm of the gradients of the log-likelihood with
respect to the parameters of the layer, i.e. (F1.8)) Vo, (E (mm)) :
[Left and Middle] shows the two layer-wise gradients separately, [Right] shows their interaction in scatter plot. In Appendix B Figures 9 11, provide our complete results, showing more layers from three likelihood- beaihoodd-basd generative models, each trained and evaluated on five datasets.
from evaluation (OOD) data points with line in this latent space. Considering example, when fixing So, (#1.n) & 77.5, large negalive values of (x).n) are in-distribution, and as they become more positive, they correspond to the out-of-distribution datasots CIFAR-10 and ImageNet33, respectively, with very high probability. This renders the layer-wise information superior over the overall gradient for use as discriminative features in OOD detection. We present our complete results in Appendix B Figs. 9-11, showing further layers (histogrammes), also for other deep generative models (VAFs Kingma & Welling (2014)) and training datasets (SVHN, CelebA, GTSRB, CIFAR-10 & ImageNet33).
3.2
The Fisher Information Metric: A principled way of measuring the size of the gradient
Having identified the limitations of using the 122 norm of the gradient, perhaps mathematically more natural way to measure the score vector's size is to use the norm induced by the Fisher information metric \ (Radhakrishna Rao, 1948), defined as
[Vol(()) Vol(x)) F7 Vol(x), (oll())
Fe Ey-n (Vole)))ooo))))))))
(2)
7

---

Published in Transactions on Machine Learning Research (10/2024)
layer
layer
0.50
0.25
Foof A9
0.00
FuF
-0.25
layer
-0.50
-1.00
Figure 5: The layer-wise FIM has large absolute diagonal values. We randomly select two layers 8g and 8, from a Glow model trained on CelebA, and randomly select max(50, |0,1) weights from each layer. We then compute slices of the FIM using the method described in Equation (3) and plot the results, with dark blue colours at coordinates (a, 3) corresponding to larger values for the corresponding element of the FIM. In order to maintain visual fidelity the plot when weights between layers vary by orders of magnitudes, we normalise row a by a factor of /Faa where Fax indicates the element of the FTM at coordinates (a, and likewise for the columns, which could be equivalently formulated as re-scaling the model parameters by this factor. The same plots using diffusion models and of the raw values Fag (without row and column-wise normalisation) presented in Appendix B.3, Figures 14 & 15.
where Fo is called the Fisher Information Malria (FIM). Intuitively, the FIM re-scales the gradients to give more equal weighting to the parameters which typically have smaller gradients, and thus the Fisher information metric accounts for and is independent of how the parameters are scaled. This in theory prevents a dependence on representation in the gradient space.
The value [Ve(<))] called the score statistic, which Rao (1948) showed to follow a x² distribution with degroes of froedom, assuming the model parameters are maximum likelihood estimates. Thus the score statistic can be used with statistical test. known as the score lest Radhakrishna Rao (1948), as used by Choi et al. (2021); Bergamin al. (2022). Deep generative models with likelihood-based objective perform form of approximate maximum-likelihooe estimation.
3.3 Approximating the Fisher Information Matrix
In practise, the full FIM has P x P entries, where P= | is the number of parameters of the deep generative model. This means that is too large to store memory, and would furthermore be too expensive to compute and invert. For example our glow implementation has 44 million parameters, and thus the FIM would require R 7, 700 terabyles to store using float32 representation. To develop computable method, we therefore need to find way to approximate the FTM. What would be good choice for this approximatio?? This problem is non-trivial due to its dimensionality. We start answering this question by computing the FIM in Eq. (2) restricted to subset of parameters in two layers with parameters 0, 02 of a Glow Kingma & Dhariwal (2018) model trained on CelebA, using the Monte-Carlo (MC) approximation
Fe, Eyroo (Ve, 1 N Vo
( Vo,
yo p
"Nte that our choice MC approximation just for the sake being able compute Fe; we here not make any further (and more principled) approximations assumptions.
8

---

Published in Transactions on Machine Learning Research (10/2024)
where Ve, refers to taking the gradient with respect to the parameters 02 in layer j and y are samples drawn from the generative model y². This computation is infeasible for larger layers of the network which may be highly informative for OOD detection, demonstrating the need for a more principled approximation tochnique.
In Fig. we illustrate the resulting (restricted) FIM estimate for two layers with N 1024. Further layers of this and other models, and when trained on other datasets are presented in Appendix B.3, Figures 15. We observe interesting pattern of diagonal doménance: The diagonal elements are significantly larger in absolute value, on average approximately five times the size of the off-diagonal elements. Hence, seemingly 'crude', yet as turns out highly efficient approximation of the layer-wise FIM is to approximate of the identity matrix, which corresponds to computing the layer-wise L norm of the gradient. This reduces the cost from inverting an arbitrary P x P with potentially large P to approximating one value for each of the layers l 02 .2..... f of the model.
multiple
3.4 A method for exploiting layer-wise gradients
We are now interested in operationalising our observations so far into an algorithm that can be used in practice.
In addition to the diagonal dominance phenomenon enabling layer-wise approximation via a diagonal matrix, recall that layer-wise gradients contain more information than the overall gradient norm as the scale of the gradient norms differs by orders of magnitudes. We are therefore motivated to consider each layer 01, 02 ......0. O, in our model scparately and combine the results as an OOD score in the second step. Specifically, if we select layer 0j we can consider a restricted model where only the parameters in this layer are variable, and the other layers are frozen. We can approximate the score statistic (2) this restricted model, whose parameters are more homogencous in nature. In practice, we take the score vector for a layer Vo,1(z)) and attempt to approximate (x))) which should follow x2 test with 00 degrees of freedom for in-distribution data. As discussed in $3.3, we approximate the FIM restricted to this layer as a multiple of the identity. For a batch of (possibly equal to 1) of data points #1n we define features sa,, which via our identity-matrix approximation should be proportional to (Z,
by
fo, (w1.))
Vo,
Va;
Σ ((xb))
(4)
Given that these layer-wise 2 norms fe, should be proportional to a x² distributed variable with a large degree of freedom, we expect log fo, to be normal-distributted Bartlett & Kendall (1946). In Fig. 4 (further results in Appendix B) we observe a good fit of log Sa, to a Normal distribution, empirically validating this holds spite of our approximations.
This gives rise to natural method of combining the layer-wise 2 norms: we simply fit Normal distributions to each log-feature log fe, independently, and then use the joint density an "in-distribution" score. Algorithms to 3 summarise our proposed method. As with other unsupervised OOD detection methods (Nalisnick et al., 2019b), we assume the existence of a small fit set of data held-out during training to accurately fit to each feature fy. Note that in practise our method is very straight-forwant to implement, requiring only a few lines of PyTorch code. Many other methods could possibly be constructed from our theoretical and empirical insights, and we will discuss potential future work in $6.
In Appendix B.5 we observe a mild performance improvement, uniformly across datasets, with the joint density approach in comparison to using Fisher's method Fisher (1938) when combining these statistics using z-scores. We hypothesise that this could be due to the density being more robust to correlation between adjacent layers as noted in Fig. 6. Our presented method does not enjoy the full invariance under rescaling of the model parameters as the true score statistic (see $3.2). However, in Appendix A.4 we show that it does satisfy invariance when rescaling each layer individually, justifying our use of the density in this sctting. Our method satisfies the desiderata of the principle of invariance (see $4), is hyperparameter free, and is applicable to any data modality and model with differentiable estimate of the log- likelhood.o
9

---

Published in Transactions on Machine Learning Research (10/2024)
Autocomeaton LayerWiee oredients
Autocomelation Difluiion Layenikse
onwoogieiiigiiioooooooo 0
Caree at ontoegie Sg/003
1--1
P-1
Figure The L ---ormms of layer-wise wise gradients have little corvelation. We select layers with parameters 0,, O and measure the correlation of the logarithm gradient Li-norms log fe; (z). Binning these correlations by the distance between the layers ji - j| and averaging across correlations of this distanco gives the above plot. We note that there is strong correlation in Lo-norm between adjacent layers, but that this correlation quickly decays for both in-distribution and out-of -iw-oo--dittibutionn data. We hypothesise that this enables our approximation of the FIM which assumes independence across layers to provide good performance.
Algorithm Algorithm for computing features
Require: Deep generative model M, with paramcters 01, 02,... O, in each of its J layers.
function GRADIENT FEATURES( ..))
for 2t in batch do 1(x;)= + (xx)) end for
D Compute the log-likelihood
vg Vo (1(((()) .... ((x)))
Compute the gradient via backpropagation
for j 1...J in layers do
f + |0, l end for end function
b Store the layer-wise L norms
Algorithm Algorithm for training models
Require: train dataset and held-out fit dataset
Train deep generative model M, with parameters 02, ........ O, in each of its J layers. for Batch æx az in fit dataset do ..... f3 + CRADIENT (EA x²))
end for
for j + 1...J in layers do
Mj MEAN(log f/ ... log )
0 <- VARIANCE((og log (N)
D Fit. Gaussians to logarithmic features
end for
Algorithm Algorithm for detecting 001) data
Given new batch of samples Y/ V....9p
f < GRADIENT PEATURES( .1... ... ....))))
S(y ... yB) log N (log μu; Diag(o²))
Set OOD score be the Gaussian negative log likelihood
10

---

Published in Transactions on Machine Learning Research (10/2024)
3.5 Application to diffusion models
A denoising diffusion model Sohl-Dickstein et al. (2015a); Ho et al. (2020) uses a chain of latent variables { achieved by gradually adding noise (represented by the conditional distributions ((xx(xx;-)) to the imitial distribution of images z Zo until the final latent variable æy is approximately Gaussian with mean zero and identity variance. The inverse process is then learned by a model approximating p (0.-1/0)) Diffusion models have gained in popularity due to their ability to produce samples with high visual fidelity and semantic coherence, making them a natural candidate for OOD detection Graham et al. (2023). Nonetheless, the full variational lower bound on the log- ligeliiedhood defined by Ho et al. (2020) is expensive to compute as it requires running [T] inference steps on the network modelling pd ( For our setup, [T] = 1000 and rumning full-forward/backe backward pass requires roughly 1 minute of compute per sample. Thus, choose one component of the variational lower bound, namely the one-step log-likelihood,
((z) ((20) = Ex (mm) log p (20)
We refer to Appendix B.4 for computational details, noting that our implementation using one sample from q(<1|<0)) only requires pass on of inference on pd (20o (1). Despite this value being very different to the intractable full log-likelihood (xn), in Figure 2 we observe the same open problem and phenomenon as Nalisnick et al. (2019a) reported for the full likelihood estimates of other deep generative models. In Appendix B.4 we perform an ablation study on this one-step component of the variational lower bound used, finding the result that for both our method and the typicality method Nalisnick et al. (2019b), the components which are computed with less noise added to the image ot used as input to the model n² (0x (axe) are more informative for OOD detection, which could intuitively be understood as the noised image Dt itself being more informative in this regard.
4 Related work
In this section we review related work which uses gradients for unsupervised OOD detection.
In concurrent work to ours, Choi et al. (2021); Bergamin et al. (2022) each present an approximation to Rao's score test (Radhakrishna Rao, 1948). They independently approached the problem from the directions of training on the given sample of OOD data Xiao et al. (2020) and application of tests from classical statistics, respectively. These methods use approximations of the FIM from the field of optimization Amari (1998); Tieleman et al. (2012); Kingma & Ba (2015), whereas we use a simpler approximation tailored to the task of unsupervised OOD detection, and complement this with our empirical observations of the FIM in $3.3. Bergamin et al. (2022) compute a score test across the whole model by approximating the FIM as a diagonal, with elements F.. = (0., log (z))²² + e for a small hyperparameter e = 10-8, which which is used in optimization for its damping effect Martens (2020) and helps to mitigate numerical insta bilities when dividing by Foo. Our method differs in that it explicitly encodes the layer-homogenett of the model (whereby parameters in the same layer have similar gradient sizes and perform similar functions in the network), and the predicted chi-square distribution of the score. We also note that layers which contain squared gradient values that are << 10-8 (see Appendix B.3 Figures 14 and 15) would have their information nullified without careful tuning of e, this can further be observed in 10 and 11 where there are entire layers which provide informativity for OOD detection and and have an 1,2 norm of 10-8 Choi et al. (2021) split the problem of OOD detoction layer-wise, but use the more complex EKFAC George et al. (2018) algorithm to account for dependencies between adjacent parameters. After some normalisation and additional processing steps, the authors compute the ROSE metric by taking the maximum feature over some pre-selected subset of layers. Our method differs as it uses holistic score influenced by all the model layers.
Nguyen et al. (2019) are interested in using VAEs to detect anomalous web traflic in a semi-supervised setting, measuring the difference between a test gradient and labelled anomalous gradients. Our method differs in that does not require anomalous examples. Kwon et al. (2020) computes cosine similarity between the gradients in the decoder of VAE and the average gradients observed during training as their OOD metric of choice. In this work, we advocate for using the size of the gradient vector rat than its angle as done in Kwon et al.
11

---

Published in Transactions on Machine Learning Research (10/2024)
(2020): our intuition is that, for well-trained model evaluated in-distribution data, close to local minimum where the gradient is flat and the variance of the angle of the gradient vector is high. In particular, when averaging over samples from the model, we have that Be-z' Ve (log (logp") (x) 0 as a distribution minimises its own cross-entropy. In their supplementary, Nalisnick et al. (2019b) note that the Maximum Mean and Kernelized Stein Discrepancy lests they use to benchmark their typicality test only achieve good performance when using the inner product of the parameter gradients k(x;, 2z) Vol(z;)"Vo((z;)) leading them use the inner product with respect the data in their experiments k( xy) = (al(a,)"val(a) Our method differs from using inner product with respect to the parameters in that it allows for information to be used from all the layers, rather than a few dominant layers, we discuss in $3.1.
To the best of our knowledge, no previous work has connected these works bar Bergamin et al. (2022)'s 's citation of Choi et al. (2021). The theoretical grounding which we provide in Proposition 1 $A.1 may explain why multiple other authors have independently found efficacy unsupervised OOD detection with gradient information.
For completeness, in Appendix C we review previous work on the use of gradients of classifiers for the task of supervised OOD detection.
5 Experimental benchmark
test | train +
SVHN
CelebA
GTSRB
CIFAR-10
ImageNet32
SVHN
0.8735
0.3469
0.8599
0.8915
typicality
(B 1)
CelebA
0.9989
0.6506
0.3680
0.2857
GTSRB
0.9261
0.8201
0.6708
0.5548
CIFAR-10
0.9829
0.7733
0.6423
0.4147
ImageNet32
0.9952
0.9251
0.8057
0.7249
SVHN
0.9880
0.9858
0.8747
0.8010
CelebA
0.9823
0.9262
0.5155
0.2997
(B = 1)
GTSRB
0.9537
1.0000
0.7546
0.9967
ours
CIFAR-10
0.9658
0.9462
0.9126
0.4377
ImageNet32
0.9976
0.9876
0.9683
0.7375
SVHN
0.9899
0.6119
0.9961
0.9983
lypicality
(B = 5)
CelebA
1.0000
0.9786
0.4737
0.4293
GTSRB
0.9997
0.8987
0.6639
0.6138
CIFAR-10
1.0000
0.9082
0.9613
0.4894
ImageNet.32
1.0000
0.9974
0.9954
0.9013
SVHN
0.9997
1.0000
0.9989
0.9976
CelebA
0.9997
1.0000
0.9525
0.8514
(B =B=5) 5)
GTSRB
0.9996
0.9999
0.9596
0.9999
ours
CIFAR-10
0.9992
0.9970
1.0000
0.6712
ImageNet32
1.0000
0.9995
1.0000
0.9480
Table 1: Comparison of the AUROC values (higher is better) of our method to the typicality test Nalisnick et al. (2019b) for batch sizes B = 1,5. We train Glow Kingma & Dhariwal (2018) models on five natural image datasets (columns) and evaluate the ability of the model-method combination to reject the other datasets (rows). Bold indicates the element-wise higher value comparing both methods.
In this section, we benchmark our OOD detection method against the typicality test. We postpone detailed description of our datasets and models to Appendix D.
We follow the consensus of previous literature Nalisnick et al. (2019a) evaluate our method on distribution pairs: training generative image models on one image distribution and testing against a surrogate out- distribution. We choose five natural image datasets SVHN CelebA, GTSRB, CIFAR-10 and ImageNet32 used in
12

---

Published in Transactions on Machine Learning Research (10/2024)
previous literature Serrà et al. (2020) and evaluate all dataset pairings. To the best of our knowledge this makes our evaluation more extensive than any previously published work in unsupervised OOD detection, a field where rigorous evaluation is especially important as erroneously high performance can be achieved by selective reporting of or fino-tuning hyperparamoters to certain out-distributions.
In Tables & 2 we compare our method against the typicality test Nalisnick et al. (2019b) using the Area Under Receiver Operating Curve (AUROC) statistic on both single sample (B- = 1) batch size (B= 5) OOD detection. We choose typicality as it is, to the best of our knowledge, the most performant method which is both model-agnostic and hyper parameter free. Performance of unsupervised OOD detection greatly depending on the model and even the image resizing algorithm applied make the inputs of uniform size Bergamin et al. (2022). To mitigate this problem we directly compare to our implmentation of Nalisnick et al. (2019b) using the same models and dataset implementations where they exist.
vary
Lest & Irain +
SVHN
CelebA
GTSRB
CIFAR-10
ImageNet332
SVHN
0.9357
0.4661
0.9007
0.8777
typicality
CelebA
0.9990
0,3860
0.3409
0.2837
(B 1)
GTSRB
0.9335
0.8197
0.6981
0.5624
CIFAR-10
0.9920
0.6968
0.4855
0.4142
ImagaNet.32
0.9986
0.8759
0.6759
0.7443
SVHN
0.9903
0.8526
0.5574
0.6214
(B 1)
CelebA
0.9551
0.5466
0.5655
0.3571
GTSRB
0.8691
0.9684
0.5622
0.5530
ours
CIFAR-10
0.9535
0.9639
0.5786
0.4710
ImageNet32
0.9363
0.9818
0.6651
0.5763
SVHN
0.9978
0.7943
0.9975
0.9961
typicality
(B = 5)
CelebA
1.0000
0.7642
0.3156
0.3621
GTSRB
0.9998
0.8336
0.6809
0.5765
CIFAR-10
1.0000
0.7808
0.8332
0.4488
ImageNet32
1.0000
0.9866
0.9675
0.9266
SVHN
1.0000
0.9970
0.8457
0.9561
(B 18=5 5)
CelebA
0.9908
0.8552
0.7734
0.4202
GTSRB
0.9716
0.9997
0.7325
0.9007
ours
CIFAR-10
0.9895
0.9992
0.8104
0.5733
ImageNet32
0.9862
1.0000
0.9309
0.8532
Table 2: Diffussion models Comparison of the AUROC values (larger values are better) of our method to the typicality test Nalisnick et al. (2019b) for batch sizes B = 1,5. We train Diffusion Ho et al. (2020) models on five natural image datasots (as columns) and evaluate the ability of the model-method combination to reject the other datasets (as rows). Bold indicates the element-wise higher value comparing both methods.
For Glow models (Table 1) our method outperforms typicality on most dataset pairings, whereas for diffusson models (Table 2) neither method dominates, although our method achieves higher average AUROC. We hypothesise that the comparative advantage our method enjoys for Glow models could be related to the model having more layers (1353 vs. 276) leading to more gradient features being available. In Appendix E table 7 note poor performance for both methods when applied to VAE model with poor sample quality, indicating that how well the model captures the dataset is the main factor driving performance of the downstream OOD detection method.
Single sample (B = 1) performance for both methods was lower for models trained on the semantically diverse datasots CIFAR-10 and ImageNet32. We mainly choose to include these datasets as in-distributions for consistency with prior work, and we would like question the implicit assumption made by these prior works that unsupervised method trained these datasets should consistently reject images from other
13

---

Published in Transactions on Machine Learning Research (10/2024)
natural image datasets. There is obvious, meaningful semantic boundary to distinguish natural image from CIFAR-10 and ImageNet.32, and thus it is not clear a even human would outperform random baseline.
As noted in $4 our method is similar to those presented in concurrent works Choi et al. (2021); Bergamin et al. (2022), we choose to use our method as a representative of this class of methods so that we may use our compute resources to robustly investigate their performance over such wide range of distribution pairings and models. We make no claim of superior performance over these methods.
6 Conclusion
We analysed an approximation to the Fisher information metric for OOD detection. Our work has two key limitations:
First, while we have provided the most extensive empirical benchmark of deep generative models, OOD and in-distribution datasets, datasets beyond images and for instance large language models should be tested. Second, while we focused on comparing it to the best performing, model-agnostic, hyperparameter-fuee OOD method, further empirical benchmarking against other methods should be conducted. Future work should investigate other, potentially more computationally expensive methods for approximating the Fisher information metric and its in OOD detection.
Acknowledgements
Sam Dauncey acknowledges support from the University of Oxford Statistics Summer Rescarch Internship programme.
Chris Holmes acknowledges support from the Medical Research Council Programme Leaders award MC _UP UP_A390 A3.0__11100000 1107, The Alan Turing Institute, Hcalth Data Research, U.K., and the U.K. Engincering and Physical Sciences Research Council through the Bayes4Health programme grant.
Christopher Williams acknowledges support from the Defence Science and Technology (DST) Group and from a ESPRC DTP Studentship.
Fabian Falck acknowledges the receipt of studentship awards from the Health Data Research UK-The Alan Turing Institute Wellcome PhD Programme (Grant Ref: 218529/7/19/7), and the Enrichment Scheme of The Alan Turing Institute under the EPSRC Grant EP/N510129/1.
We thank Joost van Amersfoort for his code repository and useful discussions.
References
Shun-ichi Amari. Natural gradient works efficiently in learning. Neural Computation, 10(2):251-276, 1998.
M. S. Bartlett and D. G. Kendall. The statistical analysis of nariance-heterogenetty and the logarithmic transformation. Supplement to the Journal of the Royal Statislical Sociely, 8(1):128 138, 1946.
Christoph Baur, Stefan Denner, Benedikt Wiestler, Nassir Navab, and Shadi Albarqouni. Autoencoders for unsupervised anomaly segmentation in brain mr images: comparative study. Medical Image Analysis, 69: 101952, 2021.
Sima Behpour, Thang Doan, Xin Li, Wenbin He, Liang Gou, and Liu Ren. Gradorth: A simple yet efficient distrf-distribution detection with orthogonal projection of gradients, arXiv, 2023.
Fedorico Bergamin, Picrro-Alexandre Mattei, Jakob Drachmann Havtorn, Hugo Senctaire, Hugo Schmutz, Lars Maaloe, Soren Hauberg, and Jes Frellsen. Model-agnostic out-of-distribution detection using combined statistical tests. In International Conference on Artificial Intelligence and Statistics. PMLR, 2022.
Christopher M Bishop. Novelty delection and neural network validation. IEE Proceedings Vision, Image and Signal processing, 141(4):217 222, 1994.
Avrim Blum, John Hopcroft, and Ravindran Kannan. Foundations of Data Science. Cambridge University Press, 2020.
14

---

Published in Transactions on Machine Learning Research (10/2024)
Anthony L. Caterini and Gabriel Loaiza-Ganem. Entropic issues in likelihood-basee OOD detection. In Melanie F. Pradier, Aaron Schein, Stephanie Hyland, Francisco J. R. Ruiz, and Jessica 7. Forde (eds.), Proceedings on 'I (Still) Can't Believe It's Not Better!" at NewrIPS 2021 Workshops. 13 Dec 2022.
Hyunsun Choi, Eric Jang, and Alexander A Alemi. Waic, but why? generative ensembles for robust anomaly detection. arXiv preprint arXiv:1810.01:: 01000999, 2018.
Jaemoo Choi, Changyeon Yoon, Jeongwoo Bae, and Myungjoo Kang. Robust out-of-distribution detection on deep probabilistic generative models, arXiv, 2021.
Thomas M. Cover and Joy A. Thomas. Asymptotic Equipartition Property, chapter 3, pp. 57-69. John Wiley & Sons, Ltd, 1991. ISBN 9780471748823..
Antonio De Maio, Steven M. Kay, and Alfonso Farina. On the invariance, coincidence, and statistical equivalence of the girt, rao test, and wald test. IEEE Transactions on Signal Processing, 5844)11967,1979, 2010.
A. Fisher. Statistical methods for research workers. Oliver and Boyd, 1938.
Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky. Domain-adversarial training of neural networks. Journal of Machine Learning Research, 17:1-35, 2016.
Saurabh Garg, Nick Erickson, James Sharpnack, Alex Smola, Sivaraman Balakrishnan, and Zachary C Lipton. Risbench: Domain adaptation under relaxed label shift. arXiv preprint Xiv:22300 030200, 2023.
Thomas George, César Laurent, Xavier Bouthillier, Nicolas Ballas, and Pascal Vincent. Fast approximate natural gradient descent in kronecker factored eigenbasis. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Garnett (eds.), Advances in Neural Information Processing Systems. 2018.
Mark S. Graham, Walter H.I. Pinaya, Potru-Daniel Tudosiu, Parashkev Nachev, Sebastion Ourselin, and Jorge Cardoso. Denoising diffusion models for -iittriiiitinnnnoonn detection. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Paltern Recognition (CVPR) Workshops, June 2023.
Jakob D. Havtorn, Jes Frellsen, Seren Hauberg, and Lars Maaløe. Hierarchical vaes know what they don't know. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning. 18-24 Jul 2021.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference Learning Representations, 2017.
Dan Hendrycks, Mantas Mazeika, and Thomas G. Dietterich. Deep anomaly detection with outlier exposure. CoRR, abs/1812.0400 044666 2018.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
Conor Igoe, Youngseog Chung, Ian Char, and Jeff Schneider. How useful are gradicnts for ood dctection really?, arXiv, 2022.
Dioderik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In Proceedings of The 33rd International Conference on Machine Learning, 2014.
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1xl convolutions. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (ods.), Advances in Neural Information Processing Systems. 2018.
15

---

Published in Transactions on Machine Learning Research (10/2024)
Polina Kirichenko, Pavel Izmailov, and Andrew G Wilson. Why normalizing flows fail to detect out-of- distribution data. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems. 2020.
Gukyeong Kwon, Mohit Prabhushankar, Dogancan Temel, and Ghassan AlRegib. Backpropagated gradient representations for anomaly detection. In Proceedings of the European Conference on Computer Vision (ECCV), 2020.
Charline Le Lan and Laurent Dinh. Perfect density models cannot guarantee anomaly detection. Entropy, 23 (12), 2021.
Erich Leo Lehmann, Joseph P Romano, and George Casella. Testing statistical hyppotheses, volume 3. Springer, 1986.
Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of dutto--distri image detection in neural networks. In International Conference on Learning Representations, 2018.
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. basergy-based out-of-distribution detection. Advances in neural information processing systems, 33:21464-21475, 2020.
James Martens. New insights and perspectives on the natural gradient method. Journal of Machine Learning Research, 21(146):1-76, 2020.
Warren Morningstar, Cusuh Ham, Andrew Gallagher, Balaji Lakshminarayanann, Alex Alemi, and Joshua Dillon. Density of states estimation for of distribution detection. In Arindam Banerjee and Kenji Fukumizu (eds.), Proceedings of The 24th International Conference on Artificial Intelligence and Statistics. 13 15 Apr 2021.
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't know? In International Conference on Learning Representations, 2019a.
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, and Balaji Lakshminarayanan. Detecting out-of- distribution inputs to deep generative models using typicality, arXiv, 2019b.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2015.
Quoc Phong Nguyen, Kar Wai Lim, Dinil Mon Divakaran, Kian Hsiang Low, and Mun Choon Chan. Geo: A gradient-based explainable variational autoencoder for network anomaly detection. In 2019 IEEE Conference Communications and Network Security (CNS). IEEE, 2019.
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshmi- narayanan. Normalizing flows for probabilistic modeling and inference. The Journal of Machine Learning Research, 22(1):2617 2680, 2021.
C. Radhakrishna Rao. Large sample tests of statistical hypotheses concerning several parameters with applications to problems of estimation. Mothematical Proceedings of the Cambridge Philosophical Society, 44(1):50-57, 1948.
Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distributio detection. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems. 2019.
Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P Kingma. Pixelenn+ Improving the pixelenn with discretized logistic mixture likelihood and other modifications. or Xio preprint ar 1rXi::1701.06a 770.055177 2017.
16

---

Published in Transactions on Machine Learning Research (10/2024)
Robin Schirrmeister, Yuxuan Zhou, Tonio Ball, and Dan Zhang. Understanding anomaly detection with deep invertible networks through hierarchies of distributions and features. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems. 2020.
Joan Serrå, David Ålvarez, Viceng Gómez, Olga Slizovskaia, José F. Núñez, and Jordi Luque. Input complexity and distri-iiittriiutiin detection with likelihood-basee generative models. In International Conference Learning Represenlations, 2020.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya. Ganguli. Doep unsupervised learning using nonequilibrium thermodynamies. In Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, Lille, France, 07-09 Jul 2015a. PMLR.
Jascha Sohl- Dink-iick,,ein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using noncquilibrium thermodynamics. In International conference on machine learning. PMLR, 2015b.
Jack Stilgoe. Who Killed Elaine Herzberg?, pp. 1-6. 6. Springer International Publishing, Cham, 2020. ISBN 978-3-030-32320-2.
Christian Szegedy, Wojeiech Zaremba, Tla Sutskover, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Pergus. Intriguing properties of neural networks. January 2014. 2nd International Conference on Learning Representations, ICLR 2014; Conference date: 14-04-2014 Through 16-04-2014.
Tijmen Tieleman, Geoffrey Hinton, al. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning, 4(2):26-31, 31, 2012.
Dennis Ulmer, Lotta Meijerink, and Giovanni Cinà. Trust issues: Uncertainty estimation does not enable
reliable ood detection on medical tabular data, In Emily Alsentzer, Matthew B, A. McDermott, Fabian Falck, Suproteem K. Sarkar, Subhrajit Roy, and Stephanie L. Hyland (eds.), Proceedings of the Machine Learning for Health NewrIPS Workshop. 11 Dec 2020.
Aaron Van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Conditional image generation with pixelenn decoders. Advances in neural information processing systems, 29, 2016.
Zhisheng Xiao, Qing Yan, and Yali Amit. Likelihood regret: An out-of-distributiio detection score for variational auto-encoder. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Newral Information Processing Systems. 2020.
Lily Zhang, Mark Goldstein, and Rajesh Ranganath. Understanding failures in out-of-distribution detection with deep generalive models. In International Conference Muchine Learning. PMLR, 2021a.
Mingtian Zhang, Andi Zhang, and Steven McDonagh. On the didstiibbiiiitoo generalization of brobabilistic image modelling. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems. 2021b.
Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization in vision: A survey. Xixiv preprint 2r00v:2003. 02503, 2021.
17

---

Published in Transactions on Machine Learning Research (10/2024)
Appendix for Approximations to the Fisher Information Metric of G Models for Out-Of-Distribution Detection
Deep Generative
A Proofs and additional theoretical results
A.1 Proof of proposition 1
Proposition 1. Let (x) and p4 (t) be two probability density functions corresponding to the same model distribution od being represented on two different measure spaces X' and T. Suppose these representations encode the same information, i.e. there exists a smooth, invertible reparameterization T' X + T such that for x E &' and E T representing the same point we have T(x) = t. Then, the gradient vector Vo is invariant to the choice of representation, and particular, Ta(logp))( = Va (x)))
Proof. Via the change-of-variables formula, we obtain
0T
(t) (x)
dx
Applying the logarithm both sides provides
81-1
(t) = 1oggx log
dx
and hence (o)logr+)()) (o(log(()x)) as required.
The smoothness assumption could be relaxed by considering the pull-back measure 1& o T I and the corresponding change-of-variable formula for Radon-Nikodym derivatives, however we omit this for brevily and relevance. This result also trivially extends to the likelihood proxy we use for diffusion models logp" (x) x1).
A.2 Representation-llvarinne of variational lower-bound gradients
Assume the same setup as in A.1, but this time with a variational Bayesian method such as a Variational AutoEncodor Kingma Welling (2014) with latent variable given by 2, decoder probability density p 22) and encoder probability density q (z(x), noting that the decoder probability density is that which depends X. The Evidence Lower Bound the log-likelihodd px())) is given by
E. X (x) (zlg)))))) log (x,
(z)())))
We can then state similar representation invariance for the ELBO.
Proposition 2. Let ELBO& be the ELBO of VAE, and let ELBO (t) be the ELBO under change of variables with invertible mapping T X + T, corresponding to two sets X' and T. Then, the gradient 0 Vo(ELBBO (t) is invariant to T.
Proof. Noting that
(t, = (x,
#T dm
while
(=(1)) = (2(x)
3 gives that:
JT
ELBO ELBO (x) + log dx
and taking the gradient wrt and & gives the result that the gradient of the ELBO with respect to the VAE's parameters is iepresentationninaariae
3To those familiar with the Borel Kolmogorov paradox this condition may seem non-obvious, but can derive it from the fact that T' does not require input from s, thus (x(()) (a())))
q (x..)
or-1
9 (2/x)
(1)
(x)
e5-1
18

---

Published in Transactions on Machine Learning Research (10/2024)
A.3 Lebesgue measure of set of bounded total variation
Proposition 3. For a E RR, define the lotal variation to be (V:=) = |0i| Σi=las xx-il. Let B(a) be the set of d-length arrays whose total variation is bounded by a:
E(a)= = ( E R (V < <a..
The Lebesgue measure of this set is given by F(()) (2n)
I[d+1)"
Proof.
Consider the volume-preserving transformation (I1, 22 va) + (x), 12,,12....e ta). where 1, = 24; -4-1- We thus see that the volume of E(a) is equivalent the volume of the d-ball in the l' -mtric, with standard result:
(2a)d ((((a)) = ....dd))). |ana [12) + [tal] < < I(d 1)
Application to MNIST We can naively apply this result to MNIST images y [0, 1] by set- ting d 28 and drawing snake pattern through our images illustrated in Figure 7, setting Vij ==
see that a = 102.9 is sufficiently large such that the whole MNIST dataset is contained in E(a). which we can compute has an approximate measure of F(a)) x 10-116.76 < 10-116, Note that this is not the tightest bound one could give; for example vertical variations are neglected and membership E(a) does not restrict x; from drifting outside the set [0, 1].
Computing this numerically for the whole MNIST dataset,
5
Figure A visual illustration of the snake pattern used to unravel an MNIST image y E [0, 1]28x28 into a string of values [0, 18781
A.4 (Weak) parameterisation in-variance of method
Let O, & be two parameter spaces of the same model p, linked by the smooth invertible reparameterisation P θ + 4, such that for & P(0) we have p" = p*. In this setting, one can derive that the Fisher Information Metric Radhakrishna Rao (1948) is invariant under P, ic that for all æ1 D2 E we have Vol(=;) Vol(xz) = Fo Val((z)) (see (2) for our notation). As we merely approximate the FIM in our method we cannot make the same guarantee for all P, we can however prove similar result P linearly rescales the layers:
Proposition As $3.4 Let 0, 0,. O, be the layers of our model. P: + be a smooth invertible reparameterisation of our model which linearly rescales the layers, ie P(0, 1.0.02.... ........ = d,d,,d262... d262...d. d, for some constants d1, dz...dju R. Then, the resulting anomaly score of our method is invariant under P.
19

---

Published in Transactions on Machine Learning Research (10/2024)
Proof.
Using the same notation as in $3.4, let ff... ....ff. fJ and fi... under θ and & respectively (see equation 4). Then, for all datapoints a and layers j we have:
ff ...ff be our layer-wise gradient L norm features
(x) [o, |d, v (x)) V (z)
(5)
Taking the logarithm and writing in vectorized form gives that:
log f (x) = log d + log f (x)
(6)
In particular, if we let 1 11" and 29 o² 24 be the corresponding sample mean and variances for θ and & in algorithm 3, we see that f log + n and α29 20 o². Hence via translation invariance of the normal distribution, our metric will be invariant under P.
A.5 Comparison classical invariance properties
In classical statistics, there is separate and notion of invariance that is incompatible with that proposed by Le Lan & Dinh (2021). The setup proposed by Lehmann et al. (1986) is one in which consider group of transformations from the input space to itself X - X which are sufficiently narrow such that there is corresponding group of transformations to the parameter space 0 - that counteract the effect of the transformation, formally defined by:
p&e P 0
(7)
with the analogy to the change-of-varian formula being that:
Ogæ
(gx) = 1x
(x)
(8)
dx
One example of where this setup is applicable is applying dilations to input space of multivariate normal distribution, whereby any lincar dilation of the input space can be counteracted by dilation of the covariance matrix. This is not the case for the arbitrary transformations f considered in Proposition of Le Lan & Dinh (2021), which we cannot guarantee to be counteracted by some transformation of a generative model's parameters. Even the simple example of the non-linear RGB-HSV transformation we give in $2.3 can only approximately be countcracted by changing the generative model's parameters. In contrast, the setup proposed by Le Lan & Dinh (2021), implicitly considers transformations from arbitrary measure spaces f X + f(X), and considers the pullback:
Pd f(X) Po f-1
The presence of the counteracting parameter transformation in equation (7) leads to score vector (oquation (6) of De Maio et al. (2010)):
03-10 term in the
Vo(logp&) = Ve 2 '0 )xz) = a0
0g-18 V (o(logg2)
(10)
Nonetheless, De Maio et al. (2010) derive that the score test statistic is invariant under this setup too. This
is a consequence of the score test statistic being both invariant in the setup of Le Lan & Dinh (2021) and
satisfying strong paramoterisation invariance, which algebraically expressed by the 0g-10 being cancelled in re-parameterisation of the FIM.
80 term in eq (10)
20

---

Published in Transactions on Machine Learning Research (10/2024)
B
Additional experimental details and results
B.1
RGB-HSV representation dependence
0.43
1.01
0.75
0.70
0.50
0.18
0.60
0.58
0.45
0.62
0.39
0.94
0.65
1.76
0.69
0.37
0.87
0.68
0.98
0.37
Figure 8: The log- likelihood heavily depends on data representation We extend Figure to the first 20 examples of the CIFAR10 dataset and their values of A BPD RGB-HHVV defined in Eq. 11. We note values of A RGB-HSS between 0.18 1.76, indicating large difference of the induced change in likelihoods.
BPD
Here compute the change in Bits Per Dimension (BPD) A RGB-HHSV for the first 20 samples of the CIFAR10 test dataset, defined as:
BPD
dussv log, (RCB(X) - log PHSV 3 x 32 x 32
RGB HSV BPD
logg d
(11)
3 x 32 x 32
where μHSV is the Lebesgue measure in HSV-space, PRGB is the Lebesgue measure in RGB-space, and PHsv and PAGB are corresponding probability density functions for any distribution defined over the set of images. We compute the Radon-Nikodym derivative duнav in 11 by computing the pixel-wise Jacobian dpnce determinants of the RGB-HSV transformation TRGB-HSV R - R² In order make the comparison fair, dequantaize pixel Tij 3 add a small amount of normally distributed noise eg NN(0, I3x3), ie set. ăÚ Zz + 255 We then note that the full RGB-HS CSV transformation factors RGB-HE HSV transformation across the pixels, and thus its Jacobian determinant factors as:
fy
dpHsv log dpnGB
log
151,533
21

---

Published in Transactions on Machine Learning Research (10/2024)
B.2 Replications of Fig. 4
In figures 9- 11 provide robust replications of Figure 4 using randomly chosen layers. The layers sorted with the seght layer being the "deepest' (ie. the closest to the latent variables). We observe that the gradients are more separated for models trained on the semantically distinct datasets SVHN, CelebA & CTSRB, mirroring the superior performance our method achieves in these cases.
Please note that large parts of the gradient distributions from the OOD datasets have been cropped out keep the plots legible.
B.2.1 Glow models
didiiiyttoon 9911
giab buttan butann imageraci3 dutrrattion
2ibe mredel
20
20
10
-22
-10
corigeruss
1ig (g2
log 1g
bg
out
nuthot vi-eee buriiuttion eeleka distibuttion gtaro
out of distr button citrra out of sioar button Imagonetse fitted dierribution
-100 -75
-50
-3.
3%
5.0
bgb
out of dist bution siln nutall-dion celebx
nubsok-tier button gren diennbutton
aub-ah-deer button imagenect? detribstion
32
12
-30
-7
33
-30
-7.
-20
-15
-3.0
-05-7
logie
log 1opilb
80% STAAA imaggeeeet
lng (ptus)
bg/6 52
Figure 9: Replication of Fig. for 4 randomly selected layers out of 1353 from Glow models trained on SVHN, CelebA, GTSRB, CIFAR-10 and ImageNett3 respectively
22

---

Published in Transactions on Machine Learning Research (10/2024)
B.2.2
Diffusion models
diz attte wit1
nibak-dadr bustor gron
auboh--ddh eitrr1 out button imagenet1? detribaiio
15
14
11
12
-11
-10
-9
-16
-12
-10
22
21
20
-3
-1
-17
-15
-5
log (Ruu))
ki 1g
Ing 6/10
bg fe
rroade Ioweree ghots
out-et
ust of dit buiion udlidar desiibattoo gosbb ousteel -ia butann cltarnn outbot distr bution imagenett2 tttedd dittibubton
-13
-10
-4
-0
-11
-10
-20
-26
-24
-22
bot
Olttesn moncee aRrv C387313
Gut of dizor bution sohn out button diemibateo cherld
caleba
out imagerac3) dutribation
-20
-34
-52
-20
-10
-32
-33
-22
20
-0
-7
-30
-19
-1.
-37
-16
bale/K Jl
lng (y
Ing (6
imagerecc
1og
Figure 10: Replication of Fig. 4 for 4 randomly selected layers out of 276 from Diffusion models trained on SVHN, CelebA, GTSRB, CIFAR-10 and ImageNet.32 respectively
23

---

Published in Transactions on Machine Learning Research (10/2024)
B.2.3 VAEs
We note generally less separation with our VAE models, mirroring the poorer performance we attain with them in appendix E
dizzibuttoon ganin
aitheb dette buttan calada
altteh cel-ddtrrbuuttoo button gitrb
out of dett button citar10
out of dber button imaqeract))
Ritee diatrbutiinn
10.7
15.0
17.5
141
20
50
1.
20
Ing (n
out of diior bution pihn
imagenati??
distribation
20
30
0.0
2.5
3.0
7.5
00.0
12.5
15.0
-30
-20
-10
30
leg
eggetl +1
VIB T23M tranved grare
of coleba distribution
dienrioudinn
3.2
bg
out of
out-eit dian butlan celeta
nut-illoionhttteee gront. dleriiiutinn ottrld
duthoh dittr button imagenet3? Riteed dietmbution
2.5
10.0
1/5
15.5
50
13
14
06
10
1/
15
balk/x.
wot-cfidkrrbbarioo cutn
out-o cecoo--dtttiubbee caleba
disttiiiiiinn glart
detripuiion crant
THIO boarsed D0
distabution smapenaiaa
10
-3
20
0.0
23
30
73
10.0
22.5
13.0
37.3
20
5)
kog (y(a2.))
10g
Figure 11: Replication of Fig. for 4 randomly selected layers out of 48 from models trained SVHN CelebA, GTSRB, CIFAR-10 and ImageNet32 respectively
24

---

Published in Transactions on Machine Learning Research (10/2024)
B.3 Additional plots of the FIM
B.3.1
Windows into the FIM of Glow model
layer
layer
layer
1.00
0.75
layeri
0.50
0.25
VF FooF BB
Fag
layer
-0.25
-0.50
-0.75
layer
-1.00
Figure 12: The strong diagonal of the FIM for glow models replicate Fig. 5 using three more randomly selected layers from our Glow model trained on CelebA.
B.3.2 Windows into the FIM of diffusion model
layer
layer
layer k
layer
0.50
0.25
layer
Fo FB
0.00
FaB
-0.25
0050
layer k
-0.55
-1.00
The strong diagonal of the FIM for diffusion models. We replicate Fig. with a diffusson model. As before, we randomly select two layers from a diffusion model trained on CelebA and plot the an approximation of the FIM, this time using the gradients of the one-step log-likelihood. Note that the first plot. its entire layer-wise FIM. Again, we normalise the rows
Figure 13:
layer selected has fewer than 50 weights, and columns by the diagonal values VFo to enable layer comparison.
25

---

Published in Transactions on Machine Learning Research (10/2024)
B.3.3 Raw, laygee-layer FIMs of Glow models
30310
10185
-3 10000
0,0010
4,001
042315
23033
0.03139
-000055
20 09099
2.0003
-.0
10-
0.0020
1.
000050
3835
00003
00.0075
500
8,0005
235
0.0003
-25
0. 200000
D100
-3.000
-0137
-3.00
-0033
Figure 14: Raw layer-wise FIMs for glow models. For each row of plots, we randomly select layers 0, from glow models trained on (going from top bottom) SVHN, CelebA, GTSRB, CIFAR-10 and ImageNet32. We then plot the raw FIM Fas values for max(50, (@)) weights in these layers, using separate colorbar per layer to account for the fact that the absolute sizes of the FTM elements vary by orders of magnitudes from layer to layer.
26

---

Published in Transactions on Machine Learning Research (10/2024)
B.3.4 Raw, langee-layer FIMs of diffusion models
16-6
16-25
30-10
36-15
425
10
00
-1.4
-3.5
3e-4
16-11
D1EETS
510010
650
-025
-0 00005
50000
1060
15-15
-0.55
Figure 15: We replicate Figure 14 with a diffusion model (again using the gradient of the 1-timestep variational lower bound as stand-in for the gradient of the log-likelihood), noting qualitative difference in the appearance of the layers.
27

---

Published in Transactions on Machine Learning Research (10/2024)
B.4 Ablation study likelihood proxies for diffusion models
In this section, we discuss the use of different parts of diffusson varialional lower bound for anomaly detection. Before doing so, for completeness we present condensed version of the theory presented in Ho et al. (2020), using the same notation.
B.4.1 Derivation of diffusion process
Let our forward process be (( with prior of the true image distribution g(æo) and our learned reverse process be (2z) with Gaussian prior p(ær) ~ NN,, I). A variational lower bound on the model log-likelihood of can be derived Ho et al. (2020) as:
E - log p0 (2o)) S 1E|T + Σ 4- + Lol
(12)
log (20)201)
where =
0 <<<< T
(xx (x++)))
(13)
t t=T
KL(g(or(2o)lp((rr)
Ho et al. (2020) also derive that, if we parameterise our forward process as adding some normally distributed
noise to It-1; and our reverse process as normally predicting this noise from a via a network e (a 1), for < T, L, can be computed a squared error:
EL,-1 ((Em,)((xo) ||e ex t)] + Cr.
(14)
Here kt and Ct are constants independent of æo, .1... 2T and 0, which thus be omitted from computations. To compute the expectation, we use one sample from the reverse process ~ g(zo), motivated by our findings in section B.4.4 which show little to no performance gains from using five samples. Thus, we define our set likelihood proxies as (14) as L((x) = |e - e (II+1,t + 1)]l for a single sample of noise e from the reverse process Zt+1 ~ ((x). Note that computing L, (x) requires only one pass through the network, making it very efficient compute. section B.4.3 we do an ablation study on the value of used, motivating our choice of Lg in the application our method.
B.4.2 On Representation Dependence in Diffusion models
When considering Le Lan & Dinh (2021)'s results pertaining to representation dependence the context of diffusion models, we arrive at the interesting question as whether the choice of representation should be considered to affect the underlying distribution of the forward process q. Clearly the value use in our method, Lu = p (zo]æ11, is representation-depeneen In the strict sense, the values Lę for t 0 aren't 't representation dependent, unless representation dependence is also considered to affect g, in which case this becomes more ambiguous. In figure 16 we report the negative result that the values of L; for > 0 also follow the pattern from Nalisnick et al. (2019a), whereby structured OOD data has higher values for L. We defor further debate on this issue to future work.
B.4.3 Ablation study on the value of t used for anomaly detection with diffusion models
In this section, we evaluate using different values of t for the likelihood proxy 1(x) = L,-1(x) which we use input for our anomaly detection method and lypicality Nalisnick et al. (2019a). We summarise our results in Figure 17 by plotting the average AUROC acheived for each method across all 20 dataset pairings. In table we provide more granular results with the AUROC for each pairing individually. We note the intuitive result that the performance our method gradually decays as t increases, corresponding to more noise being added to the sample fed into the network. Overall, the average performance of method at / is higher than
28

---

Published in Transactions on Machine Learning Research (10/2024)
Oiffuston Model
DiMuuion Moeel
50N
SVHN
-2.35
-0.43
-0.33
-0.30
-0.23
-4.2
-0.35
-022
-0.20
-0.13
-0.18
-0.24
-0.22
-2.30
-0 00
-0
CelebA
0.30
0.43
0.90
0.59
0.00
0.2
0.20
0.15
0.22
0.20
0.19
0.59
0.14
0.12
2:30
018
0.5
GTSRB
-0.30
-0.45
-0.25
-1.20
-0.35
-0.0
-0.18
-0.14
-0.12
-0.10
00
CLFAR- 10
-0.50
-020
-0.8
-2.30
1 (1:)5
1111(X1 2)
Figure 16: Lt-1 follows the pattern from Nalisnick et al. (2019a) for voriety of t values. We replicate figure for == 64 [Left] and 512 [Right] using batch size (we use this batch size for reasons of limited compute.)
Ablation study effect of the number timesteps the performance of a diffusion model using batch size 5.
0.88
ours
typicality
0.86
0.82
0.80
AUROC
32
64
128
256
512
timesteps
Figure 17: For diffusion models, L,-1 is most informative for anomaly detection for low values of We compute the AUROC values for all in/out dataset distribution pairings using 2n for n 0.1...0 . and batch size B (for reasons of limited compute).
that of typicality which achieves its maximum performance 32 (out of our model's maximum timestep of T 1000). To compute requirements, batch size for all experiments.
29

---

Published in Transactions on Machine Learning Research (10/2024)
SVHN
CelebA
GTSRB
CIFAR-10
ImageNet332
SVHN
0.9937
0.7122
0.9975
0.9969
typicality
CelebA
1.0000
0.8816
0.3443
0.5904
GTSRB
0.9990
0.8108
0.6680
0.7214
CIFAR-10 10
1.0000
0.8684
0.9172
0.4852
ImageNet32
1.0000
0.9869
0.9845
0.8800
SVHN
1.0000
0.9833
0.8214
0.9838
CelebA
0.9932
0.9284
0.8250
0.3942
(t= tt= 8)
GTSRB
0.9822
0.9998
0.7025
0.7485
ours
CIFAR-10
0.9940
0.9998
0.8750
0.5071
ImageNet32
0.9934
1.0000
0.9624
0.9046
SVHN
0.9813
0.4779
0.9970
0.9982
typicality
(t = 64)
CelebA
1.0000
0.9746
0.3288
0.6622
GTSRB
0.9966
0.7805
0.6914
0.8543
CIFAR-10
1.0000
0.9228
0.9798
0.5388
ImageNet32
1.0000
0.9907
0.9965
0.8059
SVHN
1.0000
0.9801
0.9345
0.9603
(t -t= 64)
CelebA
0.9786
0.8856
0.6389
0.5063
GTSRB
0.9762
0.9990
0.7842
0.7150
ours
CIFAR-10
0.9825
0.9996
0.7911
0.5039
ImageNet32
0.9850
0.9999
0.9255
0.8001
SVHN
0.7125
0.5117
0.9574
0.9839
typicality
=t= 512)
CelebA
0.9997
0.9984
0.3592
0.4543
GTSRB
0.9549
0.7854
0.7207
0.8194
CIFAR-10
0.9997
0.9815
0.9971
0.5154
ImageNet32
0.9998
0.9908
0.9984
0.6338
SVHN
0.9962
0.9712
0.8089
0.7857
(t = 512)
CelebA
0.7654
0.9396
0.5205
0.5272
GTSRB
0.6870
0.9496
0.7770
0.6256
ours
CIFAR-10
0.6295
0.9844
0.9326
0.4660
ImageNet32
0.6635
0.9757
0.8904
0.5432
Table 3: aue values for typicality [top] and ours [bottom], batch size 5 applied diffusion models for varied timesteps t 8, 64, 512
B.4.4 Ablation study on multiple q-samples for anomaly detection with diffusion models
In this section, we investigate if any performance improvement can be achieved by using multiple samples from to estimate Top ie I(x) = 1(xg) = Eg B (ea) log (x [x TE x (g ||e (x1, 1)||- Specifically,
we take n q-samples (1)) ... (5)) x1 I.. to define our likelihood proxy as:
(1) (5)
((x)
Σ
(
5
The AUROC values using batch size for our method and typicality Nalisnick et al. (2019b) are in table 4. We note little no performance gain for our method or typicality, motivating our use of q-sample in our implementation for efficiency.
30

---

Published in Transactions on Machine Learning Research (10/2024)
SVHN
CelebA
GTSRB
CIFAR-10
ImageNet332
SVHN
0.9251
0.9668
0.9824
0.9926
typicality
CelebA
0.9981
0.6328
0.5312
0.5500
GTSRB
0.9961
0.6768
0.6291
0.4716
CIFAR-10 10
0.9685
0.7890
0,4130
0.7174
ImageNet32
0.9959
0.3954
0.6098
0.7923
SVHN
0.9984
0.9930
0.9873
0.9756
CelebA
0.8938
0.9746
0.8140
0.6952
OnTS
GTSRB
0.8222
0.9823
0.9367
0.8728
CIFAR-10
0.9683
0.9744
0.8922
0.5666
ImageNet.22
0.9797
0.9793
0.9185 88
0.7485
Table 4: auc values for typicality [top] and ours [bottom], batch size applied to diffusion model using 5 q-samples
average performance for typicality: 0.7617, 25/50/75 quantiles: 0.6062 / 0.7532 / 0.9720
average performance for ours: 0.8987, 25/50/75 quantiles: 0.8601 0.9525 / 0.9794
B.5 Using Fisher's method in the place of density estimation
In this section, briefly investigate the use of Fisher's method Fisher (1938) to compute the final anomaly score when using the gradient -o-m statistics fe which we define in $3.4. Specifically, if modify our method by defining ('(x) (f(ff()))) - & (f(())) to be the b-th p-value from a two-tailed z-test and our final anomaly score as:
S Σ log(a (x)).
In table 5 for brevity we only report our results for a Glow model with B 1, but note the same pattern across all models. We note small performance detriment across all dataset pairings from using Fisher's method, motivating our use of density estimation.
SVHN
CelebA
GTSRB
CIFAR-10
ImageNet32
SVHN
0.9808
0.9494
0.8358
0.7514 4
CelebA
0,9633
0.8125
0,5022
0,2686
(Fisher)
GTSRB
0.9321
0.9772
0.7016
0.4482
ours
CIFAR-10
0.9398
0.9250
0.8119
0.4203
ImageNet32
0.9899
0.9734
0.9165
0.6969
SVHN
0.9880
0.9858
0.8747
0.8010
ours (density)
CelebA
0.9823
0.9262
0.5155
0.2997
GTSRB
0.9537
1.0000
0.7546
0.9967
CIFAR-10
0.9658
0.9462
0.9126
0.4377
ImageNet32
0.9976
0.9876
0.9683
0.7375
Table 5: aue values for ours (Fisher's method) [top] and ours (density estimation) [bottom], batch size
applied to glow
average performance for Fisher's method: 0.7898,
25/50/75 quantiles: 0.7004 / 0.8761 / 0.9529
average performance for density estimation: 0.8516,
25/50/75 quantiles: 0.7894 0.9500 0.9863
31

---

Published in Transactions on Machine Learning Research (10/2024)
Method
MNIST
Omniglot
WAIC
0.766
0.796
S using +ixeICNN++ and FLIF
0.967
1.000
PixelCNN Gradient norms (OneClassSVM) (ours)
0.979
1.000
S using Glow and FLIF
0.998
1.000
Glow Gradient norms (OneClassSVM) (ours)
0.819
1.000
Table 6: Table of results comparing the performance of our method on for a model trained on FaSH at detecting OOD grayscale images to the performances the S-score reported in Serrà et al. (2020) and Akalnne Information Criterion reported Choi et al. (2018)
C Supervised gradient-based methodology for classifiers
For completeness, discuss classifier based OOD detection methods using the gradient, noting that these methods are given label-information at train time and our pnpresentationninvvriian result does not directly translate over to this paradigm.
In order to compute gradients without a target label, This approach hence supervised with respect to in-distribution and OOD labels which it requires. Liang et al. (2018) propose a method called ODIN which uses the gradient with respect to the data: They backpropagate gradients the input data see how much an input perturbation can change the softmax output of classifier, following the intuition that OOD inputs may be more sensitive and prone to larger variation in the output distribution. Igoe et al. (2022) are critical of the use of classifier gradients, instead advocating that most information can be recovered from the layer representations. Behpour et al. (2023) propose projection of the gradient onto the space generated by in-distribution gradients, motivated as in Kwon et al. (2020) by the informativity of the gradient angle for OOD detection.
32

---

Published in Transactions on Machine Learning Research (10/2024)
Figure 18: Samples from Glow models. Samples from the Glow models used in our Experiments, trained on SVHN, CelebA, GTSRB, CIFAR-10 and ImageNet:32 respectively from top to bottom.
5
Figure 19: Samples from diffusion models. Samples from the denoising diffusson models used in our Experiments, trained on SVHN, CelebA, GTSRB, CIFAR-10 and ImageNet32 respectively from top to bottom.
Code, models
Our Glow implementation derives from a repository https://github. con/yblast/@GooooPPyyorrc replicating the one used in Nalisnick al. (2019a), with the only difference being we use batch size of 64 in training rather than 512. See Figure 18 for samples from models.
Our diffusion model implementation derives from PyTorch transcription at https://github. com/ of that described in Ho et al. (2020). We train using Adam with a learning rate of 3e -4 for 10 epochs. Our model has 7 = 1000 timesteps which are uniformly sampled from in training and the U-Net backbone has dimension multiplicitics of (1, 2,2,) Soe Fig 19 for samples from our models.
iucidrainss/eeoiiinng-
Our code available with pre-trained models and pre-computed gradient 1 norms at https://github. com Hemm/SaDD77///Gerrato Modee Kodels-Knovleddee Knowledge.
33

---

Published in Transactions on Machine Learning Research (10/2024)
Figure 20: Samples from VAE models. Samples from the denoising diffusion models used in our Experiments, trained on SVHN, CelebA, GTSRB, CIFAR-10 and ImageNet32 respectively from top bottom.
E
Results for poorly performant VAE
test 4 truin +
SVHN
CelebA
GTSRB
CIFAR-10
ImageNet32
SVHN
0.5680
0.4302
0.5664
0.5933
typicality
CelebA
0.4569
0.3729
0.5085
0.4868
BB = 1)
GTSRB
0.5901
0.6290
0.6484
0.6061
CIFAR-10
0.4223
0.4742
0.3454
0.4922
ImageNet.32
0.4429
0.4831
0.3736
0.5047
SVHN
0.6693
0.5541
0.5904
0.5740
(B = 1)
CelebA
0.4329
0.4337
0.4685
0.4588
GTSRB
0.5920
0.6581
0.6570
0.6629
ours
CIFAR-10
0.4343
0.5826
0.5123
0.4864
ImageNet32
0.4582
0.5941
0.5048
0.5187
SVHN
0.9978
0.7943
0.9975
0.9961
typicality
B 5)
CelebA
1.0000
0.7642
0.3156
0.3621
GTSRB
0.9998
0.8336
0.6809
0.5765
CIFAR-10
1.0000
0.7808
0.8332
0.4488
ImageNet32
1.0000
0.9866
0.9675
0.9266
SVHN
1.0000
0.9970
0.8457
0.9561
CelebA
0.9908
0.8552
0.7734
0.4202
(Ba 5)
GTSRB
0.9716
0.9997
0.7325
0.9007
ours
CIFAR-10
0.9895
0.9992
0.8104
0.5733
ImageNet32
0.9862
1.0000
0.9309
0.8532
Table 7: VAE models Comparison of the AUROC values (larger values are better) of our method to the typicality test Nalisnick al. (2019b) for batch sizes B 1,5, We train VAE Kingma & Welling (2014) models on five natural image datascts (as columns) and evaluate the ability of the model-method combination to reject the other datasets (as rows).
Our VAE Kingma Welling (2014) implementation uses entirely convolutional layers, Figure we note that the samples produced approximate the colour palate of the train datasets well but have poor semantic coherence. In table note poor performance for both our methods using this models a backbone.
34


Reviews:
[{"review_id": "YRmNvXGJbi", "reviewer": "Reviewer_U1Sv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n\nThe idea of using the FIM to reweigh the individual gradients of the test points is very well motivated, both by compelling heuristics and by an invariance argument (Proposition 1). The authors perform lots of illustrative experiments that are sometimes insightful (e.g. Figs 5 and 6). To the best of my knowledge, they are also the first to conduct these kinds of experiments for diffusion models.\n\n# Weaknesses\n\nMy main concern with this paper is that the idea of using gradients weighted by the FIM, as well as most of the insights provided in the paper, seemed to be already put forward by Choi et al. (2021) and Bergamin et al. (2022), as acknowledged by the authors in Section 4. Choi et al. (2021) used a Bayesian motivation, and Bergamin et al. (2022) the point of view of statistical tests, but their final OOD scores are virtually identical to the one proposed in this paper (the only important difference being the way to approximate the FIM). Clarifying what the paper brings additionally to the table, compared to this prior works, would make it more useful. As I mention in the \"requested changes\" section, the invariance property should also be contextualized with respect to the literature.\n\n\n# Less important issues and unclear things\n\n- Introduction (and elsewhere) : the authors mention that the fact that models give higher likelihoods to OOD “marks an open problem for deep generative models“. However, as they briefly discuss in Section 2.2, the problem is not limited to deep generative models, and a simple (high-dimensional) Gaussian behaves exactly the same way.\n\n- Introduction : \"In §2.2) we will analyse the small amount of previous work on gradient-based OOD detection, linking seemingly independent discoveries of other authors.\" What links drawn here are novel? And why not cite Choi et al. (2021) and Bergamin et al. (2022) in this §2.2)? They are surely among the \"small amount of previous work on gradient-based OOD detection\".\n\n- Section 3.5 : The authors claim that diffusion models “do not allow for exact inference of the log-likelihood”. This is not entirely true, see for instance the ODE approach of Song et al. (2021)\n\n- Section 4 : “Approximating the FIM as a diagonal, with coefficients $F_{\\alpha \\alpha} = (\\delta_\\alpha \\log p_\\theta(x) )^2 + \\varepsilon$  . (…) We also note that layers whose gradient values are << 10−8 (see Appendix B.3 Figures 13 and 14) would have their information nullified without careful tuning of ε” Since 10-8 is added to the square of the gradient, shouldn’t that be 10^-4 ? Is that really source of concern? It would be interesting to do experiments with different approximations of the FIM.\n\n- Section 4: \"Nalisnick et al. (2019b) note that the Maximum Mean and Kernelized Stein Discrepancy tests they use to benchmark their typicality test are only effective when using the Fisher kernel Jaakkola & Haussler (1998); Jacot et al. (2018) $k(xi,xj) = \\nabla_\\theta l (x_i) \\nabla_\\theta l (x_j)^T$\". Nalisnick et al. (2019b) used a modified version of the Fisher kernel, that was actually $ \\nabla_x l (x_i) \\nabla_x l (x_j)^T$.\n\n- Section 4: \"Our method differs from using Fisher kernel in that it allows for information to be used from all the layers, rather than a few dominant layers\" With the standard Fisher kernel from Jaakkola & Haussler (1998), the FIM also appears, and makes sure that all weights are properly normalized. Moreover, Bergamin et al. (2022) showed that MMD with the Fisher kernel is essentially equivalent to using your criterion.\n\n- Appendix A.3: The authors could explain the Explain better the “snake pattern”. The formula is a bit cryptic. Plotting digits would be hepful.\n\n- Proposition 2: “then, ELBO is invariant”, shouldn’t it be “the gradient of the ELBO is invariant”?\n\n- Proposition 3: There is a “< alpha” missing, both in the equation of the Prop, and in the proof\n\n# Typos\n\n\nBeginning of Section 2.2: “negative leaned likelihood”\n\nTitle of Appendix 1.2: “variatonal”\n\n\n# Additional reference\n\nSong et al., Score-Based Generative Modeling through Stochastic Differential Equations, ICLR 2021", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a way to perform out-of-distribution (OOD) detection using a pre-trained generative model $p_\\theta$. To assess if a point $x$ is OOD, the key idea is to use the magnitude of the gradient of the log-likelihood $\\nabla_\\theta \\log p_\\theta(x)$. Instead of using the standard L2 norm, they argue (using both heuristics and a theoretical argument of invariance) that this magnitude should be measured using the Mahalanobis distance induced by the Fisher information matrix (FIM) of $p_\\theta$. They discuss ways of approximating the FIM, and use their score to perform OOD detection for diverse kinds of generative models: normalizing flows, diffusion models, variational auto encoders, autoregressive models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The authors should clarify what additional scientific insight their paper provide compared to Choi et al. (2021) and Bergamin et al. (2022). One of these differences is the invariance property (Proposition 1). They should clarify how it is related to classical invariance properties of likelihood ratios and scores (e.g. De Maio, 2000, and references herein). The authors should mention earlier in the manuscript that these earlier works used the FIM in the same way, albeit with different motivations (here, the first mention of  is on page 11). What new things do the experiments of the authors bring? How are their experiments different from prior work? If the FIM approximation is indeed the only important difference, maybe doing experiments on its influence could be interesting.\n\n\n# Additional reference\n\nDe Maio et al., On the Invariance, Coincidence, and Statistical Equivalence of the GLRT, Rao Test, and Wald Test, IEEE Trans. on Signal Processing, 2010", "broader_impact_concerns": "I have no particular concern.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CDdYctzTzR", "reviewer": "Reviewer_SnJ5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The paper makes a contribution of evaluating gradient-based methods for OOD detection on deep generative models, and connecting their approach to the Fisher Information Metric. \n- The experiments are comprehensive.\n- The paper is clearly written and easy to follow.\n\nWeaknesses\n- The method is model-agnostic, but it doesn’t work as well on diffusion models compared to the Glow model: for the diffusion model experiment, the proposed method is better only around half the time. In fact, this can be seen in the experiments in Appendix B.2.2 for the gtsrb, cifar10, and imagenet3 results: the intuition that the score is highly informative for OOD detection is not validated in these plots.\n- For diffusion models the one-step log-likelihood is used, which is very different from the actual log-likelihood. It would be helpful to have some exposition on how this affects the performance of the method, or whether there are other approaches to apply the proposed method to diffusion models. \n- Regarding novelty, as authors suggested, similar methods have been independently discovered in other works and other contexts. \n\nQuestion: what’s the relationship between the hat-value and influence function? Are they the same?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a method for OOD detection using likelihood-based deep generative models. Specifically, the method uses the layer-wise gradient norms as features, fits a Gaussian distribution, and uses the negative log likelihood of the fitted Gaussian as the OOD score. Even though likelihood-based deep generative models are designed to model the data distribution, prior work finds that they assign higher likelihood to OOD data. The proposed method builds on the intuition that if the model is fitted on some distribution of data, then seeing any OOD data should affect the model and induce a large gradient. \n\nThe method is theoretically motivated by the Fisher information metric, and can be seen as an efficient approximation of it. Authors conduct experiments for both the Glow model and the Diffusion model, and compare their method to the standard typicality test on a number of datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The following suggestions are not critical to my recommendation, but would strengthen the paper\n- Providing more robust justification for using the layer-wise gradient norm as a signal (see weaknesses)\n- Providing more justification for using one inference step on the diffusion model. How does this affect the performance of the method?\n- See question about hat-value.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "kbp4DRaN09", "reviewer": "Reviewer_BFSA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper provides a principled approach to out of distribution detection by generative models. It proposes to use size of gradient information to detect such samples. \n- Further, it provides an empirically efficient algorithm to approximate such high dimensional gradients via approximation of the Fisher information matrix by diagonally dominant matrices. This essentially boils down to looking at the l2 norms of the layer wise gradients. \n\nWeaknesses: \n- Providing a more principled definition of what OOD data would satisfy the assumptions of the paper would be useful. It is not clear that the gradient would be large for any OOD sample. \n- It is not theoretically clear to me why likelihood estimation is a poor method for OOD detection. Perhaps adding a short explanation/mathematical proof will provide the paper more grounding in the literature.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper discusses the problem of out of distribution detection by generative models. The paper proposes using the gradient of the model parameters w.r.t. data to distinguish out of distribution samples. It argues that the gradient at a local minima will be small for in distribution samples and high for out of distribution ones. \n\nThe paper also provides an empirically efficient algorithm to approximate said gradient for out of distribution detection.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "In addition to addressing the weaknesses enumerated above, the discussion under \"likelihood ratio\" subsection of section 2.2 is not at all clear. I'm not sure what point is being made here and it is not clear why likelihood ratio would be a suboptimal way for OOD detection.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Despite novelty concerns, I recommend accepting this paper for TMLR publication. The work makes several important contributions to OOD detection for deep generative models: 1. It provides rigorous theoretical justification for using gradient information in OOD detection, demonstrating invariance properties of the score vector. 2. The authors present extensive empirical validations of their proposed approximations across various model types, including diffusion models. IN particular, the evaluation is more comprehensive than prior works, covering multiple training distributions and model types, providing a more robust performance measure. The application to diffusion models extends the method's applicability to state-of-the-art generative models. 

While similar methods have been independently discovered, this work offers a unique combination of theoretical insight, empirical validation, and practical applicability. In my opion, the authors have adequately addressed reviewer concerns, clarifyng relationships to classical invariance results and providing more detailed explanations of certain concepts.

I believe this work makes a sufficiently significant contribution to OOD detection for deep generative models and is suitable for TMLR publication.
Decision: Accept (poster)