Perfect density models cannot guarantee anomaly detection
Abstract
Thanks to the tractability of their likelihood, some deep generative models show promise for seemingly straightforward but important applications like anomaly detection, uncertainty estimation, and active learning.
However, the likelihood values empirically attributed to anomalies conﬂict with the expectations these proposed applications suggest.
In this paper, we take a closer look at the behavior of distribution densities and show that these quantities carry less meaningful information than previously thought, beyond estimation issues or the curse of dimensionality.
We conclude that the use of these likelihoods for out-of-distribution detection relies on strong and implicit hypotheses, and highlight the necessity of explicitly formulating these assumptions for reliable anomaly detection.
Introduction
Several machine learning methods aim at extrapolating a behavior observed on training data in order to produce predictions on new observations.
But every so often, such extrapolation can result in wrong outputs, especially on points that we would consider infrequent with respect to the training distribution.
Faced with unusual situations, whether adversarial (Szegedy et al., 2013; Carlini and
Wagner, 2017) or just rare (Hendrycks and Dietterich, 2019) a desirable behavior from a machine learning system would be to ﬂag these outliers so that the user can assess if the result is reliable and gather more information if need be (Zhao and Tresp, 2019; Fu et al., 2017).
This can be critical for applications like medical decision making (Lee et al., 2018) or autonomous vehicle navigation (Filos et al., 2020), where such outliers are ubiquitous.
What are the situations that are deemed unusual?
Deﬁning these anomalies (Hodge and Austin, 2004; Pimentel et al., 2014) manually can be laborious if not impossible, and so generally applicable, automated methods are preferable.
In that regard, the framework of probabilistic reasoning has been an appealing formalism because a natural candidate for outliers are situations that are improbable or out-of-distribution .
Since the true probability distribution density p ∗ X of the data is often not provided, one would instead use an estimator, p ( θ ) X , from this data to assess the regularity of a point.
Density estimation has been a particularly challenging task on high-dimensional problems.
However, recent advances in deep probabilistic models , including variational auto-encoders (Kingma and
Welling, 2014; Rezende et al., 2014; Vahdat and Kautz, 2020), deep autoregressive models (Uria et al., 2014; van den Oord et al., 2016b,a), and ﬂow-based generative models (Dinh et al., 2014, 2016;
Kingma and Dhariwal, 2018), have shown promise for density estimation, which has the potential to enable accurate density-based methods (Bishop, 1994) for anomaly detection.
[Figure]
In this work, we propose a new perspective on this discrepancy and challenge the expectation that density estimation should enable anomaly detection.
We show that the aforementioned discrepancy persists even with perfect density models, and therefore goes beyond issues of estimation, approximation, or optimization errors (Bottou and Bousquet, 2008).
We highlight that this issue is pervasive as it occurs even in low-dimensional settings and for a variety of density-based methods for anomaly detection.
Density-based anomaly detection 2.1 Unsupervised anomaly detection: problem statement
Unsupervised anomaly detection is a classiﬁcation problem (Moya et al., 1993; Schölkopf et al., 2001), where one aims at distinguishing between regular points ( inliers ) and irregular points ( outliers ).
However, as opposed to the usual classiﬁcation task, labels distinguishing inliers and outliers are not provided for training, if outliers are even provided at all.
Given a input space X ⊆ R D , the task can be summarized as partitioning this space between the subset of outliers X out and the subset of inliers
X in , i.e., X out ∪ X in = X and X out ∩ X in = ∅ .
When the training data is distributed according to the probability measure P ∗ X (with density p ∗ X 2 ), one would usually pick the set of regular points X in such that this set contains the majority (but not all) of the mass (e.g., 95% ) of this distribution, i.e.,
P ∗ X ( X in ) =
1 − α ∈ (cid:0) 12 , 1 (cid:1) .
But, for any given α , there exists in theory an inﬁnity of corresponding partitions into X in and X out (see Figure 1).
How are these partitions deﬁned to match our intuition of inliers and outliers?
We will focus in this paper on recently used methods based on probability density.
2.2 Density scoring
When talking about outliers, infrequent observations, the association with probability can be quite intuitive.
For instance, one would expect an anomaly to happen rarely and be unlikely.
Since the language of statistics often associate the term likelihood with quantities like p ( θ ) X ( x ) , one might consider an unlikely sample to have a low "likelihood", that is a low probability density p ∗ X ( x ) .
Conversely, regular samples would have a high density p ∗ X ( x ) following that reasoning.
This is an intuition that is not only prevalent in several modern anomaly detection methods (Bishop, 1994; Blei
[Figure]
et al., 2017; Hendrycks et al., 2018; Kirichenko et al., 2020; Rudolph et al., 2020; Liu et al., 2020) but also in techniques like low-temperature sampling (Graves, 2013) used for example in Kingma and Dhariwal (2018) and Parmar et al.
The associated approach, described in Bishop (1994), consists in deﬁning the inliers as the points whose density exceed a certain threshold λ > 0 (for example, chosen such that inliers include a predeﬁned amount of mass, e.g., 95% ), making the modes the most regular points in this setting.
X out and X in are then respectively the lower-level and upper-level sets { x ∈ X , p ∗ X ( x ) ≤ λ } and { x ∈ X , p ∗ X ( x ) > λ } (see Figure 2b).
2.3 Typicality Test
The Gaussian Annulus theorem (Blum et al., 2016) (generalized in Vershynin, 2019) attests that most of the mass of a high-dimensional standard Gaussian N (0 , I D ) is located close to the hypersphere of radius √ D .
However, the mode of its density is at the center 0 .
A natural conclusion is that the curse of dimensionality creates a discrepancy between the density upper-level sets and what we expect as inliers (Choi et al., 2018; Nalisnick et al., 2019; Morningstar et al., 2020; Dieleman, 2020).
This motivated Nalisnick et al.
to propose another method for testing whether a point is an inlier or not, relying on a measure of its typicality .
This method relies on the notion of typical set (Cover, 1999) deﬁned by taking as inliers points whose average log-density is close to the average log-density of the distribution (see Figure 2c).
Deﬁnition 1 (Cover, 1999) .
Given independent and identically distributed elements (cid:0) x ( n ) (cid:1) n ≤ N from a distribution with density p ∗ X , the typical set A ( N ) (cid:15)
[Equation]
p
[Equation]
⊂ X N is made of all sequences that satisfy:
[Equation]
where H ( X ) = − E [log p ∗ X ( X )] is the (differential) entropy and (cid:15) > 0 a constant.
This method matches the intuition behind the Gaussian Annulus theorem on the set of inliers of a high-dimensional standard Gaussian.
Indeed, using a concentration inequality, we can show that
[Equation]
(cid:17)(cid:17)
= 1 , which means that with N large enough, A ( N )
[Equation]
( p ∗ X ) will contain most of the mass of ( p ∗ X ) N , justifying the name typicality .
The role of reparametrization
Given the anomaly detection problem formulation Subsection 2.1, we are interested in reasoning about the properties a solution ought to satisfy, in the ideal case of inﬁnite data and capacity.
For x p ∗ X ( x ) (a) An example of a distribution density p ∗
X .
x f ( x ) (b) Example of an invertible func- tion f from [0 , 1] to [0 , 1] .
f ( x ) p ∗ f
( X ) (cid:0) f
( x ) (cid:1) (c)
Resulting density p ∗ f
( X ) from applying f to X ∼ p ∗ X as a function of the new axis f ( x ) .
density-based methods this means that p ( θ ) X
= p ∗ X .
This setting is appealing as it gives space for theoretical results without worrying about the underﬁtting or overﬁtting issues mentioned by
Hendrycks et al.
; Morningstar et al.
; Kirichenko et al.
; Zhang et al.
Although we work in practice on points (e.g., vectors), it is important to keep in mind that these points are actually representations of an underlying outcome.
As a random variable, X is by deﬁnition the function from this outcome ω to the corresponding observation x = X ( ω ) .
However, at its core, an anomaly detection solution aims at classifying outcomes through these measurements.
How is the choice of X affecting the problem of anomaly detection?
While several papers studied the effects of a change of representation through the lens of inductive bias (Kirichenko et al., 2020; Zhang et al., 2020), we investigate the more fundamental effects of reparametrizations f .
To sidestep concerns about loss of information (Winkens et al., 2020), we study the particular case of an invertible map f .
The measurements x = X ( ω ) and f ( x ) =
( f ◦ X )( ω ) represent the same outcome ω
(although differently), and, since x and f ( x ) are connected by an invertible transformation f , the same method applied respectively to X or f ( X ) should classify them with the same label, either as an inlier or an outlier.
The target of these methods is to essentially assess the regularity of the outcome ω .
From this, we can make the following requirement for a solution to anomaly detection.
Principle.
In an inﬁnite data and capacity setting, the result of an anomaly detection method should be invariant to any invertible reparametrization f .
Do density-based methods follow this principle?
To answer that question, we look into how density behaves under a reversible change of representation.
In particular, the change of variables formula (Kaplan, 1952), which constitutes the backbone of ﬂow-based models (Tabak and Turner, 2013;
Dinh et al., 2014; Rezende and Mohamed, 2015) (used in Choi et al., 2018; Nalisnick et al., 2018), formalizes a simple intuition of this behavior: where points are brought closer together the density increases whereas this density decreases when points are spread apart.
The formula itself is written as:
[Equation]
where
[Equation]
is the Jacobian determinant of f at x , a quantity that reﬂects a local change in volume incurred by f .
Figure 3 already illustrates how the function f
(Figure 3b) can spread apart points close to the extremities to decrease the corresponding density round 0 and 1 , and, as a result, turns the density on the left (Figure 3a) into the density on the right (Figure 3c).
With this example, one can wonder to which degree an invertible change of representation can affect the density and the anomaly detection methods presented in Subsections 2.2 and 2.3 that use it.
[Figure] [Figure]
x
[Figure]
F p
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]
Leveraging the change of variables formula 4.1 Uniformization
We start by showing that unambiguously deﬁning outliers and inliers with any density-based approach becomes impossible when considering a particular type of invertible reparametrization of the problem, irrespective of dimensionality.
Under weak assumptions, one can map any distribution to a uniform distribution using an invertible transformation (Hyvärinen and Pajunen, 1999).
This is in fact a common strategy for sampling from complicated one-dimensional distributions (Devroye, 1986).
Figure 4 shows an example of this where a bimodal distribution (Figure 4a) is pushed through an invertible map (Figure 4b) to obtain a uniform distribution (Figure 4c).
To construct this invertible uniformization function, we rely on the notion of Knothe-Rosenblatt rearrangement (Rosenblatt, 1952; Knothe et al., 1957).
A Knothe-Rosenblatt rearrangement (notably used in Hyvärinen and Pajunen, 1999) is deﬁned for a random variable X distributed according to a strictly positive density p ∗ X with a convex support X , as a continuous invertible map f ( KR ) from X onto [0 , 1] D such that f ( KR ) ( X ) follows a uniform distribution in this hypercube.
This rearrangement is constructed as follows:
[Equation]
d
[Equation]
where CDF p is the cumulative distribution function corresponding to the density p .
In these new coordinates, neither the density scoring method nor the typicality test approach can discriminate between inliers and outliers in this uniform D -dimensional hypercube [0 , 1] D .
Since the resulting density p ∗ f ( KR ) ( X ) = 1 is constant, the density scoring method attributes the same regularity to every point.
Moreover, a typicality test on f ( KR ) ( X ) will always succeed as
[Equation]
However, these uniformly distributed points are merely a different representation of the same initial points.
Therefore, if the identity of the outliers is ambiguous in this uniform distribution, then anomaly detection in general should be as difﬁcult.
x p ∗ X ( x ) (a of a dis- tribution density p ∗ X .
p * X ( x ) s ( x ) x p ∗ X ( x ) ( x f ( s ) ( x ) (c)
A continuous invertible reparametrization f ( s ) such that p ∗ f
( s ) ( X ) (cid:0) f
( s ) ( x ) (cid:1) =
s ( x ) .
f ( s ) ( x ) p ∗ f ( s ) ( X ) (cid:0) f
( s ) ( x ) (cid:1) (
[Figure]
Resulting density p
[Figure]
f (
[Figure]
( X ) from applying
( s ) to X ∼
[Figure]
X as a s ) ( x ) .
4.2 Arbitrary scoring
While a particular parametrization can prevent density-based outlier detection methods from separating between outliers and inliers, we ﬁnd that it is also possible to build a reparametrization of the problem to impose to each point an arbitrary density level in the new representation.
To illustrate this idea, consider some points from a distribution whose density is depicted in Figure 5a and a score function indicated in red in Figure 5b.
In this example, high-density regions correspond to areas with low score value (and vice-versa).
We show that there exists a reparametrization (depicted in
Figure 5c) such that the density in this new representation (Figure 5d) now matches the desired score, which can be designed to mislead density-based methods into a wrong classiﬁcation of anomalies.
Proposition 1.
For any variable X ∼ p ∗ X with p ∗ X strictly positive (with X convex) and any measurable function s : X → R ∗ + bounded below by a strictly positive number, there exists a continuous bijection f ( s ) such that for any x ∈ X , p f ( s ) ( X ) (cid:0) f ( s ) ( x ) (cid:1) =
s ( x ) almost everywhere.
Proof.
We write x to denote ( x 1 , . . .
x D − 1 , x D ) and ( x <D , t ) for ( x 1 , . .
x D − 1 , t ) .
Let f ( s ) :
X → Z ⊂ R D be a function such that
[Equation]
and ∀ d ∈ { 1 , ..., D − 1 } , (cid:0) f ( s ) ( x ) (cid:1) d = x d .
As s is bounded below, f ( s ) is well deﬁned and invertible.
By the change of variables formula,
[Equation]
If X in and X out are respectively the true sets of inliers and outliers, we can pick a ball A ⊂ X in such that P ∗ X ( A ) =
α < 0 .
5 , we can choose s such that for any x ∈ ( X \ A ) , s ( x ) = 1 and for any x ∈ A, s ( x ) = 0 .
With this choice of s and the function f ( s ) deﬁned earlier, both the density scoring and the (one-sample) typical set methods will consider the set of inliers to be ( X \ A ) while X out ⊂ ( X \ A ) , making their results completely wrong.
While we can also reparametrize the problem so that these methods may succeed, such reparametrization requires knowledge of ( p ∗ X /s )
( x ) .
Without any constraints on the space considered, individual densities can be arbitrarily manipulated, which reveals how little these quantities mean in general.
[Figure]
Since our analysis in Subsections 4.1 and 4.2 reveals that densities or low typicality regions are not sufﬁcient conditions for an observation to be an anomaly, whatever its distribution or its dimension, we are now interested in investigating whether additional realistic assumptions can lead to some guarantees for anomaly detection.
Motivated by several representation learning algorithms which attempt to learn a mapping to a predeﬁned distribution (e.g., a standard Gaussian, see Chen and
Gopinath, 2001; Kingma and Welling, 2014; Rezende et al., 2014; Dinh et al., 2014; Krusinga et al., 2019) we consider the more restricted setting of a ﬁxed distribution of our choice, whose regular regions could for instance be known.
Surprisingly, we ﬁnd that it is possible to exchange the densities of an inlier and an outlier even within a canonical distribution.
Proposition 2.
For any strictly positive density function p ∗ X over a convex space X ⊆
R D with
D > 2 , for any x in , x out in the interior X o of X , there exists a continuous bijection f : X → X such that p ∗ X = p ∗ f ( X ) , p ∗ f ( X ) (cid:0) f (cid:0) x
( in ) (cid:1)(cid:1) =
p ∗ X (cid:0) x ( out ) (cid:1) , and p ∗ f ( X ) (cid:0) f (cid:0) x ( out ) (cid:1)(cid:1) =
p ∗ X (cid:0) x
( in ) (cid:1) .
We provide a sketch of proof and put the details in Appendix A. We rely on the transformation depicted in Figure 6, which can swap two points while acting in a very local area.
If the distribution of points is uniform inside this local area, then this distribution will be unaffected by this transformation.
In order to arrive at this situation, we use the uniformization method presented in Subsection 4.1, along with a linear function to ﬁt this local area inside the support of the distribution (see Figure 7).
Once those two points have been swapped, we can reverse the functions preceding this swap to recover the original distribution overall.
[Figure]
two points x in and x out inside the hypercube [0 , 1] D , there is sometimes no L 2 -ball cen-
[Figure]
in their
[Figure]
x m containing both x in and x out
[Figure]
However, given x in and x out , one can apply an in- vertible linear transformation L such that there exists a L 2 -ball centered in their new mean L ( x m ) contain- ing both L ( x in ) and L ( x out ) .
If the distribution was uniform inside [0 , 1] D , then it is now also uniform
[Figure]
[Figure]
Since the resulting distribution p ∗ f
( X ) is identical to the original f ∗ X , then their entropies are the same
H (cid:16) p ∗ f
( X ) (cid:17)
=
H ( f ∗ X ) .
Hence, when x in and x out are respectively an inlier and an outlier, whether in terms of density scoring or typicality, there exists a reparametrization of the problem conserving the overall distribution while still exchanging their status as inlier/outlier.
We provide an example applied to a standard Gaussian distribution in Figure 8.
This result is important from a representation learning perspective and a complement to the general non-identiﬁability result in several representation learning approaches (Hyvärinen and Pajunen, 1999; Locatello et al., 2019).
It means that learning a representation with a predeﬁned, well-known distribution and knowing the true density p ∗ X are not sufﬁcient conditions to control the individual density of each point and accurately distinguish outliers from inliers.
Discussion
Fundamentally, density-based methods for anomaly detection rely on the belief that density, as a quantity, conveys useful information to assess whether an outcome is an outlier or not.
For example, several density-based methods operate in practice on features learned independently from the anomaly detection task (Lee et al., 2018;
Krusinga et al., 2019; Morningstar et al., 2020; Winkens et al., 2020) or on the original input features (Nalisnick et al., 2018;
Hendrycks et al., 2018; Kirichenko et al., 2020; Rudolph et al., 2020; Nalisnick et al., 2019).
In general, there is no evidence that the density in these representations will carry any useful information for anomaly detection bringing into question whether performance of probabilistic models on this task (e.g., Du and Mordatch, 2019; Grathwohl et al., 2019; Kirichenko et al., 2020; Liu and Abbeel, 2020) reﬂects goodness-of-ﬁt of the density model.
On the contrary, we have proven in this paper that density-based anomaly detection methods are inconsistent across a range of possible representations 3 , even under strong constraints on the distribution, which suggests that ﬁnding the right input representation for meaningful density-based anomaly detection requires privileged information, as discussed in Subsection 4.2.
Moreover, several papers have pointed to existing problems in commonly used input representations; for example, the geometry of a bitmap representation does not follow our intuition of semantic distance (Theis et al., 2016), or images can come from photographic sensors tuned to speciﬁc populations (Roth, 2009;
Buolamwini and Gebru, 2018).
This shows how strong of an otherwise understated assumption it is to suppose that the methods presented in Subsection 2.2 and Subsection 2.3 would work on input representations.
This is particularly problematic for applications as critical as autonomous vehicle navigation or medical decision-making.
While deﬁning anomalies might be impossible without prior knowledge (Winkens et al., 2020) as out-of-distribution detection is an ill-posed problem (Choi et al., 2018; Nalisnick et al., 2019;
Morningstar et al., 2020), several approaches make these assumptions more explicit.
For instance, the density scoring method has also been interpreted in Bishop (1994) as a likelihood ratio method (Ren et al., 2019; Serrà et al., 2020; Schirrmeister et al., 2020), which, on the one hand, relies heavily on the deﬁnition of an arbitrary reference density as a denominator of this ratio but, on the other hand, is invariant to reparametrization.
Inspired by the Bayesian approach from Choi et al.
(2018), one can work on deﬁning a prior distribution on possible reparametrizations over which to average (similarly to Jørgensen and Hauberg, 2020).
A Proof of Proposition
Proposition 3.
For any strictly positive density function p ∗ X over a convex space X ⊆
R D with
D > 2 , for any x in , x out in the interior X o of X , there exists a continuous bijection f : X → X such that p ∗ X = p ∗ f ( X ) , p ∗ f ( X ) (cid:0) f (cid:0) x
( in ) (cid:1)(cid:1) =
p ∗ X (cid:0) x ( out ) (cid:1) , and p ∗ f ( X ) (cid:0) f (cid:0) x ( out ) (cid:1)(cid:1) =
p ∗ X (cid:0) x
( in ) (cid:1) .
Proof.
Our proof will rely on the following non-rigid rotation f ( rot ) .
Working in a hyperspherical coordinate system consisting of a radial coordinate r > 0 and ( D − 1) angular coordinates ( φ i )
i<D , ∀ d < D, x d = r (cid:32)
d − 1 (cid:89)
i = sin( φ i ) (cid:33)
cos( φ d ) x D = r (cid:32) D − 2 (cid:89) i
= sin( φ i ) (cid:33) sin( φ D − 1 ) ,
where for all i ∈ { 1 , 2 , ...,
D − 2 } , φ i ∈ [0 , π ) and φ D − 1 ∈ [0 , 2 π ) , given r max > r 0 > 0 , we deﬁne the continuous mapping f ( rot ) as:
[Equation]
where ( · ) + = max( · , 0) .
This mapping only affects points inside B 2 (0 , r max ) , and exchanges two points corresponding to ( r 0 , φ 1 , . .
. , φ D − 2 , φ D − 1 ) and ( r 0 , φ 1 , . .
. , φ D − 2 , φ D − 1 + π ) in a continous way (see Figure 6).
Since the Jacobian determinant of the hyperspherical coordinates transformation is not a function of φ D − 1 , f ( rot ) is volume-preserving in cartesian coordinates.
Let f ( KR ) be a Knothe-Rosenblatt rearrangement of p ∗ X , f ( KR )
( X ) is uniformly distributed in [0 , 1] D .
Let z ( in ) =
f ( KR ) (cid:0) x
( in ) (cid:1) and z ( out ) =
f ( KR ) (cid:0) x ( out ) (cid:1) .
Since f ( KR ) is continuous, z ( in ) , z ( out ) are in the interior (0 , 1) D .
Therefore, there is an (cid:15) > 0 such that the L 2 -balls B 2 (cid:0) z ( in ) , (cid:15) (cid:1) and
B 2 (cid:0) z ( out ) , (cid:15) (cid:1) are inside (0 , 1) D .
Since (0 , 1) D is convex, so is their convex hull.
Let r
= 12 (cid:13)(cid:13)
z ( in ) − z ( out ) (cid:13)(cid:13) 2 and r max = r 0 + (cid:15) .
Given z ∈ (0 , 1) D , we write z (cid:107) and z ⊥ to denote its parallel and orthogonal components with respect to (cid:0)
z
( in ) − z ( out ) (cid:1) .
We consider the linear bijection L deﬁned by
[Equation]
Let f ( z )
=
L ◦ f ( KR ) .
Since L is a linear function (i.e., with constant Jacobian), f ( z ) ( X ) is uniformly distributed inside L (cid:0) [0 , 1] D (cid:1) .
If z ( m ) is the mean of z ( in ) and z ( out ) , then f ( z ) ( X ) contains B 2 (cid:0) L (cid:0)
z ( m ) (cid:1) , r max (cid:1) (see Figure 7).
We can then apply the non-rigid rotation f ( rot ) deﬁned earlier, centered on L (cid:0) z ( m ) (cid:1) to exchange L (cid:0)
z ( in ) (cid:1) and L (cid:0) z ( out ) (cid:1) while maintaining this uniform distribution.
We can then apply the bijection (cid:0)
f
( z ) (cid:1) − 1 to obtain the invertible map f = (cid:0) f
( z ) (cid:1) − 1 ◦ f ( rot ) ◦ f ( z )
such that p ∗ f
( X ) =
f ∗ X , p ∗ f
( X ) (cid:0) f
(cid:0) x ( in ) (cid:1)(cid:1) =
p ∗ X (cid:0) x ( out ) (cid:1) , and p ∗ f ( X ) (cid:0) f (cid:0) x ( out ) (cid:1)(cid:1) =
p ∗ X (cid:0) x
( in ) (cid:1) .