A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Models
Authors: Hamidreza Kamkari
Brendan Leigh Ross
Rasa Hosseinzadeh
Jesse C. Cresswell
Gabriel Loaiza-Ganem
Venue: NeurIPS 2024 spotlight (2024)
Keywords: diffusion models
deep generative modelling
manifold hypothesis
intrinsic dimension
Primary area: diffusion_based_models
TL;DR: We provide an efficient local intrinsic dimension estimator using diffusion models, outperforming traditional estimators. It aligns closely with qualitative complexity in images and scales to stable diffusion.

Abstract: High-dimensional data commonly lies on low-dimensional submanifolds, and estimating the local intrinsic dimension (LID) of a datum -- i.e. the dimension of the submanifold it belongs to -- is a longstanding problem. LID can be understood as the number of local factors of variation: the more factors of variation a datum has, the more complex it tends to be. Estimating this quantity has proven useful in contexts ranging from generalization in neural networks to detection of out-of-distribution data, adversarial examples, and AI-generated text. The recent successes of deep generative models present an opportunity to leverage them for LID estimation, but current methods based on generative models produce inaccurate estimates, require more than a single pre-trained model, are computationally intensive, or do not exploit the best available deep generative models: diffusion models (DMs). In this work, we show that the Fokker-Planck equation associated with a DM can provide an LID estimator which addresses the aforementioned deficiencies. Our estimator, called FLIPD, is easy to implement and compatible with all popular DMs. Applying FLIPD to synthetic LID estimation benchmarks, we find that DMs implemented as fully-connected networks are highly effective LID estimators that outperform existing baselines. We also apply FLIPD to natural images where the true LID is unknown. Despite being sensitive to the choice of network architecture, FLIPD estimates remain a useful measure of relative complexity; compared to competing estimators, FLIPD exhibits a consistently higher correlation with image PNG compression rate and better aligns with qualitative assessments of complexity. Notably, FLIPD is orders of magnitude faster than other LID estimators, and the first to be tractable at the scale of Stable Diffusion.

A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Models
Hamidreza Kamkari
Brendan Leigh Ross Rasa Hosseinzadeh Jesse C. Cresswell Gabriel Loniza-Ganem (hamid, brendan, rasa, jesse, gabriel)@layer6..
Layer 6 AI, Toronto, Canada
Abstract
High-dimensional data commonly lies on low-dimensional submanifolds, and estimating the local intrinsic dimension (LID) of datum the dimension of the submanifold it belongs to is a longstanding problem. LID can be understood as the number of local factors of variation: the more factors of variation datum has, the more complex it tends to be. Estimating this quantity has proven useful in contexts ranging from generalization in neural networks to detection of ouitrf-disttibution data, adversarial examples, and A Al-generated text. The recent successes of deep generative models present opportunity leverage them for LID estimation, but current methods based on generative models produce inaccurate estimates, require more than single pre-trained model, computationally intensive, or do not exploit the best available deep generative models: diffusion models (DMs). In this work, show that the Fokker-Plance equation associated with DM provide an LID estimator which addresses the aforementioned deficiencies. Our estimator, called FLIPD, is easy to implement and compatible with all popular DMs. Applying FLIPD to synthetic LID estimation benchmarks, we find that DMs implemented as fully-connected networks are highly effective LID estimators that outperform existing baselines. We also apply FLIPDD to natural images where the LID is unknown. Despite being sensitive to the choice of network architecture, FLIPD estimates remain useful measure of relative complexity; compared to competing estimators, FLIPD exhibits a consistently higher correlation with image PNG compression rate and better aligns with qualitative assessments of complexity. Notably, FLIPD is orders of magnitude faster than other LID estimators, and the first to be tractable at the scale of Stable Diffusion.
Introduction
The manifold hypothesis [7], which has been empirically verified in contexts ranging from natural images [49, 10] to calorimeter showers physics [15], states that high-dimensional data of interest in RD often lies on low-dimensional submanifolds of RRD, For a given datum æ E R,D, this hypothesis motivates using its local intrinsic dimension (LID), denoted LID(z), as natural measure of its complexity. LID(x) corresponds to the dimension of the data manifold that x belongs to, and can be intuitively understood as the minimal number of variables needed to describe xr. More complex data needs more variables to be adequately described, as illustrated in Figure 1. Data manifolds are typically not known explicitly, meaning that L.D must be estimated. Here we tackle the following problem: given dataset along with a query datum v, how can we tractably estimate LID(x)?
This is a longstanding problem, with LID estimates being highly useful due their innate inter- pretation as a measure of complexity. For example, these estimates can be used to detect outliers (27, 32], Al-generated text [64], and adversarial examples [42]. Connections between the general- ization achieved by a neural network and the LID estimates of its internal representations have also
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

8
4
8
8
8
8
8
8
8
8
8
8
0
8
8
8
6
6
6
Figure 1: (Left) A cartoon illustration showing that LID is natural measure of relative complexity. We depict two manifolds of MNIST digits, corresponding 1s and 8s, as I-dimensional and 2-dimensional submanifolds IR", respectively. The relatively simpler manifold of Is exhibits a single factor of variation ("tilt"). whereas 8s have an additional factor of variation ("dispropotioonalty") (Right) The 4 lowest- and highest-LII) datapoints from subsample of L.AION- Aestheties, measured by our method, FLIPD, applied to Stable Diffusion v1.5. FLIPD scales efficiently large models on high-dimensional data, and aligns closely with subjective complexity.
been shown [4, 8, 44, 9]. These insights can be leveraged to identify which representations contain maximal semantic content [66], and help explain why LID estimates can be helpful as regularizers [72] and for pruning large models [70]. LID estimation is thus not only of mathematical and statistical interest, but can also benefit the empirical performance of deep learning models at numerous tasks.
Traditional estimators of intrinsic dimension [22, 38, 43, 12, 30, 20, 1, 5] typically rely on pairwise distances and nearest neighbours, so computing them is prohibitively expensive for large datasets. Recent work has thus sought to move away from these model-free estimators and instead take advantage of deep generative models which learn the distribution of observed data. When this distribution is supported on low-dimensional submanifolds of R., successful generative models must implicitly learn the dimensions of the data submanifolds, suggesting they can be used to construct LID estimators. However, existing model-basee estimators suffer from various drawbacks, including being inaccurate and computationally expensive [59], not leveraging the best existing generative models [62, 71] (i.e. diffusion models [56, 25, 58)), and requiring training several models [62] or altering the training procedure rather than relying on pre-trained model [26]. Importantly, none of these methods scale to high-resolution images such as those generated by Stable Diffusion [51].
We address these issues by showing how LID can be efficiently estimated using only single pre- trained diffusion model (DM) by building on LIDL [62], a model-based estimator. LIDL operates by convolving data with different levels of Gaussian noise, training normalizing flow [50, 17, 18] for each level, and fitting linear regression using the log standard deviation of the noise as a covariate and the corresponding log density (of the convolution) evaluated at x as the response; the resulting slope is estimate of LID(x:) D, thanks to a surprising result linking Gaussian convolutions and LID. We first show how to adapt LIDI. to DMs in such way that only single DM is required (rather than many normalizing flows). Directly applying this insight leads to LIDI. estimates that require one DM but several calls to an ordinary differential equation (ODE) solver; we further show how to circumvent this with an alternative ODE that computes all the required log densities in a single solve. We then argue that the slope of the regression LIDL aims to capture the rate of change of the marginal log probabilities in the diffusion process, which can be evaluated directly thanks to the Fokker-Planne Planck equation. The resulting estimator, which we call FLIPD, is highly efficient, and circumvents the need for ODE solver. Notably, FLIPD is differentiable; this property opens up exciting avenues for future research as it enables backpropagating through LID estimates.
Our contributions are: (i) showing how DMs can be efficiently combined with LIDL in a way which requires a single call to an ODE solver; (ii) leveraging the Fokker- Fokker-Planck equation to propose FLIPD, thus improving upon the estimator and circumventing the need for an ODE solver altogether; (Mii motivating FLIPD theoretically; (iv) introducing an expanded suite of LID estimation benchmark
Pronounced "flipped", the acronym is rearrangement of "FP" from Fokker-Planck and "LID".

---

tasks that reveals gaps in prior evaluations, and specifically that other estimators do not remain accurate as the complexity of the manifold increases; (v) demonstrating that when using fully- connected architectures for diffusion models, FLIPD outperforms existing baselines - especially as dimension increases while being much more computationally efficient; (vi) showing that when applied to natural images, despite varying across network architecture (i.e., fully-connected network or UNet [52, 68]), FLIPD estimates consistently align with other measures of complexity such as PNG compression length, and with qualitative assessments of complexity, highlighting that the LID estimates provided by FLIPD remain valid measures relative image complexity; and (vii) demonstrating that when applied to the latent space of Stable Diffusion, FLIPD can estimate LID for extremely high-resolution images (~ 10 pixel dimensions) for the first time.
Background and Related Work
2.1 Diffusion Models
Forward and backward processes Diffusion models admit various formulations [56, 25]; here we follow the score-based one [58]. We denote the true data-generating distribution, which DMs aim to learn, as p(-,)) 0). DMs define the forward (Itô) stochastic differential equation (SDE),
dX. f(Xe, t)dt + g(t)dW,, Xo p(:,0),
(1)
where / RD x [0, + D and [0. - R are hyperparameters, and We denotes a D- dimensional Brownian motion. We write the distribution of X, as p(:,). The SDE in Equation 1 prescribes how to gradually add noise to data, the idea being that p(;; 1) is essentially pure noise. Defining the backward process as Y2 == X1-1, this process obeys the backward SDE [3. 24].
dY, [g t), 1 - t) f(y t)] dt g(1 t)dW, Yo ~ p(;,),
(2)
where (xx, () V ((g,((z, 1) is the unknown (Stein) score function, and W, is another D- dimensional Brownian motion. DMs leverage this backward SDE for generative modelling by using neural network 3 R. x (0, + R to learn the score function with denoising score matching [67]. Once trained, 5(x, () s(x,t). t). To generate samples y, from the model, we solve an approximation of Equation 2:
aÝ; = g2 - t)s(())) - - - -2) 0) at + g(1 - t)aW,. Go ~ p(,,
(3)
with replacing the true score and with p(: 1). a Gaussian distribution chosen to approximate p(,, 1) (depending on and g), replacing p(-. 1).
Density Evaluation DMs can be interpreted as continuous normalizing flows [13], and thus admit density evaluation, meaning that if we denote the distribution of F1-, as t(,,t,, then p(a, tu) can be mathematically evaluated for any given E RD and to E (0, 1]. More specifically, this is achieved thanks to the (forward) ordinary differential equation (ODE) associated with the DM:
[ (1(00.0) -100))))0000000 (0(())(())))) dí,
d.::
Sto x.
Solving this ODE from time to to time produces the trajectory (tt) which can then be used for density evaluation through the continuous change-of-varian formula:
log p(r, (o) = logg((,, 1) + [' ir(Vo(āt, ())dt,
(5)
where p(,, 1) can be evaluated since it is Gaussian, and where v(z, = (t))((z,)))
Throughoou this paper, we use the symbol V to denote the differentiation operator with respect to the vector- valued input, not the scalar time t, i.e V 0/0x.
3

---

Trace estimation Note that the cost of computing Vo(ă,, t) for particular 2, amounts to ((D) function evaluations of (since calls to a Jacobian-vector productt routine are needed [6]). Although this is not prohibitively expensive for single é2, in order to compute the integral in Equation 5 in practice, (Et)) must be discretized into a trajectory of length N. If we denote by F the cost of evaluating v(2,, t) or equivalendy, š( t) deterministic density evaluation is ( DF',, which is computationally prohibitive. The Hutchinson trace estimator [29] - which states that for M E RDxD, (r(M) Eg|s" Me), where ε E ?R." has mean 0 and covariance ID- is thus commonly used for stochastic density estimation; approximating the expectation with l samples from & results in a cost of O PN), which much faster than deterministic density evaluation when k << D.
2.2 Local Intrinsic Dimension and How to Estimate It
LID Various definitions of intrinsic dimension exist [28, 21, 37, 111. Here we follow the standard one from geometry: d-dimensional manifold is set which is locally homeomorphic to Rd. For a given disjoint union of manifolds and a point x in this union, the local intrinsic dimension of x the dimension of the submanifold it belongs to. Note that LID not an intrinsic property of the point x, but rather property of with respect to the manifold that contains it. Intuitively, LID(z) corresponds the number of factors of variation present in the manifold containing x, and is thus a natural measure of the relative complexity of x, as illustrated in Figure 1.
Estimating LID The natural interpretation of LID as a measure of complexity makes estimating from observed data a relevant problem. Here, the formal setup is that p(- 0) is supported on disjoint union of manifolds [10], and we assume access to a dataset sampled from Then, for a given x the support of p(- 0), we want to use the dataset to provide an estimate of (z). Traditional estimators [22, 38, 43, 12, 30, 20, 1, 5] rely on the nearest neighbours of x in the dataset, or related quantities, and typically have poor scaling in dataset size. Generative models are an intuitive alternative to these methods; because they are trained to learn p(:,0) when they succeed they must encode information about the support of p(- 0), including the corresponding manifold dimensions. However, extracting this information from a trained generative model is not trivial. For example, Zheng et al. [71] showed that the number of active dimensions in the approximate posterior of variational autoencoders [33, 50] estimates LID, but their approach does not generalize to better generative models.
Tempczyk et al. [62] proposed LIDL,, method for LID) estimation relying on normalizing flows as tractable density estimators [50, 17, 18]. LIDL works thanks to a surprising result linking Gaussian convolutions and LID [39, 62, 71]. We will denote the convolution of p(:, 0) and Gaussian noise with log standard deviation as e(: 6), i.e.
LIDI.
elz, 6) p(zo: 0)N zo; ID)
(6)
The aforementioned result states that, under mild regularity conditions on p(-, 0), and for a given x: in its support, the following holds 5 + -00
log e(x 6) = ((LID(x) - D) + O(1).
This result then suggests that, for negative enough values of (i.e. small enough standard deviations):
(8) for some constant e. If we could evaluate log e(x, 8) for various values of 5, this would pro- vide an avenue for estimating LID(x)): set some values d1.-.., &m. fit linear regression using {(ō;, log e(æ, 6,)) with 8 as the covariate and log e(x, as the response, and let 3, be the corresponding slope. follows that 8, estimates LID(x)) - D, so that L.I(x)) R D+ 3, a sensible estimator of local intrinsic dimension.
log g(x, z 6(LID(x) - D) c
Since g(x, 6) is unknown and cannot be evaluated, LIDL requires training m normalizing flows. More specifically, for each d;, normalizing flow is trained data which (0, 26: Ip) noise is added. In LIDL, the log densities of the trained models are then used instead of the unknown true log densities log e(x; 5;) when fitting the regression as described above.
Despite using generative models, LIDL has obvious drawbacks. LIDI requires training several models. It also relies on normalizing flows, which are not only empirically outperformed by DMs by

---

wide margin, but are also known to struggle to learn oow-dimesiional manifolds [14, 39, 40). On the other hand, DMs do not struggle to learn p(., 0) even when it is supported on oow-dimensional manifolds [48, 16, 40], further suggesting that LIDI. can be improved by leveraging DMs.
Estimating LID with DMs The only works we are aware of that leverage DMs for LID estimation are those of Stanczuk et al. [59], and Horvat and Pfister [26]. The latter modifies the training procedure of DMs, so we focus on the former since we see compatibility with existing pre-trained models as an important requirement for DM-based LID estimators. Stanczuk et al. [59] consider variance-explading DMs, where f = They show that, as > O, the score function s(x, 1) points orthogonally towards the manifold containing 2, or more formally, it lies in the normal space of this manifold at x. They thus propose the following L.D estimator, which we refer to as the normal bundle (NB) estimator: first run Equation until time to starting from and evaluate 8(, fo) the resulting value; then repeat this process K times and stack the K resulting D-dimensional vectors into a matrix S(x) E D K The idea here is that if to is small enough and K is large enough, the columns of S(x) span the normal space of the manifold at æ. suggesting that the rank of this matrix estimates the dimension of this normal space, namely D LID(z)) Finally, they estimate LID(z) as:
LID(z)= = D rank S(z).
Numerically, the rank is computed by performing singular value decomposition (SVD) of S(z). setting a threshold, and counting the number of singular values exceeding the threshold. Computing S(x) requires K function evaluations, and intuitively K should be large enough to ensure the columns of 5(x) span the normal space at z;; the authors thus propose using K = 4D, and recommend always at least ensuring that K > D. Computing the NB estimator costs ((KF + D²K), K), where F again denotes the cost for evaluating 8. Thus, although the NB estimator addresses some of the limitations of LIDL,, it remains computationally expensive in high dimensions.
3 Method
Although Tempczyk et al. [62] only used normalizing flows in LIDL, they did point out that these models could be swapped for any other generative model admitting density evaluation. Indeed, one could trivially train m DMs and replace the flows with them. Throughout this section we provide a sequence of progressive improvements FLIPD. We assume access to a pre-trained DM such that f(z, t) = b(t)xz for a function b [0, 1] + R. This choice implies that the transition kernel Peju associated with Equation I is Gaussian [53]:
this naive application of LIDL with DMs, culminating with
(z) mo) = N (zxi (t)xo. (t)(D))
(10)
where vi : 10 + R. We also assume that b and g are such that va and a are differentiable
and such that X(t) = a (t) is injective. This setting encompasses all DMs commonly used in practice, including variance- exploodigg auriance-preserving (of which the widely used DDPMs [25] are a discretized instance), and
[58]. In Appendix A we include explicit formulas for (1), (1). and A(1) for these particular DMs.
3.1 LIDL with a Single Diffusion Model
As opposed to the several normalizing flows used in LIDL which are individually trained on datasets with different levels of noise added, single DM already works by convolving data with various noise levels and allows density evaluation of the resulting noisy distributions (Equation 5). Hence, we make the observation that LIDL can be used with a single DM. All we need is to relate e(- 8) to DMs, (t(t) = 1, so we can easily use
the density of the DM, p(-, t). In the case of variance-explodin the defining property of the transition kernel in Equation 10 to get
/ p( 0)N ( T
pfr, t) = p(zo. Pp I (o)dro =
(11)
which equals exx, 5) from Equation when we choose = - (e5)) In turn we can use LIDI with a single variance-exploding DM by evaluating each p o - () through Equation 5. This idea
Note that we treat positive and negative superindices differently; e.g. o denotes the inverse function of o.
not 1/ the other hand o² denotes the square of
5

---

extends beyond variance- exxpodiig DMs; in Appendix B.1 we show that for any arbitrary DM with transition kernel in Equation 10, it holds that
log @( 5) = D log / (t())) log ( (t(())),, t(0):
(12)
where t(6) A
(e"). This equation relevant because LIDL requires log o( ()) yet DMs provide log p(: f): linking these two quantites as above shows that LIDL can be used with single DM.
3.2 A Better Implementation of LIDL with a Single Diffusion Model
Using Equation 12 with LIDL still involves computing log (ogg((((tt (1(((()))) ((6)) through Equation 5 for each i = I..... m before running the regression. Since each of the corresponding ODEs in Equation 4 starts at different time to t(6,) and is evaluated at a different point (b(((6,)) this means that a different ODE solver call would have to be used for each i, resulting in a prohibitively expensive procedure. To address this, aim to find an explicit formula for 0/05 log e(xr,)) 5). We do so by leveraging the Fokker- Planck equation associated with Equation 1, which provides an explicit formula for 0/Ot p(æ, t). Using this equation along with the chain rule and Equation 12, we show in Appendix B.2 that, for DMs with transition kernel as in Equation 10,
log e(x, 6) of o)===((((())) tr ((())))) + +10(((((0))) 08
=(t0);ss (13)
Then, assuming without loss of generality that ò1 < Sm we can use the above equation to define log ((x, 6) through the ODE
dlog 0(x; 6) = v (M();; 8, x) d5, log ((2; 61) = -0
(14)
Solving this ODE from O3 to 8m produces the trajectory (log 0(x, 8)) (..] Since v(t(5); s, 2) does not depend on 0(x; 8), when s = 5 the solution to the ODE above will be off by a constant, i.c. log 0(xz 8) = log e( (z, 5) + Giitt for some Cinit that depends on the initial condition of the ODE (0 in this case) but not on 8. Furthermore, while setting the initial condition to 0 might at first appear odd, recall that LIDL fits a regression using {6, log ø(2: (z, 6,)), and thus Guit will be absorbed in the intercept without affecting the slope. In other words, the initial condition is irrelevant, and we can use LIDL with DMs by using a single call to an ODE solver on Equation 14.
3.3 FLIPD: An Efficient Fokker-Plance Planck-Based LID Estimator
The LIDL estimator with DMs presented in Section 3.2 provides a massive speedup over the naive approach of training m DMs, and over the method from Section 3.1 requiring m ODE solves, Yet, solving Equation 14 involves computing the trace of the Jacobian of á multiple times within an ODE solver, which, as mentioned in Section 2. remains expensive. In this section we present our LID estimator, FLIPD, which circumvents the need for an ODE solver altogether. Recall that LIDL is based on Equation 8, which justifies the regression. Differentiating this equation yields that 0/05 log e(x,, 50) R LID(x) - for negative enough dg, meaning that Equation 13 directly provides the rate of change that the regression in LIDL aims to estimate, from which we get
(x) ~ +
log o(x, 60) D + (1(6); 8, 2) D + ((((5)): 8, x) =: FLIPD( fo). (15)
08
where to t(5g). Computing FLIPD is very cheap since the trace of the Jacobian of s has to be evaluated only once when calculating v(((6n); 8,)). As mentioned in Section 2.1, exact evaluation of this trace has cost of (DF)), but can be reduced to (kkF)) when using the stochastic Hutchinson trace estimator with ks < D samples. Notably, FLIPD provides massive speedup over the NB estimator - O(FF) vs. O(KF + -ssecccsy especially in high dimensions where K > D >>> k. In addition to not requiring ODE solver, computing FLIPD requires setting only single hyperparameter, dg. Furthermore, since v(t(o); 8, x) depends on only through f()), we can directly set to the hyperparameter rather than dg. which avoids the potentially cumbersome computation of t(5p) = x-1 (c"")) instead of setting& a suitably negative on we set to > sufficiently close to 0. In Appendix A we include explicit formulas for FLIPD( tu) for common DMs.
Finally, we present theoretical result further justifying FLIPD. Note that the O(1) term in Equation 7 need not be constant in d as in Equation 8, even if it is bounded. The more this term deviates from constant, the more bias we should expect in both LIDI. and FLIPD. The following result shows that in an idcalized linear setting, FLIPD is unaffected by this problem:

---

Theorem 3.1 (FLIPD Soundness: Linear Case), Let & be an embedded submanifold of given by a d-dimensional affine subspace. If p(. 0) supported on L. continuous, and with finite second moments, then for any E L with p(æ, 0) > 0, we have:
a log e( 6) d od-D. D.
lim 60
(16)
06
Proof. See Appendix B.3.
We conjecture that our theorem can be extended to non-linear submanifolds since it is a local result and every manifold be locally linearly approximated by its tangent plane. More specifically, o(2, 8) becomes "increasingly local as 5 - -oo" in the sense that its dependence on the values p(z0, 0) becomes negligible ô + -00 when zo not in close enough neighbourhood of æ; this is because ((0, e²x Ip) concentrates most of its mass around 0 as d -o (see Equation 6).
However, we leave generalizing our result to future work.
4 Experiments
Throughout this section, we use variance- prreervi-p DMs, the most popular variant of DMs. We provide "dictionary" to translate between the score-based formulation of FLIPD and DDPMs in Appendix C. We hope this will enable practitioners who are less focused on the theoretical aspects effortlessly apply FLIPD to their pre-trained DMs. Our code is available at https: //gi sgittuu.ccom labserraiillllla/aaaaeeeaaaeae aataab/sllppp fiidd; see Appendix D for more experimental details.
4.1 Experiments Synthetic Data
The effect of to FLIPD requires setting to close to 0 since all the theory holds in the 6 + -00 regime. It is important to note that DMs fitted
Avg. for LID 2
Avg. for Circle.
FLIPDL. ful
Avg.
FLPPI tol
Avg. for
Avg. for LID 8
cow-dimesiional mani- folds are known to exhibit numerically un- stable scores s(,, to) as to > 0 [65, 41, 40]. This fact does not invalidate FLJIPD, but it suggests that it might be sensitive choice of fo. Our first set of experiments examines the effect of to on FLIPD((r, to) by varying to within the range (0, 1).
0.00 0.25
0.75
1.00
1.00
the
(a) Mixture of Gaussians.
(b) String within a doughnut.
Figure 2: FLIPD curves with knees at the true LID.
In Figure 2, train DMs on two distributions: (i) mixture of three isotropic Gaussians with dimensions 2, 4, and 8, embedded in R (each embedding is carried out by multiplication against random matrix with orthonormal columns plus random translation); and (ii) "string within doughnut", which is a mixture of uniform distributions minor radius of 1) and 1d circle (aligning with the major circle of the torus) embedded in 3 (this union of manifolds is shown in the upper half of Figure 3). While FLIPD( to) is inaccurate to = 0 due the aforementioned instabilities, it quickly stabilizes around the true LID for all datapoints. We refer to this pattern as knee in the FLIPD curve. In Appendix D.2, we show similar curves for more complex data manifolds.
2d torus (with a major radius of 10 and
FLIPD is a multiscale estimator Interestingly, in Figure 2b we see that the blue FLIPD curve (corresponding to "doughnut" points with LID of 2) exhibits a second knee at 1, located at the to shown with a vertical line. This confirms the multiscale nature convolution-based estimators, first postulated by Tempezyk et al. [62] in the context of normalizing flows; they claim that when selecting a log standard deviation 5, all directions along which a datum can vary having log standard deviation less than are ignored. The second knee in Figure 2b can be explained by similar argument: the torus looks like Id circle when viewed from far away, and larger values of to correspond to viewing the manifolds from farther away. This is visualized in Figure 3 with two views of the "string within a doughnut" and corresponding LID estimates: one zoomed-in view where lo is small, providing fine-grad fine-grained LID estimates, and zoomed-out view where to is large, making both the string and
7

---

Table I: MAE (lower is better) concordance indices (higher is better with 1.0 being the gold standard), Rows show synthetic manifolds and columns represent LID estimation methods. Columns grouped based on whether they use a generative model, with the best results each metric within each group being bolded. Model-free
Model-hased
Syothetic Manifold
FLIPD
NB
LIDL
ESS
LPCA
String within doughnut C g
0.06
1.00
1.48
0.48
1.10
0.99
0.02
1.00
0.00
1.00
C. S B2:0
0.17
1.00
0.10
0.07
0.00
Noo R:000
1.67
21.9
2/0 + Ldan + Ulo C R.000
1.30
1.00
61.6
0.34
8.46
0.74
21.9
0.74
20.1
0.86
Nio + Nas + Noo C R-000
1.81
1.00
74.2
0.34
8.87
0.74
7.71
0.88
5.72
0.91
Fi0 + F2s + Foo E R100
3.93
1.00
74.2
0.34
18.6
0.70
9.20
0.90
6.77
1.00
M10 + Ido 4 M2o0 R 900
14.3
1.00
0.34
120
0.70
1.39
1.00
0.01
1.00
M600 S 1000
24.9
14.5
219
doughnut appear as a 1d circle from this distance. In Appendix D.3 we have an experiment that makes the multiscale argument explicit.
Finding knees As mentioned, we persistently see knees in FLIPD curves. This in line with the observations of Tempczyk et al. [62] (see Figure 5 of [62]), and it gives us a fully automated approach to setting to. We leverage kneedle [54], a knee detection algorithm which aims to find points of maximum curvature. When computationally sensible, rather than fixing lo, we evaluate Equation 15 for 50 values of and pass the results to kneedle to automatically detect the to where a knee occurs.
10
-10
0
FLIPD(. to)
10
10
Experimental setup We create a benchmark for LID evaluation on complex unions of manifolds where the true LID is known. We sample from simple distributions on low-dimensional spaces, and then embed the samples into R.. We denote uniform, Gaussian, and Laplace distri- butions U,N, and L, respectively, with sub-indices indicating LID, and a plus sign denoting mixtures. To em- bed samples into higher dimensions, we apply random matrix with orthonormal columns and then apply a random translation. For example, No0 + L20 S R 100 indicates 10- (0-dimensional Gaussian and 20 20-dimensional Laplace, each of which undergoes a random affine transformation mapping to R 100 (one transformation per component). We also generate non-linear manifolds, de- noted with F. by applying a randomly initialized D-dimensional neural spline flow [18] after the affine transformation (when using flows, the input noise is always uniform); since the flow is diffeomorphism, it preserves LID. To our knowledge, this synthetic LID benchmark is the most extensive date, revealing surprising deficiencies in some well-known traditional estimators. For an in-depth analysis, see Appendix D.4.
10
10
-10 10
Figure 3: "String within a doughnut" man- ifolds, and corresponding FLIPD estimates for different values of to (to 0.05 top and to 0.65 on bottom). These results highlight the multiscale nature of FLIPD.
Results Here, we summarize our synthetic experiments in Table using two metrics of performance: the mean absolute error (MAE) between the predicted and true LID for individual datapoints; and the concordance index, which measures similarity in the rankings between the true LIDs and the estimated ones (note that this metric only makes sense when the dataset has variability in its ground truth LIDs, so we only report it for the appropriate entries in Table 1). We compare against the NB and LIDL estimators described in Section 2.2, as well two of the most performant model-free baselines: LPCA [22, 12] and ESS [30]. For the NB baseline, we use the exact same DM backbone as for FLIPD (since NB was designed for variance-een exploding DMs, we use the adaptation to variance- preeuncceeeeeevinggg DMs used in [32], which produces extremely similar results), and for LIDL we use 8 neural spline flows. terms of MAE, we find that FLIPD tends to the best model-based estimator, particularly as dimension increases. Although model-free baselines perform well in simplistic scenarios, they produce unreliable results as LID increases or more non-linearity is introduced the data manifold. In terms of concordance index, FLIPD achieves perfect scores in all scenarios, meaning that even
8

---

784
Avg. FMNIST Avg. MNIST
8 9
6
FLIPDI- k.g)))))))
5
170
8
130
e
0.1
0.5
1.0
to
(c) Ordering high-resolation images in LAION using FLIPD (top) and PNG compression size (bottom).
FLIPD MNIST and FMNIST.
(b) Ordering small images.
Figure 4: Overview image LID: (a) shows the FLIPD curves that are used to estimate average LID MNIST and FMNIST when using MLP backbones; (b) compares images with small and large FLIPD estimates from FMNIST, MNIST, SVHN, and CIFARIC 10 when using UNet backhones; and (c) compares LAION images with small and large FLIPD estimates using Stable Diffusion (top, to 0.3) and PNG compression sizes (bottom).
when its estimates are off, it always provides correct LID rankings. We include additional results in the appendices: in Appendix D.5 ablate FLIPD, finding that using kneedle indeed helps, and that FLIPD also outperforms the efficient implementation of LIDL with DMs described in Section 3.2 that uses an ODE solver. We notice that NB with the setting proposed in [59] consistently produces estimates that are almost equal to the ambient dimension; thus, in Appendix D.6 we also show how NB can be significantly improved upon by using kneedle, although is still outperformed by FLIPD in many scenarios. In addition, in Table 7 and Table 8 we compare against other model- baselines such MLE [38, 43] and FIS [1]. We also consider datasets with single -in-dimmnsional) manifold where the average LID estimate can be used to approximate the global intrinsic dimension. Our results in Table demonstrate that although model-free baselines indeed accurately estimate global intrinsic dimension, they perform poorly when focusing on (pointwise) I LID) estimates.
4.2 Experiments with Fully-Connected Architectures on Image Data
We first focus on the simple image datasets MNIST [36] and FMNIST [69]. We flatten the images and use the same MLP architecture as our synthetic experiments. Despite using an MLP, our DMs can generate reasonable samples (Appendix 1. and the FLIPD curve for both MNIST and FMNIST shown in Figure 4a. The knee points are identified at lg 0.1, resulting average LID estimates of approximately 130 and 170, respectively. Evaluating LID) estimates for image data is challenging due to the lack of ground truth. Although our LID estimates are higher than those in [49] and [10], our experiments (Table 9 of Appendix D.4) and the findings in [62] and [59] show that model-free baselines underestimate LID of high-dimensional data, especially images.
4.3 Experiments with UNet Architectures on Image Data
When moving to more complex image datasets, the MLP backbone fails to generate high-quality samples. Therefore, we replace it with state-of-the-art UNets [52, 68) (see Appendix F.2). Surpris- ingly, we find that using kneedle with UNets fails to produce sensible LID estimates with FLIPP (see curves in Figure 10 of Appendix E. We discuss why this might be the case in Appendix 1, and from here on we simply set to as a hyperparameter instead of using kneedle. Although avoid- ing kneedle when using UNets results in increased sensitivity with respect to. argue that FLIPD remains a valuable measure of complexity as it produces sensible image rankings and is highly correlated with with PNG compression length. We took random subsets of 4096 images from each of FMNIST, MNIST, SVHN [46], and CIFAR 10 [35], and sorted them according to their FLIPD estimates (obtained using UNet backbones). We show the top and bottom 5 images for each dataset in Figure 4b, and include more samples in Appendix E.3. Our visualization shows that higher FLIPD estimates indeed correspond images with more detail and texture, while lower estimates correspond to less complex ones. Additionally, we show in Appendix E,4 that using only A: = 50 Hutchinson samples approximate the trace term in FLIPD is sufficient for small values of to-
Further, we quantitatively assess our estimates by computing Spearman's 's rank correlation coefficient between different LID estimators and PNG compression size, used proxy for complexity in the absence of ground truth. We highlight that although we expect this coefficient to be high, a perfect LID estimator need not achieve correlation of 1. As shown in Table 2, FLIPD has high correlation

---

with PNG, whereas model-free estimators do not. We find that the NB estimator corre- lates slightly more with PNG on MNIST and CIFARIO, but significantly less FMNIST and SVHN. Moreover, in Appendix E.5, we analyze how increasing to affects FLIPD by re-computing the correlation with the PNG size at different values of ln E (0,1). 1). We see that as increases, the correlation with PNG decreases. Despite this decrease, we observe an interesting phenomenon: while the image
Table 2: Spearman's is correlation between LID estimates and PNG compression size. FLIPD and NB were computed using the same UNet backbone.
Method
MNIST
FMNIST
CIFARIO
SVIIN
FLIPD
0.837
0.883
0.819
0.876
NB
0.864
0.480
0.894
0.573
ESS
0.444
0.063
0.326
0.019
LPCA
0.413
0.01
0.302
-0.008
orderings change, qualitatively, the smallest FLIPDD (-()) (o) estimates still represent less complex data
compared the highest (LIPD(:; fo), even for relatively large to- We hypothesize that for larger la. similar to the "string within a doughnut" experiment in Figure 3, the orderings correspond to baars--grained and semantic notions of complexity rather than fine-grained ones such textures, concepts that a metric such as PNG compression size cannot capture.
We also consider high-resolution images from LAION-Aesthetics [55] and, for the first time, estimate LID for extremely high-ding tigh-dimensional images with D = 3 x 512 x 512 = 786,432, We use Stable Diffusion [51]. latent DM pretrained on LAION-50 5B [55]. This includes an encoder and decoder trained to preserve relevant characteristics of the data manifold in latent representations. Since the encoder and decoder continuous and effectively invert each other, we argue that the Stable Diffusion encoder can, for practical purposes, be considered a topological embedding of the LAION- SB dataset into its latent space of dimension 4 x 64 x 64 16,384. Therefore, the dimension of the LAION-5B submanifold in latent space should be unchanged. We leave an empirical verification of this hypothesis to future work and thus estimate image LIDs by carrying out FLIPD in the latent space of Stable Diffusion. Here, we set the Hutchinson sample count to k = 1, meaning we only require a single
When we order a random subset of 1600 samples according to their FLIPD at to 0.3, the more complex images are clustered at the end, while the least complex are clustered at the beginning: see Figure and Figure 4c for the lowest- and highest-LID images from this ordering, and Figure 25 in Appendix E.6 to view the entire subset and other values of to. In comparison to orderings according to PNG compression size (Figure 4c), FLIPD prioritizes semantic complexity over low-level details like colouration.
Finally, we compare the runtimes for computing FLIPD and NB for all models using UNet backbones. We show results in Table 3. For 28 x 28 greyscale (MNIST/EMNIST) and
Table 3: Time, in seconds, estimate LID for a single image.
Method
MNIST/FMNIST SVHNICIFARI0 LAION
FLIPD
0.10
0.13
0.20
3 x 32 x 32 low-resolution RGB NB
1.6
10.8
> 9 x 103
(SVHN/CIFARIO) images, we use
k = 50 Hutchinson samples: at these dimensions, FLIPD achieves ~ 10x and ~ 100x respective speedups over NB. For 4 x 64 x 64 LAION-Aesthetics images on the latent space of Stable Diffu- sion, we use k- = 1 for FLIPD, which ensures it remains highly tractable. At this resolution NB is completely intractable: constructing S(x) for a single image x takes 2.5 hours, and computing NB would then still require performing a SVD on this 16, 384 x 65, 536 matrix.
5 Conclusions, Limitations, and Future Work
In this work we have shown that the Fokker-Planck equation can be utilized for efficient LID estimation with any pre-trained DM. We have provided strong theoretical foundations and extensive benchmarks showing that FLIPD estimates accurately reflect data complexity. Although FLIPD produces excellent LID estimates on synthetic benchmarks, its instability with respect to the choice of network architecture on images is surprising, and results in LID estimates which strongly depend on this choice. We see this behaviour as limitation, even if FLIPD nonetheless still provides a meaningful measure of complexity regardless of architecture. Given that FLIPD tractable, differentiable, and compatible with any DM, we hope that it will find uses in applications where LID estimates have already proven helpful, including OOD detection, Al-generated data analysis, and adversarial example detection.
10

---

References
[1] Luca Albergante, Jonathan Bac, and Andrei Zinovyev. Estimating the effective dimension of large biological datascts using fisher separability analysis. In 2019 International Joint Conference on Neural Networks, pages 1-8. IEEE, 2019.
[2] Alastair Anderberg. James Bailey, Ricardo JGB Campello, Michael E Houle, Henrique Marques, Miloš Radovanović, and Arthur Zimek. Dimensionality-ane aware outlier detection. Proceedings of the 2024 SIAM International Conference on Data Mining, pages 652-660, 2024.
[3] Brian DO Anderson. Reverse- timese diffusion equation models. Stochastic Processes and their Applications. 12(3):313-33 130366,,,,,,, 1982.
[4] Alessio Ansuini, Alessandro Laio, Jakob Macke, and Davide Zoccolan. Intrinsic dimension of data representations in deep neural networks. In Advances in Neural Information Processing Systems, 2019.
[5] Jonathan Bac, Evgeny M. Mirkes, Alexander N. Gorban, Ivan Tyukin, and Andrei Zinovyev. Scikit- Dimension: Python Package for Intrinsic Dimension Estimation. Entrppy, 23(10):1368, 2021.
[6] Atilum Günes Baydin, Barak A Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. Automatic Differentiation in Machine Learning: Survey. Journal of Machine Learning Research, 18: 43, 2018.
[7] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation Icarning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1788)1179, 2013.
[8] Tolga Birdal, Aaron Lou, Leonidas J Guibas, and Umut Simsekli. Intrinsic dimension, persistent homology and generalization in neural networks. Advances in Neural Information Processing Systems, 2021.
[9] Bradley CA Brown, Jordan Juravsky, Anthony L Caterini, and Gabriel Loaiza-Ganena Relating regulariza- tion and generalization through the intrinsic dimension activations. arXiv:2211.13239, 2022.
[10] Bradley CA Brown, Anthony L Caterini, Brendan Leigh Ross, Jesse C Cresswell, and Gabriel Loaiza- Ganem. Verifying the union of manifolds hypothesis for image data. International Conference on Learning Reprexentations, 2023.
[11] D. Burago, Y. Burago, and S. Ivanov. A Course in Metric Geometry. American Mathematical Society, 2022.
[12] Richard Cangelosi and Alain Goriely. Component retention principal component analysis with applica- tion to edna microarray data. Biology Direct, 2:1-21, 2007.
[13] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems, 2018
[14] Rob Cornish, Anthony L. Caterini, George Deligiannidis, and Arnaud Doucet. Relaxing bijectivity constraints with continuously indexed normalising flows. In International Conference on Machine Learning. pages 2133-2143, 2020.
[15] Jesse C Cresswell, Brendan Leigh Ross, Gabriel Loaiza- Ganee, Humberto Reyes-Gonzalle Marco Letizia, and Anthony I. Caterini. CaloMan: Fast generation of calorimeter showers with density estimation on learned manifolds, vrXiv:2211.15380 2022.
[16] Valentin De Bortoli. Convergence of denoising diffusion models under the manifold hypothesis. Transac- tions on Machine Learning Research, 2022.
[17] Laurent Dinh, Jascha Sohl- Sohl-Dickkttin, and Samy Bengio. Density estimation using Real NVP. In Interna- tional Conference on Learning Representations, 2017.
[18] Conor Durkan, Artur Bekasov, lain Murray, and George Papamakarios. Neural spline flows. In Advances in Neural Information Processing Systems, 2019.
[19] Conor Durkan, Artur Bekasov, Iain Murray, and George Papamakarios. nflows: normalizing flows in PyTorch, November 2020. URL https://doi org/ 10. www.///eoooo..... 4296287.
[20] Elena Facco, Maria d' Errico, Alex Rodriguez. and Alessandro Laio. Estimating the intrinsic dimension of datasets by minimal neighborhood information. Scientific Reports, 7():12110, 2017.
[21] Kenneth Falconer. Fractal Geometry: Mathematical Foundations and Applications. John Wiley & Sons, 2007.

---

[22] Keinosuke Fukunaga and David R Olsen, An algorithm for finding intrinsic dimensionality of data. IEEE Transactions on Computers, 100.2)176-183,, 1971.
[23] Frank E Harrell Jr, Kerry Lee, and Daniel B Mark. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in Medicine, 1544)336 1996.
[24] UG Haussmann and E Pardoux. Time reversal of diffusions. The Annals of Probability, 14(4):1188-1200) 1986.
[25] Jonathan Ho, Ajay Jain, and Picter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, 2020.
[26] Christian Horvat and Jean-Pascal Pfister. On gauge freedom, conservativity and intrinsic dimensionality estimation in diffusion models. In International Conference on Learning Representations. 2024.
[27] Michael E Houle, Erich Schubert, and Arthur Zimek. On the correlation between local intrinsic dimension- ality and outlierness. In Similarity Search and Applications: 11th International Conference, SISAP 2018, pages 197-191. Springer, 2018.
[28] Witold Hurewicz and Henry Wallman. Dimension Theory (PMS-4). Princeton University Press, 1948.
[29] Michael Hutchinson. A stochastic estimator the trace of influence matrix for Laplacian smoothing splines. Communications in Naaistics-Simulation and Computation, 1859:1055- 1989.
[30] Kerstin Johnsson, Charlotte Soneson, and Magnus Fontes. Low bias local intrinsic dimension estimation from expected simplex skewness. IEFE Transactions on Pattern Analysis and Machine Intelligence, 37(1):
196-202, 2014.
[31] Hamidreza Kamkari, Vahid Balazadeh, Vahid Zehtab, and Rahul G. Krishnan. Order-based structure learning with normalizing flows. wrXiv:2308.074800 2024.
[32] Hamidreza Kamkari, Brendan Leigh Ross, Jesse Cresswell, Anthony Caterini, Rahul G Krish- nan, and Gabriel Loaiza-Ganem. A geometric explanation of the likelihood OOD detection paradox. arXiv:2403. 18910, 2024.
[33] Diederik Kingma and Max Welling. Auto-encoding variational Bayes. International Conference on Learning Represemtations, 2014.
[34] Polina Kirichenko, Pavel Izmailov, and Andrew G Wilson. Why normalizing flows fail to detect out-of- distribution data. In Advances in Neural Information Processing Systems, 2020.
[35] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical Report, 2009.
[36] Yann LeCun. Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 (1K11)):278-221 2324, 1998.
[37] John M Lee. Introduction Smooth Manifolds, Springer, 2nd edition, 2012.
[38] Elizaveta Levina and Peter Bickel. Maximum likelihood estimation of intrinsic dimension. In Advances in Neurul Information Processing Systems, 2004.
[39] Gabriel Loaiza- Loaiza-Ganem, Brendan Leigh Ross, Jesse C Cresswell, and Authony L. Caterini. Diagnosing and fixing manifold overfitting in deep generative models. Transactions on Machine Learning Research, 2022.
[40] Gabriel Loaiza-Ganem, Brendan Leigh Ross, Rasa Hosseinzadeh, Anthony Caterini, and Jesse Cresswell. Deep generative models through the lens of the manifold hypothesis: A survey and new connections, urXiv:2404.02955 2024.
[41] Yubin Lu, Zhongjian Wang, and Guillaume Bal. Mathematical analysis of singularities in the diffuusio model under the submanifold assumption. arXiv:2301 007882, 2023.
[42] Xingjun Ma, Bo Li, Yisen Wang, Sarah M Erfani, Sudanthi Wijewickrema, Grant Schoeneheck, Dawn Song, Michael E Houle, and James Bailey. Characterizing adversarial subspaces using local intrinsic dimensionality. In International Conference on Learning Representations, 2018.
[43] David JC MacKay and Zoubin Ghahramani. Comments on "Maximum likelihood estimation of intrinsic dimension' by E. Levina and P. Bickel (2004). The Inference Group Website, Cavendish Laboratory, Cambridge University, 2005.
12

---

[44] German Magai and Anton Ayzenberg. Topology and geometry of data manifold in deep learning. arXiv:2204.0860 08624, 2022.
[45] Philipp Mayr and Vivien Petras. Cross-concordances: terminology mapping and its effectiveness for information retrieval, arXiv:0806.3765, 2008.
[46] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
[47] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, terh-peeformacce deep learning library. Advances Neural Information Processing Systems, 2019.
[48] Jakiw Pidstrigach. Score-based generative models detect manifolds. In Advances in Neural Information Processing Systems, 2022.
[49] Phil Pope, Chen Zhu, Ahmed Abdelkader, Micah Goldblum, and Tom Goldstein. The intrinsic dimension of images and its impact learning. In International Conference Learning Representations, 2021.
[50] Danilo Rezende and Shakir Mohamed. Variational Inference with Normalizing Flows. In Proceedings of the 32nd International Conference on Machine Learning, volume 37. pages 1530-1538, 2015.
[51] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High-resolution image synthesis with latent diffuusin models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
[52] Olaf Ronneherger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention-MIICAA 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234-241. Springer, 2015.
[53] Simo Särkkä and Amo Solin. Applied stochastic differential equations. Cambridge University Press, 2019.
[54] Ville Satopaa, Jeannie Albrecht, David Irwin, and Barath Raghavan. Finding "kneedle" in Haystack: Detecting Knee Points in System Behavior. In 31st International Conference on Distributed Computing Systems Workshops, pages 166-171. IEFF, 2011.
[55] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35:25278 252944, 2022.
[56] Jascha Sohl- Dickstinn,, Eric Weiss, Niru Maheswaranathan. and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256-2265, 2015.
[57] Yang Song, Conor Durkan, lain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. In Advances in Neural Information Processing Systems, 2021.
[58] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
[59] Jan Stanczuk, Georgios Batzolis, Teo Deveney, and Carola-Bibiane Schönlieh. Your diffusion model secretly knows the dimension of the data manifold. 12Xiv:2212.12611, 2022.
[60] Harald Steck, Balaji Krishnapuram, Cary Dehing-Oherije, Philippe Lambin, and Vikas C Raykar. On ranking in survival analysis: Bounds on the concordance index. In Advances in Neural Information Processing Systems, volume 20, 2007.
[61] Júlia Teles. Concordance coefficients to measure the agreement among several sets of ranks, Journal of Applied Statistics, 39(8): 1148))1749-1)) 1764, 2012.
[62] Piotr Tempezyk. Rafal Michaluk, Lukasz Gamcarek, Przemyslaw Spurek, Jacek Tabor, and Adam Golinski. LIDL: Local intrinsic dimension estimation using approximate likelihood. In International Conference on Machine Learning, pages 2123-1 2022.
13

---

[63] Piotr Tempczyk, Lukasz Garncarek, Dominik Filipiak, and Adam Kurpisz. A Wiener process perspective local intrinsic dimension estimation methods. arXiv:2406. 17125, 2024.
[64] Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Daniil Cheriaavkii, Sergey Nikolenko, Evgeny Burnaev, Serguei Barannikov, and Irina Piontkovskaya. Intrinsic dimension estimation for robust detection of gengenedd texts. Advances Neural Information Processing Systems, 2023.
[65] Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative modeling in latent space. In Advances in Neural Information Processing Systems, 2021.
[66] Lucrezia Valeriani, Diego Doimo, Francesca Cuturello, Alessandro Laio, Alessio Ansuini, and Alberto Cazzaniga. The geometry of hidden representations large transformer models. Advances Neural Information Processing Systems, 2023.
[67] Pascal Vincent. connection between score matching denoising autoencoders. Neural Computation, 23(7):1611-1674 2011.
[68] Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, and Thomas Wolf. Diffusers: State-of-the-art diffusion models. attps://githur com huggingface/ http//dfffuueeeeeeeeeeeeee 2022.
[69] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07743 2017.
[70] Fanghui Xue, Biao Yang, Yingyong Qi, and Jack Xin. Searching intrinsic dimensions vision transformers. arXiv:2204.07722, 2022.
[71] Yijia Zheng, Tong He, Yixuan Qiu, and David P Wipf. Learning manifold dimensions with conditional variational autoencoders. In Advances in Neural Information Processing Systems, 2022.
[72] Wei Zhu, Qiang Qiu, Jiaji Huang, Rohert Calderbank, Guillermo Sapiro, and Ingrid Daubechies. Ldmnet: Low dimensional manifold regularized neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2743-2751, 2018
14

---

A Explicit Formulas
A.1 Variance-Exploding Diffusion Models
Variance-exploding DMs are such that f(x,t) = with being non-zero. In this case [58]:
(t)) 1, and d² S/" (u)
(17)
Since y is non-zero, 9² is positive, so that o² is increasing, and thus injective. It follows that also injective, so that À = a/v6 is injective. Equation 15 then implies that
FLIPD (o) D (to) [tr (Vs(z,to)) + (s((x,t))l3
(18)
A.2 Variance-Pressrrinng Difftusion Models (DDPMs)
Variance-presse preservip DMs are such that
f(r, 3 and g(t) vB(t),
(19)
where B is positive scalar function. In this case [58]:
(b(1) = B(0, and o²(1)= B()), where B(I) = L// B(u)du.
(20)
We then have that
o²(() (1)
A(1)
VBB 1.
(21)
Since 3 positive, B is increasing and thus injective, from which follows that À injective as well. Plugging everything into Equation 15, obtain:
FLIPD(x, = D + (1-e e-B(o))) (Ir (Vo(e
- B(w))) (V x, 2(00) 2,to)) + l|s(e - x, 1o) 1o) (22)
A.3
Diffusion Models
Sub-variance--ressrving DMs are such that
f(x,t) 1 -()), and g(t) 3(t) (1- 2B(()), where B(t) ''' 3(u)du, (23)
and where B is a positive scalar function. In this case [58]:
((t) - B(t) and o² (1-e-B(0)?
(24)
We then have that
at)) 2 B(0) - B(t) 2 sinh ( (²B(t)
X(t)
(25)
(t)
Since 3 is positive, B is increasing and thus injective, from which it follows that is injective as well due to the injectivity of sinh. Plugging everything into Equation 15, we obtain:
FLIPD(x,to) = =D + (1 (1-e-Biee -B(oo)
(Vs(E B(t)) x. to)) + llo(E _000)) x. to) 112) (26)
15

---

B Proofs and Derivations
B.1 Derivation of Equation 12
We have that:
/ prrni ( (+(t()))= zo p(z0. 0)dza
(()(t(6))2, ((0))
(27)
/ (((((())) = ((20, Ojdro
(28)
(((())) / N 0:00 (((6)) In plro. 0)dmo
(29)
((((5)) In) p(ro, O)dro
(30)
(((O)))
M( 20 ID) p(z, 0)dzo v((())) ((10)) 0(2,)) 8),
0(z,
(31)
v((6)))
where used that (ar; azo, o² ID) Mo; zo. (o) along with the definition X(t) = (t) (t)/b(tt. It is thus easy to see that taking logarithms yields Equation 12.
B.2 Derivation of Equation 13
First, we recall the Fokker- equation associated with the SDE in Equation 1, which states that:
p(2,,t)= p((,, t) V (f(,t), Vp(a, 1 "0 tr (V)p(z,t)) (32)
We begin by using this equation to derive 0/0t log p( t). Noting that Vp(z,t) p(z,t)s(zzt) we have that:
tr (v'((z, t)) = tr (V t)s(,,,))(({,))) t))) p(=,)) tr (Vs(z,t) + (s(,,
(33)
-((x (V - +1)
(34)
Because
p(z, = p(z,( t) 0t
logp(z,t).
(35)
it then follows that
Ot log p(x, = = IV f(x,t)] - (f(z, t), s(z, t)) + 1 g (t) for (Vs(x,t)) + [|s(z, +) (36)
Then, from Equation 12 and the chain rule, we get:
d
log elx, 6)
D Dog (t(6)) log ((((6))2,
(37)
06
dō
D
d dz
log ((((1))
T (1((0))
logr
( at
o
28 (60
(38)
log
1
Dt (((())
(((()))))
]
88 (()
a
logp logp( ((ee(((()))))))))))
(39)
(1(8))
(0))
(1()))
(v(((5)
D
xr, (U(10))=,
88
at
+ log (1(((66) -.))))
(40)
16

---

Substituting Equation 36 into Equation 40 yields:
0
a
( (((0))
a a
( v(((6)) (()0((())))))))))))
D
log 0l2,8) 8)
[as (6)
08
v ((((1)))))) (M((()))))))) (v(((6)))
1 (100) (r (v (vo(v(ro)))))))) (((((())))t))))((((((((())())))) (1(((10)))) (41)
From on, to simplify notation, when dealing with scalar function h, we will denote its derivative as h' Since t(6) = X (c"), the chain rule gives:
X(((6)
00 ((6) = x()=1(e0)) x((())
(42)
06
So far, we have not used that f(x,t) t) b(t)æ, which implies that V f(x, t) Db(t) and that (f(z,t), s(x,t) = ((t s(x, Using these observations and Equation 42, Equation becomes:
0
log e(v, 0) = A((())
(((())
s(co)) D (((())
06
x(()))
((v(((())) - (xtt + =1((((((0)))000001))
1 (((O)) (1r (c(V=(V((0))) (0l))//))))))((()))))......)))))
(43)
If we showed that
A(t) 2x(1) 92 -3 (1)===20)))
(1)) - b(1)((() = 0, and that
(44)
for every I, then Equation 43 would simplify to Equation 13. From equation 5.50 in [53], we have that
$ (f) b(t)
(45)
which shows that indeed ((()-b(()))) 0. Then, from equation 5.51 in [53], we also have that
(o") (t) (())))))
+
(46)
and from the chain rule this gives that
o d'(1) 2b(1)o²(1) + g()
(47)
2t
We now finish verifying Equation 44. Since A(1) = (t)/ (t). the chain rule implies that
a())
X(I
s(t)
g2(1)
(1))(1)
g
(t)
(48)
(x)
(t())((t) - (t)
2
d'(t)((t) (t)b(t)))(t 2
(t)
(()
9²(1)
o(1)
g²(1)
(49)
(((t)-((tott) 2
2b(t)o + (t) ((t)
2
20(() (t)
(t)
g (t) o2(1),
(50)
2b()) + (t) 2b(t)=2(t)
2
which, as previously mentioned, shows that Equation 43 simplifies to Equation 13.
17

---

B.3 Proof of Theorem 3.1
We begin by stating and proving lemma which we will later use.
Lemma B.1. For any 0 and & O, there exists A < 0 such that for all A and y Rd with llvll2 > S. it holds that:
N Ia) < c e
(51)
Proof. The inequality holds if and only if
d log (2x) d8 x-26 11v12 < loog + 26,
(52)
which in turn equivalent
11 > 2 2 -28 - loog
l2g(2x)
d E20.
(53)
The limit of the right hand side as + oo 0 (and approaches from the positive side), while lly b is lower bounded by g?, thus finishing the proof.
We now restate Theorem 3.1 for convenience:
Theorem 3.1 (FLIPD Soundness: Linear Case), Let L be embedded submanifold of Th. given by a d-dimensional affine subspace. If p(: 0) is supported on L, continuous, and with finite second moments, then for any L with p(z, > O, we have:
lim
log e(x, 6) D.D.
(16)
08
Proof. As the result is invariant to rotations and translations, we assume without loss of generality
that L = {xx,, 0) E RDD x' E Rd, 0 E DD-4). Since x L, has the form æ = (x', 0) for some
Rd Note that formally p(-, not density with respect to the D-dimensional Lebesgue measure, however, with a slight abuse of notation, we identify it with p(z'), where p(-) now the d-dimensional Lebesgue density of X'. where (X',0) X p(,, 0). We will denote p()) Wi 0, Lx) ps(-). In this simplified notation, our assumptions that p(+) continuous at x' with finite second moments, and that x' is such that p(x')= > 0.
For ease of notation we use No represent the normal distribution with variance e2 on a d- dimensional space, i.c. (-)-) N( e26 e2) For any subspace S we use S denote its complement where the ambient space is clear from context. (0) denotes a ball of radius & around the origin.
We start by noticing that the derivative of the logarithm of Gaussian with respect to its log variance be computed as:
a
à log Nz = 06
(-5 log(2n) dô
|| [1a1 = -d +e 26 ||
--26
(54)
26
2
We then have that:
e 6) p5(x') x N
(55)
log e(x, 8) log (x') 8(D d) + Co
(56)
a
log e(x, 6) a8
log (x (D d).
(57)
08
18

---

where Co is a constant that does not depend on 8. Thus, it suffices to show that
limg---o 58
0 log ps (x') = O:
Po('
lim log pa()')
lim
06
(58)
6--
08
8--000
Ps(x')
26 Sa plx' y) (
lim
(59)
6-2-00
Jaa plx (u)dy
JRa plx' v) 20 N2
lim
(60)
80
SR p(1 y)NNa (d)
SRa v)( e 26 11 W
lim
(61)
I p(x' y) (v) dy
28 Igo p(x v) llv 1 N
d lim
(62)
Jg p(x' v)
00
where we exchanged the order of derivation and integration in Equation 60 using Leibniz integral rule (because the normal distribution, its derivative, and are continuous; note that p does not depend on ò so regularity on its derivative is not necessary), and where Equation 61 follows from Equation 54, Thus, proving that
25 Jaa p(x' w) N9
lim
(63)
JR p(xx - y) v)
would finish our proof.
Now let > O. By continuity of at x', there exists & > 0 such that if ||l|| < E, then
Ip(a' y) p(x')l < e. Let A be the corresponding A from Lemma B.1. Assume 6 < A and define
e1 and C2 as follows:
C1 - p(a' (2)dy, and 22 BE(O)
/ pla' -)) 1l (2)) (64) BB(0)
From Lemma B.I and p having finite second moments it follows that 0₁ E [0, 1] and that 22 E [Re 1lv 1 y)dy We have:
e 28
JKy p(z' v) llvl N 28 28 [B.1 p(z' v) 10ll + eze
(65)
Se pls' - (y)
SE p(a' y) (v) dy + cie
e 26 p(x') (o) 11 Ny + 28 SB (p(x' - v) - p(x')) N + oze
p(x') JB N2 (y) SB((O) (pla' -1) -+xx((()))))) dy) Cie
(66)
Analoguously to ej and 62, there exists cg 1] and c4 E [-d, d] so that the Equation 66 is equal
to:
28
SB (W) +
(cstesle p(a")
d-e 226 (gg(o) ll 2 (g)dy + loatoule
p(r))
I. (67)
SB..o N2 + (ertas)e p(x')
Jayto) V2(u)dy + (c) tea) p(x')
We still need to prove that lim5- I = d. Taking lim sup
and lim inf as + 00 yields:
tc
d te'tefle
d
Mar)
< fim inf lim sup 1 2
p(r))
(68)
te"te pla*)
84-20
tc
p(2))
where e/' = lim -0 &i and of = lim in G. Note that although the values of cf and c depend on e and ç, the bounds on them do not. We can thus take the limit of this inequality & and approach zero:
d d lim lim lim lim sup S d.
(69)
c.8 -0 8---00
803-00
19

---

However, note that every step up to here has been an equality, therefore
Saa plu' y) I1v12 (v)dy SR p(a' - vi (y)dy
(70)
so that does not depend on nor on &. In turn, this implies that lim inf I I lim supsupl S d = 8- -00
lim d, 6-2-00
(71)
6-2-00
which finishes the proof.
20

---

Table 4: Comparing discretized DDPM notation with score-based DM side-by-side.
Term
DDPM [25]
Score- based DM (58)
Timestep
t E (0, I,..,.......... 1....
1/T = t [0, 1]
(Noised out) datapoint
2//7 = 2t
Diffusion process hyperparameter
3
B(1/T) 3(t)
Mean of transition kernel
*(1/T) = ((()
Std of transition kernel
V1 de -(a, t)/vt-/vvv/a
a(/TT) = =((()
Network parameterization
8(2,t/I) 3(2,8)
C
Adapting FLIPD for DDPMs
Here, we adapt FLIPD for state-of-the-an art DDPM architectures and follow the discretized notation from Ho al. [25] where instead of using a continuous time index from 0 1, timestep belongs instead to the sequence .0...., I with I' being the largest timescale. We use the colour gold to [25]. We highlight that the content of this section is summary
indicate the notation used by Ho of the equivalence between DDPMs and the score-based formulation established by Song et al. [58].
As reminder, DDPMs can be viewed as discretizations of the forward SDE process of DM, where the process turns into Markov noising process:
p 2x-1) (2 VI Be (e-1,,BIIp.
(72)
We also use sub-indices instead of functions evaluated at t to keep consistent with Ho et al. [25]'s notation. This turn implies the following transition kernel:
pl (D) N(a Voizo, (1 - a.)
(73)
where 0e == 1 - Be and ā₂ = II
DDPMs model the backward diffusion process (or denoising process) by modelling a network
6: R x .,, 7} > TRD that takes in a noised-out point and outputs residual that can be used to denoise. Song et al. [58] show that one can draw an equivalence between the network <(,, () and the score network (see their Appendix B). Here, we rephrase the connections more explicit manner
where we note that:
-c(x, ((x,t)/VI- a; = s(x,t/T).
(74)
Consequently, plugging into Equation 22, we get the following formula adapted for DDPMs:
FLIPD to) = D -I- ãig tr (Ve( (Ve(var, - to)) + ||e(v 2, to)
(75)
where to to T (best viewed in colour). We include Table 4, which summarizes all of the equivalent notation when moving from DDPMs to core-based DMs and vice-versa.
21

---

Table 5: Essential hyperparameler settings for the Diffusion models with MLP backbone.
Property
Model Configuration
Learning rate
10
Optimizer
AdamW
Scheduler
Cosine scheduling with 500 warmup steps [68] 200, 400, 800, or 1000 based on the ambient dimension
Epochs
Score-matching loss
Likelihood weighting [57]
SDE drift
f(z, t) = }B(()x
SDE diffusion
(t = /B(t)
3(1)
Linear interpolation: ((() := 0.1 201
Score network
MLP
MLP hidden sizes
(4096, 2048, 2 x 1024, 3 x 512, 2 x 1024, 2048, 4096) 128
Time embedding size
D Experimental Details
Throughout all our experiments, we used NVIDIA A100 GPU with 40GB of memory.
D.I DM Hyperparameter Setup
To mimic a UNet, we use an MLP architecture with a bottleneck as our score network for our synthetic experiments. This network contains L fully connected layers with dimensions "hn,h,,...,11
h2r-1) forming a bottleneck, i.e., hr. has the smallest size. Notably, for 1 S < L, the ith transform connects layer dimensions "h,-1 x h2". and the (L + i)th layer (or the output) not only contains input from the (L+i- 1)th layer but also, contains skip connections from layer (L- i) (or the input), thus forming a linear transform of dimension "(hu+t-1 ht-i) x hL++"" For image experiments, we scale and shift the pixel intensities be zero-centered with a standard deviation of 1. In addition, we embed times t E (0, 1) using the scheme in [68] and concatenate with the input before passing to the score network. All hyperparameters are summarized Table
(or the input) layer with linear transform of
D.2 FLIPD Estimates and Curves for Synthetic Distributions
Figure 5 shows pointwise LID estimates for "lollipop" distribution taken from [62]. It is uniform distribu- tion over three submanifolds: (i) 2d "candy", (id) 1d "stick", and (iti) an isolated point zero dimensions. Note that the FLIPD estimates to 0.05 for all three submanifolds are coherent.
2.0
-1.5
FLIPD( 0.05)
1.0
Figure 6 shows the FL.IPD curve as training progresses the lollipop example and the Gaussian mixture that was already discussed in Section 4. We see that gradually, knee patterns emerge at the correct LID, indicating that the DM learning the data manifold. Notably, data with higher LID values get assigned higher estimates even after a few epochs, demonstrating that FLIPD effectively ranks data based on LID, even when the DM is underfitted.
0.5
0.0
Figure 5: The FLIPD estimates on a Lol- lipop dataset from [62].
Finally, Figure 7 presents summary of complex man- ifolds obtained from neural spline flows and high- dimensional mixtures, showing knees around the true LID.
22

---

Avg. for LID
FLIPDC- Ao)
Avg. fiD
Avg. for LTD
FLIPD(.
FLIPDC,
0.00
25
0.50
0.75
1.00
0.25
0.50
1.00
0.00
0.25
0.50
0.75
1.00
to
to
(a) After 10 epochs.
(b) After 20 spochs.
(0) After 100 epochs.
10
Avg: for LID
10
10
FLIPDI: (t)
Avg. for LID 4
8
Avg. for LID
FUPD!,
FLIDD
0.00
0.25
0.50 To
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0,50
0.75
1.00
Eg
1g
(d) epochs.
After epachs.
(0
epochs.
Figure The evolution the FLIPD curve while training DM to Lollipop (top) and manifold mixture N2 + M4 + Ng C R2th (bottom).
800
Avg. for LID 100
1000
100
Avg. for LID 10
FLIPDC, kn)
FLIPDK Aa)
900
Avg. for LID 25
FLIPD(,
Avg. for LID 50
400
25
Aug.
10
0
0.0
0.2
0.4
0.6
0.8
0.00
0.25
0.50
0.75
1.00
0.0
0.2
0.4
0.8
1.0
0g
to
F400
R800
(b)
2600
1000
(c)
Nio + A2s No 100
Figure 7: The FLIPP curve for complex and high-dimensional manifolds.
D.3 A Simple Multiscale Experiment
Tempezyk et al. [62] argue that when setting 8, all the directions of data variation that have log standard deviation below d are ignored. Here, we make this connection more explicit.
We define multivariate Gaussian distribution with prespecified cigenspectrum for its covariance matrix: having three cigenvalues
of 10-4, three eigenvalues of and four eigenvalues of 103. This ensures that the distribution is numerically 7d and that the directions and amount of data variation are controlled using the
FUPD
7
Avg. for Trained
eigenvectors and eigenvalues of the covariance matrix.
Avg. for Closed-form
For this multivariate Gaussian, the can be written in closed form; thus, we evaluate FLIPD both with and without training DM. We see in Figure the estimates obtained in both scenarios match closely, with some deviations due imperfect model fit, which we found matches perfectly when training for longer.
function in Equation 13
0.3
Figure 8: FLIPD curve for a multi- variate Gaussian with controlled co- variance eigenspectrum.
Apart from the initial knee 7, which is expected, find another at to 0.6 (corresponding to e zo 6.178 with our hyperparameter setup in Table 5) where the value of FLIPD 4. This indeed confirms that the estimator focuses solely the 4d space characterized by the eigenvectors having eigenvalues of 1 and by ignoring the eigenvalues 10-4 and which are both smaller than 6.178.
D.4 In-depth Analysis of the Synthetic Benchmark
Generating Manifold Mixtures To create synthetic data, we generate each component of our manifold mixture separately and then join them to form a distribution. All mixture components are sampled with equal probability in the final distribution. For component with the intrinsic dimension
23

---

of d, we first sample from base distribution in d dimensions. This base distribution is isotropic Gaussian and Laplace for N₂ and La and uniform for the case of Un and Fa- We then zero-pad these samples to match the ambient dimension D and perform a random D x D rotation on R (each component has one such transformation). For Fd, we have an additional step to make the submanifold complex. We first initialize a neural spline flow (using the nf nflows library [19]) with 5 coupling transforms, 32 hidden layers, 32 hidden blocks, and tail bounds of 10. The data is then passed through the flow, resulting in a complex manifold embedded in I 2 with an LID of d. Finally, we standardize each mixture component individually and set their modes such that the pairwise Euclidean distance between them is at least 20. We then translate each component so its barycenter matches the corresponding mode, ensuring that data from different components rarely mixes, thus maintaining distinet LIDs.
LIDL Baseline Following the hyperparameter setup in [62], we train 8 models with different noised-out versions of the dataset with standard deviations ch {0.01, 0.014, 0.019, 0.027, 0.037, 0.052, 0.072, 0.13. The normalizing flow backbone is taken from [19], using 10 piecewise rational quadratic transforms with 32 hidden dimensions, 32 blocks, and tail bound of 10. While [62] uses an autoregressive flow architecture, we use coupling transforms for increased training efficiency and to match the training time of single DM.
Setup We use model-free estimators from the skdim library [5] and across our experiments, we sample 10 points from the synthetic distributions for either fitting generative models or fitting the model-free estimators. We then evaluate LID estimates on uniformly subsampled set of 2 points from the original set of 10 points. Some methods are relatively slow, and this allows us to have a fair, yet feasible comparison. We do not see significant difference even when we double the size of the subsampled set. We focus on four different model free baselines for our evaluation: ESS [30], LPCA [22, 12], MLE [38, 43], and FIS [1]. We use default settings throughout, except for high-dimensional cases (D > 100), where we set the number of nearest neighbours k to 1000 instead of the default 100. This adjustment is necessary because observed a significant performance drop, particularly in LPCA, when & was too small in these scenarios. We note that this adjustment has not been previously implemented in similar benchmarks conducted by Tempozyk et al. [62], despite their claim that their method outperforms model-free alternatives. We also note that FIS does not scale beyond 100 dimensions. and thus leave it blank in reports. Computing pairwise distances high dimensions (D 2 800) on all 10 samples takes over 24 hours even with 40 CPU cores. Therefore, for D > 800, use the same 22 subsamples we use for evaluation.
Evaluation We have three tables to summarize our analysis: (4) Table 7 shows the MAE of the LID estimates, comparing each datapoint's estimate to the ground truth at a fine-grained level; (ii) Table 9 shows the average LID estimate for synthetic manifolds with only one component, this average is typically used estimate global intrinsic dimensionality in baselines; finally, (iii) Table 8 looks at the concordance index [23] of estimates for cases with multiple submanifolds of different dimensionalities. Concordance indices for sequence of estimates (LID),
n= are defined as follows:
c (ILID,) J (LID
Σ
I(LID,, < LIDma))
N
(76)
Ang SN LID.., SL.DDng
where I is the indicator function; perfect estimator will have a C of 1. Instead of emphasizing the actual values of the LID estimates, this metric assesses how well the ranks of an estimator align with those of ground truth [60, 45, 61, 31J, thus evaluating LID "relative" measure of complexity.
Model-free Analysis Among model-free methods, LPCA and ESS show good performance in low dimensions, with LPCA being exceptionally good in scenarios where the manifold is affine. As we see in Table 7, while model-free methods produce reliable estimates when D is small, as D increases the estimates become more unreliable. In addition, we include average LID estimates in Table 9 and see that all model-free baselines underestimate intrinsic dimensionality to some degree, with LPCA and MLE being particularly drastic when D 100. We note that ESS performs relatively well, even beating model-based methods in some 800-dimensional scenarios. However, we note that both the C values Table 8 and MAE values in Tables 7 and suggest that none of these estimators effectively estimate LID in pointwise manner and cannot rank data by LID effectively FLIPD.
24

---

Table 6: MAE (lower is better). Rows show synthetic manifolds and columns represent different variations of our Fokker-Planck- planec-bbsed estimators.
Syathetic Manifold
FLIPD
FLIPD kneedle
HPRegress kneedle
FPRegress
61
Lollipop in R
0.142
0.419
0.572
0.162
String within doughnut R²
0.052
0.055
0.398
0.377
Swiss Roll in R
0.053
0.055
0.087
0.161
Ls C RR
0.100
0.169
1.168
0.455
Nso C TR000
0.501
0.492
3.142
0.998
Uio + Us0 + Us0 C I2 100
3.140
1.298
5,608
10,617
Fio + F25 + F60
100
14.37
3.925
16.32
21.01
U10 Mao M200
C R 8
39.54
14.30
30.06
29.99
Model-based Analysis Focusing model-based methods, FLIPD perform poorly in ranking data based on LID. Remarkably, FLIPD achieves perfect values among all datasets; further justifying
in Table 8 that all except
relative measure of complexity. We also note that while LIDL and NB provide better global estimates in Table 9 for high dimensions, they have worse MAE performance in Table 7. This once again suggests that local level, our estimator superior compared to others, beating all baselines in 4 out of groups of synthetic manifolds in Table 7.
D.5 Ablations
We begin by evaluating the impact of using kneedle. Our findings, summarized in Table 6, indicate that while setting a small fixed to is effective in low dimensions, the advantage of kneedle becomes particularly evident the number of dimensions increases.
We also tried combining it with kneedle by sweeping over the origin o1 and arguing that the estimates obtained from this method also exhibit knees. Despite some improvement in high-dimensional settings, Table shows that even coupling with kneedle does not help.
D.6 Improving the NB Estimators with kneedle
We recall that the NB estimator requires computing rank S(x), where S(x) is K x D matrix formed by stacking the scores 8(., fo). We set to = 0.01 as it provides the most reasonable estimates. To compute rank S(x) numerically, Stanczuk et al. [59] perform a singular value decomposition on S(x) and use a cutoff threshold below which singular values are considered zero. Finding best is challenging, maximum gap. Furthermore, we see that sometimes the top few singular values are disproportionately higher than the rest, resulting in severe overestimations of the LID. Thus, we introduce alternative algorithm to determine the optimal т. For each т, we estimate LID by thresholding the singular values. Sweeping 100 different values from 1000 at geometric scale (to further emphasize smaller thresholds) produces estimates ranging from D (keeping all singular values) to (ignoring all). As varies, we that the estimates plateau over certain range of T. We kneedle to detect this plateau because the starting point of a plateau is indeed a knee in the curve. This significantly improves the baseline, especially in high dimensions: see the third column of Tables 7, 9, and compared to the second column.
Stanczuk al. [59] propose finding the two consecutive singular values with the
25

---

Table 7: MAE (lower better). Each row represents synthetic dataset each column represents an LID estimation method. Rows are split into groups based on the ambient dimension: the first group of rows shows toy examples; the second shows low-dimensional data with D = 10: the third shows moderate moderate-dimension dimensional data with D = 100; and the last two show dige-dimensooall data with D 800 and D 1000. respectively.
Model-based
Model-free
Synthetic Manifold
FLIPD
NB
NB
LIDL
ESS
LPCA
MLE
FIS
kneedle
Vanilla
kneedle
Lollipop C R2
0.419
0.577
0.855
0.052
0.009
0.000
0.142
0.094
Swiss Roll C R
0.055
0.998
0.016
0.532
0.017
0.000
0.165
0.018
String within doughnut C R
0.055
1.475
0.414
1.104
0.017
0.000
0.128
0.041
Summary (Toy Manifolds)
0.176
1.017
0.428
0.563
0.014
0.000
0.145
0.051
Ms C R
0.084
5.000
0.005
0.071
0.061
0.000
0.441
0.206
Ls C R
0.169
1.000
0.146
0.101
0.068
0.000
0.462
0.203
U6 S I220
0.324
4.994
0.933
0.123
0.153
0.000
0.451
0.186
Fs S R
0.666
1.216
0.765
0.487
0.168
0.000
0.497
0.176
N2 + + C R
0.287
5.706
1.078
0.308
0.156
0.000
0.406
0,206
C2 L4 La R
0.253
5.708
0.772
0.515
0.193
0.001
0.437
0.018
U1 u4 + Us SCC" R
0.586
5.677
3,685
0.363
0.331
0.115
0.540
0.222
F++ Fa I
0.622
5.709
2.187
1.013
0.428
0.115
0.612
0.269
Summary (10-dimensional)
0.066
0.381
0.161
0.211
0.005
0.000
0.054
0.019
Uu C R 100
0.910
30.115
0.000
1.370
0.644
0.000
1.263
Uso C R 100
0.505
50.521
0.000
0.542
1.465
0.002
7.622
Usu C R 100
0.640
1.157
1.327
0.332
2.034
21.90
39.65
Nso S 2200
0.887
52.43
0.000
0.892
0.534
0.000
5.703
Noo RR:00
0.492
0.184
2.693
0.329
1.673
21.88
39.45
Fso C B 100
1.869
20.00
3.441
1.871
3.660
16.78
34.41
Uio U1s Uso C R 100
0.868
57.96
0.890
5.869
4.988
6.749
16.12
Uio + U65 + Usa C R 100
1.298
61.58
1.482
8,460
21.89
20.06
41.05
Nio + 225 + Noo C 100
1.813
74.20
0.555
8.873
7.712
5.716
14.37
Fiu + F2s + Foo C 200
3.925
74.20
6.205
18.61
9.200
6.769
16.78
Summary (100-dimensional)
1.321
42.24
1.659
4.715
5.380
9.986
21.64
M200 C R 800
11.54
600.0
7.205
55.98
3.184
0.000
159.3
FaoD C 8000
20.46
400.0
10.15
207.2
14.48
2.919
342.6
Uno Uso U200 C R 800
14.30
715.3
18.82
120.7
1.385
0.004
72.25
Summary (800-dimensional)
15.43
571.8
12.06
128.0
6.350
0.974
191.4
No0n g 1000
3.913
100.0
24.38
10.45
14.16
219.8
819.2
210o C
R 1000
12.81
900.0
62.68
12.65
1.623
0.000
72.06
U9oo
C
R 1000
12.81
100.0
0.104
24.90
14.49
219.1
810.2
FooD
C R 1000
21.77
500.0
52.19
341.3
18.83
29.99
435.7
Summary (1000-dimensional)
12.83
400.0
34.84
97.33
12.28
117.2
534.3
26

---

Table Concordance index (higher is better with 1.000 being gold standard). Each row represents mixture of multi-dimennional manifolds, and each column represents an LID estimation method. This table evaluates how accurately different estimators rank datapoints based on their LID.
Synthetic Manifold
FLIPD
NB
NB kneedle
LIDL
ESS
LPCA
MLE
FIS
kneedle
Vanilla
Lollipop C R
1.000
0.426
0.394
0.999
1.000
1.000
1.000
1.000
String withing doughnut C R
1.000
0.483
0.486
0.565
1.000
1.000
1.000
1.000
M1 + N4 N S R
1.000
0.341
0.725
0.943
1.000
1.000
1.000
1.000
C2 L4 La _ 10
1.000
0.342
0.752
0.884
1.000
1.000
1.000
1.000
U1 - C R
1.000
0.334
0.462
0.903
1.000
1.000
1.000
1.000
+ Ç I
1.000
0.342
0.578
0.867
1.000
1.000
0.999
1.000
U10 Uso C R 100
1.000
0.467
0.879
0.759
0.855
0.897
1.000
+ + Umu C R 100
1.000
0.342
0.826
0.742
0.742
0.855
1.000
Nio + Nas + No
100
1.000
0.342
0.866
0.736
0.878
0.917
1.000
Fio + F25 + Foo 100
1.000
0.342
0.731
0.695
0.847
0.897
1.000
Uio Uso U200 C R
1.000
0.342
0.841
0.697
1.000
1.000
1.000
Table Analysis on manifolds with single global intrinsic dimension. Columns, categorized by whether or not they use a model, display various LID estimators. The first set of rows shows the average LID as global intrinsic dimension estimate and is grouped into lower- (D S 100) or higher-dimensionll (D) > 100) categories. Methods that most closely match the true intrinsic dimension are bolded. The second set of rows shows the pointwise precision of the estimators, indicated by MAE (lower is better): while previous baselines generally provide reliable global estimates, their pointwise precision is subpar, especially with higher dimensionality.
Model-based
Model-free
FLIPD
NB Vanilla
NB
LIDL
ESS
LPCA
MLE
FIS
Synthetic Manifold
Average LID Estimate
Swiss Roll C R3
2.012
2.998
1.984
2.527
2.008
2.000
2.013
2.003
Ms C 1
5.017
10.00
4.995
5.067
5.004
5.000
5.108
5.187
Cs C R
4.968
6.000
4.854
5.089
4.985
5.000
5.123
5.182
26 C 20
4.796
9.994
4.067
5.107
4.880
5.000
4.833
5.152
Fs S TR
4.722
6.216
4.461
5.182
1.890
5.000
4.829
5.131
Un C 1000 100
11.68
40.12
10.00
11.29
9.361
10.00
8.905
Uso R
31.01
80.52
30.00
30.08
28.54
29.99
22.38
Uso R 100
89.54
91.16
91.32
90.24 24
88.27
68.10
50.35
Ng0 C R 100
30.79
82.43
30.00
30.85
29.67
30.00
24.38
N80 C R 100
89.88
90.18
92.62
90.21
88.89
68.12
50.55
Fso C De 100
77.97
100.0
83.23
81.46
76.35
63.22
45.59
Mzoo C R 990
211.5
800.0
207.3 2
256.0
199.3
200.0
40.73
Faoo
C R 800
454.7
800.0
410.1
607.2
385.8
397.1
57.37
N800 C R 1000
890.3
1000.
924.4
924.4
897.3
680.2
80.78
Mi00 C R 1000
135.76
1000.
162.7
112.6
99.54
100.0
27.91
Usuo C R 1000
864.9
1000.
900.1
911.9
899.3
680.9
89.80
FroD C R 1000
582.7
1000.
552.2
841.3
481.4
470.0
64.29
Manifolds Summary
Pointwise MAE Averaged Across Tasks
R where < 100
0.633
15.20
0.924
0.561
0.954
5.506
11.83
R where D > 100
13.88
433.3
26.12
108.7
11.13
78.64
439.8
27

---

E Image Experiments
E.1 FLIPD Curves and DM Samples: A Surprising Result
Figures 9 and 10 show DM-generated samples and the associated LID curves for 4096 samples from the datasets. In Figure 10, the FLIPD curves with MLPs have clearly discemiibe knees at which the estimated LID averages around a positive number, as expected. When changing the architecture of the DMs from MLPs to UNets, however, the curves either lack knees or obtain them at negative values of LID. This makes it harder to set tg automatically with kneedle when using UNets, and thus makes the estimator non-robust to the choice of architecture. Note that our theory ensures that (LIPD( 4o) converges to LID(x:) as lo + 0 when & is an accurate estimator of the true score function; indeed, does not make any claims about the behaviour of FLIPD when to is large or when the model fit is poor. Yet, the fact that the FI.IPD) estimates obtained from UNets are less reliable than the ones from MLPs is somewhat surprising. especially given that DMs with MLP backbones clearly produce worse-looking samples (shown in Figure 9), suggesting that while the model fit is better in the former case, the LID estimates are worse. Throughout the paper, we have argued that, notwithstanding this, FLIPD estimates derived from both architectures offer useful measurements of relative complexity, and can still effectively rank data based on its complexity; refer to all the images in Figures 17, 18, 19, 20, 21, 22, 23, and 24. However, this discrepancy between FLIPD estimates in absolute terms when altering the model architecture is counterintuitive and requires further treatment. In what follows, we propose hypotheses for the signifieant variation in FLIPD estimates when transitioning from MIP to UNet architectures, and we encourage further studies to investigate this issue and to adjust FLIPD to better suit DMs with state-of-the-ar backbones.
While MLP architectures do not produce visually pleasing samples, comparing the first and second rows of Figure 9 side-by-side, it also becomes clear that MLP- MLP-generated images still match some characteristics of the true dataset, suggesting that they may still be capturing the image manifold in useful ways. To explain the unstable FLIPD estimates of UNets, we hypothesize that the convolutional layers in the UNet provide some inductive biases which, while helpful to produce visually pleasing images, might also encourage the network to over-fixate score functions can be interpreted as providing a denoiser which removes noise added during the diffusion process; our hypothesis is that UNets are partieularly good at removing noise along high- frequency directions but not necessarily along low-frequency ones. Inductive biases or modelling errors which make the learned score function deviate from might significantly alter the behaviour of FLIPD estimates, even if they result in visually pleasing images; for more formal treatment of the effect of this deviation, please refer to the work of Tempezyk al. [63], who link modelling errors to LID estimation errors in LIDL.
high-frequency features. More specifically,
The aforementioned inductive biases have been found to produce unintuitive behaviours in other settings as well. For example, Kirichenko et al. [34] showed how normalizing flows over- fixate on these high-freeuency features, and how this cause them to assign large likelihoods out- of-distribution data, even when the model produces visually convincing samples. We hypothesize that a similar underlying phenomenon might be play here, and that DMs with UNets might be over-emphasizing
directions of variation in their LID estimates. On the other hand, MLPs do not inherently distinguish between the high- and low-frequency directions and can therefore more reliably estimate the manifold dimension, even though their generated samples appear less appealing to the human eye. However, an exploration of this hypothesis outside the scope of our work, and we highlight once again that FLIPD remains useful measure of relative complexity when using either UNets or MLPs.
E.2 UNet Architecture
We use UNet architectures from diffusers [68], mirroring the DM setup in Table 5 except for the score backbone. For greyscale images, we use a convolutional block and two attention-baue based downsampling blocks with channel sizes of 1 128, 256, and 256. For colour images, we use two convo- lutional downsampling blocks (each with 128 channels) followed by two attention downsampling blocks (each with 256 channels). Both setups are mirrored with their upsampling counterparts.
28

---

E.3 Images Sorted by FLIPD
Figures 11, 12, 13, and 14 show 4096 samples of CIFARIO, SVHN, MNIST, and FMNIST sorted according to their FLIPD estimate, showing gradient transition from the least complex datapoints (e.g.. the digit in MNIST) the most complex ones (e.g., the digit 8 in MNIST). We use MLPs for greyscales and UNets for colour images but see similar trends when switching between backbones.
E,4 How Many Hutchinson Samples are Needed?
Figure 15 compares the Spearman's rank correlation coefficient between FLIPD while we use k E {1, 50} Hutchinson samples vs. computing the trace deterministically with D Jacobian-vector- products: (i) Hutchinson sampling particularly well-suited for UNet backhones, having higher correlations compared to their MLP counterparts; (fi) as to increases, the correlation becomes smaller, suggesting that the Hutchinson sample complexity increases larger timescales; (iit) for small to. even one Hutchinson sample is enough to estimate LID; (iv) for the UNet backbone, 50 Hutchinson samples are enough and have high correlation (larger than 0.8) even for to as large as 0.5.
E,5
More Analysis on Images
While we use 100 nearest neighbours in Table 2 for ESS and LPCA, we tried both with 1000 nearest neighbours and got similar results. Moreover, Figure 16 shows the correlation of FLIPD with PNG for to (0, 1), indicating a consistently high correlation at small (g. and a general decrease while increasing to- Additionally, UNet backbones correlate better with PNG, likely because their convolutional layers capture local pixel differences and high- tigh-frequency features, aligning with PNG's internal workings. However, high correlation is still seen when working with an MLP.
Figures 17, 19, 21, and 23 show images with smallest and largest FLIPD estimates at different values of to for the UNet backbone and Figures 18, 20, 22, and 24 show the same for the MLP backbone: (i) we see clear difference in the complexity of top and bottom FLIPD estimates, especially for smaller to: this difference becomes less distinct as to increases; (ii) interestingly, even for larger to values with smaller PNG correlations, we qualitatively observe clustering of the most complex datapoints the end; however, the characteristic of this clustering changes. For example, see Figures at to 0.3, or 22 and 24 at to = 0.8, suggesting that FI.IPD) focuses on coarse-grained measures of complexity at these scales; and finally (itt) while MLP backbones underperform in sample generation, their orderings are more meaningful, even showing coherent visual clustering up to 0.8 in all Figures 18, 20, 22, and 24; this is a surprising phenomenon that warrants future exploration.
E.6 Stable Diffusion
To test FLIPD with Stable Diffusion v1.5 [51], which is finetuned on a subset of LAION-Aesthetic and computed FLIPD scores for each.
we sampled 1600 images from
We ran FLIPD with to (0.01, 0.3, 0.8} and single Hutchinson trace sample. In all cases, FLIPD ranking clearly corresponded to complexity, though we decided upon to 0.3 best capturing the "semantic complexity" of image contents, All 1600 images for t = 0.3 are depicted in Figure 25. For all timesteps, we show previews of the lowest- and highest-I.ID images in Figure 26. Note that LAION is essentially collection of URLs, and some are outdated. For the comparisons in Figure 26 and Figure 4c, we remove placeholder icons or blank images, which likely correspond to images that, at the time of writing this paper, have been removed from their respective URLs and which are generally given among the lowest LIDs.
29

---

0
9
3
2
6
2
5
R
3
8
8
5
4
9
8
4
8
8
2
3
0
0
3
4
4 3 4
13
0
6
3
3
2
7
8
5
6
1
2
9
6
7
2
7
CIFARIO with UNet backbone
(h) SVHN with UNet backbone.
MNIST with UNet hackhone.
FMNIST with UNet backbonne
D
9
3
7
2
8
3
8
$
D
8
8
2
3
1
0
8
3
0
9
6
4
3
3
6
8
1
8
4
9
0
4
6
3
2
4 4
9
2
9
CIFARIO
backbone. SVHN backbone.
(g) MNIST MLP backbone.
FMNIST MLP backbone.
Figure Samples generated DMs with different score network backbones, using same seed for control. Despite the variation in backbones, images of the same cell in the grid (comparing top and bottom rows) show rough similaritics, especially on CIFAR 10 and SVHN.
3072
3072
784
fel
ELIPDC G.)
ELIDD: Gal
2000
2000
784
200
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
6
CIFARIO with UNet backbone.
(b) SVHN with UNet backbone.
(c) MNIST with UNet backbone.
(d) FMNIST with UNet backbone.
3072
3072
784
784
FUPDC.6)
FLIPDC... 4)
2000
2000
784
200
200
0.0
(e) CTFARID 10 with ML.P hackhone.
( SVHN with P backhone.
(g) MNIST with MI.P hackbane. (6) PMNIST with MLP hackhone.
Figure 10: FLIPD curves from all the different DMs with different score network backhones.
30

---

Figure 11: CIFARIO sorted (left right and top to bottom) by FLIPD estimate (UNet) at to 0.01.
31

---

O
SOST O
FEGGO
BIOSS
3 DE
SO DO WHA BB
DE SIG
ST TI SON
WAN
BIO BAUS
SESP AND BS
19 TI
AN AMI 7
TH O
NA WINSH AADIS
LO WUEF LANU POIN
WIR REA CO1 2
BO
ARE WHST 2A3 DIAL
AS
SOUAPS EES
RIS B BI77 MEEE ARR 211
CAE 65
AS
DOE
24
MEDE REG M
RE
T4K
Figure 12: SVHN sorted (left right and top to bottom) by FLIPD estimate (UNet) to 0.01.
32

---

/ 7
7 4
77
7
6 6
6
7
7 1 7
8 9
1477
9
# 4
8 4 4
4
# 4 9
7 8 0
3 3 7
5
/ 1
0
7 2 4
7 4 2
6 2
5
Y 8
< 2 
9 8 Q G w 0 4 X 0
0 4 0
8 O 
WAA.0OSTCCOOOOAAO0.............. 75 1
5 00
Figure MNIST sorted (left to right and top to bottom) by FLIPD estimate (MLP) at to 0.1.
33

---

Figure 14: FMNIST sorted (left to right and top to bottom) by FLIPD estimate (MLP) at to 0.1.
34

---

Spearmant' correlation
Spearman's cormlation
0.8
0.8
0.6
Model Backbone
UNet backbone
0.4
MLP backbone
# Hutchinson Samples
0.2
0.2
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
4g
Multiscale Spcarman's correlation CIFARIO.
Multiscale Spearman's correlation SVHN.
1.0
1.0
Spearman's correlation
Spearman correlation
0.8
0.6
0.2
0.2
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.4
0.6
0.8
(c) Multiscale Spearman's correlation on MNIST.
(d) Multiscale Spearman's 's correlation on FMNIST.
Figure 15: Spearman's correlation of FLIPD estimates while using different numbers of Hutchinson samples compared to computing the trace term of FLIPD deterministically with D Jacobian vector product calls. These estimates evaluated at different values of E (0, 1) four datasets using the UNet and MLP backhones.
Spearman's correlation
0.75
MLP backbone
UNet backhone
Spearmann's correlation
0.50
0.5
0.25
0.0
0.00
-0.25
0.5
-0.50
(a) Multiscale correlation with PNG on CIFARIO.
(b) Multiscale comeeatiin with PNG on SVHN.
Spearman's correlation
Spearman's correlation
0.6
0.4
0.4
0.2
0.2
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
to
tg
Multiscale correlation with PNG MNIST.
(6) Multiscale corrllation with PNG FMNIST.
Figure Spearman's correlation of FLIPD estimates with PNG as we sweep (0, 1) on different backbones and different image datasets.
35

---

(a) to 0.01.
(b) fo 0.1.
(c) kg 0.3.
(d) to 0.8.
Figure 17: The 204 smallest (top) and 204 largest (bottom) CIFAR10 FLIPD estimates with UNet evaluated at different lo.
(a) by 0.01.
(b) to 0.1.
(c) lg 0.3.
(d) lg 0.8.
Figure 18: The 204 smallest (top) and 204 largest (hottom) CIFAR10 FLIPD estimates with MT.P evaluated different
36

---

84
14
US5
22
16263
55
00
6
SSS
615
& 115
TO 633
SIL
G13
R0
1000
& H2 55
8
9136
76 8618 19
ESS 17
6
3344
M4
20D 121
6116
25
C
5 44
E4
1189
620
1129
4383
135
288
4999
00
5
15
86
18
78B1
1
45822
655 16
9919
336
(a) to = 0.01.
(b) fo = 0.1.
(c) tg
0.3.
(d) tg 0.8.
Figure 19: The 204 smallest (top) and 204 largest (bottom) SVHN FLIPD with UNet estimates evaluated at different lo-
77
A 7 &S A 6E
3 &
TH
15 HI AIZ 10 AOOIIOO CHO
M 6 HOO
SSSIO LO 7
8
IBS OS
S7 ESSIO SS 1944 4 18
44
9477 2
519 M SE
20 6:
CNBY & 3 A5 SO I
S S SON 4.50
3334
BIB6
SOP
BOC 5W BERT AUZZ
2
NO IAS
CON GS
GOOD IS BORR AN A
140
BY
499 BERO WI SS
07 BEL E SS
RESS $15
STEE 16
20
O SS
JO
DRO STE S
RAS HOMALIS 1S TO
STH IA 50
118
AST 1I12
19 BTE BIGG BOTH GASS
PI TIR L De
SS SY BE 4S M S IS IS
BH BIL 19 OUST DERRO TO
D 9
BEII ZO
TE ROOD 78017
12
1AAD WIRH 49 IN MIMAA LA
7 2 BAA 2999 LAVE 1B
4ST FOR PER SETT
5
58 284
7O0EN 22
TAAS IS ES 3U17 80ED DE
OS MURSE 1123 $5 5
4:
3H 1EO TA4 TO NEY L 4
2 462 BE
B
14512224
VER 8109 178
M 22
140369
869
17
66
E CH 295 14
75 G 113125 14585
20
25T 10SCE
SER SRAG
19
140
E
(a) 0g = 0.01.
(b) 0.1.
(c) 4g 0.3.
(4) lg 0.8.
Figure 20: The 204 smallest (top) and 204 largest (hottom) SVHN FLIPD with MI.P estimates evaluated different to.
37

---

2
R 5
7
3
1
V
7
6
777117477
1 19
I
17/11
Y 9
/7////
I
0///1/1
/1////
7
1
3
/
(
7
1///1113
1 9
1 9
1/117-5
2
2 ( 2
1
S
S /
/
8 V
9 I
/
1661 1
/
4 2 /
(667 1
100333311
9
3 O
891
9 1033132233221
Y 4 8 878886220
455
O 0738548663388882
6 3 A 8 8895 3
50000203882
773708007000
3 8 C 00000800
8868
2
0 20026796050
R 00111010100001111101010011111110
O 9 8
0
O 8 11
1 888888888 8088880000
8
B 8
143800668804
6 606608 3
(A) tg 0.01.
(b) = 0.1.
(6) tg 0.3.
(d) to 0.8.
Figure 21: The 204 smallest (top) and 204 largest (bottom) MNIST FLIPD with UNet estimates evaluated at different to.
2
8
8
E
/
/
6
]
8
Z
/
/
5
S
/
I
7
S
S
/
2
S
5
3
/ 0
3 0 G 0620033000000 5
2 2 8 1
O3 012089440907 3096633327
A
4 4 01007704788
3 WWw00///12211
0768845986666
0000336032213
140066680242
2 242-44/6111442550-000000000
4 15244333N65130003580000
7 4A
4 0
&
2. 0
6 T
2 9 0
O
L 8888007 80n
6988895888
O
(8) 0.01.
(b) lo 0.1.
0.3.
(d) 0.8.
Figure 22: The 204 smallest (top) and 204 largest (bottom) MNIST FLIPD with MLP estimates evaluated different In.
38

---

(n) tg 0.01.
(b) to 0.1.
(c) kg
0.3.
(d) tq -..8.
Figure 23: The 204 smallest (top) and 204 largest (bottom) FMNIST FLIPD estimates with UNet evaluated at different to.
(8) 0g 0.01.
(b) to U.1
0.3.
(d) kg
=0.8.
Figure 24: The 204 smallest (top) and 204 largest (bottom) FMNIST FLIPD estimates with MLP evaluated different In.
39

---

Figure 25; 1600 images from LAION-Acsthetio-6023. sorted by FLIPD estimates with to 0.3.
40

---

(a) to 0.01.
tb) 0.1.
(c) tu 0.8.
Figure 26: The 8 lowest- and highest-FLIPD values out of 1600 from LAION-Aestheetiss66255
evaluated at different values of fn. Placeholder icons blank images have been excluded from this comparison.
41

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See abstract, Section I, and Section 4.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We state the linearity assumption in Theorem 3.1 an idealization, we are straightforward about the lack of knees the FLIPD curves with UNets, and we highlight this in Section
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sellsspeciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
42

---

Answer: [Yes]
Justification: The derivations with diffusion models clearly state Equation 10 is assumed to be the transition kernel, Theorem 3.1 is stated clearly, and all the statements proved in Appendix B.
Guidelines:
The answer NA means that the paper does not include theoretical results, All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: In additional to clearly describing our method and providing hyperparameter details in Section D.I, we included link to our code in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
43

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
See Section 4 for a link code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/ Seellii/guides/Cooee
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ //nips. ccccccpppoo cc/public//eee www.c//ggg/eeeeeee htte//ooeSubbmeeee ieltimiissiooooooooyo for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: See Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: All the FLIPD curves show average and standard deviation across datapoints, which is clearly stated in Section 4. On synthetic data we do not report error bars across seeds since (i) we generated large enough datasets (10% datapoints, see Section D.4), and (ii) computing model-free estimates is expensive. On images, we do not report error bars across seeds since training multiple diffusion models is computationally prohibitive.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
44

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that NeurlPS Code of Ethics https: : hwwrrps...../// httpppblcc/chiicccuuieeellllllll
didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the
Answer: [Yes]
Justification:
We read the code of ethics and adhered to it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: We do not believe that improved estimates of local intrinsic dimension can be directly used in any way that would have socictal consequences.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
45

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Since we do believe our work can have negative societal consequences, we also do not believe putting safeguards around it to be necessary.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We cite PyTorch [47] as well as the papers whose code we used.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
46

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/c datasttss has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: A link to our code is included in Section 4.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourced experiments with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourced experiments with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
47

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
48


Reviews:
[{"review_id": "gzGCOzCwd6", "reviewer": "Reviewer_g4nR", "summary": "The paper addresses the challenge of estimating the local intrinsic dimension (LID) of high-dimensional data, a measure reflecting the number of local factors of variation and data complexity. Traditional methods for LID estimation have limitations such as inaccuracy, high computational demand, and dependency on multiple pre-trained models. This work introduces FLIPD, an LID estimator based on the Fokker-Planck equation associated with a single pre-trained diffusion model. FLIPD outperforms existing baselines in LID estimation benchmarks and remains tractable even for high-resolution images. The authors adapt the LIDL method to DMs, leveraging the Fokker-Planck equation to enhance efficiency and accuracy. FLIPD's performance is validated through theoretical motivations and empirical results, demonstrating its superior correlation with non-LID complexity measures and qualitative complexity assessments compared to other estimators.", "questions": "**Q1.** It seems that the proposed estimator over-estimate the LID compared to [10,48]. Can you provide some insights? \n\n**Q2.** How does the FLIPD estimator maintain accuracy and efficiency in LID estimation across varying levels of data complexity and different types of high-dimensional datasets, and what specific mechanisms within the Fokker-Planck framework contribute to these properties?", "limitations": "Not applicable", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "**1.** This paper introduces a new approach to estimating local intrinsic dimension (LID) by leveraging the Fokker-Planck equation associated with diffusion models (DMs). This new method effectively addresses several limitations of existing LID estimation techniques, such as inaccuracy, high computational demand, and the need for multiple pre-trained models. \n\n**2.** The authors thoroughly validate the performance of the FLIPD estimator through a series of benchmark tasks, demonstrating its superiority over existing baselines, particularly in high-dimensional settings.", "weaknesses": "**1.** The most critical section of this paper is Section 3.2, which introduces the core innovation of the FLIPD estimator. However, the insights from Equations (12) and (13) are not immediately clear. It is difficult to understand the insights of these derivations. Moreover, many details are deferred to appendix. The authors should provide more detailed explanations of the Fokker-Planck equation and its relevance to LID estimation. \n\n**2.** One weakness of the paper is the unclear demonstration of the computational advantages of the proposed FLIPD estimator compared to the method in [58]. The authors argue that while the method in [58] addresses some limitations of LIDL, it remains computationally expensive. However, the paper lacks a thorough theoretical analysis or experimental evidence to substantiate the claim that FLIPD is more computationally efficient.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for the positive feedback and the insightful comments. If the following responses satisfactorily address your concerns, we kindly request you consider raising your score! \n\n## Weakness 1\nThank you for bringing this concern to our attention. We have provided a clarification for Section 3.2 and how it relates to the Fokker-Planck equation, along with some additional intuition. This will be included in the final version of the paper.\n\nThe Fokker-Planck equation is a partial differential equation (PDE) that describes how the marginal probabilities of an SDE evolve over time $t$. Formally, it states that for the given SDE in Eq. (1), we have the following:\n\\begin{equation}\n\\frac{\\partial}{\\partial t} p(x, t) = - \\frac{\\partial}{\\partial x} \\left[ f(x, t) p(x, t)\\right] + \\frac{\\partial^2}{\\partial x^2} \\left[ g^2(t) p(x, t) /2\\right]\n\\end{equation}\nGiven this, it is straightforward to derive the PDE for $\\log p(x, t)$ too. While Section 3.1 explores the connections between $\\varrho(x, \\delta)$ and $\\log p(x, t)$, Section 3.2 utilizes the PDE for the evolution of $\\log p(x, t)$ to develop an ODE that describes how $\\varrho(x, \\delta)$ evolves as a function of $\\delta$. Consequently, the entire trajectory of $\\varrho(x, \\delta_1)$ through $\\varrho(x, \\delta_m)$ can be determined using a *single* ODE solver. \n\nLet us begin by explaining how Eq. (12) relates to the Fokker-Planck equation. This derivation involves two main steps: (i) rewriting the $\\log \\varrho(x, \\delta)$ term on the LHS using the log marginal probabilities from Eq. (11), and (ii) transforming the differentiation with respect to the log-standard deviation $\\delta$ into the *time domain* using the mapping $t(\\delta)$. The following derivation formally connects the LHS of Eq. (12) to the Fokker-Planck equation:\n\\begin{equation}\n\\frac{\\partial}{\\partial \\delta} \\varrho(x, \\delta) = \\frac{\\partial}{\\partial \\delta} D \\log \\gamma(\\delta) + \\frac{\\partial}{\\partial \\delta} \\log p(\\gamma(\\delta) x, t(\\delta)) \\quad \\text{Using (11)}\n\\end{equation}\n\\begin{equation}\n= \\frac{\\partial}{\\partial \\delta} D \\log \\gamma(\\delta) +  \\frac{\\partial t(\\delta)}{\\partial \\delta}  \\underset{\\text{Rewrite using Fokker-Planck}}{\\underbrace{\\frac{\\partial}{\\partial t(\\delta)} \\log p(\\psi(t(\\delta)) x, t(\\delta))}}\n\\end{equation}\n$\\frac{\\partial}{\\partial t(\\delta)} \\log p(\\cdot, t)$ can be rewritten in terms of $s(x, t)$ (Eq. (41) of Appendix C.2) and replacing the Fokker-Planck term above will yield $\\nu$ in Eq. (12). By solving the entailed ODE, we can evaluate $\\varrho(x, \\delta)$ at multiple $\\delta$s and use the linear regression in LIDL to estimate LID.\n\nFurthermore, while a general solution for Eq. (12) can be obtained, it will always differ by a constant from the true $\\varrho(x, \\delta)$ due to the unspecified and difficult-to-determine initial value $\\varrho(x, \\delta_1)$. Eq. (13) introduces an initial value problem where this constant is *artificially* set to zero, defining its solution as $\\hat{\\varrho}(x, \\delta)$. Notably, although $\\hat{\\varrho}(x, \\delta)$ and $\\varrho(x, \\delta)$ differ by a constant, we perform a linear regression on $m$ sampled points from these functions, and this constant only affects the intercept in the regression, not the slope. LIDL focuses solely on the slope and not the intercept; thus, regression on $\\hat{\\varrho}$ instead of $\\varrho$ yields the same result. Finally, we derive an LID estimator that directly extends LIDL, requiring only a single pre-trained diffusion model and a single ODE solve.\n\n## Weakness 2\nThe computational benefits are one of the main advantages of our method, and we will substantiate this further in the final version of the paper. We have included a dedicated section with time comparisons to the NB method from [58] in the general rebuttal to address this concern.\n\n## Question 1\nBoth [10] and [48] use the model-free, KNN-based MLE estimator. As outlined in Table 7 of Appendix D, MLE severely underestimates the true LID on synthetic data where the ground truth is known. We note that all the experiments in [10] and [48] are conducted on images, where the ground truth LID is unknown. Thus, in the image context, it is impossible to concretely say whether FLIPD overestimates LID, or MLE underestimates it. However, the fact that MLE underestimates LID in synthetic scenarios suggests that the estimated values on high-dimensional images are likely too low as well. Additionally, we point out that our estimate of average LID for MNIST closely aligns with that of [58], which is a more recent paper.\n\n## Question 2\nWe are not sure we entirely understand the question - please respond back with clarification so that we can better address your concerns and continue the discussion. Regardless, we would like to emphasize that the Fokker-Planck equation itself does not inherently provide better LID estimates; rather, it is a tool for efficiently extracting LID from a diffusion model trained to fit the data manifold. Our estimates are more accurate than prior methods because we use diffusion models, whereas many baseline methods are either model-free or use normalizing flows. Normalizing flows are known to struggle with data coming from low-dimensional manifolds embedded in high-dimensional space [38, A], whereas diffusion models do not exhibit these limitations [B]. In fact, we further study the NB estimator, which also uses a pretrained diffusion model, in Appendix D. We find that enhancements to the NB estimator could yield a highly accurate estimator, despite remaining intractable for high-dimensional data. This supports the notion that the high quality of our LID estimator is mainly due to the superior performance of diffusion models, rather than the Fokker-Planck equation.\n\n# References\n\n[A] Behrmann et al. \"Understanding and mitigating exploding inverses in invertible neural networks.\" ICML 2021.\n\n[B] Pidstrigach. \"Score-based generative models detect manifolds.\" NeurIPS 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LzoR4UB4cI", "reviewer": "Reviewer_MJca", "summary": "The authors propose to employ the best available generative models, i.e., diffusion models, to the estimation of local intrinsic dimension. To this end, they build upon the LIDL estimator of Tempczyk et al. [ICML, 2022], but crucially resolve a number of limitations: (1) direct application of LIDL requires training $m$ diffusion models, (2) even by enabling only a single diffusion model, LIDL still requires $m$ ODE solves, and finally (3) even by enabling a single ODE, it requires repeated estimation of the trace of a Jacobian. The result is a set of explicit formulas for the LID estimate for each of the common diffusion models, leveraging the Fokker-Planck equation associated with the Ito stochastic differential equation. The asymptotic correctness of the estimate is established for an idealized linear setting. This algorithmic contribution is further demonstrated through an enhanced intrinsic dimension benchmark, which highlights the limitations of prior techniques.", "questions": "Technical Issues\n=============\nSection 3\n- L211: In my experience, such constant factors relate to the curvature of the manifold. So, the result makes sense when the manifold is an affine subspace. In this sense, the reasoning given on L219-222 appears erroneous. That is, concentration on a point mass in the limit still leaves the intrinsic dimension intact, rather than totally detaching from the manifold structure to yield an intrinsic dimension of 0.\n\nSection 4\n- L309: It seems that this is the first mention of an MLP, which this line suggests is what was used for the synthetic experiments.\n  - Unfortunately, this seems like more than a simple lapse in communication.\n  - Rather, upon switching from those previously undisclosed MLPs to UNets, those nice findings about knees in the curve no longer hold.\n  - This critical transition comes out of the blue, since it wasn't mentioned either in the abstract or the introduction.\n  - From a first reading, it was difficult to keep track of which findings reported for MLPs carry over upon switching to UNets, where the authors claim the resulting estimates remain valuable measures of complexity; see L323.\n- FLIPD on high resolution images\n  - L359: is it correct that running FLIPD in the latent space inherits existing biases, e.g., prioritizing semantics rather than \"true\" manifold geometry?\n  - It's unfortunate that the actual results as conveyed on L345-360 seem to be misrepresented in the high level summaries given in the abstract and introduction.\n\nOther Technical Comments\n=====================\nSection 3\n- L190: I'm not sure I follow how $\\nu$ does not depend on $\\hat{\\varrho}$. Perhaps what's meant is that $\\nu$ is computed from Eq. 12, and the resulting values are fixed to define $\\hat{\\varrho}$ through the ODE in Eq. 13. That way, if $\\nu$ was computed with $\\hat{s} = s$, then Eq. 13 can be expressed in terms of $\\log \\varrho(x, \\delta)$ itself.\n\nSection 4\n- L284: Does the diffeomorphism of the flow hold for practical numerical evaluations? Did the authors observe any fluctuations in the LID due to such numerical errors? If so, can those errors be bounded?", "limitations": "I'm guessing the authors wanted to go all the way, even when UNets failed to deliver the same nice story afforded by MLPs. It's unfortunate that the story didn't go that way - or perhaps it's an opportunity? Either way, the current presentation of this story may initially appear to misrepresent the actual findings. In my view, this is a serious issue that must be addressed before publication.\n\nAll that aside, the paper does achieve a lot. Only that the story had to be a bit more complex / interesting.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- Greatly simplifies LID estimation via diffusion models, presenting a significant speed up over prior work.\n- The technique is well-principled, and generalizes to different diffusion model families, enabling explicit formulas to be derived as demonstrated by the authors.\n- The proposed benchmark (with an in-depth analysis) helps to further test against prior work, better demonstrating their shortcomings.\n- Relating LID to PNG compression size is also a valuable idea.", "weaknesses": "Unfortunately, there seem to be a number of technical issues; see below. It doesn't help that some of those issues appear out of the blue upon reading into the experiments section, and don't match the impression one gets from the high level summary given in the abstract and introduction.\n\nDisregarding those issues for a moment, it's still unsettling that UNets deviate from MLPs in terms of their FLIPD curves lacking easily detectable knees. This calls into question whether knees are the best or only way to select values of $t$, or whether there are other techniques.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We highly appreciate your comprehensive review, and we are grateful that you recognize the significant achievements of our work despite some concerns. Before addressing your points in detail, we would like to clarify a potential misunderstanding regarding the UNet vs. MLP issue. While certain patterns observed with MLPs, like knees, may not carry over to UNets, we firmly maintain that FLIPD estimates remain useful measures of relative complexity, as evidenced by our image rankings with a UNet backbone. Most importantly, the results in **Table 2** use a **UNet** backbone and show a high correlation with PNG compression size. Interestingly, we often observe an *even greater* correlation with PNG compression size when using a UNet backbone compared to MLPs, as shown in Figure 16 of the Appendix.\n\nWith that in mind, we propose FLIPD as a strong LID estimator on structured manifolds and a relevant measure of complexity for images. Once again, we thank you for bringing this up, and will ensure that the introduction and abstract are updated to more accurately reflect our contributions in the final version of the paper.\n\nWe will now provide detailed responses to the individual concerns:\n\n\n## Technical Issue (L211)\nThank you for the astute observation. Indeed, these terms tend to be related to curvature, and studying the effect of curvature in this setting seems an extremely appropriate future direction. However, our conjecture is not that this term does not depend on curvature nor that its derivative wrt $\\delta$ is exactly $0$; rather, it’s simply that its derivative goes to $0$ as $\\delta \\rightarrow -\\infty$. We indeed expect the speed of convergence to depend on the curvature. Additionally, we would like to emphasize that L211 is merely a conjecture, and it is based on the strong synthetic results we obtained, as detailed in Appendix D. For this we refer you to our experimental results, particularly those involving non-linear manifolds generated by normalizing flows denoted with $\\mathcal{F}$. It is also worth mentioning that LIDL, our baseline, also relies on the curvature-related O(1) term being well-behaved.\n\nFurthermore, the limit $\\delta \\to -\\infty$ is not related to the manifold being detached. It suggests that as $\\delta \\to -\\infty$, $\\varrho(x, \\delta)$ becomes increasingly local, and the dependence on the global characteristics of the manifold diminishes. Our intuition is that as $\\delta \\to -\\infty$, $\\log \\varrho(x, \\delta)$ becomes as local as possible and, after a certain point, the local approximation to the manifold given by its tangent space can be considered “good enough”.\n\n## Technical Issue (L309)\nWe would like to clarify that using knees is not necessarily the best way to obtain accurate LID estimates and is not a crucial part of our method. In fact, choosing a sufficiently small $t_0$ still produces very reasonable results (see Table 5 of the Appendix). Therefore, the absence of knees does not render the estimator useless. Moreover, we firmly believe that even without the knees, the FLIPD estimators with UNets are useful measures of complexity, as evidenced by the high correlation with PNG compression size shown in Table 2.\nWe acknowledge that the differences between UNets and MLPs are counterintuitive and warrant further exploration in future work. Our explanation for why this issue may arise is included in Appendix E.1. We will add more context and move some of that explanation to the main text in the final version of the paper.\n\n## Technical Issue (FLIPD on high resolution images)\nThis is an excellent point and might indeed contribute to the more semantically meaningful orderings we observe. However, we would like to mention that the encoder/decoder of Stable Diffusion is designed not just to encode semantic information; the latent space remains image-shaped and high-dimensional. Additionally, we refer you to our results in Figures 17 through 24, where the model does not use a latent space and still aligns with semantic complexity. That said, we acknowledge that using a latent space for Stable Diffusion might lead to misinterpretation of the claims in the introduction. We will ensure that these claims are made more explicit in the final version of the paper.\n\n## Technical Comment (L190)\nYour latter interpretation is correct: $\\hat{\\varrho}$ is simply *defined* using $\\nu$ and the ODE in Eq. (13). It is important to note that $\\hat{\\varrho}$ and $\\varrho$ are not the same, and $\\hat{\\varrho}$ is the solution to the ODE in (12) where the initial value is *artificially* set to zero, and thus $\\varrho$ and $\\hat{\\varrho}$ differ by a constant. In Section 3.2, we demonstrate that the constant difference between $\\hat{\\varrho}$ and $\\varrho$ is unimportant because, when a linear regression is employed (similar to LIDL), this constant only affects the intercept and not the slope; with the slope being the term that is important for computing LID. We will reword this section in the final version of the paper for better clarity.\n\n## Technical Comment (L284)\nThis is yet another excellent point! Indeed, if certain factors are not controlled, normalizing flows can distort the manifold, causing numerical changes in the LID. For example, a square stretched in one direction and squeezed in another can numerically deceive an LID estimator to consider it a line with LID=1 rather than a rectangle with LID=2. We incorporate activation normalizations and use at most 10 rational quadratic transforms in our flow-induced manifolds. This ensures the overall transformation is not ill-conditioned.\n\n\nWe thank you once again for your excellent review! Your insightful comments have improved our work. Finally, if you find that our explanation satisfactorily addresses your concerns, we kindly request that you consider raising the score!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cfDJh9wnuP", "reviewer": "Reviewer_9X9u", "summary": "This paper proposes a novel method for estimating local intrinsic dimension using an existing approximate likelihood estimator with diffusion models. The proposed approach is able to exploit a pre-train diffusion model to evaluate the log density function of the noisy version of the data (i.e., adding Gaussian noise), where solving the ordinary differential equation (ODE) can be avoided by setting $t_0$ as a hyperparameter. Compared to prior works, in which normalizing flow is used for function evaluation and multiple runs of density evaluations are needed for estimation, this framework is much more efficient and shows more promising results.", "questions": "1.\tWhat are the causes for the spikes in Fig. 2(b) and Fig. 6(c)?\n2.\tThere seems not much difference between different $t_0$’s in Fig. 26. Does it imply that the estimator is not sensitive to the hyperparameter under the case with data of large dimension?\n3.\tIn line 130, what are the ‘mild regularity conditions’ leading to Eq. (7)?", "limitations": "Limitations have been discussed.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This is a technically sound paper with some theoretical depth and practical usefulness. Although the core idea of the estimation in Eq. (8) is not new, the authors come up with a novel approach to estimate the derivative of the log density function with variance-preserving diffusion models. For real data experiments, using PNG compression as a proxy for evaluation is informative and illustrative. The overall paper is well organized and relatively clearly written.", "weaknesses": "If the claim is that the proposed estimation method is simpler and more computationally efficient, it might be useful to include complexity and/or time comparison with existing baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your positive review and are pleased that you find our paper well-organized and clearly written. In response to the identified weaknesses and questions, we have addressed them as follows:\n\n## Weakness: Including time-complexity comparisons\n\nIn the general rebuttal section, we provided a time comparison between our method and other baselines, showing that our method delivers 10x faster LID estimates on grayscale 28x28 images and 100x faster on RGB 32x32 images compared to the best baseline, and most importantly, it is the **only** method capable of scaling up to high-resolution images in a tractable manner. Given that computational efficiency is one of the most critical practical aspects of our work, we believe the added content will further solidify our claims and provide greater clarity. We will also include these details in the final version of the paper.\n\n## Questions:\n\n\n1. That is a great observation! These curves roughly indicate the rate of change in the Gaussian convolved density $\\varrho(x, \\delta)$. The observed \"spike\" in the mentioned curves is likely due to a sudden increase in the Gaussian convolution at that point, potentially caused by mixing distributions from different manifolds. For example, in Figure 6-c where the dataset is a Lollipop (a mixture of 0, 1, and 2-dimensional components), Gaussian noise added to another submanifold may cause datapoints from the 2D or 1D submanifold to mix with the isolated 0D point. A similar situation occurs in the \"string within a doughnut\" experiment of Figure 2-b. These artifacts are irrelevant for LID estimation with FLIPD, however, as the theory behind our LID estimates holds when we let $\\delta \\to -\\infty$, or equivalently, $t_0 \\to 0$. In these regimes, $t_0$ is small and such mixings do not occur.\n2. You are correct that the orderings are not sensitive to $t_0$, but only in a certain range. Note that for extremely high $t_0$ we see that the ordering does not correspond to the data complexity anymore; this is evident in Figures 17-d, 19-d, 21-d, and 23-d. We also see that in Figure 16, when $t_0 \\to 1$ the correlation between FLIPD estimates and PNG compression size significantly reduces. We would also like to refer you to our synthetic experiment \"string within a doughnut\" illustrated in Figure 3, and the multivariate Gaussian in Figure 8 of the appendix, where the choice of $t_0$ has a significant impact on the overall estimate.\n3. The regularity conditions mentioned in line 130 are those used by Tempczyk et al. [61]. Here, the support of the data-generating distribution being a disjoint union of manifolds means that this measure can be thought of as a mixture of measures on each individual manifold. The regularity condition is that each of these measures is smooth, that is, that for every chart on the manifold, the pushforward of the corresponding measure through the chart is absolutely continuous with respect to the Lebesgue measure, and that the density of this pushforward is locally bounded away from 0. While these conditions are indeed mild, they involve charts and measure-theoretic terms, which we explicitly wanted to avoid to keep the paper as accessible as possible.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aSlG94NJr7", "reviewer": "Reviewer_MdUJ", "summary": "This paper is concerned with estimating the local intrinsic dimension (LID) of a given data manifold. Intuitively, high intrinsic dimension indicates more complex data distribution and an accurate representation of the intrinsic dimension has useful applications in ML such as detecting outliers or adversarial examples. \n\nThe approach in the current work suggests incorporating diffusion models as a model-based estimator since these models learn the distribution of the data and implicity the intrinsic dimension. At a high-level the idea is the following: the noise added during diffusion can be seen as a form of Gaussian convolution applied to the data. By analyzing how the data density changes with increasing noise levels, the model is able to implicitly learn the local intrinsic dimension at different points on the manifold.\n\nFurthermore, they uncover and leverage some computational efficiencies using the Fokker-Planck equation which boil down to making a single call to an ODE solver. This computation efficiency also allows for LID estimation for high resolution images, and is the first to be able to do so.\n\nThe paper has strong theoretical background and justifications and empirical results that verify their theory. They also provide code with examples to reproduce results. Overall a really excellent and well-executed effort.", "questions": "line 107 (suggestion): see also \"A Course in Metric Geometry\" by Burago, Burago, Ivanov for related discussion about intrinsic dimension and how to compute it for metric spaces. \n\n(nit) Figure 1: For the sake of intuition, this Figure makes sense. However while it is useful to justify that lower/higher FLIPD scores indicate less/more complex images, it leaves open the question of if less/more complex images necessarily have lower/higher FLIPD scores. For example, who's to say that there aren't some images that look even more complex that the 4 you surface which have the max FLIPD values. It's just a personal note and makes figures like this feel misleading without noting that important distinction. That being said, once getting through the Appendix more comprehensive comparison between entire datasets and FLIPD scores are included. Perhaps just point to those additional plots in the appendix when discussing the Figure 1 as well.", "limitations": "N/A", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The ideas behind this paper are very well motivated and compelling. The exposition is very clear and the use of diffusion models to improve LIDL feels like a natural next step. \nAlso, with respect to the previous work of Stanczuk et al, the improvements to decrease the computational complexity is noteworthy.\n\nOverall the paper is well-written and the theoretical background and proofs are carefully written and appear complete and correct to me. There are extensive experiments including both synthetic and real-world datasets.", "weaknesses": "I have a few minor comments and questions below but overall I don't see any substantial weaknesses of the work that are worth addressing further.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate your positive feedback and are delighted that you consider our work to be \"an excellent and well-executed effort.\" In response to your questions and suggestions:\n\n1. Thank you for the great suggestion! We will make sure to include a reference to the textbook “A Course in Metric Geometry” and include appropriate discussion in the final version of the paper.\n2. You are absolutely correct to imply that LID is not a perfect measure of complexity. They are related, correlated concepts as per our observations (the correlation between PNG compression and LID, as well as the images presented in the main figure and our appendix), but there is no reason to expect them to be interchangeable. We will ensure that the distinction between the two is made more explicit in the main text.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We greatly appreciate the time reviewers have spent on our paper and are delighted to see that all four reviewers recommended acceptance. Reviewers found our method “well-principled” and “clearly written” (**MJca**, **9X9u**, **MdUJ**) and described it as “a really excellent and well-executed effort” (**MdUJ**). They also noted that our method is “thoroughly motivated” by our theory (**g4nR**) and synonymously praised our experimental results as “thorough” (**g4nR**) and posed them as enabling further testing against prior baselines (**MJca**). Additionally, two reviewers highlighted the value, informativeness, and illustrative nature of the links established between LID estimates and PNG compression size (**MJca**, **MdUJ**).

Here, we provide an in-depth discussion of the computational benefits of our method compared to other baselines. Additionally, we present time comparisons from our experiments using our GPU hardware, demonstrating that FLIPD is the **only** estimator capable of scaling up to Stable Diffusion-scale data, further highlighting the scalable nature of our approach.

Before we continue, we find NB to be the only relevant baseline for this comparison because:
1. LIDL, the only other model-based LID estimator, requires training multiple models, which is extremely time-consuming. For example, obtaining LID estimates for LAION would necessitate training multiple instances of Stable Diffusion, making it impractical. Additionally, while LIDL suggests using normalizing flows for their straightforward likelihood computation, it is well-known that normalizing flows struggle to produce high-quality images even on relatively low resolution data such as CIFAR10, let alone on high-resolution, internet-scale datasets such as LAION.
2. Traditional model-free methods, such as ESS, LPCA, or MLE, not only underperform on high-dimensional data but also require performing kNN on extremely high-dimensional data, which is impractical. In fact, as detailed in Appendix D, for baseline comparisons on these methods, we had to use subsamples on synthetic datasets with 1000 dimensions because computing LID estimates for the entire dataset would have taken more than a day.

To further substantiate our claims, we included time comparisons for our image experiments in the table below, corresponding to the results reported in Table 2 of the main text, as well as all of our Stable Diffusion results. We grouped the data in Table 2 into two categories: RGB datasets with 32x32 dimensions and 3 channels (SVHN and CIFAR10), and grayscale datasets with 28x28 dimensions (MNIST and FMNIST). Our results show how long it takes to estimate LID for a single datapoint. We see that FLIPD is more than **10 times faster** than NB on grayscale data and more than **100 times faster** on RGB data, demonstrating that as dimensionality increases, FLIPD remains the only scalable estimator. Finally, as depicted in the table, FLIPD is the only estimator capable of estimating LID for LAION.

To dive deeper into why the NB baseline cannot estimate LID for LAION, we note that the method requires $4 \times D$ forward passes of the score network to construct a matrix $\mathbf{S}(x)$ of dimension $D$ by $4D$. This is then followed by a singular value decomposition, meaning that this portion of NB scales *cubically* with $D$, the ambient dimension of the data. In our experiments with LAION images, we found that just constructing this matrix $\mathbf{S}(x)$ for a single data point takes over *2.5 hours*, and performing the singular value decomposition on such a large matrix is **intractable** using the built-in PyTorch linear algebra functionality.

Finally, for the first two columns of the table below and the results in Table 2, we used 50 JVPs for our Hutchinson estimator and a standard **UNet architecture** from the diffusers library. It is worth noting that we can adjust the number of JVPs to trade-off between accuracy and performance. For our Stable Diffusion results, we opted for a *single* JVP to estimate the LID for a given datapoint which still gave a reasonable measure of complexity while only taking *195 milliseconds*. Notably, all experiments were conducted on a single NVIDIA A100-40GB GPU.

| Dataset               | 28x28 (Greyscale)        |    32x32 (RGB)     | LAION (High-resolution) | 
|-----------------------|:--------------:|:--------------:|:--------------:|
| FLIPD | **0.101** seconds | **0.133** seconds | **0.195** seconds |
| NB                    |  1.648 seconds | 10.766 seconds | - |

Meta-review: This paper proposes a Fokker-Planck-based estimator for local intrinsic dimension estimation, which uses diffusion models in implementation and scales to high-resolution images at the level of Stable Diffusion. Both the theoretical and empirical contributions are appraised by the reviewers.

The proposed method is indeed novel and demonstrates promising performance compared with existing baselines. The paper is clearly written in general and I invite the authors to revise according to the discussion with the reviewers. (Special thanks to Reviewer MJca and the authors for the very dedicated efforts in revision and I trust the authors to make necessary polishing in the final version.) An outstanding concern about the computation efficiency in the initial review is further addressed by the authors' general rebuttal. Therefore, I am recommending acceptance of the paper.
Decision: Accept (spotlight)