Sharp Optimality of Simple, Plug-in Estimation of the Fisher Information of a Smoothed Density
Authors: Subhodh Kotekal
Venue: ICML 2025 poster (2025)
Keywords: Fisher information
minimax estimation
functional estimation
information theory
Primary area: theory

Abstract: Given independent and identically distributed data from a compactly supported, $\alpha$-Hölder density $f$, we study estimation of the Fisher information of the Gaussian-smoothed density $f*\varphi_t$, where $\varphi_t$ is the density of $N(0, t)$. We derive the minimax rate including the sharp dependence on $t$ and show some simple, plug-in type estimators are optimal for $t > 0$, even though extra debiasing steps are widely employed in the literature to achieve the sharp rate in the unsmoothed ($t = 0$) case. Due to our result's sharp characterization of the scaling in $t$, plug-in estimators of the mutual information and entropy are shown to achieve the parametric rate by way of the I-MMSE and de Bruijn's identities.

Sharp Optimality of Simple, Plug-in Estimation of the Fisher Information of a Smoothed Density
Subhodh Kotekal
Abstract
is widely known plug-in estimators are suboptimal in the unsmoothed case (t 0) (Laurent, 1997: Birgé & Mas- sart, 1995). Furthermore, the minimax rate turns out to be slower for small / > O; low amounts of Gaussian smooth- ing makes estimation of the Fisher information harder problem compared to no smoothing, which is curious as Gaussian smoothing was proposed (Goldfeld et al., 2020b) in part to alleviate other statistical difficulties. Once >> O, however, the minimax rate is shown to be decreasing in 4, and there is critical level of smoothing above which the problem is casier than if unsmoothed. The smoothed and unsmoothed settings are qualitatively different as the target density's 's support is, respectively, either unbounded or bounded. The estimation theory is materially affected since the Fisher information f * (Pr need not converge to that of f as t - O; in fact, it may even diverge (see Remark 1.2). As consequence of our results, the I-MMSE and de Bruijn identities (Guo al., 2005; Stam, 1959) can be used to show that simple plug-in type estimators achieve the parametric rate for estimating the mutual information and the entropy, which are functionals of particular interest from the perspec- tive of the Information Bottleneck theory (Shwartz-Ziv Tishby, 2017: Tishby & Zaslavsky, 2015; Goldfeld et al., 2019; 2020b). Furthermore, our error bounds for the mutual information and the entropy do not blow up t - O, which affected previous bounds in the literature (Goldfeld et al., 2020b).
Given independent and identically distributed data from a compactly supported, a-Hölder density f. we study estimation of the Fisher information of the Gaussian- -mottthee density f Pt. where frr is the density of VV (0, t). We derive the minimax rate including the sharp dependence on t and show some simple, plug-in type estimators are optimal for t > 0, even though extra debiasing steps are widely employed in the literature to achieve the sharp rate in the unsmoothed (t = 0) case. Due to our result's sharp characterization of the scaling in L, plug-in estimators of the mutual information and entropy are shown to achieve the paramet- ric rate by way of the I-MMSE and de Bruijn's identities.
1. Introduction
Due to various recent motivations from machine learning (Tishby Zaslavsky, 2015; Shwartz-Ziv & Tishby, 2017: Goldfeld et al., 2019; Song et al., 2021; Gabrié et al., 2018) and information theory (Goldfeld et al.. 2020b; Goldfeld & Greenewald, 2020), the classical topic of the estimation of some fundamental statistical functionals of probability density has seen renewed interest, now in the context of Gaussian smoothed distributions. In this article, we establish the minimax rate of estimating the Fisher information (also known as the Stein information) a smoothed density f Hypt where Pp denotes the density of (0, t).
Estimation of the Fisher information and other integral functionals are natural problems with widely appreciated motivations such as uncertainty quantification, estimation of information-thhoretiic divergences, and model selection. Hence, the unsmoothed case is, unsurprisingly, well- studied problem with a long history which precludes a comprehensive review (Laurent, 1997; Donoho, 1988; Bhat- tacharya, 1967; Cao et al., 2020; Laurent, 1996; Birgé & Massart, 1995; Bickel & Ritov, 1988; Kandasamy et al., 2015) (see (Laurent, 1997: Donoho, 1988; Cao et al., 2020; Bhattacharya, 1967) for specific results concerning the Fisher information). The program of considering the smoothed setting was recently proposed in the work of Gold- feld et al. (2020b). It is classically known that, given i.i.d. data points from a d-dimensional distribution, though the empirical measure converges to the data data- genteatnnnating distribu-
I(f + (p) (() {
00 (S (x))²))))) (())()
dz, given i.i.d. data from a compactly supported, a-Hölder density f with 2 1. A plug-in type estimator is shown to achieve the optimal rate when 1 > 0, even though it
(1)
'Department of Statistics, The University of Chicago, Chicago IL, USA. Correspondence to: Subhodh Kotekal <skotckal Sutical@@@ucicc...d.>>..
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
tion in various senses, the convergence rate can be exponen-
bound proved earlier by Birgé & Massart (1995). Moreover, when a: > 9/4 and the parametric rate 1/n is achieved, Laurent also shows the proposed estimator is semiparamet- rically efficient. The methodology and results developed in (Laurent, 1997) apply generally to integral functionals of the form f [f. ,'..., f()),, f())), though certain growth/decay conditions on I and its derivatives are imposed to ensure T is a smooth function on the relevant domain (e.g. f as- sumed bounded below by constant so that the gradient of T (u, v) = v² u is bounded when u = f(x)).
tially slow in the dimension (e.g. n-1/d in Wasserstein-1 distance for d 2 3 (Dudley, 1968)). Motivated to circum- vent this curse of dimensionality. Goldfeld et al. (2020b) suggest considering convergence of the empirical measure to the data- Mata-generating distribution after smoothing both with a Gaussian; they show the rate (in Wasserstein-1 distance, x"- divergence, and total variation distance among others) dras- tically improves to the dimension-ing dimension-independen. independent, parametric rate n 11/2 (with prefactors depending on the dimension and the Gaussian's 's variance). Adopting the smoothing proposal, a growing literature has emerged including asymptotic dis- tributional results for various smoothed distances (Goldfeld et al., 2024; Sadhu et al., 2022) and the development of the Faussian-smoother approach to optimal transport (Zhang et al., 2021: Goldfeld & Greenewald, 2020; Mena & Niles- Weed, 2019; Goldfeld et al., 2020a: Chewi et al., 2024; Ding & Niles-Weed, 2022; Chen & Niles-Weed, 2022).
These conditions preclude the theory of Laurent (1997) (and also (Laurent, 1996; Birgé & Massart, 1995)) from covering the smoothed case since f * (Pt has unbounded support and is thus not bounded below by a constant. fact, the minimax estimation rate of certain functionals of itself can actually change if f is allowed to be arbitrarily close to zero in some regions of its support. For example, the entropy is no longer a smooth functional and the minimax rate of estimating the entropy of f in squared error deteriorates from 1 Lipschitz ball with smoothness index a E (0, 2] (see (Han et al., 2020) for details). However, such results are worst- case statements, and estimation of functionals of f yps can be faster when f fulfills some regularity conditions since there is special, specific structure in the form of Gaussian convolution. Indeed, Goldfeld et al. (2020b) show plug- in estimator of the entropy achieves the parametric rate 1 (with prefactors depending on t).
Aside from convergence reasons, Goldfeld et al. (2020b) are also motivated by the Information Bottleneck theory (Shwartz-Ziv & Tishby, 2017; Tishby& & Zaslavsky, 2015) in deep neural networks; a central quantity is the mutual information between the input feature vector and the hid- den activation vector. Adopting a framework (Goldfeld et al., 2019) which relates this mutual information to an additive white Gaussian noise channel, estimation of this mutual information amounts to estimating the entropy of a smoothed density f * spr Goldfeld et al. (2020b) establish the convergence rate for the plug-in estimator (entropy of the smoothed empirical measure). Given the connection of all these functionals, estimation of the Fisher informa- tion under Gaussian smoothing is of clear interest. The smoothed Fisher information has also been recently shown to be an essential quantity in the finite-sample analysis of mean estimation; we point the reader to Appendix H for a discussion of this recent line of results.
x +n 44x to 1 + (n log n) 1) 1) if f lies in
For estimation of the Fisher information of f. Bhat- tacharya (1967) proposes the estimator I(f)
5hm (x))) dr where kn - Do is sequence of truncation hyperparameters and fr, f are kernel density es- timators of f', f respectively. Under various restrictive assumptions on the size of ʃ and on the score f'/S, an asymptotic analysis was given in (Bhattacharya, 1967) (see also (Dmitriev & Tarasenko, 1974)), and later nonasymp- totic results were derived by Cao et al. (2020). The upper bound on the error of Bhattacharya's estimator obtained in (Cao et al., 2020) is quite slow (logarithmic in rt), and so Cao et al. propose a modification which truncates the integrand in addition to the domain of integration. The pro- posed estimator is shown to achieve an error bound which is polynomial in n. Cao et al. (2020) then apply their es- timator specifically to Gaussian smoothed densities of the form f (Pt. and derive some convergence rates. However, the rates are quite rough in the sense they are only shown to decay polynomially in n with the precise exponents unclear. Furthermore, no minimax lower bounds are offered and so the question of optimality remains unresolved.
1.1. Related work
In the unsmoothed case (t = 0), a detailed estimation theory for the Fisher information is available. Laurent ( (1997) writes the Fisher information as (_,((') /s = f_, ("), with T(u, v) = u²/ /u, then proposes to use preliminary estima- tors i and f' to obtain the plug-in estimator f", T(,.f") which is then debiased through an estimator of the error E. I(f.f') - -ff.f f'). The error estimator estimates the first two terms in the Taylor expansion of Y at the point (f. f') in the direction of (f,, f'). Assuming f belongs to an Sobolev space of index a >> 1, is compactly supported on, say. [-T, z.. satisties some periodicity conditions, and is bounded below by a positive constant, Laurent (1997) shows the proposed estimator achieves the (squared error) rate + n ((a1))/( which matches the minimax lower
2

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
1.2. Main contribution
contribution is to show that for a and >> 0 such that t 2 C for some sufficiently small universal con- stant > 0 and sufficiently large universal constant C' >> 0, the sharp minimax rate is
The notation used throughout the paper are defined in Ap- pendix G. The parameter space for the minimax theory is defined as follows. For a, L > 0 define the a- a-Hölder class of probability density functions,
inf sup E ((E1- I1l) 2, FEHu
1 VTI3/44
Vnt² A
A
vi
(L) = f f 11111111++ + 10, [ f(w) du = 1,
(4)
Remark 1.1. The rate (4) essentially exhibits three regimes. In the very high noise regime t 2 1, the rate is of order 1/(vnt)). ). In the high noise regime -2/(20+1) - t S 1, the rate is of order 1/(v/(//11, and the rate specializes to n n=a/(2a+1) /Vi in the low noise regime S -2/(2++1) In Section 2, three separate estimators are proposed to address each regime separately; all are essentially of plug-in type. Remark 1.2. The minimax rate blows up as + 0, and this is because the Fisher information itself blows up. Con- sider the simple example f(u) = (ldlsn)- Note IT = [ (d.p(z, t)) /p(x, t) dz = J_ (f((x px 1) /) () S 1} dr 2 Jis- fe( 1)2 dr where the last inequality follows from (II.N(x, t) S 1 x for |x| - 1+CVE as t < 1. We have also used that (x+ 1) S cp.(x- 1) for 1r-1|s vi where < 1 is a small uni- versal constant, since 1 is small. Consider [n- (2
f is |a] (al-times differentiable on (-1,1), sup
(f(h s L for 2,1, |a] -
and ( f(aa))e)) s Lllu- Llu--'la-t for all p.p. E (-1,1))
This article focuses on the setting where L > 0 is some large universal constant, and so the notational dependence on I. will be suppressed. We will also treat a > as a fixed universal constant; all constants introduced implicitly or explicitly can potentially depend on a. Define the parameter space
Fa:= {S: + o0) supp(s) C [1-1, E Ha. and ca S f(n) S C2 for |u| 11;
1) dx x S2-1ls J. -(x-))1)) dæ x V + 00 as t - O. The conceptual reason for the blowup is that f is supported on [-1, 1] with a sharp discontinuity at the end- points since f(-1), 2 ca yet fa) = for |ar] > 1. On the other hand, for any t > 0 no matter how small, I yr is positive everywhere. Consequently, the derivative (f * ( explodes near the endpoints t + 0, causing the Fisher information to blow up.
(3)
where Ca. Ca >0 0 are some universal constants. Throughout the paper, we will assume 2 1. Consequently, for forfe E Fo. the derivative f' exists and satisfies sup/ul<1 (f)) & 1. This is already assumed in the unsmoothed case = 0 since otherwise consistent estimation (in a minimax sense) is not possible (Laurent, 1997).
Remark 1.3. As worked out in (Laurent, 1997), the minimax rate at t = 0 (ignoring conditions like periodicity and others stipulated in (Laurent, 1997)) in absolute error is + + /n
In (3), we assume the data-generating density f is bounded below by a constant in (3). Since the minimax rate for estimating I(ʃ) might depend on whether / is bounded below or not (e.g. in analogy to entropy estimation discussed earlier (Han et al., 2020)), we imagine the minimax rate for estimating I(ʃ spt) may also depend on whether ʃ is bounded below by a constant or not, even though f * yPt always has unbounded support and gets arbitrarily close to zero. Since (3) is canonical class in nonparametric statistics (Tsybakov, 2009) and is interesting in its own right, we proceed with this assumption for this first investigation; the case where f' is not bounded away from zero is left open for future work.
n (aaa((((((aa)))) provided a > 1. A quick comparison to (4) shows there exists some critical threshold of under which the estimation problem becomes harder and over which it becomes easier. This result provides a caveat to Gaussian smoothing, proposed in (Goldfeld al., 2020b) to alleviate the curse of dimensionality, even in the one- dimensional setting.
Remark 1.4. Under Gaussian smoothing, the Fisher informa- tion I, can be related to the mutual information T(J: I (p)) as well as the entropies h(f ipt) and h(f) through the celebrated I-MMSE and de Bruijn identities (Guo et al., 2005; Stam. 1959). In Section 4, we show the parametric rate of convergence for estimating these functionals can be achieved through a simple plug-in of the rate-optimal Fisher information estimator, despite the blowup in the optimal rate of Fisher information estimation as t + O.
Let (x) VeNT - 3े denote the probability density func- tion of (0, t). For density f E Fa, denote p(z, t) =)==))() (f* (2) and its (spatial) derivative 0,p(z, t) = (z))). Mn. ~ f. the goal is to estimate the Fisher information I : I(f * (px) given by (1). Our main
Given i.i.d data
Remark 1.5. There is a gap in our results for c < < C. Our minimax upper bound actually goes through for all
3

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
1 >>0 O. Rather, it our analysis of the minimax lower bound which introduces the slack. We conjecture the gap is artifact of our proof, and that sharper argument could show lower bound for e < t < C.
Here, recall [s'(w) S L for all |u| < (recall 2 is assumed throughout) as stated in (3). To motivate the def- initions (8) and (9), consider integration by parts yields Ozp(z,t) = (f (1)(2) - f()) - p4) du -00
{"1 (1)) nI) dp,
2. Minimax upper bound
and so g'(x, 1) S 0np(2, 1) S (x,,t) t). Finally, define the truncated estimators
As noted earlier, it turns out a simple plug-in strategy for estimating I is rate-optimal. Recall the notation p(a,,) t) (f * (z)((r) and 0,p(z, t) (f (a)(2) for the diffused density and its (spatial) derivative. Define the function T: (0) x R-- + with
F(,, (f(z, 1) V E(2, 1)) A AE(2,)) 0,p (z,t)) = (0.5 (0, v
(10)
(z,
(11)
Theorem 2.1. Ifa 2 1 and t < 1, then there exists C = C(a, L) such that
Tuu, v)
(5)
sup E (Zx- feFa
C
Note I, = -_ T(p(x, t), 0,p( t)) dr. By a plug-in type estimator, we mean an estimator of the form
Vntz/44'
where
i i= == T (ila, t), Onp(, (Dle,,) t)) dr,
4. = f r (f,, (1,)) (c, de,
where p and 0.p are some estimators of p and Q.p respec- tively.
with g and O.p given by (10) and (11) respectively.
Section 2.1 and Section 2.2 describe plug-in type estima- tors for the high noise (n 2/(2++1) < < 1) and the low noise (t S n (22+(+1))) regimes respectively. Section 2.3 addresses the very high noise regime (t 2 1), where plug-in strategy is essentially employed but with a slight modification to achieve the sharp rate; it is thus presented last for expositional clarity.
The optimality of a plug-in estimator is notable since such strategy is not optimal (with some other conditions such as periodicity of f and f') for estimating the Fisher infor- mation I(f) of the unsmoothed density (Laurent, 1997). Instead, Laurent (1997) employs a second order debiasing strategy to achieve the minimax rate, and even the optimal asymptotic variance in the regime where the parametric rate is achieved.
2.1. High noise regime
In the high noise regime n कमन < t < 1, it is intuitive to estimate p(æ, t) = (f (z) with the unbiased esti- mator p(z, t) Ee Pr(z (4)) It directly follows O,p(2,t) = T E f1 14), which is unbiased for O.p(2, t). Though it is appealing to use unbiased estima- tors, it turns out some truncation is convenient. Define
2.2. Low noise regime
In the low noise regime /. S n I the diffused density and its derivative will be estimated via a kernel smoothing. Suppose we have access to, say, a kemel-based density estimator ʃ such that
E(x,t)= = ca S pe(x- - 4) du,
E(f(w)- - [(H)) Sn n 2a+1 for all |M| S 1,
F(J)(())) - (1(w)) In 20-11 for all [N] < 1,
E(x,t)= = Co /" G0(x 11) duc,
(7)
which is standard (Tsybakov, 2009; Giné Nickl, 2016).
Define the preliminary estimators
and note g(x,t) S p(x, 1) S E(x,,) t) since Fa implies ca E f(1) < Ca for |p| < 1. Likewise, define
p(z, (f ())))) = 1) 1) ((1))((x-1)
(12)
e((x,t):= =
L capi(x + 1) -c) - S pe(x - 4) du,
(8)
+ S" P'))lll - p) dy.
-
(13)
e((x,1):=
The definition of O.p is motivated by noting that integration by parts implies the estimand can be written as 0xp(2,, t) (fx4))(x)) = (()
Capi(1 + 1) cappe(x - 1) T. / pe(x 11) du-
9
f6) of(x - 1) du (x+

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
μ4) du. To construct plug-in estimator of I, we will truncate the preliminary estimators as in the high noise regime.
- 1
f'()) f
The two terms will be estimated separately. The first two terms can be simply estimated through unbiased estimators,
2
(16)
(xx = (f(2, t) v E(2, t)) t), 0.p (z, (ōnp(a, 1) V (x), 0) ^ ^ (1,
(14)
nl2
(15)
L2 is estimated via plug-in, namely
12 22:= == (T
where the truncation levels are given by (6), (7), (8), and (9).
dr where 5e is given by (10) and
(17)
Theorem 2.2. a 2 1 and t < 1, then there exists C'
C(a, L) such that
(xr =
sup E ( ( SC fef
n IT
pipi mi) ( v v=((z,))
f
7
(18)
where
with E(x, t) given by (7). The regularization in q (z, t) is ap- plied since the target satisfies [__ 11px(1 ( du S
2, = 000 r (F(ott)) o.p (z) dæ,
J(a, t).
Theorem 2.3. Ifc > O, then thenfor t 2 c, we have
with i and O.p given by (14) and (15) respectively.
sup E ( -Io Ial)= 1 11/2
The dominating term in the upper bound established by The- orem 2.2 is n -2/(2a)() / /VE in the regime& & < n 21(2++1). In the high noise regime n (2 < t < 1, the rate Vnt374 achieved by the estimator in Theorem 2.1 is faster; in other words, Theorems 2.1 and 2.2 together imply the minimax upper bound 1 A n =
vnl2
SEF.
where I, == L, + Le given by (16) and (17).
Theorems 2.1, and 2.3 collectively establish the upper bound in (4).
fort < 1.
Vnrs74
2.3. Very high noise regime
3. Minimax lower bound
In the very high noise regime 2 1, we will slightly modify the plug-in strategy used in Sections 2.1 and 2.2. Consider Omp(a, 1) = (ʃ * (x)))) = f-= (1 w))( du= -Fp(x, 1) + { "up(( (w) du. Therefore,
Theorem 3.1 states a minimax lower bound for the high and low noise regimes, which establishes the optimality of the plug-in methodology described in Sections 2.1 and 2.2.
Theorem 3.1. There exists a sufficiently small universal constant >> 0 such that the following holds. If f 2 1, then
I = / (Omp(z,1) 1))² dx.
p(z,t)
inf sup E ((z -II 7. SeFa
n 20+1
Vi
1
x² 2²p(x, t) dx
fz
for t Scc.
2
muppl( - (1)(()) da
The proof of Theorem 3.1 employs a two-point construction. However, standard constructions (Tsybakov, 2009) appear not to deliver the sharp minimax lower bound, particularly for t ス n 22a++1) Instead, we employ an idea introduced by Dou al. (2024), who established sharp minimax lower bounds for score matching in score-based diffusion models (i.e. estimation of the score function 02 log p(æ, f) of the diffused density).
p(a, 1): 1 [ - - (w) de
+
_00
e [ 12 (2)((a))))
i
(plartt) 1 [ ppplx w)f(w) dx
r
Let K R + RR denote a symmetric (i.e. even) probability density function (i.e. K 2 0 and f K(x) dr = 1) which
L1 L22
5

---

Plug-in Estimation of the Fisher Information of Smoothed Density
fo
fo
f1
f2
-1
Figure cartoon of fo and fi.
Figure 2. A cartoon of fo and f2.
is supported on 1] such that K C (R), || oc is bounded by universal constant, K's first [a] derivatives have Lo norm bounded by constant (which may depend on a and L), and (0) = suPSeR ((x). The two data- generating densities deployed in the proof are
the boundary of [-1, 1]. Since fi needs to integrate to 1 since it is density, some mass needs to be displaced from somewhere else to compensate for the mass difference at the boundary; we thus take mass out near zero as shown in Figure
Theorem 3.2 states lower bound for the very high noise
fo(n) - (1uls1):
(19)
regime 2 1 which matches the upper bound presented in Section 2.3.
[i(1) = (o(m) + g(()
(20)
Theorem 3.2. There exists sufficiently large universal
where
constant C' > 0 such that the following holds. If 2 1.
then
0(1)
(-eK
C
1 1--
inf sup 2, JEF.
1
P
vnt²
(21)
1 ( p
for t 2 C.
A two-point construction is also used to prove Theorem 3.2. Let + R + R denote a function such that e (R), tb is supported on [1,, |1b/loo is bounded by universal constant, v/s first [ax] derivatives have Lxo norm bounded by a constant (which may depend on and L.), 1 (b(v) du
Here, < S p course of the proof. It is clear fo E F.. To check fi F.. the Hölder condition must be verified. The constraint p 2 e ensures fi is Hölder smooth. Traditional constructions (Tsybakov, 2009) do not introduce an extra free parameter p. but rather implicitly take p = e; idea to introduce another tunable parameter due Dou et al. (2024) as noted earlier. Further, fi integrates asserts that y integrates to 0, and so provided < p are chosen smaller than sufficiently small universal constant, it is straightforward to verify fi E F.. In the course of the proof of Theorem 3.1, the optimal choices of the tuning I
parameters be tuned in the
- ((V) = 0, and IT ('v))²² du 2 c where e > 0 is some universal con- stant. The two-point construction we use are fo given by (19) and
since Lemma E.2
f2(n1) = So(u) + 5 (
(22)
where < 1 are parameters to be tuned. Figure presents a cartoon of fo and f2- is straightforward to verify f2 F& provided e S p are chosen smaller than sufficiently small universal constant. It turns out the optimal choices are p and
parameters turn out to p x VLV and
A
nvt
I
2=
The intuition for the construction is similar to the intuition described in Remark 1.2 for the blowup of (4) as + 0. Namely, the endpoints of the interval [- are the problematic points which can cause large estimation error. Thus, fo and fi are chosen such that they are different near
In the very high noise regime, happens to be large enough to smooth out the discontinuities of f at the endpoints of [-1, 1] that it does not cause fundamental slowdowns in the estimation rate. From the perspective of the lower bound
6

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
construction, it turns out we can select our two points fo and f2 which actually agree near the boundary of [-1,1]
and Theorem 2.2 for < s S n Z0t1. Fix any 0.
Define the estimator
and still obtain a sharp lower bound. Hence, we make the choice (22), where the perturbation + E (p/p) is localized near 0 and does not affect the boundary. This kind of perturbation will be readily recognized as standard to the expert, excepting the aforementioned idea of Dou al. (2024) to introduce p.
((GP)
1 8
I, ds.
Ift > 0, then there exists C C(a, L) such that
sup E - = Vn SEF.
C
4. Estimation of mutual information and entropy: plug-in achieves the parametric rate
Proof. By (23) along with Theorems 2.3, 2.1, and 2.2, it directly follows E((T(X(:))
((X{;)))
The additive Gaussian white noise channel is canon- ical model in information theory and estimation theory. In this context, the Fisher information related to other cnormationntheoreetie quantities of fundamental interest; we will focus on mutual information and entropy here. As consequence of our results Fisher information estima- tion, it can be shown through the I-MMSE (Guo al., 2005) and de Bruijn's 's (Stam, 1959) identities (which are actually equivalent (Guo et al., 2005)) that a simple plug-in type estimator can estimate at parametric rate both the mutual information between a signal and its noisy measurement through an additive Gaussian white noise channel, as well as the entropy of the source distribution. The idea to plug in an estimator of the Fisher information into these identities has appeared before in the literature (Cao et al., 2020).
C S Vnsa Vnssts a 2011 ds
V= x vn Si 3/ ds +
C
v S - 1a ds + On
204l S
1 ds S 3े +
Iha
On -,,, Since
o+d 2a+I
2 the dominating term is
V
Estimation of the entropy of the smoothed density follows immediately since
= -(xx))=((xxx)))-- ( ((x log (2xet)
2
by the well-known property I(U:V) = h(U) n(U V) for continuous random variables U and V. It is immediate to see that the parametric rate can be achieved, as we state in Corollary 4.3 without proof.
Suppose a clean signal μ ~ f is passed through an additive Gaussian noise channel to yield the measurement X, μ ~ (u, Given i.i.d. data f11...., Mrn ~ f. consider the problem of estimating the mutual information / (Xt; 14). The I-MMSE identity (Guo et al., 2005) coupled with Brown's 's identity (Brown, 1971) relates the Fisher information of the smoothed density f * &pt to the mutual information of interest, as asserted by Theorem 4.1, whose proof is deferred to Appendix F.
Corollary 4.3. Suppose a 2 1. Let 1 (X(i be the estima- tor from Theorem 4.2 and define
1 h(f (p) = 1(Xi:1)
log (og()))
2
Ift >> 0, then there exists C = C(a, L) such that
C sup E ( h(f M(fxxi))) S Vn JEF.
Theorem 4.1. For t > O, we have the identity
1 I ! (1 (1+2) 2t d
dt 1(X::)).
(23)
t
The upper bound stated in Corollary 4.3 notably does not blow up as t + O. This is in stark contrast to the upper bound obtained by Goldfeld et al. (2020b) for the estimator
Since 1(X;; μd) + 0 as t + oo, it follows by the fundamen- tal theorem of calculus that
h(f * (pr) = h ( Σm 5,, * spt): the constant in (33) of Goldfeld et al. (2020b) is explicitly characterized in (63), where is plainly seen to diverge as 1-> 0.
00 1 1.
1(X. p)
Iz ds.
8
Via de Bruijn's identity (Stam, 1959).
Plugging in our estimator of the Fisher information (depend- ing on the regime of s) and integrating yields the parametric rate.
2 d di
I
h( * sp)),
(24)
Theorem 4.2. Suppose 1. Ler I, be the estimator in
an estimator can also be directly constructed for the entropy of the unsmoothed data density f (i.e. the case = 0). The
Theorem 2.3 for s 2 1, Theorem 2.1 for n उम <8 <1,, 1,
7

---

Plug-in Estimation of the Fisher Information of Smoothed Density
fundamental theorem of calculus' yields
Bounding the estimation error amounts carefully bound- ing the remainder term T.
1 % Ig ds.
M(f h(f 4.)
5.2. Lower bound: bounding the functional separation
Again, it is straightforward to see a plug-in type estima- tor achieves the parametric rate, stated in Corollary 4.4 without proof.
As mentioned in Section 3, the lower bounds are proved through two-point constructions and applications of Le Cam's method. A crucial ingredient is to obtain lower bounds on separation of the Fisher informations of the two constructed densities. For discussion, let us first fo- cus on bounding [ * (pr - I(fo * (p)) from below in the proof of Theorem 3.1.
Corollary 4.4. Suppose a 2 1. Let Is denote the estima- tor of the Fisher information defined in Theorem 4.2 and hff (pt) denote the estimator defined in Corollary 4.3. Fix
any > 0. Define
h(f) = h6f (4)) 2 J. Σ. ds.
1
From the upper bound perspective, the dominating term in the estimation error appears to be the linear term in the Taylor expansion of T. Therefore, it is natural in the lower bound to pursue an argument asserting the order of the functional separation is dominated by this first order term. To carry out such an argument, the linear term will need to be computed explicitly, and the remainder must be argued to be of smaller order. Proposition 5.2 gives the expansion, and its proof is deferred to Appendix B.
Then there exists C C(a, L) such that
sup E (1100) - m(f)) s Vn
feF.
5. Proof ideas
This section briefly sketches the essence of the proof strate- gies, which is to examine Taylor expansions of Y.
Proposition 5.2. If (u, u), (uo, (o) E (0, oc) R. then
5.1. Upper bound: bounding the estimation error
2
2uo (u- (w-vo) vo)+ no.u (u, 2)
T(u, v) = Tuo, vo) (u u0)+
u
uo
To establish Theorems 2.1, 2.2, and 2.3, the estimation errors will be bounded through a Taylor expansion of T defined in (5). The proof of Proposition 5.1 is deferred to Appendix A. Proposition 5.1. If (u, v), (wo, vo) E (0, oo) x IR, then Tu, w) = Tuuo vo) + I uo (u, v) where the remainder I Fo (u, v) satisfies
where the remainder satisfies
[T ug (u, u)]
|u|2 V |1o 2
SC C
[u|3 A |40l 13 |u uo!
Un 2
[u]
A
|20]
|0| V |val
|u
uo
Upl
IT Tooro (u, v)l
1u 2 A
0C (lu) V [pol)2
|0| V lvol
(fr]) A |uo))2 lu wol +
[a] A Jool
| vol
To illustrate the use of Proposition 5.2, consider it yields
where C > O is some universal constant.
I(ʃi pp)) I[fo (pt)
In Appendix C, Proposition 5.1 is employed to obtain bounds on the estimation error, by recognizing
(Ompo(z, (z, 1))²
(x) po(s,, 1)) dr
Po(s,, t)2
Oxpo(I, (x,, O-pola, t)) dx. (0(
I I
2
(1 1), p.p (x, T(p(,, 1), O.p 1)) do [T ppp (T (x, f)), Dap (x, ())lda
In Appendix B, the right hand side is carefully examined to yield lower bound on the separation of the Fisher informa- tion. Jumping to the punchline, it is essentially shown
I(f1 * (p) I(fu sp))
'The careful reader may be concerned about the continuity condition (((+g)))) l([)) as 0 needed in order to invoke the (second) fundamental theorem calculus. The desired continuity has been used implicitly in the literature and might be considered folklore by some, but an explicit discussion and proof can be found in Theorem 6.2 and Remark 10 of (Wang & Madiman, 2014) (sce also (Rioul, 2017) for discussion of other works containing this result).
1 pe(1 1)a
g(x,t) dx
2g(1)
po(
2po(x,
where po( (fo * (x) and g(z,t) (g (x). The posterior expectation can be quickly recog- nized o(a.)) E(g(n) Xt, = x) in the Bayes model 2po(m.2)
8

---

Plug-in Estimation of the Fisher Information of Smoothed Density
4 So and X₁ (), The above display
Wasserstein distances. Potential Anal., 56(4):571-595, 2022. doi: 10.10077/1111--000-999099
further motivates the choice of g made in (21) and de- picted in Figure 1, namely we wish to choose such that 2g(1) - ( X2 = z) is bounded away from zero. A sufficient choice is to select which attains its maximum at {1, -13 and is bounded away from zero here; this is congruent with the intuition offered Section 3.
Chewi, S., Niles-Weed, J., and Rigollet, P. Statistical opti- mal transport. arXiv preprint, arxiv:2407.18163 2024.
Ding, Y. and Niles-Weed, J. Asymptoties of smoothed Wasserstein distances in the small noise regime. In Ad- vances in Neural Information Processing Systems, vol- ume 35, pp. 19203-19214, 2022.
similar Taylor expansion strategy is used in the proof of Theorem 3.2 for bounding [[f2 (pt) I(fo (-))-- However, it turns out some simple manipulation enables the use of the simpler expansion of Proposition 5.1. This can be related to the intuition offered in Section 3, which notes the endpoints of [-1, 1] no longer cause slowdown in the estimation rate since there is much smoothing in the very high noise regime. We defer the details to Appendix B.2.
Dmitriev, Y. G. and Tarasenko, F. P. On the estimation of functionals of the probability density and its derivatives. Theory Probab. Appl., 18(3):628 633, 1974. doi: 10.
1137/11 18083.
Donoho, D. L. One-sided inference about functionals of a density. Ann. Statist., 16(4):1390-143) 1988. doi: 10.12 00.1214/aos/11700 1/////////166500045.
Acknowledgements
Dou, Z.,, Kotekal, S., Xu, Z.., and Zhou, H. H. From opti- mal score matching to optimal sampling. arXiv preprint, arxiv:2409.07032, 2024.
This research was supported by NSF Grant ECCS-2216912 and the Institute for Data, Econometrics, Algorithms, and Learning (IDEAL).
Dudley, M. The speed of mean Glivenko-Canter con- vergence. Ann. Math. Statist., 40:40-50, 1968. doi:
Impact Statement
11121444oms/117767977
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Gabrié, M., Manoel, A., Luneau, C., Barbier, J., Macris, N., Krzakala, F., and Zdeborová, L.. Entropy and mutual infor- mation in models of deep neural networks. In Advances in Neural Information Processing Systems, volume 31, 2018
References
Giné, E. and Nickl, R. Mathematical Foundations of Infinite- Dimensional Statistical Models, volume [40] of Cam- bridge Series Statistical and Probabilistic Mathemat- ics. Cambridge University Press, New York, 2016. doi: 10. 10.1077/B80978110733777
Bhattacharya, P. K. Estimation of a probability density function and its derivatives. Sankhyā Ser. A, 29:373-382, 1967.
Bickel, P. J. and Ritov, Y. Estimating integrated squared density derivatives: sharp best order of convergence esti- mates. Sankhyā Ser: A, 50(3))381-333 1988.
Goldfeld, Z. and Greenewald, K. Gaussian smoothed opti- mal transport: Metric structure and statistical efficiency. In Proceedings of the Twenty Third International Con- ference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pp. 3327-3333 PMLR, 26-28 28 Aug 2020.
Birgé, L. and Massart, P. Estimation of integral functionals of a density. Ann. Statist., 23(1):11-29, 1995. doi: 10. 1214/aos/1 17632//57
Brown, L. D. Admissible estimators, recurrent diffu- sions, and insoluble boundary value problems. Ann. Math. Statist., 42:855-903, 1971. doi: 10.1214/aoms/
Goldfeld, Z., Van Den Berg, E., Greenewald, K., Melnyk, I., Nguyen, N., Kingshury, B., and Polyanskiy, Es- timating information flow in deep neural networks. In Proceedings of the 36th International Conference Ma- chine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 2019.
1177693318.
Cao, W., Dytso, A., FauB, M., Poor, H. V., and Feng, G. On nonparametric estimation of the Fisher infor- mation. In 2020 IEEE International Symposium on In- formation Theory (ISIT), pp. 2216-222,, 2020. doi: 10. (00.0STT4444..220.111177
PMLR, 09-15 Jun
Goldfeld, Z., Greenewald, K., and Kato, K. Asymptotic guarantees for generative modeling based on the smooth Wasserstein distance. In Advances in Neural Information Processing Systems, volume 33, pp. 2527-2222222222 22399, 2020a.
Chen, H. and Niles- Weed, Asymptotics of smoothed

---

Plug-in Estimation of the Fisher Information of Smoothed Density
Goldfeld, Z., Greenewald, K., Weed, J., and Polyan- skiy, Y. Convergence of smoothed empirical mea- sures with applications to entropy estimation. IEEE Trans. Inform. Theory, 66(7):4368 1491,, 2020b. doi: 10.1 2090TTT.2220.2975444
Mena, G. and Niles-Weed, J. Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem. In Advances in Neural Information Processing Systems, volume 32, 2019.
Rioul, O. Yet another proof of the entropy power inequality, IEEE Trans. Inform. Theory, 63(6):355533599 2017. doi: 10. 10.1 10.11/991122211.266669999
Goldfeld, Z., Kato, K., Nietert, S., and Rioux, G. Limit distribution theory for smooth D-Wasserstein distances. Ann. Appl. Probab., 34(2):2447-22 2487, 2024. doi: 10. 1214/23-aap2028.
Sadhu, R., Goldfeld, Z.,, and Kato, K. Limit distribution theory for the smooth I-Wasserstein distance with appli- cations. arXiv, 2107.13499 2022.
Guo, D., Shamai, S., and Verdú, S. Mutual information and minimum mean-square error in Gaussian channels. IEEE Trans. Inform. Theory. 5164)1221- 2005. doi: 1000000..2000.4444000
Shwartz-Zivv, R. and Tishby, N. Opening the black box of deep neural networks via information. arXiv preprint, mexiv:1703.000 2017.
Gupta, S., Lee, J., Price, E., and Valiant, P. Finite- sample maximum likelihood estimation of location. In Koyejo, S., Mohamed, S., Agarwal, A., Bel- grave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems, vol- ume 35, pp. 30139-30149, Curran Associates, Inc., 2022. URL https: https::/prcceeeingg... neurips.. ce
Song. Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
Stam, A. J. Some inequalities satisfied by the quantities of information of Fisher and Shannon. Inf. and Control, 2:
Ptwpeee---Pppe--ooreeeeeeeee Conterence 1959.
c27 c2lcfbb0aa2e9e5559994323209 pdf.
Tishby, N. and Zaslavsky, N. Deep learning and the in- formation bottleneck principle. In 2015 IEEE Infor- mation Theory Workshop (ITW), pp. 1-5, 2015. doi: 10. 10.1109/TT2220155133316666
Gupta, S., Lee, C., and Price, E. nigh-dimensional lo- cation estimation via norm concentration for subgamma vectors. In Proceedings of the 40th International Confer- ence on Machine Learning, (CML'23. JMLR.org, 2023a.
Tsybakov, A. B. Introduction to Nonparametric Estimation. Springer Series in Statistics. Springer, New York, 2009, doi: 10. 10.100//bb1 (007/b13799. Revised and extended from the 2004 French original, Translated by Vladimir Zaiats.
Gupta, S., Lee, J. C. H., and Price, E. Finite- sample symmetric mean estimation with fisher infor- mation rate. In Neu, G. and Rosasco, L. (eds.), Proceedings of Thirty Sixth Conference on Learn- ing Theory, volume 195 of Proceedings of Machine Learning Research, pp. 4777-4830, PMLR, 12-15 Jul 2023b. URL https: http:///roced//eeeeee .ml.mlr.....................mm.m press/vee http////////gpaaaaaaaa html.
Vaart, A. W. v. d. Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 1998.
Wang, L. and Madiman, M. Beyond the entropy power inequality, via rearrangements. IEEE Trans. Inform. The- ory, 60(9):5116-5133 2014. doi: 10.1 10.1109/T 2338852.
Han, Y., Jiao, J., Weissman, T., and Wu, Y. Optimal rates of entropy estimation over Lipschitz balls. Ann. Statist,, 48 (6):3228-3250) 2020. doi: 10. 10.1214/19AASS1927
Zhang, Y., Cheng, X., and Reeves, G. Convergence of optimal transport distance with sub- gamma distributions and dependent samples. In Proceed- ings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pp. 2422-2430. PMLR, 13-15 Apr 2021.
Kandasamy, K., Krishnamurthy, A., Poczos, B., Wasserman, L., and Robins, J. M. Nonparametric von Mises estima- tors for entropies. divergences and mutual informations. In Advances in Neural Information Processing Systems, volume 28, 2015.
Gaussian smoothed Gaussian-smoothed
Laurent, B. Efficient estimation of integral functionals of a density. Ann. Statist., 24(2):659-68) 1996. doi: 10. 2214/2o/1002294455..
Laurent, B. Estimation of integral functionals of a density and its derivatives. Bernoulli, 3(2):181-211, 1997. doi: 10.2307/3318588
10

---

Plug-in Estimation of the Fisher Information of Smoothed Density
Appendices to "Sharp optimality of simple, plug-in estimation of the Fisher information of a smoothed density"
A. Proofs: upper bound
Theorems 2.1 and 2.2 involve plug-in estimators of the form
i == (x, (x, da,
Before using Proposition 5.1 to bound the error, we first present its proof.
Proof of Proposition 5.1. Observe VT u) = (- b² مي 2² For (u, v), (uw, vo) E (0, 00) x R, consider that T(u, v)
T T(uo, vo) +
c2
[u - uo) (o) + 25 (v - un) for some point (Eur Eu) on the line segment {(uo X(u uo). to E
R² :0 < À < l]. The claimed result follows immediately.
Proposition 5.1 yields
Σ. I = =
-1100 (1 (x, ()) dr x / R1(2,t) R2(x, t)
(25)
where
(x, = (1Opp (x, f) V Oxp(x, f)))
10 (x, - p(x, t)l,
(26)
(Ip (z, t) p(x: t)1
10.p (x. t) 1..p(r, t)
R2
10 (z, t) |p(z, t) O.P (x,t) Osp(z,
(27)
To prove Theorems 2.1 and 2.2, we will focus on bounding (25) in expectation. Theorem 2.3 involves slightly different estimator, but though the term of focus is different in Section A.3, the essence of the argument still involves the Taylor expansion given by Proposition 5.1.
A.1. Proof of Theorem 2.1
Recall f and Oxp are given by (10) and (11) respectively in Theorem 2.1. Examining the terms (26) and (27), it is clear that error bounds of these estimators will be needed; Proposition A.1 delivers suitable bounds, and its proof deferred to Appendix C.
Proposition A.1. If a 2 1, then
E (x, p(z, 1)) x t) nvt
[1ll/g +e ألاحا 1 1||=|>1)
D.v (x, 1) 0xple, - ()) E
E(x, t)
nf3/2
where if and O.p are given by (10) and (11) respectively.
Furthermore, estimates on the size of p and 0.p are needed as seen in (26) and (27). The proofs of Lemmas A.2 and A.3 are deferred to Appendix E.
Lemma A.2. Ift < 1, then
if < 1, T| > 1.
p(x t)
x (11 1A
21

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
Lemma A.3. Ifa 2 1, then
10pplr ()1 z + + px(( 1)2 + ((x,
(Mnalss) +
2t
1
((1b)))
Vt
With these estimates in hand, in position to bound (25) in expectation.
Proposition A.4. Ifa 2 1 and t < 1, then E (1 (1 1) dæ) S Volspt where R₁ is given by (26).
Proof. By g(x, 1) S p(x, 1) S {(z, 1) and Lemmas A.3, E.I, we have
( (a, t) Pap(z,t)) 2
E
10 (x,tt)-((z,t)
(()
E(x,
(x +12 + px(x - 1)2 + g(0; 1) (1o1s1) + (1 11
E (10 - ((x,t))
(28)
g(x, ()
g
Looking at the first term in the product (28), note <xx, t) = for |x|| S and e(x, x (1 A vi
(iz] 1)a
| c
21
for | > 1
as implied by Lemma A.2 since p(a, 1) > g(2r, 1). Furthermore, ( - 1)² S D 1) and g (3 Therefore.
1)2 3 S - 1).
(e(x + 1)2 + pp(x - 1)² + g(x, t) (1 (111/1 +
Nel 1;a vT
11//31
g(x,t)
(1al-112
(pll3 1)2 + speetr
1)2
V +)
+ - 1)) + +1) [12111)
+
E(x, t)
Vt
(1x1-11 21
1 V (pc(2 + + po(x 11) 1 (11111)
x
[1ll-13 21
1+
-
1 (1//)111
Vt
t
Let us examine the second term appearing in the product (28). It follows by Jensen's inequality, Proposition A.I, and Lemma A.2 along with p(x, t) x a (xx t) that
E ( (z. t) p(x,t)) t))
E p (I, t) - p(x, 2)
g(x, t)
g(x,t)
x [1+|11} + (11
(|
Vng(x,t) t)62/4
Vnul/ ((1111) + Vaons (Iv (12) - 1)1/2 11)1/2
111-112
1 1|
11/1
Therefore,
R2 (2, tz
1 ((1+
Vi (2i(0 pi(x (1)--11) |
((Iv 1e| - ))
[x 112
(1x1-1)²
Vi
7
25
111)1111
Vrtl/4 (Iv (|2) 1)1/2
e ررا-
Vntl/4 1 1(11/11) +
1 (1/1)11)]
dr
21/4
fi(x 1) + pi dx
1+
vntz/4
vt
S1
12

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
Vnt3/4
|0|21
1v
[
Vi
1v
1)1/2 (|2]
11/4
V8
1
-11=--1)2
41
dx
1
(|0l
- 1)2/2
31/2
Vntl/4
|x|21
1v
$1/4
Vt
42
de
Vt
(2/4)
22/4
2
Vntl/4
1
Vt)
+
Vrit3/4
1
Vt
+
t2/4
+
Vntl/4
1
t1/4
Vnt3/4
The proof is complete.
Proposition A.S. I 2 1 and t < 1, then E (1 R2(1, t) dæ)
S Volsnt where R2 is given by (27).
Proof. By g(x, 1) S p(a, 1) S <z 1) and Lemmas A.3, E.I, we have
E(R2(I, t))
E Oap (z, t)] v Oxp(2, t)) 10pp (x, - 0xp(xx
g(2,8)
fi(z +1 p((x 1) Ve( (1 + 21/4
1012113
E' (Opp (z, )xp(x,t) t)
(29)
VE(x,t)
VE(x,t)
Looking at the first term in the product (29), note g(x, t) = for |o|| S and g(r, x (1 A vi as implied by Lemma A.2 since p(x, 1) = g(x,t). t). Therefore,
| c - for
| > 1
del
fi(x + px(l 1) + Ve(a, 1) 1 111S11 + 217
[1/>1
VE(a,t)
(1x1-1²²
pelx 1) ppll -1)
e
ac
$ (1+ (( p:(2 1)) 1 1111s11
+ 12/4
1 1|11|11
Ve(a,() ()
((Iv
1)1/2 (|al -1)u2 11/
t//1
s +1 p 1)) ((is)))
1 (1)>11)
Vi
vi
Let us examine the second term appearing in the product (29). It follows by Jensen's inequality and Proposition A.1 that
/ (00.0 0 (z, t) 8ap(e,)) 1))
(r, E (1ou0 (x, - Oup(a, 0)) x
te(
Ve(z,t)
5
Vnt3/4
Therefore,
( R2 t) dx
E
1
dx vnt3/4 1 (1v (10l 1)1/2
1
[1
11/4
[(=-1)
x
1 + +tpp + pp(x 1)dz
dr
Vnt3/4
t1/4
vt
Vi
1
(1+ 1/4
11/4 + 1/4
Vnd3/T
Vnt3//
Vrt3/4
The proof is complete.
13

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
Theorem 2.1 follows direct consequence.
Proof of Theorem 2.1. Proposition 5.1 yields (25). It immediately follows from Propositions A.4 and A.5 that
su E
(īx Iz
X Vni3/T, claimed.
A.2. Proof of Theorem 2.2
In Theorem 2.2, recall j and O.P are given by (14) and (15). As they are constructed from preliminary estimators which smooth, an analogous version of Proposition A. is needed; Proposition A.6 is such version, and its proof is deferred to Appendix C.
Proposition A.6. a 2 1, then
E' (x, p(t,)) t)13)
20 S n 20+1 g(x, t),
O2p (x, t) - 0sp(E, t) E
En r c(x, + (pt(x + 1)² + pp(x 1) žम1
2(m1)
where i and O.p are given by (14) and (15) respectively.
As described in Section A, the proof Theorem 2.2 proceeds by furnishing a suitable bound for (25) in expectation.
Proposition A.7. If a 2 1 and t < 1, then E (I R1 t) dz)
2
where R1 is given by (26).
Proof. By the same calculation leading up to (28), but now using Proposition A.6, we have
E (21
(i + 1)2 + (2 1)2 + c(x,() 1 + عرراا vt
(w/>1]
E (2, t) - p(2,
E(x,1)
g(x,()
<
() 1)) +1) 1 (11111) + (el(x + 1)² + pr - 1)²
+
dل|-11
21
1 1(msn))
E(x,t)
vt
I041
V VE(x,t) t)
$ (pe + (1x(( + 1)) +1) (1eاا1) +
(Iv/bt -1
1
(lei
(ail-1)² 21
1 1(10)1)))
e
25
+
Vl
t
n FT (L(mis))+ (Iv (lx) 1)1112
|(
t1/4
Integrating over x yields
E S R1 dx
1
2041 n ~
Vt +1
n 2011
1v ([2] - 1)2/23
1v x 1
dr
1//1
1v (1x) - 11)2/2
- 1 112
+ n
dx
11/4
|=|>1
11
11
+ n I
Vi
Vi
n 20
The proof is complete.
14

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
Proposition A.8. a 2 1 Ladd t < L, then E (1 R2(I, t) dz) S n 2a+1 + Vt
where R2 is given by (27).
Proof. By the same calculation leading up to (29), but now using Proposition A.6, we have E (R2(1, !))
Clel 1)2
f1(1 - + + 1)
1) + VE((,t) LIIaSs +
0174
L [11|>1)]
E
Oop (I, - )sp(a,t))
VE(r,t)
VE(0,t) ()
< (1 + (o + 1) pi(x 1)) (1+11)) + ((IV
IV (12] 1)1/2
((دما)) 21
11/4 -لاl-11 +
41
1 1(///11)
11/4
vi
n 2तl (12( 1) + (1 1)) 27
ve(xx,)
Ve(z,t)
(1 (1+ pı(x 1)) (1lls)) + ((Iv (lol- 1)2/22
12/4
1 ||<1)>)
t1/4
Vt
Vt
n (p)(x+1)+ pr(l
p/ - 1)) +
(Iv
(1x) - 1)2/2
[1+1 112
1 [1/1/11/
+ +n f0
t1/4
Integrating over yields
E
R2 1) dx
S 71
+ n
+ n
Ppll + 1)2 + :pee - 1)2 dr
1=|S1
IV (|x] [1o]
1)2/21
1)2
22/4
11ll 1)²
+ 11
Vt e
dec
|<|>1
11/4
Vt
11 (1ar) 1)1/2
1
Osl
112
((1v
(1/0)
1)1/2
1)a
t 1/4
1)a
11
2041
e
41
dx
|x|>1
t1/4
Vt
t1/4
Vt
Vt
20+1
(1+)
(1
21/4)
20+1
(1+2VA) (2/))
2a
2a
+
Vl
Vi
71
TaTT
X 71
Vt
The proof is complete.
Proof of Theorem 2.2. It immediately follows from (25) (which is furnished from Proposition 5.1) along with Propositions A.7 and A.8 that seF. E ( I S n
+ VT
A.3. Proof of Theorem 2.3
To prove Theorem 2.3, the estimation errors of L, and L2 given by (16) and (17) will be bounded separately. It is immediate to bound the error of L1 since it is unbiased; the bound is stated in Proposition A.9 without proof. Proposition A.9. We have E (lI- - La
Vntz where L, is given by (16).
To bound the estimation error of Lz, an argument involving the Taylor expansion of T given in Proposition 5.1 will be employed. Proposition 5.1 gives
(x, - (1 1 [ papelzs m))(() du dr
Σ,2 L2
r r (x, 1), (x,
Rí (x, t) + R2 R2 dx
(30)
15

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
where
1 - ())(()) du) 2 (x, (x, A (o(x,t)l)*
(x,
10 - -p(,,,
(31)
1 (spe - (1) du V (x, t)l
1 (x, - 7 -/1- - ()) du
(x
(32)
(x, (p(x,
To bound (30), bounds the plugged-in estimator's s emrors are useful. Proposition A.1 already gives a bound on the error of Pd given by (10), and the following lemma (whose proof is deferred to Appendix C) provides an error bound for q
Lemma A.10. We have
E
q (xx
& [
- (1) du
x
g(x, 1) n25/2
where of is given by (18) and g(x, t) by (6).
With these preliminary ingredients in place, (30) can be bounded in expectation. Proposition A.11. Suppose e > 0. Ift 2 c, then E (J R1(x, 1) de)
1te 1/2
Vntz where Ri is given by (31).
Proof. Since pla,1) A ff (x, 2 ((x,1) and 1 uu (M) du v (2, S E(a.t)
g(x.))
we have after
f
invoking Proposition A. E (R1 (z. t)) x } E (10 (x, p(a, t)l) x 2
_(x,1))/2 V
Integration yields
1/2
w 28
E (R)(2x,)) ()) de do
dp
de
Vnt²
Vi
C
1/2
Vt (f-
(a-m))
1/2
VnF
Vnf2
e
28
dje
dr
lc|ll
1/2
1 6-
Vnt2
Vnf²
Vt
de
dx.
|<|>1
1/2 +1
Vnt²
as claimed.
Proposition A.12. Suppose e > 0. 1ft 2 then E (1 R2(xxt)do)) S
14--1/2 Jut
where R2 is given by (32).
Proof. The proof is very similar to the proof Proposition A.11, except Lemma A.10 is invoked in place of Proposition A.1. By the same reasoning in that proof, we have
F. (Ra(E, 1)) $ } F. ( /
1 [. (P)(o- - vnt5/4 &
1 g(c,1)1/2 g(x,1)1/2
Vntl/4
Integrating and repeating the calculations in the proof of Proposition A. yields the desired claim.
Theorem 2.3 follows immediately.
Proof of Theorem 2.3. It follows from Proposition A.9 that E ((., ((Z--.1) I.l) S vuF + + + ((12-21)) Proposition 5.1 yields (30), and (30) is bounded in expectation by Propositions A.11 and 121 which yields the desired result.
B. Proofs: lower bound
This section contains the proofs Theorem 3.1 and Theorem 3.2.
16

---

Plug-in Estimation of the Fisher Information of Smoothed Density
B.1. Proof of Theorem 3.1 The proof of Theorem 3.1 proceeds through Le Cam's two point method applied to fo and fi given by (19) and (20) respectively.
Proposition B.1. If So. J1 Far then
inf sup E ([2 (0l L. JEF.
[[[fo * (p) - I(fi ppi) -nddaa(h
Proof. The desired result follows straightforward consequence of standard and well-known results (Tsybakov, 2009).
Proposition B.2. We have (f (o) S 2
Proof. By Lemma 2.7 of (Tsybakov, 2009), it follows (fi I x2 ST. (1 (p)))))o())) dp (1 du S 2a S K (4) du = 2a p. as claimed.
So(u)
2
To furnish a lower bound on the separation of the Fisher information, namely
I(fi p.) I(fo :p)) S t2), (po(2, O.po,, t)) dx
we will use Proposition 5.2. Before doing so, we first present its proof here.
Proof of Proposition 5.2. The proof follows from direct application of Taylor's theorem once the gradient and Hessian are computed. A direct calculation shows (T(, v) (- - 2n) and (2, = complete.
2v
u
x²
The proof is
Recall throughout this section and throughout Appendix D.I, we denote po(x, t) = (fo+)))(() and pı (xx = (z) Proposition 5.2 yields the expansion
(Ompo(x, 1))²
I(Л1 pc) I(Jo * xx)+
= p-)=-
(pi (2., po(x, ()) do
Po(xr, 1)2
00
dspo(I, t)
+ 2
12pi Ozpola, t)) dx
(33)
po(x, ()
-c
+
I mooo 0xp1 (x, t)) dz.
Proposition B.3 bounds the remainder term in (33). Its proof is deferred to Appendix D.I.I. Proposition B.3. I 2 1 and 0 < tt, < 1, then
| T mntertt,eemcces((nttt.... (2
2a 20
f)) dr S
+
Vi
p²
With the remainder term handled, attention will now be directed towards the main terms of (33). Lemma B.4 simplifies the first term and Lemma B.5 simplifies the second term; their proofs are deferred to Appendix D.1.2.
Lemma B.4. There exists some universal constant > such that for we have
1- (O=po(x,t)) 2) p)(a,t)2 (pi(2, t) pu(x,t)) dr (-2
00 pı(1 1)2 g(x,t)
dr) ) seee
J po(z,t)
po(x,t)
where g(x,t) t) = (g* (x)
17

---

Plug-in Estimation of the Fisher Information of Smoothed Density
Lemma B.S. There exists some universal constant c >> O such that for
have
2 /
Oxpo(azt) (d,pI(x, - )spo(a, 20(1) So¹ po(æ,t)
pe(x 1)2 dx x E" Pu(æ,t)
+c
p
Putting together Proposition B.3 with Lemmas B.4 and B.5 delivers the following functional separation. Theorem B.6. For 2 1 and 0 < p p 1, we have
Vi CE -
2a + + -c
-c
2a
I(f pp) I(fo (p)) 2
v1 p²
for some universal constants C, c, c' >
Proof. By Proposition B.3 and Lemmas B.4, B.5, have
(I(f p.) I(fo * (:)) (%/ fr(z= 1)²
(2g(1) g(z,t) t)
de S (E" +
20
2a
vl x
(o(
(2)
pa
for some universal constants C, > We can recognize the posterior expectation
g 1 S p() 14) du F(g(1) |x =xx x)
(p, 2 Pelx - -)) du
from the Bayes model j fo and X₁ I~~ (H,,t). Since g(p) S g(1) (because K maximized the origin), it follows g(z.()
that 9(1) (2,(2.() 2 0. Therefore,
L ( 1)2
g( 1)
dr 2 / ge(l 1)2
po(x,t) 9(1) dr αa
(g((1)
2po (z, t) for some universal constant > 0. The proof is complete.
po(x,t)
Vt
With Theorem B.6 in hand, the stage is set to prove Theorem 3.1.
Proof of Theorem 3.1. Let > 0 be a sufficiently small universal constant and suppose t S c. Set p C (VIvc) where C' >> is a sufficiently large universal constant and
An
I
C
nvt
Since C' is sufficiently large and p 2 it is straightforward to verify Si E Fo. It is also obvious So E F.. Therefore, Proposition B.I yields
inf sup E ((āe-14) Z. JEF.
I(fo * pp) I(fi p.)..
Here, we have used n dK1,(fi (l/o) $ from 20 220 p (-(aa-1) and Proposition B.2. Theorem B.6 thus yields, for some universal constants C", "",d,e d" 0,
inf sup E ( I.)) 2, fer.
a C² Ea + -
2a
2a
Vi
p
By taking C > to be larger than sufficiently large universal constant, we ensure
c'ca
C'
Furthermore, consider Vi
BVT
C Vtp Ce TV7 Ce
de SC" a-1 1, d
8vt 8VF where the last inequality follows from a 2 1 and C' being
chosen sufficiently large. Likewise, taking C > 0 sufficiently large gives C'e de
Furthermore, since < c and c is
8VT
sufficiently small, we have
8v8 2 C' ". Therefore, it follows
inf sup E (Zz, -z)) feFu
-Ie) z Vt 8Vt
a d E 4.
Vt
Vntt/4
Vt
as claimed.
18

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
B.2. Proof of Theorem 3.2
Le Cam's method (stated in Proposition B.7 without proof) is applied to fo and fe given by (19) and (22) respectively, Denote (2 t) (f2 (2) throughout this section.
Proposition B.7. If fo, J2 F.. then
inf sup E ([Ze-Z)) T. JEFo
II(So pp) -I(2 pi)l nd fl.).
Proposition B.8. We have dKL (f2 fo) S 22xp.
Proof. The proof the same as the proof of Proposition B.2.
To bound the Fisher information separation [I(f2 * (p)) I(fo pt)). a Taylor expansion reminiscent of the proof of
Theorem 3.1 will be employed after some simplification. Since (2, t) - P2, t) + S-, ppp((1 =2) du, we have
I(f2 * pp)) = / (O,p2(x,t)) de
p2(x,1)
(Li (1)=())(()))))) ) p2(0r, 1) dx
x²
2
00
[" xupple 12)2(1) du de +
1
13 P2(1, 1) dx
12
12
[ 1² (2(1)) du + 1 $1000 (L, (L - H)fz(H) duy
12 +1
2
S2( du
dx
f2
82
D2(0,
/ (S (S 1)52(H) du)
2
1 1
[ 12 f2(u) du +
dr.
t t
(2(
The same expression holds for I(ʃo (p)) with fo and po in place of f2 and p2. Therefore,
I(f2 pp) I(Jo * spt) 8² [ 1² (Jo(m) - f2(1)) dp + / T(2((,, N2(x, t)) (po(or, 1), No(a, 1)) dr,
1
where Y is given by (5), No N2 () 1 upx( (2)2()) du, and ()(x, (o(()))))))))))))))))))) du. Taylor expansion (Proposition 5.1) yields
1
[
1
I(fa pp) I(fo pp))
(F(f(()) f2(11) du
Montoss. Na t)) dr
22
t2
a
[
(0) du +
00
12
Ipnle t). (2 t)) dr
t²
t2
-oc
pp'
1
Trote.s.M.aaseeaaa N2(x,, ()) dx, (34)
t2
12
where Mim t))1 S C (R, (I, RR(E,, t)) with
No(2, 1)2 v No(2, 1)2
R₂ (x, 1 Po(x, t)² A P2(x, t)2
112(1, - po(x,
(35)
1 [No(2, 1) V N2 1)]
R2(x,t) ? po(x,t) A (2(x,t) t) N2( No(a, t))-
(36)
The terms (35) and (36) are lower order terms asserted by Lemmas B.9 and B.10.
Lemma B.9, Suppose e > 0. Ift 2 c, then there exists C depending only on e such that - R1 t) dr < RR given by (35).
Ce''
where
B
19

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
Lemma B.10. Suppose c > Ift 2 e, then there exists C depending only on on e such that I R2(2, 1) dx x where R2 is given by (36).
Ce 15/2
The proofs of Lemmas B.9 and 10 are deferred to Appendix D.2. We are now ready to prove Theorem 3.2.
Proof of of Theorem 3.2. Let C > 0 be sufficiently large universal constant and let / C. Set p- 1 and e C Since p 2 is straightforward to verify fa E Fa. Since fo E Fa also, it follows by Proposition B.7 that
inf sup E ( I 2 I(fo * pp)) I[f2 4p))--
3. SEF
Here, we have used Proposition B.8 and 22 p Σ π obtain n dlr, 11 [o] E 1. Therefore, it follows from (34) along with Lemmas B.9 and B.10 that for some universal constants C² c' O, we have
inf sup E (Z p2 (d C"
+
In feFa
82
11/2
Vnt²
Here, we have used f 2 C' for C > 0 sufficiently large universal constant. We have also used | v(v) del proof is complete.
2 1. The
C. Technical tools: upper bound
This section contains the proofs of Proposition A.I, Proposition A.6, and Lemma A.10, which give error bounds for the plugged-in preliminary estimators. They deal respectively with the high noise regime, the low noise regime, and the very high noise regime.
Proof of Proposition A. is clear truncation only improves the estimation error, and so is immediate that E(( (x, () p(z, +) < (( t) () and E O.p (z, t) 0mp(ar, t) S E (10.0(z,)) t) 0mp(a, d))²). Therefore, suffices to focus p and 0zp. Note p(x,, t) and 0,p(2,, t) are unbiased estimators for p(æ, t) and 0,p(2,, t) respectively, so it suffices to compute their variances to establish the desired results. By direct calculation,
Var(pp(x - -m)) S 2m ["
(
(ar(p(z,t)) = n
E
f(u) du
du
nxt
2nt
V2rt
For |ao| 1, we directly obtain the bound Var(plu, 1)) x
s(a.t) nyt
اا since e (اسيع
Suppose |ar|
1. Then
(1el 192
Uall 11² e 21 E(x,t)
21
[m-p)
dp
dp x
n22xt -1 2xt
nvt
22T
nvt
which completes the proof for the claimed bound of Var(p(z, t)).
Next, we examine estimation of the derivative 0zp(x, t). Observe
Var(O.p(z,1)
Var(y'((x - -1i4))
n?
i=1
1
1
La ())
t2
2nt
x
2
1
(a ml?
E
du
nt3/2
Vt
(
m5
dp
nt3/2
V
2
g(xr t)
nf332"
20

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
Here, have used the inequality y e22/2
Proof of Proposition A.6. Recall that D and Opp given by (14) and (15) are obtained by truncating P and O, given respectively by (12) and (13). Since the truncation only improves estimation error, it follows by Jensen's inequality that
E (2, p(x, & E (19(2, t) - p(r,t)13 [1
2a (4) du S n 20 g(r, t),
as desired.
To prove the remaining claim, observe
Ozp (s, 0up(o,t) (1)²
E
E ( 1) 0xp(a,,()
$ 1f(-1) - f(- f(11)/pp + 1)² + (f(-1) - ((-1)) 5e - 1)2 + [ f'))) 11) 1)
On (f + 1)² p - 1)²) + 20+1 g(x,t),
ا(-م)1
where we have used Jensen's inequality to obtain the penultimate line. The proof is complete.
Proof of Lemma A.10. Since [_, upo(x ()f(m) du S E((z, t) and since truncation only improves the estimation error, it follows
E (o,,t) - uprlo- -v)pl(-))((uddd (
1 m 1 Σ
uapel mi) - 1 -pp(o- w)) du
(2
SE
nl
pispe (x pii)
Var
E'pppeel - 21)2))
nt2
1
c ااسيا dp
2 nt5/2
27rl.
g(0, 1)
x nt5/2
as claimed.
D. Technical tools:
lower bound
Appendix D.I contains the deferred proofs of results stated in the main text as well as technical lemmas used in the proof of Theorem 3.1. Likewise, the deferred material for the minimax lower bound in the very high noise regime (Theorem 3.2) is contained in Appendix D.2.
D.1. High noise and low noise regimes
This section contains a useful auxiliary lemma stating some properties of fo(m) = ] (Mis))) and to, == (fo * (z). Appendices D.1.1 and 1.2.2 respectively contain the deferred proofs for bounding the remainder term in the Taylor expansion of Proposition 5.2 and bounding the functional separation.
21

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
Lemma D.I. Lei So(p)= (elss) and po(a, (fo (x)( We have
1
po 2 (1, (x,t)) t) 11}, (z, pp(e 1) (p - 1)
0zmPo(2; + p1lx + +1) + -= 1)
2
Proof. The definition directly yields = through integration by parts, OzPo(2, t) = (fo * (z) = - +)+ + f6(4))(( - 14) dp = (pp(( + 1) f0(x 1)). With the second claim in hand, direct differentiation yields the third claim.
- 13. The second claim is proved
D.1.1. PROOF OF PROPOSITION B.3
In this section, we prove Proposition B.3. From Proposition 5.2, we have
I mate..)...mme.)))p(( Osp1 (2, f))l - (2,1 + R2(x, + Rs(x, t))
(37)
where
1xpolxx t) v 10-p1 t)1
R1 (2, t)
(x, () po(o, 1)1
(38)
[oo t) (pi (2x, 1) 1
R2(x,t)-
(xpi(x, t) - Oxpo t))²
(39)
Ipo(ær, t)] A Ipa (z, f)
1appo f)l V 10.p1 (I, t)]
Rx(x,t)
(x, pol:; (z,,)) O,Po(x,t))-
(40)
[po f) A 1p (z, t)
In order to prove Proposition B.3, it suffices to furnish suitable bounds on R1, R2, and R3. The following preliminary result is a crucial ingredient.
Lemma D.2. We have
(p) - po(xx t)1 s 2=5(
(x, - 0xpulxx 1)2 s 2a (p((o+ 1)2 pe(x - 11)2 1) 2²
/2
Proof. By direct examination and that K is bounded by a universal constant, we have
1p1(x, 1) - -Po f)1² = =1(g ( * (x))
S 20 [" K 14
2 Ge 11) dyl + +
S'
1 (H-1)
2
K
Pela - 11) dp
P
[' K
[u+1)
2
+ 20
(0 - 11
x 20 E)
22

---

Plug-in Estimation of the Fisher Information of Smoothed Density
To prove the second claim, observe that we can apply integration by parts to obtain
10.P1 t) O=po(I, t)p = (g (x))
[", g (1)) ()) du ,
2 1 (l0(-1)13p + (,x)) 1 I ((())p(( dd 1)2 + s(x,t)² + 2 S 20 (1((x+1)++
p2
Here, we have used that K' is bounded by a universal constant. The proof is complete.
We can now bound the three terms R1, R2, and R3. Lemma D.3. I 2 1 and & < 1, then I- R1(x, 1) dx 5e where Ri is given by (38).
Proof. Let examine the cases |x]| < and [x]| 1 separately. Suppose |a| s Lemmas A.3 and D.2 then yield
2a (x, pr(x + 1)2² + pr 1)2 ((x, 2n g(., 1)2 S
Vi (pr(x spr(x - 1)) + 20
g(r,t)3
Here, we have used E(x, () x for | < since < 1, as implied by Lemma A.2. We have also used upll 1)² x I Fz - 1) and (I + 1)² x 1
v pr(( (x + 1). Now suppose |x| >
Then
R1 (1( S 20 pr(x + 1)2 + 9r( 1)2
+
2a
(ل1--1 21
Sea
20
20
(Iv |0|
1)
-
1
(+1-1)2
2t
2a
e
(1|--11
21
g(z,t)
Vt
t
Vt
Here, we have used E(x, f) x (1 11 1
as implied by Lemma A.2. We have also used f 1)2 + (x +
1) x te !را-1
Putting together the bounds, we obtain 1 R1 t) dx I
20
vt +
2n +
vt
20
Vi
20
as desired.
20 Lemma D.4. Ift < L, then 1 R2(1, t) dr S p
where R2 is given by (39).
Proof. As in the proof of Lemma D.3, we analyze the cases |x|| < 1 and [x]| separately. Suppose |x|| S 1. Then since Po(x,t) t) A (x x E(x, t) x 1, it follows from Lemma D.2 that
2a Vi (pc(x ( 1) px(x 1)) + 20
R2(2, t) x (.,)(x,t) dzpola, t) z
P²
Here, we have used (r + 1)2 z f1 + 1) and pe - 1)2 < + (x ----1 -1).
Now suppose |aa| > Then g(x, 1) x 1 |
1) as implied by Lemma A.2. Therefore,
R2(1, 2a (e(x + 1)2 + pp(x - 1)2
E(1, S 20 Iv |x| - 1 1 (=z--13 112
)
2a
8a Po( dx.
g(x,t) t)
p
V
Putting together our bounds, obtain J R2(1, 1o dr. S
20
vi
+
20 claimed.
x
Lemma D.5. a 2 1 and 1, then - Ro(x, t) dr S
20
+
20
where R3 is given by (40).
p
23

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
Proof. As in the proofs of Lemmas D.3 and D.4, separately analyze the cases [a]] S and |x|| > 1. Suppose |a|| 1.
Then po( = p1 g(x; 8) = 1 as implied by Lemma A.2, and Lemmas A.3 and D.2 yield
R3 x 20 (fr( + 1) ppl( - 1) +1) (pllt 1) p2((0 1) D
22 + 2 (pr((x + 0p(x -1) + 2a
Vt
Now suppose |a] > 1. Then Lemma A.3 yields
(21-1)
R3(2, 1) S 2a Pa(x + 1) + (x 1) + E(z, 1) Tee
g(x, t)
c(x,t) (cx(x- (x
E(x,t)2
x 2a Px 1)² + tp(l + +)2
(
-
+ 1 p
+ 22a
21
(10(20 1) + pe(x + 1))
g(z, t)
Vt
((2,t) 1
+ 2n
P
Vt
S 2a
1V
| - 1
1
(1-1-1)
21
2
+
2a
220
(0i(x- + pe(x + 1)) +
20 1 P v
1-1-1)
Vi
Vi
p
Here, we have used s(x,t) x
A
v
1) 2t
as implied by Lemma A.2. The final expression follows from the fact
E(x,, t) S Putting together our bounds yields 1_ Ra(x, t) dx x 2u V + 2 p as claimed. Here, we have used p S to
obtain 20 2 20
With these bounds in hand, we are in position to prove Proposition B.3.
Proof of Proposition B.3. By Proposition 5.2 along with Lemmas D.3, D.4, and D.5 we have
r
1 a.pi (x, (x, do
( 1)
R2(I, 1) dx) + *f 00
R1 dr +
Role, dr
c 20
2n
2a
vt
p
p
2a
20
Vi
p2
since p < 1. yielding the claimed result.
1.1.2. PROOFS OF LEMMA B.4 AND LEMMA B.5
This section contains the deferred proofs of Lemmas B.4 and B.5, which are used in Theorem B.6 to establish a separation in the Fisher informations.
Proof of Lemma B.4. Note since is an even function, the mapping x + g(x, t) is an even mapping. Therefore, we have (8.vn(a.t))
(pi (xx, 1) - po( 1)) dæ = -2 fo (8.n) pola,t)
po(art)" g(x, 1) dr. Then, from Lemma D. it follows
00 (O-pa(x, 2))
- 2
polæ, g(I, t) dx
- /
000
(1p1(x + 1) - (i - 1))²
g t) dx
po(x,1)²
24

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
1 0e 9i(x - 1)2 + p2(l + 1)2 2p1(2 ()) + 1)
g(2, t) po dæ
0
po(2,t)
1
/
00
fr(l - 1) g(x,t) dr
1 1 + g(z,t) dx + ( pi(x - 10pc
g(r,t) dr. po t)
2
2
(o(
(o(
2 2 po(x,t)
po(æ,
Po(a,t)
Since K is bounded, is clear (0(2, t)l E po((,, t). Then by Lemma A.2 we have
Falx + 1)2 g(a,t) dx S e" / / 2xt e
(x11)
(L(H)S)) + (IV x
vt 1)
1
lr
(1/211) dr -
2 Jo
po(a,t) t)
e
for some universal constant 0. Here, have used < 1. The proof is complete.
Proof Lemma B.5. Let abuse notation and write (xx, (g (z) in Lemma B.4. Integration by parts gives (ng t) = ((-1),;;( + 1) g(1)pr(z - 1) + [_1 g (1)) - μ4) du. Since a + 0,g(x,t) t) and x + (x. t) are both odd mappings, follows their product is an even mapping, and Lemma D.1 gives
00 Oapola, t)
2
(dapi (x, Oxpolx, ()) dx
1o(
@epo(x,t)
4 o
po(x, Oz (z, dr
pil + pi 11 (((-1) +) -)) p())p((() [ g'(u)pi(x- -)) -))xx) dr to(
2
= =2g(1) lo pe(a 1)?
2 f0e pe(x-1)
df 1)) dr + 00 (c
dx -x-2
po(x,
Po(I,t)
Here, we have used / < and the reasoning similar to that employed in the proof of Lemma B.4. Since K' is bounded, we have
fı - 1) t)(
{ ('())po(s 1) d)d dr
5 c² fi(x - 1) (I,(((x 1 dz
p j po(
/
x
( -1
p o polz, po(z, t) dr
p
completing the proof.
D.2. Very high noise regime
This section contains the proofs of Lemmas B.9 and B. which are used show the remainder term in the Taylor expansion (34) is negligible.
Proof of Lemma B.9. Since f2(1) fo(m) for |a| < because for f2 E For it follows by Jensen's inequality
[No(2,)) (22 2 (S", Iulpp(a - ()ddd du) Likewise, we have p2(x, (20 x f pr(x - 11) du. Therefore,
Nola.))PV No (a.0) (op
S 1. Therefore, it follows from the power series expansion pa(x 11) ( ( (4)
(w) n that
[ (x, dr S
|2(x,, 1) - po(x, 1) do
00
14
ce 1)) di du
1
p
/ 1o! (x)) dx
1
4²²t dpp
2
k!
p
k=0
25

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
30
(M a² du / (x)) dz.
Here, have used that p 1, d is supported on |-1, 11, J v1v) du
0, and -_ d(v) since & is even.
Therefore, it follows from Lemma D.6
00
p++
100
(1 1) dr x
b(u)eut du
Heg
(c(x)
de
1k/2
Vi
-D0
00
p++
1/2
24
(V2T ks
12
k!
tk/2
Ccpp'
for some C > 0 depending only on c. Here, we have used < and f 2 c to obtain the final line.
Proof of Lemma B.10. The proof is very similar to the proof of Lemma B.9. From that proof, we have [ Using the power series expansion (z μ4) - (z)
< 1.
(x)
(z)
it follows
kl
L R2(0, 1) do x 12
1 N2, 1) - No 1)] dx
1
a++1 S (pt (z))
(x)) dx
P
1
[' M()uuatt del 100 Hec
vB (c(x)) dr
h!! th/2
put2
1/2
(V2TT k!
2
hd 1. 1k/2
p3
15/2
for some 0 depending only on c. As in the proof of Lemma B.9, we have used p < and 2 obtain the final
line.
Lemma D.6. Let (p(()== v2xt
_e-- e- 54
denote the probability density function of (0, Ifk 2 1 an integer, then
(-1)*
2 (x)
HeR
(x)),,
1k12
where Hey is the k-th (probabilist's s) Hermite polynomial.
Proof. Let p(x) == (x()) denote the density of the standard Gaussian distribution. Recall that He, is given by (x))(x) (-I)+He((z)y((z)) Therefore, (*) (x) (-
da
dt
10
Vt
Vi
1
(kk))())
ve
x
=
(-1)*
2
Her
v
Hes 1674
(x( as claimed.
E. Auxiliary results
This section contains some useful assertions as well as deferred proof's of technical results.
Proof of Lemma A.2. Since ca E f S Ca on its support, it follows
pxx,t) /' fi(x 11) f(v) P < 1)
26

---

1)² Plug-in Estimation of the Fisher Information of Smoothed Density
If [x]] < then clearly P (1N((, 13 x 1, and the claim is proved for this case. The second claim is Lemma 13 from (Dou et al., 2024).
Proof of Lemma A.3. Since a 2 1, follows f is differentiable on (-1,1) and [f'())) S C' for some universal constant C > O. Therefore, integration by parts yields
10,((, 1) - S", ('()) w) du
X ( 1)2 +) px(x - - + S" f'w)aa u) du
Suppose |o]| & 1. Then let us apply Jensen's inequality by treating pp(l - 11) as a density in p to obtain
Sr (())p((- - 14) dp
2
/ (1) 11) dp
$ 1f'(4) (1)s1))P -))
$ <e(x,t)
which yields the desired result in this case. Now suppose |x: > 1. Let apply Jensen's is inequality in different way by treating 1 (1uSs finite measure in j to obtain
(1) 12 s [" ['W S ('weeee- uu) du
S 1
[
1
(xx c
22
-
dp
x
E 1),
Vt
Vt
Vt
as claimed. The proof is complete.
Lemma E.1. With probability one, Oap (x. 1)) ( 1)2 + px(x 1)2 + g(x, 1) where dzP is given by (11).
Proof. The proof is immediate by the definition of O.p-
Lemma E.2. If p E (0, 1), then function g is even and So g(k) du 0.
Proof. Since K is symmetric (i.e. even), it immediately follows g(m) = g(-H) for all f E R. To show the next claim, note that direct calculation yields
((0) du = -
noc
K
11
1 11lls1 du du J K x(=11) 1 (1lls1 du
30
00
+
+ a
K
11lls1 du
du++ aa (' K (P-1) du
K
P
l+p
(u+1)
+ 0
K
dp
P
- c {" K(v) + t ( k(v) du + Cp l.f. K(v)
dlu)du+e'
0,
as claimed.
Note the statement Lemma a typo, proof yields the expression stated in Lemma A.2.
27

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
F. Proof of Theorem 4.1
: + VIZ with Z ~ NN00 1) drawn independently of μ. Let X, == %/- that is to say, we have just standardized for convenience to obtain X. = vill + Z. Note that the density of pa is f given by f(u)- == Vif(Vtu). It follows directly that tI(f * (pt) I(f (1) 1 - E (H ((()( where the latter equality is Brown's identity (Brown, 1971). Next, the I-MMSE identity (Guo et al., 2005) asserts d (X = = E(ul (x)))). Since we have the identity I p)) = ((): for all t > 0, it fol-
Proof of Theorem 4.1. Consider μ ~ f and X;
at1/ej
lows (X(:r) = (-#) 24,o (Xx: p4) = - E((e E(u (x))). Therefore, we have Is = I(f Pp) = (1 - (E((a-Ei ((I1))))))) = + (1 21 21 (Xii)) as claimed.
G. Notation
For a, E IR the notation a S denotes the existence of a universal constant c > 0 such that a S cb. The notation 2 is used to denote b S a. Additionally x b denotes E and a 2 b. The symbol := frequently used when defining a quantity or object. Furthermore, we frequently use a V == max((, b) and a A b = min(a, b). We generically use the notation LA to denote the indicator function for an event A. For two probability measures P and Q on - measurable space AA) the total variation distance is defined as drv Q) := sup AEA - Q(A)). The Kullback- Leibler divergence between P and Q is denoted as ( Q) f log (dP/dQ) dP if P is absolutely continuous with respect to Q and dkI, || Q) = oo otherwise. For a probability measure P with a density p. the entropy is denoted h(P)= = - f p(x) log p(z) dæ. We will frequently abuse notation and use the same symbols with densitics or random variables in place of measures. For random variables U Py and V Pv with joint distribution (U, V) ~ Puv, the mutual information is I(U; V)- dKL Pu @ Pv).
H. Related work on smoothed Fisher information and a finite-sample analysis of location estimation
The smoothed Fisher information has also been recently shown to play a critical role in saite-sample analysis of the fundamental statistical task of mean estimation (Gupta et al., 2022; 2023a;b). Gupta et al. (2023b) make the following, insightful observation. Consider the problem of estimating the mean 0 of a density f with variance o², given n i.i.d. samples X1...., Xn, Sample mean is an obvious choice of estimator, and by the central limit theorem, it has the asymptotic distribution vn((X - 0) = o²). However, in some cases of f. the mean 0 might also coincide with the location, and some location estimators might outperform the sample mean. For example, consider the Laplace distribution centered at 0 with variance 2, which has density f(xx) xx ele--01, Note 0 is both the mean and the location (which is also the median in this case). It turns out that the sample median, expressed as X(n/2) in order-statistic notation, beats X since Vn(X(n/2)> 0) = (O, 1) whereas vn(X - 0) = NO, 2). The asymptotic variance of sample median is half that of sample mean; in fact, sample median is the maximum likelihood estimator and is thus optimal for estimation of 0 in this example.
More generally, better asymptotic variance than 0² can be achieved in the location estimation problem. The location estimation problem is the problem of estimating the ground truth location parameter 8* in the parametric family {f(x F)RRR where f some known density. is classical (Vaart, 1998) that the maximum likelihood estimator is asymptotically normal, centered around 0*, and with variance given by the reciprocal of the Fisher information of f. This variance can be substantially smaller than o². However, f needs to be known up to translation. Gupta et al. (2023b) ask the intriguing question of whether it is possible, in the case of an unknown density ʃ that is symmetric about its mean and, whether it is also possible to attain a Fisher-in informmtioo like speedup in samiless.
Gupta et al. (2023b) construct an estimator å which, with probability at least 1 1-6 8 and with n 2 log (1/5) samples, achieves 0 - 01 < (1 + v information of f and asserts a speedup since I[f+p.] S a² + L. Their results build upon earlier work (Gupta et al., 2022: 2023b) which show that the smoothed Fisher information is fundamental quantity in a finite-sample analysis of the location
[I(]pp) for wxrlxx 1xx0² x² and = (log (8))/m)///4 Specificclly, their bound involves the smoothed Fisher
2 log(2/8)
estimation problem (where/ f is assumed known up to translation).
The work of Gupta et al. (2023b) gives a point estimator which enjoys faster convergence rates. The natural problem to
28

---

Plug-in Estimation of the Fisher Information of a Smoothed Density
consider next is hypothesis testing, equivalently, construction of confidence intervals. When f is not known, then the bound of ji is not computable since I(f (pr) is itself not known. Consequently, it is desirable to estimate the smoothed Fisher information Z(f * spt) to address these subsequent problems.
29


Reviews:
[{"review_id": "5tswCiys0I", "reviewer": "Reviewer_pEyQ", "summary": "The paper considers probability densities smoothed by Gaussian noise of variance $t$, and addresses the problem of estimating the Fisher information of the smoothed densities based on a collection of $n$ i.i.d. samples. The Fisher information can be expressed as an integral of the smoothed density and its derivative. The paper proposes a estimator, which is of the “plug-in” type, in the sense that first both the smoothed density and its derivative are estimated, and then plugged in into the Fisher information functional. The way in which the smoothed density and its derivative are proposed to be estimated depends on the variance of the smoothing Gaussian noise, and is different in three different regimes of $t$, and in general, based on properly truncated empirical PDF estimator, or on existing kernel-density estimators. The paper analyzes the expected error of these estimators. The paper also derive minimax lower bounds, which assert its rate-optimality, except in an intermediate regime for $t$ (not small enough, or not large enough), in which no lower bound is proved. The lower bounds are based on the two-point method (Le-Cam) with a proper choice of pair of densities – in the regime of low $t$, densities are sharp, and thus should be distinguishable at the edges of the support ($\\pm 1$). The bound is thus based on a pair of distributions different at these edges. In the regime of high $t$, the densities are very smooth, and thus should be distinguished at their center (around $0$). The bound is thus based on a pair of distributions different at the center. \n\nThen, using known identities (I-MMSE, de Bruijn), the estimator for the Fisher information is shown to be utilized to estimate the mutual information over the Gaussian channel (with the original density has the density of the input) and the output differential entropy. Both have $O(1/\\sqrt{n})$ rates. Finally, some of the proof ideas are highlighted – perturbation analysis of the function determines the Fisher information (Propositions 5.1 and 5.2).", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the constructive comments!\n\n__Weakness 1:__\nSuppose $f$ is an $\\alpha$-Holder density on $[-1, 1]^d$. Our result can be directly extended, and the rate is entirely expected:\n\n$$\\inf\\_{\\widehat{\\mathcal{I}}\\_t} \\sup_{f \\in \\mathcal{F}\\_\\alpha} E\\left(\\left|\\widehat{\\mathcal{I}}\\_t - \\mathcal{I}\\_t\\right|\\right) \\asymp \\frac{1}{\\sqrt{n}t^2} \\wedge \\frac{1}{\\sqrt{n} t^{(d+2)/4}} \\wedge \\frac{n^{-\\alpha/(2\\alpha+d)}}{\\sqrt{t}}.$$\n\nThis is natural from the well-known $d$-dimensional estimation rates of the plugged in density and gradient estimators.\n\nNotably, in the very high and high noise regimes, the convergence rate in terms of the sample size is the fast $\\frac{1}{\\sqrt{n}}$ rate; the curse of dimensionality is circumvented. A curse seems to appear in the low noise regime rate, but one needs to take care in the interpretation. Note the rate is the minimum of three terms, which means we *always* can achieve $\\frac{1}{\\sqrt{n}t^2}$. Therefore, one might say the curse never bites. But when $t$ is very small, one can beat $\\frac{1}{\\sqrt{n}t^2}$ and achieve the faster $\\frac{n^{-\\alpha/(2\\alpha+d)}}{\\sqrt{t}}$, which appears to suffer the curse. \n\nThe answer to this apparent conceptual puzzle is that one cannot avoid paying $\\frac{1}{t}$ raised to a power involving $d$ (in our case it is $t^{-\\frac{d+2}{4}}$), which becomes large as $t$ gets small. This phenomenon is not unique to Fisher information estimation, and has been noted earlier by Goldfeld et al. (2020) that it occurs also in smoothed entropy estimation. Since some claim in these results that the curse of dimensionality has been circumvented, some might also claim that the curse is avoided in our problem too.\n\nThough we have not checked every single detail to confirm the conjectured rate, the generalization of the proof to the multivariate case seems very standard and only involving tedious notation. One might be worried about generalizing our use of integration by parts, but a coordinate-wise argument works as the domain is $[-1, 1]^d$. Since we use plug-in  the argument is quite similar to the score estimation theory of Dou et al. (2024).\n\n__Weakness 2:__ Reviewer CeFj commented on empirics, and in our response to them we have described how the estimators can be computed. Please have a look there. Thanks!\n\n__Relation to broader scientific literature:__ Reviewer SWHe pointed to some related work; please see our response!\n\n__Other Suggestion 1:__ Thanks! In the revision, we will make a note that this quantity is also known as the Stein information. After reading your comment, we were interested in completely replacing all instances of \"Fisher information\" with \"Stein information\", but after consulting some senior colleagues (from the nonparametric statistics and information theory communities and who also keep up with ICML/Neurips/etc), we decided to stick with the term \"Fisher information\" as the paper is particularly relevant to those communities. Thanks again for your comment, and we will be sure to note that it is also known as the Stein information. \n\n__Other Suggestion 2:__ Continuing the calculation from the remark, we have $\\mathcal{I}\\_t = \\frac{1}{2}\\int\\_{-\\infty}^{\\infty} \\frac{(\\varphi_t(x+1) - \\varphi_t(x-1))^2}{P(|N(x, t)| \\leq t)} dx \\gtrsim \\int_{|x-1| \\leq \\sqrt{t}} \\varphi_t(x-1)^2dx$ where the last inequality follows from $P(|N(x, t)| \\leq 1) \\asymp 1$ for $|x| \\leq 1 + C\\sqrt{t}$ as $t < 1$. We have also used that $\\varphi_t(x+1) \\leq c \\varphi_t(x-1)$ for $|x-1| \\leq \\sqrt{t}$ where $c < 1$ is a small universal constant, since $t$ is small. Consider $\\int_{|x-1| \\leq \\sqrt{t}} \\varphi_t(x-1)^2 dx \\asymp \\frac{1}{\\sqrt{t}} \\int_{|x-1| \\leq \\sqrt{t}} \\frac{1}{\\sqrt{t}} e^{-\\frac{(x-1)^2}{t}}dx \\asymp \\frac{1}{\\sqrt{t}} P(|N(1, t) - 1| \\leq \\sqrt{t}) \\asymp \\frac{1}{\\sqrt{t}}$. Hence, $\\mathcal{I}_t \\gtrsim \\frac{1}{\\sqrt{t}}$. In the revision, we will elaborate to make this clearer to the reader. \n\n__Other Suggestion 3:__ Thanks!\n\n__Other Suggestion 4:__ Though we agree the intuition is natural, it is an classic result from statistics that the minimax rate for estimating the $r$th derivative of an $\\alpha$-Holder function in squared $L^2$ or squared pointwise error is $n^{-2(\\alpha-r)/(2\\alpha+1)}$ (we take $r = 1$ for our purposes). This result (along with results for other error metrics) is due to Charles Stone (namely, his papers *Optimal global rates of convergence for nonparametric regression*, The Annals of Statistics 10 (1982), no. 4, 1040-1053 and also *Optimal uniform rate of convergence for nonparametric estimators of a density function or its derivatives*, Recent Advances in Statistics, Elsevier, 1983, pp. 393-406). In our paper, we had only cited textbooks, but we will also cite these papers of Stone in the revision.\n\n__Other Suggestion 5:__ Thanks! Yes, $g(x, t) := (g*\\varphi_t)(x)$. We had defined it on page 17, but neglected to point it out at line 423. We will make a note of it in the revision.", "claims_and_evidence": "The paper claims that a simple plug-in achieves the minimax rate, with accurate dependency on the Gaussian smoothing variance $t$, then Holder-smoothness parameter of the density $\\alpha$ and the number of samples $n$. The analysis of the estimator and the minimax lower bound validate this claim, up to a short interval in $t$, in which the question is open. \n\nThat being said, the proposed estimator is not a vanilla plug-in estimator. Indeed, the Fisher information is expressed as a function of the smoothed density and its derivative, and those quantities are estimated and plugged into the functional. However, the Fisher information is represented by different functionals in various regimes for $t$ (especially in the very high noise regime), and the way that the smoothed density and its derivative are estimated is different in each regime (possibly truncated, as smoothed empirical density or via a kernel density estimator). From my perspective the strong theoretical results of the paper are definitely of interest, but the estimator is not very simple.", "methods_and_evaluation_criteria": "The evaluation method is standard and makes sense– minimax expected error rate for an estimator based on $n$ i.i.d. samples.", "theoretical_claims": "The theoretical claims are convincing, and I have verified the claims made in the paper, though the rigorous proofs are fully deferred to the appendix. From a quick overview of the appendix, and especially the techniques used, the proofs also appear to be convincing.", "experimental_designs_or_analyses": "Not applicable, the paper is purely theoretical.", "relation_to_broader_scientific_literature": "The problem of estimating statistical functionals is a classic problem, and the Fisher information is one of the central functionals in statistics. The paper addresses both classic works on the problem (without smoothing) and a recent line of work considering smoothed densities. \n\nIn a broader context, the motivation of the current work – Given that the Fisher information is discontinuous in $t$, and as the actual interest is in the Fisher information of the original density, why it is of interest to estimate it to begin with? The motivation questions also comes to mind given the fact that increasing $t$ from zero may actually make the problem more difficult. \n\nFinally, given the interest in diffusion models, there are many papers addressing the problem of score estimation. As Fisher information is the variance of the score, it would be interesting to relate the paper more closely to this research area.", "essential_references_not_discussed": "I am not aware of an essential reference missing.", "other_strengths_and_weaknesses": "Strengths:\n\n1) The paper is very well written with the main ideas and the merit of the results clearly explained. To the extent possible, the intuition of the proofs is explained. \n\n2) The result is a sharp (almost full) characterization of the minimax estimation rate in this problem. \n\n3) It is interesting that a plug-in estimator is optimal for $t>0$, as it is suboptimal in the unsmoothed case.\n\n4) The estimator of the Fisher information results an estimator for other functionals – mutual information and differential entropy of the smoothed density. \n\nWeaknesses:\n\n1) High dimensions: The paper addresses one-dimensional densities. One of the main motivations for Gaussian smoothing is to circumvent the curse of dimensionality. It is not discussed anywhere in the paper (and is unclear) if the approach can be directly extended to high dimensions (with the anticipated technicalities), or if it breaks down at high dimensions. \n\n2) The computational question is completely ignored, in the sense that it is not obvious how simple it is to compute the estimators – these are integrals of the estimated densities and the estimated derivatives of these densities over an infinite interval. \n\n3) The paper is not a perfect fit to ICML, as it is purely theoretical, without any actual machine-learning applications, or even connection to machine learning techniques (the “information bottleneck” motivation is rather generic. This explains my recommendation is only “accept”.", "other_comments_or_suggestions": "1) The name Fisher information is typically reserved for parametric families. Here it is somewhat hidden that the parametric family is the location. I think that in the ML community this quantity is referred to as Stein information. \n\n2) Remark 1.2: How does it follow that $I_t \\gtrsim 1/\\sqrt{t}$ ? \n\n3) In (6) and (7) there is a typo, I think that the integral should have $d\\mu$. \n\n4) If I understand correctly, the derivative of $\\alpha$ Holder function will be $(\\alpha-1)$ Holder. So in Section 2.2, for the estimate of the derivative, shouldn't the rate change to $n^{2(\\alpha-1)/(2\\alpha-1)}$ (replacing $\\alpha$ with $\\alpha-1$ in the preceding bound)?\n\n5) In line 423, how $g(x,t)$ is defined? As a smoothed version of $g? A short explanation would clarify this.", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ubqCpJO9wf", "reviewer": "Reviewer_arrG", "summary": "This paper studies estimation of the Fisher information $\\mathcal{I}(f * \\psi_t)$ of a smoothed density $\\psi_t$, where $\\psi_t$ is the Gaussian kernel of bandwidth $t$, given IID samples from a density $f$. Plug-in estimators are proposed, based on appropriately truncated and smoothed estimates of $f * \\psi_t$ and its spatial derivative. The paper first presents upper bounds for these estimators, distinguishing between three regimes of the noise magnitude $t$. The paper then presents matching lower bounds for most cases. Finally, using information theoretic identities relating the Fisher information to mutual information and entropy in certain cases, the paper presents and bounds the error of estimators for those latter quantities.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the great feedback!\n\n__Questions for Authors 1:__ Reviewer pEyQ asked the same question; please see our response. Thanks!\n\n__Questions for Authors 2:__ Thank you for the question, and we agree this point could have, and should have, been made clearer in the paper.\n\nAs you point out, we estimate the integral $\\int\\_{0}^{t} \\mathcal{I}\\_s ds$ by the plug-in $\\int\\_{0}^{t} \\widehat{\\mathcal{I}}\\_s ds$. Since your question specifically asks about the low-noise regime, let us focus our discussion by examining $t \\lesssim n^{-\\frac{2}{2\\alpha+1}}$. Then for all $s \\leq t$, we have from Theorem 2.2 that $E\\left(\\left|\\widehat{\\mathcal{I}}_s - \\mathcal{I}_s\\right|\\right) \\lesssim n^{-\\frac{\\alpha-1}{2\\alpha+1}} + \\frac{n^{-\\frac{\\alpha}{2\\alpha+1}}}{\\sqrt{s}}$. Since $s \\lesssim n^{-\\frac{2}{2\\alpha+1}}$, it follows that \n\n$$\nn^{-\\frac{\\alpha-1}{2\\alpha+1}} + \\frac{n^{-\\frac{\\alpha}{2\\alpha+1}}}{\\sqrt{s}} \\asymp n^{-\\frac{\\alpha}{2\\alpha+1}} \\cdot n^{\\frac{1}{2\\alpha+1}} + \\frac{n^{-\\frac{\\alpha}{2\\alpha+1}}}{\\sqrt{s}} \\lesssim n^{-\\frac{\\alpha}{2\\alpha+1}} \\cdot s^{-1/2} +  \\frac{n^{-\\frac{\\alpha}{2\\alpha+1}}}{\\sqrt{s}} \\asymp \\frac{n^{-\\frac{\\alpha}{2\\alpha+1}}}{\\sqrt{s}}. \n$$\n\nTherefore, we have shown $E\\left(\\left|\\widehat{\\mathcal{I}}\\_s - \\mathcal{I}\\_s\\right|\\right) \\lesssim \\frac{n^{-\\frac{\\alpha}{2\\alpha+1}}}{\\sqrt{s}}$. We can now bound the estimation error. Consider, \n\n$$\nE\\left(\\left|\\int\\_{0}^{t} \\widehat{\\mathcal{I}}\\_s ds - \\int\\_{0}^{t} \\mathcal{I}\\_s ds \\right|\\right) \\leq \\int\\_{0}^{t} E\\left(\\left|\\widehat{\\mathcal{I}}\\_s - \\mathcal{I}\\_s\\right|\\right) ds \\\\\\\\\n            \\lesssim \\int\\_{0}^{t} \\frac{n^{-\\frac{\\alpha}{2\\alpha+1}}}{\\sqrt{s}}ds \\\\\\\\\n            =\\left.\\left(n^{-\\frac{\\alpha}{2\\alpha+1}}\\right) \\cdot 2\\sqrt{s}\\right|\\_{s = 0}^{t} \\\\\\\\\n            \\asymp \\sqrt{t} \\cdot n^{-\\frac{\\alpha}{2\\alpha+1}}.\n$$\nSince $t \\lesssim n^{-\\frac{2}{2\\alpha+1}}$, we have \n$$\n\\sqrt{t} \\cdot n^{-\\frac{\\alpha}{2\\alpha+1}} \\lesssim n^{-\\frac{1}{2\\alpha+1}} \\cdot n^{-\\frac{\\alpha}{2\\alpha+1}} \\asymp n^{-\\frac{\\alpha+1}{2\\alpha+1}}. \n$$\n\nSince $\\frac{\\alpha+1}{2\\alpha+1} \\geq \\frac{1}{2}$, it follows $n^{-\\frac{\\alpha+1}{2\\alpha+1}} \\lesssim \\frac{1}{\\sqrt{n}}$, and so we have obtained the parametric rate.  Thanks again for your helpful question as these clarifications will improve the paper.\n\n__Other suggestions 1:__ Thanks!\n\n__Weakness 1:__ This is a great comment and well taken. As you point out, it suffices to know $c_d, C_d, L$ just up to order without affecting rates. It is well-known \\(f\\) can be estimated in sup-norm with a KDE $\\hat{f}$ at rate $||\\hat{f} - f||_\\infty \\lesssim \\left(\\frac{N}{\\log N}\\right)^{-\\frac{\\alpha}{2\\alpha+1}}$ with high probability using $N$ samples. This is actually much faster than we need since we only need the order of the unknown constants. For example, set $\\hat{C}\\_d := \\max\\_{|\\mu| \\leq 1} \\hat{f}(\\mu)$ and $\\hat{c}\\_d := \\min\\_{|\\mu| \\leq 1} \\hat{f}(\\mu)$, and note we have $\\hat{C}\\_d \\asymp C\\_d$ and $\\hat{c}\\_d \\asymp c\\_d$ with high probability, even if we only use a constant number of samples to fit $\\hat{f}$. Similarly, $L$ can be estimated using a kernel density estimator $\\hat{f}'$ of the derivative. Therefore, we can just siphon off a constant number of data points, estimate these constants up to order, and not affect the convergence rate.\n\nThe question about unknown $\\alpha$ is more delicate. Estimation of $\\alpha$ itself is quite complicated, but perhaps the goal is instead  the construction of an adaptive estimator of $\\mathcal{I}(f*\\varphi_t)$ (i.e. does not require knowledge of $\\alpha$) yet still achieves the minimax rate as if it were known. \n\nWe believe it may be impossible to modify our methodology to be adaptive. The issue is we make use of a density point estimator in the low noise regime. Namely, we use an $\\hat{f}$ with $E(|\\hat{f}(\\mu) - f(\\mu)|^2) \\lesssim n^{-\\frac{2\\alpha}{2\\alpha+1}}$ for all $|\\mu| \\leq 1$. Let us fix a $\\mu^* \\in (-1, 1)$.\n\nIt is a well known result due to Lepski (O. V. Lepskii. *On a problem of adaptive estimation in Gaussian white noise.* Theory of Probability \\& Its Applications, 35(3):454-466, 1991) that $\\hat{f}$ cannot achieve the minimax rate of estimating the density at $\\mu^*$ over the class $\\mathcal{F}\\_{\\alpha_1}$ and simultaneously over $\\mathcal{F}\\_{\\alpha_2}$. In particular, it can be shown that for *any* estimator $\\hat{f}$, if $\\sup\\_{f \\in \\mathcal{H}\\_{\\alpha_1}} E(|\\hat{f}(\\mu^*) - f(\\mu^*)|^2) \\lesssim n^{-\\frac{2\\alpha_1}{2\\alpha_1+1}}$, then we actually have \n\\begin{equation*}\n       \\limsup_{n \\to \\infty} \\sup_{f \\in \\mathcal{H}_{\\alpha_2}} n^{\\frac{2\\alpha_2}{2\\alpha_2 + 1}} E(|\\hat{f}(\\mu^*) - f(\\mu^*)|^2) = \\infty.\n\\end{equation*}\nTherefore, our strategy seems doomed. It is very interesting to ask if some other approach can work. Thanks very much for pointing it out.\n\n__Weaknesses 2 and 3:__ Reviewer CeFj asked a question about empirics. Please see our response there. Thanks!", "claims_and_evidence": "As noted below, I am confused about why Corollary 4.4 holds.", "methods_and_evaluation_criteria": "The paper does not include any experiments.", "theoretical_claims": "I did not read the proofs in the supplement, although the high-level descriptions in the main paper generally made sense to me.", "experimental_designs_or_analyses": "The paper does not include any experiments.", "relation_to_broader_scientific_literature": "The paper lies in the intersection of classical work on nonparametric estimation of functionals of smooth probability densities and more recent work on estimating densities after Gaussian smoothing.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "There are two main limitations of the proposed estimators that seem make them unusable in practice:\n1) The construction of the estimators assumes knowledge of the constants $c_d$, $C_d$, $L$, and $\\alpha$. However, these are rarely known in practice. Is it possible to relax this assumption (e.g., by using some surrogates for $c_d$ or $C_d$ in terms of other known quantities) in a way that does not affect the convergence rates? Or is there at least some practical way to get, e.g., the right order of magnitude for these quantities?\n2) The mutual information estimator used in Theorem 4.2 requires integrating the Fisher information estimator $\\widehat{\\mathcal{I}}_s$ over $s$ from $t$ to $\\infty$. Can this integral actually by computed? It seems unlikely to me, given the complex dependence of the estimator on $s$, but there might be some tricks that make this possible. Alternatively, is there a computable approximation that can be shown to converge at the claimed rate? Are any additional assumptions needed to show this?\n3) Related to the above points, the paper would be made stronger if it demonstrated that the proposed estimators could actually be computed and used in an real-world, or at least simulated, problem.", "other_comments_or_suggestions": "1) Typo: Eq. (2): \"for all $x, y \\in (-1, 1)$\" should be \"$\\mu, \\mu' \\in (-1, 1)$\"", "questions_for_authors": "1) The paper only seems to discuss the 1-dimensional case of a density on $\\mathbb{R}$; how do results and analysis change for a multi-variate density on $\\mathbb{R}^d$?\n2) I am confused about why Corollary 4.4 holds. The integral $\\int_0^t \\mathcal{I}_s ds$ involves estimating the Fisher information in the low-noise regime $s \\leq n^{-\\frac{2}{2\\alpha+1}}$, where Theorem 2.2 gives only a nonparametric convergence rate of order $n^{-\\frac{\\alpha-1}{2\\alpha+1}} + n^{-\\frac{\\alpha}{2\\alpha+1}}/\\sqrt{t}$. From this, how can we get the parametric rate $1/\\sqrt{n}$? This point has to be clarified for me to accept the paper.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IYwG8kSfrE", "reviewer": "Reviewer_CeFJ", "summary": "This paper studies the problem of estimating the Fisher information of smoothed probability densities falling in the $\\alpha$-Holder smooth class. The authors derive minimax rate bounds for the plug-in estimator, showing that a simple plug-in estimator is optimal for smoothed probability densities. The convergence results are further extended for mutual information and entropy estimation.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the thoughtful report!\n\n__Other suggestions 1:__ Reviewers pEyQ and arrG also asked about computation, which is related to your comment. Let us first describe how computation of the estimators $\\widehat{\\mathcal{I}}\\_t$ can be done, since it involves an integral over an infinite interval.\n\nWe can approximate the integral and still obtain the claimed statistical rate. The idea is simple, we simply truncate the integral to a large enough, bounded interval (whose length is growing in $n$). The error from ignoring the complement of the interval turns out to be negligible. For illustration, let us just discuss the high and low noise regimes, where we plug-in estimators of the density and its derivative. A similar argument will hold for the very high noise regime. Let $R > 0$ be a hyperparameter to be tuned later. We will approximate by\n$$\n\\widehat{\\mathcal{I}}\\_{t, R} := \\int_{-R}^{R} \\frac{\\widehat{\\partial_x p}^\\varepsilon(x, t)^2}{\\hat{p}^\\varepsilon(x, t)} dx. \n$$\nWe can approximate this integral by Monte Carlo, \n$$\n\\widehat{\\mathcal{I}}\\_{t, R}^{M} := \\frac{2R}{M}\\sum_{i=1}^{M} \\frac{\\widehat{\\partial_x p}^\\varepsilon(X_i, t)^2}{\\hat{p}^\\varepsilon(X_i, t)}\n$$\nwhere $\\{X\\_i\\}\\_{i=1}^{M}$ are $M$ i.i.d. points drawn uniformly in $[-R, R]$. The estimation error can be bounded as \n$$\nE\\left(\\left|\\widehat{\\mathcal{I}}\\_{t, R}^{M} - \\mathcal{I}\\_t\\right|\\right) \\leq E\\left(\\left|\\widehat{\\mathcal{I}}\\_{t, R}^{M} - \\widehat{\\mathcal{I}}\\_{t, R}\\right|\\right) + E\\left(\\left|\\widehat{\\mathcal{I}}\\_{t, R} - \\widehat{\\mathcal{I}}\\_{t}\\right|\\right) + E\\left(\\left|\\widehat{\\mathcal{I}}\\_{t} - \\mathcal{I}\\_{t}\\right|\\right).\n$$\nThe last term is exactly the statistical rate we want. The first term can be made to be of smaller order by taking \\(M\\) sufficiently large. It remains to argue about the second term. To do so, consider that for $|x| > 1$, we have from calculations similar to those employed frequently in the paper (e.g. using Lemmas A.2 and A.3)\n$$\n\\frac{\\left|\\widehat{\\partial_xp}^{\\varepsilon}(x, t)\\right|^2}{\\hat{p}^\\varepsilon(x, t)} \\leq \\frac{\\overline{\\varepsilon}'(x, t)^2}{\\underline{\\varepsilon}(x, t)} \\lesssim \\frac{\\varphi_t(x-1)^2 + \\varphi_t(x+1)^2 + \\underline{\\varepsilon}(x, t) \\frac{e^{-\\frac{(|x|-1)^2}{2t}}}{\\sqrt{t}}}{\\underline{\\varepsilon}(x, t)} \\lesssim \\left(1 \\vee \\frac{|x|-1}{\\sqrt{t}}\\right) \\cdot \\frac{1}{t} e^{-\\frac{(|x|-1)^2}{2t}} + \\frac{e^{-\\frac{(|x|-1)^2}{2t}}}{\\sqrt{t}}.\n$$\nTherefore, if we pick $R \\geq 1 + \\sqrt{Ct\\log(nt)}$ for a large universal constant $C$, we have \n$$\n\\left|\\widehat{\\mathcal{I}}\\_{t, R} - \\widehat{\\mathcal{I}}\\_t\\right| \\lesssim \\int_{|x| > R} \\left(1 \\vee \\frac{|x|-1}{\\sqrt{t}}\\right) \\cdot \\frac{1}{t} e^{-\\frac{(|x|-1)^2}{2t}} + \\frac{e^{-\\frac{(|x|-1)^2}{2t}}}{\\sqrt{t}} dx \\leq \\frac{1}{(nt)^{\\tilde{C}}}\n$$\nfor some universal constant $\\tilde{C}$ which can be taken to be sufficiently large by taking $C$ sufficiently large. Therefore, the error incurred by approximating $\\widehat{\\mathcal{I}}\\_t$ by $\\widehat{\\mathcal{I}}\\_{t, R}$ is dominated by the desired statistical rate.\n\nThe estimators in Section 4 (e.g. Theorem 4.2) can be estimated by the same Monte Carlo strategy. It can also be shown that the error of the complement can be made negligible. \n\nWe did a numerical experiment computing our Fisher information estimators on data sampled from the uniform distribution on $[-1, 1]$. Unfortunately, it is not clear to us how to present that figure in our response here, as it seems there is no capability of attaching images in author responses.\n\n__Question for authors 1:__ Thanks for the great question! We imagine perhaps you are thinking about convolving with a Cauchy density instead of a Gaussian density. We imagine it was perhaps meant to be written as $\\phi_t(x) \\propto \\frac{1}{\\left(x/\\sqrt{t}\\right)^2 + 1}$. It's a nice question as Cauchy has no moments.\n\nFrom a methodological point of view, we feel that the same plug-in strategy can be straightforwardly extended to the Cauchy case. For example, in the high-noise regime it is plausible to truncate $\\frac{1}{n}\\sum_{i=1}^{n} \\phi_t(x-\\mu_i)$ and $\\frac{1}{n}\\sum_{i=1}^{n} \\phi_t'(x-\\mu_i)$ for use as estimators of $p(x, t)$ and $\\partial_x p(x, t)$ respectively (and similar extensions for the other two regimes). To us, the heavier tail behavior does not appear to cause major obstacles. In fact, most of our intuition treats $\\phi_t$ as any kernel in a kernel-density estimator, in which case the precise form (Gaussian or Cauchy) doesn't really seem to matter. From the side of rigorous, mathematical analysis, it would seem the arguments would need to be modified to handle the Cauchy case. Our arguments frequently make use of the exponential tail of the Gaussian density to argue various remainder terms can be neglected. It is not clear to us whether serious changes to the broad proof strategy would be needed, or whether just careful, albeit tedious, technical modifications would suffice.", "claims_and_evidence": "The main claim of the paper: The plug-in estimator is optimal for smoothed probability densities, is well-supported by thorough theoretical analysis.", "methods_and_evaluation_criteria": "NA", "theoretical_claims": "I went through the proofs quickly, and they look good to me. I have not checked each detail in the proofs.", "experimental_designs_or_analyses": "NA", "relation_to_broader_scientific_literature": "The main result is of significant importance for designing efficient approximations for information quantities including Fisher information, mutual information and entropy. It complements previous results in estimators for unsmoothed probability densities.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strength:\nThe presented results are new as far as I know. The main claim is of significant importance to the community to design efficient algorithms for the estimation of information quantities.\n\nWeakness:\nThere are some restrictions on applicable probability density, e.g. only bounded densities are considered. There is also a gap for $c < t < C$.", "other_comments_or_suggestions": "It would be more convincing if the authors could provide some empirical results to verify their main claim, e.g. showing how the estimation precision changes with $t$ and $\\alpha$.", "questions_for_authors": "What will happen if the smoothing kernel is not Gaussian? e.g. $\\phi_t(x) \\propto \\frac{1}{t(x^2 + 1)}$? Do the main results still hold for such kinds of smoothing?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UIEa5adFjt", "reviewer": "Reviewer_SWHe", "summary": "This paper analyzes the minimax rate for estimation of the Fisher Information of a 1-dimensional Gaussian-smoothed density that satisfy an alpha-Holder condition from samples. It shows that variants of the simple plug-in estimator achieves the minimax rate, which varies depending on the amount of Gaussian smoothing, and proves matching lower bounds. It also shows the implications of this result on estimation of mutual information and entropy.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful review!\n\n__Essential references not discussed:__\nThanks for pointing these very relevant papers out! We plan to add the following text (perhaps with some modification to obey space constraints) to the revised manuscript.\n\n\"The smoothed Fisher information has also been recently shown to play a critical role in a finite-sample analysis of the fundamental statistical task of mean estimation (Gupta et al., 2022; Gupta et al., 2023; Gupta et al., 2023). Gupta et al. (2023) make the following, insightful observation. Consider the problem of estimating the mean $\\theta$ of a density $f$ with variance $\\sigma^2$, given i.i.d. samples $X_1,...,X_n$. Sample mean is an obvious choice of estimator. However, in some cases of $f$, the mean $\\theta$ might also coincide with the *location*, and some location estimators might outperform the sample mean. For example, consider the Laplace distribution centered at $\\theta$ with variance $2$, which has density $f(x) \\propto e^{-|x-\\theta|}$. Note $\\theta$ is both the mean and the location (which is also the median in this case). It turns out that the sample median, expressed as $X_{(n/2)}$ in order-statistic notation, beats $\\bar{X}$ since $\\sqrt{n}(X_{(n/2)} - \\theta) \\implies N(0, 1)$ whereas $\\sqrt{n}(\\bar{X} - \\theta) \\implies N(0, 2)$. The asymptotic variance of sample median is half that of sample mean; in fact, sample median is the maximum likelihood estimator and is thus optimal for estimation of $\\theta$ in this example.\n\nMore generally, better asymptotic variance than $\\sigma^2$ can be achieved in the location estimation problem. The location estimation problem is the problem of estimating the ground truth location parameter $\\theta^*$ in the parametric family $\\{f(x-\\theta)\\}_{\\theta \\in \\mathbb{R}}$, where $f$ is some known density. It is classical that the maximum likelihood estimator is asymptotically normal, centered around $\\theta^*$, and with variance given by the reciprocal of the Fisher information of $f$. This variance can be substantially smaller than $\\sigma^2$. Gupta et al. (2023) ask the intriguing question of whether it is possible, in the case of an *unknown* density $f$ that is symmetric about its mean and, whether it is also possible to attain a Fisher-information like speedup in *finite-samples*.\n\nGupta et al. (2023) construct an estimator $\\hat{\\theta}$ which, with probability at least $1-\\delta$ and with $n \\gtrsim \\log\\left(1/\\delta\\right)$ samples, achieves $|\\hat{\\theta} - \\theta| \\leq (1+\\eta)\\sqrt{\\frac{2\\log(2/\\delta)}{n \\mathcal{I}(f*\\varphi\\_t)}}$ for $t \\asymp \\sigma^2$ and $\\eta = (\\log(1/\\delta)/n)^{1/13}$. Specifically, their bound involves the smoothed Fisher information of $f$ and asserts a speedup since $\\frac{1}{\\mathcal{I}(f*\\varphi\\_t)} \\leq \\sigma^2 + t$. Their results build upon earlier work (Gupta et al., 2022; Gupta et al., 2023) which show that the smoothed Fisher information is a fundamental quantity in location estimation.\n\nThe work of Gupta et al. (2023) gives a point estimator which enjoys faster convergence rates. The natural problem to consider next is hypothesis testing, or equivalently, construction of confidence intervals. When $f$ is not known, then the error bound of $\\hat{\\mu}$ is not computable since $\\mathcal{I}(f*\\varphi\\_t)$ is itself not known. Consequently, it is desirable to estimate the smoothed Fisher information $\\mathcal{I}(f*\\varphi\\_t)$ to address these subsequent problems.\"\n\n__Questions for authors:__ \n\nThere is some quantitative intuition which can be given. We can look to the estimation rates of the plugged-in targets. Estimation of the derivative is the harder problem, and is thus the rate-dominating step. \n\nIn the high noise regime, derivative estimation is done by truncating $\\partial_x \\hat{p}(x, t) = \\frac{1}{n} \\sum_{i=1}^{n} \\varphi_t'(x-\\mu_i)$, and the truncation only improves the estimation error. This is exactly a kernel density estimator for the derivative with kernel $\\varphi_t'$ with bandwidth $\\sqrt{t}$. However, observe our target is not $f'$ but actually $\\partial_x p = f*\\varphi_t'$, so no bias is incurred. Hence, the error is given by the variance, which is well known to be $\\frac{1}{n\\sqrt{t}^3}$ in squared loss, which yields $\\frac{1}{\\sqrt{n}t^{3/4}}$ in absolute loss.\n\nIn the low noise regime, recall we express via integration by parts $\\partial_x p(x, t) = (f*\\varphi_t)'(x) = f(-1)\\varphi_t(x+1) - f(1)\\varphi_t(x-1) + \\int_{-1}^{1} f'(\\mu) \\varphi_t(x-\\mu) d\\mu$. Ignoring the truncation, we estimate by plugging in estimators for the unknown quantities. In Theorem 2.2, the term $n^{-\\frac{\\alpha-1}{2\\alpha+1}}$ comes from plugging in $\\hat{f}'$. The term $\\frac{n^{-\\frac{\\alpha}{2\\alpha+1}}}{\\sqrt{t}}$ comes from plugging in $\\hat{f}(1)$ and $\\hat{f}(-1)$. The factor $\\frac{1}{\\sqrt{t}}$ comes from $\\varphi_t(x+1) \\asymp \\frac{1}{\\sqrt{t}}$ for $|x+1| \\lesssim \\sqrt{t}$ (and likewise with $\\varphi_t(x-1)$ for $|x-1| \\lesssim \\sqrt{t}$).", "claims_and_evidence": "Claims are supported by rigorous proof", "methods_and_evaluation_criteria": "No empirical results", "theoretical_claims": "Yes, they seem to be correct -- I checked all of them.", "experimental_designs_or_analyses": "No experiments.", "relation_to_broader_scientific_literature": "Several works have looked at estimation of the mean and location of Gaussian smoothed densities with error rate depending on the Fisher Information, which relates directly to the results in this paper (see 1, 2, 3 below). There have been many works looking at density estimation of smoothed densities (for example Goldfeld et al 2020), which are mentioned. There are also previous works on Fisher Information estimation, which are also mentioned.\n\n1) Finite-Sample Symmetric Mean Estimation with Fisher Information Rate. Shivam Gupta, Jasper C.H. Lee, and Eric Price. COLT 2023\n2) High-Dimensional Location Estimation via Norm Concentration for Subgamma Vectors. Shivam Gupta, Jasper C.H. Lee, and Eric Price. ICML 2023\n3) Finite-Sample Maximum Likelihood Estimation of Location. Shivam Gupta, Jasper C.H. Lee, Eric Price, and Paul Valiant. NeurIPS 2022", "essential_references_not_discussed": "The recent works below analyze mean and location estimation of Gaussian-smoothed densities, which is directly related to the present paper, and so, should be discussed.\n\n1) Finite-Sample Symmetric Mean Estimation with Fisher Information Rate. Shivam Gupta, Jasper C.H. Lee, and Eric Price. COLT 2023\n2) High-Dimensional Location Estimation via Norm Concentration for Subgamma Vectors. Shivam Gupta, Jasper C.H. Lee, and Eric Price. ICML 2023\n3) Finite-Sample Maximum Likelihood Estimation of Location. Shivam Gupta, Jasper C.H. Lee, Eric Price, and Paul Valiant. NeurIPS 2022", "other_strengths_and_weaknesses": "It would be nice to include more quantitative intuition for the rates obtained -- currently the lower and upper bounds are explained well qualitatively, but a clear quantitative explanation is lacking.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "- Is there any clear and concise quantitative explanation that you can provide for the rates obtained?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper studies the nonparametric functional estimation problem for the Fisher information of $f * N(0, t)$ based on iid observations from $f$, where the density $f$ is assumed to be Holder-smooth of order at least 1 and bounded from both above and below. The minimax rate of estimation is worked out for this problem, which exhibits three regimes. Surprisingly, the simple plug-in approach could attain this minimax rate in all regimes. 

All reviewers are unanimously positive about this paper and appreciate the technical significance, which I concur. Small concerns are also proposed, such as the requirement that the Holder smoothness parameter is at least 1, and the knowledge of the density upper and lower bounds in the estimator. Overall I am happy to recommend acceptance. 

A minor comment: technically speaking Remark 1.2 is incorrect, as the Fisher info for the uniform distribution should be infinite, because the integral should be taken over the entire real line, not just $[-1, 1]$. Alternatively, one see that the Fisher information of a density is the Fisher information in the corresponding location model, while it is well-known that the uniform location model is not differentiable in quadratic mean because of the change on the support. The current Remark 1.2 also violates the data-processing inequality of the Fisher information.
Decision: Accept (poster)