arXiv:1708.03751v1 [] 12 Aug 2017

On ε-Admissibility
in High Dimension and Nonparametrics
Keisuke Yano 1 and Fumiyasu Komaki 1,2
1

Department of Mathematical Informatics, Graduate School of Information Science and
Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan

e-mail: yano@mist.i.u-tokyo.ac.jp; komaki@mist.i.u-tokyo.ac.jp
2

RIKEN Brain Science Institute, 2-1 Hirosawa, Wako City, Saitama 351-0198, Japan
Abstract: In this paper, we discuss the use of ε-admissibility for estimation in high-dimensional and nonparametric statistical models. The
minimax rate of convergence is widely used to compare the performance
of estimators in high-dimensional and nonparametric models. However,
it often works poorly as a criterion of comparison. In such cases, the addition of comparison by ε-admissibility provides a better outcome. We
demonstrate the usefulness of ε-admissibility through high-dimensional
Poisson model and Gaussian infinite sequence model, and present noble
results.
MSC 2010 subject classifications: 62C15, 62C20, 62G05.
Keywords and phrases: Asymptotics, Bayes risk, Decision theory,
Gaussian sequence model, Poisson model.

1. Introduction
Consider a statistical decision problem in which X is a sample space, Θ is
a parameter space, and P is a statistical model {Pθ : θ ∈ Θ} such that for
each θ ∈ Θ, Pθ is a probability measure on X . Let A be an action space and
D a decision space, comprising the whole set of measurable functions from
X to A. Let L be a loss function Θ ×R A → R ∪ {+∞} and R a corresponding
risk function defined by R(θ, δ) = L(θ, δ(x))dPθ (x) for every θ ∈ Θ and
every δ ∈ D.
Our focus is on the use of ε-admissibility. For ε > 0, ε-admissibility is
defined as follows: an estimator δ is ε-admissible if and only if there exists
no estimator δ̃ such that for every θ ∈ Θ, R(θ, δ̃) < R(θ, δ) − ε. In other
words, δ is ε-admissible if for any other estimator δ̃, δ is not inferior to δ̃ at
some θ ∈ Θ when ε is subtracted from the risk of δ. For δ ∈ D, the infimum
1

K. Yano and F. Komaki/On ε-Admissibility

2

of possible values of ε such that δ is ε-admissible is denoted by R(Θ, δ):
R(Θ, δ) := sup inf [R(θ, δ) − R(θ, δ̃)].
δ̃∈D θ∈Θ

If R(θ, δ) = ε > 0, then δ is ε-admissible, and if δ is ε-admissible, then
R(Θ, δ) ≤ ε. A smaller value of R(Θ, δ) is preferable. For further details, see Blackwell and Girshick (1954), Farrell (1968), Ferguson (1967),
Hartigan (1983), and Heath and Sudderth (1978). Although the concept of
ε-admissibility was once widely studied in statistical decision theory, as a
research topic, it has long been abandoned.
In the present paper, we emphasize the use of ε-admissibility as a criterion for comparing the estimators in high-dimensional and nonparametric statistical models. We show, through two important examples, that by
adding a comparison using the value of R(Θ, δ) to that using the minimax rate of convergence, the performance of estimators in high-dimensional
and nonparametric statistical models can be more successfully compared. In
high-dimensional and nonparametric models, the minimax rate of convergence in an asymptotics has been used to measure the performance of an
estimator. For example, the minimax rate of convergence of an estimator δ
in the asymptotics in which the dimension d of a parameter space Θ grows
to infinity is defined by dα , where α is the minimum number that satisfies
0 < limd→∞ inf δ∈D supθ∈Θ R(θ, δ)/dα < ∞. When using the minimax approach, the key criterion is whether or not the rate of convergence of the
estimator matches the minimax rate of convergence (Tsybakov, 2009 and
Wasserman, 2006). However, the minimax rate of convergence often fails to
clearly distinguish between estimators. In such cases, adding a comparison
using ε-admissibility can be helpful. In the present study, we investigated
the use of ε-admissibility by application to two examples: estimation of the
mean in a high-dimensional Poisson model and estimation of the mean in a
Gaussian infinite sequence model.
We first show, using estimation of the mean in the high-dimensional Poisson model, that ε-admissibility preserves the dominating result in a finite
dimensional setting, in contrast with the minimax approach. Consider estimation of the mean in a d-dimensional Poisson model with an L1 -constraint
parameter space. This estimation appears in discretization of an inhomogeneous Poisson point process model (see Appendix A). In a setting in
which d > 2 is fixed, it is known that the James–Stein type estimator θ̂JS
dominates the Bayes estimator θ̂J based on Jeffreys’ prior when using the
divergence loss; see Komaki (2004), Komaki (2006), and Komaki (2015)
and see also Ghosh and Yang (1988). Here, δ is said to dominate δ̃ if and

K. Yano and F. Komaki/On ε-Admissibility

3

only if R(θ, δ) ≤ R(θ, δ̃) for all θ ∈ Θ and there exists θ0 ∈ Θ such that
R(θ0 , δ) < R(θ0 , δ̃). Unfortunately, the minimax rate of convergence can
not determine whether θ̂JS is superior to θ̂J because the rates of convergence of both θ̂J and a minimax estimator are d. In contrast, by applying
ε-admissibility, we can decide that θ̂J is better than θ̂JS even in the asymptotic sense, because a simple calculation introduced in Section 3 will show
that limd→∞ R(Θ, θ̂J )/d > 0 and limd→∞ R(Θ, θ̂JS ) < 2.
We further show, through estimation of the mean in a Gaussian infinite
sequence model, that ε-admissibility can quantify the degree of preference
of one asymptotically minimax estimator over another. Consider the estimation of the mean in a Gaussian infinite sequence model with a Sobolev-type
constraint parameter space. This model is a canonical model in nonparametric statistics, and has been shown to be statistically equivalent to the
nonparametric regression model (Tsybakov, 2009, pp. 65–69). In this context, Zhao (2000) demonstrated that any Gaussian prior of which the Bayes
estimator is asymptotically minimax places no mass on the parameter space.
Zhao also constructed a prior of which the Bayes estimator is asymptotically
minimax and the mass on the parameter space is strictly positive. See also
Shen and Wasserman (2001). However, the goodness due to the strictly positive mass on the parameter space has yet been quantified. We show that
a modification of the prior discussed in Zhao (2000) yields an asymptotically minimax Bayes estimator that, from the viewpoint of ε-admissibility,
is superior to one based on the Gaussian prior. This is discussed in Section
4.
Finally, we address the relationship to admissibility. Any Bayes estimator
based on a prior on Θ is admissible and thus ε-admissible for any ε > 0,
so that ε-admissibility can not be used to compare such Bayes estimators.
However, in practice, estimators based on a prior that puts full mass on Θ
are rarely used, as there exist few settings in which the full information on
Θ is known in advance. The estimators discussed in Sections 3 and 4 do not
depend on knowing the full structure of Θ.
The rest of the paper is organized as follows. In Section 2, we introduce
the properties of ε-admissibility and discuss its relationship with a related
concept introduced by Chatterjee (2014), known as C-admissibility. We also
introduce the asymptotic notation. Section 5 concludes the paper. An additional demonstration using the high-dimensional Gaussian sequence model
with an L2 -constraint parameter space is provided in Appendix B.

K. Yano and F. Komaki/On ε-Admissibility

4

2. Preliminaries
2.1. Bounds for ε-admissibility
In this subsection, we provide the general lower and upper bounds for
R(Θ, δ) that are used in later sections. While these bounds are fundamental
and have been widely used in the literature of statistical decision theoretic
literature (see, for example, Chapter 5 of Lehmann and Casella (1998)), the
proofs help clarify the concept of ε-admissibility. Throughout this subsection, we use a fixed estimator δ ∈ D.
Lemma 2.1. For an estimator δ̃ that dominates δ,
R(Θ, δ) ≥ inf [R(θ, δ) − R(θ, δ̃)] ≥ 0.
θ∈Θ

Proof. The first inequality follows immediately from the definition of R(Θ, δ).
The second inequality follows since for any θ ∈ Θ, R(θ, δ) ≥ R(θ, δ̃).
Lemma 2.2. For a probability measure Π on Θ,
Z
R(Θ, δ) ≤ [R(θ, δ) − R(θ, δΠ )]dΠ(θ),
Θ

where
δΠ is the Bayes solution with respect to Π, i.e., the minimizer of
R
R(θ,
δ)dΠ(θ).
Θ
R
Proof. Since for a function f of θ, inf θ∈Θ f (θ) ≤ Θ f (θ)dΠ(θ), we have
R(Θ, δ) = sup inf [R(θ, δ) − R(θ, δ̃)]
δ̃

≤ sup
Z

δ̃

θ

Z

Θ

[R(θ, δ) − R(θ, δ̃)]dΠ(θ)
Z

R(θ, δ̃)dΠ(θ)
R(θ, δ)dΠ(θ) − inf
δ̃
Θ
Z
Z
R(θ, δΠ )dΠ(θ),
R(θ, δ)dΠ(θ) −
=
=

Θ

Θ

Θ

where the last equality follows from the definition of the Bayes solution.
Next, we describe the relationships between ε-admissibility and admissibility and between ε-admissibility and minimaxity. Although these relationships are not used in this paper, they also help clarify the nature of
ε-admissibility.

K. Yano and F. Komaki/On ε-Admissibility

5

Proposition 2.3. If δ is admissible, then R(Θ, δ) = 0. If δ is minimax with
a constant risk, then again R(Θ, δ) = 0.
Proof. The first claim holds because from the admissibility of δ, we have
inf θ∈Θ [R(θ, δ) − R(θ, δ̃)] ≤ 0 for any δ̃ ∈ D and thus supδ̃ inf θ∈Θ [R(θ, δ) −
R(θ, δ̃)] ≤ 0. The second claim holds because letting c := inf δ̃ supθ R(θ, δ̃)
yields
R(Θ, δ) = sup inf [R(θ, δ) − R(θ, δ̃)] = c − inf sup R(θ, δ̃) = 0.
δ̃∈D

θ∈Θ

δ̃

θ

2.2. Relationship to C-admissibility
The concept of C-admissibility has appeared in the recent literature on estimation under the shape restriction, and its connection to ε-admissibility
should be noted. An estimator δ is C > 0-admissible if and only if for every
other estimator δ̃, there exists θ ∈ Θ such that C × R(θ, δ) ≤ R(θ, δ̃). See
Chatterjee (2014) and Chen, Guntuboyina and Zhang (2017) for a discussion of this.
The only difference between ε-admissibility and C-admissibility is that
ε-admissibility is based on the risk difference, whereas C-admissibility is
based on the risk ratio. Chen, Guntuboyina and Zhang (2017) argues that
for a given estimator δ, the smallest value of C for which δ is C-admissible
has a minimax interpretation:
R(θ, δ̃)
.
θ∈Θ R(θ, δ)

sup{C : δ is C-admissible} = inf sup
δ̃

Likewise, the minus of the smallest value of ε for which δ is ε-admissible also
has a minimax interpretation:
− inf{ε : δ is ε-admissible} = inf sup[R(θ, δ̃) − R(θ, δ)].

(1)

δ̃ θ∈Θ

The quantity (1) itself is of interest. Orlitsky and Suresh (2015) conducted
the regret analysis based on the quantity (1) for some baseline estimator δ.
The difference between the present paper and those of Chatterjee (2014)
and Chen, Guntuboyina and Zhang (2017) is that the latter addressed a
universal bound for C irrespective of the dimension of the parameter space
and the sample size, whereas our paper uses the rate of diminution of ε
as the dimension or the sample size grows to infinity for the performance
comparison.

K. Yano and F. Komaki/On ε-Admissibility

6

2.3. Asymptotic notation
In this subsection, we set out the asymptotic notation used in later sections.
For positive functions f (d) and g(n), the relation f (d) . g(d) as d → ∞
means that
lim f (d)/g(d) < ∞.
d→∞

The relation f (d) ≍ g(d) as d → ∞ means that f (d) . g(d) and g(d) . g(d).
3. Poisson sequence model with L1 -constraint parameter space
In this section, we present further details of the P
example discussed in the
introduction. Let X = Nd , Θ = {θ = (θ1 , . . . , θd ) : di=1 θi /d ≤ 1, θi ≥ 0, i =
1, . . . , d}, and P = {Pθ = ⊗di=1 Po(θi ) : θ ∈ Θ}, where Po(λ) is a Poisson
distribution with mean λ. Let A = Rd+ with the corresponding decision space
D. Let L(θ, a) = DKL (Pθ || Pa ) with the corresponding risk function R(θ, θ̂),
where DKL (Pθ || Pθ′ ) is the Kullback–Leibler divergence from Pθ to Pθ′ :
DKL (Pθ || Pθ′ ) :=

Z


d 
X
θi
dPθ
′
dPθ =
θi log ′ + θi − θi .
log
dPθ′
θi
i=1

We discuss the performance of the following two estimators from the viewpoint of the minimax rate of convergence and that of ε-admissibility. Let
θ̂J,i (X) := Xi + 1/2, i = 1, . . . , d
and let
Pd

θ̂JS,i (X) := Pd

j=1 Xj

j=1 Xj

+1

+ d/2

(Xi + 1/2), i = 1, . . . , d.

The estimator θ̂J is the Bayes estimator based on Jeffreys’ prior and the
estimator θ̂JS is the James–Stein type estimator used in Poisson sequence
models. For further details, see Komaki (2004) and Komaki (2006).
3.1. Main results for the Poisson sequence model
We first discuss the minimax rate of convergence. The following theorem
shows that, from the minimax rate of convergence, it is impossible to determine whether θ̂JS is better than θ̂J .

K. Yano and F. Komaki/On ε-Admissibility

7

Theorem 3.1. We have
inf sup R(θ, θ̂) ≍ sup R(θ, θ̂J ) ≍ sup R(θ, θ̂JS ) ≍ d

θ̂∈D θ∈Θ

θ∈Θ

θ∈Θ

as d → ∞
Next, we discuss ε-admissibility. The following theorem shows that, from
the viewpoint of ε-admissibility, the James–Stein type estimator is superior
to the Bayes estimator based on Jeffreys’ prior.
Theorem 3.2. We have
R(Θ, θ̂JS ) . 1 . d . R(Θ, θ̂J )
as d → ∞.
The proofs of theorems are given in the next subsection.
3.2. Proofs of theorems
In this subsection, we give the proofs of Theorems 3.1 and 3.2
Proof of Theorem 3.1. This proof relies on the fact that a minimax risk is
bounded below by a Bayes risk: For a probability distribution Π on Θ, we
have
Z
(2)
inf sup R(θ, θ̂) ≥ R(θ, θ̂Π )dΠ(θ),
θ̂∈D θ∈Θ

where θ̂Π is the Bayes solution with respect to Π. Let Π be



1
1
1
1
θd
θ1
Π(dθ) = δ0 (dθ) + δd (dkθk1 ) ⊗ Dir
,··· ,
,...,d
,
d
2
2
2
2
kθk1
kθk1
where δx is the Dirac measure having a mass on x, Dir(1/2, . . . , 1/2)(dx1 , . . . , dxd )
1/2−1
is the Dirichlet distribution of which the density is proportional to x1
×
P
1/2−1
, and kθk1 := i |θi |.
· · · × xd
R
First, we show that the Bayes risk b(Π) := R(θ, θ̂Π )dΠ(θ) is of order d
by assuming that the following two claims hold:
Claim C1. b(Π) is given by
d n −d
e log(1 + ed ) + ψ(3/2) − ψ(d/2 + 1)
b(Π) =
2
+EX∼Po(d) log(d/2 + X)
−Eb∼Beta(3/2,d/2−1) EX∼Po(bd) log(1/2 + X) ,

(3)

K. Yano and F. Komaki/On ε-Admissibility

8

where ψ(·) is the digamma function, that is, the derivative
of the log of the Gamma function;
Claim C2. for any ε ∈ (0, 1), the asymptotic inequality
EX∼Po(d) log(d/2 + X) ≥ log(3d/2) + log(1 − ε) + oε (1)
holds,
where oε (1) is the o(1) term depending on ε.
Applying Jensen’s inequality to x → log(1/2 + x) yields
Eb∼Beta(3/2,d/2−1) EX∼Po(bd) log(1/2 + X) ≤ Eb∼Beta(3/2,d/2−1) log(1/2 + bd).
Since Beta(3/2, d/2 − 1) converges weakly to δ1 as d → ∞, we have
Eb∼Beta(3/2,d/2−1) log(1/2 + bd) = log d + o(1).

(4)

Note that b ∈ [0, 1] → log(1/2 + b) is bounded and continuous. Thus, from
Claims C1 and C2, from the asymptotic inequality (4), and from the asymptotic relationship that ψ(d/2 + 1) = log(d/2) + o(1) (see Olver et al. (2010)),
we have
b(Π) ≥

d
(1 + ψ(3/2) + log(1 − ε) + oε (1)).
2

Taking ε such
R that 1 + ψ(3/2) + log(1 − ε) > 0, it is shown that the Bayes
risk b(Π) := R(θ, θ̂Π )dΠ(θ) is of order d.

Next, we prove that Claim C1 holds. Let S := Dir(1/2, . . . , 1/2). For
i = 1, . . . , d, we have

R
Q
x
( dj=1 wj j )dS(w1 ,...,wd )
d × RwiQ
if xj 6= 0 for some j,
x
( dj=1 wj j )dS(w1 ,...,wd )
θ̂Π,i (X) =
R
−d

e
d × 1+e
wi dS(w1 , . . . , wd ) if all xj s are 0.
−d

K. Yano and F. Komaki/On ε-Admissibility

9

Substituting the above expression of θ̂Π into R(θ, θ̂Π ), we have
!
d
d
Pd
X
X
θi
θi log
R(θ, θ̂Π ) =
θi e− i=1 θi log(1 + ed )
+
d/d
i=1
i=1
!
d
Pd
X
1
−
θi + d − de− i=1 θi
1 + e−d
i=1
!
d
X
θi log d
−
+

−

i=1
d
X

i=1
d
X

θi

!

EX∼Po(Pd

i=1 θi )

log(d/2 + X)

θi EXi ∼Po(θi ) log(1/2 + Xi ).

(5)

i=1

Since
Z X
d

θi log θi dΠ(θ) =d log d + d

Z X
d

wi log wi dS(w1 , . . . , wd )

i

i=1

=d log d + ψ(3/2) − ψ(d/2 + 1),

taking the expectation of the right hand side of (5) over θ with respect to
Π shows that Claim C1 holds.
Finally, we prove that Claim C2 holds. We have


2 X −d
.
EX∼Po(d) log(d/2 + X) = log(3d/2) + EX∼Po(d) log 1 + √ √
3 d
d
Since for a random variable X distributed according to Po(d), (X − d)/d
converges to 0 in probability as d → ∞, we have, for any ε ∈ (0, 1),


2 X −d
EX∼Po(d) log 1 + √ √
3 d
d


2 X −d
≥ log(1 − ε) + EX∼Po(d) 1|2(X−d)/d|>ε log 1 + √ √
3 d
d
≥ log(1 − ε) + Pr(|2(X − d)/d| > ε) log(1/3)
= log(1 − ε) + oε (1),

K. Yano and F. Komaki/On ε-Admissibility

10

where X is a random variable distributed according to Po(d). This completes
the proof.
Proof of Theorem 3.2. It suffices to show that for any d ∈ N,


1 − e−d
R(Θ, θ̂J ) ≥ −d log 1 + (d/2 − 1)
+ d/2 − 1
d

(6)

and that for any d ∈ N,
1
R(Θ, θ̂JS ) ≤ .
2

(7)

The proof of (6) follows the proof that θ̂JS dominates θ̂J ; see Komaki
(2004). Applying Lemma 2.2 with δ̃ = θ̂JS yields
R(Θ, θ̂J ) ≥ inf [R(θ, θ̂J ) − R(θ, θ̂JS )].
θ∈Θ

Since
R(θ, θ̂J ) − R(θ, θ̂JS ) = EX∼Pθ

d
X
i=1

"

θi log

θ̂JS,i (X)
θ̂J,i (X)

#

+ θ̂J,i (X) − θ̂JS,i (X) ,

we have
R(Θ, θ̂J ) ≥ inf EX∼Pθ
θ∈Θ

"

d
X
i=1

Pd

θi log Pd

j=1 Xj

j=1 Xj

+1

+ d/2

#

+ (d/2 − 1) .

P
P
Since the distribution of Z = di=1 Xi is Po(µ) with µ = di=1 θi , we have
" d
#
Pd
X
X
+
1
j
j=1
inf EX∼Pθ
θi log Pd
+ (d/2 − 1)
θ∈Θ
j=1 Xj + d/2
i=1




d/2 − 1
= inf EZ∼Po(µ) −µ log 1 +
+ (d/2 − 1)
Z +1
µ∈[0,d]



d/2 − 1
≥ inf
−µ log 1 + EZ∼Po(µ)
+ (d/2 − 1)
Z +1
µ∈[0,d]



1 − e−µ
+ (d/2 − 1)
= inf
−µ log 1 + (d/2 − 1)
µ
µ∈[0,d]
= −d log (1 + d/2 − 1) + (d/2 − 1).

K. Yano and F. Komaki/On ε-Admissibility

11

Here the first inequality follows from Jensen’s inequality and the third equality from the identity
EZ∼Po(µ) [1/(Z + 1)] = {1 − e−µ }/µ.
The proof of (7) immediately follows from Lemma 2.2 with Π = δ0 , which
completes the proof of Theorem 3.2.
4. Gaussian infinite sequence model with Sobolev-type
constraint parameter space
In this section, we consider estimation of the mean in a Gaussian infinite sequence model with Sobolev-typePconstraint parameter space. Let X = R∞ .
2α 2
Let Θ = {θ = (θ1 , θ2 . . . , ) ∈ l2 : ∞
i=1 i θi ≤ B} with α > 0 and B > 0 and
∞
P = {Pθ = ⊗i=1 N (θi , 1/n) : θ ∈ Θ}. The hyperparameter α controls the
smoothness level of a true function in the nonparametric regression model
and the hyperparameter B controls the volume of the parameter space. In
this paper, we assume that α is known. Even in the setting in which α is
known, the results in this section are noble. Let A = R∞ with the corresponding decision space D and L(θ,P
a) = kθ − ak2 with the corresponding
2
∞
risk function R(θ, θ̂), where kbk2 := ∞
i=1 bi for b ∈ R . In this section, we
consider the asymptotics in which the sample size n grows to ∞.
Let θ̂G be the Bayes estimator based on the Gaussian prior
−2α−1
G = ⊗∞
)
i=1 N (0, i

and θ̂S be the Bayes estimator based on the prior
S=

∞
X

M (d)

d=1

hn

where M (d) = e−ad /

i
o 
,
⊗di=1 N (0, d2α+1 i−2α−1 /n) ⊗ ⊗∞
i=d+1 N (0, 0)

P∞

−ai .
i=1 e

Remark 4.1. The prior S is discussed in Yano and Komaki (2017), and is
a modification of the compound prior in Zhao (2000). The compound prior
C is given as follows:
C=

∞
X
d=1

M (d)

hn

i
o 
.
⊗di=1 N (0, i−2α−1 ) ⊗ ⊗∞
N
(0,
0)
i=d+1

K. Yano and F. Komaki/On ε-Admissibility

12

The modification is necessary to ensure that R(Θ, θ̂S ) remains sufficiently
small. Roughly speaking, it does require the prior mass condition under
which the prior puts nearly the full mass on Θ to make R(Θ, θ̂S ) sufficiently
small; for further details, see the proof of Theorem 4.4 below. The mass
placed on Θ by the compound prior C is strictly less than 1 even as n → ∞,
whereas that by the prior S grows to 1 as n → ∞ for a fixed B > 0; see
Lemma 4.5. To demonstrate that the compound prior C places a mass on Θ
that is strictly less than 1 even as n → ∞, we have C (Θ) ≤ Pr(N 2 ≤ B) < 1,
where N is a one-dimensional standard normal random variable.
4.1. Existing result for the Gaussian sequence model
The following existing result shows that, from the viewpoint of the minimax
rate of convergence, θ̂G and θ̂S yield the same performance.
Lemma 4.2 (Theorem 5.1. in Zhao (2000) and Theorem 2 in Yano and Komaki
(2017)). For any α > 0 and any B > 0, we have
inf sup R(θ, θ̂) ≍ sup R(θ, θ̂G ) ≍ sup R(θ, θ̂S ) ≍ (1/n)2α/(2α+1)

θ̂∈D θ∈Θ

θ∈Θ

θ∈Θ

as n → ∞.
4.2. Main results for the Gaussian sequence model
The noble results presented in this subsection show that, from the viewpoint
of ε-admissibility, θ̂S is superior to θ̂G in the case that α = 1 and B = 1
or the case that B is sufficiently small. Numerical evaluations also show the
superiority of θ̂S over θ̂G for any α > 0 and for any B > 0. The results
are based on two theorems: Theorem 4.3 shows that R(Θ, θ̂G ) is of the same
order as the maximum risk of the estimator, which indicates that there exists
an estimator θ̂ such that
R(θ, θ̂) + O(n−2α/(2α+1) ) < R(θ, θ̂G ) < O(n−2α/(2α+1) )
for all θ ∈ Θ. Theorem 4.4 shows that R(Θ, θ̂S ) has an exponential decay.
Theorem 4.3. There exists a constant c depending only on B and α such
that the inequality
lim R(Θ, θ̂G )/n−2α/(2α+1) > c

n→∞

holds. For B = 1 and α = 1, c is taken to be strictly positive. For any α > 0,
c is taken to be strictly positive if B is sufficiently small.

K. Yano and F. Komaki/On ε-Admissibility

13

Theorem 4.4. We have

o
n a
1
R(Θ, θ̂S ) . exp − (nB) 4α+2
2

as n → ∞.

c

0.3
0.0

0.00

0.1

0.05

0.2

0.10

c

0.15

0.4

0.20

0.5

0.6

0.25

Proofs are provided in Subsection 4.3.

0

20

40

60
α

80

100

0

20

40

60

80

100

B

Fig 1. Possible choice of c in Theo- Fig 2. Possible choice of c in Theorem 4.3 with B = 1
rem 4.3 with α = 1

Although we do not provide a proof of the strict positivity of c in Theorem
4.3 for general settings, numerical evaluations (see Figures 1 and 2) show
that we can assume strict positivity of c in the case that α = 1 or in the
case that B = 1. Here, the choice of c in Figures 1 and 2 is described at the
beginning of the proof of Theorem 4.3.
4.3. Proofs of theorems
−2α−1 ).
Proof of Theorem 4.3. Let θ̂G(s) be the Bayes estimator based on ⊗∞
i=1 N (0, si
For s ∈ (0, 1), we will show that

inf [R(θ, θ̂G ) − R(θ, θ̂G(s) )]
" 
)
 2α+2 (
−2
2α+1
2α
2α
+
2
s
1
=n− 2α+1 × B
−
2α+2 2
2
4α + 1
{1 + 2α+2
{1 + s(4α+1)
}
4α+1 }

Z ∞
1
1
1
− 2α+1
dx
−
n
.
(8)
+(1 − s 2α+1 )
(1 + x2α+1 )2
0

θ∈Θ

Letting c be the supremum of the right hand side in the above inequality,
will completes the proof. The strict positivity of c for the specific settings is
proved in the last step.

K. Yano and F. Komaki/On ε-Admissibility

14

First, a direct evaluation of the risks yields
R(θ, θ̂G ) − R(θ, θ̂G(s) )
(
2  2α+1
2 )
∞
2α+1 /n
X
i
i
/{ns}
θi2
=
−
1 + i2α+1 /n
1 + i2α+1 /{ns}
i=1
(
2 
2 )
∞
X
1
1
−1
+n
−
1 + i2α+1 /n
1 + i2α+1 /{ns}
i=1


(
2  2α+1
2 )
∞
∞
2α θ 2
2α+1 /n
X
X
i
/{ns}
i
i
2α
2
i
P∞ 2α 2
−
=
j θj 
1 + i2α+1 /n
1 + i2α+1 /{ns}
j
θ
j=1
j
i=1
j=1
(
2 
2 )
∞
X
1
1
−1
+n
−
.
1 + i2α+1 /n
1 + i2α+1 /{ns}
i=1

For θ = 0,
R(θ, θ̂G) − R(θ, θ̂G(s) ) ≥ n−1

(
∞
X
i=1

1
1 + i2α+1 /n

2

−



1
1 + i2α+1 /{ns}

2 )

For θ 6= 0, a calculation in Appendix C yields
R(θ, θ̂G ) − R(θ, θ̂G(s) )
"
#

 2α+2
2α
2α + 2 2α+1 − 2α+1
1
s−2
≥B
n
2α+2 2 −
2α+2 2
4α + 1
{1 + 4α+1
{1 + s(4α+1)
}
}
(
)




∞
2
2
X
1
1
+ n−1
−
.
1 + i2α+1 /n
1 + i2α+1 /{ns}
i=1

Therefore, we have
inf [R(θ, θ̂G ) − R(θ, θ̂G(s) )]

θ∈Θ

#
"
 2α+2
2α
s−2
2α + 2 2α+1 − 2α+1
1
−
n
=B
2α+2
2
4α + 1
}2
{1 + 2α+2
{1 + s(4α+1)
4α+1 }
(
)

2 
2
∞
X
1
1
−1
+n
−
.
1 + i2α+1 /n
1 + i2α+1 /{ns}


i=1

(9)

.

K. Yano and F. Komaki/On ε-Admissibility

By convergence of the Riemann sum
number N , we have
∞
X
i=1

1
≤N
(1 + (i/N )2α+1 )2

Z

∞
0

P∞

i=1 {1

15

+ (i/N )2α+1 }−2 for a positive
∞

X
1
1
dx ≤
+ 1.
2α+1
2
(1 + x
)
(1 + (i/N )2α+1 )2
i=1

(10)

Combining (10) with (9) yields
inf [R(θ, θ̂G ) − R(θ, θ̂G(s) )]

θ∈Θ

≥B



2α + 2
4α + 1

 2α+2
2α+1

n

2α
− 2α+1

"

1
{1 +

2α+2 2
4α+1 }

−

s−2
{1 +

2α+2
2
s(4α+1) }

#

Z ∞
2α
1
1
+ n− 2α+1 {1 − s 2α+1 }
dx − n−1
2α+1 )2
(1
+
x
0
)
" 
 2α+2 (
2α
s−2
1
2α + 2 2α+1
− 2α+1
−
=n
× B
2α+2 2
2
4α + 1
}
{1 + 2α+2
{1 + s(4α+1)
4α+1 }

Z ∞
1
1
1
dx − n− 2α+1 .
+(1 − s 2α+1 )
(11)
2α+1
2
(1 + x
)
0
For B = 1 and α = 1, taking s = 0.9 confirms that the right hand side is
positive.
The choice of s follows from the direct evaluation of the integral
R∞
2α+1
)−2 dx. For an arbitrary s ∈ (0, 1) and sufficiently small B > 0,
0 (1 + x
the right hand side is positive. This completes the proof.

e be the probability disProof of Theorem 4.4. Let T = ⌊(nB)1/(4α+2) ⌋. Let S
tribution obtained by restricting S to Θ. The corresponding Bayes estimator
∞
is denoted by θ̂eS . Let D ∗ := {δ ∈ (l2 )R : δ(x) ∈ Θ}
From Lemma 2.2, it suffices to show that
Z
Z
e
inf R(θ, δ)dS(θ) + O(exp(−aT )) ≥ R(θ, θ̂S )de
S(θ).
(12)
δ

Since θ̂eS is included in D ∗ ,
Z
Z
inf R(θ, δ)de
S(θ) = inf
R(θ, δ)de
S(θ).
δ

δ∈D

K. Yano and F. Komaki/On ε-Admissibility

16

Since 1θ∈Θ = 1θ∈l2 − 1θ∈Θc , we have
Z
inf∗ R(θ, δ)de
S(θ)
δ∈D


Z
Z
1
≥
R(θ, δ)dS(θ)
inf
R(θ, δ)dS(θ) − sup
S(Θ) δ∈D∗
δ∈D ∗ θ∈Θc

 Z
Z
1
≥
R(θ, δ)dS(θ) .
inf R(θ, δ)dS(θ) − sup
S(Θ) δ
δ∈D ∗ θ∈Θc
R
Since ||θ − δ(X)||2 ≤ 2(B + ||θ||2 ) for δ ∈ D ∗ and inf δ R(θ, δ)dS =
R
R(θ, θ̂S )dS(θ), we have

 Z
Z
R(θ, δ)dS(θ)
inf R(θ, δ)dΛ(θ) − sup
δ
δ∈D ∗ θ∈E c

Z
p
1/2
c
c
4
1θ∈Θ R(θ, θ̂S )dS(θ) − (2BS(Θ ) + 2 ES [kθk ]S (Θ ))
≥

Z
c
−1 1/2
c
1θ∈Θ R(θ, θ̂S )dS(θ) − (2BS(Θ ) + 2cn S (Θ )) ,
≥
P
2α+1 < ∞. To complete the proof, we use the folwhere c := ∞
d=1 M (d)d
lowing lemma.

Lemma 4.5. There exists a constant c1 depending on a > 0 such that for
a sufficiently large n ∈ N, the inequality
1/(2×(2α+1)) ⌋

S ((Θ)c ) ≤ c1 e−a⌊(nB)
holds.

The proof of Lemma 4.5 is given after completing the proof of Theorem
4.4.
By Lemma 4.5, for a sufficiently large n > 0, we have
Z
Z

n a o
R(θ, θ̂S )de
S(θ) − O exp − T
inf R(θ, δ)de
S(θ) ≥
,
δ
2

which demonstrates that inequality (12) holds.

K. Yano and F. Komaki/On ε-Admissibility

17

Proof of Lemma 4.5. Let T = ⌊(nB)1/(4α+2) ⌋. By definition,
!
∞
d
X
X
nB
S (Θc ) =
i−1 |Ni |2 > 2α+1
M (d)Pr
d
i=1
d=1
!
∞
d
T
X
X
X
nB
−1
2
M (d)
i |Ni | > 2α+1 +
M (d)Pr
≤
d
i=1
d=T +1
d=1
!
T
d
∞
X
X
X
nB
=
|Ni |2 > 2α+1 +
M (d)Pr
M (d),
T
i=1

d=1

d=T +1

where {Ni }Ti=1 are independent random variables distributed according to
N (0, 1).
We next apply an exponential inequality for the chi-square statistics
(Lemma 1 in Laurent and Massart (2000)): for any x > d,
!
d
√
√
X
( x− d)2
2
2
2
≤ e−
|Ni | ≥ x
Pr
.
i=1

Setting x = (1/2)(nB)1/2 , we have
S (Θc ) ≤

T
X

1

1/2 (1+o(nB))

M (d)e− 8 (nB)

d=1

+

∞
X

M (d).

d=T +1

Setting nB such that the o(nB) term in the above inequality is less than
1/2 completes the proof.
5. Discussion and conclusions
In this paper, we have demonstrated the usefulness of ε-admissibility in
high-dimensional and nonparametric statistical models by presenting two
new results. These results suggest the use of ε-admissibility in conjunction
with the other criteria such as the minimax rate of convergence.
Appendix A: Poisson sequence model and inhomogeneous
Poisson point process model
In this appendix, we discuss the relationship between Poisson sequence
models and inhomogeneous Poisson point process models. Let {Nt : t ∈

K. Yano and F. Komaki/On ε-Admissibility

18

[0, 1]} be an inhomogeneous Poisson point process with an intensity function λ(·) : [0, 1] → R+ . We assume that the L1 -norm of λ is bounded by
d. Let h := ⌊1/d⌋ be a time resolution. From the independent incremental
property of an inhomogeneous Poisson point process, d random variables
Nh , N2h − Nh , · · · , N1 − Nh×d are independently distributed according to
R 2h
R1
Rh
Poisson distributions with means 0 λ(t)dt, h λ(t)dt,. . ., h×d λ(t)dt, respectively. Thus letting X1 := Nh ,X2 := N2h − Nh ,· · · , Xd := N1 − Nh×d
R1
Rh
R 2h
and letting θ1 = 0 λ(t)dt, θ2 = h λ(t)dt, . . ., θd = h×d λ(t)dt yields the
Poisson sequence model presented in Section 3.
Appendix B: Gaussian sequence model with an L2 -constraint
parameter space
In this appendix, we consider estimation in a d-dimensional Gaussian sequence model P
with L2 -constraint parameter space. Let X = Rd , Θ = {θ =
d
2
d
d
(θ1 , . . . , θd ) :
i=1 θi /d ≤ 1}, P = {Pθ = ⊗i=1 N (θi , 1)}, A = R , and
L(θ, a) = kθ − ak2 , where k · k is the l2 -norm in Rd . We compare the following two estimators: one is the maximum likelihood estimator θ̂MLE (X) := X;
the other is the James–Stein estimator θ̂JS (X) := (1 − (d − 1)/kXk2 )X.
First, the following lemma shows that the rate of convergence of θ̂MLE is
equal to that of a minimax risk, which indicates that from the viewpoint
of the rate of convergence we can not determine whether θ̂JS is superior to
θ̂MLE .
Lemma B.1 (Theorems 7.28 and 7.48 in Wasserman (2006)). We have
inf θ̂∈D supθ∈Θ R(θ, θ̂)
1
= .
d→∞
d
2

(13)

sup R(θ, θ̂MLE ) ≍ sup R(θ, θ̂JS ) ≍ d.

(14)

lim

We also have
θ∈Θ

θ∈Θ

Next, the following theorem tells that from the viewpoint of weak admissibility, θ̂JS is better than θ̂MLE as d grows to infinity.
Theorem B.2. We have
R(Θ, θ̂JS ) . 1 . d . R(Θ, θ̂MLE ).
Remark that, unlike the examples in Sections 3 and 4, we also use multiplicative constant terms lim supθ∈Θ R(θ, θ̂JS )/d and lim supθ∈Θ R(θ, θ̂MLE )/d
to compare these estimators: the former is 1/2, whereas the latter is 1.

K. Yano and F. Komaki/On ε-Admissibility

19

The proof of this theorem is a simple combination of the following lemmas.
Lemma B.3. For any d(∈ N) > 2, we have
(d − 2)2
.
2d
Proof. The proof follows the line of the well-known proof that the James–
Stein estimator dominates the maximum likelihood estimator. Applying
Lemma 2.1 with δ̃ = θ̂JS yields
R(Θ, θ̂MLE ) ≥

R(Θ, θ̂MLE ) ≥ inf [R(θ, θ̂MLE ) − R(θ, θ̂JS )]
θ∈Θ


= inf EX|θ kθ − Xk2 − kθ − X + (d − 2)X/kXkk2
θ∈Θ



(d − 2)X
(d − 2)2
+ 2 X − θ,
= inf EX|θ −
θ∈Θ
kXk2
kXk2
#
"
d
X
kXk2 − 2Xi2
(d − 2)2
+ 2(d − 2)
= inf EX|θ −
θ∈Θ
kXk2
kXk4
i=1


1
2
= (d − 2) inf EX|θ
θ∈Θ
kXk2


(d − 2)2
≥ inf
.
(15)
θ∈Θ d + kθk2

Here, the third equality follows from Stein’s lemma and from that ∂(X/kXk2 )/∂Xi =
(kXk2 −2Xi )/kXk4 . The last inequality follows from Jensen’s inequality and
from that EX|θ kXk2 = d + kθk2 . Thus, we complete the proof.
Lemma B.4. For any d(∈ N) > 2, we have
4
R(Θ, θ̂JS ) ≤ 2 − .
d
Proof. Applying Lemma 2.2 with Π := δ0 yields
R(Θ, θ̂JS ) ≤ R(0, θ̂JS ).
From (15), we have R(0, θ̂JS ) ≤ (2d − 4)/d, which completes the proof.
Appendix C: Calculation used in Theorem 4.3
In this appendix, we provide a calculation that is used in Theorem 4.3. Let
f (x, y) :=

y 2 x2α+2
.
(1 + yx2α+1 )2

K. Yano and F. Komaki/On ε-Admissibility

20

Then, the risk difference of R(θ, θ̂G ) − R(θ, θ̂G(s) ) is
R(θ, θ̂G )−R(θ, θ̂G(s) )
P
∞
2α 2 X
i2α θi2
j=1 j θj
P
=
{f (i, 1/n) − f (i, 1/{ns})}
∞
2α 2
n−1
j=1 j θj
i=1

!2 
2
∞ 

X
1
1
−
.
+
e

 1 + i2α+1 /n
1 + i2α+1 /{nB}
i=1

(16)

We will show that the following inequality holds:

R(θ, θ̂G ) − R(θ, θ̂G(s) )
#
"

 2α+2
2α
s−2
2α + 2 2α+1 − 2α+1
1
n
≥B
2α+2 2 −
2α+2 2
4α + 1
}
}
{1 + 4α+1
{1 + s(4α+1)
(
)

2 
2
∞
X
1
1
−1
+n
−
.
1 + i2α+1 /n
1 + i2α+1 /{ns}
i=1

To derive this, we show that the lower bound of f (i, 1/n) − f (i, 1/{nB})
for all i ∈ N is given by
e ≥ f (x∗ , 1/n) − f (x∗ , 1/{ns}),
f (i, 1/n) − f (i, 1/{nB})

where

1

1

x∗ := ({2α + 2}/{4α + 1}) 2α+1 n 2α+1 .
For a fixed y,
∂f (x, y)/∂x = [y 2 x2α+1 {(2α + 2) − 2αyx2α+1 }]/(1 + yx2α+1 )3 .
Letting
g(y; x) := {(2α + 2)x2α+1 y 2 − (2α)x4α+2 y 3 }/(1 + yx2α+1 )3
yields
√
√
∂f (x, y)/∂x − ∂f (x, y/ s)/∂x = g(y; x) − g(y/ s; x).
Since

∂g(y; x)
x2α+1 y
=
(4α + 4) − (8α + 2)x2α+1 y ,
∂y
(1 + x2α+1 y)4

(17)

(18)

K. Yano and F. Komaki/On ε-Admissibility

21

we have
√
∂f (x, y)/∂x − ∂f (x, y/ s)/∂x ≥ 0 if x2α+1 y ≥ (4α + 4)/(8α + 2)
and
√
∂f (x, y)/∂x − ∂f (x, y/ s)/∂x ≤ 0 if x2α+1 y ≤ (4α + 4)/(8α + 2).
Therefore,
f (i, 1/n) − f (i, 1/{ns}) ≥ f (x∗ , 1/n) − f (x∗ , 1/{ns}).
P
From the inequality that ∞
i=1 p(i)f (i) ≥ L for a function f : N → [L, ∞)
with some L ∈ R and for a probability
p on N, and from the inequality
P
2α θ 2 as a probability completes the
(17), regarding (12α θ12 , 22α θ22 , . . .)/ ∞
j
j
j=1
proof.
References
Blackwell, D. and Girshick, M. (1954). Theory of Games and Statistical
Desicions. John Wiley & Sons.
Chatterjee, S. (2014). A new perspective on least squares under convex
constraint. Ann. Statist. 42 2340–2381.
Chen, X., Guntuboyina, A. and Zhang, Y. (2017). A Note on the Approximate Admissibility of Regularized Estimators in the Gaussian Sequence Model. In eprint arXiv:1703.00542.
Farrell, R. (1968). Toward a theory of generalized Bayes tests. Ann. Math.
Statist. 39 1–22.
Ferguson, T. (1967). Mathematical Statistics: A Decision Theoretic Approach. Academic Press.
Ghosh, M. and Yang, M. (1988). Simultaneous estimation of Poisson
means under entropy loss. Ann. Statist. 16 278–291.
Hartigan, J. (1983). Bayes theory. Springer.
Heath, D. and Sudderth, W. (1978). On Finitely Additive Priors, Coherence, and Extended Admissibility. Ann. Statist. 6 333–345.
Komaki, F. (2004). Simultaneous prediction of independent Poisson observables. Ann. Statist. 32 1744–1769.
Komaki, F. (2006). A class of proper priors for Bayesian simultaneous prediction of independent Poisson observables. J. Multivariate Anal. 97 1815–
1828.
Komaki, F. (2015). Simultaneous prediction for independent Poisson processes with different durations. J. Multivariate Anal. 141 35–48.

K. Yano and F. Komaki/On ε-Admissibility

22

Laurent, B. and Massart, P. (2000). Adaptive estimation of a quadratic
functional by model selection. Ann. Statist. 28 1302–1338.
Lehmann, E. and Casella, G. (1998). Theory of Point Estimation, 2nd
ed. Springer.
Olver, F., Lozier, D., Boisvert, R. and Clark, C. (2010). NIST Handbook of Mathematical Functions. Cambridge University Press.
Orlitsky, A. and Suresh, A. (2015). Competitive Distribution Estimation: Why is Good–Turing Good. In Advances in Neural Information Processing Systems 28.
Shen, X. and Wasserman, L. (2001). Rate of convergence of posterior
distributions. Ann. Statist. 29 687–714.
Tsybakov, A. (2009). Introduction to Nonparametric Estimation.
Springer.
Wasserman, L. (2006). All of Nonparametric Statistics. Springer.
Yano, K. and Komaki, F. (2017). Scale-ratio Asymptotics in Nonparametric Minimax Estimation. In eprint arXiv:1609.00940.
Zhao, L. (2000). Bayesian Aspects of Some Nonparametric Problems. Ann.
Statist. 28 532–552.

