数理解析研究所講究録
916 巻 1995 年 112-130

112

A NOTE ON THE CONJECTURE THATTHIRD-ORDER EFFICIENCY IMPLIES FOURTH-ORDER EFFICIENCYYUTAKA KANO* (狩野 裕)
Abstract. Takeuchi [33] and Pfanzagl [23] proved that any first-order efficient estimators
are second-order efficient. Many authors e.g., Ghosh [12], have conjectured that any thirdorder efficient estimators are fourth-order efficient. Based on concentration probability of
estimators about a true parameter, this paper gives a positive answer to the conjecture ina curved exponential family with multi-structural parameters. It is seen that choice of biascorrection factors is critical.

1. IntroductionLet

be a parameter vector of interest and let an open set$\mathrm{u}\in \mathrm{R}^{q}$parameter space of . Let$\mathrm{u}$$\Lambda’(\mathrm{u}_{0})$, a neighborhood of$\mathrm{u}_{0}\in\Omega$$\mathrm{u}_{0}$be an arbitrarily fixed (inner) point of, assume that random -vectors$p$$\mathrm{x}_{1},$$\ldots,$$\Omega(\subset \mathrm{R}^{q})$. For every$\Omega$$\mathrm{x}_{n}$be a$\mathrm{u}\in$are independentand identically distributed according to a (continuous) curved exponential family withdensity(1.1)$\exp\{\theta(\mathrm{u})_{\mathrm{X}-\psi(}’\theta(\mathrm{u}))\}\mu(d\mathrm{X})$where$\mu(\cdot)$is a carrier measure on

measurable function from$\mathbb{R}^{p}$$\Omega(\subset \mathbb{R}^{q})$$\Psi_{10}=\frac{\mathrm{d}\psi}{\mathrm{d}\theta}$

and$\Psi_{11}=\frac{\mathrm{d}^{2}\psi}{\mathrm{d}\theta \mathrm{d}\theta’}$$E[\mathrm{x}_{i}]=\Psi_{10}(\theta(\mathrm{u}))$((see Amari [5], Section 4.1) andsmooth inis defined as a normalizing constant,$\mathrm{u}\in\Lambda’(\mathrm{u}_{0})$is an$\theta(\cdot)$$\mathbb{R}^{p}$-valued. The real-valued function,$\psi(\theta)=\log\int_{\mathrm{R}^{\mathrm{p}}}\exp(\theta’\mathrm{x})\mu(d\mathrm{X})$. Write$\Theta_{1}=\frac{\mathrm{d}}{\mathrm{d}\mathrm{u}},$$\psi(\theta)$,$\theta(\mathrm{u})$,. It is well-known that$=\eta(\mathrm{u})$, say)

and$\mathrm{C}\mathrm{o}\mathrm{v}[\mathrm{X}_{ii}, \mathrm{x}]=\Psi_{11}(\theta(\mathrm{u}))$($=\Psi_{11}$, say)and that the Fisher information matrix is expressed as$i_{\mathrm{u}}=\Theta_{111^{\ominus_{1}}}’\Psi$

Assume thatLet$\Psi_{11}$andfunctions from$\mathbb{R}^{p}$toare positive definite.

$i_{\mathrm{u}}$$\overline{\mathrm{x}}=\frac{1}{n}\sum_{i=1^{\mathrm{X}_{i}}}^{n}$.

and then$\mathbb{R}^{q}$$E[\overline{\mathrm{x}}]=\eta(\mathrm{u})$, smooth in. Let$N(\eta(\mathrm{u}0))$$\mathrm{g}(\cdot),$$\mathrm{b}_{g}^{(1)}(\cdot)$

anda neighborhood of$\mathrm{b}_{g}^{(2)}(\cdot)$$\eta(\mathrm{u}_{0})$be measurable. The statistics*Institute of Mathematics, University of Tsukuba, Ibaraki 305, Japan

Typeset by$A_{\mathfrak{U}}S- \mathrm{T}\vee \mathrm{H}\mathrm{X}$113, consider aare bias-correction factors, possibly to be zero. For givenclass of possibly bias-corrected Fisher consistent estimators for , defined as$\mathrm{b}_{g}^{(i)}(\overline{\mathrm{X}})’ \mathrm{S}$$\mathrm{b}_{g}^{(i)}(\overline{\mathrm{X}})’ \mathrm{S}$$\mathrm{u}$$\mathcal{F}=\{\mathrm{g}^{*}(\overline{\mathrm{x}})=\mathrm{g}(\overline{\mathrm{X}})-\frac{1}{n}\mathrm{b}(g1)(\overline{\mathrm{X}})-\frac{1}{n}\tau^{\mathrm{b}_{g}^{(}(}2)\overline{\mathrm{x}})|\mathrm{g}(\eta(\mathrm{u}))=\mathrm{u}$We say that
$O$ $(_{n}=^{1}))$$\mathrm{g}^{*}(\overline{\mathrm{x}})$for everyis first-order (or second-order) bias corrected if.

$\mathrm{u}\in N(\mathrm{u}_{0})\}$(or$E[ \mathrm{g}^{*}(\overline{\mathrm{x}})-\mathrm{u}]=o(\frac{1}{n^{2}})$. The bias-correction factors are not unique, and the class$\mathcal{F}$with one correction-

factor may be different from that with another factor. Discussing fourth-order efficiency
here, we will deal with four different bias-correction factors.
We assume that the MLE \^u(1.2)$(\overline{\mathrm{x}})$that maximizes the likelihood function$L( \mathrm{u})=\prod^{n}\exp\{\theta(\mathrm{u})’\mathrm{X}i-\psi(\theta(\mathrm{u}i=1))\}=\exp\{n(\theta(\mathrm{u})’\overline{\mathrm{x}}-\psi(\theta(\mathrm{u}))\}$, witha neighborhood of
exists and that it is Fisher-consistent and smooth inprobability for large . The assumption is met under certain regularity conditionsa$N(\eta(\mathrm{u}0))$$E[\overline{\mathrm{x}}]$$n$$1\mathrm{a}r_{\mathrm{o}}\sigma \mathrm{e}$including strong identifiability (see Kano [16]).
Let

$C$

be a class of all Borel convex sets ofclass of all Borel convex sets of$\mathrm{R}^{q}$

including the origin and letsymmetric about the origin. The classof . We expand concentration probability of
$C$

(1.3)

$\mathrm{R}^{q}$$\mathrm{g}^{*}(\overline{\mathrm{x}})$

about the true parameter vector

depend on. The coefficients
expansion is made with the help of Edgeworth expansions.
$C$

belongs toAn estimator inminimizes$A_{1}$

$C$

or to

$\mathcal{F}$’$A_{k^{\mathrm{S}}}$$C_{0}$

is said to be first-order efficientamong all the estimators infor the class

,

and

$C$

as

(or

$C$

. The

$C_{0}$) iff itfor every $C\in C$ (or $C\in C_{0}$ ). A 1OE estimator
among
for the class (or ) iff it minimizes$C$$\mathcal{F}$$C_{0})$$\mathrm{g}^{*}(\cdot)$$\mathrm{u}$$\mathcal{F}$is said to be second-order efficient
all the 1OE estimators in for every $C\in C$ (or $C\in C_{0}$ )$(2\mathrm{O}\mathrm{E})$$(1\mathrm{O}\mathrm{E})$$\mathrm{u},$be ais a subclass$C_{0}$$\mathrm{P}_{\theta(\mathrm{u})}\{\sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{X}})-\mathrm{u})\in C\}=A_{1}+\frac{1}{\sqrt{n}}A_{2}+\frac{1}{n}A_{3}+\frac{1}{n\sqrt{n}}A_{4}+\cdots$

where$C_{0}$$A_{2}$$C_{0}$

and$;3\mathrm{O}\mathrm{E}$$4\mathrm{O}\mathrm{E}$for the class

$C$

(or

are defined consecutively in the same manner.
is 1OE does not depend on the bias-corrected factors;
Whether an estimator in$\mathcal{F}$and the factorThe$\mathrm{b}_{g}^{(2)}(\overline{\mathrm{X}})$$\mathrm{l}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{n}_{\circ}\sigma$

does not influence upon

distribution of$2\mathrm{O}\mathrm{E}$$\sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{x}})-\mathrm{u})$and$3\mathrm{O}\mathrm{E}$.

(or equivalently

variate normal with mean zero and covariance matrix$G_{1}\Psi_{11}c_{1}’$and hence.

$A_{1}= \int_{c^{\mathit{1}\mathrm{V}_{q}(\mathrm{X}|0,G\Psi}}111c’1)d_{\mathrm{X}}$$\sqrt{n}(\mathrm{g}(\overline{\mathrm{X}})-\mathrm{u})$, where) is multi,$G_{1}= \frac{\mathrm{d}\mathrm{g}(\overline{\mathrm{x}})}{\mathrm{d}\overline{\mathrm{x}}},|_{\overline{\mathrm{x}}=\eta(\mathrm{u})}$114

It is known that
$G_{1}\Psi_{11}G_{1}’\geq i_{\mathrm{u}}^{-1}$and that the equality holds if and only if

first-order derivatives of the MLE \^u$(\overline{\mathrm{x}})$$G_{1}=i_{\mathrm{u}}^{-1} \ominus_{1}’=\frac{\mathrm{d}\tilde{\mathrm{u}}(\overline{\mathrm{x}})}{\mathrm{d}\overline{\mathrm{x}}},|_{\overline{\mathrm{x}}=\eta(\mathrm{u})}$, the matrix of the(see e.g., Takeuchi [33] example 6.5). As a result,

we havefor every $C\in C$ .

$\int_{C}N_{q}(\mathrm{X}|0, c_{1111}\Psi G’)d_{\mathrm{X}}\leq\int_{C}N_{q}(\mathrm{x}|0, i_{\mathrm{u}}-1)d\mathrm{X}$In other words, the MLE an estimators whose first-order derivatives are identical with$d$the MLE are all 1OE for the class .
$C$

Second-order efficiency has been discussed for a class of first-order bias-corrected

estimators$\mathrm{g}^{*}(\overline{\mathrm{x}})$which meet
$E[ \mathrm{g}^{*}(\overline{\mathrm{x}})-\mathrm{u}]=o(\frac{1}{n})$The$2\mathrm{O}\mathrm{E}$.

of the (bias-corrected) MLE was established by showing that the second te$r\mathrm{m}$of the MLE attains the lower bound, which is derive by applying Neymann-Pearson’slemma to a certain testing problem (Pfanzagl [22]; Akahira [1]). After theof the$A_{2}$$d$$2\mathrm{O}\mathrm{E}$

MLE, Takeuchi [33] and Pfanzagl [23] found that the term

estimators, so that 1OE implies$2\mathrm{O}\mathrm{E}$estimators. The basic results on$2\mathrm{O}\mathrm{E}$and that the notion of$A_{2}$

is identical for any 1OE$2\mathrm{O}\mathrm{E}$have been extended to morecannot distinguish 1OE$\circ\sigma e\mathrm{n}e\mathrm{r}\mathrm{a}1$and complicat$e\mathrm{d}$models by many authors including Akahira an Takeuchi $[2,3]$ , Hosoya [15], Taniguchi [34]$d$and Yoshi$d\mathrm{a}$$[36,37]$ .

Consider Takeuchi’s and Pfanzagl’s surprising result via an alternative criterion,
quadratic loss:(1.4)$E[ \{\sqrt{n}(\mathrm{g}^{*}(_{\overline{\mathrm{X}}))\}}-\mathrm{u}\mathrm{t}\sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{X}})-\mathrm{u})\}’]=B_{1}+\frac{1}{\sqrt{n}}B_{2}+\frac{1}{n}B3+\frac{1}{n\sqrt{n}}B_{4}+\cdots$$\mathrm{H}\mathrm{i}_{\mathrm{o}}\sigma \mathrm{h}\mathrm{e}r$,-order efficiency based on the criterion is defined in the same way as that on con-centration probability in (1.3). Since $B_{1}=G_{1}\Psi_{11}c_{1}’$ , the asymptotic covariance matrixof$\mathrm{g}(\overline{\mathrm{x}})$, we make the same conclusion on 1OE as before. We know that $B_{2}=0$ for any

estimators in$\mathcal{F}$, which means that 1OE implies

bias-correction is made and hence that$2\mathrm{O}\mathrm{E}$$2\mathrm{O}\mathrm{E}$. Notice that $B_{2}=0$ whether or notholds for the class of 1OE estimators even

without bias-correction.
Results on
is, the MLE an$3\mathrm{O}\mathrm{E}$$d$similar to those on 1OE hold under fi$r\mathrm{s}\mathrm{t}$-order bias-correction. That

estimators whose second-order derivatives coincide with those of the

115(e.g., Pfanzagl and Wefelmeyer [25]; Akahira and Takeuchi [3]; Amari [4];Taniguchi [35] . Rao [26,27,28], Ghosh an Subramanyam [13], Efron [10] and Eguchi [11]MLE are$3\mathrm{O}\mathrm{E}$

$d$

$)$

have obtained$3\mathrm{O}\mathrm{E}$of the MLE on the basis ofloss or loss of information.

$\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{d}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}_{\dot{\mathrm{C}}}$

3.

Here is an interesting question; does it hold thatmentioned “given that 1OE implies
Ghosh$\mathrm{O}\mathrm{E}$$2\mathrm{O}\mathrm{E}$$[12](\mathrm{P}\mathrm{a}\mathrm{g}\mathrm{e}64)$that$3\mathrm{O}\mathrm{E}$implies$4\mathrm{O}\mathrm{E}$. The proof of that$\mathrm{m}\mathrm{u}s\mathrm{t}$implies, it is$4\mathrm{O}\mathrm{E}$$\mathrm{n}\mathrm{a}t$? On the issue,

ural to conjecture

be very messy.”

The statement is known to be true if one takes quadratic loss as a criterion, because$B_{4}=0$ . This fact would convince us that the conjecture be rue under concentration$t$

probability as well as under quadratic loss.
In this paper, we distinguish between asymptotic efficiency for he class an for theof estimators inclass . The idea was taken by Akahira an Takeuchi [3] to study$3\mathrm{O}\mathrm{E}$$d$$C_{0}$

$d$

$C$

$t$

models with general density functions. We will show that, depending on a choice of bias-correction factors.

$3\mathrm{O}\mathrm{E}$implies$4\mathrm{O}\mathrm{E}$foror for$C$$C_{0}$In Section 2, we give preliminary results on matrix derivatives and multivariate Hermite polynomials. Section 3 describ es main results on asymptotic efficiency up to thefourth-order under sever al types of bias-correction factors. Proofs are$\circ\sigma \mathrm{i}\mathrm{v}\mathrm{e}\mathrm{n}$in Section 4.

The derivation of this paper is so-called formal, and we do not directly prove validityof the Edgeworth expansion (1.3). According to general theory, the formal expansion is
actually valid because (i) the continuity of the curved exponential family (1.2) ensures theanofthe smoothn
Cramer condition:is as umed, so that the moment of any orders exist; (iii) the class$\mathrm{s}$$s$$\psi(\theta)$$e\mathrm{s}\mathrm{s}$$\lim\sup_{||_{\mathrm{S}}||rightarrow\infty}E[e^{i(\mathrm{x}}-\eta(\mathrm{u}))]\mathrm{s}’<1;(\mathrm{i}\mathrm{i})$$C$$d\theta(\mathrm{u})$of all measurable

convex sets meets a boundary condition:
details, see Bhat acharya an Denker [9](Remark 1.4.1 and Theorem 2.1).

$\sup_{C\in C}\int_{(\partial C)}\epsilon q0\mathit{1}\mathrm{V}(\mathrm{x}|, I_{q})d\mathrm{x}=O(\epsilon)(\epsilon\downarrow 0)$

. For

$d$

$t$

2. Preliminary Results
2.1. Matrix derivatives and symmetric tensor

Let$\mathrm{x}=[x_{1}, \ldots , x_{p}]’$function. Put

an$dA(\mathrm{x})=[a_{ij}(\mathrm{x})]$$\frac{\partial A(\mathrm{x})}{\partial x_{k}}=[\frac{\partial a:j(\mathrm{x})}{\partial x_{k}}]$. Definebe an$a_{1}\cross a_{2}$matrix-valued smooth$A( \mathrm{x})=[\frac{\partial A(\mathrm{x})}{\partial x_{1}},$ $\cdots$$\frac{\mathrm{d}A(\mathrm{x})}{\mathrm{d}\mathrm{x}},=\frac{\mathrm{d}}{\mathrm{d}\mathrm{x}},$,$\frac{\partial A(\mathrm{x})}{\partial x_{p}}]$, and. The higher-order (matrix) derivatives are defined inductively byderivatives, see Kano
. For such mat
with$[17,18]$ (cf. Bentler and Lee [8]; Magnus an Neudecker [21]). Here we simply note some$\frac{\mathrm{d}}{\mathrm{d}\mathrm{x}}A(\mathrm{x})=(\frac{\mathrm{d}}{\mathrm{d}\mathrm{x}’}A(\mathrm{x})’)^{l}$$r\mathrm{i}\mathrm{x}$$( \frac{\mathrm{d}}{\mathrm{d}\mathrm{x}’})^{<k>}=\frac{\mathrm{d}}{\mathrm{d}\mathrm{x}’}(\frac{\mathrm{d}}{\mathrm{d}\mathrm{x}’})^{<k}-1>$$( \frac{\mathrm{d}}{\mathrm{d}\mathrm{x}’})^{<1>}=\frac{\mathrm{d}}{\mathrm{d}\mathrm{x}},$$d$formulas on the matrix differentiation. The proof is due to direct calculation (see Kanoand supposeandbe of orderand[18] . Let be a -vector, let$)$$\mathrm{x}$$p$$A(\mathrm{x})$$B(\mathrm{x})$$a_{1}\cross a_{2}$$b_{1}\cross b_{\underline{9}}$116that the following matrix products in the LHS are permissible. We then have$\frac{\mathrm{d}}{d_{\mathrm{X}}},$$[A( \mathrm{x})B(\mathrm{x})]=\frac{\mathrm{d}A(\mathrm{x})}{\mathrm{d}\mathrm{x}’}(I_{p}\otimes B(\mathrm{x}))+A(\mathrm{x})\frac{\mathrm{d}B(\mathrm{x})}{\mathrm{d}\mathrm{x}’}$;$\frac{\mathrm{d}}{\mathrm{d}\mathrm{x}’}[A(\mathrm{x})\otimes B(\mathrm{x})]=(\frac{\mathrm{d}A(\mathrm{x})}{\mathrm{d}\mathrm{x}’}\otimes B(\mathrm{x}))+(A(\mathrm{x})\otimes\frac{dB(\mathrm{x})}{\mathrm{d}\mathrm{x}’})(I\mathrm{i}_{a_{2}p}’\otimes I_{b_{2}})$$\frac{\mathrm{d}}{d\mathrm{t}’}[AB(\mathrm{x}(\mathrm{t}))C]=A\frac{\mathrm{d}B(\mathrm{x})}{\mathrm{d}_{\mathrm{X}}},(\frac{\mathrm{d}\mathrm{x}(\mathrm{t})}{\mathrm{d}\mathrm{t}’}\otimes C)$

where$\mathrm{x}(\mathrm{t})$is an$\mathrm{R}^{p}$-valued function of

formula. Here the matrix$K_{ab}$$\mathrm{t}$

and

$A$

;

,

and

$C$

are constant matrices in the lastof or der ab is a commu ation matrix defined by the relation:$t$$K_{ab}(\mathrm{b}\otimes \mathrm{a})=\mathrm{a}\otimes \mathrm{b}$. See Henderson and Searle [14] an Magnus anfor commutation matrices.

for arbitrary -vector a an $db$-vector$a$Neudecker$[21](\mathrm{P}^{\mathrm{a}}\mathrm{g}\mathrm{e}47)$$d$$\mathrm{b}$$d$k-foldLet us simply write $A^{<k>}=\sim A\otimes\cdots\otimes A,$ -fold right Kronecker (tensor) product of$k$the same matrix$p$-vectors$\mathrm{a}_{1},$$\ldots,$$A$. Note $A^{<0>}=1$ . The symmetric tensor for the Kronecker product of$\mathrm{a}_{k}$is denoted by$\mathit{1}\mathrm{V}_{p^{<k>}}$, which operates as$N_{p^{<k>(\mathrm{a}_{1}\otimes\cdots\otimes}} \mathrm{a}k)=\sum(\mathrm{a}_{\sigma}(1)\otimes\cdots\otimes \mathrm{a}_{\sigma}(k))/k!$,where the summation runs over all permutations $(\sigma(1), \ldots, \sigma(k))$ of 1,$($

Satakebe anof$[30](\mathrm{s}e\mathrm{c}t\mathrm{i}\mathrm{o}\mathrm{n}5.3)$$\mathrm{R}^{q}$$\mathrm{g}(\mathrm{x})$and$\mathrm{S}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{n}\mathrm{b}\mathrm{e}\mathrm{r}_{\circ}\sigma[31](\mathrm{s}_{\mathrm{e}\mathrm{c}}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}1.3)$-valued analytic function defin

about$\mathrm{x}_{0}$$e\mathrm{d}$on a neighborhood of

The matrix of the derivatives,$G_{k}$

with$\mathrm{x}_{0}\in \mathrm{R}^{p}$, has an important property:.

Define(2.2)$\Psi_{kl}=\Psi_{k\ell}(\theta(\mathrm{u}))=(\frac{d}{\mathrm{d}\theta})^{<k>}(\frac{d}{\mathrm{d}\theta’})^{<\ell>}\psi(\theta)|_{\theta}=\theta(\mathrm{u})$$\mathrm{g}(\mathrm{x})$. The Taylor series$G_{k}=( \frac{\mathrm{d}}{\mathrm{d}_{\mathrm{X}’}})^{<k>}\mathrm{g}(\mathrm{x})|_{\mathrm{x}}=\mathrm{x}0$$G_{k}\mathit{1}\mathrm{V}_{p}<k>=G_{k}$, $k)$ . See e.g.

for the symmetric tensor. Let

is expressible in the form:$\mathrm{g}(\mathrm{x})=\sum_{k=0}\frac{G_{k}}{k!}(\infty \mathrm{x}-\mathrm{x}_{0)^{<>}}k$(2.1)$\ldots$117

2.2 Multivariate Hermite polynomialsLet$\varphi(\mathrm{s})=e^{-\mathrm{s}’\Psi \mathrm{s}/2}$

witha -vector an $d\Psi p\cross p$ positive definite. The covariant$p$$\mathrm{s}$Hermite polynomials are defined as$(k\in \mathrm{N})$$H_{k}( \mathrm{s}|\Psi)=[(-\frac{\mathrm{d}}{\mathrm{d}\mathrm{s}})^{<k}>)\varphi(S]/\varphi(\mathrm{s})$,and the contravariant Hermite polynomials are$\tilde{H}_{k}(\mathrm{S}|\Psi)=(\Psi-1)^{<k}>Hk(s|\Psi)$$(k\in \mathrm{N})$.

The first few polynomials are$H_{1}(\mathrm{s}|\Psi)=\Psi S$,$H_{2}(s|\Psi)=(\Psi S)<2>-\mathrm{v}\mathrm{e}\mathrm{c}(\Psi)$$H_{3}(s|\Psi)=(\Psi \mathrm{s})<3>-3\mathit{1}\mathrm{V}_{p}<3>(\mathrm{v}\mathrm{e}\mathrm{c}(\Psi)\otimes\Psi s)$

;

,

and
$\tilde{H}_{1}(s|\Psi)=s$,$\tilde{H}_{2}(\mathrm{s}|\Psi)=s^{<}2>-\mathrm{V}\mathrm{e}\mathrm{c}(\Psi^{-1})$$\tilde{H}_{3}(s|\Psi)=s^{<3>}-3\mathit{1}\mathrm{v}<3>(p\mathrm{V}\mathrm{e}\mathrm{c}(\Psi^{-}1)\otimes s)$Let$\mathrm{z}$have a multivariate normal distribution$\mathit{1}\mathrm{V}_{p}(0, \Psi)$;.

. We then have$E[e^{i\mathrm{s}’\mathrm{z}}]=\varphi(\mathrm{s})$and$E[e^{i<k} \mathrm{s}’\mathrm{z}\mathrm{z}\mathrm{Z}]><\ell>\prime i=k+\ell(-\frac{d}{\mathrm{d}\mathrm{s}’})^{<l>}(-\frac{d}{\mathrm{d}s})^{<k>}\varphi(_{S)}$$=i^{k+\ell}(- \frac{\mathrm{d}}{\mathrm{d}s’})^{<\ell>}[Hk(_{S}|\Psi)\varphi(\mathrm{S})]$

for

$k,$

$l=0,1,2,$$[32](\mathrm{p}\mathrm{a}\mathrm{g}e239)$$\ldots$

According to Barndorff-Nielsen and Cox, we know that

(2.3)$[7](\mathrm{E}\mathrm{q}.

(5.67))$or Takemura, and hence$E[e^{i}H_{k}\mathrm{s}’\mathrm{z}(\mathrm{z}|\Psi^{-}1)]=(i\mathrm{s})^{<}k>_{\varphi(\mathrm{S}})$$E[e^{i\mathrm{z}}\mathrm{s}’\overline{H}_{k}(\mathrm{Z}|\Psi^{-}1)]=(i\Psi s)^{<}k>_{\varphi(}s)$.

We further note that
(2.4)

$E[e^{i\mathrm{s}’\mathrm{z}} \overline{H}k(_{\mathrm{Z}}|\Psi-1)\mathrm{z}^{<\ell>\prime}]=i^{k+p}(-\frac{\mathrm{d}}{\mathrm{d}s’})^{<\ell}>[(\Psi s)<k>\varphi(s)]$Recall that
Let us replace$\mathrm{z}$$\mathrm{x}_{1},$$\ldots,$with$\mathrm{x}_{n}$are an$\mathrm{i}.\mathrm{i}.\mathrm{d}$.

. sample from the curved exponential family (1.1).

$\mathrm{z}_{n}=\sqrt{n}(\overline{\mathrm{x}}-\eta(\mathrm{u}))$, and then the formulas abov still hold provided
$e$

118

that we substitutfor$\mathrm{e}\Psi_{11}$$\Psi$

and add $o(1)$ ; for example, the corr$e\mathrm{s}\mathrm{p}_{\mathrm{o}\mathrm{n}\mathrm{d}\mathrm{i}}\mathrm{n}_{\circ}\sigma$

result to (2.3)

is
(2.5)

with

$E[e\tilde{H}i\mathrm{s}’\mathrm{z}_{\hslash k}(\mathrm{z}n|\Psi_{11}-1)]=(i\Psi 11\mathrm{S})<k>(\varphi \mathrm{s})+o(1)$$\varphi(s)=e^{-\mathrm{s}’\Psi_{11}\mathrm{s}/2}$

3. Main Result.

$s$

We begin with second-order efficiency. Let$( \frac{\mathrm{d}}{\mathrm{d}\overline{\mathrm{x}}}, )^{<k>}\mathrm{g}(\overline{\mathrm{x}})|\overline{\mathrm{x}}=\eta(\mathrm{u})$expandin$\mathrm{g}(\overline{\mathrm{x}})$$\mathcal{F}$and$\mathrm{z}_{n}=\sqrt{n}(\overline{\mathrm{x}}-\eta(\mathrm{u})),$$\overline{G}_{k}=\overline{G}_{k}(\eta(\mathrm{u}))=(\frac{\mathrm{d}}{\mathrm{d}\overline{\mathrm{x}}’})^{<k>}$and the MLE \^u$(\overline{\mathrm{x}})$\^u$(\overline{\mathrm{x}})|\overline{\mathrm{x}}=\eta(\mathrm{u})$$G_{k}=G_{k}(\eta(\mathrm{u}))=$

for $k=0,1,2,$

,

\^u

.

(3.1)
$( \overline{\mathrm{x}})=\mathrm{u}+\frac{1}{\sqrt{n}}\overline{G}1\mathrm{Z}n+\frac{1}{2n}\overline{G}_{2}\mathrm{z}^{<}n2>+\frac{1}{6n\sqrt{n}}\overline{G}3\mathrm{z}_{n}^{<}+3>\ldots$$( \frac{\mathrm{d}}{\mathrm{d}\theta})^{<2>}\psi(\theta)|_{\theta=\eta}(\mathrm{u})$To discussWe

stochastically in the form:$\mathrm{g}(\overline{\mathrm{x}})=\mathrm{u}+\frac{1}{\sqrt{n}}G1\mathrm{Z}n+\frac{1}{2n}G2\mathrm{z}+\frac{1}{6n\sqrt{n}}nG<2>\mathrm{z}_{n}^{<}3+3>\ldots$

The first-order bias is given as$..$ ,$E[ \mathrm{g}(\overline{\mathrm{x}})-\mathrm{u}]=\frac{1}{2n}G_{2}\Psi_{20}+O(\frac{1}{n^{2}})$

with$\Psi_{20}=\mathrm{v}\mathrm{e}\mathrm{c}(\Psi_{11})=$.

$2\mathrm{O}\mathrm{E}$, we take$\mathrm{b}_{g}^{(1)}(\overline{\mathrm{x}})=\frac{1}{2}G_{2}\overline{\Psi}_{20}$and$\mathrm{b}_{g}^{(2)}(\overline{\mathrm{X}})$

to be unspecified. Then,,$\mathrm{g}^{*}(\overline{\mathrm{x}})=\mathrm{g}(\overline{\mathrm{x}})-.\frac{1}{2n}G\overline{2\Psi}20-\frac{1}{n^{2}}\mathrm{b}_{g}(2)(\overline{\mathrm{x}})$(3.2)$\hat{\mathrm{u}}^{*}(\overline{\mathrm{x}})=\hat{\mathrm{u}}(\overline{\mathrm{x}})-\frac{1}{2n}\overline{G}_{2}\overline{\Psi}_{2}0-\frac{1}{n^{2}}\mathrm{b}^{(}\dot{u}2)(\overline{\mathrm{x}})$,and hence(3.3)$E[ \sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{X}})-\hat{\mathrm{u}}^{*}(\overline{\mathrm{X}}))]=O(\frac{1}{n\sqrt{n}})$Here we do not necessarily specify how to estimate

TakeuchiSee also Ghosh$[33](\mathrm{P}^{\mathrm{a}}\mathrm{g}\mathrm{e}185)$$[12](\mathrm{S}\mathrm{e}\mathrm{C}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}6.4)$

Theorem 1.
$A_{\mathit{8}}sume$

thatand Pfanzagl$\mathrm{g}^{*}(\overline{\mathrm{x}})$Let$G_{2}\Psi_{20}$.

.

$[23](\mathrm{s}_{\mathrm{e}}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}7)$

showe the following theorem.
$d$.

$\mathrm{g}^{*}(\overline{\mathrm{x}})\in \mathcal{F}$and the$MLE\hat{\mathrm{u}}^{*}(\overline{\mathrm{X}})\in \mathcal{F}$be$bia\mathit{8}$-corrected as in (3.2).

is 1 $OE$ . Then,$P_{\theta(\mathrm{u})}[ \sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{x}})-\mathrm{u})\in C]=P_{\theta(\mathrm{u})}[\sqrt{n}(\hat{\mathrm{u}}^{*}(\overline{\mathrm{x}})-\mathrm{u})\in C]+o(\frac{1}{\sqrt{n}})$, that is, under thefor every $C\in C$ andestimators arefor the class .
$\mathrm{u}\in\Lambda^{r}(\mathrm{u}_{0})$$\mathit{2}OE$$bia\mathit{8}$-correction in$(\mathit{3}.\mathit{2})_{f}$any 1 $OE$

$C$

Can we say something about$2\mathrm{O}\mathrm{E}$gives a certain answer to the question.

, without bias-correction? The following theorem

119

Let \^u

Theorem 2.
$i_{\mathit{8}}\mathit{1}OE$$(\overline{\mathrm{x}})$be the $MLE$ . Assume that a$Fi\mathit{8}herconSi_{\mathit{8}}tent$

estimator$\mathrm{g}(\overline{\mathrm{x}})$. Then,$P_{\theta(\mathrm{u})}[ \sqrt{n}(\mathrm{g}(\overline{\mathrm{x}})-\mathrm{u})\in C]=P_{\theta(\mathrm{u})}[\sqrt{n}(\hat{\mathrm{u}}(\overline{\mathrm{x}})-\mathrm{u})\in C]+o(\frac{1}{\sqrt{n}})$for every$C\in C_{0}$and$\mathrm{u}\in N(\mathrm{u}0)$, that is, any 1 $OEestimator\mathit{8}$ are$\mathit{2}OE$

for the class$C_{0}$.

Some authors must have noticed the result of Theorem 2, but no explicit atement
for the class of no bias-corrected estimators has appeared. The next theregarding$\mathrm{s}t$$2\mathrm{O}\mathrm{E}$orem on$3\mathrm{O}\mathrm{E}$is well-known (e.g., Pfanzagl and Wefelmeyer [25] Theorem 1; Akahira and

Takeuchi [3] Theorem 5.1.6; Ghosh [12] Chapand the
Theorem 3. Let
that

).

$MLE\hat{\mathrm{u}}^{*}(\overline{\mathrm{X}})\in \mathcal{F}$$\mathrm{g}^{*}(\overline{\mathrm{x}})\in \mathcal{F}$$A\mathit{8}sume$$t\mathrm{e}r6$

be bias-corrected$a\mathit{8}$in (3.2).

is 1 $OE$ . Then,$\mathrm{g}^{*}(\overline{\mathrm{x}})$$P_{\theta(\mathrm{u})}[ \sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{X}})-\mathrm{u})\in C]=P\theta(\mathrm{u})[\sqrt{n}(\hat{\mathrm{u}}^{*}(\overline{\mathrm{X}})-\mathrm{u})\in C]-\frac{1}{n}\triangle_{\mathrm{u}}2,o\mathrm{g}^{*,c+(}\frac{1}{n})$for every
for any$C\in C_{0}$$C\in C_{0}$and$\mathrm{u}\in N(\mathrm{u}_{0})$if and only if
for the class$G_{2}=\overline{G}_{2}$$e\mathit{8}timat\mathit{0}ri_{\mathit{8}}\mathit{3}OE$

The actual form of
3, we see that

, where$3\mathrm{O}\mathrm{E}$of$C_{0}$$\triangle_{\mathrm{u},\mathrm{g}^{*},C}^{2}$$\mathrm{g}^{*}(\overline{\mathrm{x}})$$\triangle_{\mathrm{u},\mathrm{g}^{*},C}^{2}$, that$is_{f}$if and only ifwill bemeans$\circ\sigma \mathrm{i}\mathrm{v}\mathrm{e}\mathrm{n}$$G_{1}=\overline{G}_{1}$is nonnegative. Further,under the bias-correction in (3.2),$G_{2}=\overline{G}_{2}$and$G_{2}=\overline{G}_{2}$.

$4\mathrm{O}\mathrm{E}$. The distribution of$t$

tr ation probability, an hence how to estimate$G_{2}\Psi_{20}$$o( \frac{1}{n\sqrt{n}})$$\sqrt{n}(\mathrm{b}_{g}^{(1)}-$of concen-

is quite important to study$4\mathrm{O}\mathrm{E}$Consider the following different types of bias-corrections (or different choices ofan$d\mathrm{b}_{g}^{(2)}$

1 $OE$.

contribut es to he terms of order

$d$

$a$

in (4.13) in the proof. According to Theorem

Now we are in a position to state some results on$\frac{1}{2}G_{2}\Psi_{20})=\sqrt{n}(\frac{1}{2}G\overline{2\Psi}20-\frac{1}{2}G2\Psi 20)$$\triangle_{\mathrm{u},\mathrm{g}^{*},C}^{2}=0$.

$\mathrm{b}_{g}^{(1)}$):

Case (I): first-order bias-correction only$\mathrm{g}^{*}(\overline{\mathrm{x}})=\mathrm{g}(\overline{\mathrm{X}})-\frac{1}{2n}c\overline{2\Psi}20$;

Case (II): second-order bias-correction (1), the historical one (see$e_{\mathrm{o}}.\sigma.$, Rao et.al. [29])$\mathrm{g}^{*}(\overline{\mathrm{x}})=\mathrm{g}(\overline{\mathrm{X}})-\frac{1}{2n}G_{2}(\eta(\mathrm{g}(\overline{\mathrm{X}})))\Psi_{20}(\theta(\mathrm{g}(\overline{\mathrm{x}})))-\frac{1}{n^{2}}\mathrm{b}^{(_{\sim}}’)(g)\overline{\mathrm{x}}$so that$E[ \mathrm{g}^{*}(\overline{\mathrm{x}})-\mathrm{u}]=o(\frac{1}{n^{3}})$;

Case (III): second-order bias-correction (2), suggested by Kano $[17,19]$$\mathrm{g}^{*}(\overline{\mathrm{x}})=\mathrm{g}(\overline{\mathrm{X}})-\frac{1}{2n}G_{2}(\overline{\mathrm{X}})\Psi_{2}0(\theta(\mathrm{g}(_{\overline{\mathrm{X}})}))-n=^{1}\mathrm{b}((g\overline{\mathrm{x}}2))$

so thatCas$e(\mathrm{I}\mathrm{V})$$E[ \mathrm{g}^{*}(\overline{\mathrm{x}})-\mathrm{u}]=o(\frac{1}{n^{3}})$;: second-order bias-correction (3), based on Amari$[5](\mathrm{E}\mathrm{q}.

$\mathrm{g}^{*}(\overline{\mathrm{x}})=\mathrm{g}(\overline{\mathrm{X}})-\frac{1}{2n}G_{2}(\overline{\mathrm{X}})\Psi 2\mathrm{o}(\Psi_{1}-1(0\overline{\mathrm{X}}))-\frac{1}{n^{2}}\mathrm{b}_{g}((2))\overline{\mathrm{x}}$(4.27))$

120

so that
Here$\theta=\Psi_{10}^{-1}(\eta)$$E[ \mathrm{g}^{*}(\overline{\mathrm{x}})-\mathrm{u}]=O(\frac{1}{n^{3}})$is the inverse function of

(3.4)

.

$\eta=\Psi_{10}(\theta)$. For Cases$E[ \sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{X}})-\hat{\mathrm{u}}^{*}(\overline{\mathrm{X}}))]=O(\frac{1}{n^{2}\sqrt{n}})$$(\mathrm{I}\mathrm{I})-(\mathrm{I}\mathrm{v})$,,whereas (3.3) holds for Case (I).
We now state the main theorem.

Let

Theorem 4.

$\mathrm{g}^{*}(\overline{\mathrm{x}})\in \mathcal{F}$the four cases above. Assume that
(3.5)

and the$\mathrm{g}^{*}(\overline{\mathrm{x}})$is$MLE\hat{\mathrm{u}}^{*}(\overline{\mathrm{X}})\in \mathcal{F}$$\mathit{3}OE$

be bias-corrected as in one of. $(i)$ Under Cases (I) and (II),$P_{\theta(\mathrm{u})}[ \sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{X}})-\mathrm{u})\in C]=P_{\theta(\mathrm{u})[}\sqrt{n}(\hat{\mathrm{u}}^{*}(\overline{\mathrm{x}})-\mathrm{u})\in C]+o(\frac{1}{n\sqrt{n}})$for every$C\in C_{0}$and$\mathrm{u}\in\Lambda’(\mathrm{u}_{0})$, that is, any$\mathit{3}OE$

estimators are 4 $OE$ for the(ii) Under Cases (III) and (IV), (3.5) holds for the classare 4 $OE$ for the class

$C$

$C$

, that is, any$\mathit{3}OE$$cla\mathit{8}SC_{0}$.

estimators.

Kano $[17,19]$ and Amari [6] have established$5\mathrm{O}\mathrm{E}$of the MLE under the bias-

correction (III) or (IV) on the basis of quadratic loss, concentration probability and information loss, respectively.
4. Proofs
4.1 Auxiliary$\mathrm{r}\mathrm{e}s$ults

Differentiation of both sides in the equality$G_{k}(\Psi_{1}0(\theta(\mathrm{u})))=\overline{G}_{k}(\Psi_{10}(\theta(\mathrm{u})))$

w.r.t.

$\mathrm{u}’$and use of (2.1) shows that
(4.1a)$G_{1}=\overline{G}_{1}\Rightarrow(G_{2^{-}}\overline{G}_{2})(\Psi 11\ominus_{1}\otimes I_{p})=(G_{2^{-}}\overline{G}_{2})(I_{p}\otimes\Psi_{11}\ominus_{1})=0$,(4.1b)$G_{2}=\overline{G}_{2}\Rightarrow(G_{3}-\overline{G}_{3})(\Psi_{1}1\ominus 1\otimes I_{p^{2}})=(G_{2}-\overline{c}_{2})(I_{p}\otimes\Psi_{11}\ominus_{1}\otimes I_{p})$$=(G_{2^{-}}\overline{G}2)(I2\otimes\Psi p11\ominus_{1})$

$=0$

.

These are key relations to our derivation here.

Let$\mathrm{t}\in \mathrm{R}^{q}$and$\varphi(\mathrm{s})=e^{-\mathrm{s}’\Psi_{1}\mathrm{s}}\iota/2$. Recall that$\overline{G}_{1}=i_{\mathrm{u}}^{-1}\ominus_{1}’$. It follows from (2.3) and(4.1a) that$(G_{2}-\overline{c}_{2})E[ei\mathrm{t}^{!}\overline{G}1\mathrm{z}n(\mathrm{z}_{n0}^{<2>}-\Psi_{2})]=-(G_{2}-\overline{G}2)(\Psi_{111}\ominus i_{\mathrm{u}}^{-1})^{<2>_{\varphi}}(\overline{G}\mathrm{t})\prime 1+o(1)$(4.2a)and hence$=o(1)$

121(4.2b)$(G_{2}-\overline{G}_{2})E[e^{i\mathrm{t}}\hslash \mathrm{z}_{n}1\tilde{G}_{1}\mathrm{z}<2>(’=c2^{-\overline{c})\Psi \mathrm{u}}220^{e^{-}}+o(\mathrm{t}^{l}i-1\mathrm{t}/21)$.

Using (2.4), we have$E[e^{i\mathrm{t}’} \overline{c}_{1}\mathrm{z}n(\mathrm{Z}_{n2}2<>-\Psi 0)\mathrm{z}_{n}’]=i\frac{\mathrm{d}}{\mathrm{d}\mathrm{s}’}[(\Psi 11^{\mathrm{S}})^{<}2>\varphi(s)]|_{\mathrm{s}=\overline{G}_{1}\mathrm{t}}’+o(1)$,(4.3a)$=ie^{-\mathrm{t}’}\mathrm{u}i^{-}\iota \mathrm{t}/2N_{p}<2>[(2\Psi_{11}-\Psi_{11}\overline{c}_{1}’\mathrm{t}\mathrm{t}’\overline{G}1\Psi_{11})\otimes(\Psi_{11}\overline{c}\mathrm{t})/1]+o(1)$and hence use of (4.1a) results in(4.3b)$(G_{2}-\overline{G}2)E[ei\mathrm{t}\overline{G}_{1n}\prime \mathrm{Z}(\mathrm{z}_{n}-<2>\Psi_{20})\mathrm{Z}_{n}]/=o(1)$.

In a similar manner, we have from (2.4) and (4.1a).

$(G_{2}-^{\overline{c}_{2})}E[e(i\mathrm{t}^{\prime\overline{c}}1\mathrm{z}n\mathrm{Z}^{<2>}-\Psi 20)n\mathrm{z}]n<2>’$$=(G_{2}/-^{\overline{c}}2)( \frac{\mathrm{d}}{\mathrm{d}s’})^{<2}>[(\Psi_{11^{\mathrm{S}}})^{<}2>\varphi(s)]|_{\mathrm{s}=\overline{c}_{1}\mathrm{t}}’+o(1)$$=2e^{-\mathrm{t}’}(i_{\mathrm{u}}^{-}1/2G\mathrm{t}-\overline{c}22)\Psi<2>+\mathit{0}11(1)$,from which, it follows that
(4.4a)$(c_{2}-\overline{G}_{2})E[e(i\mathrm{t}’\overline{c}1\mathrm{z}_{n}<2\mathrm{z}n-\Psi 20>)\mathrm{Z}^{<2}]n2=>/\overline{G}’o(1)$and$(G_{2}-\overline{G}_{2})E[en\mathrm{z}(i\mathrm{t}^{l}\overline{G}\iota<2>-\mathrm{Z}_{n}\Psi 20)(\mathrm{Z}_{n}^{<2>}-\Psi_{2}0)’](G_{2^{-}}\overline{G}_{2})$(4.4b)We have used that’$=2e^{-\mathrm{t}i_{\mathrm{u}}\mathrm{t}/>}’-12(c2^{-\overline{c}_{2})}\Psi_{1}<2(1c_{2}-\overline{c}2)’+o(1)$$\overline{G}_{2}=B(\ominus_{1}/\otimes I_{p})N_{p}<2>$,

for some $B$ (see Kano [17] Lemma 3.3) anhave applied (4.1a) again to get (4.4a).

Using (2.4) and (4.1b) again, we have’$(G_{3}-\overline{c}3)E[ei\mathrm{t}’\overline{c}_{1}\mathrm{z}_{\hslash}(_{\mathrm{Z}_{n}}<3>-3(\Psi_{2}0\otimes \mathrm{z}_{n})\mathrm{Z}_{n}]<\ell>$$=i^{3+\ell}(G3-^{\overline{c})}3(- \frac{\mathrm{d}}{\mathrm{d}s’})^{<\ell>}[(\Psi_{11^{\mathrm{S}}})^{<}3>\varphi(s)]|_{\mathrm{s}=\overline{c}_{1}’\mathrm{t}}+o(1)$(4.5)

for $\ell=0,1,2$ .

$=o(1)$$d$122

4.2 Proofs of the theorems

Theorem 1 is a known result. We will give a proof of Theorem 2 and then note thedistinction between the proofs of Theorems 1 and 2.
Since
(4.6)

$G_{1}=\overline{G}_{1}$for any 1OE estimators$\mathrm{g}(\overline{\mathrm{x}})$, the difference is expressed from (3.1) as$\sqrt{n}(\mathrm{g}(\overline{\mathrm{X}})-\hat{\mathrm{u}}(\overline{\mathrm{X}}))=\frac{1}{2\sqrt{n}}(G_{2^{-}}\overline{G}2)\mathrm{Z}+np(<2>\frac{1}{\sqrt{n}}o)=O_{p}(\frac{1}{\sqrt{n}})$from which, the characteristic function of stan ardized$d$$E[e^{i\mathrm{t}\sqrt{n}(}-\mathrm{u}]\mathrm{g}(\overline{\mathrm{x}})\rangle=E’[e^{i}-\mathrm{u}.

$\mathrm{g}(\overline{\mathrm{x}})$,can be evaluated in the form:

e-\dot{\mathrm{u}}\overline{\mathrm{x}}]\mathrm{t}’\sqrt{n}(\dot{\mathrm{u}}(\overline{\mathrm{x}}))i\mathrm{t}’\sqrt{n}(\mathrm{g}(\overline{\mathrm{x}})())$$=E[e^{i\mathrm{t}’\sqrt{n}(}- \mathrm{u})\{\hat{\mathrm{u}}(\overline{\mathrm{x}})1+i\mathrm{t}’\sqrt{n}(\mathrm{g}(\overline{\mathrm{x}})-\hat{\mathrm{u}}(\overline{\mathrm{x}}))+O_{p}(\frac{1}{n})\}]$$=E[e^{i\mathrm{t}’\sqrt{n}(}-]\hat{\mathrm{u}}(\overline{\mathrm{x}})\mathrm{u})$

(4.7)

an

$+(i \mathrm{t})’E[e-\mathrm{u})\sqrt{n}i\mathrm{t}’\sqrt{n}(\hat{\mathrm{u}}(\overline{\mathrm{x}})(\mathrm{g}(\overline{\mathrm{x}})-\hat{\mathrm{u}}(\overline{\mathrm{x}}))]+o(\frac{1}{\sqrt{n}})$,$d$$E[e^{i\mathrm{t}\sqrt{n}(}’ \hat{\mathrm{u}}(\overline{\mathrm{x}})-\mathrm{u})\sqrt{n}(\mathrm{g}(\overline{\mathrm{x}})-\hat{\mathrm{u}}(\overline{\mathrm{x}}))]=\frac{1}{2\sqrt{n}}(i\mathrm{t})’(G_{2}-\overline{G}_{2})E[e]i\mathrm{t}’\overline{G}\iota \mathrm{z}_{n_{\mathrm{Z}}}<2>On+(\frac{1}{\sqrt{n}})$$= \frac{e^{-\mathrm{t}’i_{\mathrm{u}}^{-\iota}\mathrm{t}/2}}{2\sqrt{n}}(i\mathrm{t})/(c_{2}-\overline{G}_{2})\Psi_{2}0+o(\frac{1}{\sqrt{n}})$in view of (4.2b). Thus, we obtain the difference between the two characteristic functionsof$\mathrm{g}(\overline{\mathrm{x}})$and \^u$(\overline{\mathrm{x}})$as$E[e^{i\mathrm{t}’\sqrt{n}(}-] \mathrm{g}(\overline{\mathrm{x}})\mathrm{u})-E[e^{i}-\mathrm{u})]\mathrm{t}’\sqrt{n}(\hat{\mathrm{u}}(\overline{\mathrm{x}})=\frac{e^{-\mathrm{t}’i_{\mathrm{u}}^{-}\mathrm{t}/2}1}{2\sqrt{n}}(i\mathrm{t})’(c_{2}-\overline{c}_{2})\Psi_{2}0+o(\frac{1}{\sqrt{n}})$.

Inversion of the characteristic functions to distribution functions results in$\mathrm{P}_{\theta(\mathrm{u})}[\sqrt{n}(\mathrm{g}(\overline{\mathrm{X}})-\mathrm{u})\in C]-\mathrm{P}\theta(\mathrm{u})[\sqrt{n}(\hat{\mathrm{u}}(\overline{\mathrm{X}})-\mathrm{u})\in C]$(48)$= \frac{1}{2\sqrt{n}}\Psi_{20}’(G_{2}-\overline{G}_{2})’\int_{c^{\mathrm{X}\mathit{1}\mathrm{V}_{q}}}(\mathrm{x}|0, i_{\mathrm{u}}-1)d\mathrm{x}+o(\frac{1}{\sqrt{n}})$for every measurable set$C\subset \mathrm{R}^{q}$meeting a certain condition on the boundary$\partial C$, whichis satisfied by convex sets (see Bhattacharya and Denker [9] Remark 1.4.1 and Theorem2.1). The integran in (4.8) is an odd function in$d$$\mathrm{x}$. The RHS in (4.8) vanishes when $C$

is symmetric about the origin. As a result, we have$\mathrm{P}_{\theta(\mathrm{u})}[\sqrt{n}(\mathrm{g}(\overline{\mathrm{X}})-\mathrm{u})\in C]=\mathrm{P}_{\theta(\mathrm{u})}[\sqrt{n}(\hat{\mathrm{u}}(\overline{\mathrm{X}})-\mathrm{u})\in C]+o(\frac{1}{\sqrt{n}})$for every $C\in C_{0}$ . This proves Theorem 2.

Q.E.D.

123

When the bias-correction in (3.2) is made, (4.6) becomes(4.9)$\sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{x}})-\hat{\mathrm{u}}*(\overline{\mathrm{X}}))=\frac{1}{2\sqrt{n}}(c2-\overline{G}_{2})(_{\mathrm{Z}}n-\Psi_{20}<2>)+O_{p}(\frac{1}{\sqrt{n}})$,and hence we get(4.10)$E[e^{i\mathrm{t}’\sqrt{n}(\dot{\mathrm{u}}^{*}}( \overline{\mathrm{x}})-\mathrm{u})\sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{x}})-\hat{\mathrm{u}}^{*}(\overline{\mathrm{X}}))]=o(\frac{1}{\sqrt{n}})$by the virtue of (4.2a). Consequently, the difference between the characteristic functions
Q.E.D.
. This proves Theorem 1.
is of order$o( \frac{1}{\sqrt{n}})$To prove Theorems 3 an $d4$ , we need to investigat the expectation (4.10), a sort ofcross terms, up to higher-order. There are useful lemmas to do so. The basic idea of the$e$lemmas were $or$ iginated in Kano [19], who studies fifth-order efficiency. The proof will begiven in Appendix.
Lemma 1. Assume that

exponential family (1.1). Let
random q- andparameter$q’$$\mathrm{u}\in\Omega$$\mathrm{x}_{1},$$\ldots$,$\mathrm{x}_{n}$a.re an$i.i.d$. random sample from the curved$T_{n}^{(1)}=T_{n}^{(1)}(\mathrm{x}_{1}, \ldots, \mathrm{x}_{n})$and$T_{n}^{(2)}=\tau_{n}^{(2)}(\mathrm{X}_{1}, \ldots , \mathrm{x}_{n})$-vectors which may depend on the sample size
. Let$\phi=\phi(\mathrm{u}, \mathrm{t})=E[e^{i\mathrm{t}\sqrt{n}(T_{n}^{(1}-\mathrm{u})}\tau_{n}]’)(2)$$n$. Then,bebut do not depend on the$\phi$

satisfies the following

system of partial differential equations:(4.11)$\frac{d\phi}{d\mathrm{t}},$$=-\{(\mathrm{t}\prime i_{\mathrm{u}}^{-1})\otimes\phi\}+C(\mathrm{u}, \mathrm{t})$

,

where
$C( \mathrm{u}, \mathrm{t})=\frac{i}{\sqrt{n}}\frac{d\phi}{d\mathrm{u}},$Lemma 2. Let

solution to$(\mathit{4}\cdot \mathit{1}\mathit{1})$$i_{\mathrm{u}}^{-1}+iE[ei\mathrm{t}’\sqrt{n}(\tau_{n}(\iota)-\mathrm{u})T(2)\{n\sqrt{n}(\tau^{(1)}n-\mathrm{u})-\overline{G}1\mathrm{Z}_{n}\}’]$$\mathrm{t}_{k}=[t_{1}, \ldots , t_{k}, 0, \ldots , 0]’$and.

$C(\mathrm{u}, \mathrm{t})=[\mathrm{c}_{1}(\mathrm{u}, \mathrm{t}), \ldots, \mathrm{c}_{q}(\mathrm{u}, \mathrm{t})]$. The

is expressible as$\phi(\mathrm{u}, \mathrm{t})=e-\mathrm{t}i_{\mathrm{u}}l-1\mathrm{t}/2(\phi(\mathrm{u}, 0)+\sum\int^{t}\mathrm{o}(\mathrm{C}k\mathrm{u}, \mathrm{t}_{k})e\mathrm{t}’k=1qkikk/2di_{\mathrm{u}}^{-1}\mathrm{t})k$.

Here we shall use these lemmas to provide a proof of Theorem 3, which is different
and simpler compared to the existing proofs. In a similar way as in (4.7), we have
(4.12)$E[e^{i\mathrm{t}\sqrt{n}((}-)]\mathrm{g}\overline{\mathrm{x}})\mathrm{u}-’*E[e^{i}-\mathrm{u})]\mathrm{t}’\sqrt{n}(\dot{\mathrm{u}}(*\overline{\mathrm{x}})$$=(i\mathrm{t})’E[e-\mathrm{u})\sqrt{n}i\mathrm{t}’\sqrt{n}(\overline{\mathrm{u}}(*(\overline{\mathrm{x}})\mathrm{g}(_{\overline{\mathrm{X}}}*)-\hat{\mathrm{u}}^{*}(\overline{\mathrm{x}}))]$$+ \frac{1}{?}.(i\mathrm{t})’E[e^{i}\sqrt{n}\mathrm{t}’\sqrt{n}(\tilde{\mathrm{u}}(*\overline{\mathrm{x}})-\mathrm{u})(\mathrm{g}*(\overline{\mathrm{X}})-\hat{\mathrm{u}}^{*}(\overline{\mathrm{x}}))\sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{x}})-\hat{\mathrm{u}}^{*}(\overline{\mathrm{X}}))/](i\mathrm{t})$$+o( \frac{1}{n})$

.

124

The first term in the RHS in (4.12) is of order$o( \frac{1}{n})$as will be shown later an so, use of$d$(4.9) and (4.4b) results in$E[e^{i\mathrm{t}’\sqrt{n}}.-\mathrm{u}](\mathrm{g}(\overline{\mathrm{x}}))-E[e^{i}-]\mathrm{t}’\sqrt{n}(\hat{\mathrm{u}}(*\overline{\mathrm{x}})\mathrm{u})$$= \frac{e^{-\mathrm{t}’i_{\mathrm{u}}^{-1}}\mathrm{t}/2}{4n}(i\mathrm{t})’(G_{2}-\overline{c}2)\Psi<2>(112)’c-^{\overline{c}_{2}}(i\mathrm{t})+o(\frac{1}{n})$.

Inverting the characteristic functions to distribution functions, we have$\mathrm{P}_{\theta(\mathrm{u})}[\sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{x}})-\mathrm{u})\in C]=\mathrm{P}_{\theta(\mathrm{u})}[\sqrt{n}(\hat{\mathrm{u}}^{*}(\overline{\mathrm{x}})-\mathrm{u})\in C]-\frac{1}{n}\triangle_{\mathrm{u},\mathrm{g}^{*},C}^{2}+o(\frac{1}{n})$for any Borel convex set$C\subset \mathrm{R}^{q}$, where$\triangle_{\mathrm{u},\mathrm{g}^{*,c}}^{2}=\frac{1}{4}\mathrm{v}\mathrm{e}\mathrm{c}(c2-\overline{G}_{2})$.

(4.13)If$C$’$( \Psi_{11}^{<2>}\otimes\int_{C}(i_{\mathrm{u}}-i_{\mathrm{u}^{\mathrm{X}\mathrm{X}’}}i_{\mathrm{u}})l\mathrm{v}(q\mathrm{x}|0, i_{\mathrm{u}}^{-1})d\mathrm{X})\mathrm{V}\mathrm{e}\mathrm{c}(G_{2}-\overline{G}_{2})$.

is symmetric about the origin (i.e., $C\in C_{0}$ ), the integral above is shown to benonnegative; and it is positive, assuming further that the interior point of$C$

is not empty(see Pfanzagl [24] Lemma 13.2.4). Thus, Theorem 3 follows, provided that$\phi=\phi(\mathrm{u}, \mathrm{t})=E[e^{i\mathrm{t}’\sqrt{n}}-\mathrm{u}\sqrt{n}(\hat{\mathrm{u}}(*\overline{\mathrm{x}}))(\mathrm{g}(*\overline{\mathrm{X}})-\hat{\mathrm{u}}^{*}(\overline{\mathrm{X}}))]=o(\frac{1}{n})$Applying Lemma 1
evaluate$C(\mathrm{u}, \mathrm{t})$$[T_{n}^{(1)}=\hat{\mathrm{u}}^{*}(\overline{\mathrm{x}}), T_{n}^{(2)}=\sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{x}})-\hat{\mathrm{u}}^{*}(\overline{\mathrm{x}}))]$.

and using (4.9), we canin Lemma 1 as$C( \mathrm{u}, \mathrm{t})=\frac{i}{\sqrt{n}}\frac{\mathrm{d}\phi}{\mathrm{d}\mathrm{u}’}i_{\mathrm{u}}^{-1}+\frac{i}{4n}(G2-\overline{c}2)E[en(i\mathrm{t}^{\prime\overline{c}_{1}2>}\mathrm{z}\mathrm{z}_{n}-<\Psi_{20})(\mathrm{z}_{n2}-<2>\Psi 0)/]\overline{c}+o(/2\frac{1}{n})$$= \frac{i}{\sqrt{n}}\frac{\mathrm{d}\phi}{\mathrm{d}\mathrm{u}’}i_{\mathrm{u}}^{-1}+o(\frac{1}{n})$in view of (4.4a). We have from Lemma 2 and a consequence of the bias-correction$\phi=e^{-\mathrm{t}’i_{\mathrm{u}}\mathrm{t}}-1/2(E[\sqrt{n}(\mathrm{g}(*\overline{\mathrm{X}})-\hat{\mathrm{u}}^{*}(\overline{\mathrm{x}}))]+\frac{i}{\sqrt{n}}\sum^{q}i=1\int^{1_{k}}0\frac{d\phi(\mathrm{u},\mathrm{t}_{k})}{d\mathrm{u}’}e^{\mathrm{t}_{k}’i_{\mathrm{u}}^{-\iota}\mathrm{t}_{k}}/9\sim dt_{k)}i\mathrm{u}-1+o(\frac{1}{n})$$= \frac{ie^{-\mathrm{t}i_{\mathrm{u}}^{-\iota}}\prime \mathrm{t}/2}{\sqrt{n}}\sum_{i=1}^{q}\int_{0}^{t}k\frac{\mathrm{d}\phi(\mathrm{u},\mathrm{t}_{k})}{\mathrm{d}\mathrm{u}’}e^{\mathrm{t}_{k}’}di_{\mathrm{u}}-1\mathrm{t}_{k}/2i_{\mathrm{u}}tk+-1O(\frac{1}{n})$Substitution of$\phi=o(\frac{1}{n})$$\phi=o(\frac{1}{\sqrt{n}})$.

shown in (4.10) into the integrand in he RHS above leads to. The proof of Theorem 3 is complete.

$\mathrm{t}$Q.E.D.

125

Now we shall give a proof of Theorem 4. Consider Cases (III) and (IV) first. The
bias-correction factors can be expande as$d$$\frac{1}{2\sqrt{n}}G_{2}(\overline{\mathrm{X}})\Psi 20(\theta(\mathrm{g}(\overline{\mathrm{x}})))=\frac{1}{2\sqrt{n}}G2\Psi_{20}+\frac{1}{2n}c_{2}\Psi_{2}1\ominus 1c1^{\mathrm{Z}_{n}}$$+ \frac{1}{2n}G_{3}(\Psi_{20}\otimes I_{p})_{\mathrm{Z}_{n}+\mathit{0}}p(\frac{1}{n})$,$\frac{1}{2\sqrt{n}}G_{2}(\overline{\mathrm{x}})\Psi 2\mathrm{o}(\Psi_{1}-10(_{\overline{\mathrm{X}})})=\frac{1}{2\sqrt{n}}G_{2}\Psi 20+\frac{1}{2n}c_{2}\Psi_{2}1\Psi_{1}-11\mathrm{Z}_{n}$$+ \frac{1}{2n}G_{3}(\Psi_{2}0\otimes I_{p})\mathrm{z}_{n}+O(p\frac{1}{n})$an thus for a$d$$3\mathrm{O}\mathrm{E}$estimator$\mathrm{g}^{*}(\overline{\mathrm{x}})$(i.e.,$G_{1}=\overline{G}_{1}$and$G_{2}=\overline{G}_{2}$) we have$\sqrt{n}(\mathrm{g}^{*}(_{\overline{\mathrm{X}}})-\hat{\mathrm{u}}(*\overline{\mathrm{X}}))=\frac{1}{6n}(G_{3}-\overline{c}_{3}|)\{_{\mathrm{Z}}n<3>-3(\Psi_{20}\otimes \mathrm{z}_{n})\}+o_{p}(\frac{1}{n})=O_{p}(\frac{1}{n})$for he both cases. For all the cases, we shall write$t$$\sqrt{n}(\mathrm{g}^{*}(_{\overline{\mathrm{X}}})-\hat{\mathrm{u}}*(\overline{\mathrm{X}}))=\frac{1}{6n}(G3-\overline{c}’3)\mathrm{t}\mathrm{Z}^{<}n3>-3(\Psi 20\otimes \mathrm{z})n\}+\frac{1}{2n}B(\mathrm{u})_{\mathrm{Z}+\mathit{0}}np(\frac{1}{n})$(4.14)$=O_{p}( \frac{1}{n})$where$B(\mathrm{u})$,is a constant matrix of order$(G_{3}- \overline{c}_{3})(\Psi 20\otimes I_{p})-\frac{\mathrm{d}}{\mathrm{d}\overline{\mathrm{x}}},$$B(\mathrm{u})=\{$$q\cross p$

given as$[c\overline{2\Psi}_{2}0-\overline{c}\overline{2\Psi}20]|_{\overline{\mathrm{x}}=}\eta(\mathrm{u})$$(G_{3}-\overline{c}3)(\Psi_{20}\otimes I_{p})-(G_{3^{-}}\overline{c}_{3})(\Psi 21^{\ominus}1\overline{G}1\otimes\Psi_{20})$

$0$

Since

$\sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{x}})-\hat{\mathrm{u}}(*\overline{\mathrm{x}}))=O_{p}(\frac{1}{n})$for Case (I)for Case (II)for Cases (III), (IV).

, the same deriva ion as in (4.7) can apply to get$t$$E[e^{i\mathrm{t}’\sqrt{n}(\mathrm{g}}]*(\overline{\mathrm{x}})-\mathrm{u})-E[e^{i}-]\mathrm{t}’\sqrt{n}(\hat{\mathrm{u}}(*\overline{\mathrm{x}})\mathrm{u})$$=(i \mathrm{t})’E[e-\mathrm{u}\sqrt{n}i\mathrm{t}\sqrt{n}’(\hat{\mathrm{u}}^{*}(\overline{\mathrm{x}}))(\mathrm{g}(*\overline{\mathrm{x}})-\hat{\mathrm{u}}^{*}(\overline{\mathrm{X}}))]+o(\frac{1}{n\sqrt{n}})$Put$\phi=\phi(\mathrm{u}, \mathrm{t})=E[e^{i})\mathrm{t}^{l}\sqrt{n}(\overline{\mathrm{u}}*(\overline{\mathrm{x}}-\mathrm{u})\sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{x}})-\hat{\mathrm{u}}^{*}(_{\overline{\mathrm{X}}}))](=O(\frac{1}{n}))$,and then he inversion of the Fourier transform is made, so that$t$$\mathrm{P}_{\theta(\mathrm{u})}\{\sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{X}})-\mathrm{u})\in C\}$

(4.15)$=\mathrm{P}_{\theta(\mathrm{u})}\{\sqrt{n}(\hat{\mathrm{u}}^{*}(\overline{\mathrm{x}})-\mathrm{u})\in c\}$$+ \int_{C}\frac{1}{(2\pi)^{q/2}}\int_{\mathrm{R}^{q}}e^{-i\mathrm{x}’}{}^{\mathrm{t}}(i\mathrm{t})’\phi(\mathrm{u},\mathrm{t})d\mathrm{t}d\mathrm{x}+o(\frac{1}{n\sqrt{n}})$

.

126

for every Borel convex set$C\subset \mathrm{R}^{q}$. Let us apply Lemmas 1 and 2 to evaluate$\phi$. We have;$iE[e^{i\mathrm{t}’}-\mathrm{u}\sqrt{n}\sqrt{n}(\dot{\mathrm{u}}^{*}(\overline{\mathrm{x}}))(\mathrm{g}^{*}(_{\overline{\mathrm{X}})}-\hat{\mathrm{u}}^{*}(\overline{\mathrm{x}}))\{\sqrt{n}(\hat{\mathrm{u}}^{*}(\overline{\mathrm{X}})-\mathrm{u})-\overline{c}_{1^{\mathrm{Z}_{n}\}]}}$$= \frac{i}{n\sqrt{n}}E[ei\mathrm{t}’\overline{c}1\mathrm{z}n\{\frac{1}{6}(G_{3}-\overline{c}3)(_{\mathrm{Z}_{n}-}<3>3(\Psi 20\otimes \mathrm{z}_{n}))+\frac{1}{2}B(\mathrm{u})_{\mathrm{Z}_{n}\}}$.

$\{\frac{1}{2}\overline{c}_{2}(\mathrm{z}_{n}^{<}-2>\Psi_{20})\}/]+o(\frac{1}{n\sqrt{n}})$(by (4.14))(by (4.5))$= \frac{i}{4n\sqrt{n}}B(\mathrm{u})E[e^{i\mathrm{t}^{l}}n\mathrm{z}\overline{c}_{1}\mathrm{z}(\mathrm{z}^{<}2nn->\Psi_{20)’]\overline{c}_{2}’()}+\mathit{0}\frac{1}{n\sqrt{n}}$$=- \frac{e^{-\mathrm{t}’i_{\mathrm{u}}^{-}\mathrm{t}/2}1}{4n\sqrt{n}}B(\mathrm{u})[(2\Psi_{11}-\Psi_{1}1\overline{c}_{1}\mathrm{t}\mathrm{t}’\overline{G}_{1}\Psi 11)’\otimes(\mathrm{t}’\overline{G}_{11}\Psi 1)]\overline{G}_{2}’$(by$+o( \frac{1}{n\sqrt{n}})$As a result, the(4.16)$C(\mathrm{u}, \mathrm{t})$$(4.3\mathrm{a})$).

in Lemma 2 can be written as$C( \mathrm{u}, \mathrm{t})=\frac{i}{\sqrt{n}}\frac{\mathrm{d}\phi}{\mathrm{d}\mathrm{u}’}i_{\mathrm{u}}^{-1}+\frac{e^{-\mathrm{t}i_{\mathrm{u}}\mathrm{t}/2}\prime-1}{n\sqrt{n}}B(\mathrm{u})D(\mathrm{u}, \mathrm{t})+o(\frac{1}{n\sqrt{n}})$

,

where
.

$D( \mathrm{u}, \mathrm{t})=-\frac{1}{4}[(^{\underline{\eta}\Psi_{11}-}\Psi 11\overline{G}_{1}\mathrm{t}\mathrm{t}’\overline{G}_{1}\Psi 11)/\otimes(\mathrm{t}’\overline{c}_{1}\Psi_{1}1)]\overline{G}_{2}’$By Lemma 2,(4.17)$\phi(\mathrm{u}, \mathrm{t})=e-\mathrm{t}’i_{\mathrm{u}}^{-\iota}\mathrm{t}/2(E[\sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{X}})-\hat{\mathrm{u}}^{*}(\overline{\mathrm{x}}))]+\sum_{=1}\int_{0}tk-1/\mathrm{C}_{k(\mathrm{u},\mathrm{t}_{k}})e^{\mathrm{t}_{k\mathrm{u}}’}di\mathrm{t}_{k}2)kqtk$

We know hat$t$$\phi=O(\frac{1}{n})$, so that$C( \mathrm{u}, \mathrm{t})=O(\frac{1}{n\sqrt{n}})$in view of (3.3) and (4.17). Thus, sincefrom (4.16), implying.

$\phi=O(\frac{1}{n\sqrt{n}})$$C( \mathrm{u}, t)=\frac{e^{-\mathrm{t}’}\mathrm{u}i-\iota_{\mathfrak{t}/2}}{n\sqrt{n}}B(\mathrm{u})D(\mathrm{u}, \mathrm{t})+o(\frac{1}{n\sqrt{n}})$

, we

have
$\phi(\mathrm{u}, \mathrm{t})=e^{-\mathrm{t}’}i_{\mathrm{u}}^{-\iota}\mathrm{t}/2\{E[\sqrt{n}(\mathrm{g}(*\overline{\mathrm{x}})-\hat{\mathrm{u}}^{*}(\overline{\mathrm{x}}))]+\frac{1}{n\sqrt{n}}B(\mathrm{u})\sum_{=i1}q\int_{0}tkk\mathrm{d}(\mathrm{u}, \mathrm{t}_{k})dt_{k}\}$(4.18)
where$+o( \frac{1}{n\sqrt{n}})$,$D(\mathrm{u}, \mathrm{t})=[\mathrm{d}_{1}(\mathrm{u}, \mathrm{t}), \ldots, \mathrm{d}_{q}(\mathrm{u}, \mathrm{t})]$

When

.

$E[ \sqrt{n}(\mathrm{g}^{*}(\overline{\mathrm{X}})-\hat{\mathrm{u}}^{*}(\overline{\mathrm{X}}))]=o(\frac{1}{n\sqrt{n}})$and (IV), we have from (4.18) that$\phi=o(\frac{1}{n\sqrt{n}})$second statement in Theorem 4. We shallnote that$\mathrm{t}$$\phi(\mathrm{u}, \mathrm{t})$and of orderis even in$o( \frac{1}{n\sqrt{n}})$$\mathrm{a}\mathrm{r}_{\mathrm{o}}\sigma \mathrm{u}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{t}$and $B(\mathrm{u})=0$ , which is met by Cases (III)since$\mathrm{p}r$. This, along with (4.15), proves theove Theorem$D(\mathrm{u}, \mathrm{t})$$4-(\mathrm{i})$. First the reader should

is odd in . Thus,$\mathrm{t}$$(i\mathrm{t})^{\prime_{\phi(}}\mathrm{u},$. The second term in the RHS in (4.15) vanishes if

about the origin. The proof is complete.

$C$$\mathrm{t})$is odd inis symmetric
Q.E.D.

127

Kano [19] proved$5\mathrm{O}\mathrm{E}$$4\mathrm{O}\mathrm{E}$of the MLE for Case (III) as a corollary of his main result on. The proof given here is direct and much simpler.
AppendixWe$\mathrm{s}h$all give proofs of Lemmas 1 and 2.

There is a useful formula in evaluating expectations involving the score function, see:or Kano, Amarie.g., Akahi an Takeuchi$d$$r\mathrm{a}$(A. 1)$E[ \mathrm{z}_{n}’\ominus_{1}\otimes A(\mathrm{z}_{n}, \mathrm{u})]=\frac{1}{\sqrt{n}}\mathrm{t}^{\frac{\mathrm{d}}{d\mathrm{u}’}E[A(\mathrm{Z}_{n}},$where$A(\mathrm{z}, \mathrm{u})$$[17](\mathrm{E}\mathrm{q}.(4.5))$$[5](\mathrm{P}^{\mathrm{a}}\mathrm{g}\mathrm{e}124)$$[3](\mathrm{L}\mathrm{e}\mathrm{m}\mathrm{m}\mathrm{a}5.1.1)$$\mathrm{u})]-E[\frac{\mathrm{d}}{\mathrm{d}\mathrm{u}’}A(\mathrm{z}_{n}, \mathrm{u})]\}$

is a matrix-valued function measurable in$\mathrm{z}$in$\mathrm{R}^{p}$,and continuously differ-entiable in .
$\mathrm{u}$Recall that$\overline{G}_{1}=i_{\mathrm{u}}^{-1}\ominus_{1}’$. Differentiation of w.r.t.
$\phi$

gives$\frac{\mathrm{d}}{\mathrm{d}\mathrm{t}’}$$\frac{\mathrm{d}\phi}{d\mathrm{t}’}=iE[ei\mathrm{t}’\sqrt{n}(\tau^{(1)}-n\mathrm{u})\tau(n2)\sqrt{n}(T_{n}(1)-\mathrm{u})’]$);$=iE[e^{i}n-$ )$\tau \mathrm{t}’\sqrt{n}(T^{(1}\mathrm{u}(2)\{n\sqrt{n}(T_{n}(1)-\mathrm{u})-\overline{c}1^{\mathrm{Z}_{n}\}]}$(A.2)$+iE[(\mathrm{z}_{n}’\ominus_{1})\otimes(e^{i\mathrm{t}’\sqrt{n}}-\mathrm{u})T(T_{n}(1)(2))n]i_{\mathrm{u}}^{-1}$.

Exchangeability of the differentiation and expectation is permitted for exponential familiesin). Application of the formula (A.1) to the second te(see e.g., Lehmann [20]$r\mathrm{m}$$\mathrm{p}\mathrm{a}_{\mathrm{o}}\sigma \mathrm{e}59$(A.2) leads to$\frac{i}{\sqrt{n}}(\frac{\mathrm{d}}{d\mathrm{u}’}E[e^{i\mathrm{t}}-\mathrm{u}T_{n}2]-E[\frac{\mathrm{d}}{\mathrm{d}\mathrm{u}’}e^{i}n-\tau^{()}1)T^{(}2)]n)’\sqrt{n}(T_{n}^{()}1)()\mathrm{t}’\sqrt{n}(\mathrm{u}i_{\mathrm{u}}^{-1}$.

$= \frac{i}{\sqrt{n}}\frac{\mathrm{d}\phi}{d\mathrm{u}’}i_{\mathrm{u}}^{-1}-(\mathrm{t}/i_{\mathrm{u}}-1\otimes\phi)$

Q.E.D.

Substitution of this int (A.2) gives (4.11)$0$

Let us prove Lemma 2. It suffices to show that the following partial differential$e$quations (A.3) has he solution (A.6):$\mathrm{t}$

(A.3)

Put

$\frac{\mathrm{d}\phi(\mathrm{t})}{\mathrm{d}\mathrm{t}},=-(\mathrm{t}’i_{\mathrm{u}}^{-}1\otimes\phi(\mathrm{t}))+C(\mathrm{t})$$\phi(\mathrm{t})=\phi(\mathrm{t})\cdot e\sim-\mathrm{t}’i^{-1}\mathrm{t}/\mathrm{u}2$.

, and then we have$\frac{\mathrm{d}\phi(\mathrm{t})\sim}{d\mathrm{t}’}=e^{\mathrm{t}’i^{-}\mathrm{t}/2}\mathrm{u}1C(\mathrm{t})(=e^{\mathrm{t}’i^{-}\mathrm{t}/2}\mathrm{u}[_{\mathrm{C}}1(\mathrm{t})1, \ldots, \mathrm{c}_{q}(\mathrm{t})],$

That is,(A.4)$\frac{\partial\overline{\phi}(\mathrm{t})}{\partial t_{k}}=e^{\mathrm{t}’i^{-}\mathrm{t}/2}\mathrm{u}1\mathrm{c}_{k}(\mathrm{t})$$(k=1, \ldots, q)$.

$\mathrm{s}\mathrm{a}\mathrm{y})$

.

128

$\mathrm{U}\sin_{\mathrm{o}}\sigma$(A.4), we have$\phi(\mathrm{t})=\phi\sim\sim(0)+\sum_{k=1}\{^{\sim}\phi(\mathrm{t}_{k})-\phi(\mathrm{t}_{k-1})\}q\sim$$= \overline{\phi}(0)+\sum_{=k1}\int q0t_{k}\frac{\partial\phi(\mathrm{t}_{k})\sim}{\partial t_{k}}dt_{k}$

(A.5)

.

$= \phi(0)+\sum_{k1}\sim\int_{0}^{t_{k}}\mathrm{c}k(\mathrm{t}_{k}=q)e^{\mathrm{t}_{k\mathrm{u}}’}di-1\mathrm{t}_{k/2}t_{k}$Conversely, he function (A.5) is actually a solution to (A.4), provided that (A.4) has a$\mathrm{t}$solution. Thus, we obtain(A.6)$\phi(\mathrm{t})=e^{-\mathrm{t}i_{\mathrm{u}}\mathrm{t}/2}’-1(\phi(0)+\sum_{k=1}^{q}\int^{t_{k}}0/\mathrm{C}_{k()t_{k})}\mathrm{t}_{k}ed\mathrm{t}_{k\mathrm{u}}’i^{-1}\mathrm{t}_{k}2$.

ACKNOWLEDGMENT
The author would like to express his thanks to Professor M. Akahira of University ofTsukuba and Dr. M..Taniguchi of Os$a\mathrm{k}\mathrm{a}$University for invaluable discussion on higher-

order efficiency. He also thanks Professor S. Amari of The University of Tokyo for showing
his bias-correction factor (Cas$e(\mathrm{I}\mathrm{V})$in Theorem 4).

REFERENCES[1] Akahira, M. (1975). A note on the second order asymptotic efficiency of estimators in

an autoregressive process. Rep. Univ. Electro-Comm., 26, 143-149.
[2] Akahira, M. and Takeuchi, K. (1976). On the second order asymptotic efficiency of

estimators in multiparameter cases. Rep. Univ. Electro-Comm., 26, 261-269.
[3] Akahira, M. and Takeuchi, K. (1981).

$A_{\mathit{8}}ymptoti_{C}$Efficiency of Statistical Estima-

tors: Concepts and Higher Order Asymptotic Efficiency, Lecture Notes in Statistics,
Vol.7. Springer-Verlag, New York.
[4] Amari, S. (1982). Differential geometry of curve d exponential families: Curvatures andinformation loss. Ann. Statist., 10, 357-387.
[5] Amari, S. (1985).

Differential-Geometrical Methods in Statistics, Lecture Notes in

Statistics, Vo1.28. Springer-Verlag, Berlin.

[6] Amari, S. (1995). Personal communications.

129[7] Barndorff-Nielsen, O. E. and Cox, D.

R.. (1982). Asymptotic Techniques for Use in

Statistics. Chapman and Hall, London.
[8] Bentler, P. M. and Lee, Sik-Yum (1978). Matrix derivatives with chain rule and rules
for simple, Hadamard, and Kronecker products. J. Math. Psychol., 17, 255-262.
[9] Bhattacharya, R. and Denker, M. (1990).

$A_{\mathit{8}}ymptotic$

Statistics. Birkh\"auser, Basel.

[10] Efron, B. (1975). Defining the curvature of a statistical problem (with applications tosecond or der efficiency). Ann. Statist., 3, 1189-1242.
[11] Eguchi, S. (1983). Second order efficiency of minimum contrast estimators in a curved

exponential family. Ann. Statist., 11, 793-803.
[12] Ghosh, J. K. (1994). Higher Order Asymptotics. NSF-CBMS Regional Conference

Series in Probability and Statistics, Vol.4. IMS, Hayward, CA.
[13] Ghosh, J. K. and Subramanyam, K. (1974). Second order efficiency of maximum

likelihood estimates.

$Sankhy\overline{a}$, Ser. A, 36, 325-358.

[14] Henderson H. V. and Searl e, S. R. (1981). The$\mathrm{v}\mathrm{e}\mathrm{c}$-permutation matrix, the vec

operator and Kronecker products: A review. Linear and Multilinear Algebra, 9,271-288.
[15] Hosoya, Y. (1979). Higher-order efficiency in the estimation of linear processes. Ann.
$Stati_{\mathit{8}}t.,$

7, 516-530.

[16] Kano, Y. (1986). Conditions on consistency of estimators in covariance structure

model. J. Japan Statist. Soc., 16, 75-80.
[17] Kano, Y. (1993). Beyond second-order efficiency. (submitted)[18] Kano, Y. (1993). Beyond second-order efficiency: Supplement.

Technical Report,

University of Osaka Prefecture, Osaka, Japan.
[19] Kano, Y. (1994). More higher-order efficiency: Concentration probability. Technical
Reports of Statistical Group: DATA95-04, Tokyo, Japan.

[20] Lehmann, E. L. (1986). Testing Statistical Hypotheses (2nd Ed.). John Wiley, NewYork.
[21] Magnus, J. R. and Neudecker, H. (1988). Matrix Differential$cal_{C}ul\cdot us$

with Applica-

tions in Statistics and Econometrics. John Wiley, Chichester.
[22] Pfanzagl, J. (1973). Asymptotic expansions related to minimum contrast estimators.
Ann. Statist., 1, 993-1026.

[23] Pfanzagl, J. (1976). First order efficiency implies second order efficiency. In Contribution to Statistics (Hajek Memorial Volume), (J. Jureckova, Ed.), pp. 167-196,

130

Academia, Prague.
[24] Pfanzagl, J. (1985). Asymptotic ExpanSion8 for General$Stati_{\mathit{8}}tical$

Models. Lecture

Notes in Statistics, Vol.31. Springer-Verlag, Berlin.
[25] Pfanzagl, J. and Wefelmeyer, W. (1978). A third order optimum property of the

maximum likelihood estimator. J. Multivariate Anal., 8, 1-29.
[26] Rao, C. R. (1961). Asymptotic efficiency and limiting information. Proc. FourthBerkeley Symp., 1, 531-545.
[27] Rao, C. R. (1962). Efficient estimates an d optimum inference procedures in large

samples. J. Royal Statist. Soc., Ser. B, 24, 46-72.
[28] Rao, C. R. (1963). Criteria of estimation in large samples.

$Sankhy\overline{a}$, Ser. A, 25,

189-206.
[29] Rao, C. R., Sinha, B. K. and Subramanyam, K. (1982). Third order efficiency of the

maximum likelihood estimator in the multinomial distribution. Statist. Decisions,
1, 1-16.

[30] Satake, I. (1974). Senkei Daisu Gaku (Linear Algebra) (2nd Ed.). Shokabo, Tokyo (inJapanese).

[31] St$er\mathrm{n}\mathrm{b}\mathrm{e}\mathrm{r}_{\mathrm{o}}\sigma$, S. (1964). Lectures on Differential Geometry. Prentice-Hall Inc., Engle-

wood Cliffs, NJ.
[32] Takemura, A. (1991). Tahenryo Suisoku Tokei no Ki80 (The foundation of Multivari-ate$Stati_{\mathit{8}t}ical$

Inference). Kyoritsu Shuppan, Tokyo (in Japanese).

[33] Takeuchi, K. (1974). Tokeiteki Suisoku no Zenkin Riron (Asymptotic Theory of Sta-tistical Estimation). Kyoiku Shuppan, Tokyo (in Japanese).
[34]$\mathrm{T}\mathrm{a}\mathrm{n}\mathrm{i}_{\mathrm{o}}\sigma \mathrm{u}\mathrm{c}h\mathrm{i}$, M. (1983). On the second order asymptotic efficiency of estimators ofGaussian ARMA Processes. Ann. Statist., 11, 157-169.
[35] Taniguchi, M. (1991). Higher Order Asymptotic Theory for Time Series Analysis,

Lecture Notes in Statistics, Vol.68. Springer-Verlag, Berlin.
[36] Yoshida, N. (1992). Asymptotic expansions of maximum likelihood estimators forsmall diffusions via t he theory of$\mathrm{M}a1\mathrm{l}\mathrm{i}\mathrm{a}\mathrm{V}\mathrm{i}\mathrm{n}-\mathrm{v}\mathrm{v}\mathrm{a}\tan a\mathrm{b}\mathrm{e}$.

Probab.

Theory Relat.

Fields, 92, 275-311.
[37] Yoshida, N. (1993). Asymptotic expansion of Bayes estimators for small diffusions.

Probab. Theory Relat. Fields, 95, 429-450.