A Simple Derivation of the Efficiency Bound forConditional Moment Restriction Models

Naoya Sueishi

September 2015
Discussion Paper No.1531

GRADUATE SCHOOL OF ECONOMICS
KOBE UNIVERSITYROKKO, KOBE, JAPANA Simple Derivation of the Eﬃciency Bound for Conditional
Moment Restriction Models∗
Naoya Sueishi
Kobe University†
September 2015

Abstract
This study gives a simple derivation of the eﬃciency bound for conditional moment restriction models. The Fisher information is obtained by deriving a least favorable submodel in anexplicit form. The proposed method also suggests an asymptotically eﬃcient estimator, which
can be viewed as an empirical likelihood estimator for conditional moment restriction models.
Keywords: Conditional moment restrictions; Empirical likelihood; Fisher information; Leastfavorable submodel.
JEL classification: C14.

1Introduction

Models speciﬁed via conditional moment restrictions are common in econometrics, and several
estimators have been proposed in the literature. Contributors include Newey (1990), Carrascoand Florens (2000), Donald, Imbens, and Newey (2003), Zhang and Gijbels (2003), Dominguezand Lobato (2004), Kitamura, Tripathi, and Ahn (2004), and Lavergne and Patilea (2013).
The eﬃciency bound for conditional moment restriction models was originally derived
by Chamberlain (1987), who utilized the fact that any distribution can be approximatedarbitrarily well by a multinomial distribution. Severini and Tripathi (2001), who provided ageneral framework to derive the eﬃciency bound for semiparametric models using a Hilbert
space theory, obtained the eﬃciency bound for conditional moment restriction models as aspecial case.
This study gives a simple and constructive method to derive the eﬃciency bound for
conditional moment restriction models, based on the viewpoint of Stein (1956), who statedthat the Fisher information in a semiparametric model is not larger than that in any parametric∗†This research was supported by JSPS KAKENHI Grand Number 25780150.
Graduate School of Economics, 2-1 Rokkodai-cho, Nada-ku, Kobe, Hyogo 657-8501, Japan.

sueishi@econ.kobe-u.ac.jp

1Email:submodel that satisﬁes the semiparametric assumption and contains the truth. A submodel iscalled least favorable if its Fisher information takes the inﬁmum of the Fisher information overall smooth submodels, and the Fisher information in the semiparametric model is deﬁned bythat in the least favorable model. We calculate the Fisher information in conditional moment
restriction models by deriving a least favorable model in an explicit form. Severini and Wong
(1992) (hereafter: SW, 1992) showed that a least favorable submodel is obtained by ﬁnding
the minimizer of the Kullback–Leibler discrepancy from the model to the true probability
measure. Combining this result with a duality theorem in the convex analysis, we obtain theeﬃciency bound.
Furthermore, the proposed method suggests an asymptotically eﬃcient estimator, whichis essentially the same as the estimator of SW (1992) for conditionally parametric models.
Moreover, our estimator can be viewed as an empirical likelihood (EL) estimator for conditional
moment restriction models. We will observe a connection between our estimator and thesmoothed EL (SEL) estimator of Kitamura, Tripathi, and Ahn (2004).

2Fisher Information

Let (X1 , Z1 ), . . . , (Xn , Zn ) be independent copies of (X, Z) that takes values in X × Z. Thejoint law of (X, Z) is denoted by µX,Z . Furthermore, we write µX,Z = µZ|X µX , where µZ|X isthe conditional law of Z given X and µX is the marginal law of X. The parameter of interest,θ0 ∈ Θ, is characterized as a unique value that satisﬁes the conditional moment restriction∫E[m(Z, θ0 )|X] = m(z, θ0 )dµZ|X = 0 a.s. µX ,where m : Z ×Θ → Rl is a known function. We assume that µX does not provide any information on θ0 . Moreover, without loss of generality, we assume that θ0 is a scalar parameter. The
model is semiparametric in the sense that µZ|X is the inﬁnite-dimensional nuisance parameter.
Our aim is to ﬁnd the eﬃciency bound for estimating θ0 .
The model can be represented as a set of conditional probability measures {Pθ,η } that isindexed by the scalar parameter θ and the inﬁnite-dimensional nuisance parameter η. We
employ a setting that is used in Kitamura (2003) and Otsu and Whang (2011). Let MZ|x be
the set of conditional probability measures on Z given as X = x. Because a model for themarginal law of X is not speciﬁed, our model is written asPZ|X = {Pθ,η : θ ∈ Θ, η ∈ Hθ,X a.s. µX } ,where{}∫Hθ,x = νZ|x ∈ MZ|x : m(z, θ)dνZ|x = 0 .

Note that Hθ,x is the set of conditional probability measures that satisﬁes the conditional
moment restriction for a given θ ∈ Θ and x ∈ X .
The Fisher information for θ0 in PZ|X is the inﬁmum of Fisher information over all parametric submodels of PZ|X . Let η0 be the true nuisance parameter that satisﬁes µZ|X = Pθ0 ,η0 .

2Moreover, let ηθ be a smooth curve of θ that satisﬁes ηθ0 = η0 . Then we obtain a parametric submodel {Pθ,ηθ : θ ∈ U }, where U is a neighborhood of θ0 . Fisher information in thesubmodel is()2 dlog
dP
θ,η
θ
,
Edθ
θ=θ0

where the symbol E denotes the expectation with respect to µX,Z . A curve ηθ∗ that gives theinﬁmum of the Fisher informations is called a least favorable curve. The model {Pθ,ηθ∗ : θ ∈ U }is a least favorable submodel.
SW (1992) showed that a least favorable curve is obtained by minimizing the Kullback–Leibler discrepancy from Pθ,η to Pθ0 ,η0 with respect to η. Thus, the least favorable curve isobtained by solvingmin −η∈Hθ,x(∫logdPθ,ηdµZ|x)(∫dµZ|x = min −η∈Hθ,xlogdηdµZ|x)dµZ|xfor each θ and x. Moreover, a duality theorem implies that the inﬁnite-dimensional minimization problem is equivalent to the ﬁnite-dimensional dual problem:∫max log(1 + λ′ m(z, θ))dµZ|x .
λ∈RlFor instance, see Kitamura (2003).
The duality theorem implies that the least favorable submodel {Pθ,ηθ∗ } is equivalently
written as {Pθ,λθ } whose µZ|x -density is given bydPθ,ηθ∗dPθ,λθ== (1 + λθ (x)′ m(z, θ))−1 ,dµZ|xdµZ|x∫

where
λθ (x) = arg max
λ∈Rllog(1 + λ′ m(z, θ))dµZ|x .

The new nuisance parameter, λ, is ﬁnite-dimensional for ﬁxed x.
The Fisher information in {Pθ,λθ } is
([()2 )][]′dℓ(Z; θ, λθ (X))∂ℓ(Z;θ,λ(X))
∂λ(X)
∂ℓ(Z;
θ,λ(X))0θθ0θ000,E=E+dθ∂θ∂θ∂λθ=θ0where ℓ(z; θ, λ) = − log(1 + λ′ m(z, θ)). By the implicit function theorem, we have[ 2]−1 [ 2]∂λθ (x)∂ ℓ(Z; θ, λθ (X))
∂ ℓ(Z; θ, λθ (X))= −E|X
=xE|X
=x.
∂θ
∂λ∂λ′
∂λ∂θ
Because λθ0 (x) = 0 for all x ∈ X , some calculation yields that[]∂λθ0 (x)∂m(Z, θ)−1= −E [m(Z, θ0 )m(Z, θ0 )′ |X = x] E
|X = x .
∂θ∂θThus, Fisher information in PZ|X is[ []′]][∂m(Z, θ0 )∂m(Z, θ0 )−1′|X E [m(Z, θ0 )m(Z, θ0 ) |X] E|X .
Iθ0 = E E∂θ∂θThe asymptotic variance of any regular estimator for θ0 is not less than Iθ−1.
03

3Eﬃcient Estimator

SW (1992) proposed an asymptotically eﬃcient estimator for conditionally parametric models, in which a nuisance parameter is a Euclidean parameter once the value of a conditioningvariable is ﬁxed. Our original model is not conditionally parametric because η is inﬁnitedimensional even for a ﬁxed value of X. However, the model in the dual problem is conditionally parametric, and thus the estimator of SW (1992) is applicable.
An infeasible eﬃcient estimator for θ0 is the maximum likelihood estimator that maximizes∑ni=1 ℓ(Zi ; θ, λθ (Xi )). Because λθ (x) is unknown, we replace it with a sample analog. Supposethat X is compact. Then we can estimate λθ (x) byλ̂θ (x) = arg maxλ∈Rln∑(Kj=1x − Xjh)ℓ(Zj ; θ, λ),where K(·) is a kernel function and h is a bandwidth. Our estimator for θ0 isθ̂ = arg maxθ∈Θn∑ℓ(Zi ; θ, λ̂θ (Xi )),

i=1

which is essentially the same as the estimator of SW (1992).
Our estimator can also be viewed as an EL estimator. Indeed, if the model is speciﬁed
via the unconditional moment restriction E[m(Z, θ0 )] = 0, then a least favorable submodel isobtained asdPθ,λθ= (1 + λ′θ m(z, θ))−1 ,dµZ
∫whereλθ = arg max
λ∈Rllog(1 + λ′ m(z, θ))dµZ .

Thus, the resulting estimator coincides with the EL estimator of Qin and Lawless (1994). Seealso DiCiccio and Romano (1990) and Bertail (2006) for a connection between EL estimatorsand least favorable submodels.
pUnder certain regularity conditions, we have θ̂ → θ0 and√dn(θ̂ − θ0 ) → N (0, Iθ−1).
0Thus, the estimator is asymptotically eﬃcient. Note that replacing λθ with λ̂θ does not aﬀect
the asymptotic distribution of the estimator. This is because λθ is a least favorable curve (seeSection 4 of SW, 1992).
Because regularity conditions and proofs are given in SW (1992), we do not reproduce
them. To show the asymptotic results, we need to establish uniform consistency of λ̂θ (x) andits derivatives over θ ∈ Θ and x ∈ X . Thus, if X is not compact, we need to introduce atrimming function to avoid the random denominator problem.
Finally, we note a connection between our estimator and the SEL estimator of Kitamura,
Tripathi, and Ahn (2004). In our notation, the SEL estimator solvesmaxθ∈Θn ∑n∑wij ℓ(Zj ; θ, λ̂θ (Xi )),i=1 j=14

(whereKwij = ∑Xi −Xjhn
j=1 K()Xi −Xjh).

Thus, the diﬀerence between two estimators is in whether the log-likelihood is smoothed whenthe “outer loop” is solved. The local likelihood method of Tibshirani and Hastie (1987)
suggests that smoothing in the outer loop is necessary when θ0 is a function of x, which is thecase considered by Zhang and Gijbels (2003).

References
Bertail, P. (2006): “Empirical Likelihood in Some Semiparametric Models,” Bernoulli, 12,299–331.
Carrasco, M., and J.-P. Florens (2000): “Generalization of GMM to a Continuum ofMoment Conditions,” Econometric Theory, 16, 797–834.
Chamberlain, G. (1987): “Asymptotic Eﬃciency in Estimation with Conditional MomentRestrictions,” Journal of Econometrics, 34, 305–334.
DiCiccio, T. J., and J. P. Romano (1990): “Nonparametric Conﬁdence Limits by Resampling Methods and Least Favorable Families,” International Statistical Review, 58, 59–76.
Dominguez, M. A., and I. N. Lobato (2004): “Consistent Estimation of Models Deﬁned
by Conditional Moment Restrictions,” Econometrica, 72, 1601–1615.
Donald, S. G., G. W. Imbens, and W. K. Newey (2003): “Empirical Likelihood Estimation and Consistent Tests with Conditional Moment Restrictions,” Journal of Econometrics,
117, 55–93.
Kitamura, Y. (2003): “A Likelihood-Based Approach to the Analysis of a Class of Nested
and Non-Nested Models,” Unpublished Manuscript.
Kitamura, Y., G. Tripathi, and H. Ahn (2004): “Empirical Likelihood-Based Inferencein Conditional Moment Restriction Models,” Econometrica, 72, 1667–1714.
Lavergne, P., and V. Patilea (2013): “Smooth Minimum Distance Estimation and Testing with Conditional Estimating Equations: Uniform in Bandwidth Theory,” Journal ofEconometrics, 177, 47–59.
Newey, W. K. (1990): “Eﬃcient Instrumental Variable Estmation of Nonlinear Models,”
Econometrica, 58, 809–837.
Otsu, T., and Y.-J. Whang (2011): “Testing for Nonnested Conditional Moment Restrictions via Condtional Empirical Likelihood,” Econometric Theory, 27, 114–153.
Qin, J., and J. Lawless (1994): “Empirical Likelihood and Generalized Estimating Equations,” Annals of Statistics, 22, 300–325.

5Severini, T. A., and G. Tripathi (2001): “A Simpliﬁed Approach to Computing Eﬃciency
Bounds in Semiparametric Models,” Journal of Econometrics, 102, 23–66.
Severini, T. A., and W. H. Wong (1992): “Proﬁle Likelihood and Conditionally Parametric Models,” Annals of Statistics, 20, 1768–1802.
Stein, C. (1956): “Eﬃcient Nonparametric Testing and Estimation,” in Proceedings of thethird Berkeley Symposium on Mathematical Statistics and Praobability, vol. 1, pp. 187–195.
Tibshirani, R., and T. Hastie (1987): “Local Likelihood Estimation,” Journal of theAmerican Statistical Association, 82, 559–567.
Zhang, J., and I. Gijbels (2003): “Sieve Empirical Likelihood and Extensions of the Generalized Least Squares,” Scandinavian Journal of Statistics, 30, 1–24.

6