Locally Robust Semiparametric Estimation
Victor Chernozhukov
MIT

Juan Carlos Escancianoy
Indiana University

Hidehiko Ichimuraz
University of Tokyo

Whitney K. Neweyx
MIT
July 27, 2016

Abstract
This paper shows how to construct locally robust semiparametric GMM estimators,
meaning equivalently moment conditions have zero derivative with respect to the …rst stepand the …rst step does not a¤ect the asymptotic variance. They are constructed by adding
to the moment functions the adjustment term for …rst step estimation. Locally robust
estimators have several advantages. They are needed for valid inference following …rst step
selection, see Belloni et. al. (2012, 2014), and are less sensitive to the speci…cation of the…rst step. They are doubly robust for a¢ ne moment functions, where moment conditions
continue to hold when one …rst step component is incorrect. Locally robust moment
conditions also have smaller bias that is ‡atter as a function of …rst step smoothing leading
to improved small sample properties. Series …rst step estimators confer local robustness
on any moment conditions and are doubly robust for a¢ ne moments, in the direction of
the series approximation. Many new locally and doubly robust estimators are given here,
including for economic structural models. We focus on estimators that use cross-…tting in
the …rst step and give simple asymptotic theory for these.
Keywords: Local robustness, double robustness, semiparametric estimation, bias, GMM.
JEL classi…cation: C13; C14; C21; D24.
Department of Economics, MIT, Cambridge, MA 02139, U.S.A E-mail: vchern@mit.edu.
Department of Economics, Indiana University, Bloomington, IN 47405–7104, U.S.A E-mail: jescanci@indiana.edu.
zFaculty of Economics, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033. E-mail:ichimura@e.u-tokyo.ac.jp.
x
Department of Economics, MIT, Cambridge, MA 02139, U.S.A E-mail: wnewey@mit.edu.
y

1

1

Introduction

There are many economic parameters that depend on nonparametric …rst steps. Examples
include games, dynamic discrete choice, average consumer surplus, and treatment e¤ects. This
paper shows how to construct GMM estimators that are locally robust to the …rst step, meaningequivalently that moment conditions have a zero derivative with respect to the …rst step andthat estimation of the …rst step does not a¤ect their in‡uence function.
Locally robust moment functions have several advantages. Belloni, Chen, Chernozhukov, andHansen (2012) and Belloni, Chernozhukov, and Hansen (2014) showed that local robustness, also
referred to as orthogonality, is important for correct inference about parameters of interest whenmodel selection methods are used to choose the …rst step. Locally robust moment conditions
are also nearly correct when the nonparametric part is approximately correct. This robustness
property is appealing in many settings where it may be di¢ cult to get the …rst step completely
correct. Furthermore, local robustness implies the small bias property analyzed in Newey, Hsieh,
and Robins (1998, 2004; NHR henceforth). As a result asymptotic con…dence intervals basedon locally robust moments have actual coverage probability closer to nominal than for othermoments. Also, bias is ‡atter as a function of …rst step bias for locally robust estimators thanfor other estimators. This tends to make their mean-square error (MSE) ‡atter as a functionof smoothing also, so their performance is less sensitive to smoothing. In addition, by virtueof their smaller bias, locally robust estimators have asymptotic MSE that is smaller than otherestimators in important cases and undersmoothing is not required for root-n consistency. Finally,
asymptotic variance estimation is straightforward with locally robust moment functions, because
the …rst step is already accounted for.
Locally robust moment functions are constructed by adding to the moment functions theterms that adjust for (or account for) …rst step estimation. This construction gives moment
functions that are locally robust. It leads to new estimators for games, dynamic discrete choice,average surplus, and other important economic parameters. Also locally robust moments thatare a¢ ne in a …rst step component are globally robust in that component, meaning the moments
continue to hold when that component varies away from the truth. This result allows construction of doubly robust moments in the sense of Scharfstein, Rotnitzky, and Robins (1999) andRobins, Rotnitzky, and van der Laan (2000) by adding to a¢ ne moment conditions an a¢ neadjustment term. Here we construct many new doubly robust estimators, e.g. where the …rst
step solves a conditional moment restriction or is a density.
Certain …rst step estimators confer the small bias property on moment functions that are
not locally robust, including series estimators of mean-square projections (Newey, 1994), sieve
2

maximum likelihood estimators (Shen, 1996, Chen and Shen, 1997), bootstrap bias corrected…rst steps (NHR), and higher-order kernels (Bickel and Ritov, 2003). Consequently the inference
advantages of locally robust estimators may be achieved by using one of these …rst steps. These…rst steps only make moment conditions locally robust in certain directions. Locally robust
moments have the small bias property in a wider sense, that the moments are nearly zero as
the …rst step varies in a general way. This property is important when the …rst step is chosen
in very ‡exible, data based ways, such as by model selection as in Belloni et. al. (2014).
First step series estimators have some special robustness properties. Moments without theadjustment term can be interpreted as locally robust because there is an estimated adjustment
term with average that is identically zero. This property corresponds to …rst step series estimators conferring local robustness in the direction of the series approximation. Also, …rst stepseries estimators are doubly robust in those directions when the moment functions and …rst step
estimating equations are a¢ ne.
The theoretical and Monte Carlo results of NHR show the bias and MSE advantages of locally robust estimators for linear functionals of a density. The favorable properties of a twicing
kernel …rst step versus a standard …rst step found there correspond to favorable properties of
locally robust moments versus original moments, because a twicing kernel estimator is numerically equivalent to adding an estimated adjustment term. The theoretical results show thatusing locally robust moment conditions increases the rate at which bias goes to zero but only
raises the variance constant, and so leads to improved asymptotic MSE. The Monte Carlo results
show that the MSE of the locally robust estimator is much ‡atter as a function of bandwidthand has a smaller minimum than the original moment functions, even with quite small samples.
Advantages have also been found in the literature on doubly robust estimation of treatmente¤ects, as in Bang and Robins (2005) and Firpo and Rothe (2016). These results from earlier work suggest that locally robust moments provide a promising approach to improving theproperties of semiparametric estimators.
This paper builds on other earlier work. Locally robust moment conditions are semiparametric versions of Neyman (1959) C( ) test moments for parametric models, with parametricextensions to nonlikelihood settings given by Wooldridge (1991), Lee (2005), Bera et. al. (2010),
and Chernozhukov, Hansen, and Spindler (2015). Hasminskii and Ibragimov (1978) suggested
an estimator of a functional of a nonparametric density estimator that can be interpreted asadding the …rst step adjustment term. Newey (1990) derived the form of the adjustment termin some cases. Newey (1994) showed local robustness of moment functions that are derivativesof an objective function where the nonparametric part has been "concentrated out," derived
the form of the adjustment term for many important cases, and showed that moment functions
based on series nonparametric regression have small bias. General semiparametric model resultson doubly robust estimators were given in Robins and Rotnitzky (2001).

3

NHR showed that adding the adjustment term gives locally robust moments for functionals
of a density integral and showed the important bias and MSE advantages for locally robust
estimators mentioned above. Robins et. al. (2008) showed that adding the adjustment term
gives local robustness of explicit functionals of nonparametric objects, characterized some doubly robust moment conditions, and considered higher order adjustments that could further
reduce bias. The form of the adjustment term for …rst step nonparametric estimation has been
derived for a variety of …rst step estimators by Pakes and Olley (1995), Ai and Chen (2003), Bajari, Chernozhukov, Hong, and Nekipelov (2009), Bajari, Hong, Krainer, and Nekipelov (2010),
Ackerberg, Chen, and Hahn (2012), Ackerberg, Chen, Hahn, and Liao (2014), and Ichimura
and Newey (2016), among others. Locally and doubly robust moments have been constructed
for a variety of estimation problems by Robins, Rotnitzky, and Zhao (1994, 1995), Robins andRotnitzky (1995), Scharfstein, Rotznitzky, and Robins (1999), Robins, Rotnitzky, and van derLaan (2000), Robins and Rotnitzky (2001), Belloni, Chernozhukov, and Wei (2013), Belloni,
Chernozhukov, and Hansen (2014), Ackerberg, Chen, Hahn, and Liao (2014), Firpo and Rothe(2016), and Belloni, Chernozhukov, Fernandez-Val, and Hansen (2016).
Contributions of this paper are a general construction of locally robust estimators in a GMM
setting, a general nonparametric construction of doubly robust moments, and deriving bias andother large sample properties. The special robustness properties of …rst step series estimators arealso shown here. We use these results to obtain many new locally and doubly robust estimators,
such as those where the …rst step allows for endogeneity or is a conditional choice probability inan economic structural model. We expect these estimators to have the advantages mentioned
above, that model selection methods can be used in the …rst step, the estimators have appealingrobustness properties, smaller bias and MSE, are less sensitive to bandwidth, have closer tonominal coverage for con…dence intervals, and standard errors that can be easily computed.
Section 2 describes the general construction of locally robust moment functions for semiparametric GMM. Section 3 shows how the …rst step adjustment term can be derived and shows the
local robustness of the adjusted moments. Section 4 introduces local double robustness, showsthat a¢ ne, locally robust moment functions are doubly robust, and gives new classes of doubly
robust estimators. Section 5 describes how locally robust moment functions have the small bias
property and a smaller remainder term. Section 6 considers …rst step series estimation. Section 7 characterizes locally robust moments based on conditional moment restrictions. Section
8 gives locally robust moment conditions for conditional choice probability estimation discretegame and dynamic discrete choice models. Section 9 gives asymptotic theory.

42Constructing Locally Robust Moment Functions

The subject of this paper is GMM estimators of parameters where the sample moment functions
depend on a …rst step nonparametric estimator. We refer to these estimators as semiparametric.
We could also refer to them as GMM where nonparametric estimators are “plugged in”the moments. This terminology seems awkward though, so we simply refer to them as semiparametric
GMM estimators. We denote such an estimator by ^, which is a function of the data z1 ; :::; znwhere n is the number of observations. Throughout the paper we will assume that the data
observations zi are i.i.d. We denote the object that ^ estimates as 0 , the subscript referring to
the parameter value under the distribution that generated the data.
To describe the type of estimator we consider let m(z; ; ) denote an r 1 vector of functionsof the data observation z; parameters of interest , and a function that may be vector valued.
The function can depend on and z through those arguments of m: Here the function
represents some possible …rst step, such as an estimator, its limit, or a true function. A GMM
estimator can be based on a moment condition where 0 is the unique parameter vector satisfying
E[m(zi ;0;0 )] = 0;(2.1)

and 0 is the true . Here it is assumed that this moment condition identi…es : Let ^ denote
some nonparametric estimator of 0 . Plugging in ^ to obtain m(zi ; ; ^ ) and averaging over ziP^ a positive semi-de…nite
gives the estimated sample moments m(^ ) = ni=1 m(zi ; ; ^ )=n: For W
weighting matrix a semiparametric GMM estimator is^ = arg min m(^ m(^ )T W^ );2B

where AT denotes the transpose of a matrix A and B is the parameter space for .
^ that minimizes the asymptotic variance of pn( ^As usual a choice of W0 ) will bepa consistent estimator of the inverse of the asymptotic variance of nm(^ 0 ): Of course that^e¢ cient W will include adjustment terms for the …rst step nonparametric estimator ^ . This^ also gives an e¢ cient estimator in the wider sense shown in Ackerberg, Chen, Hahn,
optimal W^ makes ^ e¢ cient in a semiparametric model where the onlyand Liao (2014). The optimal W
restrictions imposed are equation (2.1).
To explain and analyze local robustness we consider limits when the true distribution of asingle observation zi is F , and how those limits vary with F over a general class of distributions.
This kind of analysis can be used to derive the asymptotic variance of semiparametric estimators,
as in Newey (1994), and is also useful here. Let (F ) denote the limit of ^ when F is the true
distribution of zi . Here (F ) is understood to be the limit of ^ under general misspeci…cation
where F need not satisfy the conditions used to construct ^ : We also consider parametric models
F where denotes a vector of parameters, with F equal to the true distribution F0 at = 0:
5

We will restrict each parametric model to be regular in the sense used in the semiparametrice¢ ciency bounds literature, so that F has a score S(z) (derivative of the log-likelihood in manycases, e.g. see Van der Vaart, 1998, p. 362) at = 0 and possibly other conditions are satis…ed.
We also require that the set of scores over all regular parametric family has mean square closurethat includes all functions with mean zero and …nite variance. Here we are assuming that theset of scores for regular parametric models is unrestricted, the precise meaning of the domain of(F ) being a general class of distributions. We de…ne local robustness in terms of such familiesof regular parametric models.
Definition 1: The moment functions m(z; ; ) are locally robust if and only if for all
regular parametric models@E[m(zi ; 0 ; (F ))]
= 0:
@
=0
This zero pathwise derivative condition means that moment conditions are nearly zero as the…rst step limit (F ) departs from the truth 0 along any path (F ): Below we use a functional
derivative condition, but for now this pathwise derivative de…nition is convenient. Throughout
the remainder of the paper we evaluate derivatives with respect to at = 0 unless otherwise
speci…ed.
In general, locally robust moment functions can be constructed by adding to moment functions the term that adjusts for (or accounts for) …rst step estimation. Under conditions discussed
below there is a unique vector of functions (z; ; ) such that E[ (zi ; 0 ; 0 )] = 0 and
p

1 X
nm(
^ 0) = pm(zi ;
n i=1
n

1 X
p
;
^
)
=
fm(zi ;
0
n i=1
n

0;

0) +

(zi ;

0;

0 )g + op (1):(2.2)

Here (z; 0 ; 0 ) adjusts for the presence of ^ in m(z; 0 ; ^ ). Locally robust moment functions
can be constructed by adding (z; ; ) to m(z; ; ) to obtain new moment functionsg(z; ; ) = m(z; ; ) + (z; ; ):For g^( ) =Pn

i=1 g(zi ;(2.3); ^ )=n; a locally robust semiparametric GMM estimator is obtained as^ = arg min g^( )0 W
^ g^( ):In a parametric setting it is easy to see how adding the adjustment term for …rst step
estimation gives locally robust moment conditions. Suppose that the …rst step estimator ^ isa function of a …nite dimensional vector of parameters where there is a vector of functionsh(z; ) satisfying E[h(zi ; 0 )] = 0 and the …rst step parameter estimator ^ satis…es
1 X
ph(zi ; ^ ) = op (1):
n i=1
n

6

(2.4)

For H = @E[h(zi ; )]=@ j = 0 the usual expansion givesp

1 X
h(zi ;
H 1p
n i=1
n

n( ^

0) =

0 ) + op (1):For notational simplicity let the moment functions depend directly on (rather than ( )) andso take the form m(z; ; ): Let M = @E[m(zi ; 0 ; 0 )]=@ : Another expansion givesp

1 X
m(zi ;nm(^ 0) = p
n i=1
n

0;

0) + M0;0)

1 X
fm(zi ;
=p
n i=1

p

n( ^

0 ) + op (1)nM H 1 h(zi ;0 )g + op (1):Here we see that the adjustment term is(z; ; ) =M H 1 h(z; ):(2.5)

We can add this term to the original moment functions to produce new moment functions of
the formg(z; ; ) = m(z; ; ) + (z; ; ) = m(z; ; ) M H 1 h(z; ):Local robustness of these moment functions follows by the chain rule and@E[g(zi ;
@

0;

)]

=
= 0

@E[m(zi ;0;

M H 1 h(zi ; )]

)
@

=M

M H 1 H = 0:

= 0

Neyman (1959) used scores and the information matrix to form such g(z; ; ) in a parametric
likelihood setting, where g(z; 0 ; 0 ) has an orthogonal projection interpretation. There thepurpose was to construct tests, based on g(z; ; ); where estimation of the nuisance parameters
did not a¤ect the distribution of the tests. The form here was given in Wooldridge (1991) for
nonlinear least squares and Lee (2005), Bera et. al (2010), and Chernozhukov et. al. (2015)
for GMM. What appears to be new here is construction of locally robust moment functions byadding the adjustment term to original moment functions.
In general the adjustment term (z; ; ) may depend on unknown components that are
not present in the original moment functions m(z; ; ). In the above parametric example thematrix M H 1 is unknown so its elements should really be included in , along with : If we
do this local robustness will continue to hold with these additional components of because
E[h(zi ; 0 )] = 0: For notational simplicity we will take to be the …rst step for the locallyrobust moment functions g(z; ; ) = m(z; ; ) + (z; ; ), with the understanding (z; ; )will generally depend on …rst step functions that are not included in m(z; ; ).
In general semiparametric settings the form of the adjustment term (z; ; ) and local
robustness of g(z; ; ) can be explained in terms of in‡uence functions. We will do so in the7next Section. In many interesting cases the form of the adjustment term (z; ; ) is alreadyknown, allowing construction of locally robust estimators. We conclude this section with animportant class of examples.
The class of examples we consider is one where the …rst step ^1 is based on a conditional
moment restriction E[ (zi ; 10 )jxi ] = 0 for a residual (z; 1 ) and instrumental variables x. The
conditional mean or median of yi given xi are included as special cases where (z; 1 ) = y
1 (x)and (z; 1 ) = 2 1(y < 1 (x)) 1 respectively, as are versions that allow for endogeneity where 1
depends on variables other than x. We take ^1 to have the same limit as the nonparametric twostage least squares (NP2SLS) estimator of Newey and Powell (1989, 2003) and Newey (1991).
Thus, ^1 has limit 1 (F ) satisfying
1 (F ) = arg min EF [fEF [12(zi ; 1 )jxi ]g2 ];and EF denotes the expectation under the distribution F . Suppose that there is 20 (x) in themean square closure of the set of derivatives @E[ (zi ; 1 (F ))jxi ]=@ as F varies over regularparametric models such that
@E[m(zi ; 0 ; 1 (F ))]=@E[ 20 (xi )@E[ (zi ; 1 (F ))jxi ]
]:
@

(2.6)

Then from Ichimura and Newey (2016) the adjustment term is(z; ; ) =2 (x;) (z; 1 ):(2.7)

A function 20 (x) satisfying equation (2.6) exists when the set of derivatives @E[ (zi ; 1 (F ))jxi ]=@is linear as F varies over parametric models, @E[m(zi ; 0 ; 1 (F ))]=@ is a linear functionalof @E[ (zi ; 1 (F ))jxi ]=@ ; and that functional is continuous in mean square. Existence of
20 (x) then follows from the Riesz representation theorem. Special cases of this characterization of 20 (x) are in Newey (1994), Ai and Chen (2007), and Ackerberg, Chen, Hahn, andLiao (2014). When @E[m(zi ; 0 ; 1 (F ))]=@ is not a mean square continuous functional of@E[ (zi ; 1 (F ))jxi ]=@ then …rst step estimation should make the moments converge slowerpthan 1= n, as shown by Newey and McFadden (1994) and Severini and Tripathi (2012) for
special cases. For linear functionals Santos (2011) gives a more general result that allows fornonidenti…cation of 10 :There are a variety of ways to construct an estimator (zi ; ; ^ ) of the adjustment term to
be used in forming locally robust moment functions, see NHR and Ichimura and Newey (2016).
A relatively simple and general one when the …rst step is a series or sieve estimator is to treat
the …rst step as if it were parametric and use the parametric formula in equation (2.5). Thisapproach to estimating the adjustment term is known to be asymptotically valid in a variety
of settings, see Newey (1994), Ackerberg, Chen, and Hahn (2012), and Ichimura and Newey(2016). For completeness we give a brief description here.
8We parameterize an approximation to 1 as 1 = ( ) where is a …nite dimensional vectorof parameters as before. Let m(zi ; ; ) = m(zi ; ; 1 ( )) and ^ i denote an estimator of 0
solving E[h(zi ; 0 )] = 0; that is allowed to depend on observation i: Being a series or sieve
estimator the dimension of and hence of h(z; ) will increase with sample size. Also, let I(i)be a set of observation indices that can also depend on i: An estimator of the adjustment term
is give by(zi ; ; ^ ) =^ i ( )H
^ 1 h(zi ; ^ i ); M^i ( ) =Mi

X @h(zj ; ^ i )X @m(zj ; ; ^ i )
^i =
;H
:
@
@

j2I(i)

j2I(i)^ i , and H^ i depend only on observations other
This estimator allows for cross …tting where ^ i , M
than the ith : cross …tting is known to improve performance in some settings, such as in "leave
one out" kernel estimators of averages, e.g. see NHR. This adjustment term will lead to locallyrobust moments in a variety of settings, as further discussed below.

3In‡uence Functions, Adjustment Terms, and Local RobustnessIn‡uence function calculations can be used to derive the form of the adjustment term (z; ; )and show local robustness of the adjusted moment functions g(z; ; ) = m(z; ; ) + (z; ; ):To explain in‡uence functions note that many estimators are asymptotically equivalent to asample average. The object being averaged is the unique in‡uence function. For example, in
equation (2.2) we are assuming that the in‡uence function of m(^ 0 ) is g(z; 0 ; 0 ) = m(z; 0 ; 0 )+(z; 0 ; 0 ): This terminology is widely used in the semiparametric estimation literature.
In general an estimator ^ of a true value 0 and its in‡uence function (z) satisfyp1 X(zi ) + op (1); E[ (zi )] = 0; E[ (zi ) (zi )0 ] exists.
0) = p
n i=1
n

n(^

The function (z) can be characterized in terms of the functional (F ) that is the limit of ^under general misspeci…cation where F need not satisfy the conditions used to construct ^. As
before, we allow F to vary over a family of regular parametric models where the set of scores forthe family has mean square closure that includes all mean zero functions with …nite variance. As
shown by Newey (1994) the in‡uence function (z) is then the unique solution to a derivativeequation of Van der Vaart (1991),@ (F )= E[ (zi )S(zi )]; E[ (zi )] = 0;
(3.1)@
as F (and hence S(z)) varies over the general family of regularly parametric models. Ichimura
and Newey (2016) also showed that when (z) has certain continuity properties it can be9computed as@ (F h ) h; F = (1)F0 + Ghz ;
(3.2)
h !0
@
where Ghz is constructed so that F h is in the domain of (F ) and Ghz approaches the point mass
at z as h ! 0:
These results can be used to derive the adjustment term (z; ; ) and to explain local
robustness. Let (F ) denote the limit of the …rst step estimator ^ under general misspeci…cation
when a single observation has CDF F; as discussed above. From Newey (1994, pp. 1356-1357) weknow that the adjustment term (z; 0 ; 0 ) is the in‡uence function of (F ) = E[m(zi ; 0 ; (F ))]where E[ ] denotes the expectation at the truth. Thus (z; ; ) can be calculated as in equation(3.2) for that (F ): Also (z; ; ) satis…es equation (3.1) for (F ) = E[m(zi ; 0 ; (F ))]; i.e. forthe score S(z) at = 0 for any regular parametric model fF g;(z) = lim

@E[m(zi ; 0 ; (F ))]= E[ (zi ;@Also, (z;0;0;0 )S(zi )]; E[

(zi ;

0;

0 )] = 0:(3.3)

0 ) can be computed as@E[m(zi ; 0 ; (F h ))] h; F = (1)F0 + Ghz ;h !0@The characterization of (z; 0 ; 0 ) in equation (3.3) can be used to specify another local
robustness property that is equivalent to De…nition 1. We have de…ned local robustness as thederivative on the left of the …rst equality in equation (3.3) being zero for all regular parametricmodels. If that derivative is zero for all parametric models then (z; 0 ; 0 ) = 0 is the uniquesolution to this equation, by the set of scores being mean square dense in the set of mean zerorandom variables with …nite variance. Also, if (z; 0 ; 0 ) = 0 then the derivative on the left isalways zero. Therefore we have(z;

Proposition 1:

0;

0) =

(z;

0;

lim

0 ) = 0 if and only ifm(z; ; ) is locally robust.

Note that (z; ; ) is the term in the in‡uence function of m(^ 0 ) that accounts for the …rst
step estimator ^ : Thus Proposition 1 gives an alternative characterization of local robustness,that …rst step estimation does not a¤ect the in‡uence function of m(^ 0 ). This result is asemiparametric version of Theorem 6.2 of Newey and McFadden (1994). It also formalizes thediscussion in Newey (1994, pp. 1356-1357).
Local robustness of the adjusted moment function g(z; ; ) = m(z; ; ) + (z; ; ) follows from Proposition 1 and (z; ; ) being a nonparametric in‡uence function. Because
def(z; ; ) is an in‡uence function it has mean zero at all true distributions, i.e.
(F ) =
R(z; 0 ; (F ))F (dz)
0 identically in F: Consequently the derivative in equation (3.1) iszero, so that (like Proposition 1) the in‡uence function of (F ) is zero. Consequently, under10

appropriate regularity conditionspn = op (1): It then follows that
1 X
pg(zi ;n i=1
n0; ^) =pnm(
^ 0) +

=

p

Pn

i=1

n =

p

(zi ;

0 ; ^ )=n has a zero in‡uence function and so

1 X
g(zi ;nm(^ 0 ) + op (1) = p
n i=1
n

0;

0 ) + op (1);(3.4)

where the last equality follows by equation (2.2). Here we see that the adjustment term is zerofor the moment functions g(z; ; ). From Proposition 1 with g(z; ; ) replacing m(z; ; ) itthen follows that g(z; ; ) is locally robust.
Proposition 2: For the in‡uence function (z; 0 ; 0 ) of (F ) = E[m(zi ;adjusted moment function g(z; ; ) = m(z; ; ) + (z; ; ) is locally robust.

0;(F ))] theR(z; 0 ; (F ))F (dz) 0
Local robustness of g(z; ; ) also follows directly from the identity
as discussed in the Appendix. Also, the adjusted moments g^( 0 ) have the same asymptotic
variance as the original moments, as in the second equality of equation (3.4). That is, adding(z; ; ) to m(z; ; ) does not a¤ect the asymptotic variance. Thus the asymptotic bene…tsof the locally robust moments are in their higher order properties. Other modi…cations of
the moments may also improve higher-order properties of estimators, such as the cross …tting
described above (like "leave on out" in NHR) and the higher order bias corrections in Robins
et. al. (2008) and Cattaneo and Jansson (2014).

4

Local and Double Robustness

The zero derivative condition in De…nition 1 is an appealing robustness property in and of itself.
Mathematically a zero derivative is equivalent to the moments remaining closer to zero than as
varies away from zero. This property can be interpreted as local robustness of the moments tothe value of being plugged in, with the moments remaining close to zero as varies away fromits true value. Because it is di¢ cult to get nonparametric functions exactly right, especially inhigh dimensional settings, this property is an appealing one.
Such robustness considerations, well explained in Robins and Rotnitzky (2001), have motivated the development of doubly robust estimators. For our purposes doubly robust moments
have expectation zero if just one …rst stage component is incorrect. When there are only two …rst
stage components this means that the moment conditions hold when only one of the …rst stage
components is correct. Doubly robust moment conditions allow two chances for the moment
conditions to hold.
It turns out that locally robust moment functions are automatically doubly robust in a
local sense that the derivative with respect to each individual, distinct …rst stage component
is zero. In that way the moment conditions nearly hold as each distinct component varies in11

a neighborhood of the truth. Furthermore, when locally robust moment functions are a¢ nefunctions of a distinct nonparametric component they are automatically globally robust in thatcomponent. Thus, locally robust moment functions that are a¢ ne in each distinct nonparametriccomponent are doubly robust.
These observations suggest a way to construct doubly robust moment functions. Startingwith any two step semiparametric moment function we can add the adjustment term to get a
locally robust moment function. When we can choose a …rst step of that moment function sothat it enters in an a¢ ne way the new moment function will be doubly robust in that component.
To give these results we need to de…ne distinct components of : A distinct component is
one where there are parametric models F with that component varying in an unrestricted way
but the other components of not varying. For a precise de…nition we will focus on the …rst
component 1 of = ( 1 ; :::; J ).
Definition 2: A component

1 ofis distinct if and only if there is F such that

(F ) = ( 1 (F ); 20 ; :::; J0 );and

1 (F ) is unrestricted as F

varies across parametric models.

An example is the moment function g(z; ; 1 ; 2 ) = m(z; ; 1 ) + 2 (x; ) (z; 1 ), whereE[ (zi ; 10 )jxi ] = 0: In that example the two components 1 and 2 are often distinct because 1
depends only on the conditional distribution of zi given xi and 20 (x; ) depends on the marginal
distribution of xi in an unrestricted way.
Local robustness means that the derivative must be zero for any model, so in particular it
must be zero for any model where only the distinct component is varying. Thus we have
Proposition 3: If 1 is distinct then for g(z; ; ) = m(z; ; ) + (z; ; ) and regularparametric models F as in De…nition 2,@E[g(zi ;0;

1 (F );

20 ; :::;

@

J0 )]

= 0:

This result is an application of the simple fact that when a multivariable derivative is zero
the partial derivative must be zero when the variables are allowed to vary in an unrestricted
way. Although this fact is simple, it is helpful in understanding when local robustness holds
for individual components. This means that locally robust moment functions automaticallyhave a local double robustness property, that the expectation of the moment function remains
nearly zero as each distinct …rst stage component varies away from the truth. For example, fora …rst step conditional moment restriction where g(z; ; ) = m(z; ; 1 ) + 2 (x; ) (z; 1 ), theconclusion of Proposition 3 is@E[m(zi ;0;

1 (F )) +20 (xi )@12(zi ; 1 (F ))]= 0:

In fact, this result is implied by equation (2.6), so by construction g(z; ; ) is already locally robust in 1 alone. Local robustness in 2 follows by the conditional moment restriction
E[ (zi ; 10 )jxi ] = 0:
Moments that are locally robust in a distinct component 1 will be globally robust in 1 if
= ( 1 ; 20 ; :::; J0 )0 and
1 enters the moments in an a¢ ne way, meaning that for any 1 andany z,g(z; ; (1) 0 + ) = (1) g(z; ; 0 ) + g(z; ; ):(4.1)
Global robustness holds because an a¢ ne function with zero derivative is constant. For simplicity
we state a result when F can be chosen so that (F ) = (1) 0+though it will hold moregenerally. Note that hereE[g(zi ;0;(F ))] = (1)E[g(zi ;0;0 )] +

E[g(zi ;0;)] =E[g(zi ;Here the derivative of the moment condition with respect to is just E[g(zi ;3 gives the following result:
Proposition 4: If equation (4.1) is satis…ed and there is F with0
1 ; 20 ; :::; J0 ) then E[g(zi ; 0 ; 1 ; 20 ; :::; J0 )] = 0:

0;

0;

)]:

)] so Proposition(F ) = ((1) 10 +

Thus we see that locally robust moment functions that are a¢ ne in a distinct …rst step
component are globally robust in that component. This result includes many existing examplesof doubly robust moment functions and can be used to construct new ones.
A general class of doubly robust moment functions that appears to be new and includes
many new and previous examples has …rst step satisfying a conditional moment restriction
E[ (zi ; 10 )jxi ] = 0 where (z; 1 ) and m(z; 0 ; 1 ) are a¢ ne in 1 : Suppose that E[m(zi ; 0 ; 1 )]is a mean-square continuous linear functional of E[ (zi ; 1 )jxi ] for 1 in a linear set : Then by
the Riesz representation theorem there is (x) in the mean square closure of the image ofE[ (zi ; 1 )jxi ] such thatE[m(zi ;0;1 )] =

E[ (xi )E[ (zi ; 1 )jxi ]] =E[ (xi ) (zi ; 1 )]; 1 2 :(4.2)

Let 20 (x) be any function such that 20 (xi )(xi ) is orthogonal toand g(z; ; ) =m(z; ; 1 ) + 2 (x; ) (z; 1 ): Then E[g(zi ; 0 ; 1 ; 20 )] = 0 by the previous equation. It also
follows that E[g(zi ; 0 ; 10 ; 2 )] = 0 by E[ (zi ; 10 )jxi ] = 0: Therefore g(z; ; 1 ; 2 ) is doubly
robust, showing the following result:
Proposition 5: If m(zi ; 0 ; 1 ) and (zi ; 1 ) are a¢ ne in 1 2withlinear andE[m(zi ; 0 ; 1 )] is a linear, mean square continuous functional of E[ (zi ; 1 )jxi ] then there is
20 (x) such that g(z; ; 1 ; 2 ) = m(z; ; 1 ) + 2 (x; ) (z; 1 ) is doubly robust.
13Section 3 of Robins et. al. (2008) gives necessary and su¢ cient conditions for a moment
function to be doubly robust when 1 and 2 enter the moment functions as functions evaluatedat observed x: Proposition 5 is complementary to that work in deriving the form of doubly robust
moment functions when the …rst step satis…es a conditional moment restriction and m(z; ; 1 )can depend on the entire function 1 .
It is interesting to note that 20 such that E[g(z; 0 ; 1 ; 20 )] = 0 for all 1 2 is not uniquewhen does not include all functions of x, the overidenti…ed case of Chen and Santos (2015).
This nonuniqueness can occur when there are multiple ways to estimate the …rst step 10 using
the conditional moment restrictions E[ (zi ; 10 )jxi ] = 0. As discussed in Ichimura and Newey(2016), the di¤erent 20 (xi ) correspond to di¤erent …rst step estimators, with 20 (xi ) = (xi )corresponding to the NP2SLS estimator.
An important case is a linear conditional moment restrictions setup up like Newey and Powell(1989, 2003) and Newey (1991) where(z; 1 ) = y

1 (w); E[yi10 (wi )jxi ] = E[

(4.3)

(zi ; 10 )jxi ] = 0:

Consider a moment function equal to m(z; ; 1 ) = v(w) 1 (w)for some known functionv(w); where the parameter of interest is 0 = E[v(wi ) 10 (wi )]. If there is (x) such that v(wi ) =E[ (xi )jwi ] then we haveE[m(zi ;0;1 )] = E[v(wi )f 1 (wi )= E[ (xi )f 1 (wi )10 (wi )g] = E[E[10 (wi )g] =(xi )jwi ]f 1 (wi )10 (wi )g]

E[ (xi ) (zi ; 1 )]:It follows that g(z; ; ) = m(z; ; 1 ) + 2 (x) (z; 1 ) is doubly robust for 20 (x) = (xi ). Interestingly, the existence of with v(wi ) = E[ (xi )jwi ] is a necessary condition for root-n consistentestimability of 0 as in Severini and Tripathi’s (2012, Lemma 4.1). We see here that a doubly
robust moment condition can always be constructed when this necessary condition is satis…ed.
Also, similarly to the above, the 20 (x) may not be unique.
Corollary 6: If m(z; ; 1 ) = v(w) 1 (w), equation ( 4:3) is satis…ed, and there is (x)such that v(w) = E[ (x)jw] then g(z; ; 1 ; 2 ) = v(w) 1 (w)+ 2 (x)[y1 (w)] is doubly
robust for 20 (x)(x) orthogonal to .
A new example of a doubly robust moment condition corresponds to the weighted averagederivative of 10 (w) of Ai and Chen (2007). Here m(z; ; 1 ) = v(w)@ 1 (w)=@w
for somefunction v(w). Let f0 (w) be the pdf of wi . Assuming that v(w) 1 (w)f0 (w) is zero on theboundary of the support of wi ; integration by parts gives
E[m(zi ;0;1 )] = E[v(wi )f 1 (wi )10 (wi )g]; v(w) = f0 (w)14

1@[v(w)f0 (w)]=@w:

Assume that there exists (x) such that v(wi ) = E[ (xi )jwi ]. Then as in Proposition 5 a doubly
robust moment function isg(z; ; ) = v(w)

@ 1 (w)@w+

2 (x)[y1 (w)]:

A special case of this example is the doubly robust moment condition for the weighted averagederivative in the exogenous case where wi = xi given in Firpo and Rothe (2016).
Doubly robust moment conditions can be used to identify parameters of interest. In general,if g(z; ; 1 ; 2 ) is doubly robust and 20 is identi…ed then 0 may be identi…ed fromE[g(zi ;for any …xed

1 when the solution

0;1;20 )] = 0;

0 to this equation is unique.

Proposition 7: If g(z; ; 1 ; 2 ) is doubly robust, 20 is identi…ed, and for some 1 theequation E[g(zi ; ; 1 ; 20 )] = 0 has a unique solution then 0 is identi…ed as that solution.
Applying this result to the NPIV setting gives an explicit formula for certain functionals of
10 (w) without requiring that the completeness identi…cation condition of Newey and Powell(2003) be satis…ed, similarly to Santos (2011). Suppose that v(w) is identi…ed, e.g. as for theweighted average derivative. Since both w and x are observed it follows that a solution 20 (x)to v(w) = E[ 20 (x)jw] will be identi…ed if such a solution exists. Plugging in 1 = 0 in theequation E[g(zi ; 0 ; 1 ; 20 )] = 0 gives
Corollary 8: If v(wi ) is identi…ed and there exists 20 (xi ) such that v(wi ) = E[ 20 (xi )jwi ]then 0 = E[v(wi ) 10 (wi )] is identi…ed as 0 = E[ 20 (xi )yi ].
Note that this result holds without the completeness condition. Identi…cation of 0 =E[v(wi ) 10 (wi )] for known v(wi ) with v(wi ) = E[ 20 (xi )jwi ] follows from Severini and Tripathi(2006). The formula 0 = E[ 20 (xi )yi ] could be used to construct an estimator of 0 from anestimator for 2 ; see Santos (2011) for such construction and asymptotic results.
Another new example of a doubly robust estimator is a weighted average over income valuesof an average (across heterogenous individuals) of exact consumer surplus bounds, as in Hausman
and Newey (2016). Here y is quantity consumed, w = x = (x1 ; x2 )0 ; x1 is price, x2 is income,10 (xi ) = E[yi jxi ], price is changing between x1 and x1 , and B is a bound on the incomee¤ect. Let v2 (x2 ) be some weight function and v1 (x1 ) = 1(x1x1x1 )e B(x1 x1 ) : For theRmoment function m(z; ; 1 ) = v2 (x2 ) v1 (u) 1 (u; x2 )duthe true parameter 0 is a boundon the average of equivalent variation over unobserved individual heterogeneity and income. Let15

f10 (x1 jx2 ) denote the conditional pdf of x1 given x2 . Note thatZ
E[m(zi ; 0 ; 1 )] = E[v2 (x2i ) v1 (u)[ 1 (u; x2i )10 (u; x2i )]du]= E[f10 (x1i jx2i ) 1 v1 (x1i )v2 (x2i )f 1 (xi )=E[ 20 (xi )E[yi1 (xi )jxi ]];10 (xi )g]

20 (x) = f10 (x1 jx2 )1

v1 (x1 )v2 (x2 ):Then it follows by Proposition 5 that a doubly robust moment function isZg(z; ; ) = v2 (x2 ) v1 (u) 1 (u; x2 )du+ 2 (x)[y1 (x)]:
When the moment conditions are formulated so that they are a¢ ne in the …rst step Proposition 4 applies to many previously developed doubly robust moment conditions. Data missing
at random is a leading example. Let 0 be the mean of a variable of interest w where w is notalways observed, y 2 f0; 1g denote an indicator for w being observed, and x a vector of covariates. Assume w is mean independent of y conditional on covariates x. We consider estimating
0 using the propensity score P0 (xi ) = Pr(yi = 1jxi ). We specify an a¢ ne conditional moment
restriction by letting 1 (x) = 1=P (x) and (z; 1 ) = 1 (x)y 1: We have 0 = E[ 10 (xi )yi wi ]; asis well known. An a¢ ne moment function is then m(z; ; 1 ) = 1 (x)yw: Note thatE[m(zi ;0;1 )] = E[E[yi wi jxi ]f 1 (xi )20 (xi ) =10 (xi )g] =E[ 20 (xi ) (zi ; 1 )];10 (xi )E[yi wi jxi ]:Then Proposition 5 implies that a doubly robust moment function is given byg(z; ; ) =1 (x)yw

2 (x)[ 1 (x)y1]:This is the well known doubly robust moment function of Robins, Rotnitzky, and Zhao (1994).
This example illustrates how applying Propositions 4 and 5 require specifying the …rst step
so that the moment functions are a¢ ne. These moment conditions were originally shown to be
doubly robust when the …rst step is taken to be the propensity score P (x). Propositions 4 and5 only apply when the …rst step is taken to be 1=P (x). More generally we expect that particular
formulations of the …rst step may be needed to make the moment functions a¢ ne in the …rst
step and so use Propositions 4 and 5 to derive doubly robust moment functions.
Another general class of doubly robust moment functions depend on the pdf 1 of a subset of variables xi and are a¢ ne in 1 : An important example of such a moment function
R
is the average where 0 = f0 (x)2 dx and m(z; ; 1 ) = 1 (x): Another is the density
weighted average derivative (WAD) of Powell, Stock, and Stoker (1989) where m(z; ; 1 ) =2y @ 1 (x)=@x. Assume that E[m(zi ; 0 ; 1 )] is a function of 1
10 that is continuous in16

the normwithR[ 1 (u)1=22: Then by the Riesz representation theorem there is
10 (u)] duE[m(zi ;0;1 )] =Z

20 (u)[ 1 (u)10 (u)]du:

The adjustment term for m(z; ; ), as in Proposition 3 of Newey (1994), is (z; ; ) =R
2 (u) 1 (u)du: The corresponding locally robust moment function isZg(z; ; 1 ; 2 ) = m(z; ; 1 ) + 2 (x)2 (u) 1 (u)du:20 (x)

(4.4)
2 (x)

(4.5)

This function is a¢ ne in 1 and 2 separately so when they are distinct Proposition 4 implies
double robustness. Double robustness also follows directly fromZ
Z
Z
E[g(zi ; 0 ; )] =20 (u)[ 1 (u)10 (u)]du +2 (u) 10 (u)du2 (u) 1 (u)duZ=[ 2 (u)20 (u)][ 1 (u)10 (u)]du:Thus we have the following result:
Proposition 9: If m(zi ; ; 1 ) is a¢ ne in 1 and E[m(zi ; 0 ; 1 )] is a linear function ofR
1=22
f 1 (x); then for 20 (x) from equation
10 (x)g dx
1
10 that is continuous in the normR(4.4), g(z; ; ) = m(z; ; 1 ) + 2 (x)2 (u) 1 (u)du is doubly robust.

We can use this result to derive doubly robust moment functions for the WAD. Let (xi ) =E[yi jxi ] 10 (xi ). Assuming that (u) 1 (u) is zero on the boundary, integration by parts givesZ
E[m(zi ; 0 ; 1 )] = 2E[yi @ 1 (xi )=@x][@ (u)=@u]f 1 (u)
0 = 2
10 (u)gdu;
so that(x)=@x: A doubly robust moment condition is thenZ
@ 1 (x)@ (x)@ (u)g(z; ; ) = 2y+2
2
1 (u)du:@x@x@u

20 (x) = 2@

The double robustness of this moment condition appears to be a new result. As shown in Newey,
Hsieh, and Robins (1998), a "delete-one" symmetric kernel estimator based on this moment
function gives the twicing kernel estimator of NHR. Consequently the MSE comparisons of
NHR for twicing kernel estimators with the original kernel estimator correspond to comparisonof a doubly (and locally) robust estimator with one based on unadjusted moment conditions, as
discussed in the introduction.
It is interesting to note that Proposition 9 does not require that 1 and 2 are distinct
nonparametric components. For the average density 1 (x) and 2 (x) both represent the marginaldensity of x and so are not distinct. Nevertheless the moment function g(z; 0 ; ) = 1 (x)0+172 (x)

R

1 (u) 2 (u)du is doubly robust, having zero expectation if either

1 or

2 is correct.

This example shows a moment function may be doubly robust even though 1 and 2 are notdistinct. Thus, there are doubly robust moment functions that cannot be constructed using
Proposition 4.
All of the results of this Section continue to hold with cross …tting. That is true because theresults of this Section concern the moment and their expectation at various values of the …rst
step, and not the particular way in which the …rst step is formed.

5

Small Bias of Locally Robust Moment Conditions

Adding the adjustment term improves the higher order properties of the estimated moments
though it does not change their asymptotic variance. An advantage of locally robust moment
functions is that nonparametric smoothing bias is relatively small. To describe this advantage it
is helpful to modify the de…nition of local robustness. In doing so we allow F to represent a moregeneral object, an unsigned measure (charge). Let k k denote a seminorm on F (a seminorm
has all the properties of a norm but may be zero when F is not zero). Also, let F be a set of
charges where m(z; 0 ; (F )) is well de…ned.
Definition 3: m(z; ; ) is locally robust if and only if E[m(zi ;for F 2 F.

0;(F ))] = o(kF

F0 k)

De…nition 1 requires that (F ) have a zero pathwise derivative. De…nition 3 requires a zero
Frechet derivative for the semi-norm k k, generally a stronger condition than a zero pathwise
derivative. The zero Frechet derivative condition is helpful in explaining the bias properties of
locally robust moment functions.
Generally a nonparametric estimator will depend on some vector of smoothing parameters
b. This b could be the bandwidth in a kernel estimator or the inverse of number of terms ina series estimator. Suppose that the limit of ^ for …xed b is (Fb ) where Fb is a "smoothed"version of the true distribution that approaches the truth F0 as b ! 0. Then under regularity
conditions m(^ 0 ) will have limit E[m(zi ; 0 ; (Fb ))]: We can think of kFb F0 k as a measureof the nonparametric (smoothing) bias in Fb . Similarly E[m(zi ; 0 ; (Fb ))] is a measure of thebias in the moment conditions caused by smoothing. The small bias property (SBP) analyzed
in NHR is that the expectation of the moment functions vanishes faster than the nonparametricbias as b ! 0:
Definition 4: m(z; ; ) and Fb have the small bias property if and only if E[m(zi ;o(kFb F0 k) as b ! 0.

18

0;(Fb ))] =As long as Fb 2 F, the set F in De…nition 3, locally robust moments will have bias thatvanishes faster than the nonparametric bias kFb F0 k as b ! 0: Thus locally robust moment
functions have the small bias property.
Proposition 10: If m(z; ; ) is locally robust then m(z; ; ) has the small bias property
for any Fb 2 F.
Note that the bias of locally robust moment conditions will be ‡at as a function of thenonparametric bias kFb F0 k as that goes to zero. This ‡atter moment bias can also make theMSE ‡atter, meaning the MSE of the estimator does not depend as strongly on kFb F0 k for
locally robust moments as for other estimators.
By comparing De…nitions 3 and 4 we see that the small bias property is a form of directional
local robustness, with the moment being locally robust in the direction Fb . If the moments are
not locally robust then there will be directions where the bias of the moments is not smaller
than the smoothing bias Fb : Being locally robust in all directions can be important when the…rst step is allowed to be very ‡exible, such as when machine learning is used to construct the…rst step. There the …rst step can vary randomly across a large class of functions making theuse of locally robust moments important for correct inference, e.g. see Belloni, Chernozhukov,
and Hansen (2014).
This discussion of smoothing bias is based on sequential asymptotics where we consider
limits for …xed b. This discussion provides useful intuition but it is also important to consider
asymptotics where b could be changing with the sample size. We can analyze the precise e¤ectof using locally robust moments by considering an expansion of the average moments. Let m =PnPn~
i=1 g(zi ; 0 ; 0 )=n; (z) = (z; 0 ; 0 ), and F denote the empirical
i=1 m(zi ; 0 ; 0 )=n, g =distribution. We suppose that ^ = (F^ ) for some estimator F^ of the true distribution F0 . Let(F ) = E[m(zi ; 0 ; (F ))]. By adding and subtracting terms we have
Z
~
^
^
~
(z)[F^ F~ ](dz);(5.1)
m(^ 0 ) = g + R1 + R2 + R3 ; R1 =Z^^
^ 3 = m(R2 = (F )(z)F^ (dz); R^ 0) m(F^ ):
RR^ 2 isThe object(z)F^ (dz) =(z)[F^ F0 ](dz) is a linearization of (F^ ) = (F^ )(F0 ) so R
^ 3 is a stochastic equicontinuity remainder
a nonlinearity remainder that is second order. Also R
of a type familiar from Andrews (1994) that is also second order.
The locally robust counterpart g^( 0 ) to m(^ 0 ) has a corresponding remainder term that
~ 1 . To see this let ^(z) = (z; 0 ; (F^ )) and note that theis asymptotically smaller than RRmean zero property of an in‡uence function will generally give ^(z)F^ (dz) = 0: Then byRg^( 0 ) = m(
^ 0 ) + ^(z)F~ (dz) we have19

Proposition 11: IfR^(z)F^ (dz) = 0 then g^( 0 ) = g + R^1 + R^2 + R
^3;
^1 =
R

Z

[ ^(z)(z)][F^

F~ ](dz):

Comparing this conclusion with equation (5.1) we see that locally robust moments have the~ 1 has been replaced bysame expansion as the original moments except that the remainder R^ 1 : The remainder R^ 1 will be asymptotically smaller than R~ 1 under su¢ cient
the remainder R
regularity conditions. Consequently, depending on cross correlations with other terms, thelocally robust moments g^( 0 ) can be more accurate than m(^ 0 ). For instance, as shown by
NHR the locally robust moments for linear kernel averages have a higher order bias term thatconverges to zero at a faster rate than the original moments, while only the constant term in
the higher order variance is larger. Consequently, the locally robust estimator will have smaller
MSE asymptotically for appropriate choice of bandwidth. In nonlinear cases the use of locallyrobust moments may not lead to an improvement in MSE because nonlinear remainder terms
may be important, see Robins et. al. (2008) and Cattaneo and Jansson (2014). Nevertheless,using locally robust moments does make smoothing bias small, which can be an important
improvement.
In some settings it is possible to obtain a corresponding improvement by changing the …rst
step estimator. For example, as mentioned earlier, for linear kernel averages the locally robust
estimator is identical to the original estimator based on a twicing version of the original kernel(see NHR). The improvement from changing the …rst step can be explained in relation to the~ 1 ; that is the di¤erence of the integral of (z) over the estimated distribution F^ andremainder R~ 1 E[R~ 1 ] should be aits sample average. Note that F^ F~ will be shrinking to zero so that R~ 1 ] is the most interesting term. If E[F^ ] = Fb
second order (stochastic equicontinuity) term. E[Rand integrals can be interchanged thenZ
Z~
E[R1 ] =(z)Fb (dz) =(z)[Fb F0 ](dz):When a twicing kernel or any other higher order kernel is used this remainder becomes second
order, depending on the smoothness of both the true distribution F0 and the in‡uence function(z); see NHR and Bickel and Ritov (2003). Thus, by using a twicing or higher order kernel
we obtain a second order bias, so all of the remainder terms are second order. Furthermore,
series estimator automatically have a second order bias term, as pointed out in Newey (1994).
Consequently, for all of these …rst steps the remainders are all second order even though themoment function is not locally robust.
The advantage of locally robust moments are that the improvement applies to any …rst
step estimator. One does not have to depend on the particular structure of the estimator,
such as having a kernel of su¢ ciently high order. This feature is important when the …rst
20

step is complicated so that it is hard to analyze the properties of terms that correspond to~ 1 ]: Important examples are …rst steps based on large dimensional model selection. In thatE[Rsetting locally robust moment are very important for obtaining root-n consistency; see Belloni
et. al. (2014). Locally robust moments have the advantages we have discussed even for very
complicated …rst steps.

6

First Step Series Estimators

First step series estimators have certain automatic robustness properties. Moment conditions
based on series estimators are automatically locally robust in the direction of the series approximation. We also …nd that a¢ ne moment functions are automatically doubly robust in these
directions. In this Section we present these results.
It turns out that for certain …rst step series estimators there is a version of the adjustment
term that has sample mean zero, so that g^( ) = m(
^ ): That is, locally robust moments are
numerically identical to the original moments. This version of the adjustment term is constructed by treating the …rst step as if it were parametric with parameters given by those of theseries approximation, and calculating a sample version of the adjustment described in SectionP
2. Suppose that the coe¢ cients ^ of the …rst step estimator satisfy ni=1 h(zi ; ^ )=n = 0. Let^ = n 1 Pn @h(zi ; ^ )=@ ; and^ ( ) = n 1 Pn @m(zi ; ; ^ )=@ ; H
M
i=1
i=1
(z; ; ^ ) =^ ( )H^ 1 h(z; ^ )
M

(6.1)

be the parametric adjustment term described in Section 2, where ^ includes the elements of^ ( ) and H^ and there is no cross …tting. Note thatM
1X
(zi ; ; ^ ) =
n i=1
n

n
X
11
^
^
M ( )Hh(zi ; ^ ) = 0:n i=1

It follows that g^( ) = m(^ ); i.e. the locally robust moments obtained by adding the adjustment
P
term are identical to the original moments. Thus, if ni=1 h(zi ; ^ )=n = 0; we treat the …rst
step series estimator as parametric, and use the parametric adjustment term the locally robust
moments are numerically identical to the original moments. This numerical equivalence results is
an exact version of local robustness of the moments in the direction of the series approximation.
In some settings it is known that (z; ; ^ ) in equation (6.1) is an estimated approximation to(zi ; ; 0 ), justifying its use. Newey (1994, p. 1369) showed that this approximation property
holds when the …rst step is a series regression. Ackerberg, Chen, and Hahn (2012) showedthat this property holds when the …rst step satis…es certain conditional moment restrictions
or is part of a sieve maximum likelihood estimator. It is also straightforward to show that
this approximation holds when the …rst step is a series approximates to the solution to the21conditional moment restriction E[ (zi ; 10 )jxi ] = 0. We expect that in general (z; ; ^ ) is an
estimator of (z; ; 0 ).
We note that the result that g^( ) = m(^ ) is dependent on ^ not varying with the observations
and on being constructed from the whole sample. If we use cross …tting in any form then the
numerical equivalence of the original moments with their locally robust counterpart will generally
not hold. Also g^( ) 6= m(^ ) will generally occur when di¤erent models are used for di¤erent
elements of . Such di¤erent models will often be present when model selection methods, suchas machine learning, are used for constructing the estimators of the di¤erent elements of : Seefor example Chernozhukov et. al. (2016).
There are interesting cases where the original moment functions m(z; ; ) with a series
estimator for are doubly robust in certain directions, with E[m(zi ; 0 ; 1 )] = 0 when 1 isa series approximation to 10 : Here we show this directional double robustness property forseries estimators of solutions to conditional moment restrictions and orthogonal series density
estimators. Consider …rst a conditional moment restriction where the residual (z; 1 ) is a¢ nein 1 , m(z; ; 1 ) is also a¢ ne in 1 , and the …rst step is a linear series estimator. Supposethat the series estimator approximates 10 by a linear combination pK0 of a vector of functions
pK (w) = (p1K (w); :::; pKK (w))0 . Let q K (xi ) be a K 1 vector of instrumental variables and ^P
be the instrumental variables estimator solving a moment condition ni=1 q K (xi ) (zi ; pK0 ^ ) = 0.
Under standard regularity conditions the limitof ^ will solve the corresponding population
moment condition E[q K (xi ) (zi ; pK0 )] = 0: Let 20 (x) satisfy equation (4.2). Then if 20 (xi ) =0 K
q (xi ) for some it follows thatE[m(zi ;

0; p

K0

)] =

E[ 20 (xi ) (zi ; pK0

)] =

0

E[q K (xi ) (zi ; pK0)] = 0:Thus we have the result
Proposition 12: If m(zi ; ; 1 ) and (zi ; 1 ) are a¢ ne in 1 2 with linear, E[m(zi ; 0 ; 1 )]is a mean square continuous functional of E[ (zi ; 1 )jxi ]; and 20 (xi ) satisfying E[m(zi ; 0 ; 1 )] =E[ 20 (xi ) (zi ; 1 )] also satis…es 20 (xi ) = 0 q K (xi ) for some then E[m(zi ; 0 ; pK0 )] = 0.
The property shown in the conclusion of Proposition 12 is a directional double robustness
condition that depends on 1 being equal to a series approximation to 10 and on 20 (x) being
restricted. These restrictions are not required for double robustness of g(z; ; ) = m(z; ; 1 ) +2 (x) (z; 1 ): We will have E[g(zi ; 0 ; 20 ; 1 )] = 0 for all 1 , and not just for the 1 that are aseries approximation to 10 ; and for any 20 (x) and not just one that is a linear combination ofq K (x). For series …rst steps the the original moment functions will be doubly robust in certain
directions just as they are locally robust in certain directions.
Previous examples of Proposition 12 are given in Newey (1990, p. 116), Newey (1999), andRobins et. al. (2007). Proposition 12 allows for endogeneity and m(z; ; 1 ) to depend on the22

entire function 1 . The condition that the instruments q K (xi ) have the same dimension as theapproximating functions pK (w) allows for more than K instrumental variables. As is well known,
any IV estimator of can be viewed as having only K instruments q K (x), each one of whichis equal to a linear combination of all the instrumental variables. Here the existence of suchthat 20 (xi ) = 0 q K (xi ) is restrictive. It is not su¢ cient that 20 (xi ) be any linear combinationof all the instrumental variables. We must have 20 (xi ) equal to a linear combination of the
instruments q K (xi ) used in estimating . This result also extends to the case where an in…nite
number of instrumental variables are used in the limit. In that case q K (xi ) can also be interpreted
as an in…nite dimensional linear combination of instrumental variables.
To illustrate, consider again the weighted average derivative example discussed above wherem(z; ; 1 ) = v(w)@ 1 (w)=@w; (z; 1 ) = y
1 (w), and there is 20 (x) such thatE[ 20 (x)jw] =f0 (w) 1 @[f0 (w)v(w)]=@w:(6.2)

Suppose that the …rst step is a linear instrumental variables (IV) estimator with right-hand side
variables pK (wi ) and instruments q K (xi ) and let
be the limit of the IV coe¢ cients. From
Proposition 12 it follows that if there is a such that 0 q K (x) = 20 (x) thenE[v(wi ) @pK (wi )0=@w]

0 = E[m(zi ;

0; p

K0

)] = 0:Thus, the weighted average derivative of the linear IV estimator will be consistent when 20 (x)is a linear combination of q K (x):The case where v(w) is 1; wi is Gaussian, and E[xi jwi ] is linear in wi is interesting. Partialout constants and means so that (wi0 ; x0i )0 has mean zero. Let pK (wi ) = wi and let qi = q K (xi ) be
any linear combination xi such that = E[qi wi0 ] is nonsingular. Normalize wi and qi so that eachhave an identity variance matrix. Then f0 (w) 1 @f0 (w)=@w = w: Note that E[qi jwi ] = wi ; so
1 K
q (x):Thus the conditions of Proposition 12that equation (6.2) is satis…ed with 20 (x) =are satis…ed, giving the following result
Corollary 13: If yi = 10 (wi ) + "i , E[xi "i ] = 0; E[yi2 ] < 1; wi is Gaussian, and E[xi jwi ]is linear in wi then for instruments equal to any linear combination qi of xi with cov(qi ; wi )nonsingular;E[@ 10 (wi )=@w] = cov(qi ; wi ) 1 cov(qi ; yi ):We can give a simple, direct proof that only uses qi and "i uncorrelated, as is assumed in
Corollary 11, rather than the conditional moment restriction we have been focusing on. Withmeans partialed out we have E[wi ] = E[qi ] = 0 so thatcov(qi ; wi ) 1 cov(qi ; yi ) = (E[qi wi0 ]) 1 E[qi yi ] = (E[qi wi0 ]) 1 E[qi 10 (wi )]= (E[E[qi jwi ]wi0 ]) 1 E[E[qi jwi ] 10 (wi )] = (E[ wi wi0 ]) 1 E[ wi 10 (wi )]= (E[wi wi0 ]) 1 E[wi 10 (wi )] = E[@ 10 (wi )=@w];
23

where the fourth equality follows by E[qi jwi ] = wi and the last equality holds by Stoker (1986)
and wi Gaussian. This result generalizes that of Stoker (1986) to NPIV models where the righthand variables are Gaussian. Further generalizations to non Gaussian cases can be obtained byletting q K (x) and pK (w) be nonlinear in x and w.
Orthogonal series density estimators have a property analogous to Proposition 12. Suppose now that pK (x) is orthonormal with respect to Lebesgue measure on ( 1; 1) so that
R K
p (u)pK (u)0 du = I: An orthogonal series pdf estimator is ^1 (x) = pK (x)0 ^ ; where ^ =
R K
Pn K
p
(x
)=n
has
limit
=
p (u) 10 (u)du: Suppose that E[m(zi ; 0 ; 1 )] is a continuousii=1
linear functional of 1
10 so that by the Riesz representation theorem there is 20 (x) withR
E[m(zi ; 0 ; 1 )] =with 0 pK (x) = 20 (x) then by pK (u)20 (u)[ 1 (u)10 (u)]du: If there isorthonormal and equation (4.4) we have
Z
Z
K0
K
0
0 K
E[m(zi ; 0 ; p)] =p (u)[pK (u)0
20 (u)[p (u)10 (u)]du =10 (u)]duZZ00= 0 [ pK (u)pK (u)0 du]
pK (u) 10 (u)du = 0= 0:
Thus we have the following result:
Proposition 14: If i) m(z; 0 ; 1 ) is a¢ ne in 1 ; ii) E[m(zi ; 0 ; 1 )]0 is a functionalR
1=22[ 1 (u); and iii) 20 (xi ) =of 1 (x)10 (u)] du
10 (x) that is continuous in the norm
0 K
K0p (xi ) for some then E[m(zi ; 0 ; p)] = 0.
The orthogonal series estimators of linear functionals of a pdf discussed in Bickel and Ritov (2003) are examples. Those estimators are special cases of the estimator above whereRm(z; ; 1 ) =for prespeci…ed 20 (u): Proposition 14 implies that the or20 (u) 1 (u)duthogonal series estimator of 0 will be consistent if 20 (u) is a linear combination of the approximating functions. For example, if pK (x) is vector of polynomials of order K 1 then theorthogonal series estimator of moments of up to order K 1 is consistent for …xed K:

7

Conditional Moment Restrictions

Conditional moment restrictions are widely used in econometrics to identify parameters of interest. In this Section we expand upon the cases already considered to construct a wide variety of
locally robust moment conditions. In particular we extend above results to residuals that may
depend on parameters of interest with instrumental variables that can di¤er across residuals.
Here we depart from deriving locally robust moments from the adjustment term for …rst step
estimation. Instead we extend the form of previously derived locally robust moments to themore general setting of this Section.
24

To describe these results let j = 2; :::; J index conditional moment restrictions, j (z; ; 1 )denote a corresponding residual, and xj be corresponding conditioning variables. We will consider construction of locally robust moment conditions when the true parameters of interest 0
and a …rst step 10 satisfy conditional moment restrictions
E[ j (zi ;

0;

(7.1)

10 )jxji ] = 0; (j = 2; :::; J):

Here 1 is speci…ed to include all functions that a¤ect any of the residuals j (zi ; ; 1 ): We
continue to assume that the unconditional moment restriction in equation (2.1) holds, thoughm(z; ; 1 ) could be zero, with identi…cation of 0 coming from the conditional moment restrictions of equation (7.1). We will discuss this case below.
In this setting we consider locally robust moment conditions having the form
g(z; ; ) = m(z; ; 1 ) +JXj (xj ;

(7.2)

) j (z; ; 1 );j=2

where j (x; ); (j = 2; :::; J) are unknown functions satisfying properties discussed below. These
moment functions depend on J nonparametric components = ( 1 ; :::; J ). By virtue of theconditional moment restrictions these moment functions will be doubly robust in ( 2 ; :::; J );meaning that E[g(zi ; 0 ; 10 ; 2 ; :::; J ) = 0. They will be locally robust in 1 if for the limit
1 (F ) of ^1 and all regular parametric models F as discussed in Section 2,@
E[m(zi ;@If @E[m(zi ;

0;

J
X
@
E[ j (zi ;0 ; 1 (F ))] + E[
j0 (xji )@
j=2

1 (F ))]=@(@E[ 2 (zi ;0;

1 (F ))jxji ]] = 0:

(7.3)

j =0 is a linear mean-square continuous function of0;

1 (F ))jxji ]=@; :::; @E[ J (zi ;0;

1 (F ))jxji ]=@)j =0

and the mean-square closure of the set of such vectors over all parametric submodels is linear
then existence of j0 (xj ), j 2 satisfying equation (7.3) will follow by the Riesz representation
theorem. In addition, if m(z; 0 ; 1 ) and j (z; 0 ; 1 ), (j 2), are a¢ ne in 1 then we will havedouble robustness in 1 similarly to Proposition 12. Summarizing we have
Proposition 15: If equation (7.3) is satis…ed then g(z; ; ) from equation (7.2) is locallyrobust. Furthermore, if i) m(z; 0 ; 1 ) and j (z; 0 ; 1 ); (j2) are a¢ ne in 1 2 withlinear; ii) E[m(zi ; 0 ; 1 )] is a mean square continuous functional of E[ j (zi ; 0 ; 1 )jxij ]; (j 2)then there is j0 (x); (j 2); such that g(z; ; ) is doubly robust.
For local identi…cation of

we also require thatrank(@E[g(zi ; ; 0 )]=@ j = 0 ) = dim( ):
25

(7.4)

A model where 0 is identi…ed from semiparametric conditional moment restrictions with
common instrumental variables x is a special case where m(z; ; ) is zero and xj = x; (j 2):
In this case let (z; ; 1 ) = ( 2 (z; ; 1 ); :::; J (z; ; 1 ))0 : The conditional moment restrictionsof equation (7.1) can be summarized as
E[ (zi ;0;10 )jxi ] = 0:This model is considered by Chamberlain (1992) and Ai and Chen (2003, 2007, 2012). We allow
the residual vector (z; ; 1 ) to depend on the entire function 1 and not just its value at somefunction of the observed data zi . Also let '(x) = [ 2 (x); :::; J (x)] denote an r (J 1) matrix offunctions of x. A locally robust moment function g(z; ; ) = '(x) (z; ; 1 ) will be one which
satis…es De…nition 1 with g(z; ; ) replacing m(z; ; ); i.e. where@E[g(zi ; 0 ; (F ))]@E[ (zi ; 0 ; 1 (F ))jxi ]= E '(xi )= 0;@@for all regular parametric models. We also require that equation (7.4) is satis…ed.
To characterize local robustness here it is helpful to assume that the set of pathwise derivatives of E[ (zi ; 0 ; )jxi ] varies over a linear set as the regular parametric model F varies. To
be precise we will assume that 1 2 for linear and for 2 we letm (x; ) =@E[ (zi ;

0;

10 +

)jxi ]@=0denote the (J 1) 1 random vector that is Gateaux derivative of the conditional expectation
E[ (zi ; 0 ; 1 )jxi ] with respect to the …rst step 1 in the direction : We assume that m (x; )is linear inand that the mean square closure M of the set fm (x; ) :2 g equals
the mean-square closure of the set f@E[ (zi ; 0 ; 1 (F ))jxi ]=@ g as F varies over all regularparametric models. The local robustness condition can then be interpreted as orthogonality ofeach row '(xi )0 ej of '(x) with M in the Hilbert space of functions of x with inner productha; bi = E[a(xi )0 b(xi )]; where ek is the k th unit vector. Thus the condition for locally robustg(z; ; ) = '(x) (z; ; 1 ) is thatE['(xi )m (xi ; )] = 0 for all

2 :We refer to such '(xi ) as being orthogonal. They can be interpreted as instrumental variables
where the e¤ect of estimation of 1 has been partialed out.
There are many ways to construct orthogonal instruments. For instance, given a r (J 1)
matrix of instrumental variables A(x) one could construct corresponding orthogonal ones '(xi )as the matrix where each row is the residual of the least squares projection of the corresponding
row of A(x) on M . We focus on another way of constructing orthogonal instruments that leads
26

to an e¢ cient estimator of 0 . Let (x) denote some positive de…nite matrix with smallest eigenvalue bounded away from zero, so that (xi ) 1 is bounded. Let ha; bi = E[a(xi )0 (xi ) 1 b(xi )]denote an inner product and note that M is closed in this inner product by (xi ) 1 bounded.
Let A~k (xi ; A; ) denote the residual from the least squares projection of the k th row A (x)0 ek
of A(x) on M with the inner product ha; bi : Also let '(xi ; A; ) be the matrix with k th row
A~k (xi ; A; )0 (xi ) 1 ; (k = 1; :::; r): Then for all 2 ;A (xi )0 ek~ i ; A; ) (xi ) 1 m (xi ; )] = 0;
A~k (xi ; A; ) 2 M ; E['(xi ; A; )m (xi ; )] = E[A(x~ A; ) can be interpreted as theso that '(xi ; A; ) are orthogonal instruments. Also, A(x;solution tominfM (x):M (x)0 ek 2M ;k=1;:::;rg

M (xi )g (xi ) 1 fA(xi )E[fA(xi )M (xi )g0 ]where the minimization is in the positive semide…nite sense.
The orthogonal instruments that minimize the asymptotic variance of GMM in the class of
GMM estimators with orthogonal instruments are given by' (xi ) = '(xi ; A ;@E[ (zi ; ; 10 )jxi ]); A (xi ) =@0;(xi ) = V ar( (zi ;0;10 )jxi ):= 0

To see that ' (xi ) minimizes the asymptotic variance note that for any orthogonal instrumental
variable matrix '(x)~ i; A ;G = E['(xi )A (xi )0 ] = E['(xi )A(x)0 ] = E['(xi ) (zi ;

0;

10 )

(zi ;

0;

0
0
10 ) ' (xi ) ];where the …rst equality de…nes G and the second equality holds by '(xi ) orthogonal. Since thep^ !
instruments are orthogonal the asymptotic variance matrix of GMM estimator with W
W
is the same as if ^1 = 10 : De…ne mi = G0 W '(xi ) (zi ; 0 ; 10 ) and mi = ' (xi ) (zi ; 0 ; 10 ): The
asymptotic variance of the GMM estimator for orthogonal instruments '(x) is(G0 W G) 1 G0 W E['(xi ) (zi ;

0;

10 )

(zi ;

0;

0
0
0
1
10 ) '(xi ) ]W G(G W G)

= (E[mi mi 0 ]) 1 E[mi m0i ](E[mi mi ]) 10 :The fact that this matrix is minimized in the positive semide…nite sense for '(x) = ' (x) follows
from Theorem 5.3 of Newey and McFadden (1994) and can also be shown using the argumentin Chamberlain (1987).
Proposition 16: The instruments ' (xi ) give an e¢ cient estimator in the class of IV
estimators with orthogonal instruments.
The asymptotic variance of the GMM estimator with optimal orthogonal instruments is~ i; A ;(E[mi mi 0 ]) 1 = E[A(x27)~ i; A ;(xi ) 1 A(x)0 ]) 1 :This matrix coincides with the semiparametric variance bound of Ai and Chen (2003). Estimation of the optimal orthogonal instruments is beyond the scope of this paper. The series
estimator of Ai and Chen (2003) could be used for this.

8

Structural Economic Examples

Estimating structural models can be di¢ cult when that requires computing equilibrium solutions. Motivated by this di¢ culty there is increasing interest in two step semiparametric methods
based on a …rst step nonparametric estimation of conditional choice probabilities (CCP). Thistwo step approach was pioneered by Hotz and Miller (1993). In this Section we show how locallyrobust moment conditions can be formed for two kinds of structural models, the dynamic discrete choice model of Rust (1987) and the static model of strategic interactions of Bajari, Hong,
Krainer, and Nekipelov (2010, BHKN). It should be straightforward to extend the constructionof locally robust moments to other more complicated structural economic models.

8.1

Static Models of Strategic Interactions

We begin with a static model of interactions where results are relatively simple. To save space we
describe the estimator of BHKN while only describing a small part of the motivational economic
structure. Let x denote a vector of state variables for a …xed set of individuals and let y denotea vector of binary variables, each one representing a choice of an alternative by an individual.
Let the observations zi = (yi ; xi ) represent repeated plays of a static game of interaction and
10 (x) = E[yi jxi = x] the vector of conditional choice probabilities given a value x of the state.
In the semiparametric estimation problem described in Section 4.2 of BHKN there is a known
function r(x; ; 1 (x)) of the state variable x, a vector of parameters , and a possible value(x) of the conditional choice probabilities such that the true parameter 0 satis…es
E[yi jxi = x] = r (x;0;

10 (x)) ,This model can be used to form moment functionsm(z; ; 1 ) = A(x)[yr (x; ; 1 (x))];where A(x) is a matrix of instrumental variables; see equation (17) of BHKN.
To describe locally robust moment functions in this example, let r (x; ; 1 ) = @r(x; ; 1 )=@ 1
where 1 here denotes a real vector representing a possible value of 10 (x). Then, it follows from
Proposition 4 of Newey (1994), as discussed in BHKN, that the adjustment term for …rst step
estimation of 10 (x) = E[yi jxi = x] is(z; ; 1 ) =A(x)r (x; ; 1 )[y28

1 (x)]:

This expression di¤ers from BHKN in the appearance of 1 (x) at the end of the expression
rather than r (x; ; 1 (x)), which is essential for local robustness. The locally robust moment
conditions are theng(z; ; 1 ) = A(x)fyr(x; ; 1 (x))

r (x; ; 1 (x))[y1 (x)]g:

For a …rst step estimator ^ (x) of the conditional choice probabilities, the locally robust sample
moments will be
1X
A(xi )fyig^( ) =n i=1
nr(xi ; ; ^1 (xi ))r (xi ; ; ^1 (xi ))[yi^1 (xi )]g

Here the locally robust moments are constructed by subtracting from the structural residuals
r a linear combination of the nonparametric residuals from the …rst step. Using these moment
functions should result in an estimator of the structural parameters with less bias and the otherimproved properties of locally robust estimators mentioned above.
The optimal instruments here are the same as discussed in BHKN. Let I denote the identity
matrix, set H(x) = I @r(xi ; 0 ; 10 (xi ))=@ 1 ; and let (xi ) = H(xi )V ar(yi jxi )H(xi )T denotethe conditional variance of H(xi )(yi
10 (xi )). The optimal instruments are given by
A (xi ) =@r(xi ;0;@

10 (xi ))0(x)where A denotes a generalized inverse of a positive semi-de…nite matrix A.
This model can also be viewed as a special case of the conditional moment restrictions
framework with residual vector (z; ; 1 ) = (yr(x; ; 1 (x)))T . An orthogonal
1 (x); yinstrument that gives the above locally robust moment function is A(x)[ r (xi ; ; 1 (xi )); I]:
Here the locally robust moment function only depend on one …rst step function 1 (x). Thisfeature is shared by all nonparametric generated regressors setups where r(x; ; 1 ) depends
only on regressors that are included in the nonparametric …rst step 1 (x). The static model ofstrategic interactions leads to this structure. The situation is not so simple in other structural
economic models, as we see next.

8.2

Dynamic Discrete Choice

Dynamic discrete choice estimation is important for modeling economic decisions, Rust (1987).
In this setting we …nd it helpful to describe the underlying economic model in order to explain
the form of the moment conditions. Semiparametric moment conditions can be constructed forthese models using nonparametric estimators of the conditional choice probabilities given thestate variables, Hotz and Miller (1993). In this Section we give locally robust moment conditions
based on this approach. We do this for the in…nite horizon, stationary, dynamic discrete choice29model of Rust (1987). It is straightforward to derive locally robust moment conditions for otherstructural econometric models. We also focus here on the case of data on many homogenous
individuals, but discuss how the approach extends to time series on one individual.
Suppose that the per-period utility function for an agent making choice j in period t is givenby
Ujt = vj (xt ; 0 ) + jt ; (j = 1; :::; J; t = 1; 2; :::)
where we suppress the individual subscript i for notational convenience. The vector xt is theobserved state variables of the problem (e.g. work experience, number of children, wealth) andthe vector is unknown parameters. The disturbances t = f 1t ; :::; Jt g are not observed by the
econometrician. As in the majority of the literature we assume that t is i.i.d. over time withknown CDF that has support RJ and is independent of the state process xt and that xt is Markov
of order 1. Let denote a time discount parameter, v(x) the expected value function, yjt 2 f0; 1g
the indicator that choice j is made and vj (xt ) = vj (xt ; 0 ) + E[v(xt+1 )jxt ; ytj = 1] the expectedvalue function for choice j. Also let v~j denote a possible realization of v~j (xt ) = vj (xt ) v1 (xt );so that v~1
0. Let v~ = (~v2 ; :::; v~J ) and Pj (~v ) = Pr(~
vj + jt
v~k + kt ; v~1 = 0; k = 1; :::; J);(j = 1; :::; J) denote the choice probabilities associated with the distribution of t : Here wenormalize to focus on the di¤erence with v1 (x) throughout. Let v~(x) = (~v2 (x); :::; v~J (x))0 . LetTbe a vector of …rst step functions with true values aj0 (xt ) =a (x) = ( a1 (x); :::; aJ (x))
Pr(ytj = 1jxt ). From Rust (1987) we know that forv (xt )); (j = 1; :::; J):
aj0 (xt ) = Pj (~v(xt ) = E[maxfvj (xt ) + jt gjxt ] = v1 (xt ) + E[maxf~
vj (xt ) + jt gjxt ]:jj

From Hotz and Miller (1993) we know that P (~v ) = (P1 (~v ); P2 (~v ); :::; PJ (~v ))0 is a one-to-one
function of v~; so that inverting this relationship it follows that E[maxj f~
vj (xt ) + jt gjxt ] is afunction of a0 (xt ); say E[maxj f~
vj (xt ) + jt gjxt ] = H( a0 (xt )); for some function H( ) (e.g. forbinary logit H( a0 (xt )) = :5772 ln( a10 (xt ))). Then the expected value function is given byv(xt ) = v1 (xt ) + H( a0 (xt ))To use these relationships to construct semiparametric moment conditions we normalize
v1 (x; ) = 0 and make an additional assumption for v1 (x). The additional assumption is thatE[v(xt+1 )jxt ; 1] does not depend on xt : With this normalization and assumption we have aconstant choice speci…c value function for j = 1; that is v1 (x) = v1 ; withv1 (xt ) = 0 + E[v(xt+1 )jxt ; yt1 = 1] = E[v(xt+1 )jyt1 = 1] = v1 :A su¢ cient condition for constant v1 (xt ) is that j = 1 is a "renewal" choice where the distributionof the future state does not depend on the current state. In the Rust (1987) example this stateis the one where the bus engine is replaced.
30With this normalization and assumption we now have
vj (x) = vj (x;0) +E[v1 + H( a0 (xt+1 ))jxt = x; ytj = 1]

= vj (x;0) +v1 + E[H( a0 (xt+1 ))jxt = x; ytj = 1];

v~j (x) = vj (x;0) +E[H( a0 (xt+1 ))jxt = x; ytj = 1]

E[H( a0 (xt+1 ))jyt1 = 1]; (j = 2; :::; J):The choice speci…c expected value di¤erences v~j (x) have a parametric part vj (x; 0 ) and a
nonparametric part that depends on J 1 additional nonparametric regressions b (x; a ) =( b1 (x; a ); :::; b;J 1 (x; a ))T and an unknown parameter c where
bj0 (x;Leta ) = E[H( a (xt+1 ))jxt = x; yt;j+1 = 1]; j = 1; :::; JT
1 (x) = ( a (x) ;

b (x;

T
a) ;

1; c0 ( a ) = E[H( a (xt+1 ))jyt1 = 1]:Tc ( a )) be a vector of …rst step objects andv~j (x; ; 1 ) = vj (x; ) + [ b;j (x; a )~(x;c ( a )]; v; 1 ) = (~v2 (x; ; 1 ); :::; v~J (x; ; 1 ))Tdenote the semiparametric choice speci…c expected value di¤erences. Semiparametric moment
conditions can then be formed by plugging in a nonparametric ^1 estimator of the …rst step into
the expected di¤erences and plugging those into the choice probability. Let yt = (yt1 ; :::; ytJ )Tdenote the vector of choice indicators for period t and z = (y1T ; xT1 ; :::; yTT ; xTT )T be the vector
consisting of the observations on choice and state variables xt for each time period t: Also let
At (xt ) be an r 1 vector of functions of xt where r is the dimension of : Then for eacht = 1; :::; T 1 we can form semiparametric moment conditions as
mt (z; ; 1 ) = At (xt )[ytP (~v (xt ; ; 1 ))]:

To derive locally robust moment functions we can derive the adjustment term for estimationof 1 . The …rst step function 1 is more complicated than previously considered. It depends on
two unknown conditional expectations, E[yt jxt ] and E[ jxt ; j], (j = 2; :::; J): From Newey (1994,
p. 1357) we know that the adjustment term will be the sum of two terms, each adjusting for
one of the two conditional expectations while treating the other as if it was equal to the truth.
In the Appendix we give a general form for each of these adjustment terms. Here we apply thatgeneral form to derive the corresponding locally robust moment functions for dynamic discrete
choice.
We begin by deriving the adjustment terms for b and c because they are simpler thanfor a . The adjustment term for b and c are obtained by applying Proposition A1 in the
Appendix. Let ~b (F ); ~c (F ) have the same form as b and c except that E[ jxt ; yt ] is replaced
by EF [ jxt ; j]: For Ht+1 = H ( a0 (xt+1 )) and 1 = E[yt1 ] let
bj (z;; 1 ) = [yt;j+1 = a;j+1 (xt )]fH( a (xt+1 ))b (z;; 1 ) = ( b1 (z; ; 1 ); :::;

c (z;

; 1;

1) =

1
1

b;J 1 (z;yt1 H( a (xt+1 ))31; 1 ))T ;c ( a ):

b;j+1 (xt ;a )g;Then for e a (J1)

@E[mt (zi ;0;

1 vector of 10 s we havea0 ; ~b (F ); ~c (F ))]@= E[ tb (zi ;0;1;2 )S(zi )];@P(~v (xt ; ; 1 )) b (z; ; 1 )
@~
v
@P
(~
v (xt ; ; 1 ))]e c (z; ; 1 ;+ E[A(xt )
@~
v

t
b (z;

; 1; ) =A(xt )1 )g

The adjustment term for estimation of a (x) is obtained by applying Proposition A2. This
term is somewhat complicated because a (x) is evaluated at x = xt+1 rather than the conditioning argument xt of its true value We assume that xt is stationary over time so that xt+1 and xt
have the same pdf, eliminating the ratio of pdf’s in the conclusion of Proposition A2. Let a (F )have the same form as 10 except that 10 (x) is replaced by EFt [yt jxt = x]: Also let
2j0 (x;ytj@P (~v (xt ; ; xt ))A(xt )
jxt+1 = x ; (j = 1; :::; J@~v
aj (xt ); 1) = E

30 (x;

1) =

1
1

1);

E[yt1 jxt+1 = x]jx=xt :Then we have@E[mt (zi ;0;a (F );b0 ;@ta (z; ; 1 ; 2 ; 3 ;c0 )]= E[ ta (zi ;

1) =

0;

1;

f 2 (x; ; 1 )@H( a (xt ))[yt@ a2 )S(zi )];E[A(xt )Pv~(~v (xt ; ; 1 ))] 3 (xt ;1 )gea (xt )]:

We can now form locally robust moment conditions as
gt (z; ; 1 ; 2 ; 3 ;1 ) = mt (z;; 1) +ta (z;; 1; 2; 3;

1) +

t
b (z;

; 1;

1 ):

With data zi that is i.i.d. over individuals these moment functions can be used for any t toestimate the structural parameters : Also, for data for a single individual we could use a timeP
average Tt=11 gt (z; ; 1 ; 2 ; 3 ; 1 )=(T 1) to estimate ; although the asymptotic theory wegive does not apply to this estimator.
Bajari, Chernozhukov, Hong, and Nekipelov (2009) derived the adjustment term for the more
complicated dynamic discrete game of imperfect information. Locally robust moment conditions
for such games could be formed using their results. We leave that formulation to future work.

9

Asymptotic Theory for Locally Robust Moments

In this Section we give asymptotic theory for locally robust estimators. In keeping with thegeneral applicability of locally robust moments to a variety of …rst steps we consider estimation32

and conditions that have the weakest conditions we can …nd for the …rst step. In particular, theconstruction here only requires that the …rst step converge at rate slightly faster than n 1=4 in
norms speci…ed below, much weaker conditions than in some of the literature. This formulation
allows the results to be applied in settings where it is challenging to say much about the …rst
step other than its convergence rate, such as when the …rst step comes from model selection.
The locally robust form of the moment conditions is essential for this formulation.
To specify such weak conditions on the …rst step we use cross …tting, as discussed above.
Chernozhukov et. al. (2016) gives results with cross …tting that allow for moment functions thatare not smooth in parameters. Here we focus on the smooth in parameters case. cross …tting has
been previously used in the literature on semiparametric estimation. See Bickel, Klaasen, Ritov,and Wellner (1993) for discussion. This approach is di¤erent than that previously adopted inwork on semiparametric estimation, as in Andrews (1994), Newey (1994), Chen, Linton, and van
Keilegom (2003), Ichimura and Lee (2010), where cross …tting was not used and the moment
conditions need not be locally robust. The approach adopted here leads to general and simple
conditions.
The estimator is formed by grouping observations into L distinct groups. Let I` ; (` = 1; :::; L)partition the set of observation indices f1; :::; ng. Let ^ ` be the …rst step constructed from allobservations not in I` . Consider sample moment conditions of the form
1 XX
g(zi ; ; ^ ` ):n `=1 i2IL

g~( ) =`

We consider GMM estimators based on these moment functions. This is a special case of thecross …tting described earlier. Also, leave one out moment conditions are a further special casewhere each I` consists of a single observation. We focus here on the case where the number ofgroups L is …xed to keep the conditions as simple as possible.
An important intermediate result is that the adjustment term for the …rst step is zero byvirtue of g(z; ; ) being locally robust, that is
p

1 X
n~
g( 0) = pg(zi ;
n i=1
n

0;

(9.1)

0 ) + op (1):With cross …tting this result holds under relatively weak and simple conditions:
Assumption 1: For each ` = 1; :::; L, i)
R> 1; C > 0 we haveg(z; 0 ; ^ ` )F0 (dz)Rpkg(z; 0 ; ^ ` ) g(z; 0 ; 0 )k2 F0 (dz) ! 0; ii) forpp
C k^ `n k^ `! 0.
0 k ; and iii)
0k

Lemma 17: If Assumption 1 is satis…ed then equation (9.1) is satis…ed.
This Lemma is proved in the Appendix. There are two important components to this result.
33

One component is a stochastic equicontinuity resultp

1 X
pg(zi ;
n i=1
n

n~
g( 0)

1 X
p
n`
n `=1
L

0;

0)

Z

g(z;

p

! 0;

0 ; ^ ` )F0 (dz)where n` is the number of observations with i 2 I` : Assumption 1 i) is su¢ cient for this result.
Assumption 1 i) is a much weaker stochastic equicontinuity condition than appears in much of
the literature, e.g. Andrews (1994). Those other conditions generally involve boundedness ofsome derivatives of ^ : In contrast Assumption 1 i) only requires that ^ ` have a mean squareconvergence property. The cross …tting is what makes this condition su¢ cient. Cattaneo andJansson (2014) have also previously weakened the stochastic equicontinuity condition and established the validity of the bootstrap for kernel estimators under substantially weaker bandwidthconditions than usually imposed.
The second component of the result is thatZppng(^ ` ) ! 0; g( ) = g(z; 0 ; )F0 (dz):This component follows from Assumptions 1 ii) and iii). By comparing Assumption 1 ii) with
De…nition 2 we see that this condition implies local robustness in the sense that the Frechet
derivative of g( ) is zero at 0 . Assumption 1 ii) will generally hold with = 2 if g( ) is twice
continuously Frechet di¤erentiable. In that case Assumption 1 iii) become the n1=4 rate condition
familiar from Newey and McFadden (1994) and other works. The more general 1 < < 2 caseallows for the …rst Frechet derivative of g( ) to satisfy a Lipschitz condition. In this case
Assumption 1 iii) will require a convergence rate of ^ that is faster than n1=4 :We note that previous results suggest that n1=4 convergence of ^ may be stronger than isneeded. As shown in Robins et. al. (2008) and Cattaneo and Jansson (2014) the variance termspin ng(^ ) are of the same order as the variance term of a nonparametric estimator, ratherpthan being the order of n times those variance terms. The arguments for these weaker results
are quite complicated so we do not attempt to give an account here. Instead we focus on therelatively simple conditions of Assumption 1.
Another component of an asymptotic normality result is convergence of the Jacobian term
~ )=@~g ( )=@ : The conditions we impose to account for the Jacobian term are standard. Let G(@~
g ( )=@ denote the derivative of the moment function.
Assumption 2: There is a neighborhood N of 0 such that i) g(zi ; ; ) is di¤erentiable inon N with probability approaching 1 ii) there is 0 > 0 and d(zi ) with E[d(zi )] < 1 such thatfor 2 N and k
0 k small enough@g(zi ; ; )@@g(zi ; 0 ; 0 )@d(zi )(k
34

0k

0

+k

0k

0

);

iii) E[k@g(zi ;

0;

0 )=@

k] < 1; iv) k^ `p0k

! 0; (` = 1; :::; L):

De…ne
G = E[@g(zi ; ; 0 )=@ j = 0 ]pLemma 18: If Assumption 2 is satis…ed then for anypwith probability approaching one and @~g ( )=@
! G:

!

~(
0; g

) is di¤erentiable at

With Lemmas 17 and 18 in place the asymptotic normality of semiparametric GMM follows
in a standard way.
pp
^ !
Theorem 19: If Assumptions 1 and 2 are satis…ed, ^ ! 0 ; W
W , G0 W G is nonsingular, and E[kg(zi ; 0 ; 0 )k2 ] < 1 then for = E[g(zi ; 0 ; 0 )g(zi ; 0 ; 0 )0 ];
p
d
0
1 0
0
1
n( ^
0 ) ! N (0; V ); V = (G W G) G W W G(G W G) :It is also useful to have a consistent estimator of the asymptotic variance of ^. As usual suchan estimator can be constructed as
^ 0W
^ G)
^ 1G
^ 0W
^ ^W
^ G(
^ G
^ 0W
^ G)
^ 1;
V^ = (GL
1 XX@~
g ( ^) ^^; =g(zi ; ^; ^ ` )g(zi ; ^; ^ ` )0 :G=@n `=1 i2I`

Note that this variance estimator ignores the estimation of , which works here because themoment conditions are locally robust. Its consistency will follow under the conditions of Theorem
19 and one additional condition that accounts for the presence of ^ in ^ :~ with E[d(z~ i )2 ] <Theorem 20: If the conditions of Theorem 19 are satis…ed and there is d(z)1 such that for k
0 k and k
0 k small enoughthen V^pkg(zi ; ; )g(zi ;0;0 )k~ i )(k
d(z

0k

0

+k

0k

0

);

! V:

In this Section we have used cross …tting to obtain relatively simple conditions for asymptotic
normality of locally robust semiparametric estimators. It is also known that in some settings
some kinds of cross …tting improves the properties of semiparametric estimators. For linear
kernel averages it is known that the leave one out method eliminates a bias term and leads toa reduction in asymptotic mean square error; e.g. see NHR and the references therein. Also
Robins et. al. (2008) use cross …tting in higher order bias corrections. These results indicate
the some kind of cross …tting can lead to estimators with improved properties. For reducing
higher order bias and variance it may be desirable to let the number of groups grow with thesample size. That case is beyond the scope of this paper.
35

10

APPENDIXWe …rst give an alternative argument for Proposition 2 that is a special case of the proof of
Theorem 2.2 of Robins et. al. (2008). As discussed above, (z; 0 ; 0 ) is the in‡uence functionof the functional (F ) = E[m(zi ; 0 ; (F ))]. Because it is an in‡uence function it has meanRzero at all true distributions, i.e.
(z; 0 ; (F0 ))F0 (dz) 0 identically in F0 : Since a regularparametric model fF g is just a subset of all true models, we haveZ(z; 0 ; (F ))F (dz) 0;
identically in . Di¤erentiating this identity at@E[ (zi ; 0 ; (F ))]@= 0 and applying the chain rule gives=

E[ (zi ;(10.1)0;0 )S(zi )]:+@E[ (zi ; 0 ; (F ))]@=0

Summing equations (3.3) and (10.1) we obtain@E[g(zi ; 0 ; (F ))]@==0@E[m(zi ; 0 ; (F ))]@= E[ (zi ;0;=00 )S(zi )]E[ (zi ;0;=00 )S(zi )] = 0:

Thus we see that the adjusted moment functions g(z; ; ) are locally robust.
Next we derive the form of the adjustment term when a …rst step is E[ jx; y = 1] for somebinary variable y with where yi 2 f0; 1g. Consider a …rst step function of the form 1 (x) =E[wi jxi = x; yi = 1]. Let (xi ) = E[yi jxi ]: Note that E[wi jxi ; yi = 1] = E[yi wi jxi ]=E[yi jxi ] sothat@EF [wi jxi ; yi = 1]
= E[ (xi ) 1 fyi wi@= E[ (xi ) 1 yi fwi

E[yi wi jxi ]gE[wi jxi ; yi = 1](yi(xi ))gS(zi )jxi ]E[wi jxi ; yi = 1]gS(zi )jxi ]:Suppose that there is (xi ) such that
@EF [m(zi ; 0 ; 1 (F ))]@EF [wi jxi ; yi = 1]
= E[ (xi )]@@= E[ (xi ) (xi ) 1 yi fwi E[wi jxi ; yi = 1]gS(zi )]:Then taking the limit gives the following result:
Proposition A1: If there is (x) such that @E[m(zi ;1]=@ ] then the adjustment term is(z; ; ) = (x) (x) 1 yfw
360;

1 (F ))]=@= E[ (xi )@EF [wi jxi ; yi =E[wi jxi = x; yi = 1]g:

Next we derive the adjustment term when a nonparametric regression is evaluated at a
variable di¤erent than the one being conditioned on in the regression. Note that for (v) with@E[ (vi )EF [yi jxi = x]jx=wi ]E[E[ (vi )jwi ]EF [yi jxi = x]jx=wi ]
=
@
@
Z
=@

=@

Z

E[ (vi )jwi = w]EF [yi jxi = w]fw (w)dw=@[fw (x)=fx (x)]E[ (vi )jwi = x]EF [yi jxi = x]fx (dx)=@= E[ffx (xi ) 1 fw (xi )E[ (vi )jwi = w]jw=xi (yiE[yi jxi ])gS(zi )]Taking limits gives
Proposition A2: If there is (v) such that @E[m(zi ;x]jx=wi ]=@ then the adjustment term is0;

1 (F ))]=@(z; ; ) = fx (x) 1 fw (x)E[ (vi )jwi = x](yi= @E[ (vi )EF [yi jxi =E[yi jxi = x]):Next we give the proofs for the asymptotic normality results.
Proof of Lemma 17: LetZg( ) = g(z; 0 ; )F0 (dz); ^ i` = g(zi ;0; ^ `)

g(^ ` )g(zi ;

0;0 ); (i 2 I` );

` =

1X^
i` :
n i2I
`

Also, let Z ` denote a vector of all observations zi for i 2= I` : Note that by construction^
E[ i` jZ ` ] = 0, so for any i; j 2 I` ; i 6= j; it follows by zi and zj independent conditionalon Z ` that E[ ^ 0i` ^ j` jZ ` ] = E[ ^ 0i` jZ ` ]E[ ^ j` jZ ` ] = 0 Furthermore
Z
2
E[ ^ i` jZ ` ]kg(z: 0 ; ^ ` ) g(z; 0 ; 0 )k2 F0 (dz):Therefore, for n` equal to the number of observations in group `; Assumption 1 i) implies
Z
2
n`
1 X
0
^
E[ ` ` jZ ` ] = 2
E[ i` jZ ` ]kg(z: 0 ; ^ ` ) g(z; 0 ; 0 )k2 F0 (dz) = op (n` =n2 ):
2
n i2I
n
`

p
Standard arguments then imply that for each ` we have ` = op ( n` =n). It then follows that
"
#
n
L
X
p
p X
p
1
p
n g~( 0 )g(zi ; 0 ; 0 ) g(^ ) = n` = op ( n` =n) ! 0:
n i=1
`=1
It also follows by Assumption 1 ii) and iii) thatpn kg(^ )k

p

nC k^
37

0k

p

! 0:

The conclusion then follows by the triangle inequality. Q.E.D.
~ ) = @~
^ = @g(zi ; 0 ; 0 )=@ :Proof of Lemma 18: Let G(
g ( )=@ when the derivative exists and Gp^ ! G: Also, by Assumption 2 i), ii),
By the law of large numbers and Assumption 2 iii), G~ ) is well de…ned with probability approaching one Pn d(zi )=n = Op (1) by the Markoviii) G(
i=1
inequality, and by the triangle inequality,
1 XX
fd(zi )(n `=1 i2I`!
n
1X
d(zi ) (
n i=1
L

~ )
G(

^
G

0

0

0

0

+ k^`
+

L
X
`=1

0k

k^`

0

)g

0k

0

p

) = Op (1)op (1) ! 0:

The conclusion then follows by the triangle inequality. Q.E.D.

Acknowledgements
Whitney Newey gratefully acknowledges support by the NSF. Helpful comments were provided
by M. Cattaneo, J. Hahn, M. Jansson, Z. Liao, J. Robins, R. Moon, A. de Paula, J.M. Robin,
participants in seminars at Cornell, Harvard-MIT, UCL, and USC.

REFERENCES
Ackerberg, D., X. Chen, and J. Hahn (2012): "A Practical Asymptotic Variance Estimator
for Two-step Semiparametric Estimators," The Review of Economics and Statistics 94: 481–498.
Ackerberg, D., X. Chen, J. Hahn, and Z. Liao (2014): "Asymptotic E¢ ciency of Semiparametric Two-Step GMM," The Review of Economic Studies 81: 919–943.
Ai, C. and X. Chen (2003): “E¢ cient Estimation of Models with Conditional Moment Restrictions Containing Unknown Functions,”Econometrica 71, 1795-1843.
Ai, C. and X. Chen (2007): "Estimation of Possibly Misspeci…ed Semiparametric Conditional
Moment Restriction Models with Di¤erent Conditioning Variables," Journal of Econometrics
141, 5–43.
Ai, C. and X. Chen (2012): "The Semiparametric E¢ ciency Bound for Models of Sequential
Moment Restrictions Containing Unknown Functions," Journal of Econometrics 170, 442–457.
Andrews, D.W.K. (1994): “Asymptotics for Semiparametric Models via Stochastic Equicontinuity,”Econometrica 62, 43-72.
Bajari, P., V. Chernozhukov, H. Hong, and D. Nekipelov (2009): "Nonparametric andSemiparametric Analysis of a Dynamic Discrete Game," working paper, Stanford.
38

Bajari, P., H. Hong, J. Krainer, and D. Nekipelov (2010): "Estimating Static Models ofStrategic Interactions," Journal of Business and Economic Statistics 28, 469-482.
Bang, and J.M. Robins (2005): "Doubly Robust Estimation in Missing Data and Causal Inference Models," Biometrics 61, 962–972.
Belloni, A., D. Chen, V. Chernozhukov, and C. Hansen (2012): “Sparse Models andMethods for Optimal Instruments with an Application to Eminent Domain,”Econometrica 80,
2369–2429.
Belloni, A., V. Chernozhukov, and Y. Wei (2013): “Honest Con…dence Regions for Logistic
Regression with a Large Number of Controls,”arXiv preprint arXiv:1304.3969.
Belloni, A., V. Chernozhukov, I. Fernandez-Val, and C. Hansen (2016): "Program
Evaluation and Causal Inference with High-Dimensional Data," Econometrica, forthcoming..
Bera, A.K., G. Montes-Rojas, and W. Sosa-Escudero (2010): "General Speci…cation
Testing with Locally Misspeci…ed Models," Econometric Theory 26, 1838–1845.
Bickel, P.J., C.A.J. Klaassen, Y. Ritov, and J.A. Wellner (1993): E¢ cient and Adaptive
Estimation for Semiparametric Models, Springer-Verlag, New York.
Bickel, P.J. and Y. Ritov (2003): "Nonparametric Estimators Which Can Be "Plugged-in,"Annals of Statistics 31, 1033-1053.
Cattaneo, M.D., and M. Jansson (2014): "Bootstrapping Kernel-Based Semiparametric Estimators," working paper, Berkeley.
Chamberlain, G. (1987): “Asymptotic E¢ ciency in Estimation with Conditional Moment Restrictions,”Journal of Econometrics 34, 1987, 305–334.
Chamberlain, G. (1992): “E¢ ciency Bounds for Semiparametric Regression,”Econometrica 60,
567–596.
Chen, X. and X. Shen (1997): “Sieve Extremum Estimates for Weakly Dependent Data,”
Econometrica 66, 289-314.
Chen, X., O.B. Linton, and I. van Keilegom (2003): “Estimation of Semiparametric Models
when the Criterion Function Is Not Smooth,”Econometrica 71, 1591-1608.
Chen, X., and A. Santos (2015): “Overidenti…cation in Regular Models,”working paper.
Chernozhukov, V., C. Hansen, and M. Spindler (2015): "Valid Post-Selection and PostRegularization Inference: An Elementary, General Approach," Annual Review of Economics 7:
649–688.
Chernozhukov, V., D. Chetverikov, M. Demirer, E. Duflo, C. Hansen, W. Newey(2016): "Double Machine Learning: Improved Point and Interval Estimation of Treatment andCausal Parameters," MIT working paper.

39

Firpo, S. and C. Rothe (2016): "Semiparametric Two-Step Estimation Using Doubly Robust
Moment Conditions," working paper.
Hasminskii, R.Z. and I.A. Ibragimov (1978): "On the Nonparametric Estimation of Functionals," Proceedings of the 2nd Prague Symposium on Asymptotic Statistics, 41-51.
Hausman, J.A., and W.K. Newey (2016): "Individual Heterogeneity and Average Welfare,"Econometrica 84, 1225-1248.
Hotz, V.J. and R.A. Miller (1993): "Conditional Choice Probabilities and the Estimation of
Dynamic Models," Review of Economic Studies 60, 497-529.
Ichimura, H., and S. Lee (2010): “Characterization of the Asymptotic Distribution of Semiparametric M-Estimators,”Journal of Econometrics 159, 252–266.
Ichimura, H. and W.K. Newey (2016): "The In‡uence Function of Semiparametric Estimators," CEMMAP working paper.
Lee, Lung-fei (2005): “A C( )-type Gradient Test in the GMM Approach,”working paper.
Newey, W.K. (1990): "Semiparametric E¢ ciency Bounds," Journal of Applied Econometrics 5,
99-135.
Newey, W.K. (1991): ”Uniform Convergence in Probability and Stochastic Equicontinuity,”
Econometrica 59, 1161-1167.
Newey, W.K. (1994): "The Asymptotic Variance of Semiparametric Estimators," Econometrica
62, 1349-1382.
Newey, W.K. (1999): ”Consistency of Two-Step Sample Selection Estimators Despite Misspeci…cation of Distribution,”Economics Letters 63, 129-132.
Newey, W.K., and D. McFadden (1994): “Large Sample Estimation and Hypothesis Testing,"
in Handbook of Econometrics, Vol. 4, ed. by R. Engle, and D. McFadden, pp. 2113-2241. North
Holland.
Newey, W.K., and J.L. Powell (1989) "Instrumental Variable Estimation of Nonparametric
Models," presented at Econometric Society winter meetings, 1989.
Newey, W.K., and J.L. Powell (2003) "Instrumental Variable Estimation of Nonparametric
Models," Econometrica 71, 1565-1578.
Newey, W.K., F. Hsieh, and J.M. Robins (1998): “Undersmoothing and Bias Corrected
Functional Estimation," MIT Dept. of Economics working paper 72, 947-962.
Newey, W.K., F. Hsieh, and J.M. Robins (2004): “Twicing Kernels and a Small Bias Propertyof Semiparametric Estimators,”Econometrica 72, 947-962.
Neyman, J. (1959): “Optimal Asymptotic Tests of Composite Statistical Hypotheses,”Probability
and Statistics, the Harald Cramer Volume, ed., U. Grenander, New York, Wiley.

40

Pakes, A. and G.S. Olley (1995): "A Limit Theorem for a Smooth Class of Semiparametric
Estimators," Journal of Econometrics 65, 295-332.
Powell, J.L., J.H. Stock, and T.M. Stoker (1989): "Semiparametric Estimation of Index
Coe¢ cients," Econometrica 57, 1403-1430.
Robins, J.M., A. Rotnitzky, and L.P. Zhao (1994): "Estimation of Regression Coe¢ cients
When Some Regressors Are Not Always Observed," Journal of the American Statistical Association 89: 846–866.
Robins, J.M. and A. Rotnitzky (1995): "Semiparametric E¢ ciency in Multivariate Regression
Models with Missing Data," Journal of the American Statistical Association 90:122–129.
Robins, J.M., A. Rotnitzky, and L.P. Zhao (1995): "Analysis of Semiparametric Regression
Models for Repeated Outcomes in the Presence of Missing Data," Journal of the American
Statistical Association 90,106–121.
Robins, J.M.,and A. Rotnitzky (2001): Comment on “Semiparametric Inference: Question
and an Answer Likelihood”by P.A. Bickel and J. Kwon, Statistica Sinica 11, 863-960.
Robins, J.M., A. Rotnitzky, and M. van der Laan (2000): "Comment on ’On Pro…leLikelihood’ by S. A. Murphy and A. W. van der Vaart, Journal of the American Statistical
Association 95, 431-435.
Robins, J., M. Sued, Q. Lei-Gomez, and A. Rotnitzky (2007): "Comment: Performance ofDouble-Robust Estimators When Inverse Probability’Weights Are Highly Variable," Statistical
Science 22, 544–559.
Robins, J.M., L. Li, E. Tchetgen, and A. van der Vaart (2008) "Higher Order In‡uence
Functions and Minimax Estimation of Nonlinear Functionals," IMS Collections Probability andStatistics: Essays in Honor of David A. Freedman, Vol 2, 335-421.
Rust, J. (1987): "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold
Zurcher," Econometrica 55, 999-1033.
Santos, A. (2011): "Instrumental Variable Methods for Recovering Continuous Linear Functionals," Journal of Econometrics, 161, 129-146.
Scharfstein D.O., A. Rotnitzky, and J.M. Robins (1999): Rejoinder to “Adjusting For
Nonignorable Drop-out Using Semiparametric Non-response Models,”Journal of the American
Statistical Association 94, 1135-1146.
Severini, T. and G. Tripathi (2006): "Some Identi…cation Issues in Nonparametric Linear
Models with Endogenous Regressors," Econometric Theory 22, 258-278.
Stoker, T. (1986): "Consistent Estimation of Scaled Coe¢ cients," Econometrica 54, 1461-1482.
Tamer, E. (2003): "Incomplete Simultaneous Discrete Response Model with Multiple Equilibria,"
Review of Economic Studies 70, 147-165.
41

van der Vaart, A.W. (1991): “On Di¤erentiable Functionals,” The Annals of Statistics, 19,
178-204.
van der Vaart, A.W. (1998): Asymptotic Statistics, Cambride University Press, Cambridge,
England.
Wooldridge, J.M. (1991): “On the Application of Robust, Regression-Based Diagnostics toModels of Conditional Means and Conditional Variances,”Journal of Econometrics 47, 5-46.

42