We provide adaptive inference methods for linear functionals of sparse linear approximations to the conditional expectation function. Examples of such functionals include average derivatives , policy effects , average treatment effects, and many others. The construction relies on building Neyman-orthogonal equations that are approximately invariant to  perturbations of the nuisance parameters , including the Riesz representer for the linear functionals . We use L1-regularized methods to learn approximations to the regression function and the Riesz representer, and construct the estimator for the linear functionals as the solution to the orthogonal estimating equations. We establish that under weak assumptions the estimator  concentrates in a 1 /root n neighborhood of the target with deviations controlled by the normal laws, and the estimator attains the semi-parametric efficiency bound in many cases. In particular, either the approximation to the regression function or the approximation to the Riesz representer can be "dense" as long as one of them is sufficiently "sparse". Our main results are non-asymptotic and imply asymptotic uniform validity over large classes of models\to  .
We provide adaptive inference methods , based on l1 regularization methods, for regular (semi-parametric) and non-regular (nonparametric) linear functionals of the conditional expectation function. Examples of regular functionals include average treatment effects , policy effects from covariate distribution shifts and stochastic transformations, and average derivatives. Examples of non-regular functionals include the local linear functionals defined as local averages that approximate perfectly localized quantities: average treatment, average policy effects, and average derivatives, conditional on a covariate subvector fixed at a point. Our construction relies on building Neyman orthogonal equations for the target parameter that are approximately invariant to small perturbations of the nuisance parameters . To achieve this property we include the linear Riesz representer for the functionals in the equations as the additional nuisance parameter . We use l1-regularized methods to learn approximations to the regression function and the linear representer, in settings where dimension of (possibly overcomplete) dictionary of basis functions P is much larger than N. We then estimate the linear functional by the solution to the empirical analog of the orthogonal equations. Our key result is that under weak assumptions the estimator of the functional concentrates in a L /root (n) neighborhood of the target with deviations controlled by the Gaussian law, provided L/root(n)\to 0; L is the operator norm of the functional, measuring the degree of its non-regularity, with L diverging for local functionals (or under weak identification of the global functionals) .
R
for
, based on l1 regularization methods, for regular (semi-parametric) and non-regular (nonparametric)
D
sparse linear approximations to
R
such
regular
R
derivatives
treatment effects
R
, average treatment
from covariate distribution shifts and stochastic transformations, and average derivatives. Examples of non-regular functionals include the local linear functionals defined as local averages that approximate perfectly localized quantities: average treatment, average policy
R
many others. The
average derivatives, conditional on a covariate subvector fixed at a point. Our
R
Neyman-orthogonal equations
Neyman orthogonal equations for the target parameter
A
small
R
, including the
. To achieve this property we include the linear
R
linear functionals
functionals in the equations as the additional nuisance parameter
R
L1-regularized
l1-regularized
R
Riesz representer, and construct the estimator for the linear functionals as
linear representer, in settings where dimension of (possibly overcomplete) dictionary of basis functions P is much larger than N. We then estimate the linear functional by
R
orthogonal estimating equations. We establish
empirical analog of the orthogonal equations. Our key result is
A
of the functional
R
1
L
R
n
(n)
R
normal laws, and the estimator attains the semi-parametric efficiency bound in many cases. In particular, either the approximation to the regression function or the approximation to the Riesz representer can be "dense" as long as one of them is sufficiently "sparse". Our main results are non-asymptotic and imply asymptotic uniform validity over large classes of models
Gaussian law, provided L/root(n)
A
0; L is the operator norm of the functional, measuring the degree of its non-regularity, with L diverging for local functionals (or under weak identification of the global functionals)