We develop a theory for estimation of a high-dimensional sparse parameter \theta defined as a minimizer of a population loss function L_D(\theta,g_0)which, in addition to \theta, depends on a, potentially infinite dimensional, nuisance parameter g_0. Our approach is based on estimating \theta via an \ell_1-regularized minimization of a sample analog of L_S(\theta, ), plugging in a first-stage estimate , computed on a hold-out sample. We define a population loss to be (Neyman) orthogonal if the gradient of the loss with respect to \theta, has pathwise derivative with respect to g equal to zero, when evaluated at the true parameter and nuisance component. We show that orthogonality implies a second-order impact of the first stage nuisance error on the second stage target parameter estimate. Our approach applies to both convex and non-convex losses, albeit the latter case requires a small adaptation of our method with a preliminary estimation step of the target parameter. Our result enables oracle convergence rates for \theta under assumptions on the first stage rates, typically of the order of n^{-1/4 . We apply our theory to high-dimensional versions of standard estimation problems in statistics and econometrics, such as: estimation of conditional moment models with missing data, estimation of structural utilities in games of incomplete information and estimation of treatment effects in regression models with non-linear link functions.
We propose an l1-regularized M-estimator for a high-dimensional sparse parameter that is identified by a class of semiparametric conditional moment restrictions (CMR). We estimate the nonparametric nuisance parameter by modern machine learning methods. Plugging the first-stage estimate into the CMR, we construct the M-estimator loss function for the target parameter so that its gradient is insensitive (formally, Neyman-orthogonal) with respect to the first-stage regularization bias. As a result, the estimator achieves oracle convergence rate \sqrt{k \log p/n . We apply our results to conditional moment models with missing data, games of incomplete information and treatment effects in regression models with non-linear link functions.
R
develop a theory for estimation of
propose an l1-regularized M-estimator for
D
\theta defined as a minimizer of a population loss function L_D(\theta,g_0)which, in addition to \theta, depends on a, potentially infinite dimensional, nuisance parameter g_0. Our approach is based on estimating \theta via an \ell_1-regularized minimization of a sample analog of L_S(\theta,
R
), plugging in a
that is identified by a class of semiparametric conditional moment restrictions (CMR). We estimate the nonparametric nuisance parameter by modern machine learning methods. Plugging the
R
, computed on a hold-out sample. We define a population loss to be (Neyman) orthogonal if the gradient of the loss with respect to \theta, has pathwise derivative
into the CMR, we construct the M-estimator loss function for the target parameter so that its gradient is insensitive (formally, Neyman-orthogonal)
R
g equal to zero, when evaluated at the true parameter and nuisance component. We show that orthogonality implies a second-order impact of the first stage nuisance error on the second stage target parameter estimate. Our approach applies to both convex and non-convex losses, albeit the latter case requires a small adaptation of our method with a preliminary estimation step of the target parameter. Our result enables oracle convergence rates for \theta under assumptions on the first stage rates, typically of the order of n^{-1/4
the first-stage regularization bias. As a result, the estimator achieves oracle convergence rate \sqrt{k \log p/n
R
theory to high-dimensional versions of standard estimation problems in statistics and econometrics, such as: estimation of
results to
D
estimation of structural utilities in
D
estimation of