Learning Before Testing: A Selective Nonparametric Test for
Conditional Moment Restrictions∗

Jia Li†

Zhipeng Liao‡

Wenyu Zhou§

January 20, 2022

Abstract
This paper develops a new test for conditional moment restrictions via nonparametric series
regression, with approximating series terms selected by Lasso. Machine-learning the main features of the unknown conditional expectation function beforehand enables the test to seek power
in a targeted fashion. The data-driven selection, however, also tends to distort the test’s size
nontrivially, because it restricts the (growing-dimensional) score vector in the series regression
on a random polytope, and hence, effectively alters the score’s asymptotic normality. A novel
critical value is proposed to account for this truncation effect. We establish the size and local
power properties of the proposed selective test under a general setting for heterogeneous seriallydependent data. The local power analysis reveals a desirable adaptive feature of the test in
the sense that it may detect smaller deviations from the null when the unknown function isless complex. Monte Carlo evidence demonstrates the superior finite-sample size and powerproperties of the proposed test relative to some benchmarks.
Keywords: conditional moments; Lasso; machine learning; series estimation; uniform inference; variable selection.
JEL Codes: C14, C22.
∗

We thank Francis Diebold, Xu Cheng, Bruce Hansen, Jack Porter, Frank Schorfheide, Xiaoxia Shi, Kenneth West,

Zhenlin Yang, Jun Yu and seminar participants at the Chinese University of Hong Kong, Singapore Management
University, University of Wisconsin-Madison, University of Pennsylvania for their comments.
†
School of Economics, Singapore Management University, Singapore; e-mail: jiali@smu.edu.sg.
‡
Department of Economics, UCLA, Log Angeles, CA 90095; e-mail: zhipeng.liao@econ.ucla.edu.
§
International Business School, Zhejiang University, Haining, Zhejiang 314400, China;
wenyuzhou@intl.zju.edu.cn.

e-mail:1

Introduction

Testing conditional moment restrictions is an important topic in econometrics. One approach is
to nonparametrically estimate the conditional moment function via a series regression (Andrews(1991a), Newey (1997)) and then test whether the function is zero in a uniform sense (Belloni,
Chernozhukov, Chetverikov, and Kato (2015), Li and Liao (2020)). In practice, however, it is often
difficult to decide which series terms should be employed to approximate the unknown function:
Using too few may induce bias, but using too many hurts efficiency. To address this issue, it
seems natural to apply some machine-learning-based variable selection procedure such as the Lasso(Tibshirani (1996)) and its variants. Although such methods may achieve the so-called “oracle
property” in large samples, they are unlikely to completely meet that theoretical ideal in finite
samples. Ignoring the sampling variability in the selection step may thus lead to possibly severe
size distortions in the subsequent test (see Section 4 for concrete Monte Carlo evidence).
The main contribution of this paper is to propose a new critical value for the nonparametrictest, which properly accounts for the effect of the preliminary Lasso-based selection. Our analysis
reveals that the first-stage selection affects the second-stage inference by restricting the seriesregression score on a random polytope. Since the asymptotics of the series estimator is capturedby the (growing-dimensional) Gaussian coupling for the score vector, this restriction effectively
results in a form of truncated normality, which explains the size distortion of the “naive” criticalvalue directly constructed from the conventional asymptotic Gaussian approximation. The novel
critical value proposed in this paper accounts for the truncation effect, and it greatly improves thetest’s size control in finite samples as shown in our simulation study.
To better understand the power property of the proposed test, we also characterize local alternatives against which the test is consistent. The power analysis clarifies an adaptive feature of the
proposed selective test: The test is able to detect smaller deviations from the null if the deviation
has a simpler form. In the extreme case when the unknown function can be approximated by abounded number of series terms (but with a priori unknown identities), the test achieves consistency nearly—up to a logarithmic factor—at the parametric rate. In the worst-case scenario in
which the unknown function is “very complex” (e.g., all covariates have equal predictive power),
the power of the selective test deteriorates to the same level as the benchmark non-selective test.
But in general, the former is shown to be more powerful than the latter.
The remainder of this paper is organized as follows. We present the econometric method2and the related asymptotic theory in Section 2, followed by a couple of extensions in Section
3. Section 4 demonstrates the finite-sample performance of the proposed test in a Monte Carloexperiment. Section 5 concludes. The Appendix provides requisite implementation details anda pedagogical example. Technical proofs and additional simulation results are relegated to theOnline Supplemental Appendix.

2

Main theory

We present our main theory in this section. Section 2.1 describes the testing problem and somerelated background. Section 2.2 presents the new selective test, with its theoretical properties
established in Section 2.3.

2.1

The testing problem and some backgroundWe start with introducing the econometric setting. Consider a series (Yt , Xt> ), 1 ≤ t ≤ n, ofobserved data, where Yt is scalar-valued and Xt takes values in a compact set X ⊆ Rd .1 Denotethe conditional expectation function of Yt given Xt byg(x) ≡ E [Yt |Xt = x] ,x ∈ X,

with the associated residual term t ≡ Yt − g (Xt ). Our econometric interest is to test the null
hypothesis
H0 : g(x) = 0 for all x ∈ X ,(2.1)

against its complementary alternative, that is, g(x) 6= 0 for some x. In some applications, Yt
may depend on an unknown finite-dimensional parameter θ∗ and, if so, we may emphasize thisdependence explicitly by writing Yt (θ∗ ). The testing of conditional moment restrictions arises
routinely from various empirical settings. To help fix ideas, we briefly consider two prototype
examples.
Example 1 (Forecast Rationality). Suppose that at time t a forecaster produces a one†
period-ahead forecast Ft+1|t for the next period’s target variable, denoted Ft+1
(e.g., inflation).

It is well-known that given a time-t information set It , the optimal forecast that minimizes the†mean-squared-error loss is the conditional expectation E[Ft+1|It ]. Therefore, if Ft+1|t is optimal,†
the forecasting error Yt = Ft+1|t −Ft+1
should satisfy E [Yt |Xt ] = 0 for any Xt in the It information
1

We consider scalar-valued Yt mainly for ease of exposition. The econometric method can be trivially extended

to accommodate multivariate Yt .

3

set. In this setting, a test for (2.1) can be interpreted as a test for forecast rationality, as studied
by Hansen and Hodrick (1980), Brown and Maital (1981), and Romer and Romer (2000), among
others.
Example 2 (Euler and Bellman Equations). In dynamic equilibrium models, the equilibrium
is often characterized by Euler equations in the form of conditional moment restrictions. The
classical example is Hansen and Singleton’s (1982) study of consumption-based asset pricing, inwhich the one-period-ahead pricing equation takes the form
βU 0 (Ct+1 , γ)
E
R
−
1
X
t+1
t = 0,U 0 (Ct , γ)

(2.2)

where U 0 (·, γ) is the representative agent’s marginal utility function with a preference parameterγ, β is the discounting factor, Ct is the consumption process, Rt+1 is the return of an asset,
and Xt is the state variable underlying the dynamic model. Equation (2.2) can be written in0

t+1 ,γ)the form of (2.1) by setting θ∗ = (β, γ) and Yt (θ∗ ) = βUU 0(C(Ct ,γ) Rt+1 − 1. Similar equilibrium

conditions can also be derived from Bellman equations; see, for example, Li and Liao (2020). In
the macroeconomic setting, the parameter θ∗ is often, though not always, calibrated based on
external data and auxiliary models.
As seen from these examples, the Yt variable may play different roles in different contexts
and sometimes may involve a finite-dimensional parameter θ∗ that may be estimated or calibrated
depending on the style of empirical research. In addition, it is generally important to accommodate
time-series dependence for these applications. In the remainder of Section 2, we shall assume that
Yt is directly observed so as to focus on the main innovation of the present paper. It is relatively
straightforward to allow for the presence of an unknown θ∗ , and we will develop that extension in
Section 3.
Testing the hypothesis in (2.1) is econometrically nontrivial, as it concerns the global property of the conditional expectation function g (·). In practice, empiricists often take “parametricshortcuts” to bypass that nonparametric functional inference problem. The simplest way to doso is to integrate out the conditioning variable Xt and test the unconditional moment restriction
E [Yt ] = 0. This amounts to regressing Yt on a constant term and then conducting a t-test. To
incorporate the conditioning information in Xt , it is common to run a linear regression
Yt = a + b> Xt + et ,(2.3)

and then test whether the coefficients are all zero. The pros and cons of the parametric approach
are also well understood. On one hand, if the observed data were known to be generated under4

the conjectured specification, the parametric approach would clearly be the simplest and the mostefficient way to carry out the test. On the other hand, if the null is violated in a way that is“orthogonal” to the given parametric specification, the test will have little power in detecting it.
In applied work, non-rejections may thus be challenged by a critical reader, because the parametric
test is designed to seek power only in very specific directions that are generally hard to justify on
the ground of economic theory.
A natural way to address this issue is to make the regression more flexible by including additional nonlinear terms. Following Andrews (1991a) and Newey (1997), one may formalize thismore general approach as a nonparametric series regression so as to directly attack the functional
inference. Consider a collection of approximating basis functions (pj (·))1≤j≤m such as polynomials,
splines, trigonometric functions, wavelets, etc., and set P (·) ≡ (p1 (·), . . . , pm (·))> . We may regress
Yt on P (Xt ) and construct the associated nonparametric series estimator for g (·) as
!−1 n
!
n
X
X
ĝ(·) ≡ P (·)>
P (Xt ) P (Xt )>P (Xt ) Yt .
t=1

(2.4)

t=1

With the number of series terms m → ∞, the specification of this regression becomes increasingly
more flexible in larger samples, so that the series approximation will approach the true unknown
function. The test can then be carried out by examining whether the estimated function ĝ (·) is
statistically zero in a uniform sense. A theoretical subtlety stems from the fact that the uniform
inference for the series estimator is a non-Donsker problem due to the growing dimensionality of
the regressors.2 Li and Liao (2020) show that the estimation error function ĝ (·) − g (·) can bestrongly approximated, or coupled, by a sequence of divergent Gaussian processes in a time-series
setting for general heterogeneous mixingales; also see Belloni, Chernozhukov, Chetverikov, andKato (2015) for a similar analysis in the cross-sectional setting. Based on that theory, one may
test g (·) = 0 using a “functional t-test” based on the sup-t statistic supx∈X |ĝ (x)| /ŝ (x), whereŝ (·) is an estimator of the standard error function of ĝ (·). The null hypothesis is rejected if the
sup-t statistic is greater than a critical value determined by the strong Gaussian approximation.
A more detailed discussion is given in Section 2.2 below.
The advantage of the nonparametric approach is that it speaks directly to the original hypothesis (2.1), whereas the parametric approach concentrates only on some of its implications. Thatbeing said, the flexibility of the nonparametric approach comes with an efficiency cost, which manifests theoretically in the relatively slow rate of convergence of the nonparametric estimator. The
cost is also easily understood in practical terms. Indeed, if g (x) happens to be a linear function in2

That is, the ĝ (·) estimator does not satisfy a functional central limit theorem (i.e., Donsker theorem) as consid-

ered in Pollard (2001), Andrews (1994), and van der Vaart and Wellner (1996), among others.

5x, adding (a growing number of) higher-order polynomial terms in the series regression ought tobe a “waste.” In practice, the null hypothesis may be rejected by a simple parametric test, whereas
the theoretically “omniscient” nonparametric test may be less powerful and fail to reject.
The above discussion clarifies the trade-off between flexibility and efficiency in the present
testing context: Flexibility requires the inclusion of a large number of regressors that might beuseful, but in order to achieve sharper inference, it would be better to focus on a small numberof regressors that are actually useful for capturing the main features of the alternative. Our goal
is to improve this trade-off margin. A reasonable approach is to first properly select a subset
of approximating functions in the spirit of “feature extraction,” and only use them to run theseries regression and construct the sup-t test statistic. Given this goal, as well as the leastsquares structure of the series regression, the Lasso method is clearly the most natural choice for
implementing the selection.
We refer to this proposal as the selective test. The aforementioned construction of the test
statistic is arguably straightforward in view of the prior literature on series estimation and Lasso.
The key challenge for carrying out the selective test, however, is to properly determine its criticalvalue. This turns out to be highly nonstandard. A seemingly reasonable approach is to treat
the Lasso-selected subset of approximating functions “as given” (i.e., ignoring the fact that it isdata-driven), and compute the critical value in the same way as in the benchmark non-selective
test (Belloni, Chernozhukov, Chetverikov, and Kato (2015), Li and Liao (2020)). However, as weshall show in the simulations (see Section 4), this approach may result in quite nontrivial size
distortions. This motivates us to develop a correction for the critical value so as to account for theselection effect. Our analysis is in spirit akin to the sequential inference theory commonly seen in
econometrics (cf. Section 6 of Newey and McFadden (1994)). But, unlike the conventional setting,our “first-stage estimator” may be viewed as set-valued (in the form of a selection event), andit affects the second-stage through a fairly complicated truncation of the support of the coupling
Gaussian process that drives the asymptotics of the series estimator. We now turn to the details.

2.2

The selective test

We construct the selective test in this subsection. Let M denote the index set associated with acollection (pj (·))1≤j≤m of candidate approximating functions. For ease of discussion, we identify
M with the associated collection of approximating functions, and refer to it as a dictionary. We
assume that the size of M grows asymptotically (i.e., m → ∞ as n → ∞), though its dependence
on n is kept implicit in our notation for simplicity. For any nonempty subset S ⊆ M, we denote
PS (·) ≡ (pj (·))j∈S , which collects a subset of approximating functions selected by S. The specific6ordering of the pj (·) components is irrelevant, because our statistics of interest are all invariant to
the ordering. We refer to S as a selection and denote its cardinality by |S|. Analogous to (2.4),
the series estimator for g (·) based on the selection S is given by>ĝS (·) ≡ PS (·)

n
X

!−1
>

PS (Xt ) PS (Xt )

t=1

n
X

!
PS (Xt ) Yt

.

(2.5)

t=1

Note that this includes the non-selective estimator ĝ (·) defined in (2.4) as a special case corresponding to S = M. The standard error function associated with ĝS (·) is given byq−1
σS (·) ≡ PS (·)> Q−1
S ΣS QS PS (·),

PP
where QS ≡ n−1 nt=1 E PS (Xt )PS (Xt )> and ΣS ≡ Var[n−1/2 nt=1 PS (Xt )t ]. We may estib S ≡ n−1 Pn PS (Xt )PS (Xt )> and estimate ΣS using a (growing-dimensional)
mate QS via Q
t=1

b S , following known results inheteroskedasticity and autocorrelation consistent (HAC) estimator Σ
the literature.3 The standard error function σS (·) can then be estimated viaqb −1 PS (·).
bSQ
b −1 Σ
σbS (·) ≡ PS (·)> Q
S
S

(2.6)

Finally, we define the sup-t test statistic associated with the selection S asn
TbS ≡ supx∈X

1/2 ĝ (x)SσbS (x).

(2.7)

We use Lasso to perform a data-driven selection from the dictionary M. In some empiricalapplications, the user may consider a subset M0 ⊆ M of regressors to be important a priori (possibly based on economic reasoning) and like to “manually” select them into the series regression.
To accommodate this type of customization, we design a selection procedure that always includes
the prior choice set M0 and relies on Lasso to select additional regressors from the remainder set
Mc0 ≡ M \ M0 . For example, the user may insist on using a constant and a linear term in theseries regression, but is uncertain about which higher-order polynomial terms should be includedin addition. In this situation, they may put the constant and linear terms in M0 , and let Lasso to“machine-learn” whether and which additional terms are needed. The role of Lasso in this designis thus to assist the user’s choice rather than dictating it. Below, we maintain a mild convention
3b S to be the classical Newey–West estimator or more generally the HAC estimators studied by
We may take Σ

Andrews (1991b). The consistency and rate of convergence of these HAC estimators have been established in a
general time-series setting with growing dimensions by Li and Liao (2020); see their Lemma B3. The consistencyb S towards QS follow a law of large numbers of growing-dimensional matrices; see, forand rate of convergence of Q
example, Lemma 2.2 in Chen and Christensen (2015) and Lemma B2 in Li and Liao (2020).

7

that M0 contains at least the constant term (which is also our recommended default choice);
this ensures the selected set of regressors to be non-empty, and hence, avoids an uninteresting
degeneracy.
This “Lasso-assisted” selection is implemented as follows. Given the user’s prior choice M0 ,we perform a Lasso estimation with the resulting estimator given by


n


X
X
1
Lasso
(Yt − P (Xt )> β)2 + λn
ωj |βj | ,β̂≡ argminβ∈Rm  2
c
t=1

(2.8)

j∈M0

where λn is a sequence of penalty parameters commonly seen in Lasso-type problems, and (ωj )j∈Mc0
is a collection of positive weights. Note that the L1 penalty is applied only to the remainder set
Mc0 , whereas the coefficients in the prior choice set M0 are unrestricted. A simple choice of thepPωj weights is to set ωj = 1 identically or ωj = n−1 nt=1 pj (Xt )2 (see, e.g., Zhao and Yu (2006),
Bickel, Ritov, and Tsybakov (2009), and Belloni and Chernozhukov (2011)), but the more general
setting in (2.8) also accommodates the adaptive Lasso (Zou (2006)). In Appendix A.1, we provide
a concrete data-driven choice of these penalty parameters and establish its theoretical validity
within our econometric framework. Due to the L1 penalty, many coefficients indexed by Mc0 will
be shrunk to zero. Our Lasso-assisted selection is then given by
o
[n
Lasso
L ≡ M0j ∈ Mc0 : β̂ j
6= 0 ,(2.9)

which consists of the user’s ex ante choice M0 and Lasso’s ex post selection from Mc0 . The
corresponding selective test statistic is defined asn
TbL ≡ TbS S=L = supx∈X

1/2 ĝL (x)σbL (x).

(2.10)

A large value of the test statistic TbL signifies a violation of the null hypothesis (i.e., g (·) 6= 0).
The remaining task is to determine a critical value for TbL . Before elaborating our proposal,
it is instructive to first review how the critical value may be constructed in a simpler benchmark
scenario with a nonrandom selection S, as studied in Belloni, Chernozhukov, Chetverikov, andKato (2015) and Li and Liao (2020).4 These authors show that TbS can be strongly approximatedby the supremum of a Gaussian process under the null hypothesis. More precisely, there exists a
eS ∼ N (0, ΣS ) such thatsequence of |S|-dimensional Gaussian random vectors N
TbS − TeS = op (1) ,4ePS (x)> Q−1
S NS
where TeS ≡ sup.
σS (x)
x∈X(2.11)The theory of Belloni, Chernozhukov, Chetverikov, and Kato (2015) and Li and Liao (2020) does not explicitly

involve selection, and hence, corresponds to the case with S = M. But it is easy to see that their inference theory
can be trivially adapted to accommodate any nonrandom selection S.

8

The 1 − α quantile of TeS can thus be used as a critical value for TbS at significance level α. A
feasible version of this critical value can be estimated as the 1 − α quantile ofb −1 Ne∗PS (x)> Q∗
SSe
TS ≡ sup
,
σ
b
(x)
S
x∈X

(2.12)e ∗ , conditional on data, is N (0, Σb S )-distributed.
where NS
A seemingly natural way to construct TbL ’s critical value is to directly apply this benchmark
theory by plugging in S = L. However, this approach turns out to suffer from nontrivial size
distortion as shown in our simulation study below. To address this issue, we need to account for
the effect of selection and adjust the critical value accordingly. The remainder of this subsection
is devoted to this task.
More notation is needed. Let In denote the n-dimensional identity matrix,  ≡ (t )1≤t≤n , andG ≡ (g(Xt ))1≤t≤n . By convention, all vectors are column vectors. For any S ⊆ M, denote PS ≡(PS (X1 ), . . . , PS (Xn ))> . When S = M, we suppress its subscript by simply writing P = PM .
e S denote the residual matrix obtained from projecting PS onto PM . That
In addition, let P
0e S ≡ Dn PS , where Dn ≡ In − PM (P> PM )−1 P> . Finally, we define ŝ as a |L \ M0 |is, P
0
0
M0
M0
Lasso
Lasso
dimensional vector that collects the signs of β̂ j
for j ∈ {j : β̂ j6= 0} \ M0 .
We first characterize the selection event. By the Karush–Kuhn–Tucker conditions for the Lasso
problem (2.8), the selection event can be represented by a system of linear-inequality restrictions onP
n−1/2 P>  = n−1/2 nt=1 P (Xt ) t , which is the score vector of the series regression using the entire
dictionary of regressors.5 Specifically, for any nonrandom selection S satisfying M0 ⊆ S ⊆ M anda sign vector s ∈ {±1}|S\M0 | , we haveno{L = S, ŝ = s} = n−1/2 P>  ∈ Π(S, s, λn ) .

(2.13)

Here, Π(S, s, λn ) is an m-dimensional (random) polytope given by






−1/2

diag (s) (AS z + cS ) > n
λn bS (s) and mΠ(S, s, λn ) ≡ z ∈ R :
,




−1/2
0
0
0
−1/2
0


n
λn bl,S (s) < AS z+cS < nλn bu,S (s)where diag (s) is a diagonal matrix with its diagonal components given by s,


−1


e>
e S\M
e>
 cS ≡ n1/2 P
P
P
S\M0
S\M0 Dn G,
0



−1

0
−1/2 e >
>
>

e
e
e
e
 cS ≡ n
PM\S In − PS\M0 PS\M0 PS\M0
PS\M Dn G,(2.14)(2.15)

0

5

See Lemma SA.1 in the Supplemental Appendix for details, which extends a similar result in Lee, Sun, Sun, andTaylor (2016) by allowing for the prior choice set M0 and penalty weights ωj .

9and bS (s), b0l,S (s), b0u,S (s), AS , and A0S are directly observable quantities. The latter observable
quantities do not pose any difficulty in our theoretical analysis (though they are needed for implementation). We thus defer their somewhat complicated definitions to Appendix A.1 to streamline
the discussion; see (A.1). On the contrary, the random vectors cS and c0S are unobservable because
G involves the unknown g (·) function. For this reason, the structure of the polytope Π(S, s, λn )is not directly observed, either.
The significance of the above (non-asymptotic) characterization is that it precisely depicts
the relation between the Lasso-assisted selection and the subsequent series estimation in finite
samples, through their common dependence on the score vector n−1/2 P> . To see this moreclearly, recall that for any given selection S, the asymptotic normality (formulated in terms ofstrong Gaussian coupling) of the ĝS (·) estimator is driven by the score n−1/2 P>
S , which is asubvector of n−1/2 P> . However, when S is selected by Lasso with a particular sign configurations, the score n−1/2 P>  is restricted within the polytope Π(S, s, λn ). This restriction would modify
the score’s asymptotic normality into a form of truncated normality. Roughly speaking, the datadriven selection may make the originally exogenous conditioning variables “effectively endogenous”
through truncating the support of the score. A failure to account for this effect would generally
lead to size distortion. It is important to note that this type of size distortion is distinct from theusual small-sample phenomenon that central limit theorems may not “kick in” sufficiently well ina moderately sized sample; indeed, the aforementioned truncation effect would arise even if thescore is exactly normally distributed (say, in a Gaussian model with fixed design).
We now propose a new critical value to adjust for the truncation effect. The key is to constructa feasible approximation for the unobserved polytope Π(S, s, λn ) for any given S that contains M0 .
As mentioned above, the polytope is not directly observed because G is unknown. To construct
an approximation for G, we regress Y on PS with the resulting regression coefficient given by−1bbS ≡ P> PS
P>
S Y.
S

(2.16)We then apply a truncation on this least-squares estimator to obtain β̃ S , with its jth component
given bynoβ̃ S,j ≡ bbS,j · 1 |bbS,j | ≥ log(n)n−1/2 σ
bS,j ,(2.17)

where bbS,j denotes the jth component of bbS , and σ
bS,j is the estimated standard error of bbS,jb −1 Σ
bSQ
b −1 .6 The n-dimensional vector
obtained as the square-root of the jth diagonal element of Q
SS

6

The intuition for using the truncation is as follows. If the estimator b
bS,j corresponds to a zero coefficient in the−1/2bpopulation, bS,j /(nσbS,j ) is approximately N (0, 1). In addition, these “zero” t-statistics are uniformly boundedby the log (n) factor with probability approaching 1. The truncation shrinks these noisy estimates of zero directly10G is then approximated by the (truncated) projection PS β̃ S . Plugging this approximation into
(2.15), we further obtain (after simplifying the expressions) approximations for cS and c0S in theform ofbcS = n1/2 β̃ S\M0 ,b
c0S = 0,where β̃ S\M0 is the subvector of β̃ S extracted in accordance with S \ M0 as a subset of S. A
feasible proxy for Π(S, s, λn ) can then be obtained by replacing (cS , c0S ) with (bcS , bc0S ) defined in(2.14), that is,diag (s) (AS z + n1/2 β̃ S\M0 ) > n−1/2 λn bS (s) andb
Π(S,s, λn ) ≡ z ∈ Rm :
n−1/2 λn b0l,S (s) < A0S z < n−1/2 λn b0u,S (s)






.

(2.18)e ∗ be an m-dimensional standard
We are now ready to construct the new critical value. Let Ne ∗ as theGaussian random vector that is independent of the data. For a given selection S, define NSb 1/2 Ne ∗ extracted in accordance with S as a subset of M, and use it to compute Te∗
subvector of ΣSM
as described in (2.12). We then set
b 1/2 Ne ∗ ∈ Π(S,
b


s,
λ
)
P∗ TeS∗ ≥ u, Σ
n
M


=α ,
cvS,α ≡ inf u ∈ R :
b 1/2 Ne ∗ ∈ Π(S,bP∗ Σs, λn )

(2.19)Me ∗ given data.7 Our proposed critical value iswhere P∗ (·) denotes the conditional distribution of N
obtained by evaluating cvS,α at S = L, that is,
cvL,α ≡ cvS,α S=L .

(2.20)The selective test rejects the null hypothesis in (2.1) if TbL > cvL,α .
The intuition for the proposed critical value is as follows.8 Note that the (conditionally) Gausb 1/2 N
e ∗ provides a distributional approximation for the score vector n−1/2 P> . Sincesian vector ΣMe ∗, Σ
b 1/2 Ne ∗ and Te∗ provide a joint distributional approximation
TeS∗ is formed using the subvector N
S
S
M
−1/2
>
for the score nP  and the sup-t statistic TbS under the null hypothesis. As such, the joint
asymptotic behavior of the test statistic and the selection event {n−1/2 P>  ∈ Π(S, s, λn )} is capb 1/2 N
e ∗ ∈ Π(S,btured by that of Te∗ and {Σs, λn )}. The critical value described in (2.19) is simplySM

to zero. This noise-reduction generally leads to better performance in finite samples.
7e ∗ . A computationally more
This critical value may be computed by simulating the Gaussian random vector N
efficient method is to sample directly from the truncated normal distribution in restriction to the selection event.
The Matlab package accompanying this paper follows the latter computational strategy by using the minimax tilting
algorithm proposed in Botev (2016).
8
Appendix A.2 provides a pedagogical example, where we use a simple linear regression model to illustrate more
concretely the effect of model selection on the subsequent inference and the intuition for the new critical value.

11defined as a tail quantile of the conditional distribution of TeS∗ in restriction to the “coupling”be ∗ ∈ Π(S,b 1/2 Ns, λn )}, which captures how the polytope restriction on the score
selection event {ΣM

vector distorts the distribution of the sup-t statistic.9
We close this subsection with a couple of remarks. We first note that our strategy for correcting
the critical value is not restricted to the Lasso method. For the other methods such as the group
Lasso (Yuan and Lin (2006)) and the elastic net (Zou and Hastie (2005), Zou and Zhang (2009)),
one may modify the underlying Karush–Kuhn–Tucker conditions accordingly and characterize theselection event in a similar fashion as (2.13). Critical values may then be constructed from thecorresponding conditional coupling distributions. Secondly, we stress that our analysis focuses on
testing whether g (·) = 0, and hence, our inference concentrates on the null hypothesis. Anotheropen question is how to make uniform inference for the unknown function g (·) also under thealternative, while properly accounting for the selection effect. The latter question is more challenging because, under “local” alternatives, the selection may miss “moderate” features of g (·),
and lead to non-negligible biases for inference.10 This is not an issue (in terms of size control) for
our hypothesis-testing problem because under the null g (·) is known to be zero.

2.3

Asymptotic properties of the selective test

We now establish the asymptotic properties of the selective test. We shall show that the proposed
test has valid size control under the null hypothesis. We also analyze the test’s power under local
alternatives so as to theoretically clarify how the Lasso-assisted selection helps improve power. In
this subsection, we focus on the baseline setting in which Yt is directly observed. The result canbe straightforwardly extended to allow Yt to depend on some unknown parameter θ∗ ; see Section
3 below. We start with introducing a few regularity conditions.
9

One may wonder whether the size correction can be automatically achieved via resampling methods such as thebootstrap. We investigate this possibility through a simulation study in Section SC.2 of the online supplemental
appendix. The simulation results show that a test based on the i.i.d. bootstrap tends to be very conservative andhave poor power (even if there is no serial dependence in the data). The theoretical investigation of resampling
methods for the selective test is beyond the scope of this paper and is left for future research.
10
The uniform nonparametric inference with a data-driven selection of series terms should be distinguished froma recent strand of literature on “selective inference.” For example, in a Gaussian linear model, Lee, Sun, Sun, andTaylor (2016) study the inference for the coefficients of a submodel selected by Lasso. That research question is very
different from making uniform nonparametric inference, because it would effectively shift the inferential target fromg(·) to a statistically selected submodel; the latter is a “moving target” that could be very different from the original
object of interest. That being said, selective inference may be fruitfully used in many other econometric problems,as demonstrated in the recent interesting work by Cox and Shi (2019), Liao and Shi (2020), Andrews, Kitagawa,and McCloskey (2021a,b); our coupling-based inference technique might be useful to extend that line of research to
growing-dimensional or functional settings for general serially dependent data.

12

Assumption 1. (i) The eigenvalues of QM and ΣM are bounded from above and away from zero;en such that(ii) there exists a sequence of m-dimensional standard Gaussian random vectors N
n−1/2

n
X

1/2

en + op (log(n)−1 );P (Xt ) t = ΣM N

t=1

b M − QM k + kΣb M − ΣM k = op ((m1/2 log(n))−1 ); (iv) m = o(n) and log(ζ L ) = O(log(m)),(iii) kQnwhere ζnL ≡ supx1 ,x2 ∈X kP (x1 ) − P (x2 )k / kx1 − x2 k.
Assumption 1 imposes high-level conditions that are similar to those used in prior work on uniform series-based inference. Condition (i) is fairly standard for series estimation; see, for example,
Andrews (1991a), Newey (1997), and Chen (2007). Condition (ii) requires that the scaled score
P
sequence n−1/2 nt=1 P (Xt )t admits a Gaussian coupling in the growing-dimensional setting (i.e.,m → ∞), which may be verified by applying Yurinskii’s coupling for i.i.d. data, or the theoryof Li and Liao (2020) in the more general time-series setting for heterogeneous mixingales. Thiscondition is crucial for our purpose of making uniform functional inference and it also implicitly
imposes the binding restriction within our analysis on how fast m may grow with the sample size
(especially in the presence of series dependence); see Li and Liao (2020) for a more detailed techb M and Σb M , which cannical discussion.11 Condition (iii) pertains to the convergence rates of Q
be verified under primitive conditions as shown in Chen and Christensen (2015) and Li and Liao(2020). Condition (iv) is trivially satisfied by commonly used series basis.
To set the stage for the local power analysis, we consider a sequence of data generating processes
under which E [Yt |Xt = x] = gn (x), where gn (·) is a (possibly) drifting sequence of functions. These
functions are assumed to satisfy the following.
Assumption 2. (i) There exists a sequence (b∗n )n≥1 of m-dimensional constant vectors such thatsup n1/2 gn (x) − P (x)> b∗n = O(1);x∈X(ii) there exists a subset R ⊆ Mc0 such that minj∈R |b∗n,j | > 0 and b∗n,j = 0 when j ∈ Mc0 \ R.
Assumption 2(i) states that the gn (·) function may be approximately represented by thegrowing-dimensional b∗n vector, which specifies how gn (·) loads on the basis functions. This is11

If one restricts attention to the setting with i.i.d. data, it might be possible to generalize our result to allow m

to grow faster possibly under some additional sparsity restriction (which is not assumed here). We do not pursuethat extension in the present paper because our primary goal is to accommodate serial dependence commonly seenin time-series settings so as to facilitate macro and finance applications, and certain applied-micro applications
involving panel data. Establishing a theory for m > n under sparsity is not our main focus, but might be aninteresting topic for future research; this could be technically challenging in a setting with general serially dependent
data (e.g., mixingales) as considered here.

13well understood in series estimation, for which comprehensive results are available from the literature on numerical approximation (see, e.g., Chen (2007)); this setup also directly accommodates
linear specifications with “many regressors.” Given this representation, condition (ii) further introduces a “relevance set,” R, which marks all basis functions in Mc0 (on which the Lasso selectionis active) with nonzero loadings. Note that R is empty under the null hypothesis, but it plays animportant role under the alternative.
Intuitively, if the user knew the (actually unknown) structure of R a priori, it would be natural
to combine it with their prior choice M0 to form an “oracle” selection
M? ≡ M0 ∪ R,
which is arguably the best one may wish to obtain from any selection algorithm (e.g., Lasso). The
M? set thus depicts the intrinsic complexity of gn (·) given the user’s ex ante choice (including thedictionary M and the prior choice M0 ). In this sense, gn (·) is the most complex when M? = M,
because one would use all basis functions to conduct the series estimation. On the other extreme, if
M? is “sparse” in the sense that it contains only a few elements, gn (·) is “effectively parametric,”
and hence, relatively simple to uncover. Consistent with this logic, our theory presented below
shows that the selective test satisfies an adaptive property, namely, it is more powerful when
the alternative is less complex. In our analysis, it turns out that the aforementioned notion ofcomplexity may be more precisely quantified as (with k · k denoting the Euclidean norm)κ (M? ) ≡ sup kPM? (x)k ,x∈Xwhich is a non-decreasing function of M? with respect to the partial order of set inclusion. Hence,a larger M? corresponds to a higher value or faster divergence rate of κ (M? ), and vice versa.12
We also need the following condition on the Lasso penalty. For any m1 × m2 real matrix
P 1
A = [Aij ]1≤i≤m1 ,1≤j≤m2 , we denote kAk1 ≡ max1≤j≤m2 m
i=1 |Aij |.
Assumption 3. The penalty parameters λn and {ωj }j∈Mc0 satisfy|R|1/2 n−1/2 λn maxj∈R ωj + log(n)(n log(m))1/2+= op (1)λn minj∈Mc0 \R ωjn1/2 minj∈R |b∗n,j |(2.21)

and, for some fixed η ∈ (0, 1),
maxj∈R ωje>Pe −1 e > e(P≤1−ηR R ) PR PMc0 \Rminj∈Mc0 \R ωj1(2.22)

with probability approaching 1.
12In our theory, M? is allowed to contain a growing number of elements. Therefore, κ (M? ) is typically a divergent

sequence of positive numbers and its “magnitude” is gauged by its growth rate to infinity.

14

Assumption 3 mainly ensures that the Lasso estimator described in (2.8) is sign-consistentunder the null and alternative hypotheses.13 This condition is high-level in nature and it does notdirectly pin down any specific penalty scheme. In Appendix A.1, we provide a concrete feasible
choice that fulfills this technical condition.
We are now ready to state the asymptotic size and power properties of the selective test, whichis the main result of this paper. Below, for two sequences of positive numbers an and bn , we writean  bn if an ≥ cn bn for some strictly positive sequence cn → ∞.
Theorem 1. Under Assumptions 1, 2, and 3, the following statements hold for any significancelevel α ∈ (0, 1/2): (a) The selective test has asymptotic level α under the null hypothesis (2.1),
that is, P(TbL > cvL,α ) → α; (b) the selective test is consistent against any local alternative thatsatisfies
sup |gn (x)|  κ (M? ) log(n)1/2 n−1/2 ,(2.23)

x∈X

that is, P(TbL > cvL,α ) → 1.
Part (a) of Theorem 1 shows that the selective test has valid size control under the null
hypothesis. Part (b) further establishes the consistency of the test against local alternatives thatsatisfy condition (2.23), with the “boundary” of the local neighborhood (under the uniform metric)
characterized by the κ (M? ) log(n)1/2 n−1/2 rate.
The local power result deserves some additional discussion. Its key significance is to provide
a sense in which the selective test is adaptive with respect to the complexity of gn (·) as gaugedby κ (M? ). That is, the test is able to consistently detect a faster-vanishing nonzero sequence ofsupx∈X |gn (x)| when the gn (·) function is easier to approximate (i.e., M? is smaller), despite thefact that this information is unknown a priori. This is an important improvement relative to thebenchmark non-selective method (cf. Belloni, Chernozhukov, Chetverikov, and Kato (2015) andLi and Liao (2020)). Indeed, the non-selective method employs the entire dictionary M of basis
functions, which can be considered as a corner case of the selective test corresponding to the mostconservative prior choice M0 = M. The power of the non-selective test is thus always dictated by
the fast-diverging sequence κ (M), and hence low, regardless of the actual complexity underlying
the data generating process (i.e., it is non-adaptive).
To further appreciate the adaptiveness of the selective test, we consider another “corner” casein which the set M? only contains a bounded number of elements. This corresponds to a situation
13

Recall that R is empty under the null hypothesis. By convention, we set the maximum and minimum of acollection of nonnegative numbers over the empty set to 0 and ∞, respectively. Under this convention, Assumption
p3 can be reduced to a simpler form under the null hypothesis, that is, n log(m)/(λn minj∈Mc0 ωj ) = op (1).

15

in which gn (·) under the alternative has a parametric but a priori unknown form. In this case,κ (M? ) is bounded and the κ (M? ) log(n)1/2 n−1/2 rate can be simplified as log(n)1/2 n−1/2 , which isessentially the n−1/2 parametric rate. Attaining this nearly parametric rate is remarkable because
the user was “prepared” to conduct a nonparametric analysis, as they did not know beforehandthat gn (·) has a simple form, let alone its specific parametric specification among a large number(i.e., 2m with m → ∞) of possibilities. In sharp contrast, the non-selective method has power onlyat the well-known and much slower nonparametric rate.

3

Extensions

We consider two extensions for our baseline method developed in the previous section. Section 3.1
presents a different version of the selective test based on an alternative test statistic. Section 3.2
describes two approaches for handling unknown finite-dimensional parameters.

3.1

Selective test with an alternative test statistic

The TbL test statistic is based on the supremum of the (studentized) series estimator for g (·), whichquantifies deviations from the null under the uniform metric on the functional space. This is anatural choice in the nonparametric setting, as g (·) is the model primitive in that context. That
noted, the underlying econometric idea can be easily extended to accommodate the other types oftest statistics. In this subsection, we provide a concrete example to demonstrate how our baseline
theory may be modified for that purpose.
Consider an alternative test statistic defined as the maximum of the t-statistics associated withindividual regression coefficients in the series regression. Since this statistic is directly based on the
regression coefficients, it is perhaps better suited than TbL for studying linear models with “many”
regressors. Specifically, for each given selection S, we define the test statistic as
n1/2 bbS,j
TbS0 ≡ max,
σbS,j1≤j≤|S|(3.24)

where we recall that bbS,j is the jth component of the series regression coefficient bbS (see (2.16))
and σbS,j is the estimated standard error obtained as the square-root of the jth diagonal element
−1
b Σ
bSQ
b −1 . Its feasible distributional “coupling” is given byof Q
SS −1 ∗ 
b N
e
Q
S j
S
0∗
TeS ≡ max,
(3.25)σbS,j1≤j≤|S|e ∗ is a subvector exacted from
where the [ · ]j operator extracts the jth component of a vector and NS
b 1/2 Ne ∗ in accordance with S as a subset of M for some generic m-dimensional standard normalΣM16e ∗ . Analogous to (2.19), we define the critical value for the new test statistic as
random vector Nb 1/2 Ne ∗ ∈ Π(S,bP∗ TeS0∗ ≥ u, Σ
s,
λ
)
n
M
0


cvS,α
≡ inf u ∈ R :=α .
(3.26)

b 1/2 Ne ∗ ∈ Π(S,
b
P∗ Σ
s,
λ
)
n
M
Finally, we set
TbL0 = TbS0 S=L ,

0
0
cvL,α
≡ cvS,α.
S=L(3.27)

0 . Similar toThis alternative “sup-b” selective test rejects the null hypothesis in (2.1) if TbL0 > cvL,αTheorem 1, we have the following result for its asymptotic properties.
Theorem 2. Under Assumptions 1, 2, and 3, the following statements hold for any significancelevel α ∈ (0, 1/2): (a) The sup-b selective test has asymptotic level α under the null hypothesis(2.1), that is, P(Tb0 > cv 0 ) → α; (b) the sup-b selective test is consistent against any local
LL,αalternative that satisfiesmax

1≤j≤|M? |b∗n,j  log(n)1/2 n−1/2 ,(3.28)

0 ) → 1.
that is, P(TbL0 > cvL,α3.2

The case with unknown parameters

So far, we have analyzed the selective test in the baseline setting in which Yt is directly observable.
As the examples in Section 2.1 show, Yt may depend on an unknown parameter θ∗ in some empiricalapplications. In this subsection, we describe how the selective test may be applied in this moregeneral setting. Below, we write Yt (θ) to emphasize the dependence of Yt on a generic parametervalue θ ∈ Θ and, correspondingly, use TbL (θ) and cvL,α (θ) to denote the selective test statistic
and the critical value (recall (2.10) and (2.20)) computed using Yt = Yt (θ). The null hypothesisof interest concerns the conditional moment restriction evaluated at the true value θ∗ , that is,
H0 : g (·) = 0, where g (x) ≡ E [Yt (θ∗ )|Xt = x].
Arguably the most straightforward approach for dealing with the unknown parameter is toconstruct the Anderson–Rubin confidence set by inverting the selective test. Specifically, for eachcandidate parameter value θ ∈ Θ, we implement the selective test for the null hypothesis
H0,θ : E [Yt (θ) |Xt = x] = 0 for all x ∈ X .
The 1 − α level Anderson–Rubin confidence set for the true value θ∗ is then constructed asnoCS1−α ≡ θ ∈ Θ : TbL (θ) ≤ cvL,α (θ) ,17which collects the θ’s such that the selective test does not reject. By the duality between test andconfidence set, Theorem 1 implies that θ∗ ∈ CS1−α with probability approaching 1 − α. We reject
the original null hypothesis (i.e., g (·) = 0) when the confidence set CS1−α is empty.
The Anderson–Rubin approach has a well-known desirable feature that it is robust against the
weak/partial identification of the unknown parameter θ∗ . This issue is particularly relevant for theempirical analysis of macro-style models (see, e.g., Stock and Wright (2000)). Although making
this type of robust inference on θ∗ is not our primary goal, it is a “free” by-product of the proposed
test.14The downside of the Anderson–Rubin approach, however, is that inverting the test for a large
number of candidate values may be computationally expensive. For this reason, we also consider
a more practical “plug-in” approach. Suppose that an estimate for θ∗ , denoted θ̂, is available.
We assume that θ̂ is n1/2 -consistent for θ∗ but do not impose any additional specific structure
on it. This agnostic setup is intentionally designed to accommodate applications in which θ̂ iscalibrated (possibly in other studies based on external datasets that are unavailable), which is quite
typical in macro-style applications (see Chodorow-Reich and Karabarbounis (2016) for interesting
examples). Below, we propose a set of conditions under which the Op (n−1/2 ) estimation error inθ̂ is asymptotically negligible for our testing purpose. Given the lack of information regarding θ̂,this is arguably the only reasonable way to proceed. The resulting plug-in selective test rejects thenull hypothesis when TbL (θ̂) > cvL,α (θ̂).
Assumption 4. (i) The estimator θ̂ satisfies n1/2 (θ̂ − θ∗ ) = Op (1); (ii) Yt (θ) is twice continuouslyP
differentiable in θ with bounded derivatives; (iii) n−1 nt=1 P (Xt )(∂θ Yt (θ∗ ))> − Γ = op (log(n)−1 ),Pwhere Γ ≡ n−1 nt=1 E P (Xt )(∂θ Yt (θ∗ ))> ; (iv) the function hn (x) ≡ E [ ∂θ Yt (θ∗ )| Xt = x] does
not depend on t, and there exist some constant r ≥ 1/2 and a matrix-valued sequence φ∗n suchthat supx∈X kφ∗n PM0 (x) − hn (x)k = O(|M0 |−r ); (v) for some constant C > 0, inf x∈X kPM0 (x)k ≥C|M0 |1/2 and |M0 | ≥ C log (m)3/2 .
Assumption 4(i) states that θ̂ is a n1/2 -consistent estimator for θ∗ , which is satisfied by mostcommonly used estimators.15 Conditions (ii)–(iv) ensure that the statistics of interest depend on
θ in a smooth manner. Condition (v) mainly requires that the size of M0 grows at least at thelog (m)3/2 rate. Note that this condition is not needed in our baseline setting (recall Theorem1). Here, we require M0 to diverge so as to ensure that the post-selection series estimation is
14

Along this line, it might be interesting to extend the selective test to the setting with conditional moment

inequalities (see, e.g., Andrews and Shi (2013), Chernozhukov, Lee, and Rosen (2013), Li, Liao, and Quaedvlieg(2020)). But these extensions are clearly beyond the scope of the current paper, and hence, left for future research.
15
Under the alternative hypothesis, θ∗ is interpreted as the pseudo-true parameter.

18

at least “moderately nonparametric.” By doing so, the statistical noise in the nonparametric test
will dominate the fast-converging estimation error in θ̂, which makes the latter asymptoticallynegligible for the nonparametric inference.
Proposition 1. Under Assumptions 1, 2, 3, and 4, the assertions in Theorem 1 hold for theplug-in selective test that rejects the null hypothesis when TbL (θ̂) > cvL,α (θ̂).

4Monte Carlo simulations

We examine the finite-sample performance of the proposed selective test in a Monte Carlo experiment. Section 4.1 presents the setting and Section 4.2 reports the results.

4.1

The setting

We consider a bivariate conditioning variable Xt = (X1,t , X2,t ) simulated as Xj,t = Zt + vj,t for
j = 1, 2, where Zt is an autoregressive process generated by
Zt = ρZt−1 + (1 − ρ2 )1/2 ηt ,and ηt , v1,t , and v2,t are i.i.d. standard normal random shocks. We set ρ = 0.5 or 0.8 so that Xt
may have different levels of persistence, whereas the variance of Zt is normalized to unity. The
dependent variable Yt is further generated according to Yt = g(Xt ) + t , whereg(x) =

δ exp(x1 + x2 ),1 + exp(x1 + x2 )

t = exp(Zt )∗t ,i.i.d.

∗t ∼ N (0, 1) .

The ∗t shock is independent of the other processes, but the disturbance term t in the nonparametric regression features conditional heteroskedasticity. The δ parameter plays a key role in our
simulation design. When δ = 0, g (·) = 0 identically, so the null hypothesis holds. When δ 6= 0,
we are under the alternative hypothesis and the magnitude of δ quantifies how far the alternative
deviates from the null. Below, we set δ = 0 for the size analysis, and set δ ∈ {0.1, 0.2, . . . , 1} totrace out a test’s power curve. The sample size is set as n = 150, 250, or 500. The number ofMonte Carlo replications is 10,000.
We examine the finite-sample size and power properties of the proposed selective test at significance level α = 5%. To implement the test, we choose the Lasso penalty parameters according
to Algorithm A in Appendix A.1, and then implement the test as described in Section 2.2. The
prior choice set M0 only contains the constant term, which is our recommended default choice.
For comparison, we also consider two other tests. The first is the non-selective test of Belloni,

19

Chernozhukov, Chetverikov, and Kato (2015) and Li and Liao (2020), which rejects the null hypothesis when TbM exceeds the 1 − α quantile of Te∗ given data; recall the definitions in (2.7) andM(2.12). The second is the uncorrected selective test, which rejects the null hypothesis when theselective test statistic TbL exceeds the 1 − α quantile of Te∗ ≡ Te∗ |S=L given data (i.e., it does not
LScorrect for the truncation effect). For simplicity, we refer to the three tests under consideration as
the selective, non-selective, and the uncorrected test, respectively.
We need a collection of basis functions to implement these tests. A natural choice is polynomials. Clearly, “plain” polynomial terms of conditioning variables tend to be highly correlated ina given sample. This may lead to numerical instability in the series estimation (e.g., the n−1 P> P
matrix may not be inverted with enough numerical precision using commonly used software) especially when a large number of series terms are involved. Following prior work (see, e.g., Li,
Liao, and Quaedvlieg (2020)), we mitigate this numerical issue by using the Legendre polynomial.
k
dk
2
,
Recall that the kth-order univariate Legendre polynomial is given by Lk (x) ≡ 2k1k! dx
k x − 1and Lj (·) is orthogonal to Lk (·) under the Lebesgue measure on [−1, 1] for j 6= k.16 To set up theseries basis using Legendre polynomials, we first rescale the X1,t (resp. X2,t ) conditioning variable
e1,t (resp. X
e2,t ). The bivariate series
onto the [−1, 1] interval to obtain a transformed variable X
e1,t )Lk (Xe2,t ) for all j, k ≥ 0.
basis is then formed by collecting Lj (XIt is worth clarifying that our econometric theory does not require the regressors in the series
estimation to be orthogonal. The construction above is not meant to achieve orthogonality among
regressors, either. Instead, we employ this construction only for the purpose of reducing theirempirical correlation so as to improve numerical stability in the practical implementation of series
regression (which would be a non-issue if the researcher had infinite numerical precision). To better
achieve this goal, a rule-of-thumb is to rescale Xj,t in a way such that the empirical distribution
ej,t is “roughly” uniform on [−1, 1], so that we may better exploit theof the transformed variable X
orthogonality property of the Legendre polynomials. Our practical recommendation is to transform
Xj,t onto [0, 1] using its empirical cumulative distribution function, which may be calibrated using
any reasonable parameterization (e.g., the normal distribution), and then rescale it linearly onto[−1, 1].
Finally, in order to examine how the finite-sample performance of the tests depends on the
pre-specified dictionary M, we consider M = {Lj (x1 )Lk (x2 ) : j, k ≥ 0, j + k ≤ p}) for p = 2, 4, 6,
and 8, so that the resulting dictionary contains m = 6, 15, 28, and 45 terms, respectively.
16

The Legendre polynomials can also be computed recursively as L0 (x) = 1, L1 (x) = x, and Lk (x) =2k−1
xLk−1 (x) − k−1
Lk−2 (x) for k ≥ 2.
k
k

20

4.2

Results

We start with discussing the results from the size analysis (i.e., δ = 0). Table 1 presents the finitesample rejection rates of the selective, non-selective, and uncorrected tests at the 5% significancelevel under the null hypothesis. Since the results for the ρ = 0.5 and 0.8 cases are similar, we shallfocus our discussion on the former for brevity.
Panel A of Table 1 shows that the selective test controls size quite well. Specifically, we observethat the test’s null rejection rates are generally very close to the 5% nominal level as long as the
sample size is not too small (i.e., n = 250 or 500), or the dictionary M is not too large (i.e., m = 6
or 15). The only visible size distortion occurs when M contains “many” series terms and thesample size is small (i.e., m = 45 and n = 150). That noted, this is actually a quite challenging
inferential scenario, because the number of candidate regressors is nearly one third of the sample
size. It is perhaps remarkable that the selective test over-rejects only by less than 4% even under
this “stress test.”
The results for the non-selective test, reported on Panel B, show a sharp contrast. First notethat the non-selective test also controls size well for small M with m = 6, which is consistent withthe asymptotic theory of Belloni, Chernozhukov, Chetverikov, and Kato (2015) and Li and Liao(2020). However, as m increases to 15, the non-selective test starts to show nontrivial over-rejection(with 23.4% rejection rate) when n = 150. We clearly see that this is a small-sample phenomenon,
because the size-distortion shrinks quickly as we increase the sample size to n = 500. The overrejection becomes substantially more severe for larger M. Indeed, when m = 45, the non-selective
test almost always (mistakenly) rejects the null hypothesis when the sample size n = 150, and it
rejects more than 60% of the time even when n = 500.
The size distortion of the non-selective test is perhaps not surprising: Since it always employs
all approximating functions in M for the series estimation, the growing-dimensional asymptotics
may not provide an adequate finite-sample approximation when the dimension grows “too fast”
relative to the sample size. From this perspective, we see why the selective test may help mitigate
this issue, in that the Lasso-assisted selection removes most candidate approximating functions(which are all irrelevant under the null hypothesis), and hence, substantially reduces the “effective
dimensionality” of the series inference.
This intuition can be further corroborated by the results shown on Panel C for the uncorrected
test. Since the uncorrected test is based on the same Lasso-assisted selection, it also benefits fromthe aforementioned dimension-reduction effect. Looking at the m = 45 case in Panel C, we indeed
see that the size distortion of the uncorrected test is much smaller than that of the non-selective

21

Table 1: Rejection Rates Under the Null Hypothesis
ρ = 0.5
n = 150n = 250

ρ = 0.8
n = 500

n = 150n = 250n = 500

Panel A: Selective Test
m=6

0.053

0.050

0.045

0.054

0.048

0.048

m = 15

0.059

0.045

0.043

0.057

0.051

0.048

m = 28

0.072

0.055

0.051

0.070

0.055

0.050

m = 45

0.088

0.066

0.053

0.078

0.063

0.053

Panel B: Non-selective Test
m=6

0.061

0.053

0.050

0.062

0.057

0.046

m = 15

0.234

0.143

0.082

0.240

0.144

0.079

m = 28

0.712

0.482

0.268

0.720

0.492

0.267

m = 45

0.958

0.853

0.620

0.959

0.852

0.619

Panel C: Uncorrected (Selective) Test
m=6

0.076

0.067

0.060

0.084

0.073

0.066

m = 15

0.143

0.120

0.109

0.148

0.132

0.114

m = 28

0.202

0.155

0.136

0.177

0.155

0.141

m = 45

0.214

0.177

0.155

0.197

0.169

0.145

Note:

This table reports the rejection rates of the selective test, the non-selective test,

and the uncorrected selective test at the 5% significance level under the null hypothesis(i.e., δ = 0). These results are generated for a variety of specifications under which theautoregressive coefficient ρ ∈ {0.5, 0.8}, the number of candidate basis functions m ∈{6, 15, 28, 45}, and sample size n ∈ {150, 250, 500}. The rejection rates are computed
based on 10,000 Monte Carlo replications.

22

test. Nevertheless, the uncorrected test still over-rejects by a nontrivial amount, and hence, isclearly inferior to the proposed selective test in terms of size control. Recall that the selective andthe uncorrected tests share the same test statistic TbL and they differ only in the construction ofcritical values. This comparison thus directly shows the necessity of accounting for the “truncation
effect” induced by the data-driven selection.
Overall, the size analysis shows that the proposed selective test has excellent size control, evenin adversarial situations with a small sample size and/or a large number of candidate approximating
functions. In contrast, the non-selective and the uncorrected tests are able to control size properly
only when m is relatively small, and may suffer from severe size distortions in general. The selective
test is clearly the most reliable method among the three.
Next, we compare the finite-sample powers of these tests. For brevity, we focus on the settingwith n = 500. Since the non-selective and uncorrected tests generally suffer from nontrivial size
distortions, directly comparing their power with that of the selective test is problematic, as themost size-distorted test may (misleadingly) appear to be the most powerful. We thus instead
focus on the size-adjusted power. Figure 1 plots the size-adjusted power curves for the selective,
non-selective, and uncorrected tests for m = 6 or 45.17
Looking at Figure 1, we first note that the size-adjusted power curves of all three tests hit thenominal size 0.05 at γ = 0 by construction and, as expected, their rejection rates are increasing in
δ; recall that δ quantifies the “distance” between the null and alternative hypotheses. From the
top row of the figure, we see that the proposed selective test and the uncorrected test have similar
power properties when m = 6, and they are more powerful than the benchmark non-selective test.
The latter finding is consistent with the intuition that the Lasso-assisted selection helps the tests
seek power in a targeted fashion.
The case with “many” series terms (i.e., m = 45) displayed on the bottom row of Figure 1
shows a more striking contrast. Indeed, the size-adjusted power of the proposed selective test is
far higher than that of the non-selective test, and the former also outperforms the uncorrected
test by a notable margin. These findings suggest that the non-selective and uncorrected tests not
only suffer from non-trivial size distortions as seen in Table 1, they also deliver worse trade-offs
between size and power than the proposed selective test, as measured by the size-adjusted power.
In summary, the simulation study above shows that the proposed selective test has excellent size
control across a broad range of scenarios, and is notably more powerful than the non-selective test.
These findings clearly demonstrate the usefulness of our proposal relative to that benchmark. We
17

The cases with m = 15 or 28 are bracketed by these two “corner” cases with similar patterns, and so, are omitted

for brevity.

23

Figure 1: Simulation Results: Size-adjusted Power Curves

Note: This figure plots the size-adjusted Monte Carlo rejection rates of the selective test (solid), thenon-selective test (dotted), and the uncorrected selective test (dashed) at the 5% significance level(highlighted by the shaded area) over δ ∈ {0, 0.1, 0.2, . . . , 1}. Results for m = 6 (resp. m = 45) are
reported on the top (resp. bottom) row. Results for ρ = 0.5 (resp. ρ = 0.8) are reported on theleft (resp. right) column. The sample size is fixed at n = 500. The rejection rates are computed
based on 10,000 Monte Carlo replications.

24

also see that the “naive” uncorrected selective test generally has nontrivial size distortion, which
confirms the necessity of adopting our novel critical value. Given these findings, we unambiguously
recommend the selective test for practical applications.18

5Concluding remarks

Conditional moment restrictions may be tested by running a nonparametric series regression.
The guidance from the conventional theory is to search for power broadly by using a relatively
large number of approximating functions in the series estimation. The cost of doing so could
be concerning in practice: If some, even many, regressors are not important for capturing themain features of the conditional expectation function, they may dilute power and, at the sametime, distort size. In view of the vast and burgeoning literature on machine-learning-based feature
selection, it appears rather natural to use this type of methods, such as Lasso, to select series terms
before running the nonparametric test. However, as this paper shows, the data-driven selection
itself may cause size distortion through restricting the score on a random polytope (which in turnaffects the score’s asymptotic normality). This take-home message complements in an interesting
way the “orthogonality-induced negligibility” phenomenon articulated by Belloni, Chernozhukov,
and Hansen (2014) in a distinct semiparametric context. Our proposed critical value is effective incorrecting for this effect. The resulting selective test exhibits improved size and power properties,
which is consistent with the theoretical intuition. In this paper, we have focused on the Lasso
method for feature selection. The underlying strategy may be applied more broadly to the othervariable-selection methods, provided that a tractable characterization of the selection event isavailable. This seems to be an interesting topic for future research.

Appendix

A.1

Implementation Details

This section provides the additional details related to the implementation of the proposed selective
test, which include (i) the exact expressions of bS (s), b0l,S (s), b0u,S (s), AS , and A0S that are needed in
the definition of Π(S, s, λn ); (ii) a feasible algorithm for determining the Lasso penalty parametersin (2.8).
18

The finite sample distributions of the number of series terms selected in the Lasso estimation are provided in

Section SC.1 of the online supplemental appendix.

25

Requisite definitions related to the selection event. We provide the precise definitions of bS (s),b0l,S (s), b0u,S (s), AS , and A0S for a given selection S satisfying M0 ⊆ S ⊆ M and a sign configuration s ∈ {±1}|S\M0 | . These quantities are used to define the polytope Π(S, s, λn ) and its proxyb
Π(S,s, λn ). Let ω S\M and ω M\S denote the subvectors of ω ≡ (ωj )j∈Mc indexed by S \ M0 and00

M \ S, respectively. For ease of notation, we write A+ ≡ (A> A)−1 A> for any matrix A with fullcolumn rank and adopt the convention that any matrix indexed by the empty set is empty. The
quantities of interest are defined as



−1

e>
e

bS (s) ≡ diag (s) (n−1 P
S\M0 PS\M0 ) diag ω S\M0 s,






e+
e > (P b0l,S (s) ≡ −ω M\S − P)> diag ω S\M0 s,

M\S

S\M
0



0
>
e+
e > (P
b
(s)
≡
ω
−
P
)
diag
ω
s,
M\S
S\M
u,S
M\S
0
S\M0







+
−1
>

e>
e

AS ≡ (n−1 P− P>Mc0 (PM0 ) , I|Mc0 | ,S\M0 PS\M0 ) , 0|S\M0 |×|M\S|






 0
+
>
>

e > (Pe+
AS ≡ − P− P>
Mc0 (PM0 ) , I|Mc0 | .
M\S
S\M0 ) , I|M\S|(A.1)

A data-driven choice of Lasso penalty parameters. We propose a feasible choice of the penalty
parameters λn and {ωj }j∈Mc0 that are needed to implement the Lasso estimation in (2.8). We
also show that it satisfies the high-level Assumption 3, and hence, is coherent within our econometric framework. This choice is used in our simulation study, and we recommend it for practicalapplications. The algorithm is given below, followed by its theoretical justification.
Algorithm A (A Recommended Choice of Penalty Parameters)
Step 1. Run a preliminary Lasso estimation with the resulting coefficient given by


n

1 X
X
p
(Yt − P (Xt )> γ)2 + n log(m) log(log(n))|γj | .
γ̂ ≡ argmin
γ∈Rm  2
c
t=1

j∈M0

Step 2. Set the weights in (2.8) as ωj = (|γ̂ j | + n−1/2 )−1 for each j ∈ Mc0 .
Step 3. Let kγ denote the cardinality of {j ∈ Mc0 : γ̂ j 6= 0} and σγ2 denote the sample variance of
1/2

Yt − P (Xt )> γ̂. Set the penalty sequence in (2.8) as λn = σγ max{kγ , 1} log(m) log(log(n)).
Proposition A1. Suppose that Assumptions 1 and 2 hold, n−1



Pn

2
2
t=1 t = σ + op (1) for somepositive constant σ2 , andmin |b∗n,j |  |R| log (n) n−1/2 .
j∈R

(A.2)

Then the penalty parameters λn and (ωj )j∈Mc0 described in Algorithm A satisfy Assumption 3 whenYt = Yt (θ∗ ). The same conclusion obtains for Yt = Yt (θ̂), if Assumption 4 also holds.
26

A.2

A pedagogical illustration for the selection effectIn subsection 2.2, we describe how the data-driven selection affects the subsequent inference. The
mechanism manifests as the restriction on the score vector n−1/2 P>  within the Π(S, s, λn ) polytope, and our proposed critical value adjusts for this truncation effect. That general discussion,
however, is somewhat abstract and notationally involved. To further clarify the main force at play,in this subsection we complement the general discussion with a pedagogical example under which
the polytope has a simple form.
The setting is as follows. Suppose that Xt is scalar-valued, the dictionary M contains only two
terms, the constant term and the linear term Xt , and the prior choice set M0 = {1}. This means,the constant term is always included in the regression and Lasso determines whether the linear
term should be included or not. The specification of the post-selection regression is thus given by Yt = a + bXt + t if L = M,
 Yt = a + t

if L = M0 .

The (full) score vector is
n−1/2 P>  = 


n−1/2
n−1/2

Pn

t=1 tPn

t=1 Xt t


.


(A.3)

The inferential analysis for the test concentrates on the null hypothesis, which under the current
setting corresponds to a = b = 0. Under the null, the collection of Π(S, s, λn ) polytopes has thePfollowing simple structure: with X̄ denoting the sample average n−1 nt=1 Xt , we have(z1 , z2 ) ∈ R2 : z2 > X̄z1 + n−1/2 λn , if S = M, s = +1,





Π(S, s, λn ) =(A.4)(z1 , z2 ) ∈ R2 : z2 < X̄z1 − n−1/2 λn , if S = M, s = −1,





 (z , z ) ∈ R2 : z − X̄z ≤ n−1/2 λif S = M0 .
1 2
2
1
n
Geometrically, the first (resp. second) set corresponds to the half-plane above (resp. below) theline with slope X̄ and intercept n−1/2 λn (resp. −n−1/2 λn ). The third set corresponds to the“stripe” between those two borderlines.
We then readily see how the selection step restricts the score vector. For brevity, we only
discuss the case with S = M and s = +1; that is, the Xt term is selected by Lasso with a positivesign. Plugging (A.3) and (A.4) into (2.13), we may characterize this selection event explicitly as
(
!
)
n
n
X
X
−1/2
−1/2
−1/2
n
Xt t > X̄ n
t + n
λn ,
(A.5)
t=1

t=1

27

Figure 2: Numerical Illustration: Effect of Truncation
Intercept

5

Slope

12

Unrestricted
Restricted

Unrestricted
Restricted

104

DensityDensity

8
3
2

6
4

1

2

0
-0.6

-0.4

-0.2

0

0.2

0.4

0
-0.6

0.6

-0.4

-0.2

a

0

0.2

0.4

0.6

b

Note: This figure plots the distributions of the regression coefficients of Yt = a+bXt +t . The data
is formed as a random sample with sample size n = 100 such that Xt ∼ N (1, 1) and t ∼ N (0, 1).
The null hypothesis is imposed by setting a = b = 0. In the unrestricted case, the least-squares
estimation is done without selection. In the restricted case, the distribution is restricted to eventon which the Xt term is selected by Lasso with a positive sign. The Lasso penalty parameter is
λ = 10 and the ω weight is normalized to 1.
which shows that the score vector n−1/2 P>  = (n−1/2

Pn

−1/2
t=1 t , nPn

t=1 Xt t ) is restricted by an

inequality constraint once the selection is made. Recall from the “textbook” regression theory thatthe asymptotic normality of the least-square estimator for (a, b), denoted (â, b̂), is driven by theasymptotic normality of the score vector n−1/2 P> . But restricting its support via (A.5) clearly
will alter its distribution, which in turn will affect that of the least-square estimates. To visualize
this effect, we plot in Figure 2 the distributions of the least-square estimates of (a, b) with andwithout the inequality restriction (A.5) computed in a numerical experiment. From the figure, we
see that the selection-induced restriction indeed “pushes” the finite-sample distribution of the leastsquares estimates notably away from the benchmark (unrestricted) normal distribution suggestedby the “textbook” theory. It is then intuitively clear that this effect will further contaminate the
distributions of “functional estimator” ĝ (x) = â + b̂x and the associated sup-t statistic, and so,leads to size distortion.
The intuition gained from this pedagogical example alludes to a more general logic. Although
the population coefficients of all regressors are zero under the null hypothesis, Lasso in any given
finite sample may select a few regressors purely due to randomness. Including extra regressors in28

this manner is far from innocuous, because the regressors are selected precisely because they appear(spuriously) important in the Lasso estimation. As such, they also tend to appear important in thesubsequent series estimation, and so, are likely to result in false rejections of the null hypothesis.
Put simply, the post-selection test might be systematically “fooled by randomness” in finite samples
due to this mechanism, whenever the theoretically optimistic “oracle” property is not actually inforce. It is worth clarifying that this concern does not apply to the familiar scenario in which an
empiricist a priori decides to include a few exogeneous control variables in the regression, say, basedon economic reasoning rather than statistical screening. The key lesson here is that the selection
step may make the originally exogeneous variables “effectively endogenous” by truncating the
support of the score. The conditional adjustment underlying our proposed critical value is exactly
designed to account for this truncation effect. This adjustment indeed provides excellent finitesample performance as we have shown in the more comprehensive numerical experiment presented
in Section 4.

References
Andrews, D. W. K. (1991a): “Asymptotic Normality of Series Estimators for Nonparametric and Semiparametric Regression Models,” Econometrica, 59(2), 307–345.
(1991b): “Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation,”
Econometrica, 59(3), 817–858.
(1994): “Chapter 37 Empirical Process Methods in Econometrics,” vol. 4 of Handbook of Econometrics, pp. 2247–2294. Elsevier, Amsterdam, Netherlands.
Andrews, D. W. K., and X. Shi (2013): “Inference Based on Conditional Moment Inequalities,” Econometrica, 81(2), 609–666.
Andrews, I., T. Kitagawa, and A. McCloskey (2021a): “Inference After Estimation of Breaks,”
Journal of Econometrics, 224(1), 39–59.
(2021b): “Inference on Winners,” Discussion paper, Harvard University, UCL, and University ofColorado.
Belloni, A., D. Chen, V. Chernozhukov, and C. Hansen (2012): “Sparse Models and Methods for
Optimal Instruments With an Application to Eminent Domain,” Econometrica, 80(6), 2369–2429.
Belloni, A., and V. Chernozhukov (2011): “High Dimensional Sparse Econometric Models: An Introduction,” in Inverse Problems and High-Dimensional Estimation, pp. 121–156. Springer.
Belloni, A., V. Chernozhukov, D. Chetverikov, and K. Kato (2015): “Some New Asymptotic
Theory for Least Squares Series: Pointwise and Uniform Results,” Journal of Econometrics, 186(2), 345– 366, High Dimensional Problems in Econometrics.

29

Belloni, A., V. Chernozhukov, and C. Hansen (2014): “Inference on Treatment Effects after Selection
among High-Dimensional Controls,” The Review of Economic Studies, 81(2), 608–650.
Bickel, P. J., Y. Ritov, and A. B. Tsybakov (2009): “Simultaneous Analysis of Lasso and Dantzig
Selector,” The Annals of Statistics, 37(4), 1705 – 1732.
Botev, Z. I. (2016): “The Normal Law Under Linear Restrictions: Simulation and Estimation via Minimax
Tilting,” arXiv preprint arXiv:1603.04166.
Brown, B. W., and S. Maital (1981): “What do Economists Know? An Empirical Study of Experts’
Expectations,” Econometrica, 49(2), 491–504.
Chen, X. (2007): “Large Sample Sieve Estimation of Semi-Nonparametric Models,” in Handbook of Econometrics, ed. by J. Heckman, and E. Leamer, vol. 6B, chap. 76. Elsevier, 1 edn.
Chen, X., and T. M. Christensen (2015): “Optimal Uniform Convergence Rates and Asymptotic Normality for Series Estimators Under Weak Dependence and Weak Conditions,” Journal of Econometrics,
188(2), 447–465.
Chernozhukov, V., S. Lee, and A. M. Rosen (2013): “Intersection Bounds: Estimation and Inference,”
Econometrica, 81(2), 667–737.
Chodorow-Reich, G., and L. Karabarbounis (2016): “The Cyclicality of the Opportunity Cost of
Employment,” Journal of Political Economy, 124(6), 1563–1618.
Cox, G., and X. Shi (2019): “Simple Adaptive Size-Exact Testing for Full-Vector and Subvector Inferencein Moment Inequality Models,” arXiv preprint arXiv:1907.06317.
Hansen, L. P., and R. J. Hodrick (1980): “Forward Exchange Rates as Optimal Predictors of Future
Spot Rates: An Econometric Analysis,” Journal of Political Economy, 88(5), 829–853.
Hansen, L. P., and K. J. Singleton (1982): “Generalized Instrumental Variables Estimation of Nonlinear Rational Expectations Models,” Econometrica, 50(5), 1269–1286.
Lee, J. D., D. L. Sun, Y. Sun, and J. E. Taylor (2016): “Exact Post-Selection Inference, With
Application to the Lasso,” The Annals of Statistics, 44(3), 907–927.
Li, J., and Z. Liao (2020): “Uniform Nonparametric Inference for Time Series,” Journal of Econometrics,
219(1), 28–51.
Li, J., Z. Liao, and R. Quaedvlieg (2020): “Conditional Superior Predictive Ability,” Review of Economic Studies, forthcoming.
Liao, Z., and X. Shi (2020): “A Nondegenerate Vuong Test and Post Selection Confidence Intervals for
Semi/nonparametric Models,” Quantitative Economics, 11(3), 983–1017.
Newey, W. K. (1997): “Convergence Rates and Asymptotic Normality for Series Estimators,” Journal of
Econometrics, 79(1), 147 – 168.

30

Newey, W. K., and D. McFadden (1994): “Large Sample Estimation and Hypothesis Testing,” Handbook of Econometrics, IV, Edited by R. F. Engle and D. L. McFadden, pp. 2112–2245.
Pollard, D. (2001): A User’s Guide to Measure Theoretic Probability. Cambridge University Press.
Romer, C. D., and D. H. Romer (2000): “Federal Reserve Information and the Behavior of Interest
Rates,” American Economic Review, 90(3), 429–457.
Stock, J. H., and J. H. Wright (2000): “GMM with Weak Identification,” Econometrica, 68(5), 1055–
1096.
Tibshirani, R. (1996): “Regression Shrinkage and Selection via the Lasso,” Journal of the Royal StatisticalSociety: Series B (Methodological), 58(1), 267–288.
van der Vaart, A., and J. Wellner (1996): Weak Convergence and Empirical Processes. SpringerVerlag.
Yuan, M., and Y. Lin (2006): “Model Selection and Estimation in Regression With Grouped Variables,”Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68(1), 49–67.
Zhao, P., and B. Yu (2006): “On Model Selection Consistency of Lasso,” Journal of Machine Learning
Research, 7, 2541–2563.
Zou, H. (2006): “The Adaptive Lasso and Its Oracle Properties,” Journal of the American Statistical
Association, 101(476), 1418–1429.
Zou, H., and T. Hastie (2005): “Regularization and Variable Selection via the Elastic Net,” Journal of
the Royal Statistical Society: Series B (Statistical Methodology), 67(2), 301–320.
Zou, H., and H. H. Zhang (2009): “On the Adaptive Elastic-Net With a Diverging Number of Parameters,” Annals of statistics, 37(4), 1733.

31