Nonparametric estimation of infinite order regression
and its application to the risk-return tradeoff ∗Seok Young Hong†

Oliver Linton‡

University of Nottingham

University of Cambridge

November 2019; Forthcoming in Journal of Econometrics
Abstract
This paper studies nonparametric estimation of the infinite order regression
E(Ytk |Ft−1 ), k ∈ Z with stationary and weakly dependent data. We propose a
Nadaraya-Watson type estimator that operates with an infinite number of conditioning variables. We propose a bandwidth sequence that shrinks the effectsof long lags, so the influence of all conditioning information is modelled in anatural and flexible way, and the issues of omitted information bias and specification error are effectively handled. We establish the asymptotic properties ofthe estimator under a wide range of static and dynamic regressions frameworks,
thereby allowing various kinds of conditioning variables to be used. We establish pointwise/uniform consistency and CLTs. We show that the convergence
rates are at best logarithmic, and depend on the smoothness of the regression,
the distribution of the marginal regressors and their dependence structure ina non-trivial way via the Lambert W function. We apply our methodology to
examine the intertemporal risk-return relation for the aggregate stock market,
and some new empirical evidence is reported. For the S&P 500 daily data from
1950-2017 using our estimator we report an overall positive risk-return relation.
We also find evidence of strong time variation and counter-cyclical behaviour
in risk aversion. These conclusions are possibly attributable to the allowanceof further flexibility and the inclusion of otherwise neglected information in our
method.
JEL Codes: C10; C58; G10.
∗

We gratefully acknowledge helpful comments from John Aston, Xiaohong Chen, Jeroen Dalderop,Paul Doukhan, Jiti Gao, Shuyi Ge, Hayden Lee, Yuan Liao, Mikhail Lifshits, Richard Nickl, Alexei
Onatski, Hashem Pesaran, Peter Phillips, Alessio Sancetta, Philippe Vieu, the Associate Editor andtwo anonymous referees. We also thank Alexey Rudenko for providing an original Russian photocopy
of Sytaya (1974), Hyungjin Lee for translating the paper, and the ERC for providing financial support.
†
Nottingham University Business School, University of Nottingham; sy.hong@nottingham.ac.uk‡
Faculty of Economics, University of Cambridge; obl20@cam.ac.uk

1

1

Introduction

Conditional expectations are crucially important in financial economics, with implications in many applications including asset returns predictability, market efficiency
and risk management. One fundamental objective is to understand the risk/return
trade-off summarized by the relationship between the expected excess return relative
to the conditional variance of returns. Due to the latency of conditional expectations however, there has been no universal agreement upon what is the best way tomeasure these objects. Differences in the approaches to modelling and estimating theconditional mean and variance has led to disagreement on their measurement, andalso, conflicting empirical evidence on their intertemporal relation. Theoretical asset
pricing models do not generally restrict the shape of the risk premium or the dynamics
of the risk return trade-off. For example, Backus and Gregory (1993) show that theshape of the relation between the risk premium and the conditional variance of returns
is largely unrestricted with increasing, decreasing, flat, or nonmonotonic patterns all
possible. Similar conclusions are drawn by studies such as Abel (1988), Gennotte
and Marsh (1993), and Veronesi (2000). Nevertheless, most empirical studies adopt a
simple linear specification.
There are some issues with the usual approaches. First, there is the risk of misspecification. For instance, some studies have relied on parametric or semiparametricassumptions such as the ARCH or stochastic volatility models, where some high degreeof structure is imposed on the return generating process. Other studies have typicallymeasured the conditional mean and conditional variance as projections onto somepredetermined variables. These approaches cannot be entirely justified, since they are
all necessarily prone to some degree of potential specification error, see Linton andPerron (2003) and Escanciano, Pardo-Fernández and Van Keilegom (2017) for further
discussions. Nonparametric modelling can be an effective solution in this context. It isa well established practical tool for analyzing time series data; see for example Härdle(1990), Bosq (1996), or Fan and Yao (2003) for a comprehensive review. A majoradvantage of this approach is that the relationship between the explanatory variables|

under study, denoted by X = (X1 , . . . , Xd ) , and the response, say Y , can be modelled
without assuming any restrictive parametric or linear structures. Stone (1980, 1982)
showed that the best achievable convergence rate (in minimax sense) is n−β/(2β+d) ,where β is a measure of smoothness and d is the dimension of the covariates.
Secondly, there may be potential bias due to the omission of necessary information.
Choosing among a few conditioning variables introduces an element of arbitrariness
into the econometric modelling of expectations. In particular, if information that2

investors consider important is neglected, then the corresponding estimates may beunreliable, Harvey (2001). Lettau and Ludvigson (2010) argued that contrasting conclusions on the intertemporal risk-return relation are largely due to the prevalent useof only small amount of conditional information in modelling the conditional mean
and variance. Indeed, such practice greatly restricts the dynamics for the variance
process and may result in poor estimates, especially when the volatility is highly
persistent, Linton and Perron (2003), Giraitis et al. (2008). For example, Pagan
and Hong (1990) estimated the conditional moments with nonparametric estimatesof E(rmt − rf t |rm,t−1 , . . . , rm,t−p ) and var(rmt − rf t |rm,t−1 , . . . , rm,t−p ), where rmt − rf t
denotes the excess market return and p = 1 or 4. Having ended up with a negativerisk-return relation using their estimates, they conjectured that the conclusion may
have been affected by their use of only a small, finite number of conditioning variables.
Noting the dependence of a GARCH process on the infinite past history of returns(with declining weights), they wrote: “[A] nonparametric estimator of σt2 appeals as asolution ..., although the fact that it operates with only a finite number of conditioning
elements makes it unable to explicitly handle a GARCH type process. ... [O]ne might
be able to establish consistency of the estimator [which deals with the infinite dependence]. As far as we are aware, however, there are no current theorems that would
justify such a conjecture.”

1.1

Overview of Results

This paper defines an estimation method that effectively addresses the aforementioned
difficulties. We propose a Nadaraya-Watson type estimator that operates with anunrestricted number of conditioning variables. We derive large sample properties of theestimator in extensive detail, thereby providing an answer to the longstanding questionin the quotation above. With a bandwidth sequence that shrinks the effects of longlags, the influence of all conditioning information is modelled in a natural and flexible
way, and both issues of omitted information bias and specification error are effectively
handled. It is worth noting that Harvey (2001) reported sensitivity of conditional
expectations estimates on what type of conditioning variables are used in modellingthe expectations. He showed with examples how several parametric/nonparametricestimates (and the estimated risk-return relationship) may vary according to the choiceof different predetermined conditioning information. In this paper, we allow for various
kinds of conditioning information. This is achieved by letting our model assumptions
cover a wide range of static and dynamic regressions frameworks. The latter includes
the autoregression framework as a special case.
3

Linton and Sancetta (2009) tackled this estimation problem of infinite order regression in the autoregression context. They established uniform almost sure consistency
for stationary ergodic data but without rates. In the conclusion, they conjectured
that the limiting distribution of nonparametric estimators could be established, andthat the rate of convergence would be logarithmic. Under strict cross-sectional andtemporal i.i.d. assumption, Mas (2012) derived a convergence rate that is consistent
with our results in the particular case they considered.
We make several contributions. First, we establish some theorems which answer
several open questions posed in the literature. Specifically, we show the pointwise
consistency of our estimator under a set of mild regularity conditions. Further, weestablish a central limit theorem for our estimator at a point under stronger conditions
as well as for a feasibly studentized version of the estimator, thereby allowing pointwise
inference to be conducted. Also, uniform consistency of the estimator is shown over acompact set of logarithmically increasing dimension. We prove that convergence rates
depend on the smoothness of the regression function, the distribution of the marginal
regressors and their dependence structure in a non-trivial way via the Lambert W
function. We elaborate how each of those factors affects the rate of convergence,and show that the best possible rate is, nonetheless, of logarithmic order in all cases
regardless of the smoothness of the regression function. This reflects the difficulty ofcapturing nonparametrically the effect of an infinite number of lags.
Second, using our estimation method we find some new empirical results. We
reveal new evidence on the dynamics of risk-return relation and its link with themacroeconomy, and add supporting evidence for explaining some major puzzles infinancial economics. To elaborate, applying our methods on the US stock market wefind a positive risk-return relationship over the past 60 years overall − which is what
asset pricing models generally postulate, e.g. Merton (1973). In particular, the relation
turns out to be highly positive and strongly statistically significant in the recent 30
years period. Moreover, we also found that there has been a strong time variationand counter-cyclicity in risk aversion and in the conditional Sharpe ratio. The timeseries of estimated risk aversion tends to move in the opposite direction to the Federal
Funds rate, a proxy for the business cycle, with the sample correlation being −0.5673.
The quarterly Sharpe ratio is also strongly counter-cyclical, rising over most periods ofrecessions. By contrast, when a standard nonparametric method is employed instead,
we noticed that these findings are not revealed, and different conclusions are reached.
We believe our new empirical findings suggest an improvement in the econometricanalysis that is attributable to allowing for extended flexibility and the inclusion of4otherwise neglected information in our method.

1.2

Technical challenges and sketch proposals for remedy

One major hurdle we face in the infinite-dimensional setting is the non-existence of theusual notion of density p(·) for the regressor X. Since there is no σ-finite Lebesgue
measure in infinite-dimensional spaces, the Lebesgue density (with respect to theinfinite product of probability measures) of the regressor cannot be defined via theRadon-Nikodym theorem. Consequently, standard asymptotic arguments for kernel
estimators are no longer valid, for example: Bochner’s lemma whereby under suitable
regularity conditions, for j = 1, 2
 Z
 
1
x−X
j
= Kj (u)p(x − uh) du
E K
hdh→ p(x)kKkjj

as h → 0(1)

where K is a multivariate kernel (see Section 2.2 below). So classical limiting theories
cannot be readily extended to our setting.
We propose to adopt and apply some ideas from the functional regression literature. There is a vast statistical literature on functional data (typical examples include
curves and images, which are infinite-dimensional in nature). Ferraty and Vieu (2002)
first studied the case where the regressor was function-valued. Masry (2005) provided
a rigorous treatment of nonparametric regression with dependent functional data inwhich X lies in a general semi-metric space, establishing the central limit theorem.
Mas (2012) derived the minimax rate of convergence for nonparametric estimation of
the regression function with strictly independent and identically distributed covariates. Ferraty and Vieu (2006) detailed a number of extensions and gave an overview
of nonparametric approaches in the functional statistics literature. Geenens (2011)
gave an up-to-date accessible summary of the literature on nonparametric functional
regression, and introduced the term curse of infinite dimensionality, which reflects theevident difficulties in nonparametric estimation of infinite-dimensional objects due toextreme data sparsity. In the finite dimensional case more smoothness can mitigate
completely the slower rate of convergence caused by dimensionality, but in the infinite
dimensional case, additional smoothness can only mildly improve the convergence rateof estimators. We discuss in the next section the difference between the functional
data framework and our discrete time framework.
There is another potential problem that may arise specifically in the infinite dimen-

5

sional setting. In the dynamic regression framework, the regressor vector Xt includes
the infinite lags of a variable Zt , say the response variable. Consequently, the classof mixing type assumptions, a popular notion of dependence in the econometrics literature, is generally not applicable. This is because measurable functions of Xt will
depend upon the infinite time-lags of Zt , and are not mixing in general, see e.g. Davidson (1994). Therefore, in order to establish asymptotic theories, an alternative set of
dependence assumptions should be imposed on the data generating process. We defer
further discussions to Section 2.1 below.
Lastly, for notations, we define an ' bn by an = bn + o(1), and cn ∼ dn byequivalence of order between the two sequences cn and dn . Also, f  g means there
exists some constant c > 0 such that limn→∞ f (n)/g(n) ≤ c. The term ‘stationarity’is taken to mean strict stationarity. Throughout, C (or C 0 , C 00 ) refers to some genericconstant that may take different values in different places unless defined otherwise.

2

Some Preliminaries

Consider the regression modelY = m(X) + ε,(2)|where the regressor X = (X1 , X2 , . . . .) is a random element taking values in somesequence space S, the response Y is a real-valued variable, and the stochastic error εis such that E(ε|X) = 0 a.s. The objective is to estimate the Borel functionm(·) = E(Y |X = ·)(3)

based on n random samples observed from a strictly stationary data generating process {(Yt , Xt ) ∈ R × S}t∈Z having some weak dependence structure. Details on theassumptions are given in Section 2.1 below.
This setting is related to the usual framework adopted for functional data, which
has been widely studied by statisticians, see Ramsey and Silverman (2002), Aneiros,
Bongiorno, Cao and Vieu (2017). Recently, successful attempts have been made to
develop theories for nonparametric inference in the functional statistics literature;
Ferraty and Romain (2010) gives a comprehensive review. A major issue in this fieldof research lies in extending the statistical theories applicable to Rd to function spaces.
In this literature, attention is usually on smooth functions that are approximated andreconstructed from finely discretised grids on some compact interval. In contrast, thesetup in our model (2) can be viewed as looking at a countable number of discrete
6

observations. Such a difference is reflected by the fact that the observed data is taken
to be a discrete process X = (Xs ) with unbounded s ∈ Z+ so that S = {f |f :N → R}, rather than X = (X(s)) with s ∈ [0, T ]k so that S = {f |f : [0, T ]k ⊂Rk → R}, e.g. curves if k = 1, images if k ≥ 2. The discrete nature of our setting
has several fundamental distinctive features that allow us to look further into many
specific practical issues.
An immediate consequence of our framework is that the tuning parameter can be
imposed on each and every dimension, allowing one to control the marginal influenceof the regressors. For instance when it is sensible to postulate that the influence ofdistant covariates is getting monotonically downweighted, one may set the marginal
bandwidths to increase in the lag horizon so as to impose higher amount of smoothingat distant lags. Depending on the nature of the regressor, S may be taken as the spaceQof all infinite real sequences R∞ := ∞j=1 Rj formed by taking Cartesian products of
the reals, or its various linear subspaces such as `∞ , `p , c. We propose to take S = R∞
so as to refrain from imposing any prior restrictions with regard to the choice of theregressor; for example, taking S to be the space of bounded sequences excludes thepossibility of having regressors with infinite support (e.g. Gaussian process).

2.1

Dependence structure and leading examples

A distinctive characteristic of time series data is temporal dependence between observations. In the nonparametric time series literature, Rosenblatt (1956)’s α-mixing
has been the de facto standard choice due to it being the weakest among the class of
mixing-type asymptotic independence conditions. Roussas (1990) established pointwise and uniform consistency of the local constant estimator under this condition,
respectively, while Fan and Masry (1992) established asymptotic normality. The αmixing condition has also been widely used in the context of dependent functional
observations, see for instance Ferraty et al. (2010), Masry (2005), and Delsol (2009).
Definition 1. A stochastic process {Zt }∞t=1 defined on some probability space (Ω, F, P )is called α-mixing (NB. ‘jointly’ α-mixing if Zt is Rd -valued, with d ∈ (1, ∞]), ifα(r) :=sup|P (A ∩ B) − P (A)P (B)|

∞
t
A∈F−∞
,B∈Fr+t

is asymptotically zero as r → ∞, where Fab is the σ-algebra generated by {Zs ; a ≤s ≤ b}. In particular, we say the process is algebraically (respectively exponentially)
α-mixing with rate k if there exists some c, k > 0 such that α(r) ≤ cr−k (respectively
7

if there exists some γ, ς > 0 such that α(r) ≤ exp(−ςrγ )).
The popularity of the α-mixing condition (note the modifier α- will occasionally beomitted if no confusion is likely) in the literature stems from the fact that it is easy towork with, see e.g. Doukhan (1994), Rio (2000) for a comprehensive survey. However,there are several limitations that have been pointed out in the literature. First, it is arather strong technical condition that is hard to verify in practice. Second, some basic
processes are not mixing. e.g. AR(1) with Bernoulli innovations, Andrews (1984).
We turn to our setting. In the static regression case it is appropriate to assume themixing condition, but in the dynamic case this condition is not generally applicable as
we now explain. Recall that the object of estimation is the conditional mean E(Yt |F),cf. (2), where the information set F is determined by the nature of the conditioning
variables. There are two leading cases: the first case is the static regression where
the information set is taken to mean σ(Xjt ; j = 1, 2, . . .), the σ-algebra generated
by the exogenous marginal regressors. The second case is the autoregression, where
Xtj = Yt−j for all j, in which case F = Ft−1 represents σ(Ys ; s ≤ t − 1), the σ-algebra
generated by the sequence of lags of the response (Ys )s≤t−1 . In fact, as for the latter
framework we may consider a more general setup, i.e. a dynamic regression, where
the information set is taken to be F = σ(Xjs , Ys ; s ≤ t − 1) for some j. Details areformally given in Assumptions A below.
In the static regression case the usual joint α-mixing condition can be assumed on
the sample data {Yt , Xt } as is usually done; since marginal regressors are observed at|

the same time t: Xt = (X1t , X2t , . . .) , assuming joint dependence does not require
additional adjustments. Indeed, it can be easily shown that joint mixing implies bothmarginal component processes and any measurable function thereof are mixing.1 In
this paper, we do not necessarily require independence between component processes{Xjt }, j = 1, 2, . . .; later we specify to what extent some dependence can be allowed(see Assumption C). It will turn out that the requirement is mild and allows sufficient
generality in application.
Moving on to the dynamic regression setting, since the regressors are taken to be
the lags of the response and/or a covariate, measurable functions of Xt depend on
infinite time-lags and hence are not necessarily mixing.2 Therefore an alternative setof dependence conditions is necessary to establish asymptotic theories for the second
1

The converse is not necessarily true unless the marginal processes are independent to each other,see Bradley (2005, Section 5).
2
Except for some very special cases; Davidson (1994, Theorem 14.9) gives a set of technicalconditions under which a process with infinite (linear) temporal dependence is α-mixing.

8

framework. We adopt the notion of near epoch dependence due to Ibragimov (1962)
for the dynamic regression setting and deal with two leading cases separately.
Definition 2. A stochastic process {Zt }∞t=1 defined on some probability space (Ω, F, P )is called near-epoch dependent or stable in L2 with respect to a strictly stationary αmixing process {ηt } if the stability coefficients v2 (r) := E|Zt −Zt,(r) |2 is asymptoticallyzero as r → ∞, where Zt,(r) = Ψr (ηt , . . . , ηt−r+1 ) for some Borel function Ψr : Rr → R.
A process that is near epoch dependent on a mixing sequence is influenced primarily by the “recent past” of the sequence and hence asymptotically resembles its
dependence structure; see e.g. Billingsley (1968), Davidson (1994), or Lu (2001) for
details. Andrews (1995) established uniform consistency of kernel regression estimators under near epoch dependence conditions. Following the usual convention, e.g.
Bierens (1983), we shall take Ψr (ηt , . . . , ηt−r+1 ) ≡ E(Zt |ηt , . . . , ηt−r+1 ). In Section 2.3
it will be shown that under suitable conditions similar asymptotic theories can bederived for both static and dynamic regression frameworks.

2.2

Local Weighting

In this section we fix the notions of local weighting and the measure of closenessbetween the data objects. Let K : [0, ∞) → [0, ∞) =: R+ be a univariate density
function and for an element u of a normed sequence space, letK(u) := K(kuk).

(4)

In our setting the properties of K are crucially important. We now group the kernel
functions into three subcategories depending on how they are generated. The firsttwo, referred to as Type-I and Type-II kernels in Ferraty and Vieu (2006) generalize
the usual ‘window’ kernels and monotonically decreasing kernels in finite dimension,respectively. Both types of kernels are continuous on a compact support [0, λ].
Definition 3. A function K : [0, ∞) → [0, ∞) is called a kernel of type−I if it
integrates to 1, and if there exist real constants C1 , C2 (with 0 < C1 < C2 ) for which
C1 1[0,λ] (u) ≤ K(u) ≤ C2 1[0,λ] (u),(5)

where λ is some fixed positive real number. A function K : [0, ∞) → [0, ∞) is called9

a kernel of type−II if it satisfies (5) with C1 ≡ 0, and is continuous on [0, λ] anddifferentiable on (0, λ) with the derivative K 0 that satisfies
C3 ≤ K 0 (u) ≤ C4
for some real constants C3 , C4 such that −∞ < C3 < C4 < 0.
The definition above suggests that the uniform kernel on [0, λ] is a type-I kernel,
and the Epanechnikov, Biweight and Bartlett kernels belong to the class of Type-II
kernels. Some of those with semi-infinite support, for example (one-sided) Gaussian,are covered by the last group, which we will call the Type-III kernels.
Definition 4. A function K : [0, ∞) → [0, ∞) is a kernel of type−III if it integrates
to 1, and if it is of exponential type; that is, K(r) ∝ exp(Crβ ) for some β and C.

2.3

Small deviations

The small ball (or small deviation) probability plays a crucial role in establishing theasymptotic theory. Let S ∗ be a sequence space equipped with some norm k.k; thenthe small ball probability of an S ∗ -valued random element Z is a function defined asϕz (h) := P (kz − Zk ≤ h) ,(6)

where h ∈ R+ . The probability is called centered if z = 0 (in which case we writeϕ(h)) and shifted (with respect to some fixed point z ∈ S ∗ \{0}) if otherwise. The
relation between the two quantities cannot be explicitly specified in general, and will
be given in terms of a Radon-Nikodym derivative (See Assumption D2 below).
The name small ball stems from the fact that we are interested in the asymptotic
behaviour of ϕz (h) as h tends to zero. The function can be thought of as a measurefor how much the observations are densely packed or concentrated around the fixed
point z with respect to the associated norm and the reference distance h. From thedefinition it is straightforward to see that ϕz (h) → 0 as h → 0, and that nϕz (h)is an approximate count of the number of observations whose influence is taken into
account in the smoothing procedure. When Z is a continuous random vector of fixed
dimension d with density p(·) > 0, it can be readily shown that the shifted small ballprobability (with respect to the usual Euclidean norm) is given byϕz (h) = Vd hd p(z) = O(hd ),10(7)where Vd = π d/2 /Γ(d/2 + 1) is the volume of the d-dimensional unit sphere.
However, when Z takes values in an infinite-dimensional normed space, it is difficult to specify the exact form of the small ball probability, and its behaviour varies
depending heavily on the nature of the associated space and its topological structure.
Due to the non-equivalence of norms in infinite dimensional spaces, it is intuitively
clear that the “speed” at which ϕz (h) converges to zero is affected by the choice of thenorm k.k. Nonetheless, a rapid decay is expected in general irrespective of the choiceof the norm due to the extreme sparsity of data in infinite-dimensional spaces.
One possible example of S ∗ is (`r , k.kr ), the space of r-th power summable sequenceequipped with the `r -norm; the centred small ball behaviour of sums of weighted i.i.d.
random variables is widely studied in the literature, see for example Borovkov andRuzankin (2008) and references therein. In this work here, we will focus our main
attention on the case of r = 2 (and take k.k to mean k.k2 unless specified otherwise).
Nevertheless, we note that the results derived in this paper can be extended to thecase of r > 2 as long as the regularity conditions are adjusted appropriately.
Writing the expected value of the kernel in terms of the small ball probability


z−Z
EK
h





kz − Zk
= EK
h



Z
=

Z
K(u) dPkz−Zk/h (u) =K(u) dϕz (uh),(8)

we are able to bypass the difficulties mentioned in the introduction, and to establish theconvergence of the integrals without requiring the existence of the Lebesgue density.
Lemma 1 Ferraty and Vieu (2006, Lemma 4.3 & 4.4). Suppose k.k is some seminorm defined on a function space. If K is type-I, then it satisfies
C1j ≤1
ϕz (hλ)Z λK j (v) dϕz (vh) ≤ C2j , j = 1, 2

(9)0

where C1 , C2 > 0 are as defined in Definition 3. When the kernel K is type-II, if
Z ε∃ ε0 > 0, C5 > 0 s.t. ∀ε < ε0 ,ϕx (u)du > C5 εϕx (ε)

(10)

0

then we haveC6j ≤1
ϕz (hλ)Z λK j (v) dϕz (vh) ≤ C7j , j = 1, 2

(11)

0

where the constants C6 = −C5 C4 and C7 = sups∈[0,λ] K(s) are strictly positive.
Under the regularity conditions of Lemma 1, (9) and (11) hold for every h > 0, so
it follows that for any kernels of type-I and II:
11

Corollary 1 If the kernel K is either type-I or type-II, then for j = 1, 2 we have
 

z−Z
1
j
E K
−→ ξj
ϕz (hλ)has

h → 0+ ,(12)

where ξ1 and ξ2 are some strictly positive real constants.

This result can be seen as an infinite-dimensional analogue of Bochner’s lemma(1): i.e., for Z ∈ Rd , h−d EK((z − Z)/h) → p(z) > 0. It is obvious that ξj is bounded
below and above by C1j and C2j , respectively (or C6j and C7j , depending on the choice of
the kernel). With specific choices of kernels and regressors we may be able to specify
the exact values of the constants in some certain cases. For example, it is straightforward to see that ξ1 = 1/λ and ξ2 = 1/λ2 when K is uniform kernel supported on [0, λ].
Remarks. (i) Lemma 1 reveals the importance of condition (10) in constructing theasymptotics when the kernel is of type-II. Whereas the condition is widely assumed
in the functional statistics literature for that reason, Azais and Fort (2013) provedthat it necessarily restricts the variable Z to be of finite dimension. In other words,
whenever (10) is valid, the topology that governs the concentration properties of Z
accounts effectively only for finite dimension. An example includes the case where Z
is associated with the semi-norm kyk := (y1 , . . . , yp , 0, 0, . . .) for some fixed positive
integer p < ∞ and y ∈ R∞ , Ferrraty and Vieu (2006, Section 13.3.3). Since thisseverely restricts the applicability of our work, we shall not consider the case of TypeII kernels.
(ii) A natural question one may then ask is whether (12) would hold for kernels withsemi-infinite support such as the Type-III kernels. In the finite Rd -framework, it is wellknown that a set of assumptions including kukd K(u) → 0 as u → ∞ is sufficient for
showing (1), see for instance Parzen (1962, Theorem 1A) and Pagan and Ullah (1999,
Lemma 1). However, in the infinite-dimensional setting the answer is negative in mostusual cases where the kernel is of exponential type (e.g. Gaussian kernel). Whereas
the lower bound of the limit can be easily constructed via Chebyshev’s inequality:with reference to Definition 4, writing V = kz − Zkβ , δ = hβ and letting cδ be somefunction of δ we have(0 <) exp(−cδ δ) ≤ [P (V ≤ δ)]−1 E exp(−cδ V ).

(13)

So the upper bound may not exist, and the rate at which the small ball probability
12decays to zero may dominate the speed at which the integral (8) converges to zero.
This claim cannot be formally verified for all general cases because (as aforementioned)
there is no unified result for the asymptotic behaviour of small deviations available.
Nevertheless, the idea can be sketched in the common case where the asymptotics ofthe distribution function (i.e. small deviation) is of exponential order: P (V ≤ δ) ∼exp(−Cδ −θ ) as δ → 0 for some constants C and θ > 0. By de Bruijn’s exponential
Tauberian theorem (see Bingham et al. (1987), Li (2012)), a necessary and sufficientcondition for such a case is the following limiting behaviour of the Laplace transformnear infinity:E[exp(−cδ V )] ∼ exp − C0θ/(1+θ)· cδ

as cδ → ∞for some constant C 0 > 0. With V = kz−Zk2 , δ = h2 , cδ = 2−1 h−2 (which corresponds
to the case of the Gaussian kernel) the difference in the order of convergence suggeststhat the right hand side of (13) is unbounded, and that the limit (12) diverges.
Due to the reasons above we shall confine our attention to Type-I kernels only here
in this work.

2.4

Bandwidth Matrix and covariates

We aim to estimate the regression operator at a point x ∈ R∞ with an R∞ -dimensional|

regressor X = (X1 , X2 , . . .) . Let H := diag(h) = diag(h1 , h2 , . . .) ∈ R∞×∞ be thebandwidth matrix. We require that a norm k.k can be admitted to the weighted
regressor values and the weighted point, and for this the bandwidth sequence must be
chosen appropriately. In particular, we let
H = hD = h × diag(φ1 , φ2 , . . .),(14)

where D ∈ R∞×∞ and h ∈ R. By Kolmogorov’s three-series theorem, the sequence of
weighted regressors {φ−1j Xj } is square summable, with probability one, provided thatthe marginal regressors Xj0 are independent with finite variance and satisfy
∞
X


2
< ∞,
E min 1, φ−2
j Xj

(15)

j=0
|

−1
so that (φ−1
1 X1 , φ2 X2 , . . .) =: Z is (`2 , k.k2 )-valued. In the autoregressive framework,

φj can be interpreted as a weight sequence that represents the “relative influence” ofthe marginal regressors, which diminishes as lags get further apart.
13

For this purpose we assume from now on that the bandwidth-weighted X and x (i.e.
|−13
Z and z := (φ−11 x1 , φ2 x2 , . . .) , respectively) are `2 -valued and normed with k.k =k.k2 . Consequently, (with an abuse of notation) we can extend the usual definitionof shifted small deviation to account for the generalized support [0, λ] and bandwidth|

vector h = (h1 , h2 , . . .) :ϕx (hλ) := P kH −1 (x − Xt )k ≤ λ= P kD−1 (x − Xt )k ≤ hλ .

(16)Equivalently, ϕx (hλ) = P (Xt ∈ E(x, hλ)), where E is the infinite-dimensional hyperellipsoid centred at x ∈ R∞ , and λ is as defined in Section 2.2. Clearly, ϕx (hλ) = ϕz (hλ).
For later reference, we also define the joint small ball probability of the regressor vectors observed at different times t and s as the joint distributionψx (hλ; t, s) := P (Xt , Xs ) ∈ E(x, λh × E(x, λh) .

3

(17)

The Estimator

We observe a sample {Yt , Xt }nt=1 with Yt ∈ R and Xt ∈ R∞ . With these data, we
propose to estimate m(x) = E(Y |X = x), x ∈ R∞ with the following local constant
type estimator:




Pn
−1
−1
K
kH
(x
−
X
)k
Yt
K
H
(x
−
X
)
Y
t
t
t
t=1
t=1

 ≡ P

 .
m(x)
b
:= P
n
n
−1 (x − X )−1 (x − X )ktt
t=1 K H
t=1 K kHPn

(18)

In practice, in the autoregression case we essentially observe only {Y1 , Y2 , . . . , Yn }rather than the full infinity, so further lags can be regarded as zeros. Similarly,
in the static case, when Xt is in Rτ for large τ we can identify this with Xt =(X1t , X2t , . . . , Xτ t , 0, 0, . . .) ∈ R∞ . So for practical applications, one may for example employ a truncation argument on the regressor (as will be done in Section 4.4 albeit with a different purpose) and let the effective dimension τ of the regressor Xt
to increase in n in the theoretical analysis.
The estimator can be viewed as an infinite-dimensional generalization of the standard multivariate local linear estimator, and is a special case of the one in Ferraty andVieu (2002), Masry (2005) and references therein for functional data. In the following
3This gives a mildP
restriction on the range of possible points at which the estimation is made; i.e.
x ∈ R∞ is such that j j −2p x2j < ∞.

14

section we will examine some asymptotic properties of the estimator.

4

Asymptotic Properties

In this section we introduce the main results of this paper. We derive some large
sample asymptotics of the proposed estimator (18). We establish consistency in both
pointwise and uniform sense, and also the asymptotic normality. All proofs are givenin the appendix.
Consider two different cases: (1) the static regression and (2) the dynamic regression. Below we specify two sets of temporal dependence conditions, either of which will
be assumed on the data generating process of the sample observations. Assumption
A1 corresponds to the static regression case where we have exogenous regressors thatare jointly observed in time in a weakly dependent manner. No restriction is needed
as regards the dependence structure between the marginal regressors, although certain
additional conditions can be potentially imposed at the later stage (see Assumptions
C below). The second option A2 concerns with the dynamic regression framework.
In this case, the notion of near epoch dependence is adopted to describe the dependence structure of the processes defined as functions of the response variables. The
assumptions below suggest that there is a trade-off between the degree of mixing andthe possible order of moments, we allow on the response variable, i.e. 2 + δ.
Assumptions A
A1. The marginal regressors X1t , X2t , . . . are exogenous variables, and the sample
data {Yt , Xt }nt=1 = {Yt , (X1t , X2t , . . .)}nt=1 is stationary and jointly arithmetically
α-mixing with rate k ≥ 2(δ +2)/δ, where δ is as defined in Assumption B4 below.
A2. Each regressor is either a lag of the response variable Yt or of a covariate Vt , i.e.
Xjt = Yt−j or Xjt = Vt−j , j ∈ N, and {Yt , Vt }nt=1 is stationary and arithmetically
α-mixing with rate k ≥ 2(δ + 2)/δ. Also, the process Kt := K(kH −1 (x − Xt )k)is near epoch dependent on (Yt , Vt ), and there exists some r = rn → ∞ such thatthe rate of stability for Kt denoted v2 (rn ) = v2 (r) satisfies
v2 (r)1/2 [ϕx (hλ)]−(2δ+3)/(2δ+2) n1/(2(δ+1)) → 0

as n → ∞.

(19)

Remark. Our model under Assumption A2 can be viewed as a generalization of
the NAARX model in Chen and Tsay (1993). The framework nests both the fully
15

autoregressive framework in which Xjt = Yt−j for all j, and the case where the regressor
vector consists only of the lags of a covariate Vt . Doukhan and Wintenberger (2008)
studied the autoregressive model of order d = ∞ under a notion of weak dependence,and showed the existence of a stationary solution. This result was further studied inWu (2011).

4.1

Pointwise consistency

Pointwise consistency of the local constant estimator was first studied by Watson(1964) and Nadaraya (1964) for i.i.d data with d = 1. Their result was extended tothe multivariate case (finite d) by Greblicki and Krzyzak (1980) and Devroye (1981).
Robinson (1983) and Bierens (1983) were amongst the earliest papers that worked on
consistency of the estimator with dependent observations (both static regression andautoregression were allowed in their frameworks), followed by Roussas (1989), Fan(1990), and Phillips and Park (1998) to name a few out of numerous papers. The caseof the functional regressor was first studied by Ferraty and Vieu (2002).
In this section we establish the pointwise weak consistency of the estimator (18)with dependent data satisfying either A1 or A2. A set of assumptions required for thetheory is now introduced, and some introductory arguments are briefly sketched.
Assumptions B
B1. The regression operator m : R∞ → R is continuous in some neighbourhood of x
B2. The marginal bandwidths satisfy hj = hj,n → 0 as n → ∞ for all j = 1, 2, . . .,where diag(h1 , h2 , . . .) = diag(h) = H is the bandwidth matrix, and the smallball probability obeys nϕx (hλ) → ∞ for every point x ∈ R∞ , where ϕx (hλ) :=P (kH −1 (x − X)k ≤ λ) → 0 as n → ∞.
B3. The kernel K is type−I
B4. The response Yt satisfies E |Yt |2+δ ≤ C < ∞ for some C, δ > 0.
B5. The joint small ball probability (17) satisfies ψx (hλ; i, j) ≤ Cϕx (λh)2 , ∀i 6= j.

B6. The conditional expectation E |Yt Ys ||Xt , Xs ≤ C < ∞ for all t, s.
Remark. The continuity assumption B1 is necessary for asymptotic unbiasednessof the estimator. It will be shown that the estimator is unbiased at every point ofcontinuity, and that the rate of convergence for the bias term can be specified upon16

imposing further smoothness condition on the regression operator, see later. Assumption B2 can be thought of as an extension of the usual bandwidth conditions thatare assumed in finite-dimensional nonparametric literature, cf. (7). As discussed before, nϕx (hλ) can be understood as an approximate number of observations that are“close enough” to x. Therefore, it is sensible to postulate that nϕx (hλ) → ∞ as
n → ∞, meaning that the point x is visited many times by the sample of data as
the size of the sample grows to infinity. This is in line with the usual assumptionthat nhd → ∞ when X ∈ Rd , in which case the small ball probability is given by
ϕx (h) ∝ hd pX (x) as noted in (7). Conditions B5 and B6 are imposed to control the
asymptotics of the covariance terms. The validity of condition B5 can be easily seen in
the Rd frameworks; for relevant discussions, see Ferraty and Vieu (2006, Remark 11.2).
To sketch the idea, we write Kt := K(kH −1 (x − Xt )k) for the sake of simplicity ofpresentation (note its dependence upon Xt ), and express the estimator (18) as
Pn Kt

Pn
1
−1
Y
(x − Xt )k Ytmb 2 (x)n
EK1 t
t=1 K kH
 = 1 Pt=1
m(x)
b
:= Pn
=
.
n
K
t
−1 (x − X )k
m
b
(x)
1
t
i=1
t=1 K kH
n
EK1

(20)

We then employ the following decomposition:mb 2 (x)mb 2 (x) − m(x)mb 1 (x)− m(x) =mb 1 (x)mb 1 (x)b 2 (x) − Emb 2 (x)] − m(x)[mb 1 (x) − Emb 1 (x)]Emb 2 (x) − m(x)Emb 1 (x) [m+,=mb 1 (x)mb 1 (x)m(x)b− m(x) =(21)

where clearly Em
b 1 (x) = 1. Below we show consistency by proving that the ‘bias part’Em
b 2 (x) − m(x) and the ‘variance part’ [mb 2 (x) − Emb 2 (x)] − m(x)[mb 1 (x) − 1] are bothnegligible in large samples. As for the latter term, it suffices to show the mean squared
convergence of mb 2 (x) − Emb 2 (x) to zero because mb 1 (x) →P 1 then readily follows.

Theorem 1 Suppose that Assumptions B1-B5 hold. Then the estimator (18) withsample observations {Yt , Xt }nt=1 satisfying either A1 or A2 is weakly consistent for the
regression operator m(x) = E(Y |X = x). That is, as n → ∞
P

m(x)
b
−→ m(x).

(22)

In the following section, we present the rates of convergence and asymptotic normality under additional regularity conditions.

17

4.2

Asymptotic Normality

Earlier studies on the limiting distribution of the standard Nadaraya-Watson estimator can be traced back to Schuster (1972) and Bierens (1987), where the case of
univariate and multivariate regressors was considered, respectively. The case of dependent samples was studied in Robinson (1983), Bierens (1983), Masry and Fan (1997),
and by many others under various model setups and different regularity conditions.
Masry (2005, Theorem 4) and Delsol (2009) established general distribution theories
for Nadaraya-Watson type estimators in a semi-metric space. Our results are different
from them in two respects. First, the difference of our framework from the functional
literature discussed in the beginning of Section 2.2 gives us further flexibility, without
which the analysis cannot be done to meet our specific needs. Second, whereas the
final results of many existing papers were given in terms of abstract functions, ourresults are presented with an explicit rate of convergence, allowing practical applications. In this section we outline the main theory and introduce some interesting
consequences thereof.
Our objective is to construct the asymptotic distribution of our estimator. Thatis, to find deterministic sequences Vn (x), Bn (x) such that
Vn−1/2 (x) m(x)b
− m(x) − Bn (x) =⇒ N (0, 1) .

(23)

In fact, under certain conditions4 we can show that the following self-normalized limiting distribution holds


∆−1
(x)
m(x)
b
−
m(x)
−
B
(x)
=⇒ N (0, 1) ,nnwhere ∆2n (x) :=

Pn

t=1 (

Pn

−2

s=1 Ks )(24)

2[Kt Yt − m(x)b
] , and as defined previously, Kt =K(kH −1 (x − Xt )k). The proof of (24) is given within the proof for Theorem 2 later.
This gives pointwise confidence intervals for m(x),bwhich can be used as a basis for
conducting standard statistical inference.
null
We now discuss some main assumptions needed for our distribution theory.
4.2.1

The bias and variance components

Regarding the asymptotic ‘bias’, we need to strengthen Assumptions B by imposing additional smoothness conditions and suitable bandwidth adjustments, just like4

Assumptions B7-B10 in section 4.2.1 below

18

the continuity assumption is extended for asymptotic normality in standard Rd cases.
These allow the exact upper bound of the asymptotic bias to be specified. Note thatalternatively, a Fréchet differentiability-type condition may be imposed. Here also
we introduce two conditions (Assumptions B9, B10) that we require for studying thevariance component.
Further Assumptions B
B7. The regression operator m : R∞ → R satisfies0m(x) − m(x ) ≤∞X

cj xj − x0j

β

(25)

j=1

for every x, x0 ∈ R∞ , and some constant β ∈ (0, 1], where {cj } is some sequencePof real constants that satisfies ∞j=1 cj ≤ 1.
B8. The marginal bandwidths satisfy hj = φj · h for some positive real numbers φj ,P
−2
where h = hn → 0 as n → ∞. We suppose that φj satisfy ∞j=1 φj < ∞ andP∞β
j=1 cj φj < ∞.
B9. The conditional variance var[Yt |Xt = u] = σ 2 (u) is continuous in some neighbourhood of x; i.e. supu∈E(x,hλ) [σ 2 (u) − σ 2 (x)] = o(1). Similarly, the crossconditional moment E[(Yt − m(x))(Ys − m(x)|Xt = u, Xs = v] = σ(u, v), t 6= s
is continuous in some neighbourhood of (x, x).
B10. Rnt := (EK1 )−1 {Kt (Yt − m(x)) − EKt (Yt − m(x))} belongs to the domain of
attraction of a normal distribution.

Remark. The first two assumptions concern with the bias component Bn in(23). Assumption B8 extends the previous bandwidth condition B2. Obviously, it isconsistent with (and stronger than) what was previously assumed in B2, since h → 0
implies the coordinate-wise convergence of each marginal bandwidths. With B8 one
can write the asymptotic bias and the order of the bias-variance balancing bandwidthin terms of the common factor h. It is possible to dispense with this condition at thecost of imposing minor modifications in B7; the asymptotic bias will then be written
in terms of the infinite sum of a weighted marginal bandwidth hj , whose convergenceneeds to be ensured. A further increment condition will be needed on φj later toelaborate the asymptotics of the variance term.
19

Assumption B7 replaces and strengthens B1, and can be thought of as a variantof Hölder-type continuity; the case of cj = 2−j and β = 1 is implied by the Lipschitz
condition. Another example of cj includes exp(−j). Indeed, under B7 the regression
operator becomes a contraction mapping, and the contribution from each marginal
dimension decreases in j. This ensures summability of the bias and allows the orderof convergence rate to be specified, cf. (28) below.
Note that in the context of autoregression where Xj ≡ Yt−j for all j, the model isgiven by
Yt = m(Yt−1 , Yt−2 , . . .) + εt .

(26)

Whether the stationary solution {Yt } indeed exists is an important question. In the
study of a class of general nonlinear AR(d) models, Duflo (1997) and Götze and Hipp
(1994) assumed what is called the Lipschitz mixing condition (or the strong contraction
condition), which is essentially (25) replaced by finite d-sum on the right hand side. In
our context, Assumption B7 plays a similar role; Doukhan and Wintenberger (2008)
P∞
showed that (25) with
j=1 cj < 1, is sufficient for the existence of a stationary
solution: for some measurable f , Yt = f (εt , εt−1 , . . .), where εt is an i.i.d. sequence.
P
Wu (2011) arrived at the same conclusion under the assumption of ∞j=1 cj = 1; thespecific restrictions on cj are chosen to reflect their findings, despite the fact that weare not restricting the error process {εt } to be an independent sequence in our model
setup.
The standard conditions B9 are assumed to deal with the asymptotics of thevariance and covariance terms. The last condition B10 is needed only for the selfnormalized CLT (24) without assuming higher moment conditions; relevant discussions can be found for example in de la Peña et al. (2009). The condition is notaffected by the temporal dependence of the DGP as the property is inherited to theapproximated sum in the Bernstein’s blocking procedure; see (77) later for details.
Lastly, before we proceed, we briefly remark that from now on the rate condition (19)is slightly strengthened as follows (modifying Assumption A2 accordingly):v2 (r)1/2 [ϕx (hλ)]−1 n1/2 → 0

as n → ∞.

(27)

With the additional assumptions introduced in this section (B7-B9), the bias and20

variance components can be specified as follows, and the CLT (23) can be constructed:∞hiX
cj j pβBn (x) := Emb 2 (x) − m(x) ≤ hβ λβ

(28)

j=1
2

σ (x)ξ2,nϕx (hλ)ξ12

Vn (x) := var [mb 2 (x)] '(29)

where λ and mb 2 (·) are as in (5) and (20), respectively. Formal derivation is done in
Section 7.2 of the appendix within the proof for Theorem 2 we introduce below.

4.2.2

Sufficient conditions for the derivation of convergence rates

Convergence rates are crucial in understanding estimators’ large sample asymptotics
and the evaluation of their performance. Below we introduce a set of conditions underwhich the rate for our estimator can be specified. We elaborate how the rate of convergence of our estimator depends crucially on (i) the distribution of marginal regressorsand (ii) their “cross-sectional” dependence structure. It is important to note thatthe conditions are sufficient but not necessary; we leave other possibilities as futurestudies, hoping that similar theories would work in a wider range of frameworks.
(i) Dependence across marginal regressors
The way how the marginal regressors are “cross-sectionally” related to each other(given each fixed time) affects the rate of convergence. We consider and allow for thefollowing dependence:
Assumption C. For every fixed t, the real-valued stochastic process formed by the4
marginal regressors {Xjt }∞j=1 has finite fourth moments, i.e. EXjt ≤ C < ∞ ∀j,t,

and is stationary and admits the following causal moving average representation:
Xjt =

∞
X

au j−u,t ,

(30)

u=0

where au is square summable, and {jt }j is an orthogonal sequence.
Remark. The dependence structure described in the assumption above is very
weak and general, and covers a large class of processes. To elaborate, a necessary
and sufficient condition for a stationary sequence to have the representation (30) is

21

linear regularity 5 , a weak notion of dependence. For example, if the stack of marginal
regressors {Xjt }j (with finite second moments) are independent of each other or αmixing between themselves, they satisfy the condition because they both imply linearly
regularity. Therefore, Assumption C is consistent with both Assumptions A1 (thestatic regression case) where Xj ’s are exogenous variables and can have any dependence
structure, and Assumption A2 (the dynamic regression case) where Xj ’s are (temporal)lags of a mixing variable. The special case when X’s are Gaussian is worth noting.
If {Xjt }j is Gaussian and linearly regular, then it has the representation above withindependent and normally distributed 0j s. An equivalent condition for linear regularityof a Gaussian sequence is simply the existence of a spectral density.
The requirement of finite 4th moment is imposed just to ensure that the squared
marginal regressors have finite second moments; this is related to the distributional
properties of the regressors and will become clear below. Note that obviously, whena lag of the response is included as in the dynamic regression framework (A2), thismakes the maximum order of moments of Yt (i.e. 2 + δ) ≥ 4due to Assumption B4.
(ii) The distribution of regressors
We can show the rate becomes available when, in addition to Condition C above,the marginal regressors satisfy some distributional properties (and when also they are“downweighted” in a certain way via bandwidths). Below we introduce these conditions and discuss them in detail. Note that as defined above, vectors Z and z are
|

|

−1
−1
−1
taken to mean (φ−1
1 X1 , φ2 X2 , . . .) and (φ1 x1 , φ2 x2 , . . .) , respectively, where the|vector x = (x1 , x2 , . . .) is the point at which estimation is made, and φ0j s are theweight coefficients on bandwidths introduced in Assumption B8 above.
Assumptions D
D1. The distribution F of Xs2 , where each Xs is the marginal regressor, is regularly
varying near zero with strictly positive index (−ρ) > 0.
D2. The induced probability measure Pz−Z is dominated by the measure PZ , and its
Radon-Nikodym density dPz−Z /dPZ =: p∗ is continuous and is bounded awayfrom zero at 0 ∈ R∞ ; i.e., p∗ (0) > 0.
D3. Further to B8, the bandwidth satisfies hj = j p h (i.e. φj = j p ) with p ∈ Π(c, β),5

See Ibragimov and Linnik (1971, Chapter 17) or Davidson (1994, Part III) for details.

22where(Π(c, β) =

∞
X

)
1
p:
cj j pβ < ∞, < p .
2
j=1

Remark. Condition D1 concerns with the marginal distributions of the regressor
vector. It is equivalent to saying that
F (1/(γx))= γ ρ,
x→∞ F (1/x)
limwhere ρ is the index of variation which is strictly negative. Under this condition,
Dunker, Lifshits and Linde (1998, cf. Conditions I and L) derived the explicit behaviour of the small ball probability. We require the function F (1/x) to be regularly
varying in order to ensure that the small ball probability is well-behaved near infinity
in the asymptotic sense. Since only those functions having strictly negative ρ satisfy
the condition, the distribution F of the squared regressor must be such that F (1/x)
decreases (as x → ∞) at a reasonable speed. By reasonable we mean that the relative
weight of decrease follows a power law, and the variation should be continuous. A
large class of common distributions satisfies this condition; for example: the Gamma,
Beta, Pareto, Uniform, Exponential, Weibull, and also the Chi-squared distribution(in which case each marginal regressor Xs is Gaussian).
Assumption D2 is about the transition of the shifted small ball probability to
the centred small deviation (whose asymptotic behaviour is more accessible), see Section 2.3 above and Mas (2012). The explicit form of the derivative (and hence ofthe relationship between the two probabilities) cannot be easily computed in general. Nonetheless, in the special case of the Gaussian process Z with some covariance
operator Σ it is known by Sytaya (1974) and Zolotarev (1986) that





1 −1/2 2
P kz − Zk ≤  ' P kZk ≤  exp − kΣzkas  → 0.
2


(31)

The reader is directed to Li and Shao (2001) for detailed discussion on this asymptotic
equivalence relation. Note that Σ can be expressed in terms of the aj constants (inAssumption C), which govern the dependence across the marginal regressors, and of
the bandwidth weights φj :cov(Z) = Σ = (DA)(A∗ D),(32)

where A = (aij ) = (ai−j ) and D = diag(φ1 , φ2 , . . .).
Assumption D3 is concerned with how the ordered marginal regressors are down23

weighted. The specific bandwidth increment condition assumed in D3 is one framework
under which the explicit behaviour of the small ball probability can be specified, (seee.g. Dunker et al. (1998)). In case the regressors are independent to each other,
the probability can also be derived when the weights are of an exponential type (i.e.
hj = ej h) up to an unknown function, or are non-increasing in a particular manner(see Gao et al. (2003)) similar to the polynomial decay. In this paper however, weshall confine our attention to the case of the polynomial law for expositional simplicity
and consistency of presentation, since the asymptotic behaviour of the small ball is
not yet known in the general case for choices other than the polynomial decay as inAssumption D3.
In practice, we would require some ordering for the marginal regressors in thestatic regressions case A1, since the influence of marginals is set to decrease via the
bandwidth adjustments as discussed just above. One practical way of doing this is
to rank them in the order of goodness of fit, or the contribution that each marginal
regressor makes in the estimation. For example, one could evaluate the sample corb t |Xjt ), where Xjt is a marginal covariate and E(Yb t |Xjt )relations between Yt and E(Yis a kernel estimate of the univariate marginal regression, and order them according
to the computed correlations. This way one can line up the marginal regressors in
the order of their relative importance. This method is motivated by the Kernel Sure
Independence Screening (KSIS) approach in Chen, Li, Linton and Lu (2018), and thereader is referred to their paper for further details.
4.2.3

The Central Limit Theorem

We now introduce the general central limit theorem. The theorem below gives thelimiting distribution of the estimator (18) with respect to mixing sample data as
described in either Assumption A1 or A2.

Theorem 2 Suppose that B2-B9 and D1-D3 hold. Let the marginal regressors Xssatisfy Assumption C. Then the estimator (18) based on the sample observations{Yt , Xt }nt=1 satisfying either Assumption A1 or A2 is asymptotically normal with thefollowing limiting distribution:
r


h
i

1+2ρp
2
nh 2p−1 exp −κ0 h− 2p−1 m(x)b
− m(x) − Bn (x) =⇒ N 0, κ1 σ 2 (x) ,(33)

where Bn (x) = O(hβ ) is the ‘bias part’ in (28) and σ 2 (x) = Var(Y |X = x) is theconditional variance defined in Assumption B9.
24

Below we explain the associated constants. Recall Assumption D1; under thatcondition, by the characterization theorem of Karamata (1933) (see e.g. Feller (1971)),
there always exists a slowly varying function `(x) satisfying F (1/x) = xρ `(x). Now fix
some p, the order of increment constant for bandwidth in Assumption D3, and denoteby L(t) the Laplace transform of X 2 . Then we have,h−1/2C` = lim `
δ→0

∗

C =



δ

4p
− 2p−1

i

Z ∞
,

ζ=−
0

(2π)(1+2pρ) (2p − 1)
Γ−1 (1 − ρ) · (2p)2p(ρ+2)−1
2p−1

·ζ

2

κ0 (K, p, F ) = C ∗∗ (CA λ−1 ) 2p−1

2p(1+ρ)
2p−1

,

C

∗∗

u−1/2p L0 (u)
du
L(u)


= (2p − 1) ·and κ1 (K, p, F ) =ζ2p

2p/(2p−1)

C ∗ C` ξ2
1+2ρpp∗ (0)ξ12 (CA λ−1 ) 1−2p,where Γ(·) is the Gamma function, ξ1 and ξ2 are the constants specified in (12) (whichsimplify in case of uniform kernel for example), λ is the upper bound of the support of
the kernel, and p∗ (·) is the Radon-Nikodym derivative defined in D2. Recall that for
the uniform (Box) kernel ξ2 = ξ12 , so they cancel out in κ2 . See Dunker, Lifshits and
Linde (1998) and also Hong, Lifshits and Nazarov (2016) for some discussions on theunderlying arguments for the formulation of these constants. To aid the exposition, we
compute and present the constants for some common, regularly varying distributionsin the table below.

Xj2 ∼ F i.i.d.
Uniform(1,b)
Gamma(α, β)

ρ
−1
−α

limx→∞ `(x) = C`−21
α −1
β α Γ(α)−1Exponential(η)
Weibull(α, β)Pareto(θ, µ)
χ21

−1
−α
−1
−1/2

η
β
µ/θ
(2/π)1/2ζn/a
απβ −1/2psin(π/2p)πη −1/2psin(π/2p)n/an/aπ2(1−2p)/2psin(π/2p)

Table 1: Examples of the key constants for some common distributions
For example, when Xj ’s are Gaussian, the constants C ∗ and C ∗∗ denoted CG∗ andCG∗∗ respectively, are given by:CG∗ =(2π)(1−p) (2p − 1)
3p−1

2 · (2p) 2p−1π2(1−2p)/2p·sin(π/2p)−p 2p−1,25

2p − 1
CG∗∗ =
2ππ
2p sin 2p2p! 2p−1.

andκ2 (K, p, a) =CG∗ C` ξ2 ξ1−2
1

−1/2 zk22e− 2 kΣp−1(CA λ−1 ) 2p−1

where z = (zj ) = (j −p xj ) = D−1 x. The exponential term in the denominator of theasymptotic variance arises from the asymptotic equivalence relationship between theshifted and non-shifted small deviation for `2 -valued Gaussian variables, cf. (31).
The CA constant represents the dependence across the marginal regressors andplays an important role. First of all, when the marginal regressors are identically
distributed and independent to each other, then CA = 1. If not independent, thespecific form of this constant is only known in the Gaussian case (by Hong, Lifshits
and Nazarov (2016, Theorem 1.1)). Specifically, when {Xj }j is centred Gaussian andsatisfies Assumption C, then given the square summable sequence aj in (30), we have
"

1
CA =
2π

Z 2π X
∞
0

#p

1/p

aj exp(ijs)

ds

.

(34)

j=0

It is worth noting that CA is a function of the spectral density of the MA(∞) process {Xjt }j denoted SX (·). Specifically, some straightforward algebra gives CA =R 2π(2π)p(p−2)/2 · [ 0 SX (s)p/2 ds]p .
The implications of the constant CA suggest an interesting finding that allowing for
dependence does not seem to incur much penalty; we conjecture that similar conclusion
would hold for regressors of different distributions than Gaussian, but leave it for futurestudies.

4.3

Optimal Bandwidth

We now discuss the issue of bandwidth optimality. As in the finite-dimensional framework, there is a bias-variance trade-off. As the bandwidth goes up, the variance getssmaller while the bias increases, and vice versa. Therefore we search for the optimal
bandwidth hopt that balances the order of those two quantities.
We first suppose that p ∈ Π(c, β), cf. D3, is given. In the i.i.d. case with Gaussian
regressor we have
s
β

h ∼

exp κ0 h−2/(2p−1)
1−p


,

nh 2p−1
so that




2
1−p
2β +
· log h − κ0 h− 2p−1 ∼ − log n.
2p − 1
26(35)

Taking h ∼ (log n)a for some a < 0 balances the leading terms on both sides:



2
1−p
2β +
· a · log log n − κ0 (log n)− 2p−1 ·a ∼ − log n.
2p − 1(36)

The explicit order a that solves (36) can be expressed in terms of n, β and p. Writingϑ := [2β + (1 − p)/(2p − 1)] and χ := 2/(2p − 1) for notational simplicity, and solving
for a we haveaopt =

ϑ·Wχ· κ0 · nχ/ϑϑ− χ log n,ϑχ · log log n(37)

where W(y) is the Lambert W function (see e.g. Olver et al. (2010)), which returns
the solution x of y = x·ex . From (37) the optimal bandwidth hopt ∼ (log n)aopt follows,in which case the asymptotic root mean squared error is of the order (log n)βaopt .
Remark. We can look for the optimal bandwidth for the cases of non-Gaussian
regressors by following exactly the same procedure as above; tedious details are omitted here. Lower value of ρ makes the rate better in general. Regarding the solution in(37), since the mapping x 7→ x·ex is not an injection, the solution may be multi-valued
on the negative domain, i.e. y < 0. This does not happen in (37) provided β ≥ 1/4(however big p is), because (1 − p)/(2p − 1) is bounded away from −1/2; in this case,the coefficient of the double logarithmic term in (36) is strictly smaller or equal to zero.
Since the log terms dominate the double logarithm in (36) as the sample size n
increases, it can be readily expected that the optimal value of a in (37) converges
to a limit in such a way that the leading orders are balanced. Below we introduce
without formal justification a trivial result that gives the lower bound (infimum) of
the optimal bandwidth (and hence of the optimal rate that balances the bias andvariance). We remark that the result below holds for other choices of the distributionof the regressors, and also for the case of dependent (non-independent) regressors asallowed in Assumption C (i.e. when CA 6= 1), as the exponent of the leading term
−2/(2p − 1) remains invariant as is clear from Theorem 2. Nonetheless, it is worth
noting that although the order of the convergence rate remains the same, the difference
in associated constants does make the speed at which aopt converge to the limit in (38).
null
Corollary 2 For any fixed choice of p ∈ Π(c, β) and the distribution F of X 2 satis-

27

fying Assumption D1, the order of the optimal bandwidth aopt satisfies

aopt ↓

2p − 1−2
asn → ∞,(38)

which suggests that the lower bound of the optimal bandwidth is given by
2p−1(log n)− 2  hopt ∼ (log n)aopt .

(39)

This result tells us the best possible performance we can expect from the optimal
bandwidth. Higher p and β makes the rate better. But, because nk (log n)−(2p−1)/2 →∞ for any real k > 0, it follows that we cannot possibly estimate the regression function at a polynomial rate. This result is consistent with and complements the findingsof Mas (2012, Theorem 3), which were obtained under the assumption of independence of regressors. The paper also considered the case where the bandwidth grows
exponentially: φj  exp(j q ) for some q > 0. Then for hj = φj h, his result suggests
exp[−(log n)2q/(2q+1) ]  hopt ∼ exp[aopt · (log n)bopt ]. Therefore, the performance is better in general in this case, although obviously a polynomial rate of convergence stillcannot be attained. It is not clear what will happen when the regressors are allowed
to be dependent in the sense of Assumption C, since the behaviour of the small ballprobability for non-independent (and non-Gaussian) sequence is not known for thecase of exponentially decaying weights.
Returning back to Corollary 3, we emphasize that the arguments are true for any
p ∈ Π(c, β). Let pmax = supp∈Π(c,β) p. Then a lower bound on the optimal bandwidth1= 1/β.
(over all p) is (log n)−(pmax − 2 ) . For example, when c = (1/2)j −2 we have pjmax

Unfortunately, it is generally the case that pmax ∈/ Π(c, β), in which case the lower
bound is not quite achievable by our method.
Remark. Regarding bandwidth selection, one possibility is the Bayesian bandwidth selection methods like proposed in Zhang, King, and Hyndman (2006). We
take as prior for h the density proportional to 1/(1 + λh2 ) and as prior for p − 1/2
the density of a χ2 (w) random variable. The hyperparameters λ, w may be chosen byexperimentation. The priors are combined with a Gaussian (least squares) density todeliver a posterior for the bandwidth. The reader is referred to Section 5 for further
discussions on the issue of bandwidth choice.

28

4.4

Uniform consistency

Uniform consistency of the Nadaraya-Watson estimator was first studied by Nadaraya(1964, 1970) and subsequently by numerous others. To mention few early papers,
Devroye (1978) weakened the regularity conditions required in the previous papers,
and Robinson (1983) proved uniform consistency for dependent sample data. In thefunctional statistics literature, uniform consistency of kernel estimators for conditionalmean is established only with respect to i.i.d. sample data so far (see for example
Ferraty et al. (2010), Ferraty et al. (2011), Kudraszow and Vieu (2013), and KaraZaı̈tri et al. (2017)) as far as the authors are concerned.
In this section, we show uniform consistency of our estimator under the (suitably
modified) regularity conditions assumed in the previous sections. We start by introducing the notion of Kolmogorov’s entropy below. For some of its earlier discussions
in statistics literature, the reader is referred to Yaracos (1985) and Mammen (1991).
Definition 5. Given some η > 0, let L(S, η) be the smallest number of open balls in
E of radius η needed to cover the set S ⊂ E. Then Kolmogorov’s η-entropy is defined
as log L(S, η).
This quantity will be used in explaining the topological restrictions we adopt tosuitably accommodate infinite dimensionality. The definition implies the dependenceof Kolmogorov’s entropy both on the nature of the space under study and the measureof proximity. It will be shown later in this section that the entropy is closely related
to the rate of convergence of the estimator, in particular, to the penalty incurred on
the rate in the uniform case. It is well known that the regression function cannot be
estimated uniformly over the entire space, e.g. Bosq (1996). In our infinite dimensionalframework, the infinite sequence spaces, if unrestricted, cannot be covered by a finite
number of balls, and that L(S, η) = ∞. We propose to consider uniform consistency
over a subset of R∞ , whose effective dimension is truncated and is increasing in sample
size n. In particular, we define the setSτ := u|(ui )i∈Z+ , uj = 0 for all j > τ, kuk∞ ≤ λ ⊂ R∞ ,(40)

where τ = τn is some increasing sequence and λ is fixed, and consider uniform consistency over this compact set. Then Kolmogorov’s entropy of the set Sτ is given as
follows:29

Lemma 2 Kolmogorov’s η-entropy of Sτ with τ = τn (→ ∞) and λ > 0 is"

log L Sτ , η = log

!τ #
√
2λ τ
+1
.
η

(41)

null
Remark. (41) is in line with common intuition; as the effective dimension τ increases, the number of balls (with some fixed radius) required to cover the set tends
to infinity. Lemma 2 can be shown by exploiting the splitting technique and then bycovering the polyhedron of increasing dimension. See appendix for details. Note thatfor λ fixed and η = ηn , Kolmogorov’s entropy log L(Sτ , η) is of order (τ log τ − τ log η).
Considering the definition of the set Sτ , in the sequel (with a slight abuse of notation) we take X to denote the regressor, but with zeros after its τ th (= τn → ∞ as n →|∞) entry; that is, X = (X1 , X2 , . . . , Xτ , 0, 0, . . .) (so that the original X is recovered
as n → ∞). Also, the regression operator and the estimator with respect to thistruncated regressor are denoted by mτ (·) and mb τ (·), respectively. All assumptions,
including the additional one to follow below, are understood to hold under these modifications.
Assumption E
E. For sufficiently large n, Kolmogorov’s η-entropy log L(Sτ , η) satisfiespnϕx (h)(log n)8+2
≤ log L(Sτ , η) ≤nϕx (h)(log n)1+

for some  ∈ (0, 1/2).

(42)

Furthermore, 0 < ϕx (h)  h < ∞ and (log n)2 /(nϕx (h)) −→ 0 as n → ∞.
Remark. The first part of Assumption E specifies the rate at which Kolmogorov’s entropy should behave with sample size n (hence in dimension τ = τn ). From the upperand lower bound it readily follows that nϕ(h) must be of order larger than (log n)6+2 .
This assumption is sufficiently general. For example, in view of the bias-variance optimal bandwidth suppose h ∼ (log n)−(2p−1)/2 so that nϕ(h) ∼ (log n)(2p−1)β . In thiscase, assumption (42) is valid as long as p is moderately large enough relative to β ≤ 1
in such a way that 6 + 2 ≤ (2p − 1)β. The second part is standard; in particular,
the last condition straightforwardly follows by (42) and only slightly strengthens the
bandwidth condition in Assumption B2.

30

For uniform consistency we shall impose a stronger condition on mixing coefficients. From hereafter, by A10 and A20 we mean Assumptions A1 and A2 but withthe arithmetic mixing rate condition strengthened to the following exponential mixing
condition (cf. Definition 1):α(r) ≤ exp(−ςrγ2 )(43)where ς > 1 and γ2 is a positive constant such that γ := 1/(γ1−1 + γ2−1 ) ≥ 1, with
γ1 defined as in Assumption B40 . When the response is assumed to be bounded (i.e.
|Yt | ≤ C), γ1 may be taken to be ∞ so that γ2 = γ ≥ 1. This stronger mixing
condition enables us to obtain exponential bounds that decay fast enough, thereby
accommodating uniformity, see appendix for details. We hope to expect the sameconclusion in this section to hold under the arithmetic mixing condition we previously
assumed, once some suitably sharper exponential inequality becomes available. Inline with the modification on the mixing rate above, we also impose a slightly stricter
condition on the response:
B40 . The response Yt is satisfies the following tail condition: There exists some positive constant γ1 and C such that P (|Yt | > u) ≤ C exp(1 − uγ1 ) for any u > 0.
For example, a Gaussian random variable satisfies B40 with γ1 < 2. The condition is
also satisfied by many unbounded variables and all those bounded ones. The main
result of this section now follows.

Theorem 3 Suppose that Assumptions B2, B3, B40 , B5-B9, D1-D3 and E hold. Let
the marginal regressors Xs satisfy Assumption C, and take τ = τn ∼ (log n). Thenthe estimator mb τ (·) with respect to sample observations {Yt , Xt }nt=1 satisfying A10 isuniformly consistent for m(x) = m(x1 , x2 , . . .) over Sτ :s
sup mb τ (x) − mτ (x) = OP hβ +x∈Sτ(log n)2 exp κ0 h−2/(2p−1)
1+2pρ

!
.

(44)

nh 2p−1

Remark. We may choose the optimal bandwidth as before; following the same
arguments in the pointwise case, choosing h ∼ (log n)a and solving for n gives
aopt =

ϑ·W

χ


χ
c
exp(−
2
log
log
n
+
χ
log
n)
+ 2χ log log n − χ log nϑϑ.
ϑχ log log n
31

(45)

And because the order of the leading terms is (log n)−(2p−1)/2 as in the pointwise
case, it is straightforward to see that the lower bound of the optimal bandwidth inCorollary 3 still continues to hold; that is, hopt  (log n)−(2p−1)/2 . This is again
invariant to the choice of distribution F of the squared regressor. It is important tonote that as before, potential cross-sectional dependence between marginal regressors
and also their distributional properties are represented via c, the collection of constants
that appear inside the exponential terms in the asymptotic variance.
The results altogether give the optimal rate of convergence of our estimator as
follows.
Corollary 3 Suppose conditions assumed in Theorem 2.4 hold. Upon choosing h ∼(log n)aopt , where aopt is as defined in (45), we have


sup m
b τ (x) − mτ (x) = OP [log n]β·aopt .

(46)

x∈Sτ

In the pointwise case the same result in Corollary 4 trivially holds, but with thedifferent optimal aopt ; it is as given in (37). In that case this rate of convergence isminimax optimal in view of Theorem 3 of Mas (2012). Although both aopt convergeto −(2p − 1)/2, the speed at which they converge is different as can be seen in theexample in Figure 1 below.

Figure 1: aopt = aopt (n) for β = 1 and p = 232

5Application to the Risk Return RelationshipThe relation between the expected excess return on the aggregate stock market the so called “equity risk premium” - and its conditional variance has long been thesubject of both theoretical and empirical research in financial economics. The riskreturn relation is an important ingredient in optimal portfolio choice, and is central
to the development of theoretical asset-pricing models aimed at explaining a host ofobserved stock market patterns. Asset pricing models generally predict a positive
relationship between the risk premium on the market portfolio and the variance of its
return. In an influential paper, Merton (1973) obtained very simple restrictions albeit
under somewhat drastic assumptions; he showed in the context of a continuous time
partial equilibrium model thatµt = E[(rmt − rf t )|Ft−1 ] = γ × var[(rmt − rf t )|Ft−1 ] = γσt2 ,(47)

where rmt , rf t are the returns on the market portfolio and risk-free asset respectively,
while Ft−1 is the market wide information available at time t − 1. The positive
constant γ is the Arrow–Pratt measure of relative risk aversion. The linear functional
form actually only holds when σt2 is constant; otherwise µt and σt2 can be nonlinearly
related, Gennotte and Marsh (1993). Further examples with a positive risk return
trade-off include the external habit model of Campbell and Cochrane (1999) and theLong Run Risks model of Bansal and Yaron (2004). However, a negative risk-return
relation is not inconsistent with (a general enough) equilibrium, Backus and Gregory(1993). Unfortunately, the empirical evidence on the risk-return relation is mixed andinconclusive. Ghysels, Santa-Clara, and Valkanov (2005), Lundblad (2005), Bali andPeng (2006), Pástor, Sinha, and Swaminathan (2008), and Ludvigson and Ng (2007)
find a positive risk-return relation, while Campbell (1987), Glosten, Jagannathan,and Runkle (1993), Harvey (2001), and Lettau and Ludvigson (2003) find a negative
relation. Still others find mixed and inconclusive evidence like French, Schwert, andStambaugh (1987), Nelson (1991), Campbell and Hentschel (1992), Linton and Perron(2003), and Whitelaw (1994). Scruggs (1998) and Guo and Whitelaw (2006) document
a positive trade-off within specifications that facilitate hedging demands. However,
Scruggs and Glabadanidis (2003) find that this partial relationship is not robust acrossalternative volatility specifications.
As already mentioned in the beginning of this paper, the main difficulty in estimating the risk-return relation is that neither the conditional expected return nor theconditional variance of the market is directly observable. The contradictory findings33

of the above studies are mostly the result of differences in the specifications and approaches to modelling the conditional mean and variance. Pagan and Ullah (1988),
and Pagan and Hong (1990) initiated the use of nonparametric methods in this setting. The latter paper argued that the risk premium µt and the conditional varianceσt2 are highly nonlinear functions of the past whose form is not captured by standard
parametric GARCH–M models. They estimated E(rmt − rf t |rm,t−1 , . . . , rm,t−p ) andvar(rmt − rf t |rm,t−1 , . . . , rm,t−p ) nonparametrically, where p ∈ {1, 4}, finding evidenceof considerable nonlinearity. They then estimated δ from the regression
rmt − rf t = δσt2 + ηt ,(48)

by OLS and IV methods, finding a negative but insignificant δ. There are a number ofdrawbacks with the Pagan and Hong (1990) approach. Firstly, as aforementioned in
the introduction, the conditional moments are calculated using a finite and small conditioning set. This greatly restricts the dynamics for the variance process. Secondly,
they only test for linearity of the relationship between µt and σt2 ; this seems to be
somewhat restrictive in view of earlier findings. Linton and Perron (2003) considered
the model where σt2 was a parametrically specified CH process (with dependence on
the infinite past) but µt = ϕ(σt2 ) for some function ϕ of unknown functional form.
They proposed an estimation algorithm but did not establish any statistical properties. They found some evidence of a nonlinear relationship. Conrad and Mammen(2008) develop the theory of estimation and inference for this model. Christensen,
Dahl, and Iglesias (2012) developed the theoretical framework by considering volatility models that are driven by observable shocks so that a full theory can be given.
Escanciano, Pardo-Fernández and Van Keilegom (2017) consider a more general classof semiparametric models. Under the semi-strong form of the efficient market hypothesis prices contain all relevant information and so the risk premium and risk themselves
can be expressed in terms of only the past history of prices. We shall use this assumption to obviate the omitted variables/endogeneity issues that have limited previous
applications in this area.

5.1

Empirical study on the US stock market

We apply our methods to the daily risk premium on the value weighted S&P500 index— the total return on the index minus the returns on T-bills6 , denoted Yt — overthe period 04 January 1950 to 30 August 2017, a total of 17,025 observations. The
6

Data obtained from Yahoo Finance and Kenneth French’s Data Library.

34

whole time period is divided into 5 subperiods: 1950:01:04-1963:02:21, 1963:02:251976:05:04, 1976:05:05-1989:05:24, 1989:05:25-2002:06:24, and 2002:06:25-2017:08:30,
to see if there is any variation in the ex-post risk and return by decades. Except for
the last subperiod where there are 3824 observations, the other five each contains 3300
observations. We suppose that both the conditional mean and variance of Yt , denotedµt and σt2 , are unrestricted nonparametric functions of the entire information set. We
estimate them for p = 4 and 12 at the points Xt = (Yt−1 , Yt−2 , . . . , Y1 , 0, 0, . . .). The
uniform kernel K(kuk) = 1[0,1] (kuk) is used, and the bandwidth sequence h of 0.00035
and 0.000125 are used for p = 4 and p = 12, respectively. These bandwidths are inaccordance with the selection methods we propose below in the end of this section.
Table 2 reports some summary statistics of the nonparametric estimates µ
bt and σ
bt2
for p = 4 over the full period (1950-2017). We present the mean, standard deviation,
skewness, kurtosis and the fitted AR(1) coefficients. The estimated conditional variance shows high persistence. Table 2 may be compared with Table I of Bali and Peng
(2006), where they report similar descriptive statistics for their realized, GARCH, andimplied volatility estimates computed using 5-minute high frequency dataset. Notethat their time period is different (1982-2002), and they present excess kurtosis.
Table 2: Summary statistics of the estimates (bµt , σbt2 )Full Period (1950-2017)
Mean

Std

µ
bt

3.1260×10−4

σ
bt2

−5

6.5894×10

Skewness

KurtosisAR(1)2.3346×10−3

1.03347

3.41940.0190−56.606676.20624 0.7033

4.8913×10√
Figure 2 reports the (annualized) estimated values, that is, ( 252 · σ
bt , 252 · µbt ),t = 2, . . . , n(= 17025) when p = 4. The result shows there is no noticeable disparity
over different time periods, although the estimates are more spread out in the more
recent periods, showing higher variability. Having a different number of observations
does not seem to affect the conclusion either, seeing from the last plot. Interestingly,
the number of negative expected excess returns is quite large; such estimates are
not inconsistent with asset pricing theory, Boudoukh et al. (1997), Whitelaw (2000),Harvey (2001). The plot of estimates evaluated when p = 12 – omitted here – reports
similar findings, except that the estimates are a bit more concentrated.
Note that to focus on the main “chunk” of the fitted values, where almost allobservations are located, the plots in Figure 2 are magnified and truncated on the35

1950/01/05 - 2017/08/31

3

0
-1
-2

2

Conditional Mean

1

1
0
-1
-2

-3
0.1

0.15

0.2

0.25

0.3

0.1

0.15

0.2

0.25

0.3

0.05

-1
-2

1
0
-1
-2

-3
0.15

0.2

0.25

0.3

0.25

0.3

1
0
-1
-2

-3
0.1

Conditional Standard Deviation

0.2

2

Conditional Mean

Conditional Mean

0

0.15

2002/06/25 - 2017/08/31

3

2

1

0.1

Conditional Standard Deviation

1989/05/25 - 2002/06/24

3

2

0.05

-1

Conditional Standard Deviation

1976/05/05 - 1989/05/24

3

0

-3
0.05

Conditional Standard Deviation

1

-2

-3
0.05

1963/02/25 - 1976/05/04

3

2

Conditional Mean

Conditional Mean

2

Conditional Mean

1950/01/05 - 1963/02/21

3

-3
0.05

0.1

0.15

0.2

0.25

Conditional Standard Deviation

0.3

0.05

0.1

0.15

0.2

0.25

0.3

Conditional Standard Deviation

Figure 2: Annualized estimates of conditional mean and standard deviation, p = 4
ranges of [−3, 3] of the y-axis and [0.0225, 0.3] of the x-axis; around 96.1% of theentire fitted values appear in the plot. In particular, among those not appearing in
the plot are those with zero estimated conditional standard deviation, which constitutearound 3% of the whole estimates. This happens when, at a point of evaluation Xt ,only one kernel in the sums returns a value of 1 and zero otherwise, so that the second
moment equals the squared first moment. One way of reducing the number of such
estimates is to increase the bandwidth. We do not proceed to this direction because
it makes the bandwidth sub-optimal and the number of those observations is rather
negligible.
Figure 3 and 4 show the estimated relationship obtained using local constantsmoothing, with the bandwidth chosen according to Silverman’s rule of thumb. The
smooths are evaluated at the 100 quantiles of the marginal distribution so that the
spacing of the covariate can be shown. The first and last 5 values are taken out, since
they tend to be extreme values in general, and including them may make the graphlook misleading. All subplots suggest that quadratic fits, i.e. including the conditionalvariance term, would be appropriate. From Figure 4, we note that when p = 12, i.e.
when the influence of distant lags is “less weighted”, we begin to see some negative
relationships in some subplots (especially in the more recent periods), which is consistent with our findings later in this section.

36

0.12

1950/01/05 - 2017/08/31

1950/01/05 - 1963/02/21

0.14

0.1

0.12

0.08

0.1

0.15

1963/02/25 - 1976/05/04

0.06
0.04
0.02

Conditional Mean

Conditional Mean

Conditional Mean

0.1

0.08
0.06
0.04

0.05

0

-0.05
0
-0.02
0.08

0.02

0.1

0.12

0.14

0.16

0
0.08

0.18

Conditional Standard Deviation

0.15

0.1

0.12

0.14

0.16

-0.1
0.08

0.18

Conditional Standard Deviation

1976/05/05 - 1989/05/24

1989/05/25 - 2002/06/24

0.2

0.15

0

-0.05
0.08

0.1

0.12

0.14

0.16

0.18

0.16

Conditional Mean

Conditional Mean

Conditional Mean

0.05

0.14
0.12
0.1

0.14

0.16

0.18

2002/06/25 - 2017/08/31

0.05
0
-0.05
-0.1
-0.15

0.08

-0.2

0.06
0.05

-0.25
0.05

Conditional Standard Deviation

0.12

0.1

0.18
0.1

0.1

Conditional Standard Deviation

0.1

0.15

0.2

Conditional Standard Deviation

0.1

0.15

0.2

Conditional Standard Deviation

Figure 3: Estimated relationship between annualized σ
bt and µ
bt ; p = 4

Now we consider some parametric analysis, and suppose the conditional meanµ(x) = E(Y |X = x) and conditional variance σ 2 (x) = var(Y |X = x) are related in aquadratic way, i.e.,µ(x) = α + γσ(x) + βσ 2 (x),

(49)

|

where θ = (α, γ, β) with α, β, γ being unknown constants. Let x1 , x2 , . . . , xq ∈ R∞
be some given points such that kD−1 (xj − xk )k > 0 for all j, k, and let µb(x) and σb2 (x)
be the estimated moments.
Then we take|b −1 Ubqθb = αb, γb, βb = Σ
q

and



2
σ
b
(x
)
i
i=1
 Pq

Pq
Pq
2
3
b


Σq = 
σ
b
(x
)
σ
b
(x
)
σ
b
(x
)
i
i
i
i=1
i=1
i=1

Pq
P
P
q
q
2
3
4
b (xi )b (xi )b (xi )
i=1 σ
i=1 σ
i=1 σ
1

Pq

b(xi )
i=1 σ



Pq

;

Pq

b(xi )
i=1 µ





bq =  Pq σ
,
U
b
(x
)b
µ
(x
)
i
i
 P i=1
q2
b (xi )bµ(xi )
i=1 σwhere q is finite.
We next derive the limiting distribution of the vector of estimated coefficientsb | , which can be used for conducting statistical inference. Define:θb := (b
α, γ
b, β)
37

0.15

0.12

0.1

0.1

Conditional Mean

0.05
0
-0.05
-0.1
-0.15

0.2

0.08
0.06
0.04
0.02

0.12

0.14

0.16

0.18

-0.02
0.1

0.2

Conditional Standard Deviation

0.15

0.1

Conditional Mean

0.05
0
-0.05

0.12

0.14

0.16

0.18

-0.1

0.14

0.16

0.18

-0.2

-0.4
0.1

0.2

1989/05/25 - 2002/06/24

0.2

0

-0.1
0.1

0.2

0.14

0.16

0.18

0.2

2002/06/25 - 2017/08/31

0.15

0.05

Conditional Standard Deviation

0.12

Conditional Standard Deviation

-0.05

0.12

-0.1

-0.3

0.1

-0.15
0.1

0

Conditional Standard Deviation

1976/05/05 - 1989/05/24

0.15

1963/02/25 - 1976/05/04

0.1

0

-0.2
0.1

Conditional Mean

1950/01/05 - 1963/02/21

Conditional Mean

Conditional Mean

0.14

Conditional Mean

1950/01/05 - 2017/08/31

0.2

0.1
0.05
0
-0.05

0.15

0.2

-0.1
0.1

0.25

Conditional Standard Deviation

0.15

0.2

0.25

Conditional Standard Deviation

Figure 4: Estimated relationship between annualized σ
bt and µ
bt ; p = 12Ω(xi ) =1

Pqi=1 σ(xi )Pq2i=1 σ (xi )




 Pq
Pq
Pq
3
2

Σq = 
σ
(x
)
σ
(x
)
σ(x
)
i 
i
i
i=1
i=1
 P i=1
Pq
Pq
q
4
3
2
i=1 σ (xi )i=1 σ (xi )i=1 σ (xi )!
!
σ 2 (xi )skew(Yt |Xt = xi )
ω1,1 (xi ) ω1,2 (xi )=:,skew(Yt |Xt = xi ) σ 4 (xi ) (kurt(Yt |Xt = xi ) + 2)
ω2,1 (xi ) ω2,2 (xi )


1
0
q
X


|
µ

Vq =
J(xi )Ω(xi )J(xi ) ; J(xi ) =  σ(xi ) 2σ (xi )  .
i=1
σ 2 (xi ) µ(xi )Here, skew and kurt denote skewness and kurtosis of Yt (conditional on Xt = xi ).
The result is a direct consequence of consistency of estimated moments and their
asymptotic independence across i.

Theorem 4 Let Assumptions B2, B3, B5-B9, and D1-D3 hold, and suppose B4 isstrengthened to require E(|Yt |8+δ ) ≤ C < ∞ for some C, δ > 0. Suppose the operatorg(·) = E(Y 2 |X = ·) satisfies Assumption B7. Suppose further that ωa,b (u) is continuous in some neighbourhood of xi for all i. Then, given the sample observations

38

{Yt , Xt }nt=1 specified in A2, we have the following limiting distribution:
r






1−p
2
−1
nh 2p−1 exp −κ00 h− 2p−1
θb − θ − Bθ =⇒ N 0, κ2 (K, p, a)Σ−1
V
Σ
,
q q
q

where Bθ is a bias terms of order hβ , κ2 is the constant defined in Section 2.4.2.

The parameters θ are estimated at the same rate as the functions µ(·) and σ 2 (·).
It may be possible to achieve faster rates of convergence by allowing q → ∞, as iscommonly done in the semiparametric literature, but we have not yet been able toestablish this rate improvement; see Chen and Christensen (2015).
With the same S&P500 data as before and the nonparametric estimates we obtained for p = 4 and reported in Figure 2, we fit the linear regression (49). Note thatthose with zero estimated variance we discussed above are removed (around 3% ofwhole data), since they can make the fitted estimates misleading and spurious. Also,
the standard deviation term is deliberately removed to allow for a direct comparison
with the results from those in the existing literature, Pagan and Ullah (1988), Pagan
and Hong (1990) and Harvey (2001). We estimate the coefficients α and β, and provide the results along with the values of t-statistics that α = 0 and β = 0 in Table
3. The first subperiod is omitted because the estimates for earlier periods may beless reliable due to being evaluated at points with many zeros. Parentheses marked
with asterisks (respectively, double asterisks) mean that the corresponding estimates
are statistically different from zero at 5% level (respectively, 1% level) of significancebased on Newey-West standard errors.
Table 3: Estimated parameters obtained using (252 · µbt , 252 · σbt2 )Full (1954-2017) and Sub Periods
Full1954-1963

1963-1976

1976-1990

1990-2003

2003-20170.072640.10046

0.01672

0.14233

0.09700

0.03761(t) (9.060)∗∗(2.2340)∗(0.4488)(5.0488)∗∗(6.7565)∗∗(2.3038)∗β0.40830-1.64391

2.9360-3.69919

0.44916

2.14394(t)(1.063)(-0.5213)(1.2656)(-2.9391)∗∗(0.6707)(2.3265)∗∗α

The result reports a positive effect (0.4083, with t = 1.063) of conditional varianceon the risk premium during the period of 1954-2017 overall. For the full period we39considered a period starting from 1954 here simply because the federal rate series,
which we analyse together with later, is available only from 1954. Including theperiod of 1950-1954 does not change the conclusion. Over the periods of 1963-1976
and 2003-2017 the risk-return relationship is strongly positive, and in the later period
the estimate is statistically significant at 1% level. In fact, the estimated risk averse
parameter β is 1.41294 (with t = 2.7317) on the last two subperiods combined (i.e. theperiod of around past 30 years), revealing evidence of strongly positive and statisticallysignificant risk-return relation in the recent time after 1990. We may compare theseresults with the findings of Pagan and Hong (1990, page 61), where they reacheda different conclusion with Monthly CRSP data over 1953-1984. They reported theestimated coefficient for conditional variance of −0.87 (with t = −0.35). The estimatedrisk aversion parameter β using our conditional expectations estimates over 1953-1984
is −0.07418 (with t = −0.0790), reporting a negative but much weaker risk-return
relation.
To investigate how the analysis we adopt in our method may have made any
difference, we repeat the same step above by computing nonparametric estimateswith using only one lag as conditioning variable. This is to replicate what was done in
the papers computed fitted means and variances based on nonparametric regression
approaches, e.g. Pagan and Hong (1990). The local constant estimation is done withthe standard Gaussian kernel and the bandwidth chosen via cross-validation. We
denote by those fitted conditional mean and variance (eµt , σet2 ), and report the leastsquares estimates for the parameters α and β below in Table 4.
Table 4 reveals a very strong and persistent negative risk-return relation throughout all time periods. Over the full period, the estimated risk aversion parameter β isaround −3.53, and this is statistically significant at 1% level based on Newey-West
Table 4: Estimated parameters obtained using (252 · µet , 252 · σet2 )Full (1954-2017) and Sub Periods
Full1954-1963

1963-1976

1976-1990

1990-2003

2003-2017

α0.149770.15097

0.156470.116930.10808

0.19830(t)(4.9068)∗∗(2.0964)∗(4.0584)∗∗(2.6367)∗∗(2.2444)∗∗(2.3897)∗∗β-3.33228-3.14701-4.30003-2.38936-0.77965-5.13079(-0.8642)∗(-1.0904)( -0.3297)(-1.4274)(t)∗∗(-2.2834)(-2.0728)

standard errors. This result implies that the conclusion Pagan and Hong (1990) obtained may have been influenced by the fact that they conditioned only on small, fixed40

lags (when forming the nonparametric estimates). In other words, incorporating further information that are neglected in estimating conditional expectations has clearly
led to some new empirical findings. This provides explanations to the conjecture
Pagan and Hong (1990) raised in their paper.

5.2Time variation and counter-cyclicity in risk aversion

Meanwhile, the results in Table 3 suggest that the risk-return relationship is strongly
time-varying. In particular, over the subperiod 1976-1990 the estimate of β was significantly lower than the other periods. To take a closer look, we conducted a rolling
regressions analysis. We set the rolling window to be 4000, roughly a quarter of thenumber of whole sample, and start estimating β from 1958:09:18. That is, we use
conditional expectations estimates over 1958:09:18-1974:12:04 to estimate β for date1974:12:05, and roll forward the window by one every time. The window size is deliberately chosen to be different from the size of 5 subperiods; this is to check if ourprevious results in Table 2 are driven by a particular choice of sample size. The timeseries of estimated parameter β shown below in Figure 5 provides an evidence thatinvestor’s average risk aversion has been varying over time.
Furthermore, we observe that interestingly, the time series of risk aversion tends
to move in the opposite direction to the federal funds rate7 ft , which is a proxy forthe business cycle fluctuations, see Figure 5. In fact, the sample correlation between
βbt and ft turns out to be −0.5673, implying that the risk aversion exhibits a countercyclical behaviour. Also, in Figure 6 we plot the time series of quarterly Sharpe ratio
and the designated recession periods by the NBER. The blue line is the ratio computed
using our estimates (bµ, σb2 ), and the red line is the one computed using the standard
nonparametric method (eµ, σe2 ). The shadings show that blue line rises over the periodof recession in general, which is a finding that is consistent with Lettau and Ludvigson(2010). Note that the red line does not behave as expected in most cases and therefore
does not quite capture counter-cyclicity.
7

Data taken from Federal Reserve Bank of St. Louis http://fred.stlouisfed.org

41

25
4

Estimated Risk Aversion
Federal Funds rate

20
2
15
0
10

-2
5

-4
1974/120
1983/01

1991/04

1999/06

2007/122017/08

Time

Figure 5: Estimated risk-return tradeoff and the federal funds rate
These findings are consistent with what is suggested and widely discussed in thefinance literature, for example Antell and Vaihekoski (2016), Campbell and Cochrane(1999), Bliss and Panigirtzoglou (2004), Smith and Whitelaw (2009), Bollerslev, Gibson and Zhou (2011), and Guo, Wang and Yang (2013).
As noted in Mehra (2012), empirical evidence for the financial theory suggesting
counter-cyclical risk return tradeoff is rather scarce and limited. Cohn et al. (2015)
wrote, “A key ingredient of many popular asset pricing models is that investors exhibit
countercyclical risk aversion, which helps explain major economic puzzles such as thestrong and systematic variation in risk premiums over time and the high volatility ofasset prices. There is, however, surprisingly little evidence for this ...”
Our findings on the time series dynamics of risk return tradeoff and their link withthe macroeconomy add a supporting empirical evidence to this issue. We reiterate thatwhen standard nonparametric method is employed, these evidence is not well revealed.
This potential improvements in the econometric analysis are possibly attributable tothe extended flexibility and the inclusion of otherwise neglected information in our
method.

42

43

-1
1950 Q1

-0.5

0

0.5

1

1.5

2

1957 Q2

1964 Q41972 Q2

Time1979 Q4

1987 Q2

1994 Q4

2002 Q2

Figure 6: Quarterly sharpe ratio (1950-2016) computed based on (bµ, σb) and (eµ, σe), respectively.
The shades are the recession periods designated by the National Bureau of Economic Research (NBER).

Conditional Sharpe ratio

2009 Q4

2016 Q4

5.3

Practical methods for choosing bandwidth

Lastly, we discuss some possible ways for selecting the bandwidth in practice. A natural choice would be direct extensions of two most extensively adopted approaches in themultivariate nonparametric regression, namely, a rule-of-thumb and cross-validation,
Green and Silverman (1993), Fan and Gijbels (1996).
We first consider a heuristic plug-in method for obtaining a rule-of-thumb bandwidth. For simplicity, we suppose that the regressors X are i.i.d. Gaussian distributed.
This way the small ball probability takes a simple form as discussed in previous sections. Further, this bypasses the need to estimate the CA term, a function of spectral
density representing the degree of dependence between regressors. With the uniform
kernel supported on [0, 1] i.e. λ = 1, the asymptotic mean squared error of our estimator is then given by
AMSE(m)b = h2β∞X!2cj j pβ+j=1

σ 2 (x)1−p2

nh 2p−1 exp − κ00 h− 2p−1

.

(50)

Denote by C the squared term in (50), and let β = 1. Now, differentiating (50) withrespect to h and equating it to zero we have


− 2
p
p+2
σ 2 (x)∂{AMSE(m)}b− 2p−1− 2p−1
κ0 h 2p−1= 2Ch +·e(p − 1)h− 2κ0 h=0∂hn(2p − 1)


p+2
p
2 
2n(2p − 1)Ch− 2p−1− 2p−1− 2p−1⇔= exp κ0 h
· 2κ0 h− (p − 1)hσ214n · C
−1
−2/7
−6/7
−4/7
= h exp 3.605h
⇔
7.21h
− 3h
(51)
σ2
where in the last line we substituted p = 4 and κ0 = C ∗∗ ≈ 3.605 (follows from astraightforward computation; see definitions in Section 4.2.2). As we can substitutethe sample variance σ
b2 into σ 2 , it now suffices to replace the squared term C with asuitable estimate.
To proceed, we impose a further model assumption and suppose m(x) =P∞

j=1 αj |xj |and |αj | ≤ cj (= Cθj for some 0 < θ < 1 and constant C 8 ). In this case, Assumption
B7 in Section 2.4.2:
0m(x) − m(x ) ≤∞X

cj xj − x0j

j=1

is satisfied via the reverse triangle inequality. A heuristic idea is to choose some C
This is a reasonable assumption because cov(Yt , Yt−k ) = O(k −ck ) for some c under Assumption
A2 and by Davydov’s inequality for covariance of mixing sequences.
844

and θ in such a way that cj ’s bound statistically significant estimates of αj ’s. Fitting
P15
the linear model upto lag 15, say: Yt =j=1 αj |xj | + εt , the estimates of αj atlags 1, 2, 9, 15 are given by 0.058, 0.0443, 0.0435 and 0.027, respectively. Therefore, we
could let C = 0.1 and θ = 0.95 for example; substituting these values back into thefirst order condition (51) yields
∞

14n
σ
b2

1 X
(0.95)j j 4
10 j=1!2=u

7/2 3.605u



3

2



7.21u − 3u ,ewhere u = h−2/7 , σb2 ≈ 9.3557 × 10−5 and n = 16820. Numerical approximation via
Matlab yields h = 0.000312.
An alternative approach would be the cross-validation, where we search for the
bandwidth that minimises the mean squared leave-one-out residuals:n

1X[Yt − mb h,−t (Xt )]2 ,n t=1g(h) :=(52)

where mb h,−t (Xt ) is the estimate obtained by ignoring tth sample. The result, as illustrated in Figure 5, suggests h = 0.00035 and h = 0.000125 for p = 4 and p = 12,
respectively. These bandwidths are the ones we used earlier in the example in thissection. Note that Yao and Tong (1998) proposed an different leave-one-out method
for choosing optimal bandwidth for dependent data. When we applied their crossvalidation for the data we considered previously however, we noticed that it suggests
way lower optimal bandwidths, and the standard approach gives a more reasonable
result. This might be because the returns data we consider is almost uncorrelated but
nonlinearly dependent.
9.6

10 -5

p=4

10.1

10 -5

p = 12

10
9.4

9.9
9.2
9.8

9

g(h)

g(h)

9.7

8.8

9.6

9.5
8.6

9.4
8.4
9.3

8.2
9.2

25

00
35
0.
0

5

03
00

00
03
0.

00
27

0.

5

02
5
00
0.

0.
0

02
00

00
22

0.

0.
0

01
5

01

01
75

00

00
0.

0.

00

01
25

0.

00
0.

00

5

00
75

00

00
0.

1
00
00

00
0.

0.

h

0.

05
00
0.

04

5

04
75

00

00

0.

0.

04
00

5

04
25
00

0.

0.

03

03

03
75
00

0.

00
0.

00

5

03
25
00

0.

0.

02

02

02
75

00

00
0.

0.

00

02
25

0.

00
0.

01

5

01
75
00

0.

00
0.

00
25

9.1

8

h

Figure 7: Cross validation choice of bandwidth given p = 4 and p = 1245

6Some concluding remarks

In this paper we studied the nonparametric estimation problem of the infinite order
regression. While we answered several open questions raised in the literature, there aresome remaining questions that we leave for future studies from a methodological pointof view. First, it is not clear how the conclusions we obtained will be changed whenthe marginal bandwidth is set to decay in a way other than polynomially. It is also a
non-trivial question whether the geometric mixing condition in the uniform consistency
result could be relaxed to allow weaker dependence of the data. Furthermore, as Linton
and Sancetta (2009) pointed out, it may be possible to achieve algebraic convergence
rates for some restricted class of functions. For example, we conjecture that given thePadditive regression model: E(Y |X = x) = m(x) = ∞j=1 mj (xj ) where x = (xj )j ∈R∞ , the rate for estimating mj (·) and m(·) would be the same, just as it was provento be so in the d-dimensional case (i.e. mj (·) = 0, ∀j > d ∈ Z+ ) by Stone (1985). In
this paper, we were unable to find an answer to this question, although we found theexistence of the curse of infinite dimensionality under a wide range of frameworks weconsidered. We leave the question for future study.
Lastly, it would be interesting to come up with a practical way to choose the parameter p or more generally the rate at which the bandwidths expand in the order ofthe covariates. This should relate to the rate of decay of influence (mixing in the autoregression case) that prevails, and perhaps this can be addressed by using tools fromthe estimation of memory properties. An alternative approach is to us a penalization
method combined with cross-validation, namely, the Bandwidth-LASSO method thatP
adds the penalty λ i |φ−1
i | to the objective function (52). The positive numbers φi are
the bandwidth weights defined previously in Section 2.4 and in Assumption B8. The
resulting choice of {φ−1i } will contain many zeros (infinite smoothing of one covariate)
depending on the tuning parameter λ, which would give a much more parsimonious
representation. The properties of this method will be investigated in the sequel.
Other quantities of interest in prediction such as the conditional median or mode
can also be studied. This could be done via nonparametrically estimating the conditional distribution P (Y ≤ y|X = ·) = E(1{−∞, y](Y )|X = ·), but would necessarily
require a slightly different set of assumptions. It is also quite easy to bring finite
dimensional predictors into the theory separately. For example, one may want toallow for slow time variation whereby t/T becomes an additional covariate and theregression function is m(x, u) with u ∈ [0, 1] and x ∈ R∞ . In this case we modify theestimator of (18) by introducing a multiplicative kernel of the form kb (u − t/T ), whereb is a bandwidth and k is a symmetric probability density function.
46

7

Appendix: Proofs of the main results

7.1

Proof of Theorem 1

Proof. From the decomposition in (21):m(x)
b− m(x) =b 2 (x) − E m
b 2 (x) m(x)[mb 1 (x) − 1]Em
b 2 (x) − m(x) m+−,mb 1 (x)mb 1 (x)mb 1 (x)we see that it suffices to show E m
b 2 (x) − m(x) → 0 and mb 2 (x) − E m
b 2 (x) →P 0, sincemb 1 (x) →P 1 will then follow from the latter and complete the proof.
We first consider the “bias component”. It is straightforward to see
!
n
1 X
Em
b 2 (x) − m(x) = E
Kt Yt − m(x)nEK1 t=1
 


1
EK1
1
=
EK1 Y1 −m(x) =E E Y1 − m(x) K1 X
EK1
EK1
EK1
hh
i i
1
E m(X) − m(x) K1 ≤ sup m(u) − m(x) −→ 0=
EK1
u∈E(x,λh)(53)

as n → ∞, where Kt is the shorthand notation for K(kH −1 (x − Xt )k) and E(x, λh)is the infinite dimensional hyperellipsoid centred at x = (xj )j ∈ R∞ with semi-axes
hj in each direction as introduced in the main text before. The second equality isjustified by stationarity that is preserved under measurable transformations, and thelast inequality is due to compact support of the kernel and continuity of the regression
operator at x (Assumption B1).
The next step concerns with the latter ‘variance component’ mb 2 − Emb 2 . We show
its mean-squared convergence to zero. Writing


n
n
1X 1
1X
m
b 2 − Emb2 =Kt Yt − E(Kt Yt ) =:Qnt ,n t=1 EK1
n t=1

(54)

we remark that the arguments to follow depend upon the temporal dependence structure of Qnt . In the static regression case, Qnt is a measurable function of Yt , X1t , X2t , . . .,and hence inherits their joint dependence structure. That is, Qnt is arithmetically αmixing with the rate specified in A1. In the dynamic regressions case (which covers
the autoregression framework), the dependence of Qnt is defined via Kt which is near
epoch dependent on (Yt , Vt ) as specified in Assumption A2; this bypasses the issue of
Qnt being dependent upon infinite past of Yt and/or Vt . We proceed with these twocases separately.
47

Case 1: Static Regression. Clearly, it is sufficient to prove var(mb 2 − Emb 2) → 0
for the mean squared convergence. Since Qnt is stationary over time we have
n

1 X
2 XX
var(Q
)
+
cov
Q
,
Q
nt
ni
nj
n2 t=1n2 1≤i<j≤n
1
2 XX= var(Qn1 ) + 2
cov Qni , Qnjnn 1≤j−i<nvar(mb 2 − Emb 2) =

(55)

n−1


2 X
1
(n − s) · cov Qn1 , Qn,s+1 =: A1 + A2 .
= var(Qn1 ) + 2
n
n s=1

(56)

Now, by (9), (11) and Assumption A it follows that
1var (K1 Y1 )
A1 =
var
K
Y
−
EY
K
=
1
1
1
1
nE 2 K1
nE 2 K1
2
2 2
2
EK1 Y1
E(E(Y1 |X1 )K1 )C→ 0 as n → ∞.
≤
=
≤
2
2
nE K1
nE K1
nϕx (λh)(57)

We now move on to the second term A2 and investigate the covariance term. Since
measurable transformations of mixing variables preserve the mixing property, using
Davydov’s inequality, see Davydov (1968, Lemma 2.1) or Bosq (1996, Corollary 1.1),
and stationarity we havecov Qn1 , Qn,s+1



K1
Ks+1
= cov Y1
, Ys+1
EK1
EK1

!

2

C{E|Y1 K1 |2+δ } 2+δ
≤.
ϕx (hλ)2 · skδ/(2+δ)(58)

In the meantime,!

cov Qn1 , Qn,s+1



K1
Ks+1
, Ys+1
EK1
EK1
!
!
!
K1
Ks+1
K1
Ks+1
≤ E Y1Ys+1+ E Y1
E Ys+1
EK1
EK1
EK1
EK1

= cov Y1

C
C0
|E
(K
K
)|
+
|E (K1 ) E (Ks+1 )|
1 s+1
ϕx (hλ)2E 2 K1C≤· ψx (λh; 1, s + 1) + C 0 ≤ C 00
ϕx (hλ)2≤(59)

by stationarity, law of iterated expectation, boundedness of regression function, andAssumption B6, B5 (along with the upper bound ψ(λh; 1, s + 1) of EK1 Ks+1 obtained
as a direct consequence of B5 following similar arguments used for Lemma 1).

48

With reference to (58) and (59), we take some increasing sequence un → ∞ suchthat un = o(n), and writen−1X

cov Qn1 ,Qn,s+1

s=1



=

uX
n −1

cov Qn1 , Qn,s+1



n−1
X

+

s=1

cov Qn1 , Qn,s+1



s=un
n−1
X


−kδ/(2+δ)+1 unCs−kδ/(2+δ),= O un +≤ C un − 1) +ϕx (hλ)2
ϕx (hλ)2
s=u
00

(60)

n


which is O ϕx (hλ)−2(2+δ)/(kδ) upon choosing un ∼ ϕx (hλ)−2(2+δ)/(kδ) .
Consequently, since k ≥ 2(2 + δ)/δ it follows that

n−1
n−1 
 2X

s
2 X
(n − s) · cov Qn1 , Qn,s+1 =1−· cov Qn1 , Qn,s+1
A2 := 2
n s=1
n s=1
n

= O n−1 [ϕx (hλ)]−2(2+δ)/(kδ) + n−2 [ϕx (hλ)]−2(2+δ)/(kδ)
= O n−1 [ϕx (hλ)]−2(2+δ)/(kδ) = o(1)(61)
by Assumption B2, and the desired result is obtained.
Case 2: Dynamic Regression.9 We return back to (54):


n
n
1X 1
1X
Qnt .
m
b 2 − Emb2 =Kt Yt − E(Kt Yt ) =:n t=1 EK1
n t=1

(62)

In this framework Kt = K(kH −1 (x−Xt )k) is a (measurable) function of (Yt−1 , Yt−2 , . . .).
Despite loosing the mixing property, Kt inherits stationarity of the mixing process{Yt }. We write Kt,(r) = Ψ(Yt , Yt−1 , Yt−2 , . . . , Yt−r+1 ) = E(Kt |Yt , . . . , Yt−r+1 ) with r as
in Assumption A2, and the measurable map Ψ. Then, Kt,(r) preserves the mixing
dependence structure of Yt with mixing coefficient α(` − (r − 1)) since σ(Ks,(r) ; s ≥t + `) ⊂ σ((Ys , . . . , Ys−r+1 ); s ≥ t + `) = σ(Ys ; s ≥ t + ` − (r − 1)).
Now write




n
n

1X 1
1X 1
m
b 2 − Emb2 =Kt,(r) Yt − E Kt,(r) Yt +Kt Yt − Kt,(r) Yt
n t=1 EK1
n t=1 EK1


n

1X 1
+
E Kt,(r) Yt − E(Kt Yt ) = R1 + R2 + R3 ,(63)
n t=1 EK1
9

For the sake of notational simplicity, we will write the proofs for the dynamic regression frameworkin terms of its autoregressive special case throughout the appendix. That is, some lags of the response
variable Yt here possibly represent lagged covariate Vt .

49

and first consider the last term R3 . Fix some increasing sequence q = qn → ∞, andwrite Yt,L := Yt 1{|Yt | ≤ q} and Yt,U = Yt 1{|Yt | > q}. Then

EYt Kt,(r) = EYt K kH −1 (x − Xt )k − EYt,U K kH −1 (x − Xt )k+ EYt,L Kt,(r) − EYt,L K kH −1 (x − Xt )k+ EYt,U Kt,(r) = D1 + D2 + D3 .

(64)

The second part of D1 is given by
EYt,U K kH −1 (x − Xt )k ≤ E|Yt |1{|Yt |>q} K kH −1 (x − Xt )k≤ q −(δ+1) E|Yt |2+δ 1{|Yt |>q} Kt ≤ Cq −(δ+1) E|Yt |2+δ 1{|Yt |>q} = o(q −(δ+1) )(65)

because 1{|Yt |>q} = o(1) as n → ∞. Following similar arguments on D3 we haveD1 + D3 = EYt Kt + o(q −(δ+1) ). So we are now left with the middle term D2 : p
D2 ≤ E |Yt,L | Kt − Kt,(r) = O q v2 (rn )(66)

by Hölder’s inequality. Therefore, from (64), (65) and (66) we see that

n 
1 X
R3 =
EKt,(r) Yt − E(Kt Yt )nEK1 t=1!
p −(δ+1) 
q v2 (rn )
q
+O
,
=o
ϕx (λh)ϕx (λh)(67)

and upon choosing q = (ϕx (hλ)/n)−1/(2(δ+1)) we have o(ϕ−1 q −(δ+1) ) = o(ϕ−1 (ϕ/n)1/2 ) =o(n−1/2 ϕ−1/2 ) = o(1). Furthermore,

O


p
1
q v2 (rn ) = O
ϕx (hλ)=O!
−1/(2(δ+1))p
ϕx (hλ)v2 (rn )n!
pv2 (rn )= o(1)[ϕx (hλ)](2δ+3)/(2δ+2) n−1/(2(δ+1))1·
ϕx (hλ)by Assumption A2, yielding R3 = o(1) and consequently R2 = op (1).

50

(68)

As for the first term that remains,


n 
n 
1 X Kt,(r) Yt − E(Kt Yt )1 X E(Kt Yt ) − E(Kt,(r) Yt )
R1 =
+
n t=1
EK1
n t=1
EK1
n


1X
=
E Qnt |Yt , Yt−1 , . . . , Yt−r+1 − R3
n t=1
!
p
 −(δ+1) nv2 (rn )1X
q=Qnt,(r) + o
+O
.
n t=1
ϕx (hλ)[ϕx (hλ)](2δ+3)/(2δ+2) n−1/(2(δ+1))(69)

Since Qnt,(r) is α-mixing, we can work with the first term by following similar arguments in the regression case. Specifically, due to boundedness of the kernel and themixing properties, the bound in (58) can be constructed. As for the constant boundconstructed in (59), we rewrite
cov Y1 K1,(r) , Ys+1 Ks+1,(r)cov Y1 [K1,(r) − K1 ], Ys+1 [Ks+1,(r) − Ks+1 ]=ϕx (λh)2
ϕx (λh)2
cov Y1 [K1,(r) − K1 ], Ys+1 Ks+1,(r)+ϕx (λh)2
cov Y1 , Ys+1 [Ks+1,(r) − Ks+1 ]cov Y1 K1 , Ys+1 Ks+1++
ϕx (λh)2
ϕx (λh)2= G1 + G2 + G3 + G4 .
The fourth term G4 ≤ C by (59). Further,
G1 ≤E(Y1 Ys+1 [K1,(r) − K1 ][Ks+1,(r) − Ks+1 ])ϕx (λh)2
E(Y1 [K1,(r) − K1 ]) · E(Ys+1 [Ks+1,(r) − Ks+1 ])v2 (r)+≤ C0→02
ϕx (λh)ϕx (λh)2

by Assumption B6 and by the fact that!
pv2 (rn )≤ϕx (hλ)!
pv2 (rn )· (n/ϕ)1/(2δ+2) −→ 0
ϕx (hλ)by (19) in Assumption A2. Following similar steps it can be shown that G2 and G3
converge to zero.
Now choosing an increasing sequence un ∼ [ϕx (hλ)−2(2+δ)/(kδ) + rn ] → ∞ such that51

rn /un = o(1), we see that (ignoring the array notation in Qnt,(r) for simplicity)n−1
X

cov Q1,(r) , Qs+1,(r)

s=1



=

uX
n −1

cov Q1,(r) , Qs+1,(r)



+

s=1
2(2+δ)
≤ C ϕx (hλ)− (kδ) + rn +n−1
Xcov Q1,(r) , Qs+1,(r)



s=un
n−1
X



2(2+δ)C(s − rn + 1)−kδ/(2+δ)− (kδ)
,
=
O
ϕ
(hλ)
x
2
ϕ
(hλ)
x
s=u
n

since the mixing coefficient for Qnt,(r) denoted α0 (n) is given by α(n−(r−1)) for n ≥ r.
It now follows by the same arguments in (61) that the first term in (69) converges tozero, yielding R1 = op (1), which is the result we desired.

7.2

Proof of Theorem 2 and 3

Proof of Theorem 2 and 3. We start by recalling the bias component (53).
Additional assumptions B7, B8 and D3 allow us to proceed further as follows:n
1 X
Bn (x) = E m
b 2 (x) − m(x) = E
Kt Yt − m(x)nEK1 t=1

 

1
EK1
1
=
EK1 Y1 −m(x) =E E Y1 − m(x) K1 X
EK1
EK1
EK1
hh
i i
1
=
E m(X) − m(x) K1 ≤ sup m(u) − m(x)EK1u∈E(x,λh)
 X

∞
∞
∞
X
X
β
β
β
β
pβ
≤ sup
cj u j − x j =cj (λhφj ) = h λ
cj j
< ∞.


u∈E(x,λh) j=1

j=1

(70)

j=1

Now rewriting the decomposition (21) asm(x)−m(x)b− Bn (x)mb 2 (x) − E m
b 2 (x) − m(x) mb 1 (x) − 1
Bn (x) · 1 − m
b 1 (x)+,=mb 1 (x)mb 1 (x)and noting that mb 1 (x) →p 1 (an immediate consequence of Theorem 1), we see thatit suffices to derive the limiting distribution ofmb 2 (x) − E m
b 2 (x) − m(x)[mb 1 (x) − 1]nni1X 1 h1X=
Kt Yt − m(x)Kt − E(Kt Yt ) + m(x)EKt =:Rnt .
n t=1 EK1
n t=1

(71)

In the rest of the proof, the way how we construct the general CLT under Assumption52

A1 is quite similar to the proofs of theorems in Masry (2005), where asymptotic normality is established in a functional context for mixing data sample. For completenessof the proof however, we will go over some of the main arguments; some relatively lessimportant details will only be briefly sketched to prevent being repetitive.
By Assumption B6, B8, D3, and the law of iterated expectations, the asymptotic
variance of the triangular array Rnt is given by
var[Kt (Yt − m(x))]
E 2K
 h1
i2
i

1
2
E Kt (Yt − m(x)) − E Kt (Yt − m(x))
= 2
E K1
)
(
h
i2 

 2
1
' 2
E σ (X)K12 + E m(X) − m(x) K12
E K1
)
(
h
i 
1
= 2
σ 2 (x)EK12 + E σ 2 (X) − σ 2 (x) K12 + o(1)EK12E K1var(Rnt ) ==EK12 2σ 2 (x)ξ2
(σ
(x)
+
o(1))
'
.
E 2 K1ϕx (hλ)ξ12(72)

Using the latter assumption of B9 and Assumptions B, and following similar arguments
as in the above and those in the proof of Theorem 1, it can be readily shown that thecovariance term is of negligible order, which together with (72) shows (29).
Meanwhile, under Assumption D2 the small ball probability can be written interms of the centered small deviation and p∗ (·), the Radon-Nikodym derivative of theinduced probability measure Pz−Z with respect to PZ :ϕx (λh) = P X ∈ E(x, λh)
X

∞


2
−2p
2 2
=P
j
xj − Xj ≤ h λ = P kz − Zk ≤ hλ
j=1

Z
=

Z
dPz−Z (u) =B(0,hλ)p∗ (u) dPZ (u)B(0,hλ)' p∗ (0) · P (kZk ≤ hλ) = p∗ (0) × PnX!
j −2p Xj2 ≤ h2 λ2 .

(73)

j=1

Given that the fourth moment of Xj is finite by Assumptions C, the latter probabilityin (73) can be explicitly specified by substituting r = h2 λ2 , A = 2p, and a = 2p/(2p−1)
in Proposition 4.1 of Dunker et al. (1998) for the i.i.d. case. When the marginal
regressors are dependent as in Assumption C, the small ball probability can be specified(by letting r = h2 λ2 CA−2 and leaving the others the same) in view of Theorem 1.1 of53

Hong, Lifshits and Nazarov (2016). In the general i.i.d. case (under Assumptions A1
and independence across marginal covariates) we haveσ 2 (x)ξ21
σ 2 (x)ξ2 C ∗ C`,=··
ϕx (hλ)ξ12
φ(h) p∗ (0)ξ12 λ 1+2ρp2p−1
where φ(h) = h(1+2ρp)/(2p−1) exp{−C ∗∗ (λh)−2/(2p−1) } andh−1/2C` = lim `
h→0



4p
− 2p−1

h

i

∗

C =

(2π)(1+2pρ) (2p − 1)
Γ−1 (1 − ρ) · (2p)2p(ρ+2)−1
2p−1

2p(1+ρ)· ζ 2p−1 .

Γ(·) is the Gamma function, ξ1 and ξ2 are the constants specified in (12), and λ is theupper bound of the support of the kernel. The constants for the dependent case canbe specified similarly.
In constructing the central limit theorem we consider the normalized statisticp√∗∗:= φ(h) · Rnt and derive the limiting distribution of (1/ n) · Rnt. We shallRnt
prove under Assumption A2 as it involves some further arguments, without which theproof just serves as the proof under Assumption A1. We make use of the standard
Bernstein’s blocking method and partition {1, . . . , n} by 2k(= 2kn → ∞) numberof blocks of two different sizes that alternate (hereafter referred to as the “big” and“small” blocks) and lastly a single block (the “last block”) that covers the remainder.
The size of the alternating blocks is given by an and bn respectively, where the one forthe “big-blocks” an is set to dominate that for the “small-blocks” bn in large sample,pi.e. bn = o(an ). Specifically, take kn = bn/(an + bn )c and an = b nφ(h)/qn c,
where qn → ∞ is a sequence of integer; it then clearly follows that an /n → 0 andpan / nφ(h) → 0. We also assume (n/an ) · α∗ (bn ) = (n/an ) · α(bn − r + 1) → 0, wheret−1∗∗α∗ is the mixing coefficient of Rnt,(r)= E(Rnt|Ft−r+1).
√ −1 Pn∗
By construction above we can write n
t=1 Rnt as the sum of the groups of

big-blocks B, small-blocks S and the remainder block R defined as
j(a+b)+a
k−1
k−1
X
X
X
1
1
∗ 

B := √
Ξ1,j = √
Rnt
n j=0
n j=0
t=j(a+b)+1(j+1)(a+b)
k−1
k−1
X
X
X
1
1
∗ 

Ξ2,j = √Rnt
S := √n j=0
n j=0
t=j(a+b)+a+1


n
X
1
1
∗ 
R := √ Ξ3,j = √ 
Rnt
.
n
n
t=k(a+b)+1

54

The aim is to show that the contributions from the small and the last remaining
block are negligible, and that the big-blocks are asymptotically independent. Consider∗∗|Yt , . . . , Yt−r+1 ),= E(Rnt
the big blocks B. Given r as in Assumption A2, and Rnt,(r)

k−1
X



1

B=√
n j=0

j(a+b)+aX
t=j(a+b)+1



k−1
X



∗
 + √1
Rnt,(r)n j=0

j(a+b)+a



X

 ∗

∗ 
Rnt,(r) − Rnt= Q1 + Q2 .

t=j(a+b)+1

As for the second term, consider
k−1 j(a+b)+a
1
1 X X
∗
∗
√ EQ2 ≤ √− Rnt
E Rnt,(r)nn j=0
t=j(a+b)+1

k−1 j(a+b)+a
1 1 X X√
E Kt Yt − Yt E(Kt |Yt , Yt−1 , . . . , Yt−r+1 )=EK1 n j=0
t=j(a+b)+1

k−1 j(a+b)+a
X
X

1
1
≤ √
n ϕx (hλ) j=0

E|Yt ||Kt − Kt,(r) |t=j(a+b)+1

k−1 j(a+b)+a
1/2 
1/2
X
X 
1
1
≤ √
E|Yt |2
E|Kt − Kt,(r) |2n ϕx (hλ) j=0
t=j(a+b)+1!
ppv2 (rn )n · v2 (rn )1=O= o(1),≤ C · √ kn an
ϕx (λh)ϕx (λh)nwhich implies that√ −1
n Q2 = op (1).

We now show asymptotic independence of terms in Q1 , on noting that Ξ01,j s are
independent if for all real tj

E

" k−1
X

exp itj Ξ1,j

#


−

j=0

k−1
Y



E exp(itj Ξ1,j )

(74)

j=0

is zero. Applying the Volkonskii-Rozanov inequality (see Fan and Yao (2003, page 72)
for example), it can be shown that (74) is bounded above by C(n/an )·α(bn −r+1) → 0,
implying asymptotic independence.

55

Moving on to the small blocks, due to stationarity we have


k−1
X



(j+1)(a+b)

1
∗ 
var 
Rnt
n
j=0 t=j(a+b)+a+1
(j+1)(a+b)(j+1)(a+b)
k−1
k−1
X
X
X
X
X
1
1
∗ 
∗


Rnt +
=
var
Rnt
,
cov
n j=0
n j6=lvar (S) =X

t=j(a+b)+a+1

=

k−1 
X

1
n j=0

∗
bn var(Rnt
)+

bn
X

t=j(a+b)+a+1



∗∗
cov(Rnt
, Rnl
) +

t6=l

1
n

k−1
bn
X
XX
j6=l



(l+1)(a+b)

X

∗ 
Rns

s=l(a+b)+a+1∗∗
cov Rn,i+w, Rn,r+w
j
l



i,j=1

= Q1 + Q2 + Q3 .
where wj = j(a + b) + a.
Regarding the first term, similar arguments used in deriving (72) yield2
ϕx (hλ)1/2 σ 2 (x)ξ2
kn bn σ 2 (x)ξ21=−→ 0
Q1 = kn bnnϕx (hλ)ξ12
nξ12

(75)

because kn bn /n ∼ bn /(an + bn ) → 0. Now moving on to Q2 and Q3 , the sum ofcovariances can be dealt with in the same manner as we did for the variance using(72), so Q2 → 0. Similarly for Q3 , implying var(S) → 0 as desired. Convergence result
for the remainder R can be established similarly, and is bounded by C(an +bn )/n → 0.
The results above suggest thatj(a+b)+a
n
k−1
k−1
X
X
X
1
1
1 X
∗
∗ 

√
R =√
Rnt + op (1) = √ηj + op (1),n t=1 nt
n j=0
n j=0

(76)

t=j(a+b)+1

and the desired result holds in view of (62) and the CLT for triangular array uponchecking the Lindeberg condition (which is omitted here due to its similarity with
Masry (2005, page 174-175)). Corollary 2 now follows because! √ PnPn∗∗√1
n n1 t=1 Rntm
b − m − Bn
t=1 Rnt
n
p
=q P
= q P
∗,2
1
1
nφ(h)∆nb∗,2t Rnt + op (1)
t Rnt
n
n
Pk−1 Pj(a+b)+a
Pk−1
∗
√1
√1
R
+
o
(1)
p
nt
j=0
j=0 ηj + op (1)t=j(a+b)+1
n
n
=r
=q P
=⇒ N (0, 1)


2
k−1 2
1
P
P
j(a+b)+ak−1
1∗
j=0 ηj + op (1)n+ op (1)j=0
t=j(a+b)+1 Rnt
n

p
nφ(h)

(77)

by Theorem 4.1 of de la Peña et al. (2009), since the denominator converges inprobability to a strictly positive quantity (σ 2 (x)ξ2 /ξ1−2 ), and that ηj belongs to the56domain of attraction of a normal distribution by definition and (76).

7.3

Proof of Lemmas 1 and 2

Proof. Lemma 1 is a straightforward extension of Lemma 4.3 and 4.4 of Ferraty andVieu (2006), and hence is omitted. Lemma 2 can be shown by noting that for eachn the τn -dimensional polyhedron D := {w = (wi )i≤τ ∈ Rτ , |wi | ≤ λ} can be covered√by ([2λ τ /ε + 1])τ number of balls of radius ε, see Chaté and Courbage (1997), andthen following the arguments of the proof of Theorem 2 in Jia et al. (2003).

7.4

Proof of Theorem 4 and 5

Proof of Theorem 4. In the sequel, we omit the subscript τ in the notations fortruncated regressor and its estimator, i.e. mτ (·) and mb τ (·) for notational simplicity.
As before, we start from the decomposition (21):
!
h
i h
i
h
i
1
m
b 2 (x) − E m
b 2 (x) + E m
b 2 (x) − m(x) − m(x) mb 1 (x) − 1 .
m(x)b− m(x) =mb 1 (x)We recall from (73) that ϕx (λh) ∼ ϕ(λh). Further, notice that the small deviation for
the truncated regressor X = (X1 , . . . , Xτ , 0, 0, . . .) denoted ϕT (λh) satisfies
ϕ(λh) = P∞X!
j −2p Xj2 ≤ h2

≤P

j=1

τ
X

!
j −2p Xj2 ≤ h2

= ϕT (λh).

(78)

j=1

Note that as implicitly mentioned in the main text, (42) is meant to hold for ϕT (λh).
In the first step of the proof we showssup mb 2 (x) − E m
b 2 (x) = OPx∈Sτ(log n)2nϕ(λh)!
.

(79)

We cover the set Sτ defined in (40) with L = L(Sτ , η) number of balls of radius ηS ndenoted by Ik , each of which is centred at xk , k = 1, . . . , L. i.e. Sτ ⊂ Lk=1
B(xk , η).

57

Then it follows thatsup mb 2 (x) − E m
b 2 (x) = max

sup1≤k≤Ln x∈Ik ∩Sτx∈Sτmb 2 (x) − E m
b 2 (x)

= max

sup

m
b 2 (x) − mb 2 (xk ) + m
b 2 (xk ) − E m
b 2 (xk ) + E m
b 2 (xk ) − E m
b 2 (x)

≤ max

sup

m
b 2 (x) − mb 2 (xk ) + max

1≤k≤Ln x∈Ik ∩Sτ
1≤k≤Ln x∈Ik ∩Sτ

sup1≤k≤Ln x∈Ik ∩SτEm
b 2 (xk ) − E m
b 2 (x)+ max mb 2 (xk ) − E m
b 2 (xk ) =: R1 + R2 + R3 ,(80)

1≤k≤Lnwhere mb 2 (xk ) = (nEK1 )−1Pn

t=1 Yt Kt,k and Kt,k = K(kH−1(xk − Xt )k).

We first consider R1 . By Lemma 1,
R1 = max

sup1≤k≤Ln x∈Ik ∩Sτmb 2 (x) − mb 2 (xk )

n




1 X
Yt K kH −1 (x − Xt )k − Yt K kH −1 (xk − Xt )k1≤k≤Ln x∈Ik ∩Sτ nEK1
t=1

= max

sup

n

X
C
Yt Kt − Yt Kt,k · 1E(x,λh)∪E(xk ,λh) (Xt ).
≤ max sup1≤k≤Ln x∈Ik ∩Sτ nϕT (λh)
t=1
Now because type-I kernels are Lipschitz continuous on [0, λ], by the triangle inequality
we havenn
1 X C 0 |Yt |
1X
−1
R1 ≤
ηh · 1E(x,λh)∪E(xk ,λh) (Xt ) =:Jt ,n t=1 ϕT (hλ)
n t=1where Jt is α-mixing under both assumptions A10 and A20 (with a different rate underA20 : α∗ (n) = α(n − τ + 1), where α(·) is the mixing rate under A10 ). Let η = log n/n2 .
Using Assumption B6 and the law of iterated expectations it is straightforward to see
that
E|Jt | ≤

Cη
.
h

(81)

Using Lemma 2 we can specify the Kolmogorov’s entropy of Sτ for η = log n/n2 :"!log n #
"
#


log n
2λn2
2λn2
√
log L S, 2
= C log+1∼ log n × log √
,
n
log n
log n
n
implying that the order of Kolmogorov’s log
entropy is of order (log n)2 .10
n2

We now apply the Fuk-Nagaev inequality (see for example, Fuk and Nagaev (1971),
10

Notice that in this case (42) is indeed satisfied with β = 1, p = 4,  = 1/4, for example.

58

or Rio (2000)) for exponentially mixing variables in Merlevède, Peligrad and Rio (2011,n1.7) with ε = ε0 [log L S, log/(nϕ(λh))]1/2 and r = (log L)/ϕ(λh), where ε0 is somen2
positive constant. Sinces2n :=n X
nX

cov (Jt , Js ) = O nϕT (λh)−1 log n

t=1 s=1

and the required tail condition holds, under Assumption A10 we obtain!
n
1X
Jt − EJt > ε
n t=1

P

=P

n
X
t=1

s
(Jt − EJt ) > nε0nlog L S, logn2nϕ(λh)

!

 
q
γ 
p

log L
r
1


−
log
n
2
n2 ε20 log L S, n2
16C nϕ(λh)
  4 nε0 nϕ(λh)  √exp≤ 4 1++
−ς
16rs2n nϕ(λh)
log L/ϕ(λh)ε0 log L
" p
#γ !
p

− r
ε0 nϕ(λh)
Cε20 log n 2 16C nϕ(λh)
exp −ς
≤ 4 1++
16r
ε0 log n
4 log n
!
"p
#γ !
p
log
L

−
nϕ(λh)nϕ(λh)Cε20 ϕ(λh) 2ϕ(λh) 16C+
exp −ςεγ0 4−γ ·≤ 4 1+
16 log n
ε0
log n
log n
!
p


nϕ(λh)
ε20 C log n
16C
0 √≤ 4 exp −(82)· e−C ( nϕ/ log n) −→ 0,
+
32
ε0
log n
where ς > 1 and γ ≥ 1 are as defined in Section 2.4.4, by choosing ε0 sufficiently large.
In the last inequality we exploited the fact that log(1 + ) =  + o(2 ) as  → 0.
Under Assumption A20 , a penalty of (− log n) is incurred in the squared brackets
in the inequalities above. This does not affect the conclusion (82) because τ = log n ≤√√(log n)2 ≤ nϕ/(log n)1+ ≤ nϕ/ log n by (42) in Assumption E.11
Therefore, in view of (81) it now follows that
s
!
 
n
log L S, log
η2nR1 = max sup mb 2 (x) − mb 2 (xk ) ≤ O+ OP
1≤k≤Ln x∈Ik ∩Sτhnϕ(λh)sss!
!
!
(log n)2(log n)2(log n)2
=O
+ OP
= OP
.
(83)
nϕ(λh)nϕ(λh)nϕ(λh)
11

To elaborate, this is due to the fact that y exp(−(y − g(y))) → 0 as y → ∞, as long as (y − g(y))
tends to +∞ as y → ∞ at the speed strictly faster than log y.

59

As for the second term R2 , we haves!
 (log n)2ηR2 ≤ max sup E m
b 2 (x) − mb 2 (xk ) = O=O.
1≤k≤Ln x∈Ik ∩Sτhnϕ(λh)(84)

Next we move on to the last component:
R3 = max mb 2 (xk ) − E m
b 2 (xk ) =: max Wn (xk )1≤k≤Ln(85)

1≤k≤Ln

where
n

i
1 Xh
Yt Kt − EYt Kt
Wn (x) = m
b 2 (x) − E m
b 2 (x) =nEK1 t=1


n
n
X
C
1X
Yt Kt − EYt Kt =Unt≤nϕT (hλ) t=1
n t=1
where Unt = (ϕT (hλ))−1 C(Yt Kt − EYt Kt ).
By following similar arguments in the proof of Theorem 1, it can be readily seen
that
s2n =

n X
n
X


cov (Unt , Uns ) = O nϕT (hλ)−1 .

t=1 s=1

With the exponential tail condition in B4, we apply the same Fuk-Nagaev inequality for exponentially mixing sequences we referred to in the above. Writing Ln :=nn
L S, log
and taking ε = ε0 [log L S, log/(nϕ(λh))]1/2 and r = (log n)2+ /ϕT (λh),
n2n2 ∈ (0, 1/2) for some ε0 > 0, under Assumption A10 we have

P

s


max |mb 2 (xk ) − E m
b 2 (xk )| > ε1≤k≤Ln≤ Ln · sup P
x∈S

n
X
t=1

s
Unt > nε0

≤ Ln · sup P
x∈Slog LnnϕT (λh)|Wn (x)| > ε0log Lnnϕ(λh)!

− r2
n2 ε20 log Ln≤ Ln · 4 1 +16rs2n nϕT (λh)
( √
)γ !
p
p
16Ln Cn nϕT (λh)ε0 n log n/ ϕT (λh)+
exp −ς
nε0 log n4(log n)2+ /ϕT (λh)
ε20 C log Ln≤ Ln · 4 1 +16(log n)2+ /ϕT (λh)
− (logTn)2+

60

2ϕ

(λh)

!

(p
)γ !
p
nϕT (λh)
16Ln C nϕT (λh)εγ0+exp −ς γε0log n4(log n)1+
 2
ε0 C log Ln+ CL2n exp(−ςεγ0 /4γ log L)≤ Ln · 4 exp −
32
−

2

ςε0

4
0 /32 + CL
≤ 4L−Cε
n
n

+2

.

(86)

Here we used the fact that γ ≥ 1 and (42) in Assumption E. Note that in the special
case when the response Yt is assumed to be bounded, the same result continues tohold with γ1 = ∞ (so that γ2 = γ(≥ 1)). Now noting that ς > 1, by choosing ε0 large
enough it follows thats
R3 = max mb 2 (xk ) − E m
b 2 (xk ) = OP
1≤k≤Ln(log n)2nϕ(λh)!
.

(87)

Same conclusion holds in the dynamic regression case (i.e. under Assumption
A20 ) because of the following reason. The penalty term (due to the penalised mixingrate) that incurs inside the curly bracket results in an additional multiplicative termof exp(−c(−τ )) = exp(c log n) = nc in the second term of the final bound in (86),where c := ς(ε0 /4) is fixed, and this diverges to infinity at the slower rate than
√
(c−2)
Ln
= (n2 / log n)log n·(c−2) .
Returning back to where we started, viewing m
b 1 (x) as a special case of mb 2 (x) with
Yt = 1 ∀t, we can repeat the above procedure, yielding (since E m
b 1 (x) = 1)s
sup m
b 1 (x) − 1 = OPx∈Sτ(log n)2nϕ(λh)!
.

(88)

The proof is now complete in view of (79), (80), (83), (84), (87), (88), contributions
from the bias component, and either Proposition 4.1 of Dunker, Lifshits and Linde(1998) under independence across marginal covariates, or Theorem 1.1 of Hong, Lifshits and Nazarov (2016) under general Assumption C.
Proof of Theorem 5. Given the extended moment condition upto 8 + δ, it isstraightforward to see (from Theorem 1 and 2 & 3) the consistency of σbj (xi ) for σ j (xi )for j = 1, 2, 3, 4 at every point of continuity xi , and the asymptotic normality of (b
µ, σ
b2 )
with limiting variance Ω(xi ).
Hence it suffices to show asymptotic independence of m(xb i )and m(xb 0i ) across i,
where xi and x0i are continuity points of m such that kD−1 (xi − x0i )k > 0. Following
the notations of the proof of Theorem 2 and 3, the asymptotic covariance matrix is
61

p

√ P
given by Var ( φ(h)/ n) nt=1 Rnt , and
Var(Rnt ) = Var 1· Kt,x [Yt − m(x)]
EK1,x
=E
1
0
0
·
K
[Y
−
m(x
)]
t,x
t
EK 0
1,x

A11 A12
A21 A22!
(89)

We know from Theorem 2 and 3 that as for A11 ' σ 2 (x) and A22 ' σ 2 (x0 ). So we justconsider the off-diagonal terms. Due to stationarity we see thathi
E Kt,x Kt,x0 (Yt − m(x))(Yt − m(x0 ))
h
n
on
oi
0
0
= E K1,x K1,x Y1 − m(X1 ) + m(X1 ) − m(x) Y1 − m(X1 ) + m(X1 ) − m(x )hihi= E K1,x K1,x0 (Yt − m(X1 ))(Yt − m(X1 )) + o(1) = E K1,x K1,x0 σ 2 (X1 ) + o(1)≤supσ 2 (u)E[K1,x0 K1,x ] → 0u∈B(x,h)∩B(x0 ,h)

as h → 0 since the kernels return 0 outside its compact support and kD−1 (xi −x0i )k > 0.
The desired result now directly follows via the delta method.

References[1] Abel, A. B. (1988): Stock prices under time-varying dividend risk : An exact
solution in an infinite-horizon general equilibrium model. Journal of Monetary
Economics, 22(3), 375–393.
[2] Andrews, D. W. K. (1984). Non-Strong Mixing Autoregressive Processes. Journal of Applied Probability, 21(4), 930-934.
[3] Andrews, D. W. K. (1995). Nonparametric kernel estimation for semiparametricmodels. Econometric Theory, 11(3), 560-586.
[4] Aneiros, G., Bongiorno, E. G., Cao, R. and Vieu, P. (eds.). (2017). Functional
Statistics and Related Fields. New York: Springer.
[5] Antell, J. and Vaihekoski, M. (2016). Countercyclical and Time-Varying
Risk Aversion and the Equity Premium. SSRN Working paper available athttps://papers.ssrn.com/sol3/papers.cfm?abstract id=2753537.
[6] Azais, J. M. and Fort, J. C. (2013). Remark on the finite-dimensional characterof certain results of functional statistics. Comptes Rendus Mathematique, 351(3),139-141.
62

[7] Backus, K. and Gregory, A, W. (1993). Theoretical Relations Between Risk Premiums and Conditional variances. Journal of Business and Economic Statistics,
11(2), 177-185.
[8] Bali, T. G. and Peng, L. (2006).Is there a Risk-Return Trade-Off? Evidence
from High-frequency data. Journal of Applied Econometrics, 21, 1169-1198.
[9] Bansal, R. and Yaron, A. (2004): Risks for the Long Run: A Potential Resolutionof Asset Pricing Puzzles. Journal of Finance, 59(4), 1481-1509.
[10] Bierens, H. J. (1983). Uniform consistency of kernel estimators of a regressionfunction under generalized conditions. Journal of the American Statistical Association, 78(383), 699-707.
[11] Bierens, H. J. (1987). Kernel estimators of regression functions. In Bewley, T.
F. (ed.) Advances in econometrics: Fifth world congress, 99-144, Vol. 1, Cambridge: Cambridge University Press.
[12] Bingham, N. H., Goldie, C. M. and Teugels, J. L. (1987). Regular variation.
Cambridge: Cambridge University Press.
[13] Billingsley, P. (1968). Convergence of Probability Measures. New York: John
Wiley.
[14] Bliss, R. R. and Panigirtzoglou, N. (2004). Option-implied risk aversion estimates. Journal of Finance, 59(1), 407-446.
[15] Bollerslev, T. (1996). Modeling and pricing long memory in stock market volatility. Journal of Econometrics, 73(1), 151-184.
[16] Bollerslev, T., Gibson, M. and Zhou, H. Dynamic estimation of volatility risk
premia and investor risk aversion from option-implied and realized volatilities.
Journal of Econometrics, 160(1), 235-245.
[17] Borovkov, A. A. and Ruzankin, P. S. (2008). On small deviations of series ofweighted random variables. Journal of Theoretical Probability, 21(3), 628-649.
[18] Bosq, B. (1996). Nonparametric statistics for stochastic processes: estimationand prediction. New York: Springer-Verlag.
[19] Boudoukh, J., Richardson, M. and Whitelaw, R. F. (1997). Nonlinearities in the
Relation between the Equity Risk Premium and the Term Structure. Management Science, 43(3), 371-385.
63

[20] Bradley, R. C. (2005). Basic properties of strong mixing conditions. A survey
and some open questions. Probability Surveys, 2(2), 107-144.
[21] Campbell, J. Y. (1987). Stock Returns and the Term Structure. Journal of Financial Economics, 18(2), 373-399.
[22] Campbell, J. Y. and Cochrane, J. H. (1999). By Force of Habit: A ConsumptionBased Explanation of Aggregate Stock Market Behavior. Journal of Political
Economy, 107(2), 205-251.
[23] Campbell, J. Y. and Hentschel, L. (1992). No News is Good News: An Asymmetric Model of Changing Volatility in Stock Returns. Journal of Financial
Economics, 31(3), 281-318.
[24] Chaté, H. and Courbage, M. (1997). Special issue on lattice dynamics. Physica
D, 103, 1-611.
[25] Chen, J., Li, D., Linton, O. B. and Lu, Z. (2018). Semiparametric Ultra-High
Dimensional Model Averaging of Nonlinear Dynamic Time Series. Journal of
the American Statistical Association, 113(522), 919-932.
[26] Chen, R. and Tsay, R. S. (1993). Nonlinear additive ARX models. Journal of
the American Statistical Association, 88(423), 955-967.
[27] Chen, X. and Christensen, T. M. (2015). Optimal uniform convergence rates andasymptotic normality for series estimators under weak dependence and weak
conditions. Journal of Econometrics, 188(2), 447-465.
[28] Chou, R., Engle, R. F. and Kane, A. (1992). Measuring risk aversion from excess
returns on a stock index. Journal of Econometrics, 52(1-2), 201-224.
[29] Christensen, B. J., Dahl, C. M. and Iglesias, E. M. (2012). Semiparametric
inference in a GARCH-in-mean model. Journal of Econometrics, 167(2), 458472.
[30] Cohn, A., Engelmann, J., Fehr, E. and Maréchal, M. (2015). Evidence for Countercyclical Risk Aversion: An Experiment with Financial Professionals. American Economic Review, 105(2), 860-885.
[31] Conrad, C. and Mammen, E. (2008). Nonparametric regression on latent covariates with an application to semiparametric GARCH-in-Mean models. Discussion
Paper No. 473, University of Heidelberg, Department of Economics.
64[32] Davidson, J. (1994). Stochastic Limit Theory: An Introduction for Econometricians. Oxford: Oxford University Press.
[33] Davydov, Y. A. (1968). Convergence of distributions generated by stationary
stochastic processes. Theory of Probability and Its Applications, 13(4), 691-696.
[34] Delsol, L. (2009). Advances on asymptotic normality in non-parametric functional time series analysis. Statistics, 43(1), 13-33.
[35] Devroye, L. P. (1978). The uniform convergence of the Nadaraya-Watson regression function estimate. Canadian Journal of Statistics, 6(2), 179-191.
[36] Devroye, L. (1981). On the almost everywhere convergence of nonparametric
regression function estimates. The Annals of Statistics, 9(6), 1310-1319.
[37] Doukhan, P. (1994). Mixing. New York: Springer.
[38] Doukhan, P. and Wintenberger, O. (2008). Weakly dependent chains with infinite memory. Stochastic Processes and their Applications, 118(11), 1997-2013.
[39] Duflo, M. (1997). Random iterative models. Berlin: Springer-Verlag.
[40] Dunker, T., Lifshits, M. A. and Linde, W. (1998). Small deviation probabilities ofsums of independent random variables. In: Eberlein, E. (ed.), High dimensional
probability, volume 43 of Progress in Probability., Birkhauser, Basel, 59-74.
[41] Escanciano, J.C., Pardo-Fernández, J.C. and Van Keilegom, I. (2017). Semiparametric estimation of risk return relationships. Journal of Business and Economic
Statistics, 35(1), 40-52.
[42] Fan, J. and Masry, E. (1992). Multivariate regression estimation with errors-invariables: asymptotic normality for mixing processes. Journal of Multivariate
Analysis, 43(2), 237-271.
[43] Fan, J. (1990). A remedy to regression estimators and nonparametric minimax
efficiency. Technical Report 161, Department of Statistics, University of North
Carolina at Chapel Hill.
[44] Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications.
New York: Chapman and Hall.
[45] Fan, J. and Yao, Q. (2003). Nonlinear Time Series: Nonparametric and Parametric Methods. New York: Springer.
65[46] Feller, W. (1971). Introduction to Probability Theory and Its Applications, Vol.
2. New York: Wiley.
[47] Ferraty, F., Laksaci, A., Tadj, A. and Vieu, P. (2010). Rate of uniform consistency for nonparametric estimates with functional variables. Journal of Statistical Planning and Inference, 140(2), 335-352.
[48] Ferraty, F., Laksaci, A., Tadj, A. and Vieu, P. (2011). Kernel regression withfunctional response. Electronic Journal of Statistics, 5, 159-171.
[49] Ferraty, F. and Romain, Y. (2010). The Oxford Handbook of Functional Data
Analysis. New York: Oxford University Press.
[50] Ferraty, F. and Vieu, P. (2002). The functional nonparametric model and application to spectrometric data. Computational Statistics, 17(4), 545-564.
[51] Ferraty, F. and Vieu, P. (2006). Nonparametric functional data analysis: Theory
and Practice. New York: Springer.
[52] French, K. R., Schwert, G. W. and Stambaugh, R. F. (1987) Expected Stock
Returns and Volatility. Journal of Financial Economics, 19(1), 3-29.
[53] Fuk, D. K. and Nagaev, S. V. (1971). Probability inequalities for sums of independent random variables. Theory of Probability and its applications, 16(4),643-660.
[54] Gao, F., Hannig, J. and Torcaso, F. (2003). Comparison Theorems for Small
Deviations of Random Series. Electronic Journal of Probability, 8(21), 1-17.
[55] Gennotte, G. and Marsh, T. A. (1993). Variations in economic uncertainty andrisk premiums on capital assets. European Economic Review, 37(5), 1021-1041.
[56] Geenens, G. (2011). Curse of dimensionality and related issues in nonparametricfunctional regression. Statistics Surveys, 5, 30-43.
[57] Ghysels, E., Santa-Clara, P. and Valkanov, R. (2005). There is a Risk-Return
Trade-Off After All. Journal of Financial Economics, 76(3), 509-548.
[58] Giraitis, L., Leipus, R. and Surgailis, D. (2008). ARCH(∞) and long memory
properties. In: Andersen, T. G., Davis, R. A., Kreiß, J., Mikosch, T. (Eds.),
Handbook of Financial Time Series. Berlin: Springer-Verlag.

66[59] Glosten, L., Jagannathan, R. and Runkle, D. E. (1993). On The Relation Between the Expected Value and the Volatility of the Nominal Excess Return on
Stocks. Journal of Finance, 48(5), 1779-1801.
[60] Götze, F. and Hipp, C. (1994). Asymptotic distribution of statistics in timeseries. The Annals of Statistics, 22(4), 2062-2088.
[61] Greblicki, W. and Krzyzak, A. (1980). Asymptotic properties of kernel estimatesof a regression function. Journal of Statistical Planning and Inference, 4(1), 8190.
[62] Green, P. J. and Silverman, B. (1993). Nonparametric Regression and Generalized Linear Models: A roughness penalty approach. New York: Chapman andHall.
[63] Guo, H., Wang, Z. and Yang, J. (2013). Time-Varying Risk-Return Trade-off in
the Stock Market. Journal of Money, Credit and Banking, 45(4), 623-650.
[64] Guo, H. and Whitelaw, R. (2006). Uncovering the Risk-Return Relation in the
Stock Market. Journal of Finance, 61(3), 1433-1463.
[65] Härdle, W. K. (1990). Applied Nonparametric Regression. Cambridge: Cambridge University Press.
[66] Harvey, C. (2001). The Specification of Conditional Expectations. Journal of
Empirical Finance, 8(5), 573-638.
[67] Hong, S. Y., Lifshits, M. and Nazarov. A. (2016). Small deviations in L2 -norm
for Gaussian dependent sequences. Electronic Communications in Probability,
21(41), 1-9.
[68] Ibragimov, I. A. (1962). Some limit theorems for stationary processes. Theoryof Probability & Its Applications, 23, 291-300.
[69] Ibragimov, I. A. and Linnik, Y. B. (1971). Independent and Stationary Sequencesof Random variables. Groningen: Wolters-Noordhoff.
[70] Jia, Q., Zhou, S. and Yin. F. (2003). Kolmogorov entropy of global attractor
for dissipative lattice dynamical systems. Journal of Mathematical Physics, 44,5804-5801.
[71] Kandel, S. and Stambaugh, R. F. (1990). Expectations and Volatility of Consumption and Asset Returns. Review of Financial Studies, 3(2), 207-232.
67

[72] Karamata, J. (1933). Sur un mode de croissance regulire. Theoremes fondamentaux (in French). Bulletin de la Societe Mathematique de France, 61, 55-62.
[73] Kara-Zaı̈tri, L., Laksaci, A., Rachdi, M. and Vieu, P. (2017). Uniform in thesmoothing parameter consistency results in functional regression. Functional
Statistics and Related Fields, 161-167, Springer.
[74] Kudraszow, N. L. and Vieu, P. (2013). Uniform consistency of kNN regressors
for functional variables. Statistics and Probability Letters, 83(8), 1863-1870.
[75] Lettau, M. and Ludvigson, S. (2010). Measuring and modeling variation in therisk-return tradeoff. In: Ait-Sahalia, Y., Hansen, L. (Eds.), Handbook of Financial Econometrics. Amsterdam: North-Holland.
[76] Li, W. V. (2012). Small value probabilities in analysis and mathematicalphysics. Presented at the Arizona School of Analysis and Mathematical Physics,
Tucson, Arizona, United States in March 15, 2012. Retrieved from the link
http://math.arizona.edu/∼mathphys/school 2012/WenboLi.pdf.
[77] Li, W. V. and Shao, Q. M. (2001). Gaussian processes: inequalities, small ballprobabilities and applications. Handbook of Statistics, 19, 533-598.
[78] Linton, O. B. (2009). Semiparametric and Nonparametric ARCH Modeling. In:
Andersen, T. G., Davis, R. A., Kreiß, J., Mikosch, T. (Eds.), Handbook ofFinancial Time Series. Berlin: Springer-Verlag.
[79] Linton, O. B. and Perron, B. (2003). The Shape of the Risk Premium: Evidence
From a Semiparametric Generalized Autoregressive Conditional Heteroscedasticity Model. Journal of Business and Economic Statistics, 21(3). 354-367.
[80] Linton, O. B. and Sancetta, A. (2009). Consistent estimation of a general nonparametric regression function in time series. Journal of Econometrics, 152(1),70-78.
[81] Lu, Z. (2001). Asymptotic normality of kernel density estimators under dependence. Annals of the Institute of Statistical Mathematics, 53(3), 447-468.
[82] Ludvigson, S. and Ng, S. (2007). The Empirical Risk-Return Relation: A Factor
Analysis Approach. Journal of Financial Economics, 83(1), 171-222.
[83] Lundblad, C. (2005). The Risk-Return Tradeoff in the Long Run: 1836-2003.
Journal of Financial Economics, 85(1), 123-150.
68

[84] Mammen E. (1991) Nonparametric Curve Estimation and Simple Curve Characteristics. In: Roussas G. (eds) Nonparametric Functional Estimation and Related Topics. NATO ASI Series (Series C: Mathematical and Physical Sciences),vol 335. Springer, Dordrecht.
[85] Mas, A. (2012). Lower bound in regression for functional data by small ballprobability representation in Hilbert space. Electronic Journal of Statistics, 6,1745-1778.
[86] Masry, E. (2005). Nonparametric regression estimation for dependent functional
data: asymptotic normality. Stochastic Processes and their Applications, 115(1),155-177.
[87] Masry, E. and Fan, J. (1997). Local polynomial estimation of regression function
for mixing processes. Scandinavian Journal of Statistics, 24(2), 1965-1979.
[88] Mehra, R. (2012). Consumption-Based Asset Pricing Models. Annual Review of
Financial Economics, 4(1), 385-409.
[89] Merlevède, F., Peligrad, M., Rio, E. (2011). A Bernstein type inequality andmoderate deviations for weakly dependent sequences. Probability Theory and
Related Fields, 151(3-4), 435-474.
[90] Merton, R. C. (1973). An Intertemporal Capital Asset Pricing Model. Econometrica, 41(5), 867-887.
[91] Nadaraya, E. A. (1964). On estimating regression. Theory of Probability & Its
Applications, 9(1), 141-142.
[92] Nadaraya, E. A. (1970). Remarks on non-parametric estimates for density functions and regression curves. Theory of Probability & Its Applications, 15(1),134-137.
[93] Nelson, D. (1991). Conditional Heteroscedasticity in Asset Returns: a New Approach. Econometrica, 59(2), 347-370.
[94] Olver, F. W. J., Lozier, D. W., Boisvert, R. F. and Clark, C. W. (2010). NIST
Handbook of Mathematical Functions. New York: Cambridge University Press.
[95] Pagan, A. R. and Hong, Y. S. (1990). Nonparametric Estimation and the Risk
Premium. In W. Barnett, J. Powell, and G. Tauchen (eds.), Nonparametric andSemiparametric Methods in Econometrics. Cambridge University Press.
69[96] Pagan, A. R. and Ullah, A. (1988). The econometric analysis of models withrisk terms. Journal of Applied Econometrics, 3(2), 87-105.
[97] Pagan, A. R. and Ullah, A. (1999). Nonparametric econometrics. Cambridge:
Cambridge University Press.
[98] Parzen, E. (1962). On estimation of a probability density function and mode.
Annals of Mathematical Statistics, 33(3), 1065-1076.
[99] Pástor, L., Sinha, M. and Swaminathan, B. (2008). Estimating the Intertemporal
Risk-Return Tradeoff Using the Implied Cost of Capital. Journal of Finance,
63(6), 2859-2897.
[100] de la Peña, V. H., Lai, T. L. and Shao, Q. M. (2009). Self-normalized processes:
Limit theory and Statistical Applications. New York: Springer.
[101] Phillips, P. C. and Park, J. Y. (1998). Nonstationary density estimation andkernel autoregression. Cowles Foundation discussion paper.
[102] Ramsay, J. O. and Silverman, B. W. (2002). Applied functional data analysis:
methods and case studies. New York: Springer.
[103] Rio, E. (2000). Theorie asymptotique des processus alé atoires faiblement dependants (in French). Berlin: Springer Verlag.
[104] Robinson, P. M. (1983). Nonparametric estimators for time series. Journal ofTime Series Analysis, 4(3), 185-207.
[105] Rosenblatt, M. (1956). A central limit theorem and a strong mixing condition.
Proceedings of the National Academy of Sciences, 42(1), 43-47.
[106] Roussas, G. G. (1989). Consistent regression estimation with fixed design points
under dependence conditions. Statistics & Probability Letters, 8(1), 41-50.
[107] Roussas, G. G. (1990). Nonparametric regression estimation under mixing conditions. Stochastic Processes and Their Applications, 36(1), 107-116.
[108] Schuster, E. F. (1972). Joint asymptotic distribution of the estimated regressionfunction at a finite number of distinct points. Annals of Mathematical Statistics,
43(1), 84-88.

70

[109] Scruggs, J. (1998). Resolving the Puzzling Intertemporal Relation Between theMarket Risk Premium and the Conditional Market variance: A Two-Factor
Approach. Journal of Finance, 53(2), 575-603.
[110] Scruggs, J. and Glabadanidis, P. (2003). Risk Premia and the Dynamic covariance Between Stock and Bond Returns. Journal of Financial and Quantitative
Analysis, 38(2), 295-316.
[111] Smith,

D.

R.

and

Whitelaw,R.

(2009).

Time-varying

risk

aver-

sion and the risk-return relation. SSRN Working paper available athttps://papers.ssrn.com/sol3/papers.cfm?abstract id=1663542[112] Stone, C. (1980). Optimal rates of convergence for nonparametric estimators.
The Annals of Statistics, 8(6), 1348-1360.
[113] Stone, C. (1982). Optimal global rates of convergence for nonparametric regression. The Annals of Statistics, 10(4), 1040-1053.
[114] Stone, C. J. (1985). Additive Regression and Other Nonparametric Models. The
Annals of Statistics, 13(2), 689-705.
[115] Sytaya, G. N. (1974). On certain asymptotic representations for a Gaussian
measure in Hilbert space (in Russian). Theory of Random Process, 2, 93-104.
[116] Veronesi, P. (2000): How Does Information Quality Affect Stock Returns?. The
Journal of Finance, 55(2), 807–837.
[117] Watson, G. S. (1964). Smooth regression analysis. Sankhya Series A, 26(4), 359372.
[118] Whitelaw, R. F. (1994). Time variations and covariations in the Expectation
and Volatility of Stock Market Returns. Journal of Finance, 49(2), 515-541.
[119] Whitelaw, R. F. (2000). Stock Market Risk and Return: An Equilibrium Approach. Review of Financial Studies, 13(3), 521-547.
[120] Wu, W. B. (2011). Asymptotic theory for stationary processes. Statistics and
Its Interface, 4, 207-226.
[121] Yao, Q and Tong, H. (1998). Cross-validatory bandwidth selections for regression estimation based on dependent data. Journal of Statistical Planning andInference, 68(2), 387-415.
71

[122] Yaracos, Y. G. (1985). Rates of Convergence of Minimum Distance Estimators
and Kolmogorov’s Entropy. The Annals of Statistics, 13(2), 768-774.
[123] Yu, J. and Yuan, Y. (2011). Investor Sentiment and the Mean-Variance Relation.
Journal of Financial Economics, 100(2), 367-381.
[124] Zhang, X., King, M. L. and Hyndman, R. J. (2006). A Bayesian approach to
bandwidth selection for multivariate kernel density estimation. Computational
Statistics and Data Analysis, 50(11), 3009-3031.
[125] Zolotarev, V. M. (1986). Asymptotic behavior of the Gaussian measure in `2 .
Journal of Soviet Mathematics, 35(2), 2330-2334.

72