# The Failure of Orthogonality under Nonstationarity: Should We Care About It?

Authors: Jose A. Campillo-García; Daniel Ventosa-Santaulària
Journal: Journal of Probability and Statistics
 (2011)

---

## Abstract

We consider two well-known facts in econometrics: (i) the failure of the
orthogonality assumption (i.e., no independence between the regressors
and the error term), which implies biased and inconsistent Least Squares (LS)
estimates and (ii) the consequences of using nonstationary variables, acknowledged
since the seventies; LS might yield spurious estimates when the
variables do have a trend component, whether stochastic or deterministic.
In this work, an optimistic corollary is provided: it is proven that the LS
regression, employed in nonstationary and cointegrated variables where the
orthogonality assumption is not satisfied, provides estimates that converge to
their true values. Monte Carlo evidence suggests that this property is maintained
in samples of a practical size.


---

## Body

## 1. Introduction

Two well-known facts lie behind this work: (i) the behavior of LS estimates whenever variables are nonstationary and (ii) the failure of the orthogonality assumption between independent variables and the error term, also in an LS regression.(1)
The reappraisal of the impact of unit roots in time-series observations, initiated in the late seventies, had profound consequences for modern econometrics. It became clear that (i) insufficient attention was being paid to trending mechanisms and (ii) most macroeconomic variables are probably nonstationary; such an appraisal gave rise to an extraordinary development that substantially modified the way empirical studies in time-series econometrics are carried out. Research into nonstationarity has advanced significantly since it was reassessed in several important papers, such as those of [1–5].(2)
The orthogonality problem constitutes another significant research program in econometrics; its formal seed can be traced back to [6], where a proposal to solve the identification problem is made in the estimation of demand and supply curves (see [7]). Typically, in textbooks, the method of Instrumental Variables (IVs) is proposed as a solution to the problem of simultaneous equations and, broadly speaking, whenever there is no independence between the error term and the regressors, that is, when the orthogonality assumption is not satisfied.This paper aims to study the consequences of using nonstationary variables in an LS regression when the regressor is related to the error term; this is done in a simple regression framework. The specification under particular scrutiny is(1.1)yt=α+βxt+ut.
To the best of our knowledge, the asymptotics—and the finite-sample properties—of the combination of nonstationarity, nonorthogonality between xt and uy,t, and LS estimates, have been scarcely studied (but see [8]). That said, we acknowledge that there are several comprehensive studies concerning the use of IV in the presence of nonstationarity [9, 10], for example, studied the asymptotics as well as the finite-sample properties of the IV estimator in the context of a cointegrated relationship, and proved that even spurious instruments (i.e., I(1) instruments not structurally related to the regressors) provide consistent estimates. Phillips [11] proved that, when there is no structural relationship between the regressand and a single regressor, that is, when there is no cointegration between y and x, the use of spurious instruments does not prevent the phenomenon (this is a simple extension of [5]). We derive the asymptotic behavior of LS estimates, where the data generating processes (DGPs) consist of two cointegrated variables in which the regressor bears a relationship with the error term. In this case, LS provide consistent estimates. Additionally, some Monte Carlo evidence is presented to account for the adequacy of asymptotic results in finite samples. In other words, LS estimates of the true DGP parameters, μy and βy (see (2.4) in the next section), do not require the information on the parameters of x, that is, xt is weakly exogenous for the estimation of μy and βy as defined by [12].


## 2. Relevant DGPs

This work aims to study the asymptotic properties of LS estimates when neither the orthogonality nor the stationarity assumptions are satisfied. Our approach is twofold: we assume (i) the variablex is statistically related to the innovations of y, as in the problem of independent variables measured with error and (ii) the DGPs of both variables are interdependent, as in the problem of simultaneity. All the cases studied consider nonstationary and cointegrated variables (DGP (2.1) is included because it eases the comprehension of the paper)(2.1)xt=μx+ux,t,(2.2)xt=μx+xt-1+ux,t,(2.3)xt=X0+(μx+ρy1)t+ξx,t-1+ρy2ξy,t-1,(2.4)yt=μy+βyxt+(uy,t+ρxux,t)︸innovations,
where uz,t, for z=x,y, are independent white noises with zero-mean and constant variance σz2, ξzt=∑i=0tuzi and Z0 is an initial condition. We may relax the assumptions made for the innovations; for example, we could force them to obey the general level conditions in [5, Assumption  1]. Nevertheless, although the asymptotic results would still hold in this case, our primary target concerns the problem of orthogonality between the regressor and the error term, not those of autocorrelation or heteroskedasticity. These DGPs allow for an interesting variety of cases (note that the asymptotics of the LS estimates when x and y have been independently generated by any of first three DGPs can be found, e.g., in [13]; notwithstanding, the authors can provide these cases as mathematica code upon request).(1)
Bookcase no. 1: DGP ofx is (2.1) and DGP of y is (2.4) with ρx=0. When the variables are generated in this manner, we fulfill the classical assumptions made in most basic econometrics textbooks. The variables are stationary, the innovations are homoskedastic and independent, and so forth. It is straightforward to show that: α̂→pμy, β̂→pβy and σ̂2→pσy2.(2)
Bookcase no. 2: DGP ofx is (2.1) and DGP of y is (2.4) with ρx≠0. These DGPs also represent a typical example of a problem of orthogonality in most basic econometrics textbooks. Although the variables are stationary and the innovations are homoskedastic and independent, the explanatory variable is related to the innovations of y. It is well known that the estimates do not converge to their true value. In particular, it is straightforward to show that: α̂→pμy-μxρx, β̂→pβy+ρx and σ̂2→pσy2.(3)
Bookcase no. 3: DGP ofx is (2.2) and DGP of y is (2.4) with ρx=0. These DGPs allow the relationship between x and y to be cointegrated à la [14]. Once again, asymptotic results have been known for a long time, obtaining these does not entail any particular difficulty: α̂→pμy, β̂→pβy, σ̂2→pσy2 and 1-R2=Op(T-2).(4)
Nonstationarity and non-orthogonality case no. 1: DGP ofx is (2.2) and DGP of y is (2.4). Notwithstanding, the obvious problem of orthogonality between x and the error term, the variables remain cointegrated. The artifact employed to induce the orthogonality problem can be considered as, for example, measurement errors in the explanatory variable. One should expect that, in the presence of this problem, estimates would not converge to their true value. We prove below that, contrary to expectations, this is not the case.(5)
Nonstationarity and non-orthogonality case no. 2: DGP ofx is (2.3) and DGP of y is (2.4). As in the previous case, we have a cointegrated relationship between x and y, only in this case, the problem of orthogonality between the regressor and the error term is even more explicit; the artifact employed to induce the orthogonality problem can be related to the typical simultaneous equations case. We also prove below that Least Squares (LS) provide consistent estimates.The common belief as regards the last two cases is that the failure of the orthogonality assumption induces LS to generate inconsistent estimates, even in a cointegrated relationship. In fact, when the variables are generated as in (2.2)–(2.4), the estimates of the parameter converge to their true value (note that we did not consider the case where the orthogonality assumption is not satisfied because of the omission of a relevant variable; [15] studied the later case and proved that the LS estimates do not converge to their true values). This is proven in Theorem 2.1:Theorem 2.1.
Letyt be generated by (2.4).(i)
Letxt be generated by (2.2). The innovations of both DGPs, uz,t, for z=y,x, are independent white noises with zero-mean and constant variance σz2; use yt and xt to estimate regression (1.1) by LS. Hence, as T→∞,
(a)
α̂→pμy,(b)
β̂→pβy,(c)
T-3/2tβ=Op(1),(d)
σ̂2→pσy2+ρx2σx2,(e)
T2(1-R2)→p12(σy2+ρx2σx2)/(βyμx)2.(ii)
Letxt be generated by (2.3). The innovations of both DGPs, uz,t, for z=y,x, are independent white noises with zero-mean and constant variance σz2; use yt and xt to estimate regression (1.1) by LS. Hence, as T→∞,
(a)
α̂→pμy,(b)
β̂→pβy,(c)
T-3/2tβ=Op(1),(d)
σ̂2→pσy2+ρx2σx2,(e)
T2(1-R2)→p12(σy2+ρx2σx2)/[βy(μx+ρy1)]2.Proof.
See AppendixA.These asymptotic results show that a relationship between the innovations ofyt and xt—as stated by DGPs (2.2), (2.3), and (2.4)—does not obstruct the consistency of LS estimates when the variables are nonstationary and cointegrated (our results are in line with those of [8]). In other words, the failure of the orthogonality assumption does not preclude adequate asymptotic properties of LS. Furthermore, it can be said that xt is weakly exogenous for the estimation of μy and βy but not for the estimation of σ2. The formula of the variance is noteworthy and the asymptotic expression of tβ depends on the values of σx2, σy2, and ρx.In order to emphasize the relevance of this result, we modified the DGPs of the variables in an effort to strengthen the link between the DGPs and the literature on simultaneous equations. The modifications are twofold and appear in the following propositions. As in Theorem2.1, the results in proposition 1 are made under the assumption that innovations are i.i.d processes.Proposition 2.2.
Letyt and  xt be generated by
(2.5)yt-βyxt-μy=uy,t,-βxyt+xt-μx-γxt=ux,t,
where uz,t, for z=x,y, are independent white noises with zero mean and variance σz2. Let these variables be used to estimate regression (1.1) by LS. Hence, as T→∞,(1)
α̂→pμy,(2)
β̂→pβy,(3)
T-3/2tβ=Op(1),(4)
σ̂2→pσy2,(5)
T2(1-R2)→p12σy2(1-βxβy)2/(βyγx)2.Proof.
See AppendixA.Proposition 2.3.
Letyt and  xt be generated by
(2.6)yt-βyxt-μy=uy,t,-βxyt+xt-X0-μxt=ξx,t,
where uz,t, for z=x,y, are independent white noises with zero mean and variance σz2, and ξx,t=∑i=0tux,i. Let these variables be used to estimate regression (1.1) by LS. Hence, as T→∞,(1)
α̂→pμy,(2)
β̂→pβy,(3)
T-3/2tβ=Op(1),(4)
σ̂2→pσy2,(5)
T2(1-R2)→p12σy2(1-βxβy)2/(μxβy)2.Proof.
See AppendixA.The two systems, represented in (2.5) and (2.6), bear a striking resemblance to classical examples of simultaneous equations in econometrics. The fundamental variations are, (i) a deterministic trend in the variable xt in system (2.5) and (ii) a stochastic as well as a deterministic trend in system (2.6). The asymptotics of LS estimates do not show significant differences from those in Theorem 2.1. Note, however, that xt is weakly exogenous for the estimation of μy, βy, and σy2. The main result is in fact identical, that is, the failure of orthogonality between xt and the error term does not preclude the estimates from converging to their true values.Asymptotic properties of LS estimators clearly provide an encouraging perspective in time-series econometrics. Notwithstanding, we should bear in mind that asymptotic properties may be a poor finite-sample approximation. In order to observe the behavior of LS estimates in finite samples, we present two Monte Carlo experiments. Firstly, we represent graphically the convergence process ofβ̂ towards its true value, β. In accordance with asymptotic results, β̂-β→p0 as T→∞. We reproduce the behavior of the later difference in figure 1. The variables x and y are generated according to (2.3) and (2.4), respectively. The sample size varies from 50 to 700 whilst βy goes from −5 to 5. The remaining parameters appear below the figure.A brief glance at Figure1 reveals that the asymptotic results stated in Theorem 2.1 approximate conveniently the finite-sample results for T>150. For smaller sample sizes, it can be seen that the difference between the parameter and its estimates corresponds usually to approximately 1.5% or less of the value of the former (we tried different variables in the y axis (ρx,ρy1,ρy2,σy2,σy2,…); all of these trials produced similar figures).Figure 1
Finite-sample behavior ofβ̂-βy: sample size and βy range from T=50–700 and −5 to 5, respectively; ρy1=-0.15; ρy2=1.5; ρx=4; σy2=σx2=1; μy=4.20; μx=0.70. Number of replications (for each coordinate): 20; grid density: 70×30.The second Monte Carlo is built upon the same basis. In Table1, each cell indicates the sample mean of β̂-βy and, below, its estimated standard deviation (in parentheses). The number of replications is 10,000. The parameter values used in the simulation are explicit within the table. The variables, x and y, are generated according to (2.3) and (2.4), respectively. Sample size ranges from T=50–700; ρy1=-0.15; ρx=4; σy2=σx2=1; μy=4.20; the error term is a white noise with variance σϵ2=1.Table 1
Finite-sample behavior ofβ̂-βy: mean and standard deviation.
DGP parametersSample sizeβyμxStatistic50100200500700−0.9−1.5Mean1.5·10-23.1·10-37.8·10-48.6·10-55.6·10-5Stand.dev.(1.8·10-3)(1.7·10-4)(1.8·10-5)(1.1·10-6)(3.9·10-7)−0.75Mean1.5·10-23.3·10-37.6·10-41.0·10-45.3·10-5Stand.dev.(1.9·10-3)(1.6·10-4)(1.8·10-5)(1.1·10-6)(3.9·10-7)0.75Mean1.4·10-23.2·10-38.0·10-41.0·10-45.8·10-5Stand.dev.(1.7·10-3)(1.6·10-4)(1.8·10-5)(1.0·10-6)(3.9·10-7)1.5Mean1.5·10-23.0·10-37.2·10-41.0·10-45.8·10-5Stand.dev.(1.8·10-3)(1.6·10-4)(1.8·10-5)(1.1·10-6)(3.9·10-7)0.9−1.5Mean2.2·10-36.2·10-41.4·10-42.1·10-51.1·10-5Stand.dev.(2.4·10-4)(2.8·10-5)(3.4·10-6)(2.1·10-7)(8.0·10-8)−0.75Mean2.4·10-36.0·10-41.5·10-41.9·10-51.7·10-5Stand.dev.(2.3·10-4)(2.8·10-5)(3.4·10-6)(2.2·10-7)(7.9·10-8)0.75Mean2.4·10-35.8·10-41.3·10-42.2·10-55.4·10-6Stand.dev.(2.3·10-4)(2.9·10-5)(3.5·10-6)(2.2·10-7)(8.0·10-8)1.5Mean2.2·10-35.6·10-41.2·10-42.1·10-51.1·10-5Stand.dev.(2.4·10-4)(2.9·10-5)(3.4·10-6)(2.2·10-7)(7.9·10-8)Table1 shows that LS estimates of a nonstationary relationship with a nonorthogonality problem quickly converge to their true value; with a sample size as small as 50 observations, the difference between βy and its estimate averages, at most, 0.015, and represents a deviation from the true value of 1.5%; in many other cases, the deviation is even smaller, of order 10−3–10−4. These differences tend to diminish further as the sample size grows. In fact, when there are 700 observations, the order of magnitude of such differences oscillates between 10−5–10−8. We performed the same experiment with autocorrelated disturbances AR(1) with ϕ=0.7 (data available upon request); using such disturbances severely deteriorates the efficiency of the LS estimates although β̂-βy still converges to zero; we do not focus on this issue because, as mentioned earlier, neither autocorrelation nor heteroskedasticity are under scrutiny in this work.


## 3. Concluding Remarks

Using cointegrated variables in an LS regression where the regressor is not independent of the error term does not preclude the method from yielding consistent estimates. In other words, it is proven that, under these circumstances, the regressor remains weakly exogenous for the estimation ofμy and βy (and for σy2 in systems (2.5) and (2.6)) as defined by [12]. Furthermore, the finite-sample evidence indicates that LS provide good estimates even in samples of a practical size.Notwithstanding, one should note the striking resemblance between the properties of the DGPs used in the propositions and those of variables belonging to a classical simultaneous-equation model. It may be possible that the estimation of such models, even if the macroeconomic variables they are nourished with are not stationary, would yield correct estimates. Of course, such a possibility rules out the existence of structural shifts, parameter instability, omission of a relevant variable, or any other major assumption failure.



---

*Source: 329870-2011-03-07.xml*