Inferring causality in time series data
By: Shay Palachy
Tags: Causality, Time Series Analysis, Statistics, Causal Inference, Data Science

Background: Notions of causality in time series data

Throughout the years a number of different notions of causality were suggested by scholars of statistics and economics. I give here an overview of the major ones. This part is based, in part, on [Eichler, 2011] and [Runge, 2014]. 📖

Granger causality⁸

The earliest concept of causality for time series data was suggested by Granger [1969, 1980], building on a notion from [Wiener, 1956]. It is based on contrasting the ability to predict a stochastic process Y using all the information in the universe, denoted with U, with doing the same using all information in U except for some stochastic process X; this is denoted with U\X. The core idea is that if discarding X reduces the predictive power regarding Y, then X contains some unique information regarding Y, and we thus say that X Granger-causes Y.

More formally:

Let X and Y be stationary stochastic processes.

Denote with 𝒰ᵢ=(Uᵢ₋₁,…,Uᵢ₋∞) all the information in the universe until time i, and with 𝒳ᵢ=(Xᵢ₋₁,…,Xᵢ₋∞) all information in X until time i.

Denote with σ²(Yᵢ|𝒰ᵢ) the variance of the residual of predicting Yᵢ using 𝒰ᵢ at time i.

Denote with σ²(Yᵢ|𝒰ᵢ\𝒳ᵢ) the variance of the residual of predicting Yᵢ using all information in 𝒰ᵢ at time i except 𝒳ᵢ.

Definition 1: If σ²(Yᵢ|𝒰ᵢ) < σ²(Yᵢ|𝒰ᵢ\𝒳ᵢ) then we say that X Granger-causes Y, and write X⇒Y.

Definition 2: If X⇒Y and Y⇒X we say that feedback is occurring, and write X⇔Y.

As noted by Granger himself, the requirement of having access to all the information in the universe is extremely unrealistic. In practice U is replaced by a limited set of observed time series X, with X∈X, and the above definition reads X Granger-causes Y with respect to X.

Finally, this definition does not specify the prediction method used for σ², and thus allows for both linear and non-linear models, but the use of the variance to quantify the closeness of prediction restricts this notion of causality to causality in mean.

This notion is usually referred to as strong Granger causality; other related notions of causality are Granger causality in mean [Granger 1980, 1988] and linear Granger causality [Hosoya 1977, Florens and Mouchart 1985].

Instantaneous causality: A related kind of causality, modifying Granger causality slightly, is instantaneous causality [Price, 1979]. We say that X and Y has instantaneous causality between them if, at time i, adding Xᵢ to the information set helps to improve the predicted value of Yᵢ.

More formally:

Let X and Y be stationary stochastic processes.

Denote with 𝒰ᵢ=(Uᵢ₋₁,…,Uᵢ₋∞) all the information in the universe until time i, and with 𝒳ᵢ=(Xᵢ₋₁,…,Xᵢ₋∞) all information in X until time i (in both cases, not including information from time i itself).

Definition 3: If σ²(Yᵢ|𝒰ᵢ∪{Xᵢ}) < σ²(Yᵢ|𝒰ᵢ) then we say that there is instantaneous causality between X and Y.

Note that this type of causality is not directional, but rather symmetric, and it can be shown that if the above definition holds then the symmetric statement — that σ²(Xᵢ|𝒰ᵢ∪{Yᵢ}) < σ²(Xᵢ|𝒰ᵢ) — also holds(see [Lütkepohl, 2007] for proof). Thus, we do not say that X instantaneously causes Y, but rather that there is instantaneous causality between X and Y.

Multi-step causality: In a bivariate system, if the 1-step ahead forecasts of one variable cannot be improved by using the information in the other variable, the same holds for all h-step forecasts for any h=1,2,…, and so the 1-step ahead criteria is sufficient to define Granger causality. This result does not hold anymore if the information set contains additional variables. [Lütkepohl and Müller, 1994]

Thus, in a multivariate system, we say that variable Xᵢ is h-step causal for another variable Yᵢ if the information in Xᵢ helps improve the j-step forecasts of Yᵢ for some j=1, 2, …, h.

Sims causality

In an influential paper, [Sims, 1972] showed — in the context of covariance stationary processes, and restricted to linear predictors — that in the bivariate case the definition of Granger causality is equivalent to parameter restrictions of the moving average or distributed lag representations of the processes x[t], y[t]. When the system is covariance stationary it can be represented as:

Equation 8: The Sims representation for covariant stationary processes

where aⱼ, bⱼ, cⱼ and dⱼ are constants and u[t] and v[t] are mutually uncorrelated white noise processes. Sims shows that the condition x[t] does not Granger cause y[t+1] is equivalent to cⱼ or ⱼ being chosen identically zero for all j.

In contrast to Granger’s definition, which considers temporal precedence in the form of a link from the past to the present, Sims’ notion considers temporal precedence in the form of a link from the present to the future. As a consequence, the potential causal relationship considered runs from the dependent variable to future values, or ‘leads’, of the regressor.

While at the time of its introduction in [Sims, 1972] it was presented as an equivalent definition to Granger’s, it was since contrasted with it and was shown to be inequivalent when the measure of uncorrelatedness of time series used is independence [Florens and Mouchart, 1982]; rather, it is shown that Granger causality is a stronger condition, and that while Granger causality implies Sims causality, the inverse is not true.

In spite of this inequivalency, most statistical tests for causal inference in time series data focus on Granger’s definition. However, at least in the case where a vector autoregressive (VAR) model is used, these tests can be modified to test for Sims’ causality (see here for an example highlighting the difference between the tests for the linear case).

Structural Causality

Introduced by White and Lu (2010), structural causality assumes that the data-generating process (DGP) has a recursive dynamic structure in which predecessors structurally determine successors. Specifically, for two processes X — the potential cause — and Y — the response, we assume they are generated by

Equation 9: The structural causality DGP

for all t∈Z. Here, the process Z includes all relevant observed variables while the realizations of U=(Uₓ, Uᵧ) are assumed to be unobserved, and the functions q[x,t] and q[y,t] are assumed to be unknown.

Observe that this dynamic structure is general, in that the structural relations may be nonlinear and non-monotonic in their arguments and non-separable between observables and unobservables. The unobservables may be countably infinite in number. Finally, this system may generate stationary processes, non-stationary processes, or both.

A structural notion of causality can then be defined:

The authors then go on to analyze the relations between Granger causality and this notion of structural causality. Additionally, building on classical notions of Granger causality, they introduce two extensions of it: weak Granger causality and retrospective weak Granger causality.

Finally, the authors construct practical tests for their two notions of Granger causality. Specifically, weak Granger causality is shown to be detectable by testing for the conditional independence of the response Y and the potential cause X given the history of the response and the near-histories of the observable covariates of the processes.

Intervention causality

The use of intervention as the basis for a statistical theory of causality inference, as championed by Judea Pearl, can be traced back at least to the early 90’s [Pearl and Verma, 1991] [Pearl, 1993]. It’s application to time series data, however, has begun to receive rigorous treatment only recently [White, 2006] [Eichler and Didelez, 2007]. This approach to causality is closely related to the use of impulse response analysis in economics.

Eichler and Didelez define a set of possible intervention regimes corresponding to different possible types of interventions in a multivariate stationary time series X with d components. Interventions are denoted by the intervention indicator σ which takes values in {∅, s∈ 𝓢}; I use Xₐ to denote a component in X, and Xᶸ to denote a subset of components in X, where a∈V and U⊂V for V={1,…d}. I also use σₐ to denote an intervention in component Xₐ.

Intervention types are:

Idle regime: When σₐ(t) = ∅, Xₐ(t) arises naturally without intervention. A.k.a. the observational regime. Atomic interventions: Here 𝓢=X, the domain of xₐ, and σₐ(t) = x⦁ denotes an intervention forcing Xₐ(t) to assume the value x⦁. Conditional intervention: Here 𝓢 consists of functions

g(xᶸ(t-1))∈X , U⊂V , such that σₐ(t)=g means Xₐ(t) is forced to take on a value that depends on past observations of Xᶸ(t=1). Random intervention: Here 𝓢 consists of distributions

meaning that Xₐ(t) is forced to arise from such a distribution.

Then, under some assumptions ensuring an intervention is an isolated exogenous change of the system, the average causal effect (ACE) of interventions in X according to strategy s on the response variable Y[t`] is defined to be (assuming w.l.o.g. that 𝔼[Y[t`]]=0):

Equation 10: The average causal effect (ACE) of interventions according to strategy s

Thus, the ACE[s] can be regarded as the average difference between no intervention and intervention strategy s. Additionally, different strategies can be compared by considering ACE[s₁]−ACE[s₂], or other functionals of the post-intervention distribution ℙ[s](Y[t`]).

Now, a priori there is no reason why data that is not collected under the intervention regime of interest should allow estimation of the ACE. However, the authors then go on to show the possibility of expressing the ACE in terms of quantities that are known or estimable under the observational regime, using what they call the back-door criterion.

I find this a very elegant reconciliation of causality in time series data with the the highly influential concept of intervention-based causality in general.

Additionally, a recent paper by Samartsidis et al. provides a thorough review of other methods for assessing the causal effect of interventions from aggregate time-series observational data for the specific case of binary interventions [Samartsidis et al. 2018].