# Mathematical Issues in the Inference of Causal Interactions among Multichannel Neural Signals

Authors: Young-Jin Jung; Kyung Hwan Kim; Chang-Hwan Im
Journal: Journal of Applied Mathematics
 (2012)

---

## Abstract

Within the last few decades, attempts have been made to characterize the underlying mechanisms of brain activity by analyzing neural signals recorded, directly or indirectly, from the human brain. Accordingly, inference of functional connectivity among neural signals has become an indispensable research tool in modern neuroscience studies aiming to explore how different brain areas are interacting with each other. Indeed, remarkable advances in computational sciences and applied mathematics even allow the estimation of causal interactions among multichannel neural signals. Here, we introduce the brief mathematical background of the use of causality inference in neuroscience and discuss the relevant mathematical issues, with the ultimate goal of providing applied mathematicians with the current state-of-the-art knowledge on this promising multidisciplinary topic.


---

## Body

## 1. Introduction and Background

Traditional functional neuroimaging studies have focused on the functional specification of brain areas. However, only a limited amount of information regarding the underlying neuronal mechanisms can be obtained when such spatial specification is studied. Recently, research interests have shifted toward describing how different brain areas interact with each other, with the hope of better understanding the functional organization of the cortical network [1–7]. Correlation [1, 2], coherence [3], phase locking value [4], mean phase coherence [5], and mutual information [6, 7] have been used to estimate functional interaction between multiple neural assemblies. These methods have been applied to signals obtained via many different functional neuroimaging modalities such as electroencephalography (EEG), local field potential (LFP), intracranial EEG (iEEG), magnetoencephalography (MEG), and functional magnetic resonance imaging (fMRI). Recent advances in neural signal analysis have also enabled the estimation of direction of information flow between different cortical areas [8–12] even beyond conventional correlation-based functional connectivity analyses.Over the past few decades, a number of measures for “directional” coupling between neural activities have been developed [8–13] and applied to various fields in both basic and clinical neuroscience [14–24]. Although a variety of causality estimators have been widely used for characterizing the mechanisms of neuronal networks, notable limitations and issues still exist that require intervention by applied mathematicians. For example, multivariate autoregressive (MVAR) model-based causality estimators do not accurately infer information flow between nonstationary and/or highly nonlinear neural signals. The determination of model order and the dependency on the analysis sample size are other issues that should be addressed in future studies. Furthermore, most non-MVAR-based causality estimators can only be used to infer causality between two signals, and thus need to be extended to the case of multichannel (≥3) signal analyses [12, 13, 25].Here, we introduce several mathematical signal analysis methods for estimating directional coupling between neural activities, all of which have been widely used in basic and applied neuroscience. Additionally, this paper attempts to illustrate the important mathematical issues that need to be addressed to improve the conventional causality estimators, with the aim to stimulate interest in this imperative multidisciplinary research topic among applied mathematicians.


## 2. MVAR-Based Causality Estimators

Recently, a number of causality estimation techniques have been developed to infer causality among multiple neural signal generators. The MVAR model—a linear multivariate time series model with a long history of application in econometrics [8]—has been frequently applied for causality estimations. The MVAR model is an extended version of the autoregressive (AR) model, a simple approach to time series characterization that assumes that for any given univariate time series, its consecutive measurements contain information regarding the process that generated it. The AR model can be implemented by modeling the current value of any variable as the weighted linear sum of its previous values. In the AR model, the value of a time series x at time t, xt can be estimated using:(2.1)xt=α0+∑k=1pαkxt-k+et,
where α, p, and et represent AR-matrix coefficients, the model order, and the uncorrelated Gaussian random process with a zero mean, respectively.


### 2.1. Granger Causality

Granger causality [8] has been proposed in the field of econometrics to quantify the causal relationship between two different time series. Specifically, this simple technique uses an MVAR model to linearly predict the future values of x and y, vectors of deterministic variables. The MVAR model attempts to estimate the value of xt using:
(2.2)xt=α0+∑k=1pαkxt-k+∑k=1pβkyt-k+wt,
where α and β represent the AR-matrix coefficients and wt the uncorrelated multivariate Gaussian random process with a zero mean. In contrast to (2.1), where the current value of a time series is estimated as the weighted sum of its previous values, the current value xt in (2.2) is estimated using the previous values of two signal vectors x and y. We can judge whether there exists the Granger causality from y to x by inspecting whether the past information from both time series significantly improves the prediction of the future of x, rather than using the past information from x alone. In other words, if the prediction error for the MVAR model (wt) is smaller than that for the AR model (et), it can be concluded that y causes x. In this way, Granger causality can be evaluated using
(2.3)F≡(RSS0-RSS1)/MRSS1/(T-2M-1),
where
(2.4)RSS0=∑i=1Tet2,RSS1=∑i=1Twt2,
where T represents the number of observations. To assess the statistical significance of the estimated Granger causality, the F-test with the null hypothesis, H0: βk=0 (i.e., yt does not influence the generation of xt) is generally used. If βk=0 for all k=1,2,…,p, the Granger causality value F becomes zero as RSS0 equals RSS1. Conversely, if the null hypothesis is rejected, that is, F is sufficiently large, it can be concluded that yt causes xt.To test this hypothesis, a traditionalF-test derived from an ordinary least squared regression for each equation can be used. To test the statistical significance of F, the cumulative F distribution is first estimated, after which the probability of the F value can be calculated by PGC=1-CDF(F), where CDF represents the cumulative distribution function and PGC represents the probability of Granger causality. For example, PGC=1 would indicate that no causal interaction exists between two time series, while PGC=0 would signal a strong directional influence (yt→xt).However, the MVAR model is problematic when estimating the appropriate model orderp. Basically, most model order estimation methods are based on the maximum likelihood principle, which allows the determination of the highest possible model order in MVAR signal modeling. Akaike information criterion (AIC) [26] is also based on this concept and was the earliest method to estimate MVAR model orders. As AIC generally chooses larger than optimal model orders, the Bayesian information criterion (BIC) [27]—which is based on the Bayes estimator—was developed by Schwarz. The BIC generally penalizes free parameters more strongly than the AIC, and thus provides more accurate estimates of MVAR model orders. Although several modifications of the AIC and BIC have been recently developed [28–35], the estimation of accurate and reliable model orders remains an important issue.


### 2.2. Directed Transfer Function

Directed transfer function (DTF) is a widely used tool in identifying information flow between multichannel neural signals. Even though both Granger causality and DTF are based on MVAR modeling, the DTF procedure differs slightly from Granger causality. As described above, Granger causality uses the variance of prediction errors to estimate the causal interaction, while DTF uses a matrix transfer function derived from MVAR model coefficients [9, 36]. In the framework of the MVAR model, a multivariate process of DTF can be described as a data vector X of N source signals: Xt=(X1(t),X2(t),…,XN(t))T. The MVAR model can then be constructed as
(2.5)Xt=∑k=1pAkXt-k+Et,
where Et represents a vector composed of white noise values at time t, Ak is an N×N matrix composed of the model coefficients, and p is the model order of MVAR. Note that (2.1) is a special case of (2.5) when N=1. The MVAR model is then transformed into the frequency domain as follows:
(2.6)X(f)=A-1(f)E(f)=H(f)E(f),
where f denotes a specific frequency and the H(f) matrix represents the so-called transfer matrix, which is defined as
(2.7)H(f)=A-1(f)=(∑k=0pAke-i2πfk∆t)-1,A0=-I,
where I is the identity matrix.The DTF can then be defined in terms of the elements of the transfer matrixHij as
(2.8)γij2(f)=|Hij(f)|2∑m=1k|Him(f)|2,
where γij2(f) denotes the ratio between inflow from signal j to signal i and all inflows to signal i and k represents the number of signals. The DTF ratio γij2(f) ranges from 0 to 1, with values approaching to 1, suggesting that signal i is caused by signal j, whereas values approaching to 0 indicating that no information flow from signal j to signal i exists at a specific frequency.


### 2.3. Partial Directed Coherence

Partial directed coherence (PDC) was proposed by Baccalá and Sameshima as a frequency domain counterpart to Granger causality [11] and is based on a spectral representation of (2.5), defined as
(2.9)A(f)=∑k=0pAke-i2πfk∆t,A̅(f)=I-A(f),
where I is an identity matrix [37]. The estimate of PDC from Xm to Xn is defined as(2.10)PDCXn←Xm(f)=A̅n,m(f)∑k=1Q|An,k(f)|2,
where A̅n,m is the (n,m)-th element of A̅.


### 2.4. Modified MVAR-Based Estimators

Although the MVAR-based causality estimators described above have been shown to be useful in many neuroscience problems, they are not applicable for all types of neural signals. For example, signals which have severely unbalanced model residual variances are not appropriate forPDC. Accordingly, several additional modified causality estimators, including generalized partial directed coherence [38, 39], Geweke’s Granger causality [40, 41], Wiener Granger causality [42], and direct directed transfer function [43], were subsequently developed to broaden application.The generalized partial directed coherence (gPDC) was first proposed by Baccalá et al. to circumvent the numerical problem associated with time series scaling [38], by which a variance stabilization of the frequency domain representation of lagged causality could be achieved [39]. The gPDC estimator is defined as
(2.11)πij(w)(f)=(1/σi)A̅ij(f)∑k=1N(1/σk2)A̅kj(f)A̅kj*(f),
where σi represents the variance of the ith input process. gPDC was modified from PDC to improve the identification of causal interactions between signals with severely unbalanced model residual variances [39].The Geweke’s Granger causality is derived from Geweke’s formulation [40, 41] and is defined as(2.12)Ik→l(f)=(Zkk-(Zlk2/Zll))|Hlk2||Sll(f)|,Il→k(f)=(Zll-(Zkl2/Zkk))|Hkl2||Skk(f)|,
where Skk(f) and Sll(f) represent the individual power spectra of sites k and l, respectively, and the expressions for Hlk can be found in (2.6). Zkk, Zll, Zlk, and Zkl are elements of the covariance matrix Z for the noise vector of the bivariate model. Geweke’s Granger causality at frequency f is expressed as the fraction of the total power at the frequency at one site that can be explained by the causal influence from the other. As seen in (2.12) and (15), Geweke’s Granger causality can be evaluated solely using the bivariate model. Recently, Bressler and Seth [42] introduced Wiener-Granger causality and discussed its merits and limitations in various neuroscience applications [42].The direct directed transfer function (dDTF) was proposed by Korzeniewska [43] for the analysis of direct information transfer among brain structures using local field potentials. To calculate the dDTF, partial coherence χij and full-band frequency DTF (ffDTF) ηij were independently defined as(2.13)χij2(f)=Mij2(f)Mjj(f)Mii(f),ηij2(f)=|Hij(f)|2∑f∑m=1k|Him(f)|2,
where Mij represents the minor produced by removing the ith row and jth column of a spectral matrix S. In multivariate signals, partial coherences may provide more specific information regarding causal interactions among signals than ordinary coherences [44]. The value of dDTF is defined as the product of the above two variables and can be expressed as
(2.14)δij(f)=χij2(f)ηij2(f).The dDTF method was proposed to circumvent some problems associated with DTF, specifically its inability to differentiate between the direct and indirect connections [43].


### 2.5. Examples of Practical Applications

To date, Granger causality has been extensively applied to the analysis of neural signals [45–58]. For examples, Hesse et al. used Granger causality to assess directed interdependencies between neural signal generators related to the Stroop task [45]. Seth also demonstrated that Granger causality may represent a useful tool for determining how interregional directional coupling is modulated by behavior [46]. Moreover, Sato et al. proposed a wavelet-based Granger causality, which they applied to fMRI signals [47]. Tang et al. applied both a blind source separation algorithm and Granger causality to the analysis of a high-density scalp EEG dataset and assessed the top-down and bottom-up influences [48]. Gow et al. demonstrated the potential value of combining Granger causality analyses with multimodal imaging to explore the functional architecture of cognition [49].The DTF algorithm has also been extensively applied to various aspects within neuroscience, particularly to the analyses of electrophysiological signals such as EEG, MEG, and iEEG, because frequency-domain analysis is generally required in these modalities. Franaszczuk et al. first applied the DTF algorithm to the localization of ictal onset zones in temporal lobe epilepsy patients [59, 60]. Astolfi et al. demonstrated that the DTF algorithm could be used to assess the time-varying functional connectivity patterns from noninvasive EEG recordings in human [61]. Babiloni et al. investigated cortical causal interactions from combined high-resolution EEG and fMRI data and showed that DTF was able to unveil the direction of the information flow between the cortical regions of interest [62]. Kuś et al. attempted to characterize EEG activity propagation patterns in beta and gamma bands during finger movements, demonstrating that short-time DTF can successfully identify frequency selective information from EEGs [63]. Ding et al. and Wilke et al. also applied the DTF algorithm to EEG and iEEG signals acquired from intractable partial epilepsy patients in order to better describe the structure of seizures in terms of space, time, and frequency [64, 65]. Kim et al. applied a source localization technique called FINEs in addition to DTF to iEEG signals and verified that directional connectivity analysis could be a useful tool to identify epileptogenic sources located outside of the iEEG electrodes [66].As with DTF, PDC analyses have recently been applied to a variety of practical applications: Sun et al. demonstrated that PDC is a useful tool for evaluating changes in cortical interdependences in the context of different psychotic or mental states and can also be used to diagnose affective disorders [21]. Similarly, Zhang et al. used PDC to estimate cortical interactive networks during the mental rotation of Chinese characters, demonstrating different changes in cortical networks according to task difficulty [18]. Furthermore, Zhu et al. studied the effects of brain development and aging on cortical interactive network pattern, demonstrating that the PDC analysis of EEG is a powerful approach for characterizing brain development and aging [24].


## 2.1. Granger Causality

Granger causality [8] has been proposed in the field of econometrics to quantify the causal relationship between two different time series. Specifically, this simple technique uses an MVAR model to linearly predict the future values of x and y, vectors of deterministic variables. The MVAR model attempts to estimate the value of xt using:
(2.2)xt=α0+∑k=1pαkxt-k+∑k=1pβkyt-k+wt,
where α and β represent the AR-matrix coefficients and wt the uncorrelated multivariate Gaussian random process with a zero mean. In contrast to (2.1), where the current value of a time series is estimated as the weighted sum of its previous values, the current value xt in (2.2) is estimated using the previous values of two signal vectors x and y. We can judge whether there exists the Granger causality from y to x by inspecting whether the past information from both time series significantly improves the prediction of the future of x, rather than using the past information from x alone. In other words, if the prediction error for the MVAR model (wt) is smaller than that for the AR model (et), it can be concluded that y causes x. In this way, Granger causality can be evaluated using
(2.3)F≡(RSS0-RSS1)/MRSS1/(T-2M-1),
where
(2.4)RSS0=∑i=1Tet2,RSS1=∑i=1Twt2,
where T represents the number of observations. To assess the statistical significance of the estimated Granger causality, the F-test with the null hypothesis, H0: βk=0 (i.e., yt does not influence the generation of xt) is generally used. If βk=0 for all k=1,2,…,p, the Granger causality value F becomes zero as RSS0 equals RSS1. Conversely, if the null hypothesis is rejected, that is, F is sufficiently large, it can be concluded that yt causes xt.To test this hypothesis, a traditionalF-test derived from an ordinary least squared regression for each equation can be used. To test the statistical significance of F, the cumulative F distribution is first estimated, after which the probability of the F value can be calculated by PGC=1-CDF(F), where CDF represents the cumulative distribution function and PGC represents the probability of Granger causality. For example, PGC=1 would indicate that no causal interaction exists between two time series, while PGC=0 would signal a strong directional influence (yt→xt).However, the MVAR model is problematic when estimating the appropriate model orderp. Basically, most model order estimation methods are based on the maximum likelihood principle, which allows the determination of the highest possible model order in MVAR signal modeling. Akaike information criterion (AIC) [26] is also based on this concept and was the earliest method to estimate MVAR model orders. As AIC generally chooses larger than optimal model orders, the Bayesian information criterion (BIC) [27]—which is based on the Bayes estimator—was developed by Schwarz. The BIC generally penalizes free parameters more strongly than the AIC, and thus provides more accurate estimates of MVAR model orders. Although several modifications of the AIC and BIC have been recently developed [28–35], the estimation of accurate and reliable model orders remains an important issue.


## 2.2. Directed Transfer Function

Directed transfer function (DTF) is a widely used tool in identifying information flow between multichannel neural signals. Even though both Granger causality and DTF are based on MVAR modeling, the DTF procedure differs slightly from Granger causality. As described above, Granger causality uses the variance of prediction errors to estimate the causal interaction, while DTF uses a matrix transfer function derived from MVAR model coefficients [9, 36]. In the framework of the MVAR model, a multivariate process of DTF can be described as a data vector X of N source signals: Xt=(X1(t),X2(t),…,XN(t))T. The MVAR model can then be constructed as
(2.5)Xt=∑k=1pAkXt-k+Et,
where Et represents a vector composed of white noise values at time t, Ak is an N×N matrix composed of the model coefficients, and p is the model order of MVAR. Note that (2.1) is a special case of (2.5) when N=1. The MVAR model is then transformed into the frequency domain as follows:
(2.6)X(f)=A-1(f)E(f)=H(f)E(f),
where f denotes a specific frequency and the H(f) matrix represents the so-called transfer matrix, which is defined as
(2.7)H(f)=A-1(f)=(∑k=0pAke-i2πfk∆t)-1,A0=-I,
where I is the identity matrix.The DTF can then be defined in terms of the elements of the transfer matrixHij as
(2.8)γij2(f)=|Hij(f)|2∑m=1k|Him(f)|2,
where γij2(f) denotes the ratio between inflow from signal j to signal i and all inflows to signal i and k represents the number of signals. The DTF ratio γij2(f) ranges from 0 to 1, with values approaching to 1, suggesting that signal i is caused by signal j, whereas values approaching to 0 indicating that no information flow from signal j to signal i exists at a specific frequency.


## 2.3. Partial Directed Coherence

Partial directed coherence (PDC) was proposed by Baccalá and Sameshima as a frequency domain counterpart to Granger causality [11] and is based on a spectral representation of (2.5), defined as
(2.9)A(f)=∑k=0pAke-i2πfk∆t,A̅(f)=I-A(f),
where I is an identity matrix [37]. The estimate of PDC from Xm to Xn is defined as(2.10)PDCXn←Xm(f)=A̅n,m(f)∑k=1Q|An,k(f)|2,
where A̅n,m is the (n,m)-th element of A̅.


## 2.4. Modified MVAR-Based Estimators

Although the MVAR-based causality estimators described above have been shown to be useful in many neuroscience problems, they are not applicable for all types of neural signals. For example, signals which have severely unbalanced model residual variances are not appropriate forPDC. Accordingly, several additional modified causality estimators, including generalized partial directed coherence [38, 39], Geweke’s Granger causality [40, 41], Wiener Granger causality [42], and direct directed transfer function [43], were subsequently developed to broaden application.The generalized partial directed coherence (gPDC) was first proposed by Baccalá et al. to circumvent the numerical problem associated with time series scaling [38], by which a variance stabilization of the frequency domain representation of lagged causality could be achieved [39]. The gPDC estimator is defined as
(2.11)πij(w)(f)=(1/σi)A̅ij(f)∑k=1N(1/σk2)A̅kj(f)A̅kj*(f),
where σi represents the variance of the ith input process. gPDC was modified from PDC to improve the identification of causal interactions between signals with severely unbalanced model residual variances [39].The Geweke’s Granger causality is derived from Geweke’s formulation [40, 41] and is defined as(2.12)Ik→l(f)=(Zkk-(Zlk2/Zll))|Hlk2||Sll(f)|,Il→k(f)=(Zll-(Zkl2/Zkk))|Hkl2||Skk(f)|,
where Skk(f) and Sll(f) represent the individual power spectra of sites k and l, respectively, and the expressions for Hlk can be found in (2.6). Zkk, Zll, Zlk, and Zkl are elements of the covariance matrix Z for the noise vector of the bivariate model. Geweke’s Granger causality at frequency f is expressed as the fraction of the total power at the frequency at one site that can be explained by the causal influence from the other. As seen in (2.12) and (15), Geweke’s Granger causality can be evaluated solely using the bivariate model. Recently, Bressler and Seth [42] introduced Wiener-Granger causality and discussed its merits and limitations in various neuroscience applications [42].The direct directed transfer function (dDTF) was proposed by Korzeniewska [43] for the analysis of direct information transfer among brain structures using local field potentials. To calculate the dDTF, partial coherence χij and full-band frequency DTF (ffDTF) ηij were independently defined as(2.13)χij2(f)=Mij2(f)Mjj(f)Mii(f),ηij2(f)=|Hij(f)|2∑f∑m=1k|Him(f)|2,
where Mij represents the minor produced by removing the ith row and jth column of a spectral matrix S. In multivariate signals, partial coherences may provide more specific information regarding causal interactions among signals than ordinary coherences [44]. The value of dDTF is defined as the product of the above two variables and can be expressed as
(2.14)δij(f)=χij2(f)ηij2(f).The dDTF method was proposed to circumvent some problems associated with DTF, specifically its inability to differentiate between the direct and indirect connections [43].


## 2.5. Examples of Practical Applications

To date, Granger causality has been extensively applied to the analysis of neural signals [45–58]. For examples, Hesse et al. used Granger causality to assess directed interdependencies between neural signal generators related to the Stroop task [45]. Seth also demonstrated that Granger causality may represent a useful tool for determining how interregional directional coupling is modulated by behavior [46]. Moreover, Sato et al. proposed a wavelet-based Granger causality, which they applied to fMRI signals [47]. Tang et al. applied both a blind source separation algorithm and Granger causality to the analysis of a high-density scalp EEG dataset and assessed the top-down and bottom-up influences [48]. Gow et al. demonstrated the potential value of combining Granger causality analyses with multimodal imaging to explore the functional architecture of cognition [49].The DTF algorithm has also been extensively applied to various aspects within neuroscience, particularly to the analyses of electrophysiological signals such as EEG, MEG, and iEEG, because frequency-domain analysis is generally required in these modalities. Franaszczuk et al. first applied the DTF algorithm to the localization of ictal onset zones in temporal lobe epilepsy patients [59, 60]. Astolfi et al. demonstrated that the DTF algorithm could be used to assess the time-varying functional connectivity patterns from noninvasive EEG recordings in human [61]. Babiloni et al. investigated cortical causal interactions from combined high-resolution EEG and fMRI data and showed that DTF was able to unveil the direction of the information flow between the cortical regions of interest [62]. Kuś et al. attempted to characterize EEG activity propagation patterns in beta and gamma bands during finger movements, demonstrating that short-time DTF can successfully identify frequency selective information from EEGs [63]. Ding et al. and Wilke et al. also applied the DTF algorithm to EEG and iEEG signals acquired from intractable partial epilepsy patients in order to better describe the structure of seizures in terms of space, time, and frequency [64, 65]. Kim et al. applied a source localization technique called FINEs in addition to DTF to iEEG signals and verified that directional connectivity analysis could be a useful tool to identify epileptogenic sources located outside of the iEEG electrodes [66].As with DTF, PDC analyses have recently been applied to a variety of practical applications: Sun et al. demonstrated that PDC is a useful tool for evaluating changes in cortical interdependences in the context of different psychotic or mental states and can also be used to diagnose affective disorders [21]. Similarly, Zhang et al. used PDC to estimate cortical interactive networks during the mental rotation of Chinese characters, demonstrating different changes in cortical networks according to task difficulty [18]. Furthermore, Zhu et al. studied the effects of brain development and aging on cortical interactive network pattern, demonstrating that the PDC analysis of EEG is a powerful approach for characterizing brain development and aging [24].


## 3. Non-MVAR-Based Causality Estimators

### 3.1. Transfer Entropy

Information theoretic measures have widely been utilized to quantify mutual dependence between time series. Although standard time-delayed mutual information can estimate mutual dependence between neural signals, it is not able to distinguish information flow [12]. To circumvent this issue, Schreiber developed a new causality estimator named transfer entropy (TE), on the basis of the entropy rate,
(3.1)hx=〈-log⁡[P(xn+1∣xn)]〉,
where <·> denotes an expectation value, P(x) represent the probability of x, P(xn+1∣xn) is the conditional probability of xn+1 given xn, and n is the time sample position. To estimate the information flow, the conditional entropy rate of xn+1 given both yn and xn(3.2)hx|y=〈-log⁡[P(xn+1∣xn,yn)]〉
has to be introduced. This indicated the average uncertainty about the future state (=xn+1) of x(t), conditional on the current state (=yn) of y(t) as well as on its own current state (=xn). The transfer entropy can be defined as the difference between hx and hx|y [67], in the following form:(3.3)Ty→x=∑P(xn+1,xn,yn)log⁡⁡P(xn+1∣xn,yn)P(xn+1∣xn),
where P(xn+1,xn,yn) is the joint probability, evaluated by the sum of all available realizations of  (xn+1,xn,yn) in time series.Many researchers now apply the TE algorithm to the field of neuroscience [67–71], as TE has been demonstrated to be more sensitive to nonlinear signal properties than the conventional MVAR-based causality estimators [69]. However, TE analyses are restricted to bivariate situations and require substantially more data samples than MVAR-based causality estimators.


### 3.2. Phase Slope Index

To robustly estimate the direction of information flow in multivariate time series, Nolte proposed a new causality estimator called phase slope index (PSI) [13], basic assumption of which states that mixing does not affect the imaginary part of the complex coherency of a multivariate times series [72]. Measured data Yt are assumed to be a superposition of two sources Xt and additive noise Et(3.4)Yt=Xt+BEt,
where B represents a mixing matrix that merges the additive noise into the measurement channels. The measured data are then divided into K segments and used to calculate the cross-spectral density as follows:
(3.5)Sij(f)=1K∑kzi(f,k)zj*(f,k),
where zi(f,k) represents the Fourier transform of the ith channel data and kth segment and Sij is the cross-spectral matrix between ith and jth time series. PSI is defined as(3.6)Ψ̃ij=I(∑f∈FCij*(f)Cij(f+δf)),
where(3.7)Cij(f)=Sij(f)Sii(f)Sjj(f)
is the complex coherency, δf is the specific frequency resolution, F is the frequency band of interest, and ℑ(·) denotes the imaginary part. Finally, the PSI is normalized using its standard deviation and is expressed as(3.8)Ψ=Ψ̃std(Ψ̃),
Nolte et al. presented several computer simulations, via which the relative performances of Granger causality and PSI were compared. In these simulations, PSI was found to perform better than Granger causality in inferring causal relationship between signals with nonlinear interactions. As the PSI is a nonparametric approach, it has several key advantages over conventional parametric approaches represented by the MVAR models. For instance, the PSI not only requires a lower computational load than the MVAR-based approaches, but it is also independent from the signal’s stationarity. However, the PSI has a limitation in that it is also a pairwise metric of directional interactions and is thereby vulnerable to the ambiguity between direct and indirect influences [25].


### 3.3. Nonlinear Granger Causality

To estimate causal interactions between the nonlinear bivariate neural signals, nonlinear Granger causality (NGC) was developed [73, 74]. The basic concept of NGC is similar to TE in that NGC concludes that y(t) does not cause x(t) if the value of hx in (3.1) is comparable to hx|y in (3.2). Gourévitch [37] defined the nonlinear Granger causality as follows:(3.9)NGCx←y=C2(xn+1,xn,yn)C2(xn,yn)-C2(xn+1,xn)C2(xn),
where C2 is the correlation integral of order 2. This correlation integral was proposed by Grassberger [75]. For any given vectorial signal dimension L and length of signal T, the correlation integral of order q is defined as(3.10)Cq(X)=(1T-L∑t=L+1T(1(T-L-1)∑s=L+1,s≠tT1{||X(t)-X(s)||<r})q-1)1/(q-1),
where ||·|| represents the maximum norm, 1A is 1 in a set A, 0 otherwise, and r is a positive scalar. The bivariate version for two signals X and Y of the same dimension L and the same length T is expressed as(3.11)C2(X,Y)=1(T-L)(T-L-1)∑t=L+1T∑s=L+1,s≠tT1{||X(t)-X(s)||<r}1{||Y(t)-Y(s)||<r}.


### 3.4. Partial Nonlinear Granger Causality

Recently, Gourévitch et al. proposed a new method for estimating nonlinear causal interactions [37], termed partial nonlinear Granger causality (PNGC). The PNGC algorithm is able to estimate direct causality from Xm to Xn when Q signals are considered. PNGC is defined as(3.12)PNGCxn←xm=C2(Xnf,X1p,…,XQp)C2(X1p,…,XQp)-C2(Xnf,X1p,…,Xm-1p,Xm+1p,XQp)C2(X1p,…,Xm-1p,Xm+1p,XQp),Although PNGC showed promising results when applied to complex systems, it is still dependent on model order and scale [37]. Consequently, if nonlinearity is suspected, PNGC should be used only as a complementary tool.


## 3.1. Transfer Entropy

Information theoretic measures have widely been utilized to quantify mutual dependence between time series. Although standard time-delayed mutual information can estimate mutual dependence between neural signals, it is not able to distinguish information flow [12]. To circumvent this issue, Schreiber developed a new causality estimator named transfer entropy (TE), on the basis of the entropy rate,
(3.1)hx=〈-log⁡[P(xn+1∣xn)]〉,
where <·> denotes an expectation value, P(x) represent the probability of x, P(xn+1∣xn) is the conditional probability of xn+1 given xn, and n is the time sample position. To estimate the information flow, the conditional entropy rate of xn+1 given both yn and xn(3.2)hx|y=〈-log⁡[P(xn+1∣xn,yn)]〉
has to be introduced. This indicated the average uncertainty about the future state (=xn+1) of x(t), conditional on the current state (=yn) of y(t) as well as on its own current state (=xn). The transfer entropy can be defined as the difference between hx and hx|y [67], in the following form:(3.3)Ty→x=∑P(xn+1,xn,yn)log⁡⁡P(xn+1∣xn,yn)P(xn+1∣xn),
where P(xn+1,xn,yn) is the joint probability, evaluated by the sum of all available realizations of  (xn+1,xn,yn) in time series.Many researchers now apply the TE algorithm to the field of neuroscience [67–71], as TE has been demonstrated to be more sensitive to nonlinear signal properties than the conventional MVAR-based causality estimators [69]. However, TE analyses are restricted to bivariate situations and require substantially more data samples than MVAR-based causality estimators.


## 3.2. Phase Slope Index

To robustly estimate the direction of information flow in multivariate time series, Nolte proposed a new causality estimator called phase slope index (PSI) [13], basic assumption of which states that mixing does not affect the imaginary part of the complex coherency of a multivariate times series [72]. Measured data Yt are assumed to be a superposition of two sources Xt and additive noise Et(3.4)Yt=Xt+BEt,
where B represents a mixing matrix that merges the additive noise into the measurement channels. The measured data are then divided into K segments and used to calculate the cross-spectral density as follows:
(3.5)Sij(f)=1K∑kzi(f,k)zj*(f,k),
where zi(f,k) represents the Fourier transform of the ith channel data and kth segment and Sij is the cross-spectral matrix between ith and jth time series. PSI is defined as(3.6)Ψ̃ij=I(∑f∈FCij*(f)Cij(f+δf)),
where(3.7)Cij(f)=Sij(f)Sii(f)Sjj(f)
is the complex coherency, δf is the specific frequency resolution, F is the frequency band of interest, and ℑ(·) denotes the imaginary part. Finally, the PSI is normalized using its standard deviation and is expressed as(3.8)Ψ=Ψ̃std(Ψ̃),
Nolte et al. presented several computer simulations, via which the relative performances of Granger causality and PSI were compared. In these simulations, PSI was found to perform better than Granger causality in inferring causal relationship between signals with nonlinear interactions. As the PSI is a nonparametric approach, it has several key advantages over conventional parametric approaches represented by the MVAR models. For instance, the PSI not only requires a lower computational load than the MVAR-based approaches, but it is also independent from the signal’s stationarity. However, the PSI has a limitation in that it is also a pairwise metric of directional interactions and is thereby vulnerable to the ambiguity between direct and indirect influences [25].


## 3.3. Nonlinear Granger Causality

To estimate causal interactions between the nonlinear bivariate neural signals, nonlinear Granger causality (NGC) was developed [73, 74]. The basic concept of NGC is similar to TE in that NGC concludes that y(t) does not cause x(t) if the value of hx in (3.1) is comparable to hx|y in (3.2). Gourévitch [37] defined the nonlinear Granger causality as follows:(3.9)NGCx←y=C2(xn+1,xn,yn)C2(xn,yn)-C2(xn+1,xn)C2(xn),
where C2 is the correlation integral of order 2. This correlation integral was proposed by Grassberger [75]. For any given vectorial signal dimension L and length of signal T, the correlation integral of order q is defined as(3.10)Cq(X)=(1T-L∑t=L+1T(1(T-L-1)∑s=L+1,s≠tT1{||X(t)-X(s)||<r})q-1)1/(q-1),
where ||·|| represents the maximum norm, 1A is 1 in a set A, 0 otherwise, and r is a positive scalar. The bivariate version for two signals X and Y of the same dimension L and the same length T is expressed as(3.11)C2(X,Y)=1(T-L)(T-L-1)∑t=L+1T∑s=L+1,s≠tT1{||X(t)-X(s)||<r}1{||Y(t)-Y(s)||<r}.


## 3.4. Partial Nonlinear Granger Causality

Recently, Gourévitch et al. proposed a new method for estimating nonlinear causal interactions [37], termed partial nonlinear Granger causality (PNGC). The PNGC algorithm is able to estimate direct causality from Xm to Xn when Q signals are considered. PNGC is defined as(3.12)PNGCxn←xm=C2(Xnf,X1p,…,XQp)C2(X1p,…,XQp)-C2(Xnf,X1p,…,Xm-1p,Xm+1p,XQp)C2(X1p,…,Xm-1p,Xm+1p,XQp),Although PNGC showed promising results when applied to complex systems, it is still dependent on model order and scale [37]. Consequently, if nonlinearity is suspected, PNGC should be used only as a complementary tool.


## 4. Mathematical Issues in Causality Inference

### 4.1. Issues in MVAR-Based Causality Inference

The most popular causality estimators—GC [8], DTF [9], and PDC [11]—as well as their modifications are based on MVAR modeling of neural signals. The MVAR modeling is highly dependent on the selection of model orders: too low order may not provide an exact expression of the signal feature, while too high model order may result in overfitting. Thus, the correct choice of an MVAR model orders is critically important for precise causality inference. Although several methods have been proposed to estimate proper model orders (like AIC [26], BIC [27], deviance information criterion (DIC) [33], minimum description length (MDL) [30], focused information criterion (FIC) [34], minimum message length (MML) [28], and others [29, 31, 33]) and some investigators have attempted to compare performances of different model order determination criteria [35, 76], no golden rule exists for the model order selection, and further research is clearly needed.Moreover, MAVR-based causality estimators guarantee accurate causality inference only when datasets (signals) satisfy stationary conditions [9, 77], whereby their multivariate probability distribution is not affected by timeshift. At the very least, the mean, variance, and autocorrelation of multivariate time series should not vary over time. Unfortunately, these conditions cannot be satisfied in most cases, and thus, some mathematical transformations are often required to make the time series become roughly stationary. Nolte et al. demonstrated that MVAR-based approaches typically fail to estimate causal interactions between neuronal signals that are not stationary [13]. Although several stationary tests (e.g., unit root test [78] and Sargan and Bhargava test [79]) have been introduced to assess whether or not a timeseries is stationary, most were not verified in practical neural signals. In neuroscience applications, issues of stationarity also have to be carefully dealt with by considering an empirical appraisal of the participants’ behavioral states [11].Another critical limitation affecting the reliability of causality estimators is the linear modeling of neural signals [80]. Neural time-series signals can take several forms; for example, spikes, noisy signal, and highly correlated signals, may have a nonlinear form [37]. Accordingly, it is imperative to develop techniques for causality analysis that accommodate nonlinear time series, as most current studies on the causal network inference do not verify signal linearity, nor do they account for nonlinearity. Specifically, many MVAR-based models (such as PDC) are not robust to simple nonlinear linkage [37].Generally, MVAR-based causality estimators require the appropriate selection of signal sample number. In one study, Schlögl assessed the dependency of several MVAR algorithms on the number of time samples, demonstrating that sufficient numbers of samples are required to obtain a reliable estimate of causal interactions among neural signals [81]. Moreover, Schlögl also showed causality inference to be highly dependent on both MVAR estimation methods as well as model order in cases with the same number of time samples. As the number of time samples is generally limited in most practical examples, a more systematic approach to reliably determine the number of time samples and appropriate MVAR estimators should be developed in future studies.


### 4.2. Issues in Non-MVAR-Based Causality Inference

While most non-MVAR-based causality estimators, such as PNGC, nonlinear Granger causality, TE, and PSI, were introduced to circumvent the well-described problems of MVAR-based causality estimators, many can only be applied to causality inferences of bivariate neural signals. As such, further research is required to extend bivariate causality inferences to include multivariate (more than three) causality inference. Furthermore, a method for determining the proper model order in PNGC remains an ongoing problem [37], as with MVAR-based estimators.


## 4.1. Issues in MVAR-Based Causality Inference

The most popular causality estimators—GC [8], DTF [9], and PDC [11]—as well as their modifications are based on MVAR modeling of neural signals. The MVAR modeling is highly dependent on the selection of model orders: too low order may not provide an exact expression of the signal feature, while too high model order may result in overfitting. Thus, the correct choice of an MVAR model orders is critically important for precise causality inference. Although several methods have been proposed to estimate proper model orders (like AIC [26], BIC [27], deviance information criterion (DIC) [33], minimum description length (MDL) [30], focused information criterion (FIC) [34], minimum message length (MML) [28], and others [29, 31, 33]) and some investigators have attempted to compare performances of different model order determination criteria [35, 76], no golden rule exists for the model order selection, and further research is clearly needed.Moreover, MAVR-based causality estimators guarantee accurate causality inference only when datasets (signals) satisfy stationary conditions [9, 77], whereby their multivariate probability distribution is not affected by timeshift. At the very least, the mean, variance, and autocorrelation of multivariate time series should not vary over time. Unfortunately, these conditions cannot be satisfied in most cases, and thus, some mathematical transformations are often required to make the time series become roughly stationary. Nolte et al. demonstrated that MVAR-based approaches typically fail to estimate causal interactions between neuronal signals that are not stationary [13]. Although several stationary tests (e.g., unit root test [78] and Sargan and Bhargava test [79]) have been introduced to assess whether or not a timeseries is stationary, most were not verified in practical neural signals. In neuroscience applications, issues of stationarity also have to be carefully dealt with by considering an empirical appraisal of the participants’ behavioral states [11].Another critical limitation affecting the reliability of causality estimators is the linear modeling of neural signals [80]. Neural time-series signals can take several forms; for example, spikes, noisy signal, and highly correlated signals, may have a nonlinear form [37]. Accordingly, it is imperative to develop techniques for causality analysis that accommodate nonlinear time series, as most current studies on the causal network inference do not verify signal linearity, nor do they account for nonlinearity. Specifically, many MVAR-based models (such as PDC) are not robust to simple nonlinear linkage [37].Generally, MVAR-based causality estimators require the appropriate selection of signal sample number. In one study, Schlögl assessed the dependency of several MVAR algorithms on the number of time samples, demonstrating that sufficient numbers of samples are required to obtain a reliable estimate of causal interactions among neural signals [81]. Moreover, Schlögl also showed causality inference to be highly dependent on both MVAR estimation methods as well as model order in cases with the same number of time samples. As the number of time samples is generally limited in most practical examples, a more systematic approach to reliably determine the number of time samples and appropriate MVAR estimators should be developed in future studies.


## 4.2. Issues in Non-MVAR-Based Causality Inference

While most non-MVAR-based causality estimators, such as PNGC, nonlinear Granger causality, TE, and PSI, were introduced to circumvent the well-described problems of MVAR-based causality estimators, many can only be applied to causality inferences of bivariate neural signals. As such, further research is required to extend bivariate causality inferences to include multivariate (more than three) causality inference. Furthermore, a method for determining the proper model order in PNGC remains an ongoing problem [37], as with MVAR-based estimators.


## 5. Conclusion

Here, we summarized the mathematical techniques used in causality estimation, all of which have been extensively applied to infer causal relationships among multichannel neural signals. We also described the limitations of current methods and presented several ongoing problems, some of which may be of interest to applied mathematicians. We hope that this paper will serve as a useful guide for researchers in the field of applied mathematics and helps raise awareness of this important research topic.


## Author Contribution

Y.-J. Jung and K.-H. Kim are co-first authors and contributed equally to this work.



---

*Source: 472036-2011-12-29.xml*