SlowGuess/NeurIPS_2023_OCR huggingface dataset:
in GC1 exhibited a faster initial oscillation than those in GC2 (Fig 3a-d, bottom panels). From visual inspection of the time courses, the driving signal in each component pair appears to lead the driven signal. Note that while the spatial topographies are flipped with respect to the cued hand, the time courses are qualitatively similar between left and right motor imagery.
The power spectra of the recovered components, as well as those of several individual electrodes, are depicted in Figure S2. Of note, it is interesting that the driving signal of GC2 has relatively large  $10\mathrm{Hz}$  "mu" power [33], whereas the corresponding driven signal has relatively low  $10\mathrm{Hz}$  power. This appears to be consistent with the driving signal being preparatory ( $10\mathrm{Hz}$  activity is synchronized) and the driven signal reflecting execution ( $10\mathrm{Hz}$  desynchronization).
The magnitude of Granger causality between all pairs of electrodes (represented here as  $-J$  so that positive values indicate a strong relationship) is depicted in Fig 3e for the right motor imagery condition (see Fig S3 for the results from left motor imagery). The single strongest relationship is 0.12, and it is difficult to interpret the overall pattern of Granger causalities. The matrix of Granger causality strengths for the recovered components is shown in Fig 3f. In addition to larger magnitudes compared to those observed between individual electrodes (as high as 0.39 for  $y_{1} \rightarrow z_{1}$ ), the bottom row of the matrix suggests that  $y_{2}$  drives both  $z_{1}$  and  $z_{2}$ . Given the left frontal topography of  $y_{2}$  and the more central topographies of  $z_{1}$  (ipsilateral to cued hand) and  $z_{2}$  (contralateral to cued hand), this is suggestive of an EEG source that projects to both the activated and inhibited sources.
For comparison to existing approaches, the spatial topographies of the components recovered by PCA, ICA, and MVARICA [23] are depicted in Fig S4. The components recovered by the proposed method were not evident in those yielded by these conventional approaches (compare Fig 3 with Fig S4). Moreover, it is notable that the lateralization of the forward models with respect to the side of the cued hand is not readily apparent in Fig S4.
Resting-state fMRI
At rest, the human brain exhibits stereotyped patterns of activity in the form of resting-state networks (RSN) [34, 35, 36]. Perhaps the most well-known of these is the default-mode network (DMN) [37], a collection of brain regions that appears to become active during introspection and mind-wandering. A previously collected fMRI dataset [38] from  $n = 20$  healthy adults in the resting state (eyes open) was employed here to investigate whether GCA could recover components (i.e., sets of brain regions) that reflect the presence of these previously reported networks. As the original study investigated the effects of near-infrared brain stimulation on fMRI activity, only the first 10 minutes of the 30 minute recording (i.e., the segment occurring before the onset of stimulation) was analyzed here. The input data was parcellated into  $D = 150$  regions-of-interest (ROIs) following a standard brain atlas [39]. The time series of each ROI was formed by computing the mean of all grey matter voxels in the ROI at each time sample. The temporal aperture was set to  $L = 4$  samples ( $\approx 11$  s) and the number of components to  $P = 6$ . Block covariance matrices were pooled across participants prior to analysis, leading to a group analysis. The recovered components were then analyzed by examining the individual brain regions most strongly expressed by the forward model [29] of each driving and driven component.
The five regions with strongest expression in the forward model of each driving and driven component are listed in Table 1. Interestingly, the ROI most strongly expressed in the driving signal of GC1 was the right angular gyrus, one of the three hubs of the DMN [37] (Fig 4a, top row). The left angular gyrus was the fourth strongest driving signal in GC1. The driven regions of GC1 were bilateral visual areas and included the primary visual cortex (Fig 4a, bottom row). The finding of the DMN and visual networks in the first Granger pair is consistent with this eyes-open, resting dataset [40].
The remaining Granger components tended to cluster into networks that have also been previously reported in the literature. For example, the driving signal of GC2 consisted of the precuneus and posterior cingulate, the other two hubs of the DMN (the pericallosal sulcus borders the posterior cingulate; Fig 4b, top). The driven portion of GC2 was comprised of a contiguous region in the parietal cortex (Fig 4b, bottom), an area belonging to the "sensorimotor" network [40].
The analysis also identified a network in the right frontal region that appeared in the driving portion of GC3 as well as the driven portion of GC4 (Fig 4c, top and Fig 4d, bottom). A set of ROIs in the left temporal cortex was identified in the driving signal of GC4, while the expression of the insular
Figure 4: Granger components measured on resting state fMRI data. (a) (Top) The driving signal of GC1 expressed the left and right angular gyrus, a hub of the DMN. (Bottom) The corresponding driven signal identified a visual network including the primary visual cortices. (b) The driving portion of GC2 included the other hubs of the DMN, namely the posterior cingulate (PCi) and the precuneus (PCN). Another commonly reported resting-state network, the sensorimotor region, was expressed in the driven component of GC2. (c)-(f) The brain regions comprising Granger components 3 through 6 were generally clustered into contiguous regions, some of which matched known RSNs (e.g. the "Salience Network" in the driving signal of GC6).
cortex and anterior cingulate in driving GC6 is consistent with another well-known RSN known as the "salience network" [41].
Overall, GCA recovered components whose spatial distribution is consistent with multiple RSNs previously reported in the fMRI literature. Note that the GCs measured here tended to capture RSNs that interact with one another, as opposed to interacting brain regions from the same RSN.
There are several limitations of the present work. The proposed method is limited to capturing components that are expressed linearly in the observations. In the high-dimensional case (large  $LD$ ), estimation of the block covariance matrices  $\Sigma_{1:L}$  and  $\tilde{\Sigma}$  is challenging, particularly in the case of low sample sizes. Regularization approaches more sophisticated than the Tikhonov approach employed here may alleviate this challenge [42].
Although the components recovered by GCA are ordered according to the strength of the Granger causal relationship, this work has not proposed a methodology for selecting the number of components  $P$  to estimate (or retain). One post hoc approach is to perform statistical significance testing on the recovered Granger component pairs to assess the likelihood of a non-spurious relationship. This may be accomplished in a non-parametric fashion by employing the method on surrogate data lacking any Granger causal structure, for example by randomizing the phase spectrum of the observed signals [43]. Furthermore, this work has not considered a methodology for selection of the autoregressive
Table 1: Granger components of resting-state brain activity. Listed are the five ROIs most strongly expressed in the forward models of the driving and driven components of the first  $P = 6$  Granger components. Where applicable, the ROIs have been labeled according to previously reported resting-state networks: Visual, Default Mode Network (DMN), Central Executive Network (CEN), Salience Network, Sensorimotor Network.
y1	z1	y2	z2
1 (L) G pariet inf-Angular	(R) Pole occipital	(R) S pericallosal	(R) G postcentral
2 (L) G occipital middle	(L) Pole occipital	(L) G precuneus	(L) G postcentral
3 (L) G parietal sup	(R) G+S occipital inf	(R) G cingul-Post-dorsal	(R) S postcentral
4 (R) G pariet inf-Angular	(L) G+S occipital inf	(L) S pericallosal	(L) G+S subcentral
5 (L) S intrapariet and P tran	(R) S collat transv post	(R) S subparietal	(L) S postcentral
y3	z3	y4	z4
1 (R) S front inf	(L) S subparietal	(L) S temporal sup	(R) S front inf
2 (R) G front middle	(L) G cingul-Post-ventral	(L) G temporal middle	(R) G front inf-Triangul
3 (R) S precentral-inf-part	(L) G cingul-Post-dorsal	(L) S interm prim-Jensen	(R) S interm prim-Jensen
4 (R) S intrapariet and P tran	(R) G cingul-Post-dorsal	(L) G front sup	(R) G front middle
5 (R) G front inf-Triangul	(R) S subparietal	(L) G temp sup-Lateral	(R) S front middle
y5	z5	y6	z6
1 (R) G occipital sup	(R) S front middle	(R) S circular insula sup	(L) G parietal sup
2 (L) G occipital sup	(R) G front middle	(L) S circular insula sup	(R) G parietal sup
3 (R) S oc middle and Lunatus	(L) S interm prim-Jensen	(R) Lat Fis-post	(L) S intrapariet and P tran
4 (R) S oc sup and transversal	(R) S orbital lateral	(L) G insular short	(R) G precuneus
5 (L) G+S occipital inf	(R) G+S transv frontopol	(L) G+S cingul-Mid-Ant	(R) S intrapariet and P tran
model order  $L$ . Knowledge of the underlying systems was utilized in the two real-world examples here to select the history lengths (i.e., 500 ms for EEG and 4 samples for fMRI).
In defining the GCA objective (3), it has been assumed that the Granger causal relationships between latent variables are unidirectional: if  $y$  Granger-causes  $z$ , then  $z$  does not Granger-cause  $y$ . An extension of GCA that instead maximizes net Granger causality [12] and accounts for bidirectional relationships may facilitate the recovery of pairs of sources that exhibit temporal dependencies in both directions.
This work has proposed an approach to blind source separation that is applicable when the underlying sources are related by a Granger causal relationship (i.e., the past of one signal forecasts the present of another). In contrast to conventional approaches that require statistical independence for identification, the present approach exploits temporal dependencies to recover pairs of components with a directed relationship. A coordinate descent algorithm that decomposes a multivariate dataset into pairs of components, ranked by the strength of the temporal dependency, was then developed and evaluated on simulated and real datasets. Future efforts may extend this line of work to permit non-iterative solutions to the linear GCA problem, or propose modifications that allow one to recover nonlinear temporal dependencies.
The code that was employed to generate the experimental results is available at https://github. com/dmochow/gca.
The author would like to thank the anonymous Reviewers for providing many useful comments and critiques that helped to improve the quality of the paper.
[1] David R Hardoon, Sandor Szedmak, and John Shawe-Taylor. Canonical correlation analysis: An overview with application to learning methods. Neural computation, 16(12):2639-2664, 2004.
[2] Aapo Hyvarinen. Independent component analysis: recent advances. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 371(1984):20110534, 2013.
[3] Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. *Econometrica: journal of the Econometric Society*, pages 424–438, 1969.
[4] John Geweke. Measurement of linear dependence and feedback between multiple time series. Journal of the American statistical association, 77(378):304-313, 1982.
[5] John F Geweke. Measures of conditional linear dependence and feedback between time series. Journal of the American Statistical Association, 79(388):907-915, 1984.
[6] Craig Hiemstra and Jonathan D Jones. Testing for linear and nonlinear granger causality in the stock price-volume relation. The Journal of Finance, 49(5):1639-1664, 1994.
[7] Nicola Ancona, Daniele Marinazzo, and Sebastiano Stramaglia. Radial basis function approach to nonlinear granger causality of time series. Physical Review E, 70(5):056221, 2004.
[8] Daniele Marinazzo, Mario Pellicorò, and Sebastiano Stramaglia. Kernel method for nonlinear granger causality. Physical review letters, 100(14):144103, 2008.
[9] Adam B Barrett, Lionel Barnett, and Anil K Seth. Multivariate granger causality and generalized variance. Physical Review E, 81(4):041907, 2010.
[10] Lionel Barnett and Anil K Seth. The mvgc multivariate granger causality toolbox: a new approach to granger-causal inference. Journal of neuroscience methods, 223:50-68, 2014.
[11] Alex Tank, Ian Covert, Nicholas Foti, Ali Shojaié, and Emily Fox. Neural granger causality. arXiv preprint arXiv:1802.05842, 2018.
[12] Irene Winkler, Danny Panknin, Daniel Bartz, Klaus-Robert Müller, and Stefan Haufe. Validity of time reversal for testing granger causality. IEEE Transactions on Signal Processing, 64(11):2746-2760, 2016.
[13] Clemens Brunner, Martin Billinger, Martin Seeber, Timothy R Mullen, and Scott Makeig. Volume conduction influences scalp-based connectivity estimates. Frontiers in computational neuroscience, 10:121, 2016.
[14] Frederik Van de Steen, Luca Faes, Esin Karahan, Jitkomut Songsiri, Pedro A Valdes-Sosa, and Daniele Marinazzo. Critical comments on eeg sensor space dynamical connectivity analysis. Brain topography, 32:643-654, 2019.
[15] Helmut Lütkepohl. New introduction to multiple time series analysis. Springer Science & Business Media, 2005.
[16] Harold Hotelling. Relations between two sets of variates. In *Breakthroughs in statistics*, pages 162-190. Springer, 1992.
[17] Shotaro Akaho. A kernel method for canonical correlation analysis. arXiv preprint cs/0609071, 2006.
[18] Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. Deep canonical correlation analysis. In International conference on machine learning, pages 1247-1255. PMLR, 2013.
[19] Pieter W Otter. On wiener-granger causality, information and canonical correlation. Economics Letters, 35(2):187-191, 1991.
[38] Grzegorz M Dmochowski, Ahmed Duke Shereen, Destiny Berisha, and Jacek P Dmochowski. Near-infrared light increases functional connectivity with a non-thermal mechanism. Cerebral Cortex Communications, 1(1):tgaa004, 2020.
[39] Christophe Destrieux, Bruce Fischl, Anders Dale, and Eric Halgren. Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature. Neuroimage, 53(1):1-15, 2010.
[40] Malaak N Moussa, Matthew R Steen, Paul J Laurienti, and Satoru Hayasaka. Consistency of network modules in resting-state fmri connectome data. 2012.
[41] Vinod Menon and Lucina Q Uddin. Saliency, switching, attention and control: a network model of insula function. Brain structure and function, 214:655-667, 2010.
[42] Karl Werner, Magnus Jansson, and Petre Stoica. On estimation of covariance matrices with kronecker product structure. IEEE Transactions on Signal Processing, 56(2):478-491, 2008.
[43] James Theiler, Stephen Eubank, André Longtin, Bryan Galdrikian, and J Doyne Farmer. Testing for nonlinearity in time series: the method of surrogate data. Physica D: Nonlinear Phenomena, 58(1-4):77-94, 1992.