Title: Attentive Transfer Entropy to Exploit Transient Emergence of Coupling Effect
Venue: NeurIPS 2023
Accepted: yes
Keywords: Directed coupled network reconstruction; Neuronal dynamics; Mutual information estimator; Attention mechanism; Transfer entropy.
Abstract: We consider the problem of reconstructing coupled networks (e.g., biological neural networks) connecting large numbers of variables (e.g.,nerve cells), of which state evolution is governed by dissipative dynamics consisting of strong self-drive (dominants the evolution) and weak coupling-drive. The core difficulty is sparseness of coupling effect that emerges (the coupling force is significant) only momentarily and otherwise remains quiescent in time series (e.g., neuronal activity sequence). Here we learn the idea from attention mechanism to guide the classifier to make inference focusing on the critical regions of time series data where coupling effect may manifest. Specifically, attention coefficients are assigned autonomously by artificial neural networks trained to maximise the Attentive Transfer Entropy (ATEn), which is a novel generalization of the iconic transfer entropy metric. Our results show that, without any prior knowledge of dynamics, ATEn explicitly identifies areas where the strength of coupling-drive is distinctly greater than zero. This innovation substantially improves reconstruction performance for both synthetic and real directed coupling networks using data generated by neuronal models widely used in neuroscience.

Attentive Transfer Entropy to Exploit Transient Emergence of Coupling Effect

Abstract

We consider the problem of reconstructing coupled networks (e.g., biological neural networks) connecting large numbers of variables (e.g., nerve cells) for which state evolution is governed by dissipative dynamics consisting of strong self-drive which dominates the evolution and weak coupling-drive. The core difficulty is sparseness of coupling effect, which emerges with significant coupling force only momentarily and otherwise remains quiescent in time series (e.g., neuronal activity sequence). Here we propose an attention mechanism to guide the classifier to make inference focusing on the critical regions of time series data where coupling effect may manifest. Specifically, attention coefficients are assigned autonomously by artificial neural networks trained to maximise the Attentive Transfer Entropy (ATEn), which is a novel generalization of the iconic transfer entropy metric. Our results show that, without any prior knowledge of dynamics, ATEn explicitly identifies areas where the strength of coupling-drive is distinctly greater than zero. This innovation substantially improves reconstruction performance for both synthetic and real directed coupling networks using data generated by neuronal models widely used in neuroscience.

1 Introduction

In this work, our task is to infer coupling relationships between observed variables based on time series data and reconstruct the coupled network connecting large numbers of these variables (see Figure 1a). Assume the time series x it records the state evolution of variable i governed by coupled dissipative dynamics, as represented by a general differential equation ˙ x it = g ( x it ) + ∑ B ij f ( x it , x jt ) , where g and f are self- and coupling functions respectively. The parent variable influences the evolution behavior of the child variable via the coupling function f . Note that these two functions are hidden and usually unknown for real systems. The asymmetric adjacency matrix B represents the directional coupling relationship between variables. Hence, the goal is to infer matrix B from observed time series x it , i = 1 , 2 , . . . , N where N is the number of variables in the system. If B ij = 1 , the variable j is a coupling driver (parent) of variable i , otherwise it is zero.

In neural dynamics (e.g., biological neuronal systems observed via neuronal activity sequences), the coupling effect is usually too weak to be detected, making less applicable the classic unsupervised techniques used across multiple research communities to infer coupling relationships [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]. This difficulty manifests in three aspects. First, the dynamics contains selfdrive and coupling-drive. The strength of the coupling force f ( x it , · ) is usually many orders of

( b )   M ethod

( a)   R eco n st r u ct   co u p l ed   n et w o r k

acti vi t y

sequence

OrderedPairofSequences

{(xiyi)}=1

Input

Coupling

Transfer

Stage

Attention

Entropy

Estimator

{ai}=1

{(xi,yi) *ai=1

MI1

Ml2

2

Stage

Binary

Classifier

Infer

ATEn

Max{logiti]i=1

magnitude smaller than self-drive g ( x it ) , and the latter dominates the evolution of variable state. Second, the behavior of the coupling-drive can be chaotic, unlike in linear models [11, 12]. The resulting unpredictability and variability of system state means that coupling force can be significant momentarily and otherwise almost vanish, as illustrated in Figure 4 (gray lines). This dilutes the information in time series that can be useful for inferring the coupling relationship. Third, in the heterogeneous networks common in applications, some variables are hubs coupled with many parent variables, among which it is difficult to distinguish individual drivers.

When coupling effects are weak and sparse, we do not observe clearly the principle of Granger Causality, whereby the parent variable can help to explain the future state change in its child variable [13]. Rather, when we train a model [5, 8] for prediction task on the neuronal activity sequences, the model only exploits the historical information of the child variable itself and that from parent variables is ignored. We posit that coupling-drive makes a negligible contribution to state evolution in the majority of samples of time series data. In other words, only in a small fraction of samples is the information of parent variables effective in predicting the evolution of child variables. Taking as an example the gradient algorithm to minimise the regression error over all samples ∑ t ( x it -ˆ x it ) 2 , the adjustment of model parameters from the tiny samples corresponding to significant coupling force is negligible, but these are the only samples which could induce the model to exploit coupling effects in reducing regression error. Similarly, for transfer entropy [2], which measures the reduction in uncertainty which a potential parent variable provides to a potential child variable, there is no significant difference in measured value between ordered pairs of variables with and without coupling relation.

To overcome the difficulty, we introduce coupling attention to identify the moments when coupling effect emerges. We design an objective function, A ttentive T ransfer En tropy ( ATEn ), comprising a weighted generalisation of transfer entropy. In order to maximize ATEn , an artificial neural network is trained to autonomously allocate high attention coefficients a t at times t where information of parent variables effectively reduces the future uncertainty of child variables, and ignores other positions by setting a t close to zero.

However, coupling attention also detects high transfer entropy regions produced by factors unrelated to coupling-drive, e.g., noise in empirical samples, which leads to spurious identification of spurious coupling effects between variables, which disturb the inference. To ameliorate this, we consider

utilising known relationships between variables, a small number of which could be inferred, e.g., by manual neuron tracing in connectome [14]. We leverage this limited number of known relationships for more effective detection of connectivity which advances in technology are making increasingly simple and cheap to observe [15]. Specifically, we add a binary classification model to perform more sophisticated inference under the guidance of coupling attention to focus on these critical regions and recognize different patterns between real and spurious coupling effect. We deal with this coupling relation inference task by way of small sample supervised learning. Although training and test data have a distribution shift in the setting of small samples, they arise through an identical underlying generation process. Thus, if the model provides an insight into the underlying dynamics the coupling-drive in our task - then the understanding acquired from small samples can be effectively utilised in the test environment [16, 17, 18, 19]. The role of coupling attention is to help the classification model gain this insight. Our contributions are summarized as follows:

We draw on the coupling attention mechanism to identify the positions of time series at which coupling effect emerges and guiding a classification model to infer coupling relation by focusing on these critical positions. Without any prior knowledge of dynamics, this mechanism determines the areas where the coupling force is substantially different from zero.

By formulating Transfer Entropy as the difference between two types of mutual information, and based on the dual representation of Kullback-Liebler (KL) divergence, we design a differentiable metric, Attentive Transfer Entropy, as the objective function of the proposed coupling attention.

Our method significantly improves performance on synthetic and real directed coupling networks using the data generated by four well-known neural dynamic models, and the number of labels required is very small compared to the size of the coupled networks.

2 Background

2.1 Definition of Transfer Entropy

The transfer entropy, an information-theoretic measure, is able to detect information flow between time series X and Y . Transfer Entropy measures the degree of non-symmetric dependence of Y on X , defined as [2]:

where x ( l ) t = ( x t , ..., x t -l +1 ) and y ( k ) t = ( y t , ..., y t -k +1 ) and k , l are lengths of memory. For an uncoupled system ( X and Y independent) that can be approximated by a Markov process of order k , the conditional probability to find Y in state y t +1 at time t +1 satisfies p ( y t +1 | y ( k ) t , x ( l ) t ) = p ( y t +1 | y ( k ) t ) , and so TE ( X → Y ) vanishes.

2.2 Mutual Information Neural Estimation

The mutual information is equivalent to the KL divergence between the joint distribution P XY and the product of the marginal distributions P X ⊗ P Y [20, 21]. The KL divergence D KL admits the neural dual representation [22, 23]:

where the supremum is taken over parameter space Θ and f θ is the family of functions parameterized by the neural network with parameters θ ∈ Θ . The mutual information neural estimator is strongly consistent and can approximate the actual value with arbitrary accuracy [23].

3 Method

3.1 Neural Estimator of Transfer Entropy

By the conditional Bayes formula and adding a marginal distribution of Y , we derive the transfer entropy as the difference between two types of mutual information. An intuitive description is provided in Figure 2, and the derivation is placed in Appendix A.

In these expressions, y r is sampled from Y randomly and independently of the time step t . The first term MI ( Y t +1 , ( Y ( k ) t , X ( l ) t )) quantifies the reduction in the uncertainty of the future state y t +1 from knowing the historical information y ( k ) t and x ( l ) t . The second term MI ( Y t +1 , Y ( k ) t ) is the reduction in uncertainty simply from knowing y ( k ) t . By connecting Eq. 4 and Eq. 2, we define a differentiable estimator of transfer entropy as:

Transfer entropy, and even mutual information, is difficult to compute [24], especially for highdimensional or noisy data. In Appendix B, we offer a theoretical proof for the consistency and convergence properties of Transfer Entropy Neural Estimation, and examine its bias on a linear dynamic system where the true values of transfer entropy can be determined analytically.

3.2 Attentive Transfer Entropy

The main difficulty in our task is that the coupling effect in certain nonlinear dynamical systems is too weak to be recognized by classic techniques. We discuss the limitation of the iconic transfer entropy in detail that it works well when the three true distributions, i.e., one joint distribution and two conditional distributions in Eq. 1, can be estimated perfectly. However, sparse coupling effects are easily masked if the estimated probability density deviates even slightly from the real distribution. These momentary sources of evidence of coupling drive are like outliers in the total distribution of a time series dominated by self-drive. In order to make the transfer entropy provide a clear distinction between coupling and non-coupling pairs, we need to highlight the positions where

Figure 2: Visual interpretation of transfer entropy and its attentive version. Transfer Entropy is derived as the difference of two types of mutual information: MI ( Y t +1 , ( Y t , X t )) (blue area) quantifies the reduction in uncertainty of future state y t +1 from knowing current states ( y t , x t ) , and MI ( Y t +1 , Y t ) (green area) is same but only y t is known. Attention coefficients a t (yellow area) are assigned to each position of time series by coupling attention mechanism to maximize Attentive Transfer Entropy. For brevity, k = l = 1 here.

while avoiding the problem of distribution approximation. We do so by defining ATEn as:

In this expression, a t ∈ [0 , 1] is the attention coefficient at time step t and the collection A of attention coefficients is the attention series. Comparison of Eq.1 and Eq.6 reveals that the transfer entropy can be viewed as a simplified version of ATEn in which attention coefficients are uniformly set to one: ∀ t , a t = 1 . Because each position has an equal contribution to estimation, the value of transfer entropy is dominated by the majority of positions where coupling effect is negligible, i.e., where p ( y t +1 | y ( k ) t , x ( l ) t ) ≈ p ( y t +1 | y ( k ) t ) . Similarly to transfer entropy, ATEn is derived as the difference of two mutual informations, but ATEn incorporates the scheme of attention assignment. By connecting Eq. 6 and Eq. 2, we define a differentiable estimator of ATEn as:

where T is the total number of time steps and L = T -max( k, l ) . The expectation on the distribution of variables is adapted into the mean over time series.

3.3 Application of Coupling Attention

The overall framework of our model is presented in Figure 1b. In addition to two neural networks f θ and f ϕ for mutual information estimation, we employ another neural network g α for coupling attention assignment. Rather than approximating distributions, the neural network g α learns to maximize ATEn given by Eq. 7 via gradient descent. However, relying solely on coupling attention mechanism would lead to erroneous identification of spurious coupling effects as high transfer entropy regions which can also arise from non-coupling factors, e.g., noise in empirical samples. For more sophisticated inference, we augment our method with a binary classifier h η guided by coupling attention to focus on high transfer entropy regions and recognize different patterns between real and spurious coupling effect. The classifier takes as training samples a small number of ordered pairs of variables, among which labels for both coupling and non-coupling relationships are represented.

Then, the training process is divided into two independent stages: coupling attention learning and classification learning. The objectives in the first stage are:

where L 1 , L 2 is the expectation of the first and second sup term of Eq.7 on training set respectively. We update ( f θ , f ϕ ) and g α alternately. A small learning rate is required to maintain training stability, otherwise the g α may fall into a trivial solution where attention is almost zero throughout the time series. The objective in the second stage is:

where L 3 is the binary cross entropy and the notation η | α indicates that coupling attention remains fixed during the second stage of training. The downstream classifier is sensitive to the upstream scheme of attention assignment 1 . Implement of our method is presented as Alg. 1 in Appendix C.

4 Experiment

4.1 Setup

Directed coupling networks. For synthetic networks, we use Scale-Free (SF) graph model [25], which could generate networks with controllable structure characteristics. For real networks, we select six neurological connectivity datasets as presented in Table 1, each from a different species: Cat, Macaque, Mouse, C. elegans, Rat and Drosophila.

Dynamic models. We use four dynamic models for neural activity simulation widely used in the field of neuroscience: Hindmarsh-Rose (HR), Izhikevich (Izh), Rulkov and FitzHugh-Nagumo (FHN). Dynamic equations are provided in Table 2, and segments of generated time series are represented in Figure 4.

Baselines. We compare our method with eight baselines. Unsupervised learning: (1) Granger causality test (Ganger) [1]; (2) Transfer Entropy using the Kraskov-estimator (TE_Kraskov) [26], a standard method for TE calculation; (3) Convergent cross mapping (CCM) [3]; (4) Latent convergent cross mapping (Latent CCM) [27]; (5) PCMCI [6] and (6) PCMCI + [28] using partial correlation to quantify coupling strength. Unsupervised learning with validation set: (7) Transfer Entropy Neural Estimator (TENE), as in Eq. 5. The estimator is trained on the test set unsupervised and terminates upon achieving optimal performance on the validation set. Small sample learning: (8) Classifier with traditional attention mechanism [29], which was initially developed for computer vision tasks.

Evaluation metrics. We quantify the performance of methods by evaluating the metrics, specifically the area under the receiver operating characteristic curve (AUROC).

Training details. We employ a 4-layer convolutional neural network for model g α and h η , and a 5-layer fully-connected neural network for model f θ and f ϕ . We use the ADAM [30] optimizer with initial learning rate of 10 -3 for the classifier h η and 10 -4 for the others. The learning rate decays exponentially by gamma = 0 . 999 per epoch. The batch size for stage 1 is 32 and for stage 2 is 10. The number of training epochs is 400. We run all experiments in this work on a local machine with two NVIDIA V100 32GB GPUs. See codes in Supplementary Materials for more details.

( a)

( b )

( c)

( d )

( e)

( f )

1.0

1.0

1.0-

0.9

0.9

0.9

AUROC

0.8

0.8

0.8

0.7

--+--

--+--

TENE

ATEn

0.7

0.6

0.7

0.6

--+--

--+

TE_Kraskov

Classifier

0.6

0.5

0.5

0.5

20

60

100

140

180

10

20

30

40

50

0.1

0.2

0.3

0.4

0.5

1.0

1.0

1.0

0.9

0.9

0.9

0.8

AUROC

0.8

0.8

0.7

0.7

0.6

0.7

0.6

0.5

0.6

0.5

0.5

0.05

0.1

0.2

0.3

0.4

1

5

10

20

30

80

60

40

19

0

4.2 Ablation study

Our method's innovation can be decomposed into three components: Firstly, a trainable transfer entropy calculator (TENE) is proposed; Secondly, TENE is combined with attention to detect region of high transfer entropy; Thirdly, the classifier is guided by coupling attention to focus on regions where the coupling effect may emerge. Here, we provide a detailed ablation study to determine which component is the key to performance improvement, and show results in Figure 3. We observe that Classifier alone (blue lines) exhibits the worse performances than our method (green lines) in most environments. This indicates that the spurious features learned by Classifier, despite quickly reaching low loss on the small training set, are less related to the properties of coupling effect and result in poor generalization capability on test set. When the size of train set increases, the performance of Classifier alone become comparable to that of ATE n (see Figure 3a). TENE (cyan lines) exhibits performance marginally superior to transfer entropy estimated using the standard Kraskov method (TE_Kraskov; purple lines). The ATE n (green lines) shows a substantial performance improvement compared with baselines, which indicates that the classifier with the guidance of coupling attention can extract features hidden in activity sequences that are closely related to coupling effect and result in superior generalization. This also demonstrates the importance of identifying and focusing on these critical regions of activity sequences. Moreover, our method is robust against changes to experimental conditions other than mean degree and measurement noise, a limitation we discuss in Sec. 5.

4.3 Insight into the coupling-drive of underlying dynamics

In Figure 4, we demonstrate the ability of coupling attention to catch the transient emergence of coupling effect. The gray lines in Figure 4 represent the change of coupling force from parent to child variable over time, and are generated by the coupling term Γ in Table 2. The absolute value of the coupling force rises (the gray lines spike) at occasional moments when the behavior of a

( c)   Fi t zH ugh-N agum o

( b )   I zh i kevi ch

( a)   H i n d m arsh-Rose

Em ergence  of

Rem ai n

dorm ant

2

50

2

Power

1

0

0

0

Parentvariable

-1

Childvariable

-50

Whu

0

100

200

300

400

0

100

200

300

0

200

400

1.0

1.0

1.0

Coupling-driveforce

Strength

Traditionalattention

0.5

Couplingattention

0.5

0.5

0.0

LNLLLN

LANULI

0.0

0.0

0

100

200

300

400

0

100

200

300

0

200

400

t

t

t

parent variable substantially influences the evolution of its child variable, and remains almost zero at other times. The orange lines representing the coupling attention keep in step with the gray lines, indicating that the coupling attention mechanism recognizes the effect of coupling force in reducing the uncertainty of the child variable and pays attention to the areas where coupling force is significant. In contrast, the light pink lines in Figure 4 representing the traditional attention [29] remain close to their maximum value, indicating it is insensitive to changes in coupling force. This would lead its classifier to extract features from any part of the entire time series, rather than focusing on areas where coupling effects may emerge.

4.4 Performance on real networks

We test our method on six empirical neurological connectivity datasets using activity sequences data generated by four neuronal dynamic models. Compared with the baselines, our method usually substantially improves reconstruction performance on real coupled networks, as shown in Table 3. The performance of classical unsupervised methods, for which all positions in the time series are treated equally, is limited by the paucity of distinct coupling effects. In conclusion, our method slightly

increases cost, due to the need for label collection, but obtains a substantial boost in performance compared with those unsupervised methods in this class of coupling network reconstruction tasks. We also observe that the reconstruction performances on identical dynamics across distinct connectomes are significant different (e.g., AUROC of ATE n on Izh across C.elegans in Table 3 and Rat in Appendix Table 5), indicating that the underlying network architecture plays an crucial role in shaping the behavior of coupling dynamics.

5 Limitations

Our methodology has limitations (i.e., cases for which performance improvement is less): 1. Dense networks, where a variable is coupled with many driving variables and substantial coupling forces can emerge from distinct parents at overlapping times, making individual drivers harder to distinguish (see green line in Figure 3b); 2. Intense noise, which makes the coupling attention mechanism falsely identify high transfer entropy regions. The downstream classifier then extracts spurious features, leading to the reduction of its generalization (see Figure 3e); 3. Strongly coupled systems, which are dominated by synchronization phenomena in which the dynamic behaviors of all variables are similar.

In addition, we assume the state evolution of variables in the coupled network is uniformly governed by a group of dynamic equations. However, if the evolution behaviors of variables in the system obey different underlying dynamics, it becomes critical to carefully select the small training samples, in which the patterns of coupled driving effects are representative. Failure to do so may result in the poor generalization of the classifier. That is our next research direction.

6 Related Work

Coupled Network Reconstruction. Several common methods of causal inference [31, 4, 6, 9] are based on conditional independence relations, but differ in the design of condition-selection strategies or choice of conditional independence test. Granger Causality [1] is extended to nonlinear dynamics by using neural networks to represent nonlinear casual relationships [8, 5]. Many methods of causal discovery assume that the coupled network is a directed acyclic graph. However, directed cyclic graphs are common in real systems. Conventional frameworks assume separability, i.e., that information about coupling drive are not contained in the parent variable itself. To address the non-separability issue, Convergent-cross mapping [3] and its variations [32, 27] measure the extent to which the historical record of child can reliably estimate states of the parent in reconstructed state space. However, sparse causal effect in neuronal dynamics, particularly in the presence of noise, may lead parent and child time series to appear statistically independent, so that their contribution to state estimation is hard to recognize.

Mutual Information Estimation. Ref. [23] built on a dual representation of KL divergence [22] to offer a parametric mutual information neural estimator (MINE) which is linearly scalable in dimension as well as sample size, and is also trainable and strongly consistent. They also discussed another version of MINE based on the f -divergence representation [33, 20]. Using the technique of Noise-Contrastive Estimation (NCE) [34], based on comparing target and randomly chosen negative samples, Van den Oord et al. [35] proposed InfoNCE loss, minimization of which maximizes a mutual information lower bound. An important application of this contrastive learning approach has been extracting high-level representations of different data modalities [36, 37, 38, 39]. In our work, we extend MINE for transfer entropy estimation.

Attention Mechanisms. The attention mechanisms identify key areas in the data by learning a set of weight distributions. Spatial-based attention [29, 40, 41] involves generating attention scores from spatial regions of feature maps, while channel-based attention [42, 43] optimises the representation of each channel. Self-attention [44, 45] encodes interactions among all input entities and crossattention [46, 47, 48] introduces the interaction of two domains further. However, it is recognised that attention mechanisms need to be tailored to the specific problem at hand [49]. In our work, we tailor attention mechanism for coupling relationship inference by accommodating the selection of temporal regions that correspond to the transient emergence of coupling effect in neural activity sequences.

7 Discussion

The problem of reconstructing coupling networks from observational data is fundamental in multiple disciplines of science including neuroscience, since it is a prerequisite foundation for the research about structure analysis and behavior control in coupling networks. Especially, several countries have recently launched grand brain projects, and one important goal is to map the connectomes (i.e., directed links between neurons) of different species.

We draw on coupling attention to guide machine learning models to infer coupling relationships while focusing on the specific areas where casual effect may emerge. We show that this mechanism identifies weak coupling effects ignored by classical techniques, and helps machine learning models gain insight into the coupling dynamics underlying time series data.

Acknowledgments and Disclosure of Funding

This work is supported by STI2030-Marjor Project 2021ZD0204500, National Natural Science Foundation of China (Grants No. T2225022, No. 12161141016, No. 12150410309), Shuguang Program of Shanghai Education Development Foundation and Shanghai Municipal Education Commission (Grant No. 22SG21), Shanghai Municipal Science and Technology Major Project (Grant No. 2021SHZDZX0100), Shanghai Municipal Commission of Science and Technology Project (Grant No. 19511132101), and the Fundamental Research Funds for the Central Universities.

References

Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: Journal of the Econometric Society , pages 424-438, 1969.

Thomas Schreiber. Measuring information transfer. Physical Review Letters , 85(2):461, 2000.

George Sugihara, Robert May, Hao Ye, Chih-hao Hsieh, Ethan Deyle, Michael Fogarty, and Stephan Munch. Detecting causality in complex ecosystems. science , 338(6106):496-500, 2012.

Jie Sun, Dane Taylor, and Erik M Bollt. Causal network inference by optimal causation entropy. SIAM Journal on Applied Dynamical Systems , 14(1):73-106, 2015.

Meike Nauta, Doina Bucur, and Christin Seifert. Causal discovery with attention-based convolutional neural networks. Machine Learning and Knowledge Extraction , 1(1):312-340, 2019.

Jakob Runge, Peer Nowack, Marlene Kretschmer, Seth Flaxman, and Dino Sejdinovic. Detecting and quantifying causal associations in large nonlinear time series datasets. Science Advances , 5(11):eaau4996, 2019.

Andreas Gerhardus and Jakob Runge. High-recall causal discovery for autocorrelated time series with latent confounders. Advances in Neural Information Processing Systems , 33:12615-12625, 2020.

Alex Tank, Ian Covert, Nicholas Foti, Ali Shojaie, and Emily B Fox. Neural Granger causality. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(8):4267-4279, 2021.

Atalanti A Mastakouri, Bernhard Schölkopf, and Dominik Janzing. Necessary and sufficient conditions for causal feature selection in time series with latent common causes. In International Conference on Machine Learning , pages 7502-7511. PMLR, 2021.

Lipeng Cui and Jack Murdoch Moore. Causal network reconstruction from nonlinear time series: A comparative study. International Journal of Modern Physics C , 32(04):2150049, 2021.

Shohei Shimizu, Patrik O Hoyer, Aapo Hyvärinen, Antti Kerminen, and Michael Jordan. A linear non-Gaussian acyclic model for causal discovery. Journal of Machine Learning Research , 7(10), 2006.

Feng Xie, Ruichu Cai, Biwei Huang, Clark Glymour, Zhifeng Hao, and Kun Zhang. Generalized independent noise condition for estimating latent variable causal graphs. Advances in Neural Information Processing Systems , 33:14891-14902, 2020.

Niklas Pfister, Peter Bühlmann, and Jonas Peters. Invariant causal prediction for sequential data. Journal of the American Statistical Association , 114(527):1264-1276, 2019.

Hanchuan Peng, Erik Meijering, and Giorgio A Ascoli. From diadem to bigneuron, 2015.

Wenjing Wang, Christina K Kim, and Alice Y Ting. Molecular tools for imaging and recording neuronal activity. Nature chemical biology , 15(2):101-110, 2019.

Elias Bareinboim and Judea Pearl. Transportability from multiple environments with limited experiments: Completeness results. Advances in Neural Information Processing Systems , 27, 2014.

Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, et al. Interaction networks for learning about objects, relations and physics. Advances in Neural Information Processing Systems , 29, 2016.

Karima Makhlouf, Sami Zhioua, and Catuscia Palamidessi. Survey on causal-based machine learning fairness notions. arXiv preprint arXiv:2010.09553 , 2020.

Dana Pessach and Erez Shmueli. A review on fairness in machine learning. ACM Computing Surveys (CSUR) , 55(3):1-44, 2022.

Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. Advances in Neural Information Processing Systems , 29, 2016.

R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670 , 2018.

Monroe D Donsker and SR Srinivasa Varadhan. Asymptotic evaluation of certain markov process expectations for large time. iv. Communications on Pure and Applied mathematics , 36(2):183-212, 1983.

Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In International Conference on Machine Learning , pages 531-540. PMLR, 2018.

Liam Paninski. Estimation of entropy and mutual information. Neural Computation , 15(6):11911253, 2003.

K-I Goh, Byungnam Kahng, and Doochul Kim. Universal behavior of load distribution in scale-free networks. Physical Review Letters , 87(27):278701, 2001.

J. T. Lizier. JIDT: An information-theoretic toolkit for studying the dynamics of complex systems. Frontiers in Robotics & Ai , 2014.

Edward De Brouwer, Adam Arany, Jaak Simm, and Yves Moreau. Latent convergent cross mapping. In International Conference on Learning Representations , 2020.

Jakob Runge. Discovering contemporaneous and lagged causal relations in autocorrelated nonlinear time series datasets. In Conference on Uncertainty in Artificial Intelligence , pages 1388-1397. PMLR, 2020.

Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. Cbam: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision , pages 3-19, 2018.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Peter Spirtes and Clark Glymour. An algorithm for fast recovery of sparse causal graphs. Social Science Computer Review , 9(1):62-72, 1991.

Adam Thomas Clark, Hao Ye, Forest Isbell, Ethan R Deyle, Jane Cowles, G David Tilman, and George Sugihara. Spatial convergent cross mapping to detect causal relationships from short time series. Ecology , 96(5):1174-1181, 2015.

XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory , 56(11):5847-5861, 2010.

Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pages 297-304. JMLR Workshop and Conference Proceedings, 2010.

Aaron Van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv e-prints , pages arXiv-1807, 2018.

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning , pages 1597-1607. PMLR, 2020.

Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Cost: Contrastive learning of disentangled seasonal-trend representations for time series forecasting. In International Conference on Learning Representations , 2021.

Yeping Hu, Xiaogang Jia, Masayoshi Tomizuka, and Wei Zhan. Causal-based time series domain generalization for vehicle intention prediction. In NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications , 2021.

Maciej Koch-Janusz and Zohar Ringel. Mutual information, neural networks and the renormalization group. Nature Physics , 14(6):578-582, 2018.

Ting-I Hsieh, Yi-Chen Lo, Hwann-Tzong Chen, and Tyng-Luh Liu. One-shot object detection with co-attention and co-excitation. Advances in Neural Information Processing Systems , 32, 2019.

Dingguo Shen, Yuanfeng Ji, Ping Li, Yi Wang, and Di Lin. Ranet: Region attention network for semantic segmentation. Advances in Neural Information Processing Systems , 33:13927-13938, 2020.

Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 7132-7141, 2018.

Hang Zhang, Chongruo Wu, Zhongyue Zhang, Yi Zhu, Haibin Lin, Zhi Zhang, Yue Sun, Tong He, Jonas Mueller, R Manmatha, et al. Resnest: Split-attention networks. arXiv preprint arXiv:2004.08955 , 2020.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems , 30, 2017.

Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. Non-local neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 7794-7803, 2018.

Peizhao Li, Jiuxiang Gu, Jason Kuen, Vlad I Morariu, Handong Zhao, Rajiv Jain, Varun Manjunatha, and Hongfu Liu. Selfdoc: Self-supervised document representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5652-5660, 2021.

Chun-Fu Richard Chen, Quanfu Fan, and Rameswar Panda. Crossvit: Cross-attention multiscale vision transformer for image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 357-366, 2021.

Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch, Catalin Ionescu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer, et al. Perceiver io: A general architecture for structured inputs & outputs. In International Conference on Learning Representations , 2022.

Mohammed Hassanin, Saeed Anwar, Ibrahim Radwan, Fahad S Khan, and Ajmal Mian. Visual attention methods in deep learning: An in-depth survey. arXiv preprint arXiv:2204.07756 , 2022.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis article expands the concept of transfer entropy to include attentive transfer entropy. Implementing attentive transfer entropy on connectomes of realistic biological networks demonstrated that this extension encompasses the transient coupling effect within the connectivity constraints of realistic biological networks.\n\nSTRENGTHS:\nThis paper provides convincing empirical evidence that the attentive transfer entropy is particularly useful in understanding coupling effects of realistic biological networks. They achieve this through first conducting ablation study on toy models of simple connectivity (full connectivity?) and comparing the proposed attentive transfer entropy with multiple commonly used baselines.\n\nWEAKNESSES:\nIt is a bit confusing that the classifier with conventional attentional mechanism works almost as well as the ATEn in many scenarios (e.g., the Izh dynamics in C.elegans) and overall in Mouse connectome. In addition, it seems which dynamics being used to simulate the sequences may also influence the performance, but there was no discussion on that.\n\nQUESTIONS:\n1) The Classifier outperforms ATEn in mouse connectome across all dynamics. Does this mean ATEn may not work well in very sparse networks? \n2) In many instances, the transfer entropy without attention mechanism performs significantly worse than both classifier and ATEn. Even the conventional attentional mechanism appears to be of great help. Is it possible to compare or discuss in some details the distinction between classifier and ATEn attention mechanisms in real-world networks? \n3) The authors mentioned in a couple places that the train/validate/test split is 20/100/1000. How many samples are there in total? Is it possible to elaborate more training details?\n\nLIMITATIONS:\nThis study employed predetermined dynamics, which is hardly realistic for functioning networks in the real world. The sensitivity to the choice of dynamics may hinder this method's potential utility for analyzing experimental results.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes \"attentive transfer entropy\" as a way to infer connectivity of real and synthetic coupling networks. The submission suggests extending a 'neural estimator of mutual information' to transfer entropy and adding \"attention\" to focus on moments in time where the influence is larger.\n\nSTRENGTHS:\n* higher performance than other methods this estimator is compared to.\n* fewer data necessary than other methods this estimator is compared to.\n\nWEAKNESSES:\n* The introduction of attention appears to be ad hoc, a mathematical/information theoretic justification of the attention mechanisms seems to be missing. \n* The methods seems a direct extension of the Mutual information neural estimator, the only technical innovation seems to be the attention mechanism which seems to lacks theoretic grounding.\n* The whole theory (including the analytical toy example) considers discrete time dynamics but the methods is then applied to inference of couplings in systems of coupled differential equations. Something seems to be missing here to link the two descriptions (continous time and discrete time).\n\nMinor issues:\n* Table 1 is a bit confusing. The second column is \"brain, brain cortex, neural, brain\". This seems rather unspecific. What is the actual brain region in the cat an macaque data? Why don't you use a full connectome, e.g. of a fly (https://www.science.org/doi/10.1126/science.add9330)\n* Weird notation in first equation. Why is t and index, usually ODEs are written as d x(t) /dt =f(x(t)) and not d x_t /dt=f(x_t)\n\nQUESTIONS:\n* Is the estimate of the transfer entropy assumed to be independent of the time-discretization? If yes, please provide evidence for that (e.g. show that changing dt of the numerical integration of the ODE does only have a very minor effect on the estimate of transfer entropy).\n* How does the proposed method coping with confounders (ie. unobserved/unrecorded neurons)?\n* In the single neuron models in the section 4 there is no noise added, so asymptotically, what is the source of entropy in this system? In other words: If the network dynamics is not chaotic, asymptotically the dynamics is expected to relax onto a fixed point or limit cycle. In both cases of fixed points and limit cycle, the true transfer entropy should be zero (observing Y very long is enough to predict it's future even without knowing X) so the full transfer entropy should be zero. What I am confused about is: Is this setup only quantifying some kind of 'transient' transfer entropy? Or is the approach relying on chaotic network dynamics? If yes, how does changing the intensity of network chaos affect the estimate? Or does the method actually implicitly rely on all nodes receiving independent noise? Is the 'numerical noise' enough? I am not expert on transfer entropy and I might be misunderstanding something here, but some clarification seems necessary.\n* Why is noise added to the data sets? (\"Gaussian measurement noise is added with mean zero and standard deviation 1%/10% that of the original time series for synthetic/real networks respectively.\")\n* What are the units of time in figure 3? Units of membrane time constant?\n* How is this method dealing with identifiability issues, e.g. when different network structures can lead to the same dynamics?\n* Could the proposed method be combined with causal interventions? E.g. can it tell me the neuron/synapse it has the biggest uncertainty about, so I could apply activity perturbations/causal interventions to infer if the neuron/synapse is connected or not?\n* Why is the mean connectivity in the mouse data 1.1 according to table 1? It seems that compared to a real brain, possibly not all existing connections are in the data set. How does this change your conclusions? In other words: Real biological cortical neurons have a mean degree more arround 10^3 to 10^4 (see e.g. Potjans, Diesmann 2014), would your method still work for neural networks with realistic degree?\n\nLIMITATIONS:\nLimitations (mentioned above) are already addressed in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors extend the MiNE method [23], of using a neural network to estimate mutual information, to estimating transfer entropy between two coupled time series. When the coupling is weak on average due to being intermittent, they propose a differentiable attention mechanism that learns to transiently gate the transfer entropy calculation to only these intermittent moments. They test their method to identify coupling in networks created using simple neuron models connected as per connectivity datasets.\n\nSTRENGTHS:\n-\tThe authors combine several new techniques to address an important issue of connectivity determination from neural recordings / time series.\n\nWEAKNESSES:\n-\tThe authors straightforwardly extend the theory of the neural estimation of mutual information to their neural estimation of transfer entropy without attention in Appendix A.. However, they do not say anything about their attention based method. Can it be shown that it is also somehow consistent / convergent?\n-\tIt appears to me that all the ‘connectomes’ apart form the C.Elegans one are between macro brain regions. Yet these have been used to connect individual neuron models. Can the authors not use local connectivity of cortical neurons from say the Blue Brain project and deduce this connectivity?\n-\tUltimately the method is only tested on simulated datasets even though the connectivity is based on macro-connectivity of animal brains. Numerous Ca-imaging or spike-train recordings of hundreds of neurons in various small regions of the brain are available. Yet the authors do not seem to have applied their method to these. Thus it calls into question whether their method is suitable for real-world data.\n\nQUESTIONS:\nAlready stated in weaknesses.\n\nMinor:\n- L126: ‘adjusting $a_t$ in Eq. 5’ -> ‘adjusting $a_t$ in Eq. 6’ since Eq. 5 does not contain $a_t$.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel method to reconstruct coupled networks from observational time series data, where the coupling effects between variables are sparse and weak. The authors propose a new attention mechanism called coupling attention is introduced to identify critical regions in time series where coupling effects momentarily emerge amidst dominant intrinsic dynamics. The authors design a differentiable objective function called Attentive Transfer Entropy (ATEn) to train the coupling attention model in an unsupervised, data-driven manner.\n\nSTRENGTHS:\nATEn is well motivated and the use of Mutual Information Neural Estimation as a component to both estimate and optimize attention parameters makes sense.\n\nThe concept of using attention to identify sparse coupling effects in time series appears original for this reviewer and the method leverages the use of neural networks well. The method could likely be extended in the future to other neural architectures like transformers, therefore this work is a good initial foray into further work and improvements.\n\nText is well written and clear and the ablation experiments make sense.\n\nWEAKNESSES:\nThere needs to be more clarity in the experimental setup:\n\nSpecifically, the way overfitting is prevented in the proposed ATEn method. Like Mutual Information Neural Estimation method, parameters are optimized in Transfer Entropy, which can result in perfect estimates of the mutual information (due to overfitting). How have the authors addressed this?\n\nWhat are the inputs to fθ and fφ and how are they different from the inputs to the attention and classifier models? Is there a feature space beyond the scalar timeseries of the neurons?\n\nQUESTIONS:\nFor the Hindmarsh-Rose experiments, why does it appear as though the child variable is leading the parent variable in the time-series in Figure 4?\n\nWhat is the reason for making the attention and classifier convolutional networks while fθ and fφ are fully-connected networks?\n\nFor clarification, Figure 4 TENE refers to using Equation 5?\n\nLIMITATIONS:\nNo obvious negative societal impacts. Refer to Weaknesses for constructive suggestions for improvement.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-30"}