Causal Attention to Exploit Transient Emergence of Causal Effect
Authors: Xiaolei Ru
Xin-Ya Zhang
Jack Murdoch Moore
Gang Yan
Venue: Submitted to ICLR 2023 (2023)
Keywords: causal attention mechanism
coupling-drive
sparse causal effect
neural dynamics
causal network reconstruction

Abstract: We propose a causal reasoning mechanism called $\textit{causal attention}$ that can improve performance of machine learning models on a class of causal inference tasks by revealing the generation process behind the observed data. We consider the problem of reconstructing causal networks (e.g., biological neural networks) connecting large numbers of variables (e.g., nerve cells), of which evolution is governed by nonlinear dynamics consisting of weak coupling-drive (i.e., causal effect) and strong self-drive (dominants the evolution). The core difficulty is sparseness of causal effect that emerges (the coupling force is significant) only momentarily and otherwise remains dormant in the neural activity sequence. $\textit{Causal attention}$ is designed to guide the model to make inference focusing on the critical regions of time series data where causality may manifest. Specifically, attention coefficients are assigned autonomously by a neural network trained to maximise the Attention-extended Transfer Entropy, which is a novel generalization of the iconic transfer entropy metric. Our results show that, without any prior knowledge of dynamics, $\textit{causal attention}$ explicitly identifies areas where the strength of coupling-drive is distinctly greater than zero. This innovation substantially improves reconstruction performance for both synthetic and real causal networks using data generated by neuronal models widely used in neuroscience.

Under review as a conference paper at ICLR 2023
CAUSAL ATTENTION TO EXPLOIT TRANSIENT EMER- GENCE OF CAUSAL EFFECT
Anonymous authors Paper under double-blind review
ABSTRACT
We propose causal reasoning mechanism called causal attention that can im- prove performance of machine learning models on class of causal inference tasks by revealing the generation process behind the observed data. We consider the problem of reconstructing causal networks (e.g.. biological neural networks) con- necting large numbers of variables (e.g., nerve cells), of which evolution is gov- erned by nonlinear dynamics consisting of weak coupling-driv (i.e., causal effect) and strong self-drive (dominants the evolution). The core difficulty is sparseness of causal effect that emerges (the coupling force is significant) only momentarily and otherwise remains quiescent in the neural activity sequence. Causal attention is designed to guide the model to make inference focusing the critical regions of time series data where causality may manifest. Specifically, attention coefficients are assigned autonomously by a neural network trained to maximise the Attention- extended Transfer Entropy, which is a novel generalization of the iconic transfer entropy metric. Our results show that, without any prior knowledge of dynamics. causal attention explicitly identifies areas where the strength of coupling-dris drive is distinctly greater than zero. This innovation substantially improves reconstruction performance for both synthetic and real causal networks using data generated by neuronal models widely used in neuroscience.
INTRODUCTION
In this work, our task is to infer causal relationships between observed variables based on time series data and reconstruct the causal network connecting large numbers of these variables. Assume the time series Zue record the time evolution of variable governed by coupled nonlinear dynamics, as represented by a general differential equation xi g( + Σ Biy ( where and f are self- and coupling functions respectively. The parent variable influences the dynamic evolution of the child variable via the coupling function f. Note that these two functions are hidden and usually unknown for real systems. The asymmetric adjacency matrix represents the causal, i.e., directional coupling relationship between variables. Hence, the goal is to infer matrix B from observed time series Ta. = 1, ., N where is the number of variables in the system. If Byy = 1, the variable i is a coupling driver (parent variable) of variable j. otherwise it is zero
The key challenge is that the causal effect in neural dynamics (e.g.. biological neural systems ob- served via neuronal activity sequences) is too weak to be detected, rendering powerless classic supervised techniques of causal inference across multiple research communities Granger (1969);
Schreiber ( (2000): Sugihara et (2012); Sun al. (2015): Nauta al. (2019); Runge et al. 2019))
Gerhardus & Runge 2020) Tank et al. 2021) 1): Mastakouri et al. (202T) This difficulty manifests in
three aspects. First, the dynamies contains deif- and coupling-drin drive. The strength of coupling fz ) is usually many orders of magnitude smaller than self-drive g g and the latter dominates evolution. Second, the behavior of the coupling-drive is chaotic, unlike linear models Shimizu et al. (2006); et al. (2020). The resulting unpredictability and variability of system state means that coupling force can be significant momentarily and otherwise almost vanish, as illustrated in Figure 3 (gray lines). This dilutes the information in time series that can be useful for inferring the causal relationship. Third, in the heterogeneous networks common in applications, some variables hubs coupled with many parent variables, among which it is difficult to distinguish individual causes. When causal effects are weak, we do not observe clearly the principle of Granger Causality, whereby the parent variable can help to explain the future change in its child variable Pfister et al.

---

Under review as a conference paper at ICLR 2023
l) for 2019). Rather, when train machine learning model Nauta al. (2019); Tank et al. (202
prediction task on the neuronal activity sequences, the model only exploits the historical informa- tion of the child variable itself and that from parent variables is ignored. We posit that coupling-drin makes a negligible contribution to dynamic evolution in the majority of samples of time series data. In other words, only in small fraction of samples is the information of parent variables effective in predicting the evolution of child variables. Taking as an example the gradient algorithm to minimise the regression error over all samples (x - Hit) )2, the adjustment of model parameters from the tiny samples corresponding to significant coupling force is negligible, but these are the only samples which could induce the model to exploit causal effects in reducing regression error. Similarly, for transfer entropy [Schreiber] (2000). which measures the reduction in uncertainty which potential parent variable provides to a potential child variable, there no significant difference in measured value between ordered pairs of variables with and without causality.
To overcome the difficulty, we propose causal reasoning mechanism causal attention - to iden- tify the moments when causal effect emerges. We design an objective function, Attention-extended Transfer Entropy (AcTE), comprising a weighted generalisation of transfer entropy. In order to maximize AE the causal attention mechanism trains neural networks to autonomously allocates high attention coefficients a at times where information of parent variables effectively reduces the uncertainty of child variables, and ignores other positions by setting a, close to zero. If we consider each value in a time series as a feature, the operation of attention allocation is also equivalent to removing the non-causal features Kusner et al. (2017); Hu al. (202T).
However, noise in empirical samples may also produce high transfer entropy regions, which leads spurious causal effects even when using causal attention. We add a binary classification model to perform more sophisticated inference under the guidance of causal attention to focus on these critical regions and recognize different patterns between noisy and sparse emergence of causal effect. We deal with this class of causal inference task by way of small sample supervised learning. Although training and test data have a distribution shift in the setting of small samples, they arise through an identical underlying generation process. Thus, if the model provides insight into the underlying dynamics- - the coupling-dris for causal inference - then the understanding acquired from small samples can be effectively utilised in the test environment [Bareinboim & Pearl (2014); Battaglia jet al. (2016); Makhlouf et al.] (2020); | |Pssachh & Shmueli (2022). The role of causal attention is to help the classification model gain this insight. Our contributions summarized as follows:
1. We introduce causal attention, a causal reasoning mechanism to identify the positions of time series which causal effect emerges and guide classification model infer causality focusing on these critical positions. Without any prior knowledge of dynamics, the mecha- nism determines the areas where the coupling force is substantially different from zero
2. By formulating Transfer Entropy as the difference between two types of mutual informa- tion, and based on the dual representation of Kullback-Liebler (KL) divergence, we design a differentiable metric, Attention-extended Transfer Entropy, as the objective function of the proposed causal attention mechanism.
3. Our method significantly improves performance on synthetic and real causal networks us- ing the data generated by five well-known neural dynamic models, and the number of labels required is very small compared to the size of the causal networks.
Our methodology has limitations (i.e., cases for which performance improvement less): Dense networks, where a variable is coupled with many driving variables such that their causal effects overlap and are harder to distinguish. 2. Intense noise, which makes the casual attention mechanism falsely identify high transfer entropy regions. The downstream classifier then extracts non-causal features, leading to the reduction of its generalization. 3. Strongly coupled system, which is domi- nated by synchronization phenomena in which the dynamic behaviors of all variables are similar.
2 BACKGROUND
2.1
DEFINITION OF TRANSFER ENTROPY
The transfer entropy, an information-thhoreee causality measure, is able to detect information flow between time series X and Y. Transfer Entropy measures the degree of non-symmetric dependence
2

---

Under review as a conference paper at ICLR 2023
of Y on X, defined Schreiber (2000):
TE(X -)= Ep (Ve+1. (8) , log (vt+1 wt+1 101 (k) 11
Ui+1 y/. (8:)
where x; pled system (X and Y independent) that be approximated by a Markov process of order k. the conditional probability to find Y in state 1 at time t + 1 satisfies p (Ve+1 Vs 224
(Izz, x2-1+1) and yi (e. x4- and k, I are lengths. For an uncou-
(k)
p Mt+1 )
2.2 MUTUAL INFORMATION NEURAL ESTIMATION
The mutual information is equivalent to the KL divergence between the joint distribution Pxy and the product of the marginal distributions Px @ Py NNwozin et al. (2016); Hjelm et al. (2018). The KI divergence D admits the neural dual representation Donsker & Varadhan (1983) Belghaz)
all.. 2018):
MI(X,Y) - Dez (Pxy Pv)
sup Exxy [So] log PE (e/e eee
where the supremum is taken over parameter space θ and is the family of functions parameterized by the neural network with parameters 0 E O. The mutual information neural estimator is strongly consistent and can approximate the actual value with arbitrary accuracy |Belghazi al. (2018).
METHOD
3.1
NEURAL ESTIMATOR OF TRANSFER ENTROPY
7--2
Yt-1
y1
Yit1
Y44
Xz.2
Xx
X2
Xx11
Xz12
TEx-y = MilYeta. (x.X)) MI(V++x KD
ATEx-y = MI (V. X.)4) MI(K+1 Vel4)
Figure 1: Visual interpretation of transfer entropy and its attention-extendee version. The Transfer Entropy is derived as the difference of two types of mutual information: M (Y++1, (Y,, Xx)) (blue area) quantifies the reduction uncertainty of future state V/+1 from knowing current states (3t. 21), and MI (Y(+1, Yi) (green area) is same but only is known. The attention coefficients (yellow area) are assigned to each position of time series by the causal attention mechanism maximize the Attention-extender Transfer Entropy. For brevity, k = l= here.
By the conditional Bayes formula and adding marginal distribution Y,, we derive the transfer en- tropy as the difference between two types of mutual information. An intuitive description is provided Figure [ and the derivation is placed in Appendix A.
TE(X-M
P vu (1))) ,x
-Ex (w++1.11)) log P (JetI. (k)
=
Σ (11-1. _a! ,x log p (vr) (1( , (1)
(3)
(vr) (x)
-MI (Yet2, (x(,x("))) MM Y((1))((11)
(4)
3

---

Under review as a conference paper at ICLR 2023
In these expressions, Ur is sampled from Y randomly and independently of the time step t. The first term MI (Y++1, (y x(")) quantifies the reduction in the uncertainty of the future state
(!) The second term MT (Y+1,, y( is Ve+1 from knowing the historical information y/ (k) and Zy
the reduction in uncertainty simply from knowing yr By connecting Eq. and Eq. we define a differentiable estimator of transfer entropy as:
TE(X YY sup E P M xxt) [so] log F. Do (x))) [cfo]
sup P p(rniaa [So] - log 10 (PPCyanoopy) [elu])
(5)
4
Transfer entropy, and even mutual information, is difficult to compute Paninski (000)), especially for nigh-dimensional or noisy data. In Appendix B, we offer a theoretical proof for the consistency and convergence properties of Transfer Entropy Neural Estimation, and examine its bias linear dynamic system where the true values of transfer entropy can be determined analytically.
3.2 ATTENTION ENTENTION-EXTENDED TRANSFER ENTROPY
The main difficulty in our task is that the causal effect in certain nonlinear dynamical systems is too weak to be recognized by classic techniques. We discuss the limitation of the iconic transfer entropy in detail that works well when the three true distributions, i.e.. one joint distribution and two conditional distributions in Eq. [ can be estimated perfectly. However, sparse causal effects are easily masked if the estimated probability density deviates even slightly from the real distri- bution. These momentary sources of evidence of coupling drive are like outliers in the total dis- tribution of a time series dominated by self-drive. In order to make the transfer entropy provide clear distinction between causal and non-causal pairs, need to highlight the positions where Vt+1 I wt _z( > p (yc+1 ( and filter out other times by adjusting a in Eq. 6 all while avoiding the problem of distribution approximation. We do so by defining AcTE as:
(k) (1)
P Utt1
V1 1t
(8) _x( AcTE(X +1)= Σ ai (1++1.8
(1) log
(6)
(1t+1 | Vi
=M1 (Ye41,
A) M1 (Yet1, (2(k) IA).
(7)
In this expression, at [0, 1] is the attention coefficient at time step and the collection of attention coefficients is the attention series. Comparison of [q[ and reveals that the transfer entropy can be viewed as a simplified version of AeTE in which attention coefficients are uniformly set to one: VI, d; = 1. Because each position has equal contribution to estimation, the value of transfer entropy dominated by the majority of positions where causal effect is negligible, i.e.,
where p [y++1 1/: (k) (1) p ye+1 Me (R) Similarly to transfer entropy, AcTE is derived the
difference of two mutual informations, but AeTE incorporates the scheme of attention assignment.
By connecting Eq. 7 and Eq. 2 we define differentiable estimator of AeTE as:
( sup e
Σ ae fo (UL+1.8( .x. log
(K)
( e e xt))
L
Σ at fo (JetI. log ( '' as e clo(vr.v))))
sup
(8)
L
where is the total number of time steps and T maxax (k: 1). The expectation on the distribution of variables is adapted into the mean over time series.
3.3 CAUSAL ATTENTION MECHANISM
The overall framework of our model is presented in Figure 2 In addition to two neural networks fo and Sa for mutual information estimation, employ another neural network ga for causal attention
4

---

Under review as a conference paper at ICLR 2023
Input Data
Figure 2: Graphical illustration of causal attention mechanism framework. input sample is the time series on order pair variables with shape [2, LJ. Stage I: the neural network 9a assigns the attention efficients {a)] The neural networks fo and fg forming transfer entropy estimator estimate mutual information M 1 and M2 (first and second terms in Eq. 77. Stage 2: the inferred probability of causality is Sigmoid( I Σ a; logil:).
Binary Classifier
Causal Attention
Transfer Entropy Fstimator
[logu:]f
[a:]]
M
M
Stage
Stage
7
lngit.
assignment. Rather than approximating distributions, the neural network 9o learns to maximize AeTE given by Eq. X via gradient descent. However, the occurrence of high transfer entropy regions may appear due noise in empirical samples. For more sophisticated inference, method with a binary classifier h, guided by causal attention to focus on high transfer entropy regions and recognize different patterns between noise and sparse emergence of causal effect. The classifier takes causality and non-causality as class labels. Then, the training process is divided into two independent stages: causal attention learning and classification learning. The objectives in the first stage are:
augment
0, argmax L1 L2 6,0/a
QK + argmax L1 L2,
(10)
aif.b
where L1, L2 is the expectation of the first and second sup term of Eq on training set respectively. We update (Jo; fa) and Ja alternately. A small learning rate is required to maintain training stability, otherwise the 9a may fall into trivial solution where attention is almost zero throughout the time series. The objective in the second stage
f argmin C3,
(11)
where L3 is the binary cross entropy and the notation y a indicates that causal attention remains fixed during the second stage of training. The downstream classifier is sensitive to the upstream scheme of attention assignment. In experiments, there exists an optimal loss interval for training the attention model 9a. We stop the first stage training when the loss value of objective Eq, T0 is stable in this interval, and then the downstream classifier h. usually obtains the best generalization perfor- mance. For different dynamics, their optimal intervals are different and we find them empirically) Details on the implementation causal attention mechanism are provided in Appendix C.
4 EXPERIMENT
We describe our experiment setup and extensively evaluate the causal attention mechanism on neu- ronal dynamics coupled on both synthetic and real causal networks.
4.1
SETUP
Causal networks. For synthetic networks, we generate ten groups of Erdōs-Rényi (ER) and scale- free (SF) directed networks with one hundred nodes (i.e., variables) uniformly and with mean degree varying from 5 to 41 by adjusting the probability for edge creation in ER and the total number of
An alternative design, which we have not yet implemented, would involve joining the first and second stages. Attention model ga. would be trained by not only maximizing AcTE but also responding to feedback from the classifier, and would find the balance between Eq. 10 and Eq autoomatically.
5

---

Under review as a conference paper at ICLR 2023
edges in SF. Symmetric links (both 2; - 2j and z; xz2) can exist. For each of network pa- rameters we consider five independently generated instances. For real networks, we select five neu- rological connectivity datasets presented in Table [ each from different species: Cat. Macaque, Mouse, Worm and Rat.
Dataset
Region
#Nodes
#Edges
Mean degree
Table I: Statistical information of
five real networks: dataset name, type of network, number of nodes, number of edges and mean degree (k). Detailed introduction are pro- vided in Appendix E.
Cat
Brain
65
1139
17.5
Macaque
Brain
242
4090
16.9
Mouse
Cortex
195
214
1.1
Worm
Neural
272
4451
16.4
Rat
Brain
503
30088
59.8
Table Equations of the five dynamical models considered. B is the asymmetrical adjacency matrix of the causal network, recording causal relationships between nodes. B 1 if variable is the parent of variable j. otherwise B, = 0. In these expressions, I describes the coupling-drive while other terms represent self-drive. The detailed configuration of dynamical parameters provided in Appendix D.
Dynamics
Equations
1. = app + bp
Texs +
Hindmarsh-Ro
4: = c- dp? 9.
18, = (p. po) ml
9e (Vyn pc) Σm B,, cxp( Osya)))
CV =I 9u V2) m. (V) (V Vo.) 9K Vx)
Mecris-Lecar
à = (n (V) n), r - B: (n: n:)
1. = 0.01o + 5v + 140 u, I+r
Izhikevich
1. = a(br, - m.) r == s. Biza5
B
Fy m))
w. (u,), Fa(u;, u. 2/1;
Rulkov
1
((y) 3 B15/ (1 +1++ exp(A (ug -.)))
à == a bu ov² + do -u
FitzHugh-Naguma
= =tev - w). r= -e -. Bu (e, w.)
Dynamic models. We use five dynamic models for neural activity simulation widely used in the field of neuroscience: Hindmarsh- Rose (HR), Morris-Lecar (Morris), Izhikevich (Izh), Rulkov and SitzHugh-NNguun (FHN). Dynamic equations are provided in Table 2 and segments of generated time series are represented in Figure
Evaluation metrics. We measure the following metries: (1) the area under the receiver operating characteristic curve (AUROC); and (2) the area under the precilion-recal curve (AUPRC).
Baselines. We compared our method with seven basclines: (1) Granger causality test (Ganger) Granger (1969): (2) Transfer Entropy (TE), in Eq. [ (3) Convergent cross mapping (CCM) Sug- ihara et al. (2012): (4) Latent convergent cross mapping (Latent CCM) [De Brouwer al. (2020); (5) PCMCI Runge et al. (2019) and (6) PCMCI+ Runge (2020) using partial correlation to quan- tify causal strength; (7) Classification model with Convolutional Block Attention Module (TA) Woo
al. (2018).
Training details. We employ the convolutional neural network for model Ja and hn and the fully- connected neural network for model fo and fo- We use the ADAM Kingma & Ba (2014) optimizer with the initial learning rate of 10-4 for classifier h. and 10 for the others. The batch size is 10. For synthetic networks, we select randomly twenty ordered pairs of variables as a training/validation set and four hundred ordered pairs a test set. For real networks, the sample set scheme is provided equal samples with causality and without causality. The total time step T of time series is 50,000. Gaussian measurement noise is added with mean zero and stan- dard deviation 1%/10% that of the original time series for symthetic/real networks respectively. We all experiments in this work local machine with two NVIDIA Tesla V100 32GB GPUs.
in Table 3 All sets are composed

---

Under review as a conference paper at ICLR 2023
verieble
Child variable
Power
-1
0
200
400
600
800
100
200
300
400
500
1.0
1.0
Coupling force
Strength
Treditional attention
Strength
0.5
Causal attention
0.5
Emergence of causal effect
Remain dormant
300
500
(a) Hindmarsh- Rose
(b) Izhikevich
Power
100
150
200
250
300
Strength
Strength
0.5
0.5
0.0
100
200
300
400
500
50
100
250
(c) Fiteluugh-Ngumoo
(d) Rulkov
Figure 3: Insight into the coupling drive of dynamics. Top panel in each subplot: segment of time series from ordered pair of variables with causal relationship (blue is parent and green is child). Bottom panel in each subplot: the absolute value of coupling force (gray line), traditional atten- tion (light pink line), and causal attention (orange line). (a) Hindmarsh-Rose; (b) Izhikevich; (c)
FiizHugh-Nagumo; (d) Rulkov.
4.2 RESULT
Insight into the coupling-drive of underlying dynamies. The gray lines in Figure represent the change of coupling force from parent child variable over time, and generated by the coupling term in Table The absolute value of the coupling force rises (the gray lines spike) at occasional moments when the behavior of a parent variable substantially influences the evolution of its child variable, and remains almost zero at other times. The orange lines representing the causal attention keep step with the gray lines, indicating that the causal attention mechanism recognizes the effect of coupling force in reducing the uncertainty of the child variable and pays attention to areas where coupling force significant. Figure dd, the causal attention focuses on two separated regions where the coupling forces have concentrated bursts. In contrast, the light pink lines representing the traditional attention remain close to their maximum value, indicating insensitive changes in coupling force. This leads its classifier to extract features throughout the whole time series (instead of focusing on causal features). The traditional attention are not designed for causal reasoning and cannot accommodate the selection of features that correspond the causal information.
1.0
1.0
1.0
1.0
AUMRICCA
WUROC.C CA
AUmIC T4
0.7
AUROC 14
0.7
0.7
0.7
A.MICTT
0.6
0.5
0.6
2.6
20
Degres
Figure 4: Comparison of classifiers synthetic causal networks, CA: Causal Attention. Examples of dynamical model: (a) HR on ER; (b) Izh on ER; (c) HR on SF; (d) Izh on SE.

---

Under review as a conference paper at ICLR 2023
Table Performance comparison on real causal networks. Values the first column of each dataset are AUROC, and values in the second column are AUPRC. Each point contains the mean and stan- dard deviation computed in five experiments with randomly sampled raaning/validationnesst set in Cat (top, 10/10/500) and Mouse (bottom, 10/10/90) connectomes. Results other three connec- tomes are shown in Appendix F.
Hindmarsh-Rose
Morris-Lecar
Izhikevich
Rulkov
FitzHugg-Nagumoo
Granger
0.50=0.01 0.50e0.01
0,57a0.01 0.5710.01
1.59:0.02
0.5920.02
0.6520.01
0.66:0.01
0.53+0.01
0.5320.01
TE
0.59#0.01
0.5420.01
0.4420.01
0.49+0.09
0.40:0.03
0.4420.02
0.74x0.01
0.73+0.02
0.57=0.02 0.62e0.01
CCM
0.7640.01
0.76e0.02
0.5840.02
0.6010.02
0.51e0.01
0.51a0.01
0.5710.01
0.5740.01
0.8840.02
0.6840.02
Latent CCM
0.6920.02
0.8920.01
0.5510.02
0.5510.02
0.4840.03
0.4820.03
0.5110.01
0.5120.02
0.6420.02 0.8420.01
PCMCI
0.53=0.01
0.58e0.01
0.49:0.01
0.49:0.01
0.48:0.01
0.49s0.01
0.6810.01
0.6610.01
0.53e0.01
0.53s0.01
PCMCI*
0.57+0.01
0.5740.01
0.52x0.02
0.52+0.01
0.52:0.02
0.52s0.01
0.63#0.02
0.63+0.02
0.56e0.01
0.58a0.01
0.78+0.01 0.76e0.02
0.61#0.01
0.59+0.01
0.53+0.01
0.52+0.01
0.53+0.01
0.5240.01
0.67+0.01 0.64e0.02
ATE
0.80=0.01
0.6240.01
0.92=0.01
0.89w0.01
0.70:0.01
Hindmarsh-Rose
Morris-Lecar
Izhikevich
Rulkov
FitzHugh-Nagumo
Granger
0 0.62:0.03 0.62x0.02
0.5120.02
0.51:0.01
0.54
0.5420.02
0.89:0.01
0.89s0.02
0.2120.04
0.2110.01
TE
0.40=0.03
0.44x0.02
0.53+0.05 0.62+0.06
0.57=0.02
0.96e0.03
0.33e0.04 0.40sa002
0.30e0.03 0.33e0.01
CCM
0.56.0.05
0.5840.04
0,2410.05
0.2410.01
0,4720.04
0.47s0.04
0.5810.01
0.5810.02
0.58s0.05
0.58s0.04
Latent CCM
0.4720.01
0.4720.01
0.5120.02
0.5110.01
0.5120.01
0.5120.02
0.5310.02
0.5320.02
0.53e0.01
0.53a0.01
PCMCI
0.50:0.02
0.50e0.01
0.4610.03
0.4610.01
0,50=0.02
0.50e0.01
0.7710.03
0.77a0:03
0.13e0.02
0.13:0.01
PCMCI*
0.53+0.03
0,50#0.01
0.3810.04
0.3810.01
0.50s0.05
0.50s0.02
0.7910.02
0.7940.01
0,21#0.09
0.21a0.01
TA
0.97=0.02 0.95e0.04
0.51+0.02
0.53+0.02
0.51e0.03
0.51e0.02
0.89+0.01 0.83+0.05
0.94s0.01 0.89+0.01
ATE
0.98+0.01 0.96:0.02
0.85+0.01 0.82+0.01
0.9240.03 0.91+0.03
0.98:0.01 0.96:0.02
0.89+0.04 0.85+0.05
Performance on test sets. Compared with the baselines, our method usually substantially improves reconstruction performance both synthetic and real causal networks, shown in Figure 4 and Table 3 In contrast, the classifier with traditional attention mechanism (TA) obtains low losses on training sets but has poor generalization on sets, highlighting that mere statistical correlation for causal inference is unstable and can be spurious Cui & Athey| (2022). The performance of classical unsupervised methods, for which all positions in the time series are treated equally, is also limited by the paucity of causal effects. These patterns demonstrate the importance of identifying and focusing on critical regions, which we achieve via the causal attention mechanism. In conclusion, our method slightly increases cost, due to the need for label collection, but obtains a substantial boost in per- formance compared with those unsupervised methods in this class of causal network reconstruction tasks.
The performance of all methods tends to decrease as the average network degree grows. Networks with larger average degree are more likely to exhibit synchronization of variables which makes it harder to distinguish cause and effect. Furthermore, a single variable in these denser networks can have many parent variables, and substantial coupling forces emerge from distinct parents at overlapping times, making individual drivers harder to distinguish. In this circumstance, a slight variance in the scheme of causal attention assignment may cause fluctuations in the performance of the downstream classifier. Robustness of the proposed method to measurement noise and sequence length is presented Appendix F.
RELATED WORK
5.1 CAUSAL NETWORK RECONSTRUCTION
Conventional frameworks assume separability, i.e., that information about causes not contained in the caused variable itself. Several common methods SSpites& & Glymour (199I); Sun et al. (2013); Runge et al. (2019): Mastakouri al, (2021) are based conditional independence relations, but
8

---

Under review as a conference paper at ICLR 2023
differ in the design of condition-selection strategies choice of conditional independence test. Granger Causality Granger (1969) is extended to nonlinear dynamics by using neural networks to represent nonlinear casual relationships Tank et al. (2021); :; Nauta et al. (201)). Many methods directed acyclic graph. However, directed eyclic graphs common in real systems. To address this non-separability issue, Convergent-cross mapping (Sugihara et al. (2012) and its variations Clark et al. (015);; De Brouwer et al. (2020) measure the extent to which the historical record of child can reliably estimate states of the parent in reconstructed state space. However, sparse causal effect in neuronal dynamics, particularly in the presence of noise, may lead parent and child time series to appear statistically independent, so that their contribution to state estimation is hard to recognize.
of causal discovery assume that the causal network
5.2 MUTUAL INFORMATION ESTIMATION
Belghazi et al. Belghazi et al. 2018 built on a dual representation of KL divergence Donsker & Varadhan 1983) to offer parametric mutual information neural estimator (MINE) which is linearly scalable in dimension as well as sample size, and is also trainable and strongly consistent. They also discussed another version of MINE based on the f-divergence representation Nguyen et al. (2010); Nowozin et al. (2016). Using the technique of Contt-Cvnt Estimation (NCE) Gutmann & Hyvärinen| (2010), based comparing target and randomly chosen negative samples, Van den Oord et al. Van den Oord et al. (2088) proposed InfoNCE loss, minimization of which maximizes a mu- tual information lower bound. An important application of this contrastive learning approach has been extracting high-level representations of different data modalities Chen et al. (2020); Woo et al. (20TT) Hu et al. (202T); Koch-Janusz & Ringel (2018). In our work, we extend MINE for transfer entropy estimation.
CONCLUSION
The problem of reconstructing causal networks from observational data is fundamental in multiple disciplines of science including neuroscience, since a prerequisite foundation for the research about structure analysis and behavior control in causal networks. Especially, several countries have recently launched grand brain projects, and one important goal to map the connectomes (i.e., directed links between neurons) of different species.
We proposed a novel mechanism, causal attention, to guide machine learning models to infer causal relationships while focusing the specific areas where casual effect may emerge. We showed that this mechanism identifies weak causal effects ignored by classical techniques, and helps machine learning models gain insight into the coupling dynamics underlying time series data. Our method needs a small set of samples (i.e., a small number of known causal links), and thus raises an open problem worthy of future pursuit: for large complex systems, how to select the small number of ordered pairs of nodes that offer general pattern for identification of sparse causal effects.
REFERENCES
Elias Bareinboim and Judea Pearl. 'Transportabllity from multiple environments with limited exper- iments: Completeness results. Advances Neural Information Processing Systems, 27, 2014.
Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, et al. Interaction networks for learning about objects, relations and physics. Advances in Neural Information Processing Systems, 29, 2016.
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In International Conference on Machine Learning, pp. 531-540. PMLR, 2018
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, pp. 1697- PMLR, 2020.

---

Under review as a conference paper at ICLR 2023
Adam Thomas Clark, Hao Ye, Forest Isbell, Ethan R Deyle, Jane Cowles, G David Tilman, and George Sugihara. Spatial convergent cross mapping to detect causal relationships from short time series. Ecology. 9665):1174-1181, 2015.
Peng Cui and Susan Athey. Stable learning establishes some common ground between causal infer- ence and machine learning. Nature Machine Intelligence, 4(2):110-115: 10-115, 2022.
Edward De Brouwer, Adam Arany, Jaak Simm, and Yves Moreau. Latent convergent cross mapping. In International Conference on Learning Representations, 2020.
Monroe D Donsker and SR Srinivasa Varadhan. Asymptotic evaluation of certain markov process expectations for large time. iv. Communications on Pure and Applied mathematics, 36(2):183- 212, 1983.
Andreas Gerhardus and Jakob Runge, High-recall causal discovery for autocorrelated time series with latent confounders. Advances in Neural Information Processing Systems, 33:12615-12225 2020.
Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: Journal of the Econometric Sociery. pp. 438, 1969.
Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 297-304, JMLR Workshop and Conference Proceedings, 2010.
Devon Hjelm, Alex Fedorov, Samuel Caavoie-Marchildoo. Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. arXiv preprint
2018.
Yeping Hu, Xiaogang Jia, Masayoshi Tomizuka, and Wei Zhan. Causal-based time series domain generalization for vehicle intention prediction. In NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications, 2021.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Maciej Koch-Janusz and Zohar Ringel. Mutual information, neural networks and the renormaliza- tion group. Nature Physics, 14(6))578-582, 2018
Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual faimesss. Advances in Neural Information Processing Systems, 30, 2017.
Karima Makhlouf, Sami Zhioua, and Catuscia Palamidessi. Survey causal-based machine learn- ing fairness notions. arXiv preprint arXiv:2010.09553, 2020.
Atalanti A Mastakouri, Bernhard Schölkopf, and Dominik Janzing. Necessary and sufficient con- ditions for causal feature selection in time series with latent common causes. In International Conference on Machine Learning, pp. 7502-7511. PMLR, 2021.
Meike Nauta, Doina Bucur, and Christin Seifert. Causal discovery with attention based convolu- tional neural networks. Machine Learning and Knowledge Extraction, 1(1):312-340) 2019.
XuanLong Nguyen, Martin J Wainwright, and Michael Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 56(11):5847-586 2010.
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. Advances in Neural Information Processing Systems, 29, 2016.
Liam Paninski. Estimation of entropy and mutual information. Neural Computation, 15(6): 1191 1191- 1253, 2003.
10

---

Under review as a conference paper at ICLR 2023
Dana Pessach and Erez Shmueli. A review faimesss in machine learning. ACM Computing Surveys (CSUR), 55(3):1 44, 2022.
Niklas Pfister, Peter Bühlmann, and Jonas Peters. Invariant causal prediction for sequential data.
Journal of the American Statistical Association, 114(527):1264-122) 2019.
Jakob Runge. Discovering contemporaneous and lagged causal relations in autocorrelated nonlinear time series datasets. In Conference on Uncertainty in Artificial Intelligence, pp. 1388-1397
PMLR, 2020.
Jakob Runge, Peer Nowack, Marlene Kretschmer, Seth Flaxman, and Dino Sejdinovic. Detecting and quantifying causal associations in large nonlinear time series datasets. Science Advances, (11 (1))au9949 2019.
Thomas Schreiber. Measuring information transfer. Physical Review Letters, 85(2):461, 2000.
Shohei Shimizu, Patrik O Hoyer, Aapo Hyvärinen, Antti Kerminen, and Michael Jordan. A linear non-Gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.
Peter Spirtes and Clark Glymour. An algorithm for fast recovery of sparse causal graphs. Social Science Computer Review, 91):622-7, 1991.
George Sugihara, Robert May, Hao Ye, Chih-hao Hsich, Ethan Deyle, Michael Fogarty, and Stephan Munch. Detecting causality complex ecosystems. science, 338(6 106):49--500, 2012.
Jie Sun, Dane Taylor, and Erik Bollt. Causal network inference by optimal causation entropy.
SIAM Journal on Applied Dynamical Systems, 14(1):73-106, 2015.
Alex Tank, Ian Covert, Nicholas Foti, Ali Shojaic, and Emily B Fox. Neural Granger causality.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(8):4267-42799 2021.
Aaron Van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic-
tive coding. arXiv e-prints, pp. arXiv-1807, 2018.
Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Cost: Contrastive learn- ing of disentangled seasonal-trend representations for time series forecasting, In International Conference on Learning Representutions, 2021.
Sanghyun Woo, Jongchan Park, Joon-Yon YounYoung Lee, and So Kweon. Cbam: Convolutional block attention module. In Proceedings of the European Conference Computer Vision, pp. 19, 2018.
Feng Xie, Ruichu Cai, Biwei Huang, Clark Glymour, Zhifeng Hao, and Kun Zhang, Generalized independent noise condition for estimating latent variable causal graphs. Advances in Neural Information Processing Systems, 33:148911-448 14902, 2020.

---

Under review as a conference paper at ICLR 2023
A DERIVATION
Here we present derivation showing that the transfer entropy equals the difference between two types of mutual information:
(yt+1 (M) xz("
TE(X -YY Ep (1e+1.3h (k) x2) log
(12)
Me+1 | 3/ (8))
Applying the conditional Bayes formula (y (ylx) log term of equation 12
p(u.t)
on the numerator and denominator in the p(a)
P (1041100 Mi (A) (9)
p (2())
-
(K)
(1)) log
(13)
(3-11.1 (A)) P (((())
Adding the marginal distribution of time series Y to the numerator and denominator simultaneously:
p(u+1100 " (2 p()-)p u8 ,x( 22("
TE(X +Y)= = (M++1.V1% ,x( ) log (0)
(14)
p
p(y.)a(u??))
ye+1, Ve (1)
(1)
Σ Me+1,11 , ax()) log
(Brr) V/
(N)
xx
P (ueti, (k)
Σp (VcrI. log
(15)
p(rr) p ye
MT ( x(") MI (Y+1,, (V+t,,,())
(16)
In these expressions, is sampled from time series Y randomly each time step and independently of the time step I.
B NEURAL ESTIMATOR FOR TRANSFER ENTROPY
B.1 CONSISTENCY
Definition. A neural estimator Š( Y), which uses n samples from the data distribution to estimate statistic S(X, Y) on variables X, Y is strongly consistent if for any e there exists positive integer N and a choice of neural network such that:
Vn 2 N, S(X,Y) 5(X, Y)n I almost everywhere (a.e.)
(17)
The Mutual Information Neural Estimator (MINE) depends choice of neural network and the number samples n from the data distribution Belghazi et al. (2018). Let fe be the family of functions parameterized by the neural network with parameters θ. MINE is defined
MT(X, Y),, sup F. Fpin) [fo] - llog
F. (Fppe ere [el]])
(18)
Theorem Belghazi et al. 2018 MINE is strongly consistent.
The Transfer Entropy Neural Estimator (TENE) consists of two independent MINE and depends on choice of neural network and sample number n. TENE is defined as:
TE(X + Y)n Mi (Yeth, (y xx,)))))) -Mi (Yet))
(19)
12

---

Under review as a conference paper at ICLR 2023
We use MTW), MI MT." MT, as abbreviations of MI (Y++1, (,(0)x!)) and MI (Y44,, (( MI (Yet2, (y, xx and MI (, respectively.
We will prove the following:
Theorem 2. TENE is strongly consistent.
Proof. Let > 0. By Theorem I, we can choose neural networks and integers N1, N2 and such that
Vn 2 NI,
M MT
S $r/2,
(20)
M -M
Vn N2,
S 5//2,a.e.
(21)
Letting N max {Ni, N2), for 2 N and for some neural network we have, a.e.,
(M M1(21 (MI, -MI, [2]
Vn 2 N. TE(X +Y - TE(X +)
(22)
(M -TT (M MT n
[21
(23)
$ (M - -MI,""
(M - -MI, .2))
(24)
S e/2 e/2
(25)
The proof is complete.
B.2 VARIATION OF BIAS VARY WITH DIMENSION AND NOISE
We examine the performance of TENE for the considered class of neural networks on linear dynamic system, consisting of variables X and Y defined as:
T++1 = O.Et + Ex
(26)
1/+1 By: ge + Ey
(27)
We set a 3 = 0.5 and 6.r = 62 ~ ((0,02) The true values of transfer entropy TE X - Y) in this simple coupled system can be determined analytically Kaiser Schreiber (2002). We can increase the dimension of the system by considering multiple independent copies of variables X and Y, in which case the mutual information and transfer entropy scale linearly with the dimension of the system. For each considered dimension, standard deviation a, and coupling strength in an interval from -0.4 to 0.4, we generate time series of length 50,000. We also consider an alternative non-parametric estimator of mutual information, the Kraskov estimator Kraskov et al. (2004) with == nearest neighbours. Fig we compare the results of MINE with the analytic formula and Kraskov estimator. MINE shows marked improvement over the Kraskov estimator, especially when variables are high-din nigh-dimensional. Comparing Fig/(ja.b) a.b) or (c,d) shows that the amplitude of the driving Gaussian noise has little influence estimates. Interestingly, i.e, as X and become more independent, the Kraskov estimator can suggest a negative value of the mutual information. i.c.. we estimate that M,, (Y++1, Y,, X,) < M (Ye+1, Y.). We deduce that irrelevant information about the nearly independent variable X, interferes with the estimation of the mutual information by the Kraskov estimator.
coupling strength Je grows small,
C ALGORITHM
Details the implementation of causal attention mechanism are provided in Algorithm
13

---

Under review as a conference paper at ICLR 2023
(a) Dim=3 Std=1
(c) Dim=20 Std=1
Analytic
0.25
Kraskov
Nerual
1.5
0.20
Transfer entropy
1.0
0.15
0.10
0.05
-0.4
-0.3
-0.2
-0.1
0.0
0.1
0.2
0.3
0.4
-0.4
-0.3
-0.2
-0.1
0.0
0.1
0.2
0.3
0.4
Coupling strength
Coupling strength
(b) Dim-3 Std-5
(d) Dim-20 Std-5
0.30
0.25
1.5
0.20
Transfer entropy
1.0
0.5
0.0
-04
-0.3
-0.2
-0.1
0.0
0.1
0.2
0.3
0.4
-0.4
-0.3
-02
-0.1
0.0
0.1
0.2
0.3
0.4
Coupling strength
Coupling strength
Figure 5: True and estimated transfer entropy versus coupling strength ge- The dimension and stan- dard deviation (std) a of system noise is indicated in the titles of subplots.
Algorithm Causal Inference by Causal Attention
Input: Small samples S, part with causality S C S, segment length for training İ < L
0, di ax 7/ <- initialize network parameters
2: repeat until (L, La) is stable in optimal loss interval
3: for all samples do
4: Choose t, [1 L] randomly
5: Take segment { yi 1
6: Produce joint samples in Eq.
( a.") fo+2 etc.
7:
8:
end for
9:
Assign causal attention A
Compute L1, La on 5 0. + 0 + V VoL2
11:
Update parameters
12:
Recompute L1, La on
13:
Update parameters
V a (L, -22)
14:
repeat until L3 convergence
Compute L3 on S
16:
Update parameters 7 17 V,Ls
14

---

Under review as a conference paper at ICLR 2023
D MODEL BRAIN DYNAMICS
Here we give detailed information about five neuronal dynamics applied to modeling membrane potential and relevant quantities in biological connectomes. We input to each causal discovery algo- rithm the coordinate corresponding to the neuron membrane voltage potential, because this variable most likely to be experimentally accessible.
D.1 HINDMARSH- ROSE DYNAMICS
The spikes of activity in neurons are considered an important part of the brain's information process- ing Borges et al. (2018): Rabinovich et al. (2006). Hindmarsh and Rose Hindmarsh & Rose (1984) (HR) proposed phenomenological neuron model that is simplification of the Hodgkin Hodgkin-Huxley model model Hodgkin Huxley 1952). The HR model is described by
ap" + bp + Lext
dp² q
= (s (p- (p)po) n)
where p(t) is the action potential of the membrane, q()) is related the fast current and n(() is sociated with the slow current. Presynaptic neurons with an action potential Pj coupled by chemical synapses to neurons modifying its action potential p; according to
p = ap, + bp? n + Text +
Bi
r gr (Vsyn p.) E + exp(-A Owyn))
where j.j = N,, N is the number of neurons, is the chemical coupling strength and B describes neurons' chemical connections. The chemical synapse function modeled by the above sigmoidal function, with ,yn = 1.0. We use parameters = 6 = 3, 1, u = 5, = 4. 0.005, Po 1.60, coupling strength S 0.1, Vayn 2, À 10, and external current Text 3.24, for which HR neurons exhibits chaotic burst behavior.
D.2 MORRIS-LECAR DYNAMICS
Morris and Lecar Morris Lecar 1981 suggested simple two variable model describe os- cillations in barnacle's ' giant muscle fiber. The Morris- model has became quite popular in computational neuroscience community due its biophysically meaningful and measurable param- eters, which consist of a membrane potential u receiving instantaneously activated Ca current and a more slowly activated K current n evolving according to:
CV =I 92 (V Vi.) 3c. =X(V) (noc (V) n)
(V) (V) Vca) 9Kn (V VK) +I()
where
(V 1/2)
Mco (V)
2 + tanh
V/2
[V [V- Vz)
noo (V) = 2
+ tanh
V4
(V Vis)
X(V) cosh
(2V))
with the coupling term
r(V) = 9. 3 Bij (nz - n4),
with parameters C 20μF/
/ /cm/ gL 2mmhho cmm Vz -50m5 50 gc. 4mmho/cm² Vcs
100mV, = Smmho/ cmm Vx = -70mV, Vi = 0mV, V/2 = 15mV, V/3 = 10mV, V4
10m A= 0.1 s1, and applied current = 344A/cm²
15

---

Under review as a conference paper at ICLR 2023
D.3 IZHIKEVICH DYNAMICS
lzhikevich dynamics reproduce spiking and bursting behavior of known types of cortical neurons, and combine the biological plausibility of Boadgkin-Huxle--yeee dynamics and the computational efficiency of integrate-and-fire neurons neurons[zziieviide Izhikevich (2003). The equations governing Izhikevich spike
dynamics are:
è = 0.04c + 50 + 140 - u + +++ + Dc E Bizuz
ú albv u)
with the auxiliary after-spike resetting
if +30mV, then { ut u t u d
Here, variable e represents the membrane potential of the neuron and u represents a membrane recovery variable, which accounts for the activation of K*ionic currents and inactivation of Na ionic currents, and it provides negative feedback to u. Here, we use the parameters = 0.2, b 2, c = -5, 16, = -99. After the spike reaches its apex (+30m V), the membrane voltage and the recovery variable are reset. If e skips over 30, then it first is reset to 30, and then to so that spikes have equal magnitudes.
D.4 RULKOV DYNAMICS
The Rulkov model is map-based neuron model with surprising abundance of features, such as periodic and chaotic spiking, and bursting. The Rulkov map is an abstract mathematical model, although shares some specific features with others neuron models closer to experimental observa- tions. We use synthetic time series where each neuron is simulated using the Rulkov model Eroglu al. (2000). which has two variables, и and w, evolving at different timescales as described by at = (u(t 1), utt + 1)) = F(x(()) = (Fi (u(t), w(t)), F2(u(t), w(t))), with
3
Fi (u, w) = 1 + u2
+ w + r (u)
and F2(u, w) = w vu-c
The two variables reflect the two important time scales of neuron model. The variable represents the fast dynamics of the system and usually models the membrane voltage of the neuron, whereas w is the slow variable and represents the variations of the ionic recovery currents. Different combi- nations of parameters a and give rise to different dynamical states of the neuron, such as resting, tonic spiking, and chaotic bursts. As for the coupling, we consider chemical synaptic coupling, that is, H (x,, x,) = (h (u, ,,), 0) with h [u,, Uz) = (u: - V&) I' (uy). where
r (u;) 1 + exp (A (u, - θ.)]
and electrical synaptic coupling, H (I,, 2.j) (h (u, uj) 0). with (uz uy) = Uj uy. In the chemical coupling, V, is parameter called the reverse potential. Here, we use the parameters with 4.4, a 0.001,, V, = 20, O = 0.25 and 10.
D.5 FITZHUGII-NAGUNE DYNAMICS
A HitzHugh-Nagume neuron comprises two-dimensional system of smooth ODEs, cannot exhibit autonomous chaotic dynamics and bursting. Adding noise allows for stochastic burst- ing FitzHugh| (1961). The equations governing the FitzHugh-Nagumo neuronal network dynamics
are
+ by + c2² + d -u T
E(ev - u)
with the coupling term
T(o;) = - - By v:).
The FitzHugh-Nan Nagumo dynamics capture the firing behaviors of neurons with two components. The first component e represents the membrane potential, which contains self- and interaction dynamics,
16

---

Under review as a conference paper at ICLR 2023
and the second component represents recovery variable. To simulate the shape of each spike, the time step in the model must be relatively small, e.g., T = 0.25 ms. Here, we use the parameters a = 0.28, b 1, c 0, d 1, 0.04, 12.5. Moreover, the parameters in the CartHuug-Nagumne model can be tuned so that the model describes spiking dynamics of many resonator neurons.
D.6 TIME SERIES GENERATION
To obtain the time series from above neural dynamics, we Runge- Kutta method with variable- step to solve the ordinary differential equation of Hindmarsh-Roo and Morris-Lecar dynamics with sample interval 7 0.1. Izhikevich dynamics are solved by the Euler formula with time step h 0.05. For the Rulkov map we consider a unit sample interval. The total time step of time series T = 50, 000 in both synthetic and real networks.
E REAL BRAIN CONNECTOMES INFORMATION
E. CAT CONNECTOME
The cat connectivity dataset comprises a description of cortical connections in the cat brain Scannell et al. (1995), connectivity resulting from comprehensive literature search of anatomical trac- ing studies in the cat cortex. Detailed information on the delineated regions, including information on the used parcellation scheme, abbreviations and possible overlap with other parcellation schemes, as well as information on the physiological characteristies of these regions, is given in the appendix of the original study Ref. [Scannell al. (1995). The connectivity dataset incorporates data of one hemisphere. including 65 regions and I139 interregional macroscopic axonal projections de Reus &
van den Heuvel (2033).
E.2 MACAQUE CONNECTOME
The macaque connectivity data set used in this study comprises anatomical data from 410 tract tracing studies collated in the online neuroinformatics data base CoCoMac (http://cocomacc.ogg first analyzed and made publicly available in Ref. Modha & Singh (011)) In the present study they focused primarily on an analysis of the comnectivity among regions of the cerebral cortex. The cortical connection matrix was extracted from the primary connection data by removing all subcortical (thalamus, basal ganglia, brainstem) regions. addition, regions that did maintain at least one incoming and one outgoing connection were also removed to ensure that the network was strongly connected. The remaining connection data set used in this study consisted of 242 regions and 4090 directed projections represented in binary format (connection present 1, connection absent = 0) Harriger et al. (012)).
E.3 MOUSE CONNECTOME
The Allen Mouse Brain Connectivity Atlas uses enhanced green fluorescent protein (EGFP)- expressing adeno-associated viral vectors to trace axonal projections from defined regions and cell types, and high-throughput serial two-photon tomography to image the EGFP-labelled axons throughout the brain. This systematic and standardized approach allows spatial registration in- dividual experiments into common three dimensional (3D) reference space, resulting in whole- brain connectivity matrix. The Allen Mouse Brain Connectivity Atlas freely available, foun- dational resource for structural and functional investigations into the neural circuits that support behavioural and cognitive processes in health and disease Oh et al. (2014).
E.4 WORM CONNECTOME
All the chemical and gap junction synapses, the connectome, in the posterior nervous system of the C. elegans adult male are identified by serial section electron microscopy Jarrell et al. (2012). The feasibility of comprehensive synapse-level nervous system reconstruction by this method was a primary reason for the initial selection C. elegans as an experimental model. They developed PC-based software platform to facilitate assembly of connectome from electron micrographic
17

---

Under review as a conference paper at ICLR 2023
images. The connectome is of a single adult animal and produced from series of 5000 serial thin sections of 70 to 90 nm encompassing the posterior half the body.
E.5 RAT CONNECTOME
Because resliceable 3D brain models for relating systematically and topographically different par- cellation schemes are still in the first phases of development, it is necessary to rely on qualita- tive comparisons between regions and tracts that are either inserted directly by neuroanatomists or trained annotators, or are extracted or inferred by collators from the available literature. To address these challenges, Ref. [Bota et al. (2012) developed publicly available neuroinformatics system, the Brain Architecture Knowledge Management System, including exemplar for constructing inter- related connectomes at different levels the mammalian central nervous system organization, and presented the latest version of the BAMS rat macroconnectome.
Information about the above datasets is summarized Table 4
Table 4: Statistical information of six real networks: dataset name, type of network, number of nodes, number of edges, mean degree (k)), and data acquisition method.
Dataset
Region
#Nodes
#Edges
Mean degree
Sensor
Cat
Brain
65
1139
17.5
Tract tracing studies
Macaque
Brain
242
4090
16.9
Tract tracing studies
Mouse
Cerebral Cortex
195
214
1.1
Electron microscopy
Worm
Neural
272
4451
16.4
Flectron Microscopy
Rat
Brain
503
30088
59.8
Neuroanatomical experiments
F ADDITIONAL EXPERIMENTS
F.1 PERFORMANCE OF TRANSFER ENTROPY ON REAL CAUSAL NETWORKS
Experiment Results on Cerraque/C.llegan/ee connectome are provided in Table Taable the supplement of main text Table Classic methods have limited performance across various neural dynamics unfolding on real causal networks especially in a noisy environment: we add Gaus- sian measurement noise with mean zero and standard deviation 10% that of the original time series. As we discuss main text Sec. 3.1, sparse causal effects are easily masked in metric iconic trans- fer entropy when noise causes estimated probability densities to deviate even slightly from the true distributions.
Table 5: Performance comparison train/ cain/validaionn/eestt set 50/50/500.
Macaque connectome. The sample number in
Hindmarsh-Rose
Morris-Lecar
Izhikevich
Rulkov
FitzHugh-Nagumo
Granger
0.50:0.01 0.50a0.01
0.4810.02
0.4840.01
0.50+0.02
0.50s0.01
0.5410.01
0.54a0.01
0.5410.01
0.5410.01
TE
0.4410.01
0.49&0.01
0.53+0.05
0.5210.05
0.54a0.03
0.51a0.02
0.4810.07
0.49a0.04
0.43e0.02 0.45a0.01
CCM
0.4400.02
0.44+0.02
0.4940.01
0.49H0.01
0.51e0.01
0.51s0.01
0.55+0.01
0.56+0.01
0.58+0.01
0.56#0.01
Latent CCM
0.5140.02
0.51+0.01
0.53+0.02
PCMCI
0.4700.01
0.47s0.01
0.51+0.01
0.5110.01
0.50e0.02
0.50e0.01
0.50+0.01
0.50+0.01
0.52:0.01
0.52:0.01
PCMC1*
0.47+0.00
0.47+3.01
0.48+0.01
0.48#0.00
0.51=0.02
0.51e0.01
0.52+0.00 02
0.5240.00 01
0.52e0.01
0.52+0.01
0.68:0.03
0.51g0.03
0.5810.01
0.55:0.01
0.52:0.04
0.50e0
0.5110.02
0.5120.00
0.54g0.01
0.52:0.01
ATE
0.7120.04
0.65x0.09
0.60:0.02
0.59:0.01
0.66x0.08
0.57e0.08
0.63:0.02
0.59:0.02
18

---

Under review as a conference paper at ICLR 2023
Table Performance comparison on C.elegans connectome. train/validation/festt set is 50/50/500.
The
sample
number
in
Hindmarsh-Rose
Morris-Lecar
Izhikevich
Rulkov
FitzHugh-Nagumo
Granger
0.50=0.01
0.50e0.01
0.9610.01
0.5620.01
0.6420.01
0.6120.02
0.7120.01
0.7120.01
0.61200.2 02.0 0.6420.01
0.7420.01
0.73:0.02
0.6120.01
0.58+0.00 02
0.56e0.01
0.56e0.02
0.80+0.01
0.80+0.01
0.52e0.01 0.48e0.01
CCM
0.7940.02
0.79e0.02 02
0.5410.02
0.6410.02
0.82:0.02
0.62s0.02
0.6710.02
0.6740.02
0.74a0.01
0.74a0.01
Latent CCM
0.7840.01
0.7840.01
0.5210.01
0.5240.01
0.5840.04
0.5620.03
0.5310.04
0.5310.03
0.8820.01
0.88s0.01
PCMCI
0.53=0.01
0.53e0.01
0.50+0.02
0.50+0.01
0.53+0.02
0.53e0.01
0.86s0.01
0.88x0.01
0.57s0.01
0.59x0.01
PCMCI*
0.53+0.02
0.53e0.01
0.52x0.01
0.52+0.01
0.56:0.01
0.56a0.01
0.6910.02
0.69x0.02
0.63e0.01
0.63a0.01
0.92+0.01 0.91e0.01
0.67+0.03
0.88+0.03
0.52=0.01
0.54x0.01
0.56+0.01
0.54+b.01
0.85+0.01
0.84:0.01
0.93:0.01
0.92:0.01
0.8120.01
0.78c0.01
0.76:0.01
0.7120.01
0.97=0.01
0.97:0.01
0.7810.01 0.72:0.02
Table 7: Performance comparison on Rat connectome. The sample number in train/validdaiinntest
set is 100/100/500.
Hindmarsh-Rose
Morris-Lecar
Izhikevich
Rulkov
FitzHugh-Nagumo
Granger
0.50=0.01 0.50s0.01
0.43+0.02
0.43+0.01
0.50=0.01
0.50s0.01
0.48+0.01
0.4840.01
0.52+0.02 0.5260.01
TE
0.57+0.02
0.62+0.01
0.3360.02
0.41=0.01
0,48+0.08 08
0.54m0.08
0.37+0.01
0:43+0.01
0.1100033 0.47=0.03
0.63e0.02
0.6360.02
0.48+0.02
0.48+0.02
0.54e0.05
0.54ed.
0.50+0.01
0.50+D.01
0.52+0.08
0.52+0.02
Latent CCM
0.63:0.02
0.8810.02
0.40:0.01
0.4010.01
0.5820.01
0,56e0.02
0.5310.02
0.5310.01
0.5120.03
0.51s0.02
PCMCI
0.51=0.01
0.51=0.01
0.4410.02
0.4420.01
0.53e0.01
0.53e0.00
0.46:0.01
0.46:0.01
0.48:0.02
0.48=0.01
PCMC
0.5110.02
0.51a0.01
0.4260.02
0.4210.01
0.56g0.01
0.55a0.01
0.4410.
0.4410.01
0.49s0.01
0.49a0.01
TA
0.82+0.03
0.78e0.02
0.70+0.01
0.6740.01
0.82:0.02
0.58e0.03
0.73+0.02
0.74a0.01
0.7460.04 0.74+0.02
ATE
0.83+0.03
0.83+0.03
0.78:0.01 0.78a0.01
0.68:0.04
0.64:0.03
0.83:0.01
0.79:0.01
0.84:0.02 0.8110.01
F.2
DATA EFFICIENCY OF CAUSAL ATTENTION MECHANISM AGAINST THE LENGTH TIME SERIES
Taking Cat and Mouse connectome as examples, we show the results our method trained on the time series data with different lengths, i.c. total time steps, in Figure 6 The size of training and test sets is the same as the scheme in the main text Table 4. Overall, the AUROC/AUPRC scores tend get higher as the length increases, but this tendency is not significant. It indicates that the causal attention mechanism extracts sufficient causal features for causal inference even over short time series.
F.3 ROBUSTNESS OF CAUSAL ATTENTION MECHANISM AGAINST THE INTENSITY OF NOISE
We show the results of our method trained on time series data added different intensities of noise (measurement noise rather than intrinsic noise of dynamics) in Figure 7 Except for dynamics of Izh and FHN on Cat connectome, the AUROC/AUPRC scores are stable within the range 2%-10% of standard deviation. implies that causal attention mechanism is robust by the sample noise level.
19

---

Under review as a conference paper at ICLR 2023
AUPRC
Lengtht-104
2
Lengthc-109 u
Cat
Lengthi
0.5
0.7
0.9
1.0
0.5
0.6
0.7
0.8
0.9
1.0
0.5
0.6
0.7
0.8
0.9
1.0
AUROC/AUPRC
AUROC/AUPRC
AUROC/AUPRC
Morris-Lecar
Izhikevich
Lengthr-11
2
Length
Cat
0.5
0.7
0.9
AUROCIAUPRC
AUROC/AUPRC
Rulkov
FitzHugh-Nagumo
AUPRC
Lengthx110
4
Mouse
AUROC/AUPRC
AUROC/AUPRC
AUROC/AUPRE Izhikevich
Hindmarsh-RRose
Morris-Lecar
Lengthe110
2
Lengthe
0.7
0.8
0.7
1.0
AURDC/AUPRC
AUROC/AUPRE
FitzHugh-Nagumo
Figure 6: AUROC/AUPRC scores of the causal attention mechanism trained time series data of different lengths. The blue bar is AUPRC, the green bar is AUROC. The x-axis indicates the scores and y-axis represents the total time step of time series.
F.4 How MUCH DOES CAUSAL ATTENTION MECHANISM BRING THE DISTANCE BETWEEN DISTRIBUTIONS OF TRAINING AND TEST SET CLOSER TOGETHER
The causal attention mechanism helps the classifier reveal the generation processing underlying the data, i.e.. coupling-drive in problem of causal inference and alleviate the dilemma of distribution shift in scene of small samples. The causal attention mechanism refines the content of samples (critical regions) and thus reduces the distribution dimension of the entire dataset. To quantify this shortened distance, we ask, how many additional training samples does traditional machine learning need to achieve the same level of generalization as our method? Taking the Cat Connectome as example, we train the classifier with traditional attention mechanism by gradually expanding the size of training set, and provide the results Figure The green horizontal lines represent the AUROC value of our method using ten ordered pairs with causality (0.8% of edges in Cat Connectome) while, to achieve the same performance, the traditional classifier needs approximately samples for HRR/zhh/Moris/Rulkov dynamics respectively (the blue lines cross the green lines). It also indicates that our method provides significant saving in labels collection, which is significant given that the procedure for checking connections in organisms is cumbersome in practice.
20

---

Under review as a conference paper at ICLR 2023
10
AUPRC AUROC
10
10
Noisel15)
Notse/99)
Nolso
Cat
0.5
0.6
0.7
0.8
0.9
1.0
0.5
0.6
0.7
0.8
0.9
1.0
0.5
0.6
0.7
0.8
0.9
1.0
AUROC/AUPRC
AUROC/AUPRC
AUROC/AURRC
Hindmarsh-Rowe
Morris-Leca
Izhikevich
Noiet%)
Nolcot
Cat
0.5
0.6
0.7
0.8
0.9
0.6
0.7
0.8
0.9
1.0
AUROC/AUPRE
AUROC/AUPPC
IRulkov
FitzHugh-Nagumm
10
10
10
AUROC
Noise(%)
Noiset%)
Nolset
Mouse
0.8
10
AUROCIAUPRC
AUROCYAUPRC
AUROCIAUPRE
Hindmarsh-Ross
Morris-Lecar
Izhikevich
10
Natsel%)
Noiser%)
Mouse
2
0.7
0.8
0.9
1.0
0.7
0.8
0.9
1.0
AUROCIAUPRC
AUROCIAUPRC
IRulkov
FitzHugh -aagumo
Figure 7: AUROC/AUPRC scores of the causal attention mechanism trained time series data added different intensity of noise. The blue bar is AUPRC, the green bar is AUROC, and the y-axis indicates the percentage of standard deviation of Gaussian measurement noise in the original time scries.
0.58
0.7
0.8
traditonal attention
0.54
0.8
causal attention
10
20 Porcentage:76/ Rullow
Percentage196 (0
Percontage/9)
Pencontags(55) (4 Maris
du lah
Figure 8: Size of training set that the traditional classifier requires to achieve same level of general- ization method. The x-coordinate indicates the percentage of the ordered pairs with causality in training set to the total edges in causal network. Examples of dynamical model: (a) HR; (b) Izh; (c) Morris; (d) Rulkov.
21

---

Under review as a conference paper at ICLR 2023
REFERENCES
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In International Conference on Machine Learning, pp. 531-540. PMLR, 2018.
FS Borges, Ewandson I. Lameu, Kelly C larosz, Paulo R Protachevicz, Iberè I. Caldas, Ricardo I. Viana, Elbert EN Macau, Antonio M Batista, and Murilo da Silva Baptista. Inference of topology and the nature of synapses, and the flow of information in neuronal networks. Physical Review E, 97(2):022303, 2018.
Mihail Bota, Hong- Wei Dong, and Larry W Swanson. Combining collation and annotation efforts toward completion of the rat and mouse connectomes in bams. Frontiers in neuroinformatics, 6: 2, 2012.
Marcel A de Reus and Martijn P van den Heuvel. Rich club organization and intermodule commu- nication in the cat connectome. Journal of Neuroscience, 33(32): (11229):1929-129333 2013.
Deniz Eroglu, Matteo Tanzi, Sebastian van Strien, and Tiago Pereira. Revealing dynamics, commu- nities, and criticality from data. Physical Review X, 10(2):021047, 2020.
Richard FitzHugh. Impulses and physiological states in theoretical models of nerve membrane. Biophysical journal, 1(6):445-466, 1961.
Logan Harriger, Martijn P Van Den Heuvel, and Olaf Sporns. Rich club organization of macaque cerebral cortex and its role in network communication. PLoS ONE, 7(9):e46497, 2012.
James L Hindmarsh and RM Rose. A model of neuronal bursting using three coupled first order differential equations. Proceedings of the Royal Society of London. Series B. Biological Sciences, 221(122))877-022 1984.
Alan Lloyd Hodgkin and Andrew Fielding Huxley. Propagation of electrical signals along giant nerve fibres. Proceedings of the Royal Society of London. Series B-Biological Sciences, 140 1879):177-183, 1952.
Eugene M Izhikevich. Simple model of spiking neurons. IEEE Transactions neural networks, 14 (6):1569-1572, 2003.
Travis Jarrell, Yi Wang, Adam E Bloniarz, Christopher A Brittin, Meng Xu, J Nichol Thomson, Donna G Albertson, David H Hall, and Scott W Emmons. The connectome of decision-making neural network. Science, 337(6093) 444, 2012.
Andreas Kaiser and Thomas Schreiber. Information transfer in continuous processes. Physica D: Nonlinear Phenomena, 166(1-2):43-62, 2002.
Alexander Kraskov, Harald Stöghauer, and Peter Grassberger. Estimating mutual information. Phys- ical review E, (9(6))06613) 2004.
Dharmendra S Modha and Raghavendra Singh. Network architecture of the long-distance pathways in the macaque brain. Proceedings of the National Academy of Sciences, 107(30): 13485-)193485134999 2010.
Catherine Morris and Harold Lecar. Voltage oscillations in the barnacle giant muscle fiber. Biophys- ical journal, 35(1):193-2) 193-213, 1981.
Seung Wook Oh, Julie A Harris, Lydia Ng, Brent Winslow, Nicholas Cain, Stefan Mihalas, Quanxin Wang, Chris Lau, Leonard Kuan, Alex M Henry, et al. mesoscale connectome of the mouse brain. Nature, 508(7495):20-2214 2014.
Mikhail Rabinovich, Pablo Varona, Allen I Selverston, and Henry DI Abarbanel. Dynamical principles in neuroscience. Reviews of Modern Physics, 78(4):1213, 2006.
Jack W Scannell, Colin Blakemore, and Malcolm P Young. Analysis of connectivity in the cat cerebral cortex. Journal of Neuroscience, 15(2):1463-1483, 1995.
22


Reviews:
[{"review_id": "smp2_Kf_bgA", "reviewer": "Reviewer_Ube5", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a novel mechanism of attention to train neural networks to identify the coupling parameters of complex dynamical system under small perturbations. The problem belongs to the space of dynamical systems with strongly dissipative elements with small coupling strength. The idea is inspired by Schriber, 2000 to use transfer entropy to measure uncertainty in the causal relationships between variables. The results shown large improvements respect to other methods. And the attention variables shown in Fig. 3 appear to capture well the state-space volumes where external perturbations from other neurons/elements are expected to have larger impact. When the neurons are near the resting state perturbations are quickly erased, while exiting the saddles through the unstable separatrix amplifies the perturbation effect and the signal-to-noise ratio is expected to be more favorable. So, the method makes quite a lot of sense to me.\n\nMy recommendation to the authors is to present the limitations of this type of methodologies right after the contributions in the introduction. So, a new reader can quickly understand where and how this methodology can be applied. One of the limitations is the degradation of performance as the degree increases. It’s certainly problem dependent but it can be useful to know in what problem one can potentially apply this methodology. Another limitation is for dynamical systems with strong perturbations when, for example, synchronization phenomena dominate the system dynamics. The authors also show in the appendix that the method is quite sensitive to noise, some something should be written in this regard. It also appears to me that there’s quite a lot of effort setting up the right balance between the learning rates and the attention variable such that the cross-entropy loss leads to good generalization. Having guardrails on how to set the balance between the a-values and the cross-entropy loss would facilitate extending this method to other contexts. \n\nAnother more specific comment that I wish the authors could have addressed is the intended consequences of the mechanism of attention in the context of stable and unstable (saddles) fixed points in dissipative dynamical systems. The fixed points and their separatrices form the skeleton of trajectories and the saddles are the escape routes away of a resting state. Moreover, the neuron models the authors can have bistability and another neuron can flip the child one from resting state to the limit cycle, for example. Once in the limit cycle there will be very tiny modifications on the frequency of oscillations, but the pulse that switched the neuron away from the resting state is short lived. So, if one understands the bifurcations of the individual neuron dynamics, then we can infer where in the state-space external perturbations will lead to the largest information gains and, perhaps, we do not need the attention mechanism.\n  \nMinor comments:\nIn the appendix, injecting noise in a dissipative dynamical system is not equivalent as adding 10% noise to the time series. The noise gets integrated through any SDE solver. It leads to compression of the noise in the highly dissipative regimes and gets amplified nearby the unstable saddles. So, it does work in a similar way as the attention mechanism (or works against it), because it can also be used to identify the state-space volumes where the entropy gets amplified. \n\n\nIntro: Replace the rhetorical question: “Why are information from parent ..?” by a direct sentence.\n\nATE is commonly used as average treatment effect in classic causal effect estimations. So, it may lead to confusion when people from other fields read this paper in the context of causality.\n\n\n\n", "summary_of_the_review": "Novel contribution well supported by plenty of experiments and extensive appendix.", "strength_and_weaknesses": "My recommendation to the authors is to present the limitations of this type of methodologies right after the contributions in the introduction. So, a new reader can quickly understand where and how this methodology can be applied. One of the limitations is the degradation of performance as the degree increases. It’s certainly problem dependent but it can be useful to know in what problem one can potentially apply this methodology. Another limitation is for dynamical systems with strong perturbations when, for example, synchronization phenomena dominate the system dynamics. The authors also show in the appendix that the method is quite sensitive to noise, some something should be written in this regard. It also appears to me that there’s quite a lot of effort setting up the right balance between the learning rates and the attention variable such that the cross-entropy loss leads to good generalization. Having guardrails on how to set the balance between the a-values and the cross-entropy loss would facilitate extending this method to other contexts. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "OqyE5TB5m0", "reviewer": "Reviewer_pNYJ", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a causal attention model to identify the causal effects in the time series data. This model is trained by maximizing the Attention-extended Transfer Entropy. Beyond the causal attention model, a binary classification module is introduced to mitigate the negative effect of noises. The experiments on both synthetic and real datasets show good performance.\n", "summary_of_the_review": "Overall, after balancing the positive and negative points, I think this paper needs to be further polished since many details need to be explained.", "strength_and_weaknesses": "Strength:\n1) Identifying causal relations of time-series data plays an important role in AI applications, such as science and human-robot interaction.\n2) Compared with the original methods using transfer entropy to detect causal-effect relations, this paper learns a reweighted transfer entropy to refine the construction.\n3)  This method shows great performance in the experiments. \nWeaknesses:\n1) \"Causal attention\" is not accurate. This paper applies a reweighted transfer entropy to promote the construction but does not learn an attention model for causal structure prediction. It is suggested to modify the title and claims in the paper. For example, detect causal effects in time-series data with reweighted/attentive transfer entropy.\n2) What is the advantage of reweighting? The insight into using reweight is not clear. Why add a few parameters to reweight can improve the performance dreamily? \n3) Is the model identifiable? If yes, it is better to give corresponding proof and assumptions.\n4) Could the author provide a detailed ablation study? It seems to be important to identify which component is the key to performance improvement.\n5) Could this method be used for i.i.d data? If not, the boundary of the method should be clarified. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "JRiPfPEdvQF", "reviewer": "Reviewer_F8Hs", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies a cyclic causal model where values of every variable $X_i$ at time step t are decided by a differential equation $\\frac{d x_i}{d x}  = g(x_i) + \\sum_{j} B_{ij}f(x_i, x_j)$. The learner's goal is to recover the structural functions $f$ and $g$ and the coefficients $B_{ij}$ from the observational data. The authors propose a new loss function for this learning task, called attention-extended transfer entropy, which encourages certain conditional dependence in the model. Finally, simulations were performed on both synthetic and dynamical models to validate the proposed approach. ", "summary_of_the_review": "This paper studies a cyclic causal model where values of every variable $X_i$ at time step t are decided by a differential equation $\\frac{d x_i}{d x}  = g(x_i) + \\sum_{j} B_{ij}f(x_i, x_j)$. The learner's goal is to recover the structural functions $f$ and $g$ and the coefficients $B_{ij}$ from the observational data. The authors propose a new loss function for this learning task, called attention-extended transfer entropy, which encourages certain conditional dependence in the model. However, this paper provides little theoretical guarantee for the proposed method. Due to the lack of clarity, it is unclear how this paper contributes to the existing literature and how novel the proposed method is. ", "strength_and_weaknesses": "This paper may have some interesting ideas. The definition of attention-extended transfer entropy (ATE) seems intriguing. However, due to the lack of clarity, it is unclear how this paper contributes to the existing literature and how novel the proposed method is. I will elaborate on the following.\n\nFirst, the authors claim that \"our task is to infer causal relationships between observed variables based on time series data and reconstruct the causal network connecting large numbers of these variables.\" However, this inference task is never formally defined. What is the performance measure of the learning task? How does one measure the quality of the reconstructed causal networks? Should we measure the L1 / L2 distance of the learned parameters with the actual parameters of the underlying model, or should we measure of the divergence between simulated and observed samples? Unfortunately, I tried to read through the paper but could not find answers.\n\nSecond, the paper describes the proposed algorithm. However, it does not perform any analysis of the algorithm's theoretical guarantee, concentration properties, and sample complexity. It is unclear how the proposed algorithm improves the existing baseline. To support the proposed method, the authors have to resort to empirical evaluation.\n\nAs for the experiments, I appreciate the authors' efforts in including various synthetic and dynamical models. Unfortunately, the clarity issue remains. For instance, the authors state \"compared with the baselines, our method usually substantially improves\nreconstruction performance on both synthetic and real causal networks, as shown in Figure 4.\" Again, it is unclear how the reconstruction performance is measured here. Without such information, it is hard to evaluate and compare the proposed method with the existing baseline.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject