Tangent Space Causal Inference: Leveraging Vector Fields for Causal Discovery in Dynamical Systems
Authors: Kurt Butler
Daniel Waxman
Petar Djuric
Venue: NeurIPS 2024 poster (2024)
Keywords: causal discovery
convergent cross mapping
manifolds
dynamical systems
differential geometry
Primary area: causal_inference
TL;DR: We improve convergent cross mapping by explicitly considering vector fields instead of individual predictions.

Abstract: Causal discovery with time series data remains a challenging yet increasingly important task across many scientific domains. Convergent cross mapping (CCM) and related methods have been proposed to study time series that are generated by dynamical systems, where traditional approaches like Granger causality are unreliable. However, CCM often yields inaccurate results depending upon the quality of the data. We propose the Tangent Space Causal Inference (TSCI) method for detecting causalities in dynamical systems. TSCI works by considering vector fields as explicit representations of the systems' dynamics and checks for the degree of synchronization between the learned vector fields. The TSCI approach is model-agnostic and can be used as a drop-in replacement for CCM and its generalizations. We first present a basic version of the TSCI algorithm, which is shown to be more effective than the basic CCM algorithm with very little additional computation. We additionally present augmented versions of TSCI that leverage the expressive power of latent variable models and deep learning. We validate our theory on standard systems, and we demonstrate improved causal inference performance across a number of benchmark tasks.

Tangent Space Causal Inference: Leveraging Vector Fields for Causal Discovery in Dynamical Systems
Daniel Waxman' Department of Electrical and Computer Engineering Stony Brook University Curt.butles daniel. waxman, petar djuric) @stonybrook.edu
Kurt Butler"
Petar M. Djurié
Abstract
Causal discovery with time series data remains a challenging yet increasingly important task across many scientific domains. Convergent cross mapping (CCM) and related methods have been proposed to study time series that are generated by dynamical systems, where traditional approaches like Granger causality are unreliable. However, CCM often yields inaccurate results depending upon the quality of the data. We propose the Tangent Space Causal Inference (TSCI) method for detecting causalities in dynamical systems. TSCI works by considering vector fields as explicit representations of the systems' dynamics and checks for the degree of synchronization between the learned vector fields. The TSCI approach is model- agnostic and can be used drop-in replacement for CCM and its generalizations. We first present basic version of the TSCI algorithm, which is shown to be more effective than the basic CCM algorithm with very little additional computation. We additionally present augmented versions of TSCI that leverage the expressive power of latent variable models and deep learning. We validate our theory on standard systems, and we demonstrate improved causal inference performance across a number of benchmark tasks.
Introduction
The discovery of causal relationships is one of the most fundamental goals of scientific work. When causal relationships are known and understood, we can explain the behavior of a system and understand how our actions or interventions upon the system will affect its behavior of reasoning is fundamental in many problem domains, such as medicine, environmental policy, and economics. However, is not always possible to perform interventions and observe their effects. For example, medical practitioners might only have one chance to prescribe a medication to help a patient. As another example, an ecologist might find that performing large experiment is forbiddingly expensive or otherwise infeasible. Due to these concerns, there remains great interest in the problem of observational causal inference, where one infers causation without manipulating the system under study directly.
This kind
For time series data, the most prominent tool for observational causal inference is Granger causality [1).. Granger causality operates on the assumption that if one system (the cause) is driving changes another system (the effect), then the cause should have unique information about what will happen to the effect. This is encoded in an assumption called separability, which posits that the information needed to predict the future behavior of the effect not contained in the effect itself. However, the separability condition runs counter to the behavior of coupled dynamical systems: if long histories of the effect can forecast the cause, then an autoregressive model with the appropriate lag may forecast the effect from itself. This effect is explored in the supplementary materials of 112 and we give
Equaa contribution.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

an example in Appendix B Since systems with (approximately) dynamical behavior are ubiquitous in many application domains, alternative methods have become of interest.
To address the failures of Granger causality in coupled dynamical systems Sugihara et al. 41 proposed the convergent eross mapping (CCM) method, which directly takes advantage of the topological properties of dynamical systems. CCM can be seen as an adaption of earlier work that into an algorithmic procedure for detecting causalities between time series. Using Takens' theorem, a well-known result in dynamical systems theory [42), CCM attempts to detect causality by reconstructing the state space of given invariat function that maps between reconstructed state spaces, called a cross map. The key assumption is that the cross map only exists if the original systems were dynamically coupled. In this sense, the "causality" of CCM deviates from the popular Pearlian framework [27]. and is better interpreted as identifying which variables "drive" or "force" a dynamical system based on observational data Building upon this framework, several extensions of CCM have been proposed to address various technicalities and caveats of the approach and to generalize the method to new domains. However, the CCM test statistic is difficult to interpret and does not admit simple decision rule. CCM also does not explicitly learn a cross map function.
studied the synchronization of dynamical systems 2 29 35
time series, and then learning
To improve upon CCM, we propose Tangent Space Causal Inference (TSCI). TSCI detects causation between dynamical systems by explicitly checking if the dynamics of one system map on to the other. The proposed method can be seen as a drop-in replacement for the CCM test, providing an interpretable and principled alternative while remaining compatible with many of the extensions of CCM. Furthermore, the proposed method is model agnostic, meaning that it can be adapted to any method used to learn the cross map function, including multilayer perception (MLP) networks, splines, or Gaussian process regression. The only major assumption of TSCI is that the time series under study were generated by continuous time dynamical systems (i.e, by systems of differential equations). which is standard assumption in number of physical systems [36]. As result, TSCI is applicable to many of the same problems as CCM.
1.1 Related Work
Causal representation learning. The primary function of Takens' theorem in the CCM method is to yield representation of a system's 's latent state so that it may be used for cross mapping. As result,
numerous generalizations of Takens' theorem emerged in the following decades B32 33 40) Causal representation learning aims to learn hidden variables from high dimensional observations M. and principle task is to decipher the causal relationships between many, possibly redundant, observations of system [30]. In such cases, methods of dimensionality reduction can be applied to extract causal variables from the raw observations [I] This is particularly important in the processing of large, spatio-temporal data sets [36] 43. Some authors have proposed that CCM can be improved by aggregating data from multiple sources into the reconstruction of the latent states 7 which requires an awareness of how the observation data relate to the causal variables of interest.
Causal discovery with cross maps. CCM has popularized the use of cross maps as a tool for causal inference, and many variations and improvements of the basic CCM methodology have been proposed. Some of these works aim to improve the reconstruction of latent states in these models by using a Bayesian approach to latent state inference [13. where the approach is adapted for spatial/geographi data [14] or modified for sporadically sampled time series (1). Additionally, several improvements have suggested changing the way that cross map is detected: some authors have recommended varying the library length (25 or time delaying the cross mapping 44] to yield refined information. The k-nearest neighbor regression, which is used in the original CCM algorithm, can be swapped out for a radial basis function network [23), or for a Gaussian process regression model [33. Other approaches have not directly learned the cross map all; instead they have examined other aspects of the reconstructions such as their dimensionality [3] or used pairwise distance rankings as a signature of the mapping [5).
2 AAthhough not typically interpreted as such, cross maps have been suggested to be interpretable in Pearlian causal framework via the do operator [[2].
2

---

2 Proposed Method
We begin by considering the inference of causal relationship between two time series, xxt) and y(t). Our starting assumption is that both time series were generated by latent dynamical systems with states 2g (t) and 2, (t), respectively, whose behavior is governed by set of ordinary differential equations (ODEs). For motivation, we consider a particular case with a unidirectional coupling between the latent states:
<tt) h (2x (t),,
(I)
(t) = hy (t)x, dzg f. (Zz) dt
(2)
(3)
dzy
f (Zzz kyy,
dt
where the observation functions hy; hy and the time derivative functions fe, fy are assumed to be differentiable. A causal relationship between x and is evidenced by the appearance of zg in the equation for dzy, dl. The state vectors 2 and Zy could possibly have redundant information, but by writing the system in this form, we obtain an asymmetry between sn(() and y(() which will inform our inference of causation. In our notation, z appears the equation for dzz/ we will write - We now explain how the CCM method approaches this problem and how the TSCI method builds upon the CCM framework.
2.1 Convergent Cross Mapping
CCM is technique for inferring causation between time series generated by dynamical systems
, as seen Eqs. (I) to (4) The basic motivation for CCM that given an observed time series (f). one can construct a vector x which acts as a proxy for the latent states z, that generated it. Given two constructions, and y, we may detect if there is mapping between them, which provides evidence of causal relationship. From Eq. (4) since 2, influences 2 unidirectionally, the effect yt) contains more information than the cause time series stt), and result ý generally contains enough information to reconstruct X. With this in mind, we can frame CCM two-step procedure 13.. In Step I, we construct representations x and y that are proxies for 2₂ and Zy, respectively. In Step 2, we detect a mapping between reconstructions; if there is a mapping + X, then the reverse causality holds, x + y/-
To reconstruct the latent state space, as in Step 1, multiple approaches could be employed. However, in the basic CCM methodology, one uses the so-called delay embedding of xtt).
xx(1)
xrl1 -T)
x(t)
(5)
xx(t (Q 1)7)
where and Q are parameters called the embedding lag and embedding dimension, respectively, The justification that x is good proxy for Z, is given by Takens' theorem 42) which states that x and 2g are equivalent up nonlinear change of coordinates.
Theorem 2.1 (Takens' theorem 33
Let M be a compact manifold of dimension d. Let z E M evolve according to dz/dl f(z), let oor be the mapping that takes to Ze and let x(() h(((()). If Q > 2d 1, then for almost-everj triplet (f, h, T), the map De,h.
l((z)
(x))
L (2),
(6)
Lh (Q- (z))
'Several version of Takens' theorem exist, and they make useful, but mathematically distinct, statements about how common such embeddings are. The version presented here says that functions that do not produce embeddings live measure set, in the sense of prevalence [33 p. 584].
3

---

is an embedding of M into R
Since Dr,,, (t)) x(t., Takens' theorem tells us that if Zg lives manifold M, then the points x lie on a manifold Ma. which is called the shadow manifold of x [41]. Despite the number of assumptions in the statement of Takens' theorem, many generalizations of the statement exist, allowing us to justify the use of delay embedding to systems with strange attractors [33] and systems
with noise [40]. If system satisfies the conditions for Takens' theorem, in the sense that M2 is valid embedding of M, then we say that the system is generic.
Since only 2, influences sr). Takens' theorem implies that x is equivalent (up to nonlinear coordinate change) to Zz. However, due to the appearance of zg 44) both za and Zy are responsible for generating y(()) As a result, Takens' theorem suggests that ý is equivalent to (Zx;2y) as concatenated vector [39]. Since (Z.; Zy) clearly can be mapped onto Zg, the equivalence between the delay embeddings and the latent states suggests that there is mapping + X, called cross map. Thus, cross maps encode the idea that the effect time series contains information about its cause. This is encoded in the following corollary to Takens' theorem.
Corollary 2.1.1 Suppose that x + y for a generic system. Then there exists a function F such that x(I) = (V) for all I.
The proof of the corollary is provided in the Appendix.
Step 2 of CCM is then to detect if -> y by checking if cross map ý +> X exists. To this end, Sugihara et. al. [41] propose to check the predictability of the time series xx() given (t)). They use
form of k-nearest neighbors regression to produce an estimate i(t) of xx() given (t), and then they define a test statistic
CCM X = ((())
(7)
where corr is the Pearson correlation coefficient. To test the reverse causal direction, x y, one simply performs Step 2 again with the roles of x and y reversed.
2.2 Tangent Space Causal Inference
Takens' theorem and cross maps as a tool for causal inference are rooted in a solid mathematical foundation, but the CCM test does not exploit all of the properties of shadow manifolds. Namely, it does not exploit the fact that the shadow manifolds are copies of the latent manifolds. In practice, there are cases in which CCM learns cross map that appears to be reasonably predictive, but results in a false positive. Thus, a more robust algorithm would exploit more subtle properties of the hypothesized cross map. We propose TSCI as alternative to Step 2 in the CCM algorithm.
TSCI operates by checking if the ODE on one manifold, M,, can be mapped to an ODE on another manifold My- To understand how this works, we need to reframe our discussion of ODEs in terms of vector fields. Recall that given an ODE of the form,
dx
dt = n(x),
(8)
may interpret u to be velocity vector field the manifold. When evaluated. u(x) is tangent vector of the manifold My. existing in the tangent space T&M From calculus, we know that tangent vectors in M2 can be mapped to tangent vectors in T My by the Jacobian matrix J (x) at the point x By checking if the tangent vectors can be mapped in such a way, TSCI provides an alternative to the CCM causality test. A visual motivation for the TSCI methods is depicted in Fig. [
Before we can map vector fields from one manifold another, we need to check that meaningful vector fields exist on the shadow manifolds. As usual, this is also a corollary of Takens' theorem.
Corollary 2.1.2 There exists vector field on Mr such that the embedding De,,, in Takens' theorem is a time-invariant mapping. we define 70) = 4 &r., (2,, and if we let 7(1) to be the flow of a point 7(0) under the vector field u, then & (z). 7(), for all times L.
'See Appendix Afror definitions of differential geometric quantities such as the tangent space.

---

My
F(8)
X
F
M
FF(
Figure I: Visual motivation for the TSCI method. Observed time series are related to latent states evolving on some manifolds. Given observed time series xxt) and y(t), we reconstruct latent states x(t) and y(t) that reside in manifolds M2 and My, respectively. z and are causally coupled, there should exist function F that maps between these manifolds. Using the Jacobian matrix J Jr. we can map the velocity vector field on MM to a vector field on My. and use the angle between velocity vectors measure of their similarity.
The proof of the corollary is provided in the Appendix.
The corollary guarantees that we can learn ODEs that describe the dynamies of and y, once we have obtained valid reconstructions of the latent states. Let u and v be vector fields such that
dx dt
u(x),
(9)
dý
v(ÿ).
(10)
di
Because the cross map F is mapping between the two shadow manifolds, the Jacobian matrix of the cross map relates the vectors u and v. We state this formally in the following Lemma.
Lemma 2.1.1 Let Mr and My be manifolds with respective vector fields and that define their dynamics. If there exists a cross map F M,++ My then for every point x e My, we have that (F)())) (X)xxx where J (x) is the Jacobian matrix of F at X.
The proof of the lemma is in the Appendix.
While the cross map F Mr MM is mapping between points on the manifolds, the Jacobian matrix J (x) induces a mapping between the tangent spaces at & and y. We show this visually in Fig. [ Since the velocity of & can be represented by tangent vector uxx), the Jacobian matrix J allows us to map these vectors to tangent vectors My. there a cross map, then vector field and the push forward vector field J pu should match exactly. On the other hand, there should be no correlation in the absence
causal relationship, assuming quality reconstructions and plentiful data. The degree of alignment between and J F can be used test statistic for the presence of causal link. We therefore propose the TSCI test statistic,
TSCI X-
Ju,
(11)
Because the tangent vectors are centered in their respective tangent planes, we can geometrically interpret the TSCI test statistie the expected cosine similarity between the vector field u and the push forward vector field J FV,
u(F())) T J [u(F()))I [y)(y)
Bg-M, TSCL X-
(12)
In practice, the estimation quality of the cross map also depends on the location in the reconstruction space, and so the cosine similarity takes on a distribution of values (Fig. 2 and weak-to-moderate correlation may be empirically observed artifact of limited data.
5

---

da (Q-1)r) Algorithm 1 TANGENT SPACE CAUSAL INFERENCE
Input: Embeeding//vettor field matrices X, U Rxxx= and Y, V E RTXQ, Output: Score r
Learn a differentiable function F' such that X; = F(),) for all
2: for port=: 0,
I'- ,, do
3: Compute the Jacobian matrix (Yx,) E xQ of the function F location Y,
4: Push forward the tangent vector by computing Ü, VY,)
5: end for
6: Define TX-Y = C).
Algorithm 2 TANGENT SPACE CAUSAL INFERENCE WITH K-NEAREST NEIGHBORS
Input: Embedding//vector field matrices X, E TTQQ. and Y, V.Ve EEERTOO TxQ,, regression parameter
K > max(Qu., Qy)
Output: Score 7X-+Y
1: to = 0, 2, ..... 1 T
2:
Find the indices .I...., TK of the Kenearest neighbors of Y,
3:
Compute the local displacements in r-space: AX,- Xx Xt
4: Compute the local displacements in y-space: AYa Y Th Y,
5: Compute the least-squares solution to AYgJ. yJ AX;
6: Compute U, V,J
7: end for
8: Define 7X-YY = conr( (Ü, U)
Based on these analyses, we propose the TSCI method in Algorithm 1 which learns a cross-map F and returns the alignment of the vector fields and J.. CCM is sensitive to the quality of the reconstruction of the latent states because an improperly constructed shadow manifold can sabotage the method, both theoretically and empirically [6]: TSCI similarly requires that the shadow manifold be properly embedded. Because of this, we assume that the embedding vectors x and ý and the estimates of their vector fields u(x) and v(ý) each point are supplied as inputs the algorithm. variety of methods could be used to estimate the vector fields, but the simplest approach is to use finite differences,
dz(()
x(t + 1) - x(t))
dx
dr (t-rr) dt
x(t - 1 1- 1- xx(t -))
n(x)
dt
At
La(++1 -Q- 1)r) -(( (Q 1)7)
dt
where At is the sampling rate of the data. Finite differences are known to be sensitive to noise, in real scenarios a more careful approach to obtaining the time derivatives is necessary. For a practical implementation, we propose using the central fiffe-differences, which second-order accurate 16. or the derivatives interpolated by a Savitsky- Golay filter for noisy data (34].
On methods to learn the cross map function. The TSCI algorithm is notably agnostic to the regression approach used for the cross map F, with reasonable approaches including multilayer perceptron networks (MLPs), splines, and Gaussian process regression (GPR). Depending on the approach, derivatives can then be estimated from the model, either using automatic differenniation or analytical derivatives. For GPR in particular, analytical derivatives are straightforward to compute
38
Since regression can sometimes be computationally complex procedure, in Algorithm 2 we also provide a version of TSCI that is based on the k-nearest regression, analogy to CCM. For clean and dense data, this simple approach can yield accurate results, but it is generally unsuitable for noisy or sparse data. In the latter case, is preferrable to combine the TSCI approach with other methods of denoising time series, or learning of latent dynamical models of the observed time series. In particular, the learning of a latent ODE learning can be combined with the TSCI test to yield accurate causal inference.

---

Algamment of u with Jpv (rx-r) Mem=-097
M, Emnbedding Q
Mr
Embedding din
4030
-0.5
0
0.5
AE Agggeent v with Jyu (ry-x)
Moon 0.03
600
400
-0.6
0.5
Figure 2: Shadow manifolds Mr and My from the unidirectionally coupled Rössler-Loree system Eq. 13 with C' 1, and the corresponding histograms of ú u and & v. The test statistics, Tx-+Y and rx+y, correspond to the means of these distributions.
On the use of correlation coefficient. In relation to CCM, can be noted that both TSCI and CCM use correlation coefficient as their test statistic. However, critical difference between the two methods is how the usage of this statistic is justified. The correlation coefficient used in CCM analysis, called the tross-map skill [41], is used to measure the accuracy of predictions of the cross map. However, since the points being predicted in CCM live on a manifold, measuring correlation in the ambient (extrinsic) space is not well-motivated Furthermore, estimates of this correlation biased by the distribution of observations along the manifold, and as a result, a high correlation can be achieved with relatively low-accuracy prediction by guessing the general region in which the points lie.
be
In contrast, the correlation between tangent vectors (or vector fields) is more geometrically motivated quantity: the correlation a linear measure of similarity and the tangent vectors belong to (centered) linear space. Additionally, the shadow manifolds constructed to be submanifolds of Euclidean space, so correlations computed in extrinsic coordinates will match correlations computed using the intrinsic coordinates of the shadow manifold. By Lemma 2.1.1 the correlations will be identically 1 a cross map exists.
One alternative to the cosine similarity is the mutual information (MI). Specialized to the current text, the MI 7 (u; vv) quantifies the reduction in the uncertainty of given the pushforward vector JFY (9] While the information-theorede underpinning of the MI is attractive, we have two main reasons to prefer the cosine similarity: First, the MI between two (continuous) distributions can be difficult to interpret. For example, it is not obvious if [(u; J Fv) 0.5 is strong dependence or not, particularly in the case of continuous distributions. The cosine similarity, on the other hand, is upper bounded by 1, so corr( Ü, U) = 0.95 is easily interpreted as having near-perfect reconstruction. Second, accurate estimation of the MI from samples notoriously difficult task, particularly in high dimensions, and no single estimator works consistently well [10]. We provide some experiments with the MI, and show the differing performance of different estimators, in Appendix B.3
Experiments
We validate the performance of TSCI on two datasets that are popular in the literature. The first arises from a coupled Rössler-Lorenz system, where the ground truth causality is known and the causal influence can be smoothly modulated. The second example was proposed in [12] and uses sporadic time series from coupled double pendulums, and illustrates the applicability TSCI to extensions of CCM. Code implementing TSCI is available at https://githu: wtw//ootttti /urtButlerr/r/// tangentspaces All comparisons to CCM use the skcem module in Pythor Experiments were 6-Core Intel Core i5 and NVIDIA Titan RTX.
htttp:///gtthab..oomniecc////// MIT License
7

---

1.0
Correlation Coefficient
X-y (TSCI) rrsk (TSCI)
Correlation Coefficient
0.5
0.5
x-v (TSCI)
0.0
0.0
Fy-x (TSCI)
0
1
2
0
2000
4000
6000
1.0
OK-y
rrak (CCM)
(CCM)
1.0
X-r (CCM)
0.5
0.5
Y-x (CCM)
0.0
0.0
1
2000
4000
6000
C
Library Length
(a)
(b)
Figure 3: Comparison of TSCI with CCM for the Rössler-Lorenz system (true causation X + Y). The plotted lines show the median test statistic over 100 trials for both CCM and TSCI, and the shaded region indicates the 5th and 95th percentiles when (a) C' is varied from (no coupling) to (approximate general synchrony), and (b) C' is fixed to 1.0 and the library length is varied.
3.1 Unidireciionaly-CCuppee Rössler-Lorenz System
common toy system used for studying coupled dynamic systems unidirectionally-coupeee Rössler-Lorenz system (28), which we define in Eq. (13). In this system, the first three coordinates (21, 22, 23) describe a Rössler system, and they unidirectionally coupled with Lorenz-type system (Z4, 25, #6). The strength of the coupling is controlled by the parameter C. When C = 0, the two systems are disconnected, but for C there is causal influence from 21, 22, 23 21, 25. 26.
-6(22, 23,1)
21
6(21.) + 0.222,t
22
dz
d
23
6 (0.2 23, 5.7))
(13)
dii
dt
24
10( - 21,t)
26
28z4,t
26.t 20, C..z,1
[6
25, 826,1/3
Fig. we visualize the TSCI method x 22 and y= 24. First, we show the shadow manifolds Mz and M, with a set of tangent vectors on each manifold. The delay embedding dimension parameters, Qx and Qy. were selected using the felse-nearest neighbors algorithm with a tolerance of 0.005 [19] Time lags for the embeddings, TN and Ty. were selected by picking the minimal delay such that the autocorrelation function drops below a threshold [2I]. Additionally, we show the histograms of (0), where 0 is the angle between the tangent vectors at each point. For X + Y direction, which is the true causal direction, the distribution is concentrated near 1. For the < X direction, the distribution of tangent vectors is centered on O. The test statistics, rx- and - correspond to the means of each distribution, and visibly correspond to the correct causality.
We show the effects of varying the coupling strength C' in Fig. 3a where TSCI clearly shows better separation across varying C. We see the effect increasing the library length (i.e., the size of training set in nearest neighbors) for C' 1.0 in Fig, 3bb Here, rx-+y increases at similar rates for TSCI and CCM, suggesting similar data efficiency the two methods.
3.2 Double Pendulum System
To illustrate the generality of TSCI within CCM-like frameworks, we applied the TSCI methodology to the latent CCM framework, where CCM is applied to state-space reconstruction obtained via neural ODEs [12]. One reason to favor this approach when the observed time series are irregularly (i.e., with non-uniform sampling rate) sporadically (i.e., any given observation only measures subset of states) sampled. Notable for TSCI the fact that ground-truth derivatives are available, as they are directly learned in the neural ODE reconstruction.

---

Table I: Double Pendulum Experiments. Entries denote the mean + one standard deviation across
folds. Bolded directions indicate ground truth causality.
7X
Direction
Latent CCM
Latent CCM (MLP)
Latent TSCI (MLP)
X + Y
0.011 + 0.009
0.015 + 0.009
0.021 # 0.010
X + Z
0.044 + 0.008
0.055 0.011
0.077 # 0.013
Y + X
-0.003 = 0.008
-0,005 = 0.009
-0.010 0.004
Y + Z
0.003 = 0.006
-0.002 = 0.004
-0.008 = 0.013
Z + X
0.737 = 0.019
0.747 = 0.015
0.915 = 0.006
Z Y
0.475 = 0.043
0.578 0.030
0.612 = 0.053
We replicate an experiment the latent CCM paper where network of unidirectionally coupled double pendulums simulated, and observations sampled irregularly and sporadically. The authors' source code 6 was used to generate data and apply latent CCM. For more information on the data generation, Appendix A of 12
For TSCI, we learn a cross-map using an MLP between the reconstructed state spaces. To avoid tuning learning rates for every network, the parameter-free COCOB optimizer [26 and networks were trained for 50 epochs. Results can found Table
When compared to latent CCM, TSCI provided larger correlation coefficients for the true positive cases, with similarly small coefficients for the true negative cases. The use of MLPs for learning the cross map partially but not fully explains the difference in performance.
3.3 Additional Experiments
Several additional experiments appear in Appendix B We briefly describe them here and provide some commentary on their results.
Varying the embedding dimension. In Appendix B. we artificially lower quality embeddings by varying the embedding dimension. We find that the performance of TSCI and CCM similarly degrade when using embedding dimension that is very small, and that there is little harm to larger embedding dimension. In either case, the hyperparameters from a false-nearest neighbors test perform well.
Corrupting signals. In Appendix B.2, we lower the embedding quality in different ways: (1) by injecting additive noise to all observations, and (2) by adding sine wave to observations. We observe that TSCI and CCM similarly degrade when the signal-to-noise ratio altered. However, because of the larger separation in TSCI, the correct causal relation can be determined at much lower signal-to-noiss ratios.
Using mutual information. In Appendix B.3 of cosine similarity. We find that conclusions made using MI are generally similar to those from cosine similarity, but with less interpretability and significant difficulties in estimation.
we experiment with using mutual information instead
Comparisons to other causal discovery methods. As mentioned in the introduction, CCM (and hence TSCI) are specifically designed to address failures of Granger causality. In Appendix [.4] empirically verify the limitations of Granger causality on our Rössler-Lorenz toy system. We additionally test several other causal discovery methods and show their limitations in our setting.
Advantages and Limitations of TSCI
Scalability. The TSCI approach, using the K K-nearest neighbors algorithm, retains the scalability and lightweight implementation that CCM enjoys. The only additional computational complexity arises
httpp://ggtabbooo////ccrrrrrreee MIT License
http:://ggttttooooo///////tt/ttn MIT License

---

from solving a local linear system of equations, and from the estimation of derivatives. Since the linear systems have fixed size K, and since derivative estimation can be done using linear filter, the additional cost is minimal. The general version of TSCI will scale depending on the choice of the regressor used learn the eross map, and on other design decisions made to improve the efficacy of the method. However, this is not unlike the situation in which CCM is augmented with other customization options that potentially slow down the method.
Model agnosticism. Because a TSCI test can be formulated for any differentiable regression model used to learn the cross map, the approach is highly flexible and model agnostic. Additionally, because there are many potential ways in which reconstruction of latent states and learning of the velocity vector fields can be improved, the TSCI method can be incorporated into a wide variety of inference frameworks,
Quality of reconstructed states. In both CCM and TSCI, there few assumptions which warrant some justification before use, since their violation may yield to misapplication of cross map methods. The first issue is that while Takens' theorem implies that embeddings are plentiful, all embeddings are equal in quality or useful. Shadow manifolds that are sparsely or incompletely sampled, time series with trends, or otherwise data which do not accurately capture the latent manifold can lead to dubious results using CCM [6]. While TSCI is less likely to produce a false positive in these cases, the assumption that a latent manifold is well-represented by given embedding is nontrivial and critical to ensuring trustworthy performance of the method.
General synchrony. General synchrony is problem that plagues all cross map-hased methods 41 The issue is that when the causal strength of the relationship x + y is very strong, the influence of 2₂ dominates the dynamics of Zyy and Zy cannot exhibit its own independent behavior. As result, My will look similar to Ma, and the cross map will become an invertible function. As result, strong unidirectional relationship is detected as a bidirectional causal relationship. In Fig. 3 the Rössler-Lorenz system enters general synchrony near C = 3 [6]. The TSCI method appears more resistant the effects of general synchrony than CCM, but it is topological fact that C' grows, synchrony becomes inevitable.
5 Conclusion
In this paper, we presented the TSCI method for detecting causation in dynamical systems. By considering how tangent vectors map from one manifold to another, may achieve more robust detection of causal relationships in dynamical systems than with standard CCM. Key advantages of TSCI include that we may use it in many systems in which CCM would be applied, but the method is far less prone to false positives and spurious causation. We presented both a general form of the algorithm as well as k-nearest neighbor version inspired by the original CCM algorithm. Because TSCI requires us to estimate latent states and their time derivatives, there us much room for the TSCI method to be further developed in future work.
Acknowledgments and Disclosure of Funding
We would like thank the anonymous reviewers for their suggestions which improved the content and presentation of this work. This work was supported by the National Science Foundation under Award 2212506.
References
[1] Kartik Ahuja, Divyat Mahajan, Yixin Wang, and Yoshua Bengio. Interventional causal repre- sentation learning. In International Conference on Machine Learning, pages 372-407. PMLR, 2023.
[2] Jochen Arnhold, Peter Grassberger, Klaus Lehnertz, and Christian Erich Elger. A robust method for detecting interdependences: application to intracranially recorded EEG. Physica D: Nonlinear Phenomena, 134(4):419-4) 1999.
10

---

[3] Zsigmond Benkō, Adám Zlatniczki, Marcell Stippinger. Dániel Fabó, András Sólyom, Loránd Eröss, András Telcs, and Zoltän Somogyvári. Bayesian inference of causal relations between dynamical systems. Chaos, Solitons & Fructals, 185: 15142, 2024.
[4] Patrick Blöbaum, Dominik Janzing, Takashi Washio, Shohei Shimizu, and Bernhard Schölkopf. Cause-effect inference by comparing regression errors. In International Conference on Artificial Intelligence and Statistics, pages 900-909. PMLR, 2018.
[5] Leo Breston, Eric J Leonardis, Laleh K Quinn, Michael Tolston, Janet Wiles, and An- drea A Chiba. Convergent cross sorting for estimating dynamic coupling. Scientific Reports, 1 1111:20377, 2021.
[6] Kurt Butler, Guanchao Feng. and Petar M Djurić. On causal discovery with convergent cross mapping. IEEE Transactions on Signal Processing, 2023.
[7] Adam Thomas Clark, Hao Ye, Forest Isbell, Ethan R Deyle, Jane Cowles, G David Tilman, and George Sugihara. Spatial convergent cross mapping detect causal relationships from short time series. Ecology, 96(5):117--118,, 2015.
[8] Sarah Cobey and Edward Baskerville. Limits causal inference with state-space reconstruc- tion for infectious disease. PloS one, 11(12)=0019050, 2016.
[9] Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley-Interscience, USA, 2006.
[10] Pawet Czyž, Frederic Grabowski, Julia E Vogt, Niko Beerenwinkel, and Alexander Marx. Be- yond normal: On the evaluation of mutual information estimators. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[11] Povilas Daniušis, Dominik Janzing, Joris Mooij, Jakob Zscheischler, Bastian Steudel, Kun Zhang, and Bernhard Schölkopf. Inferring deterministic causal relations. In Proceedings of the Twenty-Sixth Conference on Uncertainty Artificial Intelligence, pages 143-150, 2010.
[12] Edward De Brouwer, Adam Arany, Jaak Simm, and Yves Moreau. Latent convergent cross mapping. In International Conference on Learning Representations, 2020.
[13] Guanchao Feng, Kezi Yu, Yunlong Wang, Yilian Yuan, and Petar M Djurić. Improving conver- gent cross mapping for causal discovery with Gaussian processes. In 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3692-3696. IEEE, 2020.
[14] Bingbo Gao, Jianyu Yang. Ziyue Chen, George Sugihara, Manchun Li, Alfred Stein, Mei- Po Kwan, and Jinfeng Wang. Causal inference from cross-sectional earth system data with geographical convergent cross mapping. Nature Communications, 14(1):5875, 2023.
[15] Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: Journal of the Econometric Society, pages 424-438, 1969.
[16] Michael T Heath. Scientific Computing: An Introductory Survey. SIAM, 2018
[17] Patrik Hoyer, Dominik Janzing, Joris M Mooij, Jonas Peters, and Bernhard Schölkopf. Nonlinear causal discovery with additive noise models. Advances Neural Information Processing Systems, 21, 2008
[18] Diviyan Kalainathan, Olivier Goudet, and Ritik Dutta. Causal discovery toolbox: Uncovering causal relationships in Python. Journal of Machine Learning Research, 21(37):1-5, 2020.
[19] Matthew B Kennel, Reggie Brown, and Henry DI Abarbanel. Determining embedding di- mension for phase-space reconstruction using a geometrical construction. Physical Review A, 45(6):3403, 1992.
[20] Alexander Kraskov, Harald Stöghauer, and Peter Grassherger. Estimating mutual information. Physical Review E, 6966)366, 2004.

---

[21] Dimitris Kugiumtzis. State space reconstruction parameters in the analysis of chaotic time series the role of the time window length. Physica D: Nonlinear Phenomena, 95(1):13-28. 1996.
[22] John M Lee. Introduction to Smooth Manifolds, volume 218 of Graduate Texts in Mathematics. Springer, second edition, 2012.
[23] Huanfei Ma, Kazuyuki Aihara, and Luonan Chen. Detecting causality from nonlinear dynamics with short-term time series. Scientific Reports, 4(1):7464, 2014.
[24] A. McInerney. First Steps in Differential Geometry: Riemannian, Contact, Symplectic. Under- graduate Texts in Mathematics. Springer New York, 2013.
[25] Dan Mønster, Riccardo Fusaroli, Kristian Tylén, Andreas Roepstorff, and Jacob Sherson. Causal inference from noisy series data- data-testing the convergent cross mops-ng algorithm the presence of noise and external influence. Future Generation Computer Systems, 73:52-62, 2017.
[26] Francesco Orabona and Tatiana Tommasi. Training deep networks without learning rates through coin betting. Advances in Neural Information Processing Systems, 30, 2017.
[27] Judea Pearl. Causality. Cambridge University Press, 2009.
[28] Quian Quiroga, Jochen Arnhold, and Peter Grassberger. Learning driver-response relation- ships from synchronization patterns. Physical Review E, 61(5):142, 2000.
[29] Nikolai F Rulkov, Mikhail M Sushchik, Lev S Tsimring, and Henry DI Abarbanel. Generalized synchronization of chaos directionally coupled chaotic systems. Physical Review E, (2)))988
1995.
[30] Jakob Runge, Andreas Gerhardus, Gherardo Varando, Veronika Eyring, and Gustau Camps- Valls. Causal inference for time series. Nature Reviews Earth & Environment, 4(7):487-505, 2023.
[31] Jakob Runge, Vladimir Petoukhov, Jonathan F Donges, Jaroslav Hlinka, Nikola Jajcay, Martin Vejmelka, David Hartman, Norbert Marwan, Milan Paluš, and Jürgen Kurths. Identifying causal gateways and mediators in complex spatio-temporal systems. Nature Communications, 6(1):8502, 2015.
[32] Tim Sauer. Reconstruction of dynamical systems from interspike intervals. Physical Review Letters, 72(24):3811, 1994.
[33] Tim Sauer, James A Yorke, and Martin Casdagli. Embedology. Journal of Statistical Physics, 65:579-616, 1991.
[34] Abraham Savitzky and Marcel JE Golay. Smoothing and differentiation of data by simplified least squares procedures. Analytical Chemistry, 36(8):1627-1639) 1964.
[35] Steven Schiff, Paul So, Taeun Chang, Robert E. Burke, and Tim Sauer. Detecting dynamical interdependence and generalized synchrony through mutual prediction in a neural ensemble. Physical Review E, 54(6):6708, 1996.
[36] Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE, 109(5):612-634, 2021.
[37] Skipper Seabold and Josef Perktold. statsmodels: Econometric and statistical modeling with Python. In 9th Python in Science Conference, 2010.
[38] Erean Solak, Roderick Murray-Smith, WE Leithead, Douglas Leith, and Carl Rasmussen. Derivative observations in Gaussian process models of dynamic systems. Advances in Neural Information Processing Systems, 15, 2002.
[39] Jaroslav Stark. Delay embeddings for forced systems. Deterministic forcing. Journal of Nonlinear Science, 9:255-332, 1999.
12

---

[40] Jaroslav Stark, David S Broomhead, Michael Evan Davies, and Huke. Delay embeddings for forced systems. II. Stochastic foreing. Journal of Nonlinear Science, 13:519-577, 2003.
[41] George Sugihara, Robert May, Hao Ye, Chih-hao Hsich, Ethan Deyle, Michael Fogarty, and Stephan Munch. Detecting causality in complex ecosystems. Science, (386610)))49-5000 2012.
[42] Floris Takens. Detecting strange attractors in turbulence. In David Rand and Lai-Sang Young, editors, Dynamical Systems and Turbulence, Warwick 1980, pages 366-381, Berlin, Heidelberrg 1981. Springer.
[43] Gherardo Varando, Miguel-Angel Femández-Torres, Jordi Muñoz-Marí, and Gustau Camps- Valls. Learning causal representations with Granger PCA. In UAI 2022 Workshop on Causal Reprexentation Learning, 2022.
[44] Hao Ye, Ethan R Deyle, Luis Gilarranz, and George Sugihara. Distinguishing time-delayed causal interactions using convergent cross mapping. Scientific Reports, 5(1):14750, 2015.
[45] Hao Ye and George Sugihara. Information leverage in interconnected ecosystems: Overcoming the curse of dimensionality. Science, 353(6302):222-92. 2016.
[46] Yujia Zheng, Biwei Huang. Wei Chen, Joseph Ramsey, Mingming Gong, Ruichu Cai, Shohei Shimizu, Peter Spirtes, and Kun Zhang. Causal-learn: Causal discovery in Python. Journal of Machine Learning Research, 25(60):1-8, 2024.
13

---

A Important Facts About Manifolds
In this appendix, we review some important background about manifolds. Much more complete references include the authoritative [22] (especially chapters 1, 2, 3, and 8), or the more accessible [24].
We say that a function F is smooth if it has derivatives of all orders. A diffeomorphism is a function F which is smooth, invertible, and F-1 is also smooth. We say that two subsets M and N of R" are diffeomorphic if there exists diffeomorphism such that F(M) = N.
A subset M of 12th is called d-dimensional manifold if for every point in M, there exists an open set U2 such that
1. There is a diffeomorphism Fg IR" + Uz.
2. The image of the HHI {y R" Md+1 Yn 0} under F2 is given by
(H, U2 M.
Without a loss of generality. we can assume that (0) x in this definition. This allows us to easily define tangent vectors. A vector R" is called tangent vector of M at the point there exists a path (t) in H.s such that
dt (F(6)))) = =
where Fr is the diffeomorphism in the above definition. The tangent space T,MM is defined to be the set of all tangent vectors of M at x. Note that from these definitions, clear that IR" is manifold and 72 Rd d
In caleulus, define vector field u R to be function u R RR. We generalize this to manifolds in Euclidean space by defining a vector field u on M to be a mapping u M + R² such that uxx) M every point x.
An embedding of a manifold M C R" is a mapping F R" + R such that F restricted to M is diffcomorphism onto its image. If M is manifold of dimension d and F is an embedding, then m is necessarily greater than or equal to d. In general, it is an interesting question to ask how much larger m must be so that embeddings are easy find. The Whitney embedding theorem says that it a generic property that F IR. >> IRm when m 2 2d + 1. Takens' theorem refines the Whitney theorem by asserting that such embedding can be obtained from scalar observation function, by time lagging the observations sufficiently many times.
B Additional Experiments
In this appendix, we include experiments and results omitted from the main text.
B.1 Varying Embedding Dimension
One of the key parameters in both CCM an TSCI is the embedding dimension parameter Q used during reconstruction the shadow manifold. Takens' theorem suggests that once Q reaches a certain value, the manifold is embedded and the improvement of the results should saturate in principle. Of course, in real life there is curse of dimensionality associated with inference, so it is interesting to check how well the results behave as the embedding dimension is chosen to be higher than necessary. We consider the effect of varying the parameter Q using data from the Missler-Lorens system in Fig. 4
Performance of both methods improve as the dimension of the putative effect (if we are testing then this is Q2) is increased, until saturating after the manifolds are fully embedded.
B.2 Corrupted Signals
Generally, there are many non-trivial ways which the state reconstruction can be poor, e.g., presence of periodic trends (8] and underexploration of the shadow manifold 6). We consider two scenarios that deteriorate the state reconstruction quality on the Rössler-Lorent system: additive noise, and periodic trend.
14

---

TSCT (X - Y)
TSCT (Y -> xx)
2
4
0.5
10
10
-1
10
2
4
8
10
CCM (X + Y)
CCM (Y
X)
2
0"
8
-0.5
00.5
10
Q
9g
Figure 4: Comparison of TSCI and CCM for the unidirectionally coupled Rössler-Lorenz system Eq. 13 with C = Here, xxt)= = 22 (f) and y(t) (t), and the true causality is x y. The red lines indicate the values of Q₂ and Qy selected by felse-nearest neighbors with tolerance of 0.01.
Results for additive noise are presented in Fig. We fixed C' = 1.0 and varied the signal-to to-ii- ratio (SNR) from 0 60 dB by adding Gaussian noise to the observed signals. When estimating derivatives, Lavitzky-Golay filter 34 was used with window length of and order 2.
Overall, find the performance relative SNR be similar between CCM and TSCI, suggesting that inference of the vector field from noisy data is not a limiting factor in the presence of additive noise.
1.0
Correlation Coefficient
0.5
<x-y (TSCI)
0.0
Ty-x (TSCI)
0
20
40
60
1.0
Xxy (CCM)
0.5
ry (CCM)
0.0
20
Signal-to-Noise Ratio [dB]
Figure 5: Comparison of TSCI with CCM for the Rössler-Lorenz system (true causation Y) with additive noise. The plotted lines show the median test statistic over 100 trials for both CCM and TSCI, and the shaded region indicates the 5th and 95th percentiles when (a) C' is fixed to 1.0 and the signal-to-noise ratio is varied.
15

---

An alternative way to corrupt signals is with a deterministic periodic trend; we used sine wave with a period of 2x, and varied its power with respect the true signal. Results are presented in Fig. 6 Our takeaways are relatively similar to the additive noise experiment: TSCI seems to degrade similarly to CCM, but increased separation between cause and effect makes this degradation in performance less problematic. Notably, TSCI seems significantly more robust to false claims of strong causation when the relative power of the confounder large.
1.0
Correlation Coefficient
0.5
X-y (TSCI)
Y-- (TSCI)
0.0
20
-10
0
10
20
1.0
x-y (CCM)
0.5
Y--X (CCM)
0.0
20
-10
0
10
20
Relative Signal Power [dB]
Figure 6: Results of the Rössler-Lorenz system with C = 1 corrupted by additive sine signal of varying amplitude. The amplitude of the signal, relative to the sine wave, plotted in decibels on the horizontal axis.
B.3
Using the Mutual Information for Scores
One interesting alternative to the cosine similarity score is the mutual information (MI), which provides a non-linear quantification of how related u and JEV are. One immediate concern with using MI is that estimates from finite samples difficult; Stögbauer, and Grassberger (20), which has shown relevance even the era deep learning [10].
the classical estimator of Kraskov,
Results on the Rössler-Lorenz test system while varying the coupling parameter C are presented in Fig. 7 We find that MI typically shows less separation than TSCI, which brings into question its viability when the ground truth is unknown. Another issue is in the interpretation of MI estimates: not entirely clear if an MI estimate of, for example, nat indicates strong influence, and estimates are not easily normalized in the continuous setting.
1.0
Tx-Y (CS)
Ty-x (CS)
0.5
TSCI Score
0.0
0
1
2
3
4
r-ux (MI)
2
X--Y MMI)
3
C
Figure 7: Results of the Rössler-Lorenz system using the cosine similarity and mutual information while varying C. Compare to Figure 2a in the main manuscript.
16

---

B.4 Comparisons to Other Causal Discovery Methods
As argued in the main manuscript, cross-map-based methods such TSCI and CCM approach causal discovery in rather particular setting, where more mainstream methods do not apply. We empirically verify these claims on the Rössler-Lorenz. test system, using a variety of values of C. We compare against Granger causality, as well as various bivariate causal discovery methods.
We first compare against Granger causality in Table incorrectly infers bidirectional causality for all C' > 0.
2 As expected from the theory, Granger causality
We also compare against various methods from the bivariate pairwise causal discovery literature. These methods typically assume the existence of a map y = f(x) which either deterministic or subject to additive noise, which does not exist in all but the most trivial dynamical systems. This includes RECI 44, IGCI
[TI. and ANM [77. Results are presented in Tables 3 and
We find that IGCI consistently fails to detect causal edge, with weak coefficients. RECI consistently chooses the direction X - Y, but does so even when C O, which suggests that results are not actually detecting causality, but rather some other anomaly of the data. To this end, the variance ratios as described by Blobaum et al. [4] are typically large, indicating little confidence. Finally, for all C 0, ANM detects bidirectional causality.
Comparisons with Granger causality use the implementation of statsmodels isons to RECI and IGCI used the implementations of cdt [18. with minor bug-fix fix for RECI. Comparisons with ANM used the implementation of causal learn 46
All compar-
Table 2: Granger Causal Results for Rössler-Lorent System. Entries denote the median, 5th, and 95th percentile p-values across 50 trials. All valuue are from the F-test implemented in statstools.
C
p-value X - Y
p-value Y
X
0.0
0.003
0.168
000
000
0.75
0.000 000
0.000 0.000
1.5
0.000
0.000 0.000
2.25
0.000 0.00 000
0.000
0.00000000 000
3.0
0.000
0.000
Table 3: Bivariate causal discovery results for Rössler-Lorent System. For all models, negative score indicates X + and positive score indicates Y - X. All scores are based on the implementations of cdt. Entries denote the median, min, and max over 10 trials.
C
RECI
IGCI
0.0
00.02
0.035
0..22
0.75
0.00250.001
0.0344.048
00,088
0.017
1.5
-0.028
-0.0169.031
0.039
0.007
2.25
-0.029
0.040 073
3.0
0.036
0.017
0.030
C
Proofs
In this section, we provide proofs for results mentioned in the main text.
There are many different proofs and versions of Takens' theorem. This version is proved by Sauer, Yorke and Casdagli 33 and is beyond the scope of this appendix. Other
Proof of Theorem 2.1
noteworthy variations of this theorem are proved by Stark (39 and Stark et al. [40].
httpp////ftthhbboo/ssssttssststs BSD-3-Clause License
MIT License
httpes//githuu..com/yyyuuu
MIT License
17

---

Table ANM causal discovery results for and max p- value over 10 trials. All scores are based on the implementation in causal-lease
System. Entries denote the median, min,
C
X
Y
X
0.0
0.507
777
0.450
0. 096
0003
0.75
0.000
121
0.000 0.00
000
1.5
0.000
0.000
2.25
0.000..000
0.000
0.000
3.0
0.000
0.000 000000000
Proof of Corollary 2.1.1 Let assume that we have generic system, where (Zz; M for some manifold M, and x is a function Z₂ and is a function of Zy.
If + and diffeomorphic to each other. The existence of cross map is then immediate.
y both hold, then M2 and My
both diffeommrphic to M, and thus,
If a > y but not vice-versa then the dynamics of 25 are autonomous, meaning that the behavior of 2y depends only upon itself.
result, the projection map nr (Zz, Zyy) Zr can be applied directly to .M to yield new manifold M2 on which is an autonomous dynamical system. Since observes this system zy. Ma will be diffeomorphic to M since 2 only contains information about 2g. Let
2n M2 + Mr and d M - My denote the diffeomorphisms implied by Takens' theorem. We can construct the cross map then as
Fly) 2 (Tx (r))).
Proof of Corollary 2.1.2
Since the shadow manifold M produced by Takens' theorem is a smooth embedding of the original manifold M, there exists diffcomorphism 4 M - ... By Proposition 8.19 of Lee (22) p. 183], for any given vector field u defined on M, there exists a unique vector field on M that is the push forward of under the map 4. Because the dynamics the manifold derived from these vector fields, the two systems are equivalent as dynamical systems.
Suppose that F M, + My is cross map. Fix any point Ny M2 and let Yo F(xo). we consider flows forwards and backwards with respect to these vector fields, then there exists some small E > 0 such that we have curves along M2 and y, along My such that y: = F(x,) for each in the interval (-e,e). e).. By the chain rule of calculus, we have that
Proof of Lemma 2.1.1
dýo ( di.
dxo
dt
where (xx is the Jacobian matrix of the function F at the point x. Since the dynamics these spaces given by vector fields, replace the time derivatives in the equation with their vector fields:
(ýo) =
Since the point No that we chose was arbitrary, the statement holds for every point in M..
18

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction have been written to clearly state our claims and contributions of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The paper discusses the limitations and assumptions of the approach, and the possible failure modes. section discussing the advantages and limitations of the approach has also been included in the paper.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
19

---

Answer: [Yes]
Justification: We have taken care to provide precise mathematical statements and proofs. For proofs which do not provide (like the proof of Takens' theorem), reference other sources where the proof may be found.
Guidelines:
The answer NA means that the paper does not include theoretical results, All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have worked to make our results reproducible as possible. We have provided full description of the TSCI algorithm so that reader should be able to implement and reproduce method from the written manuscript. The design of the experiments has also be described within reason, so that any factor that might determine their outcome has been detailed. For minor technical details (such as the type of ODE solver used generate data) which are less likely affect the results of the experiment, we refer to the code included with the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to hosted model (e.g., s., in the of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
20

---

(d)
The authors should provide scripts to reproduce all experimental results for the new reproducible, they We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Code to reproduce our results is being included with the publication, with MIT license. The code is reasonably organized and documented, and instructions on how to use the code have been included with the code itself. When making use other code bases for comparison (e.g., with latent CCM), link to the original code base provided.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
Sublic/guides https//odesuunnisssoooolll..
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/pips..cc/publc//guue http://ddiisssseeeeee /
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have included all necessary details to understand and interpret the setup and results of the experimental section. Code is provided for the Rössler- systems, and details on all networks, optimizers, and training details were provided for latent TSCI.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
21

---

Justification: In our experiments, we have provided suitable measures of the statistical confidence of our methods. In particular, we run over several folds and report either median/percentiles mean/standard deviations of all results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative eтror rates).
If error bars reported in tables plots. The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification; We state in the experiments section that all code was run 6-Core Intel Core CPU and NVIDIA Titan RTX GPU.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http:://///aaapppscccccp////////
Question:
Answer: [Yes]
Justification: The research conducted this paper is in compliance with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
22

---

10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper includes discussion of the potential broader impact the work in the introduction.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification:
The paper does not pose any such risks for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have provided in-text mentions and citations for all assets or methods that used in this work.
23

---

Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license ge.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We have included code with the publication that we have taken to document, organize, and make understandable for the future reader. Our code publication includes proper license (MIT License) as well as instructions for use.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The current paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The current paper does not require IRB approval as it does not contain research with human or animal subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "dplbHPgOJ6", "reviewer": "Reviewer_VHEe", "summary": "The authors propose the Tangent Space Causal Inference (TSCI) method for detecting causalities in dynamic systems. TSCI works by considering vector fields as explicit representations of the systems’ dynamics and checks for the degree of synchronization between the learned vector fields. The authors present both a basic TSCI algorithm, which is lightweight and more effective than the basic CCM algorithm, as well as augmented versions of TSCI that leverage the expressive power of latent variable models and deep learning. The authors demonstrate improved causal inference performance across a number of benchmarks.", "questions": "What are the advantages of using corr(xˆ(t), x(t)) compared to Granger causal discovery, and the regression or conditional variance-based methods[1,3,6]?", "limitations": "Experimental study and comparison with additional baselines should be included to support the proposed method.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The method leverages vector fields of manifolds and improve the CCM method for causal direction detection between time series. Applying tangent space vector representation  for causal discovery is interesting. The author also provides theoretical analysis to support the algorithm.", "weaknesses": "a. The main weakness of the paper is that the experiments are insufficient to validate the method. Comparison with existing standard methods is missing. Experimental comparisons with classical Granger causal discovery and other methods are necessary. \n\nb. Theoretical analysis and comparison with existing mechanisms and methods should be included.\n\nc. Apart from comparison with classical time series causal discovery methods, comparison with bivariate causal discovery should be included to strengthen the paper. E.g. papers [1-7]\n\n\n[1]Blöbaum, P., Janzing, D., Washio, T., Shimizu, S. and Schölkopf, B., 2018, March. Cause-effect inference by comparing regression errors. In International Conference on Artificial Intelligence and Statistics (pp. 900-909). PMLR.\n\n\n[2]Khemakhem, I., Monti, R., Leech, R. and Hyvarinen, A., 2021, March. Causal autoregressive flows. In International conference on artificial intelligence and statistics (pp. 3520-3528). PMLR.\n\n[3]Ren, S. and Li, P., 2022, October. Flow-based perturbation for cause-effect inference. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management (pp. 1706-1715).\n\n[4]Daniusis, P., Janzing, D., Mooij, J., Zscheischler, J., Steudel, B., Zhang, K. and Schölkopf, B., 2012. Inferring deterministic causal relations. arXiv preprint arXiv:1203.3475.\n\n[5]Fonollosa, J.A., 2019. Conditional distribution variability measures for causality detection. Cause Effect Pairs in Machine Learning, pp.339-347.\n\n[6]Ren, S., Yin, H., Sun, M. and Li, P., 2021, December. Causal discovery with flow-based conditional density estimation. In 2021 IEEE International Conference on Data Mining (ICDM) (pp. 1300-1305). IEEE.\n\n[7]Hoyer, P., Janzing, D., Mooij, J.M., Peters, J. and Schölkopf, B., 2008. Nonlinear causal discovery with additive noise models. Advances in neural information processing systems, 21.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and consideration. We now address the weaknesses and questions mentioned above.\n\n# Weaknesses\n> The main weakness of the paper is that the experiments are insufficient to validate the method. Comparison with existing standard methods is missing. Experimental comparisons with classical Granger causal discovery and other methods are necessary.\n\nComparisons to the methods Granger causality, RECI, IGCI, and ANM have been added to our experiments. We refer to the general response to reviewers above for more information and for the test results.\n\n> Theoretical analysis and comparison with existing mechanisms and methods should be included.\n\nWe include theoretical analysis to support the method in the main manuscript. We have added more theoretical comparisons to Granger causality in the general response and a new appendix, but we also feel that comparisons between cross map-based methods and traditional causal discovery methods appear exhaustively in the CCM literature, for example, in [1-3].\n\n> Apart from comparison with classical time series causal discovery methods, comparison with bivariate causal discovery should be included to strengthen the paper. E.g. papers [1-7]\n\nComparisons to the methods RECI, IGCI, and ANM have been added to our experiments. We refer to the general response to reviewers above for more information and for the test results. We found that these methods performed underwhelmingly compared to CCM and TSCI, due to the nature of the systems under study.\n\n# Questions\n> What are the advantages of using corr(xˆ(t), x(t)) compared to Granger causal discovery, and the regression or conditional variance-based methods[1,3,6]?\n\nCCM and TSCI are intended to be used to study signals generated by coupled, deterministic dynamical systems. In this setting, Granger causal discovery fails because the separability condition is violated. We refer to the general response to reviewers above for more discussion on the matter. \n\nJust as the main assumption of Granger causality fails in coupled deterministic dynamical systems, the underlying assumptions of most bivariate causal discovery methods are also violated.The conditional variance based methods [1,3,6] are not appropriate because we typically do not expect an additive noise model. For example, from A2 of [3], we would not generally expect an invertible, monotonic function such that $g(x) = y$ to exist. This is apparent by looking at a scatter plot of $X$ and $Y$. Indeed, we wouldn't even expect a *function* to exist in general --- please see Figure 4 of the PDF attached to our general response. Such assumptions are typical in bivariate causal inference, for example in ANM or IGCI, which is why we chose to omit comparisons in the original manuscript. We verified that RECI, IGCI and ANM failed in the additional experiment described in the general response to reviewers above. \n\n# References \n\\[1\\] G. Sugihara, R. May, H. Ye, C. Hsieh, E. Deyle, M. Fogarty, and  S. Munch. Detecting causality in complex ecosystems. Science, 338(6106):496–500, 2012.\n\n\\[2\\] E. De Brouwer, A. Arany, J. Simm, and Y. Moreau. Latent convergent cross mapping. In International Conference on Learning Representations, 2020.\n\n\\[3\\] Assaad, C. K., Devijver, E., and Gaussier, E. Survey and evaluation of causal discovery methods for time series. Journal of Artificial Intelligence Research, 73, 767-819. 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xUFD73H75K", "reviewer": "Reviewer_htpT", "summary": "The authors propose a novel statistic for detecting causality in dynamical systems, which overcomes a key conceptual difficulty in the convergent cross mapping (CCM) method from Sugihara et al. [2012]. CCM is justified by the existence of a surjective \"cross map\" from the delay embedding of a driving to a driven dynamical variable in unidirectional coupling. However, the CCM test statistic is difficult to interpret because it does not directly measure whether this map can be constructed. \n\nInstead, the authors represent the dynamical equations through the time invariant flow fields induced by these delay embeddings. A cross map would induce a Jacobian transformation between these flow fields, so the authors propose testing the correlation between the flow fields of a candidate \"driven\" variable and a candidate \"driving\" variable pushed forward by the estimated Jacobian. \n\nThe authors demonstrate the clear advantage of their method for distinguishing the direction of causality in a unidirectional Rössler-Lorenz system.", "questions": "How do the authors interpret a measured Jacobian that cannot predict the correct magnitude of $\\mathbf{v}(\\mathbf{\\tilde{y}})$ from $\\mathbf{u}(\\mathbf{\\tilde{x}})$ but correctly predicts the direction? This case, of course, gives the TSCI test statistic value $1$. \n\nHow do the authors interpret their results if the measured Jacobian is invertible? Is this simply the case in which $x(t)$ and $y(t)$ are driven by the same set of variables $\\mathbf{z}$?", "limitations": "Yes. The authors adequately describe underlying assumptions and describe challenges (i.e., generalised synchrony) that obstruct informative results.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The authors give a convincing account of the difficulties with CCM and their refinements are sure to be highly impactful to the subject area.", "weaknesses": "The authors point out that the CCM statistic does not admit a simple decision rule and refer to their correlation coefficients as test statistics. However, the authors do not offer guidance as to when, e.g., a two-body dynamical system should be considered unidirectionally coupled. The authors could explore testable hypotheses such as \"$H_0$: there is no driving $x \\Rightarrow y$\", perhaps over a limited class of models. Perhaps this has been investigated in the CCM literature already and the authors could adapt existing methods for decision making. \n\nThe authors could also address their choice of correlation as their test statistic. Perhaps replacing correlation coefficients with mutual information $I(\\mathbf{u};\\mathbf{J}_F \\mathbf{v})$ should be acknowledged as a possible future refinement in the conclusion (see the first question below).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and consideration. We now address the weaknesses and questions mentioned above.\n\n# Weaknesses\n> The authors point out that the CCM statistic does not admit a simple decision rule and refer to their correlation coefficients as test statistics. However, the authors do not offer guidance as to when, e.g., a two-body dynamical system should be considered unidirectionally coupled. The authors could explore testable hypotheses such as $H_0$: \"there is no driving\", perhaps over a limited class of models. Perhaps this has been investigated in the CCM literature already and the authors could adapt existing methods for decision making.\n\nOne approach that we considered for making a decision was to assume that the two vector fields were independent and to apply a uniform distribution over the hypersphere to model their angles. However, we found this approach unsatisfactory in general because it often selected thresholds that were too small to reproduce the ground truth in our experiments. We mostly attribute this to the (sharp) null hypothesis being flawed; the theory of cross-maps does not preclude a weak or non-smooth estimator existing in the causal direction. We therefore do not generally have reason to believe the sharp null hypothesis of independence is realistic. This is also reflected in the mutual information case; the MI between time series with *no* coupling is estimated to be greater than zero, implying some correlation between samples. While MI estimators are not always accurate, they tend to underestimate the MI rather than overestimate it, so if anything, the MI with no coupling is likely to be larger than pictured in Figure 1 of the general response.\n\nThe issue of test statistics has been approached several times in the CCM literature, but in our opinion, no single approach is entirely satisfactory. Indeed, one popular approach is to simply set a threshold (say, 0.3) prior to testing. Several other approaches use \"surrogates\" by shuffling the time series. However, working with these surrogates is delicate, as we cannot easily assume that time series arising from dynamical systems are stationary. Finally, one other approach (for example, used by latent CCM) is to test the difference in the test statistic over simulations where coupling is and isn't present. We argue access to such a surrogate system is overwhelmingly rare in practice, and that this, to a certain extent, begs the question of causal analysis. We will update future versions of the manuscript to include some discussion regarding previous work on decision rules.\n\n> The authors could also address their choice of correlation as their test statistic. Perhaps replacing correlation coefficients with mutual information $I(\\mathbf{u}; \\mathbf{J}_F \\mathbf{v})$ should be acknowledged as a possible future refinement in the conclusion (see the first question below).\n\nThank you for this interesting and thoughtful suggestion. We ran an experiment to examine how the mutual information behaves compared to the cosine similarity, which will appear in an appendix of the revised manuscript. We defer to our general response to the reviewers above for information and discussion regarding this experiment.\n\n# Questions\n> How do the authors interpret a measured Jacobian that cannot predict the correct magnitude of $\\mathbf{v}(\\mathbf{\\tilde{y}})$ from $\\mathbf{u}(\\mathbf{\\tilde{x}})$ but correctly predicts the direction? This case, of course, gives the TSCI test statistic value 1.\n\nOur intuition is that this situation would be pathological and unlikely to occur in practice. In particular, it would imply that one can learn a cross map, but that the cross map identifies the wrong speed (or magnitude of velocity) consistently across the whole manifold. Generically, we would assume the resulting vector field would not be smooth, which would imply the supposed cross-map also lacked smooth structure (see e.g., Lee, Proposition 8.14). While it is possible that in some places the method over/underestimates the magnitude of the velocity, this is the result of a computational error and cannot be explained from the theoretical development. To check that the magnitude is closely tracked in our experiments, we added Figure 3 to the PDF appearing in the general rebuttal. \n\n> How do the authors interpret their results if the measured Jacobian is invertible? Is this simply the case in which and are driven by the same set of variables?\n\nThe Jacobian matrix of the cross map, as a linear map $T_xM \\rightarrow T_{F(x)}N$, is invertible whenever the two manifolds in question have the same dimensionality. This is a necessary condition for bidirectional causation, or a common driver, but it is not a sufficient condition. Mathematically, the reason is because an invertible Jacobian $\\mathbf{J}_F$ indicates _local_ invertibility of the function $F$, but it does not mean the function is *globally* invertible.\n\nIn addition, we expect that invertibility does not commonly occur in the embedded space. This is because we tend to embed in a higher dimension than the actual manifold, in order to ensure Takens' theorem holds. Thus, we would actually expect invertibility to happen on some linear subspace of the tangent plane $T_p M$, reflecting the subspace topology of $M \\subset \\mathbb{R}^n$.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vnHd4emxMV", "reviewer": "Reviewer_kcd7", "summary": "This paper proposes a novel tangential space causal inference method, TSCI, for identifying causal relationships from time series data generated by dynamical systems. TSCI considers vector fields as explicit representations of dynamic systems and checks for the degree of synchronization between the vector fields to identify the causal direction. TSCI has higher interpretability and scalability than the original CCM. Experiments on standard systems show that TSCI is much easier to identify causal directions than CCM.", "questions": "1.\tHow does TSCI perform on real systems? Can the authors demonstrate the performance of TSCI in identifying causal relationships with large scale data?\n2.\tDo the authors consider cyclic causal relationships? For example, if there are cyclic causal relationships X->Y->Z->X, can TSCI correctly identify them?\n3.\tCan TSCI correctly identify causality when the quality of reconstruction of latent states are low?\n4.\tDid the authors consider nonlinear causality? The Pearson correlation coefficient may not be effective in identifying nonlinear causal relationships.\n5.\tIn Figure 2, why cos(\\theta) in the correct direction not equal to -1?", "limitations": "The authors do not directly state the limitations in the paper. This work is limited by the quality of reconstruction of latent states, which is a difficult problem to solve, and the authors do not report the robustness of the proposed method for this problem.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.\tThis paper innovatively proposes to use tangent vector instead of delay embedding to identify causality in dynamic systems. The correlation between tangent vectors is geometrically motivated, so it has higher interpretability.\n2.\tTSCI can be combined with any differentiable regression model to learn the cross map, so it is more flexible and can adapt to model agnostic situations.\n3.\tThis paper has a solid theory and clear expression, so readers who lack theoretical knowledge can also understand this article.", "weaknesses": "1.\tIt may be unreasonable to use Pearson correlation coefficient in this paper to evaluate the correlation between tangent vectors. As far as I know, Pearson correlation coefficient can only evaluate linear correlation, if there is nonlinear correlation between tangent vectors, then Pearson correlation coefficient may not work well.\n2.\tThe experiments in this paper are not convincing. It only shows the results of algorithms identifying the causal relationships between a few variables in the toy system. It is recommended to evaluate TSCI in real systems with larger scale data. In addition, the results in this paper are not sufficient to show that TSCI is superior to CCM, because CCM also identities correct causal relationships. The authors should provide results that TSCI can correctly identify causality and CCM cannot.\n3.\tThis paper claims that TSCI is lighter and more scalable than CCM, but lacks the necessary results support.\n4.\tThis paper assumes a high quality of the reconstruction of the latent states, which may not be reasonable. As the article says, a high-quality reconstruction state does not always exist. It is recommended to conduct experiments with low latent state quality, and the robustness of TSCI and CCM for this problem has been compared.\n5.\tThe experiments in this paper lack necessary analysis. For example, the authors should further explain why TSCI is more resistant to the effects of general synchrony than CCM.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and consideration. We now address the weaknesses and questions mentioned above.\n\n# Weaknesses\n> It may be unreasonable to use Pearson correlation coefficient in this paper to evaluate the correlation between tangent vectors. As far as I know, Pearson correlation coefficient can only evaluate linear correlation, if there is nonlinear correlation between tangent vectors, then Pearson correlation coefficient may not work well.\n\nWe address this question in detail in the general response to reviewers. In short, we interpret the correlation used by TSCI as an expected cosine similarity, not Pearson correlation. Furthermore, the linear correlation is only considered after examining tangent spaces (local linearizations). We note that this is well-motivated by the theory of differential geometry; if the cross-map exists, the tangent vectors should align *exactly*.\n\n> The experiments in this paper are not convincing. It only shows the results of algorithms identifying the causal relationships between a few variables in the toy system. It is recommended to evaluate TSCI in real systems with larger scale data.\n\nWe are not aware of any real data sets that are freely available online, satisfy the assumptions of Takens' theorem, and have ground truth available. Of the articles that use real data, they are often not available online, or the causal ground truth is unknown (so any differences between TSCI and CCM cannot be interpreted). As a result, it is quite common in the CCM literature to primarily test on synthetic data. \n\n> In addition, the results in this paper are not sufficient to show that TSCI is superior to CCM, because CCM also identities correct causal relationships. The authors should provide results that TSCI can correctly identify causality and CCM cannot.\n\nWe disagree that CCM consistently identifies the correct causal relationships in the Rossler-Lorenz system. In Figure 2a, we have plotted the performance of TSCI and CCM for a wide variety of different dynamical systems. We agree that TSCI and CCM provide the same, correct conclusion for $C\\lesssim 1.5$. However, for systems with $C\\gtrsim 1.5$, a practitioner using CCM would likely (incorrectly) conclude bidirectional causality. On the other hand, TSCI correctly identifies unidirectional causality until $C\\approx 2.5$, where the effects of general synchrony are known to occur. Even with $C=1$, it is not clear that CCM consistently performs well, while TSCI clearly separates the two cross map scores (see Figure 2b).\n\n> This paper claims that TSCI is lighter and more scalable than CCM, but lacks the necessary results support.\n\nWe believe our claims regarding scalability may have been misunderstood; our claim is not that TSCI is more scalable than CCM, but rather that it enjoys the same scalability and lightweight implementation as CCM. Notably, the only added computational complexity in TSCI is solving a simple least squares problem.\n\n> This paper assumes a high quality of the reconstruction of the latent states, which may not be reasonable. As the article says, a high-quality reconstruction state does not always exist. \n\nSee the general response to reviewers for more discussion on this matter. In the manuscript appendix, we consider the effect of poor reconstruction quality in two specific cases: improper selection of the embedding dimension (appendix A.1) and the presence of additive noise in the recordings (appendix A.2). \n\n# Questions\n> How does TSCI perform on real systems? Can the authors demonstrate the performance of TSCI in identifying causal relationships with large scale data?\n\nSee the above reply to weakness 2.\n\n> Do the authors consider cyclic causal relationships? For example, if there are cyclic causal relationships X->Y->Z->X, can TSCI correctly identify them?\n\nTheoretically, cyclic causation of the form  $X \\rightarrow Y \\rightarrow Z \\rightarrow X$ is not uniquely identifiable using cross maps. It will likely detect that $X \\rightarrow Z$. There are two separate issues for this.\n\nFirst, CCM-based methods are not designed to handle mediation. This follows by carefully thinking about the motivation for CCM: if $X\\to Y$ and $Y\\to Z$, then there exist smooth functions $F: \\mathcal{M}_Y \\to \\mathcal{M}_X$ and $G:\\mathcal{M}_Z \\to \\mathcal{M}_Y$. By composition, there exists a function $\\mathcal{M}_Z \\to \\mathcal{M}_X$ as well.\n\nSecond, if $X\\to Z\\to Y$, and $Z\\to X$ as well, then $\\mathcal{M}_X$ and $\\mathcal{M}_Z$ should mutually cross map onto each other. This implies that the manifolds are diffeomorphic, and the direction of causality is indistinguishable.\n\nNote that we would not consider this a weakness of cross map methods. Rather, it is a difference in interpretation. We argue that when interpreting causality in terms of identifying directional coupling, these are the \"correct\" conclusions.\n\n> Can TSCI correctly identify causality when the quality of reconstruction of latent states are low?\n\nIt depends on the type of impurity in the reconstruction process. If vanilla TSCI is directly applied to a noisy reconstruction, it will fail. However, we are able to apply TSCI accurately to noisy data if we apply a  smoothing filter (appendix A.2). Please see our general response to reviewers for more discussion and additional experiments.\n\n> Did the authors consider nonlinear causality? The Pearson correlation coefficient may not be effective in identifying nonlinear causal relationships.\n\nThe TSCI method is a nonlinear causal discovery method. Nonlinear mappings between manifolds induce linear mappings between tangent spaces (Lemma 2.1.1). See the general response to reviewers above for discussion on the use of correlation. \n\n> In Figure 2, why cos(\\theta) in the correct direction not equal to -1?\n\nIn Figure 2, the vectors map onto each other near-perfectly, and the angle $\\theta$ between tangent vectors is near 0. Since $\\cos(0)=1$, the samples of $\\cos(\\theta)$ are near 1.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FEbDz3rcvN", "reviewer": "Reviewer_rcKb", "summary": "This paper presents a novel algorithm to perform causal inference in time series data based on the idea of convergent cross mappings (CCMs). Similar to Granger causality, the CCM paradigm infers causation by testing whether there exists a map (i.e., a predictor) from the reconstructed (unobserved) state trajectory of the effect variable to the reconstructed state trajectory of a cause variable. However, TSCI proposes to compare the (correlations of the) vector fields of the reconstructed dynamical systems rather than the reconstructed trajectories, as done by classical CCM. The approach is validated in two, low-dimensional synthetic examples.", "questions": "Some of these questions do not concern TSCI specifically, but CCM in general, and it would improve the quality of the paper if they are explained more explicitly throughout the paper. I am not an expert specifically on Granger causality and CCM, so it would be very helpful to the reader to highlight key differences more clearly.\n\n-\tIt would be great if Section 2.1. provided some basic mathematical definition and/or background on manifolds, to the degree that is needed to understand the paper. The notion of manifolds appears repeatedly, so some basic intuitions would help a lot (e.g., how do you define a manifold defined? Is a compact subspace of Euclidean space also covered by Taken’s theorem? How does this concept apply, e.g., to the experiments done in Section 3?)\n\n-\tWhy is it the case that “the angle […] is the same regardless if it is computed from intrinsic or extrinsic coordinates”? (l. 212)\n\n-\tWhat is the fundamental difference, especially conceptually and algorithmically, between CCM and Granger causality? It seems that both ideas are based on learning predictors from one time series to the other. Is the main difference the direction of the predictor (anticausal for CCM, causal for Granger) or the fact that CCM-based methods work with reconstructed trajectories?\n\n-\tIn general, what are the identifiability conditions or theoretical guarantees of CCM? Can we always infer all causal dependencies in the infinite sample limit for arbitrarily large dynamical systems? From the causal discovery/causal structure learning perspective, this seems like a “free lunch” situation and too good to be true. What assumptions on the class of dynamical systems are needed to guarantee identifying the causal dependencies?\n\n-\tCould you give more explanation for why this is the case in the introduction? “… for many systems, namely dynamical systems, the separability condition [of Granger causality] is violated” (l. 32-33)", "limitations": "The paper is doing a good job at discussing possible disadvantages of the methods in Section 4. For other limitations, see weaknesses/questions above.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Overall, the paper proposes a novel algorithmic idea that is well-motivated in the context of existing CCM literature. References to related work appear complete, but I am not an expert on Granger/CCM causal inference specifically. The paper is very well written, which makes it easy to understand the concept without having explicit background in this field of causal inference, except for certain additional background that could improve understanding further (see below).", "weaknesses": "The explanation of why the correlation coefficient over tangent vectors is better than over trajectories (in CCM) is shaky and needs to be made more precise (last paragraph of Section 2). Specifically, it would be great if the work could expand on the argument and reasoning behind: “since the points being predicted in CCM live on a manifold, measuring correlation in the ambient (extrnistic) space is not well-motivated (l. 205-206)”. Why is it not well-motivated? Given this, the work states “the tangent planes [of TSCI are] geometrically motivated” (l. 209-210). Please elaborate more here as well. In my eyes, this paragraph lies at the heart of the differences/(dis)advantages of CCM vs TSCI, so it would be great if this discussion is expanded and made more explicit.\n\nThe experimental evaluation of the work can be improved. First, as acknowledged, e.g. in lines 180-183, TSCI requires accurate embeddings/reconstructions of the state space trajectories to work well. However, none of the experiments study how the performance of TSCI and classical CCM behave/degrade as reconstruction quality decreases, which seems like a fundamental component of applying either method in practical applications. \nIn addition, in none of the experiments, CCM or latent CCM (i.e., existing works) fail to identify or misidentify the true causal effects. Thus, all baselines and TSCI achieve perfect accuracy, which suggests that the experiments may be too easy? The example systems here are very low dimensional, and the library length very long (e.g. Fig 3b). How does the performance compare in larger dimensionalities, where certain edges may be misclassified? The experiments do not distinguish which method is the better one.\n\nAlgorithms 1 and 2 are stated for 2-variable dynamical systems (with variables X and Y). Is the extension to multivariate systems simply done by applying TSCI to all pairs of variables in the system (e.g. in the experiments of Section 4.2)? Is there a curse of dimensionality involved when comparing vector fields, because we have to sample tangent vector points on the manifold? The work states that “[CCM] estimates [are] biased by the distribution of observations along the manifold” (l. 206-)”. If TSCI does not have this bias, it must sample from the whole manifold, or am I getting this wrong?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and consideration. We now address the weaknesses and questions mentioned above.\n\n# Weaknesses\n> The explanation of why the correlation coefficient over tangent vectors is better than over trajectories (in CCM) is shaky\n\nOne major reason that the use of correlation in TSCI is more principled over CCM is because it appears naturally when checking the degree of alignment between vector fields, measured by the angle. This correlation is *not* the same as Pearson correlation, because the tangent vectors are not i.i.d. Euclidean vectors and they technically exist in separate tangent spaces, but from the algorithmic perspective, correlation is the operation that results from the math. \nIn contrast, using the correlation between points on the manifold is not well-motivated because it does not clearly arise from any geometric principle related to the problem at hand.\n\n\n> TSCI requires accurate embeddings/reconstructions of the state space trajectories to work well. However, none of the experiments study how the performance of TSCI and classical CCM behave/degrade as reconstruction quality decreases\n\nWe refer to the general response to reviewers above for comments on how  CCM/TSCI is sensitive to the reconstruction quality. We note that two of the supplemental experiments in the appendix A.1 and A.2) address the degradation of performance due to under-embedding the manifolds, and due to the injection of additive noise. We have also added an additional experiment including a sinusoidal confounder to the general response.\n\n> In addition, in none of the experiments, CCM or latent CCM (i.e., existing works) fail to identify or misidentify the true causal effects. ... The experiments do not distinguish which method is the better one.\n\nFor each value of C in Figure 3, a different system was tested. In every case $C > 0$, the true causality is $X \\rightarrow Y$. For moderate values of C, CCM concludes that $X \\leftrightarrow Y$, which is an incorrect inference. In contrast, TSCI demonstrates much better separation in these experiments. \n\n\n> Algorithms 1 and 2 are stated for 2-variable dynamical systems (with variables X and Y). Is the extension to multivariate systems simply done by applying TSCI to all pairs of variables in the system? \n\nThe application of TSCI to multivariate systems proceeds largely in the same way as CCM. The general scheme is to perform this analysis pairwise across the entire network. There are likely more efficient ways to perform this inference, and we generally do not recommend TSCI (or CCM) for large networks, but this is not the focus of the current paper.\n\n> Is there a curse of dimensionality involved when comparing vector fields, because we have to sample tangent vector points on the manifold?\n\nBecause the tangent vectors can be estimated from the time derivatives of the original scalar time series, we do not anticipate a strong dependence on the dimension of the manifold. In particular, we expect in general that the tangent vectors belong to a lower dimensional subspace inherited from $\\mathcal{M}_X$. See also the experiment in Appendix A.1. \n\n> The work states that “CCM estimates are biased by the distribution of observations along the manifold”. If TSCI does not have this bias, it must sample from the whole manifold, or am I getting this wrong?\n\nThe TSCI test statistic uses samples from the whole manifold, as illustrated in Eq. (12). In particular, we take the expected value of the cosine similarity across all test samples.\n\n\n# Questions\n> It would be great if Section 2.1. provided some basic mathematical definition and/or background on manifolds, to the degree that is needed to understand the paper.\n\nThank you for this suggestion. To make the manuscript easier to read, we are adding some background material on manifolds and Takens' theorem to the appendix. The concepts apply to the systems in Section 3 because they are all generated by systems governed by deterministic differential equations. \n\n\n>  Why is it the case that “the angle is the same regardless if it is computed from intrinsic or extrinsic coordinates”?\n\nThe angle between tangent vectors is the same when computed using the intrinsic definition or the extrinsic definition. In technical language, the tangent space $T_x(M)$ at a point x along a manifold $M$ is a vector subspace of the tangent space $T_x(\\mathbb{R}^n)$ \\[1, p.80\\]. Calculation of angles in the space $T_x(M)$ corresponds to an intrinsic coordinate definition, and calculation in $T_x(\\mathbb{R}^n)$ corresponds to an extrinsic definition. The angle between two tangent vectors $u$ and $v$ in $T_x(M)$ is the same as if it were computed in $T_x(\\mathbb{R}^n)$, due to the subspace property. \n\n\n> What is the fundamental difference, especially conceptually and algorithmically, between CCM and Granger causality?\n\nPlease see our general response to reviewers above for discussion about CCM and Granger causality.\n\n> what are the identifiability conditions or theoretical guarantees of CCM? Can we always infer all causal dependencies in the infinite sample limit for arbitrarily large dynamical systems? \n\nIn the large sample limit, the shadow manifold represents the reconstructed latent states perfectly. A key assumption here is that the system is generic, as to avoid situations where symmetries of the model inhibit our inference. However, even given a deterministic dynamical system with attractors and with generic observations that permit a perfect embedding, we still have the critical issue of general synchrony (detecting $X \\leftrightarrow Y$ could be a false positive in one direction). The problem of synchrony is ultimately a topological one, because the situation that a cross map may be approximately invertible occurs even in the original state spaces.\n\n\\[1\\] A. McInerney. First Steps in Differential Geometry. Undergraduate Texts in Mathematics. New York, NY: Springer New York. 2013", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to thank the reviewers for their thoughtful comments. We address some common concerns and questions here.

# Comparisons to Granger Causality and Other Methodologies
As mentioned in the introduction, Granger causality (GC) can perform poorly on signals generated by deterministic dynamical systems. GC assumes separability, i.e., that novel probabilistic information about the future is continually generated by the cause, and this information is unique to the cause. Thus, GC is most readily applied to stochastic systems. In contrast,  deterministic dynamical systems violate separability due to the presence of deterministic rules for the system's evolution. This same violation ultimately makes the anti-causal prediction in the CCM/TSCI approaches possible: the cross-map exists because the cause leaves such a strong signature on the effect that recovering the dynamical rule for the system is possible (a consequence of Takens' theorem, see \[5\]). As a result, CCM/TSCI is often applicable specifically because the assumptions of GC are violated.

We expanded our comments in the introduction and included an appendix to better explain comparisons to GC. As this is relatively well-known in the CCM literature, we also leave several references to past discussions, e.g., the appendices of \[1\] and \[2\]. We added a comparison to GC in our Rossler-Lorenz system, showing that GC always incorrectly predicts bidirectional causality for $C > 0$ (Table 1 of the attached PDF). 

At the suggestion of Reviewer VHEe, we added additional comparisons to the methods RECI, IGCI, and ANM. These methods are expected to perform quite poorly as several of their assumptions are harshly violated. Namely, even in the situation that $X \rightarrow Y$, we would not expect that $y_t = f(x_t)$ (see e.g. Figure 4 attached), and we would especially not expect an additive noise decomposition. Our results are in Tables 1, 2, 3 in the attached PDF, and are summarized as follows: IGCI consistently fails to detect a causal edge; RECI consistently chooses the direction $X \rightarrow Y$, but does so even when $C=0$, suggesting this result is *not* actually detecting causality -- to this end, the variance ratios are typically large; finally, for all $C > 0$, ANM detects bidirectional causality.

# On the Use of Cosine Similarity, and Alternative Measures
We believe the TSCI score is best understood as a cosine similarity, not Pearson correlation. This is only computationally equivalent because vectors in the tangent space are centered. This method doesn't preclude nonlinear relationships, since tangent vectors are measured locally. By Lemma 2.1.1 in the manuscript, every smooth function $F$ induces a linear mapping $\mathbf{J}_F$ between tangent spaces which aligns the systems' vector fields.

Reviewer htpT suggested the mutual information $I(\mathbf{u}; \mathbf{J}_F \mathbf{v})$ as a possible alternative test statistic. Though lacking direct justification via Lemma 2.1.1, the use of MI seems intuitive, and in Figure 2 of the attached PDF we test MI and cosine similarity on the Rossler-Lorenz test systems. We find that MI typically shows less separation than TSCI. Two other issues are difficulties interpreting any particular value of MI, and that estimating MI from samples is a difficult, non-trivial problem.

# Comments on Existing Experiments
We believe several experiments in the appendices of our manuscript partially address reviewers' concerns. The experiment in Appendix A.1 demonstrates how poor manifold reconstruction, due to different selections of embedding dimensions, affects the resulting performance. The experiment in Appendix A.2 similarly considers the effect of additive noise in the observed signals. We recognize that these experiments should be better referenced in the main manuscript, and added text to the main manuscript explaining additional experiments, including the new experiments conducted during this rebuttal period.

# Comments on the Quality of Reconstructions
There are many ways to achieve a poor state space reconstruction, and we therefore cannot make general statements about robustness to reconstruction quality. In the appendix of the manuscript, we consider poor reconstructions due to poor selection of the embedding dimension (A.1) and additive noise (A.2). In these experiments, both CCM and TSCI behave similarly and in a manner consistent with the theory. 

Generally, there are many non-trivial ways in which the state reconstruction can be poor, e.g., presence of periodic trends \[3\] and underexploration of the shadow manifold \[4\]. A systematic study of how TSCI depends on the reconstruction quality would already provide ample material for future work. We considered one additional experiment to study how CCM and TSCI respond to a dynamical confounder in the form of an added sine wave. We visualize our results in Figure 3 of the attached PDF. These results suggest that TSCI may be robust to a moderate confouding influence, deteriorating only when the signal power of the confounder overtakes the signal power of the signals of interest. Notably, TSCI seems significantly more robust to false claims of strong causation when the relative power of the confounder is large. 


# References
\[1\] G. Sugihara, R. May, H. Ye, C. Hsieh, E. Deyle, M. Fogarty, and  S. Munch. Detecting causality in complex ecosystems. Science, 338(6106):496–500, 2012.

\[2\] E. De Brouwer, A. Arany, J. Simm, and Y. Moreau. Latent convergent cross mapping. In International Conference on Learning Representations, 2020.

\[3\] S. Cobey, and E.B. Baskerville. Limits to Causal Inference with State-Space Reconstruction for Infectious Disease. PLoS One. 2016

\[4\] K. Butler, G. Feng, and P. M. Djuric. On causal discovery with convergent cross mapping. IEEE Transactions on Signal Processing, 2023.

\[5\] J. Stark. Delay embeddings for forced systems. I. Deterministic forcing. Journal of Nonlinear Science, 9, 255-332. 1999.

Meta-review: This submission was found to tackle an important problem that receives growing attention, clearly highlights potential pitfalls and difficulties with CCM, and develops a novel algorithmic idea to determine causality in dynamical systems. It was found to be clearly written and well structured such that it is mostly easy to read and follow. I do not believe that a comparison with "static" bivariate causal discovery algorithms is required and agree with the authors that no meaningful contributions can be drawn from such a comparison. I encourage the authors to please add their main clarifying arguments from the rebuttal to the camera ready version and to potentially also think about and address some of the minor remaining concerns of the reviewers.
Decision: Accept (poster)