Tangent Space Causal Inference: Leveraging Vector Fields for Causal Discovery in Dynamical Systems

Authors: kurt  butler
daniel  waxman
petar m djurić
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 20:55:19

Abstract: Causal discovery with time series data remains a challenging yet increasingly important task across many scientific domains. Convergent cross mapping (CCM) and related methods have been proposed to study time series that are generated by dynamical systems, where traditional approaches like Granger causality are unreliable. However, CCM often yields inaccurate results depending upon the quality of the data. We propose the Tangent Space Causal Inference (TSCI) method for detecting causalities in dynamical systems. TSCI works by considering vector fields as explicit representations of the systems' dynamics and checks for the degree of synchronization between the learned vector fields. The TSCI approach is model-agnostic and can be used as a drop-in replacement for CCM and its generalizations. We first present a basic version of the TSCI algorithm, which is shown to be more effective than the basic CCM algorithm with very little additional computation. We additionally present augmented versions of TSCI that leverage the expressive power of latent variable models and deep learning. We validate our theory on standard systems, and we demonstrate improved causal inference performance across a number of benchmark tasks.

Main paper: 
 introduction: 
 The discovery of causal relationships is one of the most fundamental goals of scientific work. When causal relationships are known and understood, we can explain the behavior of a system and understand how our actions or interventions upon the system will affect its behavior [27] . This kind of reasoning is fundamental in many problem domains, such as medicine, environmental policy, and economics. However, it is not always possible to perform interventions and observe their effects. For example, medical practitioners might only have one chance to prescribe a medication to help a patient. As another example, an ecologist might find that performing a large experiment is forbiddingly expensive or otherwise infeasible. Due to these concerns, there remains a great interest in the problem of observational causal inference, where one infers causation without manipulating the system under study directly.For time series data, the most prominent tool for observational causal inference is Granger causality [15] . Granger causality operates on the assumption that if one system (the cause) is driving changes in another system (the effect), then the cause should have unique information about what will happen to the effect. This is encoded in an assumption called separability, which posits that the information needed to predict the future behavior of the effect is not contained in the effect itself. However, the separability condition runs counter to the behavior of coupled dynamical systems: if long histories of the effect can forecast the cause, then an autoregressive model with the appropriate lag may forecast the effect from itself. This effect is explored in the supplementary materials of [12, 41] , and we give an example in Appendix B. Since systems with (approximately) dynamical behavior are ubiquitous in many application domains, alternative methods have become of interest.To address the failures of Granger causality in coupled dynamical systems Sugihara et al. [41] proposed the convergent cross mapping (CCM) method, which directly takes advantage of the topological properties of dynamical systems. CCM can be seen as an adaption of earlier work that studied the synchronization of dynamical systems [2, 29, 35] into an algorithmic procedure for detecting causalities between time series. Using Takens' theorem, a well-known result in dynamical systems theory [42] , CCM attempts to detect causality by reconstructing the state space of a given time series, and then learning a time-invariant function that maps between reconstructed state spaces, called a cross map. The key assumption is that the cross map only exists if the original systems were dynamically coupled. In this sense, the "causality" of CCM deviates from the popular Pearlian framework [27] , and is better interpreted as identifying which variables "drive" or "force" a dynamical system based on observational data. 2 Building upon this framework, several extensions of CCM have been proposed to address various technicalities and caveats of the approach and to generalize the method to new domains. However, the CCM test statistic is difficult to interpret and does not admit a simple decision rule. CCM also does not explicitly learn a cross map function.To improve upon CCM, we propose Tangent Space Causal Inference (TSCI). TSCI detects causation between dynamical systems by explicitly checking if the dynamics of one system map on to the other. The proposed method can be seen as a drop-in replacement for the CCM test, providing an interpretable and principled alternative while remaining compatible with many of the extensions of CCM. Furthermore, the proposed method is model agnostic, meaning that it can be adapted to any method used to learn the cross map function, including multilayer perception (MLP) networks, splines, or Gaussian process regression. The only major assumption of TSCI is that the time series under study were generated by continuous time dynamical systems (i.e, by systems of differential equations), which is a standard assumption in a number of physical systems [36] . As a result, TSCI is applicable to many of the same problems as CCM.Causal representation learning. The primary function of Takens' theorem in the CCM method is to yield a representation of a system's latent state so that it may be used for cross mapping. As a result, numerous generalizations of Takens' theorem emerged in the following decades [32, 33, 40] . Causal representation learning aims to learn hidden variables from high dimensional observations [1] , and a principle task is to decipher the causal relationships between many, possibly redundant, observations of a system [30] . In such cases, methods of dimensionality reduction can be applied to extract causal variables from the raw observations [31] . This is particularly important in the processing of large, spatio-temporal data sets [36, 43] . Some authors have proposed that CCM can be improved by aggregating data from multiple sources into the reconstruction of the latent states [7, 45] , which requires an awareness of how the observation data relate to the causal variables of interest.Causal discovery with cross maps. CCM has popularized the use of cross maps as a tool for causal inference, and many variations and improvements of the basic CCM methodology have been proposed. Some of these works aim to improve the reconstruction of latent states in these models by using a Bayesian approach to latent state inference [13] , where the approach is adapted for spatial/geographic data [14] or modified for sporadically sampled time series [12] . Additionally, several improvements have suggested changing the way that a cross map is detected: some authors have recommended varying the library length [25] or time delaying the cross mapping [44] to yield refined information. The k-nearest neighbor regression, which is used in the original CCM algorithm, can be swapped out for a radial basis function network [23] , or for a Gaussian process regression model [13] . Other approaches have not directly learned the cross map at all; instead they have examined other aspects of the reconstructions such as their dimensionality [3] or used pairwise distance rankings as a signature of the mapping [5] .Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: The abstract and introduction have been written to clearly state our claims and contributions of the paper. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
proposed method: 
 We begin by considering the inference of a causal relationship between two time series, x(t) and y(t). Our starting assumption is that both time series were generated by latent dynamical systems with states z x (t) and z y (t), respectively, whose behavior is governed by a set of ordinary differential equations (ODEs). For motivation, we consider a particular case with a unidirectional coupling between the latent states:EQUATIONEQUATIONEQUATIONEQUATIONwhere the observation functions h x , h y and the time derivative functions f x , f y are assumed to be differentiable. A causal relationship between x and y is evidenced by the appearance of z x in the equation for dz y /dt. The state vectors z x and z y could possibly have redundant information, but by writing the system in this form, we obtain an asymmetry between x(t) and y(t) which will inform our inference of causation. In our notation, if z x appears in the equation for dz y /dt, we will write x → y.We now explain how the CCM method approaches this problem and how the TSCI method builds upon the CCM framework.CCM is a technique for inferring causation between time series generated by dynamical systems [41] , as seen in Eqs. ( 1) to (4). The basic motivation for CCM is that given an observed time series x(t), one can construct a vector x which acts as a proxy for the latent states z x that generated it. Given two constructions, x and ỹ, we may detect if there is a mapping between them, which provides evidence of a causal relationship. From Eq. (4), since z x influences z y unidirectionally, the effect y(t) contains more information than the cause time series x(t), and as a result ỹ generally contains enough information to reconstruct x. With this in mind, we can frame CCM as a two-step procedure [13] . In Step 1, we construct representations x and ỹ that are proxies for z x and z y , respectively. InStep 2, we detect a mapping between reconstructions; if there is a mapping ỹ → x, then the reverse causality holds, x → y.To reconstruct the latent state space, as in Step 1, multiple approaches could be employed. However, in the basic CCM methodology, one uses the so-called delay embedding of x(t),EQUATIONwhere τ and Q are parameters called the embedding lag and embedding dimension, respectively. The justification that x is a good proxy for z x is given by Takens' theorem [42] , which states that x and z x are equivalent up to a nonlinear change of coordinates.Theorem 2.1 (Takens' theorem [33] ) Let M be a compact manifold of dimension d. Let z ∈ M evolve according to dz/dt = f (z), let ϕ τ be the mapping that takes z t to z t+τ , and letEQUATIONis an embedding of M into R Q .Since Φ f ,h,τ (z x (t)) = x(t), Takens' theorem tells us that if z x lives on a manifold M , then the points x lie on a manifold M x , which is called the shadow manifold of x [41] . Despite the number of assumptions in the statement of Takens' theorem, many generalizations of the statement exist, allowing us to justify the use of delay embedding to systems with strange attractors [33] and systems with noise [40] . If a system satisfies the conditions for Takens' theorem, in the sense that M x is a valid embedding of M , then we say that the system is generic.Since only z x influences x(t), Takens' theorem implies that x is equivalent (up to nonlinear coordinate change) to z x . However, due to the appearance of z x in (4), both z x and z y are responsible for generating y(t). As a result, Takens' theorem suggests that ỹ is equivalent to (z x , z y ) as a concatenated vector [39] . Since (z x , z y ) clearly can be mapped onto z x , the equivalence between the delay embeddings and the latent states suggests that there is a mapping ỹ → x, called a cross map. Thus, cross maps encode the idea that the effect time series contains information about its cause. This is encoded in the following corollary to Takens' theorem.Corollary 2.1.1 Suppose that x → y for a generic system. Then there exists a function F such that x(t) = F (ỹ(t)) for all t.The proof of the corollary is provided in the Appendix.Step 2 of CCM is then to detect if x → y by checking if a cross map ỹ → x exists. To this end, Sugihara et. al. [41] propose to check the predictability of the time series x(t) given ỹ(t). They use a form of k-nearest neighbors regression to produce an estimate x(t) of x(t) given ỹ(t), and then they define a test statisticEQUATIONwhere corr is the Pearson correlation coefficient. To test the reverse causal direction, x ← y, one simply performs Step 2 again with the roles of x and y reversed.Takens' theorem and cross maps as a tool for causal inference are rooted in a solid mathematical foundation, but the CCM test does not exploit all of the properties of shadow manifolds. Namely, it does not exploit the fact that the shadow manifolds are copies of the latent manifolds. In practice, there are cases in which CCM learns a cross map that appears to be reasonably predictive, but results in a false positive. Thus, a more robust algorithm would exploit more subtle properties of the hypothesized cross map. We propose TSCI as alternative to Step 2 in the CCM algorithm.TSCI operates by checking if the ODE on one manifold, M x , can be mapped to an ODE on another manifold M y . To understand how this works, we need to reframe our discussion of ODEs in terms of vector fields. Recall that given an ODE of the form,EQUATIONwe may interpret u to be a velocity vector field on the manifold. When evaluated, u(x) is a tangent vector of the manifold M x , existing in the tangent space T xM x4 . From calculus, we know that tangent vectors in T xM x can be mapped to tangent vectors in T F (x) M y by the Jacobian matrix J F (x) at the point x. By checking if the tangent vectors can be mapped in such a way, TSCI provides an alternative to the CCM causality test. A visual motivation for the TSCI methods is depicted in Fig. 1 .Before we can map vector fields from one manifold to another, we need to check that meaningful vector fields exist on the shadow manifolds. As usual, this is also a corollary of Takens' theorem.Corollary 2.1.2 There exists a vector field u on M x such that the embedding Φ f ,h,τ in Takens' theorem is a time-invariant mapping. If we define γ(0) = Φ f ,h,τ (z x (0)), and if we let γ(t) to be the flow of a point γ(0) under the vector field u, then Φ f ,h,τ (z x (t)) = γ(t), for all times t. Figure 1 : Visual motivation for the TSCI method. Observed time series are related to latent states evolving on some manifolds. Given observed time series x(t) and y(t), we reconstruct latent states x(t) and ỹ(t) that reside in manifolds M x and M y , respectively. If x and y are causally coupled, there should exist a function F that maps between these manifolds. Using the Jacobian matrix J F , we can map the velocity vector field on M x to a vector field on M y , and use the angle between velocity vectors as a measure of their similarity.The proof of the corollary is provided in the Appendix.The corollary guarantees that we can learn ODEs that describe the dynamics of x and ỹ, once we have obtained valid reconstructions of the latent states. Let u and v be vector fields such thatEQUATIONEQUATIONBecause the cross map F is a mapping between the two shadow manifolds, the Jacobian matrix of the cross map relates the vectors u and v. We state this formally in the following Lemma.Lemma 2.1.1 Let M x and M y be manifolds with respective vector fields u and v that define their dynamics. If there exists a cross map F : M x → M y , then for every point x ∈ M x , we have that v(F (x)) = J F (x)u(x), where J F (x) is the Jacobian matrix of F at x.The proof of the lemma is in the Appendix.While the cross map F : M x → M y is a mapping between points on the manifolds, the Jacobian matrix J F (x) induces a mapping between the tangent spaces at x and ỹ. We show this visually in Fig. 1 . Since the velocity of x can be represented by a tangent vector u(x), the Jacobian matrix J F allows us to map these vectors to tangent vectors in M y . If there is a cross map, then the vector field v and the push forward vector field J F u should match exactly. On the other hand, there should be no correlation in the absence of a causal relationship, assuming quality reconstructions and plentiful data. The degree of alignment between v and J F u can be used as a test statistic for the presence of a causal link. We therefore propose the TSCI test statistic,EQUATIONBecause the tangent vectors are centered in their respective tangent planes, we can geometrically interpret the TSCI test statistic as the expected cosine similarity between the vector field u and the push forward vector field J F v,EQUATION)In practice, the estimation quality of the cross map also depends on the location in the reconstruction space, and so the cosine similarity takes on a distribution of values (Fig. 2 ), and weak-to-moderate correlation may be empirically observed as an artifact of limited data.Input: Embedding/vector field matrices X, U ∈ R T ×Qx and Y, V ∈ R T ×Qy Output: Score r X→Y 1: Learn a differentiable function F such that X t = F (Y t ) for all t 2: for t = 0, 2, . . . , T -1 doCompute the Jacobian matrix J F (Y t ) ∈ R Qx×Qy of the function F at location Y tPush forward the tangent vector by computingÛt = V t J ⊤ F (Y t ) 5: end for 6: Define r X→Y = corr( Û, U) Algorithm 2 TANGENT SPACE CAUSAL INFERENCE WITH K-NEAREST NEIGHBORS Input: Embedding/vector field matrices X, U ∈ R T ×Qx and Y, V ∈ R T ×Qy , regression parameter K > max(Q x , Q y ) Output: Score r X→Y 1: for t = 0, 2, . . . , T -1 do 2:Find the indices τ 1 , . . . , τ K of the K-nearest neighbors of Y tCompute the local displacements in x-space:∆X k = X τ k -X t 4:Compute the local displacements in y-space:∆Y k = Y τ k -Y t 5:Compute the least-squares solution to ∆Y k J = ∆X k 6:Compute Ût = V t J 7: end for 8: Define r X→Y = corr( Û, U)Based on these analyses, we propose the TSCI method in Algorithm 1, which learns a cross-map F and returns the alignment of the vector fields u and J F v. CCM is sensitive to the quality of the reconstruction of the latent states because an improperly constructed shadow manifold can sabotage the method, both theoretically and empirically [6] ; TSCI similarly requires that the shadow manifold be properly embedded. Because of this, we assume that the embedding vectors x and ỹ and the estimates of their vector fields u(x) and v(ỹ) at each point are supplied as inputs to the algorithm. A variety of methods could be used to estimate the vector fields, but the simplest approach is to use finite differences,u(x) = dx dt =      dx(t) dt dx(t-τ ) dt . . . dx(t-(Q-1)τ ) dt      ≈ 1 ∆t     x(t + 1) -x(t) x(t + 1 -τ ) -x(t -τ ) . . . x(t + 1 -(Q -1)τ ) -x(t -(Q -1)τ )     ,where ∆t is the sampling rate of the data. Finite differences are known to be sensitive to noise, so in real scenarios a more careful approach to obtaining the time derivatives is necessary. For a practical implementation, we propose using the central finite-differences, which are second-order accurate [16] , or the derivatives interpolated by a Savitsky-Golay filter for noisy data [34] .On methods to learn the cross map function. The TSCI algorithm is notably agnostic to the regression approach used for the cross map F , with reasonable approaches including multilayer perceptron networks (MLPs), splines, and Gaussian process regression (GPR). Depending on the approach, derivatives can then be estimated from the model, either using automatic differentiation or analytical derivatives. For GPR in particular, analytical derivatives are straightforward to compute [38] .Since regression can sometimes be a computationally complex procedure, in Algorithm 2 we also provide a version of TSCI that is based on the k-nearest regression, in analogy to CCM. For clean and dense data, this simple approach can yield accurate results, but it is generally unsuitable for noisy or sparse data. In the latter case, it is preferrable to combine the TSCI approach with other methods of denoising time series, or learning of latent dynamical models of the observed time series.In particular, the learning of a latent ODE learning [12] can be combined with the TSCI test to yield accurate causal inference.Figure 2 : Shadow manifolds M x and M y from the unidirectionally coupled Rössler-Lorenz system Eq. ( 13) with C = 1, and the corresponding histograms of û⊤ u and v⊤ v. The test statistics, r X→Y and r X←Y , correspond to the means of these distributions.On the use of correlation coefficient. In relation to CCM, it can be noted that both TSCI and CCM use a correlation coefficient as their test statistic. However, a critical difference between the two methods is how the usage of this statistic is justified. The correlation coefficient used in CCM analysis, called the cross-map skill [41] , is used to measure the accuracy of predictions of the cross map. However, since the points being predicted in CCM live on a manifold, measuring correlation in the ambient (extrinsic) space is not well-motivated. Furthermore, estimates of this correlation can be biased by the distribution of observations along the manifold, and as a result, a high correlation can be achieved with a relatively low-accuracy prediction by guessing the general region in which the points lie.In contrast, the correlation between tangent vectors (or vector fields) is a more geometrically motivated quantity: the correlation is a linear measure of similarity and the tangent vectors belong to a (centered) linear space. Additionally, the shadow manifolds are constructed to be submanifolds of Euclidean space, so correlations computed in extrinsic coordinates will match correlations computed using the intrinsic coordinates of the shadow manifold. By Lemma 2.1.1, the correlations will be identically 1 if a cross map exists.One alternative to the cosine similarity is the mutual information (MI). Specialized to the current text, the MI I(u; J F v) quantifies the reduction in the uncertainty of u given the pushforward vector J F v [9] . While the information-theoretic underpinning of the MI is attractive, we have two main reasons to prefer the cosine similarity: First, the MI between two (continuous) distributions can be difficult to interpret. For example, it is not obvious if I(u; J F v) = 0.5 is a strong dependence or not, particularly in the case of continuous distributions. The cosine similarity, on the other hand, is upper bounded by 1, so corr( Û, U) = 0.95 is easily interpreted as having near-perfect reconstruction. Second, accurate estimation of the MI from samples is a notoriously difficult task, particularly in high dimensions, and no single estimator works consistently well [10] . We provide some experiments with the MI, and show the differing performance of different estimators, in Appendix B.3.H d = {y ∈ R n : y d+1 = • • • = y n = 0} under F x is given by F x (H d ) = U x ∩ M.Without a loss of generality, we can assume that F x (0) = x in this definition. This allows us to easily define tangent vectors. A vector v ∈ R n is called a tangent vector of M at the point x if there exists a path γ(t) inH d such that d dt (F x (γ(t))| t=0 = v,where F x is the diffeomorphism in the above definition. The tangent space T x M is defined to be the set of all tangent vectors of M at x. Note that from these definitions, it is clear that R n is a manifold andT x R d = R d .In calculus, we define a vector field u on R n to be a function u : R n → R n . We generalize this to manifolds in Euclidean space by defining a vector field u on M to be a mapping u : M → R n such that u(x) ∈ T x M at every point x.An embedding of a manifold M ⊂ R n is a mapping F : R n → R m such that F restricted to M is a diffeomorphism onto its image. If M is a manifold of dimension d and F is an embedding, then m is necessarily greater than or equal to d. In general, it is an interesting question to ask how much larger m must be so that embeddings are easy to find. The Whitney embedding theorem says that it is a generic property that F : R n → R m when m ≥ 2d + 1. Takens' theorem refines the Whitney theorem by asserting that such an embedding can be obtained from a scalar observation function, by time lagging the observations sufficiently many times.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: The paper discusses the limitations and assumptions of the approach, and the possible failure modes. A section discussing the advantages and limitations of the approach has also been included in the paper. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
experiments: 
 We validate the performance of TSCI on two datasets that are popular in the literature. The first arises from a coupled Rössler-Lorenz system, where the ground truth causality is known and the causal influence can be smoothly modulated. The second example was proposed in [12] and uses sporadic time series from coupled double pendulums, and illustrates the applicability of TSCI to extensions of CCM. Code implementing TSCI is available at https://github.com/KurtButler/ tangentspaces. All comparisons to CCM use the skccm module in Python 5 . Experiments were run on a 6-Core Intel Core i5 and NVIDIA Titan RTX. The plotted lines show the median test statistic over 100 trials for both CCM and TSCI, and the shaded region indicates the 5th and 95th percentiles when (a) C is varied from 0 (no coupling) to 3 (approximate general synchrony), and (b) C is fixed to 1.0 and the library length is varied.A common toy system used for studying coupled dynamic systems is a unidirectionally-coupled Rössler-Lorenz system [28] , which we define in Eq. ( 13). In this system, the first three coordinates (z 1 , z 2 , z 3 ) describe a Rössler system, and they are unidirectionally coupled with a Lorenz-type system (z 4 , z 5 , z 6 ). The strength of the coupling is controlled by the parameter C. When C = 0, the two systems are disconnected, but for C > 0 there is a causal influence from z 1 , z 2 , z 3 to z 4 , z 5 , z 6 . dz dt = d dt        z 1 z 2 z 3 z 4 z 5 z 6        =        -6(z 2,t + z 3,t ) 6(z 1,t ) + 0.2zEQUATIONIn Fig. 2 , we visualize the TSCI method for x = z 2 and y = z 4 . First, we show the shadow manifolds M x and M y with a set of tangent vectors on each manifold. The delay embedding dimension parameters, Q x and Q y , were selected using the false-nearest neighbors algorithm with a tolerance of 0.005 [19] .Time lags for the embeddings, τ x and τ y , were selected by picking the minimal delay such that the autocorrelation function drops below a threshold [21] . Additionally, we show the histograms of cos(θ), where θ is the angle between the tangent vectors at each point. For X → Y direction, which is the true causal direction, the distribution is concentrated near 1. For the Y ← X direction, the distribution of tangent vectors is centered on 0. The test statistics, r X→Y and r Y →X , correspond to the means of each distribution, and visibly correspond to the correct causality.We show the effects of varying the coupling strength C in Fig. 3a , where TSCI clearly shows better separation across varying C. We see the effect of increasing the library length (i.e., the size of the training set in nearest neighbors) for C = 1.0 in Fig. 3b . Here, r X→Y increases at similar rates for TSCI and CCM, suggesting similar data efficiency of the two methods.To illustrate the generality of TSCI within CCM-like frameworks, we applied the TSCI methodology to the latent CCM framework, where CCM is applied to a state-space reconstruction obtained via neural ODEs [12] . One reason to favor this approach is when the observed time series are irregularly (i.e., with a non-uniform sampling rate) or sporadically (i.e., any given observation only measures a subset of states) sampled. Notable for TSCI is the fact that ground-truth derivatives are available, as they are directly learned in the neural ODE reconstruction. We replicate an experiment in the latent CCM paper where a network of unidirectionally coupled double pendulums are simulated, and observations sampled irregularly and sporadically. The authors' source code 6 was used to generate data and apply latent CCM. For more information on the data generation, see Appendix A of [12] .For TSCI, we learn a cross-map using an MLP between the reconstructed state spaces. To avoid tuning learning rates for every network, the parameter-free COCOB optimizer [26] 7 and networks were trained for 50 epochs. Results can be found in Table 1 .When compared to latent CCM, TSCI provided larger correlation coefficients for the true positive cases, with similarly small coefficients for the true negative cases. The use of MLPs for learning the cross map partially but not fully explains the difference in performance.Several additional experiments appear in Appendix B. We briefly describe them here and provide some commentary on their results.Varying the Embedding Dimension In Appendix B.1, we artificially lower quality embeddings by varying the embedding dimension. We find that the performance of TSCI and CCM similarly degrade when using an embedding dimension that is very small, and that there is little harm to a larger embedding dimension. In either case, the hyperparameters from a false-nearest neighbors test perform well.Corrupting Signals In Appendix B.2, we lower the embedding quality in two different ways: (1) by injecting additive noise to all observations, and (2) by adding a sine wave to observations. We observe that TSCI and CCM similarly degrade when the signal-to-noise ratio is altered. However, because of the larger separation in TSCI, the correct causal relation can be determined at much lower signal-to-noise ratios.Using Mutual Information In Appendix B.3, we experiment with using mutual information instead of cosine similarity. We find that conclusions made using MI are generally similar to those from cosine similarity, but with less interpretability and significant difficulties in estimation.As mentioned in the introduction, CCM (and hence TSCI) are specifically designed to address failures of Granger causality. In Appendix B.4, we empirically verify the limitations of Granger causality on our Rössler-Lorenz toy system. We additionally test several other causal discovery methods and show their limitations in our setting.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes]Justification: We have taken care to provide precise mathematical statements and proofs.For proofs which we do not provide (like the proof of Takens' theorem), we reference other sources where the proof may be found.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
advantages and limitations of tsci: 
 Scalability. The TSCI approach, using the K-nearest neighbors algorithm, retains the scalability and lightweight implementation that CCM enjoys. The only additional computational complexity arises from solving a local linear system of equations, and from the estimation of derivatives. Since the linear systems have a fixed size K, and since derivative estimation can be done using a linear filter, the additional cost is minimal. The general version of TSCI will scale depending on the choice of the regressor used to learn the cross map, and on other design decisions made to improve the efficacy of the method. However, this is not unlike the situation in which CCM is augmented with other customization options that potentially slow down the method.Model agnosticism. Because a TSCI test can be formulated for any differentiable regression model used to learn the cross map, the approach is highly flexible and model agnostic. Additionally, because there are many potential ways in which reconstruction of latent states and learning of the velocity vector fields can be improved, the TSCI method can be incorporated into a wide variety of inference frameworks.Quality of reconstructed states. In both CCM and TSCI, there are a few assumptions which warrant some justification before use, since their violation may yield to misapplication of cross map methods. The first issue is that while Takens' theorem implies that embeddings are plentiful, not all embeddings are equal in quality or useful. Shadow manifolds that are sparsely or incompletely sampled, time series with trends, or otherwise data which do not accurately capture the latent manifold can lead to dubious results using CCM [6] . While TSCI is less likely to produce a false positive in these cases, the assumption that a latent manifold is well-represented by a given embedding is nontrivial and critical to ensuring trustworthy performance of the method.General synchrony. General synchrony is a problem that plagues all cross map-based methods [41] .The issue is that when the causal strength of the relationship x → y is very strong, the influence of z x dominates the dynamics of z y , and z y cannot exhibit its own independent behavior. As a result, M y will look similar to M x , and the cross map will become an invertible function. As a result, a strong unidirectional relationship is detected as a bidirectional causal relationship. In Fig. 3 , the Rössler-Lorenz system enters general synchrony near C = 3 [6] . The TSCI method appears more resistant to the effects of general synchrony than CCM, but it is a topological fact that as C grows, synchrony becomes inevitable.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: We have worked to make our results as reproducible as possible. We have provided a full description of the TSCI algorithm so that a reader should be able to implement and reproduce our method from the written manuscript. The design of the experiments has also be described within reason, so that any factor that might determine their outcome has been detailed. For minor technical details (such as the type of ODE solver used to generate data) which are less likely affect the results of the experiment, we refer to the code included with the paper. 
conclusion: 
 In this paper, we presented the TSCI method for detecting causation in dynamical systems. By considering how tangent vectors map from one manifold to another, we may achieve more robust detection of causal relationships in dynamical systems than with standard CCM. Key advantages of TSCI include that we may use it in many systems in which CCM would be applied, but the method is far less prone to false positives and spurious causation. We presented both a general form of the algorithm as well as a k-nearest neighbor version inspired by the original CCM algorithm. Because TSCI requires us to estimate latent states and their time derivatives, there us much room for the TSCI method to be further developed in future work.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: Code to reproduce our results is being included with the publication, with MIT license. The code is reasonably organized and documented, and instructions on how to use the code have been included with the code itself. When making use of other code bases for comparison (e.g., with latent CCM), a link to the original code base was provided.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: We have included all necessary details to understand and interpret the setup and results of the experimental section. Code is provided for the Rössler-Lorenz systems, and details on all networks, optimizers, and training details were provided for latent TSCI.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: In our experiments, we have provided suitable measures of the statistical confidence of our methods. In particular, we run over several folds and report either median/percentiles or mean/standard deviations of all results. Guidelines: • If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: The paper includes discussion of the potential broader impact of the work in the introduction. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper does not pose any such risks for misuse. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We have provided in-text mentions and citations for all assets or methods that we used in this work.Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained? Answer: [NA] Justification: The current paper does not require IRB approval as it does not contain research with human or animal subjects. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: In this appendix, we review some important background about manifolds. Much more complete references include the authoritative [22] (especially chapters 1, 2, 3, and 8), or the more accessible [24] .We say that a function F is smooth if it has derivatives of all orders. A diffeomorphism is a function F which is smooth, invertible, and F -1 is also smooth. We say that two subsets M and N of R n are diffeomorphic if there exists a diffeomorphism such that F (M ) = N .A subset M of R n is called a d-dimensional manifold if for every point x in M , there exists an open set U x such that 1. There is a diffeomorphism F x : R n → U x .In this appendix, we include experiments and results omitted from the main text.One of the key parameters in both CCM an TSCI is the embedding dimension parameter Q used during reconstruction of the shadow manifold. Takens' theorem suggests that once Q reaches a certain value, the manifold is embedded and the improvement of the results should saturate in principle. Of course, in real life there is a curse of dimensionality associated with inference, so it is interesting to check how well the results behave as the embedding dimension is chosen to be higher than necessary. We consider the effect of varying the parameter Q using data from the Rössler-Lorenz system in Fig. 4 . Performance of both methods improve as the dimension of the putative effect (if we are testing x → y, then this is Q y ) is increased, until saturating after the manifolds are fully embedded.Generally, there are many non-trivial ways in which the state reconstruction can be poor, e.g., presence of periodic trends [8] and underexploration of the shadow manifold [6] . We consider two scenarios that deteriorate the state reconstruction quality on the Rössler-Lorenz system: additive noise, and a periodic trend. Results for additive noise are presented in Fig. 5 . We fixed C = 1.0 and varied the signal-to-noise ratio (SNR) from 0 dB to 60 dB by adding Gaussian noise to the observed signals. When estimating derivatives, a Savitzky-Golay filter [34] was used with a window length of 5 and order 2.Overall, we find the performance relative to SNR to be similar between CCM and TSCI, suggesting that inference of the vector field from noisy data is not a limiting factor in the presence of additive noise. An alternative way to corrupt signals is with a deterministic periodic trend; we used a sine wave with a period of 2π, and varied its power with respect to the true signal. Results are presented in Fig. 6 . Our takeaways are relatively similar to the additive noise experiment: TSCI seems to degrade similarly to CCM, but its increased separation between cause and effect makes this degradation in performance less problematic. Notably, TSCI seems significantly more robust to false claims of strong causation when the relative power of the confounder is large. One interesting alternative to the cosine similarity score is the mutual information (MI), which provides a non-linear quantification of how related u and J F v are. One immediate concern with using MI is that estimates from finite samples are difficult; we use the classical estimator of Kraskov, Stögbauer, and Grassberger [20] , which has shown relevance even in the era of deep learning [10] .Results on the Rössler-Lorenz test system while varying the coupling parameter C are presented in Fig. 7 . We find that MI typically shows less separation than TSCI, which brings into question its viability when the ground truth is unknown. Another issue is in the interpretation of MI estimates: it is not entirely clear if an MI estimate of, for example, 1 nat indicates strong influence, and MI estimates are not easily normalized in the continuous setting. • The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.Although not typically interpreted as such, cross maps have been suggested to be interpretable in a Pearlian causal framework via the do operator[12].Several version of Takens' theorem exist, and they make useful, but mathematically distinct, statements about how common such embeddings are. The version presented here says that functions that do not produce embeddings live on a measure zero set, in the sense of prevalence[33, p. 584].See Appendix A for definitions of differential geometric quantities such as the tangent space.https://github.com/nickc1/skccm, MIT Licensehttps://github.com/edebrouwer/latentCCM, MIT Licensehttps://github.com/bremen79/parameterfree, MIT License 


Decision: This submission was found to tackle an important problem that receives growing attention, clearly highlights potential pitfalls and difficulties with CCM, and develops a novel algorithmic idea to determine causality in dynamical systems. It was found to be clearly written and well structured such that it is mostly easy to read and follow. I do not believe that a comparison with "static" bivariate causal discovery algorithms is required and agree with the authors that no meaningful contributions can be drawn from such a comparison. I encourage the authors to please add their main clarifying arguments from the rebuttal to the camera ready version and to potentially also think about and address some of the minor remaining concerns of the reviewers.

Reviews: review: paper_summary: paper_summary: The authors propose the Tangent Space Causal Inference (TSCI) method for detecting causalities in dynamic systems. TSCI works by considering vector fields as explicit representations of the systems’ dynamics and checks for the degree of synchronization between the learned vector fields. The authors present both a basic TSCI algorithm, which is lightweight and more effective than the basic CCM algorithm, as well as augmented versions of TSCI that leverage the expressive power of latent variable models and deep learning. The authors demonstrate improved causal inference performance across a number of benchmarks.
strength_weakness: strength_weakness: a. The main weakness of the paper is that the experiments are insufficient to validate the method. Comparison with existing standard methods is missing. Experimental comparisons with classical Granger causal discovery and other methods are necessary. 

b. Theoretical analysis and comparison with existing mechanisms and methods should be included.

c. Apart from comparison with classical time series causal discovery methods, comparison with bivariate causal discovery should be included to strengthen the paper. E.g. papers [1-7]


[1]Blöbaum, P., Janzing, D., Washio, T., Shimizu, S. and Schölkopf, B., 2018, March. Cause-effect inference by comparing regression errors. In International Conference on Artificial Intelligence and Statistics (pp. 900-909). PMLR.


[2]Khemakhem, I., Monti, R., Leech, R. and Hyvarinen, A., 2021, March. Causal autoregressive flows. In International conference on artificial intelligence and statistics (pp. 3520-3528). PMLR.

[3]Ren, S. and Li, P., 2022, October. Flow-based perturbation for cause-effect inference. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management (pp. 1706-1715).

[4]Daniusis, P., Janzing, D., Mooij, J., Zscheischler, J., Steudel, B., Zhang, K. and Schölkopf, B., 2012. Inferring deterministic causal relations. arXiv preprint arXiv:1203.3475.

[5]Fonollosa, J.A., 2019. Conditional distribution variability measures for causality detection. Cause Effect Pairs in Machine Learning, pp.339-347.

[6]Ren, S., Yin, H., Sun, M. and Li, P., 2021, December. Causal discovery with flow-based conditional density estimation. In 2021 IEEE International Conference on Data Mining (ICDM) (pp. 1300-1305). IEEE.

[7]Hoyer, P., Janzing, D., Mooij, J.M., Peters, J. and Schölkopf, B., 2008. Nonlinear causal discovery with additive noise models. Advances in neural information processing systems, 21.
questions: questions: What are the advantages of using corr(xˆ(t), x(t)) compared to Granger causal discovery, and the regression or conditional variance-based methods[1,3,6]?
limitations: limitations: Experimental study and comparison with additional baselines should be included to support the proposed method.
paper_summary: paper_summary: The authors propose a novel statistic for detecting causality in dynamical systems, which overcomes a key conceptual difficulty in the convergent cross mapping (CCM) method from Sugihara et al. [2012]. CCM is justified by the existence of a surjective "cross map" from the delay embedding of a driving to a driven dynamical variable in unidirectional coupling. However, the CCM test statistic is difficult to interpret because it does not directly measure whether this map can be constructed. 

Instead, the authors represent the dynamical equations through the time invariant flow fields induced by these delay embeddings. A cross map would induce a Jacobian transformation between these flow fields, so the authors propose testing the correlation between the flow fields of a candidate "driven" variable and a candidate "driving" variable pushed forward by the estimated Jacobian. 

The authors demonstrate the clear advantage of their method for distinguishing the direction of causality in a unidirectional Rössler-Lorenz system.
strength_weakness: strength_weakness: The authors point out that the CCM statistic does not admit a simple decision rule and refer to their correlation coefficients as test statistics. However, the authors do not offer guidance as to when, e.g., a two-body dynamical system should be considered unidirectionally coupled. The authors could explore testable hypotheses such as "$H_0$: there is no driving $x \Rightarrow y$", perhaps over a limited class of models. Perhaps this has been investigated in the CCM literature already and the authors could adapt existing methods for decision making. 

The authors could also address their choice of correlation as their test statistic. Perhaps replacing correlation coefficients with mutual information $I(\mathbf{u};\mathbf{J}_F \mathbf{v})$ should be acknowledged as a possible future refinement in the conclusion (see the first question below).
questions: questions: How do the authors interpret a measured Jacobian that cannot predict the correct magnitude of $\mathbf{v}(\mathbf{\tilde{y}})$ from $\mathbf{u}(\mathbf{\tilde{x}})$ but correctly predicts the direction? This case, of course, gives the TSCI test statistic value $1$. 

How do the authors interpret their results if the measured Jacobian is invertible? Is this simply the case in which $x(t)$ and $y(t)$ are driven by the same set of variables $\mathbf{z}$?
limitations: limitations: Yes. The authors adequately describe underlying assumptions and describe challenges (i.e., generalised synchrony) that obstruct informative results.
paper_summary: paper_summary: This paper proposes a novel tangential space causal inference method, TSCI, for identifying causal relationships from time series data generated by dynamical systems. TSCI considers vector fields as explicit representations of dynamic systems and checks for the degree of synchronization between the vector fields to identify the causal direction. TSCI has higher interpretability and scalability than the original CCM. Experiments on standard systems show that TSCI is much easier to identify causal directions than CCM.
strength_weakness: strength_weakness: 1.	It may be unreasonable to use Pearson correlation coefficient in this paper to evaluate the correlation between tangent vectors. As far as I know, Pearson correlation coefficient can only evaluate linear correlation, if there is nonlinear correlation between tangent vectors, then Pearson correlation coefficient may not work well.
2.	The experiments in this paper are not convincing. It only shows the results of algorithms identifying the causal relationships between a few variables in the toy system. It is recommended to evaluate TSCI in real systems with larger scale data. In addition, the results in this paper are not sufficient to show that TSCI is superior to CCM, because CCM also identities correct causal relationships. The authors should provide results that TSCI can correctly identify causality and CCM cannot.
3.	This paper claims that TSCI is lighter and more scalable than CCM, but lacks the necessary results support.
4.	This paper assumes a high quality of the reconstruction of the latent states, which may not be reasonable. As the article says, a high-quality reconstruction state does not always exist. It is recommended to conduct experiments with low latent state quality, and the robustness of TSCI and CCM for this problem has been compared.
5.	The experiments in this paper lack necessary analysis. For example, the authors should further explain why TSCI is more resistant to the effects of general synchrony than CCM.
questions: questions: 1.	How does TSCI perform on real systems? Can the authors demonstrate the performance of TSCI in identifying causal relationships with large scale data?
2.	Do the authors consider cyclic causal relationships? For example, if there are cyclic causal relationships X->Y->Z->X, can TSCI correctly identify them?
3.	Can TSCI correctly identify causality when the quality of reconstruction of latent states are low?
4.	Did the authors consider nonlinear causality? The Pearson correlation coefficient may not be effective in identifying nonlinear causal relationships.
5.	In Figure 2, why cos(\theta) in the correct direction not equal to -1?
limitations: limitations: The authors do not directly state the limitations in the paper. This work is limited by the quality of reconstruction of latent states, which is a difficult problem to solve, and the authors do not report the robustness of the proposed method for this problem.
paper_summary: paper_summary: This paper presents a novel algorithm to perform causal inference in time series data based on the idea of convergent cross mappings (CCMs). Similar to Granger causality, the CCM paradigm infers causation by testing whether there exists a map (i.e., a predictor) from the reconstructed (unobserved) state trajectory of the effect variable to the reconstructed state trajectory of a cause variable. However, TSCI proposes to compare the (correlations of the) vector fields of the reconstructed dynamical systems rather than the reconstructed trajectories, as done by classical CCM. The approach is validated in two, low-dimensional synthetic examples.
strength_weakness: strength_weakness: The explanation of why the correlation coefficient over tangent vectors is better than over trajectories (in CCM) is shaky and needs to be made more precise (last paragraph of Section 2). Specifically, it would be great if the work could expand on the argument and reasoning behind: “since the points being predicted in CCM live on a manifold, measuring correlation in the ambient (extrnistic) space is not well-motivated (l. 205-206)”. Why is it not well-motivated? Given this, the work states “the tangent planes [of TSCI are] geometrically motivated” (l. 209-210). Please elaborate more here as well. In my eyes, this paragraph lies at the heart of the differences/(dis)advantages of CCM vs TSCI, so it would be great if this discussion is expanded and made more explicit.

The experimental evaluation of the work can be improved. First, as acknowledged, e.g. in lines 180-183, TSCI requires accurate embeddings/reconstructions of the state space trajectories to work well. However, none of the experiments study how the performance of TSCI and classical CCM behave/degrade as reconstruction quality decreases, which seems like a fundamental component of applying either method in practical applications. 
In addition, in none of the experiments, CCM or latent CCM (i.e., existing works) fail to identify or misidentify the true causal effects. Thus, all baselines and TSCI achieve perfect accuracy, which suggests that the experiments may be too easy? The example systems here are very low dimensional, and the library length very long (e.g. Fig 3b). How does the performance compare in larger dimensionalities, where certain edges may be misclassified? The experiments do not distinguish which method is the better one.

Algorithms 1 and 2 are stated for 2-variable dynamical systems (with variables X and Y). Is the extension to multivariate systems simply done by applying TSCI to all pairs of variables in the system (e.g. in the experiments of Section 4.2)? Is there a curse of dimensionality involved when comparing vector fields, because we have to sample tangent vector points on the manifold? The work states that “[CCM] estimates [are] biased by the distribution of observations along the manifold” (l. 206-)”. If TSCI does not have this bias, it must sample from the whole manifold, or am I getting this wrong?
questions: questions: Some of these questions do not concern TSCI specifically, but CCM in general, and it would improve the quality of the paper if they are explained more explicitly throughout the paper. I am not an expert specifically on Granger causality and CCM, so it would be very helpful to the reader to highlight key differences more clearly.

-	It would be great if Section 2.1. provided some basic mathematical definition and/or background on manifolds, to the degree that is needed to understand the paper. The notion of manifolds appears repeatedly, so some basic intuitions would help a lot (e.g., how do you define a manifold defined? Is a compact subspace of Euclidean space also covered by Taken’s theorem? How does this concept apply, e.g., to the experiments done in Section 3?)

-	Why is it the case that “the angle […] is the same regardless if it is computed from intrinsic or extrinsic coordinates”? (l. 212)

-	What is the fundamental difference, especially conceptually and algorithmically, between CCM and Granger causality? It seems that both ideas are based on learning predictors from one time series to the other. Is the main difference the direction of the predictor (anticausal for CCM, causal for Granger) or the fact that CCM-based methods work with reconstructed trajectories?

-	In general, what are the identifiability conditions or theoretical guarantees of CCM? Can we always infer all causal dependencies in the infinite sample limit for arbitrarily large dynamical systems? From the causal discovery/causal structure learning perspective, this seems like a “free lunch” situation and too good to be true. What assumptions on the class of dynamical systems are needed to guarantee identifying the causal dependencies?

-	Could you give more explanation for why this is the case in the introduction? “… for many systems, namely dynamical systems, the separability condition [of Granger causality] is violated” (l. 32-33)
limitations: limitations: The paper is doing a good job at discussing possible disadvantages of the methods in Section 4. For other limitations, see weaknesses/questions above.

Comments: title: Thank you
Response for Rebuttals
comment: We thank the reviewer for their response.

We repeat that the application of TSCI to arbitrary bivariate causal discovery data sets is unjustified. Without assuming an underlying dynamical manifold structure, there is no theoretical basis as to why CCM or TSCI should yield the correct causal truth. This is entirely analogous to the invalidity of RECI/IGCI/EFRE when no deterministic function $y = f(x)$ exists. It is unclear what the scientific value of applying TSCI to a bivariate causal discovery data set (e.g., the Tuebingen data) would be when there is no inherent time series structure, and the TSCI method is specifically intended for time series generated by dynamical systems. 

The theoretical advantage of TSCI and CCM, as cross mapping methods, is that they address scenarios that may be considered pathological by traditional causal discovery methods. It is well known that the separability assumption of Granger causality is violated in deterministic dynamical systems. Static causal discovery methods cannot account for spurious correlations that may occur due to autocorrelation or non-stationarity in time series data. Furthermore, static methods such as RECI, IGCI, and EFRE, make specific assumptions about the existence of a deterministic function $y=f(x)$, which are clearly violated in the time series we are interested in. 

In the PDF attached to the main author rebuttal above, we also show empirically Granger causality, RECI, IGCI, and ANM fail in the Rossler-Lorenz system. This is not a criticism of these methods, but rather a reminder that TSCI and CCM are methods specifically designed to addressthe blind spots of many traditional causal discovery techniques.
We thank the reviewer once again for their feedback and engagement with our article. We now reply to the reviewer's remaining comments.

(1) The primary issue with finding real data sets is that it is a rather particular scenario to find data which both has a nontrivial causal ground truth as well as having data with some underlying dynamical manifold structure (in the sense of Takens' theorem). While some articles applying CCM and related methods to real data exist, many of these data sets cannot be accessed without permission. Additionally, the vast majority of papers on CCM and related methods overwhelmingly study toy systems and synthetic data sets, because finding clean data with manifold structure is so rare.

Of the few articles that use real data sets to test CCM, they typically study data that has no verifiable ground truth, so even in comparison to TSCI we would be unable to verify which method is performing better. There is also somewhat of a survivorship bias present: datasets that CCM has been applied to are ones that CCM performs well on. That said, we have been and will continue to look for real data sets where we can justify the manifold structure assumption. 

Gernerally, it would be unjustified to apply CCM or TSCI to a data set without manifold structure. To make an analogy, one would not apply a graph neural network to a data set that doesn't have some inherent graph structure. Regardless if the model produces the correct result or not, we have no theoretical guarantee that the result was meaningful unless the structure hypothesis is assumed. 

One partial solution is to consider more sophisticated synthetic systems. For example, \[1\] considers a large data set of synthetic neural data. Generating synthetic data like this is advantageous because it is reproducible, and the scale of the system can be varied. If this is acceptable, we will incorporate additional experiments of this nature into the revised manuscript. 

If the reviewer is aware of any potential real data sets that would be appropriate, we will gladly incorporate experiments using them into the manuscript. It is our intention to fairly prove the robustness of the TSCI method, but after searching for some time, we are yet to find real data sets that would provide meaningful comparison between CCM and TSCI.

(2) Thank you for specifially pointing out the abstract. While we believe that what we wrote is correct, we understand that it may be easy to misinterpret. In the original abstract, we made two separate claims: (1) TSCI is a lightweight method in general, and (2) TSCI is more effective as an algorithm than CCM. In the revised manscript, we have modified the abstract to better clarify that these two claims are distinct. Specifically, the relevant part now reads:
"We first present a basic version of the TSCI algorithm, which is shown to be more effective than the basic CCM algorithm with very little additional computation. We additionally present augmented versions of TSCI that leverage the expressive power of latent variable models and deep learning."

Both TSCI and CCM can be implemented with a time complexity of $O(T \log(T))$ using KDTrees, where $T$ is the time series length. This implies that TSCI and CCM have comparable scaling, and both are lightweight. 

Our claim that TSCI is more effective than CCM is unrelated to the computational complexity, and is based on results of the experiments, where TSCI  outperforms CCM in accuracy. 



\[1\] E. De Brouwer, et al. "Latent convergent cross mapping." International Conference on Learning Representations (ICLR). 2021.
We again thank the reviewer for their feedback and engagement with our article. 

We are grateful for the reviewer's suggestions and are incorporating all of them into the revised manuscript.
We thank the reviewer once again for their thoughtful suggestions and positive view of our manuscript. 

We agree that it would be reasonable to try using the Euclidean distance between vector fields. As the reviewer mentioned, this would be consistent with the theory developed in the manuscript, and is straightforward to implement. One disadvantage of this approach, like the MI approach, is that it is unclear what principle should be used to derive a general threshold. However, we found in a simple experiment that there is still a clear difference in the distributions obtained from Euclidean distances in each direction. Our primary preference for cosine similarity arises from (1) its interpretability and (2) the ease of finding heuristics to select a threshold. For example, selecting 0.8 as a desired cosine similarity is about as arbitrary as selecting 80% variance explanation in principal component analysis. In any case, our preference does not preclude the use of alternative measures, such as the Euclidean distance or the MI, and we find these to be very interesting directions for future research.

In the revised manuscript, we added an appendix section to provide examples of alternative test statistics for the TSCI and to compare them to the cosine similarity. For now, this section includes the MI and the Euclidean distances as suggested by the reviewer.
Many thanks to the authors for their thoughtful responses. 

The difficulty of decision rules in this case is indeed intriguing! 

While I remain unconvinced that cosine similarity is an "ultimate" measure of the similarity between $\mathbf{u}$ and $\mathbf{J}_F \mathbf{v}$, I see now that mutual information is an inappropriate suggestion (perhaps a normalised version would fix this). The authors are also right to point out that cosine similarity is more straightforward to estimate than MI. 

As a somewhat late suggestion, the authors may wish to test an averaged Euclidean distance between the vector fields, which should be computationally tractable and more precisely measures the condition arising from lemma 2.1.1. (which, as far as I can tell, concerns their similarity in both magnitude and direction).

On perspectives expressed by reviewers about the scope of this paper, I would argue that the paper being explicitly in conversation with CCM---a popular approach with a very particular notion of "causality"---is a strength rather than a weakness! I feel strongly that this paper is a valuable contribution to the dynamical systems literature and has important and immediate applications. I maintain my score at 7: Accept.
Thank you for your reply and additional explanations. I will maintain my current score and suggest that the authors add the above explanations to an updated version of the paper.
Thank you for your response that addressed most of my concerns, and I have raised my score. But I still have some comments and questions:\
1.Although it is difficult to find real datasets that satisfy the assumptions of Taken’ theorem, it is suggested to evaluate TSCI on other general real datasets, especially on larger real datasets, which is conducive to proving the robustness of TSCI.\
2.The authors state in abstract that they present a basic TSCI algorithm, which is lightweight and more effective than the basic CCM algorithm, which is contradictory to their response. Is TSCI as lightweight as CCM or lighter than CCM? If TSCI is more lightweight, please give the explanation and experimental results to support this conclusion.
Thank you for the response.  The authors failed to provide a sufficiently direct and convincing comparison between bivariate causal discovery methods and the proposed method, TSCI.  Though the proposed TSCI is targeting causal discovery in dynamic systems, with powerful backbone models, e.g. neural networks, I believe the bivariate causal discovery methods, such as RECI[1], and the deep generative model-based methods CAREFL[2], EFRE[3], retain the power to capture the causal relationship between dynamic systems.  The authors need to provide a comprehensive study to compare these methods using both dynamics systems data and real-world bivariate causal discovery datasets, e.g. Tuebingen dataset. 

The theoretical advantage of the proposed method is not clear to the reviewer.  Existing bivariate causal methods may also be able to be applied to the case X->Y, where -> is a dynamic system or process.  The authors should provide convincing experimental study and analysis, and theoretical analysis to demonstrate the advantage of the proposed method. 

Therefore, I will keep the score unchanged. 

[1]Blöbaum, P., Janzing, D., Washio, T., Shimizu, S. and Schölkopf, B., 2018, March. Cause-effect inference by comparing regression errors. In International Conference on Artificial Intelligence and Statistics (pp. 900-909). PMLR.

[2]Khemakhem, I., Monti, R., Leech, R. and Hyvarinen, A., 2021, March. Causal autoregressive flows. In International conference on artificial intelligence and statistics (pp. 3520-3528). PMLR.

[3]Ren, S. and Li, P., 2022, October. Flow-based perturbation for cause-effect inference. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management (pp. 1706-1715).