Latent Convergent Cross Mapping

1 INTRODUCTION . Inferring a right causal model of a physical phenomenon is at the heart of scientific inquiry . It is fundamental to how we understand the world around us and to predict the impact of future interventions ( Pearl , 2009 ) . Correctly inferring causal pathways helps us reason about a physical system , anticipate its behavior in previously unseen conditions , design changes to achieve some objective , or synthesize new systems with desirable behaviors . As an example , in medicine , causality inference could allow predicting whether a drug will be effective for a specific patient , or in climatology , to assess human activity as a causal factor in climate change . Causal mechanisms are best uncovered by making use of interventions because this framework leads to an intuitive and robust notion of causality . However , there is a significant need to identify causal dependencies when only observational data is available , because such data is more readily available as it is more practical and less costly to collect ( e.g. , relying on observational studies when interventional clinical trials are not yet available ) . However , real-world data arising from less controlled environment than , for instance , clinical trials poses many challenges for analysis . Confounding and selection bias come into play , which bias standard statistical estimators . If no intervention is possible , some causal configurations can not be identified . Importantly , with real-world data comes the major issue of missing values . In particular , when collecting longitudinal data , the resulting time series are often sporadic : sampling is irregular ⇤Both authors contributed equally †Corresponding author in time and across dimensions leading to varying time intervals between observations of a given variable and typically multiple missing observations at any given time . This problem is ubiquitous in various fields , such as healthcare ( De Brouwer et al. , 2019 ) , climate science ( Thomson , 1990 ) , or astronomy ( Cuevas-Tello et al. , 2010 ) . A key problem in causal inference is to assess whether one temporal variable is causing another or is merely correlated with it . From assessing causal pathways for neural activity ( Roebroeck et al. , 2005 ) to ecology ( Sugihara et al. , 2012 ) or healthcare , it is a necessary step to unravel underlying generating mechanisms . A common way to infer causal direction between two temporal variables is to use Granger causality ( Granger , 1969 ) , which defines “ predictive causality ” in terms of the predictability of one time series from the other . A key requirement of Granger causality is then separability ( i.e. , that information about causes are not contained in the caused variable itself ) . This assumption holds in purely stochastic linear systems , but fails in more general cases ( such as weakly coupled nonlinear dynamical systems ) ( Sugihara et al. , 2012 ) . To address this nonseparability issue , Sugihara et al . ( Sugihara et al. , 2012 ) introduced the Convergent Cross Mapping ( CCM ) method , which is based on the theory of chaotic dynamical systems , particularly on Takens ’ theorem . This method has been applied successfully in various fields such as ecology , climatology ( Wang et al. , 2018 ) , and neuroscience ( Schiecke et al. , 2015 ) . However , as the method relies on embedding the time series under study with time lags , it is highly sensitive to missing values and usually requires long uninterrupted time series . This method is thus not applicable in settings with repeated short sporadic time series , despite their occurrence in many practical situations . To address this important limitation , we propose to learn the causal dependencies between time series by checking the existence of convergent cross mappings between latent processes of those time series . Using a joint model across all segments of sporadically observed time series and forcing the model to learn the inherent dynamic of the data , we show that our method can detect causal relationship from short and sporadic time series , without computing delay embeddings . To learn a continuous time latent representation of the system ’ s state-space , we leverage GRU-ODE-Bayes ( De Brouwer et al. , 2019 ) , a recently introduced filtering method that extends the Neural ODE model ( Chen et al. , 2018 ) . Importantly for causal inference , the filtering nature of the model makes sure no future information can leak into the past . We then check the existence of continuous maps between the learnt latent representations and infer the causal direction accordingly . In a series of increasingly challenging test cases , our method accurately detects the correct causal dependencies with high confidence , even when fed very few observations , and outperforms competing methods such as multi-spatial CCM or CCM with multivariate Gaussian process interpolation . 2 RELATED WORK . CCM to address failure of Granger causality . Granger causality ( Granger , 1969 ) provided the first significant framework to infer causal dependencies from time series . Relying on predictability between dynamical systems , it was extended to account for different limitations , such as nonlinearity ( Chen et al. , 2004 ) or instantaneous relationships ( Schiatti et al. , 2015 ) . However , the assumption of separability of information between causative and caused variables leads to the failure of the Granger paradigm for a significant number of time series coupling scenarios ( Sugihara et al. , 2012 ) ( see Appendix D for a revealing worked out example ) . Convergent Cross Mapping , a technique based on nonlinear state space reconstruction was introduced to tackle this issue ( Sugihara et al. , 2012 ) . Recently , several works have proposed extensions of CCM , such as the extended CCM , to address issues such as synchrony ( Ye et al. , 2015 ) or to improve the discrimination of the confounding case ( Benkő et al. , 2018 ) . Synchrony occurs when one time series can be expressed as a function of the other ( e.g . Y ( t ) = ( X ( t ) ) and attractors of both dynamical systems become homeomorphic to each other ( Rulkov et al. , 1995 ) . This occurs when coupling between two chaotic system is too strong . Confounding , on the other hand , occurs when two variables are causally driven by a third one . In general we say that X confounds the relation between Y and Z if X causes both Y and Z. Huang et al . ( 2020 ) also proposed to predict directly the driving time series from the driven one with reservoir computing , bypassing the delay embedding step , making it more robust to noise . However , those methods still require long regularly sampled time series . Causality for short or sporadic time series . Short time series are very common in practice and there has been some work proposing to learn causality from short time series relying on state space reconstruction . Ma et al . ( 2014 ) proposed a method for short , fully observed , unique time series . Multi-spatial CCM ( Clark et al. , 2015 ) , considered the problem of inferring causality from several short fully observed snippets of the same dynamical system by computing delay embeddings compatible with the lengths of the time series and aggregating them . In comparison , on top of addressing irregular sampling , our approach computes more informative state-space representations by sharing a model across all segments . Techniques to infer causal direction from incomplete time series have also been proposed , but all are relying on the Granger causality framework , which limits their applicability to separable dynamical systems . They use direct partial correlations on regularly sampled data ( but with missing values ) ( Elsegai , 2019 ) or generalizations of similarity measures for sporadic time series ( Bahadori & Liu , 2012 ) . To the best of our knowledge , this is the first work investigating the identification of causal dependencies from short sporadic time series using state-space reconstruction . 3 METHOD . We consider the problem of inferring a causal dependency between two temporal variables from several segments of their multivariate time series X [ t ] 2 RdX and Y [ t ] 2 RdY . We assume that X [ t ] and Y [ t ] have been generated by an unknown dynamical system . In this work , we refer to the dynamical system of a time varying variable X as the smallest dynamical system that fully describes the dynamics of X . As an example , let ’ s consider the following system of ODEs representing the dynamics of X and Y : dX ( t ) dt = f ( X ( t ) ) ( 1 ) dY ( t ) dt = g ( X ( t ) ) + h ( Y ( t ) ) . ( 2 ) The dynamical system of X is given by Equation ( 1 ) . On the other hand , the dynamical system of Y is Equation ( 1 ) + ( 2 ) as Equation ( 1 ) is required to describe the dynamics of Y . To account for the more general and most frequent case , we consider those time series are only observed in segments of finite duration . X [ t ] and Y [ t ] then consist of collections of N short time series ( X1 [ t ] , ... , XN [ t ] ) ) and ( Y 1 [ t ] , ... , Y N [ t ] ) ) respectively . Importantly , each segment of X and Y is observed concomitantly . To proceed with a lighter notation , we ’ ll drop the superscript when referring to a segment of time series . Each of those time series is also sporadic namely the are not regularly sampled and not all dimensions are observed each time . In this work , we define the notion of causality by considering the equations of the dynamical system as a structural causal model . In this framework , X causes Y if p ( Y |do ( X ) ) 6= P ( Y ) where do ( X ) is an intervention on X ( Pearl , 2009 ) . Then , if X causes Y , X is part of the dynamical system of Y ( X is required to describe the dynamics of Y ) . In the case of the example described by Equations 1 and 2 , X causes Y if g ( · ) is not a constant function . 3.1 CONVERGENT CROSS MAPPING AND TAKENS ’ THEOREM . CCM aims at discovering the causal direction between temporal variables in dynamical systems by checking if the state-space dynamics of their time series can be recovered from one another . As shown above , if X causes Y , X is then contained in the dynamical system of Y and it should be possible to recover a representation of the dynamical system of X from the dynamical system of Y . A common way to obtain a representation of a dynamical system from its time series relies on Takens ’ embedding theorem ( Takens , 1981 ) . Let X [ t ] 2 RdX be issued from a chaotic dynamical system that has a strange attractor M with box-counting dimension dM , where we define an attractor as the manifold toward which the state of a chaotic dynamical system tends to evolve . The dynamics of this system are specified by a flow on M , ( · ) ( · ) : R⇥M ! M , where ⌧ ( Mt ) = Mt+⌧ and Mt stands for the point on the manifold at time index t. This flow is encoded in the ODE of the system . The observed time series X [ t ] is then obtained through an observation function fobs ( · ) : X [ t ] = fobs ( Mt ) . Takens ’ theorem then states that a delay embedding with delay ⌧ and embedding dimension k k , ⌧ , ↵ ( Mt ) = ( ↵ ( 0 ( Mt ) ) , ↵ ( ⌧ ( Mt ) ) , . . . , ↵ ( k⌧ ( Mt ) ) ) is an embedding of the strange attractor M if k > 2dM and ↵ : RdM ! R is a twice-differentiable observation function . More specifically , the embedding map is a diffeomorphism between the original strange attractor manifold M and a shadow attractor manifold M0 generated by the delay embeddings . Under these assumptions , one can then theoretically reconstruct the original time series from the delay embedding . The simplest observation function ↵ consists in simply taking one of the dimensions of observations of the dynamical system . In this case , writing Xi [ t ] as the i-th dimension of X [ t ] , Takens ’ theorem ensures that there is a diffeomorphism between the original attractor manifold of the full dynamical system and the shadow manifold M0 that would be generated by X 0 [ t ] = ( Xi [ t ] , Xi [ t ⌧ ] , . . . , Xi [ t k⌧ ] ) . To see how this theorem can be used to infer the causal direction , let us consider the manifold MZ of the joint dynamical system resulting of the concatenation of X [ t ] and Y [ t ] . We then generate two shadow manifolds M0 X and M0 Y from the delay embeddings X 0 [ t ] = ( Xi [ t ] , Xi [ t ⌧ ] , . . . , Xi [ t k⌧ ] ) and Y 0 [ t ] : ( Yj [ t ] , Yj [ t ⌧ ] , . . . , Yj [ t k⌧ ] ) . Now , if X unidirectionally causes Y ( i.e. , Y does not cause X ) , it means that X is part of an autonomous dynamical system and that Y is part of a larger one , containing X . The attractor of Y is then the same as the one of the joint dynamical system Z . By contrast , the attractor of X is only a subset of it . From Taken ’ s theorem , it is theoretically possible to recover the original MZ from M0Y and hence , by extension , recover M0 X from M0 Y . However , the contrary is not true and it is in general not possible to recover M0 Y from M0 X . The CCM algorithm uses this property to infer causal dependency . It embeds both dynamical systems X and Y and use k-nearest neighbors to predict points on M0 X from M0 Y and inversely . The result then consists in the correlation of the predictions with the true values . We write Ccm ( X , Y ) the Pearson correlation for the task of reconstructing M0 X from M0 Y , Ccm ( X , Y ) = Corr ( M0 X , M̂0 X ) where M̂0 X stands fro the prediction of M0 X obtained from M0 Y . Importantly , this measure is nonsymmetric as an non-injective map between M0 X and M0 Y would lead to an accurate reconstruction being possible in one direction only . To infer that there is a causal link between the predictor dynamical system and the predicted one , this correlation should be high and , importantly , increase with the length of the observed time series , as the observed manifolds become denser . The potential results are then interpreted in the following way ( 1 ) X causes Y if one can reconstruct with high accuracy M0 X from M0 Y ; ( 2 ) X and Y are not causally related ( but not necessarily statistically independent ) if nor M0 X nor M0 Y can be reconstructed from the other ; ( 3 ) X and Y are in a circular causal relation if both M0 Y and M0 X can be reconstructed from the other . In the extreme case of strong coupling , the two systems are said to be in synchrony , and it becomes hard to distinguish between unidirectional or bidirectional coupling ( Ye et al. , 2015 ) .

Summary: This paper studies short, chaotic time series and uses the Taken's theorem to discover the causality between two time series. The main challenge is that for short time series, the delay embedding is not possible. Thus, the authors propose to fit a latent neural ODE and theoretically argue that they can use the Neural ODE embeddings in place of the delay maps. The authors provide two sets of experiments, both on simulation data. Unfortunately, they never tested the algorithm on real data.