Optimal Transport for Causal Discovery

1 INTRODUCTION . Determining causal direction between two variables is a fundamental and challenging causal discovery task ( Janzing et al. , 2012 ) . Conventional constraint-based and score-based causal discovery methods identify causal structures only up to Markov equivalent classes ( Spirtes et al. , 2001 ) , in which some causal directions are undetermined . To address this challenge , properly constrained functional causal models ( FCMs ) have been proposed . FCMs represent the effect as a function of its cause and independent noise , and can help identify the causal direction between two variables by imposing substantial structural constraints on model classes , such as additive noise models ( ANMs ) ( Shimizu et al. , 2006 ; Hoyer et al. , 2008 ) and post-nonlinear models ( PNLs ) ( Zhang and Hyvärinen , 2009b ) . While some of the models , such as PNLs , are highly flexible , the constraints are still restrictive and difficult to interpret and relax . Inevitably , the performance of these methods is sensitive to model assumptions and optimization algorithms , especially in real-world applications . To handle the mentioned issues , we consider FCMs from a dynamical-system view . By augmenting a time dimension for the static causal discovery task , we interpret FCMs with dynamical causal processes under the least action principle ( Arnol ’ d , 2013 ) . The new interpretation connects FCMs with a large class of models in dynamical systems . It then provides more freedom to model causal influences , possibilities to derive new causal discovery criteria , and a potential direction to generalize causal models with identifiable causal direction . In particular , we exploit the above idea by leveraging the intrinsic connection between FCMs and optimal transport . Optimal transport is originally introduced by Monge ( 1781 ) , which has been ap- FIX ( GA77 , pBTi ) plied in a large range of applications , not only because it is a natural way to describe moving particles ( Ambrosio et al. , 2012 ) but also because of its recent improvement of the computational methods ( Cuturi , 2013 ; Kolouri et al. , 2019 ) . Recently , it has also been largely applied to generative models for measuring the distance of probability distributions ( Arjovsky et al. , 2017 ; Kolouri et al. , 2018 ; Genevay et al. , 2018 ) . Among different optimal transport definitions , the L2 Wasserstein distance got extensive applications in statistics ( Rachev and Rüschendorf , 1998 ) , functional analysis ( Barthe , 1998 ) , and multiple disciplines of science ( McCann , 1997 ; Otto , 1997 ) . The dynamical formulation of the L2 Wasserstein distance is introduced by Benamou and Brenier ( 2000 ) for relaxing the computational costs . We find that in the context of the dynamical formulation , FCMs can be connected with optimal transport . Furthermore , with the dynamical interpretation of optimal transport , one can naturally understand FCMs from a dynamical-system view , which makes it possible to derive new criteria to identify causal direction . Moreover , it also enables us to develop practical algorithms with optimal transport for static causal discovery tasks in the bivariate case without learning a regression model . Accordingly , our main contributions are : 1 . Dynamical interpretation of FCMs in the bivariate case . We provide dynamical interpretations of optimal transport under the constraints of FCMs . Furthermore , we introduce a time variable , determine the underlying dynamical process under the least action principle ( Arnol ’ d , 2013 ) for the static bivariate causal discovery task , and characterize properties of the corresponding dynamical systems ( Sec . 3.1 and Sec . 3.2 ) . 2 . A criterion of determining causal direction between two variables for ANMs . We study the corresponding dynamical systems of FCMs , and prove that ANMs correspond to volume-preserving pressureless flows . Moreover , based on the divergence of their velocity fields , we propose a criterion to determine causal direction , and show that under the identifiability conditions of ANMs , it is a valid criterion for ANMs , which can be extended to PNLs directly ( Sec . 3.2 ) . 3 . An optimal transport based approach ( DIVOT ) for distinguishing cause from effect between two . variables . DIVOT inherits the advantages of one-dimensional optimal transport . It can be computed efficiently , and does not require independence tests , learning a regression model , or deriving likelihood functions for complicated distributions . Experimental results show that our method is robust to the choice of models , and has a promising performance compared with the state-of-the-art methods on both synthetic and real cause-effect pair datasets ( Sec . 4 and Sec . 6 ) . 2 PRELIMINARIES . 2.1 OPTIMAL TRANSPORT . In this work we mainly use the L2 Wasserstein distance and we introduce its framework following the notations and definitions of ( Benamou and Brenier , 2000 ) in this section . FIX ( pBTi ) The underdetermined Jacobian problem . Suppose that two density functions , ρ0 ( x ) and ρT ( x ) where x ∈ Rd , are non-negative and bounded with total mass one . The transfer of ρ0 ( x ) to ρT ( x ) is realized with a smooth one-to-one map M : Rd → Rd . The Jacobian problem is to find M that satisfies the Jacobian equation : ρ0 ( x0 ) = ρT ( M ( x0 ) ) |det ( ∇M ( x0 ) ) | , xT =M ( x0 ) , ( 1 ) where det ( · ) denotes determinant . This is an underdetermined problem as many maps can be the solutions . Thus , a natural way is to choose the optimal one in the sense of lowest cost depending on applications . A common cost function is the Lp Wasserstein distance ( Kantorovich , 1948 ) . Lp Wasserstein distance and its one-dimensional closed-form solution . The Lp Wasserstein distance between p0 and pT , denoted by Wp ( p0 , pT ) , is defined by Wp ( p0 , pT ) p = infM ∫ |M ( x0 ) − x0|pp0 ( x0 ) dx0 , where p ≥ 1 . We say that the optimal transport M∗ achieving the infimum solves the optimal transport ( or Lp Monge-Kantorovich ) problem . In this work , we mainly use the square of the L2 Wasserstein distance , denoted by W 22 . Moreover , following the approach of Kantorovich ( 1948 ) , the one-dimensional ( 1D ) Lp Wasserstein distance has a closed-form solution , e.g. , the 1D optimal solution of W 22 is M∗ = P−1T ◦ P0 , ( 2 ) where P0 and PT are the cumulative distribution functions for p0 and pT , and “ ◦ ” represents the function composition . In practice , the 1D optimal solution in Eqn . ( 2 ) can be computed with the average square distance between the sorted samples from p0 and pT ( Kolouri et al. , 2019 ) . 2.2 FUNCTIONAL CAUSAL MODELS . A FCM represents the effect Y as a function f ( · ) of the direct cause X and independent noise Ey , where function f describes the causal influence of X on Y , and Ey is the exogenous vari- able/noise . Without any additional assumption on the functional classes , the causal direction is not identifiable ( Hyvärinen and Pajunen , 1999 ; Zhang et al. , 2015a ) . Roughly speaking , this is because given variable pair ( X , Y ) , one can always construct Y = f ( X , Ey ) and another different FCM , X = f̃ ( Y , Ex ) , such that both of them are consistent with the data distribution and have independent “ noise ” ( Hyvärinen and Pajunen , 1999 ; Zhang et al. , 2015a ) . Several works further introduce proper assumptions on the functional form of the FCMs and the distributions of the involved variables , and then identify causal direction in the bivariate case , i.e. , the independence of cause and noise only holds in the causal direction . Examples include the ANM and PNL models : ANM : Y = g ( X ) + Ey ; ( 3 ) PNL : Y = h ( g ( X ) + Ey ) , ( 4 ) where g and h are nonlinear functions and h is invertible . 3 DYNAMICAL INTERPRETATION OF FUNCTIONAL CAUSAL MODELS . We first show the connection between FCMs and optimal transport in Sec . 3.1 . In Sec . 3.2 , we further elaborate the analogy between the optimal transport problem and the causal direction determination problem . We then study the optimal transport under the constraints of FCMs , show the corresponding dynamical systems of FCMs , and characterize the properties of such systems . 3.1 CAUSALITY AND OPTIMALITY OF THE MAPS IN THE JACOBIAN PROBLEM . As introduced in Sec . 2.2 , FCMs are used to approximate the true data generation process . Given the FCM , Y = f ( X , Ey ) , we rewrite it in a Jacobian equation way , xT = [ X Y ] = [ Ex f ( X , Ey ) ] =M ( [ Ex Ey ] ) =M ( x0 ) , ( 5 ) where x0 , xT ∈ R2 , their probability densities p0 , pT ≥ 0 , and M : R2 → R2 . As an analogy to the mass transfer scenario ( Monge , 1781 ) , we consider the samples of independent noise Ex and Ey as the particles of materials in the mass transfer problem and regard the map M in Eqn . ( 5 ) as a special transformation of the independent noise samples . As shown in Fig . 1 , one can consider the data points are transferred from the original positions ( which are unmeasured ) in the plane Ex–Ey at time 0 to the observed positions in the plane X–Y at time T . Such transformation considers the transfer as a causal process which moves the unmeasured independent noise x0 = [ Ex , Ey ] ′ 1 and consequently leads to the observations xT = [ X , Y ] ′ . From the perspective of FCM-based causal FIX ( pBTi ) discovery approaches , causal influences are represented by FCMs which represent the effect as a function of its direct cause and an unmeasured noise satisfying the FCM constraints : 1 “ ′ ” denotes the transpose of vectors or matrices . ( i ) The map constraint : the values of X are determined by the values of its corresponding noise , i.e. , X = Ex , while the values of the effect depend on cause X and noise Ey ; ( ii ) The independence constraint : the noise terms are independent , i.e , Ex is independent of Ey . How is causality defined with the FCM constraints connected with the optimality of maps in optimal transport ? The optimal transport M∗ with the minimal Lp Wasserstein distance is not necessary to be the causal one given by Eqn . ( 5 ) , because it has no information about the FCM constraints or the true data generation process . In other words , given two sample sets of x0 and xT , the couplings FIX ( pBTi ) given by optimal transport are not necessary to be the ones generated from the ground-truth FCM . 3.2 DYNAMICAL INTERPRETATION OF FCMS : OPTIMAL TRANSPORT UNDER THE FCM CONSTRAINTS . In this section we jointly consider causality and optimality of the maps in the Jacobian problem . It provides both a causal sense of the transformation and a dynamical view of FCMs . We first recap the dynamical formulation of theL2 Wasserstein distance , and then study such dynamical systems under the FCM constraints . Furthermore , we show their properties under the FCM and ANM constraints . Dynamical interpretation of the L2 Wasserstein distance . Benamou and Brenier ( 2000 ) formulate the L2 Monge-Kantorovich problem as a convex space-time minimization problem in a continuum mechanics framework . Fixing a time interval [ 0 , T ] , they introduce the concepts of smooth time-dependent density ρ ( t , xt ) ≥ 0 and velocity fields v ( t , xt ) . When they are clear from context , we denote them by ρ and v. Because we are considering the bivariate case , xt ∈ R2 and v ∈ R2 . Then , they give the dynamical formulation of W 22 : W 22 ( p0 , pT ) = inf ρ , v T ∫ R2 ∫ T 0 ρ ( t , xt ) |v ( t , xt ) |2dxtdt , ( 6 ) s.t . { initial and final conditions : ρ ( 0 , · ) = p0 , ρ ( T , · ) = pT the continuity equation : ∂tρ+∇ · ( ρv ) = 0 . They prove that the solutions of ( 6 ) are pressureless potential flows , of which the fluid particles are FIX ( pBTi ) not subject to any pressure or force and their trajectories are determined given their initial positions and velocities or given their initial and final positions . Suppose that M∗ is the solution given by W 22 . The corresponding flows follow time evolution equation , xt = x0 + t T v ( t , xt ) , where v ( t , xt ) = v ( 0 , x0 ) =M∗ ( x0 ) − x0 and t ∈ [ 0 , T ] . ( 7 ) The time evolution equation shows that xt is just a convex combination of x0 and M∗ ( x0 ) . As an analogy between the optimal transport problem and causal direction determination , the density ρ and the velocity v of moving particles can be considered as the probability density and the velocity of changing values of data points . Moreover , the dynamical interpretation of the L2 Wasserstein distance introduces a time variable and provides a natural time interpolation ρ ( t , xt ) of ρ0 and ρT together with the velocity field v ( t , xt ) under the least action principle . Similarly , we can also have the natural time interpolation p ( t , xt ) between p0 and pT as well as the velocity field v ( t , xt ) under the least action principle , which is the dynamical interpretation of FCMs . Dynamical L2 Wasserstein distance under the FCMs constraints . First , we introduce FCM constraints in the context of the dynamical L2 Wasserstein distance . According to time evolution equation ( 7 ) , we know that the velocity is fully determined by the initial and final values of xt . We first consider FCM constraint ( i ) . For the initial and final values of the cause , the value of its observation X is equal to its noise value Ex in Eqn . ( 5 ) . Consequently , denoting v = [ vx , vy ] ′ , where vx and vy represent the velocities along the Ex/X-axis and Ey/Y -axis respectively as shown in Fi.g 1 , we know that ∀t ∈ [ 0 , T ] , xt ∈ R , vx ( t , xt ) = 0 . As for FCM constraint ( ii ) , it implies that the initial noise of cause and effect , corresponding to Ex and Ey at the time 0 , are independent . Therefore , we have the FCM constraints for the dynamical L2 Wasserstein distance , ( I ) The map constraints : v ( t , xt ) = [ vx ( t , xt ) , vy ( t , yt ) ] ′ , ∀xt , yt ∈ R , t ∈ [ 0 , T ] , vx ( t , xt ) = 0 ; ( II ) The independence constraint : Two random variables ( cause and effect ) at the initial time have the joint probability density function p0 ( x0 ) and they are independent . Second , we characterize the properties of the dynamical optimal transport ( L2 Wasserstein distance ) under constraints ( I ) and ( II ) . According to Eqn . ( 5 ) , the form of M∗ is determined as M∗ ( x0 ) = [ Ex f ( Ex , Ey ) ] ; ( 8 ) otherwise , the FCM constraints will be violated . Moreover , the two-dimensionalW 22 under the FCM constraints can be computed with the one-dimensional W 22 as shown in Prop . 1 ( the derivation is in App . D ) , Proposition 1 . Under constraints ( I ) and ( II ) the square of the L2 Wasserstein distance between p0 and pT is W 22 ( p0 , pT ) = EEx [ W 22 ( p ( Ey ) , p ( Y |Ex ) ) ] . ( 9 ) Furthermore , we consider the constraints of ANMs and characterize the corresponding dynamical systems for now , which can be directly extended to PNLs as mentioned in Sec . 4.2 . Based on constraints ( I ) and ( II ) , we further introduce the ANM constraint , ( III ) the effect is the sum of noise and a nonlinear function of cause as defined in Eqn . ( 3 ) . Theorem 1 ( Zero divergence of the velocity field ) . Under constraints ( I ) and ( II ) , the dynamical systems given by the L2 Wasserstein distance are pressureless flows . Further under ANM constraint ( III ) , they become volume-preserving pressureless flows , of which the divergence of the velocity field , v ( t , xt ) = [ vx ( t , xt ) , vy ( t , yt ) ] ′ , satisfies div v ( t , xt ) = ∂vx ( t , xt ) ∂xt + ∂vy ( t , yt ) ∂yt = 0 , ∀t ∈ [ 0 , T ] , xt , yt ∈ R , where div is the divergence operator in vector calculus . Thm . 1 determines the corresponding dynamical systems of FCMs and ANMs by analyzing their densities and velocity fields ( the details of the proof are in App . D ) , and shows a property of the corresponding dynamical systems of ANMs . Importantly , the property indicates a potential criterion for causal direction determination , that in the causal direction the divergence of the velocity field is zero everywhere , while in the reverse direction it may not always hold . Next , we will verify the criterion rigorously , propose an algorithm based on it , and show the extension for the PNL cases .

Summary: The paper tackles the problem of causal discovery in the basic case where a pair of variables is considered. In particular, it is concerned with Functional Causal Models and how to establlish the causal direction for a pair of continuous variables. It motivates the contibution by poiting out thet the performance of availabel algorithms is sensitive to the model assumptions, which makes it difficult for practitioners to use them in real settings. Then, the main contibution is a novel dynamical-system view of Functional Causal Models which aims to identify the causal direction in the case of pairs of variables. The paper exploits connection between functional causal models and optimal transport. Then, the paper studies the problem of optimal transport by taking into account the constraint originating from the considered functional causal model.