Time Series Counterfactual Inference with Hidden Confounders

1 INTRODUCTION . Decision makers want to know how to produce desired outcomes and act accordingly , which requires causal understanding of cause and effect . In this paper , we consider applications in healthcare , where time series data on past features and outcomes are now widely available . Causality in time series have been long studied in statistics ( Box et al. , 2008 ) , and allows more powerful analysis than methods on time-independent data , like instrumental variable regression ( Stock & Trebbi , 2003 ) . However , temporal causality in statistics and econometrics focuses mainly on passively discovering time lag structure ( Eichler , 2012 ) . In contrast , decision-making applications need concrete interventions , which is more amenable to an interventionist approach to causality ( Woodward , 2005 ; Pearl , 2009 ) . To give one example , electronic health records ( EHR ) in healthcare provide an accessible history of a patient ’ s disease progression over time , together with their treatment records and their results . To identify effective treatments , a doctor may want to ask counterfactual questions ( Johansson et al. , 2016 ) , like “ Would this patient have lower blood sugar had she received a different medication ? ” Through such counterfactual analysis , medical professionals may hope to discover new cures and improve existing treatments . Similar situations arise in other use cases . For example , a user interface designer may want to ask “ Would the user have clicked on this ad had it been in a different color ? ” , substantiating their answer from counterfactual inference on clickstream data or other user behaviors . Counterfactual inference in time series has studied , assuming that all possible causal variables are observed ( Soleimani et al. , 2017 ; Schulam & Saria , 2017 ; Lim , 2018 ) . In practice , however , this assumption of perfect observability is not testable and too strong for many real-world scenarios ( Bica et al. , 2020 ) . For example , there are many ways in general to treat cancer , but each patient requires their own bespoke treatment plan based on unique characteristics of each case such as drug resistance and toxic response ( Vlachostergios & Faltas , 2018 ; Kroschinsky et al. , 2017 ; Bica et al. , 2020 ) . However , these factors are also likely to be unmeasurable in practice , or otherwise not recorded in EHRs . Detecting these hidden confounding variables is therefore crucial to avoid bias in the estimation of treatment effects . The challenge introduced by confounders in counterfactual inference was first studied in the static setting . Wang & Blei ( 2019 ) developed a two-step method that estimates confounders with latent factor models , then infers potential outcomes with bias adjustment . However , confounders in time series can have their own dynamics , and can themselves be affected by the history of interventions . Subsequently , Bica et al . ( 2020 ) introducing recurrent neural networks ( RNNs ) into the factor model to estimate the dynamics of confounders . However , this method only works in discrete time setting with a fixed time step , due to how RNNs are structured . In this paper , we consider the continuous-time setting , which is more flexible in practice and provides more insights of the underlying mechanisms ( Chen et al. , 2018 ; Rubanova et al. , 2019 ) . The continuous-time setting is particularly important for healthcare , where there are many time-varying treatments , irregularly-sampled or partially observed time series ( Soleimani et al. , 2017 ) . The classical modeling approach to dynamics uses ordinary differential equations ( ODEs ) d ( x ( t ) ) /dt = f ( x ( t ) ) , encoding domain expertise of underlying mechanisms in the explicit specification of f . In contrast , Chen et al . ( 2018 ) introduced the concept of neural ODEs by parameterizing f with neural networks , thus allowing dynamics to be described by arbitrarily complicated functions . Several extensions handle even more complicated issues like irregular sampling or switching dynamics ( Jia & Benson , 2019 ; Kidger et al. , 2020 ) . However , these methods can not be directly applied to time series counterfactual inference , as they focus on initial value problems , which can not describe interventions without explicit modification of f ( Kidger et al. , 2020 ) . Furthermore , these existing methods can only handle hidden variables by explicitly describing their dynamics and interdependency with interventions , thus limiting their utility when confounders exist . Our contributions . We propose augmented counterfactual ODEs ( ACODEs ) to predict how a continuous-time time series will evolve under a sequence of interventions . Our method augments the observed time series with additional dimensions to represent confounders . We then construct counterfactual ODEs based on the neural ODE framework to model the effects of incoming interventions . The ACODE model has three key features . First , it allows for the presence of confounders that can reduce the prediction bias . Second , the ACODE can continuously incorporate incoming interventions using neural ODEs and support irregularly-sampled time series . Third , it demonstrates state-of-the-art performance against competitive baselines for counterfactual inference in both simulation of tumor growth and real-world time series of sepsis patients treatment response . Moreover , the ACODE provides an interface between machine learning and dominant modelling paradigm described in differential equations , which allows for well-understood domain knowledge to be applied to time series counterfactual inference . To the best of our knowledge , this represents the first method for counterfactual inference with confounders in the continuous-time setting . 2 RELATED WORK . Time series counterfactual inference stems from causal inference ( Pearl , 2009 ; Eichler , 2012 ) . A large body of pioneering work in causal inference focus on causal relations such as structural causal models ( Pearl , 2019 ) and Granger causality ( Eichler , 2007 ) . Counterfactual inference , on the other hand , focus on estimating the effects of actionable interventions , which is a pervasive problem in healthcare ( Hoover , 2018 ) . In literature , the difference between the counterfactual outcomes if an intervention had been taken or not is defined as the causal effect of the intervention ( Pearl , 2009 ) . Originated from the literature on observational studies ( Shadish et al. , 2002 ) , Rubin ’ s potential outcome framework has been a popular language to formalize counterfactuals and intervention effect estimate ( Rubin , 2005 ; Imbens & Rubin , 2015 ) . The problem of hidden confounders in counterfactual inference was first studied in the static setting . Wang & Blei ( 2019 ) developed theory for adjusting the bias introduced by the presence of hidden confounders in the observational data . They found out that the dependencies in these multiple confounders can be used to infer latent variables and act as substitutes for the hidden confounders . In this paper , we are interested in considering hidden confounders in time series setting which is much more complicated than in the static setting . Not only because the hidden confounders may evolve over time , but also because they might be affected by previous interventions . On the other hand , most existing work on time series counterfactual inference including counterfactual Gaussian processes ( CGP ) ( Schulam & Saria , 2017 ) and recurrent marginal structural networks ( RMSNs ) ( Lim , 2018 ) assume there is no hidden confounders , i.e . all variables affecting the intervention plan and the potential outcomes are observed , which is not testable in practice and not true in many cases . Recently , Bica et al . ( 2020 ) applied the idea of latent factor models from Wang & Blei ( 2019 ) to the deconfounding of time series . However , their proposed method is based on recurrent neural networks , which works only with discrete and regularly-spaced time series . Differential equations have been introduced into causal and counterfactual inference in previous studies . Rubenstein et al . ( 2018 ) showed that equilibrium states of a first-order ODE system can be described with a deterministic structural causal model , even with non-constant interventions . This line of literature is centering around casual relations , which is a different focus from this work . On the other hand , differential equations with incoming information is a well-studied mathematical problem in the field of rough analysis , which is referred as controlled differential equations or rough differential equations . These approaches directly integrate with respect to incoming processes ( Friz & Victoir , 2010 ; Lyons et al. , 2007 ) . Neural Ordinary Differential Equations Neural ODEs ( Chen et al. , 2018 ) are a family of continuous-time models . Starting from an initial state z ( t0 ) , it evolves following a neural network based differential equations . The state at any time ti is given by integrating an ODE forward in time : dz ( t ) dt = f ( z ( t ) , t ; θ ) , z ( ti ) = z ( t0 ) + ∫ ti t0 dz ( t ) dt dt ( 1 ) where f is a neural network parametrized by θ . Given the initial state , states at any desired time stamps can be evaluated with a numerical ODE solver : z0 , z1 , ... , zN = ODESolve ( fθ , z ( t0 ) , ( t0 , t1 , ... , tN ) ) ( 2 ) More importantly , Chen et al . ( 2018 ) proposed to use the adjoint method to compute the gradient with respect to the parameters θ as long as f is uniformly Lipschitz continuous in z ( t ) and continuous in t. This allows ODE solvers to be used as a black box building block in large models . 3 PROBLEM FORMULATION . Consider a multivariate time series x ( t ) and continuous-time time-dependent interventions a ( t ) . The observational data consists of multiple realizations of above mentioned time series and interventions . Given that realizations are independent to each other , we only consider one realization in following part for simplicity . In a realization up to time t , we observe N time series data points and their timestamps { xi , ti } Ni=1 along with continuous-time interventions { a ( s ) : s ≤ t } . We would like to infer the potential outcome under future interventions given all historical information for any potential intervention plan { a ( s ) : s > t } . We will abuse the notation of a > t and { a ( s ) : s > t } in following sections . Our goal is to infer the following distribution : p ( x ( a > t ) |a≤t , { xi , txi } Ni=1 ) ( 3 ) where x ( a > t ) denotes the potential outcome of future time series x under future intervention a > t . Although we can not directly model this objective distribution , we can instead fit a regression model to estimate p ( x > t|a > t , a≤t , { xi , ti } Ni=1 ) from observational data ( Rubin , 1978 ) . For cases without hidden confounders , this lead to unbiased estimation of potential outcome p ( x ( a > t ) |a≤t , { xi , ti } Ni=1 ) = p ( x > t|a > t , a≤t , { xi , ti } Ni=1 ) under certain assumptions , including sequential strong ignorability ( Fitzmaurice et al. , 2008 ) : z ( a≥t ) ⊥⊥ a ( t ) |a < t , x≤t for ∀a≥t ( 4 ) This condition holds if there are no hidden confounders , which can not be tested in practice since counterfactual outcomes are never observed in practice . With the presence of hidden confounders , the above assumption is no longer valid and p ( x ( a > t ) |a≤t , { xi , ti } Ni=1 ) 6= p ( x > t|a > t , a≤t , { xi , ti } Ni=1 ) ( 5 ) Consequently , existing methods which infer conditional distribution p ( x > t|a > t , a < t , { xi , txi } Ni=1 ) from observed data would result in biased estimation of potential outcome .

Summary: The authors propose a new method, called augmented counterfactual ordinary differential equations (ACODs), to do counterfactual inference on time series data in healthcare. This is done by modelling interventions in continuous time with differential equations augmented by auxiliary confounding variables to reduce bias. They demonstrate the proposed method on tumor growth simulation and sepsis patient treatment response.