Time Series Counterfactual Inference with Hidden Confounders

1 INTRODUCTION . Decision makers want to know how to produce desired outcomes and act accordingly , which requires causal understanding of cause and effect . In this paper , we consider applications in healthcare , where time series data on past features and outcomes are now widely available . Causality in time series have been long studied in statistics ( Box et al. , 2008 ) , and allows more powerful analysis than methods on time-independent data , like instrumental variable regression ( Stock & Trebbi , 2003 ) . However , temporal causality in statistics and econometrics focuses mainly on passively discovering time lag structure ( Eichler , 2012 ) . In contrast , decision-making applications need concrete interventions , which is more amenable to an interventionist approach to causality ( Woodward , 2005 ; Pearl , 2009 ) . To give one example , electronic health records ( EHR ) in healthcare provide an accessible history of a patient ’ s disease progression over time , together with their treatment records and their results . To identify effective treatments , a doctor may want to ask counterfactual questions ( Johansson et al. , 2016 ) , like “ Would this patient have lower blood sugar had she received a different medication ? ” Through such counterfactual analysis , medical professionals may hope to discover new cures and improve existing treatments . Similar situations arise in other use cases . For example , a user interface designer may want to ask “ Would the user have clicked on this ad had it been in a different color ? ” , substantiating their answer from counterfactual inference on clickstream data or other user behaviors . Counterfactual inference in time series has studied , assuming that all possible causal variables are observed ( Soleimani et al. , 2017 ; Schulam & Saria , 2017 ; Lim , 2018 ) . In practice , however , this assumption of perfect observability is not testable and too strong for many real-world scenarios ( Bica et al. , 2020 ) . For example , there are many ways in general to treat cancer , but each patient requires their own bespoke treatment plan based on unique characteristics of each case such as drug resistance and toxic response ( Vlachostergios & Faltas , 2018 ; Kroschinsky et al. , 2017 ; Bica et al. , 2020 ) . However , these factors are also likely to be unmeasurable in practice , or otherwise not recorded in EHRs . Detecting these hidden confounding variables is therefore crucial to avoid bias in the estimation of treatment effects . The challenge introduced by confounders in counterfactual inference was first studied in the static setting . Wang & Blei ( 2019 ) developed a two-step method that estimates confounders with latent factor models , then infers potential outcomes with bias adjustment . However , confounders in time series can have their own dynamics , and can themselves be affected by the history of interventions . Subsequently , Bica et al . ( 2020 ) introducing recurrent neural networks ( RNNs ) into the factor model to estimate the dynamics of confounders . However , this method only works in discrete time setting with a fixed time step , due to how RNNs are structured . In this paper , we consider the continuous-time setting , which is more flexible in practice and provides more insights of the underlying mechanisms ( Chen et al. , 2018 ; Rubanova et al. , 2019 ) . The continuous-time setting is particularly important for healthcare , where there are many time-varying treatments , irregularly-sampled or partially observed time series ( Soleimani et al. , 2017 ) . The classical modeling approach to dynamics uses ordinary differential equations ( ODEs ) d ( x ( t ) ) /dt = f ( x ( t ) ) , encoding domain expertise of underlying mechanisms in the explicit specification of f . In contrast , Chen et al . ( 2018 ) introduced the concept of neural ODEs by parameterizing f with neural networks , thus allowing dynamics to be described by arbitrarily complicated functions . Several extensions handle even more complicated issues like irregular sampling or switching dynamics ( Jia & Benson , 2019 ; Kidger et al. , 2020 ) . However , these methods can not be directly applied to time series counterfactual inference , as they focus on initial value problems , which can not describe interventions without explicit modification of f ( Kidger et al. , 2020 ) . Furthermore , these existing methods can only handle hidden variables by explicitly describing their dynamics and interdependency with interventions , thus limiting their utility when confounders exist . Our contributions . We propose augmented counterfactual ODEs ( ACODEs ) to predict how a continuous-time time series will evolve under a sequence of interventions . Our method augments the observed time series with additional dimensions to represent confounders . We then construct counterfactual ODEs based on the neural ODE framework to model the effects of incoming interventions . The ACODE model has three key features . First , it allows for the presence of confounders that can reduce the prediction bias . Second , the ACODE can continuously incorporate incoming interventions using neural ODEs and support irregularly-sampled time series . Third , it demonstrates state-of-the-art performance against competitive baselines for counterfactual inference in both simulation of tumor growth and real-world time series of sepsis patients treatment response . Moreover , the ACODE provides an interface between machine learning and dominant modelling paradigm described in differential equations , which allows for well-understood domain knowledge to be applied to time series counterfactual inference . To the best of our knowledge , this represents the first method for counterfactual inference with confounders in the continuous-time setting . 2 RELATED WORK . Time series counterfactual inference stems from causal inference ( Pearl , 2009 ; Eichler , 2012 ) . A large body of pioneering work in causal inference focus on causal relations such as structural causal models ( Pearl , 2019 ) and Granger causality ( Eichler , 2007 ) . Counterfactual inference , on the other hand , focus on estimating the effects of actionable interventions , which is a pervasive problem in healthcare ( Hoover , 2018 ) . In literature , the difference between the counterfactual outcomes if an intervention had been taken or not is defined as the causal effect of the intervention ( Pearl , 2009 ) . Originated from the literature on observational studies ( Shadish et al. , 2002 ) , Rubin ’ s potential outcome framework has been a popular language to formalize counterfactuals and intervention effect estimate ( Rubin , 2005 ; Imbens & Rubin , 2015 ) . The problem of hidden confounders in counterfactual inference was first studied in the static setting . Wang & Blei ( 2019 ) developed theory for adjusting the bias introduced by the presence of hidden confounders in the observational data . They found out that the dependencies in these multiple confounders can be used to infer latent variables and act as substitutes for the hidden confounders . In this paper , we are interested in considering hidden confounders in time series setting which is much more complicated than in the static setting . Not only because the hidden confounders may evolve over time , but also because they might be affected by previous interventions . On the other hand , most existing work on time series counterfactual inference including counterfactual Gaussian processes ( CGP ) ( Schulam & Saria , 2017 ) and recurrent marginal structural networks ( RMSNs ) ( Lim , 2018 ) assume there is no hidden confounders , i.e . all variables affecting the intervention plan and the potential outcomes are observed , which is not testable in practice and not true in many cases . Recently , Bica et al . ( 2020 ) applied the idea of latent factor models from Wang & Blei ( 2019 ) to the deconfounding of time series . However , their proposed method is based on recurrent neural networks , which works only with discrete and regularly-spaced time series . Differential equations have been introduced into causal and counterfactual inference in previous studies . Rubenstein et al . ( 2018 ) showed that equilibrium states of a first-order ODE system can be described with a deterministic structural causal model , even with non-constant interventions . This line of literature is centering around casual relations , which is a different focus from this work . On the other hand , differential equations with incoming information is a well-studied mathematical problem in the field of rough analysis , which is referred as controlled differential equations or rough differential equations . These approaches directly integrate with respect to incoming processes ( Friz & Victoir , 2010 ; Lyons et al. , 2007 ) . Neural Ordinary Differential Equations Neural ODEs ( Chen et al. , 2018 ) are a family of continuous-time models . Starting from an initial state z ( t0 ) , it evolves following a neural network based differential equations . The state at any time ti is given by integrating an ODE forward in time : dz ( t ) dt = f ( z ( t ) , t ; θ ) , z ( ti ) = z ( t0 ) + ∫ ti t0 dz ( t ) dt dt ( 1 ) where f is a neural network parametrized by θ . Given the initial state , states at any desired time stamps can be evaluated with a numerical ODE solver : z0 , z1 , ... , zN = ODESolve ( fθ , z ( t0 ) , ( t0 , t1 , ... , tN ) ) ( 2 ) More importantly , Chen et al . ( 2018 ) proposed to use the adjoint method to compute the gradient with respect to the parameters θ as long as f is uniformly Lipschitz continuous in z ( t ) and continuous in t. This allows ODE solvers to be used as a black box building block in large models . 3 PROBLEM FORMULATION . Consider a multivariate time series x ( t ) and continuous-time time-dependent interventions a ( t ) . The observational data consists of multiple realizations of above mentioned time series and interventions . Given that realizations are independent to each other , we only consider one realization in following part for simplicity . In a realization up to time t , we observe N time series data points and their timestamps { xi , ti } Ni=1 along with continuous-time interventions { a ( s ) : s ≤ t } . We would like to infer the potential outcome under future interventions given all historical information for any potential intervention plan { a ( s ) : s > t } . We will abuse the notation of a > t and { a ( s ) : s > t } in following sections . Our goal is to infer the following distribution : p ( x ( a > t ) |a≤t , { xi , txi } Ni=1 ) ( 3 ) where x ( a > t ) denotes the potential outcome of future time series x under future intervention a > t . Although we can not directly model this objective distribution , we can instead fit a regression model to estimate p ( x > t|a > t , a≤t , { xi , ti } Ni=1 ) from observational data ( Rubin , 1978 ) . For cases without hidden confounders , this lead to unbiased estimation of potential outcome p ( x ( a > t ) |a≤t , { xi , ti } Ni=1 ) = p ( x > t|a > t , a≤t , { xi , ti } Ni=1 ) under certain assumptions , including sequential strong ignorability ( Fitzmaurice et al. , 2008 ) : z ( a≥t ) ⊥⊥ a ( t ) |a < t , x≤t for ∀a≥t ( 4 ) This condition holds if there are no hidden confounders , which can not be tested in practice since counterfactual outcomes are never observed in practice . With the presence of hidden confounders , the above assumption is no longer valid and p ( x ( a > t ) |a≤t , { xi , ti } Ni=1 ) 6= p ( x > t|a > t , a≤t , { xi , ti } Ni=1 ) ( 5 ) Consequently , existing methods which infer conditional distribution p ( x > t|a > t , a < t , { xi , txi } Ni=1 ) from observed data would result in biased estimation of potential outcome .

Summary: This paper proposed to solve an interesting problem: how do we perform counterfactual inference for time series data? The paper follows a study of the problem in the static setting: in the first step, the paper fit an augmented time series $u_t$ as additional confounders, and then perform inference based on the augmented time series. The experiments presented appear promising to the time series inference problem.