Causal Inference for Event Pairs in Multivariate Point Processes

1 Introduction . It is widely known that the gold standard for effective causal inference is through the use of interventional data such as randomized controlled trials , for measuring the impact of some treatment on an outcome of interest [ 30 ] . However , intervening in the system can often be impractical or even impossible , in which case one must conduct causal analysis using observational data alone . Observational data in the form of multivariate event streams is common in many domains including health , finance , and retail . Analogous to how an observation in an i.i.d dataset can be viewed as a sample from a joint distribution over a set of random variables , a multivariate event stream can be viewed as a sample from a multivariate point process over a set of event labels [ 10 ] , where the instantaneous rate of any event label depends on the historical occurrences of some subset of the event labels . Modeling , fitting , and predicting future occurrences given a multivariate event stream is an active area in statistics [ 9 , 11 , 40 ] , data mining [ 28 , 34 ] , and machine learning [ 32 , 16 , 29 , 44 , 7 ] . Recent work in this area leverages advances in deep learning [ 45 , 25 , 17 , 48 , 21 ] . Related work in survival analysis focuses only on pairs of events with continuous covariates [ 23 ] , involving short event streams with both continuous and discrete variables , and most notably [ 47 , 46 ] only a single occurrence of the outcome [ 24 ] . While a general theory for point process counterfactual inference has been developed [ 37 , 1 ] , practical algorithms focus on continuous covariates and hazard models [ 39 ] . Moreover , work in dynamic treatment [ 26 , 18 ] is for discrete time , motivating work for continuous-time data , which aims to reduce bias or high variance in discretizing time [ 38 , 41 , 43 ] . In 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . contrast , here we focus on the setting where data is of the form of long event streams with multiple occurrences of various types of events , including both the treatment event and outcome event . Causal inference entails drawing a conclusion about a causal relationship between potential causes and effects ; in this work , we focus on causal inference between pairs of event labels observed in event stream data , which differs from standard Granger causal graph learning [ 12 , 49 ] . Specifically , we pose the following causal inference problem for multivariate point processes : how can one meaningfully measure the causal relationship between a cause event label z and an effect event label y ? Such a causal measure would explain whether the cause label z amplifies , inhibits , or has no impact on the effect label y , while taking into account the potential effects of all other event labels x . This problem deviates from the typical causal inference setting in several ways . First , most methods assume i.i.d . observations ; in our setting , events may be correlated across time and therefore it is necessary to make certain independence assumptions in order to identify the effect of interest . Second , most causal inference settings are concerned with the expected value of a single observable outcome such as mortality . Our setting has repeated occurrences of an outcome over time and therefore involves event frequency ( or rate ) , which existing pairwise causal inference models are unable to handle and therefore necessitates new methods . Note that since event label z ’ s historical occurrences could have a complex dynamic effect on another event y ’ s occurrence at any time in a multivariate point process , there are a number of questions to address around the fundamental problem of even defining the notions of treatment , outcome and covariates , before one can study the causal effect of z on y . There are several real-world applications that motivate our work on causal inference with event stream data . For instance , one may be interested in understanding causal relations between events relevant to a diabetic patient , such as meals , insulin intake , exercise and blood glucose level changes [ 2 ] . Detecting issues in microservice or other web applications requires detecting which faults or abnormal states cause other downstream faults from system logs [ 3 ] . Example causal pairs from other domains are : food shortage and protests , which are recurring event labels in socio-economic datasets [ 27 ] , and earning call events and stock price jumps , which are recurring event labels in financial datasets [ 42 ] . All these examples have an observable outcome that occurs repeatedly , necessitating a new approach . Contributions We make several major contributions : 1 ) we propose a formalization of causal inference between a pair of event variables in a multivariate point process , 2 ) we define an average treatment effect ( ATE ) for multivariate point processes as the effect of having observed treatment in a proximal window , 3 ) we derive and obtain the equivalence of propensity scores and balancing scores for ATE in multivariate point processes , 4 ) we propose an inverse propensity score re-weighting procedure for better ATE estimation to adjust for the impact of other covariates , and 5 ) we conduct a detailed empirical evaluation on both synthetic and real datasets with baselines to show the superior performance of our proposed methods in pairwise causal inference in multivariate point processes . 2 Background . A multivariate event stream ( or event dataset ) is a sequence of events , D = { Di } Ni=1 , where each event Di includes an event label ( or synonymously type or variable ) and a time-stamp , i.e . Di = ( xi , ti ) . xi belongs to the label set L , whose cardinality is M , i.e . |L| = M , and ti is the time of occurrence , ti 2 R+ . We assume a strictly temporally ordered dataset , ti < tj for i < j , initial time t0 = 0 and end time tN+1 = T . Let z , y refer to an arbitrary pair of event labels in L. Multivariate event streams can be regarded as samples from a multivariate point process ( MPP ) , where each label has a counting process [ 10 ] . It uses conditional intensity functions y ( t|Ht ) > 0 that denote the instantaneous rate of type y given the history up to t , i.e . Ht = { ( xi , ti ) : ti < t } . In a multivariate point process , the probability of observing y as the next event at time t is : pt ( y|Htn ) = y ( t|Ht ) exp ( Z t tn X x2L x ( ⌧ |H⌧ ) d⌧ ) ( 1 ) where tn is the most recent event occurrence time before t. It can be shown that y ( t|Ht ) dt = E [ Ny ( [ t , t+ dt ] ) |Ht ] [ 10 , 33 ] , where Ny ( A ) is the number of y occurrences in time interval A . Prior work has proposed the notion of process independence among event labels ’ counting processes [ 11 ] . The basic idea is that the intensity of one type of event does not depend on certain past events once we know about specific other past events . This is an asymmetric concept , similar to Granger causality . Informally , for sets of labels x , y and z s.t . y \ z = ; , x is process independent of y given z when all labels in x have conditional intensities that do not functionally depend at any time on the history of labels in y , given the history of labels in z . Process independence is captured in graphical event models [ 11 , 15 ] , where the conditional intensity for any event label at any time t depends only on historical occurrences of its event label parents in the underlying graph . A minimal graphical representation can be used to define direct causes in multivariate point processes , analogous to those in causal networks . Here we provide a simplified definition of direct causes [ 11 ] : Definition 1 . Event label z is a direct cause of label y if z belongs to the minimal set of nodes u s.t . y is process independent of all other labels given u . A minimal set entails that none of its subsets satisfies the property . We highlight that in this work , we are interested in identifying pairwise causal relationships without learning a full graphical model . This is because it can often be computationally intensive as well as unnecessary to learn a full model when one is merely interested in pairwise causality . 3 Causal Inference in Point Processes . The Neyman-Rubin potential outcomes causal inference framework estimates the treatment effects between a treated variable and an effect for i.i.d . data [ 36 , 20 ] . We extend this framework , including average treatment effects ( ATE ) , to study how an event label z affects event label y in a MPP . In this class of models , treatment is denoted as z 2 { 0 , 1 } , where 0 is the control and 1 the treatment . The potential outcome yz for each z is the outcome if treatment z is applied . The main difficulty in causal inference stems from the fact that only outcomes from the administered treatments are observed and never any of the other outcomes . Hence it may be viewed as a missing data problem . We estimate the treatment effects between a treatment variable associated with historical occurrences of z and an outcome variable ( or response ) associated with y ’ s occurrence , under existing assumptions in the literature . Covariate variables x involve historical occurrences of labels other than z , i.e . x = L \ z . We assume all variables are observed , hence strong ignorability is satisfied ( which we will discuss further in Section 3.2 ) . Next we define treatment , outcome , and covariate variables , and then derive the propensity score for ATE computation . 3.1 Defining Treatment , Outcome and Covariates . There are many possible ways to define the treatment , outcome , and covariates in a multivariate point process . We begin with the following general formulation . We use Zt to represent the treatment variable at time t , distinguishing it from the event label z . Let Hzt be the history of occurrences of event z before t , and Kyt be the future of event y occurrences at t. Definition 2 . General Formulation for Causal Inference : for a pair of event labels ( z , y ) , the treatment variable Zt at time t is a function of z ’ s historical occurrences , i.e . Zt = fz ( Hzt ) . The outcome variable is a function of y ’ s future occurrences at t , i.e . Yt = fy ( K y t ) . The covariates Xt are a function of historical occurrences of event labels other than z , i.e . Xt = f z ( H z t ) . Treatment and Covariate Models We need to specify the functions in the general formulation of Definition 2 , for instance , we need to summarize historical occurrences Hzt . Due to potential multiple ( re ) -occurrence of treatment and outcome , the representation of history needs careful modeling based on specific assumptions at each time t. We take a simplified yet practical view of treatment , making the proximal assumption that the most recent history is sufficient for causation . We do this by assuming only a recent time window [ t w , t ) for some window w could directly have a causal impact on the outcome at time t ; such an assumption has been used previously in point process models [ 7 ] and provides an efficient and interpretable definition for treatment . It fits many real-world situations where the influence comes primarily from recent history and repeated occurrences have little additional impact on the intensity rate . It also has a strong statistical justification : matching long treatment history could result in infinite sample requirements . We assume covariates are also defined by whether they have appeared at least once in some recent time window . Outcome Model We wish to measure a statistical quantity fy ( Kyt ) associated with the outcome in a multivariate point process , based on future occurrences of y , Kyt , from time t. Many such quantities fy ( Kyt ) can be used . It is natural to consider either the instantaneous expected number of occurrences of the outcome y , E [ Ny ( [ t , t+ dt ] ) |Ht ] , in place of Kyt , or cumulative expected number of occurrences of outcome in a future duration wf , E [ Ny ( [ t , t+ wf ] ) |Ht ] . Moreover , the expected count E [ Ny ( [ t , t+ dt ] ) |Ht ] equals y ( t ) dt by definition . Note that at each time , the history can be very different with the proximal assumption , hence the rate at each time in Kyt can also be different . The cumulative counts can be similarly written as ⇤wfy ( t ) = R t+wf t y ( t ) dt for some future duration wf , which can be the total time horizon or infinity , but practically it is more meaningful for a specific duration suitable for the application . We will focus our analysis with y ( t ) as the outcome in the following , which could be directly extended to outcome ⇤wfy ( t ) . We will compare with both definitions of outcomes in experiments . Hence , our specific realization of the general formulation results in the following formulation : Definition 3 . Recent History Formulation for Causal Inference : for a pair of event labels ( z , y ) , the binary treatment variable Zwt at time t is defined by whether or not z has occurred at least once within a window w into the past from t. The outcome variable is the occurrence rate of the effect label y at time t given the treatment , Yt = y|Zt ( t ) . The covariates X w t at time t are a binary vector , depending on whether other event labels have occurred at least once in [ t w , t ) , w 2 ( 0 , t ) . In event streams , it is possible for the cause and effect to be the same label in history , for instance , consider a system with negative self-feedback ( eating too much sugar for example ) . Hence z and Xt could be identical to or contain y , but z and Xt must be disjoint sets of events . We summarize the assumptions in this formulation of causal inference : 1 ) events before t w have no impact on y ’ s occurrence rate at time t. This enables memory in time yet provides a compact representation of history , 2 ) only the occurrence of z in the window impacts y ’ s rate at time t ( regardless of the number of occurrences , hence we do not differentiate between multiple occurrences within w ) , and 3 ) the specific times of z ’ s occurrences do not further affect y ’ s rate at time t [ 16 , 29 , 15 ] . Such a model can be robust to outliers or noisy historical observations .

Summary: This paper proposes a causal inference framework for a multivariate point process model. The key idea is to form a series of binary processes Z_t and X_t using a moving window [t-w,t] to indicate whether processes Z and X have at least one event in the window [t-w,t].  An average treatment event (ATE) is defined and propensity scores are proposed for unbiased estimation of ATE. The authors use synthetic experiments and real-world datasets to demonstrate how the proposed method work compared to baseline approaches.