Causal Inference for Event Pairs in Multivariate Point Processes

1 Introduction . It is widely known that the gold standard for effective causal inference is through the use of interventional data such as randomized controlled trials , for measuring the impact of some treatment on an outcome of interest [ 30 ] . However , intervening in the system can often be impractical or even impossible , in which case one must conduct causal analysis using observational data alone . Observational data in the form of multivariate event streams is common in many domains including health , finance , and retail . Analogous to how an observation in an i.i.d dataset can be viewed as a sample from a joint distribution over a set of random variables , a multivariate event stream can be viewed as a sample from a multivariate point process over a set of event labels [ 10 ] , where the instantaneous rate of any event label depends on the historical occurrences of some subset of the event labels . Modeling , fitting , and predicting future occurrences given a multivariate event stream is an active area in statistics [ 9 , 11 , 40 ] , data mining [ 28 , 34 ] , and machine learning [ 32 , 16 , 29 , 44 , 7 ] . Recent work in this area leverages advances in deep learning [ 45 , 25 , 17 , 48 , 21 ] . Related work in survival analysis focuses only on pairs of events with continuous covariates [ 23 ] , involving short event streams with both continuous and discrete variables , and most notably [ 47 , 46 ] only a single occurrence of the outcome [ 24 ] . While a general theory for point process counterfactual inference has been developed [ 37 , 1 ] , practical algorithms focus on continuous covariates and hazard models [ 39 ] . Moreover , work in dynamic treatment [ 26 , 18 ] is for discrete time , motivating work for continuous-time data , which aims to reduce bias or high variance in discretizing time [ 38 , 41 , 43 ] . In 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . contrast , here we focus on the setting where data is of the form of long event streams with multiple occurrences of various types of events , including both the treatment event and outcome event . Causal inference entails drawing a conclusion about a causal relationship between potential causes and effects ; in this work , we focus on causal inference between pairs of event labels observed in event stream data , which differs from standard Granger causal graph learning [ 12 , 49 ] . Specifically , we pose the following causal inference problem for multivariate point processes : how can one meaningfully measure the causal relationship between a cause event label z and an effect event label y ? Such a causal measure would explain whether the cause label z amplifies , inhibits , or has no impact on the effect label y , while taking into account the potential effects of all other event labels x . This problem deviates from the typical causal inference setting in several ways . First , most methods assume i.i.d . observations ; in our setting , events may be correlated across time and therefore it is necessary to make certain independence assumptions in order to identify the effect of interest . Second , most causal inference settings are concerned with the expected value of a single observable outcome such as mortality . Our setting has repeated occurrences of an outcome over time and therefore involves event frequency ( or rate ) , which existing pairwise causal inference models are unable to handle and therefore necessitates new methods . Note that since event label z ’ s historical occurrences could have a complex dynamic effect on another event y ’ s occurrence at any time in a multivariate point process , there are a number of questions to address around the fundamental problem of even defining the notions of treatment , outcome and covariates , before one can study the causal effect of z on y . There are several real-world applications that motivate our work on causal inference with event stream data . For instance , one may be interested in understanding causal relations between events relevant to a diabetic patient , such as meals , insulin intake , exercise and blood glucose level changes [ 2 ] . Detecting issues in microservice or other web applications requires detecting which faults or abnormal states cause other downstream faults from system logs [ 3 ] . Example causal pairs from other domains are : food shortage and protests , which are recurring event labels in socio-economic datasets [ 27 ] , and earning call events and stock price jumps , which are recurring event labels in financial datasets [ 42 ] . All these examples have an observable outcome that occurs repeatedly , necessitating a new approach . Contributions We make several major contributions : 1 ) we propose a formalization of causal inference between a pair of event variables in a multivariate point process , 2 ) we define an average treatment effect ( ATE ) for multivariate point processes as the effect of having observed treatment in a proximal window , 3 ) we derive and obtain the equivalence of propensity scores and balancing scores for ATE in multivariate point processes , 4 ) we propose an inverse propensity score re-weighting procedure for better ATE estimation to adjust for the impact of other covariates , and 5 ) we conduct a detailed empirical evaluation on both synthetic and real datasets with baselines to show the superior performance of our proposed methods in pairwise causal inference in multivariate point processes . 2 Background . A multivariate event stream ( or event dataset ) is a sequence of events , D = { Di } Ni=1 , where each event Di includes an event label ( or synonymously type or variable ) and a time-stamp , i.e . Di = ( xi , ti ) . xi belongs to the label set L , whose cardinality is M , i.e . |L| = M , and ti is the time of occurrence , ti 2 R+ . We assume a strictly temporally ordered dataset , ti < tj for i < j , initial time t0 = 0 and end time tN+1 = T . Let z , y refer to an arbitrary pair of event labels in L. Multivariate event streams can be regarded as samples from a multivariate point process ( MPP ) , where each label has a counting process [ 10 ] . It uses conditional intensity functions y ( t|Ht ) > 0 that denote the instantaneous rate of type y given the history up to t , i.e . Ht = { ( xi , ti ) : ti < t } . In a multivariate point process , the probability of observing y as the next event at time t is : pt ( y|Htn ) = y ( t|Ht ) exp ( Z t tn X x2L x ( ⌧ |H⌧ ) d⌧ ) ( 1 ) where tn is the most recent event occurrence time before t. It can be shown that y ( t|Ht ) dt = E [ Ny ( [ t , t+ dt ] ) |Ht ] [ 10 , 33 ] , where Ny ( A ) is the number of y occurrences in time interval A . Prior work has proposed the notion of process independence among event labels ’ counting processes [ 11 ] . The basic idea is that the intensity of one type of event does not depend on certain past events once we know about specific other past events . This is an asymmetric concept , similar to Granger causality . Informally , for sets of labels x , y and z s.t . y \ z = ; , x is process independent of y given z when all labels in x have conditional intensities that do not functionally depend at any time on the history of labels in y , given the history of labels in z . Process independence is captured in graphical event models [ 11 , 15 ] , where the conditional intensity for any event label at any time t depends only on historical occurrences of its event label parents in the underlying graph . A minimal graphical representation can be used to define direct causes in multivariate point processes , analogous to those in causal networks . Here we provide a simplified definition of direct causes [ 11 ] : Definition 1 . Event label z is a direct cause of label y if z belongs to the minimal set of nodes u s.t . y is process independent of all other labels given u . A minimal set entails that none of its subsets satisfies the property . We highlight that in this work , we are interested in identifying pairwise causal relationships without learning a full graphical model . This is because it can often be computationally intensive as well as unnecessary to learn a full model when one is merely interested in pairwise causality . 3 Causal Inference in Point Processes . The Neyman-Rubin potential outcomes causal inference framework estimates the treatment effects between a treated variable and an effect for i.i.d . data [ 36 , 20 ] . We extend this framework , including average treatment effects ( ATE ) , to study how an event label z affects event label y in a MPP . In this class of models , treatment is denoted as z 2 { 0 , 1 } , where 0 is the control and 1 the treatment . The potential outcome yz for each z is the outcome if treatment z is applied . The main difficulty in causal inference stems from the fact that only outcomes from the administered treatments are observed and never any of the other outcomes . Hence it may be viewed as a missing data problem . We estimate the treatment effects between a treatment variable associated with historical occurrences of z and an outcome variable ( or response ) associated with y ’ s occurrence , under existing assumptions in the literature . Covariate variables x involve historical occurrences of labels other than z , i.e . x = L \ z . We assume all variables are observed , hence strong ignorability is satisfied ( which we will discuss further in Section 3.2 ) . Next we define treatment , outcome , and covariate variables , and then derive the propensity score for ATE computation . 3.1 Defining Treatment , Outcome and Covariates . There are many possible ways to define the treatment , outcome , and covariates in a multivariate point process . We begin with the following general formulation . We use Zt to represent the treatment variable at time t , distinguishing it from the event label z . Let Hzt be the history of occurrences of event z before t , and Kyt be the future of event y occurrences at t. Definition 2 . General Formulation for Causal Inference : for a pair of event labels ( z , y ) , the treatment variable Zt at time t is a function of z ’ s historical occurrences , i.e . Zt = fz ( Hzt ) . The outcome variable is a function of y ’ s future occurrences at t , i.e . Yt = fy ( K y t ) . The covariates Xt are a function of historical occurrences of event labels other than z , i.e . Xt = f z ( H z t ) . Treatment and Covariate Models We need to specify the functions in the general formulation of Definition 2 , for instance , we need to summarize historical occurrences Hzt . Due to potential multiple ( re ) -occurrence of treatment and outcome , the representation of history needs careful modeling based on specific assumptions at each time t. We take a simplified yet practical view of treatment , making the proximal assumption that the most recent history is sufficient for causation . We do this by assuming only a recent time window [ t w , t ) for some window w could directly have a causal impact on the outcome at time t ; such an assumption has been used previously in point process models [ 7 ] and provides an efficient and interpretable definition for treatment . It fits many real-world situations where the influence comes primarily from recent history and repeated occurrences have little additional impact on the intensity rate . It also has a strong statistical justification : matching long treatment history could result in infinite sample requirements . We assume covariates are also defined by whether they have appeared at least once in some recent time window . Outcome Model We wish to measure a statistical quantity fy ( Kyt ) associated with the outcome in a multivariate point process , based on future occurrences of y , Kyt , from time t. Many such quantities fy ( Kyt ) can be used . It is natural to consider either the instantaneous expected number of occurrences of the outcome y , E [ Ny ( [ t , t+ dt ] ) |Ht ] , in place of Kyt , or cumulative expected number of occurrences of outcome in a future duration wf , E [ Ny ( [ t , t+ wf ] ) |Ht ] . Moreover , the expected count E [ Ny ( [ t , t+ dt ] ) |Ht ] equals y ( t ) dt by definition . Note that at each time , the history can be very different with the proximal assumption , hence the rate at each time in Kyt can also be different . The cumulative counts can be similarly written as ⇤wfy ( t ) = R t+wf t y ( t ) dt for some future duration wf , which can be the total time horizon or infinity , but practically it is more meaningful for a specific duration suitable for the application . We will focus our analysis with y ( t ) as the outcome in the following , which could be directly extended to outcome ⇤wfy ( t ) . We will compare with both definitions of outcomes in experiments . Hence , our specific realization of the general formulation results in the following formulation : Definition 3 . Recent History Formulation for Causal Inference : for a pair of event labels ( z , y ) , the binary treatment variable Zwt at time t is defined by whether or not z has occurred at least once within a window w into the past from t. The outcome variable is the occurrence rate of the effect label y at time t given the treatment , Yt = y|Zt ( t ) . The covariates X w t at time t are a binary vector , depending on whether other event labels have occurred at least once in [ t w , t ) , w 2 ( 0 , t ) . In event streams , it is possible for the cause and effect to be the same label in history , for instance , consider a system with negative self-feedback ( eating too much sugar for example ) . Hence z and Xt could be identical to or contain y , but z and Xt must be disjoint sets of events . We summarize the assumptions in this formulation of causal inference : 1 ) events before t w have no impact on y ’ s occurrence rate at time t. This enables memory in time yet provides a compact representation of history , 2 ) only the occurrence of z in the window impacts y ’ s rate at time t ( regardless of the number of occurrences , hence we do not differentiate between multiple occurrences within w ) , and 3 ) the specific times of z ’ s occurrences do not further affect y ’ s rate at time t [ 16 , 29 , 15 ] . Such a model can be robust to outliers or noisy historical observations .

Summary: This paper describes a method for the causal analysis of long streams of event data that can be modeled as multivariate point processes (MPPs). The paper defines the average treatment effect (ATE) between pairs of events in MPPs that occur within a window, and derives how to estimate the ATE as the expected difference of average outcome rates.  When covariates differ or assumptions are violated, this estimate may be biased, but adjusting for the propensity score yields an unbiased estimator, which the paper does through inverse probability of treatment weighting (IPTW) (including a stabilized version).  Experiments on data from known, synthetic MPPs (proximal graphical event models and Hawkes processes) show that the proposed IPTW estimation method more accurately recovers the ATE than baseline conditional intensity scores.  Similarly, experiments on a diabetes dataset show that stable IPTW recovers more true causal pairs at 2 out of 3 levels of recall compared to the conditional intensity scores.  These results extend to using the cumulative rather than the conditional intensity rate.