Explaining Point Processes by Learning Interpretable Temporal Logic Rules

1 INTRODUCTION . Event sequences with irregular time intervals are ubiquitous . The inter-event times usually convey rich information regarding the underlying dynamics such as disease progression ( Liu et al. , 2015 ) . It is useful to understand events ’ generating mechanisms , as well as the occurrence reason and time . In systems where domain knowledge is rich , events generating can usually be governed by a few first-order temporal logic rules , as a compact representation of knowledge . In healthcare , the knowledge “ if a sudden fall in blood pressure is observed , vasopressors are required to be applied to patients immediately ; and then the blood pressure may return to normal afterwards '' may explain why the event “ blood pressure returns to normal from the abnormally low level `` is observed after the event “ vasopressors are used '' . This domain knowledge can be summarized as a collection of logic rules with temporal relation constraints , such as “ A happens before B ” , “ If A happens , and after 5 mins , B can happen ” , and “ If A and B happen simultaneously , then at the same time C can happen ” . However , temporal logic alone is not an ideal temporal model , as hard constraints will be too strict to model the recurrent noisy event data . Meanwhile , a large amount of literature has been devoted to modeling event data , among which temporal point process ( TPP ) models provide an elegant framework without the need to discretize the time horizon into bins and to compute the count of events within each bin . TPP models treat the interevent time as random variables and directly model the intensity function ( i.e. , occurrence rate ) of the events . However , most TPP models lack interpretability , and they can not represent domain knowledge in a human-readable form . Recently , Li et al . ( 2020 ) proposed a unified framework to marry point process intensity functions with temporal logic rules . This method employs a set of pre-specified temporal logic rules to design the intensity functions to incorporate domain knowledge . The resulting so-called temporal logic point process models are inherently interpretable and expressive . Using such logic-informed intensity functions , the model can capture nonlinear dependencies , interactions , and various temporal relations between events . It was also shown that , by specifying one or two simple temporal logic rules , many existing parametric models , e.g . the Hawkes process ( i.e . self-exciting processes ) ( Hawkes , 1971a ) and the self-correcting process ( Isham & Westcott , 1979 ) are special cases of the proposed temporal logic point process . However , in ( Li et al. , 2020 ) the temporal logic rules are required to be specified by the model builder , but they may not be known beforehand . Models constructed in such a way may suffer from model misspecification when the pre-specified logic rules are incorrect or missing . Can we automatically discover the temporal logic rules governing the event dy- namics based on historical event data alone ? This is a challenging problem since the space of possible temporal logic formulas is huge , comprising of a combination of a large number of discrete logic variables and all kinds of temporal relations . Previous inductive logic programming approaches focused mostly on deriving the relations between discrete logic variables , but have largely ignored the crucial temporal information ( Dash et al. , 2018 ; Wei et al. , 2019 ) . A generalization of these methods by further considering temporal relations in logic variables is largely missing and is in pressing needs due to the increasing availability of event data . To address the aforementioned challenges , we propose the TEmporal Logic rule LearnER ( TELLER ) algorithm for learning temporal logic rules from event sequences . TELLER is inspired by the branch-and-price algorithm ( Barnhart et al. , 1998 ) , a column generation ( CG ) algorithm for linear programming ( LP ) problems where the number of variables is too large to be considered explicitly . In the CG algorithm framework , the original LP problem is solved via two alternating procedures : the master problem and the subproblem , where the master problem is the original problem with only a subset of variables being considered and the subproblem is a new problem created to identify a new variable to be added . Similarly , TELLER also alternates between solving a master problem and a subproblem , where the master problem aims to evaluate the current rules by maximizing the likelihood and reweighting these rules as in ( Li et al. , 2020 ) , and the subproblem is to search and construct a new temporal logic rule ( by extending a current rule or adding a new short rule ) . It repeats until the likelihood can no longer be improved by adding new candidate rules . In this way , TELLER searches through the vast space of potential temporal logic rules and learns the importance weights of the discovered rules to hedge against noise in the data . Fig . 1 shows the flow . Specifically , TELLER is computationally efficient and with guarantees . It includes the candidate rules one by one in a tractable and progressive fashion , and is able to find a near-optimal set of rules . The hypothesized logic rules are in disjunctive normal form ( DNF , OR-of-ANDs ) with temporal relation constraints , and can be of various lengths . Given the temporal logic point process modeling framework , we show that the resulting objective function , i.e . likelihood , is convex , which guarantees optimal performance for our search algorithm . A set of logic rules and their importance weights are jointly learned by maximizing the likelihood . The most important collection of logic rules are guaranteed to be discovered within a fixed time limit . The uncovered rules by TELLER will not only shed light on when and what events would happen , but also why some events would happen at a specific time . Our method facilitates the human-readable knowledge exchange between experts and point process models . On one hand , the mined temporal logic rules may supplement or refine the existing knowledge ; on the other hand , human experts can easily provide feedback to modify the learned models via logic rules , as a way to add the safety of using these models in high-stake tasks like healthcare and autonomous driving . In our real experiment related to medicine , TELLER is used to learn the explanatory temporal logic rules regarding the choice and arrangement of drugs in improving patients ’ health status . Doctors were asked to justify the rules and they confirmed that these discovered logic rules are consistent with the pathogenesis and have captured the most important factors in affecting patients ’ health status . 2 BACKGROUND . 2.1 FIRST-ORDER TEMPORAL LOGIC RULES . First-order temporal logic is a form of symbolized reasoning in which each statement is a composition of temporal predicates and their relations . We formally define the interval-based temporal logic below . This type of temporal logic rule is often suitable for reasoning about events with duration , which are better modeled if the underlying temporal ontology uses time intervals . It fits well with the temporal point process models for event sequences , where the time intervals of events are explicitly modeled . Refer to ( Goranko & Rumberg , 2021 ) for a comprehensive survey of various temporal logic models . Temporal predicate . First define a set of static predicates X = { X1 , . . . , Xd } , where each static predicate Xi ( c ) is a logic random variable that defines the property or relation of entities , such as Smokes ( c ) or Friend ( c , c0 ) , where c and c0 are the entities . By adding a temporal dimension to predicate Xi , we obtain a temporal predicate { Xi ( c , t ) } t 0 , which can be viewed as a continuous-time stochastic process . Given observations , each temporal predicate will be grounded as a list of ordered 0-1 events , which take True ( 1 ) and False ( 0 ) in an alternating way with the state transition times recorded . For example , a temporal predicate { NormalBloodPressure ( c , t ) } t 0 , where entity c is referring to the patient , will take value 1 or 0 at any time t to indicate whether the patient ’ s blood pressure is normal or not , with stochastic state transition time ( see Fig . 2 ) . To simplify the notation , we will temporally drop the dependency of predicates on entities . We will write X ( c , t ) as X ( t ) instead and the grounded temporal predicate as { x ( t ) } t 0 . 2.2 TEMPORAL LOGIC POINT PROCESS . Temporal Point Process ( TPP ) provides an elegant tool to capture the dynamics of the event sequences . It is characterized by conditional intensity function , denoted by ( t|Ht ) , where history Ht is the knowledge of times of all events . By definition , we have ( t|Ht ) dt = E [ N ( [ t , t+dt ] ) |Ht ] , where N ( [ t , t+dt ] ) denotes the number of points falling in an interval [ t , t + dt ] . Here we aim to use TPP to model these 0-1 events and use the intensity function to capture the 0-1 transition rate . Temporal relation . We will use the interval-based temporal relations originally introduced in Allen ’ s seminal paper ( Allen , 1990 ) to describe the temporal relations that can exist in any two temporal predicates . There are 13 types of possible temporal re- lations , including Before , Meets , Overlap and so on ( see Appendix B for a comprehensive illustration ) , and we denote this set by R. To evaluate these temporal relations , we need to define the time interval using the state transit in/out times . For example , at time tA , the temporal predicate { XA ( t ) } t 0 takes state xA = 0 or 1 , we define ItA = ( tA1 , tA2 ] , tA 2 ( tA1 , tA2 ] , as a time interval where tA1 is the transition time that this predicate enters the state xA and tA2 is the transition time that this predicate leaves the state xA . For any two temporal predicates , given their time intervals , ItA = ( tA1 , tA2 ] and ItB = ( tB1 , tB2 ] , their temporal relation function , denoted as RA typeB ( ItA , ItB ) 2 R , is a logic function defined over the time intervals and can be evaluated by plugging in the specific time intervals . For example , RA BeforeB = { tB1 > tA2 } and RA EqualsB = { tA1 = tB1 } { tA2 = tB2 } . All these definitions of temporal relations are also given in Appendix B. Temporal logic formula . A first-order temporal logic rule f is defined as logical connectives of temporal predicates and their temporal relations . The generic form is : f : ( ¬ ) Y ( ty ) ^ Xu2X+f Xu ( tu ) ^ Xv2X f ¬Xv ( tv ) ^ Xu , Xv2Xf Ru type v ( Itu , Itv ) ( 1 ) where each temporal predicate and the temporal relations can be evaluated at ty , tu and tv along with their corresponding intervals Itu and Itv . In the above formula , negation ( ¬ ) means the head predicate Y can take a negative sign ; Xf = X+f [ X f is the set of predicates defined in f , where X f is the set of predicates as negation in the formula f , and X + f = Xf \ X f . We assume has a causal direction , and the body part of the formula indicate the evidence to be gathered from history to deduce the state of the head predicate . By this assumption , it is only valid to consider ty tu , tv . Given observed sequences of data , each predicate in X will be grounded as a list of ordered 0-1 events . The main idea of temporal logic point process ( TLPP ) ( Li et al. , 2020 ) is to use the temporal logic rules ( 1 ) as templates to selectively choose combinations of events from history as evidence to infer the transition rate of these 0-1 events . In this way , the structure of the intensity function will be informed by the pre-specified temporal logic rules . Below we provide more details . Logic-informed intensity function . Let Ht include all the historical trajectories of the grounded predicates in X up to t. TLPP leverages the rule set to model the state transition dynamics of the head predicates . Define ⇤ ( t ) : = ( t|Ht ) as the transition intensity for { Y ( t ) } t 0 to transit from 0 to 1 , given history up to t ; and µ⇤ ( t ) : = µ ( t|Ht ) vice versa . Assume the generating process of the 0-1 events are governed by the set of temporal logic rules F = { f1 , f2 , . . . } . First consider one formula f , to incorporate its knowledge in intensity construction , we define a formula effect ( FE ) , which aims to gather only the effective combinations of the historical predicate events defined by f as evidence to reason about the transition rate of the head predicate . To ease the notation , define the predicate index set of Xf as U , then the formula effect of f is computed as f ( t | y ( t ) , { xu ( tu ) , Itu } u2U 2 Ht ) : = f ( 1 y ( t ) , { xu ( tu ) , Itu } u2U ) f ( y ( t ) , { xu ( tu ) , Itu } u2U ) ( 2 ) where y ( t ) is the observed head predicate state at t , 1 y ( t ) is its counterfactual state , { xu ( tu ) , Itu } u2U 2 Ht is one historical combination of the body predicates ( including their states and time intervals ) in f , and f ( · ) is the clausal form of ( 1 ) , which is an alternative expression to ease the evaluation of the rule . For example , Y X is logically equivalent to its clausal form ¬Y _X . FE , which is essentially a difference between the what-if scenario and the true scenario , answers the question such as “ should Y ( t ) transit its state given logic formula f is true '' . One can check that the sign of FE can only be 1 , -1 or 0 , which can be interpreted as : sign ( FE ) = 1 indicates a positive effect for the head to transit , -1 indicates a negative effect , and 0 means no effect . Only the non-zero FE will refer to an effective combination . Let Hut be the historical trajectory specific to predicate u up to t , and one can aggregate all the valid ( i.e. , non-zero ) FEs from history by a summation over all combinations of the temporal predicate states and their intervals , i.e. , f ( t ) = X { ( xu ( tu ) , Itu ) 2H u t } u2U f ( t | y ( t ) , { xu ( tu ) , Itu } u2U ) , ( 3 ) where f ( t ) is the feature informed by logic formula f . One can refer to Fig . 2 for an illustration . For each f 2 F , one can build the rule-informed and history-dependent features f ( t ) as above , and assumes the rules are connected in disjunctive normal form ( OR-of-ANDs ) to deduce Y . Then the transition of { Y ( t ) } t 0 are modeled as monotonically increasing and non-negative functions of the weighted sum of the features : ⇤ ( t ) = exp ⇣ b0 + X f2F wf · f ( t ) ⌘ , µ⇤ ( t ) = exp ⇣ b1 + X f2F wf · f ( t ) ⌘ ( 4 ) where w = [ wf ] f2F 0 are the weight parameters associated with each rule f 2 F , and b0 , b1 are the spontaneous intensity , which are distinct for the dual intensities . Weight parameters will be shared by the dual intensities , but the calculated features f through ( 2 ) and ( 3 ) will always have opposite signs . One can think of the formula weight wf as the confidence level put on f . The higher the weight , the more influence that the formula will have on the intensity . The rationale of TLPP is that the intensity functions are constructed in a way that the yielding 0-1 event sequences will enable the set of logic rules to be more satisfied . 3 THE PROPOSED METHOD : TELLER Likelihood function and its convexity . Given { Ht } 0 < t < T , one can write out the likelihood w.r.t . the intensity ⇤ ( t ) and µ⇤ ( t ) . Suppose a realization of { Y ( t ) } 0 < t < T = { Y ( 0 ) = 0 , Y ( t1 ) = 1 , . . . , Y ( tn ) = 1 , tn < T < tn+1 } , the ( log ) likelihood are ( see proof in Appendix C ) : ` ( w , b0 , b1 ) = logL ( w , b0 , b1 ) , where : L ( w , b0 , b1 ) = ⇤ ( t1 ) exp ✓ Z t1 0 ⇤ ( s ) ds ◆ · µ⇤ ( t2 ) ( 5 ) exp ✓ Z t2 t1 µ⇤ ( s ) ds ◆ · · · exp ✓ Z T tn µ⇤ ( s ) ds ◆ , Note that ` is convex w.r.t w , b0 and b1 . This is true as the intensity function has a functional form of Eq . ( 4 ) , and this turns the log-likelihood function into a generalized linear model ( GLM ) with Poisson observations and log link ( Fahrmeir & Tutz , 2013 ) . It is well-known that the negative log-likelihood of a GLM is convex w.r.t the model parameters . This convexity property leads to a convergence guarantee for TELLER . We will design our TELLER with the backgrounds detailed in the previous section . TELLER uses TLPP as the backbone to evaluate the likelihood of the grounded 0-1 temporal predicate sequences . The likelihood will be maximized to jointly learn the set of temporal logic formulas and their weights . The problem is essentially combinatorial and requires enumerating an exponentially large set of combinations of the predicates and their signs . Our problem is even more challenging compared to traditional inductive logic programming due to the sequential properties of data with temporal information . TELLER solves this problem using the procedures as shown in Fig . 3 . We will start our exposition with the formulation of the original , or the master rule learning problem .

Summary: The paper uses a method inspired by column generation to continuously add temporal logic rules to their model to maximize the likelihood of the data. The goal is to generate a model that is highly interpretable. They compare against state of the art baselines and get good improvements.