Explaining Point Processes by Learning Interpretable Temporal Logic Rules

1 INTRODUCTION . Event sequences with irregular time intervals are ubiquitous . The inter-event times usually convey rich information regarding the underlying dynamics such as disease progression ( Liu et al. , 2015 ) . It is useful to understand events ’ generating mechanisms , as well as the occurrence reason and time . In systems where domain knowledge is rich , events generating can usually be governed by a few first-order temporal logic rules , as a compact representation of knowledge . In healthcare , the knowledge “ if a sudden fall in blood pressure is observed , vasopressors are required to be applied to patients immediately ; and then the blood pressure may return to normal afterwards '' may explain why the event “ blood pressure returns to normal from the abnormally low level `` is observed after the event “ vasopressors are used '' . This domain knowledge can be summarized as a collection of logic rules with temporal relation constraints , such as “ A happens before B ” , “ If A happens , and after 5 mins , B can happen ” , and “ If A and B happen simultaneously , then at the same time C can happen ” . However , temporal logic alone is not an ideal temporal model , as hard constraints will be too strict to model the recurrent noisy event data . Meanwhile , a large amount of literature has been devoted to modeling event data , among which temporal point process ( TPP ) models provide an elegant framework without the need to discretize the time horizon into bins and to compute the count of events within each bin . TPP models treat the interevent time as random variables and directly model the intensity function ( i.e. , occurrence rate ) of the events . However , most TPP models lack interpretability , and they can not represent domain knowledge in a human-readable form . Recently , Li et al . ( 2020 ) proposed a unified framework to marry point process intensity functions with temporal logic rules . This method employs a set of pre-specified temporal logic rules to design the intensity functions to incorporate domain knowledge . The resulting so-called temporal logic point process models are inherently interpretable and expressive . Using such logic-informed intensity functions , the model can capture nonlinear dependencies , interactions , and various temporal relations between events . It was also shown that , by specifying one or two simple temporal logic rules , many existing parametric models , e.g . the Hawkes process ( i.e . self-exciting processes ) ( Hawkes , 1971a ) and the self-correcting process ( Isham & Westcott , 1979 ) are special cases of the proposed temporal logic point process . However , in ( Li et al. , 2020 ) the temporal logic rules are required to be specified by the model builder , but they may not be known beforehand . Models constructed in such a way may suffer from model misspecification when the pre-specified logic rules are incorrect or missing . Can we automatically discover the temporal logic rules governing the event dy- namics based on historical event data alone ? This is a challenging problem since the space of possible temporal logic formulas is huge , comprising of a combination of a large number of discrete logic variables and all kinds of temporal relations . Previous inductive logic programming approaches focused mostly on deriving the relations between discrete logic variables , but have largely ignored the crucial temporal information ( Dash et al. , 2018 ; Wei et al. , 2019 ) . A generalization of these methods by further considering temporal relations in logic variables is largely missing and is in pressing needs due to the increasing availability of event data . To address the aforementioned challenges , we propose the TEmporal Logic rule LearnER ( TELLER ) algorithm for learning temporal logic rules from event sequences . TELLER is inspired by the branch-and-price algorithm ( Barnhart et al. , 1998 ) , a column generation ( CG ) algorithm for linear programming ( LP ) problems where the number of variables is too large to be considered explicitly . In the CG algorithm framework , the original LP problem is solved via two alternating procedures : the master problem and the subproblem , where the master problem is the original problem with only a subset of variables being considered and the subproblem is a new problem created to identify a new variable to be added . Similarly , TELLER also alternates between solving a master problem and a subproblem , where the master problem aims to evaluate the current rules by maximizing the likelihood and reweighting these rules as in ( Li et al. , 2020 ) , and the subproblem is to search and construct a new temporal logic rule ( by extending a current rule or adding a new short rule ) . It repeats until the likelihood can no longer be improved by adding new candidate rules . In this way , TELLER searches through the vast space of potential temporal logic rules and learns the importance weights of the discovered rules to hedge against noise in the data . Fig . 1 shows the flow . Specifically , TELLER is computationally efficient and with guarantees . It includes the candidate rules one by one in a tractable and progressive fashion , and is able to find a near-optimal set of rules . The hypothesized logic rules are in disjunctive normal form ( DNF , OR-of-ANDs ) with temporal relation constraints , and can be of various lengths . Given the temporal logic point process modeling framework , we show that the resulting objective function , i.e . likelihood , is convex , which guarantees optimal performance for our search algorithm . A set of logic rules and their importance weights are jointly learned by maximizing the likelihood . The most important collection of logic rules are guaranteed to be discovered within a fixed time limit . The uncovered rules by TELLER will not only shed light on when and what events would happen , but also why some events would happen at a specific time . Our method facilitates the human-readable knowledge exchange between experts and point process models . On one hand , the mined temporal logic rules may supplement or refine the existing knowledge ; on the other hand , human experts can easily provide feedback to modify the learned models via logic rules , as a way to add the safety of using these models in high-stake tasks like healthcare and autonomous driving . In our real experiment related to medicine , TELLER is used to learn the explanatory temporal logic rules regarding the choice and arrangement of drugs in improving patients ’ health status . Doctors were asked to justify the rules and they confirmed that these discovered logic rules are consistent with the pathogenesis and have captured the most important factors in affecting patients ’ health status . 2 BACKGROUND . 2.1 FIRST-ORDER TEMPORAL LOGIC RULES . First-order temporal logic is a form of symbolized reasoning in which each statement is a composition of temporal predicates and their relations . We formally define the interval-based temporal logic below . This type of temporal logic rule is often suitable for reasoning about events with duration , which are better modeled if the underlying temporal ontology uses time intervals . It fits well with the temporal point process models for event sequences , where the time intervals of events are explicitly modeled . Refer to ( Goranko & Rumberg , 2021 ) for a comprehensive survey of various temporal logic models . Temporal predicate . First define a set of static predicates X = { X1 , . . . , Xd } , where each static predicate Xi ( c ) is a logic random variable that defines the property or relation of entities , such as Smokes ( c ) or Friend ( c , c0 ) , where c and c0 are the entities . By adding a temporal dimension to predicate Xi , we obtain a temporal predicate { Xi ( c , t ) } t 0 , which can be viewed as a continuous-time stochastic process . Given observations , each temporal predicate will be grounded as a list of ordered 0-1 events , which take True ( 1 ) and False ( 0 ) in an alternating way with the state transition times recorded . For example , a temporal predicate { NormalBloodPressure ( c , t ) } t 0 , where entity c is referring to the patient , will take value 1 or 0 at any time t to indicate whether the patient ’ s blood pressure is normal or not , with stochastic state transition time ( see Fig . 2 ) . To simplify the notation , we will temporally drop the dependency of predicates on entities . We will write X ( c , t ) as X ( t ) instead and the grounded temporal predicate as { x ( t ) } t 0 . 2.2 TEMPORAL LOGIC POINT PROCESS . Temporal Point Process ( TPP ) provides an elegant tool to capture the dynamics of the event sequences . It is characterized by conditional intensity function , denoted by ( t|Ht ) , where history Ht is the knowledge of times of all events . By definition , we have ( t|Ht ) dt = E [ N ( [ t , t+dt ] ) |Ht ] , where N ( [ t , t+dt ] ) denotes the number of points falling in an interval [ t , t + dt ] . Here we aim to use TPP to model these 0-1 events and use the intensity function to capture the 0-1 transition rate . Temporal relation . We will use the interval-based temporal relations originally introduced in Allen ’ s seminal paper ( Allen , 1990 ) to describe the temporal relations that can exist in any two temporal predicates . There are 13 types of possible temporal re- lations , including Before , Meets , Overlap and so on ( see Appendix B for a comprehensive illustration ) , and we denote this set by R. To evaluate these temporal relations , we need to define the time interval using the state transit in/out times . For example , at time tA , the temporal predicate { XA ( t ) } t 0 takes state xA = 0 or 1 , we define ItA = ( tA1 , tA2 ] , tA 2 ( tA1 , tA2 ] , as a time interval where tA1 is the transition time that this predicate enters the state xA and tA2 is the transition time that this predicate leaves the state xA . For any two temporal predicates , given their time intervals , ItA = ( tA1 , tA2 ] and ItB = ( tB1 , tB2 ] , their temporal relation function , denoted as RA typeB ( ItA , ItB ) 2 R , is a logic function defined over the time intervals and can be evaluated by plugging in the specific time intervals . For example , RA BeforeB = { tB1 > tA2 } and RA EqualsB = { tA1 = tB1 } { tA2 = tB2 } . All these definitions of temporal relations are also given in Appendix B. Temporal logic formula . A first-order temporal logic rule f is defined as logical connectives of temporal predicates and their temporal relations . The generic form is : f : ( ¬ ) Y ( ty ) ^ Xu2X+f Xu ( tu ) ^ Xv2X f ¬Xv ( tv ) ^ Xu , Xv2Xf Ru type v ( Itu , Itv ) ( 1 ) where each temporal predicate and the temporal relations can be evaluated at ty , tu and tv along with their corresponding intervals Itu and Itv . In the above formula , negation ( ¬ ) means the head predicate Y can take a negative sign ; Xf = X+f [ X f is the set of predicates defined in f , where X f is the set of predicates as negation in the formula f , and X + f = Xf \ X f . We assume has a causal direction , and the body part of the formula indicate the evidence to be gathered from history to deduce the state of the head predicate . By this assumption , it is only valid to consider ty tu , tv . Given observed sequences of data , each predicate in X will be grounded as a list of ordered 0-1 events . The main idea of temporal logic point process ( TLPP ) ( Li et al. , 2020 ) is to use the temporal logic rules ( 1 ) as templates to selectively choose combinations of events from history as evidence to infer the transition rate of these 0-1 events . In this way , the structure of the intensity function will be informed by the pre-specified temporal logic rules . Below we provide more details . Logic-informed intensity function . Let Ht include all the historical trajectories of the grounded predicates in X up to t. TLPP leverages the rule set to model the state transition dynamics of the head predicates . Define ⇤ ( t ) : = ( t|Ht ) as the transition intensity for { Y ( t ) } t 0 to transit from 0 to 1 , given history up to t ; and µ⇤ ( t ) : = µ ( t|Ht ) vice versa . Assume the generating process of the 0-1 events are governed by the set of temporal logic rules F = { f1 , f2 , . . . } . First consider one formula f , to incorporate its knowledge in intensity construction , we define a formula effect ( FE ) , which aims to gather only the effective combinations of the historical predicate events defined by f as evidence to reason about the transition rate of the head predicate . To ease the notation , define the predicate index set of Xf as U , then the formula effect of f is computed as f ( t | y ( t ) , { xu ( tu ) , Itu } u2U 2 Ht ) : = f ( 1 y ( t ) , { xu ( tu ) , Itu } u2U ) f ( y ( t ) , { xu ( tu ) , Itu } u2U ) ( 2 ) where y ( t ) is the observed head predicate state at t , 1 y ( t ) is its counterfactual state , { xu ( tu ) , Itu } u2U 2 Ht is one historical combination of the body predicates ( including their states and time intervals ) in f , and f ( · ) is the clausal form of ( 1 ) , which is an alternative expression to ease the evaluation of the rule . For example , Y X is logically equivalent to its clausal form ¬Y _X . FE , which is essentially a difference between the what-if scenario and the true scenario , answers the question such as “ should Y ( t ) transit its state given logic formula f is true '' . One can check that the sign of FE can only be 1 , -1 or 0 , which can be interpreted as : sign ( FE ) = 1 indicates a positive effect for the head to transit , -1 indicates a negative effect , and 0 means no effect . Only the non-zero FE will refer to an effective combination . Let Hut be the historical trajectory specific to predicate u up to t , and one can aggregate all the valid ( i.e. , non-zero ) FEs from history by a summation over all combinations of the temporal predicate states and their intervals , i.e. , f ( t ) = X { ( xu ( tu ) , Itu ) 2H u t } u2U f ( t | y ( t ) , { xu ( tu ) , Itu } u2U ) , ( 3 ) where f ( t ) is the feature informed by logic formula f . One can refer to Fig . 2 for an illustration . For each f 2 F , one can build the rule-informed and history-dependent features f ( t ) as above , and assumes the rules are connected in disjunctive normal form ( OR-of-ANDs ) to deduce Y . Then the transition of { Y ( t ) } t 0 are modeled as monotonically increasing and non-negative functions of the weighted sum of the features : ⇤ ( t ) = exp ⇣ b0 + X f2F wf · f ( t ) ⌘ , µ⇤ ( t ) = exp ⇣ b1 + X f2F wf · f ( t ) ⌘ ( 4 ) where w = [ wf ] f2F 0 are the weight parameters associated with each rule f 2 F , and b0 , b1 are the spontaneous intensity , which are distinct for the dual intensities . Weight parameters will be shared by the dual intensities , but the calculated features f through ( 2 ) and ( 3 ) will always have opposite signs . One can think of the formula weight wf as the confidence level put on f . The higher the weight , the more influence that the formula will have on the intensity . The rationale of TLPP is that the intensity functions are constructed in a way that the yielding 0-1 event sequences will enable the set of logic rules to be more satisfied . 3 THE PROPOSED METHOD : TELLER Likelihood function and its convexity . Given { Ht } 0 < t < T , one can write out the likelihood w.r.t . the intensity ⇤ ( t ) and µ⇤ ( t ) . Suppose a realization of { Y ( t ) } 0 < t < T = { Y ( 0 ) = 0 , Y ( t1 ) = 1 , . . . , Y ( tn ) = 1 , tn < T < tn+1 } , the ( log ) likelihood are ( see proof in Appendix C ) : ` ( w , b0 , b1 ) = logL ( w , b0 , b1 ) , where : L ( w , b0 , b1 ) = ⇤ ( t1 ) exp ✓ Z t1 0 ⇤ ( s ) ds ◆ · µ⇤ ( t2 ) ( 5 ) exp ✓ Z t2 t1 µ⇤ ( s ) ds ◆ · · · exp ✓ Z T tn µ⇤ ( s ) ds ◆ , Note that ` is convex w.r.t w , b0 and b1 . This is true as the intensity function has a functional form of Eq . ( 4 ) , and this turns the log-likelihood function into a generalized linear model ( GLM ) with Poisson observations and log link ( Fahrmeir & Tutz , 2013 ) . It is well-known that the negative log-likelihood of a GLM is convex w.r.t the model parameters . This convexity property leads to a convergence guarantee for TELLER . We will design our TELLER with the backgrounds detailed in the previous section . TELLER uses TLPP as the backbone to evaluate the likelihood of the grounded 0-1 temporal predicate sequences . The likelihood will be maximized to jointly learn the set of temporal logic formulas and their weights . The problem is essentially combinatorial and requires enumerating an exponentially large set of combinations of the predicates and their signs . Our problem is even more challenging compared to traditional inductive logic programming due to the sequential properties of data with temporal information . TELLER solves this problem using the procedures as shown in Fig . 3 . We will start our exposition with the formulation of the original , or the master rule learning problem .

Summary: This paper builds upon the work of [1] and proposes an algorithm to simultaneously estimate the logic rules and the model parameters.  The estimation problem is formulated as the contained optimization problem (6), with both continuous and discrete parameters to optimize over. The proposed TELLER algorithm divided the original problem into a sequence of Restricted master problems and gradually expand the search space for a global optimum.  Subjective constraints and expert opinions can be incorporated into the search algorithm as well.   [1] Shuang Li, Lu Wang, Ruizhi Zhang, Xiaofu Chang, Xuqin Liu, Yao Xie, Yuan Qi, and Le Song. Temporal logic point processes. In International Conference on Machine Learning, pp. 5990–6000. PMLR, 2020.