Learning to Select Exogenous Events for Marked Temporal Point Process

1 Introduction . In recent years , marked temporal point processes ( MTPPs ) have shown tremendous potential in modeling the arrival process of the asynchronous events in continuous time [ 65 , 46 , 66 , 16 , 22 , 17 , 52 , 82 , 33 , 13 , 60 , 83 , 74 , 49 , 77 , 27 , 53 , 47 ] . They are extensively used in a wide variety of applications , e.g. , information diffusion in social networks [ 66 , 77 , 16 , 43 , 12 , 85 , 76 ] , improving human learning [ 76 , 74 ] , location modeling [ 83 ] , epidemic forecasting [ 49 , 66 ] , etc . At the outset , an MTPP event consists of two elements— its arrival time and the mark , where the latter encodes the category of the event , e.g. , the topic of a blog post , the sentiment associated with a tweet , disease of an individual , etc . In the context of learning an MTPP , we observe two types of events— endogenous events which occur due to influence of the previously observed events and , exogenous events which are driven by the external sources or unobserved events , rather than being influenced by the previously observed events . For example , a post in social media may be influenced by some external news source , rather than the previous messages [ 26 , 60 , 13 ] ; in epidemic process , an infection may be triggered by community transmission where the source is other than any previously known infections [ 50 ] , etc . In such applications , the observed events appear unlabeled , i.e. , they are not tagged with their sources of influence . Consequently , the existing MTPP models [ 21 , 12 , 84 , 78 , 66 ] resort to the superposition of exogenous and endogenous MTPPs . However , such approaches view an event being exogenous or endogenous with equal likelihood and therefore , they do not focus on identifying which events are more likely to be exogenous than others . Consequently , they often remain oblivious to the presence of the exogenous events , which often preclude these models from realizing their full potential . While 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . De et al . [ 13 ] tackle this problem in the particular context of opinion dynamics in social networks , their formulation focus only on the mark dynamics and , skirt the effect of exogenous events on the arrival times . As a result , their framework can not be extended to a generic MTPP model , where the arrival time is often a key component . 1.1 Our contributions . In order to respond to the above limitations , we make the following contributions . Learning in conjunction with selection of exogenous events . We design TPP-SELECT , a novel event selection algorithm which , for any given MTPP model , simultaneously filters the exogenous events from a stream of unlabeled events as well as train the model parameters over the remaining ( endogenous ) events ( Section 3 ) . More specifically , we seek to maximize the likelihood of all the events , jointly with respect to the set of exogenous events and the parameters of the associated MTPP model . Such a formulation induces a parameter estimation problem in conjunction with the combinatorial optimization problem for selecting the exogenous events . Characterization of the objective . Subsequently , we show that our problem is NP-hard due to the involvement of the candidate exogenous events as an optimization variable . We tackle this problem using several technical innovations . First , we reformulate it as an instance of a set function maximization problem , where the objective corresponds to the value of the maximum likelihood , as a function of the candidate set of exogenous events . Then , based on this representation , we show that this objective satisfies monotonicity and α-submodularity— a notion of approximate submodularity [ 25 , 44 , 29 ] for linear MTPP and a wide variety of nonlinear MTPPs ( Section 4 ) . These results reveal an unexplored connection between approximate submodularity and temporal point process , that is of independent interest . Approximation guarantee for greedy method in the face of imperfect training . Next , we show that these properties allow us to use an existing stochastic greedy algorithm [ 64 ] . While this algorithm is known to admit an approximation guarantee for submodular maximization , we show that it also enjoys an approximation guarantee for maximizing α-submodular functions . Moreover , we show that this algorithm also admits an approximation guarantee even when the training algorithm provides an imperfect estimation of the MTPP parameters . Finally , we perform a comprehensive evaluation of our proposal on both synthetic and real data . Our experiments with synthetically generated sequences show that TPP-SELECT can accurately select the exogenous events from a stream of unlabeled events , even though it is completely unsupervised . Our experiments on real datasets show that , despite training on a subset ( endogenous ) of events , TPP-SELECT outperforms several baselines as well as the corresponding model which is trained on the entire dataset in the presence of exogenous events . Our code is available in https : //github.com/noilreed/TPP-Select . 2 Related work . Apart from the works on MTPPs , our work is related to robust learning , subset selection , modeling external effects in different applications , active learning , etc . Robust learning . Robust learning focuses on training machine learning models in the presence of outliers and noisy labels [ 9 , 67 , 3 , 23 , 88 , 4 , 58 ] . However , exogenous events are not outlier data , they are events which follow a different model , other than the central model— which , in our case , captures the endogenous event dynamics . These robust learning methods predominantly consider i.i.d . data and focus on designing models and training algorithms which would generalize well across the data including the outliers . In contrast , the events in our setup have strong temporal dependencies . Moreover , our objective is to select the exogenous events , so that the model can be trained on the remaining ( endogenous ) events to obtain more accurate parameter estimates Data subset selection . In recent years , there is a flurry of work on data selection [ 81 , 80 , 79 , 48 , 2 , 40 , 51 , 57 , 7 , 6 , 36 ] , which predominantly select a smaller sized subset of data to facilitate efficient training , by optimizing submodular functions and their variants . However , TPP-SELECT aims to filter the exogenous events from a sequence of events to help the underlying MTPP model achieve better predictive performance . Modeling external effects . Our work is also connected to the methods that model the external effect in several applications , especially , in web or online social media [ 19 , 60 ] that aim to model the effect of exogenous influence on the information diffusion in social media . However , they do not aim to select the exogenous messages from a stream of unlabeled messages . Iwata et al . [ 34 ] aim to learn the effect of one user on the post of other users . Such a model can be used to quantify the external effect . Linderman and Adams [ 45 ] aim to learn a probabilistic model which captures latent influence present in data using mutually exciting processes combined with a random graph model . Furthermore it infers the latent network structure from the noisy observations . Tanaka et al . [ 75 ] model dynamics of event sequence in the particular application of purchase data modeling . Specifically , it combines three factors , namely user preferences , influence from other users and media advertisements . These proposals learn their model using EM algorithm combined with Gibbs sampling method . However , they do not provide an algorithm to explicitly demarcate between endogenous and exogenous events . Very recently , Liu and Hauskrecht [ 47 ] consider a semi-supervised setting of event outlier detection for MTPP . However , they assume that they have access to data without outliers , so that the underlying MTPP model can be trained on this clean data . In contrast , our work applies to an unsupervised setting , where the training data also contains the exogenous events . Optimization of submodular functions and their variants . Submodular functions have the ability to characterize a wide variety of set functions . Moreover , thanks to the existence of a simple greedy algorithm to maximize such functions , their use is widespread across several subset selection tasks [ 81 , 80 , 79 , 48 , 2 ] . Several popular set functions , e.g. , facility location , set cover , concave over modular , etc . are submodular [ 36 , 2 , 35 ] . In this context , there is a recent line of works which focus on maximizing different variants of submodular functions which include α-submodular functions and γ-weakly submodular functions [ 61 , 54 , 5 , 11 , 37 ] . In particular , our work is more focused around α-submodular functions . Subset selection in conjunction with model learning . Existing works [ 7 , 51 , 36 , 39 , 14 , 15 , 57 ] have also considered subset selection in conjunction with model parameter estimation . However , in the context of applications as well as formulations , these works differ from us . Killamsetty et al . [ 39 ] consider simultaneous parameter estimation on training set and subset selection using validation error in order to facilitate efficient learning without introducing significant error . On the other hand , the works in [ 36 , 51 , 57 , 7 ] focus on selecting coreset— a small representative subset of the entire data— together with parameter estimation . However , they consider i.i.d data and select subset for efficient training . In contrast , we partition sequence of events into exogenous and endogenous events for improved training of point process models . Finally , the works in [ 14 , 15 ] consider simultaneous parameter estimation and subset selection . Similar to our work , they aim to find model parameters and a desired data subset simultaneously by minimizing the underlying loss functions with respect these variables . However , they focus on human assisted machine learning , whereas our work considers exogenous event selection in the context of marked temporal point processes . Thus the problem formulation , results and proofs are significantly different from these works . Active learning . Active learning methods [ 81 , 28 , 69 , 42 , 24 ] aim to identify instances in the training data , in order to reduce the labelling cost . In contrast , our work aims to select samples which are not influenced by the previous events in a sequence of dependent observations . To that end , it trains the endogenous MTPP model on the rest of the events . More specifically , active learning methods aim to find a subset of instances so that the model trained over it gives accurate predictions for all the samples in the test set . In contrast , our work selects both exogenous and endogenous events and then trains two separate models ( λθ , mθ ) and ( µ , q ) so that they perform well specifically on endogenous and exogenous events respectively .

Summary: The authors propose an approach to separate exogenous and endogenous events generated by marked temporal point processes (MTPPs). In this context, an event consists of a timestamp and a mark, which is assumed to be a category or class. Many MTPPs are self-exciting or self-inhibiting so that new events are generated due to the influence of previous events--these are endogenous events. Other events are considered to be exogenous events. Given a set of events, the authors proposed approach attempts to select a subset of exogenous events using a stochastic greedy approach. Along the way, they establish some important theoretical properties regarding monotonicity and submodularity of MTPPs that allows them to derive an approximation guarantee for their algorithm. They also demonstrate reasonable empirical results on simulated and real data sets.