Continuous-time edge modelling using non-parametric point processes

1 Introduction . Continuous-time edge ( or dyadic ) data , which comprise information on edges ( between two individuals or nodes ) and their times of occurrence , arise naturally in many applications , such as sending and receiving messages , posting and answering comments on social networks , and purchasing groceries in online stores . Reciprocity is an important attribute of edge data , in that many edges are induced as responses to previous oppositely-directed edges ( e.g . a reply email is written in response to an initial message between two individuals ) . As a result , mutually-exciting Hawkes processes ( MEHP ) [ 13 , 12 ] , which regard the directed edges as events , are commonly used to analyse edge data [ 3 , 8 , 25 , 29 , 26 , 20 ] . For example , [ 3 ] uses the ME-HP to model the reciprocity of group-wise edges , and [ 20 ] uses sparse exchangeable random measures [ 4 , 28 ] to model the ME-HPs exogenous values . However , these existing methods are either limited in modelling flexibility , or inefficient to implement . For the former , the exogenous ( background rate ) functions in the ME-HP are typically assumed to be static and the endogenous ( excitation ) functions are specified parametrically , which may be inadequate for modelling complex real-world systems . E.g. , the exogenous intensities may exhibit daily or weekly periodicity , and the endogenous functions might not be strictly decreasing over time . For the latter , Markov chain Monte Carlo ( MCMC ) methods are usually adopted for model inference , which usually require high computational costs and large output storage . Without proper scalable inference methods , it is difficult to deploy these models to larger-scale datasets . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In this work , to address the above limitations , we develop continuous-time edge models ( CTEM ) based on non-parametric point processes . The CTEM models also use point processes to model data specific to nodal pairs . CTEMs have two major differences compared to existing approaches : Firstly , they place sigmoidal Gaussian process priors on the individual exogenous functions and endogenous functions . As a result , the exogenous functions are able to evolve over time and account for gradual changes and variations in the background environment ( such as , e.g. , trends or periodicity ) . Similarly , non-parametric modelling of the endogenous functions enables more behaviour choices ( e.g. , the rate of decrease in the excitation functions may vary over time , or there might be increases in mutual-excitation effects ) . Secondly , we develop an efficient variational inference method for model inference , which is more suited for large-scale data modelling than MCMC methods . By specifying K exogenous functions and K endogenous functions , we propose a family of three non-parametric CTEM models on N nodes , in which the directed edges from node i to node j are regarded as events for the ( i , j ) -th point process : • Nodal pair-specific Poisson process ( NP-PP ) : For each directed nodal pair ( i , j ) , i , j ∈ { 1 , . . . , N } , the intensity function for their Poisson process is constructed from the linear combination of K exogenous functions and nodes i , j ’ s latent exogenous features . • Nodal pair-specific mutually-exciting Hawkes process ( NP-HP ) : We additionally include the combination of K endogenous functions and nodes i , j ’ s latent endogenous features as the endogenous functions for the ( i , j ) -th ME-HP . • Mutually-exciting Hawkes process with edge clustering ( NP-HP-C ) : Rather than using nodal pair-specific endogenous functions , we associate each edge with a latent label , and assume that those edges with the same label are allocated to the same endogenous function . The last approach may be used to model the phenomenon that many edges are often associated with certain behavioural or other patterns . For instance , messages may be grouped into particular labelled events , or edges associated with different patterns may generate different triggering effects . Inspired by inference techniques [ 5 , 31 ] which adopt the Polya-Gamma trick to form conjugate models for sigmoidal Gaussian process-modulated Poisson processes and Hawkes processes , we develop mean-field variational algorithms to enable fast inference for the proposed models . In addition we also construct a fast variational inference solution to the Dirichlet-Hawkes process related models [ 7 , 19 , 24 ] , which typically heavily rely on Sequential Monte Carlo [ 6 ] methods . In summary , the merits of the proposed CTEM family include : ( 1 ) Modelling flexibility : The proposed models are highly flexible as they are able to describe both exogenous and endogenous functions using non-parametric sigmoidal Gaussian processes . The experimental results on four real-world data sets show clear periodic patterns in both exogenous and endogenous functions . ( 2 ) Inferential efficiency : To the best of our knowledge , this is the first work to develop fast variational inference for the continuous-time edge modelling problem , making it suitable for large-scale relational modelling settings ; ( 3 ) Modelling choice exploration : The CTEM family permits different structural choices in modelling continuous-time edge data , including nodal pair-specific intensity functions and latent label determined endogenous functions . 2 Background on mutually-exciting Hawkes processes for edge data . The mutually-exciting Hawkes process [ 12 , 13 ] is a special case of the temporal point process ( TPP ) . TPPs are realised through sets of events in a time interval [ 0 , T ] , and are parameterised by an intensity function λ ( t ) , which states that the probability of an event occurring in any infinitesimal time interval [ t , t+ dt ) is λ ( t ) dt . TPPs which are independent of the history of the process ( e.g . a Poisson point process ) are not well-suited to describe edge data ( which comprise directed events from one node to another ) , as the occurrence of edges may be the result of responses to previous oppositely-directed edges . For example , receiving a message from a person would likely increase the probability of sending a response message to that person over the following time period . The ME-HP provides a natural way to model the reciprocity of edge data . Its intensity functions λij ( t ) , which describes edge generation from node i to node j at time t , depend on the historical edges from node j to node i . We use the triplet ( im , jm , tm ) to denote an edge , meaning that among all edges , the m-th one is initiated from node im to node jm and occurred at time tm . Using fij ( t ) and gij ( t ) as the exogenous function and endogenous function from node i to node j respectively , the intensity function λij ( t ) and the likelihood L ( · ) of the ME-HP over all M edges is λij ( t ) = fij ( t ) + ∑ m′ : im′=j∩jm′=i gij ( t− tm′ ) , ( 1 ) L ( { λij ( t ) } ( i , j ) : i̸=j | { ( im , jm , tm ) } Mm=1| ) = ∏ ( i , j ) : i ̸=j e− ∫ T0 λij ( t ) dt ∏ m′ : im′=i∩jm′=j λij ( tm′ ) . ( 2 ) ME-HP with block models : Stochastic block models [ 22 , 21 , 14 ] are a classical way to describe static edge data , via partitioned nodal groups and group-wise edges . Several methods [ 3 , 8 , 25 , 29 , 26 , 20 ] have incorporated stochastic block models within the ME-HP to describe edge data reciprocity . Among these , the HPGP-IRM [ 25 ] is the most similar model to ours , with the generative process ν ∼ CRP ( 1 : N ) , λij ( t ) = γν ( i ) ν ( j ) + ∑ m′ : im′=j∩jm′=i βij ( Fi , Fj ) gij ( t− tm′ ) ( 3 ) where ν is a Chinese Restaurant Process ( CRP ) [ 23 ] group partition on all N nodes , γν ( i ) ν ( j ) is the exogenous intensity from person i ’ s group ν ( i ) to person j ’ s group ν ( j ) , βij ( Fi , Fj ) is the endogenous scalar measuring the feature similarity between person i and person j through their features Fi , Fj , and the occurrence times { tm } m follow a Hawkes process ( HP ) with intensity function λij ( t ) . Similar to other approaches , the HPGP-IRM specifies parametric functions for both exogenous and endogenous functions , and its MCMC-based approach to inference is not practical in large-scale settings . Further , the absence of edge clustering capability means that intrinsic structure among persons , especially the sharing of endogenous functions , can not be explored . 3 Non-parametric point processes for continuous-time edge modelling . The family of Continuous-Time Edge Modelling ( CTEM ) methods is inspired by three motivations : constructing highly flexible models to adequately describe the complexities of real world edge data ; enabling fast variational inference ; and exploring different modelling choices for studying continuoustime edge data . The following sections introduce three non-parametric point processes , in which the intensity functions are generated from sigmoidal Gaussian processes . 3.1 Nodal pair-specific non-parametric Poisson processes ( NP-PP ) . The first approach ( NP-PP ) is to use nodal pair-specific Poisson processes to model the continuoustime edge data . This involves K exogenous functions σ ( fk ( t ) ) , where fk ( t ) is a random function generated from a Gaussian process and σ ( x ) = 1/ ( 1 + exp ( −x ) ) is the sigmoidal function to ensure the positiveness of the intensity values . Each node i is assumed to have K-length positive latent exogenous features πi ∈ [ R+ ] 1×K , in which the k-th entry πik represents node i ’ s affiliation with the k-th exogenous feature . The intensity function of the Poisson process for generating edges from node i to node j is thus a linear combination of { σ ( fk ( t ) ) } k weighted by πi , πj , with the corresponding generative process described as follows : 1 . Generate fk ( t ) ∼ GP ( 0 , κ ( f ) k ) for group features k = 1 , . . . , K , where GP ( 0 , κ ) denotes a Gaussian process with mean 0 and covariance function κ ( · , · ) . 2 . Generate { πik } Ni=1 ∼ Gam ( aπ , bπ ) for group features k = 1 , . . . , K , where Gam ( a , b ) denotes a Gamma distribution with mean a/b and variance a/b2 . 3 . Generate the time stamps of edges from node i to node j as : { ( i , j , tm′ ) } m′ ∼ PoissonProcess ( λij ( t ) ) , with intensity function λij ( t ) = ∑ k πikπjk · σ ( fk ( t ) ) . Under this setting , the intensity functions between any nodal pair are symmetric λij ( t ) = λji ( t ) . Together , πik and πjk scale the k-th exogenous intensity onto the interval ( 0 , πikπjk ) . Nodes with similar feature-affiliation values are more likely to generate edges . The L1-norm of the latent features of a node πi indicates its edge activity , since a larger value means it is more likely to induce edges . This non-parametric setting provides greater modelling flexibility than those using static or piecewiseconstant exogenous functions . I.e. , the NP-PP allows each exogenous function to change over time , while most existing approaches [ 3 , 8 , 25 , 29 , 26 , 20 ] use a time-constant exogenous intensity value , even though real-world settings are known to vary in this manner . [ 8 ] also used Poisson processes to model the edges , however , their intensity functions are piecewise-constant with the changepoints and intensty values almost pre-fixed , which is substantially less flexible than our approach . However , without incorporating endogenous functions , the NP-PP can not describe the reciprocity property of continuous-time edges . A direct consequence might be overestimated exogenous intensities , as they need to explain all edges , and consequently lower model performance .

Summary: The authors propose a family of models based on mutually-exciting Hawkes process that aim at modeling continuous time edges. The framework is based on a variational inference procedure for a fast and efficient computation. The models have been quantitatively and qualitatively evaluated with respect to baselines, showing that, in general, they outperform the baselines in terms of accuracy.