Continuous-time edge modelling using non-parametric point processes

1 Introduction . Continuous-time edge ( or dyadic ) data , which comprise information on edges ( between two individuals or nodes ) and their times of occurrence , arise naturally in many applications , such as sending and receiving messages , posting and answering comments on social networks , and purchasing groceries in online stores . Reciprocity is an important attribute of edge data , in that many edges are induced as responses to previous oppositely-directed edges ( e.g . a reply email is written in response to an initial message between two individuals ) . As a result , mutually-exciting Hawkes processes ( MEHP ) [ 13 , 12 ] , which regard the directed edges as events , are commonly used to analyse edge data [ 3 , 8 , 25 , 29 , 26 , 20 ] . For example , [ 3 ] uses the ME-HP to model the reciprocity of group-wise edges , and [ 20 ] uses sparse exchangeable random measures [ 4 , 28 ] to model the ME-HPs exogenous values . However , these existing methods are either limited in modelling flexibility , or inefficient to implement . For the former , the exogenous ( background rate ) functions in the ME-HP are typically assumed to be static and the endogenous ( excitation ) functions are specified parametrically , which may be inadequate for modelling complex real-world systems . E.g. , the exogenous intensities may exhibit daily or weekly periodicity , and the endogenous functions might not be strictly decreasing over time . For the latter , Markov chain Monte Carlo ( MCMC ) methods are usually adopted for model inference , which usually require high computational costs and large output storage . Without proper scalable inference methods , it is difficult to deploy these models to larger-scale datasets . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In this work , to address the above limitations , we develop continuous-time edge models ( CTEM ) based on non-parametric point processes . The CTEM models also use point processes to model data specific to nodal pairs . CTEMs have two major differences compared to existing approaches : Firstly , they place sigmoidal Gaussian process priors on the individual exogenous functions and endogenous functions . As a result , the exogenous functions are able to evolve over time and account for gradual changes and variations in the background environment ( such as , e.g. , trends or periodicity ) . Similarly , non-parametric modelling of the endogenous functions enables more behaviour choices ( e.g. , the rate of decrease in the excitation functions may vary over time , or there might be increases in mutual-excitation effects ) . Secondly , we develop an efficient variational inference method for model inference , which is more suited for large-scale data modelling than MCMC methods . By specifying K exogenous functions and K endogenous functions , we propose a family of three non-parametric CTEM models on N nodes , in which the directed edges from node i to node j are regarded as events for the ( i , j ) -th point process : • Nodal pair-specific Poisson process ( NP-PP ) : For each directed nodal pair ( i , j ) , i , j ∈ { 1 , . . . , N } , the intensity function for their Poisson process is constructed from the linear combination of K exogenous functions and nodes i , j ’ s latent exogenous features . • Nodal pair-specific mutually-exciting Hawkes process ( NP-HP ) : We additionally include the combination of K endogenous functions and nodes i , j ’ s latent endogenous features as the endogenous functions for the ( i , j ) -th ME-HP . • Mutually-exciting Hawkes process with edge clustering ( NP-HP-C ) : Rather than using nodal pair-specific endogenous functions , we associate each edge with a latent label , and assume that those edges with the same label are allocated to the same endogenous function . The last approach may be used to model the phenomenon that many edges are often associated with certain behavioural or other patterns . For instance , messages may be grouped into particular labelled events , or edges associated with different patterns may generate different triggering effects . Inspired by inference techniques [ 5 , 31 ] which adopt the Polya-Gamma trick to form conjugate models for sigmoidal Gaussian process-modulated Poisson processes and Hawkes processes , we develop mean-field variational algorithms to enable fast inference for the proposed models . In addition we also construct a fast variational inference solution to the Dirichlet-Hawkes process related models [ 7 , 19 , 24 ] , which typically heavily rely on Sequential Monte Carlo [ 6 ] methods . In summary , the merits of the proposed CTEM family include : ( 1 ) Modelling flexibility : The proposed models are highly flexible as they are able to describe both exogenous and endogenous functions using non-parametric sigmoidal Gaussian processes . The experimental results on four real-world data sets show clear periodic patterns in both exogenous and endogenous functions . ( 2 ) Inferential efficiency : To the best of our knowledge , this is the first work to develop fast variational inference for the continuous-time edge modelling problem , making it suitable for large-scale relational modelling settings ; ( 3 ) Modelling choice exploration : The CTEM family permits different structural choices in modelling continuous-time edge data , including nodal pair-specific intensity functions and latent label determined endogenous functions . 2 Background on mutually-exciting Hawkes processes for edge data . The mutually-exciting Hawkes process [ 12 , 13 ] is a special case of the temporal point process ( TPP ) . TPPs are realised through sets of events in a time interval [ 0 , T ] , and are parameterised by an intensity function λ ( t ) , which states that the probability of an event occurring in any infinitesimal time interval [ t , t+ dt ) is λ ( t ) dt . TPPs which are independent of the history of the process ( e.g . a Poisson point process ) are not well-suited to describe edge data ( which comprise directed events from one node to another ) , as the occurrence of edges may be the result of responses to previous oppositely-directed edges . For example , receiving a message from a person would likely increase the probability of sending a response message to that person over the following time period . The ME-HP provides a natural way to model the reciprocity of edge data . Its intensity functions λij ( t ) , which describes edge generation from node i to node j at time t , depend on the historical edges from node j to node i . We use the triplet ( im , jm , tm ) to denote an edge , meaning that among all edges , the m-th one is initiated from node im to node jm and occurred at time tm . Using fij ( t ) and gij ( t ) as the exogenous function and endogenous function from node i to node j respectively , the intensity function λij ( t ) and the likelihood L ( · ) of the ME-HP over all M edges is λij ( t ) = fij ( t ) + ∑ m′ : im′=j∩jm′=i gij ( t− tm′ ) , ( 1 ) L ( { λij ( t ) } ( i , j ) : i̸=j | { ( im , jm , tm ) } Mm=1| ) = ∏ ( i , j ) : i ̸=j e− ∫ T0 λij ( t ) dt ∏ m′ : im′=i∩jm′=j λij ( tm′ ) . ( 2 ) ME-HP with block models : Stochastic block models [ 22 , 21 , 14 ] are a classical way to describe static edge data , via partitioned nodal groups and group-wise edges . Several methods [ 3 , 8 , 25 , 29 , 26 , 20 ] have incorporated stochastic block models within the ME-HP to describe edge data reciprocity . Among these , the HPGP-IRM [ 25 ] is the most similar model to ours , with the generative process ν ∼ CRP ( 1 : N ) , λij ( t ) = γν ( i ) ν ( j ) + ∑ m′ : im′=j∩jm′=i βij ( Fi , Fj ) gij ( t− tm′ ) ( 3 ) where ν is a Chinese Restaurant Process ( CRP ) [ 23 ] group partition on all N nodes , γν ( i ) ν ( j ) is the exogenous intensity from person i ’ s group ν ( i ) to person j ’ s group ν ( j ) , βij ( Fi , Fj ) is the endogenous scalar measuring the feature similarity between person i and person j through their features Fi , Fj , and the occurrence times { tm } m follow a Hawkes process ( HP ) with intensity function λij ( t ) . Similar to other approaches , the HPGP-IRM specifies parametric functions for both exogenous and endogenous functions , and its MCMC-based approach to inference is not practical in large-scale settings . Further , the absence of edge clustering capability means that intrinsic structure among persons , especially the sharing of endogenous functions , can not be explored . 3 Non-parametric point processes for continuous-time edge modelling . The family of Continuous-Time Edge Modelling ( CTEM ) methods is inspired by three motivations : constructing highly flexible models to adequately describe the complexities of real world edge data ; enabling fast variational inference ; and exploring different modelling choices for studying continuoustime edge data . The following sections introduce three non-parametric point processes , in which the intensity functions are generated from sigmoidal Gaussian processes . 3.1 Nodal pair-specific non-parametric Poisson processes ( NP-PP ) . The first approach ( NP-PP ) is to use nodal pair-specific Poisson processes to model the continuoustime edge data . This involves K exogenous functions σ ( fk ( t ) ) , where fk ( t ) is a random function generated from a Gaussian process and σ ( x ) = 1/ ( 1 + exp ( −x ) ) is the sigmoidal function to ensure the positiveness of the intensity values . Each node i is assumed to have K-length positive latent exogenous features πi ∈ [ R+ ] 1×K , in which the k-th entry πik represents node i ’ s affiliation with the k-th exogenous feature . The intensity function of the Poisson process for generating edges from node i to node j is thus a linear combination of { σ ( fk ( t ) ) } k weighted by πi , πj , with the corresponding generative process described as follows : 1 . Generate fk ( t ) ∼ GP ( 0 , κ ( f ) k ) for group features k = 1 , . . . , K , where GP ( 0 , κ ) denotes a Gaussian process with mean 0 and covariance function κ ( · , · ) . 2 . Generate { πik } Ni=1 ∼ Gam ( aπ , bπ ) for group features k = 1 , . . . , K , where Gam ( a , b ) denotes a Gamma distribution with mean a/b and variance a/b2 . 3 . Generate the time stamps of edges from node i to node j as : { ( i , j , tm′ ) } m′ ∼ PoissonProcess ( λij ( t ) ) , with intensity function λij ( t ) = ∑ k πikπjk · σ ( fk ( t ) ) . Under this setting , the intensity functions between any nodal pair are symmetric λij ( t ) = λji ( t ) . Together , πik and πjk scale the k-th exogenous intensity onto the interval ( 0 , πikπjk ) . Nodes with similar feature-affiliation values are more likely to generate edges . The L1-norm of the latent features of a node πi indicates its edge activity , since a larger value means it is more likely to induce edges . This non-parametric setting provides greater modelling flexibility than those using static or piecewiseconstant exogenous functions . I.e. , the NP-PP allows each exogenous function to change over time , while most existing approaches [ 3 , 8 , 25 , 29 , 26 , 20 ] use a time-constant exogenous intensity value , even though real-world settings are known to vary in this manner . [ 8 ] also used Poisson processes to model the edges , however , their intensity functions are piecewise-constant with the changepoints and intensty values almost pre-fixed , which is substantially less flexible than our approach . However , without incorporating endogenous functions , the NP-PP can not describe the reciprocity property of continuous-time edges . A direct consequence might be overestimated exogenous intensities , as they need to explain all edges , and consequently lower model performance .

Summary: -This work develops a family of novel non-parametric point processes for continuous-time edge data. In details, the model captures the periodicity exhibited in its base rate using K sigmoidal Gaussian processes(SGPs) \sigmoid(f_k(t)).  -For each pair of two nodes, the nodal pair specific coefficients \p_{ik}, \pi_{jk} is used to rescale the pair-wise base intensity of (i,j) and (j,i).   -To  capture the reciprocity, the model employs a second set of functions \sigmoid(g_k(t)) drawn from SGPs, which has more flexibility in capturing exciting behaviors, compared to the exponential decaying kernel used in Hawkes processes. - A mean-field variational scheme is derived to perform model inference. -The model is demonstrated on four real data to illustrate its estimated periodically varying base intensity and the endogenous functions. They also show the superior performance of the model in terms of training log-likelihood and link prediction.