Continuous-time edge modelling using non-parametric point processes
Authors: Xuhui Fan
Bin Li
Feng Zhou
Scott A Sisson
Venue: NeurIPS 2021 Poster (2021)
Keywords: Continuous-time Edges
Sigmoidal Gaussian Process
Variational Inference

Abstract: The mutually-exciting Hawkes process (ME-HP) is a natural choice to model reciprocity, which is an important attribute of continuous-time edge (dyadic) data. However, existing ways of implementing the ME-HP for such data are either inflexible, as the exogenous (background) rate functions are typically constant and the endogenous (excitation) rate functions are specified parametrically, or inefficient, as inference usually relies on Markov chain Monte Carlo methods with high computational costs. To address these limitations, we discuss various approaches to model design, and develop three variants of non-parametric point processes for continuous-time edge modelling (CTEM). The resulting models are highly adaptable as they generate intensity functions through sigmoidal Gaussian processes, and so provide greater modelling flexibility than parametric forms. The models are implemented via a fast variational inference method enabled by a novel edge modelling construction. The superior performance of the proposed CTEM models is demonstrated through extensive experimental evaluations on four real-world continuous-time edge data sets.

Continuous-time edge modelling using non-parametric point processes
Xuhui Fan', Bin Li2, Feng Zhou², and Scott A. Sisson
UNSW Data Science Hub, and School of Mathematics & Statistics, University of New South Wales
Shanghai Key Lab of III, School of Computer Science, Fudan University
DDepartment of Computer Science & Technology, Tsinghua University,
fxn scott.sisson/ @ @unsw.edu.aut libin @ @fudan. fudan.edu.cn, (annf/nng6288ssiinnuuaaaa
Abstract
The mutually-exciiin Hawkes process (ME-HP) is natural choice to model reciprocity, which is an important attribute of continuous-time edge (dyadic) data. However, existing ways of implementing the ME-HP for such data are either inflexible, as the exogenous (background) rate functions are typically constant and the endogenous (excitation) rate functions specified parametrically, inefficient, as inference usually relies on Markov chain Monte Carlo methods with high computational costs. To address these limitations, we discuss various approaches to model design, and develop three variants of non-parametric point edge modelling (CTEM). The resulting models are highly adaptable as they generate intensity functions through sigmoidal Gaussian processes, and so provide greater modelling flexibility than parametric forms. The models are implemented via a fast variational inference method enabled by a novel edge modelling construction. The superior performance of the proposed CTEM models is demonstrated through extensive experimental evaluations on four real-world continuous-tin time edge data sets.
processes for continuous-
1 Introduction
Continuous-time edge (or dyadic) data, which comprise information edges (between individu- als or nodes) and their times of occurrence, arise naturally in many applications, such as sending and receiving messages, posting and answering comments on social networks, and purchasing groceries in online stores. Reciprocity is an important attribute of edge data, in that many edges are induced as responses to previous oppositely-directed edges (e.g. reply email is written in response to an initial message between two individuals). As a result, mutually-exciting Hawkes processes (ME- HP)
B
12 which regard the directed edges as events, are commonly used to analyse edge data 26 20]. For example. (3] uses the ME-HP to model the reciprocity of group-wise edges, model the ME-HPs exogenous values.
3
8
25
uses sparse exchangeable random measures
However, these existing methods are either limited in modelling flexibility, or inefficient to implement. For the former, the exogenous (background rate) functions in the ME-HP typically assumed to be static and the endogenous (excitation) functions are specified parametrically, which may be inadequate for modelling complex real-world systems. E.g., the exogenous intensities may exhibit daily or weekly periodicity, and the endogenous functions might not be strictly decreasing over time. For the latter, Markov chain Monte Carlo (MCMC) methods are usually adopted for model inference. which usually require high computational costs and large output storage. Without proper scalable inference methods, difficult deploy these models larger-scale datasets.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

In this work, to address the above limitations, we develop continuous-ting time edge models (CTEM) based on non non-parametric point processes. The CTEM models also use point processes to model data specifie to nodal pairs. CTEMs have two major differences compared to existing approaches: Firstly, they place sigmoidal Gaussian process priors on the individual exogenous functions and endogenous functions. As result, the exogenous functions are able to evolve over time and account for gradual changes and variations in the background environment (such as, c.g., trends or periodicity). Similarly, non-parametric modelling of the endogenous functions enables more behaviour choices (e.g., the rate of decrease in the excitation functions may vary over time, or there might be increases in mutual-excitation effects). Secondly, we develop efficient variational inference method for model inference, which is more suited for large-scale data modelling than MCMC methods.
By specifying K exogenous functions and K endogenous functions, we propose family of three non-paraman non-parametric CTEM models on N nodes, in which the directed edges from node to node j are regarded as events for the (i,j) j-th point process:
Nodal pair-specific Poisson process (NP-PP): For each directed nodal pair (1.j), i.j E NJ. the intensity function for their Poisson process is constructed from the linear combination of K exogenous functions and nodes j'' latent exogenous features.
Nodal pair-specific mutually-excitinng Hawkes process (NP-HP); We additionally include the combination of K endogenous functions and nodes i, latent endogenous features as the endogenous functions for the (i, ME-HP.
Mutually-exciting Hawkes process with edge clustering (NP-HP-C): Rather than using nodal pair-specific endogenous functions, we associate each edge with a latent label, and assume that those edges with the same label are allocated to the same endogenous function.
The last approach may be used to model the phenomenon that many edges are often associated with certain behavioural or other patterns. For instance, messages may be grouped into particular labelled events, or edges associated with different patterns may generate different triggering effects.
Inspired by inference techniques [5 31 which adopt the Polya-Gamma trick to form conjugate models for sigmoidal Gaussian moocess-oddulated Poisson processes and Hawkes processes, we develop mean-field variational algorithms to enable fast inference for the proposed models. In addition we also construct a fast variational inference solution to the Dirichlet-Hawkes process related
models 7 24), which typically heavily rely on Sequential Monte Carlo (6) methods.
In summary, the merits of the proposed CTEM family include: (1) Modelling flexibility: The proposed models are highly flexible as they are able to describe both exogenous and endogenous functions using non-parnee -on-aarmmetric sigmoidal Gaussian processes. The experimental results on four real-world data sets show clear periodic patterns in both exogenous and endogenous functions. (2) Inferential efficiency: To the best of our knowledge, this is the first work develop fast variational inference for the continuous-time edge modelling problem, making it suitable for large-scale relational modelling settings; (3) Modelling choice exploration: The CTEM family permits different structural choices in modelling continuous-timg edge data, including nodal pair-specifi intensity functions and latent label determined endogenous functions.
2 Background on mutually-exciting Hawkes processes for edge data
The nutually-excitimn Hawkes process 12 13
is a special case of the temporal point process (TPP). TPPs are realised through sets of events in a time interval [0, 7]. and are parameterised by an intensity function A(t). which states that the probability of an event occurring in any infinitesimal time interval [t, + dt) is A(t)dt. TPPs which are independent of the history of the process (c.g. Poisson point process) are not well-suited to describe edge data (which comprise directed events from one node to another), as the occurrence of edges may be the result of responses to previous oppositely-directe edges. For example, receiving a message from a person would likely increase the probability of sending a response message that person over the following time period.
The ME-HP provides natural way to model the reciprocity of edge data. Its intensity functions Asy (L), which describes edge generation from node i to node j at time I, depend on the historical edges from node j to node i. We use the triplet (imiJm; bm) to denote an edge, meaning that among all edges, the m-th one is initiated from node im to node im and occurred at time tm- Using (t)

---

and (ty(t) as the exogenous function and endogenous function from node i to node j respectively, the intensity function A (t) and the likelihood (-) of the ME-HP over all M edges
)1)(1) [z)(1)+ +
Σ
gis lnn));
me' m'itm' Umi
(lAaa(())aanssslll( Ims 4m) (
Im=1))
II ((f))
e (cs(t)d
II m (mn
Aey(bm))
ME-HP with block models: Stochastic block models 22] are a classical way to describe static
edge data, via partitioned nodal groups and group- wise edges. Several methods [ 8 25
29 26 20)
have incorporated stochastic block models within the ME-HP to describe edge data reciprocity. Among these, the HPGP-IRM [25] is the most similar model to ours, with the generative process
CRP(1 N),
Xıj (t) ()())()) +
Σ
F Fy)sss(t (mm))))))
(3)
if
where y is a Chinese Restaurant Process (CRP)
23 group partition on all N nodes, (o(()) is
the exogenous intensity from person i's group vi)) to person j's group v(j), (F; F2) is the endogenous scalar measuring the feature similarity between person and person through their features Fú, Pj. and the occurrence times }mm follow a Hawkes process (HP) with intensity function Aıy (t). Similar to other approaches, the HPGP-IRM specifies parametric functions for both exogenous and endogenous functions, and its MCMC-based approach to inference is not practical in large-scale settings. Further, the absence of edge clustering capability means that intrinsic structure among persons, especially the sharing endogenous functions, can not be explored.
3 Non-parametric point processes for continuous-time edge modelling
The family of Continuous-Time Edge Modelling (CTEM) methods inspired by three motivations: constructing highly flexible models to adequately describe the complexities of real world edge data; enabling fast variational inference; and exploring different modelling choices for studying continuous- time edge data. The following sections introduce three con-paramttric point processes, in which the intensity functions are generated from sigmoidal Gaussian processes.
3.1 Nodal pair-specific non-parametric Poisson processes (NP-PP)
The first approach (NP-PP) is to nodal pair-specific Poisson processes to model the continuous- time edge data. This involves exogenous functions a (t)), where (t)(t) is a random function generated from a Gaussian process and (x) 1/(1 + exp(-x)) is the sigmoidal function to ensure the positiveness of the intensity values. Each node is assumed to have K-length positive latent exogenous features n, [ in which the k-th entry mik represents node i's affiliation with the k-th exogenous feature. The intensity function of the Poisson process for generating edges from node i node is thus a linear combination of (t)) weighted by n, Ty. with the corresponding generative process described as follows:
1. Generate fa(t) GP,O, xg6 ( for group features == 1...., ... K, where GP(O, x) denotes Gaussian process with mean O and covariance function k(-,)).
N 2. Generate { denotes Gamma distribution with mean a/b and variance a/b².
Gam(ar, bx) for group features k: = 1,...,
......,, K, where Gam(a, b)
3. Generate the time stamps of edges from node node as: (m,)),mm)))m PoissonProcess(Appp
with intensity function Xzy Σ. J o (fv(t)).
Under this setting, the intensity functions between any nodal pair are symmetric (z)(1)= (t). Together, ma and Tyt scale the k-th exogenous intensity onto the interval (0, TT.TTR). Nodes with similar feature-affiliation values are more likely to generate edges. The L -nom of the latent features of node indicates its edge activity, since larger value means is more likely to induce edges.

---

This non con-pramettic setting provides greater modelling flexibility than those using static piecewise- constant exogenous functions. Le.. the NP-PP allows each exogenous function to change over time, while most existing approaches B g
20 use time-constant exogenous intensity value, even though real-world settings are known to vary this manner. g also used Poisson processes to model the edges, however, their intensity functions are piecewise-constant with the changepoints and intensty values almost pre-fixed, which is substantially less flexible than our approach.
25
29
26
However, without incorporating endogenous functions, the NP-PP cannot describe the reciprocity property continuous time edges. A direct consequence might be overestimated exogenous intensi- ties, as they need to explain all edges, and consequently lower model performance.
3.2 Nodal pair-specific non-parametric Hawkes process (NP-HP)
Our second approach is use mutually-exciting Hawkes processes (ME-HP) to model the continuous- time edge data. In addition to the non-paree non-parametric exogenous functions in the first approach, the
NP-HP includes nodal pair-specific endogenous functions model the reciprocity of edge data. In
particular, the endogenous function for the (i,j)-th nodal pair is structured as E. PikPjk ( ((). where (o(g a: are random functions from sigmoidal Gaussian process and p; E ja* is node i's K K-length latent endogenous features. As a result, the occurrence of edge at time Im is either instantiated by the exogenous effect (i.c. Σn TT.TTT (ff (fm))) or triggered by one of its historical oppositely-directed edges (through the endogenous effect En PikPak o (gg
tmm) )). where m' E Hir (tm) and Hi (t) {mn (tmx < t) n (im' Imm) (j, denotes the set of historical (i.e. occurring before time stamp t) oppositely-directed (i.e. from node j to node i) edges).
Since the occurrence of an edge increases the intensity function value of its future oppositely- directed edges, the NP-HP can deseribe reciprocity well. It's generative process described as follows:
1. Generate (t) ~ GP(O, Ki (P),
(1)(t) ~ GP(O, (0)), == 1, 1....,, K.
2. Generate Tik ~ Gam (az, bm), Pik ~ Gam(ap: bp); i i= N,k= K.
3. Simultaneously generate two sets of triplets {(i, tmm)m and {(j, t.m hm, which are edges initiated from node i to node j. and from node j to node respectively, from Hawkes processes with the following intensity functions:
\1 = Σ T ((J(())) + Σ
Σ
PaPPi olgx(t bm:)),
(4)
on m/am'EM,5(1)
(ge(t) = Σ TzkTa offi(tt))
Σ
Σ
PskPak a agg tm
(5)
m/
Since the functions (o) are also drawn from sigmoidal Gaussian process, the NP-HP's endogenous functions possess greater flexibility than previous approaches, in which the endogenous functions are typically specified as exponential functions. Incorporating the product of the nodes' latent endogenous features ensures that these endogenous functions are nodal pair specific, as for the exogenous functions. However, NP-HP assumes that each nodal pair would use same endogenous functions for all its edges. This might be unable describe the changes in the edges, such as emails with different topics between two persons might generate distinct endogenous effects.
3.3 Non-parametric Hawkes process with edge-clustering effects (NP-HP-C)
The third approach introduces edge clustering mechanism for the mutually-exciting Hawkes process with nodal pair-specific exogenous functions. Here each generated edge is assigned a latent group label. The NP-HP-C generative process for edges in the time period [0, T) is then follows:
1. Generate fx (t) ~ GP (f)). gk GP(O, (o)), ==1, 1,... K 1,...,N:
R
k= 1....., K.
2. Generate Tak Gam (ax, bm),
N:
3. Generate Us Gam(a,, bw), t I,..., K, where en is the scaling variable for the k-th endogenous function (gg)(t)) (f)).
4. For each pair of nodes ((1,j) (mm') [(i,j, tig )) ((mm) ((mm)
1 S i j S N). generate triplets
),6% 22zg Im and {ū, i, ef (mn)) (mn) (m") ) 3mm" for the two directed edges
6.6%m²² ,28

---

according to the following procedure, until tu S < tM/+1. tMn S T < tt
Initialise the current time stamp at t. = and set M., M,; Then:
(a) Starting from tu, obtain two candidate event times t.g min(te)) Ki > t,y,
min (1() bele% > t.), where the two time sequences (1/)) h. 18 j are generated
from:
Mg
(1gg J PoissonProcess!
Tak Ta (fa(())
+
(mm)) a0 ((s 155 ()), (6)
m 11
Mag
PoissonProcess
mak o +
aig (oaet (t - 1/5 (]}- (7)
(Note that the second term in Eqs O6) and 7 is not involved when Mj = M₁; 0).
bay > T Tii > T, break for this nodal pair and start generating edges for other
pairs.
(b) If lig < Egi. set Maj- = Mi + 1,4 (Mas) = big and generate b/y 1 ,2 from (m' = Mçj.
(Ma)) (Mas)
if (Mis) 0, 2i5 (Ma) = 2
(M4s) = k;
(M,)) (Mwil) P(hgg ,29
Vegne)00s (ons) (t. - 1/ f(m))))
if byy (,)))
= ml, 2.g
(m²)
(M,e)) (M,z) from (m' (M,)) = Eji and generate bis 2zs
else, set Mjs = Mji + 1,t, 1j
1. 1..... Mij)
(((()))((((Ll))
(6
0, 2,24)
(M;i) k;
P(b 238 {
(M))) (M50))
"i ooe) (t+ - 15g ((m))),
if bf
(M, ml, ,298
(Mzc)
(m²)
where Mi is the number of edges from node to node j, and where ((m)) : ((m)) E (10) Hey (m))) and z.) E and latent label of the m-th edge from node to node j respectively. The above generative process produces observations between nodal pairs in the form {[(i,,, 1y ((n1)) ") (.mm)) ((m') )== ) Jayj- For
}}
K} denote the time, "source" factor (i.e., exogenous rate or endogenous rates)
: ,Zij -=]]"
notational convenience, reformat these as {6mm Imm t, bm 2m JM m=1, with denotes the edge number among all the edges, for the following discussion.
In this generative process, steps & 2 generate the exogenous function random bases and the nodes' latent features, in the same way as for the NP-HP. The exogenous function in the NP-HP-
C Σ Tak offx (1)) is also nodal-pair specific. Step 3 generates the scaling variables for K
endogenous functions. Step (.()) generates two directed edges between each nodal pair. Starting from any time stamp t., the Poisson processes with the current intensity functions (Eqs 6 and 7 are used to generate the candidate edge occurrence time stamps &.y, tys for each edge direction. If both Eaj. Eyt are larger than T. which means that the two Poisson processes do not generate edges in (t., T7. we stop generating edges for this nodal pair. Otherwise, we use the earlier candidate time stamp (e.g. take E.j if f.j < Ezz)) the actually occurring edge time, and add its triggering effect to the aypositely-directee intensity function (from the actual occurrence time).
Step 4.(b) uses the idea of branching structure [17 32 to describe the "source" factor of the edge triplet (im:Im; lm). That is, bm = 0 if the triplet Jm: (m) is instantiated by an exogenous intensity and bm = mí' (m'' E Him.... (lm)) )) if the triplet (fm; Jm, (m) is triggered by its historical pppositely-directe
edge triplet (Jm, im, ). At the same time, 2m E {l....,K' } is the latent label of (im: Jm. tm) which determines the pattern group to which Jm. tm) ).'"s triggering effect belongs. The edges with the same label use the same endogenous function. In this way, the endogenous functions are shared among the edges of all nodal pairs. Those edges triggered by the same labelled edges (through the same labelled endogenous function) or instantiated by the same labelled exogenous component are associated with the same endogenous function. This process is visualised in Figure []
As previously mentioned, NP-HP-C's clustering mechanism for the endogenous functions of edges can help modelling practical phenomena, such as urgent emails would usually result in fast response, regardless of the nodal pair. What is more, NP-HP-C uses much smaller number (K) of endogenous functions than NP-HP. This small number might be able to help NP-HP-C avoid overfitting issues.
5

---

efi(i
e(
0.2-
Eij
my
mud (pe(())
Moalf)
2.0
Moalt)
Ày
Ay(()
2.5
0.0
Big
Figure 1:
(Left panel) Visualisation of edge generation between nodes and j in the NP-HP-C. Different edge colours (red, green, blue) represent different edge-group labels. Arrows (grey-dotted and black solid arrows) link the exogenous functions and past oppositelyddireeted edges to current edges, and they together represent all potential triggering relationships between mutually-excited edges, Black solid arrows denote the actual triggering relationships. If an edge is triggered by one of its past oppositely-dires dirreced edges, their colours (i.e labels) are the same, (Right panel) From top to bottom: the exogenous functions {fr (f))
M k=1" the endogenous functions (Uk a (g) and the corresponding intensity functions Aey (t), (t) between nodes and j.
3.4 Mean-field variational inference
To implement faster posterior inference than a MCMC implementation, use the strategy of [5]
to develop mean-field variational posterior approximation [2]. In particular, [5] integrated a Poly- Gamma trick into variational approximation for the sigmoidal Gaussian process-modulated Poisson process, and 33 applied similar techniques in the simple Hawkes point process setting. For e.g. the NP-HP-C model, we aim approximate the joint posterior distribution of sigmoidal Gaussian process modulated random functions {fa (f), Ih ht) the endogenous function scaling values { hx, the latent features (Tax ( J,K, and the branching variables (bm) {bm)n and latent labels {=m }m m for all edges. Full details of the variational model and updates are provided in the Supplementary Material.
Variational distributions for bm. 2m: Since bm 2m discrete random variables, can adopt noted that when calculating the
categorical distributions as their variational distributions. likelihood for 2m. which denotes the latent label of the m-th edge (ims Jmmtm)) we need to include all the edges triggered by the m-th edge (i.e. (Jmrs fmm) bmn = m 7 (Im: = (Jmis , (m)) )).
In order to reduce the computational cost of the Gaussian process regression, we use the sparse Gaus- sian process approximation (27 to calculate E. Et), (f), E,(, and (t)) making scale to the number of edges 5
Computational complexity: The CTEM models provide efficient solution large-scale continuous-time edge data modelling. Let Ng denote the number of observed edges, NgT = Em # ((Fimmmm (fm) denote the number of potential triggering phenomena between all oppositely- directed edges, NIN denote the number of inducing points for sparse Gaussian process approximation and K denote the number of latent features. The computational cost the CTEM models then scales with O(NNE + Ng) K + Ng K),, which scales linearly to the number of edges and would be suitable for large-scale continuous- time edge datasets.
4 Related work
Of the various approaches which use the ME-HP to model continuous time edge data, the Hawkes Infinite Relational Model (Hawkes-IRM) 3 was the first to apply the ME-HP on the edges be- tween groups formed by the nodes partition. By using the stochastic block model [2] 21 [4], the Hawkes-IRM model adopted static exogenous rates. The Hawkes-Dur Latent Space Embedding model (Hawkes-DI.s DL.S) 29 created latent space vectors for nodes and used the products over these vectors construct the exogenous and endogenous functions in the ME-HP. 26 adopted the nested-
CRP M to construct hicrarchical structure in the exogenous function when using the ME-HP

---

Table I: Continuous time edge dataset information. N is the number of nodes, NE is the number of unique edges, NEC the number of edges (including their counts), and 7 is the number days.
Datasct N
NE
NEC
T
Dataset
N
NE
NEC
T
Email
793
1032
81371
17.34
College
1417
15 329
41371
21.37
Overflow
277
21 291
41710
133.60
Ubuntu
71 817576
24062
47 298
147.35
for continuous-time edges. (20 applied the same ME-HP in the framework of sparse exchangeable random measures [4] 22)) and obtained continuous-time sparse exchangeable random measures. (30]
adopted the Hierarchical Gamma Process to construct static exogenous function rates.
We note that the marginal probability of am obtained by summing over the values of 6m in their joint probability function P(=m. by.. in NP-HP-C is similar to the one used in the process (DHP) and its variants [44. The NP-HP-C model has three major differences with these DHP models: 1) These DHP models consider the particular latent feature in triggering the current event. In contrast, the NP-HP-C model simultaneously specifies the particular latent feature (through 2m and the particular historical oppositely-direst direecedd edges (through bm:): 2) We are able to develop fast variational inference algorithm for this model (Section (.4). whereas inference for DHP models relies on Sequential Monte Carlo methods; 3) The NP-HP-C model describes continuous-time edge data, whereas DHP models are mainly used for streaming text data.
Instead of using point processes model the continuous-time edges, the approaches of [9] 10 studied discrriscet networks and used Bernoulli emission distributions to model all the binary-valuee edges at each observed timestamp. They adopted Gaussian processes/Nested Gaussian processes to generate continuous-tiin time features for each node at the observed timestamps. The approach of [88 also considered continuous-time edges. It extended the Poisson IRM by using histogram or kernel approximations to model the time-varying exogenous rate function, whereas our NP-PP uses Sigmoidal Gaussian Processes approximation.
Experiments
We perform experiments on four continuous edge datasets (Table collected from the Stanford Large Social Network Dataset [16]. In each dataset the occurrence time of an edge is recorded to the nearest second. We scale them onto the interval [o. T] by dividing the recorded time by 24 x 3600, so that T' is the total number of observation days. The first two datasets (Email, College) comprise instant response-typed edges (replying to emails and sending messages), while the last two (Overflow, Ubuntu) comprise edges recording relatively slow replies (writing answers to questions or commenting on others' answers).
Experimental setup: We use the Gaussian kernel functions, R%
( (tttt ty) Bf -.5((t.--t)))) (,)((tt-t))/(((()))
(9) and Kg (1i, (4) 0000 e -..((;,-,))
in the Gaussian processes for generating random func- tions (t) and {t) respectively, where o( .(()) are scaling parameters and 8(f) g( are bandwidth parameters. These parameters are optimized by applying the ADAM algorithm [TS] on the Evidence Lower Bound of the variational distributions. The parameters a, by of the endoge- nous function scaling values Us and the parameters a,,, by of the latent feature tu are given prior distribution of Gam 0.1) and optimized through the variational inference. We usually set 400 iterations for the MF-VI algorithm. Without particular specifications, we set the number of latent features as K = 5.
Comparison methods: The CTEM models are compared to the following benchmark methods: (I) the Poisson-Infinite Relational Model (Poisson-IRM) 3 (2) the Hawkes-Infinite Relational Model (Hawkes-IRM) [3: (3) the Hawkes Dual Latent Space model (Hawkes- DLS) 29 and (4) Hawkes process with overlapping communities (HawkesNetOC) 20). We use the exponential function as the endogenous functions and implement Markov chain Monte Carlo (MCMC) methods for the Poisson-IRM and the Hawkes-IRM models to the best of our ability. Inference for the Hawkes-DI.S model and NettC is executed by using the authors' provided code. The default settings of Hawkes-NerOC and and Hawkes- DL are used in the comparison. We use the previously developed CTM model constructions, giving the non- parameettt Poisson process (NP-PP),
7

---

Table 2: Testing log- likelihhodd and Area Under ROC Curve (AUC).
Testing Log-Likelhoood
AUC
Models
Email
College
Overflow
Ubuntu
Email
College
Overilow
Ubantu
Poisson-IKe -Rsson--RMM
-22320
-58 552
780
-96
77.46
79.18
83.14
83.20
Hawkes-IRM
58341
469461
746
-90435
79.12
80.14
85.16
84.14
Hawkes-DLS
-9880
-56233
505
-93485
80.15
80.16
81.63
80.19
Haw
577 483
-57
- 88 183
193
85
83.17
81.38
83.48
82.24
NP-PP
468
151
833
136880
78.16
79.71
89.15
86.77
NP-HP
-53
729
1142277
-120419
81.78
81.57
89.58
86.78
NP-HP-C
120
-53958
-155387
-234
85.29
82.17
89.91
88.02
non-parametric Hawkes process (NP-HP), and non-parametric Hawkes process with edge clustering (NP-HP-C).
5.1 Testing Log-likelihood and link-prediction performance
The performance of each model is evaluated through its model fitting performance to the Testing edges and its capability to predict future edges. The edges of each dataset are sorted according to their occurrence times, and the dataset split into training set (the first 70% of edges) and testing set (the remaining 30%). The comparisons between the CTEM and other models are based on two criteria: the log-likelihood for the edges in the testing time period, and the AUC values randomly sampled sub-interval in the testing time period. We use Eq. (2) to calculate the log- ligeliik,hood, in which the intensity function Az (t) is either the exogenous function (for Poisson processes) or combination of the exogenous and endogenous functions (for Hawkes processes).
For the AUC calculation, first uniformly sample 100 time stamps t; in the time interval of the test dataset, and then evaluate the probability of an edge appearing between all nodal pairs in the time window [4, + 6). We compute AUC values for each time window [4c, 4 + 8). The final link-prediction performance score is the mean of these 100 AUC values on all time windows.
The results are displayed in Table 2 Due to the space limit, we have provided the standard error in the Supplementary Material. The CTEM models perform differently for the two types of datasets. For the instant-response datasets, the NP-HP and NP-HP-C produces better performance in both testing log-likelihood and AUC. This may indicate that our exogenous functions and endogenous functions fitted better than the comparison methods. However, for the slow-response datasets, the CTEM models performs worse in the testing bog-likelihoods, although still produces better prediction performance. A possible reason is that these two datasets have relatively simple patterns (here low-response activities) which can be well captured by simple rate functions. For instance, the exogenous functions for these slow-response datasets did show clear periodicities (see the right two panels in the top row of Figure 2 Simple rate functions like constant functions may well describe the exogenous effects. For the endogenous effects, they are dominated by one single endogenous function in both Overflow and Ubuntu (see the right two panels in the bottom row of Figure D. One single exponential decay function may also capture these simple patterns.
5.2 Visualisations for exogenous functions and endogenous functions
The top row in Figure 2 presents the mean exogenous function values Ee,naays Ex Teke o (fa(t) for the NP-PP, NP-HP and NP-HP-C models. With the vertical dotted lines separating the time lines into weeks, daily periodicity is quite clear for the instant-response datasets (i.c. Email, College). and approximate weekly periodicity for Overflow and Ubuntu. The weekend effect is also easily identifiable in Email, a gradual increasing trend in location and scale for College, and occasional weeks of peak high and low activity for the slow response datasets. For the latter, the large activity spikes could be explained by sudden interest in popular topics being discussed on the message boards.
In terms of performance difference within the CTEM models, the exogenous values of the NP-PP model (blue lines) are always higher than those of the NP-HP and NP-HP-C models, and the NP-HP (green lines) usually produces the smallest values. The former may occur as the NP-PP model is using the exogenous functions to explain all edges as it does not have endogenous functions. The latter may occur the NP-HP models less edges than the NP-HP-C. For the slow-response datasets,
8

---

Email
College
Overllow
Ubuntu
1P-19
NP-1P
APHIP-C
202
Email
College
Diverflow
Ubuntu
7.09916
0.645%
0.314%
5.032%
90.78%
0.023
0.217%
0.5
00 201
03825
0.044
50. 20.00%
0.313%%
0.6
0.03
0.546%
0.2
0.22
0.2
02-
0.302
0.022
0.0
0.0
0.300
0.000
Figure 2: Top row: mean exogenous functions N(N-1) Eizy Ex Tas (f)((t) for the NP-PP, NP-HP and NP-HP-C models on each dataset. The axes show the the exogenous rates (y-axis) and the number of days (r-axis). Vertical dotted lines in each panel indicate weekly intervals. Bottom row: fitted endogenous functions {Uk (g (t))}x for NP-HP-C on each dataset. Panel legends indicate the proportion of triggered edges explained by each function. In order to improve visualisation, exogenous functions with weight under 0.1 are not shown.
the average values are similar for the NP-PP and NP-HP-C models. This may be explained as the endogenous effects may be more important to the instant-response data than the slow-response data.
The bottom row in Figure 2 iillustrates the most influential endogenous functions of the NP-HP-C model. Within our expectation that endogenous effects would become weaker along with the time, the values of all main endogenous functions have sharp decreases in the beginning period. Three of the datasets also exhibit (daily or weekly) periodicity, which indicates that for these data there has been an increase in triggered responses above and beyond the increase in events attributable to background (exogenous) rates. This is less obviously the case for the College dataset, where, despite some initial periodicity, many of the endogenous functions become constant after certain time. We are unaware of any related discussions of periodic endogenous functions for these data the literature.
We have provided visualisations for other variables in the Supplementary Material.
5.3 Intensity functions (t)(() and influence of the number of latent features
The left two panels of Figure 3 show the intensity functions X (t) of the NP-HP-C for randomly selected nodal pairs periodicity is again strongly apparent in A (t) for Email, and more muted weekly periodicity for (t) in Overflow. We note that most of the observed edges (points the z-axis) located in regions where the intensity increases, indicating correct performance.
instant-respons, (Email) and slow-response (Overflow) dataset. The daily
The right two panels of Figure 3 show the influence of different number of latent features K. Predictive performance appears less sensitive to for the instant-response datasets, which perhaps have simpler structures, than the slow-response datasets, as the latter have improved performance with increasing K. Consistent with the complexity analysis, running times are linear in the number of edges. As the NP-HP-C with K = 5 requires ~ 60 seconds for one iteration with 80 000 edges, we need ~ 6 hours of for posterior inference on these edges (with 400 iterations).
Conclusion
In this work, we have explored three non-parametric point process models for continuous-time edge data. The sigmoidal Gaussian process priors for the exogenous and endogenous functions enable flexible modelling, and strong predictive performance. The developed mean-field variational

---

Email
Overflow
Overflow
0.02
0.8
Palt Sereon
60
Km3
Keg
5
0.6
9.01
0.4
MW
100
0.2
x30
10
15
25
50
100
125
Docikn
Namber Edges
Figure 3: Left two panels: example nodal pair-intensity function (t) for NP-HP-C on Email and Overflow. middle right panel: AUC prediction performance for NP-HP-C with K = 1, 1.3,5. Right most: Running time (seconds per iteration on laptop) comparison for NP-HP-C with 1,
inference algorithm allows the models to work at scale. The experimental results on four real world data sets demonstrate superior performance compared to the state-of- state-of-the-art.
Acknowledgements
Xuhui Fan and Scott A. Sisson are supported by the Australian Research Council through the Australian Centre of Excellence in Mathematical and Statistical Frontiers (ACEMS, CE140100049), and Scott A. Sisson through the Future Fellow Scheme (FT170100079). Bin is supported by the Program for Professor of Special Appointment (Eastern Scholar) at Shanghai Institutions of Higher Learning. Feng Zhou is partially funded by China Postdoctoral Science Foundation.
References
[1] Amr Ahmed, Liangjie Hong, and Alexander Smola. Nested chinese restaurant franchise process: Applications to user tracking and document modelling. ICML, pages 1426-1434, 2013.
[2] Christopher M Bishop. Pattern recognition and machine learning. springer, 2006.
[3] Charles Blundell, Jeff Beck, and Katherine A Heller. Modelling reciprocating relationships with hawkes processes. In NIPS, pages 2600-2608, 2012.
[4] François Caron and Emily Fox. Sparse graphs using exchangeable random measures. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 79(5) 2017.
[5] Christian Donner and Manfred Opper. Efficient bayesian inference of sigmoidal gaussian cox processes. The Journal of Machine Learning Research, 19(1):21002743,, 2018.
[6] Arnaud Doucet, Nando De Freitas, and Neil Gordon. An introduction to sequential monte carlo methods. Sequential Monte Carlo methods practice, pages 3-14. Springer, 2001.
[7] Nan Du, Mehrdad Farajtabar, Amr Ahmed, Alexander Smola, and Le Song. Dirichlet-hawkke processes with applications clustering continuous-tine document streams. In KDD, pages 219-228, 2015.
[8] Christopher DuBois, Carter Butts, and Padhraic Smyth. Stochastic blockmodelling of relational event dynamics. AISTATS, pages 238-246, 2013.
[9] Daniele Durante and David Dunson. Bayesian Logistic Gaussian Process Models for Dynamic Networks. In AISTATS, pages 194-201, 2014.
[10] Daniele Durante and David B. Dunson. Locally adaptive dynamic networks. The Annals of Applied Statistics, 10(4):2203 2232.
[11] Thomas L Griffiths, Michael I Jordan, Joshua B Tenenbaum, and David M Blei. Hierarchical topic models and the nested chinese restaurant process. In NIPS, pages 17-24, 2004.
[12] Alan Hawkes. Point spectra of some mutually exciting point processes. Journal of the Royal Statistical Society: Series B (Methodological), 33(3):438-44 1971.
10

---

[13] Alan G Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1):83-90, 1971.
[14] Charles Kemp, Joshua B. Tenenbaum, Thomas Griffiths, Takeshi Yamada, and Naonori Ueda. Learning systems of concepts with an infinite relational model. In AAAI, volume 3, pages 381-388, 2006.
[15] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980) 2014.
[16] Jure Leskovec and Andrej Krevl. SNAP Datasets: Stanford large network dataset collection. http:///nnap.ssaaf.... edu/ data June 2014.
[17]
David Marsan and Olivier Lengline. Extending earthquakes' reach through cascading. Science, 3 31965866)1006-10777 2008.
[18] C Matias, T Rebafka, and F Villers. A semiparametric extension of the stochastic block model for longitudinal networks. Biometrika, 105(3):665-680, 2018.
[19] Charalampos Mavroforakis, Isabel Valera, and Manuel CodrzzRodriguez. modelling the dynamics of learning activity on the web. In WWW, pages 1421-1430. 2017.
[20] Xenia Miscouridou, François Caron, and Yee Whye Teh. Modelling sparsity, heterogeneity, reciprocity and community structure in temporal interaction data. In Neur/PS, pages 2343-2352, 2018.
[21] Mark EJ Newman and Michelle Girvan. Finding and evaluating community structure in networks. Physical review 69(2):0261 2004.
[22] Krzysztof Nowicki and Tom A.B. Snijders. Estimation and prediction for stochastic block structures. Journal of the American Statistical Association, 967455):107-11087, 2001.
[23] Jim Pitman. Exchangeable and partially exchangeable random partitions. Probability theory and related fields, (02(2))145-155) 1995.
[24] Yeon Seonwoo, Alice Oh, and Sungjoon Park. Hierarchical dirichlet gaussian marked hawkes process for narrative reconstruction in continuous time domain. In EMNLP, pages 3316-3325, 2018.
(25) Xi Tan, Syed AZ Naqvi, Yuan (Alan) Qi, Katherine A Heller, and Vinayak Rao. Content-based modelling of reciprocal relationships using hawkes and gaussian processes. In UAI, pages 726-734, 2016.
[26] Xi Tan, Vinayak Rao, and Jennifer Neville. Nested crp with hawkes-gaussian processes. In AISTATS, pages 1289-1298, 2018.
[27] Michalis Titsias. Variational learning of inducing variables in sparse gaussian processes. In AISTATS, pages 567-574, 2009.
[28] Adrien Todeschini, Xenia Miscouridouand, François Caron, and Emily B Fox. Exchangeable random measures for sparse and modular graphs with overlapping communities. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 82(2):487-52) 2020.
[29] Jiasen Yang, Vinayak Rao, and Jennifer Neville. Decoupling homophily and reciprocity with latent space network models. In UAI, pages 38-43, 2017.
[30] Sikun Yang and Heinz Koeppl. The hawkes edge partition model for continuous-ting event- based temporal networks. In UAI, pages 460-469 060,469,440 2020.
[31] Feng Zhou, Zhidong Li, Xuhui Fan, Yang Wang, Arcot Sowmya, and Fang Chen. Efficient inference for nonparametric hawkes processes using auxiliary latent variables. Journal of Machine Learning Research, 21(241):1-31, 2020.
[32] Ke Zhou, Hongyuan Zha, and Le Song. Learning triggering kernels for multi- dimensinaal hawkes processes. ICML, pages 1301-1309, 2013.

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the pa- per's contributions and scope? [Yes] See Section 3 (main model part) and Section (experiments).
(b) Did you describe the limitations of your work? [N/A]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] My paper conforms to the ethics review guidelines.
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes] We provide the code and data in the Supplementary Material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See the experimental setup part in Section
(c) Did you report error bars (e.g., with respect to the random seed after running exper- iments multiple times)? [Yes] Due to the space limit, we put the average results in main content and complete results with errors are provided in the Supplementary Material.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] We mention the running time display part (right most panel in Figure 4).
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] Please see reference
16
(b) Did you mention the license of the assets? [No] The reference 16 should contain the
license.
(c) Did you include any new assets either in the supplemental material as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
12


Reviews:
[{"review_id": "kC03p1KTC7q", "reviewer": "Reviewer_Cj5q", "summary": "The paper presents a Gaussian process based nonparametric method that enables flexible modeling of temporal interacting events. With a fast variational inference method, the proposed approach shows improved empirical predictive performance on four real datasets compared to several existing work.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n\n1. A Gaussian process based nonparametric method for modeling the conditional intensity function to capture temporal events between pairs of interacting nodes (NP-HP).\n2. An extension of NP-HP to capture the clustering structures among interacting nodes, referred to as NP-HP-C by introducing latent group label similar to the stochastic blocking model\n3. Strong empirical results show competitive modeling capabilities compared to other SOTA approaches. \n\nLimits:\n\n1. Do the conditional intensity functions in (4) and (5) need some minimum positive background intensity to make sure that they are not too small to be close to zero to have numerical instability issue as mentioned by the work [Estimation of space–time branching process models in seismology using an EM–type algorithm, 2008] and [Robust identification of controlled Hawkes processes, 2020] since the sigmoid function can decay fast to be near zero.\n2. Although the experiments show competitive performance across four real datasets, it will be more convincing to use a small synthetic dataset showing that NP-HP-C is able to better capture the clustering structures compared to other stochastic blocking based baselines.\n3. On overflow and ubuntu, it shows that the proposed method gives the worst results compared to other baselines. More elaborations are needed. In particular, it will be very helpful to include the MAE of the time prediction as the evaluation metric since this is quite a standard metric used in the temporal point process literature.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No potential negative societal impacts are involved in this paper.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1.5-2 hours"}, "extra_text": {}}, {"review_id": "i3Ur0BwocZ2", "reviewer": "Reviewer_2ejX", "summary": "The authors propose a family of models based on mutually-exciting Hawkes process that aim at modeling continuous time edges. The framework is based on a variational inference procedure for a fast and efficient computation. The models have been quantitatively and qualitatively evaluated with respect to baselines, showing that, in general, they outperform the baselines in terms of accuracy.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper include many details (in particular in the supplementary material) that help comprehension. However, the math section of the main paper is not always clear. In particular, it took me a while to check what some notation meant (such as M_ij), before realising it was explained later on in the section. I would suggest to explain the notation before or at the point when it is first introduced to simplify reading.\n\nI like the experiments section. The results are clear and well explained. They also aligned with the properties of the models presented in the paper. However, the choice of K=1,3,5 seem a bit restrictive, in particular with the order of thousands of edges. I understand this is because the model scales linearly with the number of K as well as the number of edges. How much would be the computational limit of the model in terms of the edges and clusters? \n\nIn this terms I'm not fully convinced the model can really scale to large datasets, as I believe a linear dependency from the number of edges and the number of clusters would not let the model be applicable to (say) social networks.\nI think this is important as, when increasing the number of edges in the network, an higher number of clusters would in turn be required to better model the data at hand.\n\n\nMinor:\nline 33: \"oo\" instead of \"to\"\nline 146 it's", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I think limitations of the models have not been fully discussed. In particular, based on the complexity of the models it does not seem that they are able to scale to large set of edges, in particular when the number of clusters is high. This would be a relevant limitation as I argue that, when increasing the number of edges, an higher number of clusters would be needed to better describe all of the network.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "gN4kXpQF1Jk", "reviewer": "Reviewer_Kebi", "summary": "The paper proposes three  non-parametric point process variants for continuous-time edge modelling (CTEM) and the performance of the proposed framework evaluated on four real world dynamic networks. The three cases are \n1) a Poisson process in which K time series are generated according to a (sigmoidal) GP and weighted by node specific feature weights utilizing these temporal evolutions.\n2) An extension to the Hawkes process such that reciprocity is invoked by an additional sigmoidal GP weighted by other node specific feature weights defined through time to past reciprocative interactive events.\n3) A further extension incorporating edge clustering rather than node specific weights for the reciprocating effects (i.e., endogeneous functions) reducing the parameter space and enabling specific endogenous effects according to the cluster used for the specific reciprocative event of each dyad. \n\nVB is used for large scale inference and inducing points invoked to reduce complexity of the GP. The model is contrasted the Poisson and Hawkes IRM as well as Hawkes-DLS and HawkesNetOC and found on the four considered datasets to have superior performance as quantified using time-binned AUC but inferior results on two datasets for the (test I assume) log-likelihood. In particular, the edge clustering approach 3) forming the NP-HP-C is found to be the superior of the proposed approaches.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a well written and solid contribution in which multiple novel dynamic network models for directed dynamic networks are proposed. \n\n$\\textbf{Originality}$\n\nThe paper is based on a series of well-established methodologies, i.e. Hawkes process for dynamic network modeling , sparse GP for modeling smooth temporal trajectories efficiently, VB for efficient scalable inference. However, their combination is novel and the proposed framework both sound and efficient enabling accurate dynamic network analysis accounting for reciprocative relationships. The edge clustering idea is neat and the use of dynamic feature representations that are weighted in terms of node specific weights sound and useful. The inference is non-trivial and appears well implemented.\n\n\n$\\textbf{Quality}$\n\nThe approach developed is technically sound and well decribed. The experimentation is solid with reasonable comparisons included to existing methods considering a sufficient  number of (four) datasets. The results are convincing, the approach well motivated and with potential wide applications within dynamic network analysis.\n\nIt would be good to position the paper also in relation to work using GPs for dynamic network analysis, see also:\n\nDurante, Daniele, and David Dunson. \"Bayesian logistic gaussian process models for dynamic networks.\" Artificial Intelligence and Statistics. PMLR, 2014.\n\nand nested GPs\n\nDurante, Daniele, and David B. Dunson. \"Locally adaptive dynamic networks.\" The Annals of Applied Statistics 10.4 (2016): 2203-2232.\n\nThere are further more advanced Poisson Process stochastic block modeling approaches than the PP IRM worth discussing. This includes the work of:\nMatias, C., Rebafka, T., & Villers, F. (2018). A semiparametric extension of the stochastic block model for longitudinal networks. Biometrika, 105(3), 665-680.\n\nThere are no error bars reported in the results. It would strengthen the paper to have multiple restarts and provide the robustness of the results in terms of initialization of the inference procedure. In particular Table 2 does not provide error bars. The authors state in checlist point 3.c \"Due to the space limit, we put the average results in the main content and complete results with errors are provided in the Supplementary Material.\" However, I was unable to find such error bars in the supplementary for the most important Table 2 results.\n\n$\\textbf{Clarity}$\n\nThe paper is well written and in general easy to follow. However, method 3) is quite convoluted in its description and although the authors provide a reasonable attempt in describing the steps and intuition behind the generative model this section could be improved in clarity and the reader guided more carefully through the steps. \n\nThe authors set the number of features to K=5 but does not discuss this choice further nor its impact on the results. It would be good to elaborate on this.\n\nFurthermore, there are very minor comments to be considered:\n\nthese models oo -> these models to\n\nIn table 2 it is a bit confusing to call it training log-likelihood when it is in fact the test log likelihood I believe on last 30% of the available time segment.v Please clarify what the results refer to as it should be test-log likelihood. \n\n$\\textbf{Significance}$\n\nAlthough the use of Hawkes processes is well established the proposed approach is very useful in having time-evolving rates that are flexibly and well implemented through feature representations and clustering. The results are convincing and the proposed approach can have a solid impact within dynamic network analyses in general.\n\n--------------------Response to the authors rebuttal-------------------------------------\n\nI thank the authors for their responses to my concerns. It is also good that error bars are now provided and can be assessed. \nIn regards to 1) I agree that the work of the authors differ. However, the point was to position the manuscript in context of existing work utilizing GPs for dynamic network analysis. I agree that the work I referred to using GPs differ considering time-discretized network and a Bernoulli likelihood. However, the approach also utilizes GPs defined in continuous time.\n\nThe authors could have elaborated in their rebuttal on the work of:\nMatias, C., Rebafka, T., & Villers, F. (2018). A semiparametric extension of the stochastic block model for longitudinal networks. Biometrika, 105(3), 665-680.\nWhich also considers continuous time modeling using a stochastic block model including kernel and histogram approximations of the rate function.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There are no broader impact discussed of the paper nor potential negative societal impact. Enhanced abilities modeling dynamic networks and understanding their structure inevitably can also be used for surveillance, the enhancement of filter bubbles and be used to negatively influence people based on the improved understanding of their behaviors. Some of these potential negative aspects by the enhanced ability to predict behavior in social dynamic networks could be discussed.", "review_text": "", "extra_scores": {"ethical_concerns": "The analysis is based on open and publicly available datasets.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "MYJQtVvdAaU", "reviewer": "Reviewer_sJ8A", "summary": "-This work develops a family of novel non-parametric point processes for continuous-time edge data. In details, the model captures the periodicity exhibited in its base rate using K sigmoidal Gaussian processes(SGPs) \\sigmoid(f_k(t)). \n-For each pair of two nodes, the nodal pair specific coefficients \\p_{ik}, \\pi_{jk} is used to rescale the pair-wise base intensity of (i,j) and (j,i).  \n-To  capture the reciprocity, the model employs a second set of functions \\sigmoid(g_k(t)) drawn from SGPs, which has more flexibility in capturing exciting behaviors, compared to the exponential decaying kernel used in Hawkes processes.\n- A mean-field variational scheme is derived to perform model inference.\n-The model is demonstrated on four real data to illustrate its estimated periodically varying base intensity and the endogenous functions. They also show the superior performance of the model in terms of training log-likelihood and link prediction.\n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "-Originality\nThe idea of modelling time-varying base rate, and flexible endogenous functions is interesting and novel, to me.\nNonetheless, both the sigmoidal Gaussian processes and the main techniques for the proposed variational inference scheme have been exploited well before.\n\n-Quality\nThe novel point processes models and the derived variational algorithm are sound.\n\n-Clarity\nThe notations are not clear to me, e.g., why you have nodal pair specific coefficients \\rho_{ik}\\rho_{jk} in Eq.(4,5) but you change to v in Eq.(6)?\nFor experimental setup, it is not clear how to choose the number of features K, and why K is fixed for all data sets. I would consider the features may represent the latent group or clusters in the Email, Overflow and College data. Thus, some explainations are missing for the setting of K.\nIt is not clear why to chose the first 70% of edges as training data while chose remaining 30% as test data. As you are motivated, the exogenous intensity of events may have some periodical pattern. Thus, it is not clear how to judge the remaining 30% of edges still have the same periodicity, has entered a novel period that are not observed before. \n\n-Significance\nTo me, the novel point processes may have some impacts for a small group of audience who are interested in modeling excitatory behaviors of continuous-time dyadic data. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors come up with some interesting phenomenons, which cannot be fully captured using Hawkes process with constant base rate and parametric kernels. In addition, I think the proposed symmetric intensity function may also some limitations. For instance, an employee may immediately reply to an email from her/his team leader, while the leader may not reply to the employees quickly. Hence, I am wondering how to model asymmetric nodal pair-specific intensity by extending the framework.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "3 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Overall, the reviewers were positive about your work, though there are a number of suggestions that you should make to improve the manuscript. I remain a little perplexed by the performance on Overflow and Ubuntu--isn't it odd that AUC is highest but log-likelihood is lowest? Could this be due to the use of the mean-field variational Bayes approximation which can give very wrong posteriors?
Decision: Accept (Poster)