Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper presents a Gaussian process based nonparametric method that enables flexible modeling of temporal interacting events. With a fast variational inference method, the proposed approach shows improved empirical predictive performance on four real datasets compared to several existing work.
main_review: Strength:

1. A Gaussian process based nonparametric method for modeling the conditional intensity function to capture temporal events between pairs of interacting nodes (NP-HP).
2. An extension of NP-HP to capture the clustering structures among interacting nodes, referred to as NP-HP-C by introducing latent group label similar to the stochastic blocking model
3. Strong empirical results show competitive modeling capabilities compared to other SOTA approaches. 

Limits:

1. Do the conditional intensity functions in (4) and (5) need some minimum positive background intensity to make sure that they are not too small to be close to zero to have numerical instability issue as mentioned by the work [Estimation of space–time branching process models in seismology using an EM–type algorithm, 2008] and [Robust identification of controlled Hawkes processes, 2020] since the sigmoid function can decay fast to be near zero.
2. Although the experiments show competitive performance across four real datasets, it will be more convincing to use a small synthetic dataset showing that NP-HP-C is able to better capture the clustering structures compared to other stochastic blocking based baselines.
3. On overflow and ubuntu, it shows that the proposed method gives the worst results compared to other baselines. More elaborations are needed. In particular, it will be very helpful to include the MAE of the time prediction as the evaluation metric since this is quite a standard metric used in the temporal point process literature.

limitations_and_societal_impact: No potential negative societal impacts are involved in this paper.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors propose a family of models based on mutually-exciting Hawkes process that aim at modeling continuous time edges. The framework is based on a variational inference procedure for a fast and efficient computation. The models have been quantitatively and qualitatively evaluated with respect to baselines, showing that, in general, they outperform the baselines in terms of accuracy.
main_review: The paper include many details (in particular in the supplementary material) that help comprehension. However, the math section of the main paper is not always clear. In particular, it took me a while to check what some notation meant (such as M_ij), before realising it was explained later on in the section. I would suggest to explain the notation before or at the point when it is first introduced to simplify reading.

I like the experiments section. The results are clear and well explained. They also aligned with the properties of the models presented in the paper. However, the choice of K=1,3,5 seem a bit restrictive, in particular with the order of thousands of edges. I understand this is because the model scales linearly with the number of K as well as the number of edges. How much would be the computational limit of the model in terms of the edges and clusters? 

In this terms I'm not fully convinced the model can really scale to large datasets, as I believe a linear dependency from the number of edges and the number of clusters would not let the model be applicable to (say) social networks.
I think this is important as, when increasing the number of edges in the network, an higher number of clusters would in turn be required to better model the data at hand.


Minor:
line 33: "oo" instead of "to"
line 146 it's
limitations_and_societal_impact: I think limitations of the models have not been fully discussed. In particular, based on the complexity of the models it does not seem that they are able to scale to large set of edges, in particular when the number of clusters is high. This would be a relevant limitation as I argue that, when increasing the number of edges, an higher number of clusters would be needed to better describe all of the network.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: -This work develops a family of novel non-parametric point processes for continuous-time edge data. In details, the model captures the periodicity exhibited in its base rate using K sigmoidal Gaussian processes(SGPs) \sigmoid(f_k(t)). 
-For each pair of two nodes, the nodal pair specific coefficients \p_{ik}, \pi_{jk} is used to rescale the pair-wise base intensity of (i,j) and (j,i).  
-To  capture the reciprocity, the model employs a second set of functions \sigmoid(g_k(t)) drawn from SGPs, which has more flexibility in capturing exciting behaviors, compared to the exponential decaying kernel used in Hawkes processes.
- A mean-field variational scheme is derived to perform model inference.
-The model is demonstrated on four real data to illustrate its estimated periodically varying base intensity and the endogenous functions. They also show the superior performance of the model in terms of training log-likelihood and link prediction.


main_review: -Originality
The idea of modelling time-varying base rate, and flexible endogenous functions is interesting and novel, to me.
Nonetheless, both the sigmoidal Gaussian processes and the main techniques for the proposed variational inference scheme have been exploited well before.

-Quality
The novel point processes models and the derived variational algorithm are sound.

-Clarity
The notations are not clear to me, e.g., why you have nodal pair specific coefficients \rho_{ik}\rho_{jk} in Eq.(4,5) but you change to v in Eq.(6)?
For experimental setup, it is not clear how to choose the number of features K, and why K is fixed for all data sets. I would consider the features may represent the latent group or clusters in the Email, Overflow and College data. Thus, some explainations are missing for the setting of K.
It is not clear why to chose the first 70% of edges as training data while chose remaining 30% as test data. As you are motivated, the exogenous intensity of events may have some periodical pattern. Thus, it is not clear how to judge the remaining 30% of edges still have the same periodicity, has entered a novel period that are not observed before. 

-Significance
To me, the novel point processes may have some impacts for a small group of audience who are interested in modeling excitatory behaviors of continuous-time dyadic data. 

limitations_and_societal_impact: The authors come up with some interesting phenomenons, which cannot be fully captured using Hawkes process with constant base rate and parametric kernels. In addition, I think the proposed symmetric intensity function may also some limitations. For instance, an employee may immediately reply to an email from her/his team leader, while the leader may not reply to the employees quickly. Hence, I am wondering how to model asymmetric nodal pair-specific intensity by extending the framework.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper proposes three  non-parametric point process variants for continuous-time edge modelling (CTEM) and the performance of the proposed framework evaluated on four real world dynamic networks. The three cases are 
1) a Poisson process in which K time series are generated according to a (sigmoidal) GP and weighted by node specific feature weights utilizing these temporal evolutions.
2) An extension to the Hawkes process such that reciprocity is invoked by an additional sigmoidal GP weighted by other node specific feature weights defined through time to past reciprocative interactive events.
3) A further extension incorporating edge clustering rather than node specific weights for the reciprocating effects (i.e., endogeneous functions) reducing the parameter space and enabling specific endogenous effects according to the cluster used for the specific reciprocative event of each dyad. 

VB is used for large scale inference and inducing points invoked to reduce complexity of the GP. The model is contrasted the Poisson and Hawkes IRM as well as Hawkes-DLS and HawkesNetOC and found on the four considered datasets to have superior performance as quantified using time-binned AUC but inferior results on two datasets for the (test I assume) log-likelihood. In particular, the edge clustering approach 3) forming the NP-HP-C is found to be the superior of the proposed approaches.

main_review: This is a well written and solid contribution in which multiple novel dynamic network models for directed dynamic networks are proposed. 

$\textbf{Originality}$

The paper is based on a series of well-established methodologies, i.e. Hawkes process for dynamic network modeling , sparse GP for modeling smooth temporal trajectories efficiently, VB for efficient scalable inference. However, their combination is novel and the proposed framework both sound and efficient enabling accurate dynamic network analysis accounting for reciprocative relationships. The edge clustering idea is neat and the use of dynamic feature representations that are weighted in terms of node specific weights sound and useful. The inference is non-trivial and appears well implemented.


$\textbf{Quality}$

The approach developed is technically sound and well decribed. The experimentation is solid with reasonable comparisons included to existing methods considering a sufficient  number of (four) datasets. The results are convincing, the approach well motivated and with potential wide applications within dynamic network analysis.

It would be good to position the paper also in relation to work using GPs for dynamic network analysis, see also:

Durante, Daniele, and David Dunson. "Bayesian logistic gaussian process models for dynamic networks." Artificial Intelligence and Statistics. PMLR, 2014.

and nested GPs

Durante, Daniele, and David B. Dunson. "Locally adaptive dynamic networks." The Annals of Applied Statistics 10.4 (2016): 2203-2232.

There are further more advanced Poisson Process stochastic block modeling approaches than the PP IRM worth discussing. This includes the work of:
Matias, C., Rebafka, T., & Villers, F. (2018). A semiparametric extension of the stochastic block model for longitudinal networks. Biometrika, 105(3), 665-680.

There are no error bars reported in the results. It would strengthen the paper to have multiple restarts and provide the robustness of the results in terms of initialization of the inference procedure. In particular Table 2 does not provide error bars. The authors state in checlist point 3.c "Due to the space limit, we put the average results in the main content and complete results with errors are provided in the Supplementary Material." However, I was unable to find such error bars in the supplementary for the most important Table 2 results.

$\textbf{Clarity}$

The paper is well written and in general easy to follow. However, method 3) is quite convoluted in its description and although the authors provide a reasonable attempt in describing the steps and intuition behind the generative model this section could be improved in clarity and the reader guided more carefully through the steps. 

The authors set the number of features to K=5 but does not discuss this choice further nor its impact on the results. It would be good to elaborate on this.

Furthermore, there are very minor comments to be considered:

these models oo -> these models to

In table 2 it is a bit confusing to call it training log-likelihood when it is in fact the test log likelihood I believe on last 30% of the available time segment.v Please clarify what the results refer to as it should be test-log likelihood. 

$\textbf{Significance}$

Although the use of Hawkes processes is well established the proposed approach is very useful in having time-evolving rates that are flexibly and well implemented through feature representations and clustering. The results are convincing and the proposed approach can have a solid impact within dynamic network analyses in general.

--------------------Response to the authors rebuttal-------------------------------------

I thank the authors for their responses to my concerns. It is also good that error bars are now provided and can be assessed. 
In regards to 1) I agree that the work of the authors differ. However, the point was to position the manuscript in context of existing work utilizing GPs for dynamic network analysis. I agree that the work I referred to using GPs differ considering time-discretized network and a Bernoulli likelihood. However, the approach also utilizes GPs defined in continuous time.

The authors could have elaborated in their rebuttal on the work of:
Matias, C., Rebafka, T., & Villers, F. (2018). A semiparametric extension of the stochastic block model for longitudinal networks. Biometrika, 105(3), 665-680.
Which also considers continuous time modeling using a stochastic block model including kernel and histogram approximations of the rate function.


limitations_and_societal_impact: There are no broader impact discussed of the paper nor potential negative societal impact. Enhanced abilities modeling dynamic networks and understanding their structure inevitably can also be used for surveillance, the enhancement of filter bubbles and be used to negatively influence people based on the improved understanding of their behaviors. Some of these potential negative aspects by the enhanced ability to predict behavior in social dynamic networks could be discussed.
review_title: null

Meta-review: Overall, the reviewers were positive about your work, though there are a number of suggestions that you should make to improve the manuscript. I remain a little perplexed by the performance on Overflow and Ubuntu--isn't it odd that AUC is highest but log-likelihood is lowest? Could this be due to the use of the mean-field variational Bayes approximation which can give very wrong posteriors?
Decision: Accept (Poster)