Cardinality-Regularized Hawkes-Granger Model
Authors: Tsuyoshi Ide
Georgios Kollias
Dzung T. Phan
Naoki Abe
Venue: NeurIPS 2021 Poster (2021)
Keywords: point process
Hawkes process
minorization-maximization
cardinality
Granger causality
temporal events

Abstract: We propose a new sparse Granger-causal learning framework for temporal event data. We focus on a specific class of point processes called the Hawkes process. We begin by pointing out that most of the existing sparse causal learning algorithms for the Hawkes process suffer from a singularity in maximum likelihood estimation. As a result, their sparse solutions can appear only as numerical artifacts. In this paper, we propose a mathematically well-defined sparse causal learning framework based on a cardinality-regularized Hawkes process, which remedies the pathological issues of existing approaches. We leverage the proposed algorithm for the task of instance-wise causal event analysis, where sparsity plays a critical role. We validate the proposed framework with two real use-cases, one from the power grid and the other from the cloud data center management domain. 

Cardinality-Regularize Hawkes-Granger Model
Tsuyoshi Idé
Georgios Kollias
IBM Research, T. J. Watson Research Center tide@us. iim. com
IBM Research, Watson Research Center gkolliastus .im............ ..m..ccoomo.oooooococcccooom.... com
Dzung T. Phan
Naoki Abe
IBM Research, T. J. Watson Research Center phandu@us ibm.
IBM Research, J. Watson Research Center nabe@us ibm. com
Abstract
We propose a new sparse Granger-causal learning framework for temporal event data. We focus on a specific class of point processes called the Hawkes process. We begin by pointing out that most of the existing sparse causal learning algorithms for the Hawkes process suffer from a singularity maximum likelihood estimation. As a result, their sparse solutions can appear only as numerical artifacts. In this paper, we propose a mathematically well-defined sparse causal learning framework based on a reegllaiie- Hawkes process, which remedies the pathological issues of existing approaches. We leverage the proposed algorithm for the task of instance- wise causal event analysis, where sparsity plays a critical role. We validate the proposed framework with two real use-cases, one from the power grid and the other from the cloud data center management domain.
Introduction
The Hawkes process 14! is one of the most popular models for analyzing temporal events in the machine learning (ML) community. It has been applied in variety of application areas including analysis human activities on social networks 23 36 39
[5] 38 2 [4], healthcare event analysis (6, [2], search query analysis (21 20 and even water pipe maintenance (48). In the studies of Hawkes processes, there have been two major milestones to date. One is the minorization- maximization (MM) algorithm [16]. The other is Granger causal analysis through Hawkes processes.
The first milestone was marked by Veen and Schoenberg process of earthquake aftershocks, they introduced the first MM-based maximum likelihood algorithm, which is often loosely referred to as EM (expectation (expectation-maxxiizae) maximization) due to their similarity (See Eq. and the discussion below). As their paper pointed out, the standard pradient-based maximum likelihood estimation (MLE) of multivariate Hawkes processes suffered from numerical stability issues, limiting their applicability in practice.
Based on the intuition of branching
The second milestone was achieved by a few pioneers including Kim al. [8 who first proposed an approach to Granger causal learning through the Hawkes process; Zhou et al. [51 who introduced l regularized MILE of a multivariate Hawkes process in the context of Granger causal analysis; and Fichler et al. 9 who theoretically established the equivalence between the flawkss-aase causality and the Granger causality [13].
Given these achievements and the well-known importance of sparsity in Granger causal learning 3 24), the MM algorithm combined with a sparsity-enforeing regularizer would seem be a promising path for Granger-causa analysis for stochastic events. This is especially true when the main interest in analyzing causal triggering mechanism event instances since the MM framework provides instance-wise triggering probabilities as side product. Interestingly, however, the likelihood
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

0.3
0.4
(a)
(b)
0.2
0.1
(c)
11.1 111
1
100 1
B1
11 11:101
In 11
111
1.11
10
I
Figure 1: The Blawes--Grangee model allows two different levels of causal analysis: (a) instance-wise and (b) type-wise, in which well-defined sparsity is essential for causal diagnosis. (c) Example of fave-variate point process data, where each 'I" represents an event instance.
function of the MM algorithm has singularity that in fact prohibits any sparse solutions. Despite its significance, to date little attention has been paid to this issue in the ML community.
In this paper, we provide a mathematically well-defined solution to sparse causal learning on temporal event data for the first time. Specifically, we introduce novel aardinality-regulrizzz MM framework for the Hawkes process. Although cardinality- or fo-regularization is known as ideal way of en- forcing sparsity, makes optimization problems generally hard to solve. Even checking the feasibility is NP-complete [17]. We will show that the Hawkes process in fact has a semi-analytic solution under an lo-constraint, which is free from pathological issues due to the logarithmic singularity at zero, unlike existing e f1-regularization approaches [51] 46). By leveraging the well-defined sparsity and the side product provided by the MM framework, we tackle the task of instance-wist causal diagnosis in real world domains. Since any event instance occurring in the past is potential candidate cause of given event of interest, achieving sufficient sparsity in the learned causal structure has proven critical in these applications. See also Fig. [ illustration.
To the best of our knowledge, this work represents the first fo-regularized MM learning framework for the Hawkes process applied to simultaneous instance- and type-level causal analysis.
2 Related work
Related work relevant to this paper can be categorized into the following five categories.
Graphical Granger models: For multivariate temporal data, sparse graphical modeling with a (group) lasso penalty [3] 24 25 has been standard approach Granger causal analysis in the ML community. It replaced subtle independence tests with sparse matrix estimation in a systematic way. and significantly expanded the application field of Granger causal analysis. Their neural extension has also been proposed recently [40). However, most of those studies use vector autoregressive (VAR) or space models (including recurrent neural networks (RNNs)) and thus assume as the input multivariate time-series data with a regular time interval. As result, they cannot handle event sequences without certain aggregation/smnn sagregation/mmoohhnng operation and thus it is not possible to analyze causal structure of event instances. We will show that event aggregation can seriously impact significantly impact the accuracy of causal analysis Sec.
Conventional MLE: Since Hawkes' original proposal 14] gradient-based MLE has been a main- stream approach to fitting Hawkes processes (see, e.g. (34) In particular, has been used extensively in the field of seismology (32). However, as Veen and Schoenberg [42) and Mark and Weber [26] convincingly demonstrated, gradient-based MLE tends have numerical instability issues. Unless detailed prior knowledge on the data generating mechanism is available, which is indeed the case in seismology, its applicability industrial stochastic event data can be limited. Stochastic optimization can be leveraged (e.g. (3I)) but it brings in extra variability to inference. In addition to conventional MLE, there are a few other methods to fit multivariate Hawkes processes, such as least squares [47 and moment matching I They trade off the direct linkage to the likelihood principle for computa- tional stability, and are out of our scope. This paper focuses on how to enhance the MM algorithm through cardinality regularization for sparser and thus more interpretable solution.

---

Neural Hawkes models: Neural point process models typically replace all or part of the terms of
the Hawkes intensity function to capture nonlinear temporal dependencies [8] 45 27 33
IT [2].
However, most of the works either assume regular-interval time-series or lack the perspective of instance-wise causal triggering. The transformer architecture [50], on the other hand, allows extracting instance-level information, but the self-attention filter essentially represents auto-correlation rather than Granger causality, making it inapplicable to event-causal diagnosis. Also, the positional encoding approach, originally introduced in a machine translation task, where the notion of time-stamp does not exist, implicitly assumes a regular time grid. partly sharing the same limitation as VAR- and RNN- based methods. Finally, it lacks systematic pparsity-enforcie mechanisms for better interpretability.
Sparse MM algorithms: As discussed in Introduction, Zhou et al. [51] and Xu et al. [46] are the pioneers who first attempted to combine sparsity-efforeing regularizers with the MM algorithm. Unfortunately, due to the logarithmic singularity of the likelihood function, sparsity can only be achieved as numerical artifact under f and f constraints. Sections and will demonstrate this point theoretically and empirically.
Applications of Hawkes processes: Somewhat related to our motivation, there are a few recent stud- ies that pay particular attention to the duality between microscopic (instance-level) and macroscopic (type- or aggregate-level) random processes. For instance, Wang et al. between a point process and a rate equation an aggregate level, Li et al. [9] discuss macroscopic
433 discuss the relationship
information diffusion through human interactions, and Zhang et al. [49] discuss failure propagation in compressor stations. None of them, however, explicitly performs simultaneous instance- and type-level causal analysis. As discussed in Sec. [ our framework can be viewed as disciplined and effective solution to the issue of "warning fatigue," which prevails across many industries [10 28
3
Preliminaries
This section provides the problem setting and recapitulates the basics of stochastic point processes.
3.1 Problem setting
Before getting into the formal definitions, let us take a brief look at a concrete use-case from cloud data center management (See forr more details). In data center, various computer and network devices continuously produce numerous event logs, which can be viewed as marked temporal event data with "marks" being the warning types. Due to inter-connectivity of the devices, one event from a device, such as a warning of type "response time too long." " may trigger many related events downstream. The more critical the original error, the more numerous the resulting set of events tend to be. Thus for a given event instance of interest (called the target), it is desirable to find which event instances in the past are causally related to it. We call this task event causal diagnosis.
There are two essential requirements in event causal diagnosis. The first one is the ability of providing instance-specifty causal information in addition to the type-level causality. For instance, even if the i-th event type is on average likely to have a causal relationship with the j-th type, one specific instance of the i-th event type may have occurred spontaneously. A practical solution for event causal analysis must therefore perform type- and instance-level causal analysis simultaneously. The second requirement is the ability of providing sparse causal relationship. Since the number of event instances can be large, the capability of effectively shortlisting the candidates that may be causally related to a target event is essential in practice. To the best of our knowledge, our sparse Hawkes-Grange model is the first that meets these two requirements.
We are given an event sequence of N + 1 event instances:
= (lo, do), (11, (Lv.
where fn and d, are the timestamp and the event type of the rt-th event, respectively. The timestamps have been sorted in non- decreasirg order to I t S.... ...... S fx. There are D event types {1, D} with D < N. We take the first time stamp as the time origin. Hence, the remaining N instances are thought of as realization of random variables, given dg- As a general rule, we use t or u as a free variable representing time while those with subscript denote an instance.
The main goal of event causal diagnosis, which is an unsupervised learning task, is to compute the instance triggering probabilities { where a is the probability for the n-th event (n = I,...,N)
3

---

m). instance to be triggered by the i-th event (i = .,...,,), By definition, n 2 i, and
In 1, Vn (1....,NN.
We call In,n the self triggering (or simply self) probability. Note that providing {9n,) amounts to providing weighted ranking of candidate triggering (or causing) events, where the weights sum to one. We wish to have as few candidates as possible with the aid of sparse causal learning.
3.2 Likelihood and intensity function
Since all the events are supposed be correlated, the most general probabilistic model is the joint distribution of the N events. By the chain rule of probability density functions (pdf), the joint distribution can be represented as
f(li,di).....
(lo, do)) II M(nn Hn--)),
where Hn-1 denotes the event history up to 1n namely, Ha-1 {(la, do); .... (6--1,d, dn-1))- We use (()) to symbolically denote pdf. This decomposition readily leads to the definition of the base likelihood function Lo:
Lo Σ In f(n dn, Hn-1) + Σ In f(n Hn-1).
The distribution f(t dn. Hn-1) is defined on fx-1 S t co and satisfies the normalization condition in that domain. For the task of causal diagnosis, the first term in Eq. (3) plays the central role. We omit the second term in what follows, assuming f'd (H--1) is constant.
The intensity function given Hn-1 1 is defined as the probability density that the first event since l occurs. This is conditional density. When considering the density at f, the condition reads "no event occurred in (t)- Hence.
f(t d, Hnn Si,_,, du f'u d, (,H4-11)
Xa(t | 11-1))
(4)
where Xa(t)Hnn11) is the intensity function for the d-th event type, given the history Hn-1- Notice that the r.h.s. can be written de In (1 - du f(u dn. Hn-1)) Integrating the both sides and arranging the terms, we have
f(t d, Hn-1) da(t Hn-1) exp {- f du da(u Almr)
(5)
which allows representing Ly terms of the intensity:
>/f
f"a du Xd, (ul (ul3Hn-1) hn-
Lu In dd, (tn (m-11)11))
(6)
Note that the integral in the second term cannot be reduced to that of (to, (t) in general due to d, being dependent on n.. This fact sometimes ignored in the literature.
Cardinality-Regularii Hawkes-Granger Model
This section provides specific model for the intensity function and discusses the connection to Granger causality to define the Hawkes-Granger model.

---

4.1 Intensity function and Granger causality
For the intensity function in Eq. (6. we introduce a specific parameterization of the Hawkes process:
Xa(t Hn-1) Mdd
Σ i=0
Ad.d t,). tt).
where pa 2 0 and @u ---) fz) are called the baseline intensity and the decay function, respectively, of the d-th type. Ad,d, the (d, d.)-clement of matrix A R DXD, which called the impact matrix (a.k.a. triggering or kernel matrix).
The Hawkes process has potential indistinguishabiiity issues in the second term due the product form. We remove some of the indistinguishabbiity by (i) imposing the normalization condition Jg da(u) du = 1 and (ii) making Φ independent of d;- In this parameterizaton, Ad,d, is the triggering impact from event type d, to while dh represents the susceptibility of d. Also, pat represents the tendency of spontaneous occurrence. Due the arbitrariness of the time unit, the decay function has the form (u) = (a where (p()) is nondimensional function and 3a is called the decay rate. Popular choices for y are the exponential (x)(-u) and power nn1 + +u) 9-1 functions with > being a given constant.
Figure 2 illustrates the model 77. in which >a(t H4) is shown with an arbitrary decay function. We assume that Ad.d; = Ad.d, = 0 and Ad,dz > Ad,d.- The effect of the 2nd instance on (t. d) is smaller than that of the 4th due to time decay despite the larger Ad,dz- On the other hand, as shown with the dashed lines, the 1st and 3rd instances have no effect on the occurrence probability for the assumed d-th type in any future time point. This is in fact how Eichler al. [9] defined the Granger non-causality in the Hawkes model (see also (
Definition (Hawkes process and Granger non non- causality 19) If Ad,d' = 0, event instances of the d' d'-rype are Cranger-non-cause, to those of the d-th type.
Definition T states that estimating A amounts to learning a Granger causal graph. This Granger non-cause characterization holds also in nonlin- ear Hawkes models, where the r.h.s. of Eq. 7 is replaced with g(ua + Ei Aa.d, @a(t- 10--(,)) 4.). where g(.) is some nonlinear function. In that case, however, the original meaning of Ma and Ad.d, no longer holds. For example, fe con- tributes not only to spontaneous occurrences but also to causal triggering, due to cross-terms be- tween Ad and Ad,d, da in the intensity function. Also, as the MM strategy is no longer applicable, there is no natural way of defining the triggering probabilities (see the next subsection). Given that main interest lies in causal diagnosis ("who caused this?"), rather than black-box data fitting, the linear additive form Eq. 7 is particularly convenient, and hence is our primary model.
d1
d2
d3
dp
(t, d)
81
t2
t3
t4
Figure 2: Illsstaaiion of Hawkes model in Eq. 7 showing Aa(t H44) as example. The d1- and da-types are not causally related to d.
4.2 Cardinality-regulaiiie
miinoization-maximizztoon maximization framework
As we saw in Fig. 2
achieving sparsity in A is of critical importance in instance- causal analysis: directly leads to reducing the number of event candidates to be causally associated. To guarantee maximum likelihood:
sparsity, we propose the following cardinality
1
max (Lo(A, u1 3) 7||A|lo (2,A,, A,
u. 3)), Ra (M 12 us |1.113 + +
(8)
where we have defined a (BI...., Bp) and μ e (Pi...., PD) and the to norm | Alo represents the cardinality (the number of nonzero elements) of A. Also, || ll2 is the 2-norm and || ||r the Frobenius norm. M3, Pys, PA are constants for regularization strength. For T, we note that Eq. (15) can be viewed as MAP (maximum a posteriori) estimation with the Bernoulli prior (1 - (lll) Di-AAlo,, where 0.5 7 < 1 is the probability of getting 0 in the matrix elements. By taking the logarithm and equating to -7 ||Aloo. we have
r = 1n(y((1 - ->]-
(9)
5

---

With having the specific interpretation, this equation makes the choice of a value easier.
As mentioned earlier, numerically solving for MLE is known to be challenging even when mainly due to the log-sum term In Xa in Eq. (6). The MM algorithm leverages the additive structure of the Hawkes process in Eq. (7) apply Jensen's inequality in manner similar to the EM algorithm for mixture models
30 Specifically, we rewrite Eq. as Ad, (t.///n-))
Ei= ad where
n,
Hdn.: (Adad, - t.), i i= 0, ......... .
i i= nl
(10)
With arbitrary distribution Zrgt over i such that E qn,i 1 for Vn, Jensen's inequality guarantees In Ei=
o anndd gmas which leads to a lower bound of the log likelihood:
ghicch 2,
In Ei= =0=0 qnit
N
n-1
Anii du od, (
Lo 11
A
9n,s In
Udd Ud..n.n- An
Σ
Adn.d,
(1I)
An
n=1
1=
1=
where we defined An.; t, 4. The tightest bound is obtained by maximizing the r.h.s. with respect In,i under the normalization condition:
n, Un.i 4n.i- Adn (t,, (n-1) Add,,d Od, tz), + n.
Ad, (ln (n-1))
(12)
With a proper initialization, the MM algorithm estimates {gm,a} and (μ, B, A) alternately. The whole procedure is concisely summarized as
μ.. 3, arg max {L1 -llAllo R2) given fgn.)- given J.. B.A.
(13)
{gm.i} = (Eq.
(14)
Similarity the EM algorithm 30 is evident, but they differ since apply Jensen's inequality only partially in Lo. In this framework, } was introduced as a mathematical artifact in Jensen's 's inequality. However, opens new door to instance-level causal analysis. We interpret bu.r as the instance triggering probability that the 7-th instance has been triggered by the i-th instance. The MM-based Hawkes process causal both at the instance-level and the type-level. With this in mind, the Hawkes- Graner model.
we call the framework of Eqs. [13 14)
5 Sparse Causal Learning via Cardinality Regularization
This section discusses how find the solution A in Eq. (13) We leave the estimation procedure of μa and / to Appendix A (all the Appendices are in the supplementary material).
By Eqs.
8 and 11 the optimization problem of Eq. 13 with respect A is rewritten as
D
(Qu,1 Ak. - HR.(As.) - VA ||A| T||Allo: J.
max
Σ
(15)
2
k.1=1
where we have defined matrices Q and H as
f du Gd. (u).
An,
Qk.1-
Σ 54, Ha.1 A Σ Sa (n.8)
hns A
JAn..
(16)
(1),8)
Qi. represents how likely the type k and / become cause-effect pair. For ease of exposition, consider the vectorized version of the problem by defining A vec A, h 2 vec H, and g - vec Q:
Σ V (T - T T|00010 }.
max ZE
T (xm) a Im In F' hm
VA 2 >
(17)
2
m
where Jmn Am 0, TA 0 hold. This is the main problem consider in this section.
Let us first look what would happen we instead used the popular & or Ê regularizer here. For the tp norm ||>||p (Em Imm ?,,,, the following theorem holds:

---

(a)
Zm
(b)
Fm
Fm
V
V
V m
Jm
3m
3m
Figure
Three cases in Eq.
18 (a) in > e and (b) two possibilities when x E.
Theorem 1. For p 2 1, the problem maxę { Lm (2m) - is convex and has a unique solution. Let x² be the solution. The solution cannot be sparse, i.e,, I'm frror Vm, if Im O.
(Proof) The convexity follows from d² dr2 (2m) - (9m/ pm/mm VA and the convexity of the e--norm. The second statement follows from limz-50 In - -00 n
In the MM iteration, we need to start with some Qu >> O for all k. to make all the event types eligible in causal analysis. In that case, Vm, 9m >> 0, and thus any Ax, cannot be zero. Therefore, under any 2 1, including numerical artifact. One can avoid this particular issue by performing conventional MLE without the MM algorithm, but, discussed earlier, that is not viable solution in practice due to the well-known numerical difficulties of the conventional MLE 42 26]..
(l,) and 2 (f2,1), any "sparse" solution must be viewed
This situation is reminiscent of the issue with density estimation with probabilistic mixture models as discussed by Phan and Idé [35], who first introduced the notion of "e-sparsity." Here we employ similar approach. To handle the singularity at zero, we introduce small constant >> 0 and we propose solve
x² max Σ (Vm(mm) (2 > <)},
(18)
to get e-sparse solution instead of the original problem (77. where I(-) is the indicator function that returns when the argument is true and otherwise. Note that the condition VA > 0 is important Jm close zero; It makes the objective strongly convex (easily seen from the proof of Theorem [ and thus makes the problem (17) well-behaved. We remark that while it is possible use an analogous e-sparsity approach with the & 1 regularizer, may not be as well-behaved due to the piece-wise linearity of & -norm.
for stable learning when
To solve Eq.
18
first note that L m is concave and has the maximum at
2 (-hmm + h2 4VA9m
1
I'm
(19)
which is obtained by solving us define set of indices B & (m
d Ma
O. Suppose that we had a solution of Eq. 18 somehow, and let
dam.
a > <). Since the objective function Eq. (18) gets penalty
- in F. > for an Z.m. to be able to be the solution in this domain, the objective must catch up on
T at maximum (see Fig. (a) for illustration). Hence, Vm B, we have
Im >
V (E) < Mm (Tm) T.
(20)
Conversely, if these conditions met, must be in B due to the concavity of Lm Therefore, can learn whether an index m in B or not by checking these conditions.
With this fact in mind, we solve Eq. (8) ( for m E B and & B separately:
Vm B.
Im arg max [ (Im) T). 2m
(21)
Vm & B, 2 arg max v (m((m)) sm
subject to 6 Fm 2 0.
(22)
For Vm B, the optimality condition is
d dam
(V (Im) -
hm VAJm 0, Om
(23)
Jm

---

Algorithm Hawkes: sparse learning impact matrix
1: Input: 9 Vec Q, h = vee and VA 0, 0, >
2: for all m = ..... D2 do
3: compute F. with Eq. (99
4: check Eq. (20) to see if m E B.
5: if m B then
6: Fm = 3m
7: else
8:
En = min Jm,
9:
end if
10:
end for
return A vec-1 1 x (i.e., convert back to matrix)
which readily gives xm E. under Eq. 20
For Vm & B, with Lagrange multiplier Em, the Karush-Kuhn-Tuce (KKT) condition is given by
Im hm
VATm mm 0, (m xm) O. Ems
mm 0.
(24)
Im
As illustrated in Fig. (b), there are two possibilitics here: One is 2t, o e, In this case, 5m must be O, and the first equation gives zt, 2m. which holds when Im The other is x which holds when F. >
Although, can be viewed as a 'zero-ness' threshold, the solution derived is qualitatively different from naive thresholding applied the non-regularized solution æ. See Sec. 6 for an empirical validation.
Algorithm summary Algorithm summarizes LyHawkes, the proposed algorithm, which used as part of the iterative MM procedure in Eq. 13). The estimation procedure for μt, (3 can be found in Appendix
The total complexity O(N + D²), which is the same as for the existing MM algorithm. For input parameters, can be fixed to value of user's choice in 0.5 < < through Eq. (2). The parameters MA, Mg, Up are important for stable convergence. It recommended to start with small positive value, such 10-5, and increase if numerical issues occur. These parameters should eventually be cross-vaud cross-validated with independent episodes of event data, or ground through causality data. In Sec. (. we presents an approach to determine as the one that gives the break-even accuracy. If validation dataset is unavailable at all, the use of Akaike's information criterion (AIC) can be one viable approach, given that ||Al|o approximates the total number of free parameters fitted.
Experiments
Our focus in this section is to (1) show the impact of the equi-time-interve assumption of the existing Granger causal learning models, (2) demonstrate how LyHawkes produce a sparse solution, and (3) show its utility in real-world use-cases. We leave the detail of the experimental setup Appendix B.
Comparison with neural Granger models. We generated two synthetic multivariate event datasets, Sparse5 and Dense10, with a standard point process simulator tick (4) Sparse5 has D = 5 with a sparse causal graph shown Fig. (b). We generated different datasets by changing the random seed of tick, one of which has been shown in Fig. (c). Dense10 has D = 10 and was generated with a relatively dense and noisy causal graph. Both have ~ 000 event instances.
First, we compared LoHawkes with cLSTM and cMLP [4)]. state-of-the-ar Granger causal learning algorithms, on Sparse5. These correspond a nonlinear extension of autoregressive and state-space models, respectively. covering the two main time-series prediction paradigms known to date. We estimated the Granger causal matrix with many different values of the regularization parameters. We chose VAA V3. 1/ to be 0.1 and tested 7 = 0.5, 1, The goal is to retrieve the simple causal graph Fig. (b), which has 2 positive and negative edges as the ground truth, omitting the self- loofs For cLSTM and cMLP, the dataset was converted into 5-dimensional equi-interval time-series of event counts with N time points, based on a sliding window whose size ur = 10 times larger than the mean event inter-arriva time. The number of hidden units and the lag (or the context length of LSTM)
8

---

LOHawkes
<1.STM
<MIP
1.0
Recuraey
accuracy
0.0
022
0.0
0.5
0.4
0.6
100
ml
10
100:s11
Aycasl
Figure 5: Comparison of x (flattened A in each row) computed with 100 different values.
Figure 4: TN (red) and TP (blue) accuracies as a
function of log regularization strength.
were set to 100 (from [40]) and 10, respectively, for both. The mean computational time was (46, 382) seconds per parameter set for (LyHawkes, cMLP, cLSTM), respectively, on laptop PC (i7 CPU, 32GB memory, Quadro P3200 GPU).
Figure M compares the true positive (TP) and true negative (TN) accuracies as function of log regularization strength. LjHawkes uses T = 1.
Table 1: Break- accuracies in Fig.
Ly Hawaess
cLSTM
cMLP
1.00 E 0.11 0.43 = 0.09 0.31 1 0.10
The plot, which we call the contrustive accuracy plot (CAP), allows to directly choose one regularization strength and is more useful than the ROC (receiver operating characteristic) curve. This is especially true this case, where the variety of positive samples is limited. See Appendix B for more comments on CAP. The fitting curves in the CAP were computed with Gaussian process regression (GPR) [37] with optimized hyperparameters. The TP accuracy at the intersection with the TN curve are called the break-even accuracy, which can be thought of as an overall performance metric. Table summarizes the break-even accuracies, where the standard deviation was estimated with the GPR fitting. The contrast is clear. cLSTM and cMLP failed to capture the very simple causality while L,Hawkes reproduced it almost perfectly. The main reason is that the event density of self-exciting events can be highly non-uniform in time (as seen from Fig. (c)), which is inconsistent to the equi-interval assumption of the autoregressive or RNN-type models. The conclusion held over all the parameter values we tested: {0.5, 1,2] and W. E {1, 5, 10, 20).
Comparison with sparse Hawkes models. Next, compared LyHawkes with E regularrizatoon based [5I] ("L1") and fg.-reguultiizationn based [46] (LL2.1) '') models on Dense10. Figure compares the solution A" obtained by the three methods. We solved Eq. 17) 100 times for each, changing the value of the regularization strength T from 0 to 2. We used c = 0.01 for LyHawkes. All the three methods share the same values of r and 1/A 10 The matrix elements are sorted in the ascending order of the value of & in Eq. (9). As shown in the figure, L, and L2,1 produce a smooth non-sparse profile, expected from Theorem [ In contrast, LgHawkes gets more "off" entries (shown in yellow) as t increases. As clearly seen from the irregular sparsity patterns in Fig. 55) the result is different from naive thresholding on ã (Eq. (19)). At given sparsity level, our solution is guaranteed to achieve the highest possible likelihood while naive thresholding is not.
Now that we have confirmed the capability of LoHawkes in sparse Granger causal learning, let us leverage it in real-world causal diagnosis tasks.
Power grid failure diagnosis. We obtained failure event data ('Grid") of power grid from U.S. De- partment of Energy (29). The failure events represent abrupt changes in the voltage and/or current signals measured with phasor measurement units (PMUs), which are deployed in geographically distributed locations in the power grid. The network topology is not given for privacy concerns. Only anonymized PMU IDs are given. We are interested in discovering hidden causal relationship in data-driven manner from the temporal event data alone. This is a type-level causal diagnosis task.
The dataset records N = 3811 failure events labeled as "line outages" from D) = 22 PMUs over a 10-month period. We grid- rearche the model parameters based on AIC to get x (10 100- 14 10 4) for (M/, V/3 AA) and (1, 1) for (r,c). The value of corresponds to about 3% of max..1 Ar,-- We used the same 7 for the & and l2,1 regularizers. We used the power decay of 12 = 2 to capture long-tul behaviors. Figure O compares computed A, in which nonzero matrix elements are shown in black. In L, and L2, zero entries can appear only when Qv.1 happens to be numerically zero. contrast, LoHawkes enjoys guaranteed sparsity. From the computed A, a hidden causal structure among PMUs were successfully discovered. In particular, a PMU called B904 seems to be a dominatingly influential source of failures. We leave further details to another paper.

---

200
500
0.0
HOW TOTDONN
6N 400
UIM FAL
UNIOIN
UPDOIN
LIW FAI
DOW
HOS AOETDONNN
EiR NIT
NET MAS ETR
600
JIM
ETH
Figure 6: Sparsity pattern of estimated A on the Grid data with LyHawkes (left), L1 (mid- dle), and Lg,1 (right).
Figure 7: Results the Cloud data. (Left) Nonzero elements of {gm.a). (Right) Triggering probabilities for the 150th instance (i.e. (150,1)
Data center warning event diagnosis. Finally, applied LyHawees to real data center manage- ment task. We obtained = 718 warning events from a real cloud system. These events result from filtering logs emitted by network devices and each has its type. There are D 14 unique event types in our dataset ("Cloud'). This is an instance-level causal diagnosis task ("who caused this?").
Figure 7 (left) visualizes nonzero entries of ]. where those with gn, < 0.01 are omitted. As expected, {gn.s) quite sparse, and hence, event consolidation can straightforwardly performed by picking nonzero triggering probabilities. The right panel shows such example, showing 0150, in which the rightmost slot (in red) corresponds to the self probability 9150.150- For each i, its event type d, is shown below the bar. The type of the event in question, ETH_ INIT, is related to the process of initializing an Ethernet interface. Note in the figure that the self probability of this instance was computed as O, while several preceding instances of the same type have positive triggering probabilities, leading to successful suppression of duplication.
Many instances have zero triggering probability despite their time proximity (the six events with positive probabilities were within 27 seconds from the 150th event), thanks to the sparsity of For example, this dataset contains 416 instances of event type UPDOWN adding considerable noise but were appropriately ignored by our method. Unlike naive hard- windowing approaches, our framework able to sift for genuine causal relationships.
7 Concluding remarks
We proposed
sparse Granger-causal learning framework for stochastic event data. We pointed out that the existing sparse MM algorithms do not have a sparse solution the exact sense (Theo- rem []. We also showed that the existing neural Granger approaches (cMLP, cLSTM) of limited use for event data, mainly due to its equi-time-inee -iniiiia--ttel assumption. The proposed Hawkes- Granger model produces mathematically well-defined sparsity and allows simultaneous instance- and type-level causal event diagnosis with good interpretability enabled by the sparsity.
Acknowledgement T.I. is partially supported by the Department of Energy National Energy Tech- nology Laboratory under Award Number DE-OE0000911. A part of this report prepared as an account work sponsored by an agency Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, process disclosed, represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state reflect those of the United States Government or any agency thereof.
United States Government. Neither the United States
References
[1] M. Achab, E. Bacry, S. Gaïffas, I. Mastromatteo, and J.-F. Muzy. Uncovering causality from multivariate Hawkes integrated cumulants. Journal of Machine Learning Research, 18(1):6998- 7025, 2017.
10

---

[2] H. Alvari and P. Shakarian. Hawkes process for understanding the influence of pathogenic social media accounts. In Proceedings of the 2nd International Conference on Data Intelligence and Security (ICDIS), pages 36-42. IEEF, 2019.
[3] A. Amold, Y. Liu, and N. Abe. Temporal causal modeling with graphical Granger methods. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 66-75, 2007.
[4] E. Bacry, M. Bompaire, P. Deegan, S. Gaiffas, and S. V. Poulsen. tick: a Python library for statistical learning, with an emphasis on Hawkes processes and time-dependent models. Journal of Machine Learning Research, 18(214):1-5, 2018.
[5] E. Bacry, M. Bompaire, S. Gaïffas, and J.-F. Muzy. Sparse and low-rank multivariate Hawkes processes, Journal of Machine Learning Research, 21(50): 2020.
[6] E. Choi, N. Du, R. Chen, L. Song, and J. Sun. Constructing disease network and temporal progression model via context-sensitive Hawkes process. In Proceedings of the 2015 IEEE International Conference Data Mining (ICDM), pages 721-726. IEEE, 2015.
[7] K. N. Dominiak and A. R. Kristensen. Prioritizing alarms from sensor-based detection models in livestock production-A review on model performance and alarm reducing methods. Computers and Electronics in Agriculture, 133:46-67: 67, 2017.
[8] N. Du, H. Dai, R. Trivedi, U. Upadhyay, M. Gomez- RodrizRa and L. Song. Recurrent marked temporal point processes: Embedding event history to vector. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 1555-1564, 2016.
[9] M. Eichler, R. Dahlhaus, and J. Dueck. Graphical modeling for multivariate Hawkes processes with nonparametric link functions. Journal of Time Series Analysis, 38(2):225-24) 2422 2017.
[10] H. T. Elshoush and I. M. Osman. Alert correlation in collaborative intelligent intrusion detection systems-A survey. Applied Soft Computing, 1177):339-44665, 2011.
[11] Enguchard, D. Busbridge, A. Bozson, C. Woodcock, and N. Hammerla. Neural temporal point processes for modelling electronic health records. In Proceedings of the Machine Learning for Health, NeurlPS Workshop (ML4H), pages 55-113, 2020.
[12] V. Escobar. A Hawkes process model for the propagation of covid-19: Simple analytical results. EPL (Europhysics Letters), 13166)58000, 2020.
[13] C. W. Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica, 37(3):424-438, 1969.
[14] A. Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1):83-90, 1971.
[15] S. A. Hosseini, A. Khodadadi, A. Arabzadeh, and H. R. Rabiee. HNP3: hierarchical
nonparametric point process for modeling content diffusion over social media. In Proceedings of the 2016 IEEE International Conference on Data Mining (ICDM), pages 943-948, 2016
[16] D. R. Hunter and K. Lange. A tutorial on MM algorithms. The American Statistician, 58(1):30- 37, 2004.
[17] C. Kanzow, A. B. Raharja, and A. Schwartz. Sequential optimality conditions for cardinality- constrained optimization problems with applications. Computational Optimization and Applica- tions, 80(1):18055-1111 221, 2021.
[18] S. Kim, D. Putrino, S. Ghosh, and E. N. Brown. Granger causality measure for point process models of ensemble neural spiking activity. PLoS Computational Biology, 7(3):c 1000:1000110,, 2011.
[19] H. Li, H. Li, and S. S. Bhowmick. CHASSIS: Conformity meets online information diffusion. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (SIGMOD), pages 1829-18400 2020.

---

[20] Li, H. Deng, J. Chen, and Y. Chang. Learning parametric models for context-awarr query auto- completion via Hawkes processes. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining (WSDM), pages 131-139, 2017.
[21] Li. H. Deng, A. Dong, Y. Chang, and H. Zha. Identifying and labeling search tasks via query- based Hawkes processes. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 731-740, 2014.
[22] S. Li, X. Gao, W. Bao, and G. Chen. FM-Hawkes: A Hawkes process based approach for modeling online activity correlations. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (CIKM), pages 1119-1128 2017.
[23] J. C. Louzada Pinto and T. Chahed. Modeling user and topic interactions in social networks using Hawkes processes. In Proceedings of the 8th International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS), pages 58 65, 2014.
[24] A. C. Lozano, N. Abe, Y. Liu, and S. Rosset. Grouped graphical Granger modeling for gene expression regulatory networks discovery. Bioinformatics, 25(1 25(12):110-11 110-118, 2009.
[25] A. C. Lozano, H. Li, A. Vieulescu-Mizil Y. Liu, C. Perlich, J. Hosking, and N. Abe. Spatial- temporal causal modeling for climate change attribution. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 587-596, 2009.
[26] M. Mark and T. A. Weber. Robust identification of controlled Hawkes processes. Physical Review E, 101(4):043305, 2020.
[27] H. Mei and J. M. Eisner. The neural Hawkes process: A neurally self-modulating multivariate point process. Advances in Neural Information Processing Systems, 30:6754-6764, 2017.
[28] Moyne and J. Iskandar. Big data analytics for smart manufacturing: Case studies in semicon- ductor manufacturing. Processes, 5(3):39, 2017.
[29] National Energy Technology Laboratory. Award number DE-OE0000911, 2021.
[30] R. M. Neal and G. E. Hinton. A view of the EM algorithm that justifies incremental, sparse, and other variants. In Learning in graphical models, pages 355-368. Springer, 1998.
[31] M. Nickel and M. Le. Learning multivariate Hawkes processes at scale. arXiv:2002.12501. 2020.
[32] Y. Ogata. Seismicity analysis through point-process modeling: A review. In Seismicity patterns, their statistical significance and physical meaning. pages 471-507. Springer, 1999.
[33] T. Omi, Ueda, and K. Aihara. Fully neural network based model for general temporal point processes, Advances in Neural Information Processing Systems, 32:2122-2132, 2019.
[34] T. Ozaki. Maximum likelihood estimation of Hawkes' self- point processes. Annals of the Institute of Statistical Mathematics, 31(1):145-D 155, 1979.
[35] D. T. Phan and T. Idé. ly-regularized sparsity for probabilistic mixture models. In Proceedings of the 2019 SIAM International Conference on Data Mining (SDM), pages 172-180. SIAM, 2019.
[36] J. C. L. Pinto. T. Chahed, and E. Altman. Trend detection in social networks using Hawkes processes. In Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 1441-1448, 2015.
[37] C. E. Rasmussen and C. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006.
[38] M.-A. Rizoiu, L. Xie, S. Sanner, M. Cebrian, H. Yu, and P. Van Hentenryck. Expecting to be hip: Hawkes intensity processes for social media popularity. Proceedings of the 26th International Conference World Wide Web (WWW), pages 735-744, 2017.
12

---

[39] Y. Tanaka, T. Kurashima, Y. Fujiwara, T. Iwata, and H. Sawada. Inferring latent triggers of purchases with consideration of social effects and media advertisementss In Proceedings of the Ninth ACM International Conference Web Search and Data Mining (WSDM), pages 543-552, 2016.
[40] A. Tank, I. Covert, N. Foti, A. Shojaie, and E. B. Fox. Neural Granger causality. IEEE Transactions Pattern Analysis & Machine Intelligence, Early 10cess(01):111, 2021.
[41] A. C. Türkmen, Y. Wang, and J. Smola. Fastpoint: Scalable deep point processes. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD), pages 463-480, 2019.
[42] A. Veen and F. P. Schoenberg. Estimation of space-time branching process models in seismology using an EM-type type algorithm. Journal of the American Statistical Association, 103(482):614 624, 2008
[43] Y. Wang, X. Ye, H. Zhou, H. Zha, L.d Song. Linking miero event history to macro prediction in point process models. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), pages 1375-1384, 2017.
[44] O. G. Ward, J. Wu, T. Zheng, A. L. Smith, and Curley. Network Hawkes process models for exploring latent hierarchy in social animal interactions. arXiv preprint arXiv:2012.09598, 2020.
[45] S. Xiao, J. Yan, X. Yang. H. Zha, and S. M. Chu. Modeling the intensity function of point process via recurrent neural networks. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI), pages 1597-1603, 2017.
[46] H. Xu, M. Farajtabar, and H. Zha. Learning Granger causality for Hawkes processes. In Proceedings of the 33rd International Conference on Machine Learning (ICML), pages 1717- 1726, 2016.
[47] H. Xu, D. Luo, and L. Carin. Online continuous time tensor factorization based on pairwise interactive point processes. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI), pages 2905-291 2005--91-2 2018.
[48] C. Zhang, H. Wu, R. Bie, R. Mehmood, and A. Kos. Dynamic modeling of failure events in preventative pipe maintenance. IEEE Access, 6: 6:12599-12550, 2018.
[49] L.-n. Zhang, X. Zuo, and J.-w. Liu. Derivative failure of compressor station analysis based on Hawkes process. In Proceedings of the 2020 39th Chinese Control Conference (CCC), pages 4089-4096, 2020.
[50] Q Zhang, A. Lipani, O. Kirnap, and E. Yilmaz. Self-attentive Hawkes process. In Proceddiggs of the 37th International Conference on Machine Learning (ICML), pages 11183-11193 2020.
[51] K. Zhou, H. Zha, and L. Song. Learning social infectivity in sparse low-rank networks using multi-dimmnsional Hawkes processes. In Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics (AISTATS), pages 641-649, 2013.
[52] S. Zuo, H. Jiang, Z. Li, T. Zhao, and H. Zha. Transformer Hawkes process. In Proceedings of the 37th International Conference Machine Learning (ICML), pages 11692-1 1702. PMLR, 2020.
13


Reviews:
[{"review_id": "oMhVcrIbW8M", "reviewer": "Reviewer_qZ8d", "summary": "The authors a sparse Granger-causal learning framework for temporal event data following Hawkes process. The method utilizes the L0-regularized minorization-maximization (MM) learning framework, which remedies the issues of singularity in existing approaches.  The authors validated the proposed framework with two real use-cases, one from the power grid and the other from the cloud data center management domain.\n\nThe contributions of this paper seem to be as follows: (1) The proposed method is the ﬁrst L0-regularized MM learning framework for the Hawkes process applied to simultaneous instance- and type-level causal analysis. (2) The proposed method can explicitly perform simultaneous instance- and type-level causal analysis.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nThere are some unclear points in this paper as described below. Although the proposed method has some novelty and the experimental results demonstrate that the performances of the proposed method were better than those of baselines in type-level tasks, due to the lack of clarity, I cannot provide higher ratings in this paper.\n\nSpecific comments:\n1. L132 “When D > 1, the summation over n cannot be performed in the second term due to d_n being dependent on n. “ I cannot understand why the first term can be computed but the second term cannot be computed when D > 1 because the first term also uses d_n.\n2. Theorem 1 has no proof and may be better to describe as a definition (seems to be obvious for me).  \n3. L252 cLSTM and cMLP are not state-of-the-art Granger causal learning algorithms. For example, economic statistical recurrent unit (eSRU; Khanna & Tan, 2019, ICLR) and generalized VAR (Marcinkevics &  Vogt, 2020, ICLR) are more efficient and effective Granger causal learning algorithms. \n4. L268 GBR -> GPR?\n5. Although the authors mentioned that “Power grid failure diagnosis” is a type-level causal diagnosis task and “data center warning event diagnosis” is an instance-level task, are the first two experiments type-level tasks? If yes, the authors demonstrate the performances of the proposed method were better than those of baselines in type-level tasks, but they are unknown in an instance-level task because there is no comparison with baselines in “data center warning event diagnosis” task. In the appendix, the authors mentioned a hard-windowing approach, but just mentioning and no numerical results. The authors introduced other approaches in the related work section, thus the comparison with baselines in an instance-level task may be required.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitations were mentioned. The societal impact was not mentioned but the reason was described.", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "7 hours"}, "extra_text": {}}, {"review_id": "eRl6AnXJwxW", "reviewer": "Reviewer_YomF", "summary": "This work introduces a l_0-regularized multivariate Hawkes process model for learning sparse Granger causality from temporal event data. \nOn the one hand, the authors show that the prior related work cannot infer the true sparsity using l_1 or l_{1,2} regularizers. \nOn the other aspect, most neural point processes for learning Granger causality, assume the equi-time-input, which prevents them from analyzing continuously observed events. \nThe authors used synthetic data to compare their introduced one with neural point processes, and the counterpart model using l_1 and l_{1,2} regularizers, and demonstrate the new method can infer the true sparse impact matrix behind the events.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "-Originality\nTo me, the new contribution by introducing l_0 regularizer, seems to be still incremental although it is the first attempt to exploit the l_0 regularizer in temporal point processes, and the other type of regularizers cannot infer the true sparse Granger causal graphs.\n\n-Quality\nThe technical aspect of the paper appears to be sound. In particular, the new finding about the limitation of the l_1 and l_{1,2}-regularized Hawkes processes (HPs) in estimating true sparse impact matrices, is useful for related practitioners.\n\n-Clarity\nThe paper is well written. Specifically, the limitations of the prior l_1, l_{1,2} -regularized HPs in learning true sparse Granger causal graphs, are well explained and illustrated. \n\n-Significance\nTo me, the novel l_0-regularized HP , and the proof of the limitations of the prior HPs in learning true sparse Granger causal graphs, are useful and important. Nonetheless, the aforementioned contributions seem to be still insufficient to reach acceptance level. \n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "-The authors do discuss some limitations of the regularized Hawkes processes in inferring Granger causal graphs from temporal event data. \n- Overall, the contribution of the paper is useful but seems to be incremental. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "5 hours"}, "extra_text": {}}, {"review_id": "MEChfrGl3Y8", "reviewer": "Reviewer_NuTX", "summary": "The paper presents a L0-regularized algorithm to learn sparse Hawkes-Granger models. It is well motivated from the numerical instability issue of general MLE based approach to learn MHP, and the singularity issue of existing MM-based approach that may prohibit any sparse solution. The paper is clearly presented, and empirical results from both synthetic and real datasets show competitive performance compared to other SOTA methods in learning sparse hawkes-granger models.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper proposes an interesting L0-regularized algorithm to learn sparse Hawkes-Granger models. It is well motivated from the numerical instability issue of general MLE based approach to learn MHP, and the singularity issue of existing MM-based approach that may prohibit any sparse solution. It has a nice analysis of why the existing L1-regularization methods may fail to recover the sparse solution. Then, it builds a connection between the Minorization-Maximization (MM) and the \\epsilon-sparsity technique used for probabilistic mixture models. One additional benefit of introducing the auxiliary q_{n,i} into the optimization is that they are able to capture the instance level influence per past temporal event. Experiments on both synthetic and real datasets show that the new method is able to better recover sparse structures compared to other SOTA methods.\n\nLimits:\n\nThe root cause of the singularity issue lies in the fact that the intensity function has to be strictly positive. The base positive background intensity for a Hawkes process still guarantees this condition when no event happens at atll. The \\epsilon-sparsity technique trades off this issue by introducing the additional thresholding hyperparameter \\epsilon. For the existing SOTA methods, the recovered parameters of the mutual excitation matrix can also be thresholded using a similar thresholding hyperparameter \\epsilon which can be tuned using the dev dataset. I wonder how this can be compared to the proposed method. In addition, for the synthetic experiment, it will be more convincing to report the F1 metric by comparing the recovered sparsity structure to the groundtrue structure rather than showing qualitatively the sparse pattern in Figure 4. Finally, it will be more convincing to add event time prediction using MAE to evaluate the predictive performance of the new method against the baselines.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No potential negative societal impacts are involved in this paper.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2-2.5 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper addresses an important problem, reviewers consider the method sound. There were some concerns that the work is too incremental I would not over estimate this issue given the relevance of the problem.

Decision: Accept (Poster)