Weighted Clock Logic Point Process
Authors: Ruixuan Yan
Yunshi Wen
Debarun Bhattacharjya
Ronny Luss
Tengfei Ma
Achille Fokoue
Anak Agung Julius
Venue: ICLR 2023 poster (2023)
Keywords: Multivariate event data
Neuro-symbolic models
Temporal point process
Propositional logic

Abstract: Datasets involving multivariate event streams are prevalent in numerous applications. We present a novel framework for modeling temporal point processes called clock logic neural networks (CLNN) which learn weighted clock logic (wCL) formulas as interpretable temporal rules by which some events promote or inhibit other events. Specifically, CLNN models temporal relations between events using conditional intensity rates informed by a set of wCL formulas, which are more expressive than related prior work. Unlike conventional approaches of searching for generative rules through expensive combinatorial optimization, we design smooth activation functions for components of wCL formulas that enable a continuous relaxation of the discrete search space and efficient learning of wCL formulas using gradient-based methods. Experiments on synthetic datasets manifest our model's ability to recover the ground-truth rules and improve computational efficiency. In addition, experiments on real-world datasets show that our models perform competitively when compared with state-of-the-art models. 

Published as a conference paper at ICLR 2023
WEIGHTED CLOCK LOGIC POINT PROCESS
Ruixuan Yan', Yunshi Wen', Debarun Bhattachariya", Ronny Luss", Tengfei Ma², Achille Fokoue?, and Agung Julius'
'Ressselaer Polytechnic Institute IBM T.J. Watson Research Center
ABSTRACT
Datasets involving multivariate event streams are prevalent in numerous applica- tions. We present novel framework for modeling temporal point processes called clock logic neural networks (CLNN) which learn weighted clock logic (wCL) for- mulas interpretable temporal rules by which some events promote or inhibit other events. Specifically, CLNN models temporal relations between events using conditional intensity rates informed by a set of wCL formulas, which are more ex- pressive than related prior work. Unlike conventional approaches of searching for generative rules through expensive combinatorial optimization, we design smooth activation functions for components of wCL formulas that enable a continuous re- laxation the discrete search space and efficient learning of wCL formulas using gradient- brasee methods. Experiments ability to recover the ground-trute truth rules and improve computational efficiency. addition, experiments real- world datasets show that our models perform com- petitively when compared with mate-of-the-al models.
synthetic datasets manifest model's s
INTRODUCTION AND RELATED WORK
Multivariate event streams are emerging types of data that involve occurrences of different types of events continuous time. Event streams observed in wide range of applications, including but not limited to finance (Bacry et al. 2015), politics O BBrien 2010), system maintenance (Gunawar- dana al.) 2011), healtheare [Weiss& Page 2013), and social networks (Farajtabar et al.) (015). As opposed to time series data that typically comprises continuous-valued variables evolving in regular discrete time stamps, event streams involve events occurring irregularly and asynchronously tinuous time. Modeling the dynamics in event streams is important for wide range of scientific and industrial processes, such as predicting the occurrence of events of interest or understanding why some deleterious events occur so possibly prevent their occurrence. A (multivariate) tempo- ral point process (TPP) provides a formal mathematical framework for representing event streams, where a conditional intensity rate for each event measures its occurrence rate any time given the historical events in the stream (Daley & Vere-Jones 2003 Aalen et al. 2008).
con-
There has been a proliferation of research around TPPs in recent years, particularly around the use of neural networks for modeling conditional intensity rates as a function of historical occurrences (Du al. 2016 Mei & Eisner [0177] Xiao et al., 2017 Xu et al. 2017 Gao et 2020 Zhang al., Zuo et al. 2020). One stream of research studies graphical event models (GEMs) as a compact
2020
and interpretable graphical representation for TPPs, where the conditional intensity rate for any particular event depends only on the history of a subset of the events (Didelez, 2008: Gunawardana & Meek 2016). While any TPP can be represented as GEM, various models make assumptions about the parametric form of conditional intensity rates for the sake of learnability, for instance that rates are piece-wise constant with respect to occurrences within historical windows (Gunawardana al. 1 Bhattachariya et al. 2018). Ordinal GEMs(OGEM (Bhattacaarjya et al. [2200] 2021) are a recent model from this family where a conditional intensity rate depends on the order which parent events occur within the most recent historical time period.
A temporal logic point process (TLPP) framework was proposed as an alternate way to lend some interpretability to TPPs by modeling intensity rates using temporal logic rules (Li al., 2020). Although the initial work pre-specified temporal logic rules, recent work has introduced a temporal logic rule leamer (TELLER) for automatically discovering rules (Li et al. 2021). There is however

---

Published as a conference paper at ICLR 2023
the issue of scalability since TELLER exploits expensive branch-and- price algorithm to search for temporal logic rules in a discrete space. Another important limitation of this work is that TELLER's rules not informative enough to explain how the interval length between ordered events impacts the conditional intensity rate. For instance, while predicting the occurrence of diabetes, the rule that "insulin injection happens 20 minutes before eating meal" is more informative and accurate in predicting "blood glucose remains normal" than the rule that "insulin injection happens before eating meal", as the latter rule cannot expose the interval between 'insulin injection' and 'eating meal', To tackle the above limitations, we propose novel atomic predicates enriching the expressiveness of temporal logic rules as well as a differentiable framework learn rules in an end-to-end manner.
This work introduces a differentiable neuro-symbolic framework. clock logic neural network (CINN), to model TPPs by learning weighted clock logic (WCL. formulas as explanations. Firstly, event streams are converted into continuous-tima clock signals representing the time interval be- tween the last occurrence of an event and the current time. Next, we propose novel wCL to de- scribe the underlying temporal relations with relative interval length, enabling the design of a CLNN to learn the generative mechanisms. Instead of searching for temporal logic rules in some vast dis- crete space, CLNN associates every neuron with order representation assigns weights to edges to reflect the importance of various inputs, which relaxes the search space to be continuous. Moreover, architecture weights are introduced into CLNN to make the formula structure search differentiable. wCL. formula-informed intensity rates are carefully designed so that the parameters appearing in the rules can be learned through maximum likelihood estimation using gradient-based approaches. CLNN is tested on synthetic datasets to show that CLNN can recover the ground-truth rules as well as on real-world datasets to demonstrate its model-fitting performance.
logical operator and
2
PRELIMINARIES
2.1 NOTATION & BACKGROUND
Let L denote the set of event labels, and M denote the number of event labels. An event stream
is a sequence of events including time stamps, denoted as D= ((4 ((), (22, t2)..., (Iv, ty)]. where
4 E R+ denotes a time stamp between the beginning time to = 0 and end time 1N+1 = T, and 4 E L is the event label that happens 4,. We refer 'event label' and 'label' interchangeably. Every event label [ E L has an associated conditional intensity rate describing the occurrence rate of given the history up to In multivariate temporal point processes, conditional intensity rates describe the dynamics of events. Let Hi {(44, (4) t, < {} denote the historical events up to time t. The conditional intensity rate of event label is denoted as > (t)He)) Specifically, (t(())))) describes the expected number of occurrences of event label / in an infinitesimal interval [1,++ At] given the history He, i.e., (eet)He)) = limar-of [t + At) - ((())) where (1) denotes the number of event label I's occurrences up to L.
label
Example rumning example of an event stream with 11 events of labels is shown in Figure [(a)
CLNN
Output
Masking Junction
Cioching Junction
POC
Event
Masked event stream
Event clocks
soc
formula
Intensity
1
10
12
21
25
27
30
rate
(a)
(b)
Figure (a): An event stream example with 11 events of M 4 event labels over 30 days. (Integer-valued time stamps utilized for easy interpretation, note that the proposed approach also works for t; E R). (b): The overall workflow of the proposed method (POC: paired order cell, SOC: singleton order cell, AC: architecture cell, details presented in Section 2.2
2.2 ORDER REPRESENTATIONS FOR EVENT STREAMS
The overall workflow of the proposed framework is visualized Figure [(b) The raw event streams first go through a masking function to generate the masked event streams, which are then trans- formed into event clocks using clocking function. The event clocks are given inputs to the clock logic neural network (CLNN) to learn interpretable wCL formulas and the intensity rate of event occurrences. The following sections provide a detailed explanation for each module in Figure ((b)
We are interested in exploring the effect of temporal ordering between event labels and the occur- rences of causal event labels in historical window on the occurrence rate of particular event label,
2

---

Published as a conference paper at ICLR 2023
where the generative mechanism is expressed interpretable formulas. An event stream up to may include multiple occurrences of the same event label, thus a masking function is required to mask out duplicated event labels in the history for accessing the ordering information at any t. Here adopt a technique similar to Bhattachariya et al. (2020) for extracting distinct event labels from H4.
Definition (Masking Function) A masking function T(-) is a function that takes an event stream as input and returns new event stream that is a subset of the input stream and contains no dupli- cated event labels. Mathematically, I())) is applied to Hz = {(1,,tt) and converts it into a new stream Hi = (g) He :1 + ly if J').
We consider the following two masking functions as per Bhattacharjya et al. (2020) due to simplic- ity: 'first' masking and 'last' masking. The 'first" (resp. 'last') masking function keeps the first (resp. last) occurrence of an event label in an event stream.
Example 1 (cont.) Let H1s {(A, 1), 3), (A, 6), (D, (),,),C. (D, 12)). The "first' mask- ing function converts it to Hi = (A,11), (B,3). (D, 8). (0), and the 'last' masking function converts it to His {B, 3), (A, 6), 10,
With the masked event history Hy, we define two order representations for the order relationship between any two event labels and the occurrence of an event within a historical window t. t.
Definition 2 (Paired Order Representation (POR) A paired order representation is defined as [4 E [4] where [4] denoles two-element permulation of a subset of C. A paired order repre- sentation for Hi, can be obtained by arranging any two distinct labels in Hi in a sequential order.
Definition 3 (Singleton Order Representation (SOR)) A singleton order representation is de- noted as [, u1,] E L x R_, representing event label I E L occurred within the past vi, time units, where Wi is variable to learn through a process that will be explained in Section 3.3
Example (cont.) With first masking, an example of paired order representation for Hi's can be [A, B] representing "A happens before B" or [B, C*) representing "B happens before C". The overall order representation for Hig is expressed as [A, B, D, C}. which can be derived from the paired order representations: [A, B), [B, Dj. [D. C). A singleton order representation example of His can be expressed as [B, 10.5), meaning B happened in the past 10.5 days.
2.3 WEIGHTED CLOCK LOGIC FORMULA
To adapt HC to continuous- signals from HC to describe the time passed since the last occurrence of a label. A clocking function is introduced to convert łÿ into a clock signal Cj denoting the time interval length between ij and t. fincting,
signals that can be described by logical statements, we extract clock
Definition 4 (Clocking Function) A clocking function E(-) converts Hi into a vector of clock sig-
nals as (t) = [ea c2(),..... eas (t)]² E R4 with (t) denoting the clock signal for event label i E L. where (t) is computed as c, = t-t, if (lg,tt) tz) E Hi, and 11 i, and ct(t) = Z otherwise. Note that is a user-defind,, large positive number to indicate event label i not happening in Hi-
Example 1 (cont.) Taking the 'first' masked event stream Hig = ((A, 1), (B, (D, (C, as an example, the event clocks extracted C3(13) [1, 10, 1,
The event clocks can essentially provide the ordering between any two event labels in that the differ- ence between any two event labels' clock signals reflects which event label happens first. As shown in the diabetes prediction example in the Introduction section, the time interval between ordering events notably important in explaining and predicting an event label's occurrence. contrast to (Li et al. 2020 202I) which only learns the temporal ordering relation between event labels, define a paired order predicate (POP) with a learnable parameter U4,, to describe the time interval between two ordered event labels I and 1j and a singleton order predicate (SOP) with a learnable parameter U1 to describe the occurrence of label ly within a historical window Ut follows.
Definition 5 (Paired Order Predicate) A POP describes the order between two labels 4114, Lils C, 44 + li, denoted Tpop ;= (iy) =c, Cty > Wa, where W1, E IR is parameter to learn. A positive 1 means 1; huppened before for at least 21 time units, and a negative 11 means 15 happened before 1; for at most -W1,, time units. A POP is used in the POC of Figure I(b)
Definition 6 (Singleton Order Predicate) An SOP describes a causal label 1 E L occurring within the past ue time units, defined as Tsop == Ci M1, < O, where M1 E RR is a learnable parameter.
3

---

Published as a conference paper at ICLR 2023
Instead of taking heuristic approach for some underlying combinatorial search problem for a given
set of temporal predicates (Bhattachariya et al. 2020 2021 Li et al. 2021) to uncover the effective
order relations, this work proposes a differentiable learning model to learn suitable singleton and
paired order predicates among all the possible choices of order predicates through a gradient-based approach. The scheme of weighted signal temporal logic (wSTL.) in Yan et al. (2021) (2022) is
exploited to build weighted clock logic (wCL) formulas that are logical compositions of singleton and paired order predicates. The syntax of wCL is recursively defined (Mehdipour et al. 2021):
= pop T2o 1 Φ1 A 62 ok Φ1 V 2 VV V
where .1,-.. @& are wCL formulas, - denotes negation, A denotes logical conjunction, V denotes logical disjunction, w;; 2 0,j 1.... k denotes non-negative weights assigned to $1.-.. , in the conjunction and disjunction operations. A wCL formula can describe the characteristies of Hi, thus the conditional intensity rate of event ( given H. can be equivalently denoted (t))().
Remark 7
The syntax above means each W formula can be built by using predicates in Tppp or Toop and then by recursively applying the or the A the V operations.
Example (cont.) A wCL J.Iffullla example is & (CA (CA-CB cB > 1)² A (ec < <3)0.05, The first and second clauses read " A happened before B for at least one day" and "C happened less than 3 days ago", respectively. Note that & is satisfied by the event stream up to = 13 in Figure The two clauses have weights of 1 and 0.05, reflecting the first clause is more important than the second one.
WEIGHTED CLOCK LOGIC POINT PROCESSES
3.1 TRUTH DEGREE OF WEIGHTED CLOCK LOGIC
To quantitatively measure the satisfaction degree of a wCL formula e over the event clocks (t()), i.e., how well does ø describe the underlying patterns of C'(t), we propose smooth activation functions (AFs) to compute the truth degree, denoted p(C', o, t) [0, 1], defined as (Riegel et al. 2020):
pCC', J.4 pop- t) sigmoid( Ct, (t) ct (t) U1,2)),
(2)
P(C', ypp 1) sigmoid(4, - a (4))
(3)
p(C', -(,)) = 11-1ppp 1-p((C, e,t).
(4)
In contrast to the combinatorial search of the temporal logic predicates in Li al. 22021 the smooth design of AFs in (2) 4 benefits the maximum likelihood estimation problem shown later in Section 3.6 by allowing it learn the parameters in the POP and SOP through gradient- basee methods. Next, we present the design of activation functions (AF) for the A operator. Here we use a 2-ary conjunction operator to motivate the design. Let / = p(C', 21 A 22 1) [0, 1]. Intuitively, /a is low when either input is low, and pd is high when both inputs are high. Here we adopt similar idea to Sen et al. 2002 for capturing the low and high. A user-defined hyperparameter a e [ is introduced to aid the interpretability of low and high such that /a represents high if pd E [a, 1] and low if p^ E [0, 1 - a). Considering the importance weights, a low input with a zero weight should not impact the output, which implies pd should be low when both inputs low. With these considerations, the AF for the A operator is defined as follows: (See Appendix A for more details.)
,CC', 11 02
= =f(3
Σ
w,(1 plC', og ())),
(5)
subject B Σ w - a) > a Σ wja 51-6 aa,
where f(x) max (0, min 1}} clamps the truth degree into [0, 1], wy 2 0 and 3 are
parameters to learn. By De Morgan's ' law (Hurley 2014), the AF for the V operator is defined as
p(C', Φ1 V V V $ = =f(1 B + Σ my (,,(),,
(6)
subject to
1 1- 3+ Σ
wia 2 a, 3+ Σ
wy (1 a) S a.
1=1
4

---

Published as a conference paper at ICLR 2023
An event stream with M event labels would generate P (M-22) paired order predicates and M singleton order predicates. If conjunction or disjunction operator takes these predicates as inputs, how it recognizes the effective order predicates in describing the event dynamics becomes a critical issue. By carefully designing the AFs in (5) ( the logical operators exhibit the following properties so as to recognize effective inputs. This is a critical advantage over Bhattacharjya et al. 202T);
M!
(202T) in that it allows a differentiable search of the suitable predicates among all the possible choices of order predicates in an end-to-end manner. Here we illustrate the properties for A with two inputs, which can be generalized to k-ary inputs. (See Appendix B for more details.)
Theorem 8 The AF for the A operator with two inputs exhibits the following properties.
1) Nonimpact for zero weights: If = 0.. = 1,2, p(C', 2,,t) has no impact on p(C", 01 12,
2) Impact ordering: If p(C', 11, - p(C', 22 t). and wi 2 W2, then Opic' "oi''.nnn00t 0p(C' 0p(C'.01.0)
0p(C' aa At: Op(C' 02.1)
2
3) Monotonicity: 15 p(C', 03 t))) fBB ΣE- wy - (p(C', 0z + d))), d20.
Architecture
Ang
(t)
pCe' 4b.1)
p .tt)
Stagleton order
oelection rodd
logical
alov
Arthhtecture
Mpsp
Painod order weight
Singleton-order weight
PoP selection
node
SOP solection
A
rode
A.B pop
R.C rI pop
Weop
pags
pap
Pop
pap
Hoop
C (t)
CB (t)
cc(t)
CA
cc(E)
ca(t)
(a)
(b)
Figure 2: CLNN Structure. (a): Continuous relaxation of the search space using weights. (b): The learned discrete model structure for o = (Trppp A TI T V (T
3.2 LEARNING OF PAIRED ORDER REPRESENTATION
With the smooth AFs designed in 2 (6), neuro symbolic model called clock logic neural network (CLNN) can be designed for any given WCL formula é in which every neuron has corresponding symbolic representation. A typical CLNN for o (TAB A B=BBC) (T is visualized as Fig. 2(b) which can be considered as the discrete structure obtained by learning the parameters of the model in Figure 2(a) and keeping the dominant components. Here 6 can be interpreted before B for at least MBB time units or B happens before C' for at least "Bc time units) and A happens within the past A time units." This part describes the continuous relaxation of the search space by designing a paired order cell, a singleton order cell, and an architecture cell for learning the paired order representation, singleton order representation and the formula structure.
"(A happens
Paired Order Cell (POC). A POC is a directed acyclic graph (DAG) comprising two paired order predicate (POP) nodes and one logical node for the operator, shown as orange block in Figure hud,
li.t 2(a) The two POP nodes represent Tpop and (id. sharing the same parameter W..,, where Thop
denotes "I, happened before I, for at least 41,, time units" and dydd denotes "I, happened before 14,ly 1g.ds 4 for at least Wr,4, time units". Each POP has an associated weight Wpop or Wpop to be learned, and the A operator forces one of the two weight parameters to dominate the other one such that the learned POR is consistent with the event stream. For example, the POC in Figure 2(a) aims to learn the POR between A and B, whose discretized version would be either Topp or Too
*A.B
B.A
An event stream with M event labels can generate P2 M!
PORs between any two event (M-
5

---

Published as a conference paper at ICLR 2023
labels, resulting in (P POCs. Similar to learning the POR between any two events, the discrete order representations for the entire history Hy can be learned using a POP selection node (as shown in Figure 2 that takes the outputs of all the POCs as input and identifies the important PORs. The learning of the POCs essentially becomes learning the w. 3 in (5) for the POCs and the POP selection node, well W1,, in (2) for the POPs through back propagation. The discrete PORs can be acquired by keeping the top-k strongest POCs and the dominant POPs.
3.3 LEARNING OF SINGLETON ORDER REPRESENTATION
Singleton Order Cell (SOC). The learning of SOR is accomplished by an SOC, which is displayed as a green block in Figure [ An SOC is a DAG comprising M singleton order predicate (SOP) nodes and one SOP selection node for the A operator. An SOP node represents Soop that takes 2g (1) as input and returns the truth degree of Toop over 01 (1). The SOP selection node has the same functionality as the POP selection node. The A operator in the SOP selection node assigns a nonnegative weight to every SOP node and learns the importance weights w and 3 to extract the dominant SORs affecting the conditional intensity rate the most. The learning of the SOC is thus learning the w, 39 in (5) for the SOP selection node and 4r in (3) for the SOPs through back propagation. The discrete SORs can be determined by keeping the top-k strongest SOPs.
3.4 LEARNING OF FORMULA STRUCTURE
Architecture Cell (AC). For a given set of PORs or SORs, their conjunction or disjunction will behave differently and have distinct meanings. For instance, given two causal formulas 6 = (CA -C CB > 1)² A [cc < 5)² and Φ22 (CA CB > VCC < 5) for the occurrence of event label D, Φ1 means "(A happens before B for least 1 time unit) and (C happens within the past time units) simultaneously will cause D to happen", whereas @ means "(A happens before B for at least 1 time unit) or (G happens within the past 5 time units) alternatively will cause to happen." The afore-mentioned cells can learn the order representations. Nevertheless, whether their outputs should be connected by the A or V operator needs to be determined. Here we consider the outputs of the POCs and the SOCs having two choices of being connected by A or V operator, each of which is associated with an architecture weight a or aare that enables continuous learning of the two choices; this is also called differentiable architecture search (Liu et al. 2019). An architecture cell introduced for learning the model architecture, which comprises two logical nodes representing a operator and V operator well logical selection node (LSN), shown as the blue block in Figure ((a) Let = (PI,..., } denote the set of inputs for each logical operator. Subsequently, the conjunction operator takes p as input and returns p/ f(8r - 2j=1 wef (1 11-20 P;)), and the dis- input and returns p" f(1 BY + ش p,), The LSN represented
junction operator takes by 2 takes 'd and pK as inputs and returns their weighted sum, where the weights are computed using the softmax of the architecture weights as shown below: efare
Pg xCC', 0,t)
Σ
mE(A.) m''EAAV
The task of architecture search then reduces to learning the architecture weights a a and the w, 3 in (5) (6) for the two logical operators, which can be executed simultaneously while learning parameters in the POCs and SOCs. The outcome of the architecture search process is discrete architecture obtained by retaining the logical operator with the strongest architecture weight.
3.5 WCL-INFORMED INTENSITY FUNCTION
Forward propagation
L1
Backward
The output of a CLNN is the truth degree of o over C' which is incorporated into modeling the conditional intensity rates. The modeling process aims discover the generative mechanism as WCL formulas for every 1 E C. In other words, larger value of p(C', Φ. t) should reflect that & has a greater impact on the occurrence of a particular label. For example, if the wCL for- mula for affecting the occurrence of event label D given as == (( A (=Epp) )²², it means if á is satisfied or the truth degree of Φ is high, then it has a strong impact on the occurrence of D. where the impact can be promoting or inhibiting the occurrence of D. In terms of the relation between the truth degree and the con- ditional intensity rate, the higher the truth degree p(C', , Φ, l), the greater its impact ApI Note
propagation
Ano
PCC' 41.0)
PCC' 4.,,
...
CLNN-4+
CLNN-
C':Clock signal for
Figure
The overall learning framework for n wCL formulas.

---

Published as a conference paper at ICLR 2023
that the occurrence of one event label may depend multiple wCL formulas. This work follows the assumption that the impact of multiple formulas are additive in predicting the intensity rate, similar to Li et al. (2020). To incorporate set of wCL formulas 1 f 22 .... Qn into the modeling of the conditional intensity rate, we define a WCL formula-informed conditional intensity rate as:
Anol (t) = exp( Σ
Wl,p(C', , 0i t) p),
where Wo; is the weight of Φi, and is a bias term that allows for spontaneous occurrence without the influence from Φ.
3.6 MAXIMUM LIKELIHOOD ESTIMATION
Suppose event stream D contains nu occurrences of event L, for which the occurrence time stamps denoted as f,, f12....,,, Mt..- Let to 0, Mlatt T. Based on the conditional intensity function
in 8 the likelihood for label I over the event stream is calculated (Daley & Vere-Jones 2003):
(-/M/tt
(o) A exp (-frn (
L
II 2=0
exp
(9)
The corresponding log-likelihood for event label I is expressed as LLI = (- (s +
Ei b (Aog (tt, ))]. The total log-likelihood of all the events in D is thus LLD = Eiec LLnn
During the training process, we train the model parameters for each event label separately, Specifi- cally, the maximum likelihood estimation problem for event label I can be formulated as follows:
min LLL
(10)
s.t.
vΦ E 4 v1 < k
KA 1I
Σ ieLu
Winh a) 0.
Σ Wiia 1 -a, a, iel
(11)
VΦ
E 1 < /' < ky 1 Bk +
Wi, 2 ax
BB
Wipe (1 -0) 5 -0, (12)
tEly
iEIy
1i. 2 0, 3 2 0, w 2 0, 3r 2 0, W1, 2 0,
where K (resp. x is the number of A (resp. V) operators in Φ. Is (resp. 1g-) denotes the inputs to the k-th A (resp. w'th V) operator. Please see Appendix A for more details about the above formulation. The overall learning framework is shown in Figure in which the forward propagation computes LL, by using n CLNNs; each learns a wCL formula Φ; and the backward propagation updates the parameters CLNNs using projected gradient descent.
EXPERIMENTS
We conduct several experiments on synthetic and real-world datasets to demonstrate the efficacy of our proposed model. Simultaneously, we compare with state-of-the-art (SOTA) models. The exper- iments are run using the AdamW optimizer in Pytorch (1.10.2) on Windows 10 system desktop with 16-core CPU (i7. 3.60GHz) and 32 GB RAM. Our code is available Https://ICLRCCLM,
4.1 MODELS
Multivariate Hawkes Process (MHP) [Bacrr et al., 2017 ]: conventional multivariate Hawkes process utilizing exponential kernel function to describe the conditional intensity rate, which involves a decay rate and an infectivity matrix characterizing the inter-dependence among events. This model implemented in the rick) library, where the learning problem is posed as convex quadratic programming problem with fixed decay rate.
Proximal Graphical Event Model (PGEM) ((Bhattachariya et al., 2018 A type of GEM that models event data by considering whether a parent in some underlying graph happens in proximal (recent) window.

---

Published as a conference paper at ICLR 2023
Ground vuá
CLNN's role
(eA cp 1,221) A (ex
8.00) 1:41 oxp 0.82)
4.387" 10.60/ (cp
TILLER's rule
before D before before D. before D and C before D
OGEM jab's
Excitunio: [C. (C. 12, Cj Intibitor:: [4), (C,A), 14.C]
Table 1: Comparison of rule discovery for CLNN, TELLER, and OGEM-tab on the Syn-1 dataset.
Ordinal Graphical Event Model (OGEM) [(Bhattacharjya et al. 2020: 2021D]: An ordinal GEM that models the impact of the order of events on the conditional intensity rate. OGEM-tab (resp. OGEM-tree) refers to an OGEM that adopts a tabular (resp. tree) representation of orders.
Temporal Logic Rule Learner (TELLER [[ et al. 2021). This is method to learn first-order temporal logic rules explaining the generative mechanism of TPPs. The rule discovery process is formulated as maximum likelihood estimation problem solved by branch-and- algorithm.
4.2 SYNTHETIC DATASETS
The first part of this experiment demonstrates CLNN's capability of recovering ground-truth rules using three synthetic datasets generated by CLNN with pre-specified formula structure and param- eters, including _, in Tpop as well as the importance weights w and bias / in (5) for logical operators, and the Wy and in (8) for the conditional intensity rate.
Experimental Setting. Each synthetic dataset contains 1, 000 event streams partitioned into three sets: training (70%), validation (15%), and test (15%). Every dataset is generated using a wCL formula with wg = 3 and p -5. The truth value threshold is set for representing an event not occurring in Hi is set as 2 = 1.57 where Taax is the maximal ending time among all the event streams. During the training process, we initialize the parameters using four approaches (see Appendix C.5 for more details) and report the best one, and CLNN aims to recover the manually set parameters.
0.5, and the clock signal
Results. The ground-truth rule &1 for generating the first synthetic dataset (Syn-1) with L = {A, B,C. and the rules discovered by CLNN, TELLER, and OGEM-tab are summarized in Table Results for the other synthetic datasets are presented in Appendix C The rules are learned using the 'last' masking method, which was also used for data generation. The experimental results show an accurate recovery performance of CLNN in terms of order representation recovery and pa- rameter identification. The unweighted version of the ground truth rule reads: "If A happens before B for at least 1 time unit and A happens before C' for at least 3 time units, then D will happen". The rule of TELLER only reflects the temporal relation between events A, B,C and 1) but unable to capture the temporal relation between A and B or A and C, which does not match the ground-truth rule. In OGEM-tab's s rule, [4] denotes a single parent. We show the top 3 excitation and inhibitory rules from OGEM-tab, where excitation (resp. inhibitory) means À is higher (resp. lower) than the λnj with all Ve = O. The excitation rules of OGEM-tab do not match the ground ground-truth rule. In contrast, the rule discovered by CLNN (Φ1) assigns larger weights to the paired order predicates A.B = -(CA-or cB > 1.21) and pop A.C = (cA -cc >> 3.00) and small weights to the other predicates, Tpop
where the interval values of 1.21 and 3.00 are both learned. By ignoring the small weights, Φi can be interpreted as "If A happens before B for at least 1.21 time units and A happens before C for at least 3.00 time units, then D will happen", meaning the paired order representations discovered by CLNN match well with the ground truth. Moreover, CLNN's rules are more expressive than TELLER and OGEM as provides a detailed interval length between two ordered labels.
To show the computational efficiency of our gradient-based learning, we compare the runtimes of CLNN and TELLER on the synthetic datasets in Table Notably, CLNN not only recovers the cor- rect order representations but also was two orders of magnitude faster on average (5.62 s vs 635.99 s). In addition, CLNN can learn more expressive order representations that describe both the order relation between two events and their interval length.
WCL formula
di
01,1
61.
Average
CLNN TELLER
5.20
4.60
4.95
7.73
5.62
252,91
286.83
925.58
1078.66
635.99
Table 2: Runtime (s) for CLNN and TELLER on synthctic datascts.
4.3 REAL-WORLD DATASETS
Linkedln [(Xu et al. 2017)]. An event dataset related to job hopping records of 3, 000 LinkedIn users in 82 iT companies. Each event stream records a user's check-in time stamps for different companies or the time stamps for role change within the same company. We filter the dataset to popular companies as per Bhattachaari et al. (2020), resulting in 1, 000 users.
http://ggthaaaamm////ggggggggunn Point Processes ICLR
8

---

Published as a conference paper at ICLR 2023
Mimic II [(Saeed al., 2011] An event dataset concerning health records of patients from Inten- sive Care Unit (ICU) visits over 7 years. A patient's 's event stream records each visit's time stamp and the corresponding diagnosis. We filter out sequences with few visits, resulting in 650 patients.
Stack Overflow [(Grant & Betts 2013)]. An event dataset that is related to the badges awarded to website, the Stack Overflow. Each user's event stream records the badges that he/she receives at various time stamps. We keep the event streams with one or more of 20 types of badges and sample 1, 000 users from the dataset used in Du al. (2016).
users in the question-answernen
Experimental Setup. Each dataset is partitioned into three sets: training (70%), validation (15%), and test (15%). For simplicity, 2, set O to study the ordering representations. 'he truth value threshold is a = 0.5, and = 1.57maxe same as the setting for the synthetic datasets, and the number of subformulas is = 5, and the parameters are initialized as random numbers from uniform distribution on [0, 1). CLNN is trained on the training set, and the validation set is utilized for model selection during training. Model fit is evaluated using log-likelihood on the test set.
2021 to use the log- Results. We follow a similar trend to Bhattachariya et al. (2018) 2020
likelihood for evaluation of the model's performance. The log- ligliikllhoodd on the real-world datasets is reported in Table 3 where DR denotes the difference ratio the difference between CLNN and the best SOTA divided by the absolute value of best SOTA. CLNN's result chosen as the better one among the 'first' or the 'last' masking. Notably, CLNN outperforms the baseline models on the Linkedln dataset (13.40% advantage) and achieves a competitive result on the MIMIC dataset (1.63% loss only). It is observed that PGEM achieves better result on the Stack Overflow dataset. In Stack Overflow, one type of badge can be awarded only when a user receives a particular badge multiple times, for example, the 'Epic' badge is awarded only when carning 200 daily reputations 50 times, depending on the 'Mortarboard' badge acquired while answering or asking questions. CLNN and OGEMs apply masking methods the data, which may not capture the above dependence. In contrast, PGEM models data without masking, making it more suitable for this dataset.
Datasel
N (A events)
M (labels)
MIIP
PGEM
uGEM
OGEM free
TELLER
CLNN
DR
LinkedIn
2932
10
-1593
-1462
-1478
.1418
-1548
-1228
13.40%
MIMIC
2419
15
-567
-500
-474
-429
-645
-436
-1.6%%
Stack Overflow
71254
20
-52543
48323
.49344
-49192
-71101
-50981
-5.50%
Table 3: Dataset information and log-likelihood for all models on the real-world datasets.
Case Study, The primary strength of CLNN over the SOTA models is that it can describe the generative mechanism as wCL formulas. being more expres- sive and potentially providing more de- tailed information. CLNN can be de- ployed as a valuable tool for assisting domain specialists in knowledge discov- cry from event data, Here we showcase the above strength of CLNN using an il- lustrative example. We select the experimental result on company F of the LinkedIn dataset to demonstrate the expressivity of CLNN's rules, which are shown in Table Here we specify the model to learn five formulas, four of which are inhibitory, and one exhibits excitation. One inhibitory formula has a weight of 0.05, thus not reported in Table Each formula shows the dominant single- ton or paired order predicates. Notably, CLNN learns expressive wCL formulas that describe how the logical composition of paired order predicates and(or) singleton order predicates affect role change in the company F. CLNN's rules are more expressive than TELLER and as expressive as OGEM-tab for describing the occurrence of causal event within a specific historical window.
Rules
Effect
(cp Cst 0.000 ((r cy)0.72
Inhibitory
d9 ((en 0.45 100e (ep < 0.055p00
Excitation
CLNN
0r (ca > cr 1:30 A (c. "
Inhibitory
Gg (Cs < 0.84 A (Cy LOBj
Inhibitory
TELLER
FI. [C.
FI. FI. [D.
Excitation
OGEM-tab
[F.A] Al
Excitation
[A]
Inhibitory
Table 4: Formulas and their effect as learned by CLNN, TELLER and OGEM-tab on company F of LinkedIn.
5 CONCLUSION
In this paper, proposed a novel neuro-symbolic model, CLNN, to learn interpretable wCL for- mulas from multivariate event data. Experimental results using synthetic and real-world datasets demonstrate CLNN's expressiveness in recovering ground-truth rules in multivariate temporal point processes. Further, CLNN can be trained using gradient- based methods, which improve the learning speed compared to the SOTA.

---

Published as a conference paper at ICLR 2023
6 ACKNOWLEDGEMENT
This research is sponsored by the Rensselaer-IBM AI Research Collaboration http://aire.rpi.ddu part the IBM AI Horizons Network; the National Science Foundation under Grant CMMI- 1936578; and the Defense Advanced Research Projects Agency (DARPA) through Cooperative Agreement D20AC00004 awarded by the U.S. Department of the Interior (DOI), Interior Business Center. The content of the information does not necessarily reflect the position or the policy of the Government, and no official endorsement should inferred.
REFERENCES
Odd Aalen, Ornulf Borgan, and Hakon Gjessing. Survival and Event History Analysis: A Process Point of View. Springer Science Business Media, 2008.
Emmanuel Bacry, lacopo Mastromatteo, and Jean- Françorançois Muzy. Hawkes processes in finance. Market Microstructure and Liquidity, 1(01):1550005, 2015.
Emmanuel Bacry, Martin Bompaire, Philip Deegan, Stéphane Gaiffas, and Søren V Poulsen. tick: A Python library for statistical learning, with an emphasis on Hawkes processes and time-dependent models. The Journal of Machine Learning Research, 18(1):9337-7941, 2017.
Debarun Bhattachariya, Dharmashankar Subramanian, and Tian Gao. Proximal graphical event models. Advances in Neural Information Processing Systems 31:
2018.
Debarun Bhattacharjya, Tian Gao, and Dharmashankar Subramanian. Order-dependent event mod- els for agent interactions. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp. 1977-1983, 2020.
Debarun Bhattachariya, Tian Gao, and Dharmashankar Subramanian. Ordinal historical dependence in graphical event models with tree representations. In Proceedings of the Conference on Artificial Intelligence (AAAI), pp. 6759-6767, 2021.
Yuanda Chen. Thinning algorithms for simulating point processes. Florida State University, Talla- hassee, FL, 2016.
Daryl Daley and David Vere-Jones. An Introduction to the Theory of Point Processes, Volume I: Elementary Theory and Methods. Springer, 2003.
Vanessa Didelez. Graphical models for marked point processes based on local independence. Jour- nal of the Royal Statistical Society: Series B (Statistical Methodology), 70(1):245-264, 2008.
Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguer and Song- Recurrent marked temporal point processes; embedding event history to vector. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1555-1564, 2016.
Mehrdad Farajtabar, Yichen Wang, Manuel Gomez Rodriguez, Shuang Li, Hongyuan Zha, and Le Song. COEVOLVE: A joint point process model for information diffusion and network co- evolution. In Advances in Neural Information Processing Systems (NeurIPS), volume 28, pp. 1954-1962. 2015.
Tian Gao, Dharmashankar Subramanian, Karthikeyan Shanmugam, Debarun Bhattachariya, and Nicholas Mattei. A multi-channel neural graphical event model with negative evidence. In Pro- ceedings of the Conference on Artificial Intelligence (AAAI), pp. 3946-3953, 2020.
Scott Grant and Buddy Betts. Encouraging user behaviour with achievements: An empirical study. In Proceedings of the 10th Working Conference on Mining Software Repositories, MSR '1, pp- 65-68. IEEE Press, 2013.
Asela Gunawardana and Chris Meek. Universal models of multivariate temporal point processes. In Artificial Intelligence and Statistics, pp. 556-563. PMLR, 2016.
10

---

Published as a conference paper at ICLR 2023
Asela Gunawardana, Christopher Meek, and Puyang Xu. A model for temporal dependencies in event streams. Advances Neural Information Processing Systems (NeurIPS), 24, 2011.
Patrick J Hurley. A Concise Introduction to Logic. Cengage Learning. 2014.
Shuang Li. Lu Wang, Ruizhi Zhang, Xiaofu Chang, Xuqin Liu, Yao Xie, Yuan Qi, and Le Song. Temporal logie point processes. In International Conference on Machine Learning, pp. 5990- 6000. PMLR, 2020.
Shuang Li, Mingquan Feng, Lu Wang, Ahdelmajid Essofi, Yufeng Cao, Junchi Yan, and Le Song- Explaining point processes by learning interpretable temporal logic rules. In International Con- ference Learning Representations, 2021.
Hanxiao Liu, Karen Simonyan, and Yiming Yang. DARTS: Differentiable architecture search. In International Conference on Learning Representations, 2019.
Noushin Mehdipour, Cristian-loan Vasile, and Calin Belta. Specifying user preferences using
weighted signal temporal logic. IEEE Control Systems Letters, 566):2006-2011, 2021.
Hongyuan Mei and Jason Eisner. The neural Hawkes process: A neurally self-modulating multi- variate point process. Advances in Neural Information Processing Systems (NeurIPS), 30:6757- 6767, 2017.
Sean O''Brren. Crisis early warning and decision support: Contemporary approaches and thoughts future research. International Studies Review, 12(1):87-104, 2010.
2010.
Ryan Riegel, Alexander Gray, Francois Luus, Naweed Khan, Ndivhuwo Makondo, Ismail Yunus Akhalwaya, Haifeng Qian, Ronald Fagin, Francisco Barahona, Udit Sharma, et al. Logical neural networks. arXiv preprint arXiv:2006. 13/55, 2020.
Mohammed Saeed, Mauricio Villarroel, Andrew T Reisner, Gari Clifford, Li-We Lehman, George Moody, Thomas Heldt, Tin H Kyaw, Benjamin Moody, and Roger G Mark. Multiparameteer in- telligent monitoring in intensive care (MIMIC-II): public-access intensive care unit database. Critical Care Medicine, 39(5):952, 2011.
Prithviraj Sen, Breno WSR de Carvalho, Ryan Riegel, and Alexander Gray. Neuro-symbolic in- ductive logic programming with logical neural networks. Proceedings of the Conference on Artificial Intelligence (AAAI), volume 36, pp. 8212-8219, 2022.
Jeremy C. Weiss and David Page. Forest-based point process for event prediction from electronic health records. In Machine Learning and Knowledge Discovery in Databases, pp. 547-562, 2013.
Shuai Xiao, Junchi Yan, Xiaokang Yang, Hongyuan Zha, and Stephen Chu. Modeling the intensity function of point process via recurrent neural networks. In Proceedings of the Conference on Artificial Intelligence (AAAI), volume 31, pp. 1597-1603, 2017.
Hongteng Xu, Dixin Luo, and Hongyuan Zha. Learning Hawkes processes from short doubly- censored event sequences. International Conference on Machine Learning, pp. 3831-3840.
PMLR, 2017.
Ruixuan Yan, Agung Julius, Maria Chang, Achille Fokoue, Tengfei Ma, and Rosario Uceda-Sosa. STONE: Signal temporal logic neural network for time series classification. In 202/ International Conference on Data Mining Workshops (ICDMW), pp. 778-787. IEEE, 2021.
Ruixuan Yan. Tengfei Ma. Achille Fokoue, Maria Chang, and Agung Julius. Neuro-symbolic mod- cls for interpretable time series classification using temporal logic description. In 2022 IEEE In- ternational Conference on Data Mining (ICDM), pp. 618-627 0277,, 2022. doi: 10. 10910//44444444 2022.00072.
Qiang Zhang, Aldo Lipani, Omer Kirnap, and Emine Yilmaz. Self-attentive Hawkes process. In International Conference on Machine Learning, pp. 11183-11193. PMLR, 2020.
Simiao Zuo Haoming Jiang, Zichong Li. Tuo Zhao, and Hongyuan Zha. Transformer Hawkes process. In International Conference Machine Learning, pp. 11692-11702. PMLR, 2020.

---

Published as a conference paper at ICLR 2023
A FORMULATION OF LOGICAL CONSTRAINTS & OBJECTIVE FUNCTION
The optimization problem in TTPp is formulated by maximizing the log-likelihood subject to the logical constraints for the A and operators. This section discusses the details of the formulation for the two logical constraints and how to formulate the optimization problem while considering the logical constraints. Without loss of generality, we illustrate the formulation of the constraints for the A operator, and the constraints for V operator can be derived from the constraints for the A operator using De Morgan's 's law.
Logical constraints for A operator.
Let æ, y E [0, 1] denote the inputs the A operator, and f(x, y) denote the quantitative satisfaction of A. The conventional characteristic of the A operator is illustrated as follows: 1) [(x,, y) is low when either input is low, and 2) f(x, y) is high when both inputs are high. However, associate each input with nonnegative weight, implying the input with zero weight should not affect the output. In other words, if low imput has zero weight, should not affect the output of (x. y). Therefore,
require the A operator exhibit the following characteristics: 1) fxx y) is low when both inputs are low, and 2) f'x, y) is high when both inputs are high. Here we introduce a user-defined hyperparameter
E [ 1] to capture low vs. high: x E [0, a) represents low and x E [ax, 1] represents high. According the above characteristics, we have (Sen et al. 2022
f(z, v) S a, VI, 00, 1.1--a). a),
(13)
f(x, 1) a, Vx, [a,
Here we follow a specific choice of f by using triangular norm (t-norm) and define the quantitative satisfaction function of as (Riegel et al. 2020)
p(C', .01 A 2012
f(3
Σ
wo(1 O(C',o,,0)
(14)
/=1
2
subject to B -
Σ
W (1 a) 2 0
Σ
Wja S aa
(15)
j=1
where f(2) max min(2, 1}} is introduced to clamp the truth value into the range of [0, 1].
Logical constraints for V operator.
By using De Morgan's 's law, we could derive the quantitative satisfaction function and the logical constraints for the V operator with 2 inputs as follows:
pCC', V 02 202
= B
+
w/ ( dj,{)))
(16)
j=l
2 waa 2 a, 1 1 1-11-- Σ w (1 a) $ a. (17)
2
subject to
1 3+
5=1
j=1
Here show the characteristics of the activation functions for the A and V operators using Figure (ag shows the truth value of the A operator with a = 0.7. Figure shows the truth value of the A operator with a 0.9. It can be distinetly observed that f(x, y) is close O when both and y are low, and f far y) is close to when both & and are high. In addition, the unconstrained region for a 0.9 is larger than the unconstrained region for a 0.7. Figure [howssowws the truth value of the V operator with a = 0.7. It is obvious that f(x, 3) is close to O when both a and y are low, and fax y) close to I when both and y are high.
In general, we could extend the quantitative satisfaction for the A and V operators in 14 to k-ary conjunction and k-ary disjunction. The k-ary conjunction formulation expressed as follows.
CCC,, 00 2z
= =f Σ wy (1 ((',,,,))));
(18)
12

---

Published as a conference paper at ICLR 2023
0.7
0.7
07
0.0
044
0.4
0.4
0.7
0.7
0.4
0.4
0.7
0
0.4
0.4
G.7
(a)
(b)
(c)
Figure 4: Plot of truth degree for (a) CLNN-A with 0.7, (b) CLNN-A with a = 0.9, (c) CLNN-
with a 0.7.
subject
B
wy 0) 2 a Σ wja S 1
(19)
The k-ary disjunction formulation is expressed as follows.
k
p(C', 61 V 00
ox .t)
=f - 3
(20)
j=1
subject to
1 B
wiaa 2 a, 1 B Σ
a) a. (21)
With the above constraints, we can formulate the maximum likelihood estimation problem as
min
LL4
(22)
s.t. Vo 4, 41 < k < KA 1. Bx
Σ
Wi a) 2 a, 3x
Σ
wiia I a,
(23)
1ET
ieh
VΦ 4 Y1 < K < KY 1
3k +
a a,
3r Σ
Win a) S 1 a.
iElw
ieW
(24)
In this paper, we set a = 0.5, thus the constraints in 19
become
W 2 28 1.
Wy S 28 1,
(25)
23 1 1
W/ 2
Reformulating the above constraints, have
Σ w: 2,3 1,
(26)
2 0.5,
w; 2 0.
(27)
The above constraints hold for each conjunction operator in ø. Therefore, we can incorporate the constraints in 26 into the objective function, which becomes
K
min
LLI
WiAh 23g 1)2
(28)
13

---

Published as a conference paper at ICLR 2023
subject to WiA 2 0, Bk 0.5, Ia 11 K Vg T
(29)
Similarly, we propose set of logical constraints for the V operator as 21 If we set a 0.5, the constraints in 21 become
k
Σ
2 28 1,
i=1
Σ 1=1
Wi S 28 1,
(30)
23 - 11-12 121220 0,
Wi 2
Reformulating the above constraints, we have
Σ f1
wi = 2,3 1,
(31)
B 2 0.5.
Wy 2 0.
(32)
The above constraints hold for each disjunction operator in ø. Therefore, we can incorporate the constraints in (31 into the objective function. The maximum likelihood estimation problem then becomes
min
- LL +
Wi.k 23, + 1)²
WiKK velu
23w + 1)² (33)
1= EE
subject to
Wiik 2 0, 3k 2 0.3, 4, v1 K Vg E
Wi,k' 2 20 2 O.5. Vi E J, 1A < k
VΦ E $
B PROOF OF THEOREM 8
The activation function designed for the A operator satisfies the properties of nonimpact for zero weights. impact ordering, and monotonicity. Without loss of generality, present the proof for the A operator connecting two clauses, which can be generalized to the A operator connecting k-ary clauses.
Proof 1 Here present the proof for the activation function for the A operator satisfying each property mentioned above.
Nonimpact for zero weights.
This means if wy = 0, = 1,2, then p(C', ,6. L) should have impact on p(C', ot ^ dh 1). Without loss of generality, we suppose W1 = O, thus we have
plC'. Sm A A0T2 t) =( 0 p(C', 11, w2 (1 p(C', (2, t)))
(34)
f(3 w2 (1 plC', 62 t)))
meaning p(C', ti, 1) has no impact on p(C', Dt A 22 l).
Impact Ordering
This means the truth degree of subformula with higher weights has a greater impact on p(C", T A / Mathematically, we need to prove that p(C' 01.t) p(C", Φ2, t) and
w, 2 W2 then
Op(C", N02 (1,)),
Op(C', 111 A A 20rtt)
(35)
Op(C",022
14

---

Published as a conference paper at ICLR 2023
As f(x) max(0, man }}}, have
0, if x <0
df
1,
if <<<<>,
(36)
dx
0, xf>>1. 1.
153 183 13--E=11:3 u1(( pp(C,, ,0; 1)) 00r B ΣE= wy(1 p(C'(((C_ (j,
then we have
Op(C', " A 1 Öp(C', of 1022 ,t) Op(C', 01.t)
Öp(C", 22,t) 0. (37)
Also, if0 8 Σ3=1 wy
.o.,t) then we have
2 0(3 - Σ3 wj(1 (1 - p(C', .(,,,,,)) w1 Σ
-(-- Óp(C", 11)
wj(1 ,,
(38)
j=1
and
a(B j= wj(1 ppppppCC_, o(,)))))))) w2(B - Σ Öp(C', 22,t)
2 wy wy(11p(C''.
(39)
j=l
As wh 2 w2, the following holds:
Op(C', 0th $2 Op(C',0"" A 2 Op(C", 02,t)
z
(40)
Op(C',011)
which proves the impact ordering property holds.
Monotonicity.
This means p(C', 01 60 .t) increases monotonicallyy over p(C', ie
f(B Σ wy (1 o(,)), - f(B - Σ wy(1 () for 0. (41)
First, note that 3 - E}=1 wT(( - pCC 35 1)) can be rewrilten as
2 B
Σ w7((1 o(,t)) = 3 - wi - W2 + wip(C', 01,t) + w2p(C', 22,t). (42)
2=1
This implies f(B E p(C', Kz. f)) is monotonically increasing p(C', 21,t) I) and p(C", 02, Also, from the proof of impact ordering we know f(x) max min/2, 1}} is monotonically nondecreasing, we can show that
f(B Σ (1,11p(C',, 5=1
S f(B
Σ
wy (1 ,,,, d)), 20. (43)
Thus the property of monotonicity is satisfied.
C EXPERIMENT RESULTS OF SYNTHETIC DATASETS
Dataset Generation. In the experiments on synthetic datasets, we manually generate 3 synthetic datasets considering different settings, where the details and results for the first synthetic dataset is reported in Section 4.2 Each setting considers a different order representation, different number of event labels different intensity of causal event labels.
15

---

Published as a conference paper at ICLR 2023
ADio
P(C', 11
pap
Ct Cg
A
pop
Cc
==l
A
A
A
4.8 pop
B.A pop
A.C pop
D.D Popp
D.A pop
C.D
pop
Ca (t)
(t(()
Cc(C)
Cp(t)
Figure 5: Model structure of Φ1 for generating the first synthetic dataset.
C.1 SYNTHETIC DATASET-1 (SYN-1).
Generation process. The first synthetic dataset contains 4 event labels: A, B, C, and D, where D the event for prediction, and A, B, C are causal events. The wCL. formula used generate event D in the first synthetic dataset is set as
Φ (CA cB 1)' A -CA- - 3));
(44)
whose unweighted version reads as "If A happens before B for at least time unit and happens before C' for at least 3 time units, then D will happen."
Here we consider event labels A, B, as free predicates, whose occurrences are generated by homogeneous Poisson process. The homogeneous intensity rate for A, B, C are set as ÅA = 0.2, An 0.2, and Ag 0.2. The algorithm used to generate instances of A, B, is described as
Algorithm Chen 2016).
Algorithm Simulation of a homogeneous Poisson process with intensity rate X. Input:
Intensity rate A, simulation horizon T
Output:
Occurrence time stamps T {tz}
1: Initialize n = 0, to 0;
2: while True do
uniform(0, 1); 4: Let w = (n
3: Generate
5: Set lu+1 In w.
6: if 1/+11 > then
7:
return T (tx {to]k=1,2....
8: else
9: Set
1;
10:
end if
11:
end while
With the above algorithm, we can generate the occurrences of event labels A, B, and C. Next, we build a CLNN for Φ = (CA - cB > 1)a (CA ca > 3) to caleulate the conditional intensity rate Àp &, whose model structure is shown in Figure 5 After obtaining \ (t), we could use
Algorithm Chen 2016) to generate the occurrence of D.
Results. The rules learned by CLNN, TELLER, and OGEM- tab on the first synthetic dataset are presented in Table where the paired order predicate among the candidates with the highest
16

---

Published as a conference paper at ICLR 2023
Algorithm 2 Simulation of an inhomogeneous Poisson process with intensity rate X(t).
Input:
intensity rate X(t), simulation horizon T
Output:
Occurrence time stamps T {tx} 1: Initialize n. = m = 0, l0 80 = 0, Ā suPosiSri A(O);
2: while 8m. 7 do
3: Generate uniform random variable ~ uniform(0, 1);
4: Let - In u/>;
5: Set 8m+1
We; Generate D ~ uniform((01);
6:
7:
if D S X(5m+1) then
8:
la+1 = 8m41i + 1; end if
9:
10:
11:
+ 1;
12:
In S T then
13:
return (4,),=1,,....
14:
else
return (1x)2=1220..
16: end if
end while
Dataset
Syn-1
N (# events)
(A,
Ground truth
Φ1 = (CA CB 1)² A (CA Cc 3)
CLNN's rule
(CA CB > 1.21) 1.52 A (CA - c > 3.00) A (CA
CD > 0.82) (CB c0 4.33)
(CB CD
10.69) A (cp - cc > -6.57)
TELLER's s rule
A before D, before D, C before D, before D and
before D
Excitation: [B]. [C), [C, BJ, [B, (A C. BJ. [A, B, C Inhibitory: [A]. [B, A], [B. A, CJ, [C. B, A], [A. B). [A. CJ. [B. A] [C. A. B1, [C, A]
OGEM-tab's rule
Table 5: Comparison of rule discovery for CLNN and TELLER on the Syn-I I dataset.
weight is presented. It can be clearly observed that by truncating the predicates with small weights, we could obtain the formula as
(CA - cB > 1.21)1.52 (CA- - cc > 33.00)1.41
(45)
which matches well with the ground-truth rule. However, TELLER cannot capture the paired order representation between A and B or A and C. OGEM-tab captures the order representation [A, B) and [A, C) as inhibitory causes, which contradicts the ground-truth rule.
C.2 SYNTHETIC DATASET-2 (SYN-2).
Generation Process. The second synthetic dataset contains 5 event labels: A, B, C, D and F, where E is the event for prediction, and A, B, C, D are causal events. The wCL formula used to generate the occurrence event in the second synthetic dataset set as
6² (c. cB 0.5) A (ca cc 1.5) A (cc cp 2
(46)
whose unweighted version reads as "If A happens before B for at least 0.5 time units, A happens before C for at least 1.5 time units, and C happens before D for at least time units, then will happen."
17

---

Published as a conference paper at ICLR 2023
ADio (t)
"pop
AB
> 0.5
pCc² 22
"pop"
> 1.5
"pop' Cp Cp
V
A
A
A
A
A.R Th pop
R.A Ppop
ACC Eppp
C.d pop
BC Tt
C,B pap
C.D pop
D.C pop
D.E. Pop
E.D Epop
Ppppp
(t(t)
(t(()
Cc
(t
CE (t)
Figure 6: Model structure of بN for generating the second synthetic dataset.
The occurrence of events A, B, C and D are generated using Algorithm [ in which Aa AB Ac = Ap = 0.2. After obtaining the occurrence of A, B, C and D, we simulate the generation of event label E' using Algorithm 2 in which the intensity rate > (t) is computed using the model shown in F
Results. The rules learned by CLNN, TELLER and OGEM-tab on the second synthetic dataset are presented in Ta where the paired order predicate with the highest weight presented. can be clearly observed that by truncating the predicates with small weights, CLNN learns wCL. formula as:
(47) whose order representation match well with the ground-truth rule. Nevertheless, TELLER's rule only capture the ordering between A. B and E, whereas the ordering between A and B or B and C' or and are not learned. OGEM-tab's rules can only capture the relation between event label D and event label E can excite the occurrence of event label E, whereas not able capture the dependence of event label E's occurrence on the order relation between and or and C or C and D.
Φ (CA- cB > 0.77) A (cA-co - cc > 2.09) (cc- - cp > 2.60) 1.06
Dataset
Syn-2
N (# events)
N 5, (A,,C,,,
Ground truth
62 (CA (CA-CB> (A5 A(BB) 155) A(cc -DD
2)
CLNN's rule
(CΛ en > 0.77) A -CA-c0 - > 1.19 A ((ca
cD) > -5.00) (() cg) -2.74)0.09 (CB
cC > -9.31) A (cg CD > -8.54) A (CB
Cy 2.07) A ((cc - cp) > 2.60) 1.06 ((ce CE)
-.27))..03 A ((CD- CE) 1.17)
TELLER's s rule
A before E, B before E, and B before E, and before E
Excitation: (D). [D, EJ, [ε] [E, DJ
OGEM-tab's 's rule
Inhibitory: [D, A], [A], [A, D1, [A.DEE], IE. D. A. EJ, EJ. [E. A}, [D, E. A), [A, E. I.. D]
Table 6: Comparison of rule discovery for CLNN and TELLER on the Syn-2 dataset.
C.3 SYNTHETIC DATASET 3 (SYN-3).
The third synthetic dataset is generated using a more interesting scheme by combining the generation schemes of the first synthetic dataset and the second synthetic dataset. The third synthetic dataset
18

---

Published as a conference paper at ICLR 2023
2 piiaa (t)
131.1)
C4 > -2
App
C6
C4
w
A
V
A
V
w
A.P pop
R.A pop
A.C pop
C.A pop
A.D pop
DA pop
C.D. pon
D.F POT
(t)
Ct())
cc(t)
(t)
Figure 7: Model structure of Φ.,1 for generating the occurrence of D in the Syn-3 dataset.
Aster (t)
p(c² 232.2
Cy
-4
Fr
Eg >
co 4e >
A
A
ir pop
pop
pon
pop
pon
poy
pppp
Top
Pop
(t)
(t)
(t)
(t)
(g(t)
Figure 8: Model structure of δ3.2 for generating the occurrence E in the Syn-3 dataset.
includes five event labels: A, B, C. D and F. Here we consider A, B, and C as the causal events for the occurrence of D) and A, B,C, and as the causal events for the occurrence of F. The occurrence of events A, C' are generated using Algorithm [ in which d 0.2, A 0.2, and A, = 0.2. The wCL formula used to generate the occurrence of event D is set as
Φ = (cB CA > 2) A (ec - CA cA -5)
(48)
whose unweighted version reads as "If happens before B for less than time units, and A happens before C' for less than time unit, then D will happen." The generation of D's occurrence follows Algorithm where A (t) computed using the model shown in Figure 7 We call the third synthetic dataset at this step as Syn-3.1.
After obtaining the occurrences of events A, B, C, and D, could simulate the occurrence of E using the following formula:
S = (cB
-5) ^ (cc cB - A - cc -3)1
(49)
Similarly, the generation of E's occurrence follows Algorithm 2 where the intensity rate Ss (t) computed using the model shown in Figure g We call the Third synthetic dataset at this step as Syn-3.2.
Results.
The rules learned by CLNN, TELLER, and OGEM-tab on the cause of event D in the third synthetic dataset are presented in Table where the paired order predicate with the highest weight among the two candidates is reported. It can be clearly observed that by truncating the predicates with small
19

---

Published as a conference paper at ICLR 2023
weights, CLNN learns wCL formula as
2 (cB CA > -1.55-8 A (cc CA > -3.90)
(50)
whose order representation match well with the ground-truth rule. On the other hand, TELLER's rule only reveal the temporal relation between event labels A, B, C and D, but does not capture the temporal relation between event labels A and B or A and C. In addition, we could observe that OGEM-tab does not capture that is a parent event of D.
Dataset
Syn-3.1
N (# events)
N 5, 5 (A,,
Ground truth
@ = (CB - CA C1> -2) (cc - CA > -5)
CLNN's rule
(CB - CA
-1.85) A (cc CA > -3.90) A ((CD CA)
-16.25) 0.33 ^ (Icc - CB) > -3.01) (CD - cB > -7.37) A
(CD cC > -7.55)0
TELLER's rule
before D. B before D, C' before D
Excitation: [A], [A, B, D1, [B, D. AI, (D. AI, (), BI, A.
OGEM-tab's rule
D1. [D]. [B, A, D1. [D, B, A]
Inhibitory: [A, B), [B, D), (B), [A, D, BD,
Table 7: Comparison of rule discovery of Φ3,1 for CLNN and TELLER on the Syn-3.1 dataset.
The rules learned by CLNN, TELLER, and GEM on the cause of event in the third synthetic dataset are presented in Table g in which the discrete wCL formula learned by CLNN is
$ = (CB CA > - -3.94) 1.49 A (cc - cB > 3.02)2..0 A ((CD - ec) > -2.00) 1.92
(51)
It is obvious that Φ3.2 is able to learn the temporal relation between A and B, B and C. and C and D. However, TELLER's 's rules only reflect the temporal relation between A, B, C and E, which cannot give the information about the temporal relation betwee A and B, or B and C, or C' and D. OGEM-tab's rule indicate that it considers event labels A, D, F. as the parent events of D, which does not match with the ground-truth parent set.
Dataset
Syn-3.2
N (# events)
N 5, L (A, C,
Ground truth
23.2 = (cn- CA > -5)² A (CC CB > -4)² A
(CD Cc > -3)1
CLNN's rule
(CB-CA> -3.94) A(CC-CA
9.12)
((CD - CA) > -1.42)0.13 A ((E ca) >
-3.88)0.16 (cc-CB > -3.02) 200 ^ A(CD - CB >
-.. A (CE cB -7.30) 0.04 A (CD
cc) > 2.00) 1.92 A ((cg -cc) > -5.30) 0.09 A ((CE cD) > -1.77
TELLER's rule
before E, 1 before F, C before E
Excitation: [A, D). [D, A}, [D, EJ. [E], [A, D,
OGEM-tab's rule
E). (D. E, A}, [E, A], [A, EJ, [E, A, DJ, [A, E, DD. A, ET, [E, D,
Inhibitory: [A], [D], [E, D]
Table 8: Comparison of rule discovery of 63.2 for CLNN and TELLER on the Syn-3.2 dataset.
C.4 QUANTITATIVE COMPARISON OF CLNN'S S RULES WITH GROUND TRUTH
To quantitatively evaluate the difference between the ground-truth rules and the rules learned by CLNN, we adopt the Jaccard similarity score to assess the learned formulas against the ground truth. Let & denote the set of paired ordering representations from the ground-truth rule, and C denote the set of paired ordering representations from the learned rules, the Jaccard similarity score
is calculated J cng For TELLER and OGEM-tab, the ordering representations are extracted
cUg
20

---

Published as a conference paper at ICLR 2023
Jaccard for Syn-1
Jaccard socre for Syn-2
1.0
1.0
0.8
0.6
Value
0.4
0.2
0.05
0.05
0.05
0.05
0.0
CLNN
TELLER Models
OGEM
CLNN
TELLER Models
OGEM
(b)
Syn-3.1
Jaccard socre for Syn-3.2
1.0
100
0.8
Value
0.6
Value
0.6
0.4
0.2
0.09
0.0
0.0
CLNN
TELLER Models
OGEM
CLNN
TELLER Models
OGEM
(c)
(d)
Figure 9: Comparison of ground-truth rules with CLNN's rules in terms of Jaccard similarity score for a) Syn-1, b) Syn-2, Syn-3.1, d) Syn-3.2.
from the excitation rules. The comparison of Jaccard similarity score for the synthetic datasets is shown in Figure where the Jaceard similarity score of is manually set the minimum threshold 0.05 for clarity purposes. It is clearly observed that the Jaccard similarity scores for CLNN is higher than the ones for TELLER or OGEM, implying the rules discovered by CLNN are more consistent with the ground truth.
C.5 STABILITY ANALYSIS OF CLNN'S RULES WITH RESPECT TO INITIALIZATION
To further validate the model's stability in learning wCL rules, different parameter initialization methods are carried out, including:
1. rand - parameter initialization as random numbers from a uniform distribution on the in- terval [0, 1):
2. randn random numbers from a normal distribution with mean and variance 1;
3. ones constant values of 1;
4. xavier random numbers from uniform distribution on the interval where n the dimension of the parameter.
The rules learned by CLNN for the above parameter initializations summarized in Table 9 By inspecting the rules for different initialization methods, clear that CLNN can still recover the correct paired order representations even if initializing the learning process from different position. In the meantime, the logic formulas learned by CLNN are stable as the variance of learned parameters relatively small.
21

---

Published as a conference paper at ICLR 2023
Datasel
Initialization
Rules
Ground truth
(Ca -
Cy
1)
(cg - co > 3)
rand
Φ= [CA
1.21)
[cs 00 3.00)
Syn 1
randn
0 [CA > 1.21
1:00 Ac1 c0 3.32)
#= [64 Gg 1.17]
1 [ex 00 3.14)
xavier
0= lox cy> 1.12)
(es 00 3.20
Ground truth
Φ (CA - on > 0.5) (ca -6 1.5) A (ec
2)
rund
④ (CA op > 0.77) Aea 60 2 2.00 15 (ee [p) 2.60)
Syn 2
randn
0
(ca
cB 0.80) " A (ea 60 192)
(ee op) 1.74)
Ones
(ca <p> 1.00
18 Aca -0
1.92 122
(1ee
cp) > 2.03 1.
cca
cg
> 0.97
ACe - Fo 207
(
ppl
1977
Ground truth
0= (cp-c44 - -4 -2) ^ (cg c21> -c1--5 -5)
rund
Φ loy <4 1,85 A lce 64 3.90
Syn 3.1
randn
0= (0g - -198]
(cc -4 -3,89)
ones
01.1 = (cp 61 1.94) Are - 64 > -3.68)233
xavier
05.1 = (C8 04 -189) A00 - 64 > -3.92)
Ground truth
[cn > 5). A (co
4)² A (cp -0 > -3)
rund
@ (og
> 3.94) A (cc 6/ > 3.02 mm ((e (6) > 2.00)
Syn 3.2
randa
@ (CB > -3.79) n (rc -00 -3.4
A ( - -0) > -1.88
ones
0 (cp CΛ > -3.33) 1.00 (cc cp > 3.09)
88 A (lep 60) > -1.25)
xavier
6 (CR > -3.71) A cc > 33.0) A (Cp 00) > -1.8)
Table 9: Comparison of rules learned by CLNN for different parameter initialization methods.
C.6 ANALYSIS LOGICAL CONSTRAINTS ON THE LL
In this part, we investigate the effect of the interpretability using experiment of the impact of log- ical constraints on the model's performance. The log-ikelihood on the synthetic datasets for CLNN with and without logical constraints is summarized in Table TO, Table T0 demonstrates that the log- likelihood for CLNN with logical constraints is higher than the log- lig-liiel for CLNN without constraints, implying that interpretability (logical constraints) is helpful improve the performance.
Dataset
CLNN with constraints
CLNN without constraints
Syn 1
-7821
-8716
Syn - 2
-6075
-6942
Syn 3.1
-10898
-11583
Syn 3.2
-10919
-11230
Table 10: Comparison of L.I. for CLNN with and without logical constraints.
D EXPERIMENT RESULTS OF REAL-WORLD DATASETS
D.1 LINKEDIN DATASET
The LinkedIn dataset is a collection of job hopping records between 82 IT companies of 3,000 Linkedln users. Each event stream represents a user's check-in time stamps for different com- panies or role changes within the same company. Here we select 1000 users' event streams to compose the dataset by filtering out the event streams with uncommon companies, resulting in 10 event labels: = (A, B,C,D. D, E, F, G, H,, JJJ.. Here we set the number of formulas as 5. i.e., Tb {$1, Φ2, Φ3, 64, Φs), each of which embodies a model structure shown in Figure 2(a) and CLNN aims to learn the parameters for each formula. The weight parameters in the paired order cell or the singleton order cell are initialized as random variables following Gaussian distribu- tion, and the bias terms of conjunction or disjunction operators are initialized as 1. The architecture weights are initialized random variables following Gaussian distribution, and the formula im- pact weights and bias are initialized as Gaussian random variables. The detailed log likeelhodood for each event label is summarized in T1
22

---

Published as a conference paper at ICLR 2023
Event Label
Log-likelihood
-180.59
B
-177.80
-89.49
-140.31
E
-132.83
-76.63
G
-106.23
-103.33
-95.51
-125.45
Table 11: Log likelihood for each event label in the Linkedln dataset.
D.2 MIMIC II DATASET
MIMIC dataset is obtained from the intensive care unit research database that consists of 25,328 intensity care unit stays. The records include laboratory data, therapeutic intervention profiles such as nursing progress notes, discharge summaries and others. Here we restrict the event types to the diagnosis of patients and filter out the shorter event sequences with few visits, ending up with 650 patients and 15 event labels: L {1, 2, 8, 9, 11, 12, 14, 20, 21, 22, 23, 26, 27, 42, 47). Similar to the setting for the LinkedIn dataset, where the initialization of parameters follow the same setting as the LinkedIn dataset. The detailed log- likelihood for each event label is presented in Table [2]
Event Label
log-likelihood
-72.14
-62.33
-5.98
-51.34
11
-43.64
-25.81
14
-69.73
20
-5.66
21
-6.08
-10.47
23
10064
26
227.08
27
-27.42
42
-5.95
47
-10.54
Table 12: Log likelihood for each event label in the MIMIC II dataset.
D.3 STACK OVERFLOW DATASET
Stack Overflow is question-and and-wrswer website spanning a wide range domains. A badge rewarding scheme is exploited to encourage users to participate in the questioning and answering activities. The badge system of Stack Overflow comprises 81 types of non-topical badges, including the badges that can be awarded only once and the badges that can be awarded several times. The dataset (Du et al. 2016) was obtained by first filtering out the badges that can be awarded only once, then restricting to the users who have acquired at least 40 badges from 2012-01-01 to 2014-01- 01, from which the badges have been awarded more than 100 times are selected as the determinate dataset. Our dataset specified badges and then randomly sampling 1000 users to obtain 1000 event streams. The detailed log-likelihood for each event label in the Stack Overflow dataset is summarized in Table 13
acquired by retaining the event streams with one more of the 20 types of
23

---

Published as a conference paper at ICLR 2023
Event Label
Log-likelihood
-3791
-1451
-538
-17656
-3574
-3559
-1381
-1330
-10961
-1105
11
-189
12
2012
13
-673
-1340
15
-406
16
-117
17
-186
18
-330
19
-282
20
-100
Table 13: Log likelihood each event label in the Stack Overflow dataset.
Dataset
CLNN with SOP
CLNN without SOP
LinkedIn
-1228
-1344
MIMIC II
-436
-480
Stack Overflow
-50981
-51195
Table 14: Comparison of log-likelihood for CLNN with and without SOP on the real-world datasets.
D.4 ANALYSIS OF EXPRESSIVENESS ON MODEL'S PERFORMANCE
In this part, we conduct an experiment by training the CLNN without the singleton order cell (SOC) on real-world datasets show the effectiveness of the singleton order predicates. The comparison of log- liglliiel for CLNN with SOC and CLNN without SOC is summarized in Table 14] As evidenced by Table 14 the log-likelihood of CLNN with SOP is higher than the log-likel ligeeiilloood of CLNN without SOP, meaning enriching the expressiveness of wCL formulas can better explain the generative mechanism of events.
24


Reviews:
[{"review_id": "vhT_x9KjJT", "reviewer": "Reviewer_7kNw", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies event streams and proposes a neural-network-based method that learns weighted clock logic to model temporal point process. Compared to traditional combinatorial optimization approaches, this method is more expressive in terms of representing weighted clock logic formulas as neural network activation function; easier to learn through gradient descent. ", "summary_of_the_review": "This paper proposes a novel approach to learn temporal point processes using neural networks which learn weighted clock logic formulas. Though the presentation is clear, the major concerns of this paper are the lack of motivation and competitive empirical results. I am not an expert on this topic, but I think the authors can improve this paper by providing clear motivation/theoretical insights, or detailed explanation/verification of all the claims (such as interpretability and effectiveness).", "strength_and_weaknesses": "The approach of using neural networks to model the time interval length is novel. The writing of this paper, especially the technical part, is sound and with minor issues. There are some insights of how to relax logical formulas that people can draw from this paper.\n\nThe major concern of this paper is that there are no theoretical insights or motivations for the proposed method. The authors present the architecture of the clock logic neural networks without motivating the reason behind them, nor do they explain how the smooth relaxation or POC/SOC benefits problem-solving, either in the method section or in the experiments section.\n\nIn section 2, though the examples are easy to follow, the authors introduce more than enough terminologies which increases the complexity of understanding. For example, it seems only \"paired order predicate\" is used in the rest of this paper so other terminologies can be simplified. \n\nThe author claims to learn \"interpretable\" rules without detailed explanation/verification, either in the main method section or in the experiments section. \n\nThe synthetic dataset is too simple and thus easy to learn. The \"effectiveness\" of the proposed method needs to be verified on larger datasets or real-world datasets. The empirical results on real-world datasets are not competitive enough compared to existing methods. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "fUtuFn0xlZ", "reviewer": "Reviewer_ndLd", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper introduces a framework of 'clock logic neural networks', which use a special neural network architecture to train temporal process models that can then be interpreted in terms of weighted clock logic formulae.  The paper provides a detailed overview of weighted clock logic, and proves that the network architecture enforces the desired properties necessary for this interpretation.  Training can be achieved directly through projected gradient descent.  The experimentation on synthetic and real datasets shows that the CLNN learns more informative rules (incorporating interval duration information) than alternative SOTA approaches, and achieves competitive performance in terms of the out-of-sample log-likelihood.", "summary_of_the_review": "An interesting neuro-symbolic approach for training NN models with interpretability in terms of temporal logic.  The paper will be of interest to those working in logic, deep-learning and model interpretability.", "strength_and_weaknesses": "Strengths:\n\n- The paper is presented very thoroughly, and the theoretical properties of the model are well established.\n- Developing neuro-symbolic approaches is an important problem, both from the point of view of model interpretability, and modeling complex dependencies; the paper offers an elegant solution by formulating a differentiable model which has a logical interpretation.\n- The experimentation is somewhat condensed but convincing on both synthetic and real problems.\n\nWeaknesses:\n\n- As above, the experimentation could be expanded.  Mostly, the interpretation is done by simply listing the formulae extracted by the network (Tables 1 and 4) and inviting comparison by inspection.  It would be beneficial to develop a metric to assess the formulae found against the ground truth for various methods.\n- The network training incorporates constraints on architecture and parameters (via projected gradient) to ensure interpretability.  It would be interesting to see the log-likelihood performance of similarly sized NN models without these constraints, to gauge if the interpretability is helping or hindering performance.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "cQtxU8jSh1", "reviewer": "Reviewer_ryco", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an approach for learning interpretable logical rules for event data, where each event is represented by a timestamp and a discrete event type.\nThe main idea of the proposed approach is to represent the logical rules and their combinations as nodes in a neural network. By optimizing the weights of the neural network with gradient descent, the relevant logical rules are learned from the data. This is a significant improvement compared to earlier works, where the rules are learned using combinatorial optimization tools.", "summary_of_the_review": "The paper advances the state of the art in the problem of learning interpretable logic rules from multivariate event streams.\nThe proposed approach offers a clear improvement over existing methods both in terms of expressivity and runtime.", "strength_and_weaknesses": "Strengths:\n- The proposed framework has several clear advantages compared to the existing approaches:\n    - The learned logic rules take into account the time between the events (not just their ordering), which can be important in some applications\n    - Thanks to using a differentiable relaxation of the discrete search space, the training time for the proposed approach is ~100x faster compared to the existing state-of-the-art method\n    - The predictive performance of the proposed model is competitive with other approaches\n- The examples introduced throughout the paper are helpful for understanding the contents.\n- The experimental setup is described very clearly, which should make the results easy to reproduce.\n\n\nI haven't found any major weaknesses in the paper. The only part that could be explored in more detail is the stability of the learned logic rules with respect to different random initializations of the network weights and different choices of the  threshold parameter  $\\alpha$.\n\nNotes:\n- Typo: $c$ should be replaced with $l$ in Equation 9\n- Figure 1: Would be helpful to mention that the approach can also work with continuous timestamps, and that discrete timestamps are chosen for illustration purposes.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "MM2MOUtWosQ", "reviewer": "Reviewer_LMp2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a method for learning models of multivariate event streams, which are time series of events that may be may occur irregularly and synchronously at points in continuous time.  Specifically, they introduce clock logic neural networks (CLNNs), which model temporal point processes using formulas to represent conditional intensity rates.  Learning in this context involves a relaxation of the satisfiability predicate on formulae and then gradient-based maximum likelihood optimization.  ", "summary_of_the_review": "- Well-motivated and interesting problem\n- Sound approach but unclear the relevance of the result.  I'm willing to be convinced otherwise\n- I’d suggest focusing on the core claims and providing more evidence", "strength_and_weaknesses": "Strengths:\n\n- Attacks an interesting and useful problem with a well-motivated approach\n- The method is performant (in absolute execution time) and has the potential to be a practical tool\n- The general approach of relaxation for learning multivariate event streams could generalize to richer representations\n\nWeaknesses\n\n- The experimental results are mixed.  More importantly, it’s not clear when I should expect this method to perform well or poorly.  I would not wish to penalize this paper for not beating SOTA in every situation, but it’s not apparent when and why I should use this approach over any other.  The method performs best on the LinkedIn dataset.  Is this coincidental, or is there some salient property there that makes a difference?\n- The primary claimed advantage of CLNNs of SOTA is expressiveness.  While this is to some degree true by construction — CLNNs can represent the relationships between interval lengths and the conditional intensity — there’s not much evidence to demonstrate that it matters in its ability to model a richer class of datasets.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster