Preventing Conflicting Gradients in Neural Marked Temporal Point Processes
Authors: Tanguy Bosser
Souhaib Ben Taieb
Venue: Accepted by TMLR ()

Abstract: Neural Marked Temporal Point Processes (MTPP) are flexible models to capture complex temporal inter-dependencies between labeled events. These models inherently learn two predictive distributions: one for the arrival times of events and another for the types of events, also known as marks. In this study, we demonstrate that learning a MTPP model can be framed as a two-task learning problem, where both tasks share a common set of trainable parameters that are optimized jointly. We show that this often leads to the emergence of conflicting gradients during training, where task-specific gradients are pointing in opposite directions. When such conflicts arise, following the average gradient can be detrimental to the learning of each individual tasks, resulting in overall degraded performance. To overcome this issue, we introduce novel parametrizations for neural MTPP models that allow for separate modeling and training of each task, effectively avoiding the problem of conflicting gradients. Through experiments on multiple real-world event sequence datasets, we demonstrate the benefits of our framework compared to the original model formulations.

Published in Transactions on Machine Learning Research (01/2025)
Preventing Conflicting Gradients in Neural Marked Temporal Point Processes
Tanguy Bosser
tangug.bosser@uuonn..c...
Deportment of Computer Science University of Mons
Souhaib Ben Taich
souhaib.bentaich bentaieh @wmbzuai.ccmm ac.ac.
Mohamed bin Zayed University of Artificial Intelligence University of Mons
Reviewed on OpenReview: https: // openreveew. net/ pett/formm? id= idt JSSPtb
Abstract
Noural Marked Temporal Point Processes (MTPP) are flexible models to capture complex temporal inter-dependencies between labeled events. These models inherently learn two predictive distributions: one for the arrival times of events and another for the types of events, also known as marks. In this study, we demonstrate that learning MTPP model can be framed two-task learning problem, where both tasks share common set of trainable parameters that are optimized jointly. We show that this often leads to the emergence of conflicting gradients during training, where task-specific gradients are pointing in opposite directions. When such conflicts arise, following the average gradient can be detrimental to the learning of each individual tasks, resulting in overall degraded performance. To overcome this issue, we introduce novel parametrizations for neural MTPP models that allow for separate modeling and training of each task, elfectively avoiding the problem of conflicting gradients. Through experiments multiple real-world event. sequence datasets, remonstrate the benefits of our framework compared to the original model formulations.
1 Introduction
Sequences of labeled events observed in continuous time at irregular intervals are ubiquitous across various fields such as healthcare (Enguehard al.] 2020). finance (Bacry et al. 2015). social media (Farajlabar et al. 20117), and seismology {Ogata| 1998). numerous application domains, an important problem involves predicting the timing and types of future
historical data. Marked Temporal Point Processes (MTPP) (Daley & Vere-Jones, 2003) provide mathematical framework for modeling sequences of events, enabling subsequent inferences on the system under study. The Hawkes process, originally introduced by Hawkes (Hawkes] 1971), is a well-known example of MTPP model that has found successful applications in diverse domains, including finance (Hawkes) 2018), crime analysis (Egesdal et al. 2010), and user recommendations (Du et al. 2015). However, the strong modeling assumptions of classical MTPP models often limit their ability to capture complex event dynamics (Mei & Eisner 2017). This limitation has led to the rapid development of more flexible class of neural MTPP models, incorporating recent advances in deep learning (Shchur et al. 2021).
called
This paper argues that learning a neural MTPP model can be interpreted as two-task lcarning problem where both tasks share a common set of parameters and are optimized jointly. Specifically, one task focuses on learning the distribution of the next event's 's arrival time conditional on historical events. The other task involves learning the distribution of the categorical mark conditional on both the event's arrival time and the historical events. We identify these tasks as the time prediction and mark prediction, respectively. While parameter sharing between tasks can sometimes enhance training efficiency (Standley et. al. 2020 it may

---

Published in Transactions on Machine Learning Research (01/2025)
also result in performance degradation when compared to training each task separately. A major challenge in the simultaneous optimization of multi-task objectives is the issue of conflicting gradients (Liu et al. 202 This term describes situations where task-specific gradients point in opposite directions. When such conflicts arisc, gradient updates tend to favor tasks with larger gradient magnitudes, thus hindering the learning process of other concurrent lasks and adversely affecting their performance. Although the phenomenon of conflicting gradients has been studied in various fields (Chen et al.) 2018 2020; Yu et al., 2020 Shi et al.) 2023), its impact on the training of neural MTPP models remains unexplored. We propose to address this gap with the following contributions:
(1) We demonstrate that conflicting gradients frequently occur during the training of neural MTPP models. Furthermore, we show that such conflicts can significantly degrade a model's predictive performance on the time and mark prediction tasks.
(2) To prevent the issue of conflicting gradients, we introduce novel parametrizations for existing neural TPP models, allowing for separale modeling and training of the time and mark prediction lasks. Inspired from the success of (Shi et al. 2023), our framework allows to prevent gradient conflicts from the root while maintaining the flexibility of the original parametrizations.
(3) We want to emphasize that our approach to disjoint parametrizations does not assume the independence of arrival times and marks. Unlike prior studies that assumed conditional independence (Shchur et al. 2020; Du et al. 2016), we propose a simple yet effective parametrization for the mark conditional distribution that relaxes this assumption.
(4) Through series of experiments with real-world event sequence datasets, show the advantages of our framework over the original model formulations. Specifically, our framework effectively prevents the emergence of conflicting gradients during training, thereby enhancing the predictive accuracy of the models. Additionally, all our experiments are reproducible and implemented using common code base"]
2 Background and Notations
A Marked Temporal Point Process (MTPP) is a random process whose realization is a sequence of events S = {c, = (t,, k)) Each event 6 E S is an ordered pair with an arrival time t; E [0, T) (with ty = 0) and a categorical label k E K = (1..., K} called mark. The arrival times form a sequence of strictly increasing random values observed within a specified time interval (T, i... 0 St < < ... tn S T. Equivalently, 7. = 4 E R is an event inter-arrival time. We will use both reprosentations interchangeably throughout. the paper. If c.-1 is the last observed event, the occurrence of the next event e; in (f:-1, oo can be fully characterized by the joint PDF [ k k(He), where H4 = {(4,, E 442 < f) is the observed process history. For clarity of notations, we will use the notation 's' of (Daley & Jones 2008) to indicate dependence on He, i.c. f"(T,k) = k(H,) This joint PDF can be factorized as f'(T, k) = (k)T)) where f*(r) is the PDF of inter-arrival limes, and p" (k)T) is the conditional PMF of marks. A MTPP can be equivalently described by its so-called marked conditional intensity functions (Rasmussen) 2018). which are defined for 4-1 <t < 4 as xi(1) = (t) FF), where [+) is the conditional CDF of arrival-times and IK From the marked intensities, we can also define the marked compensators Ag(()= = fr (s)ds.. Provided
that certain modeling constraints are satisfied, each of f (T,k), (t(() and (t) fully characterizes MTPP and can be retrieved from the others [Rasmussen]
2018
Enguehard et al. 2020
Neural Temporal Point Processes. To capture complex dependencies between events, the framework of neural MTPP (Shchur et al.] 2021 Bosser & Ben Taieb 2023) incorporates neural network components into the model architecture, allowing for more flexible models. A neural MTPP model consists of three main components: (1) An event encoder that learns representation e, E Rdd. for each event e, S, (2) history encoder that learns a compact history embedding h; E RR of the history Hi, of event e;, and (3) a decoder that defines a function characterizing the MTPP from h; e.g. At)... Let kT; 0),
(f 0) be valid model of f (T, k), where the set of trainable parameters 0 lies within the parameter space O
httpe://gitaab.cc//aaaerboosssss
2

---

Published in Transactions on Machine Learning Research (01/2025)
To train this model, we dataset S = {Si, ...., ) where each sequence St comprises my events with
arrival times observed within the interval [0, 7' and 1 = ,.... L. The training objective is the average sequence
negative log-likelihood (NLL) (Rasmussen 2018), given by
1 Σ i=1
n log ( k1, 0) - log(1 1og(1-F' -T:
[(0:5)
(1)
which is optimized using
stochastic gradient descent
RRuder 2017
3 Conflicting Gradients in Two- Task Learning for Neural MTPP Models
Consider the factorization of (",,,, k 0) into (T,,, 0) and p" (kr, 0). where f k), f p" (k
0). Substituting this decomposition into the NLL in equation and rearranging terms, we obtain:
1 Σ
((0:8)= - L
log f ( 0) - log(1 - FT(TT)))
log p" (kL,sT.::00)
(2)
L
(r(0,8)
Exr(0,S)
This shows that the total objective function S) consists of two sub-objectives: Cr(O; S) and (m revealing that learning an MTPP model can be interpreted as a two-task learning problem. The first objective, Cr SO; relates to modeling the predictive distribution of (inter-)arrival times, which we refer to as the time prediction task Tr. The second objective, Lar S0, concerns modeling the conditional predictive distribution of the marks p" O), which we call the mark prediction task TM-
Conflicting gradients. Assuming that (0) and (0) are differentiable, let 9r = (eCr and 9M (O) denote the gradients of (0) and Las (0), respectively, with respect to the shared parameters As discussed (Shi et al. 2023), when gr and gM are pointing in opposite directions, ie. gr 9M < O, an update step in the direction of negative gr for 0 will increase the loss for task Tas, and inversely for task Tr if update step is taken in the direction of negative gM- Such conflicting gradients can be formally defined as follows.
Definition 1 (Conflicting, gradients (Shi et al. ΦTм E [0, 2m] be the angle between the gradiends 9r and 9M- They are said to be conflicting with each other if cos ΦTM < O.
2023) Let
(a) Conflicting
(b) Non-conflicting
9M
9M
The smaller the value of cos ΦTM E [[-1.0], 0]. the more severe the conflict between the gradients. Figure 4 illustrates these conflicting gradients. Idcally,
gT
ΦTM
ΦTM
gT
want the gradients to align during optimization (i.e. cos ạтM² 0) to encourage positive reinforcement between the two tasks, or to be simply orthogonal (i.e. cos ΦтM = =). Conflicting gradients, especially those with significant differences in magnitude, pose substantial challenges during the optimization Specifically, if gr and gM conflict, the update step for 0 will likely be dominated by the gradient of whichever task gr or 9M has the greater magnitude, thereby disadvantaging the other task. The degree of similarity between the magnitudes of these two gradients can be quantified using metric known as gradient magnitude similarity, defined follows:
Figure 1: Conflicting gradients
multi-task learning objectives Yu et al. 2020
Definition 2 (Gradient Magnitude Similarity Yu et al. 2020)) The gradient magnitude similarity between 9r and gM is defined GMS 2l|gr la 1g where ||. [|2 is the In-norm.
1| +1g
A GMS value close to indicates that the magnitudes of gr and gas are similar, while a GMS value close to suggests a significant difference between them. Ideally, we aim minimize the number of conflicting gradients and maintain GMS close to to ensure balanced learning across the two tasks. However, a
We explicitly omit the dependency of fo and Las on simplify notations.
3

---

Published in Transactions on Machine Learning Research (01/2025)
THP
SAHP
FNN
LNM
RMTPP
CG: 0.57 GIMS: 0.65 TPL: 0.63
DG: 0.61 GIS:: 0.70 TPI: 0.47
CC: 0.50 CIMS: TPL: 0.67
CG: 0.50 CMMS: 0.61 0.87
CG: 0.47 GMS: 0.46 0.92
0.150
0.333
0.20
0.150
5.35
LastFM
Proportion
0.35
0.25
0.300
03225
5.30
0.20
6.30
0.075
0.305
0.075
0.05
6.05
0.050
0.023
0.025
9.00
0.30
0.00
0,003
0.006
9T
THP
SAHP
FNN
RMTPP
[G: 0.82 GMS:
0.88 TPL: 0.70
CG 0.87 GMS: 0.90 TR: 0.30
CG: 0.38 GMS: 0.67 TPL: 0.60
CG: 0.39 GMS. 0.45 TRL: 0.95
CG: 0.31 GNS: 0.65 TPL: 0.88
0.20
0.350
9:32
0.125
0.20
Propertion
MOOC
0.30
0.35
3.00
0.20
0.075
0.20
0.30
0.04
0.25
2.050
0.05
0.52
3.00
0.30
0.00
0.00
-10
-0.5
5.0
2.5
3.2
-2.0
-0.3
2.5
0.3
3.0
-1.0
-0.5
0.3
5.5
2.5
-1.0
-0.5
5.0
0.5
1.0
-2.0
-0.3
0.5
05
10
009
6PM
CO8 61
008 QP
CO8
Figure 2: Distribution of cos ørw during training for the different baselines on MOOC and LastFM. CG refers to the proportion of cos ΦTM- 0 observed during training. The distribution is obtained by pooling the values of ΦTM over 5 training runs, and gradients that are conflicting correspond to the red bars.
low GMS value among conflicting gradicnts does not specify which task is being prioritized. Therefore, we introduce the time priorily inder to address this issue, which is defined as follows:
Definition 3 (Time Priority Index) The time priority inder between conflicting gradients gr and gM with cos ΦTM < 0 is defined as TPI = 1 (119rl12 11g (12) where 1(.) is the indicator function.
If gr and gMs are conflicting and omtimization prioritizes task Tr, then the TPI takes the value 1. Conversely, if task TM is prioritized, the TPI takes the value 0.
Do gradients conflict in neural MTPP models? To explore this question, we perform a preliminary experiment with common neural MTPP basclines that either learn (t, k; 0), \i f), or (t: 0): THP (Zuo et al. 2020), SAHP (Zhang et al., 2020), FNN (Omi et al. 2019), LNM (Shchur et al., 2020 and RMTPP Duu et al. 2016). We aim to determine whether conflicting gradients occur during their training. For this purpose, each model is trained to minimize the NLL defined in (2) using sequences from two real-world datasets: LastFM (Hidasi & Tikk 2012) and MOOC (Kumar et al. 2019). For optimization, we rely on the Adam optimizer (Kingma& & Ba 2014) used by default in neural MTPP training with learning rate a = 10-3 At every gradient update, we calculate the gradients gr and 9M with respect to the shared parameters
(t,,
recording values of cos ΦтM, GMS, and TPL. Figure 2 shows the distribution of cos ΦTM across all S training iterations, along with the average values of GMS and TPI, and the proportion of conflicting gradients (CG), computed as
Σee =G= 1 (cos dTTM <0) CG -
(3)
E CoS 67M
where 6Tм refers to the angle between g and gis at training iteration s. Gradients that are conflicting during training correspond to the red bars. We observe that some models, such as THP and SAHP on MOOC, frequently exhibit conflicting gradicnts during training, as indicated by a high value of CG. Conversely, while other models show a more balanced proportion of conflicting gradients, these are generally characterized by low GMS values, which may potentially impair performance on the task with the lowest magnitude gradient. In this context, the data. indicates that optimization generally tends to favor Tr during optimization, as suggested by an average TPI greater than O.5. In Section G our experiments show that the combined inffuence of a high proportion of conflicts and low GMS values during training can significantly deteriorate model performance on the time and mark prediction tasks. We provide additional visualizations for other datasets in Appendix HH.5] which confirm the above discussion and further suggest that conflicting gradients emerge frequently during the training of neural MTPP models.
ii practice conflicts are computed layer-wise for each layer of the model (e.g the weights W of fully-comnectdd layer).
4

---

Published in Transactions on Machine Learning Research (01/2025)
4 A Framework to Prevent Conflicting Gradients in Neural MTPP Models
Given the observations from the previous section, our goal is to prevent the occurrence of conflicting gradients during the training of neural MTPP models with the NLL objective given in (2). To accomplish this, we first propose in Section 4.1 a naive approach that leverages duplicated and disjoint instances of the same model. Then, to avoid redundancy in model specification, we introduce in Section 4.2 novel parametrizations for
neural MTPP models. We finally show in Section 1.3 how our parametrizations enable disjoint modcling and training of the time and mark prediction tasks.
Existing MTPP models generally fall into three categories based on the chosen parametrization: (1) Intensity- based approaches that model the marked intensily function (1; 0) Zuuo el al. 2020), (2) Densily- bused approaches that model the joint density f k; O) (Shchur et al. (020). and (3) Compensator-based approaches that model the marked compensators (t; 0) (Omi ct al. 2019). We denote these different approaches as joint paramelriaations because they define a lunction that involves both the arrival time and the mark. To enable disjoint modeling of the time and mark prediction tasks, we consider the factorization of these functions into the products of two components: one involving function of the arrival-times, and the other involving the conditional PMF of marks:
x1(1;) = (1; O)p (M|1; 0) di OA) 0),
(4)
(t; = (t; 0)p' (k|t; 0).
(5)
()(t;0) [" (s; 0) 0)p (k|ss; 0sds
(6)
where (1; ((:00) (C0)) and A (1( = Mr, (s; dds are respectively the ground intensity and the ground
compensalor of the process. Similarly to 2 two-task NLL objective involving the r.h.s of expressions 4
and (6) can be derived, where task Tr now consists in learning either (t; A (t; 0). We provide the expression of these two-task NLL objectives in Appendix (A). The factorizations presented in expressions [4]
and 5 show that, to enable disjoint modeling of time and mark prediction functions, we need to specily p" (A:lt; Ow)) with parameters Oas, and either (t; 0r), X Or), or A²(1; with parameters 0r. As a naive approach, we first explore obtaining these functions from duplicated instances of the joint parametrizations (t(t: 0), (t: 0) or (t, , k;; 0), as presented next.
4.1 A Naive Approach to Achieve Disjoint Parametrizations
Consider a model that parametrizes (t; 0), such as THP (Zuo et al. 2020). To obtain a disjoint parametriza- tion of X (1; fr) and p" (k|t; 0,,, we can parametrize two identical lunctions (E( and (%(1; 0xx) from the same model and for all à E K, where Or and Os are disjoint set of trainable parameters. From these two functions, we can finally derive (1; and p" (k:|4; 0m) as
A% (1: OM) Σ X (1; and p" p'(111;0s))
K
(l;
E X (l:
effectively defining the desired disjoint parametrization. In the presence of conflicts during training, we can show that a gradient update step for the shared model Ag 0) leads to higher loss compared to the duplicated model in 7 with disjoint parameters Or and 0 Indced, suppose that 0, 0r and O all initialized with the same 6* at training iteration N. Assuming that Lr and Las are differentiable, let
gr VOLT (0") and gM VoLM (0*).
Denoting ØтM as the angle between gr and gM, we have the following corollary of Theorem 4.1. from Shi et al. 2023
Corollary 1. Assume that Lr and LM are differentiable, and that the learning rate a sufficiently small. If cos 9TM 0, then L 1((0)+1,00 00+11)) < L(0++1).
5

---

Published in Transactions on Machine Learning Research (01/2025)
This result essentially indicates that model trained with disjoint parameters leads to lower loss after gradient update if conflicts arise during training, i.e. cos ΦтM < O. We provide the proof in Appendix C Naturally, expression 7 and Corollary | remain valid for models that parametrize (t; or f (T, 0), as these functions can be uniquely retrieved from (t 0).
4.2 Novel Disjoint Parametrizations of Neural MTPP Models
In this section, we introduce an alternative approach to 7 to achieve disjoint parametrizations of Tr and TM- Specifically, we introduce novel parametrizations of existing neural MTPP model that directly parametrize p" (k:tt; a and either (t; 0r), (t;t0rr) or (t; thereby avoiding the unnccossary redundancy in model specification required by the method in (7). For query time / 2 4411, we define history representation h ENC((e1, .... e.-1):0,,
Rd where ENC(- ;0).) denotes the history encoder with parameters 0n, The encoder ENC(- ; 0x) is general and encompasses any encoder architecture typically found in the neural MTPP literature, such as recurrent neural networks (RNN) (uu et al. 2016 Shchur et al. 2020 or self-attention mechanisms Zuo et al. 2020 Zhang et al. 2020
A general approach to model the distribution of marks. Given a query time t 2 t;-1 and its corresponding history representation h, we propose to define the conditional PMF of marks (k]];; 0 using the following simple model:
p" (k) = 0s 0s0 (W20R (W1 [h|[log(T]] + b,)) + bz),
(9)
where = 1- 4-11; so is the softmax activation function, Wi (ddxx(d++1), bi E Rdi, W₂ RKxd, b2 E CRK and || mcans concatenation. Here, ONs (W1, W2, b1, b2, 0x) Despite its simplicity, this model is flexible and capable of capturing the evolving dynamics of the mark distribution between two events. Note that equation 9 effectively captures deeendeeeesssss between arrival times and marks. Moreover, by removing log(t- - 1 from expression
19, we obtain a PMF of marks that is independent of time, given Hr.
Intedsity-based parametrizations. We first cousider MTPP models specified by their marked intensities, namely THP (Zuo et al., 2020) and SAHP (Zhang al. 2020). We propose revising the original model formulations to directly parametrize (t), while p tk( systematically derived from expression (
Furthermore, although RMTPP (Du et al. 2016) is originally defined in terms of this decomposition, we extend the model to incorporate the dependence of marks on time.
SAHP+. While the original model formulation parametrizes Xt),,, we adapt its formulation to define:
x (t: = - [os (++ (17 (x)exp(-<l f(-1)))],
(10)
where 1 E C is vector of I's allowing to define the ground intensity as sum over C different, representations. In [10 ( h), n == (W,h) and y oc W W.) where as and ag are respectively the softplus and
G activation functions (Hendrycks & Gimpel 2023 W p. W,, W,, E RCxd, are learnable parameters
and 0r = (W #: W,, W,)
THP+ Following a similar reasoning, the original formulation of THP is adapted to model X*()) instead of (t):):
(t; - as wt l 1- 4:-1 + Wh + wh+b)]
(11)
4:-1
where w, E RRC W E RCx,, b Rd, and Or (W, wi, b. O.).
RMTPP+. We retain the original definition of X (t) proposed in RMTPP:
xt =exp wr(t 1--1) + wTThh h + b).
(12)
where W. R,, wr E RR b e R. and 0T (wr, wn, 0z)- A major difference between the original formulation of RMTPP and our approach lies in p² (k/T: QM) being defined by (h) which alleviates the original model assumption of marks being independent of time given Hi-

---

Published in Transactions on Machine Learning Research (01/2025)
Density-based parametrizations. The decomposition of the joint density in expression 5 has previonsly been considered by (Shchur et al., (020) with the LogNormMix (LNM) model. However, similar to RMTPP, this model assumes independence of marks on time, given Hz. In LNM we relax this assumption by using expression to paramotrize p" (k:tt), while maintaining a mixture of log-normal distributions for (T):::
M (T; = Σ m==
(log T - pm))
p" (m)
exp
(13)
mo mo 2T
20
where Pm (W + by)m, p" (m) 0z (W pph phphhbbbp bp) m. om exp(W,.t + b m with W ps WH: W. E Mxdx and bp, by. by E RM, M being the number of mixture components. Here 0r ( W ps Wa, by bp. b, 0n). Again, parametrizing the mark distribution using 9 relaxes the original assumption of marks being indepen- dent of time given Hz-
Compensator-based parametrizations. Integrating compensator-based neural MTPP models into our framework requires to define A' (t; 0r,, this way retrieving the decomposition in the r.h.s. of [ Specifically, we extend the improved marked FullyNNN model (FNN) (Omi et al. 2019 Enguehard et al. 2020 into FNN+, that models (t) and p" (kk(),, instead of (t):
A (1; = -
G (11-)),
(14)
(t) = 1 [os (W(oas where W [Cxd, wi, bj RRd, Wn E 12dh xdn, b2 RR, and aas is the Gumbel-softplus activation function. Here, Or (W, WA, wi, b1, b2, 0.). Similarly to the previous models, use ( to define p" (k[tt.
t.-1)++ + With + b,) + bb)]]
(15)
Training different history encoders. The different functions defined in this section, p" (k|T; 0Mr), (T; 0r), X (t; 0T), and A (t; 0r), share a common set of parameters 0, through a common history representation h. To enable fully disjoint modeling and training of the time and mark predictive functions, we define two distinct history representations:
hr ENCr (e Rdh and hMs E (: E Rd","
(16)
where ENCT(: 1)),)) and ENCar(: 0,,,) are the time and mark history encoders, respectively, while θ and AM. represent the sets of disjoint learnable parameters of the two encoders. By using hr for (T;0r), X (1tt; and A (t; fr), and hM for p" (k]T; 0as F we have defined completely disjoint parametrizations of the decompositions in and (5) Using separate history encoders further enables the model to capture information from past event occurrences that are relevant to the time and mark prediction tasks separately. In this paper, without loss of generality, we compute hy and hMs by training two GRU encoders that sequentially process the set of event representations in (ci...., .e.....,e: e-11]
4.3 Disjoint training of the time and mark tasks.
Let us model OT) from 13 and p" (k(r;; OM) from 9 using distinet history encoders such that Or and fι are disjoint set of trainable parameters. By injecting these expressions in find that the NLL now is sum over two disjoint objectives Cr S) and LM (M S), i.e.
C(0r,0N:5)
1 - 1=1
Σ
n:
log ("(,, log(1 - (T;
L
TEE
log p" (kr,s
1=1
(r(Or.S)
Exr (S)
(17)
meaning that the associated tasks T and TM can be learned separately. This contrasts with previous works, such as (Shchur et al., 2020) and (Zuo et al., 2020), in which shared parameters between f and p* does not allow for disjoint training of ( In our implementation, we minimize expression (17 through a single
40,, now replaced by fr. in fr. and by 0M 0M-
7

---

Published in Transactions on Machine Learning Research (01/2025)
pipeline by specifying different carly-stopping criteria for C1 (0r, and LM Note that similar
decomposition can be obtained from any of the parametrizations presented in Sections 4.1 and 4.2 Finally, we would like to emphasize that disjoint training of tasks TT and Ta through (17) does not imply independence of arrival-times and marks given the history. In fact, in our parametrizations, this dependency remains systematically captured by (9)
5 Related Work
Neural MTPP models. To address the limitations of simple parametric MTPP models (Hawkes 1971
Isham & Westcott 1979h, prior studics focused designing more flexible approaches by leveraging recent advances in deep learning. Based on the parametrization chosen, these neural MTPP models can be generally classified along three main axis: density-based, intensity-based, and compensator-basel Intensity- approaches propose to model the trajectories of future arrival-times and marks by parametrizing the marked intensitics (t).). In this line of work, past event occurrences are usually encoded into a history representation
using RNNs (Du et al. 2016 Mei & Fisner 2017 Guo et al. 2018b Türkmen et al.,
2019: Biloš et al.
2019
Zhu et al. 2020 or self-attention (SA) mechanisms (Zuo et al. 2020 Zhang et al.
2020 Zhu ct al.
2021
Yang et al. 2022 Li et al., 2002).. However, parametrizations of the marked intensity functions often come at the cost of being unable to evaluate the log- ligllekei in closed- form, requiring Monte Carlo integration.
This consideration motivated the design of compensator-based approaches that parametrize Ag(t) using fully-connected neural networks Omi et al. (2019), or SA mechanisms (Enguehard et al.] 2020), from which (t( can be rieved through differentiation. Finally, density-based approaches aim at directly modeling the joint density of (inter-)arrival times and marks f*(r, k). Among these, different family of distributions have been considered to model the distribution of inter-arrival times (Xiao et al. 2017
Lin ct al. 2021 b.
Notably [Shchur el al. (2020) relies on a mixture of log-normal distributions (Shehur et al.
2020 to estimate 2021 However,
(T()), model that then appeared in subsequent works (Sharma et al. 2021 Gupta et al.
the original work of Shchur et al. (2020) assumnes conditional independence of inter-arrival times and marks given the history, which is alleviated in (Waghmare et al. 2022). Nonetheless, common thread of these parametrizations is that they explicitly enforce parameter sharing between the time and mark prediction tasks. As we have shown, this often leads to the emergence of conflicting gradient during training, potentially hindering model performance. For an overview of neural MTPP models, we refer the reader to the works of Shehur et al. 2021), (Lin al.) 2022) and (Bosser & Ben Taieb 2023).
Conflicting gradients in multi-task learning. Diverse approaches have been investigated in the literature to improve interactions between concurrent tasks in multi-task learning problems, thereby boosting performance for each task individually. In this context, a prominent line of work, called gradient surgery methods, focuses on balancing the different tasks at hand through direct manipulation of their gradicnts.
These manipulations either aim at alleviating the differences in gradient magnit between tasks Chen
et al.] 2018 Sener & Koltun
2018
Liu et al.] 2021c), or the emergence of conflicts (Sinha et al. 2018
Maninis et al.] 2019 Yu et al.
2020 Chen al. 2020: Wang et al. 2020 Liu al.] 2021a Javaloy & Valera
2022 Alternative approaches to task balancing have been explored based on different criteria, such as task
prioritization (Guo et al.
2018a), uncertainty (Kendall et al. 2018), or learning pace (Liu et al. 2019). Our methodology relates more to branched architecture search approaches (Guo et al. 2020 Bruggemanu et al.)
2020: Shi et al. 2023), where the aim is set on dynamically identifying which layers should or should not be shared between tasks based on a chosen criterion, e.g. the proportion of conflicting gradients. Specifically, Shi et al. (0023) recently showed that gradient surgery approaches for multi-task learning objectives, such as GradDrop (Chen et al., 22020), PCGrad (Yu et al., 2020), CAGrad (Liu al., (021aa), and MGDA (Sener & Koltun 2018), cannot effectively reduce the occurrence of conflicting gradients during training. Instead, they propose to address task conflicts directly from the root by turning shared layer into task-specific layers if they experience conflicting gradicnts too frequently. Inspired by their success, our framework follows a similar approach: the time and mark prediction lasks are parametrized on disjoint set of trainable parameters to avoid conflicts from the root during training. We want to emphasize that our goal is not to propose a gencral-purper gradient surgery method to mitigate the negative impact of conflicting gradients. Instcad, we want demonstrate that conflicts can be avoided altogether during the training of neural MTPP models by adapting their original parametrizations.
8

---

Published in Transactions on Machine Learning Research (01/2025)
Base
(b) Base+
(c) Base+
(sC())p) 0t,
[NCY):0oa) (FC(;)0)
(1; 01 0PA))
H4 - [NC(: 8.) (EC(: 0,
Hi + ( ENC(=;0) 0.))
Hi ENC/ (ys)) (ac(: p- (kit: OMa.)
[BC(::u) - (Aitt O2s, 0n)
Figure 3: Graphical representation of the base, '+', and '++" setups.
6
Experiments
Datasets and baselines. We conduct an experimental study to assess the performance of our framework in training the time and mark prediction tasks from datasets composed of multiple event sequences. Specifically, explore the various novel neural TPP parametrizations enabled by our framework, detailed in Section
4. These are compared to their original parametrikations We use five real-world marked event sequence
datasets frequently referenced in the neural MTPP literature: LastFM (Hidasi & Tikkk 2012). MOOC, Reddit (Kumar et al. 2019) Github (Trivedi et al. 2019), and Stack Overflow (Du et al. 2016). We provide descriptions and summary statistics for all datasets in Appendix [ We consider as base models THP Zuo et a 2020); SAHP (Zhang al.) 2020). RMTPP (Du et al. 2016), FullyNN (FNN) (Omi et al.
2019), Morm (LNM) (Shchur al. 2020 and SMÜRF-THP (STHP). To highlight the different. components of our framework that lead to performance gains, we introduce the following settings:
(1) Shared History Encoders and Disjoint Decoders. A common history embedding, denoted as h is used. while the two functional terms from equations 44) and (5) are modeled separately as detailed in Section 4.2 Here, the functions (T; 0r), (t; 0r); A (t; 0r) and [k(kTT;0x) share common parameters via h. Models trained in this setting are indicated with a '+' sign, e.g. THP+.
(2) Disjoint History Encoders and Disjoint Decoders. In contrast to the previous configuration, distinct history embeddings, hr for time and hM for marks, p' (k]T; 0Mr separately. This separation allows for the independent training of the time prediction and mark prediction tasks as described in Section 4.3 Models trained within this setting are labeled with a "++' symbol, e.g., ++
used to define f (T: 0r), Xt (t; (1:1; and
Compared to the base models, these configurations allow us to assess the impacts of (1) isolating the parameters for the decoders in the time and mark prediction tasks, and (2) using distinct history embeddings for each task, enabling fully disjoint training. A graphical illustration of these configurations is shown in Figure We will often refer to these sctups as base, basc+, and base+ throughout the text. The distinction between LNM (RMTPP) and LNM+ (RMTPP+) stems from the modeling of the PMF of marks using our model in 19, which relaxes the conditional independence assumption inherent in the base model. To maintain a fair comparison, wo ensure that each configuration controls for the number of parameters, keeping them roughly equivalent across settings to confirm that any observed performance improvements are not merely due to increased model capacity. Finally, all models are trained to minimize the average NL. given We provide further training details in Appendix B
Metrics. To evaluate the performance of the different baselines on the time prediction task, we report the C term in (17g computed over all test sequences. Following (Dheur Ben Taieb 2023), we also quantify the (unconditional) probabilistic calibration of the fitted models by computing the Probabilistic Calibration Error (PCE). Finally, evaluate the MAF in event inter-arrival time prediction. To this end, we predict the next 7 as the median of the predicted distribution of inter-arrival times, iee. F (F*))-(0055 07), where the quantile function (F*)-1( (=; ;r) is estimated using a binary search algorithm.
Similarly, for the mark prediction task, we report the average Las term in 17 and quantify the probabilistic calibration of the mark predictive distribution by computing the Expected Calibration Error (ECE) Naeini et al. 2015), and through reliability diagrams (Guo et al. 2017 Kuleshov al. 2018). Additionally, by predicting the mark of the next event as k argmax p² TK we can assess the quality of the point kek
predictions by means of various classification metrics. Specifically, we compute the Accuracy@n for values of n in (1,3, 5) and the Mean Reciprocal Rank (MRR) (Craswell 2009) of mark predictions. Lower Lr, LM
PCE and ECE is better, while higher Accuracy@n and MRR is better.
Forr the remainder of this paper, these models will be referred to as "base models".
Note that for the base and base+ methods, the two terms 17 are functions on shared parameters.

---

Published in Transactions on Machine Learning Research (01/2025)
CB 60 GH53 TRA
B.CA GN T CGG 0M5-6 37 TRL6-66
06:301 0P
Clc 99 G.6 TP 62
CG.0 50 0960 50 TRA
CG GNG TR:0.2
0G0.6 GH6.077 TRI3-47
B.CG SM6-153 TRI6.66
THP ENC
THP DEC
SAHPIENC
SAMP DEC
FNN ENC
DEC
laxFF
Proportion
COS
cor
005 6PM
CO5
1PE6-77
GMS
0060, 6.5 12
CO ON5O 32 1966.)
66 AMKLEAAR THASS
O00 54 6M5.0R2 TRIaSa
THP-ENC
THR.DEC
SAHP-ENC
SAMP-DEC
DEC
Proportion
MDOC
CO5 PMM4
COC
005
005 PM
CSG
Figure 4: Distribution of cos ΦTM during training at the encoder (ENC) and decoder (DEC) heads for THP, SAHP and FNN in the base and base+ setup on LastFM and MOOC. "B" and "+" refer to the base and base+ models, respectively, and the distribution is obtained by pooling the values of øras over 5 training runs. As the decoders are disjoint in the base+ setting, note that cos ΦтM is not defined.
6.1 Results and discussion
We present the Lr and Las metrics for the base, base+, and b configurations across all datasets in Table [ The PCE, MAE, ECE, MRR, and Accuracy@1335 metrics, which reflect the subsequent discussion, are given in Appondix H
Distinet decoders mitigate gradient conflicts. Based on the LT and LM metrics, we note a consistent. improvement when moving from the base to the base+ setting for THP, SAHP, and FNN. This underscores the benefits of using two distinct decoders for time and mark prediction tasks with base+, leading to improved predictive accuracy compared to the base models. Figure 3 shows the distribution of cos dras during training for THP, SAHP and FNN for both base and base+ on the LastFM and MOOC dataset, along with the average GMS and TPI for conflicting gradients. We would like too emphasize that both base and base+ share the some encoder architecture, which allows for a direct comparison of the distribution of cos ΦтM between the two settings during training. Appendix H provides detailed visualizations for other baselines and datasets. With the base model, a significant proportion of severe conflicts (as indicated by cos ΦтM in [-,, -05)) is often observed for the shared parameters of both encoder and decoder heads, typically with low GMS values. Additionally, with the base model, the TPI values suggest that these conflicting gradients at the encoder heads predominantly favor Tr (i.c., TPI > 0.5). In contrast, basc+ inherently prevents conflicts at the decoder by separating the parameters for each task. During training with base+,, there is also a noticeable reduction in the severity of conflicting gradients for the shared encoder parameters, as evidenced by a more concentrated distribution of cos ØтM around O. Moreover, the TPI values indicate that base+ generally achieves a more balanced training between both tasks, which further contributes to enhancing their individual performance. While we note that. the GMS values do not consistently improve between the base and base+ settings, improvements with respect to LT and LM suggest that this effect is offsset by reduction in conflicts during training. Finally, while LNM and RMTPP already avoid decoder conflicts in their base models by decomposing the parameters, explicitly modeling the dependency of marks on time with base+ further enhances mark prediction performance.
Disjoint training enhances mark prediction accuracy. Returning to Table [ moving from the base+ the base+ setting often leads to improvements in the Eys metric for most baselines, while the Cr metric generally remains comparable between the two configurations. Although conflicting gradients are typically reduced when moving from base to base+, this pattern indicates that the residual conflicts primarily hinder the mark prediction task. In contrast, base+ effectively eliminates these conflicts by using distinct history representations for each task. A significant advantage of base+ is that it allows one task to continue training after the other has reached convergence. For example, Figure 5 illustrates the validation losses LT and Lm for SAHP++ on MOOC. Thanks to disjoint training, the LM metric
10

---

Published in Transactions on Machine Learning Research (01/2025)
10
FIN
Confidence
Confidence
1.0
Accuracy
Prequency
64 30
63 34
36
04
36
92 04
66 68 59
04 24
1.5 06
92
04
86 02
34
Confidence
Confidence
Figure 6: Reliability diagrams of p (klT; 0) (left) and (T; (right) for the base models and their "++' extensions on the LastFM dataset. Frequency and accuracy aligning with the black diagonal correspond to perfect calibration. The reliability diagrams are averaged over 5 splits, and error bars refer to the standard error.
Table 1: Cr and LM results of the different sctups across all datascts. The values are computed over splits, and the standard error is reported in parenthesis. Best results are highlighted in bold.
FM Gitbab
LaxFFM
MO0C
Reddit
Stack O.
MOOC
Github
Reddit
Stack O.
THP
714.0 (16.5) 93.3 (1:4)
(27.3
30.9 (1-3)
105.3 (0-7)
THP
-945.1 (41-3) -135.8 8 (1:3)
.242.5 5 (36.5)
-72 (7.33 (2.3) -84.0 (1
THP-
695.8 (15-7)
76.9 (1:2) 120.5 (16.0)
43.7 (1:2)
183.3 (0.7)
THP-
-981.1
(45.6)
-130 6 (1:3)
.25577 7 (42.0)
-55 (2.3)
-M..7 (1.4)
THP 4
061.1 (18.1)
70.9 (1.0)
112.1 (18.4)
40.4 (1.3)
108.0 (0.7)
THP +
-1087.3 (47.0)
-186.0 (2.1)
-271.0
(50.8)
-87.9 (2.0)
-84.2 (1.4)
4THIT
(1.1)
(0.7)
STMD4
(32.2)
(1.4)
STHP
096.5 (15.9)
(1.4)
114.7
(14.0)
106.0 (0.6)
STHP
-1014.0
(42.1) -182.0
-27700
(2:1)
.83.3 (1.4)
SAHP
825.5 (25.6)
169.0 (2.2)
138.0 (39-1)
77.9 (4:7)
108.4 (1:0)
SAHP
-1285.8
(57.9) -208. 0 (3.5)
-366.3 5 (57-4)
-72.9
(1:8)
-80-7 (1:4)
SARP4
740.0
(24.6)
73.8 (1.0)
116.8 (15.9)
48.4 (1.3)
183.3 (0.7)
SAHP+
-1820.4
(58.4)
-258 8 58:5)
.358 1 (67 0)
.94
(3:3)
.80.0 (1.4)
SAMP++
654.8
(17.3)
71.0 (1:1)
114.1
(15.2)
40.5 (0.6)
103.1 (0.7)
SARP+
-1320.0
(60.5)
-293.8
(3.0)
-366.0
(50.5)
-05.4
(2-1)
-77.4 (1:9)
LNM
655.2
(15.8) 86.6 (1:2)
116.5 (15
43.4
106.5 (0.7)
LNM
-1326.3
(55.8)
-310.0 0 (3.6)
-390 (59.8)
-96.4
(2.0)
-91.0
(1.0
LNM4
668.6 (15.8)
(1:3)
112.3 (35.1)
41.4
103.2 (0.7)
LNM+
-1320.4
(60.5)
-310.6
-37877
(59.4)
-96.4 (2.0)
-91.1 (1.5)
LNM++
B37.2
(19.4)
(1.1)
111.5
(15.2)
40.8
103.2 (0.7)
LNM++
-1334.7
(58.5)
-307
(3.9)
-381.2
(59.9)
-96
(2.2)
-90.5 (1.4)
PXN
739.5
(25:2) 78.8 (1:3)
113.5 (35.0)
47.0 (1.3)
107.3 (0.6)
PNN
-1276.2
(38.6) -280.
(3.2)
-383 (37.2)
-75 3 (2:5)
-81.2 (1:5)
FNN4
672-1
(17.9)
72.3 (1:1)
111.6 13
41.2 (0.9)
183.2 (0.7)
PNN+
-1324.8
(59.6)
-302.2 (3.6)
-3813 (b7.0)
-94.4 (2.1)
-90.0 (1.8)
F
(10.2)
(1:0)
109.5 (15.0)
40.1 (1.0)
103.1 (0.7)
FNN++
-1324.2 (08.1) -300 (3:7)
.365.0
(50.7)
-00.1
(2:2)
(1.4)
RMTPP
654.6 (15.6)
87.0 (1.2)
126.4 (18.3)
41.4 (0.9)
106.5 (0.7)
RMTPP
-1052.9
(48.0)
-178.8
(1.8)
-208.0 (64.3)
-88.0 (2.2)
-83.3 (1.4)
RMTPP4
661
(16.3)
74.8 (1:3)
115.7 (15.4)
22.0 (1.0)
103.0 (0.7)
RMTPPP
-1040.7
(15.5)
-187.8 (3.1)
.279 3 (49.0)
-S7 (2.0)
-83.4 (1.4)
AMTPP++
654.3 (16.7)
71.4 (1.1)
112.6 (15.3)
40.6 (1.2)
183.1 (0.7)
RMTPP44 ++
-1071.1
(50.3)
-182.2 (1.9)
-287.2 (52.6)
-56.0 (2.0)
-82.8 (1:4)
can be further optimized for additional epochs after training of the fr metric ceases due to overfitting, thus achieving gains in mark prediction performance. This fcature is absent in base and base+,, where both LM and Lr metrics rely on a shared set of parameters. In Figure 5 Or is fixed after the vertical red line, resulting in a constant validation L for the remaining training epochs of the model.
90
140
130
110
Roanc 6,
120
100
Reseme
-1 33
80
Finally, referring back to the preliminary experiment on Figure 2 the results with respect to Lr and Lm in Table [ suggest that conflicting gradients are mostly detrimental to model per- formance if (1) they are in great proportion during training (i.e. high CG) and (2) if they are associated to low GMS values. For instance, on Figure 2 THP and SAHP exhibit high CG associated to high GMS values, whereas the remaining models conversely show a more balanced CG, but with lower GMS values. We find that model performance often improves both these scenarios when preventing conflicting gradicnts altogether in the base++ setting.
200 Epach
400
200
400
Epoch
Figure 5: Validation curves of the LT and La components for SAHP++ on MOOC.
Reliability Diagrams. Figure 0 presents the reliability diagrams for the predictive distributions of arrival- times and marks for base and base+ on LastFM. The diagrams show that the basc+ models are generally better calibrated than their base counterparts, as evidenced by the bin accuracies aligning more closely with the diagonal. This improvement aligns with the results in Table [ where a lower Cas indicates better accuracy. However, improvements in the calibration of arrival times between base and basc+ models are less noticeable, suggesting that conflicting gradients during training predominantly affect the mark prediction task. We provide reliability diagrams for other baselines and datasets in Appendix H
11

---

Published in Transactions on Machine Learning Research (01/2025)
CG 60 GH53 de TRA Di 060 52 GNG0058 22
C. GN T D: CGO So 045-6 d8 TR
06:351 0P6 T4:34
CG.0 50 0960 50 TRA
CG GNG TR:0.2
D: 0.066.99 6M6.071 TRUO-
B.CG SM6-153 TRI6-66
THP ENC
THP DEC
SAHPIENC
SAMP DEC
FNN ENC
FNN DEC
lasFF
Proportion
COS PPM
cor
005 6PM
CO5 $MM
615- 1P50.47 Osee 39 69 19
Di C 6M5 19E6.03
66 AMKCEAAR THASS
OS0 32 6MS.08 TRiA 36
THP-ENC
THR.DEC
SAHP-ENC
FNN-ENC
DEC
Proportion
MDOC
0O5 PMM4
COC
005
005 PM
CSG
Figure 7: Distribution of cos ΦTM during training at the encoder (ENC) and decoder (DEC) heads for THP, SAHP and FNN in the base and base-D setup on LastFM and MOOC. "B" and 'D' refer to the base and base-D models, respectively, and the distribution is obtained by pooling the values of Φтм over 5 training runs. As the decoders are disjoint in the base- setting, note that cos ΦTM is not defined.
Table 2: LT and LM results for the base, base-D, and base-DD settings across all datasets. The values are computed over 5 splits, and the standard error is reported in parenthesis. Best results are highlighted in bold.
LastFM
MOOC
Reddit
Stack O
LastFM
MOOC
Gitbubb
Reddit
Stack
THP
714.0 (16.5) 98.3 (1.4) 128.3
(17.3)
39.9 (1.3) 108 (0.7)
THP -945.1
(41.3) -135.9
(1:3) 222.8 (38.5) -72.3 (2:3) .44.0 (1.4)
THP-D
677.1
(10.4)
(1:2)
121.4
(10.8)
30.1 (0.0)
104.7 (0.7)
THP-D -905.1
(50.2)
-104.8
(1.8)
228.0
(47.2)
-0.3 (1.9)
.85.3 (1.4)
THP-DD 007.2 (10.2)
70.8 (1.2)
118.5
(14.0)
38.2
(1.0)
104.4 (0.0)
THP-DD -1023.8 (53.0)
-102.1 (4:3)
27000 (55.2)
-2.0 (2.1)
-84.8 (1:4)
SAHP
825.5 (25.6)
163.0 (2.2) 139.0 (19:1)
77.8 (4.7) 105.4 (1.0)
SAHP -1253.8
(57.9) -200.0 (3.5) -346.5 (07.4) .72.9 (1.8)
-99.7 (1.4)
SAHP--
632.1
(32.0)
83.3 (1:5)
128.8
(18.5)
56.0 (1.0)
105.1 (0.7)
SARP-D -1318.1 (57.9)
-286.7 (3.7) -308-7 (58.3)
-82.9 (2.3)-80.1
(1.3)
SAHP-DD 692.1 (19.7)
89.0 (1-4)
115.4
(15.3)
51.0 (0.5)
101.8 (0.6)
SAMP-DD-1319.88 (39.3)-294.0 (3.6) (366)-66.8 (59.1) -87.0 (3.5) -87.1 (2.0)
739.5
(25-2)
113.5
(15-4)
47.0
(1.4)
(0.0)
-1276.2 (58.6) -280.9
-363.1
(57:2)
(2:5)
(1:3)
FNN-D
(19-7)
T6.6
(1-2)
(15.4)
(0.7)
FNN-D
(39.7)-286.4
(3.5)
FNN-DD
670.0 (18.1)
111.4
(1.6) 108.7
FNN-DD -1294.4 (57.0) -286.3
.362.6 (56.4)
-82.7 (2.2)
-91.4 (1.4)
Isolating the impact of conflicts on performance. Our experiments reveal that the base+ and base+ settings result in decrease of conflicting gradients and to enhanced performance with respect to the time and mark prediction tasks. However, for THP, SAHP and FNN, these scttings do not enable to disentangle performance gains brought by decrease in conflicts from the ones brought by modifications of the decoder architecture. To address this limitation, we introduce the following two settings based on the duplicated model approach of Section 4.1
(1) Shared History Encoders and Duplicated Decoders. The functions (1; OT) 0r) and p (k|t; are obtained through 77) from two identical parametrizations of the same base decoder. However, similar to the base+ setting, a common history encoder h is used, meaning that these functions still share parameters via On Models trained in this setting are indicated with a "DD" sign, c.g. THP-D.
(2) Disjoint History Encoders and Duplicoted Decoders. This setting differs from the previous one in the use of two distinct history embeddings hy and hM in 77), implying that (t; 0r) (t; 07) and p" (k]t; O are now completely disjoint parametrizations. We use the label ''DD' to denote the models trained in this setting, e.g. THP-DD.
In contrast to base+ and base++, the base-D and base-DD settings retain the same architecture as the base model, enabling to directly evaluate the impact of conflicting gradients on performance. In Table report the performance with respect to Lt and Ls for THP, SAHP, and FNN trained in the base, base-D and basc-DD settings. We follow the same experimental sctup as before, and maintain the number of parameters comparable between the different settings for fair comparison. We almost systematically observe improvements on both tasks when moving from the base to the base-D or base-DD settings. Moreover, these performance gains are often associated to a decrease in (severe) conflicts during training, as shown on Figure 7. Furthermore, when comparing the results between Tables D and 2 we note that our base+ and basc++ parametrizations often show improved performance compared to the base-D and base-DD DD settings, especially
12

---

Published in Transactions on Machine Learning Research (01/2025)
the mark prediction task. This highlights that the benefits of our parametrizations extend beyond the prevention of conflicts to achieve greater predictive performance. We provide more visualizations in Appendix
H.6
7 Conclusion, Limitations, and Future Work
Learning a neural MTPP model can be essentially interpreted two-task learning problem, in which one lask is focused on learning a predictive distribution of arrival times, and the other on learning a predictive distribution of event types, known as marks. Typically, most neural MTPP models implicitly require these of trainable parameters. this paper, we demonstrate that this parameter
two tasks share a common sharing leads to the emergence of conflicting gradients during training, often resulting degraded performance each individual task. To prevent this issue, introduce novel parametrizations of neural MTPP models that enable separate modeling and training of each task, effectively preventing the occurrence of conflicting gradients. Through extensive experiments on real-world event sequence datasets, we validate the advantages of our framework over the original model configurations, particularly in the context of mark prediction. However, we acknowledge several limitations in our study. Firstly, our focus was solely on categorical marks. Investigating conflicting gradients in more complex scenarios, such as temporal graphs (Trivedi et al. 2019
Gracious & Dukkipati 2023) spatio-temporal point processes (Zhou & Yu 2023; Zhang al.) 2023
presents promising avenue for future research. Secondly, analysis limited neural MTPP models
trained using the negative log-likclihood. Extending our framework to other proper scoring rules (Brehmer
et al. 2023 is also a potential area for future exploration.
13

---

Published in Transactions on Machine Learning Research (01/2025)
References
Emmanuel Bacry, Iacopo Mastromattco, and Jean-François Muzy. Hawkes processes in finance. ArXi preprinl, 2015.
Souhaib Ben Taieb. Learning quantile functions for temporal point processes with recurrent neural splines. Proceedings of The 25th International Conference on Artificial Intelligence and Statistics (AISTATS), 2022.
Marin Biloš, Bertrand Charpentier, and Stephan Günnemann. Uncertainty on asynchronous time event prodiction. Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS) 2019.
Tanguy Bosser and Souhaib Ben Taieb. On the predictive accuracy of neural temporal point process models for continuous-time event data. Transactions of Machine Learning Research (TMLRR), 2023.
Jonas Brehmer, Tilmann Gneiting, Marcus Herrmann, Warner Marzocchi, Martin Schlather, and Kirstin Strokorb. Comparative evaluation of point process forecasts. Annals of the Institute of Statistical Mathematics, 76:47-71, 2023.
Glenn W. Brier. Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1): 1-3, 1950.
David Bruggemann, Menelaos Kanakis, Stamatios Georgoulis, and Luc Van Gool. Automated search for resource-efficien branched multi-lask networks. Proceedings of the 31st Brilish Machine Vision Conference (BMVC), 2020.
Zhao Chen, Vijay Badrinarayanan Chen- Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. Proceedings of the 35th International Conference on Machine Learning (ICML), 2018
Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov. Just pick sign: Optimizing deep multitask models with gradient sign dropout. Proceedings of the 34th International Conference Newral Information Processing Systems (NeurIPS), 2020.
Nick Craswell. Mean reciprocal rank. Encyclopedia of Dulabase Systems, Springer, 2009.
Daryl Daley and David Vere-Jones. An introduction to the theory of point processes, volume i: Elementary theory and methods. Springer, 2003.
Daryl J. Daley and David Vero-Joncs. An introduction to the theory of point processes. volume ii: General theory and structure. Springer, 2008.
Victor Dheur and Souhaib Ben Taieb. large-scale study of probabilistic calibration in neural worw regression. Proceedings of the 40th International Conference on Machine Learning (ICML), 2023.
Nan Du, Yichen Wang, Niao He, Jimeng Sun, and Le Song. Time-sensitive recommendation from recurrent user activitics. Proceedings of the 28th International Conference Neural Information Processing Systems (NeurIPS), 2015.
Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodrigue,, and Le Song. Recurrent marked temporal point processes: Embedding event history to vector. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Dala Mining (KDD), 2016.
Mike Egesdal, Chris Fathauer, Kym Louie, Jeremy Neuman, George Mohler, and Erik Lewis. Statistical and stochastic modeling of gang rivalries Los Angeles. SIAM Undergraduate Research Online, 2010.
Joseph Enguchard, Dan Busbridge, Adam Bozson, Claire Woodcock, and Nils Y. Hammerla. Neural temporal point processes for modelling electronic health records. Proceedings of Machine Learning Research (PMLR), 136:85-1133 2020.
14

---

Published in Transactions on Machine Learning Research (01/2025)
Mehrdad Farajtabar, Yichen Wang, Manuel Gomez Rodriguez, Shuang Li, Hongyuan Zha, and Le Song. Coevolve: A joint point process model for information ion diffusion and network co-evolution, 2017. Journal of Machine Learning Research, 18, 1-49.
Manuel Gebetsbergora, Jakob W. Messner, Georg J. Mayr, and Achim Zeilois. Estimation methods for nonhomogeneous regression models: Minimum continuous ranked probability score versus maximum likelihood. Monthly Weather Reniew, 146((1))4323 4338, 2018.
Tilmann Gneiting and Adrian E. Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359-375 2007.
Tony Gracious and Ambedkar Dukkipati. Dynamic representation learning with temporal point processes for higher-order interaction forecasting. Proceedings of the 37th AAAT Conference on Artificial Intelligence (AAAI), 2023.
Chuan Guo, Gcoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration modern neural networks. Proceedings of the 94th International Conference on Machine Learning (ICML), 2017.
Michelle Guo, Albert Haque, De-An Huang, Serena Yeung, and Fei-Fei. Dynamic task prioritization for
multitask learning, Proceedings of the 2018 European Conference on Computer Vision (ECCV), 2018a.
Pengsheng Guo, Chen-Yu Lee, and Daniel Ulbricht. Learning to branch for multi-task learning. Proceedings of the 87th International Conference on Machine Learning (ICML), 2020.
Ruocheng Guo, Jundong Li, and Huan Liu. Initiator: Noise-contrastive estimation for marked temporal point process. Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI), 2018b.
Vinayak Gupta, Srikanta J. Bedathur, Sourangshu Bhattacharya, and A. De. Learning temporal point processes with intermittent obscrvations. Proceedings of The 24th International Conference on Artificial Intelligence and Slatistics (AISTATS), 2021.
Alan G. Hawkes. Point spectra of some mutually exciting point processes. Journal of the Royal Statistical Society: Series B (Methodological), 33(3):438 443, 1971.
Alan G. Hawkes. Hawkes processes and their applications to finance: A review. Quantitative Finance, 18(2): 193 198, 2018.
Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (GELUs). ArXiv preprint, 2023.
Balazs Hidasi and Domonkos Tikk. Fast
tensor factorization for context-aware recommendation from implicit. feedback. Proceedings of the 2012 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD), 2012.
Valcrie Isham and Mark Westcott. A sclf-corrocting point process. Stochastic Processes and Their Applications, 8(3):335-33) 347, 1979.
Adrián Javaloy and Isabel Valera. Rotograd: Gradient homogenization in multitask learning. Proceedings of the 2022 International Conference on Learning Representations (ICLR), 2022.
Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. Proceedings of the 2018 IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Proceedings of the 2014 International Conference on Learning Representations (ICLR), 2014.
Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon. Accurate uncortainties for deep learning using calibrated regression. Proceedings of the 35th International Conference Machine Learning (ICML), 2018.
15

---

Published in Transactions on Machine Learning Research (01/2025)
Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in temporal interaction networks. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2019.
Zichong Li, Yanbo Xu, Simiao Zuo, Haoming Jiang, Chao Zhang, Tuo Zhao, and Hongyuan Zha. SMURF- THP: Score matching- uncertainly quantification for transformer Hawkes process. Proceedings of the 40th International Conference on Machine Learning (ICML), 2023.
Haitao Lin, Cheng Tan; Lirong Wu, Zhangyang Gao, and Stan. 7. Li. An empirical study: Extensive deep temporal point process. Arx preprint, 2021.
Haitao Lin, Lirong Wu, Guojiang Zhao, Pai Liu, and Stan Z. Li. Exploring generative neural temporal point process. Transactions on Machine Learning Reseurch (TMLR), 2022.
Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Couflict-averse gradient descent for multi-task learning. Proceedings of the 35th International Conference on Neural Information Processing Systems (NeurIPS), 2021a.
Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. Proceedings of the 35th International Conference Neural Information Processing Systems (NeurIPS), 2021b.
Liyang Liu, Yi Li, Zhanghui Kuang, Jing-Hao Xue, Yimin Chen, Wemming Yang, Q, Liao, and Wayne Zhang, Towards impartial multi-task learning. Proceedings of the 2021 International Conference on Learning Representations (ICLR), 2021c.
Shikun Liu, Edward Johns, and Andrew J. Davison. End-to-end multi-tast learning with attention. Proceedings of the 2019 IEEE/CVF Conference Computer Vision and Pattern Recognation (CVPR), 2019.
Kevis-Kokitas Maninis, Ilija Radosavovic, and lasonas Kokkinos. Attentive single-tasking of multiple tasks. Proceedings of the 2019 IEEE/CVF Conference on Compuler Vision and Pallern Recognition (CVPR), 2019.
Hongyuan Mei and Jason Eisner. The neural Hawkes process: A neurally self-modulating multivariate point process. Proceedings of the 31st International Conference on Neural Information Processing Systems (NeurIPS), 2017.
Mahdi Pakdaman Naeini, Gregory F. Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning, Proceedings of the 29th AAAI Conference Artificial Intelligence (AAAI), 2015.
Yosihiko Ogata. Space-time point-process models for carthquake occurrences, 1998. Annals of the Institute of Statistical Mathematics, 50(2):379 402.
Takahiro Omi, Naonori Ueda, and Kazuyuki Aihara. Fully neural network based model for general temporal point processes. Proceedings of the 33rd International Conference on Neural Information Processing Systems (NeurIPS), 2019.
Jakob Gulddahl Rasmussen. Lecture notes: Temporal point processes and the conditional intensity function. ArXie preprint, 2018.
Sebastian Ruder. An overview of gradient descent optimization algorithuns. ArXie preprinl, 2017.
Ozan Sener and Vladlen Koltun. Multi-uask learning as multi-objective optimization. Proceedings of the 32nd International Conference on Neural Information Processing Systems (NeurIPS), 2018.
Karishma Sharma, Yizhou Zhang, Emilio Ferrara, and Yan Liu. Identifying coordinated accounts on social media through hidden influence and group behaviours. Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery and Data Mining (KDD), 2021.
16

---

Published in Transactions on Machine Learning Research (01/2025)
Oleksandr Shchur, Marin Biloš, and Stephan Günnemann. Intensity-free learning of temporal point processes. Proceedings of the 2020 International Conference on Learning Regresentations (TCLR), 2020.
Oleksandr Shehur, Ali Caner Türkmen, Tim Januschowski, and Stephan Gümnemann. Neural temporal point processes: review. Proceedings of 13th Joint Conference on Artificial Intelligence (IJCAI), 2021.
Guangyuan Shi, Qimai Li, Wenlong Zhang, Jiaxin Chen, and Xiao Mtno-Ming Wu. Recon: Reducing conflicting gradients from the root for multi-task learning. Proceedings of the 2023 International Conference on Learning Representations (ICLR), 2023.
Ayan Sinha, Zhao Chen, Vijay Badrinaran Badrinarayanan, and Andrew Rabinovich. Gradient adversarial training of neural networks. ArXiv preprint, 2018.
Trevor Standley, Amir R. Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? Proceedings of the 37th International Conference on Machine Learning (ICML), 2020.
Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. Representation learning over dynamic graphs. Proceedings of the 2019 International Conference on Learning Representations (ICLR), 2019.
Ali Caner Türkmen, Bernie Wang, and Alex Smola. Fastpoint: Scalable deep point processes. Proceedings of the 2019 European Conference on Machine Learning and Knowledge Discovery in Databases (ECML. PKDD), 2019.
Govind Waghmare, Ankur Debnath, Siddhartha Asthana, and Aakarsh Malhotra. Modeling inter-dependence between time and mark in multivariate temporal point processes. Proceedings of the 31sl ACM Internalional Conference on Information & Knowledge Management (CIKM), 2022.
Zirui Wang, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models. Proceedings of the 2020 International Conference on Learning Representations (ICLR), 2020.
Shuai Xiao, Junchi Yan, Stephen M. Chu, Xiaokang Yang, and Hongyuan Zha. Modeling the intensity function of point process via recurrent neural networks. Proceedings of the 31st AAAT Conference on Artificial Intelligence (AAI), 2017.
Chenghao Yang, Hongyuan Mci, and Jason Eisner. Transformer embeddings of irregularly spaced events and their participants. Proceedings of the 2022 International Conference on Learning Representations (ICLR), 2022.
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning, Proceedings of the 34th International Conference on Neural Information Processing Systems (NeurIPS), 2020.
Qiang Zhang, Aldo Lipani, Omer Kirnap, and Emine Yilmaz. Self-attentive hawkes processes. Proceedings of the 37th International Conference on Machine Learning (ICML), 2020.
Yixuan Zhang, Quyu Kong, and Feng Zhou. Integration-free training for spatio-temporal multimodal covariate deep kernel point processes. Proceedings of the 37th International Conference on Neural Information Processing Systems (NeurIPS), 2023.
Zihao Zhou and Rose Yu. Automatic integration for spatio-temporal neural point processes. Proceedings of the 37th International Conference on Neural Information Processing Systems (NeurIPS), 2023.
Shixiang Zhu, Henry Shaowu Yuchi, and Yao Xic. Adversarial anomaly detection for marked spatio-temporal streaming data. Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020.
17

---

Published in Transactions on Machine Learning Research (01/2025)
Shixiang Zhu, Minghe Zhang, Ruyi Ding, and Yao Xic. Deep fourier kernel for self-attentive point processes.
Proceedings of The 24th International Conference on Artificial Intelligence and Stalistics (AISTATS), 2021.
Simiao Zuo, Haoming Jiang, Zichong Li, Tuo Zhao, and Hongyuan Zha. Transformer Hawkes process.
Proceedings of the 37th International Conference Machine Learning (TCML.), 2020.
18

---

Published in Transactions on Machine Learning Research (01/2025)
A
Additional Forms of the Negative Log-Likelihood
Consider a dataset S = ..., St}. where each sequence Si comprises ny events with arrival times observed within the interval [0, T and = 1....,L. L. The average sequence negative log-likelihood (NLL) for these sequences can be expressed as a function of the marked intensity X² (1(10T) or the compensator A (1;0r) as follows:
L my
log X (tr,i 0T) + S." (r(
L
C(Or,0M:5)
- -
Σ 1=1
log p" (ki,lTT,,
(18)
Cr(0r:5)
Ex (0mr8))
and
L(0,, 0
1
d log
- ^ (4si Orl) di
1 + +A(T;Or))
T
log p (k1,d Oms).
(19)
1=1
(r(0r:5)
Cas (Par:))
B Alternative Scoring Rules
The NLL in (1) has been largely adopted as the default scoring rule for learning MTPP models (Shchur et al., 2021). However, our framework can be extended even further by using alternative scoring rules other than the NLL for assessing the mark and time prediction tasks. Let $1, S and 5 be (strictly) consistent scoring rules for (Tr;0r) p" (k|T; 0M) and 1 (T: Or), respectively. Given sequence S of n events (i.e.
L = 1), the scoring rule
n
C(0,8) - Σ (T(()))) St F² (T(TOOT))
Sp (kilT;iMM))
(20)
(r(Or.S)
(0
is (strictly) consistent for the conditional joint density f (T, k; 0r, 0m) restricted to the interval [0, T BBrehmer ct al.) 2023). Using the LogScore for 51, 5m and S in equation 20 reduces the NLL (17 One can also use other choices tailored to the specific task. For instance, one can choose to use the continuous ranked probability score (CRPS) (Gneiting & Raftery (007)) for S to evaluate the predictive distribution of inter-arrival times (Ben Taieb 2022). Similarly, the Brier score (Brier) 1950) can be used for SP to evaluate the predictive distribution of marks. Contrary to the local property of the LogScore, both the CRPS and the Brier score are sensilive to dislance, in the sense that they reward predictive distributions that assign probability mass close to the observed realization (Gebetsbergera et al. 2018). Nonetheless, the choice between local and non-local proper scoring rules has been generally subjective the literature. While exploring alternative scoring rules train neural MTPP models an exciting research direction, we leave it future work and train the models exclusively the NLL in 17
C Proof of Corollary 1
Consider a base model (t: 0) with trainable parameters 0, and disjoint parametrization of X (²; 0r) and p' (klt; 0x) obtained in 7 from two identical (t; 07) and Xi(t; Oar) with trainable parameters Or and OM, respectively. The NLL losses for 0 and (Ør. 0m)) respectively write
((0) = (0) + (O),
(21)
C((Or, 0m)) Cr(0r) + (0xs)
(22)
Suppose that at training iteration s E N, 0, Or and 0s are all initialized with the same 6", This implies that (t(:; (), (r; and Xt(t; QM) are all identical. Assuming that Cx and Las are differentiable, the gradient update steps for 0, Or and 0M are
0++1 0-
aggr + gM), 07 0² -agr and 0 0" -ogM:
(23)
19

---

Each sequence corresponds to a user listening to music records over Published in Transactions on Machine Learning Research (01/2025)
where > 0 is the learning rate and
gT = VoLT (0") = Cr,Cr(O) and gM = VaLM (0*) = Von LM (0").
(24)
Denoting ØTM as the angle between gr and gM, have the following corollary of Theorem 4.1. from Shi et al. 2023
Corollary 1. Assume that Cr and LM are differentiable, and that the learning rate a is sufficiently small. If 008 ΦTM < 0, then C((07", ( }) < L(0++1).
Proof. Let us consider the first order Taylor approximations of the total loss L Lr + LM near 0a- and (0T OM respectively:
= (0*+1 -0") (0 -0)) 9M o(a).
(25)
[( OM! = L()*) + (07+) - 0-) + (0 0¹) o(a).
(26)
Taking the difference between C( [( 0 1) and ((0*+)) yields
[(10T .0 ((e = (O 0 1 gr + (0 0 10' +
(27)
= oghgr + ag. gas + o(a)
(28)
2allgr ||lgg/|ooo ΦTM + o(a)).
(29)
Provided that a is sufficiently small, this difference is negative if cos o4 O, where drM is the angle between gr and gM-
Datasets
We use real-world event sequence datasets for our experiments:
LastFM Hidasi & Tikk 2012
lime. The artist the song is the mark.
MOOC (Kumar et al. 2019 Records of students' activities on an online course system. Each sequence corresponds to a student, and the mark the type of activity performed.
Github (Trivedi et al. 2019): Actions of software developers on the open-source platform Github, Each sequence corresponds a developer, and the mark is the action performed (e.g. fork, pull request,...).
Reddit (Kumar el al.
Sequences posts to sub-reddits that users make on the social website Reddit, Each sequence user, and the sub-reddits to which the user posts is considered as the mark.
2019
Stack Overflow (Du 2016). Sequences of badges that users reccive over time on the website Stack Overflow, A sequence is specific user, and the type of badge received is the mark.
We employ the pre-processed version of these datasets described in [osssr & Ben Taieb, 2023) which can be openly accessed at this url: https://www dropbox. com/soo////o///o /hom//sh/aaa jutpmm/m2777770000k nav___naa_a______ tracking=1 (MIT License). Specifi- cally, each dataset is filtered to contain the 50 most represented marks, and all arrival-times are rescaled the interval [0,10] to avoid numerical instabilitics. To save computational time, the number of sequences Roddit is reduced by 50%. Each dataset randomly partitioned into train helti//vaidaaiinnttee ttest splits (60%/20% 220% The summary statistics for each (filtered) dataset is reported in Table 3
20

---

Published in Transactions on Machine Learning Research (01/2025)
Table 3: Datasets statistics
#Scq.
Bventt
Mean Length
Max Length
Min Length
#Marks
LastFM
856
193441
226.0
6396
2
50
MOOC
7047
351160
49.8
416
2
50
Github
173
20656
119.4
4698
3
8
Reddit
4278
238734
55.8
941
2
50
Stack Overflow
7959
569688
71.6
735
40
22
E
Training details
Training details. For all models, we minimize the average NLL in 17 on the training sequences using mini-batch gradient descent with the Adam optimizer (Kingma & Ba 2014 and learning rate of 10-3, For the base models and the base+ setup, an carly-stopping protocol interrupts training if the model fails to show improvement the total validation loss (i.e., LT + LT) for consecutive epochs. Conversely, in the + setup, two distinct early-stopping protocols are implemented for the LT and La terms, respectively. If one of these terms does not show improvement for 50 consecutive epochs, we freeze the parameters of the associated functions (c.g. 0r for (T: (r)) and allow the remaining term continue training. Training is ultimately interrupted if both early-stopping criteria are met.
In all setups, the optimization process can last for maximum of 500 epochs, and we revert the model parameters to their state with the lowest validation loss after training. Finally, we evaluate the model by computing test. metrics on the test. sequences of each split. Our framework implemented a unified codebase using PyTorc All models were trained on machine equipped with an AMD Ryzen Threadripper PRO 3975WX CPU running 4.1 GHz and Nvidia RTX A4000 GPU.
Encoding past events. To obtain the encoding e, d of an event e, (4,, k.) in He, we follow the work of (Enguchard et al. 2020 by first mapping t to vector of sinusoidal functions:
4,/2-1
ef
sin (ayli) @ cos (azl)) E R
(30)
==
and @ is the concatenation operator. Then, a mark embedding e E Rd for ky is generated as ea = E'k, where E* RR4 x is a learnable embedding matrix, and K2 E 1)K the one-hot encoding of k4. Finally, we obtain e, through concatenation, i.c. e, = [e]]le?]
where aj x 1000 2:
Hyperparameters. To ensure that changes in performance are solely attributed to the features enabled by our framework, we control the number of parameters such the baseline's capacity remains equivalent. across the base, baset, base+ base-D, and base-DI setups. Notably, since STHP inherently models the decomposition of the marked intensity, the base and base+ configurations are equivalent. Also, LNM, RMIPP and STHP are equivalent between the base+ and the base- settings, and between the base+ and basc-DD settings. Hence, we only consider these models in the basc and base+ scttings. Furthermore, provides the total nunber of trainable parameters for each setup when trained on the LastFM dataset, as well as their distribution across the encoder and decoder heads. For all baselines and setups, we use chose
Table f
single encoder layer, and the dimension de of the event encodings is set to 8. Additionally, value of M 32 for the number of mixture components. It is worth noting that (Shchur et al. 2020 found MogNormMii to be robust to the choice of M. Finally, we set the number of GCIF projections to C = 32.
F
Computational Time
We report in Table equation the average execution time (in seconds) for a single forward and backward pass on all training sequences of all datasets. The results are averaged over 50 epochs. We notice that the computation of two separate embeddings by and hy in the base+ setup inevitably leads to an increase
hhttps//pyttrrc..o
21

---

Published in Transactions on Machine Learning Research (01/2025)
Table 4: Number of parameters for each baseline when trained on the LastFM dataset. The distribution of parameters between the encoder and decoder heads reported in parenthesis.
THP
STHP
SAHP
FNN
LNM
RMTPP
Base
14720 (0.66/0.34)
15588 (0.68/0.32)
15939 (0.65/0.35)
13930 (0.67/0.33)
13619 (0.69/0.31)
Base-t
14786 (0.64/0.36)
18586 (0.5/0.5)
15210 (0.67/0.33)
16083 (0 65/0.35)
139-16 (0.67/0.33)
13669 (0.69/0.31
Basc++
14602 (0.63/0.37)
(0.5/0.5)
15514 (0.67/0.33)
14961 (0.67/0.33)
13340 (0.66/0.34)
13063 (0.67/0.33)
Base-D)
15512 (0.66/0.34)
15412 (0.67/0.33)
16083 (0.65/0.35)
Base-DD
15252 (0.66/0.34)
15464 (0.68/0.32)
10446 (0.65/0.35)
in execution time, which appears more pronounced for larger datassess such Reddit and Stack Overflow. However, the increased computational complexity is generally offset by improved model performance, detailed in Section 6.1
Table 5: Average execution time (in seconds) for single forward and backward pass on all training sequences for all datasets, Results are averaged over 50 epochs.
LNM
RMTPP
FNN
THP
SAHP
STHP
Base
++
Base
+
++
Base
+
++
Base
Base
+
Base +
LastFM
2.2 2.23
3.26
1.91
1.94
2.94
4.29
3.66
4.74
2.96
2.57 3.62
4.82
3.42
4.48
3.08
\ 4.17
MOOC 3.97
3.99
5.6
3.33
4.37
6.36
6.28
5.19
5.91
7.74
5.61
\ 6.72
Gitlmb 0.34
0.39
0.51
0.34
0.34
0.5
0.76
0.65
0.8
0.38
0.43
0.58
0.51
0.55
0.69
0.49
\ 0.64
Reddit
8.74 9.53
11.5
7.99
8.14
11.79
19.13
16.47
20.32
9.75
9.78
13.5
13.63
11.99
15.9
11.59
\ 15.54
Stack O. 16.02 16.03
22.98
14.08
14.54 20.31
32.83
28.15
33.95
16.32
16.81
23.1
21.11
20.77
26.45
19.84
\ 26.07
G An Alternative Approach to Model the Joint Distribution
As detailed in Section 4.2 our parametrization LNM+ alleviates the conditional independence of arrival- times and marks made LNM (Shchur et al., 2020). Relatedly, [Waghmare et al. (2022) also proposed an extension of LNM that relaxes this assumption, although their approach differs from ours in some key aspects. Specifically, their work parametrizes f (r]k:: 0r) as distinct mixture of log-normal distributions for cach mark k; and p" (k; 0ws) is obtained by removing the temporal dependency in (g For further reference, we denote this model as LNM-Joint. Although both LNM+ and LNM-Joint aim to model the joint distribution f* (T,K), some conceptual differences separate the two approaches:
By design, LNM-Joint cannot be trained in the base setup as prevents the decomposition the NLL into disjoint. Lr and LM terms. Indeed, suppose that we use two distinct history representations ht and hM parametrize (r]kk; 0r) and p" (k; Osr) respectively, as detailed in Section 4.2 Here, Or and Ou are disjoint set of learnable paramcters. The NLL of training sequence S = { {(T( k observed in [0. T] would write
[(0),0M:S)= = Σ log (T, log (1-F' F (T;0r, > log p" (k:; 0m),
(31)
Ess (PxrS)
where Fx (T; 0r: _ quently, the NLL cannot be disentangled into disjoint Lr and LM terms, proscribing disjoint training in the base+ setup. Converscly, choosing to parametrize (T; 0y) and p" (k|r; axr) as done in our framework leads to the decomposition
Ek (s (k;; Pm )ds depends on both Or and 0as- Conse-
17 as FT (T) is solely function of 0r.
Both (r)kk 0r) and p² (x)) rely common history embedding
22

---

Published in Transactions on Machine Learning Research (01/2025)
2. For LNM-Joint, M mixtures must be defined for each k leading to M x K log-normal distributions in total. Conversely, in INM+, (T; (r) does not scale with K, and p" (r|k; 0as) requires equivalent. number of parameters as (k; 0M) in LNM-Joint.
For completeness, we integrate LNM-Joint in our code base using the original implementation as reference. In Table (6), we compare its performance against LNM+ on the time and mark prediction tasks terms of the LT, LM, and accuracy metrics, following the experimental setup detailed previously. We observe improved performance of LNM+ compared to LNM-Joint the mark prediction task (Eas and accuracy), and competitive results the time prediction task (CT). Despite both approaches modelling the joint distribution, our results suggest that the dependency between arrival times and marks is more accurately captured by p" (klT; 0M) than by (r
Table 6: Lr, CM and Accuracy results for LNM+ and LNM-Joint on all datasets. The values are computed over 3 splits, and the standard error is reported in parenthesis. Best results are highlighted in bold.
LasFFM
MOOC
Gitbub
Redda
Stack O.
AM
LNM+
668.6 (15.6)
77.1 (1.3)
112.3 (15.1) 41.4 (1.1)
103.2 (0.7) 117.3 (15.5) 42.9 (0.9)
Jom
67133 (17.1)
127.0 (6.4)
108.8 (0.7)
Accuracy
LNM
0.24 (0.01)
0.52 (0.0)
0.67 (0.01)
0.52 (0.0)
0.49 (0.0)
Joint-LNN
0:23 (0.01)
0.23 (0.03) 0.64 (0:01)
0.82 (0.0)
0:47 (0.0)
-1820.4 (90.5)
-810.6 (3.7) -378.7 (89.4) 4) -90.4 (2.0)
-01.1 (1.3)
Joint-LNM -1820.2 (58.3) -308.9 (3.3) -38..0 (60.0) -94.3 (2.0)
-00.0 (1.4)
H Additional results
H.1 Evaluation metrics
Tables and give the PCE, ECE, MRR, and accuracy&(1,3.5 for the base, base and base+ setups across all datasets. The metrics are averaged over 3 splits, and the standard error is given in parenthesis. We note that the results not discussed in the main text are consistent with our previous conclusions. Specifically, we observe general improvement with respect to mark related metrics (i.e. ECE, MRR, (1, when moving from the base models to the base- or base+ setups. Finally, the PCE metric does not always improve between the base+ and ++ setups, suggesting that the remaining conflicting gradients the encoder head in the base+ mostly detrimental to the mark prediction task.
Finally, we report. the results with respect to the MAE metric in Table TO We notice that lower MAE values do not systematically match the lower values of Lr or PCE in Tables and 7 indicating that the MAE may not be entirely appropriate to evaluate the time prediction task. As discussed in Shehhr et al.) 2021). noural MTPP models are probabilistic models that enable the generation of complete distributions over future events. In this context, point prediction metries, like MAE, are deemed less suitable for evaluating MTPP models because they consider single point predictions into account. In contrast, the NLL and calibration scores directly evaluate the entire predictive distributions, and should be therefore favored compared to point prediction metries.
H.2 Reliability diagrams
Figures to 10 show the reliability diagrams of the predictive distribution of arrival-times and marks for all models in the base and basc+ setups on all datascts. In most cases, we observe improved mark calibration for the base+ setup compared to the base models, in accordance to the ECE results of Table 7 Additionally, improvements with respect to the calibration the predictive distribution arrival times is in general less prevalent, corroborating our discussion in the main text. This observation is also in coherent with the PCE results of Table 7 Nonetheless, we observe substantial time calibration improvements for SAHP and FNN when trained in the base++ setup on MOOC and Reddit.
23

---

Published in Transactions on Machine Learning Research (01/2025)
Table 7: PCE and ECE results of the different setups across all datasets. The values are computed over 5 splits, and the standard error is reported in parenthesis. Best results are highlighted in bold.
PCB Githab
ECE Grehwb
LM
MODC
Reddit
Stack O.
LaatM
MOOC
Reddit
Stack O.
THP
0.28 (0.0) 0.37 (0.0) 0.2 (0.01) 0.12 (0.0) 0.01 (0.0)
THP
0.23 (0.03)
0.07 (0.0) 0.09 (0.02) 0.02 (0.0) 0.04 (0.02)
TMP+ T
0.27 (0.01)
0.37 (0.0)
0.19 (0.01) 0.07 (0.0) 0.01 (0.0)
TMP+
0.05 (0.00 (0.01)
0.02 (0.0) 0.07 (0.01) 0.03 (0-01) 0.01 (0.0)
0.28 (0.01)
0.37 (0.0) 0.19 (0.02) 0.05 (0.0) 0.01 (0.0)
THP++
0.03 (0.0)
0.02 (0.0) 0.07 (0.02) 0.02 (0.0) 0.01 (0.0)
STHP
0.29
(0.01) 0.37
(0.0)
0.20 02) (0.0) 0.01 (0.0)
9THP+
0.05 (0.0)
0.08 (0.0) 0.00 (0. (2) 0.08 (0.0) 0.02 (0.0)
STHP++
0.29 (0.01) 0.36 (0.0)
(0.02)
(0.0)
(0.0)
STRP+4
(0.0)
(9.0)
(0.01) 0.03 (0.01) 0.03 (0.01)
SARP
0.06 (0.01) 0.12 (0.0)
0.07 (0.0) 0.1 (0.01) 0.01 (0.0)
SAHP
0.11 (0.01)
0.13 (0.0) 0.05 (0.02) 0:08 (0.01) 0.03 (0.0)
8ARP4
0.05 (0.01)
0.03 (0.0)
0.04 (0.01) 0.01 (0.0) 0.0 (0.0)
8AHP-4
0.06 (0.02) 0.02 (0.0) 0.07 (0.02) 0.03 (0.01) 0.01 (0.0)
3ARP4-4 +
0.04 (0.01) 0.08 (0.0)
0.04 (0.01)
0.01 (0.0) 0.04 (0.0)
SAHP 4- +
0.03 (0.0)
0.02 (0.01) 0.07 (0.01) 0.02 (0.0) 0.01 (0.0)
LNM
0.08
0.01
0.02
LNM
0.00 (0.02)
0.07 (0.01) 0.05
0.08 (0.01) 0.08 (0.01)
(0.01) 0.01
(0.0)
(0.0)
LNM
0.06 (0.03)
(0.01)
(0.01)
(0.0)
+
0.03 (0.0) 0.01
(0.0)
+
0.02 (0.0)
0.02 (0.0)
(0.01)
(0.0)
(0.0)
FNN
0.04 (0.0) 0.00 (0.0)
0.04 (0.01)
0.07
(0.0) 0.05 (0.0)
FNN
0.05 (0.01)
0.00 (0.0) 0.05 (0.0) 0.04 (0.01) 0.02 (0.0)
FNN+
0.03 (0.0)
0.01 (0.0)
0:03 10 01)
0.01 (0-0) 0.01 (0-0)
FNN+
0.04 (0.01)
0.02 (0.0) 0.00 (0.0) 0.02 (0-0) 0.01 (0-0)
+
0.02 (0.0)
0.01 (0.0)
0.02 (0.0)
0.01 (0.0) 0.01 (0.0)
FNN++
0.03 (0.0)
0.02 (0.0)
0.07 (0.02) 0.02 (0.0) 0.01 (0.0)
AMTPP
0.25 (0.01)
0.28 (0.0)
0.18 (0.01)
0.03 (0.0) 0.01 (0.0)
RMTTP
0.05 (0.03)
0.05 (0:01)
0:07 (0.0) 0.03 (0.0) 0.03 (0.02)
RMTPP4
0.26 (0.01)
0.27 (0.01)
0.18 (0.01)
0.08 (0.0) 0.01 (0.0)
RMIPP-- 4
0.06 (0.03)
0.03 (0.01) 0.00 (0.0) 0.03 (0.01) 0.01 (0.0)
RMTPP |
0.26 (0.0) 0.29 (0.0) 0.17 (0.01) 0.04 (0.0) 0.01 (0.0)
RMTPP 4 + 0.04 (0.01) 0.02 (0.0) 0.00 (0.01) 0.02 (0.0) 0.01 (0.0)
Table 8: Accuracy@1 and Accuracy@3 results of the different setups across all datasets. The values are computed over 5 splits, and the standard error is reported in parenthesis. Best results are highlighted in bold.
Accuracy03 Gitbub
LaatFM
MOOC
Gitbub
Reddit
Stack
LanFM
MOOC
Reddit
Stack
THP
0.18 (0.01) 0.4 (0.0) 0:59 (0.02) 0.83 (0.0) 0.18 (0.0)
THP
0.36 (0.01)
0.75 (0.0) 0.86 (0.01) 0.91 (0.0) 0.82 (0.0)
THT+
0:21 (0:03)
0-52 (0.0)
0.64 (0:01)
0.82 (0.0)
0.4M (0.0)
THP+
0:37 (0.01)
0.8 (0.0)
0.88 (0.0) 0.9 (0.0)
0.83 (0.0)
THP++
0.25 (0.01) 0.58 (0.0)
0.07 (0.01)
0.82 (0.0) 0.40 (0.0)
THP +
0.42 (0.01)
0.83 (0.0)
0.89 (0.0) 0.9 (0.01) 0.88 (0.0)
8THP4
0.2 (0.01) 0.48 (0.0) 0.63 (0.01)
0.81 (0.0)
0.48 (u.0)
STHP+
0.36 (0.01)
0.78 (0.0) 0.85 (0.01) 0.89 (0.0) 0.83 (0.0)
8THP +++
0.21 (0.01) 0.48 (0.0) 0.85 (0.01) 0.81 (0.0) 0.48 (0.0)
8THP++
0.37 (0.03)
0.78 (0.0) 0.89 (0.0) 0.89 (0.0) 0.82 (0.0)
BAHP
(0.0) 0.36 (0.01) 0.0 (0.01) 0.09 (0.02) 0.48 (0.0)
8AHP
(0.0)
0.57 (0.01) 0.88 (0.01)
0.79 (0.02)
(0.0)
8AHP4 +
0.14
(0:01) 0.64 (0.0)
0.65 (0.01) 0.82 (0.0) 0.40 (0.0)
SAHP
0:29 (0.01)
0.81 (0.0) 0.88 (0.0) 0.9 (0.0) 0.88 (0.0)
SAHP++
(0.01) 0.56 (0.0) 0.87 (0.01) 0.82 (0.0) 0.49 (0.0)
SARP++
(0.01)
0.82 (0.0)
(0.0)
(0.0)
(0.0)
LNM
0.21 (0.01) 0.43 (0.0) 0.64 (0.01) 0.81 (0.0) 0.47 (0.0)
LNM
0.38 (0.02)
0.76 (0.0) 0.58 (0.0)
0.59 (0.0) 0.62 (0.0)
LNM+
0.24 (0.01)
0.52 (0.0) 0.67 (0.01) 0.82 (0.0) 0.49 (0.D)
LNM+
0.41 (0.01)
0.8 (0.0)
0.89 (0.0)
0.9 (0.0) 0.83 (0.0)
LNM4++
0.25
(0.01) 0.54 (0.0) 0.67 (0.01) 0.82 (0.0) 0.48 (0.0)
LNM++
0.44 (0.01)
0.81 (0.0)
0.89 (0.0)
0.9 (9.9) 0.83 (0.0)
FNN
0.13
0.51
(0.01)
FNN
0.3 (0.03)
FNN-
0:23 (0.01)
0.82
FNN+
0.4
(0.0)
+
(0.01)
(0.0)
(0.01) 0.82 (0.0) 0.40 (0.0)
(0.01)
0.52 (0.0)
(0.0)
(0.0)
(0.0)
RMTPP
0.23 (0.01) 0.42 (0.0) 0.61 (0.02)
0.82 (0.0) 0.47 (0.0)
RMTPP
0.39 (0.01)
0.76 (0.0) 0.80 (0.0)
0.9 (0.0) 0.82 (0.0)
RMTPP4 +
0.23 (0.03) 0.63
(0.0)
0.64
(0.01)
0.83
(0.0) 0.40 (0.0)
RMTPP
0.39 (0.01)
0.8 (0.0)
0.57 (0.01)
0.9 (0.0)
0.88 (0.0)
BMTPP44 ++ + 0.24 (0.01)
0.58
(0.0)
0.87 (0.01)
0.82
(0.0) 0.49 (0.0)
RMTPP++
+ 0.42 (0.01)
0.82 (0.0)
0.89 (0.0)
0.9 (0.0) 0.83 (0.0)
Table 9: Accuracy@5 and MRR results of the different setups across all datasets. The values are computed over 5 splits, and the standard error is reported in parenthesis. Best results are highlighted in bold.
Accuracy R5 Gishub
MRR Gitbab
LastPM
MO00
Reddic
Stack O
LasFM
MO00
Reddit
Stack O.
THP
0.48 (0.01) 0.87 (0.0)
(0.01)
THP
(0.01) D.6 (0.0)
0.74
(0.0)
TRP+
0.47 (0.01) 0.58
0.66 (0.0)
0.02 (0.0) 0.98 (0.0)
THP+
0.34 (0.01) 0.68 (0.0)
0.77 (0.01) 0.87 (0.0) 0.07
THP++
0.52 (0.01)
0.97 (0.0)
0.93 (0.0) 0.93
THP++
0.38 (0.01)
(0.01)
0.87 (0.0) 0.67 (0.0)
STMP-
0.46 (0-01) 0.88 (0.0) 0.86 (0.0)
0.02 (0.0) 0.93 (0.0)
STHP+
0.33 (0.01) 0.04 (0-0) 0.77 (0 01)
0.80 (0.0) 0.07 (0.0)
STHP++
0.48 (0.m1 0.89 (0.0) 0.97 (0.0)
0.91 (0:0) 0.92 (0.0)
STHP++
0.34 (0.01) 0.64 (0.0) 0.78 (0.0)
0.80 (0.0) 0.67 (0.0)
SAHP
0.22 (0.01) 0.67 (0.01) 0.85 (0.0)
0.84 (0.02) 0.91 (0.0)
SAHP
0.16 (0.0) 0.5 (0.01)
0.71 (0.00 (0.01)
0.76 (0.02) 0.67 (9.n)
SAHP-
0.39 (0.01) 0.58 (0.0)
0.97 (0.0)
0.82 (0.0)
0.95 (0.0)
SAHP4
0.27 (0.01)
0.60 (0.0)
0.78
0.86 (0.0) 0.67 (0.0)
SAHP4+
0.52 (0.03)
0.9 (0.0)
0.97 (0.0)
0.93 (0.0)
0.95 (0.0)
SAHP++
0.38 (0.01)
0.7 (0.0)
0.79 (0.01)
0.87 (0.0) 0.87 (0.0)
LNM
0.47 (0.02) 0.58 (0.0) 0.98 (0.0)
0.82 (0.0) 0.92 (0.0)
LNM
0.35 (0.01) 0.62 (0.0) 0.77 (0.0) 0.86 (0.0) 0.86 (0.0)
LNM+
0.61 (0.01)
0.89 (0.0)
0.97 (0.0)
0.93 (0.0)
0.93 (0.0)
LNM+
0.57 (0.01) 0.88 (0.0)
0.79 (0.0)
(0.0) 0.67 (0.0)
LNM++
0.54 (0.01)
(0.0)
0.97 (0.0)
0.93 (0.0)
0.98 (0.0)
LNM-+
(0.01)
(0.0)
(0.01)
(0.0)
(0.0)
ENN
0.4 (0.01)
0.88 (0.0) 0.07 (0.0) 0.02 (0.0) 0.91 (0.0)
FNN
0.27 (0.01) 0.07 (0.0)
0.79 (0.01)
0.85 (0.0) 0.06 (0.0)
RNN+
0.5 (0.01)
0.9 (0.0) 0.97 (0.0)
0.93 (0.0)
0.93 (0.0)
FNN+
0.58 (0:01)
0.7 (0.0)
0.79 (0.0)
0.87 (0.0) 0.67 (0.0)
FNN+
0.53 (0.01)
0.9 (0.0) 0.97 (0.0) 0.93 (0.0) 0.93 (0.0)
FNN+-
0.38 (0.01)
0.7 (0.0)
0.8 (0.0)
0.87 (0.0) 0.67 (0.0)
RMTPP
(0.01)
(0.0)
0.66 (0.0) 0.03 (0.0) 0.92 (0.0)
RMTPP
0.36 (0.01) 0.61 (0.0) 0.70 (0.01)
(0.0) 0.60 (0.0)
(0.0) 0.96 (0.0)
0.95
(0.0)
RMTPP-
(0.01)
(0.0) 0.97 (0.0) (0.0) 0.95 (0.0)
RMTPP-+
(0.01) (0.0) (0.01) (0.0) (0.0)
24

---

Published in Transactions on Machine Learning Research (01/2025)
Table 10: MAE results of the different setups across all datasets. The values are computed over 5 splits, and the standard crror is reported in parenthesis. Best results are highlighted in bold.
MAE Github
LastEM
MOOC
Reddit
THP
0.01624 (0.00056) 0-13332 (0.00096)
0.0935 (0.01328)
0.15257 (0.00822) 0.09885 (0.00097)
THP+
0.01925 (0.00165)
0.13638 (0.0016)
0.09769 (0.014)
0.15811 (0.00079)
0.09972 (0.00077)
THP++
0.01744 (0.00103)
0.13365 (0.00134)
0.09327 (0.01162)
0.17627 (0.00359)
0.09926 (0.00069)
STHP+
0.03717
(0.00978)
0. 17422 (0.01899)
0.11691 (0.00889)
0.15864
(0.00409)
0.09883 (0.0007)
STHP
0.08356 (0.01887)
0.17057 (0.02083)
0.1029 (0.01195)
0.22542
(0.02801)
0.09931 (0-00056)
SAHP
0.0137 (0.00055)
0.12319 (0.00174)
0.084199
(0.01092)
0.18839
(0.00389)
0.0992 (0. (0.00066)
SAHP
0.01364 (0.00056)
0. 12318 (0.00191)
0.07491 (0.0117)
0.13427
(0.00216)
0.09853 (0.00071)
S
0.01358 (0.0006)
O. 0.12138
(0.00197)
0.07127 (0.0102)
0.13168 (0.00206)
0.10285 (0.00057)
FNN
0.03563 (0.0138)
0.16602 (0.0487)
0.41847 (0.19173)
0.23207 (0.05185)
0.266 (0.09399)
FNN+
0.01055 (0.00043)
0.07732 (0.00034)
0.07093 (0.00902)
0.13157 (0.00208)
0.09299 (0.00072)
ENN++
0.01054 (0.00042)
0.07711
(0.00039)
0.07102 (0.0093)
0.13148
(0.00231)
0.09305 (0.00064)
LNM
0.0111 (0.00049)
0.13516 (0.00078)
0.08706 (0.01153)
0.13494
(0.00202)
0.09494 (0.00075)
L
0.01131 (0.00054)
0.13548 (0.0011)
0.08715 (0.0114)
0.135 (0.00213)
0.09517 (0.00069)
4
0.01098 (0.00049)
0.13083
(0.00096)
0-08694
(0.01185)
0.1361 (0.00209)
0.09476 (0.00071)
RMTPP
0.01619 (0.00097)
0.1208
(0.00084)
0.12939 (0.02772)
0.13577
(0.00223)
0.09522 (0.00069)
RMTPP-
0.01677 (0.00 00.00072)
0.12059
(0.00083)
0.114116 (0.02624)
0.1362 (0.00227)
0.09563 (0.00085)
RMTPP++
0.01705 (0.00058)
0.12011 (0.00079)
0.09183 (0.0106)
0.13772 (0.00192)
0.09656 (0.0008)
(a) LastFM
1.0
Accuracy
0.8
0.6
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
10
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
022
0.4
0.6
0.8
1.0
0.2
0.4
9.6
08
1.0
Confidence
Accuracy
SAMP11
LNW
0.4
0.2
0.0
Confidence
1.0
Frequency
0.8
0.6
59
02
0.4
1.0
0.0
9.2
6.4
0.6
08
1.0
02
0-2
0.4 6.8
10
0.9
9.2
0.4
06
08
Confidence
Frequency
1MP1
SAMP-1
HMI
0.4
0.2
0.0
Confidence
Figure 8: Reliability diagrams of the predictive p" (kkTT; 0x) (top) and (r, (bottom) on LastFM. Frequency and Accuracy aligning with the black diagonal corresponds to perfect calibration. The results are averaged over 5 splits, and the error bars correspond to the standard error.
25

---

Published in Transactions on Machine Learning Research (01/2025)
MOOC
Accuracy
0.0
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.5
9.8
10
0.2
0.4
0.5
1.0
0.2
9.4
0.6
0.8
1.0
0.2
0.4
9.6
0.8
1.0
0.2
0.4
0.6
08
1.0
Confidence
Accuracy
THP--
5AMP
Th-M--
LXIM++
BMTPP ++
STMP-+
0.0
Confidence
1.0
Frequency
0.8
0.6
0.4
0.2
50
0.2
0.4
0.9
0.5
1.0
0.0
02
3A
0-8
2.8
3.0
0.0
0.2
5.4
0.5
08
10
03
54
0.4
0.5
3.5
100
05
02
24
0.5
5.8
100
0.0
2.2
0.4
0.
08
10
Confidence
Frequency
THP
SAMP--
LIVS--4
BNT7P4
STHP
0.0
Confidence
(b) Github
Accuracy
0.8
SAIP
LKM
RMTPO
0.6
0.4
0.2
0.0
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.2
1.0
3.0
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
Confidence
1.0
Accuracy
IMP
SAMPI
-MN--
LIN
HMNI HORT
SiMFee
0.8
0.5
0.2
Confidence
1.0
Frequency
0.8
0.6
0.4
0.2
0.4
3.0
0.
0.2
0.4
0.6
08
30
63
02 BNTT74
6,8
1.0
0.
3.2
0.4
06
08
30
Confidence
Trequency
TIP+
SANP
STMP-
0.0
5.4
3.0
5.2
3.2
0.6
05
08
Confidence
Figure 9: Reliability diagrams of the predictive p² (k|r; 0x) (top) and fT(r,0r) (bottom) on MOOC anf Github. Frequency and Accuracy aligning with the black diagonal corresponds to perfect calibration. The results are averaged over splits, and the error bars correspond to the standard error.
26

---

Published in Transactions on Machine Learning Research (01/2025)
(a) Reddit
Accuracy
0.0
0.2
0.4
0.6
0.8
1.
0.2
0.4
0.5
9.8
0.2
0.4
0.5
1.0
0.2
9.4
0.6
0.8
1.0
0.2
0.4
9.6
0.8
1.0
0.2
04
0.6
08
1.0
Confidence
Accuracy
0.8
THP-+
SAHPH
PMN-+
L-W
5ThO44
0.5
0.2
0.0
Confidence
1.0
Frequency
0.8
0.6
50
0.2
5.4
0.6
0.8
3.0
0.0
02
34
66
38
1.0
0.0
0.7
5.4
0.
08
10
64
52
0.4
0.
388
1.0
02
02
34
9.6
58
1.0
0.0
3.2
0.4
06
08
18
Confidence
Frequency
TiPP+
RMT
STAPPe
0.2
0.0
Confidence
(b) Stack Overflow
Accuracy
0.2
0.4
0.6
08
1.
6.2
0.4
0.6
A8
1.0
0.2
0.4
0
58
100
0.2
2.4
0.6
0.8
1.5
0.2
0.4
36
0.
1.8
0.2
0.4
0.6
00
1.5
Confidence
Accuracy
THP
SAHPI
FMN-+
RMT R0
Sioefe
0.2
0.0
Confidence
1.0
Frequency
0.8
SAMP
PMN
STHT-
0.4
0.2
0.4 PMN
0.4
0.5
08
1.0 Confidence
0.2
5.2
0.4
0.6
08
Frequency
THP
SAMP--
LIMS-+
RNTIN
STHP-
0.4
0.2
0.0
0. 2.0 02
Confidence
Figure 10: Reliability diagrams of the predictive p² (k]T;0M) (top) and f* (bottom) on Reddit and Stack Overflow. Frequency and Accuracy aligning with the black diagonal corresponds to perfect calibration. The results are averaged over 5 splits, and the error bars correspond to the standard error.
27

---

Published in Transactions on Machine Learning Research (01/2025)
H.3 Conflicting gradients remain harmful as capacity increases
To assess whether conflicting gradients remain detrimental to predictive performance for higher-capacityy models, we train THP, SAHP and FNN in the base and base-DD settings while progressively increasing the number of trainable paramcters. Figure [1 shows the evolution of the proportion of CG, GMS and TPI during training for these models on LastFM and MOOC, along with the evolution of their test Cr and La as a function of number of parameters. For each capacity (25K, 50K; 75K and 100K parameters), we maintain the distribution of parameters between the encoder and decoder heads constant at 0.67 1..7...3. Note that only report the CG, GMS, and TPI values for the base models, the base-DD setting is by definition free of conflicts. We note that increasing model's capacity has a limited impact on the CG, GMS and TPI values, as well as model performance with respect to both Lr and LM- In contrast, differences in performance between the base and base-DD setups are more significant, suggesting that conflicting gradients remain harmful to performance even with increased model capacity.
H.4 Scaling the loss does not efficiently address conflicts
Our findings in Figure ! suggest that conflicting gradients generally tend to favor TT at the encoder heads during optimization, as illustratd by TPI values > 0.5. To better balance tasks during training, a natural approach would consist in scaling the contribution of Tr in (2) to reduce its impact on the overall loss, i.e.
1 --C Cr Strain) Las (0; Swaiu),
L(0 Strain)
(32)
5
where 2 is scaling coefficient. To assess the effectiveness of this method, train the base THP, SAHP and FNN models on the objective in (32) following the experimental setup detailed in Section E For these models, we report in Figure 12 the evolution of the training CG, GMS and TPT values, along with their unscaled test Lr and Las. We observe that the occurrence of conflicting gradients marginally impacted by larger values of s. However, as scaling increases, the magnilude of gr diminishes, which translates inlo decreased values of GMS and TPI. Consequently, optimization begins favor TM, and improvements on LM can be observed. Nevertheless, the crashed TPI and GMS values are at the root of significant degradation with respect to Cr, offsetting the gains on Tas Although specific value of could lead to trade-off between tasks, models trained in our base+ or + settings generally show improved performance with respect to both tasks simultaneously, as shown in Table [
28

---

Published in Transactions on Machine Learning Research (01/2025)
(a) LastEM
-
CG
- GMS
TP1
- Base
Base-DD
-900
0.85
-950
0.60
-1000
25K
25K
50K
-1200
-1250
0.6
-330
800
-1350
700
-1400
0.60
-125
FNN
0.55
725
11350
25K
S0C
25K
50K
Number trainable parameters
(b) MOOC
-
CG
GMS
TR
Base
Base DD
0.85
-120
-300
THE
-15
200
0.8
0.8
220
150
100
25K
25K
25K
50K
0.7
275
0.6
280
FNN
0.5
-285
-200
0.4
74
100K
25K
50K
75K
25K
50K
Number of trainable parameters
Figure 11: Left: Evolution of CG, GMS, TPT with increasing model capacity for the base THP, SAHP and FNN during training on LastFM and MOOC. Right: Evolution of the test CT and Cas in the base and base-DD DD settings.
29

---

Published in Transactions on Machine Learning Research (01/2025)
(a) LastFM
0.6
750
-800
700
-000
650
0.0
-1000
SAHP
0.4
-1000
-1200
0.0
850
1200
800
ENN
0.4
-1250
750
1300
-1350
Scaling coefficient
(b) MOOC
CG
GMS
-
TP
-50
-75
100
0.25
-125
0.00
175
0.6
250
0.4
-200
125
0.0
-250
100
-220
100
0.6
FNN
-240
0.0
-20
Scaling coefficient
Figure 12: Left: Evolution of CG, GMS, TPI with scaling coefficient. & for the base THP, SAHP and FNN during training on LastFM. Right: Evolution of the test. Lr and CM in the base and base-DD settings.
30

---

Published in Transactions on Machine Learning Research (01/2025)
H.5 Distributions of conflicting gradients between the base and base+ settings
Figures (13) to (14) show the distributions of cos Φтм during training, as well as the proportion of conflicting gradients (CG), their average GMS, and their average TPI for all models in the base and base+ setups on all datasots. As discussed in the main text, severe conflicts (cos ΦтM < -0.5) often arisc when training neural MTPP models in the base setup. For THP, SAHP, and FNN, the base+ setup reduces the occurrence of severe conflicts at the encoder heads, and prevents conflicting gradients to appear at the decoder heads. For LNM and RMTPP, we only show the distribution of cos ΦтM at the encoder heads as the decoders are by design free of conflicting gradients in the base and base+ setups. For these models, the base- setup does not change significantly the distribution of conflicting gradients during training, which aligns with expectations. Hence, for LNM+ and RMTPP+, we attribute performance gains on the mark prediction task to the relaxation of conditional independence between arrival times and marks via
C3. 90 ERS:3 IRAZE 6.
B: CURLM GRD IP CGO GVB- S7 TREE
DU-3 OROIE TRID06
CD D95 TAE
BI C0 UMBIS TPE6.3
CS0.99 SHG:0 TRI13
BI CDC0 49 SPSISS TRED,S6
THP-ENC
THC-
SAHPIENC
SAHP DEC
PNN-ENC ENC
FNN DEC
LaaFM
Proportion
0.3
3.0
-1.0
-0.5
2.0
0.3
05
05
1.0 -2.0
-18
-03
16
05
Cos 9M
cor
0OX 6T
CoX 6MM
FOR 91
B: OGd 5o GRE T7
B: G23AA GPSC146 TRE392
Call G45:3.63 T95082
CG S42004 TIU033
4: 00:6 GM2 17:0.55
LMMMNNN
RMTPP-ENC
STMP-ENC
LazzFM
Proportion
4.32
3.83
1.0
10
5D
THP-ING
THNBEC:
SAHPNEN
SAHP-DEC
PIN-EEC
FNN-DDC
Proportion
MOOC
5..0
-3.5
60
3.5
3.5
-3.5
-05
0.5
3.0
-.00
0.0
6.5
3.0
-1.0
-05
8-3
05
1.0
-0.0
-0.5
0.5
2.0
-2.
-05
86
65
58
CS:0.39
17:000
LNINENN:
RMIPP-ENC
STHPENE
Proportion
MODC
CO5 V7
C75 6M
Figure 13: Distribution of cos ΦтM during training at the encoder (ENC) and decoder (DEC) heads for all baselines in the base and base+ setup on LastFM and MOOC. 'B' and "+" refer to the base and base+ models, respectively, and the distribution is obtained by pooling the values of ΦтM over 5 training runs. As the decoders are disjoint in the base+ setting, note that cos ØTM is not defined.
31

---

Published in Transactions on Machine Learning Research (01/2025)
GMS 02550.32111 87
000 OPEUUUS T7E0S?
4: CG SM500/S 1P//2 90
325
SAMP-DEC
Progorton
Github
0.83
1.5 -1.9
-3.0
C.-aa..5 52..522200 G450022 TR800
CG0.5 GNE TRI0.33
8333
STHPLENC
Proportion
0:225
Github
9333
8.003
CBS 5TM
CO5 PM
COS AM
Clio Sa 075 17:6.77
BI SE 6SM5 9o 1PE5,07 es Coin 0M516 35
OS03 65 6 19
0006.3 0.75 IWOSS
OC:E 30 GRE25.75 TR15.2
D: C20.76 CHSUE,32 1RN0-32
COBSS 6/65 TP
B: CG-00000 SP33.65 1I10.3
THP-ENC
THP-DEC
SAHPENC
SAHP DEC
FNN-ENC
FNN DEC
Ruddit
Proportion
CO5 PMM
005 6TM
CO5 8PM
COG $10
CO0 6TH
CE GAAK TRESSAT
4s cad.3 crecu 24 77:00 eq
4: 00-055 2245 mouu4
C. GMS 6776 TPic S6
LNM.ENC
RMTTP ENC
STHP ENC
Progartion
Reddit
043
-20
-15
3.0
0.5
3.0
-3.0
-0.5
6.
0.5
3.0
-3.0
-05
6.0
65
3.0
B:cost-e 42 GM60 T19924
00:3-4 CPS-082 17:330
C500 0M5 64 TIR
00:003 CHOUULES TRISLE
0:C0:550 SP5:0 1110
325
THP ENC
THP DBC
SAHPIENC
SAHP DEC
FMN ENC
FNN DEC
Stack Overflow
Proportion
0:33
3.0
-3.5
3.0
3.5
10--10
-65
3.5
055
3.0
-10
0.3
65
-1.
-25
6.3
85
1.0
-10
-0.5
0.5
1.0
-18
-05
3.6
65
67M
BICOO 26 GHESOL TRUOLD
00:349 SNE007 TRU623
CGEA4 GNGETSTRREER
UNINENC
RMTPP ENC
STHP-EN ENC
Stack Overllow
422
Proportion
COu 6'
C06
Figure 14: Distribution of cos ΦTM during training at the encoder (ENC) and decoder (DEC) heads for all baselines in the base and base+ setup on Github, Reddit, and Stack Overflow. "B" and '+" refer to the base and base+ models, respectively, and the distribution is obtained by pooling the values of Ørw over 5 training runs. As the decoders are disjoint in the base+ setting, note that cos ΦTM is not defined.
32

---

Published in Transactions on Machine Learning Research (01/2025)
H.6 Distributions of conflicting gradients between the base and base-D settings
Figure (1) show the distribution of cos ΦTM during training, as well as the proportion of conflicting gradients (CG), their average GMS, and their average TPI for THP, SAHP, and FNN in the base and base-D D setups on all datascts. We observe that using two identical and task-specific instances of the same decoder in the base-D setup for the time and mark prediction tasks mitigates the occurrence of conflicts. The decrease in conflicting gradients during training in turn associated to increased performance with respect to both tasks. We refer the reader to the discussion in Section of the main text for further details.
60 OV6236 TRALE
00-352 OPECEES TRISE4
CGO 50 OREO DO TREG
CG8.4 ONGGE TFEO.2
CG SPS333 TRERLEE
THP-ENC
THP-DEC
SAMPENC
SANP-DEC
FNN-ENC
PNN DEC
LanFM
Proportion
02
0.2
0.0
-3.0
-0.5
3.0
3.5
3.0
-1.0
-0.5
3.5
6.5
3.0
-1.
-85
0.3
05
-1.0
-35
6.3
0.5
1.0
-0.0
-0.5
0.5
2.0
-28
-05
3.
05
CR 88 GM5-3 IN TR:6.37
DCEE GMS 6.4 TAI3.3
R-CG3 6M5 99
CGI.0 GIG 0.6 T96
CRAN GIICELSS TRU
THP ENC
THP OBC
SAHP ENC
SAHP DEC
FNN ENC
FNN DEC
Proportion
MOOC
0.03
-1.0
60
3.5
3.0
-05
6-3
0.5
10.1
0.0
6.5
-10
-15
6.3
0.5
1.0
-0.
-0.5
0.5
-18
-05
1.5
65
COLO OPS 0.82 1P:0 be
B:C0: SF GNG 63 17E3.07
COVD 64 OMSIB 1P16.39
IN COIOUE 6I550 TRNOLE
095/0
IN CO-SS SP5/366 TRERUS
THI-ENC
THI-DEC
SAHMKNN
SAHP-DEC
FNN-ENC
PNN DEC
9.2
Proportion
Github
0:33
0.09
1.5
0.5
3.5
100
1.0 1/3
0.5
0.5
1/5
0.5
05
1/7
3.5
1.9
05
90
05
CO5
COC 61
61M
0O5
6M
000
91
000
CCC9 GP5:0-79 1776
B: P.COO G15.0 54 1715.77
DG-0 83 GPS:339 TE30:73
C.ot-33 6.76 TIN
00:0 30 GRE26.75 TT15.2)
D: CG0 24 046:0 72 19:3.34
B: C200.6 CIECE,S1 1110
DC:00:40 ONG0074 1110.34
B:C0-0 SP50.6 1I1R.35
THP-INC
THP-DEC:
SNHPENC
SAMP-DEC
FNN-ENC
FNN--DC
Prpprtion
Reddk
9.2
0:30
0:85
3.0 -1.0
-6.3
3.0
-1.
-85
0.3
8.5
1.5
15
1.5
-2.0
-0.5
-1.8
-05
55
0.5
cus 9PM
C00 67M
003 41M
0130 61M
0120
97M
C00 6T
C6.0 43 GM5.002 TW:334
B: 6: 87103
CR-3 GHS 083 TRI
D: 03c6-43 6.36 TWA
35.447 G450 d4 TRE
D: OG0 GV5:8 TR-6-30
B: 06:355 G.S. TRISSE
05:0/42 S4S TR
8: CG-5555 5V53 AS TRIAEE
325
THP ENC
THP DEC
SNHPIENC
SAHP DEC
PWN ENC
FNN DEC
Stack Overtoow
Proportion
222
0.33
0.83
3.0
-05 3.0 -1.
-85
0.3
05
13
85
1.0
-20
-0.
-18
-05
25
0.5
CO5 9MM
CO5 6mM
005 6M
CO5 8MM
Co2
9M
Co0 6m
Figure 15: Distribution of cos ΦтM during training at the encoder (ENC) and decoder (DEC) heads for all baselines in the base and base-D setup all datasets. 'B' and 'D' refer to the base and base- models, respectively, and the distribution is obtained by pooling the values of ΦrM over 5 training runs. As the decoders are disjoint in the base-D setting, note that cos ar is not defined.
33


Reviews:
[{"review_id": "fDqBDIGuHL", "reviewer": "Reviewer_6Ygz", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths:\n- The paper is very well written and the approach is well motivated.\n- The proposed approach is relatively simple yet very effective across different models.\n- Very thorough experimental section they investigated the usage of different parameterization schemes across  6 different models and 5 different datasets.\n- Over all I think its a strong paper with strong empirical results. \n\n\n\n# Weakness:\n- Results on more recent models like AttNHP.\n- GMS was introduced in the beginning of the paper as way to ensure that both gradients have the same magnitude the different parameterization schemes proposed in the paper did not help in this particular metric (since the have different gradients now I guess it doesn't matter since each model is separate) but it seemed confusing to introduce metric and not use it for comparison similarly to TGI.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes to model MTPP as two multi-task learning problems, where one task focuses on learning the distribution of the next event's arrival time conditional on historical events, and the other learns the distribution of the categorical mark conditional on both the event's arrival time and the historical events. The paper argues that modeling these two jointly with common model parameters might result in a sub-optimal solution due to conflicting gradients between the two tasks. The conflicting gradient refers to having two gradients in opposite directions where one has a significantly larger magnitude than the other, so the learning process tends to favor tasks with larger gradient magnitudes.\n\nThe paper proposes to use the following measures: (a) conflicting gradient as the cos $\\phi$ angle between the gradient of the two tasks, (b) gradient magnitude similarity (GMS) to measure the difference in magnitude of the two gradients; the paper argues that this only measures that there is a significant gradient difference to measure the direction of the difference (c) they propose the time priority index (TGI) between conflicting gradients to determine the direction of the conflict. Ideally, cos $\\phi$>0, while GMS is around 1. They then performed experiments on a set of MTPP models and showed that they exhibit conflicting gradients.\n\n# Approach:\n\nThe paper proposes to solve this problem by disjoint parametrizations of neural MTPP models they investigate:\n\n- \"Shared History Encoders and Disjoint Decoders,\" where a common history encoder encodes time and marks predictive history together, while the two predictive functions are modeled independently through separate decoders. They referred to this model as Base +.\n\n- \"Disjoint History Encoders and Disjoint Decoders.\" Each function has an independent history encoder and an independent decoder. They referred to this model as Base ++.\n\n- \"Shared History Encoders and Duplicated Decoders\" instead of having two decoders trained independently, they have two identical parametrizations of the same base decoder. They referred to this model as Base -D\n\n- \"Disjoint History Encoders and Duplicated Decoder\" is similar to Base ++, but instead, the decoders are duplicates of each other, as in Base -D. They refer to this as Base-DD\n\nHere, base-D and base-DD settings retain the same architecture as the base model.\n\n# Experiments\n\nThey evaluate these parameterization approaches on 6 different models and 5 different datasets.\n\n## Results:\n\n- Comparing regular training vs. Shared History Encoders and Disjoint Decoders (Base vs. base+), they reported that (a) generally conflicting gradient decrease, (b) GMS seems not to have changed much or often worsens, (c) decrease in both losses of mark and time.\n\n- When shared History Encoders and Disjoint Decoders are compared to Disjoint History Encoders and Disjoint Decoders (base+ vs. base++), mark loss improves.\n\n- Comparing regular training to Disjoint History Encoders and Decoders (Base vs. base++), it was found that the base++ models were better calibrated than the base models.\n\n- When regular training is compared to Shared History Encoders and Duplicated Decoders (Base vs. base-D), the conflicting gradient decreases, and task accuracy improves.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Please explicitly mention how you get CG in the text from cos $\\phi$.\n- Figures 5 and 7  is missing y axes.\n- Please comment on the GMS and TGI results in different experiments.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "br0NrDkHxd", "reviewer": "Reviewer_fNFi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n*) The paper is well written and the aim is well motivated.\n\n*) The suggested methods appear to be effective at bypassing the negative impact of conflicting gradients.\n\nWeaknesses:\n\n*) 3 runs do not allow for accurate estimates of the standard error (e.g. in Table 1). The results would be strengthened by increasing the number of runs.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "It is shown that the training of Marked Temporal Point Process (MTTP) constitutes a two-task learning problem. During training, the gradients of the different tasks can counteract each other. In the paper, the authors propose different ways of parametrizing the model to alleviate this issue. Multiple experiments are conducted showing improved alignment between the gradients (where applicable) as well as slightly improved performance on other metrics.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "*) Definition 3: \n\nThe formulation is unclear:\"s.t. $cos(\\phi_{TM})<0$\" seems out of place at its current position. For example,  \"... conflicting gradients g_T and g_M (i.e. when $cos(\\phi_{TM})<0$) ...\" would be clearer.\n\n*) Page 4: \"...most models also show a certain proportion of positive gradients ($cos \\phi_{TM} \\geq 0$) ...\"\n\n\"positive gradients\" -> \"non-conflicting gradients\"\n\nThis is a bit of an understatement, in many of the models the proportion of positive and negative alignment are almost equal. In Section 6, it would be interesting to comment on whether a bigger improvement in terms of the evaluation metrics is observed for models that had a higher proportion of conflicting gradients in Fig. 2.\n\n*) Figure 6:\n\n(left) (right) -> (right)", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "kg3J7DePnw", "reviewer": "Reviewer_JXsD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. It studies that conflicting gradients issue in the neural marked temporal point process, showing that this issue existed in multiple public models such as THP, SAHP and etc.\n2. It proposed a novel framework to address the conflicting gradient by disjointly modeling the conditional probability and intensity function in terms of history. \n\nWeaknesses:\n1. Need more discussion for the results. For the Table 5, since base model and base+ model has different parameters, comparing the cosine distance between gradients is not straight-forward. More discussion would be preferred.\n2. Although Table 1 show the benefits from disjoint modeling. But that benefits may comes from the model sizes since disjoint modeling would double model parameters for history modeling. More experiments and discussions are needed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work first investigated and validated that the inference in neural marked temporal point processes would be affected by the conflicting gradients issue in which the gradients from the temporal event modeling has opposite direction with the gradients from the mark events. To address this conflicting gradients issue, it proposed a framework in which it proposed disjoint history encoders/decoders.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See Weakness", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors have successfully addressed the reviewer feedback, which has strengthened the submission. All reviewers are in agreement that after modifications, the submission is ready for acceptance, which I agree with.
Decision: Accept (poster)