SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where the first term is the negative loglikelihood of the event sequence  $(\mathcal{L}_{nll}$  equals to  $-\mathcal{L}_{ll}$  defined in equation 1) and the second term refers to a surrogate loss to pull selected keys closer to corresponding query in the retrieval process.  $\alpha$  is a scalar to control the importance of the surrogate loss. Given the learned parameters, we may wish to make a minimum Bayes risk prediction about the next event via the thinning algorithm (Mei & Eisner, 2017; Yang et al., 2022).
Asynchronous Refresh of Prompt Pool. The prompts may lead to the variable contextual representation of the event as the parameters of the based model are continually updated. To accelerate training, we propose to asynchronously update all embeddings in the prompt pool every  $C$  training epochs.
Datasets and Evaluation Setup We conduct our real-world experiments on three sequential user-behavior datasets. In each dataset, a sequence is defined as the records pertaining to a single individual. The Taobao (Alibaba, 2018) dataset contains time-stamped user click behaviors on Taobao shopping pages with the category of the item involved noted as the event type. The Amazon (Ni, 2018) dataset contains time-stamped records of user-generated reviews of clothing, shoes, and jewelry with the category of the reviewed product defined as the event type. The StackOverflow (Leskovec & Krevl, 2014) dataset contains two years of user awards on a question-answering website: each user received a sequence of badges with the category of the badges defined as the event type. See Appendix D.1 for dataset details.
We partition Taobao and Amazon datasets into 10 consecutively rolling slides (namely 10 tasks) and partition the StackOverflow dataset into 6 rolling slides (namely 6 tasks). For the Taobao dataset, each slide covers approximately 1 day of time; for the Amazon dataset, each slide covers 2 years of time; for the StackOverflow dataset, each slide covers approximately 5 months time. The subset in each task is split into training, validation, and test sets with a  $70\%$ ,  $10\%$ ,  $20\%$  ratio by chronological order. Each task has no overlap in the test set. For a detailed discussion, a demonstration of the evaluation process is provided in Figure 9 in Appendix D.3.
Metrics. Following the common next-event prediction task in TPPs (Du et al., 2016; Mei & Eisner, 2017), each model attempts to predict every held-out event  $(t_i, k_i)$  from its history  $\mathcal{H}_i$ . We evaluate the prediction  $\hat{k}_i$  with the error rate and evaluate the prediction  $\hat{t}_i$  with the RMSE.
Base models. While our proposed methods are amenable to neural TPPs of arbitrary structure, we choose two strong neural TPPs as our base models: NHP (Mei & Eisner, 2017) and AttNHP (Yang et al., 2022), an attention-based TPP whose performance is comparable to or better than that of the NHP as well as other attention-based models (Zuo et al., 2020; Zhang et al., 2020).
Competitors. With NHP and AttNHP as base models, we trained PromptNHP (Pt-NHP) and PromptAttNHP (Pt-ANHP) in the proposed prompt-augmented setup and compared with 7 baselines.
- PretrainedTPP. PretrainedNHP (Pre-NHP) and PretrainedAttNHP (Pre-ANHP) represent NHP and AttNHP learned at the first task (time step) and not trained any longer.
(a) Taobao dataset
(b) Amazon dataset
Figure 3: Performance of all the methods on Taobao (up), Amazon (middle) and StackOverflow (down). In each figure, the subfigures from left to right are the evolution of type error rate and the time RMSE of each task, the average error rate, and the average time RMSE of all the tasks.
(c) StackOverflow dataset
- RetrainedTPP. RetrainedNHP (Re-NHP) and RetrainedAttNHP (Re-ANHP) refer to TPPs retrained at every sliding widow.
- OnlineTPP. As there is no prior work on online neural TPPs, we use online Hawkes process OnlineMHP (O-TPP) (Yang et al., 2017), trained in an online manner without any consideration for knowledge consolidation.
- CLTPP. The concurrent work (Dubey et al., 2022), to the best of our knowledge, is the only neural TPP with CL abilities proposed so far. Based on their work  $^{4}$ , we implement CL-NHP (CL-NHP) and CLAttNHP (CL-ANHP) as two variants of the hypernetwork-based CLTPPs.
Implementation and Training Details. For a fair comparison, they (except O-TPP which is a classical TPP model) are of similar model size (see Table 2 in Appendix D.4). For Pt-NHP and Pt-ANHN, we set  $M = 10$ ,  $N = 4$ ,  $L_{p} = 10$  for both datasets. During training, we set  $C = 2$  by default and explore the effect of asynchronous training in Analysis IV of section 4.2. More details of the implementation and training of all the methods are in Appendix D.5.
The main results are shown in Figure 3. Pre-NHP and Pre-ANHP work the worst in most cases because of inadequate ability to handle the distribution shift in the event sequence. Besides, O-TPP has a similarly poor performance because of two reasons: first it is a classical (non-neural) TPP with weaker representation power of modeling event sequence compared to its neural counterparts; second as a traditional online learning method, it easily loses memory of previously encountered data and suffers from catastrophic forgetting. Retraining at every task (Re-NHP and Re-ANHP) achieves
They have not published the code yet.
Figure 5: Effect of temporal prompt and prompting function of PromptTPP.
Figure 6: Effect of hyperparameters of PromptTPP on Amazon dataset.
moderate results but it also causes catastrophic forgetting. Not surprisingly, CL-NHP and CL-ANHP perform better than retraining, by applying a regularized hypernetwork to avoid forgetting. However, the hypernetwork relies on task descriptors built upon rich meta data, which limits its applicability and performance in our setup (and in real applications as well!). Lastly, our methods (both Pt-NHP and Pt-ANHP) work significantly better than all these baselines across the three datasets: they substantially beat the non-CL methods; they also consistently outperform CL-NHP and CL-ANHP by a relative  $4\% - 6\%$  margin on both metrics, thanks to our novel design of the CtRetroPromptPool, which successfully reduces catastrophic forgetting (see Analysis 0).
Analysis 0: How models perform on previous tasks after learning new events? We aim to validate that the improvement in performances indeed is due to the alleviation in catastrophic forgetting instead of simply a better fit on the current task. We use ANHP trained on task 9 and Pt-ANHP continuously trained on task 9, re-evaluate them on previous tasks and see how the metrics changed. Specifically, on Figure 4,
(i) Each number on the curves of ReANHP and CL-ANHP corresponds to the performance difference on the test
set of task  $i$ ,  $i < 9$  using ANHP trained on task 9 vs ANHP trained on task  $i$ .
Figure 4: The performance drop when re-evaluating the 0-8-th tasks using model trained on the 9-th task on Amazon dataset.
(ii) Each number on the curves of Pt-ANHP corresponds to the performance difference on the test set of task  $i$ ,  $i < 9$  using Pt-ANHP trained until (including) task 9 vs Pt-ANHP trained until task  $i$ .
See from Figure 4, on both metrics, we see the drop in performance (i.e., error rate / RMSE increases) of Pt-ANHP is much less significant than ANHP, indicating Pt-ANHP stores well the knowledge of previous tasks, which largely alleviates catastrophic forgetting.
Figure 7: Effect of asynchronous refresh and prompt related components of PromptTPP on dataset.
Analysis I: Does stronger base TPP model naively improve CL? Our method builds upon a backbone TPP and understanding this question is important for fair comparisons and future research. From Figure 3, Re-ANHP makes no consistent improvement against Re-NHP on average CL performance, which indicates a stronger TPP is not a solution for CL without being appropriately leveraged. Besides, for the CL-based methods, CL-ANHP is tied with CL-NHP on Taobao and makes a limited advancement against CL-NHP on Amazon, while Pt-NHP and Pt-ANHP perform closely on both datasets. Therefore, we can conclude that, although AttNHP is a more robust base model than common non attention-based TPP, i.e., NHP, it is not necessarily translated to CL performance.
Analysis II: Temporal prompt vs standard prompt. For a fair comparison, we initialize a pool of standard prompts without time-varying parameters by fixing their temporal components  $P_{t}$  to be an all-ones matrix and incorporate it into the base model AttNHP. This method is named Pt-ANHP-standard. With other components fixed, we compare Pt-ANHP-standard with Pt-ANHP to validate the effectiveness of the temporal prompt introduced in section 3.2.
Figure 5a shows that Pt-ANHP achieves better performance on both datasets: the introduction of temporal prompts slightly improves the RMSE metric and reduces the error rate with a larger margin. We did the paired permutation test to verify the statistical significance of the improvements. See Appendix D.6 for details. Overall, on both datasets, we find that the performance improvements by using the temporal prompts are enormously significant on error rate (p-value  $< 0.05$ ) and weakly significant on RMSE (p-value  $\approx 0.08$ ).
Analysis III: How to better attach prompts? We explore how to attach prompts and enhance their influences on overall performance. We compare three types of prompting: ① Naive Prompting (N-P), where the retrieval and prompting are performed after the event embedding layer: we replace  $h_i$  with  $x_i$  in equation 7, presuppose the selected prompts to  $x_i$  and pass it to the rest structure of TPP. ② Prompt Tuning (Pro-T): which concurrently presuppose the prompts to query, key, and value, introduced at the end of section 3.3. ③ Prefix-Tuning (Pre-T), proposed in the main body of section 3.3, which is the prompting method used in PromptTPP.
In Figure 5b, we observe that Pre-T leads to better performance on both datasets compared to those two variants. Despite its empirically better performance, the architecture of Pre-T is actually more scalable and efficient when attached to multiple layers since it results in unchanged output size:  $h_i^{Pre - T} \in \mathbb{R}^D$  remains the same size as the input while  $h_i^{Pro - T} \in \mathbb{R}^{(L_p*N + 1)\times D}$  increases the size along the prompt length dimension.
Analysis IV: Efficiency of our method. We examine the efficiency of our method in two steps:
- Firstly, seen from Table 2, on both datasets, Pt-NHP / Pt-ANHP leads to a  $12\% / 8\%$  total parameter increase to the base model, which in fact causes a marginal impact on training speed: Figure 7a shows that learning curves of Re-ANHP and Pt-ANHP  $(C = 1)$  converge at almost the same speed to achieve competitive log-likelihood, respectively.
- Furthermore, to accelerate the training (especially when introducing large size prompts), we introduce the asynchronous refresh mechanism (see section 3.4) with prompts updated in a frequency  $C > 1$  (refresh the prompt pool less frequently). We observe in Figure 7a that Taobao training with  $C = 2$  has a comparable performance with  $C = 1$  while Amazon training with  $C = 2$  improves the convergence notably.  $C = 4$  leads to no advancement.
Overall, PromptTPP only adds a small number of parameters so that it generally has the same convergence rate as the base model. The asynchronous prompt optimization scheme with  $C = 2$  improves the convergence more remarkably on the Amazon dataset. In addition, we indeed provide a complexity analysis. See Appendix D.7.
Analysis V: Effect of prompt related components of our method. Firstly we completely remove the CtRetroPromptPool design (w/o CtRroPP in Figure 7b) and use a single temporal prompt to train tasks sequentially. The performance declines with a notable drop, indicating that a single prompt suffers severe catastrophic forgetting between tasks, while our design of CtRetroPromptPool encodes task-invariant and task-specific knowledge well. Secondly, we remove the learnable key associated with prompts (w/o  $k - v$  in Figure 7b) and directly use the mean of prompts as keys. This strategy causes a moderate drop in performance. To conclude, learnable keys decouple the query and prompt learning processes and markedly contribute to the performance.
Analysis VI: Effect of hyperparameters of our method. We evaluate how the performance of PromptTPP changes as we vary three key hyperparameters: (i) prompt length  $L_{p}$ , (ii) selection size  $N$ , and (iii) prompt pool size  $M$ . Theoretically,  $L_{p}$  determines the capacity of a single prompt (which jointly encodes certain knowledge),  $L_{p} \times N$  is the total size used to preprocess the event vector, while  $M$  sets the up limit of the capacity of learnable prompts.
- Prompt length  $L_{p}$  and selection size  $N$ . From the results in Figure 6a, a too small  $L_{p}$  negatively affects results as a single prompt has a too limited ability to encode the knowledge. Besides, given an optimal  $L_{p}$ , an overly large  $N$  makes the total prompts excessively oversized, leading to underfitting and negatively impacting the results. We conclude that a reasonably large  $L_{p}$  and  $N$  enable the model properly encode the shared knowledge between the tasks of event sequences and substantially improve the predictive performance.
- Prompt pool size  $M$ . Figure 6b illustrates that  $M$  positively contributes to the performance. This is because the larger pool size means the larger capacity of the prompts.
In summary, this paper has proposed a groundbreaking framework, known as PromptTPP, for modeling streaming event sequences. By incorporating a continuous-time retrieval prompt pool, the framework effectively facilitates the learning of event streams without requiring rehearsal or task identification. Our experiments have shown that PromptTPP performs exceptionally well compared to other competitors, even under challenging and realistic conditions.
6 Limitations and Societal Impacts
Limitations. Our method uses neural networks, which are typically data-hungry. Although it worked well in our experiments, it might still suffer compared to non-neural models if starved of data.
Societal Impacts. By describing the model and releasing code, we hope to facilitate probabilistic modeling of continuous-time sequential data in many domains. However, our model may be applied to unethical ends. For example, it may be used for unwanted tracking of individual behavior.
A Discussion on Classical Schemes for Modeling Event Sequence
As shown in Figure 1, a common approach is to use sliding windows to frame the data for model training and prediction. In this setup, we discuss three classical schemes.
- Pretrained TPP. A straightforward solution is to pretrain a TPP in the first train set and use it for all the following test periods. Such an approach faces the problem of distribution shift, i.e., the new data is systematically different from the data the model was trained on Snoek et al. (2019). Take the Taobao dataset (Alibaba, 2018) for example, which contains time-stamped user click behaviors  ${}^{5}$  . We split the dataset by timestamps into 10 periods sequentially and compute the 3S statistics (Shchur et al., 2021) as a measure of the distribution of event sequences for each period. Figure 8a shows that the 3S statistics differentiate between periods while Figure 8b illustrates an increasing KL divergence of the 3S statistics between the first and later period, implying a pattern shift over time. As a result, this approach may fail to adapt to new data and produce unsatisfactory predictions.
- Retrained TPP. Another classical solution is to train a new TPP on the data of sliding windows over again. The TPP can quickly adapt to new data but may suffer from catastrophic forgetting (McCloskey & Cohen, 1989): adaptation usually implies that the model loses memory of previously encountered data that may be relevant to future predictions. For example, Figure 8a shows a large overlap in distributions of different periods on the Taobao dataset, indicating the necessity of maintaining the knowledge of existing patterns to improve generalization (Snoek et al., 2019; Wang et al., 2020).
- Online TPP. A better solution is an online approach: discretize the time axis into small intervals and then incrementally update the TPP at the end of each interval using an online algorithm. However, online models are generally more difficult to maintain and may also cause catastrophic forgetting (Hoi et al., 2021). Besides, to the best of our knowledge, apart from online classical TPPs (Yang et al., 2017; Hall & Willett, 2016), the field of online neural TPPs is much less well-studied.
B Related Work Details
Here we draw connections and discuss differences between our method to related works.
Temporal Point Process. A large variety of Neural TPPs have been proposed over recent decades, aimed at modeling event sequences with varying sorts of properties. Many of them are built on recurrent neural networks (Du et al., 2016; Mei & Eisner, 2017; Xiao et al., 2017; Omi et al., 2019; Shchur et al., 2020; Mei et al., 2020; Boyd et al., 2020). Models of this kind enjoy continuous state spaces and flexible transition functions, thus achieving superior performance on many real-world datasets, compared to classical models such as the Hawkes process (Hawkes, 1971). To properly capture the long-range dependency in the sequence, the attention mechanism (Vaswani et al., 2017) has been adapted to TPPs (Zuo et al., 2020; Zhang et al., 2020; Xue et al., 2021; Qu et al., 2023; ?; Wang et al., 2023; Shi et al., 2023) to enhance the predictive performance. However, learning the event sequence under the stream setting is largely unexplored. To the best of our knowledge, there exist two prior works (Yang et al., 2017; Hall & Willett, 2016) that propose online learning algorithms for classical TPPs while that for neural TPP have rarely been studied. We show our method works better than classical online TPPs in practice (see section 4.2).
Continual Learning. There is also a rich existing literature on CL: the models can be categorized into regularization-based methods (Kirkpatrick et al., 2017; Zenke et al., 2017), which regularize important parameters for learned tasks, architecture-based methods (Rusu et al., 2016; Mallya & Lazebnik, 2018) which assign isolated parameters for each task and rehearsal-based methods (Cha et al., 2021; Buzzega et al., 2020) which save data from learned tasks in a rehearsal buffer to train with
5Please see Appendix D.1 for the details of the Taobao dataset and Appendix D.2 for the explanations on 3S statistics and the procedure of the experiment on distribution shift.
Figure 8: Analysis of distribution shift on Taobao dataset.
the current task. In retrospect, we realize a concurrent work (Dubey et al., 2022) which also augments TPP with CL abilities. Important distinctions of their work from ours include: 1. setup: they use standard TPP train-valid setting while we formalize more realistic streaming setting to train/validate the models; 2. methodology: they model the event streams with a hypernetwork-based regularizer while we use a trainable prompt pool with more flexibility and generality in CL. 3. task agnostic: they rely on a task descriptor built from meta attributes of events while we do not - our method is task agnostic. As their source code is not available yet, we independently implement it and our method still outperforms it (see section 4.2).
Prompt Learning. Prompt-based learning (or prompting), as an emerging transfer learning technique in NLP, applies a fixed function to condition the model so that the language model gets additional instructions to perform the downstream task. Continuous prompts have also been proposed (Lester et al., 2021; Li & Liang, 2021) to reduce prompt engineering, which directly appends a series of learnable embeddings as prompts into the input sequence, achieving outstanding performance on transfer learning. Wang et al. (2022a,b) connect prompting and CL, which attaches prompts to the pretrained backbone to learn task-invariant and task-specific instructions. Note that it is non-trivial to apply prompting to neural TPPs (See Analysis II and III in section 4.2), and our proposed novel framework reveals its values to event sequence modeling.
C.1 PromptTPP at Training and Test Time
The training and test time Algorithms for PromptTPP are illustrated in Algorithm 1 and Algorithm 2, respectively.
For simplicity of notations, in test time, we show how to sample the next event given one historical event sequence via the thinning algorithm (Mei & Eisner, 2017), which can be easily extended to batch-wise inference (see the implementation in EasyTPP (Xue et al., 2023)).
D.1 Dataset Details
We evaluate our methods on three industrial user behavior datasets. We provide details on the preparation and utilization of each below. For both datasets, users are associated with anonymous aliases to remove personally identifiable information (PII).
Taobao (Alibaba, 2018). This dataset contains time-stamped user click behaviors on Taobao shopping pages from November 25 to December 03, 2017. Each user has a sequence of item click events where each event contains the timestamp and the category of the item. Following the previous work (Xue et al., 2022), the categories of all items are first ranked by frequencies, and the top 19 are kept while the rest are merged into one category, with each category corresponding to an event type. We work
Algorithm 1 PromptTPP at training time of the  $T$ -th task.
Input: Train set  $\{s_{train}\}$ . CtRetroPromptPool  $(\mathcal{K},\mathcal{V}) = \{(k_i,P_i)\}_{i = 1}^M$  (inherited from the previous task), score function  $\varphi$ , loss weight  $\alpha$  and asynchronous refresh frequency  $C$ .  
Output: Trained base model with a encoder  $f_{\phi_{enc}}$  and a decoder  $f_{\phi_{dec}}$ ; trained CtRetroPromptPool  $(\mathcal{K},\mathcal{V}) = \{(k_i,P_i)\}_{i = 1}^M$ .  
1: procedure TRAIN(\{strain\},  $(k_i,P_i)\}_{i = 1}^M,\varphi ,\alpha ,C)$   
2: for epoch_id in total_epochs:  
3: Draw a mini batch  $B$   
4: For illustration purposes, we use batch size = 1 here. The computation here can be easily extended to batch size ≥ 1.  
5: for  $s_{[0,T]}$  in  $B$ :  
6: Update the loss  $\leftarrow$  CALCLOSS(s[0,T],  $f_{\phi_{enc}},f_{\phi_{dec}},\alpha$ )  
7: Update  $f_{\phi_{enc}},f_{\phi_{dec}}$  by backpropagation.  
8: if epoch_id %C == 0: Update  $(k_i,P_i)\}_{i = 1}^M$  by backpropagation.  
9: procedure CALCLOSS(s[0,T],  $f_{\phi_{enc}},f_{\phi_{dec}},\alpha$ )  
10:  $\mathcal{L}\gets 0$ .  
11: ▷ Recursively compute the loss.  
12: for e@t in  $s_{[0,T]}$ :  
13: ▷ Compute the likelihood loss; the technical details can be found in Mei & Eisner (2017) and Yang et al. (2022).  
14:  $\mathcal{L}_{event}\gets$  Take a sum of log intensity at the event time by calling CALCINTENSITY(s[0,t],e@t,  $f_{\phi_{enc}},f_{\phi_{dec}},(k_i,P_i)_i = 1$ ).  
15:  $\mathcal{L}_{non\_ event}\gets$  Integrate log CALCINTENSITY(s[0,t],e@t) over inter-event time interval.  
16:  $\mathcal{L}_{nll}\gets \mathcal{L}_{non\_ event} - \mathcal{L}_{event}$   
17: ▷ Compute the matching loss.  
18:  $\mathcal{L}_{matching}\gets \sum_{K_{top-N}}\varphi (f_{\phi_{enc}}(e@t),kr_j)$   
19:  $\mathcal{L}\gets \mathcal{L}_{nll} + \alpha \mathcal{L}_{matching}$   
20: return  $\mathcal{L}$   
21: procedure CALCINTENSITY(s[0,t],e@t,  $f_{\phi_{enc}},f_{\phi_{dec}},(k_i,P_i)_i = 1$ )  
22: s[0,t]  $\leftarrow$  Append e@t to history s[0,t].  
23: Encode s[0,t] by  $f_{\phi_{enc}}$  to generate the hidden state  $h_t$ .  
24: Matching the index  $r_{jj = 1}^{N}$  based on equation 7.  
25: Select Top-N prompts  $\{P_r_i\}_{i = 1}^N$ .  
26: Prepend  $\{P_r_i\}_{i = 1}^N$  to  $h_t$  and pass to the decode  $f_{\phi_{dec}}$  to generate the intensity  $\lambda_e(t),e\in$  {1,...,E}.  
27: return  $\lambda_e(t),e\in \{1,\dots,E\}$
on a subset of 4800 most active users with an average sequence length of 150 and then end up with  $K = 20$  event types.