Long-range Modeling and Processing of Multimodal Event Sequences
Authors: Jichu Li
Yilun Zhong
Zhiting Li
Feng Zhou
Quyu Kong
Venue: ICLR 2026 Poster (2026)
Keywords: Temporal Point Process
Multimodal LLM
Primary area: learning on time series and dynamical systems

Abstract: Temporal point processes (TPPs) have emerged as powerful tools for modeling asynchronous event sequences. While recent advances have extended TPPs to handle textual information, existing approaches are limited in their ability to generate rich, multimodal content and reason about event dynamics. A key challenge is that incorporating multimodal data dramatically increases sequence length, hindering the ability of attention-based models to generate coherent, long-form textual descriptions that require long-range understanding. In this paper, we propose a novel framework that extends LLM-based TPPs to the visual modality, positioning text generation as a core capability alongside time and type prediction. Our approach addresses the long-context problem through an adaptive sequence compression mechanism based on temporal similarity, which reduces sequence length while preserving essential patterns. 
We employ a two-stage paradigm of pre-training on compressed sequences followed by supervised fine-tuning for downstream tasks. Extensive experiments, including on the challenging DanmakuTPP-QA benchmark, demonstrate that our method outperforms state-of-the-art baselines in both predictive accuracy and the quality of its generated textual analyses.

Published as a conference paper at ICLR 2026
LONG-RANGE MODELING AND PROCESSING OF MULTIMODAL EVENT SEQUENCES
Jichu Li Yilun Zhong' Zhiting Li", Feng Zhou 1.21 Quyu Kong"i
Center for Applied Statistics and School of Statisties, Renmin University of China 2 "Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing 3 IIndependent Researcher
{lijichu52, kongquyu] gmail 1 com, feng. zhou@ruc. edu. en
ABSTRACT
Temporal point processes (TPPs) have emerged as powerful tools for modeling asynchronous event sequences. While recent advances have extended TPPs to handle textual information, existing approaches are limited in their ability to gen- erate rich, multimodal content and reason about event dynamics. A key challenge is that incorporating multimodal data dramatically increases sequence length, hin- dering the ability of attention-based models to generate coherent, long-form tex- tual descriptions that require long-range understanding. In this paper, we propose novel framework that extends LLM- TPPs the visual modality, position- ing text generation as a core capability alongside time and type prediction. Our approach addresses the long-context problem through adaptive sequence pression mechanism based on temporal similarity, which reduces sequence length while preserving essential patterns. We employ two-stage paradigm of pre- training on compressed sequences followed by supervised fine-tuning for down- stream tasks. Extensive experiments, including on the challenging DanmakuTPP- QA benchmark, demonstrate that our method outperforms state-of saute-of-the-ar base- lines in both predictive accuracy and the quality of its generated textual analyses. Code is publicly available at https///gitub.coo//cccni/.MMM
INTRODUCTION
Temporal point processes (TPPs) (Daley & Vere-Jones 2003 2007) are statistical models designed for analyzing asynchronous event sequences continuous time. They have found widespread ap- plications across diverse domains including neuroscience (Zhou et al. 2021). ), finance (Bacry et al.] 2015), crime analysis (Mohler 2013), and epidemiology (Rizoiu al. 2018). While traditional sta- tistical TPPs can model event timing and dependencies, they often lack the expressiveness needed to capture complex real-world patterns. Recent advances in deep learning have led to the devel- opment of deep TPPs (Du et al. 2016 Mei & Eisner [017]; Zuo et al. 2020 Meng et al.) 2024), which demonstrate superior modeling capabilities. Building the success of large language models (LLMs), Language-TPP TPP (Kong et al., (025)) further extended TPPs to incorporate textual informa- tion, showing the potential of integrating natural language processing with temporal modeling.
However, real-world event sequences are increasingly becoming multimodal, extending beyond just temporal and textual information. For instance, in video streaming platforms, user comment streams (known as Danmaku) contain not only timestamps and textual content but also associated visual frames from the video (Jiang et al.) 2025). Similarly, traffic monitoring systems capture accident re- ports that include temporal information, textual descriptions, audio recordings. and visual evidence from surveillance cameras. Current TPP models, including Language- TPP, remain largely confined to limited modalities and lack the capability to effectively model and generate rich multimodal con- tent such as images alongside text and temporal information.
"Equal contribution.
1 Corresponning authors.

---

Published as a conference paper at ICLR 2026
Extending deep TPPs to be fully generative in multimodal settings presents two primary chal- lenges. First, existing TPP models lack a unified framework for multimodal generation. While Language-TPP TPP has progressed in incorporating text, it cannot handle visual content or generate text that is conditioned on it. Different modalities require distinct encoding strategies (Lahat et al. 2015), and truly generative framework must not only encode this heterogeneous data but also produce contextually aware content- as generating textual description for future event that cor- rectly references its associated visual information. To address this, we propose a unified framework that extends Language-TPPe TPP's template- cemplate-based based approach to incorporate visual information, enabling comprehensive multimodal event modeling and generation.
Second, incorporating multimodal data dramatically increases sequence length, hindering the ability to generate coherent, long-form text that depends on long-range dependencies. This in- flation occurs because non-textual modalities are themselves tokenized into sequences; for example, single image commonly decomposed into hundreds of patch-based tokens (Dosovitskiy et al,) 2021. In event sequences where each point is associated with such data, the total length N can be- come exceptionally large. This makes the (NN)) complexity of the Transformer's 's (Vaswani 2017) self-attention mechanism critical bottleneck. This prevents the model from accessing the full his- tory needed to generate comprehensive analytical report or answer summary questions. To mitigate this, we propose adaptive compression mechanism that leverages temporal similarity reduce sequence length while preserving the dynamics required for effective long-range text generation.
In summary, to address the above challenges, we propose Multimodal Temporal Point Processes (MM-TPP), a unified framework that tackles these issues through the following contributions:
(1) We propose MM-TPP, extension of Language-TPP event time and type while generating rich text conditioned on visual, textual, and temporal inputs.
multimodal framework that predicts
(2) We introduce adaptive compression mechanism based temporal similarity that reduces sequence length, making the generation of long-form, context-aware text computationally feasible.
(3) We construct TAXI-PRO, a new multimodal TPP dataset by augmenting the classic NYC Taxi dataset with visual map patches and textual descriptions, providing a valuable benchmark.
(4) Comprehensive experiments on two multimodal TPP datasets, including the DanmakuTPP-QQ benchmark, demonstrate the superiority of MM-TPP in predictive accuracy compared to SOTA base- lines. Furthermore, qualitative analysis reveals that our model generates significantly more insightful and coherent textual analyses, showcasing its superior reasoning capabilities.
2 RELATED WORKS
Deep TPPs. Deep learning-basse TPPs were first introduced by Du et (2016) using an RNN- based framework. Follow-up works improved this approach, including Mei & Eisner (017)); |; Yang al. (2018): Omi al. (2019): Soen al.] (202T): Chen al. (2024)) Recurrent models offer efficient inference, with constant-tima prediction and linear training cost, but suffer from limited parallelism and poor modeling of long-range dependencies. To address these issues, Transformer- based TPPs emerged, starting with Zuo et al. (2020) and Zhang et al.] (2020). and later extended by works such as Zhu et al. (002)) ; Zhou et al. (202)); Yang et al. (2022); Meng et al. (2024); Panos 2024). These models enable parallel training and better capture long-range dependencies via self-attention, but require higher computational cost cost-both time and memory scale as O
In contrast to these approaches, MM-TPP not only handles long-range dependencies efficiently but also extends TPP modeling to generate and reason over rich multimodal content.
Covariate TPPs. Covariate TPPs enhance traditional models by incorporating contextual informa- tion to better explain event dynamics and improve predictions. Early works focused on structured covariates-es e.g. Meyer et al.] (2012) used demographic data for epidemic modeling, Xiao et al. (017)) proposed a dual-LSTM for combining covariates and event history, and Adelfio & Chiodi 2021 Gajardo & Müller (2021) applied similar ideas to earthquake and COVID-19 data. In prac- tice, many covariates are unstructured, such as text, images, or audio, making their integration into TPPs challenging. Some studies tackled this by using word frequeney features (Zhu & Xie 2022) or BERT embeddings (Zhang et al.] 2023). More recent efforts leverage LLMs to jointly model MM-TPP builds on
temporal and textual information, as in Liu & Quan 2024 Kong et al. 20255
2

---

Published as a conference paper at ICLR 2026
Qwen2.5 LM Decoder
Dent
Typect
Pypect
Pypet
lose Tima tales
Image
Ewart token
Vision Encoder
Compressed
Eeart Type tolan
Rored The
Event Sequence
Image salen
video frames
Simller tokere
Sirn la Lvanle
(Teet, Type, Time, Imagei
Sequence
a
(b)
Figure I: Overview of the MM-TPP framework. (a) Model Architecture: The framework built upon Qwen2.5-VI, where multimodal coons--oossssingg of time, type, text, and image informa- tion-are are tokenized into unified token sequences. Visual content encoded by the vision encoder and fused with other modalities via the language model decoder to perform autoregressive next- event prediction. (b) Adaptive Long Sequence Compression: Events with similar time intervals ([T; Ti- < A) are compressed into special token. This strategy enables efficient processing of long event sequences while preserving temporal structure and key event information.
this direction by incorporating multiple types of unstructured covariates speciaiialsss.ciillllll...l images and text-into a unified TPP framework, and further enables multimodal generation for future events.
3 PRELIMINARIES
This section introduces the foundational concepts and background knowledge for our proposed model, including TPPs and LLM-based TPPs modeling.
3.1 TEMPORAL POINT PROCESSES
TPPs are probabilistic models designed for sequences of discrete events occurring in continuous
time. A typical realization of TPP is denoted by S {(t, e)) where each (t,, e.) represents an event of type e; E E {1.... , E that occurs at time l E (), satisfying 0 < 4 < < IN T. Here, N denotes the random number of events observed in the time window (0, 7). There are multiple equivalent formulations to characterize TPPs. One common approach is to model the conditional distribution of future events given the past. Let Hr denote the event history up to and including time t. We define p(f++1. ei+1 event occurring time f,41 with type c++1. given the past history He Under this formulation, the probability of observing sequence S can be factorized as follows:
1==1
the conditional density function for the next
p(S) I H....) P(T H)w)),
i=1
where 1 1 P(T) Hex) represents the probability that events occur in (tN,T), P(T) Hey) Jey p(t | Hew) dt is the cumulative density of the next event time after tv. and p(t Hi,) = ΣE D(le H.,). standard to decompose the conditional joint density as:
p(l,e (H;) p(( HH,,) p(e H),,
Therefore, based the above formulations, the log-likelihood of a TPP can be derived as follows:
logp(S) Σ lo -lo)) (og.pee)(tt...)) log (1 - P(T) Hex)).
(1)
3.2 LANGUAGE-TPP
Recent advances in LLMs have enabled new approaches to TPP modeling. Early studies adapted pre-trained LLMSs to temporal event data using customized tokenization and supervised fine-
3

---

Published as a conference paper at ICLR 2026
tuning Liu & Quan 2024 Xue et al.) 2023 Shi et al. 2024). Following this direction, Kong et al. (2025) proposed Language- a framework for modeling event sequences of the form
(ti, ei, m where each event includes a timestamp, type, and textual description. The model captures dependencies across the sequence and jointly predicts future event attributes.
To achieve this, Language-TPP adopts a decoder-only causal Transformer and introduces several components. Each event is tokenized by applying a standard language tokenizer e; and mi while t; is encoded using a byte-level strategy with 256 special tokens, allowing each 32-bit times- tamp to be represented by four tokens. The resulting tokens are organized into a structured event template, where each event is enclosed by < start_to _event/ |> and < led_of__verrs |> to- kens, and each component is prefixed by a modality-specifie marker (e.g., < <|time.prefix > for timestamps). The model then performs autoregressive generation until an end-of-sequence to- ken is produced, from which the timestamp, type, and description of future events can decoded. Language- TPP is trained in two stages. First, event sequences converted into token sequences and the backbone LLM is pre-trained using standard next-token prediction. In the second stage, prompt-response pairs are constructed for each subtask (time prediction, type classification, and description generation). and fine-tuning is performed on the response portion using next-token loss.
While Language- TPP effectively incorporates textual information into TPP modeling, it is limited to a single modality and faces scalability issues when handling long multimodal sequences. Our proposed MM-TPP extends this framework by incorporating both visual and textual modalities, and introduces a temporal similarity- compression mechanism to address the sequence length explosion in multimodal settings.
METHODOLOGY
To model multimodal event sequences with rich visual and textual information under long-context constraints, propose MM-TPP, extension of Language- TPP. It jointly models temporal, cat- egorical, textual, and visual data, and introduces a compression mechanism to enhance long-range dependency handling.
4.1 MODEL FRAMEWORK
Fig. 1a provides overview of the framework of MM-TPP. Our model is built upon Qwen2.5- VL Bai et al., 2025), a SOTA pre-trained multimodal LLM. The entire framework follows a sequence-to-sequence paradigm, taking multimodal event history sequence input and autore- gressively predicting various attributes of future events. Extending the annotations from Kong et al.) 2025 given a multimodal event sequence (t,, m2, ()) where &4 denotes the event timestamp, e represents the event type, m; the textual description, and v; is the associated visual description, the goal is to model the dependencies within such a sequence and predict the time, type, textual content of future events.
4.2 MULTIMODAL EVENT SEQUENCE TOKENIZATION
To handle multimodal event data with language models, we design tokenization scheme that con- verts each event tuple (t,, e;, m,, v;) into a unified token sequence suitable for autoregressive mod- eling. Temporal information encoded following the design of Language-TPP, where each 32-bit time interval is decomposed into four bytes based on its memory layout. These bytes are mapped 256 special tokens, <| byte 1 to <byyte_ 255 >>, allowing for compact and precise repre- sentation of time intervals. Categorical information, such as event types, is handled using dedicated special tokens (e.g., < typee_ >> to < type_5| |> for six event categories), enabling unified and interpretable modeling of discrete labels.
For textual information, event descriptions (e.g., user comments) are tokenized using the built-in tokenizer of Qwen2.5-VL, which converts raw text into standard language tokens that can be directly processed by the model. Visual information is integrated differently: instead of converting image pixels into tokens, insert placeholder token < image.pad > the appropriate position in the sequence. During model execution, the corresponding image is passed through Qwen2.5- V-L'ss vision encoder to produce visual embeddings, which then aligned with the placeholder token.
4

---

Published as a conference paper at ICLR 2026
This design allows for deep fusion of visual, textual, and temporal features while keeping the token sequence compact and language-model-cone compailom
To organize these multimodal components, define a structured event template. Each event is enclosed by < slstart.oo of event >> and < end of event >> tokens. Within the event, cach modality is marked by prefix token: <| t.ime-starte |> for time, < <Itype_star streesstart// for type, | > for text, and <| aassionstartttt > for image. This structured template pro- vides consistent and interpretable format for encoding complex multimodal events. A full example of the uncompressed event format is shown in Appendix
<
4.3 ADAPTIVE LONG SEQUENCE COMPRESSION
The inclusion of text and image tokens in multimodal event sequences significantly increases input length, posing challenges for Transformer- based LLMs due to high computational cost and limited context windows. To mitigate this, we introduce an adaptive compression strategy based on temporal similarity between events. In many real-world settings- arrive in bursts or follow periodic rhythms, resulting in similar inter-event intervals. MM-TPP lever- ages this observation by compressing sequences of temporally similar events, as shown in Fig. lb
video comment streams- events
During input construction, we define a temporal similarity threshold A and compare the interval T, 4 1 of the current event with that of the previous one T;-1- If the difference between the two is smaller than the threshold, i.e., I T=-1| < A, event i is deemed temporally similar to event 1. In this case, instead of encoding using the full multimodal template, is replaced with a single special token <| s iimimii lar event >>, thereby reducing the number of tokens. Details on choosing an appropriate value for A are discussed in Appendix [A.2] Otherwise, the time intervals differ significantly, event i is represented in full using the multimodal template defined in Section [.2 For a concrete step-by-step illustration of this sequential compression mechanism applied to consecutive events, please refer Appendix A.I
This strategy enables compact representation dreds of tokens-uset using only few < | Limmilrr-eent >> tokens, while preserving key events with distinctive timing patterns. As a result, MM-TPP can utilize the fixed context window (e.g., 4096 tokens) more effectively, allowing it to model longer event histories and capture long-range dependencies with improved efficiency.
dense event clusters-potenniaall requiring hun-
It is worth noting the distinction between our proposed sequence-level compression (inter-event) and standard representation-level reduction techniques (intra-event) commonly employed in efficient MLLMs, such token pruning or merging (Bolya et al.] 2023 Chen al. 2024a Dhouib al. 2025 Alvar et al. 2025). While these methods effectively reduce computational cost by exploiting spatial redundancy within individual modalities (e.g., visual patches), they are less applicable the structured components of TPPs. Unlike images, key components of TPPs such as timestamps (t,), event types (ez), and text deseriptions (m,) contain dense semantic information. Every time value and word is precise and necessary for understanding the event sequence, so removing them risks losing critical logic. Therefore, we prioritize a sequence-level strategy that compresses temporally similar events. This approach effectively captures the bursty nature of TPPs and allows the model to accommodate a much longer history within the limited context window. We leave the design of more advanced, hybrid compression strategies that adapt to more complex and diverse scenarios for future work.
4.4 TRAINING AND INFERENCE
Model Training. To enable the model to understand complex multimodal event structures and the proposed compression strategy, we adopt two-stage training framework inspired by Language-TPP In the first stage, continued pre-training is conducted on large-scale token sequences constructed from the defined multimodal event templates, which may include both full and compressed repre- sentations. This stage helps the model adapt to the new event format and learn the semantics of special tokens such < slsmmlml wiilla e.eeette > and type X > The objective follows standard next-token prediction:
f. Σ
Lssugel (8) =
log ( xes),
(2)
L
i=1
5

---

Published as a conference paper at ICLR 2026
where 21,...,42 ,. denotes the event token sequence.
The second stage involves supervised fine-tuning for downstream tasks, including time prediction, type classification, and text generation. Training samples are formatted into prompt-response pairs, where the prompt consists of a compressed historical segment and a task-specific instruction to- < yppe soype--reicct.... prediction]
ken (e.g., <itime prediction and the response contains the ground-truth tokens (e.g.. byte tokens for time, type token, or text sequence). The training loss applied only the response tokens:
natural language query),
1 Σ log po(ry Prompt, R
Stuge (0)
(3)
where "l...., -' denote the response sequence. This two-stage process enables the model to learn both general sequence modeling and task specific reasoning over multimodal event data.
Model Inference. During inference, the model follows an autoregressive decoding strategy sim- ilar to Language-TPP. Given historical sequence and a task-specific instruction, sequentially generates output tokens. Byte tokens are decoded into poaaingg-pong intervals for time prediction, categorical tokens indicate event types, and language tokens are interpreted as text for generation or QA tasks. Since Qwen2.5-VL (Bai et al. (025) does not support image generation, our current model focuses on time, type, and text outputs. Future extensions may leverage omni-modal models such Chameleon (Team 2025) enable image generation for predicted events.
5 EXPERIMENTS
We conduct comprehensive experiments to evaluate the effectiveness of our proposed MM-TPP multiple dimensions, including overall performance against SOTA baselines, text generation quality, the impact of adaptive compression mechanism, and the contribution of multimodal infor- mation integration.
5.1 EXPERIMENTAL SETUP
Our MM-TPP model is built upon the Owen2.5-VL-31 architecture and fine-tuned using LoRA (Low-Rank Adaptation) (Hu et al. 202T). All experiments are conducted on a single NVIDIA RTX 4090 GPU. For Stage (continued pre-training), we train the model for 5 epochs with a learning rate of 10-4 In Stage (supervised fine-tuning), we also use a learning rate of 10-4 for epochs. The temporal similarity threshold for our compression mechanism is set to A 0.2, which we found to offer good balance between compression and performance. During inference, we set the decoding temperature 0.05 and perform each evaluation 5 times with different random seeds to ensure robustness. Full hyperparameter details can be found in Appendix
Baselines.
We compare MM-TPP with several SOTA TPP models, including NHP Mei & Eisner
2017). SAHP Zhang et al. 2020). THP (Zuo et al. 2020). RMTPP (Du al. 2016 AttNHP (Mei
al. 2022
TPP-LLM (Liu & Quan 2024), S2P2 (Chag al., 2025) .nd Language-TTe -Tapa (Kong
et al. 2025). ). All models except TPP-L.M and Language-TPP TPP are trained and evaluated using the Easy-TPP framework (Xue al. 2024). while TPP-LLM and Language-TPP
reproduced using its official implementation. For fair comparison, we use a unified maximum epoch setting across all baselines, select the pest-peeforming checkpoints for testing, and repeat the entire training and evaluation pipeline times with different random seeds.
Datasets. Most existing TPP benchmarks limited to timestamps and event types, with very few incorporating both textual and visual data. We evaluate MM-TPP on two distinct multimodal datasets address this gap:
DanmakuTPP (Jiang et al. 2025). An established dataset that contains extremely long quences- monnc with over 1,000 events, and some exceeding 9,600. Each event includes a times- tamp, textual content (e.g., video comments), and visual information extracted from corresponding video frames. also provides the DanmakuTPP-o TPP-QA subset for assessing complex multimodal reasoning capabilities.

---

Published as a conference paper at ICLR 2026
Table 1: Prediction performance comparison on multimodal TPP datasets for event times and types. Results reported in terms of RMSE(L) ACC(f) with standard deviations. Best results are in bold.
DanmakuTPP
TAXI-PRO
Model
RMSE(L)
ACC( (()
RMSE(L)
ACC(% % (()
NHP
5.4540 10.036
30.74 10.08
0.4494 10012
75.93 10.20
SAHP
5.4245 10.006
27.38
0.3526 10.006
75.37 11.30
THP
5.4001 10.002
24.64 10.06
0.3736 10.008
75.31 11.00
RMTPP
5,4633 =0,076
23.82 +0.50
0.3830 +0.008
76.47 #0,20
AtNHP
5,4384 =0.077
28,92
+0.40
0.4049
+0.007
69.37 #0,80
S2P2
5.5490 =0.016
31.48
+0.13
0.5735
+0.035
75,73 #0.6
TPP-LLM
5.3035 =0.012
24.59
+0.70
0.3336
+0.008
71.09 #0.20
Language-TPP
5.3845 =0.022
22.62
+0.07
0.3376
#0.001
75.27 #0.08
MM-TPP
5.2987 =0.018
27.62 +0.02
0.3310 =0.001
77.56 #004
TAXI-PRO. A new dataset we introduce by augmenting the classic NYC Taxi dataset (Whong 2014) with rich multimodal content. To create a complementary evaluation scenario, enrich each event with 224 x 224 map image patches centered at each GPS coordinate, as well as natural language descriptions incorporating landmarks, coordinates, and ride metadata (see Appendix B for details). In contrast to DanmakuTPP, sequences in TAXI-PRO are significantly shorter (aver- age length of 36), providing complementary evaluation scenario where our proposed compres- sion mechanism is not required.
Metrics. We evaluate model performance using several task-specific metrics. For timestamp pre- diction, report root square error (RMSE); for event type classification, use accuracy (ACC); and for text generation, we measure token-level quality using perplexity (PPL.. For question answering tasks in DanmakuTPP-QA we follow the original evaluation protocol-using ACC and RMSE for closed-ended questions, and conducting qualitative analysis for open-ended responses.
5.2 OVERALL PERFORMANCE COMPARISON
We first compare MM-TPP against all baseline methods on the complete datasets. Both datasets split into training, validation, and test with an 8:1:1 ratio. For DanmakuTPP, we constrain the number of encoded events using maximum token limit of 4096 tokens during training and perform evaluation on the first 200 events of each test sequence. For TAXI-PRO. the 4096 token limit is sufficient to encode complete sequences without truncation. As shown in Table MM-TPP delivers state-of-the-arr performance across both datasets. DanmakuTPP, achieves the lowest RMSE (5.2987) for time prediction, while its type accuracy (27.62%) is highly competitive and substantially improves upon Language-TPP The advantages are even clearer on TAXI-PRO, where MM-TPP outperforms all baselines on both metrics, establishing new benchmark with leading RMSE of 0.3310 and an accuracy of 77.56%.
5.3 PERFORMANCE ON MULTIMODAL QUESTION ANSWERING
key strength of our framework lies in its capacity for complex reasoning over multimodal event streams. To evaluate this, adopt the DanmakuTPP-ON
benchmark (Jiang et al. (025). which specifically designed to assess temporal, visual, and textual reasoning. The benchmark comprises 8 closed-ended tasks for quantitative evaluation and 2 open-ended tasks that require generating de- scriptive analytical reports, evaluated qualitatively. Detailed descriptions of all 10 tasks are provided in Appendix
Closed-ended QA Tasks. We compare MM-TPP with SOTA LLMs and MLLMs eight closed-
ended tasks, using baseline results reported in Danmaku TPPBench (Jiang et al. 2025). ). As shown in Table 2 MM-TPP achieves highly competitive performance across the benchmark and attains leading results on several tasks. Notably, it consistently outperforms the fine-tuned Qwen2.5-VL-33 from the original benchmark, despite utilizing the same underlying base model. This performance
7

---

Published as a conference paper at ICLR 2026
Table 2: Comparative evaluation on the DanmakuTPP-QA closed-ended tasks. Metries are ACC (1) and RMSE (J). The proposed MM-TPP model is highlighted. Best results are in bold and the second best results are set in underline.
Model/ / Task Metries
T-1 ACC 1
T-2 RMSE +
T-3 RMSE +
T-4 RMSE +
T-5 RMSE J
T-6 RMSE
T-7
T-8 ACC 1
ACC 1
LLMs
0.33
27.64
134.45
0.65
0.56
10.67
32.67
Qwen2.5 2-2.5533B-BInntnuee
25.33
1.52
122.69
0.36
0.24
16.67
38.17
Swen2.5-72B-Instae
0.67
1.28
123.45
0.30
0.46
0.46
16.00
43,83
Qwen3-8B
6.67
1.80
123.59
0.32
0.45
19.33
41.50
Swen3-30B-A3D
0.67
1.33
121.96
0.20
0,40
23.00
43.67
Qwen3-235B-A222
8.67
1.39
120.79
0.30
0.31
0.29
10.33
32.50
L.mma-3t-70B-nnstruue
1.67
1.11
121.49
0.26
0.27
0.22
17.00
33.33
DeepSeek-V3 V3
25.00
1.30
121.30
0.34
0.26
0.22
13.67
34.5
MLLMs
9.67
11.61
124.99
0.46
0.82
0.66
8.33
22.17
Owen2.5-VI-322
8.0
1,26
124.02
0.35
0.51
0.38
12.67
22.17
Owen2.5-VL-72.
0.33
1.14
121.25
0.28
0.47
0.41
15.98
47.17
Gemma3-21 2-B
0.33
1.33
121.32
0.28
0.27
0.20
15.67
36.17
Finetuned
Qwen2.5-VI-3B
27.0
1.35
220.43
0.05
0.16
0.08
15.33
43.00
MM-TPP
27.33
1.49
190.45
0.05
0.10
44.00
gap highlights the strength our framework, which explicitly captures the structured nature of multimodal event requences-unlins conventional fine-tuning approaches.
Open-ended QA and Report Generation. We assess MM-TPP's capacity for generating high- quality analytical text-a core capability that sets it apart from purely predictive models. On the two open-ended tasks from the DanmakuTPP-QA benchmark, MM-TPP consistently produces sig- nificantly more insightful and coherent text compared to existing MLLMs. To further evaluate its generalization ability, we prompt MM-TPP to generate descriptive analytical reports on the TAXI- PRO dataset, which lacks predefined QA tasks and has not been used fine-tune the model for generative tasks. Remarkably, the quality the generated narratives for TAXI-PRO's temporal data also surpasses that of leading MILMs, which are often limited to basic descriptive analysis of the event sequence. In contrast, MM-TPP uncovers high-level spatio-temporal patterns, demonstrating a robust capacity to synthesize multimodal event streams into coherent, insightful narratives even in zero-shot setting. Detailed qualitative comparisons of MM-TPP and baseline MLLM outputs for both datasets are provided in Appendix
5.4 EFFECTIVENESS OF COMPRESSION STRATEGY
To assess the effectiveness of adaptive compression strategy, compare the performance of MM-TPP with and without this mechanism on the DanmakuTPP dataset (note that compression is applied TAXI-PRO due its shorter sequences). The key advantage of compression lies in its ability to fit longer event histories within fixed context window. Under 4096-token constraint, the uncompressed model can accommodate extends this to an average of 292 events, with a maximum of 2008. This extended context allows the model better capture long-range dependencies, leading to noticeable performance gains. result, the compressed model outperforms its uncompressed counterpart, achieving lower RMSE (5.2987 vs. 5.5551) and higher accuracy (27.62% vs. 25.87%) on predictive tasks. To further verify the effectiveness of compression strategy, compared MM-TPP against naive bascline that randomly drops events. As detailed in Appendix(D) [D] the random drop strategy leads to significant performance degradation, confirming that preserving temporal causality is critical extending context.
average of 113 events, whereas the compressed variant
In addition, conduct further experiments evaluate the effectiveness of our compression method from a language modeling perspective. Specifically, we measure PPL on the encoded event se- quences from the test without truncation, standard and widely-used metric for assessing the
8

---

Published as a conference paper at ICLR 2026
HMT Curgesseed
27:52 2505 2319
RMSE
Number of Evente Group
Toomn-rr: A-0.05
6-05
2541
Model
Model
(a)
(b)
(C)
Figure 2: Evaluation of the compression mechanism and key hyperparameters on the DanmakuTPP
dataset. (a) PPL comparison on the test set. Our compressed model consistently maintains lower PPL than the uncompressed version, indicating effective Long-range modeling. (b, c) Ablation results for context length and similarity threshold (A) on RMSE and ACC, showing optimal performance with larger context and A 0.2.
Table Ablation study results for MM-TPP. The table evaluates the contribution of visual informa- tion (vs. text-only variants) and the effect of model size (vs. 7B model). Metrics are RMSE(() and ACC(T). Best results are in bold.
DanmakuTPP
TAXI-PRO
Model
RMSE())
ACC(%)(1
RMSE())
ACC(%)(1)
MM-TPP
5.2987 20018
27.62 :002
0.3310 10.01
77.56 10:4
Language-TPP
5,3845 40.022
22.62 #0.77
0.3376 #0.001
75.27 20.08
MM-TPP (text only)
5,4654
20000
23,64 10006
0.3388 #:0001
76.70 10008
MM-TPP (7B)
5.0533 #0002
26.98
0.3337
76.16 #0005
quality of language models (Zhao et al. 2023). As shown in Fig. 2a
perplexity increases with sequence length-a well-known challenge in sequence modeling. Notably, our compressed model consistently achieves lower PPL across all lengths, with the gap widening sequences grow longer. These results highlight a key strength of our approach: by preserving more long-range context within fixed token budget, the compression mechanism mitigates performance degradation on long and complex event streams.
5.5 ABLATION STUDY
In this section, we present ablation studies to examine the impact of four key design choices in MM-TPP: (i) the incorporation of visual information, (ii) model scale, (iii) context window size (controlled via the maximum token limit), and (iv) the temporal similarity threshold used in the compression mechanism.
Impact of Visual Information. To assess the contribution of visual input, we compare the full MM-TPP model (text image) with two text-only variants: MM-TPP (text only) and Language-Tie TPP. All models are evaluated under controlled conditions with a fixed number of events in the context window, ensuring that sequence length does not confound the results. As shown in Table 3 MM- TPP consistently outperforms both baselines across datasets, indicating that visual features provide additional predictive value. The integration of image improves both timestamp and type prediction accuracy, highlighting the complementary role of visual context alongside textual information.
Impact Model Size. To assess the role of model capacity, we replace the base model Qwen2.5- VL-3B with larger Qwen2.5-V L---7B, and evaluate on DanmaluTPP and TAXI-PRO datasets using identical training procedures and hyperparameters. This experiment investigates whether scaling up the model improves event prediction performance. Results in Table 3 show that scaling to 7B yields mixed results. While the larger model significantly improves time prediction (RMSE) on the complex DanmakuTPP dataset, the original 3B model outperforms it across all other metrics. This

---

Published as a conference paper at ICLR 2026
suggests larger model's capacity is most beneficial for long- tangge temporal patterns but may hinder performance on simpler tasks, possibly due to overfitting.
Impact of Context Length. To examine the effect of context window size, reduce the maxi- mum token limit from 4096 to 2048 during both training and inference on the DanmakuTPP dataset. and 2c] this reduction results in clear drop in performance. A shorter context
As shown in Figs. and window limits the model's s access to historical events, hindering its ability to capture long-range de- pendencies critical for accurate temporal modeling. These results highlight the importance of main- taining a sufficiently large context window when dealing with long and complex event sequences.
Impact of Temporal Similarity Threshold. We evaluate how the performance of our compres- sion mechanism varies with the temporal similarity threshold A. Specifically, we compare the de- fault setting A 0.2 with a stricter threshold (A = 0.05) and looser one (Á = 0.5), as motivated in Appendix A.2 As shown in Figs. 2b and 2c both alternatives lead to performance drops. A small threshold (A = 0.05) results limited compression, offering little gain in contextual coverage. In contrast, a large threshold (A 0.5) causes over-compr,siion, merging dissimilar events and discarding important temporal signals. These results confirm that A = 0.2 strikes good balance between compression efficiency and temporal fidelity.
6 CONCLUSION
In this paper, we introduce MM-TPP, novel framework that advances TPP modeling from purely predictive to a generative paradigm. By incorporating visual information alongside temporal, cate- gorical, and textual data, our model can not only predict future events but also generate rich, descrip- tive text that analyzes and explains event dynamics. We address the critical challenge of long-context modeling in multimodal sequences through adaptive compression strategy that exploits tempo- ral similarity. Extensive experiments demonstrate that MM-TPP consistently outperforms SOTA baselines in standard prediction tasks and, more importantly, exhibits superior text generation and reasoning capabilities on the challenging Danmaku'PPP-Q benchmark. While our framework cur- rently lacks image generation, opens new avenues for vision-enhanced temporal modeling and provides a solid foundation for future research in generative multimodal TPPs.
ACKNOWLEDGMENTS
This work was supported by the NSFC Project (No.62576346), the MOE Project of Key Research Institute of Humanities and Social Sciences (22JJD1 10001), the fundamental research funds for the central universities, and the research funds of Remmin University of China (24XNKJ13), and Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing.
ETHICS STATEMENT
Our work adheres to the ICLR Code of Ethics. The datasets used are either public benchmarks (DanmakuTPP) or derived from public data (NYC TLC). For our constructed TAXI-PRO dataset, ensured no personally identifiable information was included, focusing only aggregated spatio- temporal patterns. We acknowledge the potential for biases inherited from the pre-trained language model backbone. Our framework intended for research purposes analyze event dynamics, and we foresee no direct negative societal consequences from its use.
REPRODUCIBILITT STATEMENT
We have made every effort to ensure the reproducibility our research. The complete source code for our MM-TPP model, including training and evaluation procedures, will be provided in the supplementary materials. All crucial hyperparameters for both the pre-training and supervised fine-tuning stages documented in Appendix C Furthermore, details of our experimental setup, baselines, and evaluation metrics are described m Section 5 to facilitate fair comparison and repli- cation of our results.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Giada Adelfio and Marcello Chiodi. Including covariates in a space-time point process with appli- cation to seismicity. Statistical Methods Applications, 30(3):947-971, 2021.
Saeed Ranjbar Alvar, Gursimran Singh, Mohammad Akbari, and Yong Zhang. Divprune: Diversity- based visual token pruning for large multimodal models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 9392-9401, 2025.
Emmanuel Bacry, lacopo Mastromatteo, and Jean-Françoi, Muzy. Hawkes processes in finance. Market Microstructure and Liquidity, 1001):1550055, 2015.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025.
Daniel Bolya, Cheng- Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your VIT but faster. In International Conference on Learning Repre- sentations, 2023.
Yuxin Chang, Alex Boyd, Cao Xiao, Taha Kass-Hout, Parminder Bhatia, Padhraic Smyth, and An- drew Warrington. Deep continuous-time state-space models for marked event sequences. In Proceedings of the 39th Conference on Neural Information Processing Systems (NeurlPS), 2025.
Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision- language models. In European Conference Computer Vision, pp. 19-35. Springer, 2024a.
Zhiheng Chen, Guanhua Fang, and Wen Yu. On non-asymptotic theory of recurrent neural networks in temporal point processes. arXiv preprint arXiv:24066.00634) 2024b.
Daryl Daley and David Vere-Jones. An introduction to the theory of point processes. vol. prob- ability and its applications, 2003.
Daryl Daley and David Vere-Jones. An introduction to the theory of point processes: volume II: general theory and structure. Springer Science & Business Media, 2007.
Mohamed Dhouib, Davide Buscaldi, Sonia Vanier, and Aymen Shabou. Pact: Pruning and clustering-based token reduction for faster visual language models. In Proceedings of the Com- puter Vision and Pattern Recognition Conference, pp. 14582-14592. 2025.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image worth 16x words: Transformers for image recognition at scale, 2021.
Nan Du, Hanjun Dai. Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodrigues and Song. Recurrent marked temporal point processes: embedding event history to vector. In International Conference on Knowledge Discovery and Data Mining. 2016.
Álvaro Gajardo and Hans-Georg Müller. Point process models for covid-19 cases and deaths. Jour- nal of Applied Statistics, pp. 1-16, 2021.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation large language models, 2021.
Yue Jiang, Jichu Li, Yang Liu, Dingkang Yang. Feng Zhou, and Quyu Kong. Danmakutppbench: A multi-modal benchmark for temporal point process modeling and understanding. arXiv preprint orXiv:2505.1841/ 2025.
Quyu Kong, Yixuan Zhang, Yang Liu, Panrong Tong, Engi Liu, and Feng Zhou. Language-tpe Integrating temporal point processes with language models for event analysis. arXiv preprint arXiv:2502.07 2025.
11

---

Published as a conference paper at ICLR 2026
Dana Lahat, Tülay Adali, and Christian Jutten. Multimodal data fusion: overview of methods, challenges, and prospects. Proceedings of the IEEE, 103(9):1499-1477, 2015.
Zefang Liu and Yinzhu Quan. TPP-LLM: Modeling temporal point processes by efficiently fine- tuning large language models. arXiv preprint. 2024.
Hongyuan Mei and Jason Eisner. The neural Hawkes process: A neurally self-modulating multi- variate point process. In Advances in Neural Information Processing Systems, 2017.
Hongyuan Mei, Chenghao Yang, and Jason Eisner. Transformer embeddings of irregularly spaced events and their participants. In International Conference on Learning Representations, 2022.
Zizhuo Meng, Ke Wan, Yadong Huang, Zhidong Li, Yang Wang, and Feng Zhou. Interpretable Transformer Hawkes processes: Unveiling complex interactions in social networks. In Interna- tional Conference on Knowledge Discovery and Data Mining, 2024.
Sebastian Meyer, Johannes Elias, and Michael Höhle. space-time conditional intensity model for invasive meningococcal disease occurrence. Biometrics, 68(2):607-616, 2012.
George Mohler. Modeling and estimation of multi-source clustering in crime and security data. The Annals of Applied Statistics, pp. 1525-1539, 2013.
Takahiro Omi, Naonori Ueda, and Kazuyuki Aihara. Fully neural network based model for general temporal point processes. In Advances in Neural Information Processing Systems, 2019.
Aristeidis Panos.
Decomposable Transformer point processes. In Advances in Neural Information Processing Systems, 2024.
Marian Andrei Rizoiu, Swapnil Mishra, Quyu Kong, Mark Carman, and Lexing Xie. Sir-hawkes: on the relationship between epidemic models and Hawkes point processes. In The Web Confernce, 2018.
Xiaoming Shi, Siqiao Xue, Kangrui Wang. Fan Zhou, James Zhang, Jun Zhou, Chenhao Tan, and Hongyuan Mei. Language models improve event prediction by few-shot abductive reasoning. In Advances in Neural Information Processing Systems, 2024.
Alexander Soen, Alexander Mathews, Daniel Grixti-Cheng, and Lexing Xie. Unipoint: Universally approximating point processes intensities. In AAAI conference on artificial intelligence, 2021.
Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models, 2025.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503. 19786, 2025.
A Vaswani. Attention all you need. Advances Neural Information Processing Systems, 2017.
Chris Whong, Foiling NYC's taxi trip data, 2014.
Shuai Xiao, Junch Yan, Xiaokang Yang. Hongyuan Zha, and Stephen Chu. Modeling the inten- sity function of point process via recurrent neural networks. In AAAI Conference on Artificial Intelligence, 2017.
Siqiao Xue, Yan Wang, Zhixuan Chu, Xiaoming Shi, et al. Prompt-augmented temporal point process for streaming event sequence. In Advances in Neural Information Processing Systems, 2023.
Siqiao Xue, Xiaoming Shi, Zhixuan Chu, Yan Wang, Hongyan Hao, Fan Zhou, Caigao Jiang, Chen Pan. James Y. Zhang, Qingsong Wen, Jun Zhou, and Hongyuan Mei. Easytpp: Towards open benchmarking temporal point processes. In International Conference on Learning Representa- tions, 2024.
Chenghao Yang, Hongyuan Mei, and Jason Eisner. Transformer embeddings of irregularly spaced events and their participants. In Proceedings of the Tenth International Conference on Learning Representations, 2022.
12

---

Published as a conference paper at ICLR 2026
Guolei Yang. Ying Cai, and Chandan K Reddy. Recurrent ppatooltempora point process for check-in time prediction. International Conference on Information and Knowledge Management, 2018.
Qiang Zhang, Aldo Lipani, Ömer Kimna,, and Emine Yilmaz. Self-attentive Hawkes process. In International Conference Machine Learning, 2020.
Yixuan Zhang, Quyu Kong, and Feng Zhou. Integration training for spatio-temporal multimodal covariate deep kernel point processes. In Advances in Neural Information Processing Systems, 2023.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang. Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303. 1(2), 2023.
Feng Zhou, Yixuan Zhang, and Jun Zhu. Efficient inference of flexible interaction in spiking- neuron networks. In International Conference on Learning Representations, 2021.
Zihao Zhou, Xingyi Yang, Ryan Rossi, Handong Zhao, and Rose Yu. Neural point process for learning spatiotemporal event dynamics. In Learning for Dynamics and Control Conference, 2022.
Shixiang Zhu and Yao Xie. Spatioteemporal-exxualll point processes for crime linkage detection. The Annals of Applied Statistics, 16(2):115--111 2022.
Shixiang Zhu, Minghe Zhang, Ruyi Ding, and Yao Xie. Deep fourier kemnll for self-attentive point processes. In International Conference on Artificial Intelligence and Statistics, 2021.
Simiao Zuo, Haoming Jiang, Zichong Li, Tuo Zhao, and Hongyuan Zha. Transformer Hawkes process. International Conference on Machine Learning, 2020.
13

---

Published as a conference paper at ICLR 2026
A DETAILS OF ADAPTIVE SEQUENCE COMPRESSION
A.1 ILLUSTRATION OF COMPRESSION MECHANISM
To clarify how the adaptive compression operates on consecutive events, we provide a step-by-step illustration using hypothetical sequence of three events: A, B, and C, occurring that order. Let T; = t4 - t denote the inter-event interval for event i. The compression decision is made sequentially based on the pairwise comparison of adjacent intervals:
Step 1 (Processing Event B): The model compares the interval of event B (TB) with the interval of the immediately preceding event (TA). If the absolute difference satisfies (B TAl A, event B is considered temporally similar to A and is replaced by single special token < | weimmlaa..eeeee
The model then proceeds to event C. It compares its in- terval (TC) with the interval of the preceding event B (TB). Note that this comparison is based on the calculated time intervals regardless of whether B was compressed in the to- ken sequence. If TC TBl A, event C' is also replaced by the < sismmii event |> token.
Step 2 (Processing Event C):
This sequential process ensures that compression is applied locally based on the dynamic evolution of event density, allowing the model to efficiently encode bursts of events with similar distinct timing patterns.
A.2 TEMPORAL. SIMILARITY THRESHOLD SETTING
In our framework, the temporal similarity threshold A is key hyperparameter for adaptive sequence compression. Specifically, for each event, we define the interval π = t, f.-1 and compare it with the previous interval T(-1- If the absolute difference between the two intervals satisfies [Ti T A, event i deemed temporally similar to event directly controls the granularity of sequence modeling and the preservation of fine-grained temporal dynamics, as only sufficiently similar events are compressed into single representation.
and is eligible for merging. This mechanism
To determine an appropriate value for A, we conducted comprehensive statistical analysis of the absolute differences between adjacent inter-event intervals (/T. (fT:-T-1) -T-- on the DanmakuTPP training dataset. As illustrated in Fig. 3] the empirical distribution of these differences is highly skewed, with a large proportion of values concentrated near zero. This indicates that event intervals are frequently very similar to their predecessors, validating the potential for compression strategy based on temporal similarity.
To ground our threshold selection in the data's structure, we examined the distribution's 's quantiles, summarized in Table 4 Based on this analysis, we selected candidate thresholds of 0.05, 0.2, and 0.5 seconds. These values correspond to meaningful points in the distribution:
A 0.05 is conservative threshold, corresponding to approximately the 25th percentile. It compresses only the most similar event pairs, preserving most the original sequence structure.
A 0.2 corresponds approximately the 50th percentile (the median), offering a balance between compression rate and information preservation.
A = 0.5 is a more aggressive threshold, near the 75th percentile, which merges a larger portion of the sequence achieve higher compression ratio.
The positions of these thresholds relative to the distribution marked in Fig. This principled, data-driven selection ensures that our adaptive compression mechanism is both statistically grounded and practically effective for multimodal temporal point process modeling.
14

---

Published as a conference paper at ICLR 2026
800,000
= 0.05 A - 0.2
Table 4: Quantiles of the absolute difference be-
Frequency
600,000
à - 0.5
tween adjacent inter-event intervals ((;;-TZ -
400,000
Percentile
Value (s)
0,050
0.007
200,000
0.100
0.017
0,200
0,045
0.8.0
0,250
0.063
Absolute Difference of Intervals (s)
0.500
0.214
Figure Distribution of the absolute differ- ence between adjacent inter-event intervals on the training set. Vertical lines indicate the tested thresholds A.
0.750
0.590
0,900
1.330
0,950
2.115
1.000
1787.000
B TAXI-PRO DATASET CONSTRUCTION
We construct the TAXI-PRO dataset by augmenting the original NYC Taxi dataset with multimodal information (images and text descriptions). This section provides detailed information about the data construction pipeline.
Original Taxi Data. The raw taxi trip data is sourced from the New York City Taxi and Limou- sine Commission (NYC TLC) public dataset, consisting of files trip-data_l. asv through trip-data.12. csv. This dataset is identical the source used by the EasyTPP official taxi dataset and originates from NYC TLC's open data initiative !
We construct the TAXI-PRO dataset with the following setups:
Image. We leverage the coordinates of each event and construct an image crop of the map the location. High-resolution map tiles are generated from OpenStreetMap data using tools like contextill iiy. The base map covers the entire Manhattan spondence between pixel coordinates and geographical boundaries (resolution: 6071 x6307 pixels). For each event location, extract 224x 224 pixel patch centered at the event ordinates. Edge cases are handled by padding with gray background to maintain consistent patch dimensions.
with strict corre-
Text. We generate the textual mark for each event following a fixed template incorporat- ing landmark information, coordinates, passenger count, and trip distance. For example, "Picked up al Times Square (40.7580, -73.9855), 2 passengers".
Event type. We focus exclusively on Manhattan and subdivide it into three regions: lower Manhattan, midtown Manhattan, and upper Manhattan, resulting in event types for com- binations of regions and pickup/dropoff events.
To provide concrete visualization of our data structure, Fig. showcases the composition of single event in the TAXI-PRO dataset. It highlights the multimodal nature of each data point, com- bining the geographical map patch, the templated text description, and the categorical event type. When selecting the final 2000 sequences, we employ a greedy algorithm with variance minimization ensure balanced event type distribution, avoiding imbalance.
C DETAILED HYPERPARAMETER SETTINGS
The specific hyperparameters used for each stage of our training procedure are summarized in T We employed the Adam W optimizer and utilized bfloat16 mixed- precissin for all experiments to enhance computational efficiency.
Hyperparameters
"FOlL ing NYC's Taxi Trip Data: https:t////// https:///ahiishhoog..cc///ooaaaa fo__tofl_____xiii
15

---

Published as a conference paper at ICLR 2026
Time_since__orr lest_t 0,0333,
event_type (pickup at upper__mahhaeee mete____ahattnn)
text: "Picked at Central Park West (40.800018, -73.944778), 1 passenger".
Image:
Figure 4: An illustration of single event in the TAXI-PRO dataset. Each event comprises four key components: (1) a 224x224 map image centered on the event's coordinates, (2) a generated textual description including landmark and trip details, (3) categorical event type and (4) timestamp.
Table Hyperparameter Setting
Hyperparameter
Value
Base Model
Qwen2.5 ---22.---LL-BB VL-3B
LoRA Rank
16
LoRA Alpha
64
LoRA Dropout
0.1
Stage Learning Rate
le-4
Stage Epochs
5
Stage 2 Learning Rate
le-4
Stage 2 Epochs
Batch Size
Max Token Length
4096
Temporal Threshold A
0.2s
Inference Temperature
0.05
D COMPARISON WITH RANDOM DROP BASELINE
To rigorously evaluate the effectiveness of our proposed adaptive compression mechanism, we com- pared it against naive compression baseline: Random Drop. This baseline randomly discards events from the sequence during both training and inference phases.
We conducted experiments the DanmakuTPP dataset with drop probabilities of = 25% and p 50%. The results, compared with our MM-TPP (Adaptive Compression) and the uncompressed baseline, are presented in Table
As shown Table 6 the Random Drop strategy results in a clear performance degradation com- pared to our adaptive approach. Even with moderate drop rate of 25% the model's predictive accuracy and timestamp precision are inferior to MM-TPP. This degradation likely because event sequences in TPPs contain strict causal dependencies; randomly removing events disrupts the his- torical context necessary for the model to reason about future dynamics (e.g., self-exciting patterns). In contrast, adaptive compression strategy selectively merges only temporally similar intervals (which often represent redundant burst signals) while preserving the critical anchor events and the overall semantic continuity of the sequence.
16

---

Published as a conference paper at ICLR 2026
Table 6: Performance comparison between MM-TPP's adaptive compression and naive Random Drop strategies on the DanmakuTPP dataset. The proposed adaptive method significantly outper- forms random dropping, highlighting the importance of preserving temporal structure.
Model / Setting
RMSE (4)
ACC (%) (1)
MM-TPP
5.2987=00018
27.62=00.2
AMMTPPUUncmpper
5.5551
25.87
MMTPP-Ranee SaATTP-RRnddo-Drop (p 25%)
5.3792:20014
20.06
MMTTP-Random-DD (p 50%)
5.425940.009
24.79=0.03
E DANMAKUTPP-QA DETAILS
Table 7: Detailed descriptions and evaluation metrics for the ten tasks in the DanmakuTPP-QA benchmark. Closed-ended tasks are evaluated quantitatively (Accuracy or RMSE), while open- ended tasks are assessed via qualitative analysis.
Task Description
Evaluation Metries
Task Type
Task-1
Danmaku burst peak counting
ACC
Closed-ended
Prediction of the next
Task-2
RMSE
Closed-ended
Danmaku timestamp
Prediction of the next Danmaku burst peak timestamp
Task-3
RMSE
Closed-ended
Assessment of average sentiment polarity
Task-4
RMSE
Closed-ended
Sentiment polarity prediction for the next Danmaku
T
RMSE
Closed-ended
Sentiment polarity prediction for the next Danmaku burst peak
Task-6
RMSE
Closed-ended
Event type inference for the next Danmaku
Task-7
ACC
Closed-ended
Prediction of Top-2 triggering event types for the next burst peak
Task-8
ACC
Closed-ended
Analysis of global sentiment dynamics and the underlying drivers
Task-9
Qualitative Analysis
Open-ended
Causal attribution analysis for specific Danmaku burst peak formation
Task-10 10
Qualitative Analysis
Open-ended
E.
TEN QUESTION- ANSWERING TASKS
This section provides a detailed breakdown of the ten reasoning tasks that constitute the DanmakuTPP-QA benchmark. The benchmark is designed to evaluate model's temporal-visual- textual reasoning capabilities through series of diverse challenges. As summarized in Table 7 these tasks are divided into eight closed-ended, quantitatively evaluated tasks and two open-ended tasks that require qualitative assessment of generated reports.
E.2 MM-TPP'S PERFORMANCE ON OPEN-ENDED TASKS
In this section, we conduct a qualitative evaluation of MM-TPP's ability to generate high-quality analytical reports for open-ended tasks. We compare its outputs against two SOTA, general-purpose
17

---

Published as a conference paper at ICLR 2026
Multimodal Large Language Models (MLLMs): Qwen2.5-VL-3B (Bai al. 2025) and Gemma3- 27B (Team et al. 2025). Our analysis reveals that MM-TPP consistently produces significantly more insightful and relevant reports, highlighting the value of its specialized architecture.
The qualitative advantage is not only evident on the DanmakuTPP- -A benchmark, where MM-TPP was fine-tuned, but also generalizes robustly the TAXI-PRO dataset in a zero-shot setting. To substantiate these claims, we provide a series of comparative examples in the following sections. Given the consistent nature of the performance gap, we present two representative examples for each of the open-ended tasks (Task 9 and Task 10) and for the TAXI-PRO report generation task illustrate the consistent superiority of MM-TPP's analytical capabilities. Here we summarize the qualitative advantage as follows:
Task 9 (Danmaku sentiment dynamies analysis): MM-TPP conveys information more effectively by focusing on essential sentiment dynamics, follows a from overall trend key inflection points and summary, and produces outputs that human evaluators find more readable and natural.
clear analytical structure
Task 10 (Danmaku peak burst reason analysis): Beyond capturing peak bursts, MM- TPP pinpoints sub-peaks with fine temporal precision, enabling more rigorous view of self-exciting dynamics. It also ties specific event types, such as quote/reference and hu- mor/meme, directly to the triggering mechanisms of bursts, clarifying how content cate- gories drive cascading responses. Unlike the general-purpor into tangential scene descriptions and instead highlights the key drivers, while framing the explanation explicitly in terms of TPP theory. This balance of precision, focus, and domain-specific grounding makes the analysis both accessible and academically robust.
MLLMs, it avoids drifting
TAXI-PRO: MM-TPP can delivers high-level insights by identifying core TPP patterns in the driver's 's behavior. It moves beyond simple route tracking to model how events mutu- ally influence one another, revealing strategic responses to spatio-tempora dynamics such as the formation of self-exciting event clusters in high-demand corridors while MLLMs, proficient in language generation, tend to produce outputs that are more descriptive than analytical.
TAXI-PRO Example_1
MM- TPP's generated text on TAXI-PRO (Example 1)
Prompt:
You expert data andlyst specializing pemporl event sequences. Your lask is to analyze chronological log single taxi driver's activities in NYC, which includes actions like pickups and drop aff' at various Incations. Based on the provided event sequence, please conduct comprehensive analysis and genernte detailed Analysis several paragraphs. Note that this sequence NOT VIDEO! All time intervals are converted to four bytes according to the IEEE 754 standard.
[TPP data >
Answer:
The event sequence reveals driver's dynamic movement acmes multiple iconic York City (NYC) neighborhoods, primarily centered around the Upper West Side, Theuter District, and suroundiing areas. The driver initiales journey the Upper West Side, then rapidly transitions through nrigh-denstty calturally rich locations such as the Financial District and Triheca, indicating a high volume of pedestrian dense traffic activity. This pattern aligns with self-exciting TPP dynamics, where initial events trigger subsequent ones through social or cconomic stimuli.
The driver consistently serves mix of pickups and drops off. The average trip duration is under 10 minates, with notable short haul excursions (< 1.5 miles), implying rapid-fire driver tumover or high-frequency service needs.
six passengers, suggesting high-visiility or possibly shared vehicle model,
each sees
Visually, the map shows tight clustering of points, especially between 40.76 40.78 and 40.69 40.74, aligning with known densest city blocks like the Theater District and surrounding high-activity zones. The presence of multiple "Dropped off" entries within <'1 miles suggosts that driver likely functioning in high-density urban environment where such proximity commoo.
Temporal clustering indicates high-intensily period. potentially early morning or late-night when public transpotationn or walking pattems denser. The driver's puttern of serving different numbers of passengers in quick succession implies shift haved service model, where each location functions as micro huh, possibly responding to fluctuating passenger flow.
Unexpectedly, despite the high frcquency of short trips. there no significant drop average speed, suggesting that the driver is possibly in city with low-speed zones high- pedestrian friction. Additionlly, the lack significant negutive sentiment correlates with high-frequency, short-interval events, indicating rourine and possibly socially or culturally motivated transportation needs rather than anomalics.
In conclusion, this sequence reflects high intensity, short haul transportation patlern typical in densely pupulated urban centers, with temporal and spatial clustering pointing structured driver behavior modulated by local gengraphy and timing dynamics.
18

---

Published as a conference paper at ICLR 2026
Qwen's generated text on TAXI-PRO (Example_1)
Prompt:
You are an expert data analyst specializing in spatio-temporal event sequences, Your task is to analyze a chronological log of single taxi driver's activitics in NYC, which includes actions like pickups and drop-off's at various locations. Based on the provided event sequence, pleuse conduct comprehensive andlysis and generate a detailed Analysis in several paragraphs.
< IPP _dota >
Answer:
The provided data logs a single taxi driver's activities in New York City over approximately 8.5 hours, capturing a series of pickups and drop-aff's with details time, location, number of passengers, and distance traveled.
The sequence begins with pickup at Times Square, followed by I 4.5miile trip to Hell's Kitchen. The driver then picks up two more Tribeca, covering 2:22 miles. Subsequent trips involve short distances within Theater District and Chelsea, highlighting the high demand for shoet taxi rides in these areas.
passengers Hell's Kiteben and drops them
Notable longer trips include a 3.6 3.67-mile journcy from the Theater District to Central Park West and 5.20-mile trip from Chelsea to Harlem. These longer trips demonstrate the driver's cupability hundle diverse travel needs. The mujority of the driver's activities. however, focus on short, frequent trips in midlown and lower Manhattan, particularly in areas like Tribeca, Chelses, and the Theater Dis- triet. The final recorded activities include 5.55-mile trip from Tribecn Midtown East and m-90- trip from the Theater District Times Square, concluding the sequenoe.
In summary, the driver operates elliciently in high- dehaede areas, Irequently handling short trips while occasionally taking longer jour- neys. This pattern reflects the dynamic nature of taxi services in New York City's 's husiling urban environment.
Gemma's 's generated text on TAXI- PRO (Example 1)
Prompt:
You expert data analyst specializing spatio-tempona event sequences. Your task is to analyze chronological log single taxi driver's activities in NYC, which includes actions like pickups and drop-off's at various locations. Based on the provided event sequence, please conduct comprehensive analysis and generate a detailed Analysis in several paragraphs.
< PTPP Pddota| >
Answer:
period of approximately 8.5 minutes. The driver demonstrates highly active pattern, completing 36 events- encomparsing pickups and dmp--ofs within this short timeframe. The driver's operations are concentrated within Manhattan, with significant activity the Midtown, Chelsea, Tribees, and Theater District arcas, with excursions to Harlem and Battery Park. The frequent. short trips suggest a focus on high-demand. quick-turmarount fares, potentially catering to tourists or commuters, The driver consistently picks up and drops off passengers, indicating efficient utilization of time berween fares. The initial segment of the log (0-2.5 minutes) shows rapid sequence of pickups and drop-offs primatily between Times Square, Hell's Kitchen, Tribeca, and Chelsea. This suggests the driver may have been positioned in a high-demund area initially and quickly responded to available requests. The trip distances during this period
This analysis details the activity log of taxi driver
relatively short (ranging from 0.06 2.22 miles), reinforcing the iden of quick, localized trips. A notable pattem emerges around the 3-5 minute mark, with a shift towards longer trips, including a 3.67-mile journey Central Park West and 5 5.20-mile trip to Harlem. This indicates a willingness accept longer fares, potentially maximize carnings reposition for better opportunities. However, the driver quickly returns to shorter, more frequent trips in the latter part of the log.
The driver's passenger load varies between I and passengers per trip. The majority of trips carry passengers, suggesting preference or availability for larger groups The linal few events (6.8. minutes) show a return to a puttern of short trips within the Midlown and drop-off Times Square The driver's ability quickly transition between pickups and dmp--ofs, with varying trip lengths, highlights their efficiency and responsiveness to demand. The consistent activity throughout the log suggests dedicated and active driver, effectively navigating the dynamic landscape New York City services. Further amalysis, including correlating this data with time of day and external events, could reveal more nuanced patterss in the driver's behavior and the demand for taxi services in specific areas.
Triheen areas, culminating
TAXI-PRO Example 2 MM-TPP's generated text on TAXI-PRO (Example 2)
Prompt:
You are an expert data analyst specializing in spatio-temporal event sequenees. Your task is to analyzo chronological log of single taxi driver's activities in NYC, which includes actions like pickups and drop-ofl's at various locations. Based on the provided event sequence, please conduct comprehensive unalysis and generate a detailed Analysis in several paragraphs. Note that this sequence is NOT VIDEO! time intervals are converted to four bytes according to the IEEE 754 standard.
< ITPP _dotaa >>
Answer:
The event sequence reveals complex pattern of multi-modal interactions between a single driver and multiple locations in New York City (NYC). The temporal clustering suggests high-frequency activity, likely driven by surge pricing or peak travel times. The driver initiates sequence al 40.764388, -739977577 (Times Square), then repeatedly picksup and drupped at multiple locations including Hell's Kitchen, Triheca, Chelsea, and Battery Park, indicating rapid fire sequence of events. This pattern aligns with self- exciting TPP dynamics, where initial events trigger subsoquent ones through social economic stimuli.
The majority of trips involve two passengers, suggesting shared rides or multi-destination excursions, while others
pessener.
This implies dual activity modes; some are passive observers (egg. watching from a distance), while others are active participunts, pos- sibly duc short-term availability or limited visibility.
The spatial distribttion shows a core-te-periphee
pattern, with frequent inter-reginaal movements, such as from central to peripheral city zones. average 1.66 mile trip suggests relatively short excursions, potentially indicating micro mobility patterns or commutte behavior.
A notable temporal shift occurs around 14:00-15:00, where multiple high-deasity drops are observed, coinciding with pesk city traffic and surge pricing surges. This temporal clustering supports TPP riggering offects, where past events (e.g.. previous drops) excite future ones.
Unexpectedly. no significant uogative sentiment correlates with high engagement, suggesting positive reinforeement loops possiblyy due
19

---

Published as a conference paper at ICLR 2026
to social interaction famillarity with the city. The uniformity of event types (all 10%) indicates bread-based excitation, rather than single driver unemaly.
In summary, sequence reflects a TPP rPP-driven driver behavior, shaped by multi modal interactions between time, location, modali- ties of action. The driver exhibits self- excitatcittionn, temporal clustering, and regional dependency, all hallmarks of complex event-Iriggered processes in urhun transporiation systems.
Qwen's generated text on TAXI-PRO (Example_2)
Prompt:
You are expert data analyst specializing in spatio-temporal event sequences. Your task is to analyze chronological log of single taxi driver's activitics in NYC, which includes actions like pickups and drop-off's at various locations. Based on the provided event sequenee, pleuse conduct comprehensive analysis and generate detailed Analysis in several paragraphs.
< [TPP .data >
Answer:
The provided event sequence details the activitics of a single taxi driver in New York City over approximately ten hours. The log captures series pickups and drop-offs, each recorded with precise timestamps, passenger counts, and geographic coordinates.
The driver begins their shift in the Upper West Side, picking up four passengers and dropping them off in the Theater District after 10 minutes. Over the next few hours, the driver operates primarily in the midtown area, making multiple short trips between the Theater Districi, Herald Square, and Central Park West. These trips range from 0.73 to 1.35 miles and involve Around the one- hour mark, the driver completes a 1.1 1-mile trip from the Theater District to Central Park West and then returns to the Theater District. This pattern of short, frequent trips continues until the 2:5-hour mark, when the driver takes longer 2. 2.39-mile trip from the Theater District to Midtown East. This is followed by a significant gap in activity, suggosting break or reduced activity.
to three passenger.
When driver resumes. they pick up six passengers in Tribeca and drop them off complete another long trip from the Financial District to SoHo, covering 2.43 miles. These longer trips indicate a shift to differcnt parts of the city where demand is lower but the trips are more protitable.
Financial District, a 1.79-mile trip. They then
The driver returns the midiown West, the longest trip recorded. The final recorded trip is a 0.99-mile journey from Midtown Fast to Central Park West, hringing the total number trips 36 the approximately ten-hour period.
making several
trips, including 4.80 mile journey from Herald Square Central Park
In summary, the driver peimarily operates in the midtown area. making frequent short trips, while occasionally taking longer, potentially more profitable
other parts of the city. This balanced approsuc likely maximizes both efficiency and profitability.
Gemma's 's generated text on TAXI- PRO (Example_2)
Prompt:
You are an expert data analyst specializing spatio-temporal event sequences. Your task is to analyze a chronological log of single taxi driver's activities in NYC, which includes actions like pickups and drop-off's at various locations. Based on the provided event sequence, please conduct comprehensive analysis and generate a detailed Analysis in several paragraphs.
< ITPP data >
Answer:
This dalasel details the activity of a single taxi driver over approximately 10 minutes, revealing dynamic pullern of pickups and drop- offs across Manhattan. The driver hegins the Upper West Side, completing relatively quick trip the Theater District with full load of four passengers. This initial segment suggests a pocential commute or scheduled pickup. Following this, the driver quickly pivors to serving shorter trips, picking up two passengers in the Theater District and dropping them off in Herald Square, picking up three passengers in the Theater District and dropping them off in Central Park West. This shift indicates move towards more opporunistic. shorter-distance fares, typical of a driver circulating in a busy area. The frequent changes in passenger count throughout the initial phase consistent group or pro-booked long- ding-distncc rides.
(4, 2, 3) suggest the driver isn't catering The middle portion of the log (belween approximately 5 and minutes) shows a significuni change in the driver's operational area and passenger volume. A substantial jump time occurs before the driver picks up six passengers in Tribecn, indicating a longer travel time to reach this southern location. This is followed by series of pickups and drop-offs primarily concentrated in the Midtown area. including Times Square, Hell's Kitchen, driver may have accepied a larger group booking or is capitalizing on high demand areas during pesk hours. The distances of these trips vary, ranging from short hops within Midtown to a longer haul to Central Park West, demonstrating adaptability to passenger roquests. Finally, the last few events (around 9-1 minutes) show the driver returning to pallern shorter trips, picking six passengers in Midtown East and dropping them off in Central Park West. The consistent high passenger count during this period suggests the driver is actively seeking maximize occupancy. The overall pattern reveals driver who flexible, adapting varying trip lengths and passenger numbers. The initial phase focuses on shorter, more dispersed trips, while the middie and cnd phases demonstrate Midlown and surrounding areas.
Herald Square, consistently carryngg full complement six passengers. This suggests
preference for maximizing passenger capacity, particularly good understanding of Manhattan's 's geography and demand hotspots, efficienly navigating between different neighborhoods to secure fares. The data doesn't provide information on wait times or idle periods, which would be valuable for a more complete assessment of the driver's elliciency.
driver's route demonstrates
Task 9 Example
MM-TPP's generated text on Task 9 (Example.1) 1)
Prompt:
You are an expert model for analyzing bollet comments. Bascd sequence comments and their timing intervals, your give a sentiment unalysis of the time window. All the comments are in Chinese and all the time intervals are converted to four byles acconding HEEE standard.
< (T PPPPP _data >
[question] > Please provide a detailed analysis the seutiment treed in the bullet comments over time.
Answer:
The video exhibits predominantly pusitive semtiment with minor Iluctuations. Initially, there's a sharp rise followed by stabilization around positive polarity. Key inflection points include early peaks approximately 00:00:30 and 00:01:00, with slight dips around
20

---

Published as a conference paper at ICLR 2026
00:0130 and 00:02:00. Significant scutiment shifts occur 00:03:00, showing a gradual doeline but remaining positive. Overall, video muintains consistenily positive emotional trajectory with minimal negative deviations.
Qwen's generated text on Task 9 (Example_1)
Prompt:
You are an expert model for analyzing bullet comments. Based on sequence comments and their timing intervals, your task give sentiment unalysis of the time window. All the commentts Chinese. Question:
Please provide a detailed analysis of the sentiment trend the bullet comments over time.
Answer:
The sentiment trend
comments for the generally starts neutral to positive tone, as viewers are likely intrigued by the title and the initial seenes of the video. As the vidco progresses. the comment frequency increases, indicating heightened engagement. Around the 20-second mark, there is noticeable spike comments,, suggesting a particularly engaging or exciting moment the video, which could be drumatic visual indicating moments high excitement or awe. By the mid-point of the video (around 120 seconds), the comments stabilize but contirue ta show enthusiasm, with viewers expressing admiratiion for content sharing video, the comment frequency slightly decrenses, but the overall positive sentiment persists, with many viewers leaving final thoughts oncouragements. This tread suggests that the video successfully maintains viewer interest and elicits consistently positive response.
impactful statement. The sentiment remains largely posilive throughout, with occasional spikes
own experiences. Towards the
Gemma's generated text on Task 9 (Example_1)
Prompt:
You expert model for analyzing bollet comments, Based sequence comments and their timing intervals, your task
give a sentiment analysis of the time window. All the comments are in Chinese.
Question: provide detailed analysis < ITPP _ata
bullet comments over
Answer:
The bullet comment density indicates consistently high engagement throughour the video, with numerous peaks suggesting moments of particular interest or impact. Engagement starts slowly, builds steadily, and then remains very active with frequent spikes from around seconds onward. There's clear overall decline in engagement, remains high until the very cnd the data provided. Without comment text, it's impossible to determine what sentiment is being expressed (positive, oegative, surptise. etc.). However, the consisteut high volume suggests the video generally captivating elicits respumse from viewers, likely related exciting content described in the title ( ("Million Volt Voltgee Capture!"). "" The frequent spikes likely correspond to visually or narratively dramatic moments.
Task Example 2
MM-TPP TPP's generated text on Task 9 (Example 2)
Prompt:
You are expert model for analyzing bollet comments, Based on the sequence of comments and their timing intervals, your task give a sentiment unalysis of the time window. All the comments are in Chinese and the time intervals are converted to four bytes according the IEEE 754 standard.
< PTPP data >>
[guestiin]
provide detailed unalysis
trend bullet
over
Answer:
predominantly positive tmjectory minor fluctuations. Initially, there's a sharp followed by stabilization around the 0.4 mark. Key inflection points include carly peaks and valleys that settle into consistent positive range. Significant sentiment shifts nocur at the heginning with a steep increase and towards the end with a slight dealine. Overall, the video maintains positive emotional with minimal abrupt changes, indicating stable uplifting viewer experience.
sentiment polarity
Qwen's generated on Task 9 (Example_2)
Prompt:
You are an expert model for analyzing boller comments, Based on the sequence of comments and their timing intervals, your task to give a sentiment analysis of the time window. All the comments are Chinese.
Question: Please provide detailed analysis the sentiment trend the bullet comments time.
< ITP data >
Answer: The sentiment trend in the bullet comments shows a clear pattern over time. Initially. from to 60 seconds, there is a very high density comments, andicating high excitement positive reactions
are immedistely engaged euthusiastic. From 60 120 seconds, the comment density moderates, suggesting steady engagement with mixed reactions viewers remain interested but less intensely. In the mid-phase (120 300 seconds). the deasity fluctuates with occasional spikes, reflecting mixed reactions and key scenes comments.. From 300 480 seconds, the comment density lower with occusional spikes, indicating reactions and deeper engagement with the content. Finally, from 480 633 seconds, there is a higher density of comments with a significant spike the end, showing increased engagement and final thoughts viewers share their overall impressions and reactions to the conclusion. Overall, the vidco maintains viewer interest throughout, with notable peaks at the beginning and cnd, and steady engagement
that spark
subduued
21

---

Published as a conference paper at ICLR 2026
Gemma's generated text on Task 9 (Example.2
Prompt:
You are an expert model for analyzing ballet comments. Based on the sequence of comments and their timing intervals, your task to give a sentiment mnalysi of the time window. comments Chinese.
Question: Pleuse provide a detailed analysis of the sentiment trend in the bullet comments over time. ITPP _dota >
Answer:
The bullet comments show high initial engagement (0. 10s1,,,, dipping slighly before building to sustained high density from 20s omward. Several peaks comment frequency (amunnd 20-40s, and 100-150s) suggeu impactful moments the film. The very end (550c*) maintains extromely high comment volume, indicating strong final reactions. Essentially, the audience was consistently cugaged, with specific scenes eliciting particularly strong respenses throughout viewing experience. Without the comment content, impossible to say if these peaks are positive or negutive, only that they represent heightened attention.
Task 10 Example
MM-TPP's generated text on Task 10 (Example.1)
Prompt:
You are an expert model for unalyzing bullet comments. Based on the sequence of comments, their timing interval, and the ussociated video frames, your task is analyre detail causes hurst peaks in hullet comments and give a detailed analysis report. Chinese and all the time intervals are converted to four bytes according the IEEE 754 standard.
the comments
< (TPP data >
[question] Within the time window: [0, 1.5], please analyze concise and fluent paragraph the of the burst peaks in comments.
Answer:
The bullet comment hurst at timestamp 0 exhibits a sharp initial spike, Followedd by rapid fire clusters at 0.08 0.Iss 0.3 0.44, and 0.6 0.7s. dynamics -early comments triggered cascading audience responses. These sub- sab-peaks align with recurring quoterreference and humor/meme culture and inside jokes within the community.
indicating self-exciting
lypes (e.g.. " ( (tF ". "N (itlZ", suggesting viral textual prupagation driven meme
The average sentiment polarity of0.8 reflects mildly positive tone"*, yet the diversity comment types-including critical, amotional, and off-copic remarks- shows heterogencous audience reactions. Notably, the presence of neutral visual tags (1f 96 22" "i" contrasts with emutionally charged commentss, revealing a mulli-modal dissonance: formal attire speech settings likely prompted ironic or humorous reinterpretations, fueling meme generation and engagement.
Temporal alignment reveals the first sub-peak 0.0-0. Is) was dominated hy nonsense text and emotional reaction,, possibly triggered by the sudden visual shift or unexpected content juxtaposition. The second cluster (0.3-0.4s) saw increased social interaction and personal experience, indicating viewers began farming collective interpretations or dehates. Finally, the third wave (0.6-0.74 featured more critical .comment and quote/rcference, reflecting deeper engagement with thematic comparisons (c.g. "RE/> vs. "I IX").
social.interaacion
This burst illustrates TPP self-xxcitation. where early comments acted as triggers prompting a cascade of related and reactive posts. The multimoddal mismatch between fornal visuals and hunoroustextual responses underscores audience interprelation, not just on- screen content, drives engagement. The result is a complex interpaa of textual virality, temporal clustering, and visual framing, typical of interactive streaming platforms.
Qwen's 's generated text on Task 10 (Example_1)
Prompt:
You an expert model analyzing bullet comments. Based the sequence of comments, their timing interval, and associated video frames, your task is to analyze in detail the causes of the hurst peaks in hullet comments and give a detailed analysis report. All comments Chinese.
Question: Within the time window: [0. 1.5), please analyze in concise and flucat paragraph the causes of the burst peaks in bullet comments.
JTPP_datta
Answer:
Within the time window [0, soconds, burst peaks ballet comments primarily driven by the strong temporal self-cxcitation in the Temporal Point Process (IPP). These peaks cluster around 0.48-0.58 and 1.U6-1.66 seconds, coinciding with the appearance of formal visual cues such as "ifi 3 le." "Mh be." and "M These visual elements set a moek-scrious or authoritative tone, which catalyzes multi-modal irony and prompts viewers to juxtapose the visuals with satirical culturally referential commentary. The recur- ring references to sutirical labels like ( Kl8 125 and $ 58555 indicate a leedback loup where one humorous comment inspires similar responses, particularly the humor/meme and quoteheeeeeence categories. Despite neutral positive average sentiment (0.384), the bursts of emotional reaction and nonsense text comments suggest sentiment heterogencity driven by entertainment rather than criticism. The sub-peak intervals, as 1.25-1.46 seconds, reinforce this pattern, with repeated mentions & 3F " and other metu commentary further amplifying the engagement. Overall, the burst is best explained by sell-exciting humor cascades that resonate culturally and create continuous stream of tronic and playful interactions.
Gemma's generated text on Task 10 (Example. 1)
Prompt:
You an expert model analyzing bullet comments. Based the sequence of comments, their timing interval, and associated video frames, your task is to analyze in detail the causes of the burst peaks in bullet comments and give a detailed analysis report. All comments
Chinese.
Question: Within the time window: [0, 1.5], please analyze in concise and fiuent paragraph the causes of the hurst peaks in buller comments.
IPPP data
Answer:
The bullet comments primarily driven a combination of satirical commentary and ironic juxtaposition. evidenced by
22

---

Published as a conference paper at ICLR 2026
frequent references to shows like 1 and 8 12 alongside the formal setling of debate like scenario. This creates strong self-exciting loop where humerous meme-like comments (areund 14.29% trigeer similar responses, amplified by visual cues of the scene particularly the presence of ligures in "il 94 * engaged in "Miif". While the sentiment is generally pusilive, bursts correlate with moments of playful mockery and cultural references, suggesting viewers are engaging iranic commentary rather than expressing strong negative reactions. The visual elements, rather than dampening engagement, seem to catalyze it, fostering dynamic where the formal setting is ironically contrasted with the absurd or culturally resonant commentary, leading to a cascade of humorrus responses.
Task 10 Example 2
MM-TPP's generated text on Task 10 (Example 2)
Prompt:
You are an expert model for unalyzing bullet comments, Based on the sequence of comments, their timing interval, and the associated peaks in bullet comments and give a detailed analysis report. the comments are in Chinese and all the time intervals are converted to four hytes according to the IEFE 754 standard.
video frames, your task is analyze detail the causes
ITPP P.P.AA data >
[question] Within the time window: [393.5, 5), please analyze in detail the causes of the burst peaks bullet comments.
Answer:
The builet.
burst penks 395 seconds, with dense timestamps revealing sub peaks around 394.8 395.0 and 395.6 335.8. indicating rapid-fire engagement intervals. These clusters align with humor/meme (e.g. "1:5!" "55555"), emotional reactions (" interactions, suggesting
18 19 10:32 dist 1il eascading responses.
10 (R..."
exciting TPP pattern where early comments trigger
The dominant object tagttttgg-aaaa 11.30 +M + -imply a gameplay highlight or sereenshot moment that resonated emotionally and hemorously with viewers. Semiment polarity averages 0.423 (positive), consistent with excitement and meme-like commentary rather thun crilical or ool-c remarks.
Temporal alignment shows that the inirial spike (99455-39488
confusion anticipation ("?", "Et A1 SURE J"), likely triggered by a visually unexpected event. This is followed by a surge of numerichumorous texts ("1! 5! "5"), indicating real-time voting or scoring rituals common gaming videns, reinforcing a triggering effect among viewers.
Notably, despite the positive sentiment, some critical or off-topic comments appear (e.g. "T'EE"), saggesting mixed but engaged audience. The repeated use of "5" and "I!5!" reflects a coordinated or meme-drivea response pattern. consistent with self-cxcitation in TPP models- where similar comments cluster closely time.
Multi modal correlation reveals that the visual context (gumeplay frame lugged (+) prompted rapid textual engagement, especially humor and social mimicry, amplifying the burst. No negutive sentiment spikes observed, ruling controversy -rivee engagement. Instead, the burst reflects synchronous emotional and interactive behavior, driven by shared gaming culture and visual triggers.
Qwen's ' generated text on Task 10 (Example 2)
Prompt:
You an expert model analyzing bullet comments. Based the sequence of comments, their timing interval, and associated video frames, your task is to analyze in detail the causes of the burst peaks in bullet comments and give a detailed analysis report. All comments
Chinese.
Question: Within the time window: [393.5, 396.5l. please analyze in detail the causes of the burst peaks in bullet comments. < [TPP >
Answer:
Within the time window [393.5, 396.5), the burst peaks in ballet comments are primarily driven by the convergence of visual and tex- captures critical mement the game, likely high-difticully challenge ("Hb3 [2 ") which serves as visual trigger for the audience. This visual cue prompts surge in com- ments, characterized humor memes (e.g. "ft "W"". "FulR (E-F" and social interaction g... numerical repetitions "I1 1111111111 "55555"). The mildly posirive sentiment (0. (0.386) reflects the excitement and playful banter among viewers, who are anticipating the un- predictable oulcomes of the challenging gume segment. The rapid fire clustering of similar phrases and numerals indicates sell exciting Hawkes proces, where initial comments stimulate casende of responses, creating palse collective engagement. The temporal align- ment of these comments with the critical gamoplay moment further reinforces the multi-modal interaction, where visual cues and textual responses work tandem to drive the burst peaks.
tual elements that clicit strong emotional and social interactions.
vidco
Gemma's generated text on Task 10 (Example 2)
Prompt:
You are an expert model for analyzing bullet comments. Based on the sequeace of comments, their timing interval, and the associated video frames, your task is analyze detail causes the burst peaks in bullei comments and give detailed analysis report. the comments are in Chinese.
Question: Within time window: [393.5, 396.5], please analyze in detail causes of the burst peaks in buller comments.
ITPP dota
Answer:
Based the provided data, the burst bollet comments between 393.5 and 396.5 scconds primarily driven by a combination of game- play events and coordinated social interaction, best explained by option D. The peaks align with moments likely depicting a challenging questions
in game scenatio (" it U 2t Jade difficuliyy. irggeerng emotional reactions strong element of meme- based coordination is present, evidenced by repetitive numerical expressions | "55555") and phrases like "MI H indicating viewers are engaging playfol, culturally resonant interaction scripes. This suggests self-exciting Hawkes process dynamic where initial comments stmulate cascading responses. amplified the visual cue of the gumeplay screenshot acting Istent trigger for rapid fire textual engagement. The mildly positive sentiment (0.386) further supports this interprelation of excitement and playful benter rather than nogativity or criticism.
viewers. Simultaneussly,
23

---

Published as a conference paper at ICLR 2026
F PROMPT DETAILS
Here we present the detailed prompt templates used for preprocessing temporal point process datasets. Due to the similarity between processing the two datasets, we only present the templates of TAXI-PRO.
Special Tokens Here present all special tokens added to the Qwen2.5-VL tokenizer vocab- ulary and used in our work to structure the prompts in Table 8 We note that < [im_start] > < [im_end] >,< [vision_start] >,< |vision_end >,< jimage_pa] are native special tokens from the Qwen2.5-VI. tokenizer.
Table 8: Descriptions of Special Tokens
Special Token
Description
start of event] lend of _evnnt|
Marks the start of an event.
Marks the end of an event.
Simm__strrs timd_end]
Marks the start of timestamp.
Marks the end of a timestamp.
Marks the start of an event type. Marks the end of event type.
Lype_end]
lext_ssarrl start) lear_end)
Marks the start of the text an event. Marks the end of the text of an event.
time prediction type prediction) A
Task token for next event time prediction.
Task token for next event type prediction.
quession)
Task token for Question-Answering
byte 0|
| 255 >
Byte tokens for representing event time intervals float32 numbers.
Hype_0) 0
1type.88
Type tokens for representing event types.
Event Template
Here we present the event template in Fig. 5
Event Template:
<|start_of. _
[ftime_starth-
<bbyte pyyte_66lxx(byte_1d66
boyt__144[=(byte_125)
<ttim__end)>
<|text_stort("'Jstt got
back from Dubail
<[text_end|>
<tyyee_startl>
[type_]lx=[fype_endd
< dvvsioo_start/>>
paddge___ddl>>>
end))
<lend_of_event)
Figure 5: A complete example of an uncompressed event representation
Prompt Template Here in Table we provide a sample generated event sequence from the TAXI- PRO dataset used in Stage-1 training process. We also provide two prompt-response pair samples from the TAXI-PRO dataset used in Stage-2 next event type finetuning and next event time finetuning in Table I0 and Table []] The construction logic for all three templates is largely consistent, with minor variations in the system prompt.
24

---

Published as a conference paper at ICLR 2026
Table 9; Event Sequence sample from TAXI-PRO
Prompt:
< [im_start] >system
You are an intelligent urban mobility analysis model specializing in NYC taxi patterns. Your task is to predict the precise time interval until the next taxi event in Manhattan. You will be given a history of taxi trips, where each event includes its type (e.g. Uptown Pickup, Downtown
Drop-off), the time elapsed since the last event, and map image of the location.
[ém_end] >
Event Sequence:
>
time_start) >< bytee_0 >< byte__0 >< bbyte_0) >< bytee_0| >< [time_end]
type start > < type 0] ><
type end] >>
text start > Picked up at Tribeca 140.711086 -74.016106, passengers. < |text end) >
wisioo_start)
limage_padl
|issionend >
end_off_vent
start_oof
time_start
><
byyte_63 62 >< (hyte_162
>< |byte.34| >< [byte_34| >< [time.end]
tyype_taart]
type-3) >< typpe_edd]
ttert start
> Dropped off from Tribeca (40.711086, -74.016106) to Times Square
(007756698, 777.9821222 22)), passengers, 2.87 miles trip.< |lext_ wiision_saat]] >< image_pad >< |vision_end >
-eveet
start_of-event >
tiime_stort >< bytte. 62) Ibyte_199 196] (byte.68) bbyte_68)
>< [time.end
tyypessaar) typpe_4| >< Itype_eene end] >
|tert_ start]
Picked up Upper West Side (40.799252, -73.970146), passengers.
leat_ende >
|vision_starl
Smnagepadd) >< wiisionennd
eevd-of-event)
sstat_oofeveen >
time start) >
< bbyte 63 > bbyte 17 > (byte 16 >< bbtte 161|
>< [time end]
type. 1| >< Itype. end] >
staat_ Dropped off from Upper West Side (40.799252, -73.970146) to Tribeca (40.7 40.714455, -74.014008), passengers, 4.37 miles trip.< [eezt_end] >
|vision_star]] >< |image_padl >< |vision_end] efdeof-event)
Table 10: Prompt-response pair sample for next event time finetuning from TAXI-PRO
Prompt:
< [im_start] start] >system
You intelligent urban mobility analysis model specializing in NYC taxi patterns. Your task to predict the precise time interval until the next taxi event in Manhattan. You will be given history of taxi trips, where each event includes its type (e.g. Uptown Pickup, Downtown Drop-off), the time elapsed since the last event, and a map image of the location. Analyze the patterns in event types and timing forecast the time the next event. Your response must be exactly four byte tokens representing the time interval. jim end]
Event Sequence History: sttatt_of_event >
time-stort) >< bbyte_0 >< bytee_0 >< [byte_0] >< [byte_0] >< [time_end] >
type_start] >< type__] >< [type_end] >
text_start] > Picked up Tribeca (40.71 1086, -74.016106), passengers. < |teatendd
vision_start! >< limage_pad] >< pision_end >
end of event] >
starl of event]
25

---

Published as a conference paper at ICLR 2026
lime lime.star() starl| >
bylee 62 > < Byle 162 >< bbyle.34 34|
><
Ibyle 34 Uimee.end
lype_start] ><
Mype_3) >< [ypeeend]]
[teat_start] Dropped off from Tribeca (40.711086, -74.016106) to Times Square 40.757698, 73.987...2122 124), passengers, 2.87 miles trip.< [teztendd
|vision start)
image >< |vision
>
start_of _event] >
lime_star >< bylee 62) >< 1byle_166| >< bbyle_68 68] >< bbyle.00 >< [time_end >
lype_start) >< kype-41 >< [yppeend]]
|text_start > Pickkd
at Upper West Side (40.799252, -73.970146), 1
passengers.< Itext endl >
vision start] >>
jimage. pad] >< vision end) >
end.of-event
time Aime.predictio)
Response:
[byte_63 >< |byte_17| >< [byte_16] ><
1byte_161 >
Table 11: Prompt-response pair sample for next event type finetuning from TAXI-PRO
Prompt:
< [im_start] >system
You are an intelligent urban mobility analysis model specializing in NYC taxi patterns. Your task is to predict the type of the next taxi event in Manhattan based on sequence of past events. You will be given a history taxi trips, where each event includes its type (e.g. Uptown Pickup, Downtown Drop-off), the time elapsed since the last event, and map image of the location. Analyze the patterns in timing and location to forecast the next action. Your response must be single token representing one of the six event types. jim_end]
Event Sequence History:
|start.of foart_of_eventt >
[time_starl) > >< |byle_0 _0 bylle00 bbyle. 0| >< (byle.0) 0 >< [lime
type-slart) > < Lyppe_0] ><
Hypeeendd] >>
teæt_staart > Picked up Tribeca (40.71 -74.016106). passengers. |tert_end
vision_start)
limageppdl >< [vision_end] >
end of event
start.of -event >
time__taar) >< byte 62) > 1byte_162 >< [byte_34| >< [byte_34] >< [tme__ndd
typpestarrt) type 3) >< [yype_nnd]
|teat_stardl > Dropped off from Tribeca (40.711086, -74.016106) to Times Square (40.757698, -73.98 122), passengers, 2.87 miles trip.< [tezt_end] |vision start >< immage pad] >< |vision end] > lend of event >
timee_stom >< byte_ 62 > [byte_190 196] >< [byte_68| >< [byte_68| >< [time_end] type-start] >< type_44 >< [type_end] >
|teat_start] Picked up Upper West Side (40.799252. -73.970146), passengers.<5 [text_end] >
vision starts] >< limage pad| >< wision_end]
lend of evend] >
lypeepreediction
26

---

Published as a conference paper at ICLR 2026
Response: [yppe_1
G THE USE OF LARGE LANGUAGE MODELS
In the preparation of this manuscript, we utilized LLM as writing-assistivg tool. Its application was strictly limited to improving the language and presentation of the paper, which included correcting grammatical errors, rephrasing sentences for enhanced clarity and conciseness, and refining para- graph structure. The core scientific contributions, including research ideation, methodology, and analysis, were conceived and executed entirely by the human authors. The authors have reviewed and edited all text and take full responsibility for the final content of the paper.
27


Reviews:
[{"review_id": "sIuHTK19AT", "reviewer": "Reviewer_F8w3", "summary": "This paper extends TPPs to the visual domain for modeling event sequences, avoiding traditional issues in modeling multimodal data via TPPs such as long sequence length. The authors implement this extension by introducing an adaptive sequence compression mechanism that retains key patterns, and pretrains on compressed sequences and then fine-tunes for specific tasks. The authors evaluate their method on the DanmakuTPPQA benchmark.", "questions": "See weaknesses. Generally speaking, the paper is very clear.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The extensive documentation of details of the experiment setup are appreciated.\n\n- The experiments are extensive. The authors compare against a range of TPP approaches, and select fair experiment details for an appropriate comparison. Selected metrics seem appropriate.\n\n- Standard deviations for experiment results are presented in Table 1 alongside key metrics.\n\n- The paper is well-written and very detailed. I do not have many questions after reading through the experiments section.\n\n- The ablation study included in Sec. 5.5 is very helpful for understanding details of the proposed method and how it relates to other approaches.", "weaknesses": "- Given that only two benchmarks were selected for evaluation, it would be helpful to add a sentence or two explaining why these datasets were selected and why more are not needed for a fair evaluation.\n\n- Figure 1 is difficult to read due to the font choices and text size.\n\n- It would be interesting to consider other backbone architectures beyond Qwen-2.5 and its different sizes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hNmCYg0PjM", "reviewer": "Reviewer_X9AS", "summary": "This paper proposes MM-TPP, a multi-modal, generative framework for temporal point processes that models multi-modal event data and predicts the next event’s time/type while generating text. To cope with very long sequences induced by multimodal tokenization, the authors introduce an adaptive temporal-similarity compression that replaces runs of events with similar inter-arrival times by a special `
` token, enabling longer effective context within a fixed window. The system is built on Qwen2.5-VL-3B with a two-stage training recipe including continued pre-training and SFT. Experiments on DanmakuTPP and a newly proposed TAXI-PRO dataset show improved performance over strong TPP baselines and Language-TPP.", "questions": "See weaknesses", "limitations": "", "rating": 6, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. **First multi-modal TPP dataset.** This article proposed TAXI-PRO, which may be a useful testbed for future work.\n2. **Simplicity and effectiveness.** The `
` token is easy to implement and effectively increases the total event counts in a single context window.", "weaknesses": "1. **Evaluation breadth.** Only DanmakuTPP is chosen as existing benchmarks to be evaluated, which could be biased.\n2. **Lack to efficient MLLM baselines.** The author should include token pruning baselines as baseline methods (such as [1]), as they are closely related to the addressed topic.\n\n[1] An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models (ECCV 2024)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4UWZtLTjvv", "reviewer": "Reviewer_dt4W", "summary": "This paper proposes a unified framework, multimodal temporal point processes (MM-TPP), that extends Language-TPP to predict event time and type while generating rich text conditioned on visual, textual, and temporal inputs. The authors further introduce a compression mechanism to reduce sequence length. Finally, they introduce a benchmark TAXI_PRO to evaluate their proposed method.", "questions": "Please see above.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well organized.  \n\n2. The topic of using LLMs for multimodal event sequences is worth exploring in the research community. This paper targets this important problem.\n\n3. The paper provides the code to help reviewers better understand of the proposed method.", "weaknesses": "There are some concerns and questions about this paper:\n\n1.\tIn Section 4.3, the authors mention that temporal similarity between events can be calculated to reduce sequence length. However, how is this similarity calculated? The authors do not seem to mention the calculation method.\n\n2.\tIn video understanding, I learned that only 64 frames are needed to help the model understand what is happening in the video. So, for the video input mentioned in the paper, how many video frames are actually input into the model?\n\n3.\tTo reduce sequence length, did the authors try keeping the input sequence length constant while using token merging or pruning methods? How does this method perform compared to the methods mentioned in the paper?\n\n4.\tAmong the comparison methods listed in Table 1, only Language-TPP was published in 2025, while the others were published in 2022 or earlier. Could the authors update the comparison methods or include more recent methods published in 2024 or 2025 for a fairer comparison?\n\n**Please note that I am not very familiar with this research area. I would suggest that the ACs place greater weight on the comments from the other reviewers.**", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FWfAK87BJD", "reviewer": "Reviewer_r5oN", "summary": "The paper proposes a unified encoding paradigm that integrates time, event type, text, and image to achieve multimodal TPP modeling. It introduces an adaptive temporal compression mechanism, where adjacent events with time differences below a defined threshold are replaced by a special token—representing temporally similar events), enabling contextual compression and enhancing the model’s long-range sequence modeling capability.", "questions": "1. When events are compressed, will the compression be applied consecutively? For example, if there are three events A, B, and C, after compressing ⟨A, B⟩, will it further compress to ⟨A, B, C⟩? If so, this should be clarified in the paper.\n\n2. When a particular moment is controversial — for instance, when some comments (danmaku) are positive while others are negative — does the current method only consider the earliest event? This could indicate a potential limitation.\n\n3. I suggest adding a comparative experiment that randomly drops some events, representing the most naive form of compression, and comparing it against the proposed adaptive compression method. This would make the results more convincing.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The target task has high practical value and represents an important direction in the field of long-term reasoning and multimodal generation.\n\n2. The proposed method in this paper is concise and effective. The proposed time-similarity-based compression mechanism can significantly reduce sequence length.\n\n3. The experimental design of the paper is very comprehensive, with particularly thorough ablation studies.", "weaknesses": "The temporal compression approach proposed in the paper may have certain limitations in some scenarios, as it only addresses the compression of the number of events but not the compression within individual events.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)