RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility
Authors: Haoyu He
Haozheng Luo
Yan Chen
Qi Wang
Venue: NeurIPS 2025 poster ()
Keywords: Human mobility prediction
large language models (LLMs)
spatio-temporal forecasting
Primary area: applications
TL;DR: We propose RHYTHM, a new foundation model for human mobility prediction that combines temporal tokenization, hierarchical attention, and a frozen LLM backbone to model multi-scale spatio-temporal patterns efficiently.

Abstract: Predicting human mobility is inherently challenging due to  complex long-range dependencies and multi-scale periodic behaviors. To address this, we introduce RHYTHM (Reasoning with Hierarchical Temporal Tokenization for Human Mobility), a unified framework that leverages large language models (LLMs) as general-purpose spatio-temporal predictors and trajectory reasoners. Methodologically, RHYTHM employs temporal tokenization to partition each trajectory into daily segments and encode them as discrete tokens with hierarchical attention that captures both daily and weekly dependencies, thereby quadratically reducing the sequence length while preserving cyclical information. Additionally, we enrich token representations by adding pre-computed prompt embeddings for trajectory segments and prediction targets via a frozen LLM, and feeding these combined embeddings back into the LLM backbone to capture complex interdependencies. Computationally, RHYTHM keeps the pretrained LLM backbone frozen, yielding faster training and lower memory usage. We evaluate our model against state-of-the-art methods using three real-world datasets. Notably, RHYTHM achieves a 2.4% improvement in overall accuracy, a 5.0% increase on weekends, and a 24.6% reduction in training time. Code is publicly available at https://github.com/he-h/rhythm.

RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility
Haoyu Het
Haozheng Luo'
Yan Chen'
Qi R. Wang
Northeastern University
Northwestem University
(he. haoyul, q- anng)ssnothe edu
bluouu. northweer edu, ychen@mortthwestern..d edu
Abstract
Predicting human mobility is inherently challenging due to complex long-range de- pendencies and multi-scale periodic behaviors. To address this, we introduce RHYTHM (Reasoning with Hierarchical Temporal Tokenization for Human Mobility), unified framework that leverages large language models (LLMs) general-purpose spatio-temporal predictors and trajectory reasoners. Method- ologically, RHYTHM employs temporal tokenization to partition each trajectory into daily segments and encode them discrete tokens with hierarchical atten- tion that captures both daily and weekly dependencies, thereby quadratically re- ducing the sequence length while preserving cyclical information. Additionally, we enrich token representations by adding pre-computed prompt embeddings for trajectory segments and prediction targets via frozen LLM, and feeding these combined embeddings back into the LLM backbone to capture complex interdependencies. Computationally, RHYTHM keeps the pretrained LLM back- bone frozen, yielding faster training and lower memory usage. We evaluate our model against state-of-the-art methods using three real-world datasets. Notably, RHYTHM achieves a 2.4 % improvement in overall accuracy, a 5.0% increase on weekends, and 24.6% %.%%%%%% reduction in training time. Code is publicly available attps://gitte com/ he- h/ rhythm.
Introduction
Human mobility shapes transportation systems [6. 62], informs epidemic control strategies [63, 8], and guides sustainable city planning [4], making accurate movement prediction essential for optimizing infrastructure, managing disease spread, and building resilient communities [42]. Yet human trajectories exhibit long-range dependencies, spatial heterogeneity [78, 19], and dynamic influences such as weather anomalies or special events [7, 36), producing non-stationary, multi-scale spatio-temporal patterns.
To address this challenge, we introduce RHYTHM (Reasoning with Hierarchical Temporal Tokenization for Human Mobility), human mobility foundation model that rethinks mobility modeling via structured temporal abstraction. We posit that human mobility, like language, follows compositional structures: daily routines form tokens, and weekly rhythms form higher-order syntax. RHYTHM operationalizes this analogy by tokenizing time into segments and reasoning over them with frozen large language model (L.LM).. This framework unites multi-scale temporal tokenization with the reasoning capabilities of pretrained LLMs, delivering a scalable and general approach for mobility prediction with reduced computational cost.
The design of RHYTHM is inspired by inherent patterns of human mobility. People's 's movements are random; they follow an underlying order marked by recurring daily and weekly rhythms [12, 24, 18]. Notably, Song et al. [58] quantify this regularity by showing that 93% of daily trajectories are
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

predictable, underscoring the critical role of cyclical temporal context in mobility modeling. Capturing this cyclical regularity requires models that can jointly represent local behaviors (e.g., morning commutes) and global temporal dependencies (e.g.. weekly routines). Yet existing approaches fall short: Markov and RNN-based methods either disregard long-term periodicity or suffer vanishing gradients over long sequences [76, 19, 22], while transformer-base -based methods treat time as static, failing to disentangle multi-scale temporal patterns [27. 72]. To bridge this gap, we decompose each trajectory into meaningful segments, tokenizing each into discrete representations that capture local patterns through intra-segment attention. These segment tokens are then pooled into higher-level representations, enabling inter-segment attention to model long-range dependencies across days, as illustrated Figure 1, thereby reducing sequence length and quadratically lowering attention cost. Each segment token is augmented with pre-computed semantic embeddings derived from frozen L.M,, enriching temporal tokens with contextual meaning before being processed by the L.M backbone for reasoning.
Recent advances demonstrates LLMss' remarkable capabilities not only as se- quential representation extractors to capture the spatio-temporal attention patterns but also as reasoning models [13, 9]. Prior works [25, 59, 17, 21] demonstrate their reasoning capabili- ties through techniques such few- shot prompting [9), chain-of-thought reasoning [50, 49, 66], and in-context learning [16]. However, mobility- specific models like PMT [71] and ST- MoE-BERT [26] lack the capability to leverage LLMs for modeling complex correlations in human flows, limiting
Model
RHYTHM
Day
Day
Day
Day!
Day
Figure 1: Motivation for RHYTHM. Instead of processing entire trajectories as a continuous sequence, RHYTHM seg- ments them into tokens better capture periodic patterns.
their predictive performance. By integrating an LLM- reasoning module, RHYTHM more effectively models these complex interdependencies. To maintain scalability, RHYTHM adopts parameter-efficient adaptation strategy by freezing the pretrained LLM and avoid extensive fine- tuning. This design captures fine-grad -re-grained spatio-temporal dynamics, deep semantic context, and leverages LLM reasoning- all while minimizing computational and memory overhead making RHYTHM ideally suited for deployment in resource-constrained, real-world environments.
Contributions. We propose RHYTHM, unified, computationally efficient framework that cap- tures both temporal dynamics and cyclical patterns, as illustrated in Figure 2. Our contributions are as follows:
We introduce temporal tokenization that encodes daily mobility patterns as discrete tokens, reducing the processed sequence length while capturing cyclical and multi-scale mobility dependencies through hicrarchical attention mechanism.
We design efficient prompt-guided approach that integrates semantic trajectory information and task description with segment embeddings, enhancing RHYTHM's ability to interpret complex mobility patterns.
We propose parameter-efficient adaptation strategy using frozen pretrained LLMss, reducing train- able parameters to 12.37% of the full model size and achieving a 24.6% % reduction in computational cost compared to other baselines.
Empirically, we evaluate RHYTHM on three real-world mobility datasets, demonstrating superior performance compared to state-of-the-art models. RHYTHM achieves 2.4% improvement in prediction accuracy, with a 5.0 % increase on weekends.
Related Work
In this section, we provide a brief overview of related work, with detailed review in Appendix B.

---

LLM (Embedder)
Troen
Traring
Hoden Ree.
ty Eyan
(a)
Seat exbedzing
Sugruet antoucciing)
Emoedding
Freteddes pioner ETagertory
Erbeatzed prompt Deact
Outpul Projection
[pra-computed
trajectory
day
Friday.
Embedder
trajectoryy consists recorcis, each record
coordiaate as follows: 08:30: De- 130, medd2); oi
Large Language Model (Backbone)
Tokerszation
18:00: den 185, 1n-42).
Text
Key transtions:... Main stay
2.00
2.6
Jocal onei......
Temporal Tokenization
(b)
Temporal Takenization
mobality environments.
Inter-segment Attantion
Bpatio-Temporal Embedding
Pooling
TASK:
historicall movementt pattarns, pred locations (Swuurdag)...
Intra mang-a Attomtion
Day
Day
Futurn Thresterph
Figure 2: The proposed architecture of RHYTHM. Our framework processes historical trajectories through spatio-temporal embedding and temporal tokenization (b), capturing local and global depen- dencies via hierarchical attention. Segment representations enriched with semantic embeddings from trajectory information, while future timestamps incorporate task description context (a). This combined sequence passes through frozen LLM backbone with output projection to generate location predictions.
Mobility Prediction. Human mobility prediction progresses from probabilistic approaches [75, 22] to deep learning architectures, as demonstrated in recent studies. Sequence models like LSTM [37] and attention mechanisms [19] improve temporal modeling, while graph-based methods [55, 14] integrate spatial relationships. Transformer architectures [65, 77, 45] further enhance long-range dependency modeling but struggle with the hierarchical structure of mobility patterns. Recent work with LLMS [20, 64) shows promise but typically treats mobility as generic sequences, ignoring the inherent periodicity of human movement.
Cross-domain Adaptation of LLMs. LLMs emerge as powerful natural language processing systems and quickly evolve into general- purposs foundation models capable of reasoning and generation tasks [9, 1j. Their remarkable adaptability has enabled successful applications in computer vision [5, 53], speech [69, 46], biomedicine [82, 44, 57], time series forecasting [11, 81], and finance [31,70) While many adaptations rely on parameter-efficient fine-tuning methods like LoRA [29], recent approaches maintain frozen L.MMs by utilizing them sequential representation extractors, preserving their semantic capabilities while reducing computational costs [40, 32, 2]. To the best of our knowledge, RHYTHM is the first approach that adapts frozen LLMs mobility prediction without compromising the model's reasoning capabilities or requiring extensive fine-tuning.
3 Method
In this section, we introduce RHYTHM, an LLM-based deep architecture tailored for prompt-guided representation learning of spatio-temporal patterns with its periodicity, shown in Figure 2. In the following, we first define the problem and then introduce the model structure of RHYTHM, including its computational efficiency and theoretical guarantees.
3.1 Problem definition
Let X (x1, X2, ØT} denote a user's historical trajectory, where each x; = (t4,4) consists of timestamp and a location 4; L from finite set of locations L. Given sequence of future timestamps T = {tr+1, tr tr+H) } with prediction horizon H, the goal is to predict the corresponding future locations y (lr+1. 12 ........... 1,,,,,) Formally, seek function
f: (X, T) + V,
which maps historical trajectories and future timestamps to the user's future locations.
3

---

3.2 Model structure
Spatio-Tempore Feature Encoding. For each observation 24, we construct temporal embeddings to capture cyclical patterns in human movements:
E
plempporal
E ToD (E
where -||- indicates concatenation, TO represents the time-of-day embedding (capturing 24rhour cycles), and EDoW represents the day-of-week embedding (capturing weekly patterns). These are learnable embeddings that map discrete temporal indices to continuous representations, with E, E.omporal E RR with D matching the backbone LLM'' input dimension.
Spatial embeddings Expaial
I2D for location 4 is defined as:
E spatial
Loo (1:) (L))l((Womm)) |la;, lon:"" bhwod))
where EL denotes the categorical location embedding, and the second term projects the geographic coordinates (lat,, lon,) into the embedding space. Here, Woood E Rdloo is the projection matrix and deword denotes the projected dimension.
The spatio-temporal embedding E, E Rº is obtained by element-wise addition:
E, E
temporal
E
For future timestamps without known locations or missing historical records, the spatial component is set zero, allowing the model operate temporal information alone while preserving dimensional consistency.
Temporal Tokenization. Human mobility patterns exhibit inherent multi-scale temporal structures that span both short-term routines (c.g., daily activities) and long-term periodicities (c.g.. weekly rhythms) [58, 24). To effectively model these dynamies, RHYTHM employs temporal tokenization mechanism that effectively disentangles local patterns from global dependencies, inspired by Liu al. [40]. Formally, partition the embedded sequence X' into N non-overlapping segments each capturing meaningful temporal intervals (e.g., daily patterns):
{s1, 82, . SN
S; (E( E(5-1) ELL} for 1,2, .,,,,.
where each segment s; has length L (number of time steps). Within each segment, we employ intra-segment attention to model local temporal dependencies:
Attention(s.).
Our attention mechanism follows pre-norm transformer architecture with gated forwafo network as introduced by Dubey et al. [17], which enhances gradient flow during training and increases model expressivity. The implementation details are provided in Appendix C. To enable efficient modeling of cross-segment dependencies, we apply a learnable pooling operation that condenses each segment into discrete token representation:
SE, Pool (E()).
The resulting segment tokens (SE, SE ... ... SEn) } undergo inter-segment attention capture broader temporal context and long-range dependencies:
SELN Attention(SE
yielding refined segment-level embedding SE; E RDD that integrates contextual information across multiple temporal scales. By reducing the effective sequence length from T to N while preserving both fine-graines temporal dynamics and long-term dependencies, our approach addresses the computational challenges of modeling extended mobility trajectories.
Semantic Context Integration. Prior work such as FPT [81] employs LLMs as general-purpose sequential representation extractors. However, models like those in Wu et al. [67], Nie et al. [48] typically discard semantic embeddings and other critical information. For instance, timestamp attributes (e.g., day of the week, hour of the day) are essential for capturing chronological patterns in

---

human mobility, while spatial details Such prediction.
such as coordinates provide additional context for accurate
While recent works [40, 68, 80) begin to incorporate semantic information in sequential data, they typically employ traditional embedding approaches that fail to holistically capture the rich contextual information inherent in mobility patterns. Our work addresses this limitation by developing a mobility- based semantic embedding method that leverages detailed trajectory information. key challenge in utilizing LLMs for mobility prediction is balancing information richness with computational efficiency. Unlike approaches such as LLM-Mob [64] that rely on extensive prompting- whiic can lead to excessive context lengths and computational overhead, our approach breaks trajectory information into smaller, gegment-specifit pieces that retain essential mobility patterns while ensuring shorter prompts for each component. This decomposition significantly improves computational efficiency without sacrificing semantic richness. For each segment token, we provide informative trajectory descriptions, and for each future timestamp, we provide task descriptions and timestamp information, clarifying the expected output as shown in Appendix D. These prompts are then processed by pretrained LLMs to generate semantic embeddings. We adopt strategy that uses the special end-of-sequence (<EOS>) token for positional embedding. thereby integrating semantic information without extending the overall context length.
Technically, we define the semantic embedding TE; E RR for each token as follows:
TE.- SSleetLss(LLM(Pronptttt-----
Similarly semantic embedding of task description for future timestamp is defined as TET (LLM
Notably, TE pre-computed using the LLM, so runtime forward pass through the language model is not required during training.
Semantic-Temporal Alignment for Mobility Prediction. Since the latent space of the LLM encompasses both temporal tokens and semantic tokens, the semantic embedding can be aligned with the corresponding time span without extending the context length. Consequently, the combined embedding CE, for segment obtained by elementwise adding the segment embedding SE, and semantic embedding SE,
CE;- SE, + TE,
Here, TE, serves role similar to positional embeddings [61], while avoiding the sequence length overhead incurred by prompt concatenation [32]. Similarly, the combined embedding for future timestep T + j is computed as CEN+j = EN+ + TEy, following the combined embedding for N segments.
After obtaining the enriched embeddings CE,, we feed them into the backbone of RHYTHM-a pretrained LLM. The LLM processes these embeddings through its deep layers, performing in-context reasoning over the aligned temporal and semantic information, and yields contextualized hidden representations h; from its last hidden layer.
h4
LLM( CE.).
Then we apply an output projection layer to map the LLM's final representations to set of logits corresponding to candidate locations.
X(1,+)|X,7) = softmax(W. + b,),
where W, E R|4XDD These logits are then used to determine the most likely location predictions,
thereby generating human mobility forecasting as defined our problem statement. See Appendix for implementation details.
3.3 Computational Efficiency
RHYTHM achieves computational and parameter efficiency through several complementary design choices. Semantic embeddings are computed once clc--offfieeesssse the frozen LLM prior to model training, thereby eliminating any need for language-model inference at runtime. Simultaneously, our temporal tokenization mechanism significantly reduces sequence length from T H to N + H, thereby decreasing the quadratic attention complexity from O(TT + H)²) to O(NN + H)2), which is particularly valuable when processing extended mobility histories. Furthermore, we keep the LLM
5

---

backbone parameters frozen during training, resulting in faster convergence and reduced memory requirements. The combination of these approaches enables RHYTHM to efficiently process long mobility trajectories while maintaining strong predictive performance (as shown in Figure 3), making suitable for large-scale mobility prediction tasks where computational resources may constrained.
3.4 Theoretical Guarantee
We emphasize that our design choices provide strong theoretical guarantees. Employing an LLM as a universal sequential representation extractor provides two key advantages: (1) ensures the convergence of output values, as demonstrated in Zhou et al. [81, Theorem E.2]. and (2) it guarantees uniform distribution of the feature space the last hidden layer of the LLM, outlined in Zhou et al. [81, Theorem E.3]. Together, these properties enable LLMs to enhance the learning capability of the final multi-layer perceptron layer. Additionally, since our model is transformer-base [54] demonstrates that the transformer architecture is special case of modern Hopfield networks. Our approach has a guaranteed upper bound on memory retrieval error in LLMs [30, Lemma 3.2]. These theoretical benefits reinforce our method, and our results provide validation.
Ramsauer et al.
4 Experiment
In this section, we conduct experiments to demonstrate the performance and efficiency of RHYTHM. We evaluate the performance of RHYTHM on three real-world mobility datasets and compare with several state-of-the-art baselines. We also conduct series of ablation studies to investigate the effectiveness of the proposed strategies.
Models. To evaluate the model's performance on mobility prediction, we use multiple pretrained LLMs as the backbone of RHYTHM. These models are obtained from Hugging Face along with their pretrained weights. The specific LLM variants used are detailed in Appendix G.4.
Evaluation Metrics. For mobility prediction, we employ Accuracy@1 @k, where candidate locations are ranked based on model-predicted probabilities, and a prediction is considered correct if the true location is among the top <nd Mean Reciprocal Rank (MRR) to evaluate ranking performance. These metrics have been shown to correlate well with human mobility prediction tasks [23, 19]. We also utilize Dynamic Time Warping (DTW) [47] and BLEU [51] as real-world metrics to evaluate the performance. DTW quantifies their spatial alignment, while BLEU measures the n-gram similarity between predicted and ground-truth trajectories. Detailed descriptions about the evaluation metrics can be found in Appendix G.I.
Datasets. We evaluate our approach on three real-world datasets collected from the cities of Kumamoto, Sapporo, and Hiroshima sourced from YJMob100 [74]. Each day is divided into 48 time slots (each representing 30 minutes), though not every slot contains an observation. Each dataset is divided into training, validation, and test sets based days, with 70%, 20%, and 10% of the data allocated to each set, respectively. More details about the dataset can be found in Appendix F.
Settings. The temporal resolution is 30 minutes. In experiment, we window with 336 time slots and set the prediction horizon to 48 time slots (1 day). Also, we set the segment length as 48 time slots for experiments. The model is trained using cross-entropy loss to maximize prediction accuracy across the target locations.
7-day lookback
4.1 Overall Performance
To assess the efficiency of RHYTHM on human mobility prediction, we compare RHYTHM with several -f-te-of-the-art baselines. In this experiment, we evaluate the models on the test datasets using FP16 precision. We conduct each evaluation three times with different random seeds and present the average for each metric.
Baselines. To evaluate the performance of RHYTHM, we compare to LSTM-based models, transformer-based models, and L.L--based models. For the transformer-basee models, we conduct experiments with PatchTST [48], PMT [71], ST-MoE-BERT [26], CMHSA [27], iTransformer [39]

---

and COLA [65]. Among these models. ST-MoE-BERT PMT and COLA are the state-of ttate-of-the-an aht models for human mobility prediction. PatchTST and iTrrasformer are two powerful transformer- based models for time series forecasting. We add the spatiotemporal embedding to the input of those transformer-basee time-series models for fair comparison. For the I.I.M-based models, we conduct experiments with Time-LLM [32] and Mobility-LLLMM [23]. Time-LLM is a ofate-of-the-art model for time series forecasting using LLMs. We also add the spatiotemporal embedding to the input of Time-LLM for fair comparison. Additionally, in order to make fair comparison, we use the Llama-3.2 IB' as the pretrained LLM model for fine-tuning TLMe-LLM. Mobility-LLLM is a versatile LLM-based framework designed for multiple mobility tasks. For the -STM-based models, we conduct experiments with LSTM [34] and DeepMove [19].
Results. Table 1 show that RHYTHM outperforms the baselines across three datasets in most metrics. On the Sapporo and Hiroshima dataset, RHYTHM achieves the best performance in all evaluation metric. These findings underscore the effectiveness of RHYTHM in mobility prediction tasks. CMHSA and PMT may perform better in Accuracy @3 on Kumamoto due to their specialized attention mechanisms that effeetivvly capture mid-range candidate locations in this region. Despite sharing an LLM- architecture, Mobility-LLMN underperforms compared to RHYTHM, likely because it was primarily designed for visiting intention tasks requiring rich semantic context. In contrast, RHYTHM leverages temporal tokenization and LLM to model multi-scale spatio-temporal dependencies, prioritizing precise location likelihood maximization. This design focus enables RHYTHM to excel in top-rank precision metrics. Overall, RHYTHM achieves 2.4% improvement in Accuracy @1 I and 1.0% Accuracy @5 respectively compared to the best baseline model.
Table 1: Performance of RHYTHM and baselines on the Kumamoto, Sapporo, and Hiroshima datasets. The evaluation metrics include Accuracy @k for different values of k. The reported results averaged over three runs; variance values are omitted all are < 2%. The best results highlighted in bold, and the second-best results are underlined. RHYTHM demonstrates superior performance compared to baselines across most configurations.
Kumamoto
Sapporo
Hiroshima
Model
Acc@1 1
Acc@3
Acc
Acc@1
Acc@3
Acc@5
Acc@1 1
Acc@3
Acc@5
LSTM
0.2652
0,4799
0,5472
0.2310
0.3940
0.4526
0.2129
0.3775
0,4415
DeepMove
0.2779
0.4986
0.5683
0.2825
0.4672
0.5264
0.2804
0,4810
0.5477
PatchTST
0,2751
0,5018
0,5716
0.2703
0.4582
0.5168
0.2752
0.4839
0,5522
TTransformer
0,2609
0,4724
0,5412
0.2696
0.4500
0.5070
0,2804
0.4857
0.5523
Time-LLM
0,2712
0.4848
0,5535
0.2792
0.4746
0.5352
0.2698
0.4753
0.5426
CMHSA
0,2862
0.5182
0.5887
0.2890
0.4901
0.5525
0.2874
0.5001
0.5684
PMT
0.2697
0.4475
0.5187
0.2878
0.4896
0.5522
0,2850
0.4982
0.5668
COLA
0.2864
0.5186
0.5896
0.2847
0.4865
0.5497
0,2874
0.5013
0.5708
TT-MoE-BERT
0.2862
0.5155
0,5871
0.2869
0.4856
0,5480
0.2839
0.4925
0.5601
Mobility-LLMM
0.2666
0.4793
0.5448
0.2838
0.4703
0.5288
0.2826
0.4856
0,5525
RHYTHM-Llama-1
0.2929
0.5200
0.5835
0.2931
0.4876
0,5502
0.2913
0.5027
0.5753
RUYTHM--Gnnne Gemmaa2B
0.2923
0.5191
0,5932
0.2943
0.4896
0,5545
0.2953
0.5074
0.5798
RHYTHM-LJama-3B
0,2941
0.5205
0.5947
0.2938
0.4875
0.5523
0.2929
0.5032
0.5756
Geographical Evaluation. We evaluate RHYTHM against baseline models using BLEU and DTW, which respectively measure n-gram similarity and spatial alignment error between predicted and ground-truth trajectories. As shown in Table 2, RHYTHM scores the best DTW performance on Sapporo, demonstrating superior spatial alignment. While COLA leads in BLEU scores for all cities, RHYTHM ranks second in Kumamoto. This highlights a key trade-off between exact sequence matching and minimizing spatial deviations. One potential explanation is that COLA employs post-hoc adjustment technique that recalibrates predictions to better align with the long-tail frequency distribution of locations, which may enhance mid-tier accuracy by mitigating overconfidence in dom- inant locations. Notably, RHYTHM significantly outperforms LSTM-based methods and transformer baselines by leveraging temporal tokenization and prompt-guided reasoning to enhance sequential coherence and spatial precision. This results in an optimal balance for world mobility tasks. For MRR, RHYTHM consistently outperforms all baselines, achieving a 1.44% improvement over
latrpe://huggiingacc.. cw/meta- tanett-llammae 11w.ammmaaaaaammamam aaaaamaaaaa L1w.a-3.mm........
7

---

the best baseline and demonstrates its superior ranking capability across diverse mobility patterns. Additional experimental results and extended evaluations are reported in Appendix H.
Table 2: Performance comparison of RHYTHM with baselines using geographical metrics. The evaluation metrics include DTW (L), BLEU (1), and MRR (t). The best results are highlighted in bold, and the second-best results are underlined.
Kumamoto
Sapporo
Hiroshima
Model
DTW
BLEU
MRR
DTW
BLEU
MRR
DTW
BLEU
MRR
LSTM
5014
0.1564
0.3860
4507
0.1716
0.3270
5908
0.1544
0.3113
DeepMove
4630
0.1746
0,4021
3818
0.1959
0.3887
4981
0.1933
0.3959
PatchTST
5251
0.1315
0,4021
4099
0.1784
0.3773
5021
0.1884
0.3945
iTransformer
6178
0.1275
0.3796
4074
0.1780
0.3730
5094
0.1789
0.3977
Time-LLM
5984
0,1285
0.3912
3915
0.2145
0.3902
5126
0.1988
0.3872
CMHSA
4490
0.1810
0.4158
3786
0.2299
0.4034
4841
0.2289
0,4086
PMT
4536
0.1524
0.3720
3799
0.2017
0.4026
4851
0.2009
0.4065
COLA
4446
0.2064
0.4164
3793
0.2496
0.3996
4840
0.2445
0.4095
ST-MoE-BERT
4691
0.1557
0,4151
3796
0.2102
0,4001
4889
0.2117
0.4031
Mobility-LLM
5603
0.1649
0,3858
3911
0.1917
0.3902
4985
0.2056
0.3990
WHYTHM-IIlama-
4478
0.1793
0.4216
3745
0,2496
0.4045
5059
0,2083
0.4069
RHYTHM-Gemma-2B
4416
0.1928
0.4205
3995
0,2019
0,4065
4857
0.2109
0.4173
RHYTHM-Llama-33
4470
0.1814
0.4220
4035
0.1917
0,4048
4935
0.2093
0,4140
Transferability. To demonstrate that RHYTHM transfers well across pretrained LLMs, we vary the size of the pretrained backbone and train it on the mobility prediction datasets (see Table 1 for detailed results). In our experiments, we change the size of the pretrained model in RHYTHM and test them on the mobility prediction datasets. We use the Llama-3.2-11 Llama-3.2-3B, and Gemma- 2-2B model as the pretrained backbone of RHYTHM. The results indicate that the performance of RHYTHM improves as the model size increases. Notably, Llama-3.2-3B and Gemma-2-2B model outperforms the Llama-3.2-- B. model in most metrics. This result demonstrates the performance of RHYTHM scales with LLM size and suggests that larger models may achieve even greater performance improvements
larger datasets. Note that models are pretrained with 30 epochs. It's plausible that .lama-3.2233 model requires more epochs to fully converge and realize its full performance potential compared with Llama-3.2-11 model. However, Llama-3.2-3B model still achieves competitive performance compared to Llama-3.2- 1B model. Overall, Llama-3.2-3B model demonstrated 0.40% improvement in Acc @ compared to Llama-3.2-11 model, highlighting the scalability of RHYTHM.
RoVEnN By sAYY IEW 28
0.60
DeepSook
Accuracy@
0.29
DocpMoove
PNT
MeE-BETT
Mohity-1MM
Accuracy
0.57
Llama Qwen
Phi
Gemma
0.28
TImcuM
0.55
PaRdTTT Maansfoma
0.27
0.53
3.0
3.2
3.4
3.6
3.8
4.0
4.2
4.4
3.0
4.0
Iogio per Epach [s])
Log:a(Time per Epoch (s])
Figure 3: Training Speed vs. performance of RHYTHM and baseline models on the Sap- poro Dataset.
Figure 4: Efficiency comparison of alternative LLMs, evaluated by the same configuration of Tablle 4.
Training Speed. To evaluate the training speed of RHYTHM, we conduct experiments on Sapporo using the same training configuration. We run these experiments on a single NVIDIA A100 GPU with 40GB of memory. The results are shown in Figure 3. RHYTHM reduces the number of trainable parameters to only 12.37% of the full model size, reflecting its parameter-efficien design. In terms of runtime, it achieves a 24.6% reduction in training time compared with the best best-performing baseline, while remaining faster than most other models, being 80.6% faster than LLM-based methods on average. Although RHYTHM is slower than lightweight models such as LSTM, DeepMove, PStchTST, and iTransformer, it substantially outperforms them accuracy. Moreover, RHYTHM maintains computational efficiency comparable to PMT. COLA and TT-MoE-BERTI despite having
8

---

significantly higher parameter counts, demonstrating its parameter-efficien design and scalable architecture. Furthermore, RHYTHM's training speed scales predictably with parameter count: Llama-3B is 2.2 times slower than Llama- I model, while Gemma-2-2B shows a 1.9 times slowdown. detailed breakdown of preprocessing time, storage cost, and training time across datasets provided in Appendix I.
Daily and Weekly Trend Analysis. We analyze the periodic accuracy trends of RHYTHM and baselines on Sapporo. measuring performance fluctuations across daily and weekly intervals in Figure 5. RHYTHM demonstrates distinct performance characteristics: achieving 5.0% and 3.4% improvements during weekends and evening peak hours respectively, while showing comparable performance during highly regular periods like nighttime and standard weekday working hours. This pattern reveals fundamental Beight-RRyTHHi excels precisely when mobility prediction becomes a complex decision-make makinn task rather than simple pattern matching. During regular hours, mobility is largely deterministic with fixed routines where traditional models' pattern memorization suffices; however, weekends and transitional periods involve nuanced choices influenced by multiple contextual factors, aligning with findings from Barbosa et al. [3] on weekend variability. In these complex scenarios, RHYTHM's hierarchical attention captures both local daily context and global weekly patterns, while the LLM backbone provides reasoning capabilities decision points. This makes RHYTHM particularly valuable for real-world applications where handling irregular, unpredictable periods is crucial for system reliability, even if simpler models suffice for deterministic segments.
model non-routine
0.32
0.48
Accuracy@1
Accuracy@11
0.30
0.29
0.32
0.27
0.24
Sun
Mon
Tue
Wed
Thu
Fri
Sat'
00:00
04:00
08:00
12:00
16:00
20:00
DeepMove
ST-MoE-BERT
COLA
RHYTHM
Figure 5: Weekly (left) and daily (right) accuracy trends of RHYTHM and baselines on Sapporo.
These results illustrate that prediction performance fluctuates over both daily and weekly intervals.
4.2 Method Analysis
In this section, we perform ablation studies assess the effectiveness of the proposed strategies and test the scaling behavior of RHYTHM.
Ablation study. All experiments utilize Llama-3.2-11 3.2--BB as the backbone model. To evaluate our key components, we conduct ablation studies across three datasets, as shown in Table 3. Removing temporal tokenization significantly degrades performance by 5.39%, while climinating hicrarchical attention (HA) results in 0.90% decrease, demonstrating that structured temporal encoding is the most critical element of our framework. Regarding semantic enhancement, our findings indicate that both trajectory information and task description prompts contribute substantially to RHYTHM's effectiveness, with their removal causing a combined performance drop of 1. 1.82%. Task descriptions yield marginally higher impact than trajectory in information, with their removal causing an additional 10% performance decrease compared to omitting trajectory information. Further ablation studies examining the contribution of different design choices are included in Appendix J.
Scaling Behavior. Scalability is critical factor in the success of large-scale models. We assess RHYTHM's ' scalability by analyzing its performance in different model sizes. We conduct experi- ments using pretrained LLMs of varying sizes, including OPT, Llama-3.2, DeepSeek-R1, Gemma-2, Phi-2, and Qwen 2.5 detailed in Table 6. As shown in Table 4, the predictor's 's prediction performance generally improves as the number of LLM parameters increases. This observation is consistent with scaling law dynamics in large models [33]. This scaling behavior highlights the trade-off between predictive performance and adaptation cost. To capture this balance, we assess RHYTHM's 's scalability across three dimensions: model performance, parameter size, and training and inference speed (time

---

Table Ablation study on each module in RHYTHM. We evaluate each module's contribution to overall performance. The best results are highlighted in bold. Each module significantly influences RHYTHM's performance across all datasets.
Kumamoto
Sapporo
Hiroshima
Model
Acc@0
Acc
Acc
Acc
Acc@1
Acc@15
Acc
Acc
Acc
RHYTHM
0.2929
0.5200
0.5835
0.2938
0.4866
0,5502
0.2913
0.5027
0.5753
w/o HA
0.2917
0,5163
0,5881
0,2901
0.4856
0,5481
0.2895
0,4946
0.5657
w/o token
0.2801
0,5049
0,5764
0.2768
0.4775
0,5409
0.2749
0,4812
0,5535
w/o Traj info.
0.2914
0.5176
0,5891
0,2879
0,4842
0,5472
0.2858
0,4916
0.5633
w/o Task desc.
0.2895
0.5166
0,5889
0.2883
0.4839
0.5463
0,2882
0.4934
0.5648
per epoch), shown in Figure 4. Our results indicate that the largest model, Llama-3.2-3B, achieves the best performance for human mobility prediction, while Llama-3.2-1 1B remains the most suitable choice in RHYTHM, providing an optimal balance between performance and computational cost.
Table 4: Scalability Performance on RHYTHM. We conduct experiments to evaluate the scalability of RHYTHM on Sapporo using pretrained models of varying parameter sizes. The evaluation metrics include Accuracy @k MRR, training time per epoch (in seconds), and inference time per epoch (in seconds). The best results are highlighted in bold, while the second- -best results are underlined. In most configurations, the performance of RHYTHM improves as the model size increases.
Buckbone
Training Time (s) Inference Time (s)
Acc@1 1
Acc@3
Acc@5
MRR
OPT- 125M
787
107
0.2798
0,4726
0.5231
0,3819
OPT-350M
986
224
0,2837
0.4789
0.5343
0.3923
Llama-3.2-1B
5235
359
0,2929
0.5200
0,5835
0,4216
Qwen-2.5-1.5P
9241
336
0.2897
0.4873
0.5521
0,4049
DeepSeek-R1-1.5B
7308
335
0.2921
0.5164
0.5896
0,4188
Gemma-2-2B
9928
559
0.2923
0.5191
0,5932
0,4205
Phi-2
10047
693
0.2915
0.5166
0,5892
0,4183
Llama-3.2-3B
11566
762
0.2941
0.5205
0.5948
0.4220
Conclusion
This paper proposes RHYTHM, an efficient and scalable framework for mobility prediction. RHYTHM leverages temporal tokenization with hierarchical attention mechanisms to model spatio- temporal dependencies while incorporating semantic embeddings to capture cyclical patterns. The integration of frozen pretrained L.MMs reasoning engines enables RHYTHM to interpret nuanced decision-making processes that influence mobility choices, particularly scenarios with irregular or non-routine movement patterns, at reduced computational costs. Empirical results demonstrate that RHYTHM significantly outperforms state-of-the-an methods in accuracy. Moreover, its high scalability allows for the seamless integration of different pretrained LLMs in a plug-and-play manner, offering a flexible and efficient prediction framework.
Limitations. It is worth noting that RHYTHM has certain limitations. Its performance depends heavily on the quality of pretrained LLMs, which were designed for language tasks rather than mobility prediction. If these models patterns accurately. RHYTHM does not adopt an autoregressive prediction strategy; although widely studied in time-series modeling [40], we instead emphasize holistic sequence prediction to capture broader contextual dependencies. Future extensions may integrate autoregressive decoding to more closely mimic step-by-step human mobility decisions. In addition, while freezing pretrained LLMMs improves efficiency, RHYTHM's training time remains high, limiting its practicality in some applications. Despite these challenges, RHYTHM provides novel framework for mobility prediction, advancing efficiency and accuracy. Future work will focus on refining fine-tuning and quantization methods [43, 15, 73] improve scalability and reduce resource demands.
essorcc-constrainedd they may fail to capture user mobility
10

---

Acknowledgments and Disclosure of Funding
The authors would like to thank Mingzhen for insightful discussions and the anonymous reviewers for their constructive comments. H.H. and Q.R. W.'s work is supported by the National Science Foundation (NSF) under Grant Nos. 2125326. 2114197. 2228533, and 2402438, as well as by the Northeastern University iSUPER Impact Engine. H.L. partially supported by the OpenAl Researcher Access Program. This research was supported in part through the computational resources and staff contributions provided by the Quest High Performance Computing facility at Northwestern University, which is jointly supported by the Office of the Provost, the Office for Research, and Northwestern University Information Technology. Any opinions, findings, conclusions, or recom- mendations expressed in the paper are those of the authors and do not necessarily reflect the views of the funding agencies.
References
[1] Ibrahim M Alabdulmohsin, Behnam Neyshabur, and Xiaohua Zhai. Revisiting neural scaling laws in language and vision. Advances in Neural Information Processing Systems, 35:22300-
22312, 2022.
[2] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: visual language model for few-shot learning. Advances in neural information processing systems, 35:
23716-23736, 2022.
[3] Hugo Barbosa, Marc Barthelemy, Gourab Ghoshal, Charlotte R James, Maxime Lenormand, Thomas Louail, Ronaldo Menezes, José Ramasco, Filippo Simini, and Marcello Tomasini. Human mobility: Models and applications. Physics Reports, 734:1-74, 2018.
[4]
Michael Batty. The new science of cities. MIT press, 2013.
[5] William Berrios, Gautam Mittal, Tristan Thrush, Douwe Kicla, and Amanpreet Singh. Towards language models that preprint arXiv:2306. 16410, 2023.
Computer vision through the lens of natural language. arXiv
[6] Lufs MA Bettencourt, José Lobo, Dirk Helbing, Christian Kühnert, and Geoffrey B West. Growth, innovation, scaling, and the pace of life in cities. Proceedings of the national academy of sciences, 104(17):3301-300) 2007.
[7] Sebastiano Bontorin, Simone Centellegher, Riccardo Gallotti, Luca Pappalardo, Bruno Lepri, and Massimiliano Luca. Mixing individual and collective behaviors to predict out-of-routine mobility. Proceedings of the National Academy of Sciences, 122(17):24414488 122, 2025.
[8] Dirk Brockmann, Lars Hufnagel, and Theo Geisel. The scaling laws of human travel. Nature, 439470755:462-465, 2006.
[9] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 33:1877-1901, 2020.
[10] Oriol Cabanas- Tirapu, Lluís Danús, Esteban Moro, Marta Sales-Parde and Roger Guimerà. Human mobility is well described by closed-fom form gravity- like models learned automatically from data. Nature Communications, 16(1)::1366 2025.
[11] Ching Chang, Wei-Yat Yao Wang, Wen-Chih Peng, and Tien- Fu Chen. LIm4ts: Aligning pre- pre-trained Ilms as data-efficient time-series forecasters. ACM Transactions on Intelligent Systems and Technology, 16(3):1-20 2025.
[12] Eunjoon Cho, Seth A Myers, and Jure Leskovec. Friendship and mobility: user movement location-based social networks. Proceedings of the 17th ACM SIGKDD international conference Knowledge discovery and data mining, pages 1082-1090, 201
11

---

[13] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham. Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240): 1-113, 2023.
[14] Weizhen Dang, Haibo Wang, Shirui Pan, Pei Zhang, Chuan Zhou, Xin Chen, and Jilong Wang. Predicting human mobility via graph convolutional dual-attentive networks, In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pages 192-200, 2022.
[15] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36, 2024.
[16] Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng. Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. A survey on in-context learning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1107-1128, Miami, Florida, USA, November 2024. Association for Computational Linguistics.
[17] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.
Kadian, Ahmad Al-Dahle,
[18] Nathan Eagle and Alex Pentland. Reality mining: sensing complex social systems. Personal and ubiquitous computing, 10:255 268, 2006.
[19] Jie Feng, Yong Li, Chao Zhang, Funing Sun, Fanchao Meng, Ang Guo, and Depeng Jin. Deepmove: Predicting human mobility with attentional recurrent networks. In Proceedings of the 2018 world wide web conference, pages 1459-1468, 2018.
[20] Jie Feng, Yuwei Du, Jie Zhao, and Yong Li. AgentMove: A large language model based agentic framework for zero-shot next location prediction. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume Long Pupers), pages 1322-1338, Albuquerque, New Mexico, April 2025. Association for Computational Linguisties. ISBN 979-8-891766 76-189-6.
1:
[21] Luciano Floridi and Massimo Chiriatti. Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines, 30:681-694, 2020.
[22] Sébastien Gambs, Marc-Olivier Killijian, and Miguel Núñez del Prado Cortez. Next place prediction using mobility markov chains. In Proceedings of the first workshop on measurement, privacy, and mobility, pages 1-6, 2012.
[23] Letian Gong, Yan Lin, Xinyue Zhang, Yiwen Lu, Xuedi Han, Yichen Liu, Shengnan Guo, Youfang Lin, and Huaiyu Wan. Mobility-LLMM Learning visiting intentions and travel preference from human mobility data with large language models. In The Thirty-eighth Annual Conference on Neural Information P Processing Systems, 2024.
[24] Marta C Gonzalez, Cesar A Hidalgo, and Albert- Barabasi. Understanding individual human mobility patterns. nature,
2008
[25] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025.
(26) Haoyu He, Haozheng Luo, and Qi Wang. St-moe-bert: spatial-temporal mixture-of-experts framework for long-term cross-city mobility prediction. In Proceedings of the 2nd ACM SIGSPATIAL International Workshop on Human Mobility Prediction Challenge, pages 10-15, 2024.
[27] Ye Hong, Yatao Zhang, Konrad Schindler, and Martin Raubal. Context-aware multi-head self-attentional neural network model for next location prediction. Transportation Research Part C: Emerging Technologies, 156:104315, 2023.
12

---

[28] Shang-Ling Hsu, Emmanuel Tung, John Krumm, Cyrus Shahabi, and Khurram Shafique. Trajgpt: Controlled synthetic trajectory generation using a multitask transformer- spa- tiotemporal model. In Proceedings of the 32nd ACM International Conference Advances in Geographic Information Systems, pages 362-371, 2024.
[29]
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021.
[30] Jerry Yao-Chieh Hu, Maojiang Su, En jui kuo, Zhao Song, and Han Liu. Computational limits of low-rank adaptation (loRA) fine-tuming tunin for transformer models. In The Thirteenth International Conference Learning Representations, 2025.
[31] Allen H Huang, Hui Wang, and Yi Yang. Finbert: A large language model for extracting information from financial text. Contemporary Accounting Research, 40(2):806 84000 2023.
[32] Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y. Zhang, Xiaoming Shi, Pin- Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, and Qingsong Wen. Time-LLM: Time series forecasting by reprogramming large language models. In The Twelfth International Conference on Learning Representations, 2024.
[33] Jared Kaplan. Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models, arXiv preprint vrXiv:201..086/1 2020.
[34] Dejiang Kong and Fei Wu. Hst-lstm: A hierarchical spatial-temporal long-short term memory network for location prediction. In Ijcai, volume 18, pages 2341-2347, 2018
[35] Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4): 1234-1234111 1240, 2020.
[36] Weiyu Li, Qi Wang, Yuanyuan Liu, Mario L Small, and Jianxi Gao. A spatiotemporal decay model of human mobility when facing large-scale crises. Proceedings of the National Academy of Sciences, 119(33)=e20004411 19, 2022.
[37] Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. Predicting the next location: recurrent model with spatial and temporal contexts. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016.
[38] Yifan Liu, Xishun Liao, Haoxuan Ma, Brian Yueshuai He, Chris Stanford, and Jiaqi Ma. Human mobility modeling with limited information via large language models. arXiv preprint arXiv:2409. 0.009./7499 /7495, 2024.
[39] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024.
[40] Yong Liu, Guo Qin, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Autotimes: Autoregressive time series forecasters via large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[41] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Leurning Representations, 2019.
[42] Massimiliano Luca, Gianni Barlacchi, Bruno Lepri, and Luca Pappalardo. A survey on deep learning for human mobility. ACM Computing Surveys (CSUR), 55(1):1-44, 2021.
[43] Haozheng Luo, Chenghao Qiu, Maojiang Su, Zhihan Zhou, Zoe Mchta, Guo Ye, Jerry Yao- Chieh Hu, and Han Liu. Fast and low-cost genomic foundation models via outlier removal. arXiv preprint arXiv:2505.00598, 2025.
[44] Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie- Liu. Biogpt: generative pre-trained transformer for biomedical text generation and mining. Briefings in bioinformatics, 23(6)bbac409, 2022.
13

---

[45] Yingtao Luo, Qiang Liu, and Zhaocheng Liu. Stan: Spatio-temporal attention network for next location recommendation. In Proceedings of the web conference 2021, pages 2177-2185, 2021.
[46] Soumi Maiti, Yifan Peng. Shukjae Choi, Jec-weon Jung, Xuankai Chang, and Shinji Watanabe. Voxtlm:
Unified decoder-only models for consolidating speech recognition, synthesis and speech, text continuation tasks. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 13326-13330. IEEE, 2024.
[47] Meinard Müller. Dynamic time warping. Information retrieval for music and motion, pages 69 84, 2007.
[48] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. The Eleventh International Conference on Learning Representations, 2023.
[49] Zhenyu Pan, Haozheng Luo, Manling Li, and Han Liu. Conv-coa: Improving open- domain question answering in large language models via conversational chain-of-action. arXiv preprint arXiv:2405.17882 17822, 2024.
[50] Zhenyu Pan, Haozheng Luo, Manling Li, and Han Liu, Chain-of-action: Faithful and multimodal question answering through large language models. In The Thirteenth International Conference on Learning Representations, 2025.
[51] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings the 40th annual meeting of the Association for Computational Linguistics, pages 311-318, 2002.
[52] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, beghppefformacc deep learning library. Advances in neural information processing systems, 32, 2019.
[53] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pages 8748-8763. PMLR, 2021.
[54] Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Lukas Gruber, Markus Holzleitner, Thomas Adler, David Kreil, Michael K Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. Hopfield networks is all you need. In International Conference on Learning Representations, 20211
[55] Xuan Rao, Lisi Chen, Yong Liu, Shuo Shang, Bin Yao, and Peng Han. Graph-flashhack network for next location recommendation. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 1463-1471, 2022.
[56] Filippo Simini, Marta C González, Amos Maritan, and Albert-László Barabási. A universal model for mobility and migration patterns. Nature, 484(7392))96-110 2012.
[57] Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge. Nature, 620(7972):722-111 2023.
[58] Chaoming Song, Zehui Qu, Nicholas Blumm, and Albert- Barabási, Limits of predictabil- ity in human mobility. Science, 327(5968):
2010.
[59] Qwen Team. Qwq-32b: Embracing the power of reinforcement learning. March 2025.
[60] Maria Tsimpoukelli, Jacob L Menick, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. Multimodal few-shot learning with frozen language models. Advances in Neural Information Processing Systems, 34:200-212 2021.
[61] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, tukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
14

---

[62] Geoff Vigar. The politics of mobility: transport, the environment, and public policy. Taylor & Francis, 2002.
[63] Qi Wang, Nolan Edward Phillips, Mario L Small, and Robert J Sampson. Urban mobility and neighborhood isolation in america's is 50 largest cities. Proceedings of the National Academy of Sciences, 1 11(33))7735--7740, 2018.
[64] Xinglei Wang, Meng Fang, Zichao Zeng, and Tao Cheng. Where would i go next? large language models as human mobility predictors. arXiv preprint arXiv:2308. 15197, 2023.
[65] Yu Wang, Tongya Zheng, Yuxuan Liang, Shunyu Liu, and Mingli Song. Cola: Cross-city mobility transformer for human trajectory simulation. In Proceedings of the ACM on Web Conference 2024, pages 3509-3520, 2024.
[66] Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of -f-thughttt prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 24437724-24877, 2022.
[67] Dennis Wu, Jerry Yao-Chiel Hu, Weijian Li, Bo- Yu Chen, and Han Liu. STanhop: Sparse tan- dem hopfield model for memory-enhance time series prediction. In The Twelfth International Conference on Learning Representations, 2024.
[68] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting, 2022.
[69] Shang Wu, Yen-Ju Lu, Haozheng Luo, Maojiang Su, Jerry Yao-Chich Hu, Jiayi Wang, Jing Liu, Najim Dehak, Jesus Villalba, and Han Liu. SPARQ: Outlier-free speechLM with fast adaptation and robust quantization, 2025.
[70] Shijie Wa, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: large language model for finance. arXiv preprint arXiv:2303. 2023.
[71] Xinhua Wu, Haoyu He, Yanchao Wang, and Qi Wang. Pretrained mobility transformer: A foundation model for human mobility. arXiv preprint arXiv:2406.02578. 2024.
[72] Yongji Wu, Defu Lian, Shuowei Jin, and Enhong Chen. Graph convolutional networks user mobility heterogeneous graphs for social relationship inference. In IJCAI, pages 3898-3904, 2019.
[73] Guangxuan Xiao, Ji Lin, Mickael Seznee, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pages 38087-38099, PMLR, 2023.
[74] Takahiro Yabe, Kota Tsubouchi, Toru Shimizu, Yoshihide Sekimoto, Kaoru Sezaki, Esteban Moro, and Alex Pentland. Yjmob100k: City-scale and longitudinal dataset of anonymized human mobility trajectories. Scientific Data, 11(1):39) 2024.
[75] Dingqi Yang, Daqing Zhang, Vincent W Zheng, and Zhiyong Yu. Modeling user activity preference by leveraging user spatial temporal characteristics in lbsns. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 45(1):129-142, 2014.
[76] Dingqi Yang. Benjamin Fankhauser, Paolo Rosso, and Philippe Cudre-Mauroux. Location prediction over sparse user mobility traces using mnns. In Proceedings of the twenty-ninth international joint conference on artificial intelligence, pages 2184-2190, 2020.
[77] Song Yang, Jiamou Liu, and Kaiqi Zhao. Getnext: trajectory flow map enhanced transformer for next poi recommendation. In Proceedings of the 45th International ACM SIGIR Conference on research and development in information retrieval, pages 1 1144-1 1153, 2022.
[78] Wei Zhai, Xueyin Bai, Yu Shi, Yu Han, Zhong-Ren Peng, and Chaolin Gu. Beyond word2vec: An approach for urban functional region extraction and identification by combining place2vec and pois. Computers, environment and urban systems, 14:1-12, 2019.
15

---

[79] Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross cross- dimension dependency for multivariate time series forecasting. In The eleventh international conference on learning representations, 2023.
[80] Haoyi Zhou, Shanghang Zhang, Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting, 2021.
[81] Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, and Rong Jin. One fits all: Power general time series analysis by pretrained LM. In Thirty-seventh Conference Neural Information Processing Systems, 2023.
[82] Zhihan Zhou, Weimin Wu, Jicke Wu, Lizhen Shi, Zhong Wang, and Han Liu. Genomeocean: Efficient foundation model for genome generation, 2025.
16

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
The abstract and introduction clearly state the main claims made in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in Section 5,
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-texx system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess..
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
17

---

Justification:
We provide theory guarantees of RHYTHM in Section 3.4.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide all the details necessary to reproduce the main experimental results in the paper. The details of the experiments are included in both the main paper and the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
18

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The code and data, including detailed instructions to reproduce the main experimental results, are publicly available at btttp://githuul htte//eehh/rhythee
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
wttpeauuisssisoooosoooooooooooee onPolicy) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nips. wwtppblicc///cccccccccccccccclct wwwpc//ggss//ee//eu /Ctpe//boeSebieeeece wwweesuubisssionPee ssiissolicyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We introduce the experimental setting main paper and provide additional details in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We report the values in all tables and observed stable results with less than 2%
variance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
19

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We introduce the compute resources in Appendix G.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: penttss:/neurrte /wwrrps.....p//
Answer: [Yes]
Justification: We have carefully reviewed and followed the NeurIPS Code of Ethics through- out our research process.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the broader impacts in Appendix A.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
20

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our model does not have such a risk misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have properly credited all assets used in research and respected their licenses and terms of use.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
21

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. cww/co/////oo/o datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [NA]
Justification:
We do not release new assets in our work.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/codeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: We do not include crowdsoureing experiments or research involving human subjects in this paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (TRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: We do not include crowdsourcing experiments or research involving human subjects in this paper.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
22

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
We only use LLMs to assist with writing and formatting the paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: httpp:://neurppee 200/ for what should or should not be described.
23

---

Supplementary Material
A Broader Impact
24
Extended Related Work
24
C Attention Implementation Details
26
Prompt Design Examples
26
Implementation Details
27
Dataset
28
Experiment Settings
28
G.I Evaluation Metrics
28
G.2
Computational Resource
G.3
Hyperparameters
................................
28
G.4
LLM variants
29
H Additional Experimental Results
29
H.1 Autoregressive vs Non-autooon von-autoregrressve Strategy
................................
H.2
Deployment efficiency compared to LLM-basee baselines
29
I Resource Requirements and Computational Cost
30
L.1
Dataset Preprocessing Training Resource Usage
................................
1.2
30
................................
J
Additional Ablation Studies
30
J.1
Segment Length Sensitivity
................................
J.2
Impact of Pretrained LLMs Computational Cost of RHYTHM Components
................................
31
J.3
A Broader Impact
This paper introduces and generalization of foundation model applications in spatio-temporal domains. While the work does not have immediate societal implications, it lays the groundwork for future applications in urban planning, public health, disaster response, and transportation. However, the model may inadvertently encode amplify biases present in the training data, potentially leading inequitable outcomes in mobility predictions.
foundation model for human mobility, aiming to improve the reliability
B Extended Related Work
Mobility Prediction. Human mobility prediction has evolved from foundational statistical models to advanced deep learning frameworks. Physics-inspired models such as the gravity model [10] and the radiation model [56] predict aggregate population flows using distance and opportunity metrics but lack individual-level detail. To address this, probabilistic approaches like Markov chains [22] and tensor factorization [75] emerge, modeling location transitions the user level. While these methods improve personalization, they struggle with sparse trajectories and higher-order dependencies inherent in real-world mobility data.
Deep learning introduces sequence-aware architectures like LSTM [37], which capture local temporal contexts, and attention-enhanced variants [19] that address vanishing gradients. However, these models often overlook cyclical patterns. Hybrid approaches like Graph-Flashback [55] and GC- DAN [14] integrate graph structures to model spatial relationships, but their reliance fixed-lengh

---

sequences limits scalability for long term forecasting. Transformers [61] revolutionize the field with self-attention mechanisms for long-term dependency modeling. Innovations like STAN [45] combine spatial-temporal attention for next-PO] recommendation, while COLA [65] extends this to cross-city dynamies. GETNext [77] further refines predictions by disentangling individual preferences from population flows. Despite these advances, Transformer- based methods remain timestamp-centric incurring quadratic complexity for multi-day sequences and failing to explicitly model hierarchical periodicities (e.g., daily vs. weekly rhythms).
While Transformer- based approaches improve long-teny pengtterm dependency modeling, they still rely on timestamp-centrii encoding and struggle with hierarchical periodicities. Recent work explores large language models (LLMs) as an alternative, leveraging their strong generalization capabilities for mobility tasks [23, 38]. Studies like LLM-Mob [64] and AgentMove [20] leverage prompt engineering for next-location prediction and trajectory user linking, while TrajGPT [28] generates synthetic visits via autoregressive decoding. However, these approaches treat mobility sequences as generic token streams, neglecting structured periodic patterns and the modality gap between natural language and spatio-temporal data. Unlike existing LLM-based methods that treat mobility sequences as generic tokens, our approach uses temporal tokenization to explicitly model structured periodicity (daily/weekly cycles), thereby mitigating modality mismatches and capturing multi-scale dependencies for improved long-term mobility prediction.
Time Series Foundation Models. Existing time series foundation models can be divided into two categories: transformer- based models and language-baud basedd models. For transformer-bast based time series models [67, 39, 48), prior studies focus on transformer architecture and self-attention mechanisms to capture temporal dependency in time series data. For instance, PatchTST [48] introduces a patch- based self-attention mechanism to capture long-range dependencies in time series data. STanHop [67] and Crossformer [79] employ hierarchical self-attention to capture temporal dependencies and hierarchical structures in time series data. For language- time series models [40, 32], prior studies adapt the LLMs to time series data and achieve state-of-the-art performance in time series forecasting tasks. For instance, AutoTime [40] introduces novel autoregressive structure to capture the temporal dependency in time series data. Time- [32] employs a large language model to capture the complex transitions of time series data. However, these models struggle to capture the inherent complexity of human mobility- with its abrupt location shifts and temporal dynamics- whereas RHYTHM leverages a novel spatio-tempone embedding paired with an autoregressive framework effectively model these intricate transitions.
Cross-domain Adaptation of LLMs. LLMs have evolved from specialized natural language processing systems into versatile foundation models capable of sophisticated reasoning across diverse tasks [1, 9]. Their wansformer-based architecture and extensive pretraining have enabled remarkable transfer capabilities to domains beyond text. computer vision, models like CLIP [53] align visual and textual representations for zero-shot recognition, while in time series analysis, approaches such as One-Fits-AA [81] and LLM4TS [11] demonstrate competitive forecasting through tokenized numerical sequences. In biomedicine, BioBERT [35] and BioGPT [44] demonstrate significant gains on clinical NLP benchmarks, while instruction-tunee models like Med-PaL.M approach expert- level medical QA performance [57]. In finance, domain-speciff LLMs such as FinBERT [31] and BloombergGPT [70] substantially outperform general- information extraction.
general-purpose purpose models on sentiment analysis and
Rather than computationally expensive full fine-tuning, parameter-efficient adaptation strategies have gained prominence. Low-Rank Adaptation (LoRA) [29] introduces trainable low-rank matrices into attention layers, while more recent approaches keep LLMs entirely frozen by using lightweight adapters that project non- linguisii inputs into the model's embedding space. In vision, prefix-tuning methods [2, 60] train small encoders to produce "prompts" " for frozen LLMs, while time series approaches [40, 32] employ projection layers to convert numeric sequences into token embeddings.
Applications of LLMs to human mobility modeling remain limited, with existing approaches relying primarily on snrameter-intensiv adaptation. Mobility-LLA. [23] employs partial fine-tuning, while LLM-Mob [64] leverages in-context learning but lacks structured temporal modeling. In contrast, RHYTHM maintains fully frozen LLM backbone, preserving the model's pre-trained knowledge while introducing a specialized spatio-temporal framework that efficiently adapts to mobility data characteristics.
25

---

C Attention Implementation Details
Our attention mechanism implements pre-norm transformer block enhance training stability, with gated feed- cordarr network for improved expressivity. The mathematical formulation of our attention block is as follows:
Z LayerNorm(X (X) + Multi-Head Attention(LayerNorm((Y
7 = Z + (LayedFFN(Lyye/Noommm3 (2)),
where is the input sequence. The multi-head attention operation computes:
Multi-Head(X X)
[head] |lheady ... [head]] Wouts XWW,, XW,,;(XWa.))
head, Softmax
XW, We.i
Vdk
with h attention heads, where 9, WK,i, Wo,i Rdxdd are the query, key, and value projection matrices for the i-th head, and Wout E Rdd is the output projection matrix. The gated feed-forward network incorporates an adaptive gating mechanism:
GatedFFN( FFN( (2)= O ( gate Z), FFN(Z) = W2 GFLU(W1
where denotes the sigmoid function, represents element-wise multiplication, and W gute Rdxd the learnable gating matrix. The foed-forward network expands the hidden dimension by a factor of 4, with W1 R4dxd and W2 Rddxdd Dropout is applied after both the attention and feed-forward operations to prevent overfitting.
D Prompt Design Examples
Trajectory Information
This is the trajectory of user <User_ _D> of day <Day_ID> which is a <fay_of_Week>. The trajectory consists of <ti> records, each record of
coordinate is as follows:
08:30: (X=136, Y=42);
09:00: (X=136, Y=42)
09:30: (X=137, Y=41); ;
10:00:
(X=146, Y=37);
10:30:
(X=145, Y=38);
11:00:
(X-144, Y-38);
11:30:
(X-135, Y-41);
12:00:
(X-135, Y-42):
12:30:
(X=135, Y=42);
13:00:
(X=135, Y=42)
Key transitions: At 10:00: (X-137, Y-41)- (X-146, Y-37); At 11:30: (X-144,
Y-38) + (X-135, Y-41).
Main stay locations: (X=136, Y=42) from 08:30 to 09:30 (0.5 hours); ; (X=145, Y=38) from 10:00 to 11:00 (0.5 hours); ; (X=135, Y=42) from 11:30 to 13:00 (1.5 hours)
Task Description
You mobility prediction assistant that forecasts human movement patterns urban environments. The city represented 200 200
grid of cells, where each cell is identified by coordinates (),Y). The
26

---

coordinate increases from left (0) to right (199), and the Y coordinate increases from top (0) to bottom (199).
TASK: Based User <User_I>>'s historical movement patterns, predict their locations for Day <Day_ID> ID> (<Day_of. Neek>). The predictions should capture expected locations 30-minute intervals throughout the day (48 time slots). The model should analyze patterns like frequent locations, typical daily routines, and time-dependent behaviors to generate accurate predictions of where this user is likely to be throughout the day.
previous days' trajectory data contains information about the user's typical movement patterns, regular visited locations, transition times, and duration of stays. Key patterna to consider include: home and work locations, morning and evening routines, lunch-time behaviors, weekend vs. weekday differences, and recurring visit patterns.
E
Implementation Details
Algorithm RRYTHM - Overall Pipeline
Require: Trajectory X = [(t,,4))= (timestamps and location IDs); prediction horizon (tr+1...., tt } segment length L; frozen LLM
1: function PRECOMPUTEESEMATTICCSX L, (tr+1,-.... (T LLM)
2: N T/TL
3: for = 1 to N do
trajectory information
4: TE; + Selectaas(LLM((Proppp,,
5: end for
6: TET + SelectLas(LLM(PPomm)) (It tT4H }))
task description
7: return (TE,) TE
8:
end function
9: function PREDICT(X, (rp) (TE) TET, L,LLM)
10: for i = to T do
embed time + location into token space
11:
E Espatial SpatialEmbed(())
Elemporal &- Temporallmmeel(4)
12:
13:
E. - temporal
E
spatial
end for
Partition (E into N T/L segments 8; = (F(-1))4+::L)
16: for
= to N do
b intra-segment attention, then pool to one token
17:
()) IntraAttention(ss)
18:
SE; +- Pool(E())
19:
end for
20: (SE,)N
InterAttention ((SEE)
D model long-range dependency additive alignment of semantics at segment level
21: for
1 N do
22:
CE₂ < SE, TEE
23:
end for
24:
for j= 1 to H do
> future-time tokens + task semantics
EN+j & TemporalEmbec((rr) CEN+j + EN+; + TET
26:
27:
end for
b1:N+H + LLM
frozen backbone forward
29:
P1:11 (eolmax(Pro/ToCClssesss//a
30:
return {arg max p/
31: end function
27

---

Dataset
We provide the detail of datasets used in this paper as shown in Table 5.
Table 5: Dataset Statistics
City
Users
Duration
Spatial Resolution
Places
Kumamoto
3k
75 days
500m x 500m
40k
Sapporo
17k
75 days
500m x 500m
40k
Hiroshima
22k
75 days
500m x 500m
40k
G Experiment Settings
G.1 Evaluation Metries
Accuracy@t @k measures proportion correct predictions within top-k ranked locations:
Accuracy @k
Σ
1(IT+i E top-k(fr+i)),
H
1=1
where 1(.) is the indicator function and Pr++ is the predicted probability distribution.
Mean Reciprocal Rank (MRR) evaluates quality ranked predictions:
H
MRR
H
rank(
2=1
where rank(Ir+:) is the rank position of the true location.
Dynamic Time Warping (DTW) measures spatial similarity between trajectories:
DTW(V. min Σ d(lT+i,(+5))
(KJ)ER
where æ is valid warping path and d(..-) the Euclidean distance.
BLEU quantifies n-gram overlap between predicted and ground-truth sequences:
N
BLEU = BP exp
Σ
w, log Pr
where Pn, is n-gram precision, wn is the weight for each n- n-gram level, and BP is brevity penalty.
G.2 Computational Resource
We perform all experiments using single NVIDIA A100 GPU with 40GB of memory and 24-core Intel(R) Xcon(R) Gold 6338 CPU operating at 2.00GHz. Our code is developed in PyTorch [52] and utilizes the Hugging Face Transformer Library² for experimental execution.
G.3 Hyperparameters
We present the hyperparameters used in the training stage for each model. Embeddings for time- of-day and day-of-week, the categorical location embedding, and the coordinate projection all use hidden dimensions of
For model training, 128, 128, 256, and 128, respectively. We use Adam [41] as the optimizer. conduct a systematic hyperparameter search, exploring learning rates from the set (1 x 10 4 3 x 10 5 x 10 10-4) and weight decay values from {0, 0.001, 0.01). Through extensive validation experiments, we determine the optimal configuration for each dataset. All models are trained with a consistent batch size of 64 across all datasets for fair comparison. The final hyperparameter settings are selected based performance the validation set.
10-4,3
Pattpe://hugginggaace http:///c...ccocccccceeeeeeeeeee
28

---

G.4 LLM variants
Our experiments deployed multiple foundation language models as text embedders and frozen back- bones within RHYTHM to evaluate cross-scale performance. Table 6 presents the pre-trained models accessed through the Hugging Face Transformers library, ranging from 125M to 3B parameters.
Table 6: Pre-trained language models employed backbones in RHYTHM.
Model
Parameters
HuggingFace Repository
OPT-125M
125M
1reebook/opt-125mm
OPT-350M
350M
facebook/opt-35500
Llama-3.2-15
1.24B
meta-llama//Llama-32--1B
Qwen-2.5-115B
1.54B
Owen/Qween.55-1.5B
DeepSeek-R1-1.5t
1.78B
Gemma-2-2B
2.61B
googlefgemma-2222-it
Phi-2
2.78B
microsoft/phi-22
Llama-3.2-3B
3.21B
meta-llama/llamaa3222333
H Additional Experimental Results
H.1
Autoregressive vs Non-autoregreesive Strategy
Table compares RHYTHM under autoregressive and non-autoreggessive strategies. The non- autoregressive approach delivers comparable accuracy while being over two orders of magnitude faster.
Table 7: Comparison of RHYTHM with autoregressive and non-autoregressive prediction strategies. Results are reported using Acc@ I, Ace@3, Acc@5, and computational time per iteration (s/iter). The best results highlighted in bold.
Model
Time (s/iter)
Ace@1 1
Acc@3
Ace@5
Non eooneuutoeerressvve
0.96
0.2929
0.5200
0.5835
Autoregressive
39.80
0.2884
0,5247
0.5801
H.2 Deployment efficiency compared to LLM-based baselines
Table 8 reports deployment efficiency for RHYTHM compared with LLM-based baselines, evaluated under both GPU and CPU inference. On GPU, RHYTHM requires substantially less memory and achieves lower latency than TimeLLM, while remaining competitive with Mobility-LLM, On CPU, RHYTHM also demonstrates favorable latency and moderate RAM usage, underscoring its practicality for deployment resource-cone constreecn environments.
Table 8: Deployment efficiency of RHYTHM, TimeLLM, and Mobility-LLM. Deployment efficiency of RHYTHM, TimeLLM, and Mobility-LLM. We report model size, GPU memory footprint (GRAM), inference latency GPU and CPU, and RAM usage.
GPU
CPU
Model
Size (MB)
GRAM (MB)
Latency (ms)
RAM (MB)
Latency (ms)
RHYTHM
5841.9
5741.4
261.6 0.8
12497.9
39.5 0.9
TimeLLM 3762.7
11213.1
392.7 = 9.5
18677.8
64.2 = 2.5
Mobility-LLM
4880.7
9872.8
192.1 + 5.2
10552.2
32.3 1.9
29

---

I Resource Requirements and Computational Cost
In this section, detail RHYTHM's resource requirements to provide a practical perspective on the computational cost of deploying RHYTHM.
1.1 Dataset Preprocessing
Time and storage costs of semantic embedding generation are detailed in Table 9.
Table Preprocessing cost RHYTHM across datasets.
Dataset
Preprocessing Time (h)
Storage Size (GB)
Kumamoto
3.1
1.6
Sapporo
15.9
9.1
Hiroshima
20.1
11.8
1.2 Training Resource Usage
Training requirements with varying sequence lengths are summarized in Table 10, including GPU memory (GRAM) and runtime per epoch. The results show that memory consumption and training time do not increase linearly with sequence length, demonstrating that RHYTHM can handle long sequences moderate additional cost due to its temporal tokenization design.
Table 10: Training cost with varying sequence lengths.
Sequence Length (2)
GRAM (MB)
Time/Epoch (min)
48
7126.3
23.1
168
7469.5
24.6
336
8202.0
26.5
672
9826.4
30.9
Additional Ablation Studies
J.1 Segment Length Sensitivity
We analyze the impact of varying temporal segment length L on prediction accuracy and efficiency in Table 11. Smaller segments (e.g., L 1, 30 minutes) capture fine-grained details but result in excessive fragmentation and substantially higher computation time. In contrast, very large segments (e.g., L 96, 2 days) reduce runtime but blur meaningful daily mobility boundaries, leading to degraded accuracy. Daily segmentation (L = 48) achieves the best balance, yielding the highest predictive performance while keeping iteration time under second. These findings validate choice of 48 as a balanced temporal unit for mobility modeling.
Table 11: Effect of varying segment length L prediction accuracy and runtime. The best results are highlighted in bold.
Segment Length (L)
Segments (N)
Acc@1
Acc@3
Are@5
Time (s/iter)
1 (30 min)
336
0,2801
0.5049
0.5764
6.59
24 (12 hr)
14
0.2883
0.5124
0.5792
1.10
48 (1 day)
7
0.2929
0.5200
0,5835
0.96
96 (2 days)
3
0.2851
0.5087
0.5743
0.93
30

---

J.2 Impact of Pretrained LLMs
RHYTHM benefits substantially from pretraining: the pretrained LLM variant achieves the strongest accuracy, while randomly initialized and LLM-free variants lag behind as shown in Table 12.
Table 12: Comparison of RHYTHM with pretrained, randomly initialized, and no-LLM variants on Kumamoto. Pretraining provides the best accuracy across all metrics.
RHYTHM Variant
Acc@1
Ace@3
Acc@5
w/ pretrained LLM (ours)
0.2929
0.5200
0,5835
w/ randomly initialized LLM (frozen)
0.2556
0.4842
0,5313
w/o LLM (only attention)
0,2749
0.4921
0.5623
J.3 Computational Cost of RHYTHM Components
Table 13 shows how different architectural choices affect model size and training time. Temporal tokenization and hierarchical attention both contribute to reducing runtime without significantly increasing parameter count, highlighting their role in RHYTHM's efficient design.
Table 13: Trainable parameters (absolute and relative) and training time per epoch for different architectural configurations of RHYTHM on Kumamoto.
Configuration
Trainable Params (MB)
Share of Total (%)
Time/Epoch (min)
Full RHYTHM (frozen LLM)
152
12.37
31
Unfrozen LLM w/ LoRA
200
16.27
102
w/o Temporal Tokenization
148
12.00
53
Only Attention Module (no L.LM)
152
12.37
16
w/o HA
39
3.25
20
31


Reviews:
[{"review_id": "wCe8e0wH36", "reviewer": "Reviewer_BxoE", "summary": "RHYTHM introduces hierarchical temporal tokenization that chops each trajectory into daily segments, pools them with inter-segment attention, and feeds the enriched tokens augmented by pre-computed prompts to a frozen LLM backbone for next-location prediction. The design trims attention complexity, retains multi-scale periodicity, and pairs a parameter-efficient adaptation strategy with prompt-guided semantic context. The experiment section shows its superior comparing with the SOTA models.", "questions": "Refer the weakness part", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. Hierarchical temporal tokenization compresses each user trajectory into daily “segment tokens,” allowing segment attention to preserve daily and weekly rhythms while shortening sequences\n2. The model keeps the large language model’s weights fixed and extracts semantic embeddings obtained from the frozen LLM’s final hidden state of prompt-encoded inputs as rich, static embeddings, delivering substantial compute savings without sacrificing accuracy.\n3. On the benchmark, RHYTHM raises Accuracy@1 by 2.4% overall (5% on weekends) while cutting training time by 24.6%, outperforming strong mobility baselines such as COLA and ST-MoE-BERT.\n4. Comprehensive ablations and scaling studies show that tokenization delivers the largest share of gains, prompts add a further boost, and attention savings scale favorably up to billion-parameter backbones.\n\nWeaknesses\n1. The frozen LLM backbone may not adapt to domain-specific nuances (region-specific behaviors or transit rules). Semantic prompts are predefined, not dynamically learned or updated.\n2. While temporal patterns are captured hierarchically, spatial information is encoded via simple embeddings without incorporating topology, limiting fine-grained spatial understanding. Also space and time were modeled jointly via embeddings but omits explicit relational modeling like graphs or trajectory continuity, which may weaken performance in connectivity-aware tasks.\n3. Evaluation is confined to the Japanese-only YJMob100K dataset; results on other continents or longer temporal spans are important yet missing (longer than 1 day window).\n4. Predictions are made in a single shot (non-autoregressive), without iterative feedback, performance on rolling horizons remains untested.", "was_revised": true, "final_justification": "The authors provided detailed and rebuttals, including additional experiments demonstrating generalization to a non-Japanese dataset  and longer prediction horizons. Thanks for these new results.\n\nWhile the spatial modeling remains relatively simple, this is a reasonable tradeoff for the paper's focus on scalability and efficiency. I will keep my overall rating at 4. If other reviewers are similarly satisfied with the clarifications, please consider this a vote to accept. I have also increased the clarity score to 4 based on the improved understanding after the rebuttal.", "rebuttal": "> **Reviewer’s Comment**:  The frozen LLM backbone may not adapt to domain-specific nuances …\n\n\n**Response**: We thank the reviewer for this insightful comment. We address both concerns below:\n- **Frozen LLM Backbone and Domain-Specific Nuances**: We appreciate the reviewer's perspective and would like to clarify that the frozen backbone is a  **deliberate design choice** that offers several advantages for domain adaptation:\n  - While the LLM backbone remains frozen, domain-specific adaptation occurs through lightweight learnable components: spatio-temporal embeddings capture location-specific patterns, temporal tokenization models region-specific daily/weekly rhythms, and learnable projection layers map domain-specific features into the LLM's representation space. The frozen LLM serves as a powerful representation extractor that processes these domain-adapted features through its deep reasoning capabilities, which has been explored in [1, 2].\n  - Our experiments demonstrate successful adaptation across three distinct cities. The model achieves consistent performance improvements (2.4-2.7% over best baselines), despite their different urban characteristics and transit patterns. This validates that our architecture effectively captures regional behavioral differences without modifying the LLM backbone.\n  - From a theoretical perspective, as noted in Section 3.4, frozen LLMs provide guaranteed convergence properties and uniform feature distribution, ensuring stable learning of domain-specific patterns through the trainable components.\n- **Static Semantic Prompts**: Our semantic prompts are contextually rich and incorporate dynamic information. Each prompt is specifically generated based on user trajectory segments, temporal context (day-of-week, time patterns), segment-specific transitions and stay patterns, and task-specific prediction requirements. \n  - By generating these embeddings offline before training, we eliminate the need for expensive LLM forward passes during the training process. For the Kumamoto dataset, the one-time prompt generation and processing takes approximately 3 hours on a single A100 with 40G memory.\n  - We acknowledge the reviewer's suggestion about dynamic prompt learning. In our revised version, we will add a discussion on prompt tuning techniques that could potentially enhance performance by allowing prompts to adapt during training.\n\n\n> **Reviewer’s Comment**: While temporal patterns are captured hierarchically, spatial information is encoded via simple embeddings without incorporating topology…\n\n\n**Response**: We thank the reviewer for this valuable observation about spatial modeling and spatio-temporal relationships.\n \nOur current approach uses categorical location embeddings with coordinate projections rather than incorporating spatial topology or explicit relational structures. We acknowledge that including topological information and explicit spatio-temporal relationships could provide additional spatial understanding and potentially enhance performance in connectivity-aware tasks.\n\nHowever, our current design prioritizes computational efficiency and scalability, which motivated our choice of joint spatio-temporal embeddings. This design decision allows us to maintain training efficiency (24.6% faster) while still achieving state-of-the-art performance (2.4% improvement over baselines), suggesting that the LLM's reasoning capabilities partially compensate for the simplified spatial representation.\n\nWe agree that explicit modeling of spatial relationships is an important direction that we will explore in future work.\n\n> **Reviewer’s Comment**: Evaluation is confined to the Japanese-only YJMob100K dataset; results on other continents or longer temporal spans are important yet missing (longer than 1 day window).\n\n\n**Response**: We thank the reviewer for raising this important point about dataset diversity and temporal evaluation scope.\nMobility datasets are inherently limited due to privacy concerns and collection difficulties. YJMob100K [3] is one of the few publicly available large-scale mobility datasets with 75 days of data and sufficient user records, ensuring reproducibility of our results.\n\nTo address the reviewer's concern, we have conducted additional experiments on a Boston dataset with the same configuration (10k users). The results show consistent performance improvements, demonstrating that RHYTHM generalizes beyond Japanese cities.\n| Model | Boston |  |  |\n|-------|--------|--------|--------|\n|  | Acc@1 | Acc@3 | Acc@5 |\n| LSTM | 0.5156 | 0.6216 | 0.6534 |\n| DeepMove | 0.5608 | 0.6582 | 0.6874 |\n| PatchTST | 0.5494 | 0.6634 | 0.6912 |\n| iTransformer | 0.5404 | 0.6373 | 0.6575 |\n| TimeLLM | 0.5371 | 0.6323 | 0.6582 |\n| MHSA | 0.5598 | 0.6475 | 0.6868 |\n| PMT | 0.5803 | 0.6922 | 0.7248 |\n| COLA | 0.5751 | 0.6867 | 0.7190 |\n| ST-MoE-BERT | 0.5754 | 0.6804 | 0.7081 |\n| Mobility-LLM | 0.5811 | 0.7174 | 0.7540 |\n| RHYTHM-LLaMA-1B | 0.5810 | 0.7275 | 0.7621 |\n| RHYTHM-Gemma-2B | 0.5856 | 0.7300 | 0.7652 |\n| RHYTHM-LLaMA-3B | **0.6014** | **0.7375** | **0.7791** |\n\n\n\n\n\n\nRegarding longer temporal spans, we conducted experiments with 3-day prediction horizons on the Kumamoto dataset. The results indicate that RHYTHM still outperforms most of the baselines in this configuration.\n\n\n| Model | Kumamoto |  |  |\n|-------|--------|--------|--------|\n|  | Acc@1 | Acc@3 | Acc@5 |\n| LSTM | 0.2486 | 0.4724 | 0.5504 |\n| DeepMove | 0.2607 | 0.4889 | 0.5666 |\n| PatchTST | 0.2684 | 0.4970 | 0.5764 |\n| iTransformer | 0.2621 | 0.4902 | 0.5661 |\n| TimeLLM | 0.2696 | 0.4981 | 0.5759 |\n| MHSA | 0.2680 | 0.4983 | 0.5757 |\n| PMT | 0.2805 | 0.5104 | 0.5858 |\n| COLA | 0.2830 | 0.5131 | 0.5888 |\n| ST-MoE-BERT | 0.2747 | 0.5068 | 0.5817 |\n| Mobility-LLM | 0.2845 | 0.5126 | 0.5875 |\n| RHYTHM-LLaMA-1B | 0.2865 | 0.5164 | 0.5919 |\n| RHYTHM-Gemma-2B | **0.2919** | 0.5189 | **0.5925** |\n| RHYTHM-LLaMA-3B | 0.2918 | **0.5191** | **0.5925** |\n\n\n\n\n\n\n\n\n> **Reviewer’s Comment**: Predictions are made in a single shot (non-autoregressive), without iterative feedback, performance on rolling horizons remains untested.\n\n\n**Response**: We thank the reviewer for highlighting this important aspect of our prediction approach.  To address this concern, we have conducted additional experiments comparing autoregressive and non-autoregressive prediction strategies. Table below shows the performance and computational speed comparison on rolling horizons.\n| RHYTHM (Kumamoto) | Time (s/iter) | Acc@1 | Acc@3 | Acc@5 | \n|--------|-------|--------|--------|--------|\n | Non-autoregressive |  **0.96**  |  **0.2929**  | 0.5200 |  **0.5835**  |\n  | Autoregressive | 39.80 | 0.2884 |  **0.5247**  | 0.5801 |\n\n\nThe results demonstrate that our non-autoregressive approach achieves comparable performance while being **41× faster** than the autoregressive variant. This significant computational advantage makes RHYTHM particularly suitable for real-time deployment scenarios. In our revised version, we will add a detailed discussion on why our temporal tokenization with hierarchical attention provides sufficient context for accurate multi-step predictions without requiring expensive iterative feedback mechanisms.\n\n[1] Jin, Ming, et al. \"Time-llm: Time series forecasting by reprogramming large language models.\"ICLR 2024.\n\n\n[2] Liu, Yong, et al. \"Autotimes: Autoregressive time series forecasters via large language models.\" NeurIPS 2024.\n\n[3] Yabe, Takahiro, et al. \"YJMob100K: City-scale and longitudinal dataset of anonymized human mobility trajectories.\" Scientific Data 2024."}, {"review_id": "rmUEz55bKM", "reviewer": "Reviewer_UqUH", "summary": "This paper introduces RHYTHM (Reasoning with Hierarchical Temporal Tokenization for Human Mobility), a unified framework that leverages large language models (LLMs) as general-purpose spatio-temporal predictors and trajectory reasoners. Its main innovations are the introduction of hierarchical attention mechanisms and the enrichment of token representations by adding pre-computed prompt embeddings for trajectory segments and prediction targets via a frozen LLM. It improves performance and reduces training time on three datasets.", "questions": "Please refer to the Weaknesses.", "limitations": "Yes, the authors have discussed the limitations in the paper.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths：\n\n1.\tThe manuscript is well-written with clear descriptions.\n2.\tThe authors have conducted a wide range of experiments to verify the effectiveness of the proposed method. \n\nWeaknesses：\n\n1.\tNovelty is limited. The main technical design is not new and most of the modules have been investigated. For example, the temporal tokenization module has been studied in previous work (PMT, ST-MoE-BERT, etc.), and this paper simply replaces the attention mechanism for the complete sequence with a simple hierarchical attention mechanism. Another innovation, the efficient prompt-guided approach, has also been studied in many other LLM-related tasks.\n2.\tThe models used in the performance experiment and the efficiency experiment are not consistent. If the author chose LLaMA-1B as the final LLM, the results of Gemma-2B and LLaMA-3B should not be bolded in the performance experiment table, but should be presented as additional results. Otherwise, it will mislead readers to think that LLaMA-3B is the final model.\n3.\tThe efficiency experiment design is too simple. The author has been explaining that the hierarchical attention mechanism and the frozen LLM with an efficient prompt-guided approach can improve the efficiency of the model. Therefore, the author should provide detailed efficiency experiments for each part, such as the efficiency after eliminating the hierarchical attention mechanism, or the efficiency of using an unfrozen LLM. Furthermore, the two efficiency results of 12.37% and 24.6% mentioned in the Contributions part did not appear in the Experiment section.\n4.\tIn the ablation experiment, each module seems to have only slight improvements. Because temporal tokenization is not the innovation of this paper, it should not appear in the ablation experiment. The performance improvements of other parts are also very weak, which will make readers question whether the sota performance achieved in this paper comes from the stated design or a higher performance baseline. It is recommended that the author use a step-by-step ablation experiment and give the most basic baseline performance.\n5.\tFrom the method section, the number of segments N in the Temporal Tokenization module seems to be an important parameter. However, the author did not discuss it in detail or conduct ablation experiments. In the settings section, the author seems to set the segment length to 48 (1 day). If it is set to 24 or 96 (or even larger), what impact will it have on the model performance and efficiency?\n6.\tThe results in Daily and Weekly Trend Analysis show that the model is more inclined to predict certain time periods such as weekends or evening peaks, and its performance in highly regular periods is significantly lower than other models. This result conflicts with the author's statement in the introduction: the author stated that the hierarchical attention mechanism can better model the periodicity of daily or weekly activities, but for the more periodic weekday period, the model performance is worse. The author should give a reasonable explanation for this.", "was_revised": true, "final_justification": "The author's response addressed my concerns. Referring to the reviews of other reviewers, I improved my score.", "rebuttal": "> **Reviewer’s Comment**: Novelty is limited…\n\n\n**Response**: We thank the reviewer and respectfully clarify our novelty:\n- **First to bridge frozen LLMs with structured spatio-temporal data**: RHYTHM is the **first** to combine hierarchical temporal abstraction with semantic prompt injection into a frozen LLM. Unlike in-context learning [1] or fine-tuning-based methods [2], we freeze the entire LLM—including attention and FFN layers—and only train a lightweight hierarchical encoder and projection head. This design is parameter-efficient (12.37% trainable parameters) and scalable to large models.\n- **First to demonstrate LLM scaling laws in the mobility domain**: We conduct a comprehensive analysis (Table 4, Fig. 4) across multiple LLM backbones (LLaMA, DeepSeek, Gemma, …) to quantify trade-offs between model size, training time, and prediction accuracy—a **first** in human mobility modeling.\n- Efficient hierarchical architecture for multi-scale mobility dynamics: Compared to prior work, existing human mobility foundation models either rely on in-context learning [1] or require fine-tuning within the LLM backbone [2]. However, fine-tuning FFN layers incurs substantial computational cost. In contrast, we avoid modifying the LLM backbone. Instead, we enable spatio-temporal and semantic fusion by training only the hierarchical attention encoder and generation head. This approach significantly reduces resource demands while maintaining strong generative capacity over spatial-temporal data, which lies in a fundamentally different latent space than text.\n- Hierarchical temporal tokenization for multi-scale mobility modeling: While prior work (e.g., PMT, ST-MoE-BERT) employs standard attention over long sequences, RHYTHM introduces temporal tokenization that encodes daily mobility patterns as discrete tokens. These tokens are processed through explicit hierarchical attention to capture both short-term (daily) and long-term (weekly) periodic dependencies. Crucially, our design eliminates the need for full-length spatio-temporal sequences, enabling the model to efficiently and accurately reason over long mobility histories without incurring the quadratic cost of full-sequence attention. This leads to both reduced training overhead and improved scalability.\n- Empirical strength across real-world datasets: RHYTHM demonstrates strong real-world performance, achieving a 2.4% improvement in overall prediction accuracy, a 5.0% gain on weekends, and a 24.6% reduction in training time over prior state-of-the-art baselines (Table 1, Fig. 3, Fig. 5). These results are consistent across three urban mobility datasets.\nTaken together, RHYTHM provides a novel, efficient, and scalable framework for mobility prediction. It is not a repackaging of prior ideas but a careful integration that enables structured non-textual inputs to leverage the reasoning power of LLMs—without requiring costly full-model fine tuning. In our revised version, we will clarify our writing to better highlight our novel contributions.\n\n> **Reviewer’s Comment**: The models used in the performance experiment…\n\n\n**Response**: We thank the reviewer for this important clarification about model consistency and presentation. We apologize for the confusion in our presentation, and we will revise our tables accordingly. Importantly, we want to emphasize that even with LLaMA-1B model, RHYTHM achieves 1.8% improvement over the best baseline, demonstrating strong performance without requiring larger models. \n\n\n> **Reviewer’s Comment**: The efficiency experiment design is too simple…\n\n\n**Response**: We thank the reviewer for highlighting the need for more detailed efficiency experiments. We acknowledge this oversight and have conducted comprehensive efficiency analysis for each component.\n\n\nHere are the detailed efficiency measurements on the Kumamoto dataset (single A100 GPU):\n\n\n| Configuration | Trainable Params | Time/Epoch |\n|--|----|--|\n| Full RHYTHM (frozen LLM) | 152M (12.37%) | 31 min |\n| Unfrozen LLM w/ LoRA | 200M (16.27%) | 1h 42min |\n| w/o Temporal Tokenization | 148M (12.00%) | 53 min |\n| Only Attention Module (no LLM) | 152M (12.37%) | 16 min |\n| w/o HA | 39M (3.25%) | 20min |\n\n\nThese results demonstrate that each design choice contributes to efficiency. To clarify the metrics mentioned in our contributions: The 12.37% refers to trainable parameters. The 24.6% reduction in training time is calculated by comparing RHYTHM with mobility prediction baselines. In our revised version, we will add this clear explanation in the experiment section.\n\n\n\n\n\n\n> **Reviewer’s Comment**: In the ablation experiment, each module…\n\n\n**Response**: We thank the reviewer for this important feedback about our ablation study. We conducted a more comprehensive analysis to clarify each component's contribution.\n\n| Model | Acc@1 | Acc@3 | Acc@5 |\n|---|---|---|---|\n| RHYTHM | 0.2929 | 0.5200 | 0.5835 |\n| w/o HA | 0.2917 | 0.5163 | 0.5881 |\n| w/o token | 0.2801 | 0.5049 | 0.5764 |\n| w/o attention module | 0.2721 | 0.5033 | 0.5795 |\n| w/o Traj info. | 0.2914 | 0.5176 | 0.5891 |\n| w/o Task desc. | 0.2895 | 0.5166 | 0.5889 |\n| w/o Traj info + Task desc  | 0.2865 | 0.5155 | 0.5906 |\n| w/o LLM | 0.2749 | 0.4921 | 0.5623 |\n\n\nThe results reveal that while individual components show modest improvements, removing the LLM backbone causes a significant 6.2% performance drop, demonstrating that our approach's strength comes from the **synergistic integration** of LLM reasoning with mobility-specific components.  Regarding temporal tokenization, while segmentation techniques exist in other domains, our specific application to mobility prediction is novel - as illustrated in Figure 1, we segment trajectories into **semantically meaningful daily patterns** rather than arbitrary windows, enabling the model to capture cyclical human behaviors. This contributes 5.4% improvement when integrated with our LLM framework.\n\nIn our revised version, we will update the ablation section with clearer explanations of how each component contributes to the overall system performance.\n\n> **Reviewer’s Comment**: From the method section, the number of segments N…\n\n\n**Response**: We thank the reviewer for identifying this important parameter that deserves more detailed analysis. We have conducted comprehensive experiments with different segment lengths to understand their impact on both performance and efficiency.\n\n\n| Segment Length ($L$) | Segments ($N$) | Acc@1 | Acc@3 | Acc@5 | Time (s/iter) |\n|--|----|--|----|---|---|\n| $L$=1 (30 min) | 336 | 0.2801 | 0.5049 | 0.5764 | 6.59 |\n| $L$=24 (12 hours) | 14 | 0.2883 | 0.5124 | 0.5792 | 1.10 |\n| $L$=48 (1 day) | 7 | **0.2929** | **0.5200** | **0.5835** | 0.96 |\n| $L$=96 (2 days) | 3 | 0.2851 | 0.5087 | 0.5743 | 0.93 |\n\n\nThe results reveal an interesting trade-off: $L$=48 (daily segments) achieves optimal performance, aligning with natural mobility rhythms. Smaller segments ($L$=1) suffer from excessive fragmentation and computational overhead, while larger segments ($L$=96) lose important daily pattern boundaries.\n\nOur choice of $L$=48 captures complete daily cycles while maintaining computational efficiency. This validates that temporal tokenization at semantically meaningful boundaries is more effective than arbitrary segmentation.\n\nIn our revised version, we will add this ablation study and include a detailed discussion on how segment length affects the model's ability to capture mobility patterns at different temporal scales.\n\n\n> **Reviewer’s Comment**: The results in Daily and Weekly…\n\n**Response**: Thank you for raising this important point about the seemingly contradictory performance patterns. This is a crucial finding that reveals the fundamental strength of our approach. This contradiction with our claims about modeling periodicity actually reveals a deeper insight about our approach.\n\nRHYTHM's hierarchical attention mechanism captures periodicity not just as **repetitive patterns** but as **contextual decision points**. During highly regular periods, mobility is largely deterministic - people follow fixed routines with minimal variation. Traditional models like LSTM and PMT, which rely on simple pattern matching, excel here because the prediction task reduces to memorizing frequent transitions.\n\nHowever, during transitional periods (evening peaks) and flexible periods (weekends), mobility involves complex decision-making influenced by multiple factors. Here, RHYTHM's advantage emerges:\n- The LLM backbone provides reasoning capabilities to handle non-routine decisions\n- Hierarchical attention captures both local context (what happened today) and global patterns (typical weekend behavior)\n- Semantic prompts encode rich contextual information beyond simple location sequences\n\nHierarchical attention is optimized for capturing **global temporal dependencies** across days and weeks, which may actually dilute its focus on simple repetitive patterns within regular hours. RHYTHM considers broader context while powerful for complex decisions, and introduces unnecessary complexity for deterministic periods.\n\nThis explains the 5.0% improvement on weekends - RHYTHM excels when prediction requires understanding context and variability rather than just memorizing fixed patterns. During regular weekday hours, the simpler baseline models are sufficient because the task is essentially deterministic.\n\nIn our revised version, we will clarify that our \"periodicity modeling\" refers to capturing periodic variations in behavior complexity rather than just repetitive patterns, making RHYTHM particularly valuable for real-world applications where handling irregular periods is crucial for overall system reliability.\n\n\n\n\n[1] Wang, Xinglei, et al. \"Where would i go next? large language models as human mobility predictors.\" arXiv 2023.\n\n\n[2] Gong, Letian, et al. \"Mobility-llm: Learning visiting intentions and travel preference from human mobility data with large language models.\" NeurIPS 2024."}, {"review_id": "0oqtelTCCj", "reviewer": "Reviewer_Hudz", "summary": "The paper introduces a novel approach named RHYTHM, which leverages hierarchical temporal patterns for reasoning tasks, utilizing various pre-trained language models as fixed backbone networks to evaluate performance across different scales. The authors meticulously detail their model architecture and provide comprehensive information to ensure the reproducibility of their experimental results.\n\nKey contributions of the paper include:\n\n* Proposing and clearly delineating a new model architecture capable of effectively capturing complex patterns within temporal data.\n* Conducting systematic hyperparameter searches to optimize model performance, identifying optimal configurations such as ideal learning rates and weight decay values.\n* Demonstrating performance comparisons across multiple large-scale pre-trained language models ranging from OPT-125M to LLaMA-3.2-3B, providing insights into scalability and effectiveness.\n* Releasing their source code and datasets publicly, along with detailed instructions that enable other researchers to reproduce their experimental results, thus promoting openness and transparency in research.\n\nIn summary, this paper makes significant contributions to the fields of time series analysis and natural language processing by innovatively applying existing technologies and maintaining a high level of transparency regarding experimental details. These contributions serve as valuable references for further research in these domains.", "questions": "1. You said \"This design captures fine-grained spatio-temporal dynamics, deep semantic context, and leverages LLM reasoning—all while minimizing computational and memory overhead—making RHYTHM ideally suited for deployment in resource-constrained, real-world environments.\" in Page 2, Line 65. \n* Suggestion: Please include benchmarks comparing RHYTHM’s inference latency, peak memory usage, and model size against one or two representative baselines (e.g., TimeLLM or Mobility-LLM) on identical hardware. If possible, report these numbers under a constrained setting (e.g., edge GPU or CPU only).\n2. The choice of daily segments (L = 48 time slots) for temporal tokenization is justified by general human routines, but it remains unclear how sensitive RHYTHM’s performance is to this segmentation granularity.\n* Suggestion: Please provide an ablation study or sensitivity analysis that varies the segment length and reports its impact on different metrics and computational cost.\n3. The paper enriches segment tokens with pre-computed LLM prompt embeddings for both historical trajectories and future timestamps. However, the additional offline computation cost and storage requirements for these embeddings are not quantified.\n* Suggestion: Please report the time and memory overhead required to generate and store the semantic embeddings for a standard dataset (e.g., Sapporo), and compare it to the savings from freezing the LLM backbone.", "limitations": "No.\n\n**Broaden the Scope of Limitations**\n\n    While Appendix B discusses technical limitations—reliance on pretrained LLM quality, non-autoregressive prediction, and high training time—these are all efficiency or modeling concerns. The paper would benefit from explicitly acknowledging how these limitations might affect different deployment scenarios (e.g., edge devices vs. cloud servers), and the bounds within which RHYTHM remains practical (e.g., maximum sequence length, model sizes).", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "# Strengths\n## Quality\nThe paper demonstrates high-quality research through its detailed description of the model architecture, training procedures, and experimental settings. The authors have ensured that all necessary details are provided to reproduce their results, including open-sourcing the code and datasets with clear instructions (Appendix A). \n## Clarity\nThe paper is well-written and easy to follow. It provides a comprehensive explanation of the methods used, making it accessible to both experts and non-experts in the field.\n## Significance\nThe proposed approach addresses important challenges in time series analysis and natural language processing by leveraging hierarchical temporal patterns, which can have broad applications.\n## Originality\nThe use of hierarchical reasoning within a time series context represents a novel contribution to the field. This innovative application of existing technologies showcases original thinking.\n# Weaknesses\n## Quality\nWhile the reproducibility aspect is strong, the paper does not provide explicit information about the computational resources required for each experiment or an estimate of the total compute needed. Including such details would help other researchers gauge the feasibility of replicating the experiments.\n\nIn conclusion, the paper presents a solid contribution to the field with notable strengths in quality, clarity, significance, and originality. Addressing the identified weaknesses would further enhance the manuscript's impact and readability.", "was_revised": true, "final_justification": "Thank you for your reply! Your responses have sufficiently addressed my concerns.", "rebuttal": "> **Reviewer's Comment**: While the reproducibility aspect is strong, the paper does not provide explicit information about the computational…\n\n\n**Response**: We thank the reviewer for raising this important point for reproducibility. We provide comprehensive computational resource requirements below. All experiments are run on a single NVIDIA A100 GPU (40GB) with batch size 64.\n\n\n| Dataset | Users | Preprocessing Time | Training Time/Epoch | Total Time (20 epochs) |\n|---------|-------|--------------------|---------------------|------------------------|\n| Kumamoto | 3k | 3.1 h | 0.5 h  | 10.4 h |\n| Sapporo | 17k | 15.9 h | 2.9 h | 20.4 h |\n| Hiroshima | 22k | 20.1 h | 3.8 h | 22.1 h |\n\n\nFurther reproducibility details of inference, pre-computation, and deployment are discussed in the following responses.\n\n\n> **Reviewer’s Comment**: You said \"This design captures fine-grained spatio-temporal dynamics, deep semantic context, and leverages LLM reasoning…\n\n\n**Response**: We thank the reviewer for requesting concrete deployment metrics to support our efficiency claims. We have conducted comprehensive benchmarking on both GPU and CPU to demonstrate RHYTHM's suitability for resource-constrained environments.\n\n\n| Model | Model Size  | Peak GRAM | Inference Latency (GPU) | Inference Latency (CPU) | RAM |\n|-------|------------|-------------|------------------------|-------------------------|----|\n| RHYTHM |  5841.90 MB  | 5741.4 MB |261.62 ± 0.8 ms | 39.5 ± 0.9 s | 12497.92 MB |\n| TimeLLM |3762.72 MB | 11213 MB |392.74 ± 9.5 ms |64.2 ± 2.5 s | 18677.76 MB |\n| Mobility-LLM | 4880.73 MB | 9872.8 MB |192.11 ± 5.2 ms |32.3 ± 1.9 s | 10552.16 MB |\n\n\nThis benchmarking was performed on NVIDIA A100 (GPU) and Intel Xeon Gold 6338 (CPU) with 100 samples.  RHYTHM maintains a peak GPU memory (GRAM) footprint of 8.2 GB, significantly lower than TimeLLM's 22.4 GB, enabling deployment on more modest hardware. All models show expected slowdowns on CPU, but RHYTHM's 39.5s latency remains practical for batch processing scenarios. We will expand our discussion on practical deployment considerations in resource-constrained environments in the revised manuscript.\n\n\n\n\n> **Reviewer’s Comment**: The choice of daily segments (L = 48 time slots) for temporal tokenization is justified by general human routines…\n\n\n**Response**: We thank the reviewer for raising this important question about the sensitivity of our temporal tokenization to segmentation granularity. We have conducted a comprehensive sensitivity analysis varying the segment length to understand its impact on performance and computational cost.\n\n\n| Segment Length ($L$) | Segments ($N$) | Acc@1 | Acc@3 | Acc@5 | Time (s/iter) |\n|----------|---------|--------|--------|--------|------------|\n| $L$=1 (30 min) | 336 | 0.2801 | 0.5049 | 0.5764 | 6.59 |\n| $L$=24 (12 hours) | 14 | 0.2883 | 0.5124 | 0.5792 | 1.10 |\n| $L$=48 (1 day) | 7 | **0.2929** | **0.5200** | **0.5835** | 0.96 |\n| $L$=96 (2 days) | 3 | 0.2851 | 0.5087 | 0.5743 | 0.93 |\n\nOur sensitivity analysis reveals that RHYTHM's performance peaks at $L$=48 (daily segments), with a 4.5% improvement over the finest granularity ($L$=1) and 2.7% over coarser segmentation ($L$=96). The computational cost scales dramatically with smaller segments due to quadratic attention complexity - $L$=1 requires 6.9× more computation than $L$=48.\n\n\nThis empirical validation confirms that daily segmentation is not just theoretically motivated by human routines but also optimal in practice. The performance degradation at other granularities demonstrates that our temporal tokenization effectively captures the natural periodicity of human mobility patterns.\n\n\nIn our revised version, we will include this sensitivity analysis in Section 4.2, providing deeper insights into how segmentation granularity affects the balance between capturing fine-grained patterns and computational efficiency.\n\n\n\n\n> **Reviewer’s Comment**: The paper enriches segment tokens with pre-computed LLM prompt embeddings for both historical trajectories and future timestamps…\n\n**Response**: We thank the reviewer for highlighting the need to quantify the offline computation and storage costs of our semantic embeddings. We have conducted detailed measurements across all three datasets.\n\n\n| Dataset | Users | Days | Offline Generation Time | Storage Size |\n|---------|-------|------|------------------------|--------------|\n| Kumamoto | 3k | 75 | 3.1 h | 1.6 GB |\n| Sapporo | 17k | 75 | 15.9 h | 9.1 GB | \n| Hiroshima | 22k | 75 | 20.1 h | 11.8 GB |\n\n\nThese measurements were performed on a single NVIDIA A100 GPU (40GB memory) with batch size 128. The memory overhead is 31.24 GB. The storage requirements include both trajectory and task embeddings. \n\n\nThe generation time scales linearly with user count, and storage remains manageable even for 22k users. This one-time preprocessing eliminates redundant LLM inference during training, reducing epoch time from hours to minutes while enabling deployment on resource-constrained hardware without LLM infrastructure. We will add a dedicated section on these costs in the revised manuscript.\n\n\n\n\n> **Reviewer’s Comment**: While Appendix B discusses technical limitations—reliance on pretrained LLM quality …\n\n\n**Response**: We thank the reviewer for encouraging us to provide more concrete deployment guidance. \n\n\nWe summarize the training setup and performance of RHYTHM across different devices and sequence lengths in the tables below.\n\n\n\n\n| GRAM     | Training Time per Epoch (GPU) | Training Time per Epoch (CPU) | RAM        | Device  |\n|----------|-------------------------------|-------------------------------|------------|---------|\n| 8202 MB  | 26.5 min                      | -                             | 24995.84 MB| 2080 Ti |\n| -        | -                             | 2h 52min                      | 57896.56 MB| CPU     |\n\n\n\n\n\n\n\n\n| Sequence Length | GRAM        | Training Time per Epoch (GPU) |\n|-----------------|-------------|-------------------------------|\n| 48              | 7126.3 MB   | 23.1 min                      |\n| 168             | 7469.5 MB   | 24.6 min                      |\n| 336             | 8202 MB     | 26.5 min                      |\n| 672             | 9826.4 MB   | 30.9 min                      |\n\n\n\n\n\n\nAs demonstrated in our previous response to question 1, RHYTHM can run on CPU-only devices (**39.5s inference**), making it viable for edge deployment where real-time constraints are relaxed. For resource-constrained scenarios like a single RTX 2080, we have verified that using batch size 8 with up to 75 days of history is feasible to fit within 8GB memory constraints. Due to our temporal tokenization strategy, RHYTHM handles long sequences with less memory and faster speed than previous methods.\n\n\nOur frozen LLM design and temporal tokenization strategy makes RHYTHM practical across diverse deployment scenarios from edge devices to cloud servers. We will expand Appendix B with deployment guidelines in our revised version.\n\n\n> **Reviewer’s Comment**: In Section 4.2, the text refers to an ablation result “as shown in ??,”...\n\n\n**Response**: Thanks for catching this! This is a LaTeX \\ref typo that has already been fixed in the manuscript. All \"??\" placeholders will be replaced with the correct Table/Figure numbers."}, {"review_id": "XOswq5OEXX", "reviewer": "Reviewer_Gba1", "summary": "The specific long-term dependence and multi-scale spatiotemporal patterns of human mobility make the trajectory \nprediction of human mobility difficult. To this end, the authors proposed a new method called RHYTHM, which includes \nTemporal Tokenization, Prompt-Guided Embedding and parameter-efficient adaptation strategy. \nRHYTHM has obvious indications in the three real-world mobility datasets and has reached a new SOTA level.", "questions": "Questions:\n1.None of the formulas in the article are numbered. Is this an operational error by the author?\n2.The third point of innovation is not very distinct.\n3.The descriptions in some places are not clear enough.\n4.There is?? In line 317.\n\nIf the author's answer can convince me, I am willing to improve my score.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1.The method builds embeddings at a more fine-grained level, that is, from day-of-week or hour-of-day, \nachieving better performance and reducing the computational load.\n2.The introduction of LLM enables the model to have more powerful performance.\n\nWeaknesses:\n1.What does \"categorical location\" mean? An example can be given to illustrate.\n\n2.The description of dividing into segments by day-of-week or hour-of-day is not clear enough there. \nAs can be seen from Figure 2, the input Day 1,... Day n. But what L introduced in line 135 is in units \nof day-of-week or hour-of-day?. If it is a mixture of the two, how is it mixed?\n\n3.In line 290, it is suggested that to compare the computing speed of the attention mechanism more fairly, \nRHYTHM can remove the computing part of the LLM and only calculate the processing time of the attention \nmechanism (because the processing of the LLM itself takes time). This can better demonstrate the computational \nadvantages compared with previous sequence processing methods such as PatchTST and iTransformer.\n\n4.There is a very confusing aspect of the innovation point. \nThe author claims to have proposed the parameter-efficient adaptation strategy, \nbut this point is not mentioned in the method, and the LLM is also frozen and will not be trained. \nSo I don't know where this innovative point lies. If only a fully connected layer is added for training after the LLM, \nrather than the lora fine-tuning of the LLM itself, then this innovative point should not be very prominent.\n\n5.Generally, the prior feature extraction ability of LLM is utilized and should be verified in the generalization experiment, \nwhich can better reflect the significance of LLM.\n\n6.Ablation experiments can be conducted for more detailed verification. \nFor instance, only \"day-of-week\" or \"hour-of-day\" is encoded in terms of time, or either the timestep encoding or the spatial information encoding is excluded.", "was_revised": true, "final_justification": "The author has addressed my doubts very well. However, in terms of methodology, they merely applied techniques and methods commonly used in most fields, that is, extracting pre-trained features with a frozen LLM. Therefore, I only raised the rating to 4.", "rebuttal": "> **Reviewer’s Comment**: What does \"categorical location\" mean? An example can be given to illustrate.\n\n\n**Response**: We thank the reviewer for requesting clarification on \"categorical location.\" \n\nIn our approach, each location is treated as a discrete category. In the YJMob100K dataset [1], the city is divided into a 200×200 grid, where each grid cell is a unique location category. Each location becomes a categorical ID that is then encoded using one-hot encoding. This allows the model to learn location-specific patterns efficiently.\n\nIn our revised version, we will add a clearer explanation in Section 3.2.\n\n\n> **Reviewer’s Comment**: The description of dividing into segments by day-of-week or hour-of-day is not clear enough there…\n\n\n**Response**:  We thank the reviewer for pointing out this in our segment description.\n\nTo clarify, $L$ represents the number of time steps (records) in each segment, not day-of-week or hour-of-day units. In the YJMob100K dataset, each record captures location every 30 minutes, so for our 1-day segments, $L$=48 (48 half-hour intervals = 24 hours).\n\nThe segmentation process works as follows: Each segment $s_i$ contains $L$=48 consecutive time steps representing one full day. For example, Day 1 segment contains records from 00:00-23 :30 (48 records), Day 2 segment contains records from 00:00-23:30 of the next day (48 records), and so on. $L$ is a tunable hyperparameter that users can adjust to match their desired segment length, similar to standard practice in time series forecasting.\n\nThe day-of-week and time-of-day mentioned in our temporal embeddings (line 117-119) are attributes within each timestamp, not segmentation units. Each of the 48 records within a segment has its own temporal embedding encoding both time-of-day (which hour) and day-of-week (Monday-Sunday).\n\nIn our revised version, we will clarify this to avoid any confusion about the segmentation granularity.\n\n\n> **Reviewer’s Comment**: In line 290, it is suggested that to compare the computing speed of the attention mechanism more fairly…\n\n\n**Response**: We thank the reviewer for this valuable suggestion about comparing computational speed more fairly.\n\n\nWe have conducted additional experiments comparing RHYTHM with and without the LLM backbone to isolate the computational impact of our attention mechanism on the Kumamoto dataset. The table below shows the detailed timing comparison:\n\n\n| Model Configuration | Time per Iteration (s) | Training Time per Epoch (s) | Acc@1 |\n|---------------------|------------------------|----------------------------|--------|\n| iTransformer | 0.51 | 1084| 0.2609 |\n| PatchTST | 0.42 | 912| 0.2751 |\n| RHYTHM w/o LLM (attention only) | 0.42 | 924 | 0.2749 |\n| RHYTHM w/ frozen LLM (ours) | 0.96 | 1828| 0.2929 |\n\n\nThe results show that while incorporating the frozen LLM increases computation time by approximately 2.3×, it provides significant performance gains (+6.5% Acc@1). Notably, RHYTHM without LLM achieves comparable performance to PatchTST with similar computational cost, while outperforming iTransformer despite iTransformer’s higher computational overhead. The attention mechanism itself is highly efficient, demonstrating that our temporal tokenization design successfully reduces computational complexity as intended. The additional time from the frozen LLM is justified by the substantial accuracy improvements and is still more efficient than alternative LLM-based approaches. \n\n\nIn our revised version, we will include this detailed timing analysis to provide a more comprehensive view of the computational trade-offs in our design.\n\n\n> **Reviewer’s Comment**: There is a very confusing aspect of the innovation point… The third point of innovation is not very distinct.\n\n\n\n**Response**: Thank you for the opportunity to clarify. \n\n\nThe novelty of our parameter-efficient fine-tuning lies in fully freezing the LLM backbone—including attention and FFN layers—while training only the hierarchical attention encoder and output head. Prior human mobility prediction models typically train from scratch, which is computationally expensive. Beyond this, our key contribution is enabling spatial-temporal data to interface with frozen LLMs. Since the latent space of such data differs fundamentally from textual inputs, we introduce hierarchical attention and prompt-guided semantic embeddings to bridge the modality gap. Unlike prior methods that use in-context learning [2, 3] or unfreeze the LLM [4], RHYTHM is the **first** to support structured spatial-temporal representation with frozen LLMs efficiently with less computational resources needed. Our novel temporal tokenization further reduces training and deployment costs by shortening sequence length while preserving temporal dependencies.\n\n\nWe will revise Section 3 to more clearly articulate these technical innovations and their distinctions from existing approaches.\n\n\n\n\n\n\n\n\n> **Reviewer’s Comment**: Generally, the prior feature extraction ability of LLM is utilized and should be verified in the generalization experiment, which can better reflect the significance of LLM.\n\n\n**Response**: We thank the reviewer for this important question about verifying the LLM's generalization capabilities.\nTo demonstrate that the pretrained LLM's feature extraction abilities specifically benefit mobility prediction, we conducted the following experiments:\n\n\n| Model Variant | Acc@1 | Acc@3 | Acc@5 |\n|---------------|--------|--------|--------|\n| RHYTHM w/ pretrained LLM (ours) | 0.2929 | 0.5200 | 0.5835 |\n| RHYTHM w/ randomly initialized LLM (frozen) | 0.2556 | 0.4842 | 0.5313 |\n| RHYTHM w/o LLM (only attention) | 0.2749 | 0.4921 | 0.5623 |\n\n\n\n\nThese results clearly show that the pretrained LLM's prior knowledge is crucial - using a randomly initialized LLM performs worse than even our attention-only baseline. This verifies that the LLM's pretraining on diverse text enables it to extract meaningful spatio-temporal patterns, validating the significance of leveraging pretrained LLMs for mobility prediction.\n\n\n> **Reviewer’s Comment**: Ablation experiments can be conducted for more detailed verification.\n \n**Response**:  We thank the reviewer for suggesting more detailed ablation experiments. We have conducted comprehensive ablations on individual components of our spatio-temporal embeddings.\n\n\n| Model | Acc@1 | Acc@3 | Acc@5 |\n|------------------|--------|--------|--------|\n| RHYTHM | 0.2929 | 0.5200 | 0.5835 |\n| w/o Time-of-Day | 0.2847 | 0.5089 | 0.5743 |\n| w/o Day-of-Week | 0.2881 | 0.5142 | 0.5791 |\n| w/o Both Temporal | 0.2802 | 0.5021 | 0.5698 |\n| w/o Spatial Embedding | 0.2447 | 0.4697 | 0.5409 |\n\n\nThe results show that spatial information is most critical, while temporal embeddings provide complementary benefits. Interestingly, the model still achieves reasonable performance using only sequence order. However, our explicit spatio-temporal embeddings provide significant gains.\n\n> **Reviewer’s Comment**: The descriptions in some places are not clear enough.\n\n\n**Response**: Thank you for the feedback. We will revise the manuscript to clarify the descriptions and improve overall readability in the relevant sections.\n\n\n\n> **Reviewer’s Comment**: None of the formulas in the article are numbered. Is this an operational error by the author? 2.There is?? In line 317.\n\n\n**Response**: Thank you for pointing this out. The formulas are not numbered because they are not cross-referenced elsewhere in the paper - following standard practice, we only number equations that need to be referenced later in the text. Regarding the ?? in line 317, this is a LaTeX \\ref typo. I apologize for this oversight and will ensure all reference placeholders are properly resolved in the revised version.\n\n\n[1] Yabe, Takahiro, et al. \"YJMob100K: City-scale and longitudinal dataset of anonymized human mobility trajectories.\" Scientific Data 11.1 (2024): 397.\n\n\n[2] Wang, Xinglei, et al. \"Where would i go next? large language models as human mobility predictors.\" arXiv 2023.\n\n\n[3] Beneduce, Ciro, et al. \"Large language models are zero-shot next location predictors.\" IEEE Access 2025.\n\n\n[4] Gong, Letian, et al. \"Mobility-llm: Learning visiting intentions and travel preference from human mobility data with large language models.\" NeurIPS 2024."}]

Author rebuttal:

Meta-review: The paper introduces a new approach, RHYTHM, which incorporates Temporal Tokenization, Prompt-Guided Embedding, and a parameter-efficient adaptation strategy. Experiments demonstrate superior performance compared to state-of-the-art models. Strengths include the introduction of hierarchical attention mechanisms, the enrichment of token representations, and extensive empirical evaluation. According to the reviewers’ feedback, most of the initial concerns were adequately addressed in the rebuttal. Remaining issues include the relative simplicity of the spatial modeling component and limited novelty, as some techniques applied are commonly used in related fields.
Decision: Accept (poster)