UniTraj: Learning a Universal Trajectory Foundation Model from Billion-Scale Worldwide Traces
Authors: Yuanshao Zhu
James Jianqiao Yu
Xiangyu Zhao
Xun Zhou
Liang Han
Xuetao Wei
Yuxuan Liang
Venue: NeurIPS 2025 poster ()
Keywords: Spatio-Temporal Data Mining
Foundation Model
Urban Computing
Primary area: deep_learning
TL;DR: We built the first worldwide trajectory dataset and trained a universal trajectory foundation model.

Abstract: Building a universal trajectory foundation model is a promising solution to address the limitations of existing trajectory modeling approaches, such as task specificity, regional dependency, and data sensitivity.
Despite its potential, data preparation, pre-training strategy development, and architectural design present significant challenges in constructing this model.
Therefore, we introduce **UniTraj**, a Universal Trajectory foundation model that aims to address these limitations through three key innovations.
First, we construct **WorldTrace**, an unprecedented dataset of 2.45 million trajectories with billions of GPS points spanning 70 countries, providing the diverse geographic coverage essential for region-independent modeling. 
Second, we develop novel pre-training strategies--Adaptive Trajectory Resampling and Self-supervised Trajectory Masking--that enable robust learning from heterogeneous trajectory data with varying sampling rates and quality. 
Finally, we tailor a flexible model architecture to accommodate a variety of trajectory tasks, effectively capturing complex movement patterns to support broad applicability.
Extensive experiments across multiple tasks and real-world datasets demonstrate that UniTraj consistently outperforms existing methods, exhibiting superior scalability, adaptability, and generalization, with WorldTrace serving as an ideal yet non-exclusive training resource. 
The implementation codes and full dataset are available at https://github.com/Yasoz/UniTraj.

UniTraj: Learning a Universal Trajectory Foundation Model from Billion-Scale Worldwide Traces
Yuanshao 223 James Jianqiao Yu?! Xiangyu Zhao Xun Zhou',
Liang Han', Xuetao Wei', Yuxuan Liang".
Southern University of Science and Technology, City University of Hong Kong
The Hong Kong University of Science and Technology (Guangzhou)
Harbin Institute of Technology, Shenzhen
yuanshao@ieee.co org, jqyu@ieee .rg. xianzhao@cityu. edu.
zhouxun2023@hit. edu. en, han liangghit edu. cn weixt@sustech edu. en, yuxliang@outlook com
Abstract
Building universal trajectory foundation model is a promising solution address the limitations of existing trajectory modeling approaches, such as task specificity, regional dependency, and data sensitivity. Despite its potential, data preparation, pre-training strategy development, and architectural design present significant chal- lenges in constructing this model. Therefore, introduce UniTraj. Universal Trajectory foundation model that aims to address these limitations through three key innovations. First, construct WorldTrace, unprecedented dataset of 2.45 million trajectories with billions of GPS points spanning 70 countries, providing the diverse geographic coverage essential for region-independent modeling. Sec- ond, we develop novel pre-training Brategies-Adaptiv Trajectory Resampling and Self-supervised Trajectory Masking-that enable robust learning from het- erogencous trajectory data with varying sampling rates and quality. Finally, we tailor flexible model architecture accommodate variety of trajectory tasks, effectively capturing complex movement patterns to support broad applicability. Extensive experiments across multiple tasks and real-worke datasets demonstrate that UniTraj consistently outperforms existing methods, exhibiting superior scal- ability, adaptability, and generalization, with WorldTrace serving as an ideal yet non-exclusive training resource. The implementation codes and full dataset are available https://github.com coaaa//////////iirraa
Introduction
Trajectory data, as the digital footprints of human movement, is becoming fundamental data source for understanding mobility patterns and transforming urban intelligence [3]. These spatio- temporal sequences unlock critical insights across diverse applications: from optimizing transportation networks that alleviate congestion in megacities, enhancing location-based services that personalize user experiences [18, 2). powering logistics systems that determine the efficiency of global supply chains [12, 24, 36]. Despite their significance, extracting meaningful patterns (from statistical methods deep learning [25]) of trajectory data presents profound challenges due to their inherent complexity, varying lengths, irregular sampling rates, and region-specine characteristics.
As trajectory data continues expand exponentially, three critical limitations in current approaches have become increasingly apparent: (1) Task Specificity: Current approaches are typically designed
"Work done during the internship at HKUST(GZ)) Conresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Pre- Training Strategies
Worldwide
Adaptive
Classificction
(World
Trojectory Resompling
Model Traiiing
Self super-siiddddd
Adapter
Trajectory Masking
Data collecting
Recovery
Prediction
Building Trajectory
Embeddinn
Processing
Encocie-- Encocier-Dzzdder Modilling Structure
Generation
Foundation model (Un (UniTre
Universal Trajectory Modaling
Various Tosks
Overview of this work, we propose trajectory foundation model and also collect worldwide trajectory dataset. The pre-trained UniTraj can be used as a backbone while adapters are trained for different regions and tasks.
Figure 1:
for uinle-purpose applications, limiting their generalizability and requiring substantial re-engineering for new tasks. (2) Regional Dependency: Many models are developed and trained on data from specific geographic regions, making them ineffective when applied different locations with dis- tinct mobility patterns and infrastructure. (3) Data Sensitivity: Real-world trajectory data often contains noise, irregular sampling, or missing entries, making models highly sensitive data quality and necessitating extensive preprocessing, which reduces robustness. These limitations point to universal foundation model capable of operating across diverse
fundamental gap: the absence tasks, geographic regions, and data quality levels. While foundation models have revolutionized NLP [1. 7] and CV [9, 13] by providing versatile, pre-trained architectures that generalize across domains, trajectory analysis has not yet benefited from this paradigm shift. Creating this model would transform trajectory intelligence from its current fragmented state to a unified approach with significantly enhanced generalization capabilities [27, 39].
However, building such a model presents key challenges: (1) Data preparation: The first challenge is to prepare a sufficiently diverse trajectory dataset that spans different geographic regions and appropriate sampling rates. Existing datasets lack sufficient geographic diversity and scale, also limited by proprictary restrictions and collection costs. This data scarcity severely hampers model generalizability and cross-regional research efforts on a global scale. (2) Pre-training Strategy: Developing robust and scalable pre-training strategies is another challenge. Real-world trajectory data exhibits heterogencous quality with noising, varying sampling rates, and missing points. Effective pre- pre-training must accommodate these inconsistencies while learning robust representations that transfer across diverse contexts. (3) Model Design: The last challenge involves selecting and tailoring effective model architecture. A universal foundation model requires architecture that balances adaptability across tasks with computational efficiency, capturing complex spatio-temporal dependencies without overfitting specific regional information or trajectory patterns.
To address these challenges, we introduce Universal Trajectory foundation model (UniTraj) sup- ported by three key innovations. As shown in Figure 1, we firstly construct WorldTrace, the first trajectory dataset with large-scale, high-quality, and global distribution, which provides the essential foundation for region-agnostic modeling. Then, we design several novel pre-training strategies- adaptive resampling and self-supervised masking that enable robust learning from heterogeneous trajectory data with varying sampling rates and quality, bridging the gap between regional varia- tions and inconsistent data. Finally, we design a flexible model architecture that captures complex spatio-temporal dependencies while adapting diverse trajectory tasks, creating versatile backbone for trajectory modeling. Collectively, UniT'aa achieves task-adaptive, region-independen. and data quality resilience, delivering scalable and efficient solution for trajectory analysis applications. In summary, our research makes the following key contributions:
We introduce WorldTrace, pioneering trajectory dataset spanning 70 countries with 2.45 million trajectories and billions of GPS points. Its unprecedented global diversity and quality overcome the limitations of existing region-specifie datasets, offering a comprehensive and open groundwork for facilitating trajectory modeling research.
We propose UniTraaj trained on WorldTrace and equipped with novel pre-training and masking strategies that effectively capture complex spatio-temporal dependencies. This model significantly

---

enhances generalizability across tasks and geographical contexts, adapts to the heterogeneity of data, and provides a scalable and efficient solution for a wide range of trajectory analysis upplications.
We demonstrated the effeectiveness of UniTra through comprehensive experiments on multiple trajectory analysis tasks. The results show significantly improved performance of zero-shot and fine-tuning settings, confirming its potential as versatile backbone for diverse trajectory modeling tasks, performing optimally when trained on diverse and high-quality datasets like WorldTrace.
2 Related Work
Trajectory Datasets. Trajectory datasets are foundational for advancing mobility research, yet existing collections vary (geographic coverage, data quality, and granularity) considerably in their utility and limitations. Well-known datasets, such as GeoLife [46], collected over five years by 182 users, has contributed significantly to fields like travel mode detection [5] and traffic flow analysis [19]. However, its limited geographic coverage and participant diversity restrict its generalizability. ehicle- focused datasets such as Porto [28], T-drive [42], and Electric Vehicle Data [34] provide valuable mobility insights but frequently cxhibit low or inconsistent sampling rates that complicate analysis. Synthetic alternatives like SynMob [49] offer uniform sampling but lack the regional diversity and quality variations essential for robust model development. Proprietary collections including GALA [8] and Grab-Posisi [15] contain high-quality data but remain largely inaccessible due to regulatory and commercial constraints. These limitations- geoogaaphe constraints, sampling irregularities, impede the development of universal trajectory models. The
and access essrritioos-collecctiiii community urgently needs comprehensive. openly accessible datasets with global coverage to advance trajectory modeling research and enable effective model generalization.
Foundation Models. The success of foundation models in natural language processing and computer vision, exemplified by BERT [7], GPT-3 [1]. and Vision Transformers [9], has demonstrated how large- scale pretraining can yield highly generalizable representations across diverse tasks. This paradigm has recently extended to time series and spatio-temporal domains, with models like TST [44], TimeFM [6]. and Moirai [38] leveraging Transformer architectures to capture temporal dependencies. In spatio-temporal prediction specifically, approaches such as UniST [43], Opencity [20], and ClimaX [29] have shown promise in traffic flow and climate modeling, respectively. However, these models often remain tailored to specific tasks or regions, limiting their broader applicability. Trajectory- specific models like TrajGDM [4], BigCity [, and TrajFM [23] address certain tasks but lack the scalability and robustness needed for cross-task or cross-region applications. While unsupervised learning approaches like MAE [13] and TimeFM [6] have proven effective for images and time series, trajectory modeling presents unique challenges that demand greater ficxibility to accommodate diverse mobility patterns, geographic contexts, and sampling characteristics without extensive task- specific modifications. To summary, there remains a pressing need for trajectory foundation models that unify multiple tasks within a single framework, providing robust, transferable representations that generalize across tasks and handle data variability while maintaining computational efficiency.
3 Preliminary
Definition 1: (Trajectory). A trajectory represents the sequential record of movement through space over time. Formally, trajectory 7 of length expressed as a sequence of continuously sampled GPS points: т = {p1. P2. .... Pr where each point p: (Ing,. lat,, t.) denotes the spatial coordinates (longitude and latitude) at timestamp t,. The sampling interval between consecutive points is defined as Al; = I; 4,-1, for i 2, ....., These intervals may be uniform within or across trajectories, or vary significantly based on data collection methods and environmental factors.
Definition 2: (Trajectory Dataset). A trajectory dataset comprises multiple trajectories, each capturing the movement of an object over time. Formally, it is given by D = (Ti, . | where (D) denotes the total number of trajectories in the dataset. These collections may vary in geographic coverage, sampling rates, and quality depending on their source and application scenario.
(Universal Trajectory Modeling). Building upon the above definitions, this study aims to develop a universal foundation model for trajectory data that can adapt to diverse tasks and geographic contexts while accommodating heterogeneous data sources. Formally, consider set
Problem Statement:
3

---

of trajectories D = (T.))== where each T; is defined as in Definition 1. The goal is:
F R",
which projects raw trajectory into d-dimensional representation This function F()mmust capture intrinsic spatio-temporal patterns within trajectories while demonstrating three key capa- bilities: (1) task adaptability across various applications including classification, prediction, and anomaly detection; (2) region independence, enabling zero-shot generalization to different geographic contexts; and (3) resilience to data quality variations, effectively handling inconsistent sampling rates, varying trajectory lengths, and noise without extensive preprocessing or task-specific re-engineering.
4 Methodology
In this section, we describe the methodology for developing UniTrajj, addressing the key challenges outlined in the introduction. Our approach is structured around answering three fundamental questions: (1) How to construct a diverse and high-quality trajectory dataset that enables cross-regional generalization? (2) How to develop robust and scalable pre-training strategies that accommodate heterogeneous trajectory data? and (3) How to design an effective model architecture that adapts across diverse trajectory tasks?
4.1 WorldTrace Dataset Construction
To address the data preparation challenge, we introduce World'racce, large-scale, globally distributed trajectory dataset specifically designed support universal trajectory modeling. Below, we intro- duce our data acquisition process, preprocessing pipeline, and key dataset statistics, demonstrating WorldTrace's suitability as foundation for developing robust and generalizable trajectory foundation models. Detailed information processing, analysis, and copyright can be found in Appendix A. The full dataset is available on the Hugging Face and ModelScope" platforms.
Data Acquisition. We sourced raw trajectory data from OpenStreetMap (OSM) GPS traces [30], fo- cusing on contributions uploaded between 2021- 2023 and tagged for motorized movement en- sure data currency and relevance. This approach minimizes device heterogeneity and outdated data impacts. All collected data is stored in the standard- ized GPX format (an XML schema), containing latitude, longitude, timestamps, and optional meta- data, providing a uniform structure that simplifies parsing and preprocessing. During acquisition, implemented preliminary filtering to exclude trajec- tories with obvious anomalies such coordinates outside valid ranges or duplicate entries.
Table 1: Summary statistics of WorldTrace.
Statistic
Value
Number of Trajectories
2.45 Million
Total Raw Points
8.8 Billion
Geographical Covered
70 Countries 1 sec (normalized) 08/2021 12/2023
Sampling Interval
Time Span
Avg. Duration
6 min
Avg. Distance
5.73 km
Avg. Speed
48.0 1kkm/// /h
Data Preprocessing. Our preprocessing pipeline balances preserving authentic movement patterns with removing noise and inconsistencies, which includes the following steps:
1. Normalization:
The original data had a high sampling frequency of up to 10 Hz, causing redundancy and increased storage demands. We therefore resampled trajectories to a uniform rate of one point per second (1 Hz), preserving essential motion details while reducing data size. In addition, by standardizing trajectories to 1s/point, we can perform better resampling during subsequent model training accommodate frequency inconsist issues.
2. Filtering: We discarded trajectories with fewer than 32 points covering distances below 100 meters, as such short trips often lack meaningful patterns and introduce noise. Following estab- lished practices [5], also removed trajectories containing implausible speeds (c.g.. exceeding 120 km/h), typically caused by GPS errors or anomalies. We also apply distance- and loop-based outlier detection to identify and remove trajectories that deviate markedly from the expected path.

---

3. Calibration: Given that GPS signals can suffer from errors due to building obstructions, multipath effects, and receiver noise [14], we applied map-matching techniques [40] align raw GPS points with underlying road networks. This calibration step is common practice in trajectory data processing and is widely used in data collection and related research to correct positioning errors [8, 37], improve spatial accuracy, and make trajectory analysis more reliable.
Data Analysis and Statisties. After acquiring and preprocessing the raw trajectory data, we conducted an in-depth analysis to examine the characteristics and quality of the WorldTrace dataset. Table summarizes key statistics of WorldTrace. Overall, the dataset contains approximately 2.45 million trajectories and 8.8 billion continents. The data spans August 2021 to December 2023, with an average trajectory duration of about six minutes (with normalized and an average speed of 48.0 km/h. The number of points per trajectory ranges from 32 to more than 600, averaging around 358 points. Collectively, these attributes confirm WorldTrace's 's suitability for developing universal trajectory models that can address varied spatiotemporal patterns and broad geographical contexts.
GPS points, covering 70 countries across all inhabited
I-second sampling interval), an average distance of 5.73 km,
4.2 Pre-Training Strategies
Having established diverse trajectory dataset, we develop robust pre-training strategies to learn robust and transferable spatio-temporal representations. Rather than relying on task-specific supervi- sion, we leverage unannotated trajectory data to capture both local and global movement patterns. To address the heterogeneous data quality challenges (varying sampling rates, differing lengths, and missing points) posed by real-world trajectory, we propose two strategies tailored specifically for trajectory: Adaptive Trajectory Resampling and Self-supervised Trajectory Masking. Due to space limitations, more details and analysis about pre-training strategies can be found in Appendix B.
Adaptive Trajectory Resampling (ATR). Real-world trajectory data often exhibits inconsistent sampling intervals and lengths due to diverse collection standards, device capabilities, and user behaviors. Such discrepancies challenge model generalization, as features learned under one sampling regime may not transfer to another. Inspired by common practice of multi-scale representation learning, ATR strategy addresses these issues through two complementary components:
Dynamic Multi-Scale Resampling. This approach dynamically adjusts sampling frequency based on trajectory length, ensuring shorter trajectories retain fne-grained detail while longer ones are efficiently compressed. Specifically, we design logarithmic resampling function R(n) to implement this strategy:
In(n R'mia 1)
R(n) = Rmin + Rmin)
(2)
In Amax 70 + 1)
where Timin and Dmaax define thresholds for trajectory lengths considered "short" or "long", and Remin is the minimum sampling ratio. This logarithmic function creates smooth transition in sampling density (Amin < n < Rmax). providing three key benefits: (1) preserving critical motion patterns across trajectory lengths, (2) reducing overfitting by limiting redundancy in densely sampled data, and (3) exposing the model to diverse temporal resolutions during training.
Interval Consistent Resampling. This component focuses on the sampling rate, imposing a uniform time interval At between consecutive points within each track:
T' = (Pk, ky 1 + (j - 1) 1,2, 1.2
(3)
By ensuring consistent spacing, this approach simplifies downstream modeling by creating regular temporal structures that make demendependent patterns easier to learn, while mitigating complica- tions from missing data or irregular sampling.
Combining these approaches, ATR enables models to learn representations that generalize across varying sampling rates and trajectory lengths (analysis presented in Appendix B.1), which is critical capability for universal trajectory modeling.
Sepe-iupe Trajectory Masking (STM). Trajectory data is often incomplete irregular due to device limitations, communication failures, and environmental factors. Motivated by masked auto-encoding methods from visual and language models, we introduce tailored self-supervised
5

---

trajectory masking strategy, in which part of the input trajectory is hidden, forcing the model to infer local and global dependencies. Given a resampled trajectory iP..Pa..... pp. we define a masking function MMT', r) that replaces fraction of points with [MASK] tokens:
F= (M (pi, ( (eT.
where I I 21, n and r = / To comprehensively address different data incompleteness scenarios, (see Appendix B.2 for details) we employ four complementary masking strategies:
Uniformly samples points to mask (T ~ Uniform( (1, .,, n))),, forcing the model to infer both short-range and long-range dependencies. By foreing the reconstruction of randomly omitted points, the approach enhances the model's ability to generalize to diverse gaps.
Random Masking:
Block Masking: Conceals consecutive points (TNlckk = {Λr k + 1,..., k + b - 1}) to simulate sensor failures, encouraging reconstruction of continuous segments. This approach prompts the model to utilize surrounding context for reconstructing entire missing segments, encouraging it to capture longer-range dependencies.
Key Points Masking: Identifies and masks critical turning points using the Ramer-Douglas Peucker algorithm [10]: key = {P& | dmaa(PPe P1Pr) > E} (dmax is the maximum perpendicular distance between point Pı: and line PiPn, is the threshold). This focuses learning on structurally significant points (sharp turns or notable speed changes) that define the trajectory's 's shape.
Last N Masking:
Masks final trajectory points (I (n N + 1, n N + +2...., .n}). This setting emulates real-world forecasting tasks where future data is unavailable and must be inferred from historical observations, making it particularly effective for prediction scenarios.
4.3 Universal Trajectory Modeling
To effectively leverage the diverse trajectory data and robust pre-training strategies described above, we need to design a model architecture that can capture local and global patterns while freeing itself from regional and task-specific constraints. Our motivation for adopting this structure design is as follows: (1) We need an architecture that can be generalized to a wide range of tasks without extensive restructuring, Therefore, we adopted minimal trajectory data information (latitude, longitude, and timestamp) and ignored other region- bound information such as POI and geographical context. (2) This structure uses the reconstruction of missing points in partial observations inherit the masking strategy introduced earlier. (3) The separation of encoding and decoding enables flexible application various downstream tasks through transfer learning or fine-tuning. More details about the architecture and parameters can be found in Appendix C.
proxy task and can
Building Trajectory Embedding. Effective trajectory modeling requires transforming raw spatial and temporal data into structured embeddings that capture both local and global movement patterns. To ensure the generality of the model, we only use the latitude, longitude, and time information of the trajectory, and embed the spatial and temporal components separately to form a unified representation. For the spatial component, we normalize trajectory and map them into d-dimensional space using a ID convolutional, yielding a spatial embedding h: Similarly, the temporal component, based on the time intervals At, embedded into the same d-dimensional space via a linear layer, resulting in temporal embedding h.. This decoupled design enables the model to effectively learn relative movement and temporal dependencies, and also cope with situations where one component may be absent. Beyond point-wise embedding, modeling the relationships between trajectory points is critical for understanding movement patterns. We adopt Rotary Position Encoding (RoPE) [32], which applies rotational transformations in the embedding space. The advantage of RoPE is its ability to preserve relative positional relationships while allowing for flexible encoding of spatial-tempordl patterns across varying trajectory scales.
Adaptive Representation Learning. Based on the trajectory embeddings, we use a encoder-decoder architecture with RoPE-enhanced attention mechanism to adaptively learn a general representation of trajectories. The encoder processes the visible points in a trajectory those that are unmasked during training. Given a masked trajectory + = {pi..... [MASK]EE embedding representations of the unmasked points = (h,, , h2...... hm)) (where m S n and & I) through the embedding steps. The encoder, denoted as Eg. processes these visible embeddings to generate latent representations: zen: = Eg (H). The decoder reconstructs masked trajectory points based on the latent embeddings produced by the encoder. It receives the visible embeddings and mask
Pr. }, we first extract the

---

Table 2: Performance comparison of Tri with trajectory recovery tasks. The results are reported in MAE
and RMSE with meters. Bold denotes the best results and underlinc denotes the sccond- breond-bestto results.
WorldTrace
Chengdo
Xi'an
Gf
Grab-Posisi
Porto
Methods
MAE
RMSE
MAE
RMSE
MAR
RMSF
MAF
RMSE
MAE
RMSE
MAF
RVSE
Linear
427.68
516.15
205.74
258.52
176.49
220.87
196.85
249.76
507.41
617.28
396.61
482.39
DITTR
220.35
302.47
75.19
98.68
62.85
83.43
80.04
168.25
351.20
415.16
194.37
232.59
Transformer
130.82
147.62
55.23
51.96
94.68
113.77
136.58
104.36
126.96
DeepMove
51.16
62.29
29.32
39.02
27.31
35.67
86.38
107.78
126.9
168.07
136.66
174.96
TrajBERT
70.14
34.53
43.24
112.68
TrajPM
47.64
58.92
19.10
25.09
18.86
24.13
59.34
54.24
107.64
130.69
21.15
92.96
UniTTaa (zero shot)
10:23
13.56
11.98
20.94
8.93
13,83
37.21
63.89
114,07
167.01
78.28
100.14
Improvement(())
UniTraj (fine tume)
6.94
9.67
6.92
10.41
6.50
9.93
23.23
34.70
48.95
69.23
60.18
79.76
Improvement((%)
183.09
tokens, which are initialized as learnable vectors representing missing positions. The full sequence is created by merging the encoded visible embeddings with the mask tokens, preserving the original structure of the trajectory:
(f
2; = Zenc.j if (adeex(j): i I ( [MASK]
dde Reorder
(5)
if E
where Zenc.j corresponds to the j-th encoder output. The decoder then processes the reordered The model is trained to
sequence to predict the missing trajectory points: f = Linear(D&s24 minimize the reconstruction loss between the predicted and original points the masked positions:
Ellfo. - -
(6)
KET
where (F) represents the encoder-decoder network, and i refers to the masked positions.
5
Experiments
5.1
Experimental Setups
Datasets.
We evaluate UniTraj on six diverse real-world trajectory datasets representing different col- lection scenarios, quality levels, motion patterns, and geographic regions. These include WorldTrace, Chengdu, Xi an, GeoLife, Grab-Posisi, and Porto. Detailed summary are provided in Appendix D.1.
5.2 Task Applicability Analysis
We explore the applicability and generalizability of UniTraj various data and downstream tasks, e.g., trajectory recovery, prediction, classification, and generation tasks. Due to space constraints, we provide the detailed setup and the results of generation task in Appendix D.2. It important to clarify that our work aims to develop a general-purposs trajectory foundation model that generalizes across diverse geographic regions without region-speciff dependencies, validating its effectiveness as a backbone supporting real-world trajectory applications across geographical contexts. Existing trajectory representation learning methods inherently rely on region- eggion-bound information (POIs, road networks, 16,, 22, 26, 48], which contradicts our initial goal of region-independen. modeling. Uniirraj extracts meaningful representations solely from trajectory points without requiring auxiliary geographic context. Therefore, we deliberately excluded these methods from our baseline comparison as their architectural dependency on regional knowledge fundamentally diverges from our objective of developing globally deployable model.
Trajectory Recovery. Table 2 presents comprehensive comparison of UniTraj against established baselines across six datasets, revealing patterns that illuminate fundamental capabilities in trajectory reconstruction. The performance disparity between UniTraj and previous methods is particularly pronounced in geographically diverse and quality-variabe datasets, where it demonstrates substantial resilience to regional variations. In the zero-shot setting, UniTraa achieves remarkable results, confirming it effectively captures transferable spatio-temporal patterns without requiring additional fine-tuning. The performance difference becomes particularly instructive when analyzing low-quality datascts like GeoLife and Grab-Posish with their highly irregular sampling intervals and multiple
7

---

travel modes. It demonstrate the effectiveness of our adaptive resampling strategy in handling temporal heterogeneity. The Chengdu and Xi'an datasets reveal another critical aspect of UniTraj's capabilities, models trained on high-quality data exhibit reliable transferability and achieve optimal results even in zero-shot scenarios. When fine-tuned, UniTraj achieves the lowest error scores across all datasets, demonstrating UniTraa''s superior generalizability across diverse geographic regions. For instance, on GeoLife, UniTraj's 's fine-tuned performance (MAE 23.23) reduces error by 32.73% compared to TrajBERT, showcasing its effectiveness with complex travel patterns and lower-quality data. These results validate WorldTrace's ' potential as a foundation dataset and UniTrajj's s consistent superiority in trajectory recovery tasks, with substantial improvements through fine-tuning reinforcing its udaptability and robustness.
Trajectory Prediction. Table 3 shows Uni- Traj's exceptional performance in trajec- tory prediction, different task requiring forward inference rather than reconstruc- tion. The zero-shot results merit particular attention, as they represent the most chal- lenging scenario for trajectory models. On WorldTrace, UniTraj's zero-shot MAE sig- nificantly outperforms all baselines, under- scoring the model's 's versatility in capturing universal motion patterns. When fine-tuned, the performance further improves, consis- tently achieving the best results across all evaluated datasets. This generalization capability stems from our Last-N masking strategy, which explicitly shapes the embedding space support predictive inference. These results further confirm that Tra not only generalizes remarkably well across diverse datasets but also benefits consid- erably from fine-tuning, making highly adaptable for real-world applications requiring accurate trajectory predictions.
Table 3: Performance comparison of UniTraj with trajec- tory prediction tasks.
Tracd
Chengdu
GeoLiff
Methodk
MAE
RMSE
MAE
RMSE
MAE
RMSE
Linexr
153.12
159.65
156.85
164.58
189.02
201.34
DETR
146.48
151.63
123,47
129.73
180.32
187.59
Transfomme
114.25
117.07
67.38
70.86
165.02
170.84
DeepMove
55.69
58.67
36.31
29.10
116.46
123.20
TriBERT
80.57
96.36
64.73
68.92
113.68
121,18
TrajFM
75:45
81.32
77,82
80.48
121.94
128.16
UniTraj shot)
49.85
133,60
Improvemer(())
UniTraj (line lune)
30.10
34,46
28.78
32,44
90.97
102.88
Improement(())
764.20
790 96
737.00
119.94
7:0
Trajectory Classification. Figure 2 presents classification accuracy results that reveal Uni- Traj's capacity learn discriminative represen- tations of movement modalities. Notably, even without fine-tuning, UniTraaj achieves 71.3% ac- curacy on GeoLife, outperforming several su- pervised baselines. This zero-shot performance demonstrates that the pre-trained representations inherently capture transportation mode signa- tures, where movement modality emerges as natural organizing principle. On the Grab-Posisi dataset, which presents additional challenges due to similar motion patterns for mixed travel modes (car and motorcycle). Tni achieves 79.3% accuracy after fine-tuning with a substantial im- provement over the best baseline. This improvement emphasizes UniTraj's ability to capture subtle kinematic signatures that differentiate travel modes with complex or similar patterns.
Goolne
Grab pesisi
Figure 2:
Performance comparison of classifica- tion task with GeoLife and Grab-posisi dataset.
5.3
Dataset Study
This section analyze the impact of dataset scale, quality, and diversity on model performance of UniTraj, particularly its gencralization capability across different data sources. We focus on two main experiments: (1) examining the effect of dataset scale and quality within WorldTrace, with varying data volumes (f0.01, 0.5, 1} millions) and a high-quality (obtained by further removing loops, staying dense trajectories) subset, and (2) assessing UniTraj'ss 's adaptability and effectiveness by training it on these datasets beyond WorldTrace, thus showing its potential as a foundation model.
Effect of Dataset Scale and Quality. Figure 3(a) illustrates the relationship between training data volume and model performance, revealing a phenomenon that goes beyond simple sealing laws. With increasing trajectory count from WorldTrace (from 0.5M to 2.45M). the MAE on the in-domain test set decreases dramatically, showing substantial improvement up to approximately 1M trajectories before beginning to exhibit diminishing returns. The above result indicates that larger datasets enable
8

---

Table 4: Ablation study different resampling and masking strategies six datasets.
World
Chengdu
XTan
GeoLite
Grab-posii
Porto
Methods
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
Who Dysamic Multi-scale resampl.
436.80
462.37
192.54
272.42
157.85
223.96
499.95
671.69
1933,25
2504 16
93.14
119.93
weo Interval Consistent resampl.
21.30
24.76
12.98
20.51
9.34
13:90
69.41
115.33
102.65
149.60
1721.12
2016:61
masking Rlock mosking UniTraj
21.98
11.10
133.65
162.57
76.51
101.18
15.36
7.16
48.50
198.41
238.58
10.22
13.56
20.94
8.93
13.83
114.07
167.01
23.28
the model to capture a wider range of spatio-temporal patterns. However, while increasing the dataset size from 1 million to 2.45 million trajectories results in better coverage, the model's s MAE slightly increases due to the introduction of more noise in the full dataset. In contrast, training on high-quality subset of I million trajectories, which includes curated, noise-free data, yields more reliable and consistent learning. This highlights the importance of both dataset scale and quality, with quality being especially crucial when data volume is limited.
Wurlffaac Changau
1--
MicrdTlac
100
MAL (m))
20
20
40
13
20
0.03 5.5
2.45
10
(uighqqelity)
(a) Training data volume.
(b) Training dataset.
(c) Number of Encoders
(d) Mask ratio (%)
Figure 3: The effect of amount of data volume, diversity dataset, and different parameter settings.
Effect of Dataset Diversity. In Figure 3(b), we compare UniTraj 's zero-shot performance when trained on WorldTrace and Chengdu (the highest quality dataset available), evaluated across multiple real-world datasets. Models trained on WorldTrace exhibit superior generalization across diverse datasets (e.g.. GeoLife and Porto), reflecting the broad geographic and contextual coverage of WorldTrace. Conversely, models trained on Chengdu perform best on datasets with similar density and travel modes, such as Xi' an. However, proprietary datasets like Chengdu, while offering high quality, not publicly available, limiting their applicability for universal tasks. These results demonstrate UniTraj's 's robustness and adaptability, validating WorldTrace as an ideal training resource for building universal trajectory foundation model. At the same time, the findings confirm that UniTraj can effectively leverage other datasets when necessary, further enhancing its versatility.
5.4 Model Study
We investigate architectural components, parameter settings, and pre-training strategies to assess sensitivity to parameter choices and the contributions of their core components.
Effect of Parameter Settings. Figure 3(c) Figure 3(d) and presents the results of our parameter sensitivity analysis, examining how the number of encoder blocks and the masking ratio influence model performance. As shown in Figure 3(c), increasing the number of encoder blocks from 2 to significantly reduces MAE, with performance plateauing beyond 8 blocks. This plateau suggests that while deeper architectures can improve model capacity, the benefits diminish without corresponding adjustments in data or hyperparameters [17]. Figure 3(d) demonstrates that masking ratio of 50% yields the best performance. Low masking ratios (e.g., 5%%-1% result in higher MAE due to insufficient training signal, while higher ratios (e.g.. 75%) lead to increased MAE from excessive information loss. A 50% masking ratio strikes a balance, providing the model with a strong training signal without sacrificing the context needed for effective trajectory reconstruction.
Ablation Study. Table 4 presents an ablation study, showing how different pre-training strategies affect UniTraj's performance across datasets. The performance varies across datasets, indicating the effectiveness and limitations of them depending on the specific data and task scenarios. Dynamic Multi-scale Resampling significantly improves performance across most datasets, especially GeoLife and Grab-Posisi, which have inconsistent sampling intervals and lower data quality. This suggests that dynamic resampling helps the model to adapt to heterogeneous dataset scenarios and to be adaptive for information preservation (see Appendix 1.1.1 for more details). The Interval Consistent Resampling has notable positive effecc on datasets with consistent sampling rates, such as Porto

---

and WorldTrace. It indicates that the integration of this strategy strategy can effectively separate the temporal sampling pattern from the region, enhances the generalization of the model to data sets with different sampling rates (analysis presented in Appendix B.1.2). Key Points Masking leads to substantial performance drops on high-quality datasets like Chengdu and Xi' an but appears to offer minimal benefits, or even slight disadvantages, for certain datasets. This finding suggests that adjusting adaptive masking strategies based on trajectory complexity, potentially applying selectively to trajectories with significant directional changes, while using alternative strategies for smoother paths. Block Masking shows significant effects on GeoLife and Porto, where it helps the model handle low sampling frequencies. However, its impact other datasets is more inconsistent, suggesting that it introduces an artificial challenge that may increase complexity in high-frequee frequency datasets. (we provide robustness analysis in Appendix B.2) Overall, the varying impact of UniTraj's 's pre-training strategies across datasets highlights its adaptability to different tasks and scenarios. While not all of them universally enhance performance, their combined
provides balanced training strategy, allowing for flexible configuration depending on specific dataset requirements. Fine-tuning further optimizes performance, ensuring stability and robustness across diverse tasks.
6 Conclusion
In this work, we presented UniTraj a universal trajectory foundation model designed overcome the task specificity, regional dependency, and data quality limitations of current approaches. UniTraj acts as a robust backbone that generalizes effectively across diverse tasks and regions. To support its development, we introduced WorldTrace, a high-quality global dataset with 2.45 million trajectories from 70 countries, offering broad geographic coverage, varied sampling rates, and open accessibility, Together, UniTraj and WorldTrace provide versatile, gegh-performing foundation for trajectory analysis, paving the new solution for more adaptable and efficient models in rrajectory-basee research. Future work will focus expanding the geographic and modal diversity of the WorldTrace dataset to better cover underrepresented regions and non-motorized travel. We also aim to enhance the UniTraa model by integrating contextual information, such road networks and points of interest, to improve its predictive accuracy and real-world applicability. Further optimizations to the model architecture and pre-training strategies will also be explored to boost performance and efficiency.
7 Acknowledgement
This work was mainly supported by the National Natural Science Foundation of China under Grant No. 62506097, No. 62402414 and No.62502404. This work is also supported by the Guangdong Basic and Applied Basic Research Foundation (No. 2025A 15 1501 1994). Tencent (CCF-Tencen) Open Fund, Tencent Rhino-Bird Focused Research Program), Didi (CCF-DiDi GAIA Collaborative Research Funds). Guangzhou Municipal Science and Technology Project (No. 2023A03J0011) and Guangzhou-HKUSTCCG Joint Funding Program (No. 2024A03J0620), Hong Kong Research Grants Research Grants Council's Collaborative Research Fund (No. C1043-24GF) Research Grants Council's General Research Fund (No. 12113255, Institute of Digital Medicine of City University of Hong Kong (No. NN..9325 Huawei (Huawei Innovation Research Program), Tencent (CCc-Tencent Open Fund, Tencent Rhino-Bird Focused Research Program), Alibaba ACCmmma Tech Kangaroo Fund No. 2024002), Ant Group (CCF-Ant Research Fund), Didi (CCF-Didi Gaia Scholars Research Fund), Kuaishou, and Bytedance.
Council's s Research Impact Fund
References
[1] T. B. Brown. Language models are few-shot learners. arXiv preprint arXiv:2005. /4165, 2020.
[2] Y. Chang, E. Tanin, G. Cong, C. S. Jensen, and J. Qi. Trajectory similarity measurement: An efficiency perspective. arXiv preprint 23Xiv:2311.00960, 2023.
[3] W. Chen, Y. Liang, Y. Zhu, Y. Chang, K. Luo, H. Wen, L. Li, Y. Yu, Q Wen, C. Chen, et al. Deep learning for trajectory data management and mining: A survey and beyond. arXiv preprint 240.114033.4111, 2024.
10

---

[4] C. Chu, H. Zhang, and F. Lu. Trajgdm: A new trajectory foundation model for simulating human mobility. In Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems, pages 1-2, 2023.
[5] S. Dabiri and K. Heaslip. Inferring transportation modes from gps trajectories using convolu- tional neural network. Transportation research part C: emerging technologies, 86:360-371, 2018.
[6] A. Das, W. Kong, R. Sen, and Y. Zhou. A decoder-only foundation model for time-series forecasting, arXiv preprint arXiv:2310.10688 2023.
[7] J. Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: 10Xxv:88000048055, 2018.
[8] Didi Chuxing. Gaia open datasets., 2018.
[9] A. Dosovitskiy, An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.1192) 2020.
[10] D. H. Douglas and T. K. Peucker. Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartogruphica: the international journal for geographic information and geovisualization.
((2)::12-122,, 1973.
[11] J. Feng, Y. Li, C. Zhang. F. Sun, F. Meng, A. Guo, and D. Jin. Deepmove: Predicting human mobility with attentional recurrent networks. In Proceedings of the 2018 world wide web conference, pages 1459-1468, 2018.
[12) C. Guo, B. Yang, J. Hu, and C. Jensen. Learning to route with sparse trajectory sets. In 2018 IEEE 34th International Conference on Data Engineering (ICDE), pages 1073-1084. IEEE, 2018.
[13] K. He, X. Chen, S. Xie, Li, P. Dollár, and R. Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000-16009, 2022.
[14] B. Hofmann-Wellenleee Wellenhof, H. Lichtenegger, and E. Wasle. CNSS-glober navigation satellite systems: GPS, GLONASS. Galileo, and more. Springer Science Business Media. 2007.
[15] X. Huang, Y. Yin, S. Lim, G. Wang, B. Hu, Varadarajan, S. Zheng, A. Bulusu, and R. Zim- mermann. Grab-posisi: An extensive real-life gps trajectory dataset in southeast asia. In Proceedings of the 3rd ACM SIGSPATIAL International Workshop on Prediction of Human Mobility, page 1-10, 2019.
[16] J. Jian, D. Pan, H. Ren, X. Jiang, C. Li, and J. Wang. Self-supervised trajectory representation learning with temporal regularities and travel semantics. In 2023 IEEE 39th international conference on data engineering (ICDE), pages 843-855. IEEE, 2023.
[17] J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Rad- ford, J. Wu, and D. Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.0836) 2020.
[18] H. Lan, J. Xie, Z. Bao, F. Li, W. Tian, F. Wang. S. Wang, and A. Zhang, Vre: versatile, robust, and economical trajectory data system. Proceedings of the VLDB Endowment, 15(12):3398 3410, 2022.
[19] L. Li, R. Jiang, Z. He, X. M. Chen, and Zhou. Trajectory data-based traffic flow studies: A revisit. Transportation Research Part C: Emerging Technologies, 1 11::225-244 2200, 2020.
[20] Z. Li, L. Xia, L. Shi, Y. Xu, D. Yin, and C. Huang. Opencity: Open spatio-temporal foundation models for traffic prediction. arXiv preprint arXiv:2408. 10269, 2024.
[21] Y. Liang, K. Ouyang, Y. Wang, Liu, H. Chen, Zhang, Y. Zheng, and R. Zimmermann. Trajformer: Efficient trajectory classification with transformers. In Proceedings of the 31st ACM International Conference on Information Knowledge Management, pages 1229-1237,
2022.
11

---

[22] Y. Lin. H. Wan, S. Guo. J. Hu, C. S. Jensen, and Y. Lin. Pre-training general trajectory embeddings with maximum multi-view entropy coding. IEEE Transactions on Knowledge and 2023.
Data Engineering, 36( 36(12):9037-99)
[23] Lin, T. Wei, Z. Zhou, H. Wen, J. Hu, S. Guo, Y. Lin, and H. Wan. Trajfm: A vehicle trajectory foundation model for region and task transferabilityy. PrXiv:2408.15255 2024.
[24] Y. Lin, Z. Zhou, Liu, H. Lv, H. Wen, T. Li, Li, C. S. Jensen, S. Guo, Y. Lin, et al. Unite: A survey and unified pipeline for pre-training trajectory embeddings. arXiv e-prints, pages arXiv 2407, 2024.
[25] M. Luca, G. Barlacchi, B. Lepri, and L. Pappalardo. A survey on deep learning for human mobility, ACM Computing Surveys (CSUR), 55(1):1-44, 2021.
[26] Z. Ma, Z. Tu, X. Chen, Y. Zhang, D. Xia, G. Zhou, Y. Chen, Y. Zheng, and J. Gong. More than routing: Joint gps and route modeling for refine trajectory representation learning. In Proceedings of the ACM Web Conference 2024, pages 3064-3075, 2024.
[27] G. Mai, W. Huang, J. Sun, S. Song, D. Mishra, N. Liu, S. Gao, T. Liu, G. Cong, Y. Hu, et al. On the opportunities and challenges of foundation models for geospatial artificial intelligence. arXiv preprint 2rXiv:2304.06798, 2023.
[28] W. K. Meghan O' O'Connell, morciraMatias. Eeml/pkdd 15: Taxi trajectory prediction (i), 2015.
[29] T. Nguyen, J. Brandstetter, A. Kapoor, J. K. Gupta, and A. Grover. Climax: A foundation model for weather and climate. arXiv preprint arXiv:2301. 2023.
[30]
OpenStreetMap Contributors. Openstreetmap. 2024.
[31] J. J., J. Yang. Y. Xiang, H. Wang, L. Li, R. Zhang, B. Tu, and X. Chen. Trajbert: Bert-based trajectory recovery with spatial-temporal refinement for implicit sparse trajectories. IEEE Transactions Mobile Computing, 2023.
[32] Su, M. Ahmed. Y. Lu, S. Pan, W. Bo, and Y. Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024.
[33] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[34] G. Wang, X. Chen, F. Zhang, Y. Wang, and D. Zhang. Experience: Understanding long- term evolving patterns of shared electric vehicle networks. In The 25th Annual international conference on mobile computing and networking, pages 1-12. 2019.
[35] Wang, N. Wu, X. Lu, W. X. Zhao, and K. Feng. Deep trajectory recovery with fine-grained calibration using kalman filter. IEEE Transactions on Knowledge and Data Engineering, 33(3):921-934, 2019.
[36] S. Wang, Z. Bao, J. S. Culpepper, and G. Cong. A survey on trajectory data management, analytics, and learning. ACM Computing Surveys (CSUR), 54(2):1-36, 2021.
[37] Z. Wang, K. Fu, and J. Ye. Learning to estimate the travel time. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, page 858-866 London, United Kingdom, 2018. Association for Computing Machinery.
[38] G. Woo, C. Liu, A. Kumar, C. Xiong, S. Savarese, and D. Sahoo. Unified training of universal time series forecasting transformers. arXiv preprint 20X 2024.
[39] H. Yan and Y. Li. Generative ai for intelligent transportation systems: Road transportation perspective. ACM Computing Surveys, 2025.
[40] C. Yang and G. Gidofalvi. Fast map matching, an algorithm integrating hidden markov model with precomputation. International Journal of Geographical Information Science, 32(3):547- 570, 2018
12

---

[41] X. Yu, J. Wang, Y. Yang, Q. Huang. and K. Qu. Bigcity: A universal spatiotemporal model for unified trajectory and traffic state data analysis. arXiv preprint arXiv:2412.00953, 2024.
[42] Yuan, Zheng, C. Zhang, W. Xie, Xie, Sun, and Y. Huang. T-drive: Driving directions based on taxi trajectories. In Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems, GIS '10, page 99-108, New York, NY, USA, 2010. Association for Computing Machinery.
[43] Y. Yuan, J. Ding, Feng, D. Jin, and Y. Li. Unist: prompt-empowered universal model for urban spatio-temporal prediction. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4095-4106, 2024.
[44] G. Zerveas, S. Jayaraman, Patel, A. Bhamidipaty, and C. Eickhoff. transformer-based framework for multivariate time series representation learning, In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 2114-2124, 2021.
[45] Zhao, A. Luo, Y. Liu, J. Xu, Z. Li, Zhuang, V. Sheng, and Zhou, Where to go next: spatio-temporal gated network for next poi recommendation. IEEE Transactions Knowledge and Data Engineering, 34())2512-2524, 2020.
[46] Y. Zheng, H. Fu, X. Xic, W.-Y. Ma, and Q. Li. Geolife GPS trajectory dataset User Guide, July 2011.
[47] Y. Zheng, L. Zhang, X. Xie, and W.-Y. Ma. Mining interesting locations and travel sequences from gps trajectories. In Proceedings of the 18th international conference on World wide web, pages 791-800, 2009.
[48] S. Zhou, S. Shang, L. Chen, C. Jensen, and Kalnis. Red: Effective trajectory representation learning with comprehensive information. arXiv preprint arXiv:2411. 2024.
[49] Y. Zhu, Y. Ye, Y. Wu, X. Zhao, and J. J. Yu. Synmob: Creating high-fidelity synthetic GPS trajectory dataset for urban mobility analysis. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
[50] Y. Zhu, J. Yu, X. Zhao, Q Liu, Y. Ye, W. Chen, Z. Zhang, X. Wei, and Y. Liang. Controltraj: Controllable trajectory generation with of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4676-4687, 2024.
diffusion model. In Proceedings
13

---

SUPPLEMENTARY MATERIAL
UNITRAJ: LEARNING UNIVERSAL TRAJECTORY FOUNDATION MODEL
FROM BILLION-SCALE WORLDWIDE TRACES
TABLE OF CONTENTS
Details of WorldTrace Dataset
15
A.I Data Collection
15
A.2 Data Processing
16
A.3 Data Statistics and Analysis...
16
A.4 Data Privacy and Copyright
17
B Pre-training Strategies
18
B.1 Adaptive Trajectory Resampling
18
B.2 Sclf-supervised Trajectory Masking
C Details of UniTraj
24
C.1 Overall Architecture
24
C.2 Input Representation and Embedding
24
C,3 Adaptive Representation Learning
25
C.4 Task-Specific Adaptation
26
C,5 Implementation Details
27
D Experiments Details
27
D.I Datasets
27
D.2 Tasks Applicability Study Settings
28
D.3 Dataset Study Settings
30
D.4
Model Study Settings
30
More Discussion
30
E.1 Limitation
30
E.2 Broader Impact
31
E,3 Ethics Issues
31
14

---

A Details of WorldTrace Dataset
In this section, we detail the collection of the dataset, the processing, and provide detailed analysis of the resulting dataset.
Meta Info
Data Crawling
Data Processing
Timestamp
Tags
Latitude
Longitude
Normalization
Altitude
Filtering
Lengths
GPX
file
Raw Info
JSON
Calibration
Row. Longitude
Row. Latitude
Figure 4: The process pipeline of WorldTrace dataset construction.
A.1
Data Collection
Data Source. As shown in Figure 4, the raw data for WorldTrace is sourced from the shared trajectory data platform on OpenStreetMap (OSM) [30] This platform, a public sharing project, hosts over 11 million GPS trajectories uploaded by contributors worldwide from 2004 the present. To ensure data quality and reliability, we specifically targeted contributions tagged for motorized movement to ensure data currency and relevance modern transportation networks. This approach helps minimize device heterogencity and avoids outdated data that might not reflect current infrastructure. The raw data is stored in the standardized GPX (GPS Exchange Format), an XML schema designed for exchanging GPS data between applications and web services https::/www.toppt wttp:/ooogrrii.cco//XXX/// Each GPX file contains sequences of trackpoints with the following attributes:
Latitude (decimal degrees)
Longitude (decimal degrees)
Altitude (decimal numbers)
Timestamp (ISO 8601 format)
Optional metadata (version, tags, etc.)
In addition, while crawling the original trajectory, we also crawled the basic information about the trajectory descriptions, such as the starting point, markers, time, creator, etc., which was saved JSON file.
Collection Process.
Prior to integration, our collection pipeline involved the following steps: Our collection pipeline involved the following steps:
API-based Retrieval: We use the OSM API systematically query and download GPX traces based on selected filters to ensure global coverage. In order not to increase the burden on server providers, did not use concurrent crawling, and the whole collection process lasted about months, yielding about 4.5 million raw traces.
During acquisition, we implemented preliminary filtering to exclude trajectories with obvious anomalies such as: Coordinates outside valid ranges (-90" to 90 for latitude, -180* to 180 for longitude); Duplicate or long duration consecutive points; Empty or near-empty traces (fewer than 60 seconds).
Initial Filtering:
Format Standardization: All collected data was parsed from the original GPX format and converted to unified internal format for subsequent processing.
"tuns:/wwww.oeenttemmg...ggguggu
15

---

A.2 Data Processing
Our preprocessing pipeline was designed balance preserving authentic movement patterns with removing noise and inconsistencies. The process consists of three main stages:
Normalization. The data exhibited highly variable sampling frequencies, ranging from sub- second intervals up to several seconds between consecutive points. This heterogeneity creates challenges for modeling and increases storage requirements unnecessarily. We therefore applied the following normalization procedures:
Temporal Resampling: We resampled all trajectories to a uniform rate of one point per second (1 Hz). For segments with sampling rates higher than Hz, select the first occurrence of trajectory point within each one-second window. For segments with lower sampling rates, we used linear interpolation between available points to estimate positions one-second intervals.
Coordinate Standardization: All coordinates were converted the WGS84 datum for consistency, and we ensured uniform precision across the dataset (6 decimal places for both latitude and longitude, providing 0.1m precision at the equator).
Filtering. After normalization, we implemented a multi-stage filtering process to meticulously remove trajectories that were deemed unsuitable for our analysis. This comprehensive filtering approach involved several key steps:
We discarded trajectories with fewer than 32 points (equivalent to 32 seconds after resampling) covering distances below 100 meters, as these typically represent stationary periods or very short movements with limited analytical value.
Length-based Filtering:
sreed-basel Filtering: We calculated point-to-poine to-point speeds and removed trajectories containing implausible values (e.g., exceeding 120 km/h lower 0.5 km/h in urban environments), typically caused by GPS errors or anomalies.
Distance-based Outlier Detection:
We calculated the distance between the original trajectory and the map-mater matched trajectory. Trajectories that were too far away (indicating large deviations in motion) were flagged for further inspection or removal.
Loop Detection: We identify and remove trajectories that form perfect or near-perfect loops with apparent destination by their geometry, which usually indicates the presence of clearly anomalous patterns.
Calibration. GPS signals can suffer from various errors due atmospheric conditions, satellite geometry, and physical obstructions. To improve data quality, we applied map-matching techniques to align raw GPS points with underlying road networks, using a Hidden Markov Model-based approach (or using online API) with custom emission probability function that accounts for both point-to-road distance and heading consistency. Besides, each trajectory point was enriched with derived attributes.
A.3 Data Statistics and Analysis
Overall Statistics. The final WorldTrace dataset contains:
Approximately 2.45 million trajectories.
8.8 billion raw GPS points (before normalization).
Coverage across 70 countries on all inhabited continents.
Temporal span from August 2021 to December 2023.
Average trajectory duration of approximately 6 minutes.
Average trajectory distance of 5.73 kilometers.
Average travel speed of 48.0 km/h.
Points per trajectory ranging from 32 to over 600, with an average of 358 points.
Geographic Distribution. WorldTrace offers extensive geographic coverage, as illustrated in Fig- ure 5, encompassing trajectory data from 70 countries and spanning diverse environments and
16

---

10P
10
102
10a
10
10
10c
107
10a
(a) Geographic distribution across world.
USA
CHN
CAN
KOR
AUS
JPN
104
10a
10a
(b) Trajectory counts of top 10 countries.
(c) Distribution within contiguous USA.
Figure The distribution details of WorldTrace dataset.
infrastructure types. This global distribution is visualized in Figure 5(a), highlighting dense concen- trations in North America, East Asia, and parts of Europe, with trajectory counts exceeding in the most represented regions. Figure 5(b) further details the top 10 countries by trajectory counts, with the United States, China, and Canada leading in data volume. Notably, it exhibits substantial geographic diversity, with varying densities across urban, suburban, and rural environments, The top 10 countries by trajectory count, namely, the USA, China, Canada, Germany, UK, Japan, Brazil, Australia, South Korea, and Hungary, represent a wide range of urban forms, road networks, and mobility cultures. Additionally, Figure 5(c) provides closer look at the data density within the contiguous United States, demonstrating high-resolution coverage along major road networks and urban centers. This detailed distribution underscores the dataset's ability to capture nuanced variations in trajectory data across different regions. Collectively, these figures emphasize the potential of WorldTrace to serve as robust foundation for developing ongion-independent and universal trajectory models. Its extensive geographic coverage and diverse environmental representation make it well-suited for applications that require broad and adaptable trajectory data.
A.4 Data Privacy and Copyright
To protect privacy and comply with international data protection regulations, data collection ad- hered strictly to privacy regulations and ethical guidelines. Trajectories were anonymized, and any personally identifiable information was excluded to protect user privacy. addition, raw data follows the Open Data Commons Open Database License (ODbL) license from OSM: We will share derived datasets under the same license terms to respect the data use policies of the community.
http://opendatacommonss www.mooossoo//llcceeeeeeeeeeeeee wwtp/eee//oo///......../
17

---

B Pre-training Strategies
In this section, we provide specific details on the adaptive trajectory resampling strategy and the self-supervised trajectory masking strategy, and we will provide the design motivation and theoretical analysis for these two strategies.
B.1 Adaptive Trajectory Resampling
Trajectory data heterogeneous is one of the main challenges in cross-regional and cross-device trajectory modeling. The Adaptive Trajectory Resampling strategy solves this problem through two complementary components: Dynamic Multi-Scale Resampling and Interval Consistent Resampling. We designed these two strategies with the motivation of fitting different regions and dataset qualities through diversified trajectory sampling frequencies and motion patterns. Dynamic Multi-Scale Re- sampling ensures an optimal balance between information preservation and computational efficiency across different trajectory lengths, prioritizing the retention of key motion patterns. Interval Consis- tent Resampling enhances the model's generalization ability across datasets with different sampling rates by normalizing the time dimension.
B.1.1 Dynamic Multi-Scale Resampling
As discussed in Section 4.2, we adopted logarithmic resampling ratio that adjusts the sampling rate according to the trajectory length. The resampling ratio function R(n) is designed to decrease logarithmically as the trajectory length n increases:
Rmiin
T1 2 Timax
R(n)
=
1 - - (1-RRin)er ((())
Taiin
< n < 70max
1
Timin
where Rmin is the minimum sampling ratio, and Ammin and mmux denotes the shortest and longest length thresholds, respectively. The normalization factor Φ(n) is computed as follows:
<(n) In (n Tmmin + 1)
In (Tmax mmin + 1)
Formal Definition. Here, we provide a formal definition and theoretical analysis of the above empirical results through information theory and computational efficiency perspectives. For any trajectory т = {Pi, P2: ..... Pn} consist of n spatio-temporal points. the number of points for resampled trajectory T' (P1.P2.... Pm is:
m R(n)-1 nn.
(9)
where function R(n) that determines what proportion of points to retain. The logarithmic sampling strategy guarantees bounded sample sizes for arbitrarily long trajectories while preserving critical minimum information content. Specifically:
For Mmmin: R(n) so n
For 2 Pmmxx: R(n) Rmins we set Mmna constant. Clearly, the number of sampled points is bounded above by mmxx-
To ensure boundedness, we analyze m(n) in the intermediate domain (Amin: Pmmx)).
[1 (1 Rmin)) <(n)]
(10)
Taking derivative:
d( d(R(n) m) dn
(11)
Solving this equation yields a value n² Amax ensuring that mmax is bounded. Since R(n) becomes constant for n 2 Rmaxx and increases linearly in that region, the global maximum occurs either n² or mmax However, due to the logarithmic decay of R(n), the growth of m slows, and the maximum value is achieved at a finite n² Rmaxx Hence, is bounded for all
18

---

Corollary 1: Information Preservation and Computing Efficiency Optimization
Standpoint: The logarithmic sampling function provides optimal balance between information preservation and computational efficiency across varying trajectory lengths.
Proof: Let (T)) represent the information content of trajectory Empirical studies in spatio- temporal data analysis suggest that information content typically scales sub-linearly with trajectory length, following approximately:
T(T) x n²:
(12)
where 0 < a < 1. For example, a 2 0.7 indicates that only 70% of the trajectory points contain valid feature information, and the remaining 30% are redundant. For resampled trajectory 7' with m R(n) points, the information preservation ratio 17 can be approximated as:
T(T') m )a R(n)*. I(T)
7 =
(13)
The computational cost C' of processing trajectory typically scales linearly with length:
C(T) x xn"
(14)
where 33 1 1, typically 8 = 2 for Transformer-base based models. After resampling, the computational efficiency gain yy is:
C(T)
7 C(T')
(15)
m
R(n)*
The optimal sampling function maximizes the product of information preservation and computational efficiency:
max 7) = max R(n)a. a R(n)
max R(n)a R(n)
(16)
R(n))
R(n)3
Since a < B for typical trajectory data, this is a decreasing function R(n). However, we must maintain minimum level of information, hence the constraint R(n) 2 Rmin-
When we examine the information density:
[(T) x xn
D(n)
(17)
n
we observe that it decreases as n increases, indicating diminishing information return per point in longer trajectories. An optimal sampling ratio should proportionally track this information density:
(1) x D(n) o n=_!
(18)
Our logarithmic resampling function's derivative in the intermediate domain (Amin < n. < 4mmxx) is:
dR(n) dn In
1 Rmin
(19)
Rmin
Mmiin
As n increases, the growth rate of the logarithmic function slows down, causing the rate at which the sampling rate R(n) decreases also slow down. This is closely proportional to the derivative of the theoretical optimal sampling rate:
(R x (a - 1 x 2-
(20)
dn
For example, when ~ 0.7, we have drg x ni.. This property of logarithmic functions (their rate of change inversely proportional the input value), making them naturally suited this task. Therefore, Logarithmic resampling provides a theoretically reasonable compromise: it preserves almost all of the information from short trajectories (where every point may be significant) while reducing redundancy in long trajectories (where redundancy is highest). Compared to linear functions, logarithmic functions can more naturally adapt to the information density curve across the entire trajectory length range.
19

---

Linear
Logarithmic
Min Points
Max Points
Sampling Ratio vs. Trajectory Length
1.0
Sampling Ratio
0.6
0.4
Min sampling ratio
Number Sampled Points vs. Trajectory Length
Number of Sampled Points
200
150
Max number points
100
Min number points
50
400
Original Trajectory Length
Figure 6: Illustration difference between dynamic resampling strategies with linear method.
Visualization. As shown in Figure 6, we compare in detail the proposed dynamic resampling strategy with a linear resampling strategy (where the sampling ratio R(n) decreases linearly with the length of the trajectory) regarding the sampling ratio and the sampled points. Specifically, this figure illustrates the dynamic resampling strategy compared to a linear resampling approach. The top plot displays how the sampling ratio R(n) decreases with trajectory length n.. The dynamic strategy (orange curve) follows a logarithmic decrease, ensuring a smoother transition from retaining all points for short trajectories (n < Memin to reducing redundancy for long trajectories (n 2 Remax). with a minimum sampling ratio Rmin- In contrast, the linear resampling strategy (blue curve) decreases the sampling ratio at a constant rate. The bottom plot shows the relationship between the number of sampled points and trajectory length for both strategies. The dynamic approach adjusts sampling more gradually, preserving detail for intermediate trajectories while minimizing redundancy in longer trajectories. However, linear sampling methods instead suffer from redundancy of sampling points due to the smoothly decreasing sampling rate. This dynamic resampling strategy ensures a balance between data volume reduction and the retention of critical movement details. The visual comparison highlights the adaptive nature of the dynamic strategy.
B.1.2 Interval Consistent Resampling
Consider different cities may exhibit drastically different sampling intervals due to: Varying data collection protocols (e.g., Is in City A vs. 58 in City B) and technical limitations or regional preferences in tracking technologies. This heterogeneity poses serious challenge for developing universal trajectory models, as models trained on data from one region may fail to generalize to regions with different sampling characteristics. Therefore, we performed consistent interval sampling (at random time intervals) on the original dataset to ensure its generalizability across different datasets. Specifically, ICR standardizes the temporal intervals between trajectory points, transforming
trajectory T = {x11 31. f1), (x2, V/2 12) tz),..., (En: Vn, tn)] with irregular time intervals into trajectory T' { 3/1, At), (x2, 1/2, At), (Im: Vm. At)} with uniform time intervals At f++1 - t,, for all i E [1, m 1).
Corollary 2: Temporal Regularity for
Generalization
Standpoint: Interval consistent resampling regularizes the temporal dimension of trajectory samples, enhancing the model's ability generalize across datasets with heterogeneous sampling rates.
20

---

Proof: Let D1 and D2 be two dataset of region with average sampling intervals j and Pst Assume the temporal pattern recognition task can be formalized as learning a function fo : T + Y where the learned parameters 0 should idcally be robust to sampling rate variations. For trajectories with irregular sampling, the model must learn the relationship:
(1)
(2)
y = f0((11, y1, 11), (I2, V2, 12),...,, (In, (n)).
(21)
This requires implicitly learning the distribution of time intervals P(AT), which varies across datasets. With ICR, the learning problem becomes:
y fo ((x1, vi, f1), (2/2, 1/2 t2)).... ..., (Im Bm bm)), with 644 - E2 Afrxed
(22)
where temporal intervals are now consistently fixed, eliminating the need learn dataset-specific temporal distributions,
Information Entropy Analysis. From the entropy perspective, consider trajectories from different regions r with characteristic sampling intervals At", where the distribution of intervals can be modeled as:
P( (r) (
(23)
where / is a dataset distribution with joint distribution of regions (or dataset) and sampling intervals is:
mean Jt, and variance o?? The entropy of the
((D,AT) (D) H (AT)) (D).
(24)
This high conditional entropy ((T D) creates a strong statistical correlation between regions
and temporal patterns, forcing region-specific model adaptations. Interval Consistent Resampling transforms the original trajectory t into T' where t/41
Atried Ví E [1.m- 1] This
transformation minimizes the conditional entropy:
H(T D) =,
(25)
which effectively decoupling the temporal sampling pattern from the region. This transformation
reduces dataset-specific temporal variability, thereby bringing the conditional distributions of trajec-
tories across datasets closer in distributional space:
P(T' Di) = P(T') D2).
(26)
The reduction means the model sees more consistent input distributions, thus reducing the domain gap in learning.
For trajectory modeling tasks that focus on spatial patterns rather than absolute temporal dynamics, information loss is minimal when resampling preserves relative temporal order and approximate speed relationships. For a trajectory with velocity profile v(t) = (p++1 - pts/(t++1 - tt;,, the constraint:
1p/+1 Pill
Afreed
(27)
||++1
pill
At;
ensures that relative speed information is preserved even absolute time intervals normalized.
B.2 Self-supervised Trajectory Masking
suff-supervised Trajectory Masking (STM) forms critical component of UniTraj's pre-training strategy, enabling the model to learn robust representations from incomplete trajectory data. While we introduced the concept in the main paper, this appendix provides a more detailed examination of the theoretical foundations, implementation details, and empirical justifications for our masking approach. Our Self-supervised Trajectory Masking framework implements four complementary masking strategies (as illustrated in Figure 7), each designed to simulate different types of real-world data incompleteness and encourage specific learning objectives:
B.2.1 Random Masking
Random Masking applies a uniform probability distribution to select trajectory points for masking, where each point has an equal chance of being masked regardless of its position or significance. Formally, we select a subset of indices:
Innd Uniform 21,,
(28)
21

---

Random
Block
trajectory
Key Points
Last N
Figure 7: Illustration of the difference masking strategies.
to mask. This strategy forces the model to develop both local and global dependencies, as must learn to infer missing points from surrounding context without relying predictable patterns. Random masking is general masking strategy used to simulate sensor failures or temporary GPS signal loss that often occur in random trajectories.
B.2.2 Block Masking
Block Masking conceals consecutive segments of the trajectory by selecting starting point & and masking b consecutive points:
I [k. + 1...., k 1}, for some k.
(29)
This approach simulates extended sensor failures, tunnels, urban canyons where trajectory data may be unavailable for continuous periods. The strategy challenges the model to reconstruct substantial missing segments by understanding the broader movement context, encouraging the development of long-range dependencies and trajectory continuity reasoning.
B.2.3 Key Points Masking
Algorithm 1 Ramer-Douglas Peuckr Peucker (RDP) Algorithm
1: RDP(T, 5, e, e)
2: Initialize max distance dmax f-
3: Initialize index &o + -1
4: for i= s +
to 5: Calculate the distance from p; to P&Pe: d;
do
6: if d, dmax then
7: Update max distance dmux +- dj
8: Update index is t-
9: end if
end for
11: if dmxx
then Tka + RDP(T, k; c)
13: Tright < RDP(T, k, e, c)
return {Pk] U Tien
Tright
15: else
return {ps,Pe}
17: end if
The key points masking adopt the Ramer- Ramer-Dougle Seuuglas-Peucker (RDP) algorithm [10], which simplifies trajectory by retaining points that are farthest from the line PiPa connecting the first and last points. The indices are determined by
Iey = {pk dmaax (pk, PIPn) > e).
(30)
where e is predefined threshold, and dmax max {d(pt. PIPw.) 2 & - 1 is the maximum distance measures deviation from this line. As summarized Algorithm 1, the RDP algorithm iteratively identifies the point Ph that maximizes dmax d(pk, p.)) ). If dmax point Pa treated as key point and included the mask set Ikey. This process recursively applied to the trajectory segments Tish {pi..... iPi.....Pl. .Pt) and Tright {pk... ... Pn. J. isolating critical points
the corresponding
22

---

for masking. By focusing on these pivotal points, the model is challenged to reconstruct essential trajectory segments, reinforcing its understanding key structural patterns within trajectories.
B.2.4 Last N Masking
Last N Masking systematically removes the final N points of each trajectory:
Last
{n N + 1, N n2).
(31)
This strategy explicitly simulates trajectory prediction scenarios where future positions must be forecasted based on historical observations. By incorporating this masking approach during pre- training, the model develops capabilities directly applicable to trajectory prediction tasks, creating a natural bridge between self-supervised pre-training and downstream forecasting applications.
Corollary 3: Robustness through Comprehensive Masking
Standpoint: Supe-vupe Trajectory Masking improves the robustness and generalization ability of the model incomplete and heterogeneous trajectory data through comprehensive masking strategy, enabling the model to learn more effective trajectory representations.
Proof: Let the trajectory data space be D, with true data distribution denoted P(T). In real-
world applications, due to device limitations, communication failures, and environmental factors, the observed trajectories are often incomplete irregular, and their distribution is denoted as P(f). The incompleteness of trajectory data can be formalized as conditional distribution P( T), representing the probability of observing incomplete + given complete trajectory т.
STM can be formalized set of masking functions {M,..M...... Ma]. each corresponding to different masking strategy. For a resampled trajectory T' = {P1,P2, ,. the masking function M; transforms as:
: M,,(T, m2) = (pI,..... MA..) .P.)
(32)
where I, S (1,2,....
n is the index set of masked positions and r, = |1// is the masking ratio. InntrmmttionTThoree Analysis. From information -theiorttcc perspective, STM introduces an artificial information bottleneck that forces the model to learn efficient representations. We define the model objective as minimizing the reconstruction loss:
(()) = - (d(s((M((Trr.))
(33)
where d is a chosen distance metric.
During training, the model needs learn the joint distribution P(T, T:) and estimate the conditional distribution P(T 4z). By Bayes' theorem:
P(T (.) P( TTP) P(F.)
(34)
By using diverse masking strategies, the model learns to estimate P(T | +;) across different types of masked trajectories, which equivalent to learning the true trajectory distribution P(T) and the various degradation mechanisms P(+; T).
Optimality Theory of Diversity Complementary Masking Strategies. A key innovation in STM is the use multiple complementary masking strategies. We define the coverage region of the union of masking strategies as:
C(IMI...., Mc))
- (I...... max PM, d7,
(35)
where PM, (7) denotes the distribution of incomplete trajectories generated by masking strategy M,,
We assert that for suitable masking ratio and diverse set of masking strategies (MM,...,MM, the combined coverage region satisfies:
C( Me))>
C((M))). iE(I.....k
(36)
This inequality indicates that the joint use of diverse masking functions provides strictly better coverage possible incomplete trajectories than any individual strategy.
23

---

The advantage of combining multiple masking strategies in STM over using single masking strategy can also be theoretically justified by comparing the expected reconstruction error. Assume the real-world conditional distribution of incomplete trajectories is Preal T). For a single masking strategy M,, let the generated distribution be PM, r). Then the expected reconstruction error under this distribution is:
P(r) [d()(
(37)
can be shown that training with a mixture of multiple masking strategies leads to lower bound on this error compared to using any single strategy. This is because the mixture of diverse masking strategies better approximates the true real-world distribution of incomplete trajectories:
k
Σ PM. (F (T))
KL Prea(( (T)
<
min KL (PrauF (E()....))
[T PM, (f [T)
(38)
k
Here, (-Il-l) denotes the Kullback-Leinne divergence.
C Details of UniTraj
In this section, we provide detailed implementation of UniTraj, including the architecture and parameter settings.
C.1 Overall Architecture
The TniTraj model adopts encoder-decodes architecture based on transformer blocks, designed to process trajectory data with minimal regional dependency and maximum task adaptability. Figure illustrates the overall framework of UniTrajj which consists of several key components: spatio- temporal tokenization, encoder, decoder, and rotary emhedding layers.
Our model takes trajectory points that have already undergone adaptive resampling (ATR) and masking (STM) as described in Appendix B. The input trajectories are represented as sequences of latitude-logiitud coordinates and timestamps: the total number of points after resampling. Unlike previous approaches that rely on region- features road network information, UniTraj operates solely these basic coordinates, enhancing its universal applicability across diverse geographic contexts.
{IIng lat, 4 ji = 1, 2, ..... n). where is
Sponal
Terpoora Sequence
Laver Noensiiztion
Retryy Entedain
Dyzamic Resompling
Inserval Cussitent
Resampling
Nammalirat to Original Cosy
Get Time Inienall
Resamping
Liazar
Imner syar
Runáin Masking
Black Madking
Spatle
Tolenization
Key Points Masking
Masking
Trajetory Minking
Reshuttle
Layer Nocmalization
ULi Tokens
Teassforner
Spural--eppeal Tekenization
Transommer
Layer
+
Activation
1x
FnocdesDessd Sticture
Receder Teap.
Layer Noom
UniTraj
Ensuler
Decuder
Transfermer
Figure 8: The main architecture and components of UniTraj.
C.2 Input Representation and Embedding
Tokenization. To enhance numerical stability and generalization, all input coordi-
nates are normalized relative to the first point in the trajectory (z,, 31) (Ing, .Ing, lat, llat).
24

---

For the spatial component, we project the normalized coordinates into a a d-dimensional space using ID convolutional neural network, yielding spatial embedding
Conv ConvID([es) D( 0:]:
(39)
where 8, represents the learnable parameters of the convolutional layer. We use kernel size of 1 with no stride to capture local spatial dependencies. Similarly, the temporal component, based on the time intervals At,, is embedded into the same d-dimensional space via linear layer, resulting in temporal embedding:
h = We A. br:
(40)
where W2 E Rdx1 and by E Rd are learnable parameters. The final embedding for each trajectory point is obtained by element-wise addition the spatial and temporal components:
h h h
(41)
This dual-tokenization captures both spatial and temporal dynamics, enabling the model to learn relative movement and temporal dependencies effectively.
Rotary Positional Encoding (RoPE). In addition to encoding the spatial and temporal details of each trajectory point, it is essential to capture the relative positional relationships between points. These relationships enable the model to comprehend the movement sequence and the timing between points, both crucial for accurate trajectory modeling. To achieve this, we employ Rotary Position Encoding (RoPE) [32], which maintains the relative positional information between points by rotating the trajectory embedding vectors. Given the combined spatial-temporal embeddings h; for point in the trajectory, RoPE applies a rotational transformation:
RoPE(h.) = /cos 02 sin 0
h h
(42)
sin 0:
cos 0;
where h and h that varies proportionally with the position index i. Specifically, 0 is calculated as 0; 100003473 where & is the index the embedding dimension, and d the total dimension of the embedding.
(2) are the first and second halves of the embedding h., and 0, is rotation angle
The main advantage of RoPE is its ability to preserve relative positional information through rotational symmetry. This ensures that the relative distance and directional relationships between points are maintained, enabling the model to capture both local patterns (e.g., short-term movements) and global patterns (e.g., longerange directionality) within a trajectory. By encoding these relative positions. RoPE strengthens the model's ' capacity to understand movement dynamics across varying scales.
C.3 Adaptive Representation Learning
The UniTraj employs an encoder-decoder architecture [13] tailored for trajectory data. The encoder and decoder use Transformer blocks [33] with RoPE-powered self-attention mechanisms to capture dependencies within trajectory embeddings.
Encoder. Given a masked trajectory 7 = (pı,.... [MAS Pn. we first extract the em- bedding representations of the unmasked points H = (h1, h2, h2,..., h. (where m and i & I) through the tokenizer and positional encoding steps. The encoder Eg processes the visible (unmasked) points in a trajectory to generate contextualized representations. consists of Le transformer blocks, each incorporating:
1. Multi-head Self-attention with RoPE: As described previously, we apply RoPE to the self- attention mechanism:
@ROPP Kf
ROPE
Attention Q, K, V) softmax
V
(43)
Vdl.
where QROPE and KROPE are the query and key matrices with RoPE applied.
2. Feed-forward Network (FFN): A two-layer FFN with GELU activation:
FFN( (r) W2 GELU(W; z b1) + b2
(44)
25

---

3. Layer Normalization and Residual Connections: Each sub-block is wrapped with layer normal-
ization (Pre-LN) and residual connections:
H' LayerNorm H AAttention(H) (H))
(45)
H' LayerNorm H' +FFN(H')
(46)
The encoder's output is set of hidden representations H (h, points.
for the m visible
Decoder.
The decoder reconstructs the masked points based on the contextualized representations from the encoder. It operates by combining the encoder's embeddings with mask tokens and processing them through La transformer layers:
1. Input Combination: The decoder input consists of both the encoder outputs for visible points and the mask token embeddings for masked positions:
(f
h, = h if Index(j), i I }). musk
H = Reorder
(47)
ifeel
where h mask represents the mask token embeddings for all masked positions.
2. Decoder Transformer Blocks:
The combined input is processed through La transformer blocks, each with the same structure the encoder blocks (self-attention with RoPE, FFN, layer nor- malization, and residual connections). The self-attention mechanism allows information to flow between visible and masked positions:
H TransformeBBlocc(H""
(48)
for E {1.2..... Ld).
3. Output Projection: The final layer projects the decoder's representations for the masked positions back to coordinate space:
(2,, 0j) = Wo ht + bo
(49)
where j indexes the masked positions, and W. R2xd and b, R22 are learnable parame- ters. These projected coordinates are then transformed back to the original coordinate system (Ing,, låt,) = ( + Ing Uj lat:).
UniTraj trained using self-supervised learning approach with a reconstruction loss function. For each trajectory, apply masking strategies (random, block, key points, last N), and the model trained to reconstruct these masked points:
L
1
Σ ((),,0;;) (z,, 11 jel
(50)
where I is the set of masked positions, and
||
denotes the L,2 norm.
C.4 Task-Specific Adaptation
For downstream applications, UniTraj can be used in two primary ways:
The pre-trained model's 's encoder can be directly applied to extract trajectory representations for various tasks without further training. We use the pre-trained UniTraj
Zero-shot Transfer:
backbone and attach task-specific Multi-Layer Perceptron (MLP) adapters to the output:
Lnall
MU
(51)
The MI.P adapter typically consists of 2-3 layers with non-linear activations:
H ada
W2 ReLU(W, HH + b1) + b2
(52)
where He is the output from the UniTrraj encoder.
2. Fine-tuning: Update all parameters of the backbone and adapters with specific dataset.
26

---

For different downstream tasks, we design specific adapter architectures:
For Trajectory Recovery/Prediction:: We can directly use UniTraj's decoder as an adapter without any additional modifications.
For Trajectory Classification: The adapter includes pooling operations followed by fully con- nected layers to produce class logits.
For Trajectory Generation: The adapter interfaces with generative models by providing condi- tioned trajectory embeddings.
C.5 Implementation Details
Additionally, we summarize the list key hyperparameters and popllmenntaiionsseciitte settings that may be used in the implementation of UniTraj in Table 5. Specifically, our model contains 8 encoders and decoders, each using heads in the attention layer. The model has approximately 2.38 million parameters, allowing dimension to 128 and employ RoPE to capture spatial and temporal relationships effectively. Our model can handle arbitrary length of the number of trajectory points and pad it to length of 200. Naturally, due to the use of rotational positional embedding, our model holds extension capability and supports a maximum length of 512. In addition, when performing the dynamic resampling strategy, we set the minimum number of sampling points to 36 and the maximum to 600, and its minimum sampling rate 0.35. Finally, we provide the probability of using various masking strategies during training, which can be further adapted to the specific task as we discussed in Section 5,4 and Table 4.
balance complexity and computational efficiency. We set the embedding
Table General parameters setting for UniTraj.
Parameter
Setting value
Refer range
Encoder Blocks
8
2
Decoder Blocks
2
Attention Heads
2
Encode Dim
128
64 ~ 256
Parameters of Model (Millions)
2.38
Mask ratio
0.5
0.25 0.75
Trajectory Length Padding
200
36 ~ 256
Maximum Length Padding
512
Minimum Trajectory Points
36
Maximum Trajectory Points
600
Minimum Sampling ratio
0.35
Random Masking
0.7
Key Points Masking
0.15
Block Masking
0.05
Last N Masking
0.1
D Experiments Details
We the Adam optimizer and mean square error loss with initial learning rate of 10-3 with learning rate scheduler. The model is trained for 200 epochs with batch size of 1024. and early stopping is applied based on validation performance. All experiments were conducted using PyTorch, where the foundation model is trained on NVIDIA A100/L40s 40GB GPUs and the baseline experiments are performed RTX 2080 Ti.
D.1 Datasets
We evaluate the performance of the proposed model using six diverse real-world trajectory datasets. Each dataset represents different data collection scenarios. quality levels, motion patterns, and geographic regions, providing comprehensive test of the capabilities of UniTraj.
WorldTrace:
WorldTrace is our proposed large-scale, globally distributed dataset, which we describe in detail in Section 4.1. We curated high-quality subset of 1.1 million trajectories from
27

---

the original dataset, which have been filtered to remove long stops and loops. Of this subset, million trajectories are designated for model training combined with resampling or masking strategies, with the remaining 100,000 reserved for testing without any operation. To ensure consistency and enable independent zero-shot evaluations, the testing dataset normalized to a sampling interval of 3 seconds per point.
Chengdu [8]: The Chengdu dataset comprises over one million urban mobility trajectories collected from taxis operating in Chengdu, China, reflecting daily commuting and transportation patterns in densely urbanized area. It features dense, high-frequenn points that provide detailed insights into active urban environments.
(3-second for most trajectories) sampling
Xi'an [8]: Similar to Chengdu, the Xi' an dataset includes millions of taxi trajectories gathered in Xi' an, China, focusing movement patterns within another densely populated Chinese city. The data, collected during November 2016, captures the traffic dynamics and urban mobility behaviors specific to this region.
The Ge dataset is a widely used trajectory dataset collected over three years by 182 users, primarily in Beijing, China. It is mainly distinguished by wide variety of travel modes, including walking, cycling and driving. With this data, we can study the trajectory movement patterns and behavioral habits of different travel modes. Besides, this dataset suffers from irregular and often long sampling intervals, which limit its granularity and quality for trajectory analysis.
GeoLife [47]:
Crab-Posisi [15]: Sourced from Southeast Asia, this dataset contains 84,000 ride-hailing trajecto- ries, predominantly from the Grab service in cities such as Jakarta and Singapore. The variable sampling intervals across these trajectories provide insights into urban mobility patterns unique to Southeast Asian metropolises.
Porto [28]: The Porto dataset consists of taxi trajectories collected Porto, Portugal, capturing trips between different areas of the city. Although it provides valuable insight into taxi mobility within the city, the dataset has a relatively low sampling frequency, with long intervals (15 seconds) between data points.
D.2
Tasks Applicability Study Settings
D.2.1 Trajectory Recovery
In this experiment, randomly mask 50% of trajectory points and test the recovery performance. Specifically, we evaluate UniTraj in both zero-shot (trained solely on WorldTrace) and fine-tuned settings (trained on World'rrace and then fine-tuned each respective dataset), aiming to under- stand its adaptability with and without task-specific training. Additionally, we compare UniTraj against a diverse range of baselines, including traditional deep learning models (Linear, DHTR [35], Transformer [33], and DeepMove [11J) and pre-trained models (TrajBERT [31] and TrajFM [23]). Performance metries include Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) with meters, computed based on geographic distance:
1 n
MAE
1y4
Dil.
(53)
RMSE
(yr
(54)
72
where y, and ÿJ are the real and recovered coordinates, respectively.
D.2.2 Trajectory Prediction
In this task, we focus on predicting future trajectories based on historical trajectory points. Following the setup [23] in previous work, we predicted the locations of five future points. The baseline settings and evaluation metrics are consistent with those used for the trajectory recovery task, and experiments conducted on WorldTrace, Chengdu, and Life datasets.
D.2.3 Trajectory Classification
The Trajectory Classification task conducted on two datasets, GeoL and Grab-Posisi. this task, we will only use the encoder module of the UniTraj as backbone and then add classification header.
28

---

We compare UniTraa in two settings: without fine-tuning (wo/ft), where only the classifier head is trained, and with fine-tuning (ft), where the entire model is updated. For baselines, we following prior literature [21] use representative classification models including GRU, LSTM, STGN [45], and TrajFormer [21]. Performance reported by classification accuracy:
n Acc Σ
(y6 ii);
1y6s
(55)
where yi and Ús are the predicted and true labels, respectively, and I(-) is a indicator function. Following the general settings of previous work, selected four travel modes from the Geolife dataset, namely walking, bus, bike, and driving. For the Grab-Posisi dataset, there are two travel modes: and motorcycle.
D.2.4 Trajectory Generation
In this task, we follow the approach in prior work [50], assessing trajectory generation using sequences of road segments that represent trajectories without explicit temporal attributes. Specifically, use Control'rra as a downstream task for trajectory generation, where we replace the road segment extraction component (RoadMAE) of the ControlTraj with UniTraj's 's encoder, testing the effectiveness of the embedded representation. The evaluation includes density error metrics [50]:
(G (G+O) 1 D 2
(G (G+O),
Density Error = JSD(G]JO) = D(I
(56)
2
where G is the distribution of the generated trajectories in the city (which divides each city into grids 16x16 size and calculates the count of trajectory points associated with each grid), and O is the distribution of the original trajectories. JSD(.) is the Jenson -aasonnShannon divergence for two distributions.
(a) Original.
(b) ControlTraj.
(c) ControlTrraj UniTraj.
(d) Original.
(e) ControlTraj.
(1) ControlTraj UniTraa.
Figure 9: Performance comparison of trajectory generation task with Chengdu dataset (first row), and transfer to Xi' an dataset (second row).
For the this task, UniTraj demonstrates its versatility through integration with existing generative frameworks. By replacing ControlTraj's road segment extraction module with UniTraj, achieved

---

5.1% 1.1% reduction in density error (from 0.0039 to 0.0037) when trained and generated on the Chengdu dataset. This improvement, though modest in magnitude, represents a significant advance in trajectory fidelity. More impressively, when transferring the generation capability to Xi an without retraining- challenging cross-region scenario-the ennhane-enhaaced generator maintains a density enror of 0.0152. In contrast, the baseline ControlTrra experiences a 0.0171 density error when transferred across regions. This cross-region resilience further validates UniTraj's ability to capture universal trajectory patterns that transcend specific geographic contexts. We also show the heatmap visualizations to measure the accuracy and realism generated trajectories in Figure 9, where brighter regions indicate denser trajectories and darker regions indicate sparser ones. Detailed analysis of the generated trajectories reveals that UniTraj UniTraj-enhancee enhanced generation produces more realistic speed variations, particularly in complex road segments such as intersections, sparse or dense areas. In summary, the above results underscore UniTraj's 's potential for robust and transferable trajectory generation, proving its effectiveness in both familiar and novel geographic settings.
D.3 Dataset Study Settings
Effeet of Dataset Scale and Quality. This task focuses on the impact of dataset size and quality on UniTraj performance. We analyze WorldTrace for the effects of different amounts and qualities of training data. Specifically, we further process the complete WorldTrace dataset by removing cyclic trajectories, removing trajectories with too many stopping points and sparse trajectories. In total, we partitioned subset of high-quality trajectory data numbering I million items, and further partitioned a subset of 10,000, 500,000 trajectory data for UniTraj training.
Effect of Dataset Diversity. The task assessed the impact of using different data coverage (i.e., geographic diversity) on the model. We evaluate the zero-shot performance of UniTraj trained on the WorldTrace and Chengdu datasets, respectively, and tested on multiple real-world trajectory datasets. We chose the Chengdu dataset for comparison because it has very high data quality and has the identical me collection standards as the Xi' an dataset.
D.4 Model Study Settings
For setting the number of encoders decoders for the model, we adopt the following scheme (en- coders: 2,4,6,8,12), [decoders:2,2,4444.6) (attention he We believe that asymmetric encoder-decoder architecture can significantly reduce the number of parameters while maximizing the performance of the model. And the scaling law between the number of model parameters and the size of the data will be one of the considerations in our future research and model architecture design.
E
More Discussion
E.1
Limitation
While UniTraa represents significant advancement in universal trajectory modeling, several limita- tions remain that warrant acknowledgment and future investigation. Despite WorldTrace' 's umprece- dented geographic coverage spanning 70 countries, data distribution remains uneven, with certain regions (particularly in Africa and parts of Asia) underrepresented, potentially limiting model perfor- mance in these areas. Additionally, our focus on motorized movement may restrict generalization to non-motorized mobility patterns, such as pedestrian trajectories with distinctly different motion properties. The computational resources required for training and deploying UniTraj at scale present practical challenges for resource-constraines environments, necessitating more efficient architectures or distillation approaches. From a technical perspective, UniTraj relies solely on coordinate and temporal information, lacking integration of contextual features like road networks, traffic conditions, and points of interest that could further enhance predictive accuracy. Addressing these limitations rep- resents promising directions for future research, potentially through expanded geographic coverage, multimodal trajectory integration, architecture optimization, context-aware modeling, and continual learning techniques. Nonetheless, we believe that the proposed UniTraa and WorldTrace datasets will contribute to the development of the entire community towards a more generalized, global view of trajectory analysis.
30

---

E.2
Broader Impact
This work presents both promising opportunities and notable concerns for society. Positively, this universal trajectory model could popularize mobility intelligence across diverse regions, enabling improved transportation systems in underserved areas without extensive local data collection. The model could drive more efficient urban planning, reduce traffic congestion and emissions, and enhance logistics optimization globally. However, this technology could also enable more pervasive monitoring capabilities, raising surveillance concerns if misused. Additionally, there exists potential for widening technological disparities between resource-rich and essource-constrained organizations. Balancing these implications requires commitment to privacy-preserty techniques and equitable access policies to ensure this technology advances social welfare while minimizing potential harms.
E.3
Ethics Issues
The development and application of large-scale trajectory foundation models raise important ethical considerations, which have been carefully addressed throughout this research. primary concern is data privacy and the risk of re-identification. To mitigate this, we implemented a robust anonymization protocol: during data collection from OpenStreetMap. we deliberately excluded user identifiers, and the data was subsequently processed through filtering, resampling, and map-matching. This multi-step process significantly abstracts the trajectories from their original form, ensuring the publicly shared dataset is thoroughly anonymized.
Another critical issue the potential for misuse, particularly for large-scale surveillance. The design of UniTraj inherently addresses this risk at the model level by learning from aggregated, generalized movement patterns rather than individual user paths, making incapable of reconstructing original trajectories tracing personal identities. To further prevent the learning of localized patterns, we will release any model versions fine-tuned on private datasets and will advocate for clear guidelines for responsible use.
Finally, we acknowledge the potential for biases arising from the uneven geographical distribution of training data. To address this, we are employing data augmentation techniques to mitigate the effects of data scarcity in certain regions. We analyze these inherent biases and their potential societal consequences, and we are committed to ongoing efforts to enhance data diversity and model faimness in future work to ensure our research aligns with best practices in responsible AI.
31

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
All claims have been confirmed in the methods and experiments section.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed this in detail in Appendix E of the supplementary materials.
32

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
We provide analysis and proof in Appendix of the supplementary materials.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide code and sample data in the supplementary materials. In addition, we provide detailed parameter settings in the Appendix C.5.
Guidelines:
The answer NA means that the paper does not include experiments.
33

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. w
For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We provide code and data https://gihubb.om/Yas/////
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: /Cod/Soodemiiiceeeee SoooeeSumiissooPPooo
/ //nips... /tippscc/pput wttp:cccppppppcccccccee/ococpcca
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
34

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
We provide detailed data and hyperparameters settings in the Appendix C.5.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: We report results by taking the average of multiple rounds of experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We declare the computing resources used in the experiments in Appendix D.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
35

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /ttp://eeuippppp.e ctt.ipp...ppppppae cctppblic///ccccccccccccccccclct wtwhhiiiiieeeeeeee....
Answer: [Yes]
Justification: We ensure that no personally identifiable information is included in the data.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discussed this topic in Discussion Appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: This paper is not related to this risk.
Guidelines:
The answer NA means that the paper poses such risks.
36

---

Released models that have high risk for misuse or dual-use use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We declare the license of the used data in Appendix A.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: licensing, acquisition, and processing the dataset introduced in this paper, and ensure that there no privacy risks.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
37

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the L.M used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [Yes]
Justification: We use LLM to improve language expression and visualization results.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https : ttttps:/nneuree htt/e//rps..eeooefeeeeeeeeeeeeee www.e/ooeeeeeeee/0000000 for what should or should not be described.
38


Reviews:
[{"review_id": "bVr42tw2AY", "reviewer": "Reviewer_9q8j", "summary": "This paper presents UniTraj, a unified foundation model designed for learning from spatiotemporal trajectories across diverse geographic and application domains. The model is based on a Transformer encoder-decoder architecture and is pretrained using self-supervised learning on a newly constructed large-scale dataset called WorldTrace, which contains 2.45 million trajectories from over 70 countries.\n\nTo improve generalization across trajectory types and regions, the authors introduce two domain-specific pretraining techniques:\n- Adaptive Trajectory Resampling (ATR), which adjusts sampling density based on trajectory length to preserve structure while normalizing sequence size.\n- Self-supervised Trajectory Masking (STM), which applies a variety of masking strategies (random, block, key-point, and last-N) to simulate missing or partial observations during training.\n\nUniTraj is designed to support multiple downstream trajectory tasks—including trajectory recovery, future prediction, semantic classification, and trajectory generation—using lightweight task-specific adapter modules. The model achieves strong results across a range of datasets and tasks, including in zero-shot and cross-region generalization settings, demonstrating its potential as a general-purpose trajectory learning framework.", "questions": "- Why did you choose a Transformer encoder-decoder architecture rather than an encoder-only (like BERT) or encoder-decoder with causal masking? Would an encoder-only model suffice for non-generative tasks?\n- Are you open-sourcing the dataset?\n- How does performance degrade on low-sample regions?\n- What’s the inference latency and memory footprint of UniTraj?\n- How sensitive is the model to trajectory length at inference?\n- Provide per-continent error to diagnose regional bias.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- Comprehensive Evaluation: The paper evaluates UniTraj on four diverse tasks—trajectory recovery, future prediction, classification, and generation—across a variety of real-world datasets from different countries. This supports the claim of broad generalizability.\n- Effective Pretraining Techniques: The proposed self-supervised strategies—Adaptive Trajectory Resampling (ATR) and Self-supervised Trajectory Masking (STM)—are thoughtfully designed for trajectory data and show consistent benefits in ablations.\n- Robust Performance: UniTraj consistently outperforms existing baselines, including zero-shot transfer across cities, demonstrating strong cross-domain generalization.\n- Novel Application of Pretraining Strategies: While inspired by techniques from vision and NLP, ATR and STM are well-adapted to the trajectory domain and have not been widely applied at this scale.\n\nWeaknesses:\n- Potential Data Bias: WorldTrace is constructed from OpenStreetMap GPS traces, which are crowd-sourced and known to be biased toward car-based travel in urban regions, particularly in North America and Western Europe. This could limit the model’s generalization to underrepresented regions or modes of transport (e.g., walking, cycling, transit).\n- No Multi-Agent Modeling: Inter-agent interactions (e.g., traffic, crowds) are not modeled, which limits applicability to domains where joint behavior is essential.\n- Methodologically Incremental: ATR and STM are useful adaptations, but not new techniques. The architecture is based on a standard Transformer encoder-decoder.", "was_revised": true, "final_justification": "The rebuttal addressed many of my original questions, especially around efficiency, geographic generalization, and length sensitivity. These responses increased my confidence in the applicability of UniTraj.", "rebuttal": "> W1\n\n----\n\nThank you for raising the important point regarding potential data bias. We agree that crowd-sourced data can exhibit geographic and modality bias. To address this concern, we provide two complementary pieces of evidence that demonstrate UniTraj’s generalization ability beyond those biases:\n\n1. We explicitly evaluated UniTraj on GeoLife and Grab-Posisi, which include mixed transportation modes beyond the WorldTrace dataset, including walking, cycling, and public transit. As shown in **Table 2** of the main results section, UniTraj achieves competitive performance on these datasets. It indicates that the model effectively generalizes to non-motorized movement patterns and non-developed regions, even without being trained on those exact distributions.\n\n2. To further investigate the potential geographic bias, we analyzed UniTraj’s performance across different continents in the WorldTrace dataset. The results are summarized below:\n\n   |            | North America | Asia  | Europe | Africa | Oceania | South America |\n   | ---------- | ------------- | ----- | ------ | ------ | ------- | ------------- |\n   | MAE        | 10.58         | 9.42  | 10.03  | 10.61  | 10.12   | 9.98          |\n   | RMSE       | 13.84         | 12.38 | 13.29  | 13.46  | 13.17   | 12.89         |\n   | Percentage | 67.97         | 24.68 | 4.40   | 0.20   | 1.52    | 1.23          |\n\nThese results show that no region suffers from severe degradation, and in fact, performance in Asia and South America is comparable or better than that in North America or Europe. This suggests that UniTraj does not overfit to overrepresented regions, and that its self-supervised, region-agnostic design (e.g., no POIs, no road networks) allows for stable cross-continental generalization.\n\n\n\n\n> W2\n\n----\n\nWe appreciate the reviewer highlighting this limitation. We agree that explicitly modeling inter-agent interactions will improve the performance of the trajectory model. We would like to clarify that the core motivation for making this decision is as follows:\n\n- Our current design is motivated by the need to establish a strong, generalizable, and region-agnostic framework. We hope it can be usable across domains and cities where inter-agent annotations are unavailable or infeasible to collect at scale.\n- Modeling multi-agent interactions typically requires high-resolution temporal alignment, shared agent IDs, and consistent scene-level context, which are often missing in large-scale, real-world GPS datasets.\n- Despite modeling trajectories individually, UniTraj can achieve remarkable performance. This suggests that our self-supervised trajectory learning framework captures meaningful dynamics even without explicit interaction modeling.\n- UniTraj is designed with a modular architecture: while it focuses on per-agent trajectory encoding, inter-agent relations could be integrated in downstream pipelines. This allows for plug-and-play compatibility with interaction-aware components without retraining the backbone.\n\n\n\n\n\n> W3\n\n----\n\nWe thank the reviewer for this observation and valuable comment. We agree that the core components of UniTraj are built upon established foundations. However, we would like to clarify that the methodological novelty of UniTraj as outlined below:\n\n- **Domain-Specific Innovations:**  We designed a unique pre-training strategy based on the unique properties of trajectories. ATR  is a novel combination of multi-scale and interval-consistent resampling, tailored to address the heterogeneity of real-world GPS trajectories, including variable sampling intervals, missing points, and inconsistent granularity. We adapted RoPE to encode both spatial and temporal relationships jointly, rather than treating positions as a single sequence, thereby capturing the continuous and multi-dimensional nature of trajectory data.\n\n- **Unified Pretraining:** We also introduce four complementary masking strategies (STM), which are uniquely adapted to trajectory semantics. This is not a direct reuse of BERT-style masking but a task-aware design. UniTraj brings together several trajectory-specific pretext tasks (e.g., next-location prediction, trajectory completion) under a single framework, enabling the model to learn diverse mobility patterns across different spatial and temporal scales.\n\n- **Empirical Impact:** Despite being an incremental architecture, our approach substantially advances in trajectory modeling, as evidenced by superior performance on diverse and challenging benchmarks.\n\nWe are committed to proving that, with the right pretraining strategy and data design, even a standard encoder-decoder architecture can achieve powerful generalization and reusability across multiple domains. \n\n\n\n> Q1\n\n----\n\nThank you for this insightful question regarding our architectural choices.\n\n1. **Why Not Encoder-Only (e.g., BERT)?**\n\nWe selected an encoder-decoder architecture to better support a broad range of trajectory modeling tasks within a single, unified framework. The encoder-decoder structure, with its “destruction and reconstruction” paradigm, allows the model to more effectively learn complex spatio-temporal dependencies in trajectory data. This design is also more flexible and plug-and-play, making it adaptable to diverse downstream tasks. Additionally, the decoder can act as a projector, further enhancing the model’s versatility for various applications.\n\n2. **Why Not  Causal Masking?**\n\nFor many trajectory-related tasks, especially classification or masked point recovery, utilizing both past and future context is highly beneficial. Standard causal masking restricts the model to accessing only past information, which limits its ability to recover missing or irregularly sampled trajectory points. By adopting a flexible masking strategy rather than strict causal masking, our model can seamlessly handle both prediction and imputation tasks, thus improving its overall utility and efficiency.\n\n\n\n> Q2\n\n----\n\nThank you for your interest in the dataset. We will **open-source the worldtrace dataset** and ensure that its release fully complies with all relevant copyright and privacy regulations. Data sharing will follow appropriate protocols to protect individual privacy and respect data usage agreements. \n\n\n\n> Q3, Q6\n\n----\n\nWe thank the reviewer for this important question. To evaluate performance across regions with different sample densities, we conducted a **continent-level analysis** using trajectories from each region. The results are summarized below:\n\n|            | North America | Asia  | Europe | Africa | Oceania | South America |\n| ---------- | -------- | ----- | ------ | ------ | ------- | ------ |\n| MAE        | 10.58         | 9.42  | 10.03  | 10.61  | 10.12   | 9.98          |\n| RMSE       | 13.84         | 12.38 | 13.29  | 13.46  | 13.17   | 12.89         |\n| Percentage | 67.97         | 24.68 | 4.40   | 0.20   | 1.52    | 1.23          |\n\nFrom the table, we can observe that despite large differences in data volume, UniTraj maintains stable error metrics across all regions, with no drastic degradation in low-sample settings. These results demonstrate that UniTraj maintains robust generalization capabilities, even in regions with limited training data. \nWe attribute this to the model’s region-agnostic design and pretraining objectives that effectively support generalization to underrepresented geographies, even when region-specific data is sparse.\n\n\n\n> Q4\n\n----\n\nWe thank the reviewer for raising this practical concern. We conducted a comparison of inference latency, memory footprint, and FLOPs among UniTraj and two representative baselines (TrajBERT and TrajFM). The experiment was set up on a computing server with an Nvidia GeForce RTX 2080 Ti GPU and Intel(R) Xeon(R) Silver CPU. The results are as follows:\n\n|          | Time 1k/ samples | Memory (MB) | FLOPs (M) |\n| -------- | ---------------- | ----------- | --------- |\n| UniTraj  | 0.131            | 4241        | 321.68    |\n| TrajBERT | 0.142            | 5785        | 644.57    |\n| TrajFM   | 0.139            | 6685        | 644.51    |\n\nFrom the above analysis, we can see that the three models have very similar inference times and can quickly complete the processing of thousands of samples. However, Unitraj requires approximately half the FLOPs of TrajBERT and TrajFM, and its memory usage is also significantly reduced. These results demonstrate that UniTraj is not only more efficient in terms of memory and computation but also provides faster inference, making it well-suited for large-scale and real-time trajectory applications.\n\n\n\n> Q5\n\n----\n\nWe thank the reviewer for this insightful question. To assess sensitivity to trajectory length, we grouped test samples by their length and computed the MAE and RMSE for each group:\n\n|      | All   | <60   | 60--120 | 120--180 | 180>  |\n| ---- | ----- | ----- | ------- | -------- | ----- |\n| MAE  | 10.25 | 9.91  | 9.39    | 9.89     | 10.60 |\n| RMSE | 13.43 | 13.05 | 12.53   | 13.04    | 13.79 |\n\nFrom the table, we can observe that uniTraj demonstrates stable performance across all trajectory length ranges, with optimal results on medium-length trajectories (60–180). There is only a slight decrease in performance for very short (<60) and very long (>180) sequences. For very short trajectories, the limited amount of information makes it challenging for the model to fully capture spatio-temporal dependencies. For very long trajectories, the increased sequence length introduces more variability and noise, making the modeling task more difficult.\nIt is worth noting that for long sequences (>180), the MAE increases by only about 3.5% compared to the overall average. This indicates that UniTraj maintains robust generalization even when handling trajectories with longer durations and higher complexity. In summary, UniTraj achieves consistently strong and stable performance across a wide range of trajectory lengths."}, {"review_id": "dZ0JcVdtzi", "reviewer": "Reviewer_MnX3", "summary": "This paper introduces UniTraj, a universal trajectory foundation model designed to overcome the limitations of existing trajectory modeling approaches—namely, task specificity, regional dependency, and sensitivity to data quality. The authors present three major contributions: (1) the construction of WorldTrace, a large-scale, diverse, and high-quality trajectory dataset sourced from over 70 countries and containing 2.45 million trajectories and 8.8 billion GPS points; (2) a set of novel pre-training strategies, including Adaptive Trajectory Resampling and Self-supervised Trajectory Masking, which enable robust learning from heterogeneous, sparsely sampled trajectory data; and (3) a flexible encoder-decoder architecture using spatial-temporal embeddings and rotary positional encodings for general-purpose trajectory modeling tasks.\nExtensive experiments across recovery, prediction, classification, and generation tasks demonstrate UniTraj's superior performance in both zero-shot and fine-tuning scenarios. Ablation studies validate the effectiveness of each pre-training component, and the model shows strong generalization across datasets with varying quality and geographic contexts. The work is a substantial and timely contribution to spatiotemporal AI, bringing foundation model thinking to the mobility and trajectory modeling domain. The ideas are novel, well-motivated, and empirically validated, making it a strong candidate for acceptance at NeurIPS.", "questions": "(1) What are the model’s training and inference times on large-scale datasets? Could the authors provide FLOP or runtime comparisons against baselines?\n(2) How is user privacy preserved? What safeguards exist beyond removing identifiers, especially when releasing samples or models?", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strength \n(1) The paper tackles a critical and timely challenge in trajectory modeling: building a universal foundation model that generalizes across tasks, geographies, and data qualities. The authors clearly identify the fragmentation in current approaches and the missed opportunity to bring foundation model principles—successful in NLP and vision—to the mobility domain. \n(2) A key contribution is the curation of the WorldTrace dataset, which provides an unprecedented scale (2.45M trajectories, 70 countries) and diversity for trajectory modeling. \n(3) The proposed pre-training strategies are carefully designed for the trajectory domain’s unique characteristics, including irregular sampling and incomplete data. The dynamic resampling technique handles varying trajectory lengths, while the masking strategies simulate missing points in both local and global contexts. \n(4) UniTraj achieves consistent state-of-the-art results across a wide range of tasks and datasets. \n\nWeakness\n(1) The model does not provide any analysis of computational efficiency or inference latency, which is crucial for deployment.\n(2) The paper lacks discussion of ethical implications or potential privacy risks related to GPS trace use and trajectory prediction.", "was_revised": true, "final_justification": "After read the rebuttal of the authors and the discussion, I decide to keep my score unchanged. The authors have addressed my concerns on computational efficiency and data privacy issues during model training. Generally I think this is a good paper with clear contritions on universal trajectory foundation model which has broad application scenarios. I recommend to accept the paper.", "rebuttal": "> W1\n\n------\n\nThank you for your comment regarding computational efficiency and inference latency. While the main submission prioritized modeling and generalization, we now provide evidence that UniTraj offers superior computational efficiency over representative trajectory models:\n\n| **Model** | **Paras (M)** | **Inference Time (s/1k) ↓** | **Memory (MB) ↓** | **FLOPs (M) ↓** |\n| --------- | ------------------ | --------------------------- | ----------------- | --------------- |\n| UniTraj   | 2.38               | 0.131                       | 4241              | 321.68          |\n| TrajBERT  | 3.30               | 0.142                       | 5785              | 644.57          |\n| TrajFM    | 3.28               | 0.139                       | 6685              | 644.51          |\n\nThe above result was conducted on  a server with Nvidia GeForce RTX 2080 Ti GPU and Intel(R) Xeon(R) Silver CPU. We can gain the following insights:\n\n- **Parameter Efficiency:** UniTraj requires significantly fewer parameters than both TrajBERT and TrajFM, reducing memory and storage demands during deployment.\n- **Inference Speed:** All three models achieved very fast inference speeds, with less than 0.14 seconds per 1,000 samples.\n- **Memory Usage:** UniTraj consumes less memory than both TrajBERT and TrajFM under identical batch and sequence settings.\n- **Lower Computational Cost:** UniTraj has the lowest FLOPs, indicating better computational efficiency.\n\nThese results demonstrate that **UniTraj achieves superior computational efficiency and faster inference with a much smaller model footprint**, making it better suited for real-world deployment scenarios, especially those with hardware or latency constraints.\n\n\n\n> W2\n\n------\n\nWe appreciate the reviewer’s important observation regarding ethical and privacy considerations. While the main paper focused on technical contributions due to space constraints, we also carefully considered the ethical and privacy aspects of trajectory data usage in  **Appendix A.4**. Below are key clarifications:\n\n1. **Source and Anonymity:**  All GPS trajectories used in our work were collected from OpenStreetMap’s public GPS trace platform, which operates under the Open Data Commons Open Database License (ODbL). These traces are user-contributed and anonymized, containing no personal identifiers, and are limited to public movement paths (e.g., roads, highways).\n2. **No Individual-Level Modeling:** UniTraj does not infer or use any user identity or metadata. It models aggregate spatio-temporal patterns, not individual behavior, making re-identification infeasible.\n3. **Model Application Scenarios** The model is trained and evaluated exclusively on transportation-scale movements, not on private or personal location histories. Its intended applications are in domains such as urban mobility analysis, logistics, and transportation planning, not surveillance or individualized tracking.\n\nWe will include a dedicated description in the revised manuscript to explicitly discuss these ethical considerations and privacy safeguards. Thank you again for bringing this critical aspect to our attention.\n\n\n\n> Q1\n\n------\n\nThank you for highlighting the importance of computational efficiency at scale. We have benchmarked UniTraj and recent baselines (TrajBERT, TrajFM) on large-scale datasets, reporting both FLOPs and runtime metrics. The experiment was set up on a computing server with an Nvidia GeForce RTX 2080 Ti GPU and Intel(R) Xeon(R) Silver CPU.\n\n| **Model** | **Time per 1k Samples (s)** | **Memory (GB)** | **FLOPs (M)** |\n| --------- | --------------------------- | --------------- | ------------- |\n| UniTraj   | 0.131                       | 4,241           | 321.68        |\n| TrajBERT  | 0.142                       | 5,785           | 644.57        |\n| TrajFM    | 0.139                       | 6,685           | 644.51        |\n\nSpecifically, we can see that the three models have very similar inference times and can quickly complete the processing of thousands of samples. In addition,  UniTraj requires substantially fewer FLOPs than both baselines, reflecting its computational efficiency. Therefore, UniTraj offers superior efficiency and scalability for both training and inference, making it well-suited for deployment on large-scale trajectory data. \n\n\n\n> Q2\n\n----\n\nThank you for raising the crucial issue of user privacy and identifiers. Below are the safeguards we incorporated or the dataset held:\n\n- **Data Source and Licensing:** All trajectory data used in UniTraj comes from **OpenStreetMap (OSM) GPS traces**, which are **public, anonymized, and shared under the ODbL**. The original data contains **no user identifiers, device IDs, or metadata** that can be used to trajectory individuals.\n\n- **Preprocessing for De-Identification:** During dataset construction (see Appendix A), we applied filtering and spatial-temporal normalization, including short trips, loops, and localized trajectories. In addition, we also use map-matching to align raw points to road networks, further abstracting fine-grained individual movement.\n\n- **Sample Released:** The publicly shared dataset includes only a processed, curated subset of trajectories.\n\n- **Model-Level Safeguards:** UniTraj learns from aggregated movement patterns, not individual users, which cannot reconstruct or trace back individual identities or raw trajectories. Moreover, we do not publish fine-tuned models on private or proprietary datasets, avoiding potential overfitting to any local patterns."}, {"review_id": "DxqCoSSu9x", "reviewer": "Reviewer_GMYB", "summary": "This paper introduces UniTraj, a general-purpose trajectory foundation model designed to overcome the limitations caused by task dependency, regional specificity, and inconsistent data quality. The authors first construct a large-scale dataset called WorldTrace, which includes over 2.45 million trajectories and 8.8 billion GPS points collected from 70 countries. This dataset focuses on motorized movement and is carefully processed using 1Hz sampling normalization and map matching to ensure quality and consistency.\n\nTo train the model, the authors propose a two-part pre-training strategy. The first part is Adaptive Trajectory Resampling (ATR), which resamples trajectory points based on their length and ensures more uniform time intervals. The second part is Self-supervised Trajectory Masking (STM), which combines several masking strategies, random, block-based, keypoint-based, and endpoint masking, to improve robustness and generalization. These strategies allow the model to learn rich and transferable representations.\n\nThe architecture of UniTraj is based on a flexible Transformer framework. It uses RoPE and masked modeling techniques during pre-training. The encoder is designed to be versatile and can be used both for zero-shot inference and fine-tuning on specific downstream tasks. As a result, UniTraj demonstrates strong performance across various trajectory-related tasks, including reconstruction, prediction, classification, and generation. It consistently outperforms existing models, such as TrajFM and TrajBERT, particularly in zero-shot transfer settings across various geographic regions.", "questions": "The paper is generally well-structured and well-written. The following question is what I wanted to clarify:\n\n- For zero-shot results in Table 2, the authors mentioned “In the zero-shot setting, UniTraj achieves remarkable results, confirming it effectively captures transferable spatio-temporal patterns without requiring additional fine-tuning. The performance difference becomes particularly instructive when analyzing low-quality datasets like GeoLife and Grab-Posisi, with their highly irregular sampling intervals and multiple travel modes.”\n    - It is interesting to note the quality of data, and this conjecture holds. For example, GeoLife can cleaned up with their modes and may re-sampling is helpful to make the data more formatted as expected.", "limitations": "Trajectory data often have privacy issues, but OSM data and their motorized traces seem to public and well-granulated. The authors mentioned some issues in E.2., but it is better to include them in the main text in my opinion.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n\n- The concept of a general trajectory foundation model is convincingly supported by scale and experiments.\n- The WorldTrace dataset is valuable for the community due to its size, diversity, and quality.\n- The model shows strong zero-shot performance in reconstruction, classification, and generation tasks.\n- Experiments are carefully designed, including ablations and analysis of geographic bias.\n\n### Weaknesses\n\n- The model architecture uses many known techniques (RoPE, masked modeling, encoder-decoder), so structural novelty is limited.\n- The design ignores contextual features like POIs or road networks, and focused on motorized data, which may reduce performance in some settings.\n- The need for high-end hardware (e.g., A100 GPUs) may limit accessibility and reproducibility.", "was_revised": true, "final_justification": "The paper is generally well-written.\n\nAlthough some components seem not to be novel from existing research literature, I respect the FM's study that has a broader technical impact in various fields; after discussing my concerns with the authors, I'll keep my score to push this paper to be accepted for technical discussions.", "rebuttal": "> W1\n\n----\n\nThank you for your valuable comment regarding the architectural novelty of our work. UniTraj indeed leverages established components such as RoPE, masked modeling, and the encoder-decoder framework. But our contribution goes significantly beyond their direct application. We have customized and innovatively extended these components to address the unique challenges of trajectory data. Specifically:\n\n- **Adaptive Trajectory Sampling:** We introduce the adaptive sampling mechanism to dynamically handle the varying density and irregular intervals typical in trajectory data. This allows the model to better represent dense urban traces and sparse intercity movements, enhancing its generalization and robustness.\n- **Trajectory-Aware Masked Modeling:** Unlike conventional masked modeling in NLP or vision, we designed four dedicated masking strategies specifically for spatio-temporal sequences with irregular sampling, frequent missing points, and noise, ensuring effective learning from real-world GPS traces.\n- **Domain-Specific Encoding:** We adapted RoPE to encode both spatial and temporal relationships jointly, rather than treating positions as a single sequence, thereby capturing the continuous and multi-dimensional nature of trajectory data.\n- **Unified and Multi-Granularity Pretraining:** UniTraj brings together several trajectory-specific pretext tasks (e.g., next-location prediction, trajectory completion) under a single framework, enabling the model to learn diverse mobility patterns across different spatial and temporal scales.\n\nIn summary, we construct UniTraj based on proven deep learning modules, primarily driven by extensive experimental validation and customized modifications based on trajectory data.  The unified design represents a potential direction for trajectory data modeling.\n\n> W2\n\n----\n\nThank you for highlighting the limitation regarding contextual features and the focus on motorized trajectory data. We appreciate this important perspective and would like to clarify our design choices:\n\n- **Generality:** Our primary objective was to develop a region-independent and modality-agnostic trajectory foundation model that can learn from raw GPS traces with minimal reliance on external data sources. While incorporating such contextual features can improve performance in localized settings, they limit transferability and are not consistently available across regions, particularly in developing countries or less-mapped areas.\n- **Model Robustness to Diverse Settings:** Our self-supervised masking strategies and adaptive resampling help the model handle sparse, irregular, and low-speed trajectories effectively. Therefore, despite the focus on motorized data in WorldTrace, UniTraj demonstrates strong performance on multimodal datasets, such as GeoLife and Grab-Posisi, which include walking, biking, and mixed transport modes.\n- **Contextual Feature Integration:** We agree that incorporating contextual information such as POIs and road networks can further enhance model performance. Although our current work does not exploit these features, UniTraj is modular by design and readily extended to integrate such signals as additional inputs or embedding channels in future work.\n\n\n\n> W3\n\n----\n\nThank you for pointing out concerns about hardware requirements and reproducibility. We would like to clarify that our use of A100 GPUs was primarily for accessing the training and evaluation process on large-scale datasets, and is not a necessity for running UniTraj.  Inference and fine-tuning can be performed efficiently on widely available consumer hardware, such as an NVIDIA 2080Ti. Specifically, on 2080Ti, UniTraj holds 321.68 M FLOPs. Inference time: 0.131 seconds and Memory usage: 4,241 MB for 1000 samples.\n\nMost importantly, UniTraj is a lightweight architecture (2.38M parameters) that is significantly smaller and more efficient than prior baselines (e.g., TrajBERT and TrajFM), allowing smooth operation on non-server-grade GPUs and even for edge-level deployment with further compression. Below is our comparison of three representative models.\n\n| **Model** | **Paras (M)** | **Inference Time (s/1k) ↓** | **Memory (MB) ↓** | **FLOPs (M) ↓** |\n| --------- | ------------------ | --------------------------- | ----------------- | --------------- |\n| UniTraj   | 2.38               | 0.131                       | 4241              | 321.68          |\n| TrajBERT  | 3.30               | 0.142                       | 5785              | 644.57          |\n| TrajFM    | 3.28               | 0.139                       | 6685              | 644.51          |\n\n\n\n\n\n> Q1\n\n----\n\nThank you for your insightful observation regarding the impact of data quality and preprocessing. We agree that if sufficient prior information, such as mode labels or regularized sampling intervals, were available, data cleaning and resampling could further improve model performance. \n\nThis aligns directly with the motivation behind our ATR strategy, which is designed to handle irregular, noisy, and multimodal sampling patterns without requiring external priors.\nBy randomly constructing a large number of possible trajectory behaviors, we aim to evaluate UniTraj's robustness and transferability under various dynamic constraints.\n\nIn our manuscript, we provide a preliminary corollary to support this perspective. Specifically, if we can convert the existing trajectory dataset to a distribution that better matches the target city, the intrinsic uncertainty (entropy) of the modeling task is reduced, making the learning problem better posed and easier for any model, like UniTraj.\n\n\n\n> Limitation\n\n----\n\nThank you for raising the important issue of privacy in trajectory data. We fully agree that privacy is a fundamental concern in mobility data research. While our experiments utilize public and well-processed datasets such as OSM-based traces, which are less likely to contain personally identifiable information. We also recognize that privacy risks may be relevant.\n\nAs you pointed out, we discussed this issue in **Section E** of the appendix. In response to your suggestion, we will move and expand this discussion into the main text of the revised manuscript. Apart from general privacy risks related to trajectory data, we will also emphasize the handling of public and anonymous datasets in this study, as well as how to ensure privacy removal.\n\nThank you again for your suggestion, which will help us make the manuscript more comprehensive and responsible."}, {"review_id": "LkBAAxEHzJ", "reviewer": "Reviewer_KnAU", "summary": "This paper introduces a universal trajectory foundation model called UniTraj, which aims to address the limitations commonly found in existing trajectory modeling methods, such as task specificity, region dependency, and data sensitivity. To this end, the authors introduced three key innovations: first, they constructed a large-scale global trajectory dataset called WorldTrace, which contains trajectory samples from 70 countries. Second, they developed novel adaptive trajectory resampling and self-supervised trajectory masking pre-training strategies, enabling the model to learn robustly from heterogeneous data with different sampling rates and quality. Finally, they designed a flexible encoder-decoder model architecture to adapt to various trajectory tasks and effectively capture complex motion patterns. Extensive experiments demonstrate that UniTraj significantly outperforms existing methods across multiple tasks, both in zero-shot and fine-tuning settings, proving its exceptional scalability, adaptability, and generalization capabilities.", "questions": "1. In self-supervised pretraining, the authors set specific application ratio for four different masking strategies. Could the authors briefly explain how this ratio combination (e.g., 70% random masking) was determined? Is model performance sensitive to this ratio?\n\n2. The UniTraj uses RoPE to capture the sequential relationships between trajectory points. Considering that there are other position encoding schemes in Transformer models, why was RoPE chosen, and what are its specific advantages?\n\n3. From the performance table of the trajectory recovery task (Table 2), it can be seen that compared with high-quality datasets, the model has significantly higher errors on low-quality datasets (such as GeoLife and Grab-Posisi). Can the authors provide some qualitative analysis of the model's failure cases based on the existing experimental results?", "limitations": "Yes.", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The research objectives of this paper hold promising insights. The author identifies the preparations required and challenges faced in building a foundation model, and the paradigm may inspire subsequent related research.\n\n2. The authors constructed the WorldTrace dataset, which is unprecedented in trajectory analysis in terms of its scale and geographical diversity. This dataset provides a solid training foundation for the model in this paper, and also alleviates the problems of data scarcity and geographical bias faced by the community.\n\n3. The pre-training strategies (ATR and STM) proposed in this paper are designed based on the specific characteristics of trajectory data and can effectively address the common issue of inconsistent data quality in this kind of problem.\n\n4. The experimental section of this paper is comprehensive and well-founded. The authors evaluated the model on six different real-world datasets, including WorldTrace, thoroughly validating its cross-regional adaptability and supporting the claim that it is a universal foundation model.\n\n5. This paper provides detailed supplementary materials, including data acquisition, processing, and pre-training strategy details, which effectively help readers understanding.\n\nWeaknesses:\n\n1. Although the proposed WorldTrace dataset covers 70 countries, the distribution of the data is not representative of some parts of Africa and Asia. This bias may affect the generalizability of the model in underdeveloped regions or regions with sparse data.\n\n2. To achieve region independence, UniTraj was designed to use only the coordinates and timestamp information of trajectories, intentionally ignoring rich contextual information such as road networks, points of interest (POIs). This is a core design choice, but it may also result in a performance bottleneck.\n\n3. The paper mentions that the model uses an asymmetric encoder-decoder structure and provides the number of parameters. However, there is a lack of detailed reasons for choosing this particular asymmetric design over other configurations.", "was_revised": true, "final_justification": "This work proposes a new dataset and foundation model for trajectory data analysis. Considering the long-term scarcity of relevant datasets for research, I believe this work can bring a new perspective to the field. Therefore, I recommend accepting this paper.", "rebuttal": "> W1\n\n------\n\nWe appreciate the reviewer’s thoughtful observation regarding potential geographic imbalance in the WorldTrace dataset. We agree that ensuring global representativeness is crucial for universal modeling. As acknowledged in **Appendix E.1** (Limitations), some regions are less represented in WorldTrace due to public data availability and the current state of community-contributed GPS traces. \n\nDespite these imbalances, UniTraj still demonstrates remarkable generalization across a variety of regions, as evidenced by:\n\n1. As shown in **Table 2 and Table 3**, UniTraj performs strongly even in regions outside the dominant geographic areas, like Grab-Posisi (from Southeast Asia), contributors of WorldTrace.\n2. In zero-shot settings, UniTraj shows notable robustness on these datasets, demonstrating its cross-regional generalization capacity, even with partial underrepresentation.\n3. The self-supervised pretraining strategies (e.g., ATR and STM) were designed explicitly to mitigate region-specific overfitting by forcing the model to learn from varying sampling rates, motion patterns, and missing data. \n\n> W2\n\n------\n\nWe thank the reviewer for highlighting this critical design trade-off. Indeed, our decision to omit contextual features such as road networks and POIs was a **deliberate and principled choice** to align with our core objective: **developing a globally deployable and region-independent trajectory foundation model**. Our core motivation for acting in this manner is as follows:\n\n1. Many prior models rely on region-specific information (e.g., local road topologies, POIs, and semantics), which hinders transferability across cities or countries due to significant infrastructure, semantics, and data availability differences.\n2. Our goal was to train a universal backbone to generalize across diverse geographies and support plug-and-play adaptation without requiring aligned contextual data from every target region. As noted in Lines 240–245 of the main paper, this decision helps UniTraj avoid hard-coded dependencies and enables zero-shot generalization to the unseen areas.\n\nDespite this minimalist design, UniTraj achieves superior performance in zero-shot and fine-tuning settings across diverse datasets and tasks (see Tables 2 & 3, Figure 2). This demonstrates that the model effectively captures complex movement patterns and spatio-temporal dependencies using only core trajectory signals. It also generalizes robustly to unseen regions, which would not be feasible if region-dependent features were required.\n\n\n\n\n\n> W3\n\n------\n\nThank you for your constructive feedback and for highlighting the need to clarify our reasoning behind the asymmetric encoder-decoder structure in UniTraj.\n\n- **Motivation:** Our objective was to balance model expressiveness with computational efficiency during pre-training on large-scale trajectory data. The encoder is tasked with learning rich representations from observed trajectory segments, and thus we allocate more layers and capacity to it. The decoder is used only during pre-training to reconstruct the masked trajectory points. During downstream tasks, the decoder is discarded, and only the encoder is used. This design enables efficient fine-tuning and inference.\n- **Empirical Support:** We conducted model ablation and parameter sensitivity experiments (**Appendix D.4, Figure 3c**), which show that increasing the encoder depth improves representation quality up to a saturation point, while deeper decoders provide diminishing returns during reconstruction. This design reduces overfitting risk and improves generalization.\n\n\n\n> Q1\n\n------\n\nThank you for your insightful question regarding the masking design. The masking ratio combination was determined empirically through a grid search on the validation set using the WorldTrace and Chengdu datasets. We observed that prioritizing random masking (50%) provided the strongest generalization and robustness for local and global dependencies, while the remaining ratios introduced targeted challenges relevant to specific downstream tasks.\n\nResults in Section 5.4 (see Figure 3(d)) show that model performance is moderately sensitive to the masking ratio. Using less than 50% random masking or omitting certain masking types degraded performance, while the chosen combination achieved the best balance between reconstruction accuracy and transferability.\n\n\n> Q2\n\n--------\n\nThank you for highlighting the importance of discussing  RoPE.\n\nWe selected RoPE over other positional encoding schemes for the following specific reasons:\n\n- **Relative Position Awareness**: Unlike sinusoidal or absolute positional embeddings, RoPE encodes relative distances between points via rotation in embedding space, which is critical for trajectory data where relative motion (e.g., turns, acceleration) matters more than absolute position indices.\n\n- **Continuous and Scalable**: RoPE naturally supports continuous and extrapolatable temporal sequences, making it better suited for modeling variable-length trajectories without discrete position limits or index overflow.\n\n\n\n> Q3\n\n------\n\nWe are very grateful to the reviewers for their meticulous reading and valuable insights. The higher errors on low-quality datasets such as GeoLife and Grab-Posisi are primarily due to their **irregular and sparse sampling intervals**, **higher levels of GPS noise**, and greater diversity in travel modes. Qualitative analysis of failure cases reveals two main patterns:\n\n- Infrequent or uneven sampling often leads to large spatial gaps between points, making it challenging for the model to accurately reconstruct complex maneuvers (e.g., sharp turns or mode switches) that occur between observations.\n- Poor GPS quality and inconsistent data can introduce outliers or implausible points, which may mislead the model during both pretraining and recovery, resulting in larger reconstruction errors.\n\nIn addition, from our ablation studies (**Table 4**), we observed that removing **dynamic resampling** or **key point masking** causes **sharp degradation** on these datasets, indicating that **temporal heterogeneity and spatial-level geometry** are major failure modes. In some failure cases, the model **over-smooths short, abrupt turns** or **fails to reconstruct loops or detours**, especially when large sections are masked."}]

Author rebuttal:

Meta-review: The manuscript introduces UniTraj, a novel trajectory foundation model addressing task specificity, regional dependency, and data quality issues. It presents the WorldTrace dataset, innovative pretraining strategies, and a Transformer-based architecture with RoPE, achieving strong zero-shot and cross-regional performance.

Reviewers recognize the dataset’s scale, pretraining innovations, and comprehensive evaluation but note limited architectural novelty, geographic bias in WorldTrace, and insufficient ethical discussion (e.g., re-identification, surveillance risks). The rebuttal addresses efficiency, bias, and privacy concerns, committing to an ethics section.

I recommend acceptance. However, the authors should revise the ethics-related section addressing re-identification, surveillance, consent, and bias. The paper would make substantial contributions to spatiotemporal AI if ethical concerns are resolved.
Decision: Accept (poster)