DiffTraj: Generating GPS Trajectory with Diffusion Probabilistic Model
Authors: Yuanshao Zhu
Yongchao Ye
Shiyao Zhang
Xiangyu Zhao
James Yu
Venue: NeurIPS 2023 poster (2023)
Keywords: Trajectory Generation
Diffusion Model
Urban Computing
Spatial-temporal Data Mining
TL;DR: We propose a spatial-temporal diffusion probabilistic model for GPS trajectory generation.

Abstract: Pervasive integration of GPS-enabled devices and data acquisition technologies has led to an exponential increase in GPS trajectory data, fostering advancements in spatial-temporal data mining research. Nonetheless, GPS trajectories contain personal geolocation information, rendering serious privacy concerns when working with raw data. A promising approach to address this issue is trajectory generation, which involves replacing original data with generated, privacy-free alternatives. Despite the potential of trajectory generation, the complex nature of human behavior and its inherent stochastic characteristics pose challenges in generating high-quality trajectories. 
In this work, we propose a spatial-temporal diffusion probabilistic model for trajectory generation (DiffTraj). This model effectively combines the generative abilities of diffusion models with the spatial-temporal features derived from real trajectories. The core idea is to reconstruct and synthesize geographic trajectories from white noise through a reverse trajectory denoising process. Furthermore, we propose a Trajectory UNet (Traj-UNet) deep neural network to embed conditional information and accurately estimate noise levels during the reverse process. Experiments on two real-world datasets show that DiffTraj can be intuitively applied to generate high-fidelity trajectories while retaining the original distributions. Moreover, the generated results can support downstream trajectory analysis tasks and significantly outperform other methods in terms of geo-distribution evaluations.

DiffTraj: Generating GPS Trajectory with Diffusion Probabilistic Model
Yuanshao Zhu Yongchao Ye', Shiyao Zhang', Xiangyu Zhao?; James J.Q. Yu?
Southern University of Science and Technology
2 City University of Hong Kong
Uniierriity of York
(zhuys2019, (ma22868))mmai) sustech. edu.
zhangay@sustech. edu. cn xi kianzhaoccityu.o edu hk j james yu@york. ac. uk
Abstract
Pervasive integration of GPS-enabled devices and data acquisition technologies has led to exponential increase in GPS trajectory data, fostering advancements spatial-temporal data mining research. Nonetheless, GPS trajectories contain personal geolocation information, rendering serious privacy concerns when working with raw data. A promising approach to address this issue is trajectory generation, which involves replacing original data with generated, privacy-free alternatives. Despite the potential of trajectory generation, the complex nature of human behavior and its inherent stochastic characteristics pose challenges in generating high-qualig trajectories. In this work, we propose a diffusion probabilistic model for trajectory generation (DiffTraj). This model effectively combines the generative abilities of diffusion models with the spatial-temporal features derived from real trajectories. The core idea to reconstruct and synthesize geographic trajectories from white noise through a reverse trajectory denoising process. Furthermore, we propose a Trajectory UNet (Traj-UNet) deep neural network to embed conditional information and accurately estimate noise levels during the reverse process. Experiments on two real-world datasets show that DiffTraa can be intuitively applied generate high-fidelity trajectories while retaining the original distributions. Moreover, the generated results support downstream trajectory analysis tasks and significantly outperform other methods in terms of geo-distribution evaluations.
Introduction
GPS trajectory data play erucial role in numerous spatial-temporal data mining applications, including urban traffic planning, business location selection, and travel time estimation [43, 42). Despite the substantial progress in urban applications through GPS trajectory analysis, few attention was paid to data accessibility and privacy concerns [41]. For example, real-world trajectory data, which contain sensitive personal geolocation information, raise significant privacy issues when using directly [38, 44]. Additionally, the time-consuming and labor-intensive nature of trajectory data collection leads to challenges in obtaining accessible, publicly available datasets that preserve privacy. Therefore, it is essential to develop methods that facilitate the efficient use of trajectory data in urban applications while safeguarding personal privacy. One promising approach involves generating synthetic trajectories by learning from real trajectory distributions and replacing privacy-sensitive
"Coresponding author
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

real trajectories with these synthetic counterparts. The generated trajectories allow for equivalent data analysis outcomes and support high-level decision decision-making without compromising privacy.
However, generating GPS trajectory that accurately reflect real-world distributions will encounter the following challenges in practice. First, most cities contain various regions with diverse functions and population densities, resulting in non-independent and identically distributed trajectories among these regions [44, 40J. This complex distribution makes it difficult to learn global trajectory model that captures regional nuances. Second, the inherent stochastic nature of human actívities makes each trajectory unique, posing challenges in modeling and predicting individual movement patterns [41, 16]. Third, external factors, such as traffic conditions, departure time, and local events, significantly influence personal travel schedules [36]. Accounting for these factors and understanding their impact on the correlations between individual GPS locations within a trajectory adds to the complexity of modeling trajectory generation accurately.
To address the above challenges, we propose a trajectory generation method based on the spatial-tempore diffusion probabilistic model, which can effectively capture the complex behaviors of real-world activities and generates high-quality trajectories. The core idea behind this method is to perturb the trajectory distribution with noise through a forward trajectory diffusion process and then recover the original distribution by learning the backward diffusion process (denoising), resulting in highly flexible trajectory generation model [11, 28]. We propose this framework based on three primary motivations: (i) The diffusion model is a more reliable and robust method of generation than canonical methods [35]. (ii) Human activities in the real world exhibit stochastic and uncertain characteristics [18, 10), and the diffusion model reconstructs data step-by-ste from random noise, making it suitable for generating more realistic GPS trajectory. (iii) Since the diffusion model generates trajectories from random noise, it eliminates the risk of privacy leakage.
While the diffusion model shows promise for generating high-quality trajectories, it is non-trivial task when applied to GPS trajectory generation. Firstly, the spatial-temporal nature of trajectory data presents complexity to the diffusion process, requiring the model to consider the spatial distribution of GPS points and the temporal dependencies among them. Secondly, external factors such as traffic conditions, departure time, and regions can significantly impact human mobility patterns [44], posing challenges for modeling these factors within the diffusion framework. Finally, the inherently stochastic nature of human activities demands that the diffusion model capable of capturing a wide range of plausible trajectory patterns, which can be difficult due to the need to balance diversity and realism in the generated trajectories. Addressing these challenges crucial for successfully applying the diffusion model to GPS trajectory generation and ensuring crivacy-preserving accurate, and computationally efficient outcomes.
Building on the identified motivations and addressing the limitations of applying the diffusion model to trajectory generation, we propose a Diffusson probabilistic model Trajectory generation (DiffTraj) framework. This framework can generate infinite trajectories while preserving the real-world trajectory distribution. DiffTraj incorporates apatial-temporte modeling of raw trajectories without the need for additional operations, allowing for direct application to trajectory generation tasks. Crucially, the synthetic trajectories generated by DiffTraj ensure high generation accuracy while maintaining their utility, offering a promising solution to privacy-preserving GPS trajectory generation. To summarize, the contributions of this work are concluded as follows:
We propose a DiffTraj framework for trajectory generation, leveraging the diffusion model to simulate real-world trajectories while preserving privacy. In addition, this framework can be applied to generate high-fidelity trajectory data with high efficiency and transferability. To the best of our knowledge, our work is the first exploration of trajectory generation by the diffusion model.
We design a novel denoising network structure called Traj-UNet, which integrates residual blocks for spatial-temporal modeling and multi-scale feature fusion to accurately predict the noise level during denoising- Meanwhile, Traj- UTaj-UNet integrates Wide and Deep networks [6] to construct conditional information, enabling controlled trajectory generation.
We validate the effectiveness of DiffTraj on two real-world trajectory datasets, showing that the proposed methods can generate high-fidelity trajectories and preserve statistical properties. At the same time, the generated trajectories can support downstream trajectory analysis tasks with replaceable utility and privacy-preservin.
2

---

2 Related Work
Trajectory Data Synthesizing. Existing methods for trajectory data can be generally divided into two main categories: non-generative and generative [4]. Non-generative methods include perturbing real trajectories [1, 37] or combining different real trajectories [21]. Adding random or Gaussian perturbations protects privacy but compromises data utility by altering the spatial-temporal characteristics and data distribution. Striking a balance between trajectory utility and privacy protection is challenging [4]. Although Mehmet et al. generated forged trajectories by mixing different trajectories, this method relies on massive data and sophisticated mixing processes [21].
Besides, the principle of the generative method is to leverage deep neural networks that learn the Rempialtemmpoal distribution underlying the real data. New trajectory data are therefore generated by sampling from the learned distribution. Liu et al. proposed a preliminary solution that employs generative adversarial networks (GAN) for trajectory generation, yet it failed go further towards a detailed design[33], Subsequently, some works divided the city map into grids and performed trajectory generation by learning the distribution of trajectories among the grids [38, 24). However, there is a trade-off' between generation accuracy and grid size. Meanwhile, researchers used the image generation capability of GAN convert the trajectory data into images for time-series generation [4, 31J. while the transformation between images and trajectories imposed an additional computational burden. Compared with previous methods, DiffTraj uses the diffusion model for trajectory generation, which can better explore the spatial and temporal distributions without additional data manipulation.
Diffusion Probabilistic Model. The diffusion model is a probabilistic generative model, which was first proposed by Sohl-Dickstein et al. [27] and then further improved by Ho et al. [11] and Song et al. [29]. A typical diffusion model generates synthetic data via sequential processes, i.e., forward process that gradually perturbs the data distribution by adding noise on multiple scales and reverse process that learns how recover the data distribution(35] In addition, researchers have made extensive attempts to improve generative sample quality and sampling speed. For example, Song et al. proposed a Mon-Markovian diffusion process to reduce the sampling steps [28]. Nichol et al. proposed to learn the variance of reverse processes allowing fewer sampling steps [22], and Dhariwal al. searched for the optimal structure of the reverse denoising neural network to obtain better sampling quality. As a new type of advanced generative model, diffusion models have achieved superior performance over alternative generative models in various generative tasks, such as computer vision [25], natural language processing [15], multi-modal learning [2, 19], and traffic forecasting [32]. Nevertheless, the diffusion probabilistic model calls for efforts in apatial-tempont trajectory data generation. To the best of our knowledge, this work is a pioneering attempt that uses diffusion probabilistic model to generate GPS trajectory.
3 Preliminary
In this section, we first introduce the definitions and notations we use in this paper and then briefly present the fundamentals of the diffusion probabilistic model.
3.1 Problem Definition
Definition 1: (GPS Trajectory). A GPS trajectory is defined as sequence of continuously sampled
private GPS location points, denoted by x = {pI, P2.... }. The i-th GPS point is represented as a tuple p. = [lat, Ing.l. where Ing, and lat, denote longitude and latitude, respectively.
Problem Statement: (Trajectory Generation). Given a set of real-world GPS trajectories, X' = {x², x² x...... ..... ). where each x' = {pt, P P', is sequence of private GPS location points. The objective of the trajectory generation task is to learn a generative model, G, that can generate synthetic GPS trajectories, y { y² ........ yy", where each y² {qi. of .., } such that:
The generated trajectories y² preserve the spatial-temporal characteristics and distribution of the real trajectories x'. In practice, there are various physical meanings, such as the regional distribution of trajectory points, the distance between successive points, etc.
Similarity:
* Utility: The generated trajectories y maintain utility for downstream applications and analyses.
Privacy: Privacy preserved, meaning that the generated trajectories y do not reveal sensitive information about the individuals associated with the real trajectories x".
3

---

3.2 Diffusion Probabilistic Model
The diffusion probabilistic model is powerful and flexible generative model, which has gained increasing attention in recent years for its success in various data generation tasks [11, 28, 29]. In general, the diffusion probabilistic model consists of two main processes: forward process that gradually perturbs the data distribution with noise, and reverse (denoising) process that learns to recover the original data distribution.
Forward process. Given a set of real data samples 2o ~ q (æo), the forward process adds T time-steps of Gaussian noise N (-) it, where T is an adjustable parameter. Formally, the forward process can be defined as Markov chain from data zo to the latent variable æ7:
q ( 1o) = II q (x, ( -=1
x(-1); q (xx x1-1) = (zi V B,0r- B,1)
(I)
< <2 <.. 3T) the corresponding variance schedule. Since it is impractical to back-propagate the gradient by sampling from a Gaussian distribution, we adopt reparameterization trick to keep the gradient derivable [11] and the æ, can be expressed as 2t = v a + /1 are, where ~ NN I) and a; = III 31).
Here, {Be E (0,
Reverse process. The reverse diffusion process, also known the denoising process, aims to recover the original data distribution from the noisy data æy N00 I). Accordingly, this process can be formulated by the following Markov chain:
T
Po (Zorr) = (xrr) II [=1
Po (xt-1 xx); Po (x1-1 xx)) N (x:11: Mo (xt, t) oo (2x, t)²
I))
(2)
where Me (z4, t) and oo (zt, t) are the mean and variance parameterized by 0, respectively. Based on }3 B1, the parameterizations of 14g and o
the literature [11], for any Br T-a; defined by:
1-or-1 Be (1
1) and
Br co (3 1) and as (ax,,) = A3
Pg (wt, 1)
2z
vou
v1 āz
4 DiffTraj Framework
In this section, we present the details of the DiffTraj framework shown in Fig. 1, which applies the diffusion model to trajectory generation. The primary goal of Diff is to estimate the real- world trajectory distribution (zo xx6)) using a parameterized model pe (x) xg"). Given a random noise 2T (0, I), DiffTraj generates synthetic trajectory æ.j. conditioned the observations 25 The reverse (generation) process from the Eq. (2) can be reformulated for DiffTraj as:
Po [x/_ -; x == N (2%-1: M4 (x x aa (x); x00)²
(4)
In this context, My serves as a conditional denoising function that takes the conditional observation 2o as input. We then extend the Eq. (3) to account for the conditional observation as follows:
Hg (xt.t
2 = Mo (zi, t, co (a t xx)))
88 (z),t
x de (at,
Training. In practice, the reverse trajectory denoising process can be summarized as learning the Gaussian noise eg (xf.t) | z through æf, t and xg (refer to Fig. 1). Then solving μ4 (z), x;6)) according to Eq. (3). This process can be trained by following objective [11]:
min L(O) == min E.zo.r ||6-
(Vorso + v1 ā+e, 1) 1)
The above equations show that the core of training the diffusion model is to minimize the mean squared error between the Gaussian noise and predicted noise level eø (æt ==)).
Sampling. Given the reverse process presented in Sec. 3.2, the generative procedure can be summarized sampling the zं -V I), and iterative sampling based øt-1 pe (zf_1 æf, x)). The final sampled output zz.

---

Sinusoidal
Xr
po 1k
Traj-UNet
EB
Xt-1
00
Deep
An illustration for trajectory generation with diffusion model. (Left) Forward and the reverse process (multiple GPS trajectories presented). (Right) Coupled the neural network model structure for reverse denoising.
Figure 1:
4.1 Traj-UNet Architecture
While developing the DiffTraj framework, accurately predicting the noise level at each diffusion time step non-trivial. model must capture these complex spatial-tempores dependencies accurately predict the noise level at each diffusion time step. Inaccurate noise predictions can lead to excessively noisy trajectories, resulting in unrealistic patterns. Thus, we employ Traj-UNet to predict the noise of each diffusion time step in Diff'rraj, i.c.. Ep (æ t x5"). Specifically, we construct a network based on the UNet architecture [26]. which has been extensively used in generative applications involving diffusion models [35, 20, 5]. The Traj-UNet is composed of two parts: down-sampling and up-sampling, each featuring multiple ID-CNN- stacked residual network blocks (Resnet block) with kernel size of 3 in each layer. An attention mechanism-basee transitional module is integrated between the two parts [30] (Detailed information about Traj-U can be found Fig. of the Appendix A). To enhance the learning of noise for each diffusion time step, Traj-UNet incorporates time step embedding and then fed to each block. Specifically, we employ Sinusoidal embedding [30] to represent each as 2e-dimensional vector. Subsequently, apply two chared-paramesd fully connected layers and add them to the input of the Resnet block.
Input trajectory. It is important to note that the input/output data for this model is two-dimensional trajectory tensor ([2, length)), with each dimension representing the longitude and latitude of the trajectory, respectively. Considering CNN-based models can only accept fixed-shape data, we initially up-sample or down-sample the training trajectories using linear interpolation methods [3]. The generated trajectories are then transformed to the desired length using the same approach.
Why use CNN based structure? In DiffTraj, we adopt CNN-based UNet structure over other temporal processing modules such as recurrent neural networks (RNNs) and WaveNet for several reasons. First, CNN-based methods can alleviate the vanishing gradient problem often encountered in RNNs. enabling more efficient learning of long-term dependencies. Second, compared to models like RNNs, CNNs hold a smaller number of parameters and offer greater computational efficiency, which is crucial during the reverse process. Finally, the UNet architecture excels in various generative tasks, particularly when combined with diffusion models [28, 11, 7]. Its multi-level structure with skip connections enables multi-scale feature fusion, helping the model better capture local and global contextual information in GPS trajectory data. We found that RNN and WaveNet-based architectures led to worse trajectory quality, as will be presented in Sec. 5.
4.2 Conditional Generation
Various external factors, such as the region of the trip and departure time, influence real-world trajectories. This conditional information should provide meaningful guidance for the generation process, ensuring that synthetic trajectories exhibit similar patterns and behaviors. In the proposed DiffTraj framework, a Wide & Deep network [6] structure is employed to effectively embed conditional information, enhancing the capabilities of the Traj-UNet model. The wide component emphasizes memorization, capturing the significance of individual features and their interactions, while the deep component concentrates on generalization, learning high-level abstractions and feature combinations. This combination effectively captures both simple and complex patterns the trajectory. For given trajectory, several numerical motion properties (c.g., velocity and distance)
5

---

and discrete external properties (e.g., departure time and regions) are derived. A wide network is utilized for embedding the numerical motion attributes, whereas a deep network employed for the discrete categorical attributes. The wide network consists of a fully connected layer with 128 outputs, capturing linear relationships among motion attributes. In regard to the discrete categorical attributes, the deep network initially embeds them into multiple 128- 128-dimensional vectors, concatenates these vectors, and subsequently employs two fully connected layers to learn intricate feature interactions. Finally, the outputs generated by the Wide & Deep networks are combined and integrated into each Resnet block.
4.3 Ensuring Generation Diversity
In practice, it should be avoided that conditional information results in a model with overly smooth or deterministic behavior patterns, which could undermine the intended privacy protections. To regulate the diversity of the generated trajectories and prevent the DiffTraj from following the conditional guidance too closely, we employ the classifier-free diffusion guidance method [12]. Specifically, we jointly train conditional and unconditional diffusion models with a trade-off between sample quality and diversity by using& guiding parameter w. That is, the noise prediction model can be written as:
(7) where @ denotes vector in which all conditional information is O, essentially not encoding any information. When w = 0, the model is a general diffusion model with conditional information only. By increasing the wi the model will focus more on unconditionally predicting noise (i.e., more diversity) when generating trajectories. Therefore, we can strike a balance between maintaining the quality of the generated trajectories and increasing their diversity, ensuring that the model does generate overly deterministic behavior patterns.
E9 = (1 + W) (zj, ax) WEB (a²7, Ø))
4.4 Sampling Speed up
As described in Sec. 3.2, DiffTraj relies on a large Markov process to generate high-quality trajectories, rendering slow reverse diffusion process. Therefore, is a significant challenge for the proposed model to generate valid and usable trajectories in reasonable time frame. To address this issue, we adopt a non-Markov diffusion process approach with the same optimization objective [28], thus allowing the computation of efficient inverse processes. Specifically, steps with the skip-step method presented in [23]. The corresponding set of noise trajectories changes []. Through this approach, the sampling steps during trajectory generation
sample every [T/S]
to (r....., rs} can be significantly reduced from T steps to S steps. Compared to the typical diffusion model [11], this method can generate higher quality samples in fewer steps [28].
4.5 Discussion on Privacy
The DiffTraj inherently protects privacy in GPS trajectory generation due its generative nature and the manner in which it reconstructs synthetic data from random noise. It offers effective privacy protection through two key aspects. (1) Generative Approach: The DiffTraj generates trajectories by sampling from learned distribution, rather than directly relying on or perturbing real-world data. By reconstructing trajectories from random noise during the reverse diffusion process, the model effectively decouples synthetic data from specific real data points. This ensures that the generated trajectories do not contain personally identifiable information or reveal sensitive location details, thus protecting their privacy. (2) Learning Distribution: The Traj-UNet learns the distribution of real trajectories and generates synthetic ones that exhibit similar patterns and behaviors, which avoids direct replicating individual real trajectories. The diverse and realistic synthetic trajectory generation ensures that sensitive information about specific individuals or their movement patterns is not inadvertently leaked. Morcover, it prevents the possibility of reverse engineering real trajectories from synthetic ones. By combining these aspects, the DiffTraj offers robust privacy protection for GPS trajectory generation, addressing concerns about data privacy while maintaining utility.
5 Experiments
We conduct extensive experiments on two real-world trajectory datasets to show the superior performance of the proposed DiffTraj in GPS trajectory generation. In this section, we only include

---

the basic setup, comparison of the main experimental results, and visualization analysis. Owing to space limitation, we leave other details (dataset, baselines, network structure, hyperparameters, etc.) and more results (conditional generation, case studies, etc.) in the Appendix, All experiments are implemented in PyTorch and conducted on a single NVIDIA A100 40GB GPU. The code for the implementation of DiffTraj is available for reproducibility".
5.1 Experimental Setups
Dataset and Configuration. We evaluate the generative performance of DiffTraj and baselines on two real-world GPS trajectory datasets of Chinese cities, namely, Chengdu and Xi'an an. A detailed description of the dataset and statistics is available in Appendix B. and experiment about Porto dataset is shown Appendix C.I.
Evaluation Metries. To fully evaluate the quality of the generated trajectories, we follow the design in [8] and use four utility metrics different levels. Specifically, 1) Density error measures the geo-distribution between the entire generated trajectory and the real one, which measures the quality and fidelity of the generated trajectories at the global level. For trajectory level, 2) Trip error measures the distributed differences between trip origins and endpoints, and 3) Length error focuses on the differences in real and synthetic trajectory lengths. Moreover, we use 4) Pattern score to calculate the pattern similarity of the generated trajectories (see Appendix B.2 for details).
Baseline Methods. We compare DiffTraa with series of baselines, including three typical generative models (VAE [34], TrajGAN [9] and D [38]) and range of variants based on the diffusion model approach. Specifically, Diffwave is a diffusion model based on the WaveNet structure [14], while Diff-LSTE replaces the ID-CNN-based temporal feature extraction module with an LSTM. In addition, Diff- scatter and Diff ww/ WUoeNet can be considered as ablation studies of the proposed method to evaluate the effectiveness of diffusion models and UNet structures, respectively. Furthermore, DiffTraj-wo/ /wo indicates no integration of the conditional information embedding module, which is used to evaluate the efficiency of the Wide & Deep module for trajectory generation. Finally, we also compare Random Perturbation (RP) and Gaussian Perturbation (GP), two classes of non-generative approaches, to compare the superiority of generative methods in privacy preservation. The details of the implementation are presented Appendix B.3.
5.2 Overall Generation Performance
Table 1 presents the performance comparison of DiffTraj and the selected baseline methods on two real-world datasets. Specifically, we randomly generate 3, 000 trajectories from each generative method and then calculate all metrics. From the results, we observe several trends that highlight the advantages of the proposed DiffTraj method. Non-generative methods, RP and GP, exhibit poor performance due to their reliance on simple perturbation techniques, which fail to capture the complex spatial-temporal dependencies and motion properties of the real GPS trajectories. This leads to lower-quality synthetic trajectories compared to generative models. Generative models, VAE and TrajGAN, show better performance than RP and GP but are still inferior to DiffTraj (or MifTrrj--o//on) This is because VAE and TrajGAN primarily focus on spatial distribution and may not adequately capture the temporal dependencies within trajectories. Additionally, the DiffTraj is designed to reconstruct the data step-by-step from random noise, which makes it more suitable for generating real GPS trajectory that reflect the inherent stochastic nature of human activities.
Diff-LSTM achieves good results in some metrics compared to the model without UNet, but falls short of Difi'rraj due to the differences in the backbone network. The LSTM-based approach in Diff-LSTM is inherently designed for sequence processing and may be less effective in capturing spatial dependencies in the data compared the CNN-based approach used in DiffTraj. The CNN-based backbone in DiffTraj can better model local spatial patterns and multi-channel dependencies more effectively, leading to higher-quality generated trajectories. Nevertheless, such results advocate the employment of diffusion models in future studies related to spatial-temporal data generation.
Ablation Study. The other diffusion model variants, Diffwave, Diff-scatter, and Diff- Uoffe, do not perform as well as DiffTrai due to differences in their network structures or the absence of the UNet architecture. The Traj- UNet structure in DiffTraj is specifically designed for tentiall
7

---

Table 1: Performance comparison of different generative approaches.
Chengdu
Xi'an
Methods
Density (U)
Trip (U)
Length (L)
Pattern (t)
Density (U)
Trip (U)
Length (L)
Pattern (1)
RP
0.0698
0.0835
0.2337
0.493
0.0543
0.0744
0.2067
0.381
GP
0.1365
0.1590
0.1423
0.233
0.0928
0.1013
0.2164
0.233
VAE
0.0148
0.0452
0.0383
0.356
0.0237
0.0608
0.0497
0.531
TraGGAN
0.0125
0.0497
0.0388
0,502
0.0220
0.0512
0.0386
0.565
TraaaAGA
0.0117
0.0443
0.0221
0,706
0.0207
0.0498
0.0436
0.664
Diffwave
0.0145
0.0253
0.0315
0.741
0.0213
0.0343
0.0321
0.574
Diff-scatter
0.0209
0.0685
0,0693
0.0762
Diff wot
0.0356
0.0868
0.0378
0.422
0,0364
0.0832
0.0396
0.367
DiffTrajwoo/Con
0.0072
0.0239
0.0376
0.643
0.0138
0.0209
0.0357
0.692
Diff-LSTM
0.0068
0.0199
0,0217
0.737
0.0142
0.0195
0.0259
0.706
DiffTraj
0.0055
0.0154
0.0169
0.823
0.0126
0.0168
0.0203
0,764
Bold indicates the staristically performance (i.c., two--ided p 0.05) best baseline.
(a) VAE
(b) TrajGAN
(c) Diffwave
(d) Diff-LSTM
DiffTraj
(i) Real
Figure 2: Geographic visualization of generated trajectory in Chengdu (Larger view in Appendix D).
modeling and multi-scale feature fusion, enabling the accurate prediction of noise levels during denoising and resulting in high-quality generated trajectory. It is also noteworthy that satisfactory results can still be accomplished when only generating scattered locations or discarding the Traj-UNet structure. Finally, the conditional version of DiffTraj outperforms the unconditional version (DiffTraj-wo/oon) because it incorporates conditional information into the trajectory generation process. This allows the model to better account for external factors such as traffic conditions, departure time, and regions, which significantly influence human mobility patterns. This result can also be verified by the performance on the pattern indicator, where this indicator significantly outperforms other models.
Geographic Visualization. We also visualized the generated results to better present the performance of different generative models. Fig. 2 shows the trajectory distributions generated by baseline methods for Chengdu (Please refer to Fig. 10 and Fig. 11 in Appendix for Xi' an and larger view). From visualizations, we see that all the generated trajectories show geographic profiles of the city, Notably, DiffTraj and Diff-LSTM generate trajectories almost identical to the original trajectories and significantly surpass the other methods. However, Diff-LSTM performs poorly compared to DiffTraj on sparse roads, suggesting that CNNs are skilled in capturing local and global spatial features, making them more suitable for handling sparse road scenes. In addition, developing tailor-made UNet architecture is also essential. Comparing Fig. 2(c) and Fig. 2(e), we can observe that the latter able to generate visually more realistic trajectory distribution.
5.3 Utility of Generated Data
As the generated trajectory serves downstream tasks, its utility is critical to determine whether the data generation method is indeed practical. In this section, we evaluate the utility of generated trajectory through an inflow (outflow) prediction task, which is one of the important applications of trajectory data analysis [13, 39]. Specifically, we train multiple advance prediction models using the original and generated trajectory separately, and then compare the prediction performance (detail experimental setup refer Appendix C.3). As concluded in Table the performance of the models when trained on the generated data is comparable to their performance on the original data. This result shows that our generative model, DiffTraj. able to generate trajectory that retain the key statistical properties of the original one, hence making them useful for downstream tasks such as inflow and outflow prediction. However, there a slight increase error metries when using the generated data, implying there's 's still room for further improvements in trajectory generation. Nevertheless, this result encourages the use of generated, privacy-free alternatives to the original data in future studies related to trajectory data analysis.

---

Table 2: Data utility comparison by inflow/outloww prediction.
Task
Inflow (origin/generated)
Outflow (origin/generraee))
Methods
AGCRN
GWNet
DCRNN
MTGNN
AGCRN
GWNet
DCRNN
MTGNN
MSE
4.33/4.55
4.42/4.50
4,45/4.62
4.28/4.56
4.45/4.70
4,64/4.72
4.72/5.01
4.39/4.77
RMSE
2.08/2.13
2.10/2.12
2.11/2.15
2.07/2.13
2.11/2.16
2.15/2.17
2.17/2.24
2.10/2
MAE
1.49/1.22
1,50/1.50
1,51/1553
1,48/1.11
1,50/1.54
1.53/1554
1.53/1.57
1,49/1.33
0.58
0.56
Origin
Thaniter
0.07
0.05
0.00
Origh
Trarater
60
30-
70
Density
0.14
0.12
70.
60-
50-
0.175
0.150
0.200
0.225
0.04
10-
0.125
0.54
0.03
30-
0.06
3
0.100
20-
0.04
20
0.050
0.0
10
0.02
10
0.00
0.00
Chengdu
(h) Xi'an
(c) Chengdu
(d) Xi' an
Figure Additional Experiment. (a) and (b) Transfer learning; (c) and (d) Speed up sampling.
5,4 Additional Experiment
Transfer Learning. Transfer learning is an essential aspect to investigate in the context of trajectory generation, as it can demonstrate the robustness and generalization capabilities of the proposed Dajf framework. In this paper, we pre-train the DiffTraj model one city, and then continue the training in another city using a different percentage (5% ~ 100%) of data. We also compared these results with only training Traf on different percentages data. As shown in Fig. 3, 'origin' indicates that transfer learning is not applicable, while 'transfer' indicates applying transfer learning. Notably, even with just 5% of the data, the transfer learning model achieves a significantly lower error compared to the original one. As the percentage of data increases, the gap between the two models narrows. The results show that the DiffTraj model exhibits strong adaptability and generalization capabilities when applied to different urban scenarios. This allows the model to quickly adapt new urban contexts and capture the inherent mobility patterns with less data, thereby reducing training time and improving applicability to diverse real-world scenarios.
Speed up Sampling. As discussed Sec. 3.2, the diffusion model requires numerous sampling steps for gradual denoising, resulting in slow trajectory generation process. By incorporating the sampling speed-up technique outlined in Sec. 4.4, we examine the efficiency of generating 50 x 256 trajectories using well-trained DiffTraj two datasets. The results summarized in Fig. 3, where the X-axis represents the total sample steps after speed-up, and the two Y-axes indicate the time spent and Density error, respectively. All models trained with T = 500 diffusion steps consistently yield high-quality trajectories resembling the original distribution. Moreover, the model matches the outcomes of the no skipped steps method T 100, saving 81% of the time cost. However, too few sampling steps (S < 100) result in a significant divergence between the generated and real trajectory. This occurs because fewer sample steps lead to more skipped reverse operations, causing Diff-Traj to discard more noise information during denoising. Nonetheless, these results highlight the substantial efficiency improvements achieved by the speed-up methods presented in Sec. 4.4.
6 Conclusion
In this work, we propose a new GPS trajectory generation method based on diffusion model and spatial-temporal data mining techniques. This method, named DiffTraj, leverages the data generation ability of the diffusion model and learning from spatial-temporal features by Traj-UNet. Specifically, real trajectories are gradually transformed into random noise by a forward trajectory noising process. After that, DiffTraj adopts reverse trajectory denoising process recover forged trajectories from the noise. Throughout the DiffTraj framework, we develop Traj- -Net structure and incorporate Wide & Deep conditional module to extract trajectory features and estimate noise levels for the reverse process. Extensive experiments validate the effectiveness of DiffTraj and its integrated Traj- UNet. Further experiments prove that the data generated by DiffTraj can conform the statistical properties of the real trajectory while ensuring utility, and provides transferability and preeervin......gg...........

---

7 Acknowledgments
This work is supported by the Stable Support Plan Program of Shenzhen Natural Science Fund under Grant No. 202208151 R1015-23), APRC- CityU New Research Initiatives (No.9610566 Start-up Grant for New Faculty of City University of Hong Kong), CityU HKIDS Early Career Research Grant (No.9360161) Hong Kong ITC Innovation and Technology Fund Midstream Research Programme for Universities Project (No. NN//TTS/33//2MSS) Hong Kong Environmental and Conservation Fund (No. 88/2022), SIRG CityU Strategic Interdisciplinary Research Grant (No. 7020046, No.7020074) Tencent (CCF-Tencent Open Fund, Tencent Rhino-Bire Focused Research Fund), Huawei (Huawei Innovation Research Program), Ant Group (CCF-Ant Research Fund, Ant Group Research Fund) and Kuaishou.
This research was partially supported by Research Impact Fund (No.
References
[1] M. P. Armstrong, G. Rushton, and D. L. Zimmerman. Geographically masking health data to preserve confidentiality. Statistics in medicine, 18(5):497-525, 1999.
[2] O. Avrahami, D. Lischinski, and O. Fried. Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 18208-18218, 2022.
[3] J. Bergh and J. Löfström. Interpolation Spaces: An Introduction. Springer Berlin Heidelberg, 2012.
[4] C. Cao and M. Li. Generating mobility trajectories with retained data utility. In Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery & Duta Mining, pages 2610-2620, 2021.
[5] H. Cao, C. Tan, Z. Gao, G. Chen, P.-A. Heng, and S. Z. Li. A survey on generative diffusion model. arXiv preprint arXiv:2209. 02646, 2022.
[6] H.-T. Cheng, L. Koc, J. Harmsen, T. Shaked, T. Chandra, H. Aradhye, G. Anderson, G. Corrado, W. Chai, M. Ispir, et al. Wide & deep learning for recommender systems. In Proceedings of the
1st workshop on deep learning for recommender systems, pages 7-10, 2016.
[7] P. Dhariwal and A. Nichol. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34:8780-8799 2021.
[8] Y. Du, Y. Hu, Z. Zhang, Z. Fang, L. Chen, B. Zheng, and Y. Gao. Ldptrace: Locally differentially private trajectory synthesis. arXiv preprint arXiv:2302.061888 2023.
[9] Goodfellow, J. Pouget-Abadie Mirza, Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, volume 27, 2014.
[10] X. Han, X. Zhao, L. Zhang, and W. Wang. Mitigating action hysteresis in traffic signal control with traffic predictive reinforcement learning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, page 673-684, 2023.
[11] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
[12] J. Ho and T. Salimans. Classifier-free diffusion guidance. In NewrIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
[13] R. Jiang, D. Yin, Z. Wang. Y. Wang, J. Deng, H. Liu, Z. Cai, J. Deng, X. Song, and R. Shibasaki. DL-ttrfff: Survey and benchmark of deep learning models for urban traffic prediction. In Proceedings of the 30th ACM international conference on information & knowledge management, pages 4515-4525, 2021.
[14] Z. Kong, W. Ping, J. Huang, Zhao, and B. Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations, 2020.
10

---

[15] X. L. Li, J. Thickstun, I. Gulrajani. P. Liang, and T. B. Hashimoto. Diffusion- Im improves controllable text generation. arXiv preprint orXiv:2205.14217, 2022.
[16] Y. Liang, K. Ouyang, H. Yan, Y. Wang, Z. Tong, and R. Zimmermann. Modeling trajectories with neural ordinary differential equations. In IJCAI, pages 1498 1504, 2021.
[17] C.-Y. Liou, W.-C. Cheng, J.-W. Liou, and D.-R. Liou, Autoencoder for words. Neurocomputing, 139:84-96, 2014.
[18] F. Liu, D. Wang, and Z.-Q. Xu. Privacy-preserving travel time prediction with uncertainty using gps trace data. IEEE Transactions on Mobile Computing, pages 1-1, 2021.
[19] Q. Liu, F. Yan, X. Zhao, Z. Du. H. Guo, R. Tang, and F. Tian. Diffusion augmentation for sequential recommendation, 2023.
[20] M. Luca, G. Barlacchi. Lepri, and L. Pappalardo. A survey on deep learning for human mobility. ACM Computing Surveys (CSUR), 55(1):1-44, 2021.
[21] M. E. Nergiz, M. Atzori, and Y. Saygin. Towards trajectory anonymization: generalization-based approach. In Proceedings of the SIGSPATIALACM GIS 2008 International Workshop Security and Privacy in GIS and LBS, pages 52 61, 2008
[22] A. Nichol, P. Dhariwal, A. Ramesh, Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.1074) 2021.
[23] A. Q. Nichol and P. Dhariwal. Improved denoising diffusion probabilistic models. In International Conference Machine Learning, pages 8162-8171. PMLR, 2021.
[24] K. Ouyang, R. Shokri, D. S. Rosenblum, and W. Yang. A non-parametric generative model for human trajectories. In ICAI,, volume 18, pages 3812-3817, 2018.
[25] R. Rombach, A. Blattmann, D. Lorenz. P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
[26] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234-241 Springer, 2015.
[27] Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256-2265. PMLR, 2015.
[28] Song, C. Meng, and Ermon. Denoising diffusion implicit models. arXiv preprint arXiv22010.02502, 2020.
[29] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011. 2020.
[30] A. Vaswani, Shazeer, N. Parmar, Uszkoreit, L. Jones, A. N. Gomez, Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[31] X. Wang, X. Liu, Z. Lu, and H. Yang. Large scale gps trajectory generation using map based on two stage gan. Journal of Data Science, 19(1): 2021.
[32] H. Wen, Y. Lin, Y. Xia, H. Wan, Q. Wen, R. Zimmermann, and Liang. Diffstg: Probabilistic spatio-temporal graph forecasting with denoising diffusion models. arXiv preprint arXiv:2301.13629 2023.
[33] I.. Xi, C. Hanzhou, and A. Clio. trajgans: using generative adversarial networks for geo-privacy protection of trajectory data. Vision paper, 2018.

---

[34] T. Xia, X. Song, Z. Fan, H. Kanasugi, Q. Chen, R. Jiang, and R. Shibasaki. Deeprailaay A deep learning system for forecasting railway traffic. 2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), pages 51-56, 2018
[35] L. Yang, Z. Zhang, Y. Song, S. Hong, R. Xu, Y. Zhao, Y. Shao, W. Zhang, B. Cui, and M.-H. Yang. Diffusion models: A comprehensive survey methods and applications. arXiv preprint MxXiv:2209007770 2022.
[36] Y. Yuan, J. Ding, H. Wang, D. Jin, and Y. Li. Activity trajectory generation via modeling spatiotemporal dynamics. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4752-4762, 2022.
[37] A. Zandbergen. Ensuring confidentiality of geocoded health data: assessing geographic masking strategies for individual-level data. Advances in medicine, 2014, 2014.
[38] J. Zhang, Q. Huang, Y. Huang, Q. Ding, and P.-W. Tsai. Dp-trajgan: A privacy-aware trajectory generation model with differential privacy. Future Generation Computer Systems, 2022.
[39] Z. Zhang, X. Zhao, H. Miao, C. Zhang, H. Zhao, and J. Zhang. Autostl: Automated
spatio-temporal multi-task learning. arXiv preprint arXiv:2304.cm 09114, 200117420000 2023.
[40] X. Zhao, W. Fan, H. Liu, and J. Tang. Multi-type urban crime prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 4388-4396, 2022.
[41] Y. Zheng. Trajectory data mining: an overview. ACM Transactions on Intelligent Systems and Technology (TIST), 6(3):1-41 2015.
[42] Y. Zheng, Capra, O. Wolfson, and Yang. Urban computing: concepts, methodologies, and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 5(3):1-55, 2014.
[43] Y. Zheng, L. Zhang. X. Xic, and W.-Y. Ma. Mining interesting locations and travel sequences from gps trajectories. In Proceedings of the 18th international conference on World wide web, pages 791-800, 2009.
[44] Y. Zhu, Y. Ye, Y. Liu, and J. J. Q. Yu. Cross-area travel time uncertainty estimation from trajectory data: federated learning approach. IEEE Transactions Intelligent Transportation Systems, 23(12)22996-24478 2022.
12

---

A DiffTraj Details & Hyperparameters
In this section, cover the specific details of DiffTraj, including the DiffTraj framework, the TNet structure, and the implementation details.
A.1 Architecture
Tt
t
Traj-UNet
Sinusoidal
and Concatanate
CB
Wide & Deep
Down-Sampling
40
x
Up Sampling
Conditional
Traj-UNet
Figure 4: The network architecture used by DiffTraj in modeling Ea (æi,, & xb)) is divided into two modules, down-sampling and up-sampling, each containing multiple Resnet blocks.
As illustrated in Fig. 4, DiffTraj divided into two modules, i.e., down-sampling and up-sampling. and conditional module. Each down-samplin and up-sampling module consists of multiple stacked Resnet blocks. Between the two of them, transitional module based on the attention mechanism is integrated. To better learn the noise of each time step and guide the generation, DrffTraa integrates conditional module to embed the time step and external traffic information, later fed to each block. Since the CNN structure can only accept data of fixed shape, we first sample each trajectory added using linear
tensor of [2, length). Specifically, the trajectory below the set length, interpolation, and if it is greater than that, the redundant portion is removed using linear interpolation. Thus, the model will generate fixed-length trajectories, which will then be tailored to the desired length by the conditional information.
Time Step
Numerical
Categorical
Up-nmppeSampee
Sinusoidal
Embedding
Group Normalization
Linear Layer
Liner Layer
Activation
Activation
Activation
Linear
Linear Layer
Unear Layer
Convolution 1D
Linear
Project
Embedding Block
Group Normalization
Down-Sample
Activation
ResnetBlock
Convolution
Attention layer
ResnetBlock
ResnetBlock
MidAttBBlock
Down/Up Sample
Up-Sample
Figure 5: The main components Traj-UNet including Resnet block, Embedding block, and middle attention block.
13

---

The main blocks of Traj-UNet are presented in Fig. 5, i.e., Resnet block, Embedding block, and middle attention block. Among them, each sampling block stamling and up-sampling) consists of multiple Resnet blocks, each containing a series of group normalization, nonlinear activation, and ID-CNN layers. Then, Traj-UNe applies up-sampling or down-sampling to the output, where down-sampling uses max pooling and up-sampling uses interpolation. After this, Traj-UNet integrates a middle attention block, which consists of two Resnet blocks and an attention layer. Note that there are additional down-/up-sampling operations in the Resnet block. Finally, we integrate conditional embedding block to learn the diffusion time step and conditional information. For the diffusion step, employ Sinusoidal embedding to represent each and then apply two shared- parammeer fully connected layers. For conditional information, such as distance, speed, departure time, travel time, trajectory length, and starting and ending locations, we use the Wide & Deep module for embedding. After getting the diffusion step embedding and conditional embedding, sum them up and add them to each Resnet block.
128-dimensional vector,
A.2 Implementation Details
For the proposed DiffTraj framework, we summarize the adopted hyperparameters Table 3. In addition, to facilitate reproduction by the researcher, we provide reference range of hyperparameters based on the experience and general settings of the present work.
Table 3: Hyperparameters setting for DiffTraj.
Hyperparameter
Setting value
Refer range
Diffusion Steps
500
300
500
Skip steps Guidance scale
~ 10
0.1 10
(linear)
0.0001 ~ 0.05
Batch size
1024
256
Sampling blocks
2
Resnet blocks
Input Length
200
120 ~ 200
The training and sampling phase of the proposed framework is summarized in Algorithm 1 and Algorithm 2, respectively.
Algorithm Diffusion Training Phase
for 1, ...,.... do
2: Sample xo ~ -xx),,
3: Uniform (1.....T) .T)
4: I)
5: = c- co (Va VI a,c,t xg6) ||
6: 0 = 0 VoL
7: end for
Algorithm 2 Diffusion Sampling Phase
1: Sample x I))
2: for 1= T S...., do
3: Compute #4 (wj, t xx6) according Eq. (5)
4: Compute pa (x: æxf ) according Eq. (4)
5: end for
6: return Zo
14

---

B Details of the Experimental Setup
B.1 Dataset
We evaluate the performance of DiffTraj and all baselines methods on two datasets with different cities, Chengdu and Xi'an'. Both datasets collected from cab trajectory data starting from November 1, 2016, to November 30, 2016. Table 4 summarizes the statistical information these two datasets, and Fig. 6 shows the trajectory distribution and heat map of these datasets, where the deeper color indicates the more concentrated trajectory in the region. For all datasets, we remove all trajectories with lengths less than 120 and sample them set fixed length.
Table 4: Statistics of Two Real-world Trajectory Datasets.
Dataset
Trajectory Number
Average Time
Average Distance
Chengdu Xi an
3 493 918
11.42 min
7.42 km
2 180 348
12.58 min
5.73 km
(a) Chengdu Trajectory
(b) Chengdu Heatmap
(c) Xi'an Trajectory
(d) Xi'an an Heatmap
Figure 6: Origin trajectory distribution of two cities.
B.2 Evaluation Metries
As trajectory generation aims to generate trajectories that can replace real-world activities and further benefit downstream tasks, we need to evaluate the "similarity" between the generated trajectories and real ones. this work, we follow the common practice in previous studies [8] and measure the quality of the generated ones by Censon-Shannon divergence (JSD). JSD compares the distribution of the real and generated trajectories, and a lower JSD indicates a better match with the original statistical features. Suppose that the original data has probability distribution P and the generated data has probability distribution G, the JSD calculated as follows:
JS)(P,G) = Ep log P+G
+
BG
log G+P
(8)
For the evaluation, divided each city into 16 16 size grids and recorded the corresponding values for each grid. Based on this, we adopt the following metrics to evaluate the quality of the generated trajectories from four perspectives:
Density error: This global level metric that used evaluate the geo-distribution between the entire generated trajectory D (Tgen) and the real trajectory D(T).
Density Error = JSD (D(T), D (Tgen) )),
(9)
where D(-) denotes the grid density distribution in given trajectory set, and JSD(-) represents the Jenson-Shee Shannon divergence between two distributions.
Trip error: This a trajectory level metric that measures the correlation between the starting and ending points of travel trajectory. Specifically, we calculate the probability distribution of the start/end points the original and generated trajectories and use JSD measure the difference between them.
Theee datasets can be downloaded at https///uurracch.dddiccxxxn.....
15

---

Length error: This trajectory level metric to evaluate the distribution of travel distances. It can be obtained by calculating the Euclidean distance between consecutive points.
Pattern score: This is semantic level metric defined the top-n grids that occur most frequently in the trajectory. We define P and gen to denote the original and generated pattern sets, respectively, and compute the following metrics:
Pattern score = 2 Precision (P, Pgsn) x Recall (P, Pgnn
(10)
Precision (P, P gen) + Recall (P: Pgen gen)
B.3 Baselines
In this section, we introduce the implementation of baseline methods.
Random Perturbation (RP): We generate -0.01 ~ 0.01 random noise to perturb the original trajectory. This degree of noise ensures that the maximum distance between the perturbed points and the original does not exceed km
Gaussian Perturbation (GP): We generate Gaussian noise perturbed original trajectories with mean O and variance 0.01.
Variational AutoEncoder (VAE) [17, 34]: In this work, trajectories are first embedded hidden distribution through two consecutive convolutional layers and a linear layer. Then, we generate the trajectories by a decoder consisting of linear layer and two deconvolutional layers. The size of convolution kernels in convolutional and deconvolutional layers is set to 4 to ensure that input and output trajectories have the same size.
TrajGAN [9]: The trajectory is first combined with random noise and then passes through generator consisting of two linear layers and two convolution layers. Subsequently, a convolutional layer and linear layer adopted as the discriminator. The generator and the discriminator trained in an alternating manner.
Dp-TrajGAN [38]: The model is similar TrajGAN, with the difference that this model replaces CNN with LSTM.In addition, we remove the unnecessary operation of differential privacy for fair comparison.
Diffwave [14]: Diffwave is Wavenet structure model designed for sequence synthesis, which employs extensive dilated convolution. Here, we use 16 residual connected blocks, each consisting of bi-directional dilated convolution. Then they are summed using sigmoid and tanh activation, respectively, and fed into the ID CNN.
Diff-scatter:
We randomly sample GPS scatter points from trajectories and generate scatter points using a 4-layer MLP (neutrons: {128, 256, 256, 1283) and the diffusion model.
Diff-wo/UNet: This model uses only two Resnet blocks combined with single attention layer between them. Compared with DiffTraj, this model does not have a UNet-type structure, which be used evaluate the necessity of the UNet structure.
Diff-LSTM: This model has the same UNet-type structure and number of Resnet blocks as DiffTraj, and the difference is that Diff-LSTM replaces the CNN with LSTM Resnet block.
DiffTraj-wo/Con represents that the Wide & Deep conditional information embedding module is removed. The rest is the same as DiffTraj.
DiffTraj-wo//C wo/Co::
C Additional Experiments
C.1 Porto Dataset Experiment
To further validate the generalizability our approach, we conducted experiments on different Porto dataset. Through the new experiment, we can see that DiffTraj achieves the best performance on this dataset, demonstrating ability to adapt to different geographic regions and capture different trajectory patterns. These results underscore the generalizability across different cities.
16

---

Table 5: Performance comparison of different generative approaches on Porto dataset.
Methods
Density ())
Trip (L)
Length (4)
Pattern (1)
VAE
0.0121
0.0224
0.0382
0.763
TrajGAN
0.0101
0.0268
0.0332
0.765
Diffwave
0.0106
0.0193
0.0266
0,799
Diff-LSTM
0.0092
0.0141
0.0255
0.828
DiffTraj
0.0087
0.0132
0.0242
0.847
C.2 Data Utility Additional Experiments
To illustrate the performance of DiffTraj generated trajectories, use the trajectory data generated based on different baselines for the inflow/outflow prediction task. As can seen from the results in Table 6, the training results using DiffTraj generate trajectories closer to the real environment than using the baseline generation method, which is in line with the trend shown in Table 1.
Table 6: Data utility comparison by inflow/outflow prediction (different baselines).
Task
Inflow
Outflow
Bascline
GAN
VAE
Diffwave
DiffTraj
GAN
VAE
Diffwave
DiffTraj
MSE
6.78
6.79
4.62
4.50
5.97
5.81
4.75
4.72
RMSE
2.60
5.61
2.16
2.12
2.44
2.41
2.18
2.17
MAE
1.76
1.76
1.52
1.50
1.68
1.66
1.56
1.54
C.3 Data Utility Setup
In this paper, we use inflow/outllww traffic forecasting to test the utility of the generated data. Inflow/outloww traffic forecasting critical task in urban traffic management that involves predicting the volume of vehicles entering (inflow) or leaving (outflow) specific region within certain period of time. this experiment, we divided a city into 16 16 grids, where each grid represents specific region. The traffic volume entering (inflow) or leaving (outflow) each region within a certain period predicted. The primary goal of this experiment to train various prediction models using both original and generated trajectory data, comparing their prediction performance. This evaluation provides an important perspective on the real-world applicability of the data generated by Diff assessing not just the fidelity of the generated trajectories, but also their utility in downstream tasks. In the experimental setup, we train the prediction models using the generated data and the original data separately, and then test their prediction performance on real data. Advanced neural network models, such as AGCRN, Graph WaveNet, DCRNN, and MTGNN, have been employed to handle this task due to their ability to capture complex apatial-temporal dependencies in multivariate time series data. All the above models and code in this section are followed the publicly available code provided in the literature [13].
AGCRN (Adaptive Graph Convolutional Recurrent Network): AGCRN is sophisticated model for spatial-tempot forecasting, which leverages both spatial and temporal features of data. It uses graph convolution capture spatial dependencies and RNNs to model temporal dynamics, making capable of handling complex spatial-tempont sequences.
GWNet (Graph WaveNet): GWNet is designed for high-dimensional, structured sequence data. It incorporates a Graph Convolution Network (GCN) to model spatial correlations and WaveNet-like architecture model temporal dependencies. The combination allows for capturing both the spatial and temporal complexities present in high-dimensional data.
DCRNN (Diffusion Convolutional Recurrent Neural Network): DCRNN is a deep learning model designed for traffic forecasting, which handles the spatial and temporal dependencies in traffic flow data. It uses diffusion convolution operation to model spatial dependencies and architecture with scheduled sampling and residual connections to model temporal dependencies.
hnnpss//gthhbbccm/eepkkaah/uu//u Graph
17

---

MTGNN (Multivariate Time-series Graph Neural Network): MTGNN is model that captures complex spatial-tempont relationships in multivariate time series data. The model model leverages a graph neural network to model spatial dependencies and an auto-regressive process to capture temporal dependencies. It also uses a gating mechanism to adaptively select the relevant spatial-temporal components, thus improving the forecasting performance.
For accuracy comparison, we use the mean square error (MSE), root mean square error (RMSE), and mean absolute error (MAE) as metrics to assess the prediction accuracy of all methods. These three metrics are defined as follows:
N
MSB(X, X)
x 1101881 xx00²
(11)
N
RMSE(X, X)
1
Σ
(x(() -x(0)
(12)
N
MAB(X, X)
N x 8110
(13)
where X0) and {0 are the ground truth and predicted inflow/outflow value at time i, respectively.
C.4 Conditional Generation
As we described in Sec. 4.2 and Sec. 4.3, the DiffTraj framework takes into account various external factors that influence real-world trajectories, such road network structure and departure time. These factors are used to guide the generation process, ensuring the synthetic trajectories mimic real-world patterns and behaviors. The model employs Wide & Deep network to effectively embed this conditional information, enhancing the capabilities of the Traj-UNet model.
To evaluate the conditional generation capability of DiffTraj, we investigate the case of generated trajectories where the start and end regions of the trajectories were pre-defined. The model was tasked to generate 20 random trajectories adhering to these conditions. The results depicted in Fig. 7 and Fig. (The red and blue boxes indicate the starting and ending regions, respectively), effectively demonstrate proficiency in generating trajectories that align with the specified start and end regions of Traj This is observed consistently across both cities under study, reinforcing the model's 's ability to accurately incorporate and adhere conditional information. This robust capability underscores the versatility of DiffTraj in generating meaningful trajectories under specific conditions, and its applicability in real-world scenarios where such conditions often exist.
C.5 Ensuring Generation Diversity
In addition, DiffTraj is designed to generate high-quality trajectories and ensure a level of diversity that prevents overly deterministic behavior patterns, thereby upholding intended privacy protections. By integrating a classifier frre diffusion guidance method, Diff can strike a calculated balance between sample quality and diversity. To validate the capacity for generating diverse trajectories of DiffTraj, we devised an experiment that manipulates the guiding parameter, This experiment aimed to examine the quality-diversity balance in trajectories generated by DiffTraj, and how this equilibrium responds to variations in w. In this experimental setup, we studied the trajectories yielded under different w settings (specifically w E 0.1, 1, 10) while keeping the conditional information the same.
The results, as illustrated in Fig. and Fig. 8, reveal an unambiguous link between an increase in and rise in trajectory diversity. This finding affirms that Diff adeptly manages the balance between diversity and quality. As w increases, the model demonstrates a tendency to spawn more varied trajectories. This is because higher w prompts the model to place more emphasis on unconditional noise prediction and reduce the sway of the conditional information. Thus, the model grows more proficient at creating diverse trajectories, albeit potentially compromising some quality. This greater diversity is consequence of the model having fewer constraints from specific conditions, providing more latitude to explore a wider range of possible trajectories. This experiment underscores
18

---

the flexibility and control inherent in DiffTraj in balancing trajectory quality and diversity, vital characteristics for generating realistic and diverse trajectories. Therefore, serves as control knob for modulating the trade-off between trajectory quality and diversity, providing powerful tool for users to align the generated trajectories with specific application requirements.
Figure 7: Conditional trajectory generation on Chengdu. The guidance scales w of the first, second and third are 0.1, 1, 10, respectively. The rectangular box indicates the area of the assigned start and end points.
Figure 8: Conditional trajectory generation Xi' an. The guidance scales of the first, second and third rows are 0.1, 1, 10, respectively. The rectangular box indicates the area of the assigned start and end regions.
19

---

D Visualization Results
We append series of experimental results in this section due space constraints. As shown in Fig. 9, we visualize the heat map of the trajectory distribution with multiple resolutions. Specifically, we divide the whole city into 32 32, 16 16, and 8 grids, and then count the distribution of trajectories in each grid. The comparison clearly indicates that the distributions are highly consistent from all resolutions. The visualized results also verify the effectiveness of metrics in Table 1, revealing that the proposed model can generate high-quality trajectories with remarkable accuracy and retain the original distribution.
30000
14
0.04
32.40
34.22
3622
3422
0.05
0..3
0.04
TACO9 166.0
1OC de
354.9
334-00 106.8
13:.01
100
1S4 16K:9
1E4-1 104
0.03
3422 38:4
34.20
0.01
10428
506.06
204
334.25
164.09
23
103.0
106.3
10336
(a) Chengdu.
(b) Xi' an.
Figure 9: Comparison of the real and generated trajectory distributions with different resolutions. The city is divided into different size grids (32 x 32, x 16, and 8 grids).
In addition, we also show the geographic results of the trajectories generated by different generation methods for two cities, Chengdu and Xi' The visualization results are concluded Fig. 10 and Fig. 11.
(a) VAE
(b) TrajGAN
(c) Diffwave
(d) Diff-scatte
(c) Diff-wo/UNet
(f) Diff-LSTM
(g) Diff-Traj
(h) Real
Figure 10: Geographic visualization of generated trajectory in Chengdu.
The rest visualize the forward trajectory addition noise process and reverse trajectory denoising process of Diff-Traj.
20

---

(a) VAE
TrajGAN
Diffwave
(d) Diff-scatter
Diff-wo/UNet
(I) Diff-LSTM
(g) Drff-Traj
(h) Real
Figure 11: Geographic visualization of generated trajectory Xi an.
Figure 12: Forward trajectory noising process (Chengdu).
Figure 13: Reverse trajectory denoising process (Chengdu).
Figure 14: Forward trajectory noising process (Xi aina.
Figure 15: Reverse trajectory denoising process (Xi' an).
21


Reviews:
[{"review_id": "tpTpW5Ez4O", "reviewer": "Reviewer_nB5g", "summary": "This paper adapts DDPM for trajectory generation within smart cities. The major contributions include the combination of different factors and the integration of some existing modules. The experiments over two real-world datasets can demonstrate the efficacy of the proposed model.", "questions": "Please answer the questions in the weaknesses. ", "limitations": "No", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-written and easy to follow. \n2. The paper addresses an essential and important task in spatio-temporal data mining.\n3. The experiments demonstrate the effectiveness of the proposed model over two real-world datasets.", "weaknesses": "1. Technical contribution is a bit weak. DDPM is popular in the era of AIGC. The paper adapts DDPM to trajectory generation but lacks novel or innovative designs. This drawback can be also seen in recent studies that borrow the idea of DDPMs to spatio-temporal data mining, e.g., DiffSTG [1]. By the way, this paper is very similar to DiffSTG, from model design to speed-up. \n2. The paper suggests that incorporating the influence of external factors is challenging, but this claim may not be entirely supported. To the best of my knowledge, existing models can easily handle such influence, e.g., conditional GAN. As the proposed model simply integrates these factors as a condition in DDPM, it is unclear why capturing external factors would pose a significant challenge. Additionally, the paper lacks an ablation study that examines the impact of these external factors on the proposed model's performance.\n3. To establish the generalizability of the proposed model, more real-world datasets from diverse applications and domains are needed. The exclusive use of datasets from DiDi in this paper limits the scope of the model's applicability and may not reflect its performance on other datasets.\n4. The paper omits some important related work, such as [1, 2, 3]. Furthermore, [3] and [4, 5] (included in the paper) should be used as baselines for comparison to provide a comprehensive evaluation of the proposed model's performance.\n\n\nRef:\n\n[1] Wen, Haomin, et al. \"Diffstg: Probabilistic spatio-temporal graph forecasting with denoising diffusion models.\" arXiv preprint arXiv:2301.13629 (2023).\n\n[2] Yuan, Yuan, et al. \"Spatio-temporal Diffusion Point Processes.\" arXiv preprint arXiv:2305.12403 (2023).\n\n[3] Feng, Jie, et al. \"Learning to simulate human mobility.\" Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 2020.\n\n[4] Liu, Xi, Hanzhou Chen, and Clio Andris. \"trajGANs: Using generative adversarial networks for geo-privacy protection of trajectory data (Vision paper).\" Location privacy and security workshop. 2018.\n\n[5] Zhang, Jing, et al. \"Dp-trajgan: A privacy-aware trajectory generation model with differential privacy.\" Future Generation Computer Systems 142 (2023): 25-40.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for recognizing the importance of our work and for the well-written paper. We also appreciate the detailed comments posed by the reviewer. Please find below the point-to-point responses to the reviewer's comments.\n\n> [W1]\n\nThank you for your feedback on our paper's technical contributions. While our work is indeed inspired by the success of DDPMs in generative models, adapting them to spatial-temporal trajectory data presented unique challenges due to differences like data dimensionality and temporal variations. Our Traj-UNet architecture is a testament to the innovations required to address these challenges.\nWhile both our work and DiffSTG  leverage DDPM-inspired models for the spatial-temporal domain, there are notable distinctions:\n\n1. Purpose: DiffSTG focuses on traffic prediction, offering probabilistic traffic flow forecasts. In contrast, our work centers on trajectory generation, a generative task.\n2. Performance: DiffSTG has limitations in prediction, whereas our model excels in trajectory generation, underscoring DDPMs' suitability for generative tasks.\n3. Structural Designs: We explored various structures, including Wavenet and unconditional designs, eventually validating the efficacy of the Traj-UNet structure through rigorous experimentation.\n\nOur primary goal with DiffTraj was to address privacy concerns in GPS trajectory data. By melding DDPM strengths with trajectory-specific innovations, we've aimed to balance leveraging established methods and introducing novel techniques.\n\n> [W2]\n\nThank you for your comment. While models like conditional GANs can incorporate external factors, the challenge we want to emphasize is the one we have in applying DDPM to trajectory generation. We'll clarify this distinction in our revised manuscript. Our model is designed not only to capture these interactions, but we can also use it as a guide for trajectory generation (We also present the corresponding experimental results in **Appendix C.2** and **Appendix C.3**). \n\nRegarding the ablation study, we've addressed this in the experimental section (**Table 1**). The performance metrics of **DiffTraj-wo/Con**, without external conditions, are notably inferior, underscoring the significance of these external factors. These factors not only enhance model performance but also guide trajectory generation. In Fig. 7 and Fig. 8 in the Appendix, we see that DiffTraj can specify the start and end areas of trajectories based on this. In addition, we can also customize the length, distance, travel time, etc., of the generated trajectories.\n\n>[W3]\n\nThank you for pointing out the importance of the generalizability of our proposed model across diverse datasets. We wholeheartedly agree that relying solely on DiDi datasets could limit our model's perceived applicability. To address this concern, we have conducted additional experiments using the **Porto** dataset, allowing us to test our model's robustness and adaptability further. The main experimental results and metrics on this dataset perform as follows:\n\n|Methods|Density|Trip|Length|Pattern|\n  |--|--|--|--|--|\n  |VAE       |0.0121|0.0224|0.0382|0.763|\n  |TrajGAN   |0.0101|0.0268|0.0332|0.765|\n  |Diffwave  |0.0106|0.0193|0.0266|0.799|\n  |Diff-LSTM |0.0092|0.0141|0.0255|0.828|\n  |DiffTraj  |0.0087|0.0132|0.0242|0.847|\n\nOur model still achieves superior performance on the new dataset, which is comparable to its performance on the DiDi datasets. These results reinforce the generalizability of our model across different datasets and domains. We believe that including the **Porto** dataset and the DiDi datasets provides a more comprehensive evaluation of our model's performance and applicability.\n\n> [W4]\n\nThank you for your feedback. We will revisit the works you mentioned ([1, 2, 3]) and acknowledge their relevance to our study. We will incorporate a discussion of these works, highlighting their contributions and differentiating our approach from theirs. This will provide readers with a more complete understanding of the state of the art and the novelty of our proposed model.\n\nWe appreciate your suggestion to use [3] and [4, 5] as baselines for comparison. Including these works in our evaluation will indeed provide a more comprehensive assessment of our model's performance.\nAmong them, the TrajGAN in our work is exactly the model proposed in the literature [4]. The main difference between the literature [4] and the literature [5] is that the former uses CNNs, and the latter uses LSTMs. In this regard, we have reproduced both of them, and the results are as follows:\n\n||**Chengdu**||||**Xi'an**||||\n|--|--|--|--|--|--|--|--|--|\n||**Density**|**Trip**|**Length**|**Pattern**|**Density**|**Trip**|**Length**|**Pattern**|\n| TrajGAN [4]    |0.0125|0.0497|0.0388|0.502 |0.0220|0.0512|0.0386 | 0.565|\n| DP-TrajGAN [5] |0.0117|0.0443|0.0221|0.706 |0.0207|0.0498|0.0436 | 0.664|\n\nHowever, we want to clarify the core differences between our approach and the methodology in [3]. In [3], the trajectory data is converted into a grid-based representation, and they focus on simulating human behavior by predicting these grids. This grid-based approach inherently changes the nature of the data and the problem, making it a distinct task from the continuous trajectory generation in our work. Our model is designed to handle and generate exact trajectory data without converting it into a discrete grid format. Given these foundational disparities, a direct comparison might not be meaningful.\n\nNevertheless, we value the contribution of [3] to human mobility simulation and will highlight its significance while distinguishing our methodology in the revised manuscript. We trust this clarifies our stance and appreciate your feedback.\n\n"}, {"review_id": "e6wXQsdVMi", "reviewer": "Reviewer_gS1x", "summary": "The paper introduces a good approach for generating realistic GPS trajectories based on a diffusion probabilistic model. The paper addresses the challenge of generating personalized trajectories that capture both temporal and spatial dependencies while ensuring privacy preservation. \nThe paper proposes the DiffTraj framework, which combines a personalized transition matrix and a diffusion process to generate trajectories that closely resemble real-world GPS data. The personalized transition matrix captures the individual movement patterns of users, while the diffusion process introduces randomness and ensures diversity in the generated trajectories. Overall, the model is reasonable.\nThe paper presents a comprehensive evaluation of the DiffTraj framework using real-world datasets. The experimental results demonstrate that DiffTraj outperforms existing trajectory generation methods in various aspects. The proposed approach achieves high accuracy in replicating individual movement patterns while maintaining privacy by generating trajectories that deviate from the original data.\n", "questions": "1 How does the proposed diffusion probabilistic model capture both temporal and spatial dependencies in the trajectory generation process? \n2 Can you elaborate on the generalizability of your approach to different geographic areas and datasets? \n3 Can you explicitly state the limitations of your approach and discuss potential avenues for addressing them in future work? Acknowledging limitations and outlining future research directions would guide the community in further advancing trajectory generation methods.\n", "limitations": "The generalizability of the approach to diverse scenarios and datasets could be a limitation that warrants further exploration.\n", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1 The paper introduces a unique approach to GPS trajectory generation by utilizing a diffusion probabilistic model, setting it apart from existing methods that may rely on different techniques or assumptions. The proposed diffusion probabilistic model captures both temporal and spatial dependencies, allowing for more accurate trajectory generation. This consideration enhances the realism and relevance of the generated trajectories. The diffusion probabilistic model captures both spatial and temporal dependencies, enabling the generation of trajectories that reflect the underlying dynamics of user movements.\n2 Incorporating user preferences into the trajectory generation process adds a personalized aspect, enhancing the relevance and usefulness of the generated trajectories for individual users. This feature contributes to the practicality and applicability of the proposed method. The generated trajectories exhibit high diversity, capturing the variability in user behaviors and preferences. This diversity expands the range of applications where the trajectories can be utilized.\n3 The paper provides a comprehensive comparison with baseline methods, showcasing the advantages of the proposed approach. The comparisons highlight the strengths and improvements of the diffusion probabilistic model for trajectory generation.  The authors provide sufficient details and code availability, enabling other researchers to reproduce the experiments and verify the results. This transparency contributes to the reliability and integrity of the presented work.\n4 The paper discusses potential applications of the proposed method beyond trajectory generation, such as urban planning, transportation analysis, and location-based services. This discussion broadens the scope of the paper and highlights its practical relevance.\n", "weaknesses": "1 The paper could provide more in-depth discussions on the assumptions made by the diffusion probabilistic model and their potential impact on the generated trajectories. Understanding these assumptions is crucial for interpreting and contextualizing the results.\n2 No clear instruction is provided on how the hyperparameters are selected for DiffTraj. \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are delighted that the reviewer found our motivations and ideas interesting and original. Thank you for your positive opinions and insightful comments.\n\n> [W1]\n\nThank you for highlighting the importance of discussing our diffusion probabilistic model's assumptions.\n\n1. Noise assumption: we assume that the observed data is a noisy version of potentially clean data. This allows the model to reverse the noise addition process, aiming to recover the original noise-free trajectory. In this way, the quality and nature of the generated trajectories are influenced by the assumed noise distribution. If the real-world noise is significantly different from the assumed noise, this may result in discrepancies in the generated trajectories.\n2. Non-equilibrium thermodynamic assumptions: In urban mobility, we assume that \"particles\" are individuals moving in an urban environment and that the \"medium\" symbolizes the urban space itself. Through a step-by-step denoising process, the disordered particles eventually follow specific paths in the medium, i.e., road trajectories in the city.\n\n\n> [W2]\n\nThank you for pointing out the hyperparameter selection. We apologize for the oversight and appreciate the opportunity to clarify this aspect.\nThe selection of hyperparameters for DiffTraj was guided by a combination of empirical experience and adherence to general settings commonly used in similar models and tasks. Here's how we approached it:\n\n1. Empirical experience: We leveraged insights gained from previous work and experimentation to choose values (like input length) that align with the specific characteristics and requirements of our model. \n2. General Settings: We also referred to general settings and best practices in the field, considering hyperparameters that have been shown to be effective in related models and tasks. This helped ensure that our choices were grounded in established knowledge and methodologies.\n3. Iterative Refinement: While our initial selection was based on experience and general settings, we conducted iterative experimentation to fine-tune the hyperparameters. This process allowed us to identify the optimal combination that achieved the desired performance on our validation datasets.\n\nWe recognize the initial manuscript's lack of detail in this area and will expand on our hyperparameter selection process in the revised version, detailing our choices, their rationale, and any guiding references.\n\n> [Q1]\n\nThank you for your feedback. This is indeed a central aspect of our model, and I'm pleased to explain how it is achieved:\n\n1. UNet structure: Individual Blocks in the UNet structure capture the relationships between consecutive points in the trajectory, essentially capturing spatial dependencies. Meanwhile, its multilevel structure and skip connections allow for the fusion of global and local contextual information to capture temporal dependencies.\n2. External factor representation: The model represents external spatial-temporal dependencies through conditional embedding, where spatial (start/end area, travel distance, average distance) and temporal (departure time, travel time, average speed, etc.) features are encoded. \n3. Diffusion process: By iterative denoising the data through a reverse denoising process, the model captures the underlying spatial structure of the trajectories. In addition, the training objective of the diffusion probabilistic model aims to minimize the differences between the generated trajectories and the real trajectories in both spatial and temporal dimensions. This ensures the model learns to capture and reproduce the spatial-temporal patterns inherent in the training data.\n\n\n\n> [Q2] & [Limitations]\n\nThank you for highlighting the importance. We appreciate the opportunity to elaborate on this.\n\nDiffTraj is designed to capture patterns and behaviors inherent in trajectory data, which ensures the model can be adapted to different data sources and contexts. This means that the model can adapt to different geographic regions. Furthermore, Section 5.4 Transfer Learning in this paper validates this by showing that for a new city, only 5% of the data needs to be used for fine-tuning to enable the models to achieve significant performance in a new scenario. These results show that the DiffTraj model exhibits strong adaptability and generalization capabilities when applied to different urban scenarios. \nTo further validate the generalizability of our approach, we conducted experiments on a different dataset **Porto** dataset.\n\n|Methods|Density|Trip|Length|Pattern|\n  |--|--|--|--|--|\n  |VAE       |0.0121|0.0224|0.0382|0.763|\n  |TrajGAN   |0.0101|0.0268|0.0332|0.765|\n  |Diffwave  |0.0106|0.0193|0.0266|0.799|\n  |Diff-LSTM |0.0092|0.0141|0.0255|0.828|\n  |DiffTraj  |0.0087|0.0132|0.0242|0.847|\n\nThrough the new experiment, we can see that DiffTraj achieves the best performance on this dataset, demonstrating its ability to adapt to different geographic regions and capture different trajectory patterns. These results underscore the model's generalizability across different datasets.\n\n> [Q3]\n\nThank you for emphasizing the need to address our approach's limitations and potential future directions. Here are the key points:\n\n1. Our method relies on raw data for synthesis, ensuring that generated trajectories mirror real-world patterns. This dependency implies that the quality and characteristics of the raw data can influence the synthesized trajectories. While it also means that any biases or anomalies in the raw data might be reflected in the synthesized data.\n2. While our trajectory generation is innovative, it's computationally demanding. Despite this, our approach still represents a more cost-effective solution than real-world data collection costs.\n\nWhile our work has limitations, it marks a notable step forward in trajectory generation. We are dedicated to refining our approach in future research, building on our established foundation.\n\n\n\n\n"}, {"review_id": "zZrFa3DOMR", "reviewer": "Reviewer_uaXp", "summary": "The paper provides an innovative diffusion probabilistic-based model to simulate realistic GPS trajectories. The main contributions are as follows: 1) The paper introduces a diffusion-based probabilistic model that captures spatio-temporal dependencies in GPS trajectories. This model allows for personalized trajectory generation, considering individual preferences and behaviors. 2) The proposed approach generates personalized trajectories for individual users, enhancing the realism and accuracy of the generated trajectories compared to traditional methods. 3) By incorporating real-world GPS data, the generated trajectories closely resemble actual user movements, making them suitable for various location-based applications. 4) The paper demonstrates the scalability of the proposed method, enabling the generation of large-scale trajectory datasets efficiently. 5) The authors perform an extensive experiment of the proposed approach, comparing it with existing trajectory generation methods, showcasing its superiority in terms of trajectory realism and diversity.\n\nOverall, the paper presents a significant advancement in trajectory generation by introducing a diffusion probabilistic model and demonstrating its effectiveness in generating personalized and realistic GPS trajectories. The approach has the potential to impact various domains, including LBSs, transportation, and urban planning.\n", "questions": "1. How does DiffTraj handle potential privacy concerns associated with generating realistic trajectories?\n2. Could you provide more details on the selection of baseline methods for comparison and why these specific methods were chosen? Are there any plans to release the implementation or codebase of the proposed method to facilitate replication and further research in trajectory generation?\n", "limitations": "The availability and accessibility of the datasets used in the experiments should be clearly stated.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1.Generating realistic and personalized GPS trajectories has wide-ranging applications in various fields. The paper addresses this important problem and proposes a solution that has the potential to enhance applications such as LBSs, transportation, and urban planning. The diffusion probabilistic model offers a novel perspective and advances the state-of-the-art in trajectory modeling and generation.\n\n2.The proposed approach generates personalized trajectories, allowing for a more accurate representation of individual user movements. This customization increases the applicability of the method in various domains. The generated trajectories closely resemble actual user movements, exhibiting realistic patterns and behaviors. This realism enhances the reliability and usability of the trajectories for real-world applications.\n\n3.The authors present a well-designed study, including an extensive evaluation of their method using real-world datasets and appropriate statistical analysis. The experiments and comparisons with baseline methods demonstrate the robustness and effectiveness of their approach. The evaluation is thorough and includes relevant metrics, ensuring the validity and reliability of the presented results.\n\n4.The proposed approach demonstrates scalability and efficiency in handling large datasets. This characteristic makes it suitable for real-time applications, where processing speed and performance are crucial. The work presented in the paper has the potential to advance the field of trajectory generation, providing a valuable tool for generating context-aware GPS trajectories. The paper's contributions can benefit various domains that rely on precise GPS data for decision-making and analysis.\n\n5.The paper is well-written and organized, effectively conveying the methodology, experimental setup, and results. The logical flow and clear explanations contribute to the ease of understanding for readers.\n", "weaknesses": "1. Some details of experiment setup are not clearly provided, such as datasets, metrics, and baselines.\n\n2. While the paper provides detailed descriptions of the proposed method, the absence of an openly available implementation or codebase limits reproducibility and further exploration by the research community.\n\n3. It is better to discuss about the generality of the propose model. Can it be applied to other application domains?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewers for their insightful comments and perspectives. We respond to each of the points as follows:\n\n> [W1]\n\nThank you for pointing out the experimental setup. We understand the importance of these details for the reproducibility of our work.\nTo address this, we have provided an in-depth description of the experimental setup, including details about the datasets, metrics, and baselines, in the **Appendix** of the supplementary material. We apologize for any oversight in not making this more explicit in the main manuscript.\nBrief Overview:\n\n1. Datasets: **Appendix B.1** contains a detailed description of the datasets used, including their sources, characteristics, and preprocessing steps.\n\n2. Metrics: We have elaborated on the metrics employed for evaluation, providing both their definitions and the rationale behind their selection in **Appendix B.2**.\n\n3. Baselines: The baselines chosen for comparison are detailed in **Appendix B.3**, along with explanations for their relevance and the context in which they were used.\n\n4. Additional experiments: A number of additional experiments are provided in **Appendix C**, including downstream tasks, conditional generation, generating diversity, and more.\n\nIn light of your feedback, we will ensure that in the revised manuscript, we provide clear references and pointers to the relevant sections in the Appendix. \n\n> [W2]\n\nThank you for emphasizing the importance of reproducibility and the availability of our implementation.\nWe have included the implementation code of our proposed method in the **supplementary material**. We apologize for any oversight in not making this more explicit. In light of your feedback, we will ensure that in the revised manuscript, we provide clear references and pointers to the relevant sections in the supplementary material where the code is available. \n\n> [W3]\n\nThank you for emphasizing the importance of discussing the generality of our proposed model.Our model, while initially designed for trajectory generation in urban traffic mobility analysis. Here are our potential ideas for its wider application:\n\n1. Temporal series data generation: The core of the trajectory data is a sequence of continuous GPS points. This means that with appropriate training data, the model can be adapted to other domains related to time series generation.\n2. Spatial and temporal dynamics: The model's ability to capture both spatial and temporal dynamics is not limited to traffic patterns. Any application domain that involves spatial-temporal data, such as act analytics or even certain financial time series, could benefit from our approach.\n\n\n\n> [Q1]\n\nThank you for raising this important issue. The privacy issue is also what motivates and centers our work. Here's how DiffTraj addresses these issues:\n\n1. Learning trajectories distribution: While DiffTraj aims to generate realistic trajectories, it does not replicate exact real-world trajectories of individuals. Instead, it learns the general patterns and structures present in the data, ensuring that the generated trajectories are representative but not exact replicas of real-world movements. Moreover, it prevents the possibility of reverse engineering real trajectories from synthetic ones.\n2. Noise generation: DiffTraj generates trajectories by stepwise denoising from random noise. By reconstructing trajectories from random noise during the reverse diffusion process, the model effectively decouples synthetic data from specific real data points. This ensures that the generated trajectories do not contain personally identifiable information or reveal sensitive location details, thus protecting their privacy.\n\n> [Q2.1]\n\nThank you for highlighting the discussion on the selection of baseline methods. The selection of baseline methods was driven by several key considerations:\n\n1. We prioritized methods specifically designed for or shown promise in trajectory generation or related tasks. This ensures that our comparisons are directly relevant and meaningful.\n2. To provide a comprehensive evaluation, we selected methods that represent a diverse range of approaches to trajectory generation. This diversity allows us to understand the strengths and weaknesses of our method in relation to different strategies and paradigms.\n3. To validate the effect of individual components of the model (e.g., UNet, Conditional module), we performed ablation experiments on the modules. Comparison with these methods validates the contribution of the individual modules of the model.\n\n> [Q2.2]\n\nThank you for inquiring about the availability of the implementation code for our proposed method. We are committed to promoting reproducibility and further research in the field of trajectory generation. To this end, we have already provided the complete implementation of our proposed method. The codebase and necessary scripts and instructions are available in the **supplementary material** attached to our submission.\n\nIn the revised manuscript, we will ensure that we provide clear references and pointers to guide readers and reviewers to the provided code in the supplementary material. We appreciate your interest in our work and hope that the availability of our code will facilitate replication and further advancements in trajectory generation research.\n\n> [Limitations]\n\nThank you for highlighting the importance of clearly stating the availability and accessibility of the datasets used in our experiments. We have provided comprehensive details regarding the datasets, including their sources, accessibility, and any relevant licensing information, in the supplementary material. We understand the significance of these details for reproducibility and further exploration by the research community, and we apologize for any oversight in not making this more prominent in the main manuscript. We appreciate your feedback, which has been instrumental in enhancing the clarity and thoroughness of our paper."}, {"review_id": "oGRKpfg3AC", "reviewer": "Reviewer_8UQL", "summary": "The paper proposes DiffTraj: a trajectory generation model based on probabilistic diffusion. The model is trained using real spatio-temporal trajectory data, and aims to generate new trajectories with similar characteristics. This is motivated by the purpose of preserving privacy information that may be present in the actual data. Like all diffusion-based models, the proposed model has a forward and a reverse process, and the latter has an U-Net component, whose importance is highlighted in the ablation studies. Contextual information such as the starting time and location can be provided as input to the model in an encoded form, using a neural network trained on the actual dataset. The influence of this contextual information on the generated data can be controlled by a parameter. It is shown that the trajectories generated by DiffTraj are statistically more similar to the original ones (in two datasets) compared to other trajectory generation models. It is also shown that the trajectories generated by DiffTraj can be used instead of the original trajectories to train models for downstream tasks such as inflow and outflow predictions from particular blocks in the cities.", "questions": "1) I would expect the trajectories in any city to have \"clustering\" tendencies, some origins and destinations maybe \"hotspots\", some road segments (eg. highways) maybe \"hotspots\", etc. Similarly, there may be peak hours of trajectories. Or, certain types of trajectories may be more common in certain hours of the day (eg. those heading to downtown office areas during morning hours). Do you think DiffTraj can produce these features in their simulations?\n2) If not, is it possible to incorporate them into the model somehow?\n3) Can we have evaluation metrics based on time and velocity, instead of only space-based measures (trip error, length error etc)?\n4) In the downstream experiment, it is shown that various models perform almost equally when trained using the generated data instead of the actual data. But to really make the point, it should be shown that these models would have performed less well had they been trained with trajectories generated by some other baseline models\n5) What is the exact role played by the U-Net in the reverse process? Could any other architecture not have played the same role?\n6) If we want trajectories for a particular downstream task, can we get better results if we can use task-specific loss functions while training?\n7) What do the figures in Fig2 convey? Are they obtained by overlaying a set of trajectories? ", "limitations": "NA", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) A new model DiffTraj is proposed for trajectory generation. It uses the currently popular probabilistic diffusion framework, but modifies it to bring in encoded contextual information, and an U-Net component in the reverse process to predict the noise levels for every next step\n2) Good generative performance is shown, where the trajectories generated by this approach are shown to be statistically more similar to the original ones compared to those generated by other methods\n3) The authors explore various practical uses of their model, such as downstream tasks (eg. inflow/outflow prediction), transfer learning (training model on one city and adapting it to another) etc\n", "weaknesses": "1) The model considers all trajectories as IID and does not aim to utilize any notion of \"trajectory clustering\" into their model. [In their defence, the encoded contextual information may carry such clues]\n2) The downstream task experiment is introduced almost as an afterthought, though such tasks seem to be the main motivation for the work. Also, these experiments seem to be incomplete [see questions below]\n3) The model includes U-Net as an important component for the reverse process, and its success is highlighted through ablation studies. But the exact role or contribution of the U-Net is not discussed\n4) The evaluation metrics used seem to be based mainly on spatial aspects, not temporal aspects\n\nOverall, I like the broad idea of the paper. I am willing to improve my ratings if the authors can address my concerns.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We highly appreciate your high-quality review and valuable suggestions. Due to space limitations, we merged some of the weaknesses and questions you mentioned. We also added a new **one-page PDF** of the results. Please kindly check it out. We clarify your concerns below:\n\n> [W1]\n\nThank you for your insightful observation regarding the treatment of trajectories in our model. Indeed, we treat trajectories as IID to ensure broad applicability (e.g., transfer learning) across datasets. While our model does not explicitly employ \"trajectory clustering,\" the contextual information encoding effectively captures the inherent similarities and differences among trajectories. As seen in **Fig. 10**, DiffTraj effectively simulates the original trajectory distribution with brighter main roads and darker surrounding areas. In addition, we also found that using only the model to generate scatter points, i.e., Diff-scatter can also present the non-IID distribution properties.\n\n> [W2] [Q4]\n\nThank you for your constructive feedback regarding the downstream task experiment. We acknowledge the importance of this experiment and conducted the corresponding baseline experiments to address this:\n\n|| MSE/RMSE/MAE (inflow) | MSE/RMSE/MAE (outflow) |\n| -- | -- | -- |\n| GAN| 6.78/2.60/1.76|5.97/2.44/1.68|\n| VAE|6.79/2.61/1.76|5.81/2.41/1.66|\n| Diffwave|4.62/2.16/1.52| 4.75/2.18/1.56|\n| Original|4.42/2.10/1.50 | 4.64/2.15/1.53|\n\nWe train **GWNet** models using the trajectories generated by the baseline method and compare their performance with models trained using the original trajectories(Due to the limited space of the rebuttal, we give the GWNet prediction results).\n\nOur results show that the training results for trajectories generated using DiffTraj are closer to the real context than using the baseline generation method, in line with the trends shown in Table I. We believe that these additional experiments and the revised presentation will provide a more comprehensive and clearer assessment of our modeling capabilities.\n\n> [W3] [Q5]\n\nIn this work, UNet offers several distinct features:\n\n1. Feature extraction and recovery: The down sampling (encoder) of the UNet captures contextual information and low-level features in the trajectory by reducing the spatial dimensions and increasing the depth (number of channels). Up sampling (decoder) of the U-Net increases the spatial dimensions while decreasing the depth. UNet can recover high-resolution features of the trajectory.\n2. Skip Connections: These ensure the retention of fine-grained details, especially crucial during detailed data reconstruction.\n3. Combination of local and global information: UNet can consider both local and global multi-level information.\n\nThese features enable UNet to precisely predict noise levels during the reverse denoising process for trajectory generation. **Fig. 13** and **Fig. 14** visualize this reverse denoising process. In addition, We also experimented with a non-UNet structure (**Diff-wo/UNet** and **Diffwave**), but they didn't produce satisfactory results.\n\n\n\n> [W4] [Q3]\n\nWe appreciate your suggestion on time and velocity-based metrics. We tested the similarity between the generated results and the original using the **Kolmogorov-Smirnov statistic** at the time level and at the velocity level. The metrics yielded 0.94 and 0.93 for the Chengdu dataset and for the Xi'an dataset, 0.94 for both. This indicates that the generated trajectories can also simulate the distribution of the original trajectories well in terms of the temporal profile.\nIn addition, we visualize the results of both metrics for a better view. Please kindly see Figure 2 in the **rebuttal PDF**.\n\n\n\n> [Q1] [Q2]\n\nThank you for your in-depth insights. We are pleased to confirm that our proposed model can exhibit specific patterns and features. We provide visual evidence of DiffTraj's capability to simulate these patterns in the **rebuttal PDF**.\n\nFor the hotspots you mentioned, it can be seen from **Fig. 2** or **Fig. 10** that the number of trajectories clustered on different road segments is different, thus revealing the hotspots of the road. In addition, in **Fig. 9**, we compare the clustering tendencies of trajectories within the city (we use a heatmap in this paper). We can clearly observe that the clustering tendencies of the generated trajectories maintain a high level of consistency with the original trajectories.\n\nRegarding peak hours and time-specific patterns, our model can identify and simulate patterns associated with peak hours and speed attributes. This is demonstrated by a high number of peak-hour trips with slower speeds and fewer trips with faster average speeds during free periods (e.g., midnight).\n\n> [Q6]\n\nThank you for raising this important point for the use of task-specific loss functions.\nWe think incorporating task-specific loss functions can be beneficial when the generated trajectories are intended for a specific downstream task. By optimizing for a task-specific loss, the model can be guided to generate trajectories that are more aligned with the requirements and nuances of the target task. For example, if the task is related to predicting pedestrian movement in a shopping mall, the loss function might emphasize trajectories that align with typical shopping behaviors or patterns. We believe it's a valuable direction for future work.\n\n> [Q7]\n\nYes, Fig. 2 is obtained by mapping a set of trajectories onto the map (see **Fig. 10** for a larger view of the effect). This figure mainly wants to convey the following messages: \n1. Compare the geographical distribution of different trajectory generation methods on a real map. \n2. Since the trajectory drawn overlapped, this will lead to brighter colors where there are more trajectories, which can approximately reflect the non-IID distribution of all trajectories (main roads or side roads). In addition, this may partially answer your concerns in [W1] and [Q1] about the clustering of trajectories."}]

Author rebuttal:
We appreciate the insightful comments and perspectives of the reviewers, and the attached figures and tables of results are included as a supplement **PDF** to the rebuttal.

Meta-review: Initially, the paper raised several concerns:

Privacy Claims and Lack of Evidence: Although the paper prominently claims to prioritize privacy, there was a distinct lack of empirical support to validate these claims. While the authors demonstrated that training on their generated data produced results similar to the original data, there was an underlying concern about potential data copying by the generative model.

Dependence on Original Data: The necessity to rely on the original data to train the generative model posed questions about the genuine advantages and novelty of using the generated data.

Weak Comparisons: The benchmarking seemed inadequate. Relying mostly on self-comparisons or using outdated models for evaluation diminishes the paper's value as it doesn't offer a comprehensive view of its standing within current advancements in the field.

However, post the initial review:

Rebuttal Improvements: The authors have made commendable efforts to address the concerns raised during the initial review. They conducted additional experiments which, evidently, clarified and resolved some of the initial apprehensions. This proactive approach has led to a noticeable shift in the perception of the paper, with many reviewers now providing more favorable ratings.

The increased ratings from the reviewers post-rebuttal indicate a positive shift, but it's essential to ensure that all significant concerns will be thoroughly addressed in the final version.
Decision: Accept (poster)