Three-Dimensional Trajectory Prediction with 3DMoTraj Dataset
Authors: Hao Zhou
Xu Yang
Mingyu Fan
Lu Qi
Xiangtai Li
Ming-Hsuan Yang
Fei Luo
Venue: ICML 2025 poster (2025)
Keywords: 3D trajectory prediction
3D trajectory dataset
Decoupled trajectory prediction
Primary area: applications->robotics
TL;DR: We introduce 3DMoTraj, a large-scale dataset for 3D trajectory prediction from underwater vehicles, and propose a decoupled prediction method that significantly reduces the complexity of predicting 3D trajectories.

Abstract: With the growing interest in embodied and spatial intelligence, accurately predicting trajectories in 3D environments has become increasingly critical. However, no datasets have been explicitly designed to study 3D trajectory prediction. To this end, we contribute a 3D motion trajectory (3DMoTraj) dataset collected from unmanned underwater vehicles (UUVs) operating in oceanic environments. Mathematically, trajectory prediction becomes significantly more complex when transitioning from 2D to 3D. To tackle this challenge, we analyze the prediction complexity of 3D trajectories and propose a new method consisting of two key components: decoupled trajectory prediction and correlated trajectory refinement. The former decouples inter-axis correlations, thereby reducing prediction complexity and generating coarse predictions. The latter refines the coarse predictions by modeling their inter-axis correlations. Extensive experiments show that our method significantly improves 3D trajectory prediction accuracy and outperforms state-of-the-art methods. Both the 3DMoTraj dataset and the method are available at https://github.com/zhouhao94/3DMoTraj.

Three-Dimensional Trajectory Prediction with 3DMoTraj Dataset
Hao Zhou 123 Xu Yang" Mingyu Fan
Lu Qi
Xiangtai Li?
Ming- Ming-Hsuan Hsuan Yang"
Fei Luo 13
Abstract
majectory prediction
optining
((xaa)
2m/2j1a expl-2 2 1-V- [/)
x-1
[=-
With the growing interest in embodied and spatial intelligence, accurately predicting trajectories in 3D environments has become increasingly criti- cal. However, no datasets have been explicitly designed to study 3D trajectory prediction. To this end, we contribute a 3D motion trajectory (3DMoTraj) dataset collected from unmanned un- derwater vehicles (UUVs) operating in oceanic en- vironments. Mathematically, trajectory prediction becomes significantly more complex when transi- tioning from 2D to 3D. To tackle this challenge, we analyze the prediction complexity of 3D trajec- tories and propose a new method consisting of two key components: decoupled trajectory prediction and correlated trajectory refinement. The former decouples inter-axis correlations, thereby reduc- ing prediction complexity and generating coarse predictions. The latter refines the coarse predic- tions by modeling their inter-axis correlations. Ex- tensive experiments show that our method signifi- cantly improves 3D trajectory prediction accuracy and outperforms sate-of-theeart methods. Both the 3DMoTraj dataset and the method are avail- able at httpss//ggtuubcc///hooooo//////
of p0.0)) 1020
of
Paransors.
trajectoyy prediction
pe
arnootje of /Or.g.))
(Brjaajeje ex
2
8-14
8-1
y-
of Ayaay Ducgo
Pry0,0 Pesa,o. o
Aua.o. Auig, of
(uzs Pys Mes of of of Peci Paol
Desoupial injestary prodictius
seinaling
[a(x))
exp p.)
V2ro3
202 (g By 203 (3 Rc)
Decoupiigg the prodictin slatp jiftren aves
fv(N)
exp
/
spinniin
f.(e)
/2ro
20
Parmersee (Me o May og
Figure I: Illustrations of different trajectory predictions and their prediction complexities. The 2D,, 3D, and decoupled 3D predictions for each point correspond to optimizing one 2D, one 3D, and three ID Gaussian distributions, which involve optimizing 5, 9, and 6 free parameters, respectively.
which emphasize the ability to understand, reason about, and manipulate information within a three-dimensional (3D) space (Zhu et al., 2025; Ishida, 2024). Such advancement highlights the critical role of 3D trajectory prediction, which enables systems to effectively navigate and interact with dynamic environments by anticipating future movements in space. For example, in unmanned systems, accurate 3D motion path prediction for unmanned underwater vehicles (UUVs) (Cao et al., 2022) and unmanned aerial vehicles (UAVs) (Zhang et al., 2023; Huang et al., 2023) helps them avoid obstacles and plan efficient navigation routes in ocean and air environments.
1. Introduction
In recent years, the embodied (Wang et al., 2024: Kaur al., 2023; Qi al., 2024) and spatial intelligence (Huang et al., 2024) have witnessed rapid advancement, both of
'qqual contribution 'School of Computing and Information Technology, Great Bay Institute for Advanced Study/Great Bay University. Dongguan, China. *Tsinghua Shenzhen Interna- tional Graduate School, Tsinghua University. Shenzhen. China. Dongguan Key Laboratory for Intelligence and Information Tech- nology, Dongguan, China. *State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China. 'Institute of Artificial Intelli- gence, Donghua University. Shanghai, China. "Intta360 Research, Shenzhen, China. 'Nanyang Technological University, Singapore. a UUniversity California Merced, Merced, USA. Correspondence to: Fei Luo <luofei @ gbu.edu.en>.
Despite the significant importance of 3D trajectory predic- tion for embodied and spatial intelligence, most current trajectory prediction tasks, including pedestrian (Mohamed et al., 2020; Mangalam et al., 2020; Zhou et al., 2021) and vehicle (Gao et al., 2020; Zhou et al., 2023; Shi et al., 2022) trajectory prediction, focus primarily on 2D trajectories due to the scarcity publicly available 3D trajectory datasets. To this end, we contribute 3D motion trajectory (3DMo-
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Three-Dimensional Trajectory Prediction
Traj) dataset, which consists of 3D trajectories collected from UUVs across eight distinct oceanic scenarios. There are two reasons for collecting trajectories from UUVs: (1) the trajectories of multiple UUVs exhibit complex inter- actions arising from formation movements, and (2) ocean currents introduce fluctuations in UUV trajectories, signifi- cantly increasing the challenge of accurate prediction.
diction part employs three independent decoders predict future trajectories along the x-, y-, and z-axes, producing coarse prediction. The correlated trajectory refinement part then models the inter-axis correlations of the coarse predictions and generates offsets to refine them. Extensive experiments conducted on the 3DMoTraj dataset demon- strate the superior performance of our proposed method for 3D trajectory prediction.
The 3DMoTraj dataset is annotated with frame-wise inten- tions to facilitate future research. Existing 2D trajectory prediction methods have shown that incorporation of inten- tions (Rasouli et al., 2019; Liu et al., 2020; Mangalam et al., 2021; Choi et al., ;0 Zhao et al., 2021) can significantly improve prediction accuracy. Specifically, the trajectories in 3DMoTraa are annotated with both static and motion inten- tions. The static intention is defined as the octant the agent is expected to reach by the end of the prediction horizon, whereas the motion intention represents the mean velocity changes between the observed and predicted trajectories. Static intentions provide insight into the agent's final spatial goal, while motion intentions capture the agent's 's dynamic state transitions.
In conclusion, the contributions of this work are as follows:
We propose a 3D trajectory dataset named 3DMo Traj, which is collected from UUVs in ocean environments. The dataset includes frame-wise annotations for both motion and static intentions.
We analyze the increased prediction complexity of 3D trajectories compared to 2D trajectories and propose a 3D trajectory prediction method that consists of two components: decoupled trajectory prediction and corre- lated trajectory refinement. The former decouples the inter-axis correlations to generate coarse predictions. The latter models the inter-axis correlations to refine these coarse predictions.
Aside from the scarcity of available datasets, one major chal- lenge impeding progress in 3D trajectory prediction is its substantially higher prediction complexity. As illustrated in Figure 1, predicting 2D trajectory is generally equivalent to optimizing 2D Gaussian distributions with 5 parameters to approximate the possible locations of each 2D point (Mo- hamed et al., 2020). In contrast, 3D trajectory prediction requires optimizing a 3D Gaussian distribution with pa- rameters for each 3D point. As result, the complexity of predicting one 3D point is approximately twice that of one 2D point. Furthermore, the prediction complexity of 3D scenario involving N points is linearly related to 9 x N, significantly hindering the prediction of complex scenar- ios containing multiple 3D agents. Therefore, reducing the optimized parameters for each 3D point important.
We conduct extensive experiments on the 3DMoTraj dataset. The experimental results verify the superior performance of our method in 3D trajectory prediction, establishing a solid baseline for future research.
2. Related Work
Trajectory Prediction Datasets. Some trajectory datasets are collected from top-view cameras, such as ETH/UCY (Lerner et al., 2007: Pellegrini et al., 2009), PWRD, and NYGC (Zhou et al., 2011), which focus on pedestrian trajectory prediction. Other datasets, such as SDD (Robiequet et al., 2016) and inD (Bock al., 2020), are collected using flying drones and include trajectories of vehicles, bicycles, and pedestrians. With advances in sensor technology, high-quality datasets such as Apollo (Ma et al., 2019), Argoverse (Chang et al., 2019; Wilson et al.), and Waymo (Ettinger et al., 2021) use sensors, such as LiDAR, radar, high- deg-neefnntionn maps, and localization modules, to record trajectories. Recognizing the importance of agents' intentions in prediction, datasets like PIE (Rasouli et al., 2019) and LOKI (Girase et al., 2021) include trame-wise intention annotations. PIE focuses on short-term pedestrian intentions, while LOKI provides long-term intentions for hybrid road users. Although these datasets offer valuable resources for trajectory prediction, they mainly focus on 2D scenarios. To date, no publicly available datasets are specifically designed for 3D trajectory prediction.
To address the increased prediction complexity of 3D trajec- tory, we propose a novel 3D trajectory prediction method comprising two key components: decoupled trajectory pre- diction and correlated trajectory refinement. Optimizing 3D Gaussian distribution can be decomposed into optimizing three individual 1D Gaussian distributions and modeling their correlations: please refer Section A in the Appendix for detailed mathematical proofs. We adopt a divide-and- conquer strategy. Specifically,, 3D trajectory prediction is first decoupled along three independent axes to mitigate the impact of inter-axis correlations, followed by modeling these correlations. In this way, the prediction of each 3D point with 9 parameters is first simplified as the optimization of three ID Gaussian distributions with 6 free parameters, as illustrated in Figure 1, thereby reducing the overall pre- diction complexity. Therefore, the decoupled trajectory pre-
Trajectory Prediction Methods. These methods in 2D scenarios can be broadly categorized into RNN-based meth- ods, one-shot prediction methods, and intention intention-conditionaa conditioond methods. RNN-based methods iteratively predict future tra- jectory points. Representative works include S-GAN (Gupta
2

---

Three-Dimensionts Trajectory Prediction
(a) Orthographic Projection
(b) Front View
(c) Top View
(d) Side View
Figure 2: Orthographic projections, along with front, top, and side views of 3D trajectories for two typical underwater scenarios. Please refer to Figure in the Appendix for the rest of the scenarios.
et al., 2018), PIE (Rasouli et al., 2019), STGAT (Huang et al., 2019), Trajectron=++SSlzaann et al., 2020), and SR- LSTM(Zhang et al., 2020). One-shot prediction methods predict the entire future trajectory in a single step. Promi- nent approaches include SSTGCNN (Mohamed et al., 2020), MSRL (Wu et al., 2023). MRGTraj (Peng al., 2023), S- Implicit (Mohamed al., 2022), and FlowChain (Maeda & Ukita, 2023). iitentioncoonditioned methods predict fu- ture intentions and then generate trajectories conditioned on these predictions. Notable examples include LBEBM (Pang et al., 2021), TNT (Zhao et al., 2021), NSP-SFM (Yue et al., 2022), and MemoNet (Xu et al., 2022). Besides, interac- tion modeling techniques such as pooling (Mangalam al., 2020), attention (Salzmann et al., 2020), and graph neural networks (Mohamed et al., 2020) studied to capture the dynamics of multi-agent systems. Recent research also addresses real-world challenges, including domain adapta- tion (Feng et al., 2024) and observation length shift (Xu & Fu, 2024), which commonly arise in practical deployment. Furthermore, large language models and unsupervised learn- ing applied in time-series tasks (Sun al., 2024; Yang & Hong, 2022) also inspire research in trajectory prediction.
Annotation
Motion intention
Vy
"Speed up"
Static intention
Vi
0.
(x..Ye. Me)
2g
"Octant VII""
Figure 3: Illustration of annotations for motion and static intentions. Motion intention is determined by the velocities of the observed and predicted trajectories, while static inten- tion is based on the location of the trajectory's 's endpoint.
present the orthographic projections and the front, top, and side views of the 3D trajectories for two representative sce- narios in Figure 2. Notably, in addition to the complex 3D motion patterns of each UUV, the trajectories also ex- hibit fluctuations caused by ocean currents, which introduce significant challenges to the prediction task. This section provides an overview of the 3DMoTraj dataset and analyzes its statistical characteristics.
Trajectory prediction in 3D scenarios primarily focuses on predicting UAV trajectories. Current research on UAV trajectory prediction largely relies on traditional methods, such as Gaussian process regression (Xic & Chen, 2022), Kalman filters (Wang al., 2018), and principal component analysis (Zhang et al., 2023), as well simple learning- based methods, including I.STMs (Zhong al., 2022) and CNNs (Liu et al., 2021). However, these methods struggle to predict 3D trajectories in complex scenarios due to the high prediction complexity of 3D trajectories.
3.1. Dataset Construction
To enhance the diversity 3D motion behaviors, we collect trajectory data using three agents over an extended period.
Dataset Preprocess. The sampling rate for the trajectory points is 10 fps, resulting in very short time interval be- tween consecutive points and, thus, minimal spatial separa- tion. To address this issue, we first downsample the trajec- tory points to a sample rate of fps. Since each scenario includes only three agents, we further divide the trajectory data into short fragments, each containing 200 data frames. We assign unique IDs to agents in each fragment to enhance the representation of agents in the dataset. The fragments for each scenario are then randomly divided into training, validation, and test sets with 1:1:1 ratio.
3. 3D Motion Trajectory Dataset
The 3DMoTraj dataset comprises eight scenarios featur- ing diree-dimensional (3D) trajectories collected from un- manned underwater vehicles (UUVs) operating in ocean environments. The 3DD trajectories are recorded using po- sitioning devices installed on the UUVs. Each scenario includes three UUVs following predefined formations. We
Dataset Annotation. Intentions play a crucial role in tra- jectory prediction. We annotate intentions in the dataset to support further research in 3D trajectory prediction.
3

---

Trajectory Prediction
Secnario
#1
42
43
#4
#5
#6
47
#8
Frame number
765
1744
1010
1558
1861
3379
2597
Distancelm)
0.707=00.22
0.482:10:212
0.505:20.188 185
0.56910.287
0.491:00220
0.784:0.333
0.707:40:251
0.51440.273
Velocity(rws)
1,11=00.564
0.964400424
1009+03369
1.139+0.573
0.92240.05
1.567:0.666
1.415:005001
1.028:0.566
Acoolenaiion(/s>>
0.003 1,373
1406
0.005+00957
0,0024 361
0,00441 1045
40.00++1.68. 683
0.001+1.435
+1.12
Curvaturo(I/m) (/m)
0.181=2.128
0.419:3.199 194
0.21542.784
0.48613.591
0.15522.692
0.074
0.030.10.300
0.10742.259
Mation intentioo
1022,,
1:1954
1:997,111012.
1:17355--1350,
19881,
1-948.1:1188
1:3168.1:4800,
1:2562,1:9045
(velocity change)
-:440
->:1450
-1816
-111293
-11673
-x:370
->:1530
70:100
4:2114
#1:161,
1616,
#1:74,
#1:175, # 133,,
#1 #1:301,442-000
#1:522,42:5224
# #1:788,
#1:1363, #2 1243,
Static intention
#:3:722,
H1-:55
#33144,
(230099 (1-09991440 (444911
#:3:509,
#449775
#3:826, (4:8544
#33:291.:/4:5555
4 (3:1622 (/4951
#10:2182,
447786.
(Endpoint octant)
#33288,#661166
#3:1051,
#550000
#5:11 #9:3998
#5:57.#66711
#6 #66:800.66-422.
#55:145,66:111
#3:1772. #67788
#5:412.466772
#7:445,
#7:2141,88::44
N7:312, #83351
#7:196, 8:378
#7:911.1/8.997
# #17:47.p88:294
#7:2579, #8:1599 1264
# #7.556 294
Table I: Statistics information of the 3DMo dataset. For motion intention, notations 1. 4 and -> respectively represent
speed up, speed down, and constant speed. For static intention, notation #N denotes the endpoints in N-th octant.
Unlike datasets such as PIE (Rasouli et al.. 2019) and LOKI (Girase et al., 2021), which annotate intentions based solely on future trajectories, we propose utilizing both ob- served and future trajectories. Specifically, we compare the average velocity of the observed trajectory (1:) and the future trajectory (vy) to label samples with motion intention "speed up.' 'slow down', or 'constant speed', as shown in Figure 3. trajectory sample is labeled as 'speed up" if "y U. > 8, 'speed down' if Vy - U, < -ō, and "constant speed' if Juy vol S 8, with 8 set to 0.1 by default.
contains over 13000 frames, offering sufficient data for model training and validation.
Motion information. Secondly, we present the average distance, velocity, and acceleration of the trajectory points in the third to fifth rows. The average distance across all scenarios ranges from 0.4 to 0.8 meters, with corresponding average velocities between 0.9 and 1.6 meters per second, which are relatively high speeds for UUVs in ocean environ- ments. Additionally, the significant standard deviations in velocity and acceleration indicate frequent velocity changes among agents in our dataset. The distributions of distance, velocity, and acceleration, presented in the Appendix, fur- ther illustrate this variability. These characteristics make the dataset well-suited for evaluating the ability of prediction methods to handle complex velocity dynamics.
Additionally, we annotate future location intention, referred to as static intention, based on the endpoint of each trajectory sample, as illustrated in Figure 3. Specifically, we define the current point as the origin, with the velocity direction at that point serving as the x-axis. Using the Frenet-Serret frame, we establish local coordinate system, within which the endpoints are classified into one of eight octants. This octant-based labeling provides trajectory samples with static intention. As a result, our dataset includes annotations for both motion and static intentions.
Curvature information. Thirdly, calculate the aver- age 3D curvature of the trajectory points, reported in the sixth row. The lowest average curvature is 0.074 in scenario 1, while the highest is 0,486 in scenario 4. Notably, the standard deviation of curvature in each scenario is approxi- mately ten times its mean value. The curvature distributions across all scenarios, visualized in the Appendix, further support this variability. These statistics indicate that the trajectories in our dataset exhibit wide range of curvatures, providing a robust benchmark for evaluating the prediction ability of methods on complex curvilinear paths.
Dataset Usage. Following the setting in datasets ETH (Pel- legrini et al., 2009) and UCY (Lerner et al., 2007), we adopt sliding window technique to generate samples from the dataset. Each sample contains 20 frames of trajectory points, with the first 8 frames (4 seconds) as the observed trajectory and the remaining 12 frames (6 seconds) as the ground truth future trajectory. Furthermore, we recommend evaluating prediction methods using a leave-one-out cross-validatie validationn strategy. Specifically, the model is trained and validated on the training and validation sets of seven scenarios and tested on the test set of the remaining scenario. This strategy helps mitigate validation bias.
Intention information. Finally, the statisties on intention information are presented in the seventh and eighth rows. Motion intentions are evenly distributed across the labels 'speed up,' 'slow down,' and 'constant speed.' Similarly, static intentions are uniformly distributed across eight dif- ferent octants. These balanced distributions ensure that our dataset offers a clear distinction in intention labeling.
3.2. Dataset Statistics
To demonstrate the practicality of the proposed dataset for evaluating 3D trajectory prediction, we analyze its charac- teristics across several dimensions. A detailed summary of the dataset's statistical properties is presented in Table 1.
4. 3D Trajectory Prediction Method
In this section, we address 3D trajectory prediction two steps: Decoupling inter-axis correlations by independently predicting the trajectory along each axis, and Modeling inter-axis correlations to generate offsets that refine the initial predictions. The former reduces the high prediction
Basic information. Firstly, we present the number of frames for each processed scenario in the second row. Our dataset
4

---

Three-Dimensional Trajectory Prediction
Coordinati System
Kes Pakn
LSTM
LSTM,
Trajectary
Agent
yrazo
Hakke
Enosder
Prodinne
LSTMy
L.S
Trajetery
3D Coordicate System
Inpot Traeectories
Inbueding
LSTM,
LSTM,
Prodimero
Eansker
Agent
LSTM
L.TM,
Feature
Pridicter
Decoupled Trajectory Prediction
Correlated Tinjectory Refinement (SCA-LSTM)
Predicted Trajectorios
20
W
Wau
[
hug
LSTM
Slate Conrelation (SC)
L,
Legend
Elemens-wise Multiplication
x
Concalenation
LSTM
LS77 M
LST.M,
LSTM,
Hlement-wisc Addittion
Sigmoid
Atr
Attention
State Aggregation
Figure 4: The framework of our proposed 3D trajectory prediction network. In addition to the baseline encoder from LBEBM, our method consists of two key components: decoupled trajectory prediction and correlated trajectory refinement. The former independently predicts the future trajectories along each axis, thereby decoupling inter-axis correlations. The latter generates offsets to refine these predictions by modeling their inter-axis correlations.
complexity of 3D trajectories, providing a coarse prediction. The latter refines these predictions by capturing their inter- axis correlations.
illustrated in Figure 4.
Let F represent the encoded feature of pf, extracted by LBEBM while accounting for interactions and other ele- ments. We first employ three key point predictors predict key points along different axes:
Problem Formulation. Let pl (1/, y 2/) E E rep- resent the position of agent at time t. The observed tra- jectory of agent i is defined as p. = {pt = 1,... to). and the ground truth future trajectory is p {pf | to 1,...,to to + }y), where and iy the time hori- zons of the observed and future trajectories, respectively. The objective of 3D trajectory prediction is to use the ob- served trajectory P as input to predict a future trajectory !! (il 1= +,... to bpp that closely approximates the ground truth trajectory p
D (FT),, ME (x,
(1)
where (- and D. denote the MLP key point predictor and the predicted key points for the p-axis, respectively. We then extract features from the predicted key points:
(2)
where x(-) is an MLP encoder, [ denotes concatenation, and E is the extracted feature. Finally, we concatenate F with Fe and feed this concatenated feature into three trajectory predictors to generate the future trajectory for each axis:
4.1. Decoupled Trajectory Prediction
We adopt the LBEBM method (Pang et al., 2021) as our baseline. To decouple the inter-axis correlations of 3D trajec- tories, we replace LBEBM's decoder with three independent decoders to predict trajectories separately along the x-, y-, and z-axes. LBEBM is key point- conticoond prediction method that first predicts future key points and then gener- ates the future trajectories conditioned on those points. We follow the same overall procedure as LBEBM but decouple the prediction process into three parallel streams for each axis. The detailed process our decoupled prediction is
Df =( E (x.
(3)
where (.) and of represent the MLP predictor and the predicted future trajectory for the v-axis, respectively.
4.2. Correlated Trajectory Refinement.
We design a novel LSTM-based method, Cate-Correlltton and Aggregation LSTM (SCA-LSTM),, to model the inter- axis correlations of the coarse predictions and refine them.
5

---

all Thrre-Dimensional Trajectory Prediction
An overview of the SCA-LSTM is shown in Figure 4. SCA- LSTM employs a centralized architecture comprising cen- ter LSTM, node LSTMs, a state correlation (SC) module, and state aggregation (SA) module. The center LSTM ex- tracts summarized trajectory features across different axes, while the node LSTM extracts trajectory features for each axis. The SC module captures the correlation between the center and node LSTMs, using update the node features. Meanwhile, the SA module aggregates the state features from the node LSTMs to update those of the center LSTM.
We design the message correlation function with gated selection mechanism to adaptively model correlations and refine the node LSTM's cell states. Specifically, we first extract the input features for the gate from hidden states:
phen = Wheher'
(8)
F.aa Whohvs, ed
E (2,
where Whe and Who are linear transformations. Then, we construct a correlation gate using the features F.d and pu such that:
Inputs Representation. Instead of directly using raw tra- jectories, we extract their embeddings as inputs for SCA- LSTM.
- (Feo :1 bew), WE (1,
(9)
c." is the gate vector for the v-axis, Waw and bev are where 9; linear transformation and bias, and refers to the Sigmoid function. Using the gate vector, Me formulated as:
For the node LSTMs, we first concatenate the predicted future trajectory with the observed trajectory for each axis. Then, calculate and concatenate the trajectory, velocity, and acceleration of each axis. This concatenated informa- tion is passed through MLP encoder to extract embeddings:
Ma(hi,
6. ko, Weng," e." @ hi,,', ME (2,0, (10)
F = ([," v
P E {2, v,
(4)
where W is a linear transformation.
The SA Module. The SA module adopts similar strategy to the SC module but operates in reverse, aggregating the hidden states from node LSTMs update the center LSTM. Mathematically, the SA module is formulated as:
where u², Vu,, and avi" represent the trajectory, velocity, and acceleration on the y-axis, respectively. Qy is the MLP encoder for the v-axis. F`' represents the extracted feature, which is the input the node LSTM for the v-axis.
For the center LSTM, we apply max- and average-pooling operations to combine the input representations axes:
&e 1.41
Σ DE(D.E
Mothee HuVi + oe/t,
(11)
P.T MarPooling , FT);
1.++1 is the refined cell state of the center LSTM, where oq
PTT AugPooling FT, F", F)),
(5)
and M2 is the message aggregation function for the ı-axis.
P. F"),,,
To adaptively aggregate the hidden states of the node L.TMss, we design a message aggregation function that incorporates axis-wise attention and an aggregation gate, The attention mechanism assigns appropriate weights to dif- ferent axes, while the aggregation gate extracts key features for effective integration.
where F. the extracted input feature for the center LSTM.
The Center and Node LSTMs. The center and node LSTMs use vanilla LSTMs to extract features from the input representations:
oct,hce (F he-1,,0-1::0
-tt.h. hu-1, θu), WE {x,y,
To build the attention and aggregation gate, we first extract global feature F. from the hidden states of all LSTMs:
where LSTM, Og, cct, and he; represent the center LSTM, its parameters, cell state, and hidden state, while LST My. Ov, cut, and hut denote the node LSTM for the y-axis and its components. For more details on the vanilla LSTM, please refer to Section B in the Appendix. The main differ- ence between the LSTMs used in our method and vanilla LSTMs is the inclusion of the SC and SA modules, which refine their cell and hidden states.
F.o = ([Wgehic f.l W Woahhr, Wyyhy, W Wgahiz" D, (12)
where W ges W War W Way. and Wa: are linear transformations. We then project the global embedding into different LSTMs' projection spaces:
E- = Woattii
(13)
po0e = Wouddid V eE(x, (0.
The SC Module. The SC module takes two inputs: the hid- den states and cell states from the center and node LSTMs. The output is the refined cell states for the node LSTMs. Mathematically, the SC module is formulated as follows:
where Foo.l is for the center LSTM, and F is for the node LSTM of the y-axis, with Wge and W yo being lin- ear transformations. Using these projected features, the attention weight can be computed as follows:
a/ttt11 (hc t.! kir + ou v E (2,
(7)
where A1+ denotes the refined cell states for the y-axis after +1 1 iterations, and Mc represents the message corre- lation function.
exp(W, (WW,rreeWWrrrr (Weryud)) exp(W, Fepllrry ME (n.1e) (14)
Σ
(c(ay2)
6

---

Three-Dimensional Trajectory Prediction
Scenario
#2
#3
#4
#5
#6
#7
#8
Mcan
SSTGENNY (Mohamed et al., 2020)
2.86/5.19
1,05/1.47 447
1.08/1.65
2.23/3.83
1.15/1.34
2.25/4.00
4.49/8.28
2.18/3.30
2.16/3.63
MSRL. (Wu et al., 2023)
3.72/5.42
0.62/0.73
0.56/0.61
1.69/2.05
1.85/2.50
1.54/2.84
1.14/1.73
0.90/1.15
1.502.13
FlowChain (Maeda & Ukita, 2023)
1.44/3.20
0.61/0.99
0.62/1.02 1.00
0.90/1.76
0.54/0.84
1.31/2.52
1.18/2.53
0.93/1
0.94/1.86
PECNet (Mangalam et al., 2020)
0.79/1.05
0.70/1.29
0.74/1.26
1.29/2.46
0/41/0.58
1.16/1.65
0.92/1.55
1.22/2.21
0.90/1.50
LBEBM (Pang al., 2021)
0.72/0.98
0.52/0.80
0.64/1.11
0.94/1.87
0.35/0.57
1.02/1677
1.27/2.35
1.25/2.42
0.84/1.47
NPSN (Bac et al., 2022)
0.75/0.83
0.69/1.21
0.71/0.96
0.83/1.28
0.34/0.40
0.97/1.200
0.71/0.93
1.03/1 .50
0.75/1.04
TrajCLIP (Yao al.. 2024)
0.56/U.94
0,40/0.69
0.37/0.70
1.32/2.81
0.33/0.58
0.79/1.27
1.04/2.02
0.85/1.69 696
0.71/1.34
CausalHTP (Chen el al., 2021)
0.69/1.29
0.45/0.78
0.45/0.78
0. 0.79/1.40
0.49/0.87
1,11/1.95
0.977/1 .96
0.74/1 .40
0.71/1.30
MS-TIP (Nath ct al., 2024)
0.62/1.16
0.61/1.16
0.58/1.15 1.5
0.78/1.42
0,57/1.11
0.84/1.50
0.77/1.44
0.79/1.52
0.70/1.31
MRGTraj (Peng et al., 2023)
0.63/1.29
0.34/0.54
0.33/0.55
0.94/1.89
0.42/0.83
0.99/1.73
1.06/2.44
0.84/1 .63
0.69/1.36
S-Implicit (Mohamed et al., 2022)
0.54/0.87
0.43/0.72
0.47/0084
1.13/2.31
0.40/0.68
0.85/1.38
0.62/1.06
0.98/1.90 .92
0.68/1.22
Our
0.36/0.51
0.37/0.60
0,48/0.86
0.81/1.69
0.28/0.44
0.69/1.10
1.12/1.95
0.55/0.99
0.58/1.02
Table 2: Comparison with state-of-the-art methods on the 3DMoTraj dataset under best-of-20 evaluation setting. t:
SSTGCNN uses predicted Gaussian distributions to sample future trajectories. *: NPSN here adopts PECNet as the baseline.
MBGiTo
(a) Sample #1: linear trajectories (easy)
(b) Sample #2: curved trajectories (medium) (c) Sample #3: fluctuating trajectories (hard)
Figure 5: Visual comparisons between our method with MRGTraj and S-Imliicit on 3D trajectory prediction using samples featuring linear, curved, and fluctuating trajectories denoting easy, medium, and hard prediction difficulties. The first to the fourth rows are the orthographic projections and front, top, and side views of visualized results. Dotted lines with blue, green, and red colors represent the observed trajectories, the ground-trute -ruth future trajectories, and the predicted future trajectories.
where (.)² denotes transposition, and &c., is the attention weight for the v-axis. Additionally, the aggregation gate is defined as:
4.3. Loss Function
Aside from the specialized loss of the baseline method LBEBM, the loss for our method is defined as follows:
(15) where g. represents the gate vector for the v-axis, while Wee and lie are linear transformations and bias terms. After that, the message aggregation function M,, is defined as:
gh =o
buo), WE (2, 23,
L
N Σ Σ (ch + -f""" + 12"),
iE(1...,
(17)
5."0 L2(ut, (t),,
c.- ==2(u(,0))
C. = Lf -f/))
2. hee "a.."ee=. W. & hv,"" (x, (16) (he
where U.th and of denote the ground truth key points and fu- ture trajectory on the v-axis, respectively, and L22 represents the mean squared error.
where Wne is a linear transformation.
Finally, after a round of refinements through the SA and SC modules, the refined cell states used to update the hidden states via the output gate of the vanilla LSTM, as shown in Equation 30 in the Appendix. Using these updated hidden states, the LSTMs predict the offsets Av! E {x, 2) to refine the coarse predictions on each axis.
5. Experiments
5.1. Implementation Details
Our method uses observed frames to predict future 12 frames. The key points predicted are the 3rd, 6th, 9th, and
7

---

Three-Dimensional Trajectory Prediction
lteration Number #
Layer Number
Metric
Metric
2.
3
4
2
3
ADB
0.62
0.67
0.58
0.63
ADB
0.58
0.64
0.64
0.65
FDB
1.05
1.21
1.02
1.07
FDE
1.02
1.07
1.06
1.10
Table 3: Analysis of different Table 4: Analysis of different
iterations of the SC and SA layers of the SCA-LSTM on the 3raj dataset.
the 3DMoTraj dataset.
Variant
Components
Performance
ID
BD DD VL
SL
ADB
FDE
v
0.84
1.47
2
0.68
1.19
3
v
0,66
1.05
4
/ 0.58
1.02
Table 5: Ablation study on our method using the 3DMo Traj dataset. BD refers to our baseline's 'ss decoder, DD represents decoupled trajectory prediction part, VL stands for the vanilla LSTM, and SL refers to our correlated trajectory refinement part, SCA-LSTM.
Hansfie
Haistne
Lker
Figure 6: Visual comparisons before and after applying our proposed components on the baseline method. Rows one to four denote different views of the visualized results.
12th frames of the future trajectory. The primary compo- nents of our approach are implemented using MLP LSTM. Training is conducted with batch size of 70 for 100 epochs, employing the Adam optimizer with an initial learning rate of 0.0005. The learning rate decayed by 0.5 after the 5th epoch. Results are evaluated using (Average Displacement Error) ADE and (Final Displacement Error) FDE metrics.
tuating trajectories, which pose significant challenge for methods designed for 2D prediction.
5.3. Hyperparameter Analysis
Training our proposed method involves two key hyperparam- eters. One is the iterations of the SC and SA modules, and the other is the layers of the SAC-LSTM. This subsection analyzes them one after another.
5.2. Comparisons to State-of-the-art Methods
Iterations of the SC and SA modules. Each iteration of the SC and SA modules refines the cell and hidden states of the LSTM. We conduct experiments with counts of {1. 2, 3, 4} to choose the optimal number of iterations. The SAC-L layer is fixed at in this experiment to eliminate confound- ing variables. As shown in Table 3, our method performs best when the iteration number is set to 3.
We present the performance of our proposed method and benchmark it against sfate-of-the-an art methods on the 3DMo- Traj dataset, as detailed in Table 2. We modify all compared methods to accept 3D observed trajectories as inputs and predict 3D future trajectories. Additionally, we adjust their data augmentation strategies, such rotation, flipping, and reversing, to their 3D counterparts for consistency. The experimental results indicate that our method outperforms all evaluated approaches in 3D trajectory prediction. Specif-
Layers of the SCA-LSTM. When utilizing multiple SCA- LSTM layers, each layer refines the previous layer's output and serves as input for the subsequent one. Different num- bers of SAC-LSTM, layers can produce varying outcomes. To evaluate the effect of layer depth, we experiment with SAC-LSTM layers of 21, 3, 4), as shown in Table 4. The iteration count for the SC and SA modules is fixed at 3, determined from the previous experiment. The experi- mental results confirm that using a single SAC-LSTM layer achieves the best performance.
ically, our method achieves a 14.6% improvement in the ADE metric, reducing from 0.68 to 0.58, and a 16.3% en- hancement in the FDE metric, lowering it from 1.22 to 1.02, compared to the second-best method, S-Implicit. While NPSN shows FDE metric close ours, our ADE metric surpasses it by 22.7%, decreasing from 0.75 to 0.58.
Furthermore, we visualize some samples compared to lead- ing methods MRGTraj and S-Implicit, as shown in Figure 5. The three visualized samples illustrate the ability of these methods to predict linear, curved, and fluctuating trajectories with increasing difficulty levels. The visual results indicate: 1) all compared methods perform well on the x- and y-axes, reflecting their original 2D design; 2) our method surpasses MRGTraj and S-Implicit in 3D trajectory prediction, par- ticularly along the z-axis; and 3) sample #3 validates our method's marked improvement predicting complex, flue-
5.4. Ablation Study and Discussion We conduct an ablation study to evaluate our method's core components, i.e., decoupled trajectory prediction and corre- lated trajectory refinement. The results, presented in Table 5, reveal several vital insights. First, variants 1 and 2 reveal that the decoupled trajectory prediction strategy outperforms the LBEBM decoder by 19.0% in ADE/FDE, highlight- ing its effectiveness. Second, variants 2 and 3 indicate that using vanilla LSTM for refinement provides a performance
8

---

Thhree-Dimensional Trajectory Prediction
14
Orthographic projection
Top view
Front view
Side view
Figure 7: A representative failure case generated by our method on the 3DMoTraj dataset.
Dataset
ETIVUCY
SDD
Method
Paramters (M)
FLOPs (G)
Speed (8)
ADE/FDE
Method
BL
Our
BL
Our
MSRL
0.59
0.12
0.09
/
ADE
0.22
0.21
9.20
8.98
LBEBM
1.24
0.09
0.05
0.84/ 1.47
FDE
0.40
0.38
16.47
15.93
NPSN
0.22
0.14
1.29
1.04
Table 6: Discussion of the generalization ability of our method for 2D trajectory prediction using the ETH/UCY and SDD datasets. BL refers to the baseline LBEBM.
CausalHTP
0.04
0.16
2.54
0.71 1.30
MRGTraj
4.36
20.04
0.06
0.69 1.36
Our
3.41
0.24
0.08
0.58 1.02
Table 7: Comparison with state-of-the-an ahr methods in terms of parameters, computational costs, and inference speed the 3DMoTraj dataset. All methods are tested using an NVIDIA 2080 Ti GPU with an input size of 70 x 8 x 3.
boost of 2.9% 1.8%, confirming that trajectory refinement further improve prediction accuracy. Finally, variants 3 and 4 show that our proposed SCA-LSTM outperforms the vanilla LSTM by 12. 9/2.9%, validating that SCA-LSTM, which models inter-axis correlations, significantly improves trajectory refinement.
decision decision-making requirements of robotics applications.
5.6. Failure Case Analysis
We also visualize two typical samples before and after ap- plying our method, as shown Figure While the baseline performs well in predicting trajectories along the x- and y- axes, it struggles with the additional z-axis of 3D trajectories. Our method enhances the baseline's performance, particu- larly in predicting the z-axis component, further demonstrat- ing the effectiveness of our core components.
We present representative failure case in Figure 7, which illustrates that our method struggles when predicting trajec- tories with multiple sharp bends in short time intervals. In- corporating more advanced interaction modeling techniques or integrating 3D point cloud maps could provide additional structural constraints, thereby improving this case. Nev- ertheless, our primary focus is reducing the prediction complexity of 3D trajectories, we adopt simple interac- tion modeling strategy and do not use 3D point cloud maps, leading to suboptimal performance in this case. Exploring stronger interaction modeling methods and incorporating 3D environmental representations will important direc- tions for future work.
Furthermore, to validate the generalization ability of the proposed method for 2D trajectory prediction, we conduct experiments the pedestrian trajectory datasets ETH/UCY and SDD, as shown in Table 6. Our method outperforms the baseline by 4.5% and 5.0% on the ETH/UCY dataset and by 2.4% and 3.3% on the SDD dataset. These experimental results demonstrate that our decoupled prediction strategy can also improve the prediction accuracy of 2D trajectory.
6. Conclusion
In this paper, we present large-scale 3D) trajectory dataset, 3DMoTraj, which includes frame-wise annotations for both static and dynamic intentions. This dataset facilitates re- search on 3D trajectory prediction. Additionally, we propose a novel 3D trajectory prediction method that addresses the increased prediction complexity encountered when moving from 2D to 3D trajectories. Experiments on the 3DMo- Traj dataset demonstrate that our method outperforms all sfate-of-the-aee models tested, validating its effectiveness in reducing prediction complexity and enhancing predic- tion accuracy in 3D. In future work, we aim to collect a 3D trajectory dataset of unmanned aerial vehicles (UAVs) operating in air environments to further validate our method and advance research in 3D trajectory prediction.
5.5. Model Efficiency Analysis
We analyze the model efficiency of the proposed method by comparing its parameters, computational cost, and run- time performance with several state-of- ff-te-of-the-rt methods, as illustrated in Table 7. All models in this experiment are tested on an NVIDIA 2080 Ti GPU using an imput size of 70 8 x 3, where 70 represents the number of agents pre- the agent count in most
dicted mamultaneously-exxeeeeeee real-world applications. The results demonstrate that our method achieves the best performance with a relatively good model efficiency, making it suitable for deployment on em- bedded robotic systems. Additionally, with an inference speed exceeding 12 FPS, our method meets the real-time

---

Three-Dimensional Trajectory Prediction
Acknowledgements
Choi, C., Malla, S., Patil, A., and Choi, J. H. Drogon: A tra- jectory prediction model based on intention-conditioned behavior reasoning. In Conference Robot Learning, pp. 49- 63. PMLR, 2021.
This work supported in part by the National Natural Sci- ence Foundation of China under grant No. 62202308, in part by the Guangdong Research Team for Communication and Sensing Integrated with Intelligent Computing under project No. 2024K in part by the Youth Innovation Pro- motion Association CAS under grant No. 2022133, in part by the Science and Technology Commission of Shanghai under grant No. of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korean Government (MSIT) (No. RS-2024-00457888 National AI Research Lab Project).
Ettinger, S., Cheng, S., Caine, B., Liu, C., Zhao, H.,, Prad- han, S., Chai, Y., Sapp. B., Qi, C. R., Zhou, Y., et al. Large scale interactive motion forecasting for autonomous driv- ing: The waymo open motion dataset. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9710-9719, 2021.
and in part by the Institute
Feng, L.,, Bahari, M., Amor, K. M. B., Zablocki, É., Cord, M., and Alahi, A. Unitraj: A unified framework for scal- able vehicle trajectory prediction. In European Confer- ence on Computer Vision, pp.
Springer, 2024.
Impact Statement
Gao, J., Sun, C., Zhao, H., Shen, Y., Anguelov, D., Li, C., and Schmid, C. Vectornet: Encoding hd maps and agent dynamics from vectorized representation. In Proceedings of the IEEE/CVF conference on computer vision and patlern recognition, pp. 11525-11533, 2020.
The work presented in this paper aims to advance the field of 3D trajectory prediction. Our work may have potential societal impacts in various industries, including autonomous driving, drone operations, and robotics. While these impacts are still in their early stages, we believe that there is need to specifically highlight any particular ethical issues or societal consequences at this point.
Girase, H., Gang, H., Malla, S., Li, J., Kanehara, A., Man- galam, K., and Choi, C. Loki: Long term and key inten- tions for trajectory prediction. In 202/ IEEF/CVF Interna- tional Conference on Computer Vision (ICCV), pp. 9783- 9792, 2021. doi: 10. D0.1099/CCC449922220000009
References
Bae, I., Park, J.-H., and Jeon, H.-G. Non-probability sam- pling network for stochastic human trajectory prediction. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 6477-6487, 2022.
Gupta, A., Johnson, J., Fei-Fei, L., Savarese, S., and Alahi, A. Social gan: Socially acceptable trajectories with gen- erative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2255-2264, 2018.
Bock, J., Krajewski, R., Moers, T., Runde, S., Vater, L.., and Eckstein, L. The ind dataset: A drone dataset of naturalistic road user trajectories at german intersections. In 2020 IEEE Intelligent Vehicles Symposium (IV), pp- 1929-1934, 2020. doi: 10.1 1009///47402220200955555
Huang, D., Zhang, Z., Fang, X., He, M., Lai, H., and Mi, B. Stif: A spatial-temporal integrated framework for end-to-end micro-uav trajectory tracking and prediction with 4-d mimo radar. JEEE Internet of Things Journal, 10(21): 188211188211883 2023.
Cao, X., Ren, L., and Sun, C. Dynamic target tracking control of autonomous underwater vehicle based on tra- jectory prediction. IEEE Transactions on Cybernetics, 53 (3):1968-1)) 1981, 2022.
Huang, W., Wang, C., Li, Y.. Zhang, R., and Fei-Fei, L. Rekep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation. 8th Annual Con- ference on Robot Learning, 2024.
Chang, M.-F., Lambert, J., Sangkloy, P., Singh, J., Bak, S., Hartnett, A., Wang, D., Carr, P., Lucey, S., Ramanan, D., et al. Argoverse: 3d tracking and forecasting with rich maps. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2019.
Huang, Y., Bi, H., Li, Z., Mao, T., and Wang, Z. Stgat: Mod- eling apatial-tempore interactions for human trajectory prediction. In Proceedings of the IEFF/CV international conference on computer vision, pp. 6272-6281, 2019.
Ishida, S. Spatial reasoning and planning for deep embodied agents. PhD thesis, University Oxford, 2024.
Chen, G., J., Lu, J., and Zhou, J. Human trajectory prediction via counterfactual analysis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9824 9833, 2021.
Kaur, D. P., Singh, N. P., and Banerjee, B. A review of platforms for simulating embodied agents 3d virtual environments. Artificial Intelligence Review, 56(4):3711- 3753, 2023.
10

---

Three-Dimensional Trajectory Prediction
Lemnr, A., Chrysanthou, Y., and Lischinski, D. Crowds by example. In Computer graphics forum, volume 26, pp. 655-664. Wiley Online Library, 2007.
Pellegrini, S., Ess, A., Schindler, K., and Van Gool, L, You'll never walk alone: Modeling social behavior for multi-target tracking. In 2009 IEEE 12th international conference on computer vision, pp. 261-266 IEEE, 2009,
Liu, B., Adeli, E., Cao, Z., Lee, K.-H., Shenoi, A., Gaidon, A., and Niebles, J. C. Spatiotemporal relationship reason- ing for pedestrian intent prediction. IEEE Robotics and Automation Letters, 5(2):3485-3492 2020.
Peng, Y., Zhang, G., Shi, J., Li, X., and Zheng, L. Mrgtraj: A novel non- -on-eutoreeressvvee approach for human trajectory prediction. IEEE Transactions on Circuits and Systems for Video Technology, 2023.
Liu, Y., Wang, H., Fan, J., Wu, J., and Wu, T. Control- oriented uav highly feasible trajectory planning: A deep learning method. Aerospace Science and Technology, 110:106433, 2021.
Qi, Z.. Dong, R., Zhang, S., Geng, H.,, Han, C., Ge, Z., Yi, L., and Ma, K. Shapellm: Universal 3d object understanding for embodied interaction. In European Conference on Computer Vision, pp. 214-238. Springer, 2024.
Ma, Y., Zhu, X., Zhang, S., Yang, R., Wang, W., and Manocha, D. Traffiepredict: Trajectory prediction for heterogencous araffic-agents. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 6120- 2019.
Rasouli, A., Kotseruba, I., Kunic, T., and Tsotsos, J. Pie: A large-scale dataset and models for pedestrian intention estimation and trajectory prediction. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6261-6270, 2019. doi: 10. 1099C.....20
Maeda, T. and Ukita, N. Fast inference and update of prob- abilistic density estimation on trajectory prediction. In Proceedings of the IEEE/CVF International Conference Computer Vision pp. 9795-9805, 2023.
Robicquet, A., Sadeghian, A., Alahi, A., and Savarese, S. Learning social etiquette: Human trajectory understand- ing in crowded scenes. In Leibe, B., Matas, J., Sebe, N., and Welling, M. (eds.), Computer Vision - ECCV 2016, pp. 549-565, Cham, 2016. Springer International Publishing.
Mangalam, K., Girase, H., Agarwal, S., Lee, K.-H., Adeli, E., Malik, J., and Gaidon, A. It is not the journey but the destination: Endpoint conditioned trajectory prediction. In Computer Vision-ECC' 2020: 16th European Con- ference, Glasgow, UK, August 23-28, 2020, Proceedings, Part II 16, pp. 759-776. Springer, 2020.
Salzmann, T., Ivanovic, B., Chakravarty, P., and Pavone, M. Trajectron++: Dynamically-feasee feasible trajectory forecast- ing with heterogeneous data. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XVIII 16, pp. 683-700. Springer, 2020.
Mangalam, K., An, Y., Girase, H., and Malik, J. From goals, waypoints & paths to long term human trajectory fore- casting. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15233-15242, 2021.
Shi, S., Jiang, L., Dai, D., and Schiele, B. Motion trans- former with global intention localization and local move- ment refinement. Advances Neural Information Pro- 2022.
Mohamed, A., Qian, K., Elhoseiny, M., and Claudel, C. Social-stgen:: A social spatio-temporal graph convolu- tional neural network for human trajectory prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14424-14432, 2020.
cessing Systems,
Sun, C., Li, H., Li, Y., and Hong, S. TEST: Text prototype aligned embedding to activate LLM's ability for time se- ries. In The Twelfth International Conference on Learning Representations, 2024. URL ht https:h/t////tptt //tp:////////eeeeeeeeeeeeeeeeeee net / www.rooruuuuuuuuiro. Tddniddd22bbooo,,
Mohamed, A., Zhu, D., Vu, W., Elhoseiny, M., and Claudel, C. Social-im Social-implicit: Rethinking trajectory prediction eval- uation and the effectiveness of implicit maximum likeli- hood estimation. In European Conference on Computer Vision, pp. 463 4799. Springer, 2022.
Wang. B., Liu, D., Wang. W., and Peng. X. A hybrid ap- proach for uav flight data estimation and prediction based on flight mode recognition. Microelectronics Reliability, 84:253-262, 2018
Nath, A., Kabra, P., Gupta, I., Singh. P., et al. Ms-tip: Impu- tation aware pedestrian trajectory prediction. In Forty-first International Conference Machine Learning, 2024.
Wang. T., Mao, X., Zhu, C., Xu, R., Lyu, R., Li, P., Chen, X., Zhang, W., Chen, K., Xue, T., et al. Embodiedscan: A holistic multi-modal 3d perception suite towards em- bodied ai. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19757 19767, 2024.
Pang, B., Zhao, T., Xie, X., and Wu, N. Trajectory predic- tion with latent belief energy- based model. In Proceed- ings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1-144111444 2021.

---

Three-Dimensional Trajectory Prediction
Wilson, B., Qi, W., Agarwal, T., Lambert, J., Singh, J., Khandelwal, S., Pan, B., Kumar, R., Hartnett, A., Pontes, J. K., et al. Argoverse 2: Next generation datasets for self-driving perception and forecasting. In Thirty-fifil Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
Zhong, G., Zhang, H., Zhou, J., Zhou, J., and Liu, H. Short- term 4d trajectory prediction for uav based spatio- temporal trajectory clustering, IEEE Access, 10:93362- 93380, 2022.
Zhou, B., Wang, X., and Tang, X. Random field topic model for semantic region analysis in crowded scenes from tracklets. In CVPR 2011, pp. 3441-3448, 2011. doi: 10.1 10.11 1.599459...
Wu, Y., Wang, L.,, Zhou, S., Duan, J., Hua, G., and Tang, W. Multi-stream representation learning for pedestrian trajectory prediction. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 37, pp. 2875-2882, 2023.
Zhou, H., Ren, D., Xia, H., Fan, M., Yang. X., and Huang, H. Ast-gnn: An attention-bass spatio-temporal graph neural network for interaction-awant pedestrian trajectory prediction. Neurocomputing, 445:298-33 -088, 2021.
Xie, G. and Chen, X. Efficient and robust online trajectory prediction for non-cooperative unmanned aerial vehicles. Journal of Aerospace Information Systems, 19(2):143- 153, 2022.
Zhou, Z., Wang, J., Li, Y.-H., and Huang, Y.-K Query- centric trajectory prediction. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 17863-17873 2023.
Xu, C., Mao, W., Zhang, W., and Chen, S. Remember inten- trajectory prediction.
tions: serrospetive-mmmorry-boe In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pullern Recognition, pp. 6488-6497, 2022.
Zhu, H., Yang, H., Wang. Y., Yang, J., Wang. L., and He, T. SPA: 3d spatial-awareness enables effective embodied representation. In The Thirteenth International Confer- ence on Learning Representations, 2025.
Xu, Y. and Fu, Y. Adapting to length shift: Flexilength network for trajectory prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15226-15237, 2024.
Yang, L. and Hong, S. Unsupervised time-series represen- tation learning with iterative bilinear compora--pecctee fusion. In International conference on machine learning, pp. 25038 25054. PMLR, 2022.
Yao, P., Zhu, Y., Bi, H., Mao, T., and Wang, Z. Trajelip: Pedestrian trajectory prediction method using contrastive learning and idempotent networks. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Yue, J., Manocha, D., and Wang, H. Human trajectory pre- diction via neural social physics. In European conference on computer vision, pp. 376-394. Springer, 2022.
Zhang, J., Shi, Z., Zhang, A., Yang. Q., Shi, G., and Wu, Y. Uav trajectory prediction based on flight state recognition. IEEE Transactions on Aerospace and Electronic Systems, (0(3):2629-264) 2023.
Zhang, P., Xue, J., Zhang, P., Zheng, N., and Ouyang, W. Social-aware pedestrian trajectory prediction via states refinement Istm. IEEE transactions on pattern analysis and machine intelligence, 44(5):772--279,, 2020.
Zhao, H., Gao, J., Lan, T., Sun, C., Sapp, B., Varadarajan, B., Shen, Y., Shen, Y., Chai, Y., Schmid, C., et al. Tnt Target-driven trajectory prediction. In Conference on Robot Learning, pp. 905- PMLR, 2021.
12

---

Three-Dimensional Trajectory Prediction
This supplementary file provides more details demonstrate the benefits of our proposed dataset and method.
Prediction complexity analysis.
Details of vanilla LSTM.
The motion trajectory visualization.
The distance distributions for different scenarios.
The velocity distributions for different scenarios
The accelcration distributions for different scenarios.
The curvature distributions for different scenarios.
A. Prediction Complexity Analysis
The prediction complexity increases nearly twofold when transitioning from 2D (x, y) to 3D (x, y. 2) trajectories, Predicting
21) trajectory is equivalent to optimizing 2D Gaussian distributions to approximate the possible locations of each 2D point:
f(x, y)
exp (-21
pzz
y - Mv
x
Pbx
1)
(18)
2x [zzz]
Py
where Mny denotes the covariance matrix for the 2D Gaussian distribution and can be represented as follows:
PryOroy
Ezz
(19)
Therefore, the prediction of each 2D point involves optimizing 5 free parameters [M My o oz Puy]. Similarly, predicting 3D point is equivalent to optimizing 3D Gaussian distribution:
1
x fx
f(x, 2) (2 Egys
exp
x
My 2 Me
2zzz
by
(20)
2 fzz
where Days the covariance matrix for the 3D Gaussian distribution and can be represented as follows:
o² Payazog P2=0202
PuyTroy og PyzTg0z o²
PorOaOz Pyz0y0=
Days
(21)
This implies that there 9 parameters [pey Py M4 o2, og. Consequently, the complexity of predicting one 3D point approximately twice that of one 2D point.
yy o2 Pays Pyz: Paz to be optimized for predicting each 3D point.
For 3D Gaussian distribution, by substituting the covariance matrix Mzgs into Equation 20, we obtain its expanded form:
(-
1
(x pt )²
(3)
(2-
(z
[(, (1) (2/3) EmyyS|
1/2 exp
0
(22)
(x
1yy
(x pz
(z M2)
(y pyy 112)
2pzy
2pzz
2pyz
O20g
0²0=
0yO=
The exponential term of Equation 22 can be decomposed into two multiplicative components, such that Equation 22 can be expressed as follows:
1
(x (1)
(y - ,,,
(2-
[(x, (1.,,2)
exp
x
3/2
03 0z000?
2
Part-1
(23)
020000
(x
(v) pg) + 2paz
(x ux) 122)
My (2 pz)
1/2 exp
2pzy
2p
Euysl
OzOy
ax0=
0y0=
Part-11
13

---

Three-Dimensional Trajectory Prediction
where Part- and Part II denote the independent and correlation parts, respectively. Part corresponds the distribution of each variable within its dimension, while Part TT captures the interdependencies between variables in the 3D Gaussian distribution. This decomposition underscores that the overall Gaussian distribution is the product of independent Gaussian distributions (Part I) and correction factor that accounts for inter-axis correlations (Part - II).
Additionally, the independent part can further be expressed as the product three ID Gaussian distributions:
exp (- (xy k2)² 20
))
Part
II
(24)
V2mo²
vE(z,g.a)
Therefore, we propose a divide-and -nn-uuuueree strategy for predicting 3D trajectories. This approach first decouples the prediction process along three independent axes, mitigating the impact of inter-axis correlations. Subsequently, the inter-axis correlations are modeled to refine the predictions. In this way, the prediction of each 3D point with parameters first simplified to optimize three 1D Gaussian distributions with free parameters, thereby reducing the overall prediction complexity.
B. Details of Vanilla LSTM
LSTM is Recurrent Neural Network (RNN) designed to process sequential data such as time series, natural language text, etc. Traditional RNNs struggle with long sequences due the vanishing or exploding gradient problem. LSTM addresses this by incorporating gating mechanism that helps capture long-term dependencies effectively.
Input Gate
Output Gate
Yt
Ct-1
Ct
Tanh
ft
1
Ot
Forget Gate
C2
Tanh
h --
h
X₁ Figure 8: Illustration of LSTM cell.
Vanilla LSTM refers the basic form of LSTM, composed of forget gate, input gate, output gate, and cell state, as illustrated in Figure 8. At each time step t, the LSTM receives input vector Tı, previous hidden state ht-1, and previous cell state Ct-1. Then the four modules compute follows:
Forget Gate: The forget gate decides what information to remove from the cell state. It takes the previous hidden state h4-1 and current input Tt, and computes a forget ratio:
ft = oW, (he-1, by),
(25)
where:
fr: Forget gate activation vector at time
a: Sigmoid activation function,
W Weight matrix for the forget gate,
by: Bias vector for the forget gate,
h--1: Hidden state from the previous time step,
xe: Input vector the current time step.
14

---

Three-Dimensional Trajectory Prediction
Input Gate: The input gate determines what information add to the cell state. The gate activation computed
f = oWW, [1 + b.),
(26)
and the candidate memory content is:
è tanh(W. -1 + be),
(27)
where:
ie: Input gate activation vector,
d Candidate cell state vector,
Wi, Wo: Weight matrices for the input gate and cell state,
by, bo: Bias vectors for the input gate and cell state,
tanh: Hyperbolic tangent activation function.
Cell State Update: The cell state is updated using information from the forget and input gates:
Cx St ② Ct-1 + @ ct
(28)
where:
cg: Cell state vector at time t,
Ct-1: Cell state vector from the previous time step,
O: Element-wise multiplication.
Output Gate: The output gate decides the current hidden state (and the output of the LSTM). The gate activation is:
02 = o(W. [he-1. 21] bo).
(29)
and the hidden state is computed as:
h = 3 tanh(c:),
(30)
where:
o, Output gate activation vector,
h Hidden state vector at time f,
Wo Weight matrix for the output gate,
b,: Bias vector for the output gate.
In summary, the LSTM performs the following operations each time step:
1. Inputs the previous hidden state h4-1 and current input Iz-
2. Uses the forget gate to decide which past information to discard.
3. Updates the cell state with the input gate and candidate memory.
4. Computes the current hidden state using the output gate.
15

---

Three-Dimensional Trajectory Prediction
C. The Motion Trajectory Visualization
(a) Orthographic Projection
(b) Front View
(c) Top View
(d) Side View
Figure 9: Orthographic projections, along with front, top, and side views of 3D trajectories for the remaining five underwater scenarios in the 3DMoTraj dataset.
16

---

Three-Dimensional Trajectory Prediction
D. The Distance Distributions for Different Scenarios
350
684
15
Scenario #1 entere demtete
(h) Scenario #2 srancesterruu
Scenario #3
(d) Scenario #4 pensee cww.eecaaatte
05
10
(e) Scenario #5
(l) Scenario #6
Scenario #7
(h) Scenario #8
Figure 10: The distance distributions for scenarios #1 through #8.
E. The Velocity Distributions for Different Scenarios
(a) Scenario #1
(b) Scenario #2
(c) Scenario #3
(d) Scenario #4
(e) Scenario #5
(f) Scenario
(g) Scenario
(h) Scenario
Figure 11: The velocity distributions for scenarios #1 through #8.
17

---

Three-Dimension Trajectory Prediction
F. The Acceleration Distributions for Different Scenarios
200
(a) Scenario
(b) Scenario
(c) Scenario #3
(d) Scenario
Scenario #5
(f) Scenario #6
Scenario #7
(h) Sccnario #8
Figure 12: The acceleration distributions for scenarios #1 through #8.
G. The Curvature Distributions for Different Scenarios
(a) Scenario #1 Sanator Diiitative
(b) Scenario #2
(c) Scenario #3
(d) Scenario #4
Caraler Dea
Durvotue Zeralaate
(c) Sccnario #5
(f) Sccnario #6
Scenario #7
(h) Scenario
Figure 13: The curvature distributions for scenarios through #8.
18


Reviews:
[{"review_id": "gy7v6wheUB", "reviewer": "Reviewer_3PK9", "summary": "This paper proposes a 3D trajectory dataset named 3DMoTraj collected from unmanned underwater vehicles (UUVs) in ocean environments, which fills the research gap in this field.\nRegarding the setting of 3D trajectory prediction, the paper highlights the challenge of computational complexity and provides theoretic proofs.\nThe paper proposes a decoupled framework to mitigate the computational complexity, experiments demonstrate the superior accuracy of 3D trajectory prediction.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: The paper tries to mitigate the computational complexity but has not provided ablations or method comparisons on inference cost.**\n\n**A1**: At first, we clarify that the complexity our paper aims to mitigate is prediction complexity, which is crucial for optimizing 3D trajectory prediction. However, this is distinct from computational complexity. It is our fault for misunderstanding. To avoid any confusion, we will explicitly define the term of prediction complexity in our paper. Theoretical analysis in the Appendix and ablation studies confirm that our method effectively reduces the prediction complexity of 3D trajectories. \n\nComputational complexity is indeed an important aspect to evaluate the proposed method; therefore, below we compared inference costs across several state-of-the-art methods using an NVIDIA 2080 Ti GPU. To ensure fair comparisons, all tested methods were modified to accept inputs of shape 70×8×3.\n\n| Methods|Parameters (M)|FLOPs (G)|Inference time (s)|ADE|FDE|\n|:----|:----|:----|:----|:----|:----|\n|MSRL|0.59|0.12|0.09|1.50|2.13|\n|LBEBM|1.24|0.09|0.05|0.84|1.47|\n|NPSN|0.22|0.14|1.29|0.75|1.04|\n|CausalHTP|0.04|0.16|2.54|0.71|1.30|\n|MRGTraj|4.36|20.04|0.06|0.69|1.36|\n|Our|3.41|0.24|0.08|0.58|1.02|\n\nThese results show that our method performs best while maintaining relatively high model efficiency, making it suitable for deployment on embedded robotic systems.\n\n***\n**Q2: Extending 3DMoTraj dataset to general 3D scenarios is difficult due to underwater dynamics.**\n\n**A2**: From the methodological perspective, our motivation is to reduce the prediction complexity of 3D trajectory in a manner that ensures robustness across diverse scenarios. The 3DMoTraj dataset is only one of the validation scenes for our method.\n\nTo further establish its generalizability, we are constructing a large-scale 3D trajectory dataset for unmanned aerial vehicles (UAVs). Preliminary results from this dataset confirm that our method extends effectively to general 3D trajectory prediction. Please refer to **Q4** below for more details.\n\n***\n**Q3: Further discussions on trajectory refinement works, such as MTR, QCNet, SmartRefine, etc.**\n\n**A3**: We modified the trajectory refinement components of MTR [1], QCNet [2], and SmartRefine [3] by replacing them with the refinement module in our method.\n\n|Settings|ADE|FDE|\n|:----|:----|:----|\n|Decoupled prediction+MTR|0.69|1.15|\n|Decoupled prediction+QCNet|0.64|1.04|\n|Decoupled prediction+SmartRefine|0.61|1.05|\n|Decoupled prediction+SCA-LSTM (Our)|0.58|1.02|\n\nThe results show that our SCA-LSTM performs best in refining initial predictions. This outcome is expected, as SCA-LSTM explicitly models the inter-axis correlations of predicted trajectories, which are intentionally ignored in the decoupled prediction stage to simplify optimization. In contrast, the refinement modules of MTR, QCNet, and SmartRefine focus on modeling interactions between predictions, global intentions, and local maps for refinement.\n\nWe will further elaborate on the differences between our method and other 2D refinement-based methods in our paper.\n\n***\n**Q4: Provide a discussion on 3D trajectory dataset collected from unmanned aerial vehicles.**\n\n**A4**: As outlined in our conclusion, future work will involve collecting a large-scale 3D trajectory dataset from unmanned aerial vehicles (UAVs) to validate our method further. Specifically, we plan to capture trajectories across nine distinct environments. For each environment, we aim to collect six hours of trajectories, allocating three hours for training, two for validation, and one for testing.\n\nAt the current stage, we have collected two environments—large-scale indoor mall scenarios and urban road airspace scenarios—though annotations remain incomplete. We randomly selected one hour of annotated data from these two scenarios (30 minutes for training, 20 for validation, and 10 for testing) and benchmarked our approach against other state-of-the-art methods.\n\n|Methods|Large-scale indoor mall scenarios|Urban road airspace scenarios|\n|:----|:----|:----|\n|NPSN|0.72/0.97|0.83/1.31|\n|MRGTraj|0.60/0.78|0.73/1.23|\n|CausalHTP|0.54/0.71|0.68/1.18|\n|S-Implicit|0.52/0.73|0.65/1.25|\n|MS-TIP|0.47/0.77|0.62/1.12|\n|TrajCLIP|0.45/0.74|0.61/1.13|\n|Our|0.32/0.67|0.42/0.95|\n\nThe results demonstrate that our method outperforms all the competitors, validating its effectiveness in diverse 3D scenarios. We have also visualized several predicted trajectories in Figure-4 and Figure-5 at the anonymous URL <https://anonymous.4open.science/r/ICML_ID4436/README.md>, further illustrating the ability of our method in general 3D trajectory prediction.\n\n***\n**Reference**\n\n[1] Shi S, et al. Motion transformer with global intention localization and local movement refinement. NeurIPS, 2022.\n\n[2] Zhou Z, et al. Query-centric trajectory prediction. CVPR. 2023.\n\n[3] Zhou Y, et al. Smartrefine: A scenario-adaptive refinement framework for efficient motion prediction. CVPR. 2024.", "claims_and_evidence": "The paper proposes a decoupled framework to mitigate the computational complexity, but it has not provided ablations or method comparisons on inference cost.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "The theoretical claims that support the key contributions of this paper are correct.", "experimental_designs_or_analyses": "The paper has not provided experiments to support the claims of mitigating the computational complexity of the 3D trajectory prediction task.", "relation_to_broader_scientific_literature": "The key contributions of the paper are related to previous 2D trajectory prediction datasets and methods, as well as future 3D trajectory prediction tasks.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\n\n1. The paper proposes a large-scale and diverse 3D trajectory dataset collected from unmanned underwater vehicles in ocean environments, which is a good contribution to the related field.\n\n2. The computational complexity challenges discussed in this paper are well-justified.\n\n3. The performance of the proposed decoupled framework surpasses that of several methods in the 3D trajectory prediction problem, and ablations demonstrate that the proposed modules can enhance prediction accuracy.\n\nWeaknesses:\n\n1. The 3DMoTraj dataset is collected in ocean environments. Extending it to general 3D scenarios is difficult due to underwater dynamics, and the paper should elaborate on the limitations of the application scenarios more formally for clarity.\n\n2. Apart from 2D and 3D, the paper has not provided further discussions on the differences between the proposed framework and other trajectory prediction methods. Trajectory refinement is a common practice in trajectory prediction tasks, such as in MTR, QCNet,  SmartRefine, etc.", "other_comments_or_suggestions": "Please see the weaknesses.", "questions_for_authors": "Is there any available 3D trajectory dataset collected from unmanned aerial vehicles? If so, please provide a discussion on it.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CvvcWAHZG8", "reviewer": "Reviewer_cvmD", "summary": "Firstly, this paper introduces the 3DMoTraj dataset, a novel 3D trajectory dataset collected from unmanned underwater vehicles (UUVs) in oceanic environments. The dataset includes annotations for both static (endpoint octant) and motion (velocity change) intentions. Secondly, to address the increased complexity of 3D trajectory prediction compared to 2D, this paper proposes a method consisting of two components: decoupled trajectory prediction (independently predicting each axis to reduce complexity) and correlated trajectory refinement (modeling inter-axis correlations to refine predictions). The approach leverages LSTM-based modules with state-correlation and aggregation mechanisms. Experiments on the 3DMoTraj dataset demonstrate improvements over state-of-the-art methods, achieving lower Average Displacement Error (ADE) and Final Displacement Error (FDE). This paper also validates the method’s generalization to 2D datasets like ETH/UCY and SDD.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: The dataset is simulation data and has not been validated in real-world scenarios.**\n\n**A1**: While our dataset is based on predefined formation trajectories, real-world disturbances naturally cause deviations from planned paths and formation shifts. These deviations reflect real challenges in trajectory prediction. Moreover, most real-world robotic trajectories are pre-programmed rather than spontaneously generated, meaning our dataset aligns with actual robotic motion patterns, making it representative of real-world scenarios.\n\n***\n**Q2: Validating the proposed method against top-notch methods on other 3D datasets.**\n\n**A2**: It is hard to find publicly available 3D trajectory dataset. We plan to collect a more general 3D trajectory of unmanned aerial vehicles (UAVs) to validate our method. Specifically, we will capture trajectories across nine environments from the civilian field. Currently, we have collected two environments: large-scale indoor mall scenarios and urban road airspace scenarios. We randomly selected one hour of annotated data from these two scenarios and benchmarked our method against top-notch methods. For more experimental details, please see **Q1** of Reviewer cMBX.\n\n|Methods|Large-scale indoor mall scenarios|Urban road airspace scenarios|\n|:----|:----|:----|\n|NPSN|0.72/0.97 |0.83/1.31|\n|MRGTraj|0.60/0.78|0.73/1.23|\n|CausalHTP|0.54/0.71|0.68/1.18|\n|S-Implicit|0.52/0.73|0.65/1.25|\n|MS-TIP|0.47/0.77|0.62/1.12|\n|TrajCLIP|0.45/0.74|0.61/1.13|\n|Our|0.32/0.67|0.42/0.95|\n\nThe results show that our method outperforms all others, validating its effectiveness in diverse 3D scenarios. The visualized results in Figure-4 and Figure-5 at URL <https://anonymous.4open.science/r/ICML_ID4436/README.md> further illustrate its general 3D prediction capability.\n\n***\n**Q3: Many 3D trajectories are military datasets with limited application scenarios.**\n\n**A3**: 3D trajectory prediction has broad civilian applications. Beyond UAV navigation and obstacle avoidance, it supports urban air mobility, precision agriculture, disaster response, and industrial inspections, ensuring safe and efficient operations. These applications extend its impact far beyond military use.\n\n***\n**Q4: The claim that reduces the total number of parameters is misleading.**\n\n**A4**: Yes, our decoupling strategy simplifies the optimization process but does not reduce optimized parameters. We will clarify this in our paper.\n\n***\n**Q5: The effects of modifying the data enhancement strategies of compared methods.**\n\n**A5**: To ensure a fair augmentation, we conducted a comparative study of 2D augmentations (rotation, flipping, and translation) versus their 3D counterparts across several top-notch methods. \n\n|Augmentation|2D strategies|3D strategies|\n|:----|:----|:----|\n|CausalHTP|0.71/1.30|0.69/1.27|\n|MS-TIP|0.70/1.31|0.70/1.29|\n|MRGTraj|0.69/1.36|0.68/1.31|\n|S-Implicit|0.68/1.22|0.66/1.20|\n|Our |0.58/1.02|0.47/0.88|\n\nThe results show that 3D strategies provide slight improvements on others but significantly enhance our method, suggesting that 3D strategies are particularly beneficial for our model. For fairness, we use standard 2D strategies with 3D inputs for all methods.\n\n***\n**Q6: Discussion of the SC and SA modules separately.**\n\n**A6**: We conducted separate ablation studies on the SC and SA modules.\n\n| Settings|ADE|FDE|\n|:---- |:---- |:---- |\n|LSTMOnly|0.66|1.05|\n|LSTM+SC|0.64|1.04|\n|LSTM+SA|0.61|1.02|\n|LSTM+SA+SC|0.58|1.02|\n\nThe results indicate that removing the SC or SA module degrades the model's performance, highlighting their contributions.\n\n***\n**Q7: Generalization to 2D Datasets with more baselines.**\n\n**A7**: We evaluated our prediction strategy on four additional baselines using 2D datasets: ETH&UCY and SDD. To ensure a fair comparison, all models were trained and tested on the same machine.\n\n|Methods|ETH&UCY|SDD|\n|:----|:----|:----|\n|PECNet|0.30/0.48|10.02/15.79|\n|PECNet+our|0.25/0.42|9.45/15.26|\n|NPSN|0.28/0.44|8.56/14.95|\n|NPSN+our|0.25/0.39|8.34/14.36|\n|MSRL|0.20/0.36|8.36/13.85|\n|MSRL+our| 0.19/0.34 |  8.29/13.56| \n|TrajCLIP|0.21/0.35|7.69/13.31|\n|TrajCLIP+our|0.19/0.32|7.63/13.30|\n\nThese results show that our approach consistently enhances the performance of multiple baselines, further validating its generalization capability to 2D trajectory prediction.\n\n***\n**Q8: Refine the implementation principles of the SA and SC modules in Figure 4.**\n\n**A8**: We added the implementation details of the SA and SC modules in Figure 4. Please refer to Figure-1 at URL <https://anonymous.4open.science/r/ICML_ID4436/README.md>.\n\n***\n**Q9: How does your method perform in real-world scenarios?**\n\n**A9**: As discussed in **Q2**, our method has been evaluated in urban road airspace scenarios, where groups of UAVs navigate complex intersections to simulate 3D logistics transmission. These scenarios involve complex interactions. Our method performs best in this environment, demonstrating its scalability to real-world complexities.", "claims_and_evidence": "The claims in this paper are not fully supported by clear and convincing evidence for the following reasons:\n\n1. Dataset Limitations: The 3DMoTraj dataset is self-collected and may be specific to the 3D trajectory prediction model presented in this paper. The dataset is an analog simulation data based on human intervention and has not been validated for its application in real-world scenarios.\n\n2. Method Validation: The model proposed in this paper has only been evaluated on the self-constructed dataset, and has not been experimentally validated against state-of-the-art methods on other 3D datasets, so it does not indicate whether the excellent experimental results stem from the design of the model or from the bias of the specific dataset.\n\n3. Application scenario: Most of the trajectories in the real world are 2D, so many studies are based on 2D trajectories. Many 3D trajectories are military datasets, and the application scenarios of this research are limited.", "methods_and_evaluation_criteria": "The proposed method and evaluation criteria partially align with the problem of 3D trajectory prediction but suffer from critical limitations that undermine their suitability for broader validation and practical impact. First, the 3DMoTraj dataset is generated specific to this model, and it is not possible to assess the effectiveness of this method as applied in a realistic scenario. Second, the design concept of this model is simple, mainly based on LSTM and Attention mechanism, which is not innovative enough.", "theoretical_claims": "This paper makes theoretical claims about the complexity of 3D trajectory prediction and the decomposition of 3D Gaussian distributions. After reviewing the mathematical derivations, the following issues are identified:\n\n1. Validity of Parameter Complexity Analysis\nThis paper correctly argues that 3D trajectory prediction requires optimizing 9 parameters per point (3 means, 3 variances, 3 correlations), compared to 5 parameters for 2D (2 means, 2 variances, 1 correlation). This analysis is mathematically sound.\n\n2. Decomposition of 3D Gaussian Distributions\nThe decomposition of the 3D Gaussian distribution into independent 1D Gaussians and a correlation correction term (Equations 22–24) is theoretically valid. The authors correctly show that the exponential term can be split into independent (Part-I) and interdependent (Part-II) components. However, the claim that this reduces the total number of parameters from 9 to 6 is misleading. While Part-I corresponds to 6 parameters (3 means, 3 variances), Part-II still requires modeling 3 correlations (ρ_xy, ρ_xz, ρ_yz), totaling 9 parameters. The decoupling strategy simplifies the optimization process by separating independent and correlated components but does not reduce the total number of parameters.\n\nIn summary, the proof and application of the mathematical theory in this paper is correct.", "experimental_designs_or_analyses": "The experimental designs and analyses in this paper contain critical flaws that compromise the validity of the conclusions. \n\n1. Comparison with State-of-the-Art Methods\n This paper modifies 2D trajectory prediction methods (e.g., SSTGCNN, MSRL) to receive 3D trajectory inputs, and also adjusts the data enhancement strategies of these methods, which may affect the original effectiveness of these methods.\n The model proposed in this paper has only been evaluated on the self-constructed dataset, and has not been experimentally validated against state-of-the-art methods on other 3D datasets, so it does not indicate whether the excellent experimental results stem from the design of the model or from the bias of the specific dataset.\n\n2. Ablation Studies\nIn the ablation experiments, the authors did not discuss the role of the SC module and SA module in State-Correlation and Aggregation LSTM (SCA-LSTM) separately.\n\n3. Generalization to 2D Datasets\nComparison experiments with a single baseline LBEBM on the ETH/UCY and SDD datasets do not prove that the 3D Decoupled-Correlated trajectory prediction strategy proposed in this paper can improve the performance of 2D trajectory prediction.", "relation_to_broader_scientific_literature": "The approach of independently predicting each axis (x, y, z) is a very common mathematical idea. This reduces the complexity of optimizing 3D Gaussian distributions, as theoretically justified in the paper.\n\nThe SCA-LSTM module’s state-correlation and aggregation mechanisms draw inspiration from attention-based methods (e.g., Trajectron++, Salzmann et al., 2020) and graph neural networks (e.g., STGAT, Huang et al., 2019). \n\nThe paper contributes a specialized dataset and a method for UUV trajectory prediction. While its decoupled-correlated framework and intention annotations build on prior work in 2D and 3D prediction, the narrow application scope and lack of cross-domain validation restrict its broader scientific impact. To better align with the literature, future work should validate the method on diverse 3D datasets and compare against 3D-specific baselines.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "Strengths:\n\n1. The 3DMoTraj dataset focus on 3D trajectories of UUVs in oceanic environments, addressing a critical gap in underwater robotics research. Its annotations for motion and static intentions (velocity changes and endpoint octants) provide a unique foundation for intention-conditioned 3D prediction.\n\nWeaknesses:\n\n1. The 3DMoTraj dataset is self-collected and may be specific to the 3D trajectory prediction model presented in this paper. The dataset is an analog simulation data based on human intervention and has not been validated for its application in real-world scenarios.\n\n2. The model proposed in this paper has only been evaluated on the self-constructed dataset, and has not been experimentally validated against state-of-the-art methods on other 3D datasets, so it does not indicate whether the excellent experimental results stem from the design of the model or from the bias of the specific dataset.", "other_comments_or_suggestions": "It is recommended that the authors refine the implementation principles of the SA module and SC module in the model framework diagram in Figure 4.", "questions_for_authors": "Method Scalability: How does your method perform in scenarios with more agents, higher noise, or non-formation dynamics (e.g., adversarial UUV interactions)? Have you tested its scalability to such real-world complexities?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mb9pllVLCl", "reviewer": "Reviewer_cMBX", "summary": "The paper addresses the problem of 3D trajectory prediction by introducing a novel dataset and an innovative prediction framework. Building upon the 3DMoTraj dataset, they propose a dual-component prediction method that decomposes the 3D trajectory prediction task into two stages. The first stage, decoupled trajectory prediction, independently forecasts trajectories along each spatial axis to reduce overall prediction complexity. The second stage, correlated trajectory refinement, models inter-axis dependencies to generate corrective offsets that enhance the initial predictions. Extensive experiments demonstrate the superiority of the proposed approach.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: I look forward to seeing future validation on the trajectory dataset of unmanned aerial vehicles.**\n\n**A1**: As outlined in our conclusion, future work will involve collecting a large-scale 3D trajectory dataset from unmanned aerial vehicles (UAVs) to validate our proposed methodology further. Specifically, we plan to capture trajectories across nine distinct environments, including large-scale indoor mall scenarios, urban road airspace scenarios, urban low-altitude logistics corridors, dense vegetation agricultural fields, large industrial park settings, cross-sea bridge inspection sites, post-disaster debris zones, open water airspace environments, and wilderness forest environments. For each environment, we aim to collect six hours of UAV trajectory data, allocating three hours for training, two for validation, and one for testing.\n\nAt the current stage, we have collected two environments—large-scale indoor mall scenarios and urban road airspace scenarios—though annotations remain incomplete. In the indoor mall scenario, three or four UAVs simulate coordinated formation movement to mimic goods pickup and delivery tasks. In the urban road airspace scenario, four or six UAVs navigate complex intersections to simulate 3D logistics transmission environments. To evaluate our method on a more general 3D dataset, we randomly selected one hour of annotated data from these two scenarios (30 minutes for training, 20 for validation, and 10 for testing) and benchmarked our approach against other state-of-the-art methods.\n\n| Methods| Large-scale indoor mall scenarios| Urban road airspace scenarios |\n|:---- |:---- |:---- |\n| NPSN | 0.72/0.97 | 0.83/1.31 |\n| MRGTraj | 0.60/0.78 | 0.73/1.23 |\n| CausalHTP | 0.54/0.71 | 0.68/1.18 |\n| S-Implicit | 0.52/0.73 | 0.65/1.25 |\n| MS-TIP | 0.47/0.77 | 0.62/1.12 |\n| TrajCLIP | 0.45/0.74 | 0.61/1.13 |\n| Our | 0.32/0.67 | 0.42/0.95 |\n\nThe results demonstrate that our method outperforms all investigated approaches, validating its effectiveness in diverse 3D scenarios. Besides, our method performs best at urban road airspace scenarios, demonstrating the method's ability to predict complex non-formation trajectories. Moreover, our approach performs better in UAV-based 3D environments than UUV-based underwater scenarios, further confirming the greater complexity of underwater trajectory prediction due to higher resistance and turbulence. We have also visualized several predicted trajectories in Figure-4 and Figure-5 at the anonymous URL <https://anonymous.4open.science/r/ICML_ID4436/README.md>, further illustrating the ability of our method in general 3D trajectory prediction.\n\n***\n**Q2: Evaluating the computational cost and runtime performance.**\n\n**A2**: We compared our method’s computational cost, parameter, and runtime performance with several state-of-the-art methods. Specifically, we tested all models on an NVIDIA 2080 Ti GPU using an input size of 70×8x3, where 70 represents the number of agents predicted simultaneously—exceeding the agent count in most real-world applications. The results are presented below:\n\n| Methods | Parameters (M) | FLOPs (G) | Inference time (s) | ADE | FDE |\n|:---- |:---- |:---- |:---- |:---- |:---- |\n| MSRL | 0.59 | 0.12 | 0.09 | 1.50 | 2.13 |\n| LBEBM | 1.24 | 0.09 | 0.05 | 0.84 | 1.47 |\n| NPSN | 0.22 | 0.14 | 1.29 | 0.75 | 1.04 |\n| CausalHTP | 0.04 | 0.16 | 2.54 | 0.71 | 1.30 |\n| MRGTraj | 4.36 | 20.04 | 0.06 | 0.69 | 1.36 |\n| Our | 3.41 | 0.24 | 0.08 | 0.58 | 1.02 |\n\nThe results demonstrate that our method achieves the best performance with a relatively good model efficiency, making it suitable for deployment on embedded robotic systems. Additionally, with an inference speed exceeding 12 FPS, our method meets the real-time decision-making requirements of robotics applications.", "claims_and_evidence": "This paper provides sufficient experiments to support their claims.", "methods_and_evaluation_criteria": "The proposed dataset makes sense for the trajectory prediction task.", "theoretical_claims": "I have checked their claims.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "N/A.", "essential_references_not_discussed": "N/A.", "other_strengths_and_weaknesses": "Strengths:\n\n1. The 3DMoTraj dataset provides a valuable benchmark for evaluating 3D trajectory prediction algorithms in realistic settings. Its frame-wise annotations for both static and dynamic intentions offer precise descriptions of motion characteristics in 3D environments. The proposed dataset is likely to facilitate further research in this area.\n2. The authors show that the prediction complexity of 3D trajectories is nearly double that of 2D trajectories, given that a 3D Gaussian distribution requires optimizing 9 parameters. The authors demonstrate that a 3D Gaussian distribution can be decomposed into independent 1D Gaussian components along with a correction factor.\n3. The proposed methodology is well designed. The divide-and-conquer strategy, comprising decoupled trajectory prediction and correlated trajectory refinement can reduce the overall prediction complexity.\n4. Extensive experiments, including ablation studies and comparisons with state-of-the-art methods, reveal that the proposed approach significantly enhances prediction accuracy and robustness.\n \nWeaknesses:\n1. Experimental validation using a 3D trajectory dataset collected from UAVs would further strengthen the paper, given the increased complexity of UAV motion trajectories. I understand that collecting and annotating such a dataset requires substantial effort and may be impractical under current time and resource constraints; however, I look forward to seeing future work on UAV trajectory prediction.\n2. A thorough analysis of the model's efficiency is necessary. In the context of robotics applications, prediction algorithms must not only ensure accuracy but also achieve high inference speed to enable real-time decision-making. Evaluating the computational cost and runtime performance of the proposed method would strengthen the paper’s contributions.", "other_comments_or_suggestions": "N/A.", "questions_for_authors": "Refer to above sectoins.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eN87iCoQuR", "reviewer": "Reviewer_Z4gj", "summary": "In this paper, the authors address the challenge of predicting 3D trajectories, which is more complex than 2D trajectory prediction. To achieve this goal, the authors first introduce the 3DMoTraj dataset, collected from unmanned underwater vehicles (UUVs) in oceanic environments. Then, they propose a new method with two key components: decoupled trajectory prediction and correlated trajectory refinement. Based on the new dataset and solution, they reported extensive experiments to show the solution’s superior performance in 3D trajectory prediction.\n\n## Update After Rebuttal\nThe authors responded to my questions in a great way. They addressed my concerns. I do not have additional questions. I may keep my rating unchanged for this paper (weak accept) for the following reasons:\n1. The newly collected dataset is they key factor. Collecting more data with annotations is a good contribution to the community. Thus, I prefer to accept this paper. \n2.  However, the improved accuracy claim in the paper is not that convincing, which resulting in a \"weak accept\" score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: Demonstrate the generalization capability to 2D datasets on more baselines.**\n\n**A1**: We evaluated our prediction strategy on four additional baselines using two widely adopted 2D datasets: ETH&UCY and SDD. All models were trained and tested on the same machine for fair comparison.\n\n|Methods|ETH&UCY|SDD|\n|:----|:----|:----|\n|PECNet|0.30/0.48|10.02/15.79|\n|PECNet+our|0.25/0.42|9.45/15.26|\n|NPSN|0.28/0.44|8.56/14.95|\n|NPSN+our|0.25/0.39|8.34/14.36|\n|MSRL|0.20/0.36|8.36/13.85|\n|MSRL+our|0.19/0.34|8.29/13.56|\n|TrajCLIP|0.21/0.35|7.69/13.31|\n|TrajCLIP+our|0.19/0.32|7.63/13.30|\n\nThese results demonstrate that our approach consistently enhances the performance of multiple baselines, further validating its generalization capability to 2D trajectory prediction.\n\n***\n**Q2: Report the accuracy for each prediction step.**\n\n**A2**: The prediction accuracy at each step for the baseline LBEBM and our method is presented in the table below. Additionally, a visualized line chart of these results is provided; Please refer to Figure-2 at the URL <https://anonymous.4open.science/r/ICML_ID4436/README.md>.\n\n|Steps|#1|#2|#3|#4|#5|#6|#7|#8|#9|#10|#11|#12|\n|:----|:----|:----|:----|:----|:----|:----|:----|:----|:----|:----|:----|:----|\n|LBEBM|0.18|0.34|0.47|0.58|0.69|0.79|0.87|0.95|1.06|1.27|1.40|1.47|\n|Our|0.17|0.29|0.35|0.43|0.49|0.55|0.61|0.67|0.72|0.81|0.89|1.02|\n\nThe results and their line chart indicate that our method achieves progressively more significant improvements over the baseline as the prediction horizon extends. This highlights our approach’s ability to mitigate the error accumulation problem in trajectory prediction, further validating its capability to reduce prediction complexity.\n\n***\n**Q3: Discuss more recent work, for example, [1]-[4].**\n\n**A3**: We will incorporate a discussion of the recent works [1]-[4] in our manuscript.\n\n***\n**Q4: It will be better to report some failure cases.**\n\n**A4:** We visualized a representative failure case in Figure-3 at <https://anonymous.4open.science/r/ICML_ID4436/README.md>, showing that our method struggles with trajectories featuring multiple sharp bends in short time frames. A more advanced interaction modeling could improve such case. However, as our primary focus is reducing the prediction complexity of 3D trajectories, we adopt a simple interaction modeling strategy, leading to suboptimal performance in this case. In the future, we will explore specialized 3D interaction modeling for complex motion patterns.\n\n***\n**Q5: Maybe we can try GRU instead of LSTM.**\n\n**A5:** We evaluated the impact of replacing LSTM with GRU in our method, and the results are presented below:\n\n| Architecture|ADE|FDE|\n|:----|:----|:----|\n|LSTM|0.58|1.02|\n|GRU|0.59|1.06|\n\nThe results indicate that GRU performs slightly worse than LSTM in our method. However, considering GRU’s higher computational efficiency, we recommend using the GRU-based version for low latency or resource-limited scenarios.\n\n***\n**Q6: The content of Section 4.1 does not precisely match with Figure 4.**\n\n**A6:** Sorry for the typos in Section 4.1; we will refine them to ensure alignment with Figure 4.\n\n***\n**Q7: Report the profiled results (latency, power, etc.).**\n\n**A7:** We compared our method’s profiled results with several top-notch methods. Specifically, we tested all models on an NVIDIA 2080 Ti GPU using an input size of 70×8x3, where 70 represents agents' number predicted simultaneously—exceeding the agent count in most real-world applications.\n\n|Methods|Parameters (M)|FLOPs (G)|Inference time (s)|ADE|FDE|\n|:----|:----|:----|:----|:----|:----|\n|MSRL|0.59|0.12|0.09|1.50|2.13|\n|LBEBM|1.24|0.09|0.05|0.84|1.47|\n|NPSN|0.22|0.14|1.29|0.75|1.04|\n|CausalHTP|0.04|0.16|2.54|0.71|1.30|\n|MRGTraj|4.36|20.04|0.06|0.69|1.36|\n|Our|3.41|0.24|0.08|0.58|1.02|\n\nThe results show that our method achieves the best performance with a relatively good model efficiency, making it suitable for deployment on embedded robotic systems. Additionally, with an inference speed exceeding 12 FPS, our method meets the real-time decision-making requirements of robotics.\n\n***\n**Q8: Change the iteration and layer numbers on different scenes or datasets?**\n\n**A8:** It is advisable to adjust these hyperparameters for scenes with significant variations, as different environments impact agent movement to varying degrees. For instance, underwater environments introduce more excellent resistance and turbulence than aerial environments, necessitating different hyperparameters for optimal performance.\n\n***\n**Reference**\n\n[1] Feng L, et al. Unitraj: A unified framework for scalable vehicle trajectory prediction. ECCV. 2024.\n\n[2] Xu Y, et al. Adapting to length shift: Flexilength network for trajectory prediction. CVPR. 2024.\n\n[3] Heravi M Y, et al. Deep learning-based activity-aware 3D human motion trajectory prediction in construction. ESWA. 2024.\n\n[4] Shi Z, et al. Design of a UAV Trajectory Prediction System Based on Multi-Flight Modes. Drones, 2024.", "claims_and_evidence": "Overall, claims made in this paper are supported by evidence. \n1. The dataset. This paper provides detailed description of the dataset collection, including basic, motion, curvature, intention information. Besides that, the authors also report statistics information about the dataset. \n2. The proposed solution has a detailed description and complexity analysis. The additional information reported in the supplementary is also helpful. \n\nThe improved accuracy claim may need a stronger support. \nIn Table 2, the proposed solution performs better than other existing work on the newly collected dataset, e.g., MRGTraj (2023), MS-TIP(2024). However, in Table 6, on other existing 2D trajectory prediction datasets, only one solution (LBEMB, 2021) was compared, which makes the generalization capability of the proposed solution and accuracy improvement claim become a little bit less convincing.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria make sense for the trajectory prediction task. It will be better, if the authors could report the accuracy for each prediction step. In this case, readers may get more sense about the performance of the proposed solution, e.g., the model performs better at the first few steps and the performs worse at the last step, etc.", "theoretical_claims": "This paper does not have challenging mathematical proofs or theoretical claims. (1) The proposed Decoupled Trajectory Prediction and Correlated Trajectory Refinement modules should work in theory. (2) The prediction complexity analysis looks convincing.", "experimental_designs_or_analyses": "Overall, the experimental designs and analyses make sense. However, as mentioned above, it will be better to compare the proposed solution with more existing solutions on public datasets to prove the good generalization capability of the proposed solution.", "relation_to_broader_scientific_literature": "The proposed solution was adopted from the LBEBM method (Pang et al, 2021). The authors replace LBEBM’s decoder with three independent decoders to predict trajectories separately along the x-, y-, and z-axes. In terms of the newly collected dataset, it should be a good addition to the existing datasets, e.g., ETH/UCY, SDD, etc. (especially for the 3D trajectory prediction task).", "essential_references_not_discussed": "It will be better if the authors could discuss more recent work, for example:\n* Lan Feng, Mohammadhossein Bahari, Kaouther Messaoud Ben Amor, Éloi Zablocki, Matthieu Cord, and Alexandre Alahi. \"Unitraj: A unified framework for scalable vehicle trajectory prediction.\" In European Conference on Computer Vision, pp. 106-123. Cham: Springer Nature Switzerland, 2024.\n* Yi Xu, and Yun Fu. \"Adapting to length shift: Flexilength network for trajectory prediction.\" In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15226-15237. 2024.\n* Moein Younesi Heravi, Youjin Jang, Inbae Jeong, and Sajib Sarkar. \"Deep learning-based activity-aware 3D human motion trajectory prediction in construction.\" Expert Systems with Applications 239 (2024): 122423.\n* Zhuoyong Shi, Jiandong Zhang, Guoqing Shi, Longmeng Ji, Dinghan Wang, and Yong Wu. \"Design of a UAV Trajectory Prediction System Based on Multi-Flight Modes.\" Drones 8, no. 6 (2024): 255.", "other_strengths_and_weaknesses": "Other Strengths:\n1. The intuitions behind deigns were described in details, e.g., to address the increased prediction complexity of 3D trajectory, decopuled trajectory prediction and correlated trajectory refinement were proposed. \n2. The supplementary material provides more details and analysis help readers better understand their work. \n\nOther Weaknesses:\n1. It will be better if the authors could report some failure cases. It may more important for the 3D prediction task, because 3D motion has more freedom than 2D motion and has higher possibility to have failure cases. Reporting failure cases will help readers better understand the performance of the proposed solution.", "other_comments_or_suggestions": "1. Maybe we can try GRU instead of LSTM. There are some work show GRUs achieves better resutls than LSTMs. \n2. It seems like the content of Section 4.1 does not exactly match with Figure 4. For example, Equation (2) has a concatenation operation, but we cannot find it in Figure 4. Maybe we can refine the content and the figure. \n3. Even though the authors provide a complexity analusis, it will be better if the authors could report the profiled results (latency, power, etc.).", "questions_for_authors": "1. Do we need to change the values of iteration number and layer number if we apply the solution on different scenes or datasets?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper was reviewed by four experts in the field, and received three positive ratings (4, 3, 3) and one negative (2). The positive reviews commend the paper's clear justification, the value of the dataset, and the effectiveness of the proposed trajectory prediction method. The negative review appears to be based in misunderstanding. The reviewer states "The dataset is an analog simulation data", but the paper never says this, and the authors helpfully re-state that the data is real-world. The negative review further questions whether the paper's claims will generalize outside the provided dataset, and the authors defuse this concern in the rebuttal by additionally testing on some UAV and 2D data. In sum, the AC congratulates the authors on their good rebuttal, and recommends acceptance.
Decision: Accept (poster)