TrajAgent: An LLM-Agent Framework for Trajectory Modeling via Large-and-Small Model Collaboration
Authors: Yuwei Du
Jie Feng
Jie Zhao
Yong Li
Venue: NeurIPS 2025 poster ()
Keywords: LLM-based Agent
Trajectory Data Mining
Collaboration Learning
Primary area: applications

Abstract: Trajectory modeling, which includes research on trajectory data pattern mining and future prediction, has widespread applications in areas such as life services, urban transportation, and public administration. Numerous methods have been proposed to address specific problems within trajectory modeling. However, the heterogeneity of data and the diversity of trajectory tasks make effective and reliable trajectory modeling an important yet highly challenging endeavor, even for domain experts. In this paper, we propose TrajAgent, a agent framework powered by large language models (LLMs), designed to facilitate robust and efficient trajectory modeling through automation modeling. This framework leverages and optimizes diverse specialized models to address various trajectory modeling tasks across different datasets effectively. In TrajAgent, we first develop UniEnv, an execution environment with a unified data and model interface, to support the execution and training of various models. Building on UniEnv, we introduce an agentic workflow designed for automatic trajectory modeling across various trajectory tasks and data. Furthermore, we introduce collaborative learning schema between LLM-based agents and small speciallized models, to enhance the performance of the whole framework effectively. Extensive experiments on four tasks using four real-world datasets demonstrate the effectiveness of TrajAgent in automated trajectory modeling, achieving a performance improvement of 2.38%-69.91% over baseline methods. The codes and data can be accessed via https://github.com/tsinghua-fib-lab/TrajAgent.

TrajAgent: An LLM-Agent Framework for Trajectory Modeling via Large-and-Small Model Collaboration
Yuwei Du, Jie Feng" ! Jie Zhao, Yong Lit Department of Electronic Engineering, BRNist, Tsinghua University, Beijing, China {fengjie, 1iyoog17) edu. cn
Abstract
Trajectory modeling, which includes research on trajectory data pattern mining and future prediction, has widespread applications in areas such as life services, urban transportation, and public administration. Numerous methods have been proposed to address specific problems within trajectory modeling. However, the heterogeneity of data and the diversity trajectory tasks make effective and reliable trajectory modeling an important yet highly challenging endeavor, even for domain experts. In this paper, we propose TrajAgent, a agent framework powered by large language models (LLMs), designed to facilitate robust and efficient trajectory modeling through automation modeling. This framework leverages and optimizes diverse specialized models to address various trajectory modeling tasks across different datasets effectively. In TrajAgent, we first develop UniEnv, an execution environment with unified data and model interface, to support the execution and training of various models. Building on UniEmv we introduce an agentic workflow designed for automatic trajectory modeling across various trajectory tasks and data. Furthermore, we introduce collaborative learning schema between LLM-based agents and small speciallized models, to enhance the performance of the whole framework effectively. Extensive experiments on four tasks using four real-world datasets demonstrate the effectiveness of TrajAgent in automated trajectory modeling, achieving a performance improvement of 2.38% % over baseline methods. The codes and data can be accessed via com/ts ts com/tsinghe mussiinghu-fbbb fit--iibllaas lab-lab/Toe haab/Traaggen
thub
Introduction
With the rapid development of web services and mobile devices 68 61 large-scale trajectory data, such as check-in data from social network (58), have been collected, greatly facilitating research in trajectory modeling. Trajectory modeling involves the processing, mining and prediction of trajectory data, with widespread applications in urban transportation, location services and public management. The typical areas of trajectory modeling (6] 22
can be classified into five main categories: trajectory representation [21]. trajectory classification 29), trajectory prediction [60), trajectory recovery (44), and trajectory generation [5I] Each category encompasses various sub- tasks; for instance, the trajectory prediction task can be further divided into next location prediction task [31), final destination prediction task [666) and travel time estimation task [48), among others. Given the huge value of trajectory modeling in diverse practical applications, various algorithms and models [22] have been proposed to address these tasks, particularly deep learning-baad baaeed models in recent years. This has facilitated significant advancements in the field, with many tasks achieving high level of performance.
Equual contribution. Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Dl-Moold
Task
extomaible
Brved
limited
Enutning
Midel
Mi
NF
MI
Mae
training
LIM loae Apent
Iam Vaa
Data
Rlxed
linited
oxtendible
Mlddll
Multi Task Model
TrajAgent:: flsslble framework for large and mode collaburative learning
Figure 1: The paradigm of LLM based automated trajectory modeling framework TrajAgent.
However, existing methods are designed for specific tasks and datasets, making it difficult to share them across different tasks and data sources. For example, TrajFormer [29] is tailored for trajectory classification and cannot be applied in trajectory prediction or trajectory generation. Flash-back [57] is designed for sparse check-in trajectory prediction and is not suitable for dense GPS trajectory road retwork-based trajectory modeling. and the diverse nature of trajectory data-varying in resolution, format and geographical regions- existing methods can only be applied in limited task with specific data for specific regions. While some early studies have explored effective trajectory modeling via unified framework [30] 71 53].
other words, due to the heterogeneity
application scenario
they often face several limitations: 1) their performance on individual tasks lags behind that of specialized models; 2) the range of supported trajectory modeling tasks remains limited; and 3) their training and inference processes are complex and non-trivial. Thus, despite their significant contributions, there still long way to go, necessitating further exploration of more effective and reliable trajectory modeling frameworks.
In recent years, the rapid development of large language models (LLMs) 36 47 with extensive
commonsense and powerful reasoning abilities presents enable the LLM based agent (55
46 as a
new paradigm for solving complex task, such as automated software development 23
38
59 and
automated machine learning tasks [24] 42 67 (5). For example, HuggingGPT 42 utilizes LLM as manager to address various machine learning tasks with existing Al models, VisionLLM 52
investigates unified modeling for vision tasks across different vision domains. Inspired by these, we explore the potential of leveraging an LLM-based agent framework for automated trajectory modeling, paving the way toward effective and reliable trajectory modeling solution. Specifically, our approach seeks harness the capabilities of LLMs to establish collaborative framework between LLMs and various specialized models, enabling the automated and unified trajectory modeling. However, designing such an LLM-based agent for this purpose presents several significant challenges. Firstly, how to handle and integrate the diverse trajectory data and specialized models for different trajectory modeling tasks into a single, unified framework non-trivial. Secondly, the numerous steps involved in transforming raw trajectory data into the final model output are lengthy and cumbersome (6). making full automation of the process difficult and leading to a large action space for both planning and exccution. Finally, while model performance heavily depends delicate and specialized data adaptation and model optimization, the ultimate challenge lies in effectively automating the optimization of these adaptation processes.
In this paper, we propose TrajAgent, a systematic agent framework for automated trajectory modeling across diverse tasks and data. First, we design unified environment, UniEnv, to process diverse trajectory data and provide cohesive runtime environment for various trajectory modeling tasks within TrujAgent. In UniEmv, we define unified data and model interfaces to facilitate the seamless execution of different trajectory modeling methods. Building upon this environment, we develop an agentic workflow within TrajAgeni for the automatic multi-step planning and execution of trajectory modeling tasks. The diverse trajectory modeling task workflow is decomposed into four unified steps: task understanding, task planning, task exccution, and task summarization. For each step, design an expert agent to perform the corresponding operations effectively. Finally, we introduce a collaborative learning schema that integrates agent learning through reasoning with model learning through training, enabling the effective optimization of model performance for specific data and tasks. We further provide in in-depth analysis of optimization dynamics and failure modes, along with practical improvements to enhance robustness. In summary, our contributions are as follows,
To the best of our knowledge, TrajAgent is the first LLM-based agent framework for automated and unified trajectory modeling across diverse data and tasks. decomposes the trajectory modeling process into several sub-tasks, with expert agents designed to each.

---

UniEav
TrajAgent
TEAggalli Lorring reausng
Tak Eartur
Lacloo
Mald bartee
Cinnal
bes lepineds His wo Kill.
Blat
Langeara Hirary
Undervadda
Muurt lere Vineery
Chidki Trjuutay
Injucarr Irikie
Trgesary Matring Hlusig
Inppstay Aepeneaio
jabs
Poreadt Opeconser
Trgecar Reonnry
Optnindes
GrS Trijunaey Ian
Tagaour Ginuriaion
Calbbrrthe
Cullaborative Learning Schema
traasig
Togesssy Haldin Seaeery
Red Neteok Impelay Bex
LInited Modal Togsted, (011 Paransee untiage bell
Figure 2: The whole framework of TrajAgent.
To support the automated execution of various trajectory modeling tasks, we provide an unified running environment UniEnv by integrating diverse trajectory data and specialized models.
Furthermore, we develop collaborative learning schema between high-level agents and low- level models to effectively enhance the final performance of TrajAgent on targeted data and tasks, proposing closed-loop, feedback-driven optimization system that jointly adapts data augmentation strategies, model parameters, and agent reasoning based on real-time training outcomes.
Extensive experiments four representative trajectory modeling tasks across four real-world datasets demonstrate the effectiveness of the proposed framework, with the optimized model achieving performance gain of 2.38% to 69.91 % over baseline methods.
2 Methods
2.1 Overview of TrajAgent
Figure presents the comparison between work and existing works. Over the past few decades, researchers have developed various task-specific models for solving single tasks on limited datasets,
shown in the left of Figure [ Recently, some works 77] 30 53 have explored the potential of building unified models for multiple trajectory modeling tasks, as shown in the left of Figure [] However, limitations in available data and the diversity of tasks have constrained these unified models to a narrow range of tasks. Additionally, these models are not easy to train and utilize, making optimization challenging and far from being simple and user-friendly. Thus, following the success of LLM-based agentic framework in other domains, we propose to build a unify framework to enable the automated trajectory modeling via the collaboration between LLM-based agents and smaller specialized models. The framework is presented in the right of Figure [ In this framework, the LLM serves as a controller and processor, coordinating with specific smaller models accomplish specific trajectory tasks. This approach enables users to effortlessly extend support for various data and models without delving into the intricate details of individual models. essence, users can view the entire framework as a unified modeling platform, achieved through automated trajectory modeling across a variety of tasks and data. As TrajAgent A is to generate an optimized models M'. based on user query 4, optional trajectory data D, and the selected raw model M, the task definition as follows,
M' arg arg min L(M, T, A),
(1)
where T A(g). A(q. T), M A(T.D).
Figure presents the whole framework of TrajAgent. It contains three key components: (I) UniEnv, an environment with a unified data and model interface that supports the execution and training of various trajectory models; 2) Agentic Workflow, which is designed to automatically decompose and complete diverse trajectory modelling tasks; 3) Collaborative Learning Schema, an additional automated optimization module for enhancing the performance of specific model through the collaboration between M.aMe agents and specialized models with different learning mechanisms. Details of each component are presented follows.
3

---

2.2 UniEnv: Environment for Experiments
As shown in the left of Figure 2 UniEnv is comprehensive and integrated environment that bridges trajectory data, tasks, and models, providing foundational platform for trajectory modelling and analysis. It designed to support the entire lifecycle of trajectory modelling workflows, from data preparation to task execution and model optimization. UniEnv comprises four key components: rich set of datasets accompanied by processing tools, comprehensive task collection that defines and manages various task types, extensive model library with available source code, and an external tools pool for extending the capabilities of TrajAgent. Each component is seamlessly connected through unified interface, enabling agents to plan and execute trajectory modeling tasks with minimal complexity.
Task Interface: Figure summarizes the trajectory modeling tasks and associated models supported by UniEnv. The framework covers 5 fundamental trajectory modeling tasks: prediction, recovery, classification, generation, and representation, utilizing a total of 18 methods. For tasks such as prediction, recovery, and classification, we further introduce several subtasks. For example, the prediction task includes subtasks like next location prediction and travel time estimation, the classification task includes subtasks like trajectory user linking, intention prediction and anomaly detection. To enhance the clarity and effectiveness of understanding users' language queries, we provide a detailed language description for each task. This description helps extract precise task requirements from user queries and facilitates subsequent data and model selection processes.
UniEnv supports two commonly used trajectory data formats, namely Checkin trajectory (i.e., sequence of visited POIs) and GPS trajectory (i.e., sequence of gps points). These datasets, which come from different cities and with distinct forms, are pre-processing through standard pipeline that ensures compatibility across the system. Pre-processing steps are done by generated code scripts from I.LMs, include data cleaning, normalization, format transformation, which are crucial for handling inconsistencies between real-world datasets and task models. After processing, we will add description for each dataset to support efficient data selection in the subsequent stage.
Data Interface:
As previously mentioned, we support 18 models across 5 fundamental trajectory modeling tasks. To support training these models in TrajAgent, we select at least one well-know model for each task and adapt them to match the running environments in UniEnv. Furthermore, we extract the semantic context from original paper of each model with the txyz.ai A generate detailed description of model. In this description, the veri- fied data information and supported trajectory task information are recorded which supports the data and model selection in the subsequent trajectory planning stage of TrajAgent
Model Interface:
CHCSE
AutiTD
Porfutioo
OETNeT
Oemeration
Predition
F7MC
Profation
Truj Ageet
LIM
Trand
Trajetury
Liartey
Teleuaes
DepTTE
To extend the capabilities of TrajA- gent, we collect several external tools UniEmy, in- cluding paper context extraction tool txyz.ai, hyper- parameter optimization tool optuna, processing and visualization tool for trajectory data movingpandas, open street map data processing tool osmnx. Here, we also regard the LLM APIs utilized in the agen- tic workflow as one of the interface the UniEnv, including the ChatGPT API and DeepInfra for open- source LLMs.
External Tools:
MusTtE
Maching
Figure 3: The TrajAgent framework supports 5 fundamental trajectory modeling tasks, en- compassing a total of 18 methods. Detailed introduction of methods can refer to the ap- pendix A
2.3 Agentic Workflow of TrajAgent
As shown in the middle of Figure 2, the agentic workflow of TrajAgent is organized into four key modules: task understanding, task planning, task optimization, and task summary, which form an
https///ww.txxz.aa/

---

automated processing chain from user query to final result, eliminating the need for human-in in-thee loop. Specifically, the task understanding module first receives user instructions in natural language form, and analyzes and identifies the type, name, and other key information of the tasks involved. Then, the task planning module will plan for the identified task, including dataset matching and model selection. Next, the task execution module executes the planning task and cooperate with the additional performance optimization module collaborative learning schema to further improve the task performance from both the agent learning and model learning perspectives. Last, the task summary module generate an analytical report of the task based on historical interactions and decisions of TrajAgent. Following the common practice p3,, each module in TrajAgent can be regarded as a small agent, consisting of a function module for executing its core function, a memory for recording the history interaction, and a reflection module for learning practice from the memory.
Task Understanding Module: As the first module of TrajAgent workflow, task understanding module is designed to interact with user and extract detailed task information to launch subsequent stages. Given the user query, understanding module recognize the potential task name from it with the predefined supported tasks as additional input. If users ask for the out-of-scope tasks which has not been supported in UniEmv, we will directly recommend user to select task from the supported list.
Follow the task understanding module is the planning module which is designed to generate the subsequent execution plan for efficient experiments of trajectory modelling. The input of the planning module is the task name and description from the understanding module, the supported data and model with brief description from UniEm. With the carefully designed prompt, the generated execution plan will contain the data name and model name for the given task, and also the detailed model optimization plan. Due to the characteristics of different tasks and existing practice, not all the model optimization are necessary for each task. If possible, skipping the optimization step which is time-consuming and costly can accelerate the whole procedure without sacrificing performance. After generating the plan, it will start simple execution step to verify the feasibility of the plan. Once any error occurs during the execution, e.g., the model name is wrong, the planning module will obtain the feedback from UniEnv and start to regenerate new plan with the last plan as the failed history in its memory.
Task Planning Module:
Give execution plan, the task execution module is responsible for invoking UniEny to execute the experiment plan. In addition to the previously mentioned basic execution interface, another interface of this module is to call collaborative learning schema module to complete the model optimization automatically. For both interface, the task execution module will give the feedback including error information for failed cases and performance metrics for success cases.
Task Execution Module:
Task Summary Module:
After the execution module, we design a task summary module analyze the execution records to generate the optimization summary of the task. The summary contains the optimization path during the experiment and the final optimization result for the given task. User can also directly utilize the optimized model from the experiments via APIs for further applications.
2.4 Collaborative Learning Schema
Due to the geospatial heterogeneity and the diversity of trajectory data, the trajectory models usually cannot be directly transferred between data and regions. In other words, for various data in different region, the model needs to be trained from scratch. Thus, the sufficient optimization of various models with targeted data becomes emergent. In TrajAgent, we design collaborative learning schema to complete this automatic optimization and generate optimized specialized models for targeted task and data. As shown in the right part of Figure 2 the collaborative learning framework involves two levels of learning: high-level knowledge reasoning for agents and low-level data training for specialized models. The high-level agent proposes training settings for the low-level models based on its expert knowledge and experimental records. The low-level models are then trained using these settings, and their performance metrics are reported back to the high-level agent for further collaborative learning. This iterative process continues until the performance meets the predefined requirements or the maximum number of exploration epochs is reached.
2.4.1 Agent Learning via Reasoning
Following Reflexion 43 we design expert optim agent for learning from the experimental records via reasoning. The standard optim agent utilizes the history operation and related results as the
5

---

feedback to update its action in the next step. Specifically, it works as two stages, including "think" and "action". To support the "think then action", builds a long-term memory for recording all experimental data and short-term memory for historical actions. In the "think" stage, optim agent analyzes the long-term memory and meta information of experiment and generating the guidance of action in the short-term memory. In the "action" stage, optim agent analyze the results in short-term memory to generate the action. Different optimization mechanism utilize the same optim agent with different action space and optimization tips. Besides, as shown in Figure 1 our framework can also utilize LLM-based agent as the specialized model to complete the trajectory modelling tasks, Thus, design prompt optimization agent to optimize agent based specialized models. We keep all experimental records in each experiment, including the raw input trajectories, the LLM's inference results, and the reasoning process. We select the two best- pest-performing trajectories along with their corresponding inference results and reasoning process as high-scoring memory entries, which then added to the original prompt for re-run of the experiment. This process of conducting experiments and selecting results is referred to as one iteration. In each iteration, the high-scoring memory entries are updated based on the experiment results.
To validate that agents leverage causal reasoning over memory logs (not pattern matching), we conduct an ablation where memory entries were stripped of performance scores (i.e., no "good/bad" labels). As shown in Appendix Table 13] performance stagnated (AAce@5 < 0.5%), confirming that score-guies gooreg reflection is essential. This mirrors reinforcement learning's reward signal, enabling the agent to infer why certain actions succeed.
2.4.2 Model Learning via Training
By collaborating with the high-level agent, the low-level specialized models learn specific trajectory patterns tailored to the target trajectory data and tasks. To enhance performance, we introduce several optimization techniques, including data augmentation, parameter tuning, and joint optimization.
Data Augmentation. Based on the high-level optim agent, we introduce the specific action space for low-level trajectory data augmentation. For GPS/map- based trajectories, we adopt geometry- augmentation pipeline inspired by DeepMM [12]: (1) raw trajectories are first downsampled to preserve spatial topology; (2) noise is injected only within road network constraints; (3) augmented samples are validated for temporal consistency before merging with original data. For check- in trajectories, follow the practice from existing works [69 9 63 defining fixed set with ten operators for trajectory data augmentation, e.g., insert, replace, split and so on. During the optimization, the operator set with simple description is provided the optim agent, it can select optimal operator aequence(combination of operators with their simple parameters) and parameter configuration as the action, guided by training feedback. Then the specialized models are trained with the augmented trajectory data and report performance metrics. The optim agent obtain the feedback information, e.g.. performance metrics, from UniEmv continue update its memory and action.
Parameter Optimization. The action space of parameter optimization is defined based on the pa- rameters of model itself. We define a parameter configuration file for each model, the optim agent reads the configuration file and generates code as the action to update the parameters in it. To better understand the meaning of each parameter, we add comments for each parameter in the file. This kind of action space is flexible to adapt with any models.
Joint Optimization. Furthermore, we introduce the joint optimization mechanisms to further improve the performance. Due to the different working paradigms, the direct combination of two kinds of optimizations is unsuccessful. We designate the optimization order to prioritize data augmentation first, followed by parameter optimization. This means that once the performance of data augmentation stabilizes, the agent proceeds with parameter optimization. This procedure can be repeated fixed number of times until it meets the stop criteria.
3 Experiments
3.1 Settings
Data. We utilize the widely used Foursquare (FSQ) 58 and Brightkite (BGK) 7 in our framework as the default check-in trajectory data. Porto (77 and Chengdu 8 are integrated the framework as the default GPS trajectory data. Besides, we use Tencent [34] the road network based methods

---

Table 1: Performance of representative methods across five fundamental trajectory modeling tasks. For the ten subtasks, only one model is presented for each. 'DA' represents data augmentation, 'PO' denotes parameter optimization, 'PRO' denotes prompt optimization, 'JO' indicates joint optimization, ō represents performance improvements.
Taok
Trajectory Prediction
Trajectory Recovery
Trajectory Classification
Trojectory
Trajectory
SahTask
Next Loc Pre
TTE
Recovery Map-matehiing User Linking Intent Predietion
Anomaly
Generation
Representation
Metric
Acc
Acc
MAE
MAE Tencent
MAE
Actin Tencent
A
Are; Beijing
AUC
MAE
Acc05
Dataset
ESQ
ESQ
Porto
Porto
FSQ
Porto
Earthquaak
FSQ
Models
GETNest
LLM-ZS
MTTE-TTE Duty TTE TrajERT
GruphMM
S2TUL
LIMP
GM-V
DSTPP
CACSR
Origin
0.3720
0.3110
163.12
190.82
42.71
0.2014
0.5755
0.9892
+DA
0.3894
0.3258
0.6846
.PO
0.3995
0.3902
128.57
179.01
27.78
0.3477
0.757
0.9899
0.3584
0.3466
+PRO
0.9225
0.7627
+JO
0,4002
0.3350
128.57
179.01
27.78
0.3422
0,7800
0.7627
0 0.9899
0,3584
0,3472
5
7.38%
7.72%
21.18%
6.19%
34.96%
69.91%
35.57%
2.38%
22.27%
12%
Table 2: Comparison of performance on check-in trajectories for TrajAgent with different configura- tions, which demonstrate the generalization of TrajAgent across different tasks, models and datasets.
FSQ
BGK
Task
Next Location Prodiction
Trajectory User Linking
Next Location Prediction
Trajectory User Linking
Model Metries
RNN
DeepMove GETNext MainTUL. Acc@5
DPLink S2TUL
RNN
DeepMove GETNext MuL DPLink S2TCL Acc@5
Hic@5
H
Origin
0.1795
0.3422
0.3720
0.4871
0.7551
0.5755
0.4422
0,5570
0.5324
0.5908
0.8993
0,5802
+DA
0.2667
0.4018
0.3894
0.5973
0.7551
0.6846
0,5416
0.5647
0.6026
0.6836
0.9613
0.6903
+PO
0.1795
0.3422
0.3995
0.5691
0.7686
0.7570
0.5022
0.6041
0.6116
0.6683
0.9552
0.7137
+JO
0.2717
0,4018
0,4002
0.6121
0,8010
0,7802
0.5470
0.6100
0.6227
0.7145
0.9622
0.7240
51.36%
17.42%
7.58%
25.66%
6.08%
35.57% 23.70%
9.52%
16.96%
20.94%
6.99%
24.78%
to support road network based tasks and Beijing 28 with human labeled intention to support the mobility intention prediction task. Finally, to verify the effectiveness of the whole system, we utilize self-instruct method (50) with seed queries to generate 300 user queries as the experiment input.
Models. As shown in Figure 3 our framework supports 18 models spanning 5 core trajectory model- ing tasks, which are further categorized into 9 subtasks. The next location prediction task includes FPMC (3). RNN, DeepMove [13], GETNext [60], LLM-ZS [2]. The travel time estimation task comprises DeepTTE 48) and MulT-TTE 32.. The trajectory recovery task features TrajBERT 44] while the map-matching task incorporates DeepMM [14] and GraphMM [34]. The trajectory user linking task includes DPLink [8], MainTUL [5]. and S2TUL [00. The mobility intention prediction
task is supported by LIMP (8 The trajectory anomaly detection task employs GM-VSAE [33]
and the trajectory generation task includes ActSTD
(65 and DSTPP
64 Finally, the trajectory
representation method is implemented using CASCR
21
Metrics. We adopt standard practices for each task to select appropriate metries for evaluating our framework. The widely used Acc@ k metric is employed for next location prediction, map-matching, trajectory user linking, mobility intention prediction, and trajectory representation tasks. The MAE metric is utilized for travel time estimation, trajectory recovery. and trajectory generation tasks. Lastly, the AUC metric is specifically defined for the trajectory anomaly detection task.
3.2 Overall Performance and Generalization Capability of TrajAgent
In this section, we assess the overall performance of TrajAgent across various fundamental trajectory modeling tasks, as summarized in Table Furthermore, to demonstrate the generalization capability of TrajAgent across different datasets and models, present detailed results diverse trajectory data using several representative models in Table 2
As shown in Table I we select at least one representative models for 9 trajectory modelling tasks to present the effectiveness of proposed framework. As Table [ shows. TrajAgent supports a variety of widely-known trajectory models and demonstrates superior performance across multiple trajectory modeling tasks and trajectory datasets. The output of TrujAgent consistently outperforms the original methods, achieving performance gains ranging from 2.28% to 69.91 %. For instance, in the next- location prediction task, TrajAgent harnesses its agentic workflow and collaborative learning schema for automatic modeling and optimization, leading to significant performance improvements of various
7

---

Table 3: Execution success rates and task performance at each stage of the agentic workflow of TrajAgent across different LLMs. The Acc@5 obtained by evaluating the next-location prediction task, with DeepMove as the default specialized model.
LLM
Extraction
Processing Suce.
Data/Model Selection
Data Augmentation
Parameter Optim.
Joint Optim.
Suce.
Succ.
Acc@5
Succ.
Acc@5
Suce.
Acc@1
Succ. Acc
Qwen2-7B
85.00%
30%
72%
83,33%
15%
0.2015
25%
0.1833
64%
0.2668
Mistral7B-Va V3
78, 78.89%
42%
88%
90.91%
94%
0.2940
95%
0.2087
82%
0.2980
L8
69 44%
28%
81%
80.25%
18%
0.1790
11%
0.1809
65%
0.2809
Gemma2-9B
83.88 88%
12%
57%
52.63%
18%
0.1822
15%
0.1848
70%
0.2970
Bemma-22-27B
79.44%
30%
70%
88.57%
78%
0.2507
30%
0.1775
78%
0.3366
Gurb.o
88.89%
54%
100) %
82.00%
88%
0.2846
90%
0.1809
92%
0.3295
LLama3-70B
83.33%
100%
95%
86.32%
92%
0.2931
83%
0,1848
95%
0.3473
Qwen2-72B
92.22%
95%
100) %
95%
96 %
0.3925
70%
0.1816
94%
0,4333
GPT-4o-mini
95.56%
92%
100 %
98.00%
90%
0.2967
85%
0.1822
96%
0.3724
Table 4: Ablation study of TrajAgent. 'MS' stands for Model Selection, 'DA' represents Data Augmentation, 'PO' denotes Parameter Optimization, 'JO' stands for Joint Optimization. indicates decrease in performance, and f indicates improvement.
Agent Variants
MŞ
DA
PO
JO
Succ.
Acc
Suec.
Acc
Succ.
Acc
Succ.
Acc
TrajAgent
100%
98%
98%
0,3050
89%
0.1895
85%
0.3724
w/o Reflection
100%
95%.
98%
0,3028
90%1
0.1872|
82% |
0,3212,
w/o Memory 9
80%
85%.
0.17074
70%
0,2050
68%.
0.1804.
models. Specifically, it enhances the deep learning- based model GETNext by 7.58% and the LLM- based model LLM-ZS by 7.72%. Performance gain in other tasks and models are much larger, for example improvement from 34.96% to 69.91 % for trajectory recovery tasks. We observe that the improvement in the trajectory anomaly detection task is minimal, primarily due to the strong performance of the original models the dataset.
As shown in Table 2 we presents a performance comparison for check-in trajectory tasks, including the next-location prediction task and the trajectory user linking task, across three models and two datasets. The first key observation the consistent improvement observed across tasks, models, and datasets, which highlights the potential generalization of the proposed framework. For different trajectory tasks, datasets, and models, TrajAgent consistently provides transferable performance improvements, ranging from 6.08% to 35.57%. Additionally, between different models (c.g.. the top two models for each task) on specific tasks and datasets significantly narrows from 24.88% to 9.5% following the automatic optimization of TrajAgent, emphasizing the critical role of effective data augmentation and parameter optimization.
observe that the performance gap
3.3 Ablation Study and Parameter Analysis of the Agentic Workflow
Here, we select the next location prediction task as an example to demonstrate the efficiency of designs of agentic workflow and the effects of various LLMs in Table B and Table 4 During the experiment, we select DeepMove as the default specialized model for next location prediction task.
Table 3 compares the execution efficiency of TrujAgeni implemented by different LLMs, across each stages the trajectory modelling workflow. We can observe that Qwen2-72B and GPT-40- mini demonstrate the highest success rates (i.e., Succ.) across key stages such as Data Extraction, Processing, and Data/Model Selection, with over 90% success in each. In contrast, models like Gemma-2-9B and LLama3-8B struggle with lower processing and data selection success rates, which results in reduced overall performance. Their weaker performance in key optimization stages, especially in parameter selection, reflects their limitations in effectively supporting TrajAgent. These results show that TrajAgent's 's efficiency is strongly influenced by the base LLM, with models like Owen-72B and GPT-4o-mini significantly enhancing its capabilities.
We conducted an ablation study by isolating two main designs: the memory unit and the reflection mechanism, resulting in two variants: 1) w/o Reflection, where the reflection mechanism is removed, and 2) w/o Memory, where the memory unit is excluded. The experimental results are presented in Table 4 We can find that: 1) Removing either component leads to average performance declines,
8

---

9.46
P05
1020
075
03W
0.7
036
0.36
633
30
30
Thought Step
Memory Length
Optinsiation Mediol
(a) Thought steps.
(b) Memory size.
(c) Comparison with Optuna and UrbanLLM.
Figure 4: (a-b) The impact of thought steps and memory size when using Deep.Move in next location prediction tasks, (c) Compared to Optuna and UrbanLLM, Traj Agent achieves better performance in trajectory user linking tasks with S2TUL as the specialized model.
with the memory unit being especially critical for maintaining execution efficiency. 2) Interestingly, removing a component occasionally results in slight increases in success or accuracy, suggesting that some components may introduce overhead or complexity in specific stages. Overall, the combined use of the memory and reflection mechanisms is crucial for optimizing TrajAgent's performance.
Due to the importance of reflection and memory in the TrajAgent, important related parameters: 1) thought step: the number of steps that agent thought before taking action in reflection, and 2) memory size: the size of memory units in TrajAgent.
analyze the effects of two
As shown in Figure [[a] performance initially improves with more thought steps, reflecting enhanced reasoning depth. However, beyond a threshold (e.g., 20 steps), performance declines-likel due to overfitting to suboptimal action sequences or repetitive exploration without sufficient novelty, Similarly, in Figure 44b) increasing memory size beyond 10 leads to performance degradation, suggesting that excessive historical records may introduce noise or bias the agent toward previously failed strategies (a phenomenon we term the 'optimization trap').
3.4 Analysis of Optimization Failure Modes and Improvements
While TrajAgent demonstrates strong performance across diverse trajectory tasks, we observe that its optimization efficacy does not monotonically improve with increasing reasoning depth memory capacity. To understand this phenomenon and enhance robustness, we conduct an in-depth analysis of the underlying optimization dynamics.
Optimization Trap in Long Reasoning Chains: As the number of thought steps increases, the agent may converge prematurely to a local optimum. Once trapped, the agent stops exploring novel strategies and repeatedly refines the same ineffective combination. This behavior is exacerbated in weaker LLMs, which lack sufficient reasoning capacity to escape such traps. In contrast, reasoning models like DeepSeek-v3 exhibit more diverse exploration and are more likely to discover globally superior configurations within fewer steps (see Table T0 in Appendix).
Memory Saturation and Noise Accumulation: Similarly, increasing memory size improves explo- ration efficiency up to point, but larger memories introduce noisy or redundant historical records. Low-performing action sequences, if retained, can bias future decisions and lead the agent revisit failed strategies-especialll when memory is not actively curated. This "memory pollution" effect explains the performance drop in Figure 4bb Crucially, stronger reasoning models (e.g., DeepSeek- v3, Gemini-2.0) are less affected, as shown in Table To Appendix: they maintain or even improve performance with larger memories by better filtering useful experiences, whereas weaker models degrade significantly. This highlights memory content management and optimization as a key factor in mitigating memory pollution.
is also worth noting that although Figure 4(b) shows parameter optimization (PO) is less sensitive to memory size than data augmentation (DA), excessive memory (>10 entries) still degrades PO due to noise accumulation. The apparent stability in PO stems from its smaller action space, but memory pruning is equally critical for both.
Mitigation via Contrastive Reflection and Memory Pruning: To address these issues, we introduce two practical improvements: (1) Contrastive Reflection: During the reflection phase, the agent

---

explicitly compares successful and failed trials, adjusting operator parameters to avoid repeating ineffective combinations. This encourages diverse yet informed exploration. (2) Dynamic Memory Pruning: We periodically discard low-scoring memory entries and retain only high-performing trajectories as high-scoring memory entries to guide future planning. As shown in Tables 12 and in Appendix, these strategies significantly stabilize wttimmiiaaionSiimiarll... memory pruning enables consistent gains across memory lengths, with performance no longer collapsing at large capacities.
3.5 Comparison with Automated Methods
We compare TrujAgent with the deep learning- AutoML method Optuna 1 and the LLM-based automated urban task-solving framework UrbanLLM [26]. Using the trajectory user linking task with S2TUL model as example, the results are presented in Figure 4(c) UrbanLLM (represented by the blue line) is designed to directly automate the use of existing models without further optimization, resulting in the lowest performance in Figure 4(c) Meanwhile, compared to the widely used AutoML method Optuna (represented by the red line), which focuses on parameter optimization, TrajAgent (depicted by the yellow line) achieves superior results with fewer trial-and-error iterations. Furthermore, its performance can be further enhanced through joint optimization combined with automated data augmentation, resulting in a significant performance improvement of over 11.1 %. For computational overhead please of TrajAgent, please refer to Table in Appendix.
4 Related Work
In recent year, trajectory modelling 68 22 makes great progress on its core research questions, including prediction BB
Trajectory Modelling and Analytics:
57 41 60), classification [25]
18 29 45 J recovery 54 56)) and generation (37 65 51 While these specific methods accelerate
the development of trajectory modelling from different aspects, they can only handle one type of task. In other words, the automated and unified model for all the trajectory modelling task is still missing due to the heterogeneity of tasks and trajectory data. In this paper, we propose to utilize the power of LLM and agent to build a unified model framework for diverse trajectory modelling tasks, which can automatically handle various data and modelling tasks without human intervention.
Large Language Models: LLMs with extensive commonsense and outstanding reasoning abilities have been widely explored in many domains, such as mathematics (62), question answering coman--machine interaction 47. Following this direction and motivated by the exploration of LLMs' usability in urban studies tailored for urban applications, for example, CityGPT [15). UrbanI.LM (26), and Lrb [66] In contrast to these approaches, which rely on fine-tuning L.MSs with domain-specific knowledge, our work focuses on a training-free paradigm by constructing an agentic framework.
[72). and
researchers have begun developing diverse domain-specifee L.MMs
are proposed to enhance the robustness and task solving abilities of LLMs for real-world complex tasks, such as web-navigation [61 35 70 and software development 23 38 59 Besides, researchers also explore
LLM based Agents: In the general domain, agentic framework
49
46
55
the potential of applying LLM based agent for automatic research experiments [3 40 4 especially machine learning experiments 42
42 67 52]. Recently, LLM- based agent also be applied in specific
trajectory modeling tasks, egg., trajectory prediction [12] and trajectory simulation [1]. In this paper, our proposed agentic framework is designed for unified trajectory modelling which can provide automatically model training and optimization for various trajectory data and tasks.
Conclusion
In this paper, we propose TrajAgent, LLM-based agentic framework for automated trajectory modeling. Supported by UniEm,, which provides a unified data and model interface, and collaborative learning schema for joint performance optimization, TrajAgent can automatically identify and train the appropriate model, delivering competitive performance across a range of trajectory modeling tasks. TrajAgent establishes new paradigm for unified trajectory modeling across diverse tasks and datasets.
10

---

Acknowledgments
This work was supported in part by the National Key Research and Development Program of China under grant 2024YFC3307603, in part by the China Postdoctoral Science Foundation under grant 2024M761670 1670 and G2B20240384, in part by the Tsinghua University Shuimu Scholar Program under grant 2023SM235. This research is supported by Tsinghua University Mercedes Benz Institute for Sustainable Mobility.
References
[1] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2623-2631, 2019.
[2] Ciro Beneduce, Bruno Lepri, and Massimiliano Luca. Large language models are zero-shot next location predictors. arXiv preprint wxxv:2405.209663 2024.
[3] Ben Bogin, Kejuan Yang, Shashank Gupta, Kyle Richardson, Erin Bransom, Peter Clark, Ashish Sabharwal, and Tushar Khot. Super: Evaluating agents setting up and executing tasks from research repositories. arXiv preprint arXiv:2409.07444, 2024.
[4] Daniil A Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes. Autonomous chemical research with large language models. Nature, 624(7992):570-578 2023.
[5] Wei Chen, Shuzhe Li, Chao Huang, Yanwei Yu, Yongguo Jiang, and Junyu Dong. Mutual distillation learning network for trajectory-user linking. arXiv preprint arX5v:2205.03773, 2022.
[6] Wei Chen, Yuxuan Liang, Yuanshao Zhu, Yanchuan Chang, Kang Luo, Haomin Wen, Lei Li, Yanwei Yu, Qingsong Wen, Chao Chen, et al. Deep learning for trajectory data management and mining: A survey and beyond. arXiv preprint arXiv: 14151, 2024.
[7] Eunjoon Cho, Seth A Myers, and Jure Leskovec. Friendship and mobility: user movement in location-bacce based social networks. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1082-1090, 2011.
[8] Didi Chuxing. Gaia open datasets. https://outreachh didichuxing. com/research/ opendata/ent /n/ 2018.
[9] Yizhou Dang, Enneng Yang, Guibing Guo, Linying Jiang, Xingwei Wang, Xiaoxiao Xu, Qinghui Sun, and Hong Liu. Uniform sequence better: Time interval aware data augmentation for sequential recommendation. In Proceedings of the AAAI conference artificial intelligence, volume 37, pages 4225-4232, 2023.
[10] Liwei Deng, Hao Sun, Yan Zhao, Shuncheng Liu, and Kai Zheng. S2tul: A semi-supervised framework for trajectory- user linking. In Proceedings of the sixteenth ACM international conference on web search and data mining, pages 375 383, 2023.
[11] Yuwei Du, Jie Feng, Jian Yuan, and Yong Li. Cams: A powyeet agentic framework for urban human mobility simulation. arXiv preprint arXiv:2506. 13599, 2025.
[12] Jie Feng, Yuwei Du, Jie Zhao, and Yong Li. Agentmove: Predicting human mobility anywhere using large language model based agentic framework. NAACL, 2025.
[13] Jie Feng, Yong Li, Chao Zhang, Funing Sun, Fanchao Meng, Ang Guo, and Depeng Jin. Deepmove: Predicting human mobility with attentional recurrent networks. In Proceedings of the 2018 world wide web conference, pages 1469-1468, 2018.
[14] Jie Feng, Yong Li, Kai Zhao, Zhao Xu, Tong Xia, Jinglin Zhang, and Depeng Jin. Deepmm: Deep learning based map matching with data augmentation. IEEE Transactions on Mobile Computing, 21(7):2372-238.. 2020.
11

---

[15] Jie Feng, Tianhui Liu, Yuwei Du, Siqi Guo, Yuming Lin, and Yong Li. Citygpt: Empowering urban spatial cognition of large language models. In Proceedings of the 31st ACM SICKDD Conference Knowledge Discovery and Data Mining V. 2, pages 591-602, 2025.
[16] Jie Feng, Shengyuan Wang, Tianhui Liu, Yanxin Xi, and Yong Li. Urbanllava: A multi-modal large language model for urban intelligence with spatial reasoning and understanding. arXiv preprini arXiv:2506.238 19, 2025.
[17] Jie Feng, Jun Zhang. Tianhui Liu, Xin Zhang, Tianjian Ouyang, Junbo Yan, Yuwei Du, Siqi Guo, and Yong Li. Citybench: Evaluating the capabilities of large language models for urban tasks. In Proceedings of the 31 ACM SIGKDD Conference on Knowledge Discovery and Data Mining 2, pages 5413-5424, 2025.
[18] Jie Feng. Mingyang Zhang, Huandong Wang, Zeyu Yang. Chao Zhang, Yong Li, and Depeng Jin. Dplink: User identity linkage via deep neural network from heterogeneous mobility data. In The world wide web conference, pages 459-469, 2019.
[19] Shanshan Feng, Haoming Lyu, Fan Li, Zhu Sun, and Caishun Chen. Where to move next: Zero-shot generalization of Ilms for next poi recommendation. In 2024 IEEE Conference Artificial Intelligence (CAI), pages 1530-1535. IEEE, 2024.
[20] Qiang Gao, Fan Zhou, Kumpeng Zhang, Goce Trajcevski, Xucheng Luo, and Fengli Zhang. Identifying human mobility via trajectory embeddings. In IJCAI, volume 17, pages 1689-1695, 2017.
[21] Letian Gong, Youfang Lin, Shengnan Guo, Yan Lin, Tianyi Wang, Erwen Zheng, Zeyu Zhou, and Huaiyu Wan. Contrastive pre pre-training with adversarial perturbations for check-in sequence representation learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 4276-4288 2023.
[22] Anita Graser, Anahid Jalali, Jasmin Lampert, Axel Weißenfeld, and Krzysztof Janowicz. Mobil- itydl: a review of deep learning from trajectory data. Geolnformatica, pages 1-33, 2024.
[23] Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308, 20000005200,, 2023.
[24] Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec. Mlagenthench: Evaluating language agents on machine learning experimentation. In Forty-first International Conference on Machine Learning, 2024.
[25] Xiang Jiang, Erico N de Souza, Ahmad Pesaranghader, Baifan Hu, Daniel L Silver, and Stan Matwin, Trajectorynet: An embedded gps trajectory representation for point-based classification using recurrent neural networks. arXiv preprint arXiiv1105.00000
2017.
[26] Yue Jiang, Qin Chao, Yile Chen, Xiucheng Li, Shuai Liu, and Gao Cong. Urbanlim: Au- tonomous urban activity planning and management with large language models. arXiv preprint arXiv:2406.11 12360, 2024.
[27] Kaggle. Pkdd 15: Predict taxi service trajectory (i). https://wwy kaggle. kom/ c/ pkdd 15-predict- -axi-servicce -aaae---eei.e-----aaetoooo------ 1/overview 2015.
[28] songwei Li, Jie Feng, Jiawei Chi, Xinyuan Hu, Xiaomeng Zhao, and Fengli Xu. Lmp: Large language model enhanced intent-aware mobility prediction. arXiv preprint arXiv: /2832, 2024.
[29] Yuxuan Liang, Kun Ouyang, Yiwei Wang. Xu Liu. Hongyang Chen, Junbo Zhang, Yu Zheng, and Roger Zimmermann. Trajformer: Efficient trajectory classification with transformers. In Proceedings of the 31 ACM International Conference on Information & Knowledge Manage- ment, pages 1229 1000-1277,200 2022.
[30] Yan Lin, Tonglong Wei, Zeyu Zhou, Haomin Wen, Jilin Hu, Shengnan Guo, Youfang Lin, and Huaiyu Wan. Trajfm: A vehicle trajectory foundation model for region and task transferability. arXiv preprint 1255,
12

---

[31] Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. Predicting the next location: A recurrent model with spatial and temporal contexts. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016.
[32] Xin Liu, Josh Fromm. Shwetak Patel, and Daniel McDuff. Multi-task temporal shift attention networks for on-device contactless vitals measurement. Advances in Neural Information Processing Systems, 13:19940-11411, 2020.
[33] Yiding Liu, Kaiqi Zhao, Gao Cong, and Zhifeng Bao. Online anomalous trajectory detection with deep generative sequence modeling. In 2020 IEEE 36th International Conference on Data Engineering (ICDE), pages 949-960. IEFE, 2020.
[34] Yu Liu, Qian Ge, Wei Luo, Qiang Huang, Lei Zou, Haixu Wang, Xin Li, and Chang Liu. Graphmm: Graph-based vehicular map matching by leveraging trajectory and road correlations. IEEE Transactions on Knowledge and Data Engineering, 36(1):184-198, 2023.
[35] Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christo- pher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answeriig with human feedback. arXiv preprint arXiv:21 2021.
[36] OpenAI. Introducing chatgpt, 2024. Accessed: 2024-10-14.
[37] Kun Ouyang, Reza Shokri, David S Rosenblum, and Wenzhuo Yang. A non-parametric generative model for human trajectories. In IJCAI, volume 18, pages 3812-3817, 2018.
[38] Chen Qian, Wei Liu. Hongzhang Liu, Nuo Chen, Yufan Dang. Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, al. Chatdev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15174-151 15186, 2024.
[39] Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme Factorizing personalized markov chains for next-basket recommendation. Proceedings of the 19th international conference on World wide web, pages 811-820, 2010.
[40] Bernardino Romera-Pareded Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language models. Nature, 625(7995):468 475, 2024.
[41] Yibin Shen, Cheqing Jin, Jiaxun Hua, and Dingjiang Huang. Tipnet: A neural network for travel time prediction based on tensor decomposition and graph embedding. IEEE Transactions on Knowledge and Data Engineering, 34(9):451444526, 2020.
[42] Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving tasks with chatgpt and its friends hugging face. Advances Neural Information Processing Systems, 36, 2024.
[43] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024.
[44] Junjun Si, Jin Yang. Yang Xiang, Hanqiu Wang, Li Li, Rongqing Zhang, Bo Tu, and Xiangqun Chen. Trajbert: Bert-based trajectory recovery with spatial- refinement for implicit sparse trajectories. IEFF Trunsactions on Mobile Computing, 2023.
[45] Li Song, Ruijia Wang, Ding Xiao, Xiaotian Han, Yanan Cai, and Chuan Shi. Anomalous trajec- tory detection using recurrent neural network. In Advanced Data Mining and Applications: 14th International Conference, ADMA 2018, Nanjing, China, November 16-18, 2018 Proceedings 14, pages 263-277. Springer, 2018.
[46] Theodore R Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas I. Griffiths. Cognitive architectures for language agents. arXiv preprint 2tt.i::200.0222277 2023.
13

---

[47] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov. Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint urXiv:2307.0922. 2023.
[48] Dong Wang. Junbo Zhang, Wei Cao, Jian Li, and Yu Zheng. When will you arrive? estimating travel time based deep neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
[49] Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345, 2024.
[50] Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hanmameh Hajishirzi. Self-instruct: Aligning language models with self-generated instruc- tions. arXiv preprint ar arXiv:2212.1056/0 2022.
[51] Tonglong Wei, Youfang Lin, Shengnan Guo, Yan Lin, Yiheng Huang, Chenyang Xiang, Yuqing Bai, and Huaiyu Wan. Diff-mttrj: A structure-aware diffusion model for road network- constrained trajectory generation. IEEE Transactions on Knowledge and Data Engineering, 2024.
[52] Jiannan Wu, Muyan Zhong, Sen Xing, Zeqiang Lai, Zhaoyang Liu, Wenhai Wang, Zhe Chen, Xizhou Zhu, Lewei Lu, Tong Lu, al. Visionllm v2: An end-to-end generalist multimodal large language model for hundreds of vision-language tasks. arXiv preprint arX1v:2406.08334, 2024.
[53] Xinhua Wu, Haoyu He, Yanchao Wang, and Qi Wang. Pretrained mobility transformer: A foundation model for human mobility. arXiv preprint arXiv:2406.02578, 2024.
[54] Dongho Xi, Fuzhen Zhuang, Yanchi Liu, Jingjing Gu, Hui Xiong, and Qing He. Modelling of bi- directional spatio-temporal dependence and users' dynamic preferences for missing poi check-in identification. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 3458-54651 2019.
[55] Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, al. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.074 07864, 2023.
[56] Tong Xia, Yunhan Qi. Jie Feng, Fengli Xu, Funing Sun, Diansheng Guo, and Yong Li. Attmove: History enhanced trajectory recovery via attentional network. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 4494 4502, 2021.
[57] Dingqi Yang. Benjamin Fankhauser, Paolo Rosso, and Philippe Cudre-Mauroux. Location prediction over sparse user mobility traces using mns. Proceedings of the twenty-ninth international joint conference on artificial intelligence, pages 2184-21 1890,, 2020.
[58] Dingqi Yang, Daqing Zhang, and Bingqing Qu. Participatory cultural mapping based on collective behavior data in location-based social networks. ACM Transactions on Intelligent Systems and Technology (TIST), 7(3):123,, 2016.
[59] John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. Swe-agent: Agent-computer interfaces enable automated software engineering. arXiv preprint arXiv:2405.15793 2024.
[60] Song Yang, Jiamou Liu, and Kaiqi Zhao. Getnext: trajectory flow map enhanced transformer for next poi recommendation. In Proceedings of the 45th International ACM SIGIR Conference on research and development in information retrieval, pages 1144-1153, 2022.
[61] Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems, 35:20744-20757, 2022.
14

---

[62] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting language models. arXiv preprint arXiv:2210.03629, 2022.
[63] Mingjia Yin, Hao Wang, Wei Guo, Yong Liu, Suojuan Zhang, Sirui Zhao, Defu Lian, and Enhong Chen. Dataset regeneration for sequential recommendation. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3954-3965, 2024.
[64] Yuan Yuan, Jingtao Ding, Chenyang Shao, Depeng Jin, and Yong Li. Spatio-temporal diffusion point processes. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3173-3181 3184, 2023.
[65] Yuan Yuan, Jingtao Ding, Huandong Wang. Depeng Jin, and Yong Li. Activity trajectory generation via modcling spatiotemporal dynamics. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4752-4766 2022.
[66] Xiaocai Zhang, Zhixun Zhao, Yi Zheng, and Jinyan Li. Prediction of taxi destinations using a novel data embedding method and ensemble learning. IEEE Transactions on Intelligent Transportation Systems, 2 (1):688778, 2019.
[67] Yuge Zhang, Qiyang Jiang, Xingyu Han, Nan Chen, Yuqing Yang, and Kan Ren. Benchmarking data science agents. arXiv preprint arXiv:2402.17168, 2024.
[68] Yu Zheng. Trajectory data mining: an overview. ACM Transactions on Intelligent Systems and Technology (TIST), 6(3):1-41, 2015.
[69] Peilin Zhou, You-Liang Huang, Yueqi Xie, Jingqi Gao, Shoujin Wang. Jae Boum Kim, and Sunghun Kim. Is contrastive learning necessary? a study of data augmentation vs contrastive learning in sequential recommendation. In Proceedings of the ACM on Web Conference 2024, pages 3854-3863, 2024.
[70] Shuyan Zhou, Frank Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents, arXiv preprint arXiv:2307.1884. 2023.
[71] Yuanshao Zhu, James Jianqiao Yu, Xiangyu Zhao, Xuetao Wei, and Yuxuan Liang. Unitraj: Universal human trajectory modeling from billion-scale worldwide traces. arXiv preprint arXiv:2411.03859, 2024.
[72] Yuchen Zhuang, Yue Yu, Kuan Wang, Haotian Sun, and Chao Zhang. Toolqa: A dataset for Ilm question answering with external tools. Advances in Neural Information Processing Systems, 36:50117-50143, 2023.
A Appendix
A.1 Datasets
Foursquare (FSQ): This dataset consists of 227,428 check-ins collected in New York City from
04/12/2012 to 02/16/2013, Each check-in is associated with timestamp, GPS coordinates and corresponding venue-category.
Brightkite (BGK): This dataset contains 4,491,143 check-ins of 58,228 users collected from BrightKite website.
Porto: This dataset contains 1.7 million taxi trajectories of 442 taxis running in Porto, Portugal from 01/07/2013 to 30/06/2014. Each trajectory corresponds to one completed trip record, with fields such as taxilD, timestamp and the sequence GPS coordinates.
Chengdu: This dataset contains GPS trajectory records of Chengdu from 01/11/2016 to 30/1 1/2016. Each record includes taxilD, timestamp, longitude and latitude, collected and released by Didi Chuxing.
Tencent 34 : This dataset includes both a road network and set of vehicle trajectories collected
northeastern Beijing. The road network consists of 8.5K road segments and 15K edges, and the trajectory dataset contains 64K vehicle trajectories, each sampled at 15 15-second intervals.
15

---

This dataset contains check in records collected from popular social networking platform. It spans period from late September 2019 late November 2019.I also contains intent labels annotated by human for small dataset.
Beijing [28]:
UserQueries: To verify the effectiveness of the whole system, we utilize self-instruce method (50) with seed queries to generate 300 user queries as the experiment input.
More detailed information for the Check-in and GPS trajectory datasets are summarized in Table and Table 6 respectively. Note that the raw datasets are typically city-scale and data-intensive.
Loading them all into the Arentgent framework is costly. In this work, we are selecting part of data for task model training and testing during experiments.
Table 5: Statisties of the Check-in trajectory datasets.
Datasets
Foursquare (FSQ)
Brightkite(BGK)
Beijing
Num. Users
463
272
1566
Num. POIs
19870
5006
5919
Num. Trajectories
10632
22208
744813
Table 6: Statisties of the GPS trajectory datasets.
Datasets
Porto
Chengdu
Tencent
Sampling Rate
15s
38
15s
Num. Traj.
1.710.670
5,819.383
10,000
Avg. Traj. Length (m)
3522.64
2857.81
2492.01
Avg. Travel Time (s)
724.20
436.12
13903.78
Latitude Range
[41.1401, 1859 [30.6529, 30.72771|40.0222,
40.09301
Longitude Range [-8.6902, -8.5491] [104.042, (14.626) 116.349]
A.2 Models
As introduced in Figure we adopt various deep learning based and LLM based models and incorporate them into ArejAgent for solving trajectory-related tasks. According to the type of tasks they deal with, these models can be framed in the following categories:
Next Location Prediction: RNN
attention-based method like DeepMove [3], well-
performed graph-based method like GETNext 660 and two recent proposed LLM-based methods
LLM-ZS and LLMMove 19
Travel Time Estimation: DeepTTE
48 and MulT-TTE 32 to estimate the travel time for a given
GPS trajectory.
Trajectory User Linkage: widely-used DPLink (18) MainTUL (5) and S2TUL 110 are considered. We modified DPLink's training approach by using publicly available sparse trajectory datasets instead of heterogeneous mobility datasets for training.
Travel Intent Prediction: LIMP 28 which leverages the commonsense reasoning capabilities of
LLMs for mobility intention inference.
Trajectory Anomaly Detection: consider the well- well-performing method GMVSAE 33 which represent different types normal trajectories in a continuous latent space.
Trajectory Generation: ActSTD (65) which capture the spatiotemporal dynamics underlying
trajectories by leveraging neural differential equations and DSTPP [64], which defines spatial temporal point process for trajectory generation.
Trajectory Recovery: TrajBERT 44 which encode trajectory sentence and train BERT
model to get representations of trajectories.
Trajectory Map Matching: DeepMM [4] which proposes a data augmentation approach for
map-based trajectory data, and GraphMM
34 which leverages a graph-hased framework to extract features from map-based trajectory data, are considered.
Trajectory Representation: CASCSR (21). which use contrastive lcarning method to learn trajectory representations for downstream tasks.
16

---

A.3 Metrics
To evaluate the performance of all models on multiple trajectory tasks, we employ the following different metrics:
Ace@5 and H:t@5:
Acc@5 refers the percentage of the first five results predicted correctly. H measures whether at least one of the top-5 predictive results correct.
Accm: Acc, is the evaluation metric which computes the average matching degree of all trajecto- ries.For each trajectory, its matching degree is the ratio of the number of matching road segments the number of all road segments.
Acc;: Ace; is used measure the accuracy of trajectory intention inference. It is the ratio of the number of matching predicted intention each check-in to the total number of check-ins.
MAE and AUC: Mean Absolute Error (MAE) indicates the amount of deviation from the actual values. Area Under ROC Curve (AUC) measures how well the model correctly distinguishes the type of the sample.
JSD: Censen-Shannon divergence (JSD) measures the discrepancy of distributions between the gen- erated data and real- world data. Lower JSD denotes a closer match the statistical characteristics and thus indicates better generation result.
In our experiments, Acc@5 is used to measure the accuracy of trajectory prediction and agent execution, is also used to measure the downstream applications of trajectory representation. is designed for measuring the performance of map matching Ackk,Acc; is designed for measuring the performance of travel intention prediction task. Hit@5 adopted for evaluating the performance of trajectory user linkage task; MAE is employed to compute the error of travel time estimation and trajectory recovery. while the metric AUC is used to assess the performance of trajectory anomaly detection.JSD and RMSE used to measure the prediction error of the spatiotemporal domain in trajectory generation task.
All experiments are conducted on Ubuntu server equipped with 8 NVIDIA GeForce RTX 3090 GPUs. Each small model trained using single RTX 3090 GPU, while each large language model (LLM) is accessed through its corresponding API provider.
A.4 Additional results on GPS trajectory data
Table Comparison of task performance on GPS trajectories for TrajAgent with different configura- tions.
Task
TTE
Anomaly
Recovery
Model Metrics
DeepTTE MulTTTE MAE
GMVSAE AUC
TrajBERT MAE
origin +JO
8.48
129.35
0.9892
13.6667
Porto
5.85
109.85
0.9899
8.0290
8
31.01% %
15.08%
minor
41.25%
Origin +JO
7.23
166.29
0.978
54,8060
Chendu
5.95
128.57
0.984
29.8134
5
17.70%
22.68%
0.61%
54.39%
Table 7 presents the performance comparison on GPS trajectory tasks, specifically focusing on TTE and TAD tasks. The models evaluated are DeepTTE for TTE and GMVSAE for TAD, with results shown for the original configuration (Origin) and after Joint Optimization (+JO). For the Proto dataset, the original model has an MAE of 8.48, which significantly improves to 5.85 after joint optimization, resulting in % reduction in prediction error. On the Chengdu dataset, the MAE decreases from 7.23 to 5.95. As for TAD task, we find that the AUC scores were already high, the small but consistent improvements in both datasets suggest that TrajAgent's joint optimization can further refine model performance, even for tasks where models initially perform well.
17

---

A.5 Limitations and Failure Mode Analysis
In this section, we analyze some cases where the TrajAgent's optimization performance is suboptimal. The overall process is illustrated in Figure The optimization module key component affecting overall performance.
The first issue is optimization trap present in data augmentation module of TrajAgent. Specifically, refers to the situation where agent sometimes ignores the contents of the memory during the thought process. Even when the chosen parameter combination yields poor training results, the model overlooks the error feedback (i.e., the "Not good enough..." in the memory). The "optimization trap" occurs even in the best- best-performing GPT-4o-mini. As the length of the Memory increases, the impact of the optimization trap on overall accuracy grows. Once an optimization trap occurs, all memories within the same step tend to favor the same ineffective combination. We believe the causes of the optimization trap could be: (I) excessively long prompts leading to truncated inputs; (2) insufficient proportion of memory in the total input.
The second issue is the sub-optimality appears parameter optimization module of TrajAgent. This phenomenon exists across various datasets and model sizes. We believe the reasons for the poor performance might be: (1) the TrajAgent parameter optimization module is overly sensitive to the format of model outputs, treating all responses that do not meet the format requirements as invalid; (2) adjustments to certain parameters result in increased training time, reducing the total number of iterations.
To address these issues, we propose two enhancements: (1) contrastive reflection mechanism that learns from both successful and failed trials to avoid redundant exploration; and (2) dynamic memory management strategy that prunes low-performing historical actions. Experimental results (Tables 12 and IT confirm that these strategies effectively mitigate performance degradation at large step/memory sizes.
A.6 Additional Experimental Analysis
The optimization effect diminishes as the step increases: While selecting combinations of operators, the model further optimizes the configuration of each operator (e.g.. the original configuration file for "inserter" is insert nums: 1, insert_ration 0, insert ttttttte_e___e sort: maximum, percent_ no _e___oo__uuggn puaugment:: 0, ti insert_ times: 1). In a zero-shot scenario, the model explores optimization strategies based on dataset characteristics and the meaning of the operators, with a probability of converging to local optimum-i.e., finding a suboptimal combination and deeming the result sufficient, thus stopping the exploration of new combinations and selecting the best operator configuration based on this combination. We compared the Ilama3-70bb used in the paper with other reasoning models, and the results are shown in Table II (S2TUL, London, memory-length-- stronger reasoning capabilities attempt more operator combinations and have higher probability of finding better combinations in fewer steps. For instance, DeepSeek-v3 outperforms LLaMA-3- in step 4. Under the same operator combination and the same number of steps, models with stronger reasoning capabilities yield better optimization results. For instance, DeepSeek-v3 v3 outperforms -- in steps 3, 5, 6, 7, 9, 10, 11, and 13.
We found that models with
Implement reflection similar to contrastive learning between steps, such as further adjusting the parameters of each operator based on effective combinations, to avoid exploring ineffective combinations as much as possible. The improved results are shown in Table 12 (S2TUL, FSK-London, memory-lengthe
Improvement solution:
The optimization effect diminishes as the memory length increases: memory length refers the number of action proposals generated in each reasoning step. Increasing it can improve exploration efficiency but also raises the probability of falling into a local optimum. We compared the performance of llama3-70b with that of reasoning models, and the results are shown in Table 9 (S2TUL, FSK- London, step=5). Both models achieved relatively good results at memory lengthh,, but as the length increased, they faced the issue of converging to suboptimal solutions. However, models with stronger reasoning capabilities exhibited a stronger tendency to explore other combinations, thus having a higher probability of finding better combinations and escaping the "optimization trap".
18

---

Noae pane Bouak eanoo outh seroce una addrea RAW, JNS. RUCTS.
Atswer:
Next Location Prediction
USS INPUT
lnviking wetroeeteeeee yana ef cnnerooneee liedd suttttac parlorsd monbly cas irze ace amoed to coeld raily
dia.
side
Quertea:
ChEDIINNTE DESCRIPEION DEveROrion
mode
F'dlortggggg the dea Extrction Hit wor
Iratl yed macee cammocriiio
sugorel cack mode:
mado dranct homrh
whe bntelooiing Theught, Actias
pitettil sors
Apply Tiuney sinpee lleriee
chnernaree DEACRIPPION DEveROriSe
oconeet HV PERIHEAAAI LI00
enteodl profucio lyino
AMEANDNG DE OOGRATORSS
olu opecsses coplowrkies Eo opeciain
Quenttee:
Thag't duadd sauar aiclice conkkaaiin aggaraae..
Asgnestation
calo dhalll
aite Thauge, Actia
ATED ASTTS
Aenwer Stap AA
SCRATISPAD Goald wesker
Data Reansforratioo
Meewer:
Prac supronatio nutieds an
2R TERS
You une Dut Toaeiree Aogi Your ais ce
lnrraciioos luto
<CONTIO EAEERAA
Now.
Actit
that rmprrse splcn and cade
"NEANING ORERA FOESS
de moddl to concet persotm don momoor
OPTRATORSS
SHOTS
TERANSOORMAAION
Purumcter optimisation
TTRAMGOORAAATIONN PLAN
Sulect Reseecede Use
Meaver:
Baca
babd
beasing
Ia
SUBATUEEAD
day
vee. lire coeute Searee hee
Maalysis
Suggestion
Quadiax:
reriuct dut Your angerss atoagg
Locouie
Ti iili sp00n loinng indldd veleciess Aereeii Heee seletree we 2. Wilkk shaleg afda pepaeaaalli modk wilft
bejeckey eirigtak
Deep (omp exettyyssad
BAT 4.T TON
ditter with yeurci
postidee Deppleey bu pood conlecing wod ADCOMEN TATIO0 TIO0O pene contie
Figure 5: A representative example case of TrajAgent.
Table 8: Token consumption and time cost for trajectory processing with different models (traj 200). Input and output token counts are reported for two inference settings: step=1 and step=5 (with nemorr_lengther
Model
token(step=1) input
token(step=5) input
time cost
output
output
LLM-ZS
37,327
90.180
194,358
282,270
3h27min
Dut
1,108
284
7,718
1,823
1h17min
Improvement solution: Periodically optimize and update the memory organization, discard poor combinations, retain good ones, and guide the model to explore new combinations during the reflection phase. The improved results are shown in Table II (S2TUL, FSK-London, step=5).
19

---

Table 9: Step-wise @C performance and Operator Combination Order(OCO) traces for multiple large reasoning models.
10
11
12
13
14
15
16
17
L-mmkk700
ACCSS 56.25
56.28
59.10
68.16
68.18
68.39
68.
69 00.04
16.04
68. 68.39
69.04
89 06.04
87.01
69.05
68.
68.19
83.12
$8.96
OCO
11
113.91
[1,39]
11.99
11,399
[1,39]
[1,591
[1,3,9]
[1.99]
113391
[13.9]
[2.9.00]
[1,39]
11,391
[1,399]
[29,30]
[29,30]
DeepSeekerd
ACCes 56.28
79.22
59.52
61.26
81.17
68.83
09.25
69.05
K3.96
70.35
09.08
09.25
71.00
61.90
00.52
08.83
71,00
66.88
OCO
(29.10)
[1.39]
[1.39]
[2.910]
[1,399
[13,9]
[1,39]
[299.00]
[1339]
[1399]
[13.9]
[1339]
[1,39)
[1,4.]]
[1.3,9]
[1,39]
[13,599
Gemmin-220-fteh-001
ACCE5 56.25
73.16
61.09
70.35
08.83
$2.25
09:05
68.40
$7.00
65 58
79,87
83.12
71.65
84.63
$2.47
71.43
69.05
69.08
OCO
(29.10)
(13.9]
[1.3.]
[1339]
[3.9.00
[13,9]
[1,39]
[299.00]
[1.39]
[2.9.00]
[299.0]
[1339]
[2.9.00]
[2.9.00]
[1.3.9]
(1,39)
(1,39)
Table 10: Performance (ACC@5) and Operator Combination Order(OCO)) traces under different memory lengths for Llama3-70b and DeepSeek-v3.
memory-length
3
5
Llama3-70b
ACC@5
56,28 0
68.39
80.95
79.00
55.19
83.33
57.79
62.55
OCO
[1,3,9]
[2,9,10]
[2,9,10]
[1,2,9]
[2,9,10]
[1,4.9]
[2,6,10]
DeepSeek-v3
ACC@5
56,28 0
79.22
59.09
79.65
61.47
80.74
59.52
81.60
OCO
[2,9,10]
[1,3,9]
[3,6,9]
[2,5,9]
[2,9,10]
[1,4,9]
[2,9,10]
Table II: Raw step-wise performance and Operator Combination Order(OCO) traces for 1lama3-70b before and after improvement.
step
7
11
Pre-improvement
ACC@S
56.28
56.28
59.10
68.18
68.18
68.39
68.18
69.04
69.04
68.39
69.04
69.04
OCO
[]
[1,3.9]
[133.9]
[1,3,9]
[1,3,9]
[1,3,9]
[1,3,9]
[1,3,9]
[1,3,9]
[1.3.9]
[1,3,9]
Post-improvement
ACC@5
56.28
59.09
61.03
68.18
69.04
69.04
60.38
69.04
82.90
80.08
87.01
83.12
OCO
II [1,3,9] [1,3,91 [1,3,9]
11,3,91
11,3,9
[1,3,9]
[1,3,9]
[2,9,10]
12,9,101
[2,9,10]
[2,9,10]
Table 12: Performance and tool invocation Operator Combination Order(OCO) traces under different memory lengths before and after improvement.
memory-length
2
3
4
5
6
Pre-improvement
ACC@5 56.28
68.39
80.95
79.00
55,19
83.33
57.79
62.55
OCO
[1,3,9]
[2,9,10]
[2,9,10]
[1,2,9]
[2,9,10]
[1,4,9]
[2,6,10]
Post-improvement
ACC@5
56.28
68.39
69.05
80.95
81.60
84.63
80.52
85.73
OCO
[1,3,9]
[1,3,9]
[2,9.10]
[2.9,10]
[2,9,10]
[2,9.10]
[3.6,9]
Table 13: Performance comparison TrajAgent with and without utilizing score feedback in memory across optimization steps.
Step 0
4
5
7
9
10
11
12
score in memory
ACC (%)
56.28
56.28
59.10
68.18
68.18
68.39
68.18
69.04
69.04
68.39
69.04
69.04
87.01
Operatons
II
II
[1.3.]
[1.3,9]
[1,39]
[1.3.9]
[1.3.]]
[1.3.]
[1.3,9]
[1,39]
[1.3.9]
[1,3.]
[2.9.10]
Without score in memory
ACC@S (%)
56.28
45.23
47.61
58.87
Operators
1I
[1.49]
[2.9.0]]
[1.4.91
[1.49]
[1.4.9]
[1.4.]]
11.4.91
[1.4.9]
[1.4.9]
[1.4.9]
[1,4.]]
[2,6,10]
20

---

A.7
Prompt Examples
Parameter Optimization
User
Please select proper combination of hyperparameters of model in CONFIG HYPERPARAME- TERS.Adjust the selection, the combination of hyperparameters based on the main function of hyperparameters, the characteristics of the input data, the tuning principles, and memory to get high score. You should solve the task with interleaving Thought, Action and Observation steps.
<CHARACTERISTICS OF INPUT DATA>
The input temporal data contains time dietionary(key is the user Ihe value list containing all time points when the user is active in chronological order). the input user data contains user dictionary(key is the user ID, ID,the value is list containing all items that the user interacts with in chronological order).
<CONFIG HYPERPARAMETERS
{loc_emb_size 500, tim_ sin_emb_izee 10, ...J
<TUNING PRINCIPLES>
1.Start with small batch size (32-64) and small learning rate (0.001-0.01): This will help prevent overshooting and overfitting.
2. Increase batch size and learning rate: If the model is not overfitting, increasing the batch size and learning rate can help improve convergence.
3.Add dropout (0.2-0.5) to prevent overfitting: If the model is overfitting, adding dropout can help regularize the model.
4. I.nrrease embedding size: If the model is not capturing enough information, increasing the embedding size can help improve representational power.
5.Decrease learning rate and increase batch size: If the model is not converging, decreasing the learning rate and increasing the batch size can help improve stability.
<MEMORY>:
<SCRATCHPAD>:
In Thought step.you should reason how choose the combination of hyperparameters to get higher score.Plear consider following aspects:
1.Observe the hyperparameters with high scores in MEMORY, to determine the optimal hyper- parameters.
2.Use a grid search: Perform a grid search over a range hyperparameters with high scores to find the optimal combination with higher scores.
Increase batch size and learning rate, and use dropout to avoid overfitting.
4. Stop or reverse the adjusting trend if the score is decreasing. According to above aspects, please first learn experiences from MEMORY, then make plan for the action step.Please use the sentence structure 'Firstly... Then... Lastly' Let's think step by step.
Thought:
Assistant
Firstly, I will analyze the current hyperparameters in the CONFIG HYPERPARAMETERS...
Then, will consider the hidden size, which is currently set at 500.
Next, the learning rate currently at 0.001, which a good starting point.
Lastly, will monitor the model's performance closely....
Now, I will proceed with the action step to adjust the hyperparameters based on this reasoning.
21

---

Action:
1. Increase the tim_emb__tt bibz__zze from 10 to 20 to capture more temporal information.
2. Keep the loc_size at 500 for now, it seems adequate.
3. Increase the hidden_size from 500 600 enhance the model's capacity,
4. Increase the dropout_p from 0.3 to 0.4 to further prevent overfitting.
5. Set the learning_ rate to 0.005 to improve convergence speed.
6. Start with batch size of 32 and monitor the performance closely.
Observation:
After implementing these changes, I will evaluate the model's performance on a validation set.
User
In Action step, you should consider the Thought step in SCRATCHPAD, and give dict: thet:bhypermeter name:hypermeter value]. J.The hypermeter name should be the same with the raw config hypermeter names in CONFIG should be the same type as the hypermeter values CONFIG HYPERPARAMETERS Please do not add any comments to each value.
SERRPPRRAMTERSS hypermeter values
Action:
Assistant
{ floc_emb__a eem__eize', 500....)
Task Understanding
User
The description of each task is in DESCRRPICR,,,,ON.
Please parse out the task name the sentence aims address in RAW WWW_INSTTUCSS
1. The task name should match the key in the DESCRIPTION.
2. Please only output the task name. Do not output other contents.
<TASK DESCRIPTION>
Bec_Loacatio___rddittti ... Brnjectory_user__inkkeee
<RAW_INSTRUCT
want to identify the trajectory...
Assistant
Data Augmentation
User
<TASK>
Please:
I.select proper augmentation methods and use them in proper order to jointly augment the input temporal data and user data.Pleass adjust the selection and combination sequence of operators based on MEANING OF OPERATORS, CHARACTERISTICS OF INPUT DATA, and MEMORY to get a high score.
2.select proper combination of hyperparameters of each augmentation method CONFIG HYPERPARAMETTER AAI the selection, the combination of hyperparameters based the main function of hyperparameters, the characteristics of the input data, and memory to get a high score.
22

---

You should solve the task with interleaving Thought, Action and Observation steps.
<CONFIG HYPERPARAMETERS.
1. I.Ti-crop: crop_nums:iint) the size of cropping. default is 2.
(fo_ratio:(fllat) the ratio of cropping. default
crop_n_ sio___timmen (it_eettmi:i)) the number of cropping for each sequence for default setting. default is 2.
crop_ time (str choice in [maximum, minimum) choose the candidate subsequence in descending/ascending order according to its time interval variance. default 'minimum'...
2
<MEANING OF OPERATORS>
I.Ti-crop: For each user, filter sessions with fewer trajectory points than ti threshold _cro___ttimes In each session, randomly extract several segments (t_croo___ttmms) of equal length (ti threshold). Sample data segments according to the rules.
2.Tr-insert unvisited:...
<MEMORY>:
SCRATCHPADD:
In Thought youp.you should reason how to choose the combination of operators and proper combination of hyperparameters of each augmentation method get a higher score.Please consider following aspects:
1.The meaning of each operator,and the impact of adding removing operators at specific location on the score.
2. Common characteristics of operator index lists with higher scores in MEMORY.Avore using the same index list as MEMORY with scores lower than 0.1811.
3.The meaning of hyperparameters of each operator you select, and how to adjust the hyper- parameters based the characteristies of input data and hyperparameters with high scores in MEMORY.
4. Use grid search: Perform grid search find the optimal combination with higher scores.
range of hyperparameters with high scores to
7. Stop or reverse the adjusting trend if the score is decreasing.
According above sspects.please first learn experiences from MEMORY, then make plan for the action Please use the sentence structure 'Firstly... Then... Lastly' Let's think step by step.
Thought:
assistant
**Toughh:***
Firstly, will analyze the characteristies of the input data, which consists of user activity timestamps and the corresponding items they interacted with. This suggests that temporal relationships and user preferences are crucial for effective augmentation. The augmentation methods should enhance these aspects without distorting the underlying patterns.
Then, will consider the meaning of each operator and their potential impact on the score. For instance, using 'Ti-crop' can help in segmenting the data, which is essential for understanding user behavior over time....
Lastly, will select hyperparameters for each operator based on their functions and the charac- teristics of the input data.
23

---

** *ction:
**Select Augmentation Methods:** *
'crop_nums': 3
'crop_aatio": 0.1 (to allow for small portion of the data be cropped)
'crop_ n n_times":
2. "Hyperparameter Configuration:**
For "Ti-crop', will keep crop_n_times at ensure multiple segments are created....
3. **Evaluate and Adjust:**
After implementing the above configurations, I will evaluate the score...
** Observation: **
will monitor the score after each adjustment to ensure that the changes lead to an improvement. If the score decreases, will revert to the previous settings.
User
<SCRATCHPAD>
*s
**Select Augmentation Methods:***
**Ti-crop**:....
In Action step, you should consider the Thought step in SCRATCHPAD, and return a list and dictionary.
The list should contain the indices of augmentation methods in [1: 'Ti-crop', 2: 'Ti- insert unvisited... example, if you want to first use Ti-crop, secondly use Ti-insert_random. thirdly use Ti-mask, then the list should be '[1,4,8]*.
The dict:...The hypermeter name should be the same with the corresponding operator hypermeter names in CONFIG HYPERPARAMETERS and hypermeter values should the same type as the corresponding operator hypermeter values in CONFIG HYPERPARAMETERS.
Please directly output the list and dictionary.For For example:..
Action:
Assistant
[1. 3,
| [ccrop_nums:3,cmm mwm.:3,cropppppp. numss33.roo_raatoo0..... (..........
24

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We do provide proper abstract and introductions for the submission.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in the experiment analysis.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
25

---

Justification:
We do not have theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide anonymous codes for reproducing experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
26

---

Answer: [Yes]
Justification:
The codes and data can be accessed via https:/ http://anonoooss.44pg
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm ec/
public/guides/ howpes//oooosbbbsssssoooooooo
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
//nips. //nips.cc/ppl wwcppblic/guuiicuccccccc wetgguddddeeoeeubbinnnnn
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide details in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
All experiments are run more than times to obtain stable and reliable results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
27

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: All experiments are conducted on Ubuntu server equipped with 8 NVIDIA GeForce RTX 3090 GPUs. Each small model is trained using a single RTX 3090 GPU, while each large language model (LLM) is accessed through its corresponding API provider.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We do follow the requirement of NeurIPS code of Enthics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss it in the experiments and future works.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
28

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We follow the MIT license for the code and models.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
29

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do provide README for codes.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
30

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification:
Only use LLM for formatting.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
31


Reviews:
[{"review_id": "YoyOnMTrCk", "reviewer": "Reviewer_KM23", "summary": "This paper proposes TrajAgent, an LLM-based agent framework designed to automate and unify trajectory modeling across diverse tasks (e.g., prediction, classification, generation) and heterogeneous data (e.g., GPS/check-in trajectories). The framework integrates three key components: (1) UniEnv, a unified environment for data/model interfaces; (2) an agentic workflow decomposing tasks into four stages (understanding, planning, execution, summarization); and (3) a collaborative learning schema that optimizes model performance via joint reasoning (LLM agents) and training (specialized models). Experiments across 9 subtasks and 18 models show performance gains of 2.38%–34.96% over baselines. The work positions itself as the first LLM-agent solution for unified trajectory modeling.", "questions": "See weaknesses", "limitations": "Yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe integration of LLM agents with trajectory modeling is novel. The framework addresses heterogeneity in data/tasks more holistically than prior task-specific models. Extensive experiments on 4 datasets and 18 models demonstrate consistent improvements. The ablation studies (e.g., memory/reflection mechanisms) add rigor. The paper is generally well-structured, with clear illustrations of the framework and workflows.\nWeaknesses:\n•\tThe optimization process lacks critical details. For example, LLM-generated data augmentation (e.g., \"insert/replace\" GPS points) risks distorting trajectory semantics without geometric validity checks.\n•\tExperiments omit computational costs. LLM inference (e.g., GPT-4o-mini) coupled with model training could be prohibitively expensive.\n•\tDoes the LLM truly understand memory logs in agent learning, or is parameter selection overly reliant on pattern matching from few-shot examples? Can you provide evidence demonstrating LLMs’ causal understanding of memory logs beyond few-shot imitation.\n•\tHow does iterative \"few-shot example updating\" avoid overfitting to dataset-specific biases, and what safeguards ensure generalizability?\n•\tFig 4b claims memory \"has little effect\" on PO, but Sec A.5 states larger memory worsens optimization traps.", "was_revised": true, "final_justification": "After reading the rebuttal and the comments from other reviewers, my final rating is borderline accept.", "rebuttal": "## For W1 \nWe appreciate your questions and apologize for not providing sufficient details in our paper.\n1. Data Augmentation:\n  - We augment **check-in trajectory data** by considering temporal intervals and define ten operators (e.g., cropper, inserter, masker, slide window), each with configurable parameters. We employ LLM-guided optimization combined with an evolving memory pool to identify the best strategy, including operator combinations, sequencing, and parameter settings (Please refer to PauW's W4). The strategy is dynamically adjusted based on training feedback.\n   - Experiments show that, unlike sequential recommendation models (e.g., CoSeRec) perform well in common augmentation methods, different trajectory modeling architectures require tailored augmentation strategies depending on the dataset (Please refer to Table 3 in MyNr's W1).\n  - For **map/GPS trajectory data**, we adopt DeepMM's rule-based approach: first downsampling the raw data, then applying noise injection rules, and finally merging the augmented data with the original (Please refer to GraphMM in Table 2 in PauW's W2).\n\n2. Parameter Optimization:\n\n    We leverage LLMs to autonomously search for critical parameters and infer optimal configurations based on real-time training feedback.\n\n3. Joint Optimization:\n\n    By combining data augmentation with parameter tuning, we expand the optimization space, leading to further performance gains.\n\nThis integrated approach enables adaptive, task-aware optimization while maintaining computational efficiency. We will incorporate these clarifications in the revised manuscript.\n## For W2  \nThank you for raising this critical issue. Table 7(Please refer to MyNr's Q4) compares token usage between LLM methods (includes inherent LLM overhead)  and Deep learning methods (pure TrajAgent consumption)  \n*Note*: Consumption scales with step count and memory_length.  \n## For W3\nOur experiments followed zero-shot settings (see PauW's W4), allowing free exploration of operator/parameter combinations. We experimentally removed the scoring and evaluation annotations from each record in the long-term memory, which prevented the agent from distinguishing between good and bad records during its reasoning and reflection phases, thereby impairing its ability to effectively learn from successful cases. As shown in Table 8, the results demonstrate that in our original design, TrajAgent's performance consistently improved with increasing steps and memory size (improvement strategies are detailed in PauW's W4). However, when memory scoring was removed, the performance failed to show progressive enhancement despite the growing number of steps.\n| Base-Model | Version | Metric   | Step 0 | Step 1 | Step 2 | Step 3 | Step 4 | Step 5 | Step 6 | Step 7 | Step 8 | Step 9 | Step 10 | Step 11 | Step 12 |\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n| llama3-70b   | With score in memory  | ACC@5    | 56.28  | 56.28  | 59.10  | 68.18  | 68.18  | 68.39  | 68.18  | 69.04  | 69.04  | 68.39  | 69.04   | 69.04   | 87.01   |\n| | | detailed | [] | []| [1,3,9]| [1,3,9]| [1,3,9]| [1,3,9]| [1,3,9]| [1,3,9]| [1,3,9]| [1,3,9]| [1,3,9] | [1,3,9] | [2,9,10]|\n| | w/o score in memory | ACC@5| 56.28| 45.23| 78.14| 47.40| 48.05| 57.36| 47.61 | 58.22| 58.23| 58.87| 58.87| 46.32| 63.85|\n| | | detailed | [] | [1,4,9]| [2,9,10]| [1,4,9]| [1,4,9]| [1,4,9]| [1,4,9]| [1,4,9]| [1,4,9]| [1,4,9]| [1,4,9] | [1,4,9] | [2,6,10]|\nTable 8.Performance comparison before and after removing scores from memory\n## For W4\nWe appreciate your feedback! We apologize for not clarifying this in the paper - all our experiments were conducted under zero-shot settings to ensure the model can freely explore optimal solutions based on current conditions and fully leverage its reasoning capabilities (for related discussions, please refer to MyNr's Q2). The learning mechanism of our agent framework resembles the trial-and-error approach in RL, which can generalize to unseen tasks or new domains not covered by UniEnv.\n## For W5\nThanks for your valuable feedback. Addressing this issue is one of our most recent research efforts. A detailed analysis of the problem's root causes, proposed solutions, and corresponding experimental results can be found in PauW's W4.\n\n# References\n[1]Beneduce C, Lepri B, Luca M. Large language models are zero-shot next location predictors[J]. IEEE Access, 2025.\n\n[2]Liu Y, Ge Q, Luo W, et al. Graphmm: Graph-based vehicular map matching by leveraging trajectory and road correlations[J]. IEEE Transactions on Knowledge and Data Engineering, 2023, 36(1): 184-198.\n\n[3]Feng J, Li Y, Zhao K, et al. DeepMM: Deep learning based map matching with data augmentation[J]. IEEE Transactions on Mobile Computing, 2020, 21(7): 2372-2384.\n\n[4]Wang Y, Chen Y, Zhong F, et al. Simulating human-like daily activities with desire-driven autonomy[J]. ICLR 2025.\n\n[5]Hu S, Lu C, Clune J. Automated design of agentic systems[J].ICLR 2025.\n\n[6]Liu Z, Chen Y, Li J, et al. Contrastive self-supervised sequential recommendation with robust augmentation[J]. arXiv preprint arXiv:2108.06479, 2021.\n\n[7]Gandhi S, Gala R, Viswanathan V, et al. Better synthetic data by retrieving and transforming existing datasets[J]. ACL 2024 Findings.\n\n[8]Shang Y, Li Y, Zhao K, et al. Agentsquare: Automatic llm agent search in modular design space[J]. ICLR 2025.\n\n[9]Gur I, Furuta H, Huang A, et al. A real-world webagent with planning, long context understanding, and program synthesis[J]. arXiv preprint arXiv:2307.12856, 2023."}, {"review_id": "d2AKtUDIqa", "reviewer": "Reviewer_du8A", "summary": "This paper proposes an innovative LLM-based agentic framework, TrajAgent, for automating trajectory modeling. Supported by a collaborative learning mechanism, TrajAgent can automatically identify and train appropriate models, demonstrating competitive performance across various trajectory modeling tasks.", "questions": "Please refer to Weakness.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The content of the paper is coherent, with clear language and appropriate logical structure. The figures are easy to understand.\n2. The experiments on TrajAgent are comprehensive, and the results are promising.\n\nWeaknesses\n1. Many notations are not explained. For example,  In Section 2.1, the variable T is not defined in the formula. \n2. It is unclear how many agent are in the framework. Authors mentioned an \"optim agent\", however, I cannot easily find it in Fig. 2.\n3. It is unlear how Learning via reasoning is achieved. The current explaination in Sec 2.4.1 is too vague.\n4. It is unlcear how the superiority of the proposed method is shown. Though the framework is novel, can it be compared with any other agentic methods?", "was_revised": true, "final_justification": "Thanks for authors' detailed rebuttals, which have solved my conerns.", "rebuttal": "## For W1\nWe appreciate your feedback and apologize for the oversight in our writing. In Eq. (1), *T* denotes the Task. The TrajAgent first parses the trajectory modeling task from user requests, then selects an appropriate trajectory dataset based on both the request and task type, and finally chooses a suitable small model according to the dataset and task requirements.\n## For W2\nWe regret not clarifying this in the paper. As illustrated in Figure 5, each functional module is executed by an agent in sequential order: \n1) Task Parsing\n2) Model Selection\n3) Data Processing\n4) Task Planning & Execution (including data augmentation and parameter optimization)\n5) Task Summarization\n\nAs a result, there are six agents in total.\n## For W3\nWe apologize for the insufficient explanation in the manuscript. Within the TrajAgent framework, the agents perform data preprocessing and model selection by analyzing: dataset descriptions, dataset samples, task descriptions, and model descriptions extracted from original paper abstracts, along with natural language instructions. It can retry or adjust selections based on executing results of modules. Optimization strategies (operator selection/sequencing, parameter tuning) are formulated by analysing parameter descriptions, model descriptions, dataset descriptions, and operator specifications. The agent dynamically refines strategies based on real-time training results from small models.  \n\nBackground information (dataset/task descriptions etc.) is provided for the *THOUGHT* phase, where the agent analyzes information and stores plans in short-term memory. During *ACTION*, it executes configurations by writing parameters to model/operator config files.  Model deployment/training is executed via shell scripts, with results fed back to the framework. The system maintains an adaptive threshold to classify results into \"good\" (stored in long-term memory) or \"bad\" memories. Regular memory optimization purges bad memories while retaining good ones.  Periodic *REFLECTION* phases allow the agent to learn from good results and avoid bad results, extracting experiential knowledge to guide subsequent *THOUGHT* processes.\n## For W4\nWe sincerely thank you for this valuable question. For supplemental responses, please refer to MyNr's W2. Our approach can be compared with three agentic method categories:  \n1. *Automatic workflow optimizers* (e.g., AFlow, AgentSquare[8]): These focus on public benchmark problems with deterministic solutions(e.g., GSM8K), making them unsuitable for trajectory modeling tasks.  \n2. *Domain-specific problem solvers* (e.g., WebAgent[9]): While effective for targeted applications like web search, they lack transferability to other domains.  \n3. *Urban science models* (e.g., UrbanLLM): Though pretrained for urban tasks, they cannot perform online training/inference or further optimize results."}, {"review_id": "9kdTCpEDFs", "reviewer": "Reviewer_MyNr", "summary": "Summary\nThe paper introduces TrajAgent, a novel framework that leverages Large Language Model (LLM)-based agents to automate and unify a wide range of trajectory modeling tasks, including prediction, classification, recovery, generation, and representation. Recognizing the fragmentation and task-specific nature of existing trajectory modeling approaches, TrajAgent integrates:\n\nUniEnv, a unified execution environment that standardizes interfaces for data, models, and tasks;\n\nAn agentic workflow that decomposes the modeling process into interpretable steps (understanding, planning, execution, and summarization), with each step managed by an LLM-based agent;\n\nA collaborative learning schema that enables interaction between reasoning-driven agents and training-driven specialized models to optimize performance across diverse data and tasks.\n\nThe framework supports 18 models across 5 major trajectory modeling tasks and demonstrates substantial performance improvements (up to 35.57%) over baseline methods on four real-world datasets. Ablation studies and comparisons with Optuna and UrbanLLM highlight the effectiveness and generalizability of the system, especially when equipped with powerful LLMs like GPT-4o-mini and Qwen2-72B.", "questions": "1. What drives the agent's decision-making during task planning and optimization?\n2. How well does the system generalize to unseen tasks or new domains not covered by UniEnv?\n3. Can you better quantify the LLM’s contribution versus the small models in performance improvement?\n4. How costly or scalable is TrajAgent in real-world scenarios?", "limitations": "Yes.\nThe authors have done a commendable job in addressing the limitations and potential negative societal impact of their work.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe paper presents a comprehensive and well-engineered system, combining a unified execution environment (UniEnv), an LLM-based agentic workflow, and a collaborative learning schema to tackle a wide range of trajectory modeling tasks.\n\nIt supports 18 models across 5 fundamental tasks and shows solid empirical performance, with improvements up to 35.57% over baselines on multiple datasets.\n\nAblation studies and LLM comparisons (e.g., GPT-4o-mini vs. Qwen vs. LLaMA) are thoughtfully executed and lend credibility to the framework's design.\n\nThe paper demonstrates clear generalization ability: the same framework adapts well to different tasks, datasets, and models with minimal human intervention.\n\nWeaknesses:\n\nThe technical novelty in algorithm design is moderate. While the framework is cleverly assembled, many components (e.g., agentic reasoning loops, parameter tuning, prompt optimization) build on prior art without deep theoretical innovation.\n\nThe LLM agent planning and decision mechanisms are largely rule-based or heuristic, with limited discussion of failure modes or theoretical guarantees.\n\nThe collaborative learning schema—central to the performance boost—could be explained more deeply in terms of what types of feedback are most valuable and how the agent adapts over iterations.", "was_revised": false, "final_justification": "", "rebuttal": "## For W1\nThank you for your question! We apologize for not elaborating in detail in the paper. Our key innovations are:  \n1. Compared to other works that use LLMs and fixed-parameter small models to solve urban science problems (e.g., UrbanLLM), we support online training and inference for small models, using real-time inference results as feedback in the agent’s reasoning loop, leading to more reliable outcomes.  \n2. Compared to other agent frameworks that automate task execution (e.g., HuggingGPT), we introduce an online optimization mechanism on top of automation.  \n   For check-in trajectory data, we explore augmentation from the perspective of time intervals, defining ten operators (cropper, inserter, masker, slide window, etc.), each with configurable parameters. Instead of rule-based search, we use **large models + evolutionary algorithms** to optimize the memory pool, searching for the best operator combinations, sequencing, and parameter settings of each operator, while adjusting strategies based on training feedback. Experiments show that, unlike sequential recommendation models (e.g., CoSeRec[6]) perform well in common augmentation methods, different trajectory modeling model architectures require different augmentation strategies depending on the dataset (see Table 3).  \n    For map/GPS trajectory data, we explore downsampling followed by noise injection (Please refer to GraphMM in following Table).  \n3. Compared to traditional optimization tools (e.g., Optuna[9]), we use LLM agents to retrieve key parameters and infer optimal augmentation strategies based on prior knowledge and online training results. We also explore **combining data augmentation with parameter optimization** to expand the search space and improve performance\n\n\n| Model | GraphMM (Acc@5) |\n|---|---|\n| Dataset | Tencent |\n| Origin | 0.2014 |\n| +DA | 0.3258 |\n| +PO | 0.2427 |\n| +Prompt | - |\n| Joint | 0.3422 |\n\n| base-model| step | 0| 1 | 2 | 3 | 4 | 5| 6| 7| 8 | 9| 10| 11| 12| 13| 14| 15| 16| 17|\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n| llama3-70b | ACC@5| 56.28 | 56.28 | 59.10 | 68.18 | 68.18 | 68.39 | 68.18 | 69.04 | 69.04 | 68.39 | 69.04 | 69.04 | 87.01 | 69.05 | 68.40 | 68.19 | 83.12 | 88.96 |\n|| detailed | [] | [] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [2,9,10] | [1,3,9] | [1,3,9] | [1,3,9] | [2,9,10] | [2,9,10] |\n| deepseek-v3 | ACC@5| 56.28 | 79.22 | 59.52 | 61.26 | 81.17 | 68.83 | 69.26 | 69.05 | 83.98 | 70.35 | 69.05 | 69.26 | 71.00 | 61.90 | 60.82 | 68.83 | 71.00 | 66.88 |\n|| detailed | [] | [2,9,10] | [1,3,9] | [1,3,9] | [2,9,10] | [1,3,9] | [1,3,9] | [1,3,9] | [2,9,10] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,4,9] | [1,3,9] | [1,3,9] | [1,3,5,9] |\n| gemini-2.0-flash-001  | ACC@5| 56.28 | 73.16 | 61.69 | 70.35 | 68.83 | 82.25 | 69.05 | 68.40 | 87.01 | 65.58 | 79.87 | 83.12 | 71.65 | 84.63 | 82.47 | 71.43 | 69.05 | 69.05 |\n|| detailed | [] | [2,9,10] | [1,3,9] | [1,3,9] | [1,3,9] | [3,9,10] | [1,3,9] | [1,3,9] | [2,9,10] | [1,3,9] | [2,9,10] | [2,9,10] | [1,3,9] | [2,9,10] | [2,9,10] | [1,3,9] | [1,3,9] | [1,3,9] |\nTable 3. Comparative analysis of TrajAgent performance across optimization steps\n\n| Base-Model  | Version | Metric | Step 0 | Step 1 | Step 2 | Step 3 | Step 4 | Step 5 | Step 6 | Step 7 | Step 8 | Step 9 | Step 10 | Step 11 |\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n| llama3-70b   | Pre-improvement  | ACC@5  | 56.28  | 56.28  | 59.10  | 68.18  | 68.18  | 68.39  | 68.18  | 69.04  | 69.04  | 68.39  | 69.04   | 69.04   |\n||| detailed | []| []| [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9]  | [1,3,9]  |\n| | Post-improvement  | ACC@5  | 56.28  | 59.09  | 61.03  | 68.18  | 69.04  | 69.04  | 60.38  | 69.04  | 82.90  | 80.08  | 87.01   | 83.12   |\n||| detailed | []| [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [1,3,9] | [2,9,10] | [2,9,10] | [2,9,10] | [2,9,10]  | [2,9,10]  |\nTable 4. Improvement: pre- vs. post-comparison\n\n## For W2\nThank you for your valuable question—this is indeed part of our ongoing and future work. For the most common issues, solutions, and experimental results, please refer to **PauW’s W3**.  \n## For W3\nWe apologize for not providing sufficient details in the paper.  \n- **What types of feedback are most valuable**\n\n    The model treats optimization strategies (e.g., operator combinations, parameter configurations) and their corresponding training results as feedback.  \n- **How does the agent adapt during iteration**\n\n    The framework sets a dynamic threshold that increases over time. Feedback with training results above the threshold is stored as good memory, while the rest is stored as bad memory.  The framework periodically updates the memory pool, discarding bad memories and retaining good ones.  During the reflection phase, inspired by contrastive learning, the model is guided to learn from good memories and avoid bad ones.\n\n## For Q1\n\nLLMs inherently possess knowledge of deep learning and data engineering. Prior work has shown that LLMs can perform data processing based on textual descriptions[7].\n\nWithin the TrajAgent framework, LLM agents perform data preprocessing and model selection by analyzing dataset descriptions, samples, task requirements, and model specifications from  abstractions of papers, and dynamically adjusting optimization strategy for operator selection and parameter configuration based on real-time feedback from small model training, with all model deployment and training executed through automated shell script calls.  \n## For Q2\nThank you for your question. To integrate a new model, simply add Task description , data description , model description  and codes of the model (as mentioned in **Q1**)  to the corresponding part of the TrajAgent framework.  For the agent’s reasoning process, refer to **du8A’s W3**. Due to space constraints, we could not list all details in Table 2, but we can provide them in the subsequent phase. \n## For Q3\nWe apologize for the lack of clarity in the paper. In Table 1 of original paper, the difference between Origin and +JO reflects TrajAgent’s optimization effect. \n\nAdditionally, as shown in Figure 1 , the left side (dashed line) represents traditional small models, which often handle only single tasks or perform poorly across multiple tasks. In contrast, TrajAgent can handle multiple tasks and automatically select the best small model for each task.\n\n## For Q4\nThank you for this practical question. In Table 7, we provide detailed token consumption when TrajAgent uses Large model methods  and Deep learning methods.\n\nAs mentioned in Q2, TrajAgent’s framework can be easily extended to tasks/domains not yet integrated into UniEnv, which is part of our future work.  \n| Model | Token (step=1) Input/Output | Token (STEP=5) Input/Output | Time Cost  |\n|-|-|-|-|\n| LLMZS, traj=200 | 37,327 / 90,180| 194,358 / 282,270| 3h27min|\n| DutyTTE | 1,108 / 284 | 7,718 / 1,823| 1h17min |\nTable 7. Cost analysis"}, {"review_id": "Z4InmycTsW", "reviewer": "Reviewer_PauW", "summary": "This paper proposes an LLM-Agent for trajectory modeling for various practical applications, including next location prediction, travel time estimation, trajectory generation, and many other relevant tasks. The proposed framework incorporates deep learning models and LLM-based models, utilizing collaborative learning to address to effectively enhance the performance.  The proposed unified running environment can integrate various models and process diverse trajectory data. Experimental results on various trajectory tasks demonstrate that the proposed unified framework achieves superior performance compared to other recent state-of-the-art methods.", "questions": "Tiny weakness:\n\n1.\tT in Eq. (1) indicates task?\n\n2.\tMore citations should be provided to support the assumption “Following the common practice[38]”.\n\n3.\tTypos and grammatical errors. For example,  “agentic framework [44, 41, 50] are” in Line 360 of the main manuscript.\n4.\tSome recent related works are missing:\n\n[1] Zhu, Y., Yu, J. J., Zhao, X., Liu, Q., Ye, Y., Chen, W., ... & Liang, Y. (2024, August). Controltraj: Controllable trajectory generation with topology-constrained diffusion model. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 4676-4687).\n[2] Mao, X., Lin, Y., Guo, S., Chen, Y., Xian, X., Wen, H., ... & Wan, H. (2025, April). DutyTTE: Deciphering Uncertainty in Origin-Destination Travel Time Estimation. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 39, No. 12, pp. 12390-12398).\n\nThe paper presents an interesting framework that incorporates LLM-based models and deep learning models, achieving state-of-the-art performance on various trajectory tasks. The manuscript is well-organized and clearly described. Nevertheless, integrating outdated models weakens its effectiveness and superiority. The authors should provide more technical details, ablation experiments, and deeper analyses to support their assumptions. Additionally, the authors should thoroughly review and revise the manuscript to enhance the rigor in writing and architecture diagrams. Therefore, I vote for borderline reject in this review round. I would like to raise the rating if the authors can address my concerns.", "limitations": "The authors have discussed the limitations in the supplemental material.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Paper Strengths:\n\nThe main insight of this work is the unified framework which can integrate various LLM-based or deep learning models to address various trajectory tasks. The proposed framework utilizes collaborative learning to model collaborations between LLM-based models and deep learning models, enhancing the performance of various practical applications. The developed unified running environment is interesting which can integrate diverse state-of-the-art models. \n\nPaper Weaknesses:\n\n1.\tMy first concern is the technical details of each core module and architecture diagrams. The authors introduce the proposed collaborative learning in Sec. 2.4 Collaborative Learning Schema, without sufficient technical details and the underlying motivations. Some ablation experiments should be conducted to examine the effectiveness of collaborative learning schema since this module is crucial for combining LLM-based models and deep learning models. Additionally, the architecture diagrams seem confusing. In the right part of Figure 1, the connection lines for T2 and T4 have no arrows. The dotted line in Figure 1 is confusing. In the left part of Figure 2 (UniEnv), the descriptions or definitions should be included for colored rectangles below Trajectory Prediction. The authors need to modify the figures to improve the rigor.\n\n2.\tMy second concern is the effectiveness of the proposed framework. Some of the integrated methods are outdated to effectively demonstrate the capabilities of the proposed TrajAgent. The authors should integrate more recent state-of-the-art methods to evaluate the performance improvement. For example, integrating DutyTTE [Mao et al., AAAI 2025] for travel time estimation and ControlTraj [Zhu et al., ACM SIGKDD 2024] for trajectory generation.\n\n3.\tI have some concerns on the literature review. Sec. 4 Related Work is located at the last page of the manuscript, is there any specific meaning in organizing the manuscript? The summarized papers in related work should be updated. Including more recent researches and analyses may be better. Additionally, the authors should thoroughly check the references since some of mentioned methods have been accepted by conferences or journals. For example, MetaGPT [18] has been accepted by ICLR 2024, TrajectoryNet [20] was published in Annual International Conference on Computer Science and Software Engineering, and UrbanLLM [21] has been published in Findings of the Association for Computational Linguistics: EMNLP 2024.\n\n4.\tSome concerns on experimental results. Firstly, in Table 1 of the manuscript, the authors show the performance improvements across various trajectory tasks and examine the effectivenss of core optimization methods (e.g., data augmentation). However, the performances of some operations (e.g., +DA and +PO for LLM-ZS) are missing. Secondly, in Figure 4(a-b), the authors conduct the ablation experiment on thought steps and memory size. The performance is compard in terms of Accuracy@5 in next location prediction task. With the increase of thought steps and memory size, the performance of proposed framework is significantly improved. However, with excessive thought steps (e.g., thought step = 20) and larger memory size (e.g., memory length =15), the performance decreases without any analyses. The authors need to include more deeper analyses. Thirdly, in A.5 Case Study of the supplemental material, the limitations are discussed on two specific challenging issues. Including some experiments to explore possible reasons and future work would make the manuscript more persuasive.", "was_revised": true, "final_justification": "The responses have addressed my concerns, therefore, I would like to raise my rating.", "rebuttal": "## For W1\n**Ablation experiments** During the task execution phase, our goal is to leverage agentic framework to drive small models in completing various trajectory modeling tasks that LLMs are not good at, while further optimizing the performance of small models. We compared the effects of independent execution and TrajAgent joint execution, with the experimental results presented in Table 1. In the table, \"-DL\" represents our implementation method designed with reference to other large model approaches, \"-LLM\" denotes the small model, and \"Joint\" refers to the result of large models driving small models with optimization.  MAE=100% and Acc@5=0 values in the table suggest that no effective prompt-based LLM approaches were found for these tasks.\n\n**Figure** Thank you for your suggestions regarding the figures. We apologize for omitting some arrows in Figure 1. The direction of the arrows aligns with the flow of data/information, originating from the raw data (Data module) and input into the TrajAgent framework (Framework module). The large model automatically performs data preprocessing, small model selection, and small model deployment training based on the input data type and task description, then dynamically selects the optimal optimization strategy based on the training results, ultimately producing an experimental solution applicable to various downstream tasks (Task module). The left side of the dashed line represents the traditional single-model paradigm, where a single model is designed for a specific task and is only suitable for particular datasets. The right side of the dashed line represents our solution, which constructs a unified trajectory modeling framework adaptable to multiple tasks and datasets. In Figure 2, the colored blocks represent the currently integrated models for each type of task. Since the models and datasets will be further expanded in the future, the names of each model are not labeled. \n| Task(metrics, dataset)          | Trajectory Prediction<br>(Acc@5, FSQ-NYC) | Trajectory Recovery<br>(MAE, Porto) | Trajectory Classification<br>(Acc@5, FSQ-NYC) | Trajectory Generation<br>(JSD-MEAN, FSQ) | Trajectory Representation<br>(Acc@5, FSQ-NYC) |\n|-|-|-|-|-|-|\n| remove DL model | 0.3950| 100% | 0.2034 | 0.3200| 0.00 |\n| remove LLM | 0.3422| 42.71% | 0.5755  | 0.4557| 0.3100|\n| TrajAgent| 0.4018 | 27.78% | 0.7802  | 0.3200 | 0.3472 | \n\nTable 1. Ablation experiments\n## For W2\nThank you for your suggestion. We have integrated the **DutyTTE** model you proposed. For the MoEUQ method, the optimization results are shown in Table 2.  TrajAgent autonomously selects key parameters and configurations. \n\n| Model | GraphMM (Acc@5) | LLM-ZS (Acc@5) | DutyTTE (MAE) |\n|-|-|-|-|\n| Dataset | Tencent | FSQ-TKY | Tencent |\n| Origin | 0.2014 | 0.3110 | 190.82 |\n| +DA | 0.3258 | -  | - |\n| +PO | 0.2427 | 0.3302| 179.01 |\n| +Prompt | - | 0.3225 | - |\n| Joint | 0.3422 | 0.3350 | 179.01 |\n\nTable 2. Expanding models and optimization methods \n## For W3\nThank you for your reminder and suggestions. We apologize for not updating the information of the cited articles in a timely manner, which was an oversight during the resubmission process. We have updated this information in the latest version of the paper, along with the recent work mentioned in the \"tiny weakness\" section.  \n\n## For W4\nThank you for your question! \n- **Extended Models**\n    For the LLM-based method (e.g., LLM-ZS), since it uses a pre-trained model and the trajectory data does not participate in training, no data augmentation (DA) method is provided. We allowed TrajAgent to automatically optimize the key parameters mentioned in the original paper, and the results are shown in Table 2. For other small models using map/GPS trajectory data(e.g., GraphMM), we adopt the rule-based data augmentation approach from DeepMM, which first down-samples the raw data, then designs various noise-injection rules, and finally combines the processed data with the original dataset to achieve data augmentation. We integrated it into the TrajAgent framework, using agent to choose noise-injection strategies. The experimental results are shown in Table 2 (base_model=llama3-70b, step=5).  \n\n- **The optimization effect diminishes as the step increases**\n\n    While selecting combinations of operators, the model further optimizes the configuration of each operator (e.g., the original configuration file for \"inserter\" is {insert_nums: 1, insert_ratio: 0, insert_time_sort: maximum, percent_no_augment: 0, ti_insert_n_times: 1}). In a zero-shot scenario, the model explores optimization strategies based on dataset characteristics and the meaning of the operators, with a probability of converging to a local optimum—i.e., finding a suboptimal combination and deeming the result sufficient, thus stopping the exploration of new combinations and selecting the best operator configuration based on this combination. We compared the llama3-70b used in the paper with other reasoning models, and the results are shown in Table 3 (S2TUL, FSK-London, memory-length=1). We found that models with stronger reasoning capabilities attempt more operator combinations and have a higher probability of finding better combinations in fewer steps. For instance, DeepSeek-v3 outperforms LLaMA-3-70B in step 4. Under the same operator combination and the same number of steps, models with stronger reasoning capabilities yield better optimization results. For instance, DeepSeek-v3 outperforms LLaMA-3-70B in steps 3, 5, 6, 7, 9, 10, 11, and 13.\n\n    - **Improvement solution**\n\n    Implement reflection similar to contrastive learning between steps, such as further adjusting the parameters of each operator based on effective combinations, to avoid exploring ineffective combinations as much as possible. The improved results are shown in Table 4 (S2TUL, FSK-London, memory-length=1).  \n\n- **The optimization effect diminishes as the memory_length increases** \n    \n    Memory_length refers to the number of action proposals generated in each reasoning step. Increasing it can improve exploration efficiency but also raises the probability of falling into a local optimum. We compared the performance of llama3-70b with that of reasoning models, and the results are shown in Table 5 (S2TUL, FSK-London, step=5). Both models achieved relatively good results at memory_length=2, but as the length increased, they faced the issue of converging to suboptimal solutions. However, models with stronger reasoning capabilities exhibited a stronger tendency to explore other combinations, thus having a higher probability of finding better combinations and escaping the \"optimization trap\" . \n\n    - **Improvement solution**\n    \n     Periodically optimize and update the memory organization, discard poor combinations, retain good ones, and guide the model to explore new combinations during the reflection phase. The improved results are shown in Table 6 (S2TUL, FSK-London, step=5).  \n| base-model| step | 0| 1 | 2 | 3 | 4 | 5| 6| 7| 8 | 9| 10| 11| 12| 13| 14| 15| 16| 17|\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n| llama3-70b | ACC@5| 56.28 | 56.28 | 59.10 | 68.18 | 68.18 | 68.39 | 68.18 | 69.04 | 69.04 | 68.39 | 69.04 | 69.04 | 87.01 | 69.05 | 68.40 | 68.19 | 83.12 | 88.96 |\n| deepseek-v3 | ACC@5| 56.28 | 79.22 | 59.52 | 61.26 | 81.17 | 68.83 | 69.26 | 69.05 | 83.98 | 70.35 | 69.05 | 69.26 | 71.00 | 61.90 | 60.82 | 68.83 | 71.00 | 66.88 |\n| gemini-2.0-flash-001  | ACC@5| 56.28 | 73.16 | 61.69 | 70.35 | 68.83 | 82.25 | 69.05 | 68.40 | 87.01 | 65.58 | 79.87 | 83.12 | 71.65 | 84.63 | 82.47 | 71.43 | 69.05 | 69.05 |\nTable 3. Comparative analysis of TrajAgent performance across optimization steps\n\n| Base-Model  | Version | Metric | Step 0 | Step 1 | Step 2 | Step 3 | Step 4 | Step 5 | Step 6 | Step 7 | Step 8 | Step 9 | Step 10 | Step 11 |\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n| llama3-70b   | Pre-improvement  | ACC@5  | 56.28  | 56.28  | 59.10  | 68.18  | 68.18  | 68.39  | 68.18  | 69.04  | 69.04  | 68.39  | 69.04 | 69.04|\n| | Post-improvement  | ACC@5  | 56.28  | 59.09  | 61.03  | 68.18  | 69.04  | 69.04  | 60.38  | 69.04  | 82.90  | 80.08  | 87.01 | 83.12|\nTable 4. Improvement\n\n| Base-Model   | Metric   | Memory Length 0 | Memory Length 1 | Memory Length 2 | Memory Length 3 | Memory Length 4 | Memory Length 5 | Memory Length 6 | Memory Length 7 |\n|-|-|-|-|-|-|-|-|-|-|\n| llama3-70b   | ACC@5| 56.28 | 68.39 | 80.95| 79.00| 55.19| 83.33 | 57.79 | 62.55 |\n| deepseek-v3  | ACC@5| 56.28 | 79.22| 59.09| 79.65| 61.47 | 80.74| 59.52 | 81.60 |\nTable 5. Comparative analysis of TrajAgent performance across memory lengths\n\n| Base-Model   | Version | Metric   | Memory Length 0 | Memory Length 1 | Memory Length 2 | Memory Length 3 | Memory Length 4 | Memory Length 5 | Memory Length 6 | Memory Length 7 |\n|-|-|-|-|-|-|-|-|-|-|-|\n| llama3-70b   | Pre-improvement  | ACC@5 | 56.28| 68.39| 80.95  | 79.00 | 55.19 | 83.33 | 57.79| 62.55 |\n| | Post-improvement  | ACC@5| 56.28 | 68.39 | 69.05| 80.95| 81.60 | 84.63 | 80.52 | 85.73 |\nTable 6. Improvement"}]

Author rebuttal:

Meta-review: This paper introduces TrajAgent, a novel framework that leverages large language model (LLM)-based agents to automate trajectory modeling tasks through collaboration between large and small models. The key scientific contributions include: (1) UniEnv, a unified execution environment that standardizes interfaces across diverse trajectory datasets, models, and tasks; (2) an agentic workflow that decomposes trajectory modeling into interpretable steps managed by LLM agents; and (3) a collaborative learning schema enabling optimization between reasoning-driven LLM agents and training-driven specialized models. The framework demonstrates performance improvements of 2.38%-34.96% across 18 models and 5 trajectory modeling tasks on four real-world datasets.

The main strengths of the paper are as follows: (1) The integration of LLM agents with trajectory modeling represents a novel approach to addressing data and task heterogeneity in this domain. The framework's ability to automatically optimize diverse specialized models across different trajectory tasks is innovative compared to existing task-specific solutions. (2) The paper combines three well-integrated components (UniEnv, agentic workflow, collaborative learning) into a cohesive framework that handles multiple trajectory modeling architectures and data types with minimal human intervention. (3) The evaluation spans 4 datasets, 18 models, and 9 subtasks, demonstrating consistent performance improvements. The inclusion of ablation studies examining memory/reflection mechanisms and LLM comparisons (GPT-4o-mini, Qwen, LLaMA) adds experimental rigor. 

The weaknesses of the paper are as follows: (1) The submission lacked critical implementation details, particularly regarding the collaborative learning module, LLM-generated data augmentation procedures, and reasoning mechanisms behind agent decision-making.
(2) The original submission omitted crucial information about computational overhead, token consumption, and scalability considerations for real-world deployment. (3) The LLM agent planning and decision mechanisms rely primarily on heuristic approaches without deep theoretical guarantees or comprehensive analysis of failure modes. (4) Some integrated baseline models were outdated, potentially weakening the demonstration of the framework's superiority against state-of-the-art methods.

The reason for the decision: (1) the paper's solid technical merit has been demonstrated through comprehensive experimental validation across multiple datasets and models. (2) While the core algorithmic contributions represent moderate novelty through effective integration of existing techniques rather than fundamental theoretical advances, the unified framework's consistent performance improvements (2.38%-34.96%) across diverse trajectory tasks, combined with the comprehensive system design and clear presentation, justify acceptance. (3) The collaborative review process resulted in unanimous reviewer support (all rating 4) after initial concerns about technical details, computational costs, and theoretical foundations; instead of key experiments were adequately addressed. However, the weakness of the LLM agent planning and decision mechanisms relies primarily on heuristic approaches without deep theoretical guarantees or a comprehensive analysis of failure modes could be the reason for rejection. As a top-tier conference, I agree such a work to present to readers about an available way to use LLMs for an unexplored real-world application.


The rebuttal discussions: (1) Reviewer KM23 raised concerns about optimization process details, computational costs, and whether LLMs truly understand memory logs beyond pattern matching. The authors provided comprehensive explanations of data augmentation operators, joint optimization strategies, and conducted ablation studies demonstrating genuine learning rather than mere imitation. The zero-shot experimental evidence and memory scoring experiments addressed these concerns. (2) Reviewer du8A identified issues with unclear notation, missing agent descriptions, and vague explanations of learning mechanisms. Authors clarified the multi-agent architecture (6 agents total), provided detailed reasoning mechanisms, and explained the modular design rationale for efficiency and accuracy. (3) Reviewer MyNr questioned decision-making mechanisms, generalization capability, and cost scalability. Authors responded with detailed technical innovations, extensibility explanations, performance contribution analysis, and comprehensive token consumption statistics. (4) Reviewer PauW requested technical details, integration of recent methods, and deeper analysis of optimization limitations. Authors integrated DutyTTE as requested, provided extensive ablation studies on optimization step effects and memory organization, and offered improvement solutions for identified limitations.
Decision: Accept (poster)