USTBench: Benchmarking and Dissecting Spatiotemporal Reasoning Capabilities of LLMs as Urban Agents
Authors: Siqi Lai
Yansong Ning
Zirui Yuan
Zhixi Chen
Hao Liu
Venue: ICLR 2026 Poster (2026)
Keywords: large language model
spatiotemporal reasoning
urban science
Primary area: datasets and benchmarks
TL;DR: A benchmark for evaluating the urban spatiotemporal reasoning abilities of LLMs.

Abstract: Large language models (LLMs) have shown emerging potential in spatiotemporal reasoning, making them promising candidates for building urban agents that support diverse urban downstream applications. Despite these benefits, existing studies primarily focus on evaluating urban LLM agent on outcome-level metrics (e.g., prediction accuracy, traffic efficiency), offering limited insight into their underlying reasoning processes. As a result, the strengths and limitations of urban LLM agents in spatiotemporal reasoning remain poorly understood. To this end, we introduce USTBench, the first benchmark to evaluate LLMs’ spatiotemporal reasoning abilities as urban agents across four decomposed dimensions: spatiotemporal understanding, forecasting, planning, and reflection. Specifically, USTBench supports five diverse urban decision-making and four spatiotemporal prediction tasks, all running within our constructed interactive city environment UAgentEnv. The benchmark includes 62,466 structured QA pairs for process-based evaluation and standardized end-to-end task assessments, enabling fine-grained diagnostics and broad task-level comparison across diverse urban scenarios. Through extensive evaluation of fourteen leading LLMs, we reveal that although LLMs show promising potential across various urban downstream tasks, they still struggle in long-horizon planning and reflective adaptation in dynamic urban contexts. Notably, recent advanced reasoning models (e.g., DeepSeek-R1) trained on general logic or mathematical problems do not consistently outperform non-reasoning LLMs. This discrepancy highlights the need for domain-specialized adaptation methods to enhance urban spatiotemporal reasoning. Overall, USTBench provides a foundation to build more adaptive and effective LLM-based urban agents and broad smart city applications. Our project is available at https://github.com/usail-hkust/USTBench.

Published as a conference paper at ICLR 2026
USTBENCH: BENCHMARKING AND DISSECTING SPA- TIOTEMPORAL REASONING CAPABILITIES OF LLMs AS URBAN AGENTS
Siqi Lai, Yansong Ning, Zirui Yuan, Zhixi Chen, Hao Liu" The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China falail25, yning092 zyuan779 acconnnett..cc..ctt............. www.utugttt. -aast-g2.. edu. en zh whixichen@hkust-teee edu. cn, liuh@ust.hr
ABSTRACT
Large language models (LLMs) have shown emerging potential in spatiotempo- ral reasoning, making them promising candidates for building urban agents that support diverse downstream applications. Despite these benefits, existing studies primarily focus on evaluating urban L.MM agent on outcome-level metrics (e.g., prediction accuracy, traffic efficiency), offering limited insight into their underlying reasoning processes. To this end, we introduce USTBench, the first benchmark evaluate LLMs' spatiotemporal reasoning abilities as urban agents across four de- composed dimensions: spatiotemporal understanding, foreadddinnn.oocttinng planning, and reflection. Specifically, USTBench includes five urban decision- and four spatiotemporal prediction tasks, all running within interactive city environment UAgentEnv. The benchmark includes 62,466 structured QA pairs for process- based evaluation and standardized end-to-end task assessments, enabling fine-grained di- agnostics and broad task-level comparison across diverse urban scenarios. Through extensive evaluation of fourteen leading LLMs, we reveal that although LLMs show promising potential across various urban downstream tasks, they still strug- gle in long-horizon planning and reflective adaptation dynamic urban contexts. Notably, recent advanced reasoning models (e.g., DeepSeek-RI -R) trained on general agent tasks do not consistently outperform non-reasoning LLMs. This discrepancy highlights the need for dpeaai--pe adaptation enhance urban spatiotem- poral reasoning. Overall, USTBench provides a foundation to build more adaptive and effective L.Le-based urban agents and broad smart city applications. Our project is available at hhttps: http::///gtthlle www.cooo//csseee wwtssal-hkusse www.hhtttuuueenenn
INTRODUCTION
Urban systems are inherently complex and dynamic, characterized by continuous fluctuations across space and time. By learning from large-scale spatiotemporal data, traditional data-driven methods have achieved progress in prediction and decision support (Bibri & Krogstie 2017 Ullah et al.) 2020
Wei et al., 2019 Tian et al. 2025 Feng et al.)) However, they often fall short in generalizing to unseen scenarios and providing transparent reasoning for reliable decision-makinn makinn (Li et al. 2024c] Lai et 2025). Recently, the large language models (LL.Ms) (e.g., ol (Taech et al. 2024) and DeepSeek-R
(Guo et al. 2025) have emerged as intelligent urban agents (Yuan et al. [202 Li et al. [2024c] Zhou al.] 2024 Ning al.) 22025) due to their growing reasoning ability to integrate diverse information, adapt across tasks, and offer detailed interpretation through natural language. To fully leverage their potential, it is essential to systematically evaluate LLMs' spatiotemporal reasoning abilities: the capacity to infer spatiotemporal dynamics and interact with evolving urban environments. Such evaluation is key to understanding their readiness for weal-world urban challenges.
In recent literature, many efforts have been made to evaluate the urban spatiotemporal reasoning ability of L.MSs. However, summarized in Table [ they have two limitations: (1) Reliance outcome- based metrics: solving urban tasks requires multi-step reasoning, yet existing studies (Feng et al.) 2025a b) only assess outcome-based metrics (e.g., prediction accuracy, traffic efficiency), overlooking
"Corresponding author.

---

Published as a conference paper at ICLR 2026
Table Comparison of LLM benchmarks in urban tasks.
City Bench
CityGPT
UrhunPlanBecch
USTBench (Ours)
Exaluations
Li
2024b
Fong al.
2025b
Peng et al.
2025a
Zheng et
2025
Spatiotemporal Understanding
Reasoning
Forecasting
Abilities
Planning
x
Reflection
Bascline
Non- Reasoning LLM
LLMs
Reasoning LLM
X
X
Eva
Outcome- Based Metries
Metrics
Process-aase Metries
X
X
Previous Boalustion
Process-bae Process-based based Evalustion (Ours]
Spatiali Distance
Spatial:
Ouprome aood
Tursora hard Uncercandin
Corpeation
1.0 Adjacency
Reflection
Spatial: Connectivity
Planning
Congestion
DocpSock
DeopSeekkit deficit understandngg forecasting higherr better)
0.2
Prediction
underperforms Liama3 (The
Forecasting
Temporal:
Duration
Road Congestion Levels [jo, 117.00,, 14, 17.05, (4, 12.101, (s, 117:55 J4, '12:200) 117-5) (4, (1733)), (1. 117391, 1% 117000 (1, 117-55), 17:501, 17-591)
Congattoo Trand: The swaal trend decreasing
Temporal: nd
Temporal: Local Extrema
Lamm33.3: 3: Besed on the current brend decrensing congest on...
DeepSewi-:t:: The med vchibits riing compattion trund... X
Periodicity
Chronology
Prediction:
Uamaã &: The not congestion lovel at 18.001s V
DeepSeet- Rt: The next congestion level at 18100 a. X
Non-Reasoning
Qwer2: 5-328
QwG-326
Reasoning LLMs
Lamm3333-78
DeapSeak-R1
Figure I: The comparison of outcome-based and process-based evaluations.
Figure 2: The performance of leading LLMs in urban spatiotemporal reasoning.
intermediate reasoning steps. Their evaluations may mask critical reasoning deficits. For instance, the reasoning LLM, DeepSeek-R1, typically surpasses non-reasoning models. However, shown in Figure D it slightly underperforms Llama3.3 in outcome-based metrics of congestion prediction. Further reasoning process-based analysis reveals that this limitation stems from weaknesses in temporal trend understanding and forecasting. Without fine grained evaluations, such discrepancies remain unexplained. (2) Overlooking reflection reasoning: unlike static tasks, urban systems provide real-time and context-rich feedback (e.g., shifting traffic patterns), making reflection over past actions essential for agents to adapt to evolving dynamics (Wang et al. 2024c). Yet, existing evaluations fail to diagnose whether LLMs can transform this external feedback into causal links between prior actions and observed outcomes, and subsequently improve or adjust reasoning over time. These limitations prevent a comprehensive assessment of LLMs urban agents.
To this end, we introduce USTBench, the first benchmark designed systematically evaluate LLMs' spatiotemporal reasoning abilities urban agents. We first build UAgentEnv, interactive city environment spanning five typical decision decision-making and four prediction tasks. It enables agents to perceive, interact with, and respond to dynamic urban contexts. To move beyond outcome-based evaluation, we decompose urban spatiotemporal reasoning into four key processes: spatiotemporal understanding, forecasting, planning, and reflection. Each ability is assessed via question- -anwerinn (QA) pairs collected from UAgentEnv. To further explore the interplay between these reasoning processes, conduct targeted ablation studies, providing diagnostic insights into model strengths and weaknesses. Combined with end-to-end performance evaluations, this dual-level framework supports both detailed reasoning analysis and standardized downstream task evaluation.
Using USTBench, we evaluate fourteen leading LLMs, covering both non-reasoning and reasoning models. Our key contributions and findings include: (1) We construct USTBench, the first benchmark explicitly designed to evaluate the spatiotemporal reasoning capabilities of L.LMs as intelligent urban agents. It combines both process-based reasoning evaluation and standardized end-to-end task performance assessment, including 62,466 structured reasoning QA pairs and nine real-world downstream urban tasks. (2) To support this benchmark, we develop UAgentEnv, an interactive urban environment that enables nuanced benchmark dataset collection and uniform downstream task evaluation across diverse urban scenarios and tasks. (3) Despite the promising capabilities of L.Mss, extensive experiments also reveal several notable limitations: As illustrated in Figure
2

---

Published as a conference paper at ICLR 2026
LLMs excel in spatiotemporal understanding and forecasting. but typically struggles in long-term planning and reflection; Reasoning models trained on general logical or mathematical tasks do not consistently outperform non- -en-rssoning LLMs, underscoring the need for domain adaptation; Planning is a higher-order ability that LLMs struggle with, which builds upon understanding and forecasting; Reflective learning, which enables adaptation through feedback in dynamic urban environments, remains a significant weakness, even among the leading reasoning models.
PRELIMINARY
Problem 1. Urban Prediction Tasks: Given historical spatiotemporal observations o, of an urban environment, the goal of prediction is to anticipate urban future states ( 8it over a horizon A, where each state & captures key indicators across space and time.
Problem 2. Urban Decision-Making Tasks: Given an urban environment and a task I, an agent oper- ates based on observations with a policy x (o) that determines a sequence of actions {ao, Q1.... .... an to optimize the environment. The objective is to accomplish a target goal specified by t.
Definition 1. Urban LLM Agents: An urban LLM agent is a large language model-driven au- tonomous system designed to operate in dynamic urban environments. Formally, we define the urban enviromment E (S, A, O, T"), where S states the urban state space, A denotes the agent's action space, O is the observation space, and T S x A - S is the environment's is transition function. At each time step i, given a task t. the agent receives the current observation o, E O (e.g., local traffic conditions), along with history of prior observations 0:-1 and actions a,-1. Based on this context, the agent performs reasoning to either (1) execute an action a, E A (e.g., activate traffic signal), or (2) generate predictions of future urban states 15it1,-..,,i+tb. C S (e.g., estimated traffic volume).
Definition 2. Urban Spatiotemporal Reasoning: Urban spatiotemporal reasoning is the agent's ability to interpret, act upon, and adapt to urban environments characterized by spatial and temporal dynamics. Formally, given a task & and spatiotemporal observation o₁, it involves: (I) Spatiotem- poral Understanding (Shi et al. 2022 Wang & Zhao 2024]: Interpreting urban spatial structures (e.g.. road network) and temporal patterns (e.g.. traffic flow shifts). (2) Forecasting (Wang et al. based on learned spatiotemporal patterns. (3) Planning (Wang et al., 2024b): Reasoning to derive control actions a; that optimize task objectives within the current and anticipated urban context. (4) Reflection (Ji et al., 2023): Evaluating the outcomes of decisions or failures of predictions via external feedback f; and updating future reasoning accordingly improve performance over time.
2024c): Reasoning to generate predictions of future urban states {si+1..... .......... si-a)
3
UAGENTENV: INTERACTIVE CITY ENVIRONMENT
3.1 URBAN TASK SCITE AND DATA SOURCES
Urban Tasks: UAgentEny supports nine real-world urban tasks across prediction and decision- making. The prediction tasks include next POI prediction (Zhao et al., 2020). congestion forecasting (Cheng et al., 2018). socio-economic indicator prediction (Liu et al. 2023). and traffic origin- destination (OD) prediction (Yuan et al.] 2024). The decision-making tasks cover traffic signal control (Wei et al.) 2019). POI placement (von Wahl et al. (002)). route planning (Li et al. 2021). road planning Zheng et al. 2023b), and urban planning (Zheng et al. 2023a). Details are in Appendix B.I
Urban Data Collection: To ensure UAgentEnv comprehensively reflects urban dynamics and complexity, we integrate diverse publicly available real-world data across five dimensions: (I) Geospatial data: Geospatial information from OpenStreetMap (OSM) (OpenStreettMap contributors 2023), including points of interest (POIs), areas of interest (AOIs), and road networks across multiple regions worldwide. (2) Truffic: Historical traffic flow data from several urban areas (Cheng et al.
2018 /Zhng et al., 2023) to simulate realistic traffic conditions. (3) Socio-economy: Global GDP and
population data from 2000 to 2019 (Wang& & Sun 2022 WorldPop 2025) to model urban evolution. (4) Human mobility: City-wide trajectory data (New York City Taxi and Limousine Commission
2025) capturing fine-grained movements of individuals within urban regions. (5) POI ins:
Check-in records from the FourSquare (Wang et al. 2023) reflecting human mobility across POIs.
3.2
UNIFIED URBAN AGENT FRAMEWORK
We introduce unified framework for urban tasks, where the urban agent interacts with the environ- ment across nine real-world tasks. To ensure consistency, we standardize task instructions, inputs, outputs, and execution flows using simplistic instructions to showcase LLM basic reasoning abilities.
3

---

Published as a conference paper at ICLR 2026
Geospatial Data
Prediction Tasks
Decision- making Tasks
Experience
Destription Urban Observation
Traffic Dynamics
Prediction
Pacemmnn
Feedback
4. Reflection
Socio-ecanomic Indicators
Urban Agant
Conpertion Prodution
Newt FCI Prediction
Spatiotemporal
Human Mobility
Road Plamingg
Understanding
Prediction Output
Forecasting
Prediction
POI Check-ins
Traffic Signal
costrol
Boste Planring
Urian Ponning
Planning
Dedisionmmkinn
Urban Data
UAgentEnv
Action
Outpurt
Figure 3: The workflow of UAgentEnv environment.
As shown in Figure 3 the framework follows structured pipeline: (1) Each task provides the
agent with a description, data schema, and relevant domain knowledge. (2) The real-time urban spatiotemporal dynamics are delivered to the agent contextual observations. (3) The LLM agent reasons to solve the task through a modular reasoning workflow comprising spatiotemporal understanding, forecasting, and planning. Then, an action or prediction generated, aligning with the task objective and relevant past experiences retrieved from memory. (4) After receiving environmental feedback on the previous output, the agent engages in reflection to evaluate its performance and diagnose errors. Then, an informative experience will be generated and stored in the memory to guide future reasoning processes. The prompt templates are detailed in Appendix B.3
USTBENCH: URBAN SPATIOTEMPORAL REASONING BENCHMARK
4.1 PROCESS-BASED SPATIOTEMPORAL REASONING EVALUATION
USTBench provides a fine-grained assessment by decomposing urban spatiotemporal reasoning into four key processes, which represent a loop of enent-environment interaction: (1) understanding
the spatiotemporal dynamics (Shi et al., 2022 Cao et al., 2024). (2) forecasting the future states
of the environment (Wang al.) 2024c). (3) planning actions based on anticipated changes(Chang al. 2024). and (4) reflecting on environmental feedback to improve subsequent reasoning (Rene
Guven 2024). These abilities are foundational skills widely recognized (Ni et al. 2024: Kim et al.
2025) for LLM agents to interact with dynamic environments effectively.
Existing benchmarks primarily focus fundamental spatiotemporal understanding or outcome- based metrics. In contrast, USTBench offers process-based diagnostics that cover the full under- sland-forecast-pla--eee plef--eeleeetttttttee-e--------e loop, enabling systematic analysis of where reasoning succeeds fails short. The benchmark consists of 62,466 structured QA pairs spanning nine representative urban tasks, including 40% for basic spatiotemporal understanding, and 60% for higher-level reasoning in real-world task-solving. These reasoning abilities are evaluated with accuracy, delivering a coher- ent, interpretable, and empirically grounded framework for assessing LLM agents' spatiotemporal reasoning. Statistics and examples are shown in Table 2 with details provided in Appendix D.7.3
4.1.1 OBSERVATION CONSTRUCTION
To systematically evaluate LLMs' reasoning capabilities across diverse urban environments, we collect variety of task scenarios as LLM observations using UAgentEnv. Each observation encodes a snapshot of geospatial structures (e.g.. road networks) and temporal dynamics (e.g., trajectories or time-series) of the city. These serve as contextual input for LLMSs to perform reasoning over specific urban questions. We verbalize spatial observations as sparse adjacency matrices with node and edge attributes (Chen al. 2024). Temporal observations
verbalized with attribute values (e.g., visited POI or traffic volume) across discrete time intervals (Wang et al. 2023 Li et al. 2024c). Examples of observations are shown in Table 2 and detailed prompts are in Appendix D.7.3
To collect QAs from prediction tasks, we construct observations using a sliding window of length A. At each step i, extract real-world historical spatiotemporal dynamics and encode input observations in QA queries. For QAs constructed from decision- decisio tasks, we collect observations using heuristic agent that follows semi-stochastic policy to interact with the environment for
4

---

Published as a conference paper at ICLR 2026
Table 2: The required reasoning abilities of LLM agents and the statistics of our evaluation dataset.
Reasoning
Category
Observation Example
Question Example
Distince
Road longth: wodll 126m, rond2: 345m
Rank noads by their distanoes.
Adjaeeccy
Adjacency: [(ragion], region2, 341m), freplont, regions, 125m)...)
Rank the regions by proximity region [s there a path between roads and S?
Spatiotemporal
Connectivity
Connectivity: [(coad], road2, 121m). (road4, road5, 156m)...
Underttading
Daratio
Travel times; I: 34mia, roulc 2: 12min....
Kank the routes by their travel limes.
(27,000 QAs) Chronology
Trajectory: [(Shop, 19:34), (Bar. (1...)....Sppp. 20:21), (Bar, 20:1)...]]
Order oflen visited POI pairs.
Trend
Cregestion levels: [(2. 8:00) (3. 800)) (:10,..
What the tredd of the congestion?
Local Extrema
Congestion levels: [(4, 10:00).... 10:00)...444, 10:00)...
Identify high-peali hours
last days.
Periodicity
Congestion levels: [64, 9:00)... (2. (..00),..,,,2113000..
Identify the period of the waffic Blo pattern.
Next POI
Frajectory: [[Sopp, 19:34, 0m)) (Bar,
134m)_]
Product next [P] the would go.
Forecasting
Congestion
Coegestion lovels: [(2. 8:00) (3, 8:05)....(4. (4588......:5
Prodictt the congestion at 9:00
(15,336 QAx) Soclo-sconomie
Region GDP: [663M, 2009), (68M, 2010), (71M, 2011)...]
Predict the GDP 2012.
Traffic-DD
Departure vehicles: [[5. 8:00), (10, 8:0 8:55)]
How many vehacles will depart 9:00.
Signal Control
Queues: lanel: 3, lone2:
Connectivity: [Clane lanes, 500m)....
Which signal should he activared?
POI Placement
Demand: locl; 12, loc2: 23.... Distance: loct; 1021m. loch: 2033m...
Where should we placed a new station?
Phanning
(15,000 QA4)
Roule Planning
Congestion level: roadl: moad:: 2...: Connectivity: [froad], road2, 126m)...
Which road should enler next?
Road Planning
Connestivity: [(road), region2, 134m), (roa3,, region3, 234m)...]
Which road should he built next?
Usban Planning
Adjacency: [(blankl, residentiall 143m), (blankl, residential,, 315m)....
Where should we plan new hospital?
Kellection
Ground truth: (1, 10:00); Your prodiction: (1, 10:00)
Your previous prodiction wrong, let's
Retlection QAs)
Forecasting
review the spaniotemporal data and ty again.
Rellection
The facility accessibility decressed by 12%.
Verily previous decision
Planning
and its rensoning are accunte..
varying urban contexts with diverse spatiotemporal dynamics:
(o( arg max ((),a), aEA random(A) (A),
with probability -
with probability
where Q(o, a) is a simple utility function that scores the benefit of taking action a in state a (e.g.. prioritizing the green signal for lanes with the longest queues). To ensure diversity of collected scenarios, we introduce an exploration coefficient E [0, 1]. which controls the probability of selecting random action. This induces diverse decision trajectories resulting in various decision- making scenarios. At each decision time step i, the observed spatiotemporal dynamics are captured and embedded into corresponding QA instance agent observations.
4.1.2 SPATIOTEMPORAL UNDERSTANDING QA
Spatiotemporal understanding forms the first stage of problem-solving and underpins all interactions with urban environments. It requires LLMs to interpret city-scale spatial relations and identify dynamic temporal patterns within noisy urban environments. To evaluate this ability, we design QA tasks using contextual observations from Section 4.1.1 where each instance challenges the LLM to infer spatial structures or temporal patterns from the provided context. Our benchmark integrates eight widely recognized types of spatiotemporal patterns, following established definitions from prior works, enabling systematic assessment of whether LLMs can correctly extract these features.
Spatial Understanding extends from local neighborhood to long-range city-wide dependencies. (1) Distance reasoning captures the proximity between locations (Rizvi et al. 2024). (2) Adjacency rea- soning extends assessing whether multiple nearby locations form coherent spatial neighborhood
(Shi et al. 2022). ). (3) Connectivity requires multi-step inference across long-range urban topological
relations find whether viable path exists between spatial entities (Zheng et al. 2023b a)
Temporal Understanding progresses from localized analysis to long- inference, (1) Duration reasoning focuses comparing the lengths of individual events (Cao et al. (2024). (2) Local Extrema detection extends to identifying peak or off-peak periods within short temporal windows (Wang & (3) Chronology reasoning then infers the temporal order and dependencies among multiple events over broader timescales (Xiong et al. 2024). (4) Periodicity reasoning integrates long-term observations to uncover recurring seasonal cycles in urban dynamics (Wang & Zhao
Zhao 2024).
2024). (5) Trend analysis identifies persistent directional changes over extended horizons, capturing long-term growth or decline patterns (Liu et al. 2025a).
4.1.3 FORECASTING QA
Building up spatiotemporal understanding, forecasting (Wang et al. 2024c Li et al. 2024c| Tian et al., 2026) allows agents to predict future urban states. This is essential capability not only for prediction but also for enhancing action-outcome modeling in downstream planning (Fu et al.) 2025). To evaluate forecasting standalone ability, we construct QAs from the prediction tasks in
5

---

Published as a conference paper at ICLR 2026
UAgentEnv. Based on the historical observations os, the agent predicts the future urban state the next timestep 8141. The ground truth is derived from the actual observed value in the real-world data.
4.1.4 PLANNING QA
Planning (Wang et al. 2024b Chang et al., 20244) reflects the agent's ability to reason over spatiotem- poral observations and choose actions that optimize long-term urban objectives. Unlike solving static problems (e.g. mathematics, web search), planning in urban tasks requires agents to consider benefits over an extended horizon within the complex and evolving environments. To assess this ability, we construct QA instances from five urban decision-mann making tasks. Given the current spatiotemporal observation o,, the agent is tasked to select an action ag E A that best advances the task objective (e.g. reducing traffic congestion). Because real-world urban systems rarely reveal optimal actions due stochasticity and delayed feedhack, we compute the ground-truth answer through simulation-driven exhaustive search. Specifically, we evaluate all possible future action sequences over a planning horizon H and select the action of that maximizes the expected cumulative reward:
H
a = arg max A,EA
max
E
Σ
7' R(as+j)) ai
where R(a44)) is the reward when action O++y is taken. Here, the reward is the measure of progress toward the specific objective of the task (e.g., the reduction in queue length in traffic signal control). The discount factor E [0, 1] balances immediate and future rewards. The expectation is estimated via multiple rollouts account for stochasticity and ensure the reliability of the ground truth action.
4.1.5 REFLECTION QA
Unlike static problem-solving (e.g., mathematics), urban systems continuously generate time-indexed feedback (e.g. traffic states future timesteps) that reveals the consequences of prior decisions. Solving urban tasks requires not only executing accurate actions at each step, but also diagnosing and correcting past potential mistakes with feedback signals, then adapting strategies the evolving
urban environment (Ji et al. 2023 Kim et al. 2025).
To evaluate reflection reasoning, we construct QA instances in which the agent is provided with a prior action or prediction (e.g.. 0;-1 or s;-1), the current observation o, and the resulting environmental feedback St. The agent tasked with assessing prior output and determining whether the previous decision or prediction was appropriate. If not, it needs to revise its earlier outputs accordingly. Performance measured by the accuracy of corrections. This setting explicitly tests whether the agent transform external feedback into causal links between its past actions and observed outcomes, then leverage this understanding to improve and adjust strategies in subsequent steps.
4.2 END-TO-END DOWNSTREAM TASK EVALUATION
To directly evaluate LLMs' performance in real-world applications, conduct end-to-end assess- ments across nine urban tasks with UAgentEnv. Equipped with our urban agent framework, each task is uniformly evaluated using domain-specific metrics. For example, we use Mean Absolute Percentage Error (MAPE) to assess GDP forecasting accuracy over three-year window in socio- economic prediction. Congestion prediction, which classifies congestion into five levels to 4), is evaluated using accuracy and MAPE. Urban planning performance is assessed based on two criteria: accessibility to service facilities and ecological coverage. For road planning, we measure construction costs and the average travel distance to neighboring regions. The tasks mentioned above are assessed in 5.3
DppendixDddetaik. the evaluation metrics and results the remaining five tasks.
5
EXPERIMENT
5.1
BASELINE MODELS
We evaluate both non-reasoning and reasoning models with the same parameter sizes and architectures (e.g., Qwen2.5-333 5-32B vs. QwQ-32B) to isolate the effectiveness of the improvement in reasoning abilities. For non-rnnsoning LLMs, include: Qwen2.5 (7B and 32B) (Yang et al. 2024h, GLM4 (9B and 32B) GLM et al. (2024), Llama3.3-700 Grattatiori et al. (222)) and GPT-4o (Hurst et al.
2024h. For reasoning models, we evaluate: SeepSeek-RI-Distte (7B and 70B), DeepSeek-R (Guo

---

Published as a conference paper at ICLR 2026
Table 3: Performances on spatiotemporal understanding.
Spatial Understanding
Temporal Understanding
Model
Distance Adjacency Connectivity Overall Duration Local Extrema Chronology Periodicity Trend Overall
Overall
Random
0,2500
0.2500
0.2500
0,2500
0.2500
0,2500
0.2500
0.2500
0. 101 0.2200
0.2344
Non-
Reasoning LLMs
Qwen2.5-7B 7B
0,5080
0.4070
0.2513
0.3888
0.5902
0.5767
0.4710
0.6637
0,1740
0.4951
0.4552
GLM4-9B
0.5389
0.4400
0,2993
0.4261
0.6522
0,6362
0.5354
0.6438
0.1611 0.5257
0,4883
Qwen2.5.33
0.8046
0.5623
0.4537
0.6068
0.6610
0.1613 0.6317
0.6224
GLB
0.7541
0,6618
0.4555
0.6238
0.7662
0.7513
0.6414
0.7248
0.1742
06
0.6162
0,7448
0.5978
0.5113
0.6180
0,8148
0.7540
0.6630
0.7163
0,3203
0.6537
0,6403
GPT-4o
0.9295
0.6963
0.6787
0.7681
0.9288
0,8260
0.7310
0.8063
0,2110
0.7006
0.7259
Reasoning L.MMs
DeepSeek-RI-P
0,4386
0.0450
0.1724
0.5254
0,2820
0,389
0.3773
0.2580
0.3663
0,2936
GLM-Z1-9B
0.8023
0.6929
0.5627
0.6860
0.8126
0.8271
0.6717
0.8234
0.3855
0.7041
0.6973
gpt-oss-20B
0,9378
0.8444
0.4922
0.7581
0.9854
0.8563
0.7273
0.8357
0.4612
0.7732
0.7675
QwQ-32B
0.9508
0.7875
0.7450
0.8278
0.9818
0,8490
0.6810
0.8433
0.2777
0.7266
0.7645
GLM-21-328
0,9053
0,8022
0,5978
0.7684
0.9053
0.8655
0,7071
0.8162
0,2555
0.7099
0.7319
DeepSeck-R1-7088
0,9528
0.6618
0.6867
0.7671
0.9500
0,8440
0.6850
0,7850
0,4280
0.7384
0.7492
DeepSeek-RI R1
0,9310
0.8598
0.6808
0.8239
0.9492
0.8902
0.7374
0.8540
0,2502
0.7362
0.7691
04-mini
0.9798
0.8597
0.7665
0.8687
0.9930
0.8884
0.7475
0,8704
0,2340
0. 0.7467
0.7924
ct al. 2025). QwQ-32B (Team 2025).
GLM-ZI (9B and 32B) (GMM et al. 2024). gpt-oss-20B 20BB
Agaawal et al. 2025). and 04-mini (Jaech et al. 2024). DeepSeek-R provides multiple reasoning distillation variants with the same architectures and sizes, enabling fair comparisons across scales. closed-source non-reasoning and reasoning LLMs. To contextualize LLM performance in end-to-end downstream tasks, we further include traditional domain-specific baselines. Details are provided in Appendix D.2
GPT-4o and 04-mini state the state-of
5.2
SPATIOTEMPORAL REASONING QA EVALUATION
5.2.
SPATIOTEMPORAS UNDERSTANDING
Table reports model performance on spatiotemporal understanding. The evaluation results have been demonstrated to be consistent with the human evaluation. Details are in Appendix D.3 Overall, LLMs excel in interpreting urban spatiotemporal relations and patterns, with all models significantly surpassing the random baseline. Reasoning models achieve over 80% accuracy across multiple abilities. This indicates that L.MMs* broad pretraining has embedded transferable priors for urban spatiotemporal reasoning. Additionally, their performance is notably stronger in spatial understanding. reflecting the greater complexity and variability of temporal dynamics compared static spatial struc- tures, Despite these advantages, LLMs struggle with long-range spatial relations (i.e., connectivity) and long-term temporal patterns, including event chronology, and time-series periodicity or trends (e.g., traffic flow shifts), which often fall below 70% accuracy. These limitations likely stem from the pretraining on large-scale unstructured text, which provides limited exposure to multi-step reasoning over relational or sequential data. Moreover, DeepSeek-RII 1-B shows marked drop due to repetition issues (Li et al. 2023). Further analysis and more cases are provided in the Appendix D.7.3
Non- Mon-Reasoning LLMs vs. Reasoning LLMs: Reasoning models like DeepSeek-R1, QwQ and GLM-ZI generally outperform their non-reasoning versions with gains of 7-20%, highlighting the benefits of reasoning- focused post-training. However, their advantage is not consistent. Firstly, GPT- 4o often matches or exceeds models like GLM- and DeepSeek-Rl-Distiee
suggesting that general post-training on logical and mathematical problems does not always benefit urban spatiotemporal reasoning. Notably, as stated in Section [ DeepSeek-RI falls short in understanding temporal trends, exposing its limitations in urban dynamic analysis. In contrast, Llama3.3-70B excels in this ability among non-reasoning models. Interestingly, this strength has been successfully transferred to PrepSeck-R1-700 which post-trained on Llama3.3 and excels in this task. This motivates us to explore comain-adaptive methods to improve these abilities.
5.2.2 FORECASTING AND PLANNING
Table 4 summarizes model performance on forecasting and planning. The results show that most LLMs exhibit promising forecasting capabilities, with leading models achieving accuracy above 70%. However, in tasks involving long-term temporal trend analysis (e.g., congestion and traffic-OD prediction), non-reasoning base models outperform reasoning variants (e.g., Qwen-2.5 vs. QwQ. Llama3.3 vs. DeepSeek-R1). This further suggests that the general enhancement of reasoning abilities
7

---

Published as a conference paper at ICLR 2026
Table 4: Performance of LLMs in forecasting, planning, and reflection abilities.
Forseasting
Planning Road
Model
Prediction
Next POT Sucie-ccnomm Congestion Tralic-OD
Prediction
Prediction
Prediction
Overall
Traflic Signal
Control
Placement
POT
Planning Planning
Route
Urbun Planning
Overall
Reflection
0.25
0125
0.25
0.25
0.25
0.25
0.25
Non-Reasoning
LLMs
Qwen2.5-78
0.4400
016463
0.3907
0.4972
0.3700
0.290
0.2620
GL.M4-98
0.5660
0.6456
0.6830
0.4690
0.5922
0.3020
0.3890
0.2930
0.4430
0,2490
0.3330
0.2758
Qwen2. 32B
0.7473
0.5449
0,8900
0.4930
0.6938
0,5227
0.2017
0.3030
0.5483
0,3430
0.3843
0.3184
GLMM4-32
0,7050
0.464)
04668
0.2230
0.4440
0.3010
0.2087
1Jama3. 3-3708
0,7507
0.6001
0.7493
0.5110
0.6633
0.5313
0.3273
0.3563
0.6950
0.3413
0.1935
GPT-4o
0.8250
0.9029
0.7350
0.4540
0.7382
0,5620
0.2820
0.3330
0.4780
0,3380
0.3966
0,3802
Reasoming
L.MM
DeepSock-R1-78
0,4900
0.3488
014997
0.2193
013567
0.4120
0.1567
0.3000
0.4117
0.2495
0.2847
0.1066
GLM-ZI-96
0.8740
0.9159
0.6020
0,4280
0.7050
0.5280
0.4990
0.3960
0.6000
0.3160
0.4678
0,4293
gpt-oss-200 20B
0.4940
0.3560
0.3150
QwQ-328
0.9153
0.0015
0,6250
0.4987
0.736
0,5637
0.4977
0.3587
0.4225
0.5997
0.4468
0.4804
GIM--Z1-33BB
0,9230
0,8957
0,6600
0,4360
0.7309
0.5520
0.4000
0.3910
0.3660
0.3120
0.4063
0.4597
DeepSeck
0.5947
0.3227
DeepSeck-RI
0,8900
0.8706
0,6020
0,4780
0.7101
0,5160
0.3120
0.4060
0.3120
0.4390
0.3968
0.5179
mini
0,9320
0.9709
0.7360
0.5100
0.7872
0.5860
0.3420
0.4610
0.3840
0,4600
0.4466
0.5011
Distance
Adjacency
Connectiviit
Chronolagy
Trand
Extrema
Periodiclty
Forecasting
Planning
5.4-
0.5
0.4
0.4
94
08
2.5
0.5
0.33
6.2-
0.4-
0.2
03
9.3
0.2
0.2
0.3
0.2
0.0
3,0
0.2
0.2
Swer2.5-78
Deepbieck-t (inrsleekkk----iiil--ee-----
Oweri Seer2.5-78-45
Figure 4: The performance of the model with enhanced spatiotemporal understanding abilities.
does not always benefit urban scenarios. Meanwhile, the planning performance is substantially lower, revealing that current LLMs struggle to make accurate decisions aligned with long-term objectives. This disparity highlights the increased complexity of planning and reinforces claim in Section 4.1.3
that planning is a higher-order ability, dependent on and extending beyond forecasting.
Interplay Between Reasoning Abilities: Models that excel in spatiotemporal understanding gener- ally perform better in downstream reasoning, For example, gpt-oss-20B exhibits superior long-horizon temporal understanding abilities (e.g., chronology, periodicity, and trend, shown in Table 3) which directly enhances its performance in long-term temporal forecasting (e.g., socio-cconomic. conges- tion, and traffic-OD prediction). To validate this connection, we post-train Qwen2.5-7B specifically on spatiotemporal understanding (the training details are in Appendix C). As illustrated in Figure the fine-tuned model (i.e., Qwen2.5-7B-ST) not only significantly outperforms base (i.e., Qwen2.5- 7B) but reasoning (i.e., DeepSeek-RI -1-nSeee--R-Disiiiei --iisiill-Qee-77777 spatiotemporal understanding for forecasting and planning. Future research could focus on: (I) improving LLMs' comprehension of structured spatiotemporal data, and (2) integrating tools and code-based modeling to support more robust spatiotemporal pattern interpretation.
variant, confirming the benefit of improved
5.2.3 REFLECTION
Isoows shows that most LLMs achieve less than 50% accuracy on reflection, indicating that existing models struggle to interpret feedback and diagnose potential errors. Reflection essential for gener- alizing experience to subsequent reasoning, a high-level ability that facilitates long-term adaptation in dynamic urban environments. Its impact on downstream tasks is further analyzed in Section [33
We further conduct a qualitative study of reflection errors using GPT-5 (OpenAl 2025) as an LLM- judge. We examine three models with varying reflection capabilities (i.e., Qwen2.5-7B. @wwen2.5-32B. and DeepSeck-RI] and categorize reflection performance across three aspects: (1) Error types, including feedback interpretation, adaptation, and integration errors; (2) Reasoning quality, ranging from complete failure to detect errors and fixing them; and (3) Faithfulness, identifying behaviors such as overconfident but wrong reflections or inconsistent reasoning. As shown in Figure 5 DeepSeek-RI consistently demonstrates stronger reflection ability, exhibiting fewer interpretation and adaptation errors and achieving a higher correct- fix rate. However, it shows consistent challenges in dynamically integrating feedback. indicating that improved reasoning alone does not eliminate brittleness adaptation. We also observe that non-reasoning models often exhibit overconfidence even when their conclusions are incorrect. In
Limitations in Reflection Ability:
8

---

Published as a conference paper at ICLR 2026
Error Type
Reflection Category
Decision Category
Qwin2255
Qwen2.5
Occosock
Owen25 328
Deopozck
Qwen25.18
Qwen2.5-328
Dccssece
worprrdtation
Quercont Whong
Incenextert Reaeoring
Figure 5: The analysis of error type, reasoning quality, and faithfulness on reflection.
Table 5: Downstream tasks performance. Lower (4) MAPE, Cost, and Distance (Dis.) are better, while higher (f) Accuracy (Ace.), Service (Ser.), and Ecology (Eco.) are better.
Socio-ccanomic Prediction
Congestion Prediction
Urhan Planning
Road Planning
Model
MAPE (.)
(APE
Ser.
Eco. (
Dis
Classic Method
7.09%
57.05%
17.18%
0.6100
0.4310
18.95
1.99
Non- Ron-Reasoming LLMs
Qwen2.3-7B
34.57%
66. 19%
40.51%
0.3951
0.6440
20.72
1.50
Gl M4-99
58.43%
41.41%
$4.71%
0.6355
0,4507
20.59
1.50
Qwen2.5-
6.00%
24.90%
65 65.90%
0.6335
0,5209
20.56
1.55
GL.M4.320
Llamm3.3-70B
10.86%
38,88%
56. 10%
0.6561
0,5842
19.10
1.57
Reasoning
LLMs
DeepSock-RI-7B
79.23%
67.42%
37.88%
0.6348
0.6111
20.60
1:47
GLM-Z1-9B
11 11.58%
45.87%
52.01%
0.6443
0.5430
18. S0
1.33
spt-oss-20B
5.00%
23.26%
67.41%
0.5842
0.5222
18.67
1.64
QwQ-32B
89%
52.88%
0,5792
18.40
GLM-Z1-32B
7.55%
47.93%
51,22%
0.6468
0.3965
18.57
1.87
DeepSeck-R1-n
5.94%
38.78%
55.50%
0.6560
0.4711
19.42
1.13
DeepSerkRRI
5.24%.
o4-mini
4,97%
15.78%
75.73%
0.6544
0,3863
19.60
1.23
comparison, DeepSeek-RI R1 exhibits occasional inconsistencies. suggesting that faithfulness is an orthogonal challenge that is not resolved by enhanced reasoning abilities in general domains.
5.3 DOWNSTREAM TASK EVALUATION
Table prresentss the performance of LLMs on four real-world urban tasks. Generally, LLMs outper- form classic methods across prediction and secision-maaing tasks. Notably, we observe performance improvements up to 337.31 13%%% in forecasting accuracy and 53.4 53.48% in decision effectiveness. This reinforces the promise of LLMs as fiexible and robust agents for various urban spatiotemporal tasks. Similar to previous observations, models like Qwen2.5-322 and LLaMA3.3 outperform their reason- ing variants (i.e., QwQ and DeepSeek-PR -R1) on long-term prediction tasks (i.e., congestion prediction). This trend highlights the potential bottlenecks of reasoning LLMs in urban spatiotemporal reasoning.
The Impact of Different Reasoning Abilities: To evaluate the contribution of individual reason- ing components overall agent performance, perform ablations in descending order of model reasoning strength, as shown in Figure For DeepSeek-R 1, the top-tier reasoning model, remov- ing spatiotemporal understanding significantly increases prediction error and reduces downstream planning accuracy, indicating that the model relies heavily on accurate early interpretation to sup- port subsequent reasoning. Eliminating forecasting similarly degrades planning, demonstrating that DeepSeek-R effectively uses predictions to inform long-term diciisionmmking Reflection removal produces the largest performance drop, confirming that the model can meaningfully leverage feed- back to refine its outputs. In contrast, models with medium reasoning abilities (e.g.. Qwen2.5-32BB exhibit moderate sensitivity. While removing spatiotemporal understanding still harms performance, bypassing forecasting leads to slight improvement in planning, suggesting that noisy intermediate predictions may misguide downstream reasoning. For weaker models (e.g., Qwen2.5-7B), inter- mediate reasoning and reflection even be detrimental, limited reasoning capacity produces unreliable outputs that propagate additional error rather than providing corrective guidance.
RELATED WORK
Urban Agent: Urban agents have evolved from rule- based systems to RL agents and now LLM- based models. Early systems like SCOOT (Hunt et al. (982) and SCATS (Lowrie) 1990) used fixed heuristies and sensor data for traffic control. RI. approaches, such CoLight (Wei et al. 2019).

---

Published as a conference paper at ICLR 2026
Socio- aooo-ecoommccc Prediction (MAPE
Urban Planning (Service f)
0.30
0.70
0.70
0.70
0.4
0.05-
9.050
0.05
0.69
0.60
0.60
6.2
0.025
0.55
0.55
0.55
0.000
Qwen2.5578
Qwen2.5 328
DeepSeok R1
Qwen2.5-7B
Qwen2.5-328
DeepSenk-R1
Full Pipeline
Wyo Spatiotemporal Understanding
W/O Forecasting
W/O Reflection
Figure 6: The ablation of each reasoning ability on the task performance.
introduced data-driven adaptive learning using neural networks. Recently, LLM-based agents, like LLMLLight (Lai et al. 2025), UrbanGPT (Li et al. (024c), and UrbanKGent (Ning & Liu (024). leverage LLMs for tasks like traffic optimization, spatiotemporal forecasting, and urban knowledge base construction, enabling more flexible and scalable urban intelligence. LLMob (Wang al. 2024a) further frame LLMs as "urban residents" to generate realistic mobility trajectories and activity schedules, advancing agent-level behavior modeling. In contrast, UrbanLLM (Jiang et al. 2024
fine-tunes LLMs to decompose urban queries and orchestrate end-to-end planning autonomously.
Spatiotemporal Reasoning: Spatial reasoning in LLMs has been assessed by understanding spatial
relations (Mizzaee et al., 2021). Later works introduced multi-hop reasoning tasks (Shi et al.) (002)
and urban challenges (Zhan al., 2025). Notably, CityEQA (Zhao et al. 2025) extends spatial reasoning to embodied agents navigating city spaces, Temporal reasoning focuses on understanding event sequences. Studies like (Fatemi al. 2024] Chu et al. (023) examined commonsense temporal reasoning. Spatiotemporal reasoning integrates both spatial and temporal dynamics. Recent work (MOONEY et al., 2023), Quan et al., 2025) analyzed how LLMs interpret spatiotemporal patterns. In urban contexts, BTBench (Li et al. 2024b) and CityBench (Feng et al. (0255) evaluate the reasoning abilities on trajectories and interactions between spatial entities over time. Moreover, LogiCity (Li et al.
2024a) provides neuro-symbolic urban simulator for compositional, long-horizon reasoning.
LLM Complex Reasoning: L.M reasoning has seen rapid advances through chain-of ohain-of-thought (Weil ct al. 2022). which improves multi-step problem solving by encouraging intermediate reasoning. First, instruction tuning approaches, such as AgentTuning (Zeng et al. 2023), boost reasoning quality.
Recently, post-training strategies adopted by models like OpenAl-ol (Jaech et al.) 2024), DeepSeek-
R1 (Guo et al.] 2025). and QwQ (Team, 2025) incorporate RL like Proximal Policy Optimization (PPO) (Schulman et al. 2017) and Group Relative Policy Optimization (GRPO) refine reasoning through reward-guided optimization. Recent advances extend LLMs into agents operating in complex environments. PERIA (Ni et al., 2024) enhances context understanding and forecasts intermediate states to drive long-horizon planning. PreAct (Fu et al. (022)) strengthens planning by integrating explicit future-state prediction. ReflAct (Kim al. | (025) improves reliability through goal-state reflection during action execution. MoScißench (Liu et al. 2026) introduces the first multimodal scientific discovery benchmark, exposing cross-modal reasoning challenges. MM-Agent (Liu et al., 2025b) states four-stage LLM agent framework for open-ended real-world mathematical modeling.
7 CONCLUSION AND LIMITATIONS
Conclusion:
We present USTBench, the first benchmark for systematically evaluating the spatiotem- poral reasoning abilities of LLMS as urban agents. Built on the interactive environment U UAgentEnv, USTBench supports both fine-grained diagnostics of specific reasoning processes and standardized end-to-end task performance evaluations, Our evaluation of leading LLMs shows that while current models excel in spatiotemporal understanding and forecasting, they struggle with higher-order rea- soning abilities, particularly long-term planning and reflection. Notably, reasoning models trained on general logic and mathematics do not consistently outperform non-reasoning models urban-specinte tasks, underscoring the importance of domain adaptation for further enhancement.
Limitations: USTBench primarily focuses on evaluation, while methods for enhancing spatiotem- poral reasoning are underexplored. Moreover, while our study establishes a core, interpretable, and empirically testable decomposition to measure L.MM agent performance in spatiotemporal reasoning, it does not attempt to cover all aspects of LLM agents relevant to broader urban tasks (e.g., social reasoning, multi-agent interaction). Additionally, the evaluations on decision-making tasks are mainly based simulated environments, while real- world validation is essential for urban applications. Future work will explore real-world experiments on decision- decision-making to address this gap.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China (Grant No. 62572417, No.92370204), National Key R&D Program of China (Grant No. N0.2003YYF07220004) and CCF-DiDi GAIA Collaborative Research Funds.
ETHICS STATEMENT
USTBench aims to systematically evaluate the spatiotemporal reasoning capabilities LLMs in urban applications such traffic control, mobility prediction, and urban planning. To ensure privacy, our benchmark exclusively uses publicly available datasets and ensures that no personally identifiable information is included. While these evaluations advance the understanding of LLM capabilities, they also underscore the responsibility to ensure that such models are used ethically. LLM-driven urban agents could influence public infrastructure, mobility patterns, and access to services. Therefore, deploying these models without proper oversight or fail-safes could lead to unintended negative outcomes, especially for vulnerable populations in urban settings.
REPRODUCIBILITY STATEMENT
We have open-soureed all environment configurations, benchmark datasets, and evaluation scripts https://githus: com/ bsail-bkuustUSSBeenhh
The core methodology for spatiotempo- ral reasoning evaluation using UAgentEnv is described in Sections 5 To ensure reproducibilityy we provide comprehensive experimental details, including LLM configurations, baseline implemen- tations (Appendix (D.2), and evaluation metrics (Appendix (D.T). The 62,466 structured QA pairs for process-based evaluation were generated through systematic interactions with UAgentEnv, with representative samples shown Appendix [.7.3] Data processing pipelines are documented in Ap- pendices C.2 B.1 and B.2 Human evaluation procedures and results are reported in Appendix D.3
Finally, the expermental platform and runtime statistics are detailed in Appendix D.5
REFERENCES
Sandhini Agarwal, Lama Ahmad, Jason Ai. Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint ar 2508 10925, 2025.
Simon Elias Bibri and John Krogstie. Smart sustainable cities of the future: An extensive interdisci- plinary literature review. Sustainable cities and society, 31:183-212, 2017.
George Box. Box and jenkins: time series analysis, forecasting and control. A Very British Affair: Six Britons and the Development of Time Series Analysis During the 20th Century, pp. 161-215. Springer.
Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu. TEMPO: Prompt-based generative pre-trained transformer for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024.
Ma Chang, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang. Yaohui Jin, Zhenzhong Lan, Lingpeng Kong, and Junxian He. Agentboard: An analytical evaluation board of multi-turn 11m agents, Advances in Neural Information Processing Systems, 37:74325-74361 2024.
Nuo Chen, Yuhan Li, Jianheng Tang, and Jia Li. Graphwiz: An instruction-followiin language model for graph computational problems. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 353-364, 2024.
Xingyi Cheng, Ruiqing Zhang, Jie Zhou, and Wei Xu. Deeptransport: Learning spatial-temporiy dependency for traffic condition forecasting. 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1-8. IEEF, 2018

---

Published as a conference paper at ICLR 2026
Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Haotian Wang, Ming Liu, and Bing Qin. Timebench: A comprehensive evaluation of temporal reasoning abilities in large language models. arXiv preprint arXiv:2311.17677 2023.
Paul Erdos, Alfréd Rényi, et al. On the evolution of random graphs. Publ. math. inst. hung. acad. sci, 5(1):17-60, 1960.
Bahare Fatemi, Mehran Kazemi, Anton Tsitsulin, Karishma Malkan, Jinycong Yim, John Palowitch, Sungyong Seo, Jonathan Halcrow, and Bryan Perozzi. Test of time: A benchmark for evaluating Ilms on temporal reasoning. arXiv preprint arXiv:2406.09170, 2024.
Jiaxu Feng, Jindong Tian, Yan Cen, Xinyuan Wei, Yun Xiong, and Bin Yang. Physics-guide geside dual neural odes with bidirectional information exchange for air quality prediction. In UrbanAI: Harnessing Artificial Intelligence for Smart Cities.
Jie Feng. Tianhui Liu, Yuwei Du, Siqi Guo, Yuming Lin, and Yong Li. Citygpt: Empowering urban spatial cognition of large language models. In Proceedings of the 31 ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, KDD '25, pp. 591-602, New York, NY, USA, 2025a. Association for Computing Machinery. ISBN 97984007 14542.
Jie Feng. Jun Zhang, Tianhui Liu, Xin Zhang, Tianjian Ouyang, Junbo Yan. Yuwei Du, Siqi Guo, and Yong Li. Citybench: Evaluating the capabilities of large language models for urban tasks. In Proceedings of the 31 ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, KDD '25. pp. 5413-5424, New York, NY, USA, 2025b. Association for Computing Machinery. ISBN 018440071452...
Dayuan Fu, Jianzhao Huang. Siyuan Lu, Guanting Dong, Yejie Wang, Keqing He, and Weiran Xu. Preact: Prediction enhances agent's planning ability. In Proceedings of the 31st international conference on computational linguisties, pp. 1-16, 2025.
Ahmed Fawzy Gad. Pygad: An intuitive genetic algorithm python library. Multimedia tools and applications, 83(20):58029-588 -5042,, 2024.
Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. Chatglm: family of large language models from glm-1300 130b to glm-4 all tools. arXiv preprint arXiv:2406. /2793, 2024.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Daale, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama herd of models. arXiv preprint arXiv:2407.21788 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12944 2025.
PB Hunt, DI Robertson, RD Bretherton, and M Cr Royle. The scoot on-line traffic signal optimisation technique. Traffic Engineering & Control, 23(4), 1982.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint arXiv:2412.16770, 2024.
Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. Towards mitigating LLM hallucination via self reflection. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pp. 1827-1843. Association for Computational Linguisties, 2023.
Yue Jiang, Qin Chao, Yile Chen, Xiucheng Li, Shuai Liu, and Gao Cong. Urbanlim: Autonomous urban activity planning and management with large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 18 1810-1825, 2024.
12

---

Published as a conference paper at ICLR 2026
Jeonghye Kim, Sojeong Rhee, Minbeom Kim, Dohyung Kim, Sangmook Lee, Youngchul Sung, and Kyomin Jung. Reflact: World-grou goold-grounded decision making in Ilm agents via goal-state reflection. arXiv preprint arXiv:2505.15182 2025.
Siqi Lai, Zhao Xu, Weijia Zhang. Hao Liu, and Hui Xiong. Limlight: Large language models as traffic signal control agents. In Proceedings of the 31th ACM SIGKDD Conference Knowledge Discovery and Data Mining, 2025.
Bowen Li, Zhaoyu Li, Qiwei Du, Jinqi Luo, Wenshan Wang, Yaqi Xie, Simon Stepputtis, Chen Wang, Katia Sycara, Pradeep Ravikumar, et al. Logicity: Advancing neuro-symboliic ai with abstract urban simulation. Advances in Neural Information Processing Systems, 37:69840 66864, 2024a.
Huayang Li, Tian Lan, Zihao Fu, Deng Cai, Lemao Liu, Nigel Collier, Taro Watanabe, and Yixuan Su. Repetition in repetition out: Towards understanding neural text degeneration from the data perspective. Advances in Neural Information Processing Systems, 36:72888-72903. 2023.
Ke Li, Lisi Chen, and Shuo Shang. Towards alleviating traffic congestion: Optimal route planning for massive-scale trips. In Proceedings of the Twenty-Nitth International Conference on International Joint Conferences on Artificial Intelligence, pp. 1400-3306, 2021.
Wenbin Li, Di Yao, Ruibo Zhao, Wenjie Chen, Zijic Xu, Chengxue Luo, Chang Gong, Quanliang Jing, Haining Tan, and Jingping Bi. Sthench: Assessing the ability of large language models in spatio-temporal analysis. arXiv preprint arXiv:2406. 19065, 2024b.
Zhonghang Li, Lianghao Xia, Jiabin Tang, Yong Xu, Lei Shi, Long Xia, Dawei Yin, and Chao Huang. Urbangpt: Spatio-temporal large language models. In Proceedings of the 30th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 5351-5362, 2024c.
Chenxi Liu, Qianxiong Xu, Hao Miao, Sun Yang, Lingzheng Zhang, Cheng Long, Ziyue Li, and Rui Zhao. Timecma: Towards lIm-empowered multivariate time series forecasting via cross- coos-modalltyy alignment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 18780-18788, 2025a.
Fan Liu, Zhe-Rui Yang, Cancheng Liu, Tianrui SONG, Xiaofeng Gao, and Hao Liu. MM-agent: LLM as agents for real-world mathematical modeling problem. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025b.
Fan Liu, Xiaozhao Zeng, and Hao Liu. Towards multimodal data-driven scientific discovery powered by LLM agents. In The Fourleenth International Conference Learning Representations, 2026.
URL
wwt. www.:nett/ttiiiieiemo wwt.ormmmmmkkek//771e
Yu Liu, Xin Zhang, Jingtao Ding, Yanxin Xi, and Yong Li. Knowledge-infused contrastive learning for urban imagery-based socioeconomic prediction. In Proceedings of the ACM web conference 2023, pp. 4150-4160, 2023.
Pablo Alvarez Lopez, Angelo Banse, Mirko Barthauer, Michael Behrisch, Benjamin Couéraud, Jakob Erdmann, Yun-Pang Flötteröd, Robert Hilbrich, Ronald Nippold, and Peter Wagner. Simulation of urban mobility (sumo), February 2025.
PR Lowrie. Scats, sydney co-ordinated adaptive traffic system: A traffic responsive method of controlling urban traffic. 1990.
Roshanak Mirzaee, Hossein Rajaby Faghihi, Qiang Ning, and Parisa Kordjmashidi. Spartqa:: A textual question answering benchmark for spatial reasoning. arXiv preprint arXiv:2 2021.
PETER MOONEY, WENCONG CUI, BOYUAN GUAN, and LEVENTE JUHÁSZ. Towards understanding the spatial literacy of chatgpt. In ACM SIGSPATIAL International Conference, 2023.
New York City Taxi and Limousine Commission. Tle trip record data, 2025. URL https: /www nxc..po//site//ll//abottt/ wewpcccttirrrrcooreeee record- pagel Accessed: 2025-05-08.
13

---

Published as a conference paper at ICLR 2026
Fei Ni, Jianye Hao, Shiguang Wu, Longxin Kou, Yifu Yuan, Zibin Dong, Jinyi Liu, MingZhi Li, Yuzheng Zhuang, and Yan Zheng. Peria: Perceive, reason, imagine, act via holistic language and vision planning for manipulation. Advances in Neural Information Processing Systems, 37: 17541-17571, 2024.
Yansong Ning and Hao Liu. Urbankgent: A unified large language model agent framework for urban knowledge graph construction. arXiv preprint (Xii:2402.0666// 2024.
Yansong Ning, Shuowei Cai, Wei Li, Jun Fang, Naiqiang Tan, Hua Chai, and Hao Liu. Dima: An IIm-powered ride-hailing assistant didi. arXiv preprint arXiv:2503.44: 047668, 2025.
OpenAl. GPT-5 System Card. Technical report, OpenAl, August 2025. URL https://cdn
openai . httpili.commpppp///// /pttt 5- wwwsssystemmaaa pdf Accessed: 2025-11-21.
OpenStreetMap contributors.
Openstreetmap planet data, 2025. Accessed: 2025-03-31.
URL https://planet
openstreetmap. org/
Pengrui Quan, Brian Wang, Kang Yang, Liying Han, and Mani Srivastava. Benchmarking spa- tiotemporal reasoning in lims and reasoning models: Capabilities and challenges, 2025. URL
https: org/ als 250. 11618
Matthew Renze and Erhan Guven. Self-reflection in IIm agents: Effects on problem-solving perfor- mance. arXiv preprint 1eXiv:2405006682. 2024.
Md Imbesat Rizvi, Xiaodan Zhu, and Iryna Gurevych. Spare and sparp: Spatial reasoning characteri- zation and path generation for understanding spatial reasoning capability of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4750-4767, 2024.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.063477 2017.
Zhengxiang Shi, Qiang Zhang, and Aldo Lipani. Stepgame: A new benchmark for robust multi- hop spatial reasoning in texts. In Proceedings of the AAAI conference on artificial intelligence. volume 36, pp. 11321-1 1329, 2022.
Zheng Tang, Milind Naphade, Ming- Yu Liu, Xiaodong Yang, Stan Birchfield, Shuo Wang. Ratnesh Kumar, David Anastasiu, and Jenq-Neng Hwang. Cityflow: A city-scale benchmark for multi-target multi-camera vehicle tracking and re-identification. In Proceedings of the IEFE/CVF conference on computer vision and pattern recognition, pp. 8797-8806, 2019.
Qwen Team. Qwg-32b: Embracing the power of reinforcement learning, 2025. URL https
higgo/////i///o/gg-g/g/-/gg/// qwq- 32b/ Accessed: 2025-03-26.
Jindong Tian, Yuxuan Liang, Ronghui Xu, Peng Chen, Chenjuan Guo, Aoying Zhou, Lujia Pan, Zhongwen Rao, and Bin Yang. Air quality prediction with thysics-guided dual neural odes in open systems. ICLR, 2025.
Jindong Tian, Yuxuan Liang, Ronghui Xu, Peng Chen, Chenjuan Guo, Aoying Zhou, Lujia Pan, Zhongwen Rao, and Bin Yang. Arrow: An adaptive rollout and routing method for global weather forecasting. 2026.
Zaib Ullah, Fadi Al-Turjman, Leonardo Mostarda, and Roberto Gagliardi. Applications of artificial intelligence and machine learning in smart cities. Computer Communications, 154:313-323, 2020.
Leonie von Wahl, Nicolas Tempelmeier. Ashutosh Sao, and Elena Demidova. Reinforcement learning- based placement of charging stations in urban road networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. pp. 3992-4000 2022.
Jiawei Wang, Renhe Jiang, Chuang Yang, Zengqing Wu, Makoto Onizuka, Ryosuke Shibasaki, Noboru Koshizuka, and Chuan Xiao. Large language models as urban residents: An Ilm agent framework for personal mobility generation. Advances in Neural Information Processing Systems, 37:1 17:124447-1 2024a.
14

---

Published as a conference paper at ICLR 2026
Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186445, 2024b.
Tingting Wang and Fubao Sun. Global gridded gdp data set consistent with the shared socioeconomic pathways. Scientific data, 911:221, 2022.
Xinglei Wang, Meng Fang, Zichao Zeng, and Tao Cheng. Where would i go next? large language models as human mobility predictors. arXiv preprint arXiv:2308. 15197, 2023.
Xinlei Wang, Maike Feng, Jing Qiu, Jinjin Gu, and Junhua Zhao. From news forecast: Integrating event analysis in I1mmbasee based time series forecasting with reflection. Advances in Neural Information Processing Systems, 37:581 18-58153, 2024c.
Yuqing Wang and Yun Zhao. Tram: Benchmarking temporal reasoning for large language models. In Findings of the Association for Computational Linguistics ACL 2024, pp. 6389-6415, 2024.
Hua Wei, Nan Xu, Huichu Zhang, Guanjie Zheng, Xinshi Zang, Chacha Chen, Weinan Zhang, Yanmin Zhu, Kai Xu, and Zhenhui Li. Colight: Learning network cooperation for traffic signal control. In Proceedings of the 28th ACM international conference on information and knowledge management, pp. 1913-1922, 2019.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 0224822--4837, 2022.
Ran Wei. Coverage location models: alternatives, approximation, and uncertainty. International Regional Science Review, 39(1):48-766 2016.
WorldPop. Worldpop hub, 2025. URL
Siheng Xiong, Ali Payani, Ramana Kompella, and Faramarz. Fekri. Large language models can learn temporal reasoning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 10452-10470, 2024.
An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprins 15115 2024.
Dingqi Yang. Foursquare dataset. https://sites: httpa//ees..oogle.oooo/////////e
htt 2014. Accessed: 2025-05-19.
home home/Eoursquaree wtwrs/uaae-eeateeeeeeeee
Yuan Yuan, Jingtao Ding, Jie Feng, Depeng Jin, and Yong Li. Unist: prompt-empowered universal model for urban spatio-temporal prediction. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4095 4106, 2024.
Zirui Yuan, Siqi Lai, and Hao Liu. Collmlight: Cooperative large language model agents for nedwork-wide traffic signal control. arXiv preprint arXiv:2503. / 1739, 2025.
Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. Agenttuning: Enabling generalized agent abilities for 1lms. arXiv preprint arXiv:2310.12823 2023.
Weichen Zhan, Zile Zhou, Zhiheng Zheng, Chen Gao. Jinqiang Cui, Yong Li, Xinlei Chen, and Xiao-Ping Zhang. Open3dvqa: A benchmark for comprehensive spatial reasoning with multimodal large language model open space. arXiv preprint arXiv:2503. 1094, 2025.
Huichu Zhang, Siyuan Feng, Chang Liu, Yaoyao Ding, Yichen Zhu, Zihan Zhou, Weinan Zhang, Yong Yu, Haiming Jin, and Zhenhui Li. Reinforcement learning for traffic signal control. http: cont github. io 2023. Accessed on 1 December 2023.
/ c-
Pengpeng Zhao, Anjing Luo, Yanchi Liu, Jiajie Xu, Zhixu Li, Fuzhen Zhuang, Victor S Sheng, and Xiaofang Zhou. Where to go next: A spatio-tempora gated network for next poi recommendation. IEEE Transactions Knowledge and Data Engineering, 34(5):2512-22 2524, 2020.
15

---

Published as a conference paper at ICLR 2026
Yong Zhao, Kai Xu, Zhengqiu Zhu, Yue Hu, Zhiheng Zheng, Yingfeng Chen, Yatai Ji, Chen Gao, Yong Li, and Jincai Huang. Cityeqa: A hierarchical Ilm agent on embodied question answering benchmark in city space. arXiv preprint 2rXiv:2502.12532, 2025.
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangehi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume System Demonstraations), Bangkok, Thailand, 2024. Association for Computational Linguistics. URL http::tpp//aaiiii..recce abs/2403. 13372
Yu Zheng, Yuming Lin, Liang Zhao, Tinghai Wu, Depeng Jin, and Yong Li. Spatial planning of urban communities via deep reinforcement learning, Nature Computational Science, 3(9):748-762 2023a.
Yu Zheng, Hongyuan Su, Jingtao Ding, Depeng Jin, and Yong Li. Road planning for slums via deep reinforcement learning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 5706, 2023b.
Yu Zheng, Longyi Liu, Yuming Lin, Jie Feng, Guozhen Zhang, Depeng Jin, and Yong Li. Urbanplan- bench: comprehensive urban planning benchmark for evaluating large language models. arXiv preprint arXiv:2504.21027 2025.
Zhilun Zhou, Yuming Lin, Depeng Jin, and Yong Li. Large language model for participatory urban planning. arXiv preprint 240. 2024.
16

---

Published as a conference paper at ICLR 2026
USTBench:
Benchmarking and Dissecting Spatiotemporal Reasoning Capabilities of LLMs as Urban Agents
Supplementary Material
CONTENTS
Introduction
2
Preliminary
UAgentEnv: Interactive City Environment
Urban Task Suite and Data Sources
3.2
Unified Urban Agent Framework
USTBench: Urban Spatiotemporal Reasoning Benchmark
Process-based Spatiotemporal Reasoning Evaluation
4.1.1
Observation Construction
4.1.2
Spatiotemporal Understanding QA
3
Forecasting QA
1.1.4 4
Planning QA
5
Reflection Q^
End-to-End Downstream Task Evaluation
Experiment
Baseline Models
5.2
Spatiotemporal Reasoning QA Evaluation
5.2.1
Spatiotemporal Understanding
5.2.2
Forecasting and Planning
5.2.3
Reflection
5.3
Downstream Task Evaluation
6
Related Work
7
Conclusion and Limitations
10
LLM Usage
19
Details of Environments
19
B. Urban Task Suite and Environment Platform
19
B.2 Environment Configuration
19
B.3
Agent Prompt
19
17

---

Published as a conference paper at ICLR 2026
LLM Post- Post-Training
20
Instruction Construction
20
C.2
Supervised Distillation Fine- Fine-Tuning Tuning
20
Detailed Evaluations)
20
D.I
Downstream Task Evaluation Metrics
20
D.2
Baseline Configuration
D33
Human Evaluation
21
D.4
Confidence Interval of Evaluation
D.5
Runtime Estimation
................................
21
D.6
Downstream Task Performance
................................
................................
D.7
Reasoning Behavior Analysis
22
D.7.1
Cost-Effectiveness Analysis
22
D.7.2
Repetition Issues
22
D.7.3
Process-based Spatiotemporal Reasoning QA Examples
23
D.7.4
LLM Agent Prompt
35
D.7.5
Spatiotemporal Understanding Post- Training Instruction Examples
38
18

---

Published as a conference paper at ICLR 2026
A LLM USAGE
We employ LLMs to enhance the clarity and coherence of our manuscript writing, ensuring consis- tency in terminology, structure, and academic tone. addition, LLMs are utilized to generate figures and illustrations that effectively visualize experimental results, improving the overall readability and presentation quality of the paper.
B DETAILS OF ENVIRONMENTS
B. URBAN TASK SUITE AND ENVIRONMENT PLATFORM
Predicts movement patterns within urban areas, helping to understand how individuals travel between points of interest (POIs). Following the settings proposed by (Wang et al.;
Next POI Prediction:
2023 we use FourSquare datasets (Yang 2014) for evaluation.
Socio-Economic Prediction: Estimates future socio-economic indicators (i.e., GDP) based on historical observations and population density. We collect global GDP and population data from 2000 to 2019 (Wang& Sun 2022 WorldPop
2025) for future value prediction.
Congestion Prediction: Anticipates areas where traffic flow will become heavy. We follow the settings (Cheng et al. 2018 and evaluate LLMs using traffic data from Beijing, China.
Traffic-OD Prediction: Estimates vehicle flow between origin and destination regions. We follow the settings of (Yuan et al. 2024) and use taxi traffic flow datasets from New York, USA.
Traffic Signal Control: Optimizes traffic signal timing to improve traffic flow. Following the settings 2025), we use traffic flow and road network datasets from China and the US (Zhang et al. 2023). The simulation environment is built based on CityFlow (Tang et al. 2019).
of (Lai et al.
POI Placement: Determines optimal locations for urban services (e.g., shops, restaurants), ensuring strategic placement to serve the population and reduce congestion. For evaluation, we follow the settings (von Wahl et al. 2022) and collect the POI dataset Guangzhou, China.
Road Planning: Involves designing and optimizing road networks to enhance transportation ef- ficiency. This task requires analyzing road connectivity and infrastructure needs. We follow the settings and environment of (Zheng al. 2023b) and road network data from Cape Town (South Africa), Harare (Zimbabwe), and Mumbai (India) for evaluation.
Route Planning: Determines optimal paths for vehicles, considering traffic conditions, distance, and travel time. Following the settings of (Li et al. 2021). we use New York's road network and simulate traffic using the SUMO traffic simulator (Lopez et 2025).
Urban Planning: Involves designing urban spaces for sustainable and efficient cities. Following (Zheng al. 2023a), we use urban geospatial data from Beijing, China.
B.2 ENVIRONMENT CONFIGURATION
For process-based spatiotemporal reasoning evaluation, we generate QA instances using UngentEnv. Environmental observations in decision-making tasks collected using semi-stochastic policy with an exploration coefficient e = 0.1 to ensure diversity. Ground- truth answers for planning QA are derived through simulation-drive -riven exploratory process, with planning horizon H 5 and discount factor y = 0.9. During downstream task evaluation, we configure historical observation and prediction windows based on task characteristics. For socio-economic prediction, we use 6-year observation and a 3-year prediction window. For traffic-related congestonn prediction and traffic OD prediction, we adopt 12-step observation and a 12-step prediction window. In the next POI prediction task, the agent receives 30-day activity history and recent visits on the same day and predicts the next visited POL. For route planning, we synthesize urban mobility patterns using the gravity model (Li et al. 202I), calibrated real-world population distributions. The configurations for decision-making environments follow established benchmarks and experimental protocols from
prior work (Wei et al. 2019 von Wahl et al. 2022 Li et al. 2021 Zheng et al. 2023b a)
B.3
AGENT PROMPT
Our agent framework incorporates three prompt templates designed for tosk-solving, reflection, and memory storage. In the task-solving prompt, the agent is instructed perform multi-stage
19

---

Published as a conference paper at ICLR 2026
reasoning process: 1) it first interprets the spatiotemporal context of the environment; 2) then either forecasts future urban states or predicts the outcomes of candidate actions; 3) finally, the agent outputs anticipated urban states for prediction tasks selects optimal action for decision-makinn tasks. After execution and receiving environmental feedback, the agent is instructed by the reflection prompt evaluate the effectiveness of its prior decision and summarizes the outcome inform future reasoning. These experiences are subsequently aggregated and stored in memory to support continual adaptation with the summary prompt. The detailed prompt templates used in each stage are provided in Table 23
experience
C
LLM POST-TRAINING
C.1 INSTRUCTION CONSTRUCTION
We post-train Qwen2.5-7B using a synthetic instruction tuning dataset, designed to enhance the model's iss  capability to interpret spatiotemporal dynamics in urban scenarios. To generate instructions, we first prompt GPT-4o to produce diverse urban scenarios involving various entities and events. Each scenario is designed to elicit specific spatiotemporal understanding ability, prompting the model to analyze spatial relationships or temporal patterns:
Distance: We randomly assign distances between roads, routes, or urban entities. The model is asked to identify the longest or shortest elements or to compare distances between pairs of entities.
We Erdös R'enyi (ER) model (Erdos al., 1960) to generate random spatial graphs representing urban layouts. The model is then asked to determine adjacency (i.e., nearby neighbors) or connectivity (i.e., path existence between entities).
Adjacency & Connectivity:
Duration: We simulate urban events (e.g., travel, wait times) with randomly assigned durations. The model is tasked with identifying the longest shortest event comparing durations between events.
Chronology: We use POI check-in data from Tokyo (Yang) 2014) (distinct from the dataset used in USTBench). The model is tasked identify the correct temporal sequence of check-in events.
Trend, Local Extrema, and Periodicity: We leverage real-world urban time-series datasets, includ- ing PEMS04 (traffic flow), Solar (solar power output), and Electricity (power usage). The model is instructed to identify global trends (e.g., increasing/decrresinng)) local extrema (e.g., peak hours), and periodicity (e.g., daily/weekly cycles).
C.2 SUPERVISED DISTILLATION FINE-TUNING
Leveraging the constructed instructions described above, we collect responses from DeepSeck-RR -R1 using rejection sampling to ensure high-quality outputs. Representative examples of the instruction tuning data are provided in Table 24 For supervised fine-tuning, we adopt Llama-Factory Factory (Zheng
ct al. 2024) with Low-Rank Adaptation (LoRA) for training. The learning rate is set x 10-
D DETAILED EVALUATIONS
D.I DOWNSTREAM TASK EVALUATION METRICS
We evaluate LLMs' downstream task performance across nine urban tasks using task-specific metrics. Socio-economic prediction: We apply Mean Absolute Percentage Error (MAPE) assess GDP forecasting accuracy over a three-year window. Congestion prediction: This task is framed as a five-level classification task (levels 0-4), and is evaluated using MAPE and accuracy. Traffic-OD prediction: We adopt Mean Absolute Error (MAE) and Symmetric Mean Absolute Percentage Error (SMAPE) to evaluate forecasting accuracy for vehicle arrivals and departures. Next POI prediction: We use Precision and Mean Reciprocal Rank (MRR) to measure the recall quality of predictions. Urban planning: This task is assessed through accessibility to service facilities and ecological coverage. Road planning: The performance is measured by construction costs and the average travel distance to neighboring regions. POI placement: The task is evaluated by the average travel and waiting time for urban services. Traffic signal control: The performance is measured using average travel time (ATT) and waiting time (AWT) in the road network. Route planning: The task is evaluated using average travel time and network throughput.
20

---

Published as a conference paper at ICLR 2026
Table 6: Comparison of exact-match and human evaluation across spatiotemporal reasoning tasks.
Model
Evaluation Method
Spatiotemporal Understanding
Forecasting
Planning
Reflection
Exact-matchh Human-BSt Eval
0.6000
0,6500
0.4800
0.2222
Llama3.3-70B
0.6428
0.7000
0.5000
0.2556
Exact- match
0. 0.7000
0.6250
0,4400
0,4556
DeepSeek-R
Human- Eval
0.7572
0.7000
0.4800
0,5111
Table 7: Confidence Interval of Spatiotemporal Reasoning Evaluation with Accuracy (%).
Middll
Distnnee
Spotial Undersianding
Aijorency
Connectiiiy
Duration
C Chronology
Temporal Underitanding
Trend
Local Estrema
Perlodicly
Formentting
Plarningg
Refiection
Nou. Hessonnnng LLMs
2-76 -- 80) 45.70 (HCCA)) 28 (1005)) (0 40.10 (10.5%) 1-.40 57.67 (40.4)) 66,
44.72
10.04 (10.58) (5) 18
0we12.5.33 (=0
(44.100 (3)
(20UI), 7303 (20.20) 79.23 (att
(200
(20.10)
Reseoming LLMs
DesgGeck-RI-
Owen
(20035) (50()))) 3.37 (40.0r) (2.54 3190 (200))))))) (41.4))
37.73 49 (60.60
(80.88)
QwQ-326
95T8 (100) (075(1 To 100011101 18 1011523 (ATTT (100111l) (633
D.2 BASELINE CONFIGURATION
LLM Configuration: In this study, we mainly evaluate open-source LLMs using the vLLM inference framework. For proprietary models such GPT-4o and GPT-44-mini, we utilize the OpenAI API, while DeepSeek-R R1 is evaluated via the Alibaba Bailian API. All evaluations are conducted with a fixed decoding temperature of 0.1 to ensure reproducibility. Inference of open-sourced L.LMs performed on server equipped with two NVIDIA A800-80GB GPUs.
Domain-speccfic Method Configuration: To provide a comprehensive performance comparison, we also benchmark LLMs against traditional methods widely used in each domain: For time-series forecasting tasks, we use ARIMA (Box) as the baseline model. For urban planning, we employ a geometric set coverage algorithm (GSCA) (Wei 2016),, which solves geometrie set-coverage like problem by maximizing the spatial coverage of designated land-use types. For road planning, we apply genetic algorithm (Gadd (2224), where linear layer represents road features, and roads incrementally constructed based on learned sampling probabilities.
D.3 HUMAN EVALUATION
In our experiment, the LLM performance primarily evaluated using exact-match comparison with the extracted ground-truie labels to ensure reproducibility and computational efficiency across large-scale evaluation. For completeness, we incorporate human evaluation with five human experts in urban science over a randomly selected 250 QAs in our benchmark. This assessment examines not only the correctness of model answers but also evaluates the underlying reasoning process. The results are shown in Figure We find that human evaluations generally align with our exact-match assessments, supporting the reliability of our automated metric.
D.4 CONFIDENCE INTERVAL OF EVALUATION
In Table 7 we report the confidence intervals for representative models based three experiments.
D.5 RUNTIME ESTIMATION
The evaluation runtime of an LLM varies depending on the hardware or API, the specific model, and the inference platform employed. In this study, estimate runtimes for open-source LLMs using vLLM, and for GPT-4o and GPT-4o-mini through the OpenAl API. Except for DeepSeek-R1, evaluate with the Alibaba Bailian API. The estimated time of our process-based spatiotemporal reasoning evaluation is shown in Table
D.6 DOWNSTREAM TASK PERFORMANCE
The downstream task performance of representative LLMs is shown in Table 9 Notably, reasoning LLMs do not consistently outperform their non-reasoning base models in real-world urban scenarios. This indicates that advances in general mathematical and logical reasoning do not necessarily benefit urban tasks. The finding underscores the importance of developing camain-sppccfii approaches tailored to the unique challenges of urban spatiotemporal reasoning.
21

---

Published as a conference paper at ICLR 2026
Table 8: The runtime estimation results of the representative LLMs.
Model
Device API
Platform
Inference Speed
Batch Size
Total Time
Qwen2.5-32B
A800
vLLm
14.12 s/batch
32
9,9 h
GPT-4o
OpenAI API
8.20 s/batch
16
11,50 h
QwQ-32B
2 A800
vLL.m
30.21 s/batch
32
18.64 h
DeepSeek-RI
Alibaba Bailian API
97.33 s/batch
32
60.05 h
04-mini
OpenAI API
5.61 s/batch
16
7.87
Table 9: Performance on downstream urban tasks. Lower values (L) for MAPE, SMAPE, MAE, Cost, Distance, ATT, and AWT indicate better performance. Higher values (1) for Accuracy, Service, Ecology, Precision, MRR, and Throughput indicate better outcomes.
Sacio-tzomomny Prediction
Congration Prediction
Urban
Road
TrafficOD
Signal Control
Ronte Planning
Model
Planning
Planning
Prediction
Prediction
Placement
(5)
1S Acc [8)
Sere
Eeo
Coo Dis
MAE SMAPE (10 Prect MO
NTT AWT
AFT
AW
ATT
Threpu
Ron-Rensonnng L.MSs
Qwen2.5-78
34.57
66.1
40.51
0.5951
6.640
20.72 1.50
3,09
13.35
0.3787
0.188
1.21 0.56
830.26
141701
367
GLM14 9B
54.71
0,6355
0.4507
20.59
1.30
34.22
96.85
0.7017
0.4405
1009
1366.45
372
5-320
24.90
672.82
GLM64-38
9.41
25.61
63.00
0.6622
0.4715
18:44
1.52
11,20
43.20
03185
0.3786
1.32 0.36
1290.61
690
1384.54
373
1.ama3.3-200
10.86
$8.88
56 16.10
0.6561
0.1842
19.10
1.57
8.52
33.13
04863
0.3275
1.12 0.36
130484
682
1310.11
370
Reasoming
Distil-Qwen--
DeepSeekk-83 R1-
79.23
67.42
31.88
0634806611
20.60 1.47 140.05
130.84
0.2910
0.1455
1.18 0.46
1000.56 S41
1390.65
372
GLM-ZI-98
52.01
6664400540011
970.36
711.3
1283.29
371
QwQ 32B
5.64
44.39
52.88
0.6751
0.5792
18.40
1.77
8.16
36.96
0.6817
0.6013
1.19 0.54
1267,82
1417.32
373
GLM 320
7.55
47.93
51.22
0.6468
0.3965
18:57
1.87
56,46
77.46
0.6430
0.5540
1.10 0.74
132,
641.73
1331.51
370
DeepSeck-R1-
55.50
0.65600.471
1200
1267.20
375
D.7 REASONING BEHAVIOR ANALYSIS
D.7.1 COST-ERFECTIVENESS ANALYSIS
In our process-based spatiotemporal reasoning evaluation, observe that the non-reasoning model GPT-4o is comparable, and in some cases surpasses, reasoning models. To further plore this, we conduct a cost-effectiveenss anal- ysis (Figure[7), comparing model performance on spatiotemporal understanding relative to the number of reasoning tokens used. Among all models, o4- 14-mini demonstrates the highest cost- efficiency, achieving strong performance with minimal reasoning overhead, followed closely by GPT-4o. contrast, while DeepSeek-RP delivers strong performance, its reasoning pro- cesses are often verbose and time-consuming, making less suitable for real-time deployment scenarios (e.g., traffic management). These findings highlight research opportunities to develop lightweight and efficient paradigms for urban spatiotemporal reasoning, drawing inspiration from designs like 04-mini and GPT-4o.
Tamily
Qwen
Greplees et tin ma
Score
GLV-460
Token
Figure The token usage vs. score.
D.7.2 REPETITION ISSUES
We observe severe repetition issues on DeepSeek-RI-Disiill--enel Qwen2.5-7B, where it underperforms its base model, Qwen2.5-7B, on several tasks. Upon amalyzing its reasoning traces, we find that the model tends to repeat certain thought patterns, called "aha moments" (Guo et al.) 2025). without progressing toward solution. This repetitive behavior resembles ineffective in-context learning loops, and appears more pronounced in specialized domains like urban tasks. While such issues have been partially mitigated in mathematical and logical reasoning, they remain a persistent challenge in other specialized domains. Figure 19 provides examples of failed reasoning processes.
22

---

Published as a conference paper at ICLR 2026
D.7.3 PROCESS-BASED SPATIOTEMPORAL REASONING QA EXAMPLES
Table 10: The spatiotemporal understanding QA for distance relation analysis.
Task Description
We are tasked with determining the optimal location for new electric vehicle (EV) charging station to maximize benefits by effectively meeting demand. The candidate locations, along with their expected charging station coverage, demand (number of vehicles), average distance, travel time, and waiting time to existing charging stations, are provided below.
Data Schema
The number of locations that will be within the influence radius of the charging station after its construction.
coverage:
demand: The number of vehicles requiring charging near the location.
distance: The average distance that vehicles near the location must travel to reach existing charging stations.
waiting time: The average waiting time nearby existing charging stations. NaN indicates no station nearby and the highest waiting time. charging-tin time: The average charging time at nearby existing charging stations. NaN indicates no station nearby and the highest charging time.
Data
location 1:
coverage: 3.315
demand: 43531.73
distance: 265.80m
waiting time: 0.00h
charging.time: 1.80h
location
coverage: 3.316
demand: 74504.47
distance: 159.25m
wiiting-time: 0.04h
charging time: 20.61h
location 3:
coverage: 3.316
demand: 130231.14 14
-distance: 160. 19m
waiting_time: 0.04h
charging.time 21.75h
location 4:
Question
Based on the data provided, rank the locations average distance to nearby charging stations, from the farthest to the closest.
A. > 2> 3 4
B. > 4>3>
C. <1>2>3>4
D.4>1>3>>
Note
Please answer the question using the information given. Let's work this out a step-by-step way to be that have the right answer. Then provide your final answer within the XML tag <Answer>A/B/ C/ BttBB///DDD////////// Answer>
23

---

Published as a conference paper at ICLR 2026
Qwen2.5-32B The locations are ranked by average distance to nearby charging stations from the farthest to the closest as 4 >
DeepSeek-R1: The average distances for each locstion are: Location (325. 11m), Locstion 1 (285. (065,80m,, Location (160. 19m), and
Location 2 65m). Ranking from farthest closest gives the order 4 3 which corresponds to option D.
When ranking average distance to existing charging stations from farthest to closest, the order Location [325.11 m) Location (265.80 m) Location 3 (160.19 m) Location 2 (159.25 mi, which matches option
04-mini:
Figure The LLMs' reasoning on distance relation analysis.
Table 11: The spatiotemporal understanding QA for adjacency relation analysis.
Task Description
You are tasked with predicting urban uudicators--pecifiicen the GDP and population of Guangzhou for the current year. Historical data for the GDP and population the target re- gion and its nearby regions will be provided. Additionally, example region with its GDP and population records for previous years and the current year will be included for reference. Data Schema
www.iiinpvvuuuueeeee years: The GDP (in millions of USD) of region in past years. population in srrevious_years The population region in past years. connectivity: A list of tuples in the format (region_ region_2, distance), defining undirected edges between regions with the specified distance in m.
Data
Target Region 442:
years: [(8.54, 2000), (23.19, 2018)]
poppllaion.in_previousssyy previioosss.arrss [(2942.97, 2000), (2242.3,
Nearby Region 418:
wtw.prrvious_yearr [(22.55, 2000), (55.37, (55,,,,))
population_in pnwooiinnnnnpeeeeeeeeeee [(2589.67, 2000), (8729.58, 2018)]
Adjacency Info:
(442, 418, 7.58km) (442, 443, 6. 69kmm,, (442, 467, 4. 4.51km), (467, 492, 4.51km)
Example Region 192:
adp_records: [2011,59, 2000), .... (628.55, 2019)]
population_records:: [(5908.03, 2000), (11102.63, (1109) Question
Based the connectivity data, rank spatial correlation from the target region to its nearby regions, ordering them from the strongest to the weakest correlation.
A. 4.49 > 468 > 417 > 443.
B. 4.44 > 492 > 468 > 417.
C. 492 > 468 > 417 > 443.
D. 468 > 443 > 492 > 417.
Note
Please answer the question using the information given. Let's work this out a step-by-step way to be that have the right answer. Then provide your final answer within the XML tag CAnswer>A/B/C/DB /Answer>
Qwen2.5-328. The regions ranked based their shortest distance from the target region 442, with the closest regions having the strongest spetiel corresttion. The ranking from strongest to weskest correlation 467, 443, 418, 482, 488, 417. Option
DeepSeck-R1: R1: The spotial correlation determined the shortest distance from the torget region (442) to each nearby region. Based on the adjacancy date, distances are calculated follows: Region 443 (6. (5.08km), Region (9.0 9..02m 467), Ragion 468 (10. 59km via 443), and Region 417 (13.67km 418 or 443), Ranking from strongest to weskest correiation (shortest longest
distance) gives the order: 443 492 > 468 417, which correspands to option B.
04-mini: By computing shortest path distances in the adjacency 44ph-4442443 (6. ..09 km), 442+467+492 (9.02 km), 442-443-468 (10.5 59 km), and 442-643-417 (13.67 km))ww rank regione by incressing distance (strongest to weakest spetiel correlation) as
492 > 468 417. Option
Figure 9: The LLMs' reasoning on adjacency relation analysis.
24

---

Published as a conference paper at ICLR 2026
Table 12: The spatiotemporal understanding QA for connectivity relation analysis.
Task Description
You are traffic signal control agent a target intersection, managing traffic from four directions: north, south, east, and west. The intersection has 12 lanes, categorized as: NL, NT, NR, SL, ST, SR, EL. ET, ER, WL, WT, and WR. Vehicles at the intersection travel at average speed of 11 meters per second. The real-time traffic conditions for each lane are provided below. If a lane's traffic condition is not specified, indicates that vehicles currently present in that lane.
Data Schema
queue: Number of vehicles waiting at the lane for green signal.
move: Number of vehicles currently moving through the lane.
wait_time: Average wait time (in minutes) for vehicles in the queue. oceupancy: Lane occupancy rate (0 free-flow, 100% fully congested).
connectivity: A list of tuples in the format (lane_1, 1, lane 2, distance), defining directed connection between lanes with the specified distance in m. The traffic flow from lane_ (upstream)
will arrive lane 2 (downstream) in the future.
Data
Target Intersection 112:
SL lane:
queue: 37.0
move: 0
wait_ 2.86
occupancy: 112.12%
ET lane:
queue: 18.0
move: 0
wait 4.26
occupancy; 54.55%
Neighboring Intersection 140:
ST lane:
queue: 4.0
move: 0
wait_ 0.45
occupancy: 12.12%
SL lane:
queue: 3.0
move:
wait_time: 0.41
occupancy: 12.12%
Other Neighbors...
The connectivity of lanes:
(NT of 113, NT of 112, 300m), (EL of 113, NT of 112, 300m),
(WR
of 113,
112, 300m)
(NT of 113, 112, 300m) (SR
of 84, WR of 112, 300m)
Question
Based on the data provided, select the correct ranking of neighboring upstream lanes of the target intersection by the moving vehicles, ordering them from the highest to the lowest.
A. ST 140 > EL 140 = EL of 112 > other lanes
B. WR of 113 > ET of 84 ET of 112 > other lanes
C. ET of 140 = ST 140 ET of 84 > other lanes
D. WR of 113 ET of 140 SL of 140 > other lanes
25

---

Published as a conference paper at ICLR 2026
Note
Please answer the question using the information given. Let's work this out step-by-step way be that have the right answer. Then provide your final answer within the XMI tag <Answer>A/ Bttpa/////////D//////e/eeeee
Qwen2.5 5-328: The carrect ranking of neighboring upstream lanes of the target intersection by the moving vahicles is WR 113 of 140
D.
DeepSeek-R1: R1: The neighboring upstream lanes with moving vehicles are enelyzed based on the connectivity and move counts. WR
Other Therefore, the correct renking is WR 113 ET of 140 SL of 140 others, which matches option D. Option D.
highest count (5), followed
04-mini: WR has highest flow with upstream lanes have none. Option D.
vehicles, ET of 140 and of 140 ench have moving vahicla (tied), and all other
Figure 10: The LLMs' reasoning connectivity relation analysis.
Table 13: The spatiotemporal understanding QA for duration pattern analysis.
Task Description
are tasked with determining the optimal location for new electric vehicle (EV) charging station to maximize benefits by effectively meeting demand. The candidate locations, along with their expected charging station coverage, demand (number of vehicles), average distance, travel time, and waiting time to existing charging stations, provided below. Data Schema
The number of locations that will be within the influence radius of the charging station after its construction.
coverage:
demand: The number of vehicles requiring charging near the location.
distance: The average distance that vehicles near the location must travel to reach existing charging stations.
waiting time: The average waiting time at nearby existing charging stations. NaN indicates no station nearby and the highest waiting time. charging_ting time: The average charging time at nearby existing charging stations. NaN indicates no station nearby and the highest charging time.
Data
location 1:
coverage: 3.068
demand: 90827.49
distance: 188.79m
waiting time: 0.05h
charging_times 22.14h
location 2:
coverage: 3.059
demand: 105897.67 .26
distance: 197.29m
waiting time: 0.03h
charging time: 13.78h
location 3:
coverage: 3.059
demand: 12443.36
distance: 371.80m
waiting.time 0.00h
charging time: 0.96h
location 4:
coverage: 3.062
demand: 22993.54
distance: 304.35m
waiting time: 0.00h
charging_times 1.01h
Question
Based on the data provided, rank the locations by average charging time at near charging stations, from the longest the shortest.
26

---

Published as a conference paper at ICLR 2026
A.4>1>22
1,>4>>>>>
C.4>
D.1>2>4>3
Note
Please answer the question using the information given. Let's work this out in step-by-step way to be sure that we have the right answer. Then provide your final answer within the XML tag LAnswer>A/B/ / Answer>.
Qwen2.5-32B: The locations are renked by average charging time at nearby charging stations from the longest to the shortest es follows: Location (22.14h) 14h), Location (13.78h), Location (1.01h), and Location (0.95h). Option
DeepSeek-R1: The ranking based everage charging time from longest shortest determined by comparing each location's cherging_time, time. Location has the longest 22.141 14h, followed Locstion 2 (13.78h), then Location (1. (1.01h), and finaily Locstion 4 3, which corresponds to option Option D.
(0.98h). This results the order 4>
04-mini: Location has the longest average charging time (22. 12.14 followed by Location (13.78 then Location 4 (1.01 n), and finelly Locetion (0.96 h). Option D.
Figure 11: The LLMs' reasoning on duration pattern analysis.
Table 14: The spatiotemporal understanding QA for chronological pattern analysis.
Task Description
You will analyze user's historical stay records and predict their visiting patterns. Data Schema
History: The user's historical travel sequence over the past 30 days.
Recent: The user's recent visit sequence.
trajectory: A list of tuples (POI, weekday, time), representing user visits.
adjacency: A list of tuples (POL_I, POI_2. distance), representing spatial relationships between POIs.
Data
History:
Day 1:
trajectory: [(Train Station 152, Thursday, 08:10 PM)]
adjacency: N/A
Day 2:
trajectory: [(Train Station 152, Friday, 10:02 AM), (Building 20014, Friday, 06:01 PM)] adjacency: N/A
Day 3:
trajectory: [(Train Station 152, Monday, 10:01 AM), (Building 20014, Monday, 10:13 AM), (Coffee Shop 14357, Monday, 06:37 PM)]
adjacency: (Building 20014, Train Station 152, 773. (Building 20014, Coffee Shop 14357, 2886. 0m)
Day 30:
trajectory: [(Train Station 152, Monday, 09:58 AM)]
adjacency: N/A
Question
Based the user's historical stays, identify the most likely pattern in their behaviors:
A. The user usually goes to the Coffee Shop after Building.
B. The usually goes to the Building after Building.
C. The user usually goes to the Train Station after Train Station.
D. The user usually goes the Train Station after Building. Note
Please answer the question using the information given. Let's work this step-by-step way to be sure that we have the right answer. Then provide your final answer within the XML tag <Answer>A Dt/ts//////////>>.....///////// Answer>.
27

---

Days 15, Published as a conference paper at ICLR 2026
Qwen22.-32B: The user frequently transitions from Building 20014 to Train Station 152. Option D.
DeepSeek-R1: frequently
Building 20014
pettern
supported by adjacency data showing theee POls are 773m apart. While other patterns (like seme-POI visits) exiat, the Building-to-Triin sequence the consistent spatially logical
Option
04-mini: While Train-Train and Building-Train both occur 4 times, the conditional probability of going to a Train Station after Buildding after Station visit (4 aut 14). Option
visit out of 7) is much higher than going to another
Figure 12: The LLMs' reasoning on chronological pattern analysis.
Table 15: The spatiotemporal understanding QA for trend pattern analysis.
Task Description
You are analyzing traffic congestion road network of Beijing, China. Given target road and its nearby neighboring roads, along with time-stamped traffic conditions for each, perform congestion analysis.
Data Schema
Target road: The road for which congestion patterns need be analyzed.
Nearby roads: Roads within up to two hops from the target road.
speed Speed limit on the road, with levels from I (lowest) to 4 (highest).
congestion level: A series of congestion levels (from 0 to 4, where is the highest) recorded at specific times during the day.
Historical time-series data of congestion levels over the past three days, recorded specific time intervals.
congestion Level jin paasttaadaaa
Connectivity: A list of tuples in the format (road 1, road 2, distance), defining directed connec- tions between roads with the specified distance in meters.
Data
Target road:
road 33574:
speed limit:
congestion level: [(4, '17:00"), (4, '17:05"), (4, '17:10)) (4, '17:15)), (4, '17:20") (4, '17:25"), (4, 17:30"), (3, 17:35'), (3, '17:40'), (3, '17:45") (2, '17:50"), (3, '17:55")
congestion levellonneeeeeiinn letpsttonnneeee__..asttt...... .as.......... [(NNAA, '00:00"), (NN/A,, '01:00"), ... (1, '22:00"), (1, '23:00')]
Nearby roads:
road 27810:
-speed_limit,
-cnngsstio level: [(1, '17:00), (1, '17:05"), (1, '17:10)) (1, '17:15), (1, '17220) (1, '17:25)), (1, 11::3),)), (1, 17:35'), (1, '17:40'), (1, '17:45"), (1, '17:50"), (1, 117:55)]
Connectivity of roads: [(22598, 22901, 500m), (27810, 27809, 500m), (33574, 27810, 500m), (33574, 28036, 500m), (28036, 28035, 500m), (22901, 33574, 500m)]
Question
Analyze the traffic conditions for target road 33574, What is the overall trend in traffic congestion?
A. No trend
B. Increase
C. Decrease
D. No trend first, then increase
No trend first, then decrease
F. Increase first, then no trend
G. Decrease first, then no trend
H. Increase first, then decrease
I. Decrease first, then increase Note
Please answer the question using the information given. Let's work this out step-by-step way to be sure that we have the right answer. Then provide your final answer within the XML tag /At
28

---

Published as a conference paper at ICLR 2026
X Qwen2.5-328: The congestion levels for road 35574 show e peak in the late atternoon, followed by slight decrease, Historical data indicates increase congestion levels during late afterncon early evening. Nearby roads show varying congestion, with some experiencing high congestion levels. Option F.
DeepSeek-R11 target 33574 showe congestion level 4 (highest) consistently from
17:30, indicating
Starting at 1 77:5, congestion decreases to level and fluctuates slightly (briefly dropping at 17:50 before rising back to at 17:55). pottern
from the initial peak, aligning option 'No first, then decrease:
X 04-mini: Congestion on road 33574 is stable at high level from 17:00-17:30, then shows cleer decreasing trend between 17:30- Option
17:50, followed by a
(increase)
Figure 13: The LLMs' reasoning on trend pattern analysis. Table 16: The spatiotemporal understanding QA for local extrema analysis.
Task Description
You are analyzing traffic congestion in road network of Beijing, China. Given target road and its nearby neighboring roads, along with time-stamped traffic conditions for each, perform a congestion analysis.
Data Schema
Target road: The road for which congestion patterns need to be analyzed.
Nearby roads: Roads within up to two hops from the target road. speed_limit Speed limit on the road, with levels from I (lowest) to 4 (highest).
congestion_ level: A series of congestion levels (from 0 to 4, where is the highest) recorded at specific times during the day.
congestion Lenngestionneve_liiie Jewee_nn.aatttt333 3_dys:: Historical time-series data of congestion levels over the past three days, recorded at specific time intervals.
Connectivity: A list of tuples in the format (road 1, road 2, distance), defining directed connec- tions between roads with the specified distance in meters.
Data
Target road:
road 27303:
spped_limiis
congestion level: '06:00'), '06:050), (6,1, 00000000000000000000000000000000
(1, '06:20'), (1, '06:25'),
(1, '06:30"), (1, '06:35'), (2, '06:40'), (2, (,
congestion level in past days: 01,11000000000000000000000000000 (1,300)) (0..000) '04:00"),
(1, '05:00"), (1,
(1, ",,,,,0000000..8,,,000000000,110
(1,
(1,
(,,),),,,,""".0))),,.000000,,,, ",,,,, ",,,0000000000000003330000000,,1 (1,
(1, (04:00)) (1.50000 (1, '06:00'), (1,000))) (08.000, (1, '09:00'), (1, (, ",000 (1, '17:00"), (2.
(2:000") (12.00),
(1, '21:00'), (1,
'23:00')]
Nearby roads:
road 33889: speed limit 4, congestion.leves [(1, 'o0:'00:)) '6::55)]
(2.6005555))
Connectivity of roads:
[(32166 6, 27371, 500m), Question
(27371, 27370, 500m)
Based the data, identify one of the typical time periods in the past 3 days when the traffic of the target road 27303 reaches its peak.
A. 18:00-20:00
B. 00:00-02:00
C. 16:00-18:00
D. 06:00-08:00
Note
Please answer the question using the information given. Let's work this out step-by-step way to be that have the right answer. Then provide your final answer within the XML tag <Answer>a/B/c// /ttpa/////////////se......
29

---

Published as a conference paper at ICLR 2026
Qwen2.6-328:
target 27303 reaches peak during 18:00-20:00 period, as congestion highest
during this time compared to the other periods. Option A.
DeepSeek-R1: target read 27303's congestion. Leveeeieeeeeeee _i_ppet______ days first two days and 18:00 en the third day. The 18:00-20:00 window (Option A) captures these peak times most consistently across the days, making the typical peak period. Option
peaks 2] occurring 19:00-20:00
o4-mini: Over the paet three days, the target road's congestion level of occurs most often during the 18:00-20:00 windww:wwwwwwwwwooooooooooooowwo window (4 out of readings), more then any other candidate
Option
Figure 14: The LLMs' reasoning on local extrema analysis.
Table 17: The spatiotemporal understanding QA for periodicity pattern analysis.
Task Description
You are analyzing traffic congestion in the road network of Beijing, China. Given a target road and its nearby neighboring roads, along with time- stampst traffic conditions for each, perform congestion analysis.
Data Schema
Target road: The road for which congestion patterns need to be analyzed.
Nearby roads: Roads within up two hops from the target road.
speed limit Speed limit on the road, with levels from I (lowest) to 4 (highest).
congestion level: A series of congestion levels (from 0 to 4, where 4 is the highest) recorded specific times during the day.
congestion level. in past 3 days: Historical time-series data of congestion levels over the past three days, recorded at specific time intervals.
Connectivity: A list of tuples in the format (road_1, road distance), defining directed connec- tions between roads with the specified distance in m.
Data
Target road:
road 21501:
speed limit:
congestion_level. [(2, '22:00)) 122:05)), 22:11'), (2) (1, 22:20"), (1, '22:25"),
(1, 222:30),, (2, 22:355), (2, '22:40)) 22:45"), '22:50"), (1, '22:55")
congestion level in .na.iin.______. 3.dys: [(1, (00:00", (2000),))0.0))0)))..0 '04:00",
(1, '05:00'), (1, '06:00'), (1, '07:00'), '08:00"), (1, (1, (1,10))))
(1,
(1, 11,,:1000000..000000000011111111 112:00'). (1,3000)))
(2,
'2::00'), '00:0000; 0,,,,,
(06:00") 01000000100000000000000000000000
(1, '14:00"), (1,11000) '18:00'). (2:000")) (20:00"), (21:00", 22000222220....2333300000000,,.0 (1,,0)))
'05:00'), (1, '06:00'), '07:00") (4 (2, 17:00"), (2, (1, (12:00")
(1,
'09:00'), (1, (1
(14:0),, "111000000.....00000000000000010
'13:001), (1,3300))
Nearby roads:
Connectivity of roads:
Question
Based on the data, identify the most possible periodicity of congestion changes the target road 21501 over the past? 3 days.
A. 29 hours
B. 17 hours
C. 24 hours
D. 32 hours
Note
Please answer the question using the information given. Let's work this out in step-by-step way be sure that we have the right answer. Then provide your final answer within the XML tag / BB/////CDD/ Bttp/////////D//////.// Answer>.
30

---

Published as a conference paper at ICLR 2026
Qwen2. S-328: The congestion levels on the target 21501 repeat every 24 hours, inclcating a clear 24-hour periodicity. Option C. DeepSeek-RR R1: The congestion levels road 21301 show recurring peaks at 17:00 (5 PM) each day, aligning with typical evening rush These patterne repest daily, indicating 24-hour periodicity corresponding to daily traffic cycles. The 24-hour eycle (Option C) the most plausible es it matches common diurnal traific patterns.
Additionally, morning peaks
appear Days
04-mini: The strong evening congestion surge (levels rising to 2) recurs at the same local time each day, indicating a 24 hour cycle. Option
Figure 15: The LLMs' reasoning on periodicity pattern analysis.
Table 18: The forecasting QA.
Task Description
You are analyzing traffic congestion road network of Beijing, China. Given target road and its nearby neighboring roads, along with time-stamped traffic conditions for each, perform congestion analysis.
Data Schema
Target road: The road for which congestion patterns need to be analyzed.
Nearby roads: Roads within up two hops from the target road.
speed limit: Speed limit on the road, with levels from I (lowest) to 4 (highest).
congestion level: series of congestion levels (from 0 to 4, where is the highest) recorded specific times during the day.
congestion Levell_neeeennnee _in_past_3 3ddys: Historical time-series data congestion levels over the past three days, recorded at specific time intervals.
Connectivity: A list of tuples in the format (road 1, road 2, distance), defining directed connec- tions between roads with the specified distance in m.
Data
Target road:
road 28340:
speed_limit:
congestion.level. [(1, (1::0'),, (4, 11::1199110000)) (L,
119:20",, (1, '19:25',
(L, "19:30",, (1, 19:35'), (1, '1::40), (1.4) 124 (1::0")),, 10:1)))) 19:55') congeettion.level.inn wtw.eeeelliiiptt..daaaa..
'14:00"), 110000000000000000000000000000,0 (03:00)) (04000" '04:00',,
(1. '05:000),, (0,6000')
(1, '0::00'), (2, '07:00"), '08:00'), (2, '09:00')
(1,10) (1,
(1, (1, (1 (23:00",, (1, "00:000); (1010))) (1,
'04:00") 16:00,, 1,060000220000000000000000000000 (2, '14:00'), (2, (2, '15:00'), (L,00))
(2,
(1,,2.0)) (04:00")) '05:00"), (1, '06:00"), (2, '07:00'), '08:00'), (1,
(1,000, 14,000, '15:00") 15: '16:00", (2, 11,:0"),, (1.118.00) (2,0000 (1, 222,00,, (1,
(1,
Nearby roads:
road 22668:
speed limit:
congestion level: [(1,'19:00 '19:00), (1, '1::05)), (1, '19:10', (1, '19:15',, (1, '19:20'), (1, '1::25'),
(1, '19:30',, (1, 19::55", (1, '19:40'), (1, '19:45'), (1, '19:50') (1, '19:55")
road 27874:
-speed Jimit: congestion level: [(1, '19:00") (1, '19:05"), (1, '19:10', (1, '19:15"), (1, '19:20'), (1, '19:25),
(1, '19:30"), (1, '19:35"), (1, '19:40'), (1, '19:45"), (1, '19:50',, (2, '19:55")]
road 27875:
speed limit: congestion__evel [(1,'19:000 '19:00'), (1, '19:05"), (1, '1::10', (1, '19:15', (1, '19:20)) (1, '19:25"), (1, "19:30'), ) (1, 19:35'), (1, '19:40", (1, '19:45') (1, '19:50"), (2. '19:55"]
Connectivity of roads:
(27874, 28340, 500m), (27875, 27874, 500m), (28340, 22668,
50Cm) ]
Question
Based on the traffic conditions of the target road and its nearby roads, what is the most possible traffic congestion level of the target road 28340 at the next time step?
31

---

Published as a conference paper at ICLR 2026
A.
B.2
C.3
D.4
Note
Let's solve this step by step. Finally, summarize your analysis, and provide your answer in JSON format, like:
"summary": SUMMARY", "answer": "A/B/C/D"
(wen2.5-328: The current congestion levels the target 28340 and nearby roads mostly slight increase the laet few minutes. The historical data suggests that the congestion level tends to incresse slightly during certain times the day, historical pattern,
ramains relatively Given time step is 2. Option B.
congestion
likely congestion for
roads 27874 and 27875 also reached congestion level at 19:55, suggesting sustaiind or increasing traffic inflow. Historical data shows upstream
DeepSeck.R1: The
read 26340's congestion level increased to 19:50 and remained 19:55. Nearby
congestion typically drops
indicate congestion likely
level 2 in the next time step. Option B.
04-minl: After initial from 27874), the congestion likely to remein at level for the next time step. Option
19:50-19:55 road 2834D, saniing
upstream neighbor (read
Figure 16: The LLMs' reasoning on forecasting.
Table 19: The reflection QA.
Task Description
Design road network connect unconnected regions to the existing connected regions in urban area. Using the provided data, identify the roads that can be built to maximize connectivity.
Data Schema
connected_regionss A list of regions that already connected.
connectivity: A list of tuples in the format (node_.. node 2, distance), representing undirected roads between nodes with the specified distance (in km).
unconnected regions: A list of regions that are currently not connected to any other region.
region nodes: A list of boundary nodes defining the extent of each region.
available roads: A list of roads that can be built to improve connectivity.
road edge: A tuple in the format (node 1, node 2, distance), representing an undirected road that can be constructed between two nodes with the specified distance (in km).
new_roads list tuples in the format (node_1, node -2, distance), representing new roads that can be constructed after building the previous road.
Data
coanected_regionns region
region nodes: [3, 2, 1]
region 26:
"region_nodes, [4, 1, 14, 0, 13]
connectivity:
(node 1, node 3, 0. 8km) (node 0, node 1, 0. 17km), 44, node 45, 0. 0.36km)
(node
uncomnected.regions::
region 4:
region_nodes: [7, 9, 17, region 15:
rregion [33, 26, 35, 31, 32]
region 16:
region nodes: [17, 2,, 16, 21)
available.roads:
32

---

Published as a conference paper at ICLR 2026
road 7:
road edge: (node 3, node 5, 0.31
new proads:: [(node 16, node 5, 0.31km))
road 20:
road_edgee (node 17, node 9, 0.14km)
new wow_ro [(node 22, node 17, 0.1 0.17km)]
road 30:
road edge: (node 15, node 19, 0.52km)
new roads: [node 19, node 25, 0.24km))
Question
Based on the provided data, identify the most suitable road should be built next to connect the largest number of unconnected regions.
A. road
B. road 72
C. road 84
D. road 81
Note
Let's solve this step by step. Finally, summarize your analysis, and provide your answer in JSON format, like:
"summary": "YOUR SUMMARY",
"answer"; "A/B/C/D"
Qwen2.5-32B:
largest number of unconnected regions (3 regions) according the anelysis, While higher initial connectivity impact the potantial unlock road (node 16 to 5) make it the optimel choice for maximizing immediste connectivity gains, Option A.
regions regions) choica
connectivity. Option
DeepSeek-R1: Road (option A] connects the largest
number reglons,
like
48,
04-mini: Building road (3 W immediately connects unconnected regions 23, and unlocks road connact region 16-three regions in total, more than any other candidate, Option A.
Figure 17: The LLMs' reasoning planning.
Table 20: The spatiotemporal understanding QA for periodicity pattern analysis.
Task Deseription
We are tasked with determining the optimal location for new electric vehicle (EV) charging station to maximize benefits by effectively meeting demand. The candidate locations, along with their expected charging station coverage, demand (number of vehicles), average distance, travel time, and waiting time to existing charging stations, are provided below.
Data Schema
coverage: The number of locations that will be within the influence radius the charging station after its construction.
demand: The number of vehicles requiring charging near the location.
distance: The average distance that vehicles near the location travel reach existing charging stations.
travel.time: The average travel time for vehicles near the location reach existing charging stations.
The average waiting time nearby existing charging stations. NaN indicates no station nearby and the highest waiting time.
waiting time:
charging time: The average charging time at nearby existing charging stations. NaN indicates no station nearby and the highest charging time.
Data
location 1:
coverage: 3.573
demand: 22734.79
distance: 284.90m
33

---

Published as a conference paper at ICLR 2026
waitinggttin time: 0.00h charging_time: 2.38h
location 2:
location 3:
location 4:
Analysis
Coverage ranking: location (3.573) location (3.572) location (3.571) location 2 (3.571)
Demand ranking:
location 3 (121545.93) location (81328.63) location 4 (27670.77) location (22734.79)
Distance ranking:
location (284.90m) location (158.28m) location (152.72m) location 2 (151.30m)
Waiting time ranking:
location (0.04h) location (0.04h) location (0.01h) location (0.00h)
Charging time ranking: location (21.38h) > location 2 (17.03h) location 4 (8.51h) location (2.38h)
Question
Evaluate each candidate location based on the provided data. Identify which location would deliver the most substantial benefits for an EV charging station, focusing on enhancing overall station coverage.
A. location
B. location 2
C. location 3
D. location
Decision
Reason
Location offers the highest demand, which is critical for the success and utilization of the EV charging station, making it the most beneficial location despite location I's higher coverage. Environment Feedback
Station coverage increased by 181.05%
Charging time increased by 7.15%
Travel time increased by 59.22%
Waiting time increased by 83.68%
Note
Based on environment changes caused by the prediction, evaluate the correctness of the prediction and identify any weaknesses.
Verify if the Decision and its reasoning are accurate.
If the Decision is incorrect inaccurate, propose better alternative with clear rationale.
Let's solve this step by step. Finally, summarize your analysis, and provide your answer in JSON format, like:
"summary": "YOUR_SUMAARY",
"answer": "A/B/C/D"
34

---

Published as a conference paper at ICLR 2026
X Qwen2.5- 5-328: Location has highest demand, which is critical for the success of the EV charging station, Despite the high demand makes most beneficial location. However, improvements in charging and waiting time are recommended. Option C.
charging time and waiting
DeepSeek--t: The original decision selected location due its high demand. However, the environment
after building there reveal significant increnses waiting time (83.68%), charging time (7.1556), and travel time (59. 22%),, which undermine the benefits high demand. While coverage
negetive impects station may not effectively meet demand. Location 1, with the highest initial coverage, better existing metrica (lowest waiting/charging congestion, provide
substantially
experience suggest
times),
severe trade
Option A.
demand, lowest coverage times. Location offers the highest coverage plus zero waiting and the fastest charggtn yielding better network performance and user experience. Option A.
significant
waiting
Figure 18: The LLMs' reasoning on reflection.
Road Planning
POI Placement
But the question about connecting the lorgest mumber of uncormected regions. So, buildin any of mode that ocnnect non unsormeccie reg one would betw than huilding most But options givon
Atternttieely maybe the anaigais noeds consider the trade-offfa bewweon these bermos demannd is highher prisity man location
mreeriion quality
importare bink, then locatio better coverage key
coolt
better
read wit cornects
Thereforn, the mosc suhaable to build nex oither read 64, o1 Bus the question to choese from the given options: road 7, roed 72, road 64, road
But I'm not omnieey suro. Maybe ineed to on the dona more corefully Blyfully Let's Lecation Covenge 8.690...
But question about connecting the largest number uncornected regions, building
maybe anaiyeii
But
domandd tiehighhstt importans, then location bi better
factor, the tocation
So, mnongg thess, roads 72, 84, 61 cooh oonnect two unoon cotod ons, whinh bettor
1s bettrr then Zndd worte 4.
than mad on cornects one.
Thenelors, is the in autabble road to build neet either raad 72, 84, cr But the question to choose from the given optioner road 7, B. medd 84, madi81
But I'm not entirely sure. Mayoe insd to locker the duta more canetu Let's
Location
-overagge 3.000..
Figure The repetition issues of D.7.4 LLM AGENT PROMPT
RI-Distill-Qwen-
Table 21: Task-solving prompt template.
Data Schema
<data.schema>
Domain Knowledge
Data
<data.text>
Question
Based on the provided task description, data schema, domain knowledge, provided data, and experience, you may first perform spatiotemporal data analysis. Then, <task.target>,
Experience
<experience>
Note
Let's work this out in a step-by-step way to be that we have the right answer. Finally, provide your answer and summarize your analysis JSON format, like: JSON
" "answer": "summary":
"YOUR_SUMMARY"
35

---

Published as a conference paper at ICLR 2026
Table 22: Reflection prompt template.
Data Schema
<data.schema>
Domain Knowledge
<domain (domain.knowledge>
Data
<data_text>
Question
Using the provided task description, data schema, domain knowledge, and data analysis, <task target>.
You need to:
Evaluate whether the previous <deci s wwwisooooorrrreeeeeeeeee ction> correct, based the environmental feedback.
Identify any reasoning flaws or weaknesses in the original output.
If the justification.
was incorrect, propose a better alternative with clear
Conclude with high-level lesson learned. This summary must not include specific variable values or data IDs.
Experience
<experience>
Previous Action
<ddcission.or.preeccttinn
Reason
Environmental Feedback
<env.feedback>
Note
Let's work this out in a step-by-step way to ensure we have the right answer. Finally, provide your reflection ISON format as follows:
JSON
"is-correct"; "YES/NO"
"answer": "YOUR_ANSWER"
"experience": "YOURRSSUMMLLLEEEEEEEEEEEE
36

---

Published as a conference paper at ICLR 2026
Table 23: Summary prompt template.
Data Schema
<data.schema>
Domain Knowledge
deomain_knowweee
Data
<data text>
Question
Based on the task description, data schema, domain knowledge, and data sample, consolidate all experiences into smemory_num summarized experiences.
Experience
<old.experience>
<new.exoerience>
Note
Let's work this out step-by-step. Combine and restructure the information create concise and meaningful summaries. Finally, provide your answer in JSON format as follows: JSON
"YOUR EXPER TENCE. 1" "YOUR_EXPERTER CEEEEPERRRENCCE... ENCE_2"
37

---

Published as a conference paper at ICLR 2026
D.7.5 SPATIOTEMPORAL UNDERSTANDING POST-TRAINING INSTRUCTION EXAMPLES
Table 24: The spatiotemporal understanding post-training instruction for distance analysis.
Task Description
A city is planning to increase green spaces and needs to consides distances from the new park to residential areas.
Data Schema
distance: The distance to the park in km.
Data
neighborhood 0:
distance: 6km
neighborhood 1:
distance: 8. 89km
neighborhood 2:
distance: 3. 86km
neighborhood 3:
distance: 16. 1 9km
neighborhood 4:
distance: 6. 7km
neighborhood 5:
distance: 7. 9mk6
neighborhood 6:
distance: 10. 1 km
neighborhood 7:
distance: 13. 97km
neighborhood 8:
distance: 8. 39km
neighborhood
distance: 7.08km
neighborhood 10:
distance: 6. 85km
Question
How does neighborhood compare to neighborhood 3? Which is closer to the park?
A. neighborhood
B. neighborhood
Note
Let's work this out in a step-by-step way to sure that we have the right answer. Finally, provide your answer and summarize your analysis in JSON format, like:
aY JSON
"answer": "A/B"
Response <think>
Okay, let's see. The question is asking how neighborhood compares to neighborhood in terms of their distance the new park, and which one is closer. The options are for neighborhood and B for neighborhood 3.
38

---

Published as a conference paper at ICLR 2026
First, I need to look up the distances for both neighborhoods. From the data provided:
Neighborhood has a distance of 16.19 km.
Neighborhood has distance of 7.08 km.
So, comparing these two numbers, 7.08 km is definitely less than 16.19 km. That means neighbor- hood is closer to the park than neighborhood 3. Therefore, the answer should be A. Wait, let me double-check to make sure didn't 't mix up the numbers. Neighborhood 3 is over 16 km away, which is quite far, while neighborhood 9 is just over km. Yep, that's correct. The smaller the distance number, the closer they are. So neighborhood 9 is closer. The answer is A. </think>
"answer": "A"
Table 25: The spatiotemporal understanding post-training instruction for adjacency analysis.
Task Description
To enhance pedestrian safety, evaluate the number of crosswalk 1kks near schools and their relationship to traffic volume.
Data Schema
connectivity: Each tuple (entity 1, entity 2, distance) defines rected edge from entity to entity 2 with the specified distance in m.
Data
(schooo 1, crosswalk 1623m) (school 1, school 98m) (school 1, school 4, 1428m), (crosswalk 2, school 4, 1995m)
Question
Among school and crosswalk 3, which further from school 5?
Note
Let's work this out in step-by-step way be sure that have the right answer. Finally, summarize your analysis with JSON format, like:
'JSO
"summary"; "YOUR_SUMMARY"
..
Response
<think>
Okay, let's see. The task is to find which of school or crosswalk 3 has a stronger spatial correlation with school 5. The data given is the connectivity between entities, which includes schools and crosswalks.
First, the definition says that the connectivity is undirected edge between two entities with distance. So each tuple represents how those two entities are connected and the distance. I need to look at the data provided and see how school 5 connected these entities. Looking at the data entries: (school 1, crosswalk 3, 1623m) (school 1. school 5, 98m) (school 1, school 4, 1428m) (crosswalk school 4, 1995m)
School 5 is only mentioned once here, in the second tuple. So school 5 is directly connected to school with a very short distance of 98 meters. That's really close. Now, crosswalk is connected to school I but in the same tuple school I: (school I, crosswalk 3, 1623m). So the distance here is 1623 meters, which is much farther.
But how does crosswalk 3 relate school 5? Since crosswalk 3 is connected to school and school is connected to school So there's an indirect path from crosswalk to school 5 through
39

---

Published as a conference paper at ICLR 2026
school 1. The total distance would be 1623m 98m 1721m. Or is it considered as the minimal path? Either way, compared to school I's direct 98m to school 5, crosswalk much farther away.
Spatial correlation here likely depends on proximity. The closer two entities are, the stronger their spatial correlation. So since school is extremely close school 5 (98m), while crosswalk 3 is over 1.6 km away from school and hence over 1.7 km from school 5, school would have a stronger spatial correlation with school
Crosswalk 3 is not directly connected to school 5 in the data. The only connection is through school But even then, the total distance is way larger. The data doesn't 't have any other connections from crosswalk 3 to other entities except school 1. Therefore, school is directly adjacent and very close, making the stronger candidate.
Wait, but the problem is about which (school 1 or crosswalk 3) is more correlated spatially with school 5. Since school directly connected with school 5 at 98m, that's a direct and short link. Crosswalk is connected to school (1623m away) which is connected to school 5 (98m). But crosswalk itself isn't directly linked to school So spatial correlation would depend on distance between entities. School 1 is right next to school 5, making their spatial correlation strong. Crosswalk is much further away. So the answer should be school 1. </think>
"answer": "school 1",
"summary"; "School 1 has stronger spatial correlation with school 5 because it is directly connected to school 5 via a very short edge (98 m). Crosswalk 3 is connected to school but is much farther away (1623 m) and has no direct connection to school 5. Proximity strongly influences spatial correlation, and school I's adjacency to school 5 outweighs crosswalk 3's indirect, distant relationship."
Table 26: The spatiotemporal understanding post-training instruction for connectivity analysis.
Task Description
To assess the effectiveness of emergency response, evaluate the accessibility of fire stations to densely populated areas.
Data Schema
connectivity: Each tuple (entity 1, entity 2, distance) defines an undirected edge from entity to entity 2 with the specified distance in km.
Data
[dennsely populated area 1, densely populated area 7, 5km),
(densely populated area 1, fire station 3, 19km), (densely populated area 1, densely populated area 4, 6km), (densely populated area 1, densely populated area 6, 12km), (densely populated area 1, fire station 2, 4km), (fire station 2, densely populated area 5, 9km), (fire station 2, fire station 3, 1km), (fire station 2, densely populated area 6, 12km), (fire station 3, densely populated area 6, 10km), (fire station 3, densely populated area 4, 14km), (fire station 3, densely populated area 5, 8km), (fire station 3, densely populated area 7, 9km), (densely populated area 4, densely populated area 7, 15km), (densely populated area 5, densely populated area 7, 11km) Question
fire station 2 reachable from densely populated area 5?
Note
Let's work this out in a step-by-step way to be sure that we have the right answer. Finally, provide your answer and summarize your analysis in ISON format, like:
40

---

Published as a conference paper at ICLR 2026
JJJON
"answer": "Yes/No",
"summary": "YOUR SUMMARY"
<think>
Okay, let try to figure out if fire station is reachable from densely populated 5. need see if there's a path connecting them through the given edges. Let's look at the data carefully. First, let's list out all the connections provided. Each tuple an undirected edge, so movement can both ways. The nodes include densely populated areas (like areal, area7, etc.) and fire stations (like fire station 2 and 3).
Starting from densely populated area5. Let's see what connections it has. Looking through the list, I spot (fire station3, densely populated area5, 8km). Oh, right, area5 is connected to fire station3 with distance. Additionally, there's 's (fire station2, densely populated area5, 9km). Wait, is that a direct connection between fire station2 and area5? Let me check again. Yes, the tuple (fire station2, densely populated area5, 9km) exists. So if area5 is directly connected fire station2, then the answer should be yes. But wait, let me confirm if that connection is indeed there. Looking back at the data provided:
The tuples are as follows:
(fire station2, densely populated area5, 9km) one of them. So that means area5 directly connected to fire station2 with 9km. Therefore, there's direct path. No need to go through other nodes. Thus, area5 can reach fire station2 directly. So the answer should be Yes. < </think>
"answer"; "Yes", "summary"; "Densely populated area 5 is directly connected to fire station 2 via 9km edge, as per the connectivity data provided.
Table 27: The spatiotemporal understanding post-training instruction for duration analysis.
Task Description
Analyze the duration of power outages in industrial zones to develop strategies for improving grid reliability.
Data Schema
duration:
The duration of the power outage curation in h.
Data
power outage duration 0:
duration: 0.1h
power outage duration 1:
duration: 0.53h
power outage duration 2:
duration: 1.94h
power outage duration 3:
duration: 0.82h
power outage duration
duration: 1.91h
power outage curation
duration: 1.18h
power outage duration
duration: 0.8h
Question
Comparing power outage duration 5 and power outage duration 0, which is longer?
41

---

Published as a conference paper at ICLR 2026
Note
Let's work this out in a step-by-step way to be sure that we have the right answer. Finally, provide your answer and summarize your analysis in JSON format, like: JSON
"answer";
"power outage duration 5/power outage duration
<think>
Alright, let's tackle this problem step by step. I need to compare the duration of two specific power outages: number 5 and number 0. The data provided gives the durations in hours, so first I'll list those out.
Looking at the data schema, each entry is labeled "power outage duration X" with correspond- ing duration value. Specifically:
Power outage duration has duration of 0.1 hours. Power outage duration has duration of 1.18 hours. The question asks which of these two is longer. So, just need to compare the two numbers: 1.18h vs. 0.1h.
1.18 hours is clearly larger than 0.1 hours. Therefore, power outage duration is longer than duration O. can confirm by converting to minutes if necessary. 0.1 hours is 6 minutes, while 1.18 hours is about 70.8 minutes. Yep, that's much longer. other data points affect this particular comparison since we are only looking at these two instances. So the answer is straightforward. </think>
"answer":
"power outage duration 5"
Table 28: The spatiotemporal understanding post-training instruction for chronology analysis.
Task Description
You are tasked to analyze a user's historical POI stay records and visiting patterns. Data Schema
event sequence: Each tuple (event, time, weekday) represents a event that happens a specific time and day of the week.
Data
The POI visiting sequence is below:
Day 1:
trajectory: [Bike Shop 4355, Sunday, 06:00 PM), (Convenience Store 2809, Sunday, 07:14 PM) adjacency: N/A
Day 2:
trajectory: (Post Office 37, Monday, 09:54 PM), (Bank 8352, Monday, 10:05 PM), (Food and Drink Shop 6831, Monday, 10:06 PM) ] adjacency: (Bank 8352, Post Office 37, 467. 0m), (Bank 8352, Food and Drink Shop 6831, 117.0m)
3:
trajectory: [CConvnnience Store 2809, Tuesday, 08:15 PM), (Post Office 37, Tuesday, 08:15 PM), (Food and Drink Shop 1827,
Tuesday, 08:15 P
adjacency: [ (Post Office 37, Convenience Store 2809, 731. 0m), (Post Office 37, Food and Drink Shop 1827, 471.0m)
Day 4:
[[Covennience Store 2809, Thursday, 07:50 PM), (Post Office 37, Thursday, 07:58 PM), (Food and Drink Shop 1827, Thursday, 07:59 PM)
trajectory:
adjacency: I (Post Office 37, Convenience Store 2809, 731. 731.0m) (Post office 37, Food and Drink Shop 1827, 471. 7m1.0m) Day 5-30:
42

---

Day is Sunday (06:00 PM and 07:14 PM). Day 6 is Sunday (04:43 PM). Day 11: Sunday (07:20 PM). Day 16: Sunday (12:29 AM and 12:38 AM). Day 30: Sunday (05:35 PM, 06:02 PM). The latest Sunday in the dataset is Day 30, and the times there are 5:35 PM and 6:02 PMM But earlier, for example, Day 11 has an event at 7:20 PM. But that Day 11 is earlier (the 11th day), while Day 30 is the 30th day, so even though the time is earlier on Sunday, the actual date (Day 30) is later. So the Day 30's 6:02 PM is more recent than any entries in previous Sundays, because Day 30 is a later day. Published as a conference paper at ICLR 2026
Question
Which event happened most recently?
Note
Let's work this out in step-by-step way be sure that we have the right answer. Finally, summarize your analysis with JSON format, like:
JSON
"answer": "Train Statiion/Par//Convenleen Store/Government Build building/othie Other Great Outdoors/ "summary": "YOUR_SUMMARY"
...
Response
<think>
Okay, need to figure out which event happens most recently based on the data provided. Let's look through all the days and their trajectory entries. The key here is to find the latest time on the latest date.
First, I should check each day's trajectory to see the time and day of the week. All days are numbered from Day 1 to Day 30. Since each day is likely consecutive, Day 30 would be the last one in the sequence. Let me confirm with the data given. Looking at Day 30:
The trajectory is [(Train Station 274, Sunday, 05:35 PM), (Miscellaneous Shop 13944, Sunday, 06:02 PM)]
So on Sunday at 06:02 PM, the user visited Miscellaneous Shop 13944.
But wait, are there later days? No, the days go up Day 30, so that's the latest day in the dataset. Now, looking at the entries on Day 30, the events are at 5:35 PM and 6:02 PM on Sunday.
Are there any other entries in later days with later time? Let me scan through the other days just to make sure. Let's check Day 29, 28, etc. to see if any times are later in the same day. Day 29 has trajectory entry on Saturday 04:58 AM. That's way earlier. Day 28: Friday, entries up to 07:03 PM. But Saturday would come before Sunday (the next day), so Day 30 definitely after Day 28 and 29.
Double-check the times on Day 30: the latest time there is 6:02 PM (18:02) on Sunday. Miscella- neous Shop 13944 is the last entry here.
Are there any events on Day 30 that are later than 6:02 PM? The trajectory has two entries: 5:35 PM and 6:02 PM. The latest 06:02 PM. That's the most recent time in the data. But wait, maybe another day with more recent time? For example, maybe day that's also Sunday but with a later time. Let me check all other Sundays.
Looking through the data:
Therefore, the most recent event is Miscellaneous Shop 13944 06:02 PM on Sunday, So the answer would be "Miscellaneous Shop".
Wait, in the data entry for Day 30, each event in the trajectory is listed with time. Both entries are on Sunday. The Miscellaneous Shop is the later time. So yes, that's the most recent event in the entire dataset.
</think>
"answer": "Miscellaneous Shop"
43

---

Published as a conference paper at ICLR 2026
Table 29: The spatiotemporal understanding post-training instruction for trend analysis.
Task Description
You are tasked to analyze the time series data of solar energy generation power (in MW).
Data Schema
Each tuple (value, time) represents data reading in MW recorded at the specified time.
time-series:
Data
[(30.5M)) 12:00:00), 25MW, 12:10:00), 199.8MM, 12:20:00),
(29. 7MW, 12:30:00), 459.4 12:40:00), (28.2M)) 2MN, 12:50:00),
(27.05MM) 13:00:00), (26. 65MW, 13:10:00), (26. 4MW, 13:20:00),
(25. 9MW, 13:30:00), 35M.3 13:40:00), (24. 6MW, 13:50:00)]
Question
How is the trend of solar energy generation power (in MW)? First, examine the entire 12-step time series to determine the overall trend. If no clear trend emerges, divide the analysis into two segments of six steps each and assess trends separately for each half.
Experience
Let's work this out in step-by-step way to be sure that we have the right answer.
Note
Let's work this out a step-by-step way to be sure that we have the right answer. Finally, summarize your analysis with JSON format, like:
JSON
trend first, then nccresse/oo trend first, then decrease/ Increase first, then no trend/Decrease first, then no Srend/Increast first, then decrease/Decreat first, then increase", "summary"; SUMMRRY
"answer": "No (trend/tncrease/Derree
Response
<think>
Okay, let's tackle this problem step by step. The task is analyze the time series data of solar energy generation power in MW every 10 minutes from 12:00 to 13:50. The question is asking about the wend-whether it's increasing, decreasing, if there's clear trend. If the overall trend isn't 't clear, we need to split the data into two halves and analyze each separately. First, I'll look at the entire dataset. The timestamps start at 12:00 and go up 13:50, making 12 data points, each 10 minutes apart. Let's list the values in order: 30.5, 30.25, 29.8, 29.7, 29.45, 28.2, 27.05, 26.65, 26.4. 25.9, 25.35, 24.6 MW. Visually examining these values, each subsequent reading lower than the previous one. Let's check: From 30.5 to 30.25 (slight decrease), then 29.8, 29.7, 29.45... all the way down 24.6. Each step shows reduction. There's consistent downward trend without any increases. So the overall trend might be decrease.
But wait, the instruction says to first check the entire 12-step series. Then, if no clear trend, split into two halves. Let me confirm each value again. Starting with 30.5 at 12:00, every next step is lower. For example: Step 1: 30.5 to 30.25 (down) Step 2: 30.5 +30.25 -29.8 +29.7 -29.45 -28.2 (each subsequent is lower). Then from 28.2 to 27.05, down to 24.6. So, all 12 points steadily decreasing. So the overall trend for the entire period is a decrease.
Wait, no-II might need to check if the rate is consistent. For example, is the decrease steady across all data points? Let's list the values again:
44

---

Published as a conference paper at ICLR 2026
1. 30.5 2. 30.25 (-0.25) 3. 29.8 (-0.45) 4. 29.7 (-0.1) 5. 29.45 (-0.25) 6. 28.2 (-1.25) 7. 27.05 (-1.15) 8. 26.65 (-0.4) 9. 26.4 (-0.25) 10. 25.9 (-0.5) 11. 25.35 (-0.55) 12. 24.6 (-0.75) the rate of decrease varies. For instance, steps 6 and 7 have larger drops (like -1.25 and -1.15). But even then, every step is lower than the preceding one. So regardless of how much it's decreasing, the trend is consistently downward.
Wait, but the first half (first six steps) shows downward trend, and the second half (last six steps) also shows a downward trend. So both halves individually have decreasing trend. That means the entire time series has consistent decreasing trend. Therefore, the overall answer should be a decrease.
But wait- the problem says if the entire trend is unclear, split into two. But in this case, all points are decreasing. So trend clearly decrease. Hence, the answer should be a single decreasing trend for the entire period. Therefore, the summary would state that the solar generation power shows a consistent decrease across all 12 steps with no periods of increase, leading to an overall downward trend. </think>
"answer": "Decrease",
"summary": "The solar energy generation power shows a consistent decreasing trend across all 12 time steps. Starting at 30.5 MW 12:00:C0, the values decline steadily to 24.6
MW by 13:50:00, with each subsequent measurement being lower than the previous one. The rate of decrease varies slightly but remains unidirectional data also exhibit decreasing trends, reinforcing the conclusion of an overall decline."
Both the first and second halves of the
Table 30: The spatiotemporal understanding post-training instruction for local extrema analysis.
Task Description
You are tasked analyze the time series data of traffic flow.
Data Schema
time series:rres: Each tuple (value, time) represents data
reading recorded at the specified time.
Data
(336 116666666666666656,,,, 2018-02-03 00:00:00), (19.83333333332
2018-02-03 01:00:00), (40
2018-02-03 0 02:00:00),
(19. 9993333333332, 2018-02-03 03:00:001 (36.9166666666666)
2018-02-03 04:00:00), (67 (67.0833333333333
2018-02-03 05:00:00),
(106. 0006166666666666677 2018-02-03 06:00:00), (161.75, 2018-02-03
07:00:00) ...]
Question
Identify one of the local maxima peak hours of traffic flow.
Note
Let's work this out in step-by-step way be sure that we have the right answer. Finally, summarize your analysis with JSON format, like:
'JSN
"summary":
"YOUR SUMMARY"
Response
<think>
Okay, let's see. The task is to identify local maxima peak hour in the traffic flow data provided. So first, need to understand what a local maximum is in this context. A local maximum in time series a point where the value is higher than its immediate neighbors. That means, looking for
45

---

Published as a conference paper at ICLR 2026
times when the traffic flow increased to a peak and then started decreasing. Looking at the data provided, which spans from February 3rd to February 5th, each hour has a traffic value. So I need to go through each hour and check if the current hour's 's value is higher than the previous and next hour's values. Those would be the local maxima.
Let me list out the values with their times. Starting with February 3rd:
00:00: 36.416..., 01:00: 19.833... + decrease from previous.. 02:00: 14.083..., 03:00:19.333...
04:00:36.916... 05:00:67.083,
06:00:106.1666
07:00:161.75... 08:00:234.166
09:00:256.1660 166.... 10:00:279.83.. 11:00:297.5.... 12:00:309.44
13:00:301.25 - This is lower than 12:00's 309.41, so 12:00 is peak? Let's check. The 12:00 value (309.416) is
higher than 11:00 (297.5) and then decreases at 13:00. Yes, so that's local max. So 2018-02-03 12:00:00 could be a peak.
Then 14:00:285.33 15:00:276.16 16, 16:00:306.5 + comparing to 15:00 and 17:00. So 16:00's 306.5 vs previous 176.16 and next hour (17:00) 292.25. So 16:00 on the 3rd is higher than before and after, another local maximum.
Then 17:00:292.25.. then it goes down. Then 18th hour: decreases 255.1, so the peak 16:00. Looking at Feb 4th. Let me look through their hours. The data from Feb 4th starts at 00:00 again. Let's check peaks there.
For example, at 08:00 Feb 4th: 200.58..., 09:00:239.66 66, 10:00:239.66, 1 11:00:272.58, 12:38,112:0000 12:00:280.83 13:00:283.9 1,,,, 14:00:300.66 Wait, at 13:00, it's 283.91, then 14:00's 300.66, 66. Is that higher than 15:00? Next hour (15:00 Feb4) is 256.25. So 14:00 is higher than both 13:00 and 15:00; so 14:00 on Feb4 is a local peak.
Then at 19:00 Feb4, the value is 187.91, and 20:00 is 192.33 not a peak. Let me check other times.
On Feb 5th, the data shows: 05:00:288.41 06:00:390.08... 07:00:384.83. 08:00:366. 16... So here, 06:00 is higher than 05:00 and 07:00. So that's 's a peak at 06:00 on Feb5th. Then later, at 17:00 Feb5: 392.16, which is higher than 16:00 (312.25) and 18:00 (305.0). So 17:00 on Feb5 is another local maximum. That's a notable peak.
So possible candidates for local maxima are:
-2018-02-40 12:00:00 (309,41)
-201 -2018-02-03 16:00:00 (306.5)
-2018-02-04 14:00:00 (300.66)
-2018-02-05 06:00:00 (390.08)
-2088-02--- 17:00:00 (392.16)
Now, the question asks to identify one of these local maxima. The user just needs one, probably the highest the most prominent. Let's check the values. The largest numbers I see from February 5th. The 05:00 on Feb5 is 288, then 06:00 jumps to 390-allm almost 400, which a significant spike. Then at 17:00 Feb5, it goes up 392.16. Comparing those two, 392.16 16 is higher than 390.08. Both are peaks. Let me confirm:
05:00 Feb5:288.41 06:00:390.08 (peaks here, since the next hour at 07:00 384.83, which is lower) Then at 08:00:366. 16 So at 06:00, higher than before and after -peak.
Then 17:00 on Feb5 392. 16, which follows 16:00 (312.25) and precedes 18:00 (305.0). So that's another peak. So 17:00 is higher than 06:00. Which one should choose?
The task is to find "one" them. So selecting either would be correct. However, the highest local maximum would be 17:00 Feb5 (392.16). Alternatively, the spike in 06:00 on Feb5 may be because it's part of the early morning rush, but the user just wants one example. Maybe the highest one.
But let's verify there are higher points anywhere else. Looking Feb3: max is 309 at 12pm. Feb4's max 14:00 is 300.6. So Feb5 peaks at 392 and 390.
Yes. So need report one. For instance, 2018-02-05 17:00:00 with 392, 16 as the local maximum, because that's 's higher than the surrounding hours. Alternatively, the 06:00 hour on Feb5 is also a local maximum. But since the task allows any one, it's sufficient pick either. But the highest is 17:00 on the 5th. Alternatively, perhaps there is a typo, let me check the data again.
Looking at February 5th's 05:00: 288.41. 06:00: 390.08. 07:00: 384.83. So 06:00 is higher than both, so that's a local maximum. Then again, at 17:00, 392.16 is higher than 16:00's 312 and 18:00's 305, So both those times are peaks. Of these, 392 at 17:00 is higher than the 390 at 6:00. So 17:00 is a higher peak.
46

---

Published as a conference paper at ICLR 2026
So the correct answer would likely be either of these. So to answer the user's question, can choose the 17:00 on Feb5 as one example. </think>
"answer": "2018-02-05 17:00:00 identified as local maximum peak hour with traffic flow 392.16."
"summary": "A sigmificant local maximum peak occurs on 2018-02-05 at 17:00:00 with traffic flow of 392.17. This hour is preceded by a rise from 312.25 (16:00) and followed by a drop to 305.0 (18:00), confirming it as a local peak within the time series."
Table 31: The spatiotemporal understanding post-training instruction for periodicity analysis.
Task Description
You are tasked to analyze the time series data of electricity consumption (in Wh).
Data Schema
series: Each tuple (value, time) represents data reading in Wh recorded at the specified time.
Data
The time-series of electricity consumption (in Wh) in every hour is provided below:
(114. OWh, 2018-03-122 03 03:00:00) (116. OWh, 2018-03-11 04:00:00), (114. OWh, 2018-03-1 05:00:00), (115.0Wh,
([112.0h, 2018-03-11 02:00:001
2018-03-11 06:00:00), (122. 112 2018-03- 07:00:00),
(136. OWh, 2018-03-11 08:00:00)...
Question
How long is the period of electricity consumption (in Wh)?
Note
Let's work this out in a step-by-step way to be sure that we have the right answer. Finally, summarize your analysis with JSON format, like:
JSON
"summary": "YOUR_SUMMARY"
<think>
Okay, so the task here is to determine the period of the electricity consumption based on the given time series data. Hmm, the term "period" in time series usually refers to the length of a repeating cycle or pattern. Let me think.
Looking the data, the timestamps start 2018-03-1 02:00:00 and go through each hour until 2018-03-14 at 01:00:00. Let me first check the timestamps to see over how many days this data spans.
Start date: March 11th at 2 AM. The data includes March 11, 12, 13, and part of the 14th (ending AM on the 14th). So that's roughly three days. Wait, from March 11th 02:00 March 14th 01:00 is 24h 3 days minus one hour. So about 71 hours? Let's count. From the first entry to the last entry: 11th 02:00 to 14th 01:00. The total number of data points for each hour, and the entries seem consecutive with no gaps. Let me count the data points to check.
Looking at the data provided, each entry increments by one hour. Let's 's see: first entry at 02:00, next at 03:00, up the last at 01:00 the 14th. The list starts with (1 11 02:00) and ends with (14th 01:00). Since each entry is hourly, the total hours between first and last would be from 02:00 on 3/11 to 01:00 on 3/14. How many hours is that?
From 3/11 02:00 to 3/14 02:00 would be exactly days (72 hours). But since the last entry is hour, so number of data points would be 71+1? Wait, no. Let me see. The first entry is at 02:00, next 03:00, etc. So from 02:00 to 24:00 on 3/1 22 entries (hours 2-23 inclusive, which 22 hours?). Wait, no. Wait,
01:00 on 3/14, it's -1 =71 hours. But each entry corresponds
47

---

Published as a conference paper at ICLR 2026
2 AM to 11 PM is 22 hours (since 24-2=22, but with each hour included. Wait, 02:00. 03:00,...23:00
for 3/11. Then next day starts at 00:00. Hmm, the data goes through 3/11, including the next day's s recount the days covered. The data starts 1 1thh, March 02:00 and ends March 14th at 01:00. So 1 12th, 13th, and part of 14th. But the main point here isn't tt the span of the data but looking for the periodicity in the data, For electricity consumption, typically there's s daily pattern-higher during the day, lower at night. Let's check the values to see if they exhibit daily cycle.
entries. Let
Looking at the data: each day starts with lower values in the early morning, increasing during the day. For example, March 1h,, around 10:00 AM (10:00:00), there's spike to 543Wh, which seems like huge jump compared to the previous hours. Then March 1 12th, at 10:00, it's 526 Wh, also a spike. Same on March 13th: 10:00, 552 Wh. So every day at around 10 AM, there's a big increase. After that, consumption remains high during the day and starts drop in the evening. Then at 20:00 (8 PM) each day, drops to around 359Wh (March 11), 361 (March 12), 357 (March 13), then drops further to around 160-168WN in the late evening. Then this pattern repeats each day.
So that's clear daily period, where each day the consumption follows similar pattern: low at night, peaks in the late morninglafternoon, then decreases again. So the period here is 24 hours-repeattng every day.
But wait the question asks "How long is the period of electricity consumption (in Wh)?" So it's looking for the duration of the cycle period, not the data's total duration but the cycle length. Given the daily repeats, the period is 24 hours.
But need confirm if the data indeed shows daily periodicity. Let me check across the days. Let's take the first three days.
March 11th at 10:00: 543.0 Wh.
March 12th 10:00:526.0.
March 10:00:552.0.
Then each day the peak hours of around 10:00-16:00 have high values, while nighttime is lower. Therefore, the repeating cycle is daily, which is 24 hours. Thus, the period is hours. </think>
"summary": "The electricity consumption exhibits a daily periodicity of 24 hours. Patterns repeat every day with lower consumption during nightrime and higher consumption during daytime, particularly spiking around late morning to afternoon hours. This daily cycle is consistent across all provided data points from March 11th to March 14th."
48


Reviews:
[{"review_id": "jxvvn1lPBE", "reviewer": "Reviewer_UrGc", "summary": "The paper introduces USTBench, a novel benchmark designed to evaluate large language models as urban agents, focusing on their ability to reason about and act within dynamic urban environments. It proposes a comprehensive framework that decomposes urban reasoning into four stages: spatiotemporal understanding, forecasting, planning, and reflection. The benchmark is built around an interactive city simulation (UAgentEnv) based on real-world data and includes 62,466 QA pairs for fine-grained evaluation. The paper shows that while LLMs excel at spatial and temporal understanding, they struggle with long-term planning and reflection, highlighting the need for domain-specific adaptation. By providing detailed insights into each reasoning stage, USTBench aims to advance LLMs' applications in urban systems and other dynamic domains.", "questions": "1. Simulation validity: How closely does the “optimal” action derived via simulation match human expert planning in urban contexts? Have the authors conducted any expert validation?\n2. Data diversity: Does UAgentEnv include data from multiple cities? If not, how might this bias model generalization?\n3. Interpretability: Can the benchmark support interpretability analysis, such as tracing which reasoning stage most strongly correlates with task success, to guide future model improvements?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents a novel and comprehensive benchmark, USTBench, for evaluating large language models as urban agents, marking an original contribution in the field of AI for urban systems evaluation. The contributions are listed below:\n1. It decomposes of urban reasoning into four distinct capabilities (spatiotemporal understanding, forecasting, planning, and reflection), which allows process-based diagnostics instead of only outcome-level evaluation. This modular view introduces a new problem formulation that bridges urban computing and cognitive assessment of LLMs.\n2. The benchmark integrates real-world urban datasets, nine realistic urban tasks, and an interactive simulation environment (UAgentEnv). The dataset has over 62,000 structured QA pairs, and is meticulously designed to isolate and measure reasoning sub-skills, enhancing the reliability of evaluation. The experimental section is extensive, comparing multiple state-of-the-art models and providing consistent metrics for each reasoning stage.\n3. The paper is well-organized and logically presented. It effectively motivates why decomposing reasoning is critical for diagnosing model weaknesses. Figures and tables are intuitive, and the design of each reasoning subtask is clearly justified.\n4. Urban intelligence is an emerging but crucial domain for LLM applications, and the proposed benchmark fills a clear gap by providing standardized, fine-grained evaluation protocols.", "weaknesses": "While conceptually strong, the paper has several areas for improvement:\n1. Limited coverage of real-world complexity. Although UAgentEnv incorporates multiple tasks, it still simplifies many aspects of urban systems (e.g., policy constraints, multi-agent interactions). The lack of uncertainty modeling may limit generalizability to truly dynamic urban settings.\n2. Evaluation breadth. Experiments cover a good range of models, but the paper lacks ablation or fine-tuning studies showing whether models can improve with urban-specific instruction tuning or multimodal inputs. This limits insight into potential solution directions.\n3. Reflection dimension underdeveloped. The reflection tasks are intriguing, but the methodology for quantifying reflection accuracy feels underexplained. More qualitative analysis (e.g., error types, improvement dynamics) would clarify why reflection fails and how it could be improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2feypC96yd", "reviewer": "Reviewer_bS58", "summary": "The paper proposes USTBench, an evaluation benchmark to systematically measure large language models’ (LLMs) spatiotemporal reasoning abilities in urban environments. Built on an interactive environment called UAgentEnv, the benchmark covers four reasoning facets: spatiotemporal understanding, forecasting, planning, and reflection, and supports both fine-grained (process-based) and end-to-end task evaluation. The authors argue that existing urban LLM benchmarks mostly rely on outcome-based metrics (e.g., traffic efficiency, prediction accuracy) and thus may hide reasoning deficits; USTBench instead makes the intermediate reasoning step explicit and measurable, showing, for example, that reasoning-style LLMs like DeepSeek-R1 do not always outperform strong non-reasoning LLMs such as GPT-4o on urban tasks. The paper also releases datasets (62k+ structured QA pairs), environment configs, and scripts to support reproducibility.", "questions": "None", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Timely and well-scoped problem. Urban LLM agents are an emerging but under-evaluated direction. Focusing on spatiotemporal reasoning (not just traffic, not just planning) makes the benchmark conceptually coherent.\n\n2. Process-based evaluation is clearly motivated. The paper illustrates, with concrete cases (e.g. congestion trend vs prediction), that outcome-only metrics can give a misleading ranking of reasoning vs non-reasoning models. This is an actual pain point in current LLM-for-planning work.\n\n3. Comprehensive baselines. They evaluate both non-reasoning and reasoning LLMs, including recent RL-on-reasoning models (DeepSeek-R1, QwQ, GLM-Z1, o4-mini, GPT-4o), which makes the claim “general reasoning does not always transfer to urban tasks” convincing.", "weaknesses": "1. Ground-truth construction and label fidelity need more quantification. Many QA instances are generated via interactions with UAgentEnv, but the paper does not report inter-annotator agreement or error bounds for the simulation-derived “optimal” actions. For planning QAs, the “exhaustive search” over horizon H could itself be suboptimal or environment-specific.\n2. Evaluation still has a strong API/hardware assumption. The runtime table shows some models become very slow (e.g. DeepSeek-R1 via Alibaba API), which limits the practical use of the benchmark, but the paper doesn’t analyze how this affects comparability across models.\n3. Numerous grammatical errors: L107 LLMs excels; L322 Reasoning models achieving...\n4. The paper repeatedly claims that “UAgentEnv is an interactive environment for urban agents,” but in fact, the entire environment consists only of static JSON tasks (in QA format), without any state transitions, action feedback, or interactive loops.\n5. Tasks are single-step question-answering without any decision–feedback loop.\n6. Amap data is not an open API and requires a commercial license.\n7. The evaluation of CoT and few-shot results is lacking.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q7LkPJuEZY", "reviewer": "Reviewer_smgX", "summary": "This paper integrates several scenarios related to urban spatio-temporal understanding and their datasets for LLM Agent. Based on this, this paper further splits the process of solving the above scenarios using LLM Agent into four dimensions including  spatiotemporal understanding, forecasting, planning, and reflection, and then designs USTBench to evaluate LLM performance according to each of these four dimensions, as well as to provide an end-to-end overall performance evaluation.", "questions": "1. Why are LLM Agent capabilities for urban spatio-temporal understanding limited to four dimensions?\n2. How representative is the LLM chosen for this paper? How was the list of models reviewed determined for this paper? There seems to be a limited number of closed-source LLMs here, with a higher number of DeepSeek-R1 variants.\n3. How are the lists of LLMs included selected in Figure 4 and Figure 5, especially Qwen2.5-7B vs. Qwen2.5-32B used for comparison in Figure 5?\n4. Why are Table 4/Figure 4 and Table 5/Figure 5 so close? This gives the false impression that the tables and figures are related content.\n5. With this benchmark, is it possible to quantitatively assess the impact of a single dimension of performance on an end-to-end LLM agent? Or are there deeper and more comprehensive insights beyond reflection to reflect the “DISSECTING” in the title.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The idea of splitting the LLM agent into multiple key dimensions and benchmarking them independently proposed in this paper helps to help researchers understand the role played by each dimension in the whole agent autonomous execution process in a more refined way, and can deepen the understanding of the LLM capability boundary.\n2. In this paper, a significant amount of work has been invested in completing the processing of raw scenarios and datasets to QA pairs that can be used for benchmarking.\n3. This paper gives the review results and analysis of commonly used LLMs, which helps researchers to quickly understand the existing LLMs in the field.", "weaknesses": "1. This paper considers LLM agents as spatiotemporal understanding, forecasting, planning and reflection. The statement is not given any reason in this paper so that I find it very confusing. This seems to be a claim forced by the authors to match the benchmark task they designed. Therefore, a reasonable derivation to show that agents used for urban spatio-temporal reasoning tasks include mainly and only these aspects of capabilities is necessary and will complete the logical chain throughout the text.\n2. Following the weakness above, Figure 2 also shows a worrying imbalance, with forecasting, planning, and reflection occupying only a relatively small portion of the overall performance radar graph, while the rest is all about the spatial capabilities and temporal capabilities of the LLM. This seems to indicate that the benchmark proposed in this paper does not focus on or is not capable of assessing the aforementioned high-level capabilities of LLM in a holistic manner, but is only a repetitive assessment of the spatio-temporal comprehension capabilities of LLM mainly (such work already exists).\n3. In the table in the experimental section, the different models are split by horizontal lines to divide them into groups, but there is no explanation for this, which is equally confusing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "z4vaNwfowi", "reviewer": "Reviewer_WawK", "summary": "The paper introduces USTBench, a large‐scale benchmark for evaluating the spatio-temporal reasoning abilities of large language models (LLMs) acting as urban agents. USTBench is built on an interactive environment (UAgentEnv) that simulates nine realistic urban prediction and decision-making tasks (e.g., congestion forecasting, traffic-signal control, urban planning). It decomposes reasoning into four processes (understanding, forecasting, planning, and reflection with feedback) and provides 62,466 QA pairs plus standardized end-to-end task metrics. Thirteen state-of-the-art LLMs (generalist and reasoning-enhanced) are benchmarked, revealing strong performance in understanding/forecasting but persistent weaknesses in long-horizon planning and reflection.", "questions": "Mentioned in weaknesses.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- First dataset to systematically dissect spatio-temporal reasoning in urban-agent settings and to evaluate a “reflection-with-feedback” capability missing from earlier city-scale benchmarks. Also explicitly compares with STBench, CityBench, CityGPT, and UrbanPlanBench, highlighting gaps such as process-level metrics and long-horizon planning evaluation.\n- Covers 9 heterogeneous tasks, 5 data modalities, and ~62k QA pairs, providing both breadth and depth for evaluation.\nBenchmarks 13 state-of-the-art LLMs across four reasoning facets, plus ablations on reflection, yielding actionable insights for model developers.\n- Demonstrates that general reasoning post-training does not fully transfer to complex urban domains, motivating domain-adaptive techniques.\n- Detailed ablation studies and examination of failure cases offer valuable insights into the spatio-temporal reasoning challenges of current LLMs.", "weaknesses": "- The study would be strengthened by an explicit modality ablation or corruption experiment to verify which data sources (e.g., socio-economic vs. POI) are most influential in reasoning outcomes.\n- The related work section could be strengthened. In particular, the authors may consider discussing the following relevant works, which could help better situate the current contribution within the existing literature: \n   - Urban Agent [1, 2, 3]\n\n[1] Wang, Jiawei, et al. \"Large language models as urban residents: An llm agent framework for personal mobility generation.\" Advances in Neural Information Processing Systems 37 (2024): 124547-124574.\n\n[2] Li, Bowen, et al. \"LogiCity: Advancing neuro-symbolic ai with abstract urban simulation.\" Advances in Neural Information Processing Systems 37 (2024): 69840-69864.\n\n[3] Jiang, Yue, et al. \"Urbanllm: Autonomous urban activity planning and management with large language models.\" arXiv preprint arXiv:2406.12360 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)