Learning Dynamics Feature Representation via Policy Attention for Dynamic Path Planning in Urban Road Networks
Authors: Kai Zhang
Jingjing Gu
Qiuhong Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Dynamic Path Planning; Reinforcement Learning; State Representation; Dynamics Feature Representation; Policy Attention Mechanism
Primary area: reinforcement learning

Abstract: Dynamic Path Planning (DPP) in urban road networks faces fundamental challenges, as traffic conditions change rapidly over time and often render planned routes ineffective. Reinforcement Learning (RL) provides an effective way to adaptively handle such uncertainties by incorporating traffic dynamics into state, but its performance crucially depends on how these dynamics are represented. Existing approaches either rely on global traffic information, which ensures decision completeness but suffers from redundancy and high computational cost, or oversimplified local features, which are efficient but often omit critical dynamics and lead to suboptimal paths. To address this, we propose a Dynamics Feature Representation (DFR) framework that progressively refines global traffic dynamics into compact features for RL-based DPP. Specifically, we introduce a policy attention mechanism that identifies a core subset of global dynamics by extracting the top-k shortest paths, and further constructs node-related local features by intersecting with n-hop neighborhoods, enabling near-optimal policy learning. Theoretical analysis demonstrates that DFR guarantees state completeness, while empirical results confirm that, compared to classical baselines and standard RL methods, DFR significantly improves path planning performance and accelerates convergence. This work highlights the central role of feature representation in RL-based DPP and proposes a general framework that balances information sufficiency with computational efficiency, paving the way for scalable dynamic decision-making in real-world transportation systems.

Published as a conference paper at ICLR 2026
LEARNING DYNAMICS FEATURE REPRESENTATION VIA POLICY ATTENTION FOR DYNAMIC PATH PLAN-
NING IN URBAN ROAD NETWORKS
Kai Zhang", Jingjing Gu Qiuhong Wang!
Nannjing University of Aeronautics and Astronautics, Nanjing, China
"National Key Laboratory of Helicopter Aeromechanics, Nanjing, China
"Corresponding author (2 tzhangka none, gu junggiig h enuaa edu. .n
ABSTRACT
Dynamic Path Planning (DPP) in urban road networks is challenging due to rapidly changing traffic conditions that often invalidate pre-planned routes. Re- inforcement Learning (RL) can adaptively handle such dynamics, but its effec- tiveness depends on how those dynamics are represented in the state. Existing methods either rely on global dynamics, which are complete but computationally expensive, or on simplified local dynamics, which are efficient but may omit crit- ical information, leading to suboptimal routes. To address this trade-off, we pro- pose Dynamics Feature Representation (DFR) framework that progressively re- fines ligendimensional global dynamics into compact, decision-relevan features. DFR introduces policy attention mechanism to identify a core subset of dynam- ics based distance-oriented policy, and uses n-hop neighborhoods method further decouples this subset into node-related local feature sequences. This hier- archical refinement enables efficient near-optimal policy learning. Experiments realistic urban graphs show that DFR improves the performance of RL-based DPP models and accelerates convergence compared baselines. This work highlights the importance of adaptive state representation and provides a general framework for dynamic decision-making in practical urban transportation.
INTRODUCTION
With the rapid development of emerging applications such as on-demand delivery, urban logistics, and intelligent transportation systems, achieving efficient path planning in urban road networks has become a pressing demand (Gu et al., 2023) Mao et al. 2025). However, real-world traffic conditions time-varee mare-vayynng, often rendering planned routes ineffective. Dynamic Path Planning (DPP) has thus been proposed to address this issue (Du et al. (024a)), aiming to maintain robust and adaptive routing under continuously evolving traffic conditions. A common way to formulate DPP is to model the urban road network as a dynamic directed and weighted graph. Traditional methods then rely on explicit traffic flow prediction or rule-based evolution models (Guo et al.) (024) (e.g.. forecasting link speeds for a future time horizon). However, such predictions suffer from limited accuracy and poor generalization, particularly under unexpected events like accidents or road closures (Medina-
Salgado et al. 2022 Huang et al. 2022). Therefore, those approaches may be unreliable in practice.
Reinforcement Learning (RL) offers new paradigm for handling dynamics. Instead of requiring an accurate explicit model, RL incorporates dynamics into the state and learns optimal decisions through interaction (Bouktif et al. 2023 Lin et al. 2025). The key advantage is that value function estimation implicitly absorbs the statistical patterns of state transitions, thereby reducing reliance on predictive accuracy and exhibiting stronger robustness to unseen events (Nguyen et al. 2025
Despite this advantage, applying RL to DPP faces fundamental challenge (Chen al. how to effectively represent traffic dynamics effectively within the state. Existing approaches strug- gle with dilemma. Global methods that encode the entire graph dynamics ensure information completeness but are computationally prohibitive Liu et al. (2024). Local methods that use partial observations of the agent are efficient but risk overlooking essential non-local dynamics (Du et al.
2023

---

Published as a conference paper at ICLR 2026
2024b), leading to shortsighted and suboptimal routing (Francis al. 2025 More critically, in- sufficient state representation may undermine the Markov property, which leads to degraded model performance and unstable training. Therefore, it is crucial for effective RL-based DPP to identify suitable dynamics features.
To bridge this gap, we propose Dynamics Feature Representation (DFR) framework construct Markovian state representation by progressively distilling the global graph dynamics into a compact, decision-centric feature. DFR employs a novel hierarchical refinement process: 1) Task-related filtering via policy attention: We first pre-train a policy attention module that is expert in finding shortest paths. Given an source and destination, it identifies the top-k shortest paths and extracts a sparse, task-relevant subgraph from the global graph based on the nodes in those paths. The dynamics of this subgraph form the first refined state. 2) Agent-centric decoupling via n-hop neighborhoods: The above state is further refined at each timestep to focus on the agent's 's immediate context. We compute the n-hopp neighborhoods around the eurrent node and take its intersection with the policy attention subgraph. The resulting dynamics provide low-dimensional state vector that captures the essential local traffic conditions within the context of the global task. By doing so, DFR provides the RL agent with state that is both computationally efficient and informationally sufficient to approximate the Markov property.
Our core contributions summarized follows: 1) We propose the DFR framework, novel hi- erarchical approach to resolve the completeness-efficienn
trade-off in state representation for RL- based DPP. 2) We introduce two key technical innovations within DFR: a policy attention mechanism for task task-aware graph sparsification that extracts a task-related subgraph, and a r--hop neighborhood method for agent-centric feature decoupling that provides local state view. 3) We empirically val- idate that our DFR framework achieves significant improvement in performance and remarkable acceleration in convergence compared to standard baselines.
RELATED WORK
Path planning methods Traditional path planning algorithms, such as A (Hart al. 1968) and
D* Lite (Koenig & Likhachev] 20022) have been widely used in static environments. However, their reliance on pre-defined and fixed cost map makes them ill-suited for highly dynamic urban networks where travel costs change rapidly. To address this, some learning-enhanee
methods have been proposed (Du et al. 2024a Li et al., 2024). These typically involve first predicting future traffic conditions using statistical or deep learning models (Guo et al., 2024 Yang et al. 2025) and then executing a classic search algorithm on the predicted graph. However, the performance of these planners is inherently bounded by the accuracy of the traffic forecast (Tian et al. (025)- They are particularly vulnerable to distribution shift and unexpected events (e.g., accidents), often leading to suboptimal or failed routes (Ren et al.) 2025). In contrast, our method eliminates the need for explicit prediction, and we employ RL to learn policy that implicitly captures the statistics of traffic dynamics through interaction, thereby achieving greater adaptability.
State representation RL has emerged as a powerful framework for solving DPP problems (Mao) 2023 Xue et al. 2025). common challenge the design of the state representation (Francis et al.
et al. 2025). In the context of path planning
graphs previous work often resorts simplistic representations. Some methods use a local view (Zhao et al., 2025). which sacrifices global opti- mality for efficiency. Others provide a global view of the entire network (Lin et al. 2025). which is computationally prohibitive in large cities. In addition, graph neural networks offer a powerful way to process graph-structure data for DPP problem (Zang et al. 2023 [Sun et al. 2025). by encoding the graph structure and dynamics into node embeddings. While they can theoretically capture the entire graph's information, the computational and memory overhead scale with the size of the graph (Wu et al. 2020). making them impractical for real-time planning in massive urban networks. The state for decision-makir is often assumed to be given and Markovian, which is frequently not the case in practice (Kaelbling et al., 1998). Our work attemps to refine global dynamics into a compact form, drawing inspiration from feature decorrelation techniques (Huang et al., 2023).
Attention mechanisms The use of attention mechanisms for state abstraction and feature selection is well-established (Brauwers & Frasincar 202T) Hu et al.] (002)). Soft attention, as in Transformers Min et al. 2022). allows agents to weight the importance of different parts of their input. Our
2

---

Published as a conference paper at ICLR 2026
P()
Veg
Fiu-13
Prag by
Actual Poth
C-
6
fx-s
T
Time Axis
W% Wg Wa
Wi, Dynomics Sequence
Figure I: The discrete timeline of dynamics in urban road network system: ranging from to T' 1, each time & (0,1....,TT corresponds node v(k) in the path, where k 0,1, ,,.
proposed policy attention is a hard, pre-computed attention based on the structural semantics of the task. This provides strong, interpretable prior that drastically reduces the problem' 's ddimeniionality before the RI. agent even begins learning, similar spirit to knowledge distillation approaches used in other domains (Papadopoulos et al. 2021).
PROBLEM FORMULATION
Next, we formally describe the DPP problem using graph theory and Markov decision process.
3.1 DYNAMIC PATH PLANNING IN DISCRETE URBAN MAPS
Assuming that the road network of a city is represented as a directed and weighted graph G = (V,E). where V {U1. 02 ....... UN } the node set, and E (e../lu, E V, * j} the edge set. Each Ci.j = (vi, vÿ) has a connection value 1(0;, "2) and weight value w w(;,, My; f), where 1(4, = {0, 1} represents the connection relationship, and 1 indicates that node e and , are directly connected and
0 means no direct connection; and w(e,, 2y; t) E I denotes the traffic cost (such as time or distance) from v; to vy at time l. For all [w 0y5 t) < co if I(V, v2) = 1, and [w(e,, uy; O)] oc if (1v, vj) 0.
is assumed that V remains constant, while the weight :3) of each edges may vary over time The set of edge weights W, : æ-41_ = {wir, Uy; V(v,, 1ÿ) E E} is referred to as the dynamics of the urban road network system at time t. Furthermore, we define the discrete sequence of dynamics over a time horizon T as W.r (Wo. W1, W2,.. Wr-1), which characterizes the temporal evo- lution of the system. We define the path space p {p). and path p be described as sequence of nodes, and adjacent nodes are directly connected, that is, p = (o(0) u(1):.... ((m)) where the hthh node in can be represented as p = (kk) and [(ps, Pk+1) = 1, = 0, Given a source node v; E V and an goal node By V. the path space between them is defined as P (uu Uy] C P = {plu;, uy]}, and Po Pr. Vy for each path p plu;, Uz]. Figure shows the correspondence between nodes the path and dynamics sequence on the timeline.
1.
The DPP problem aims to find the optimal path with minimum traffic cost between any node pair under changing traffic dynamies formulated as a combination optimization problem over discrete path spaces. Given source 22 and goal uy, along with the temporal dynamics W r, the DPP task is defined as T = (vy, Uy; W.r) Aassuming that W.r known in advance, the traffic cost of a path p under W . is denoted as c(p; W.r). The optimal path between nodes v, and Uy can be written as
p" uy
Uys W.rl arg min clp; W.r) arg min pEP(U.))g)
Σ wopks pktii pEP())))))
This is an ideal strategy that leverages the evolution of dynamics, enabling the planner to reason about the long-term consequences of current paths and thus achieve global optimality. However, in practice, future dynamics are typically unknown or inaccurate, making it a theoretical benchmark for evaluating other strategies. To address the limitations, Markov Decision Process (MDP) is intro- duced to rigorously reason about dynamics patterns by leveraging probability theory and stochastic process and learn near-optimal policies without requiring knowledge of future traffic conditions.
It is worth noting that "dynamic" refers to the replanning of paths, while "dynamics" refers to the changes of traffic conditions in the urban road network. Dynamies is the direct cause of dynamic path planning.
3

---

Published as a conference paper at ICLR 2026
3.2 MARKOV DECISION PROCESS
Agent and environment Assuming that the vehicle is abstracted as a RL agent, and the DPP problem in urban graph can be mapped into the finite-horizon and deterministic MDP model 6-tuple M = A, R, 7; H):
State space S denotes the set of all possible states, and a state 5, E == {v², Egg fi} consists of the current node yf (with o U2 the source node) P the goal node U, Uj. and the dynamics features ft, where (vs, 0j5 W.r) is an instance of the DPP task. ft encodes current traffic conditions observable by the agent, which is a compressed representation of W . that provides environment perception relevant to the current decision. Action space A C R" is the set of possible actions taken by the agent, and the action at {0, 1, ................................ n2. } is designed to move to one of na neighbors of v in the graph. Transition function T : x A x S - [0. 1] defines the prohability of transitioning from current state 8, to next state st+1 after taking action at, that is, (( 88, dt).
Reward function R S x A x - I guides the agent to achieve the goal by assigning a numerical reward to its each transition (5r, de; 8141). ). In this paper, under dynamics W between current node of in and next node of in 81+1,
the negative value the traffic cost
R(st, (t, 8t+1) = -c(v' +1 Wi) + T(u == (ggg
where 6 E R+ is a constant reward for reaching the goal node, and e(-, is the traffic cost of two nodes (see Equation D. Discounted factor ? E [0, 1] is used balance between the future reward (Y - 1) and immediate reward (y - 0). Time horizon H E N+ is the maximum number of steps, that is, S H. If the agent cannot reach the goal within H steps, the task ends in failure.
Policy and value function Policy function S + A(A) defines strategy of the agent to decide the next action based on the current state. Given a DPP task 7 with the initial state so, a path or state sequence p = {(Se, de, 8141 ) J can be automatically generated by a, ~ * (+(ss) and 8141 T(- st, a.). The ultimate goal is to obtain the optimal policy '² that can derive the optimal path p". To quantitatively evaluate the utility of т, the state value function V S + IR and the state-action value function Q: S x A - R are introduced, which is the expectation of accumulative rewards under R at state s that the agent can obtain by following æ and T. For the convenience of calculation, V can be transformed into recursive form, which is the famous Bellman Equation,
Vs(()= Σ (alls)
Σ x (als) aEA
Σ T(dls, e'ES
a, 8') + +
(3)
aEA
Solution optimality The decision condition that policy is better than another policy is converted to the ordering relationship between their value functions. The optimal policy x" is the policy corresponding to the upper bound of V. e.g. n² max Va(s), S, and the Bellman Optimality Equation is obtained, that is,
Vn (s) (s) max E(s'' s,
Using classic DRL algorithms, such as Deep Q-Network (DQN) (Mii et al., 2015) and its variants Van Hasselt et al.) 2016 Wang et al., 2016), and Proximal Policy Optimization (PPO) (Schulman et al. 2017), the optimal value function (*(s) can be uniquely determined by Equation Even in large-scale state spaces, DQN leverages neural networks parameterize the value function and employs sample-based updates to gradually improve its estimates until convergence to V*(s).
4 METHODOLOGY
4.1 CHALLENGES
We first analyze the key challenges in constructing effective state representations in the MDP-based DPP model. In Section 3.2 the state is {et, Uyi ft), where fz encodes the dynamics features
2The here is different from the in the previous part of graph theory. Simply put, = n here represents the decision step, which corresponds to f,, on the time axis in graph theory part.
4

---

Published as a conference paper at ICLR 2026
2
E2
Kg
fx-1
En
Time Axis
W
W1
W
Win
Dynomics Sequence
Actual Path
(0W)
(06,)
(0%
F0%
%
52
Sp
0g
4g-11
π [a|s)
Optimal Policy
Figure 2: The diagram illustrating how the policy function handle dynamics sequence in MDP-based DPP. The colors of the edges in the graph represent different dynamics.
extracted from the environment (see Figure The quality of ft directly determines the agent's planning ability to reason about dynamic traffic conditions. However, the design of fa faces in- herent trade-off. When f. global dynamics, the formulation resembles a fully observable MDP, where the agent has access the complete system state. When is limited to local dynamics, the formulation becomes partially observable, in which the agent act under incomplete and uncertain knowledge of the environment. Such partial observability reflects real-world conditions, where fu- ture states are inherently unavailable. Therefore, the major challenge of dynamics feature design is sufficient yet compact: retaining the essential dynamics for decision-mann maaiinn while avoiding redun- dancy and excessive complexity. Therefore, we propose a Dynamics Feature Representation (DFR) framework, which aims to identify and encode task-relevant subsets of dynamics, striking a balance between information sufficiency and computational tractability.
4.2 DYNAMICS FEATURE REPRESENTATION
Given a DPP task т = (Ve, uyy W .T), and the state representation s; = Ugi We formalize DFR three-level process, refining the global information into compact, decision-relevant state.
W T
W
(5)
T
Global dynamics features W At decision step i, the full graph's 's edge weights W, serve as the dynamics feature fe, that is, 8, = -(e, My: Wt). where of the current node. As mentioned be- fore, although W, is extremely high-dimensional and redundant, making it unsuitable to be used directly as the state input for a RL agent. We then provides the raw input for subsequent hicrarchical processing to produce more compact, task-relevant, and node-specific features.
Task-related key features W To reduce redundancy while retaining essential information, we define global key subset (T) (T W.). where V is function that extracts partial dynamics from W. that is, W C W,. Wi significantly reduces the dimensionality while preserving the critical dynamics information required for optimal routing decisions. Formally, for task т, W. is sufficient if the optimal policy conditioned on it approximates the policy conditioned on Wi:
(u', Mgi W T (v", 0g: W.)
(6)
Node-related local features S Even after global filtering, WT' may remain high dimensionality large-scale DPP scenarios. Therefore, we further define node-related local features ("') &(W/ 2: o'), where & is feature extraction function that maps W/' to lower-dimensional subset W associated with the current node ot. W can incorporate a spatial and temporal neighborhood, forming time-aware feature sequence. Similarly, the optimality of policy is preserved, that is,
(v', Myi W!") n² (o', ogi W;)
5

---

Published as a conference paper at ICLR 2026
Step L-Given DPP mak Py. Myp)
Siep 2: wain a distance- based policy.
Step Si choose shortest paths.
Step 4: build a subgraph with paths.
Wi
Mi
Miw
Dos
Tabi) taleo >- 1500
Soap S: train cost cost-based policy with local features.
6, = M2200))
fr - Wigoot
fr, - Wi((0))
f = (g'(+*)
Figure 3: The diagram of dynamics feature design using policy attention mechanism and n-hop neighborhood method (n 2 is shown in the example). The colors of the edges in the graph represent different dynamics.
Theoretical Basis. The local feature f. W." must ensure the compactness of the state represen- tation, meaning that it preserves all decision-relevane information from the original dynamics. Pre- dictive State Representations (PSR) provide theoretical foundation for this requirement (Littman) & Sutton 2001]. PSR posits that the state of system can be defined by predictions of future observ- able outcomes given possible action sequences, without resorting latent variables. In this sense, W serves as a predictive representation of the state: it encodes sufficient information to forecast the effects of future actions, thereby ensuring that the optimal policy based on W approximates the policy based on the full dynamics sequence W. Wer:
xt (v", 0g: WT") R x² ("', W.)
Beyond spatial abstraction, DFR also preserves the temporal dependencies inherent in traffic dy- namics. Each We reflects the evolving traffic state at time t, and the refinement process (Equation operates the sequential structure W.r rather than gregating temporally adjacent representations, DFR implicitly captures short-term temporal correla- tions-such as local congestion propagation and flow continuity-whit maintaining computational efficiency. From the PSR perspective, this design enables W." to function as a predictive summary of future dynamics conditioned the current observation window, thereby preserving both spatial and temporal decision-relevant information. This mechanism aligns with the Markov assumption, which requires the current state to represent all information necessary for eerrisionmmaking
single snapshot. By filtering and ag-
Grounding DFR in PSR principles thus guarantees that the resulting representations are compact, temporally predictive, and theoretically sufficient. This not only stabilizes training and mitigates suboptimality under partial observability, but also enhances interpretability and generalization when applying RL to the DPP problem.
4.3 DYNAMICS FEATURE EXTRACTION
After formalizing the DFR framework with V for task-related key feature selection and & for node- related local feature extraction, we now instantiate these mappings with concrete methods.
Policy Attention Method. Given the global dynamics features W,, the policy attention mecha- nism employed to construct task-relevant key subset W (see Figure 3 It leverages distance- based optimal policy < which computes the shortest paths from the current node vz to the desti- nation vn under static conditions. The paths derived from T are ranked by length, and the top-k shortest paths are selected to form a subgraph G' (V', E'). The edge weights of this subgraph define W/, where the parameter controls the trade-off between completeness and compactness: a smaller k may omit critical paths, whereas larger i may introduce redundant information.
Formally, to obtain TH the MDP formulation Section 3.2 is simplified by removing the dynam- ics feature component f, from origin state representation, and the reward function is redefined as

---

Published as a conference paper at ICLR 2026
aa(s,a,s') -d(s, where d denotes the road length between the current state and the next state s'. Substituting the new state representation and reward function Ra into Equation 4 be obtained after certain number of trainning iterations.
xy can
As discussed above, DPP aims to find optimal paths under multiple objectives-Sutt as distance, travel time, and energy consumption- by training an RL agent to learn multi-objectivve optimal pol- icy. In contrast, the policy T2 used in the pre-training stage of our DFR framework is distance-bass based only, without considering temporal variations in edge weights. This corresponds to RL-based static path planning, where the goal is to learn a policy capable of generating shortest paths in a static road network. The reason for using z. that, even when the ultimate objective of DPP is distance naturally serves as one of the most fundamental constraints. Thus, emphasizing edges along the shortest paths ensures that critical dynamics influencing near-optimal planning decisions are captured, making the extraction of W both reasonable and effective. In addition, inter-node dis- tances do not vary over time as long as the topological structure of the urban graph remains constant. Consequently, the pretraining process of nī can be one-time and offline, which provides a stable and interpretable reference for subgraph extraction during DPP.
n-hop neighborhood method Based on W local features are extracted for current node of through an n-hop neighborhood method. Let (v²) denote the i-th order neighbors of v', and V' and E" are the node set and egde set of the policy attention subgraph G', respectively, The local node set of v' is then defined as Vi(u") = (o')") n V and the corresponding edges form the node-related subgraph. The weights of these edges define the local dynamics feature ft = ("" = [w(e;, ty E E' v; Uj E Vi(v")) n determines the spatial scale of W,": smaller n captures highly localized dynamics but may overlook broader context, while larger n expands coverage but increases dimensionality and computational cost.
It is noteworthy that our DFR framework incurs negligible additional computational overhead. The total cost involves updating dynamic traffic information and executing the DPP model. By leverag- ing policy attention and n-hopp neighborhoods, DFR reduces the scope of dynamics feature collec- tion to small, pre-computable subgraphs. Both policy policy-ann and no-hpp neighborhoods depend only on the fixed road network topology, allowing of- fline computation and reuse. Consequently, DFR achieves significant feature compression without affecting online efficiency, making it suitable for real-world traffic scenarios with live updates.
from a static path planning
EXPERIMENTS
5.1 EXPERIMENT SETTINGS
Scenarios The experiments are conducted on three urban road networks in China-Nanjing, the Chaoyang District of Beijing, and the Pudong District of Shanghai-sourre
from OpenStreetMap (OSM). Each road network for driving is preprocessed and modeled as directed weighted graph, as illustrated in Figure 4 For each region, we specify center node in the corresponding network and extract subgraph by including all nodes within a certain radius. Each scenario corresponds to a single DPP task with source node, a goal node, and dynamics sequence. The source and goal nodes are randomly sampled from subgraph. In this work, we take traffic time the dynamics, and the path with minimum traffic time is regarded as the optimal path (see Equation 2). Edge weight w(E, ey:t) are parameterized by congestion factor B(u, Uy;t) E [0.1, 1.5). Here, smaller indicates heavier congestion or higher traffic density, and a larger 33 corresponds to smoother traffic conditions. For instance, B(v,, uy; I) 0.1 means that the road (v;, U;) at can only be traversed at one-tenth of the base speed. Given the distance d( (v, vÿ) and the base speed un, the traffic time of a road (wt Uz) under dynamics W, is then written by
c(U,, W:) d(v,,,))/(ux (/) x 8(0,, ty:
Baselines We consider three baseline RL algorithms DQN (value-based) PPO (policy-gradient, stabilized actor criticrriicciiit for discrete actions), and GCN+DQN (graph-based based with GCN feature extrac- tion). Each algorithm is evaluated both with and without DFR. For evaluation, we adopt four metries:
As the advantages of RI.-hased approaches over traditional methods in DPP have been well established in the literature. Our work instead aim to investigate the impact of the DFR framework within the RI. paradigm.
7

---

Published as a conference paper at ICLR 2026
Subgraph
Subgraph
Subgraph 3
Map 3
(Nanjing)
(Chapyang, Beijing)
(uudong, Shanghal)
Figure 4: Urban road network of three cities or districts. For each region, a center node and radius specified, and all nodes and edges within this range are extracted form subgraph (magenta).
the average relative difference of cost between the learned paths and the ground paths, and lower is better. The ground-truth paths are computed by the dynamic Dijkstra algorithm; 2) Success Rate (SR): the proportion of scenarios in which the learned path successfully reaches the goal node, and higher is better; 3) Compactness Rate (CR): the proportion of the reduced fea- ture dimension after DFR to the original dimension, and lower is better. Planning Time (PT): the average computation time required by an algorithm to generate complete path for a given planning query, and lower is better.
1) Mean GAP:
Model Training Given subgraph, the DPP model is trained using the above algorithms over N episodes, where each episode corresponds to a new scenario. During training, the DQN updates its Q-function by minimizing the temporal-differeee
(TD) loss:
(() = E( (r(s, a, s') + + max Q(s',a', Q(s, a: =)
(10)
where 9 and 0- denotes the parameters of the Q-network and target network, respectively, y is the discount factor, and 'D is the experience replay buffer.
For all experiments we use the Adam optimizer with learning rate of 10-3 and discount factor = 0.99. The number of training episodes is fixed to 75, 600 episodes (about 200 epochs), and the model performance under the above metrics is reported. The size of the replay buffer is 10 and the batch size is 32. Each episode has a fixed horizon of 100 steps. The policy network is updated every 100 steps, and when applicable, the target network is also updated every 100 steps. ReLU is used as the activation function for all networks. For exploration, an e-greedy strategy is employed with e decaying linearly from 1.0 to 0.1. The architectures are as follows: DQN uses an MLP with 64- unit embedding layer and two 64-unit hidden layers; the target network shares this structure. PPO uses shared MIP for policy and value networks with the same layer sizes; Generalized Advantage Estimation (GAE) is applied with x 0.95, the clip ratio is 0.05, and entropy coefficient of 0.01 encourages exploration. GCN+DQN uses a graph convolutional layer for feature extraction, followed by an MLP with two 64-unit hidden layers, mirrored by the target network.
5.2 MAIN RESULTS
The experiments were conducted using three algorithms: DQN, GCN+DQN, and PPO, each with or without our DFR framework. This yields six algorithm settings, including DQN+DFR E.s. DQN+AD, GCN+DQN+DFR v.s. GCN+DQN+AD, and PPO+DFR v.s. PPO+AD, where "AD" stands for All Dynamics. Based on these parameter settings (see Section 5.1 we trained the DPP model on three different graphs (see Figure 4 and conducted statistical analysis on key metrics, in- cluding Mean GAP, SR, and CR. Since smaller values of GAP and CR indicate better performance, whereas larger SR is preferable, we plot 1 GAP and 1 CR to maintain consistent interpretation.
As shown in Figure 5 each model's performance is visualized as a triangle formed by 1 GAP, SR, and 1 - CR, with it area serving as a summary of overall performance. Across all algorithm settings,
'The source code of all experiments is publicly available at r/ 7/rbanlbynami cPathP1 anni Ag-A99K
sci /
8

---

Published as a conference paper at ICLR 2026
2QV440
OEH-0Q0-440
IPD-DM
PRO
DQU
30/+40
PO-DDR
Figure 5: Performance comparison different algorithms across three regions. "DFR" stands for framework, while "AD" refers to the use of All Dynamics features from the entire graph.
DFR-enhanced models exhibit larger triangle areas compared to their AD counterparts, indicating superior overall performance. A closer examination reveals that GCN-based models, while achiev- ing high SR, can still exhibit high GAP (i.c., low GAP). This occurs because, although GCN introduces structural representation capability, the combination of a relatively small network and high feature dimensionality limits the model's 's ability to fully exploit dynamic information. Conse- quently, the model completes planning tasks but remains largely insensitive to dynamic variations, resulting in high GAP. Incorporating DFR mitigates this limitation and improve the sensitivity of GCN-based models to dynamic changes.
Moreover, DFR substantially reduces feature dimensionality, which corresponds to an increase in 1 CR. This indicates that the model's input dimensionality decreases and the computational over- head for collecting dynamics features is reduced. After incorporating DFR, the average path plan- ning time is 8.18 + 1.74 ms for DQN/PPO and 27.26 + 6.8 for GCN+DQN. Correspondingly, DFR reduce the average time of planning paths by 85.59%, 46.08% and 79.32%, compared to DQN+AD, GCN+DQN+AD, and PPO+AD, respectively. Overall, our DFR framework achieves the fastest planning among RL baselines while maintaining high performance, highlighting that structural representation learning and dynamics feature reduction are complementary.
5.3 ABLATION STUDY
We conduct the ablation experiments of A and n Subgraph (a region of Nanjing; see the left side of Figure 4 1) k 0..4,0.4, 0.6, 0.8, 1.0, -1.0). which specifies the proportion of top-100 shortest paths used to construct the subgraph. For instance, k 0.6 selects the top 60 paths, while k = -1.0 disables policy attention. 2) n = 21, 3, 4, -1} determines the order of neighboring nodes incorporated when constructing local features. For example,
includes 2-hop neighbors, and n = - corresponds to no hop selection. We use (-1.0, -1) to represent (A: = 1.0,, n = -1).
The top of Figure shows heatmaps for Mean GAP, SR, and CR across all (k, n) configurations, and several trends can be observed from the experimental results. First, by comparing the baseline (DQN+AD or
1.0, n -)) with DON+DFR (k: 0.0, 0), the role of DFR dynam- ics feature compression can be clearly observed. The baseline exhibits SR of 0.884 and Mean GAP of 170. With the introduction of DFR and appropriate tuning of 1: and n. the model achieves higher SR and lower Mean GAP and CR. For instance, when
and A is set from 0.4 to 1.0, the Mean GAP values are 0.095, 0.113, 0. 108, and 0. 0.108, respectively, while CR remains below 5.7% in all cases. Overall, policy attention selects globally relevant dynamics, whereas n-hop neighbor- hoods captures local dependencies. Together, they enable the DFR framework to improve the model performance while minimizing input dimensionality.
Next, we aim to observe a clear relationship between the degree of neighborhood reduction, the strength of policy attention, and the overall model performance. When k is fixed, for example

---

Published as a conference paper at ICLR 2026
Mean GAP
Success Rate
Compactness Rate
9444
6130
V137
0.322
CRLL
6.00
5413
1.70
822
0:31
6113
X221
0867
6.00
840
3100
0.09
00/3
6:30
08
S4/9
17
CAR
0:11
0:23
1:344
111
<22
6.00
1.8
1:300
3330
33
0.1
coutd0 1.00
0.0
0.0
+136 113.4)
103
125
100
Figure 6: Top: Performance headmaps of the models under various combinations of k and n (k -1.0, n means no policy attention mechanism and no n-hop neighborhood selection). Botlom: Training curves under k = 0.6 with varying n.
A = 0.6, the model performance is relatively poor when is small. Specifically, for 1, the Mean GAP and SR are 0.151 and 0.723, respectively. As n increases, the performance improves 2, the Mean GAP decreases to 0.118 and the SR increases to 0.867. However, as shown in the bottom of Figure 6 further increasing n yields limited performance gains, and the curves exhibit trend of aggregation. For example, when n the Mean GAP and SR are 0.1 13 and 0.892. On the other hand, when n is fixed, increasing A generally improves perfor- mance, but & exceeds certain range, performance may decline. For example, when increases from 0.4 to 0.6, the Mean GAP rises slowly from 0.095 to 0.113, while the SR decreases from 0.908 to 0.892. In summary, compared with n, has a more complex and less predictable im- pact on model performance, which poses greater challenges for parameter tuning. Therefore, based on these systematic conclusions from small-scale experiments, it is recommend that in large-scale graph deployment, configurations with moderate A and smaller should be preferred. increases until the aggregation boundary of n is found, and can be further explored.
noticeably; for instance, when
and
CONCLUSIONS
In this paper, we present a Dynamics Feature Representation (DFR) framework for RL-based dy- namic path planning (DPP), which refines global traffic dynamies into compact, node-related local features. By incorporating policy attention mechanism and n-hopp neighborhood method in DFR, our approach enables the RL agent to effectively capture task-relevant and time-varying dynamics while reducing the dimensionality of state representation. Extensive experiments on realistic urban road networks demonstrate that our DFR framework significantly affects the performance of RL- based DPP model, highlighting the importance of appropriately designing dynamics features. Our study provides insights into how hierarchical feature refinement can improve the adaptability and efficiency of RL-based path planning under dynamic environments. However, the two parameters of l and n in DFR are manually selected in this study, which may limit its practical applicability in real-world urban road networks. In the future, -aaf-idaptive approach could automatically adjust and n to allow the DER framework to dynamically focus on the large-scale road network without manual intervention, thereby enhancing the scalability and real-world deployment potential.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Salah Bouktif, Abderraouf Cheniki, Ali Ouni, and Hesham El-Sayed. Deep reinforcement learning for traffic signal control with consistent state and reward design approach. Knowledge- Based Systems, 267:1 10440, 2023.
Gianni Brauwers and Flavius Frasincar. A general survey on attention mechanisms in deep learning. IEEE transactions on knowledge and data engineering, 35(4):3279-322 2021.
Chao Chen, Lujia Li, Mingyan Li, Ruiyuan Li, Zhu Wang. Fei Wu, and Chaocan Xiang. curl: A generic framework for bi-criteria optimum path-finding based on deep reinforcement learning. IEEE Transactions on Intelligent Transportation Systems, 24(2):1949-196) 2023. doi: 10.1109/
TITS.2022.3219555
Runjia Du. Sikai Chen, Jiqian Dong, Tiantian Chen, Xiaowen Fu, and Samuel Labi. Dynamic urban traffic rerouting with fog-cloud reinforcement learning. Computer-Aided Civil and Infrastructure Engineering, 39(6)793--81, 2024a.
Shengli Du, Zexing Zhu, Xuefang Wang, Honggui Han, and Junfei Qiao. Real-time local path planning strategy based on deep distributional reinforcement learning. Neurocomputing, 599: 128085, 2024b.
Lawrence Francis, Blessed Guda, and Ahmed Biyabani. Optimizing traffic signal control using high-dimensional state representation and efficient deep reinforcement learning. In 2025 IST. Africa Conference (IST-Africa), pp. 1-11. IEEE, 2025.
Ruixue Gu, Yang Liu, and Mark Poon. Dynamic
routing problem for scheduled de- liveries and on-demand pickups with time-related constraints. Transportation Research Part C: Emerging Technologies, 151:104139, 2023.
Kan Guo, Daxin Tian, Yongli Hu, Yanfeng Sun, Zhen Qian, Jianshan Zhou, Junbin Gao, and Baocai Yin. Contrastive learning for traffic flow forecasting based multi graph convolution network. IET Intelligent Transport Systems, 18(2):290-301, 2024.
Peter E Hart, Nils Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE transactions Systems Science and Cybernetics, 4(2):100-107, 1968.
Kai Hu, Keer Xu, Qingfeng Xia, Mingyang Li, Zhiqiang Song, Lipeng Song, and Ning Sun. An overview: Attention mechanisms in multi-agent reinforcement learning. Neurocomputing, 598: 128015, 2024.
Jizhou Huang, Zhengiie Huang, Xiaomin Fang, Shikun Feng. Xuyi Chen, Jiaxiang Liu, Haitao Yuan, and Haifeng Wang. Dueta: Traffic congestion propagation pattern modeling via efficient graph learning for prediction baidu maps. In Proceedings of the 31st ACM international conference on information & knowledge management, pp. 3172-3181, 2022.
Sili Huang, Yanchao Sun, Jifeng Hu, Siyuan Guo, Hechang Chen, Yi Chang, Lichao Sun, and Bo Yang. Learning generalizable agents via saliency-guided features decorrelation. Advances in Neural Information Processing Systems, 36:39363-39381 2023.
Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1-2):99-13 1998.
Sven Koenig and Maxim Likhachev. D* lite. Fighteenth national conference on Artificial intelli- gence, pp. 476-4833 483, 2002.
Diya Li, Zhe Zhang, Bahareh Alizadeh, Ziyi Zhang, Nick Duffield, Michelle A Meyer, Courtney M Thompson, Huilin Gao, and Amir Behzadan. A reinforcement learning- based routing algorithm for large street networks. International Journal of Geographical Information Science, 38(2):183- 215, 2024.
Jian Lin, Xintao Wang, Rui Niu, and Yifan He. A q-leamnng electric vehicle routing problem. IEEE Transactions on Intelligent Transportation Systems, 2025.
beaearning-based based hyper-heuristic for capacitated

---

Published as a conference paper at ICLR 2026
Michael Littman and Richard S Sutton. Predictive representations of state. Advances in neural information processing systems, 14, 2001.
Haiyang Liu, Chunjiang Zhu, and Detian Zhang. Global-aware enhanced spatial- graph re- current networks: A new framework for traffic flow prediction. arXiv preprint arXiv:2401.04135, 2024.
Wenzheng Mao, Liu Ming, Ying Rong, Christopher S Tang, and Huan Zheng. Faster deliveries and smarter order assignments for an on-demand meal delivery platform. Journal of Operations Management, 7 71(2):200-245, 2025.
Xiaowei Mao. Haomin Wen, Hengrui Zhang, Huaiyu Wan, Lixia Wu, Jianbin Zheng, Haoyuan Hu, and Youfang Lin. Drl4route: A deep reinforcement learning framework for pick-up and delivery route prediction. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. pp. 4628 4637, 2023.
Boris Medina-Salgan Salgado, Eddy Sánchez-DelaCruz. Pilar Pozos-Parra and Javier E Sierra. Urban traffic flow prediction techniques: A review. Sustainable Computing: Informatics and Systems, 35:100739, 2022.
Erxue Min, Runfa Chen, Yatao Bian, Tingyang Xu, Kangfei Zhao, Wenbing Huang, Peilin Zhao, Junzhou Huang, Sophia Ananiadou, and Yu Rong. Transformer for graphs: An overview from architecture perspective. arXiv preprint arXiv:2202.08455. 2022.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare G Belle- mare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 51 (18(754): 2015.
Dang Viet Anh Nguyen, Carlos Lima Azevedo, Tomer Toledo, and Filipe Rodrigues. Robustness of reinforcement learning-basee based traffic signal control under incidents: A comparative study. arXiv preprint 1.XXv::550613833 2025.
Athanasios Papadopoulos, Pawel Korus, and Nasir Memon. Hard-attention for scalable image clas- sification. Advances in Neural Information Processing Systems, 34:1 144694444777, 2021.
Dian Ren, Gongquan Zhang, C Fangrong Chang, and Helai Huang. Two-step deep reinforce- ment learning for traffic signal control to improve pedestrian safety using connected vehicle data. Accident Analysis & Prevention, 222:108161, 2025.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06344 2017.
Zheyu Sun, Ningbo Cao, Xiaoyan Shao, and Chao Cai. Gnn-rl: A graph neural network-based reinforcement learning algorithm for 3d path planning of auvs. In 2025 International Conference on Mechatronics, Robotics, and Artificial Intelligence (MRAI), pp. 310-316. IEEE, 2025.
Zhen Tian, Zhihao Lin, Dezong Zhao, Wenjing Zhao, David Flynn, Shuja Ansari, and Chongfeng Wei. Evaluating scenario- based decision-making for interactive autonomous driving using ratio- nal criteria: survey. arXiv preprint arXiv:2501.01880 2025.
Hado Van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with double q- learning. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016.
Ziyu Wang, Tom Schaul, Matte Hessel, Hado Hasselt, Mare Lanctot, and Nando Freitas. Dueling network architectures for deep reinforcement learning. In International conference on machine learning, pp. 1995-2003. PMLR, 2016.
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S Yu. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 22(1):4-24, 2020.
Junxiao Xue, Jinpu Chen, and Shiwen Zhang, Action-curiosity-baaee based deep reinforcement learning algorithm for path planning in a nondeterministic environment. Intelligent Computing, 4:0140, 2025.
12

---

Published as a conference paper at ICLR 2026
Xiaohui Yang, Shaowei Sun, and Mingzhou Liu. Adaptive multi-scale apatio-temporal convolutional network with reinforcement learning for dynamic lane-level traffic flow prediction. Array, pp- 100513, 2025.
Xiao Zang, Miao Yin, Jinqi Xiao, Saman Zonouz, and Bo Yuan. Graphmp: Graph neural network- based motion planning with efficient graph search. Advances in Neural Information Processing Systems, 36:3131-3144 2023.
Shuyuan Zhao, Wei Chen, Boyan Shi, Liyong Zhou, Shuohao Lin, and Huaiyu Wan. Spatial- temporal knowledge distillation for takeaway recommendation. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 39, pp. 13365-13373, 2025.
A USE LARGE LANGUAGE MODELS
Large Language Models (LLMs) used to assist with language polishing, improving clarity and readability of the text. All scientific content, experiments, and results were designed and conducted by the authors.
13


Reviews:
[{"review_id": "S3uVFS8DGf", "reviewer": "Reviewer_yz7p", "summary": "This paper introduces a Dynamic Feature Representation (DFR) framework for Reinforcement Learning (RL)-based Dynamic Path Planning (DPP). The DFR framework refines complex global traffic dynamics into compact, decision-relevant local features by incorporating a policy attention mechanism and n-hop neighborhood methods. This significantly improves the adaptability and efficiency of RL agents in dynamic traffic environments. Extensive experiments demonstrate that DFR improves the performance of RL-based path planning models and reduces computational overhead, offering an efficient solution.", "questions": "1.\tIn the \"Model Training\" section of 5.1, could the loss function be introduced? This would help readers understand the model’s learning process.\n\n2.\tRegarding the experiments, since the authors did not compare their method with other advanced algorithms and only demonstrated the effectiveness of their algorithm through the comparison of (-1.0, -1) in 5.2.  It should be noted that (-1.0, -1) means no policy attention mechanism and no n-hop neighborhood selection. Based on my understanding, this essentially serves as an extreme baseline setup, but more baseline descriptions are needed to justify the reasonableness of this comparison.\n\n3.\tIn line 421, \"When n = 4, the model achieves a lower mean GAP than the baseline while maintaining a comparable SR,\" which baseline does this refer to? Is it (-1.0, -1) or (0.6, -1)? The significance of this result can be further analyzed rather than just describing the observed outcome.\n\n4.\tIn model training, the experiment is set to 600 episodes, but Figure 5 only shows the results for the first 175 episodes. Why weren’t the later episodes’ results displayed?\n\n5.\tThe title and axis labels in Figure 6 are incorrect.\n\n6.\tIn the ablation study section, considering that there are some label errors in Figure 6, is this the result after 600 full episodes of training? I believe Figure 6 actually conveys the same meaning as Figure 5, and the analysis in this section seems to be redundant. The overall experimental results appear more like an ablation study, without fully highlighting the advantages of the algorithm.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "1) The DFR framework combines reinforcement learning and dynamic traffic information representation, proposing a new feature representation method.\n\n2) The figures and equations are intuitive and easy to understand, effectively conveying the main content of the research.", "weaknesses": "1)\tI think the areas that need improvement in the paper are mainly in the experimental section. Firstly, the experiments only compare the performance of the DFR framework with different k and n values, without comparing it to other advanced dynamic path planning algorithms (such as other RL algorithms), which reduces the convincing power of the method's effectiveness. \n\n2)\tSecondly, there are missing details in the experimental design and analysis, such as the training details of the model and the analysis of the experimental results in Figure 5.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pHcJ0eLTrv", "reviewer": "Reviewer_ZKPV", "summary": "This paper focuses on the trade-off between completeness and efficiency in reinforcement learning state representations for urban dynamic path planning (DPP). It proposes a hierarchical dynamic feature representation framework, DFR, which combines policy attention with an n-hop neighbourhood approach to effectively compress the state space while enhancing policy performance.", "questions": "1. Methodology Explanation and Experimental Validation:\nThe experimental evaluation appears to test only fixed combinations of k and n values. It remains unclear whether the method dynamically adjusts these parameters across cities of varying scales or under different traffic densities. The paper would benefit from a more in-depth discussion on the tuning of hyperparameters k and n—specifically, how to select these parameters to achieve optimal performance under diverse network structures and traffic conditions. Furthermore, is there a method to automate the selection of these parameters to minimize manual configuration efforts? Addressing this would significantly enhance the practicality and scalability of the proposed approach.\n\n2. Generalization Capability of the Model:\nThe current experiments are conducted solely on a road network subgraph from Nanjing, which consists of a relatively small scale (378 nodes and 864 edges). This raises concerns about whether such a subgraph is representative of larger and more complex urban traffic systems. It is essential to evaluate the model's performance on road networks from other cities, particularly on larger-scale networks (e.g., thousands of nodes and tens of thousands of edges). Further experiments across cities with varying traffic conditions and network topologies are recommended to verify the model's universality and robustness.\n\n3. Insufficient Completeness of Experimental Results:\nThe paper only compares the proposed method against baseline models such as \"No Policy Attention\" and \"No Neighborhood Expansion,\" which are essentially ablative studies. There is a lack of comparison with other established path planning methods, especially traditional dynamic programming approaches in dynamic environments.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. High innovativeness: The proposed DFR framework integrates policy attention with n-hop neighbourhoods, pioneering the combination of ‘task-relevant graph sparsification’ and ‘local dynamic feature extraction’ for DPP problems. The introduction of Predictive State Representation (PSR) theory enhances interpretability and theoretical grounding.\n\n2. Methodological Rationality: The policy attention mechanism extracts task-relevant subgraphs via a pre-trained shortest path policy, effectively reducing redundant information. The n-hop neighbourhood approach further localises states, balancing computational efficiency with information integrity.", "weaknesses": "1. Methodology Explanation and Experimental Validation\n\n2. Generalization Capability of the Model\n\n3. Insufficient Completeness of Experimental Results", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wiYfHZw1K7", "reviewer": "Reviewer_5jVk", "summary": "This paper tackles the problem of Dynamic Path Planning (DPP) in urban road networks under time-varying traffic conditions. The authors focus on the challenge of designing effective state representations in reinforcement learning (RL)-based DPP, balancing completeness and efficiency.\n\nThey propose a Dynamic Feature Representation (DFR) framework that hierarchically refines global traffic dynamics into compact, decision-relevant features. The approach consists of two stages:\n\n1. Policy Attention: Pre-trained to extract a task-relevant subgraph based on top-k shortest paths;\n2. n-hop Neighborhood: Further reduces the representation to local node-centric features.\n\nExperiments on an OpenStreetMap-based Nanjing city network show that DFR improves the convergence speed and path optimality of DQN-based models while significantly reducing feature dimensionality.", "questions": "What is the computational overhead when applying DFR to real-world traffic datasets with live updates?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper presents a clear motivation and focuses on an important and practically relevant challenge in dynamic decision-making under uncertainty. The proposed hierarchical state refinement framework is conceptually novel and effectively bridges the gap between global completeness and local efficiency in RL-based path planning. The integration of Policy Attention and n-hop Neighborhood represents an  interpretable way to distill global traffic dynamics into task-relevant local states. Moreover, the authors provide a solid theoretical grounding by connecting their approach to Predictive State Representation (PSR) theory, which strengthens the methodological rigor. The experimental design is systematic, showing how different values of (k, n) influence performance, and the released code enhances reproducibility. Quantitative results show that DFR achieves faster convergence and comparable or improved routing performance with significantly lower feature dimensionality. Overall, the work offers a meaningful step toward scalable RL in real-world traffic environments.", "weaknesses": "Despite its conceptual strengths, the paper’s experimental scope is limited to a single city (Nanjing) and relatively small subgraphs, leaving the scalability and generalization of the approach unclear. The baseline comparison is also narrow, relying solely on DQN, without evaluating against more advanced or specialized methods such as GNN-RL, actor–critic algorithms, or model-based RL frameworks. The paper would benefit from more detailed explanations of the policy attention pre-training process, including the dataset, learning stability, and computational overhead. In addition, the hyperparameters k and n are fixed and manually chosen; introducing or discussing a self-adaptive mechanism for these parameters would make the approach more robust. Finally, while the theoretical connection to PSR is strong, there is limited analysis of how the temporal dependencies of traffic dynamics are preserved beyond the spatial context.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)