Echo Planning for Autonomous Driving: From Current Observations to Future Trajectories and Back

## Abstract

Abstract Modern end-to-end autonomous driving systems suffer from a critical limitation: their planners lack mechanisms to enforce temporal consistency between predicted trajectories and evolving scene dynamics. This absence of self-supervision allows early prediction errors to compound catastrophically over time. We introduce Echo Planning, a novel self-correcting framework that establishes a closed-loop Current → Future → Current (CFC) cycle to harmonize trajectory prediction with scene coherence. Our key insight is that plausible future trajectories must be bi-directionally consistent, i.e. , not only generated from current observations but also capable of reconstructing them. The CFC mechanism first predicts future trajectories from the Bird’s-Eye-View (BEV) scene representation, then inversely maps these trajectories back to estimate the current BEV state. By enforcing consistency between the original and reconstructed BEV representations through a cycle loss, the framework intrinsically penalizes physically implausible or misaligned trajectories. Experiments on nuScenes demonstrate state-of-the-art performance, reducing L2 error by 0.04 m and collision rate by 0.12% compared to one-shot planners. Crucially, our method requires no additional supervision, leveraging the CFC cycle as an inductive bias for robust planning. This work offers a deployable solution for safety-critical autonomous systems.

## 1 Introduction

Vision-based end-to-end planning has emerged as a promising direction for autonomous driving research [16, 17, 21, 52, 40, 44, 24]. Most contemporary models partition the task into perception, motion forecasting, and planning stages and rely on dense BEV feature maps for scene understanding [5, 16, 17, 21, 52, 46, 32, 51]. Although this pipeline can leverage multiple supervision signals to regulate vehicle motion, it incurs a high computational cost from the dense BEV backbone and demands separate sub-task labels, which delays feedback on rapidly changing scenes. Recent work addresses these limitations by replacing the dense BEV grid with sparse scene representations [40, 53, 24, 49], designing compact sets of scene queries that capture the essential driving context with far lower computation and reduced model complexity.

Figure: Figure 1: Comparison between one-shot paradigms and our echo planning paradigms. (a), (b), and (c) represent different types of one-shot approaches. Both (a) and (b) construct scene representations from image features and rely on auxiliary tasks to supervise the model, thereby enhancing environmental understanding; their main distinction lies in whether dense BEV features are used. Method (c) discards the auxiliary tasks introduced in (b) and is among the first to highlight the importance of temporal supervision, though it still considers only forward verification and thus remains within the one-shot paradigm. In contrast, ours (d) presents the echo planning approach, which employs a Current $\rightarrow$ Future $\rightarrow$ Current cycle. This design enforces bidirectional self-supervision, allowing the model to validate scene understanding without additional auxiliary tasks.
Refer to caption: extracted/6472113/imgs/nips25_fig1.png

Autonomous driving planning must first perceive the scene accurately and then generate a trajectory that respects the observed context. Most current approaches follow a one-shot planning paradigm, forecasting the vehicle’s future path solely from the present scene snapshot, shown in Fig [1](https://arxiv.org/html/2505.18945v1#S1.F1) (a,b,c). Despite their advantages, existing one-shot planning paradigms exhibit a critical weakness: the absence of internal mechanisms to enforce temporal consistency between predicted trajectories and dynamically evolving scenes. Such planners rely solely on forward trajectory predictions without verifying if future states remain consistent with the originally observed scene, inevitably causing early prediction inaccuracies to amplify and propagate over time, leading to brittle and unsafe driving behaviors. Hence, an open research question persists: how can we inherently embed temporal consistency in trajectory planning without incurring additional annotation overhead or heuristic complexity?

Motivated by these challenges, we introduce Echo Planning, a novel self-correcting trajectory planning framework designed specifically to enforce intrinsic temporal coherence through a closed-loop Current → Future → Current (CFC) feedback cycle, shown in Fig [1](https://arxiv.org/html/2505.18945v1#S1.F1) (d). Although prior work [24] attempted to model temporal scene variation by feeding the current BEV representation together with the planned trajectory into a network that predicts the next BEV map, which is then supervised by the ground-truth future BEV, it enforces temporal consistency only in the forward direction and neglects the reverse constraint. In our framework, the core intuition behind Echo Planning is that a robust trajectory prediction should not only plausibly project forward from current observations but also reconstruct those observations when mapped backward. Concretely, our model first generates future trajectories using a Bird’s-Eye-View (BEV) representation of the current scene, and subsequently employs an inverse reconstruction step to predict back the current BEV state from these anticipated trajectories. This cycle ensures bidirectional consistency, inherently penalizing trajectories that are physically implausible or inconsistent with the original scene configuration.

By enforcing a Current → Future → Current BEV feedback cycle, Echo Planning (EPAD) systematically reduces the impact of evolving scene dynamics on planning, preserves temporal consistency in the driving context, and consequently boosts trajectory accuracy. Experiments on the widely used nuScenes dataset [2] confirm that our approach achieves state-of-the-art performance, sharply lowering both L2 trajectory error and collision rates relative to existing one-shot planners. Crucially, these gains are obtained without any additional supervision or handcrafted heuristics, making the method both reliable and readily deployable. Echo Planning therefore provides a systematic method to intrinsically enforce trajectory–scene consistency, advancing end-to-end autonomous driving toward safer and more dependable operation. Our main contributions are summarized as follows:

- •
Self-Correcting Planning via CFC Cycles. We pioneer a cycle-loop planner that establishes a Current → Future → Current BEV feedback loop, where predicted trajectories are continuously validated through inverse reconstruction of the current BEV state. This provides built-in error detection without requiring external supervision.
- •
State-of-the-Art Reliability with Zero Annotation Overhead. Echo Planning substantially reduces collision rates and trajectory prediction errors relative to existing one-shot planners. Our self-supervised consistency checks remain elegantly simple, making our approach broadly applicable and highly complementary to supervised planning methods.

## 2 Related work

End-to-end planning. Planning is the ultimate objective of the first phase of end-to-end autonomous driving, and the literature has grown rapidly. Early work relied on relatively simple neural networks that ignored much of the scene context and therefore offered limited interpretability [36, 7, 6, 1, 8, 37, 9]. With the advent of large-scale datasets and stronger BEV perception, many learning-based planners have been proposed [16, 45, 17, 21, 52, 20, 19, 44, 40, 27, 49, 51]. Most recent systems still follow a perception, prediction, and planning pipeline to maintain transparency. ST-P3 [16] chains map perception, BEV occupancy prediction, and trajectory planning to infer future ego motion from surround cameras. UniAD [17] introduces a unified query design that integrates detection, mapping, and motion forecasting. VAD [21] uses a vectorized scene representation to couple scene understanding with planning constraints, and GenAD [52] generates future trajectories for both the ego vehicle and other agents within a learned probabilistic latent space. A newer line of research skips the explicit perception and prediction stages to facilitate efficiency [27, 24]. BEV-Planner [27] employs ego queries to extract task-relevant cues directly from BEV features and predicts the trajectory without intermediate heads. SSR [24] leverages navigation commands to focus adaptively on salient regions and learns planning without perception supervision. However, these approaches pay little attention to temporal consistency. We address this gap with a Current → Future → Current (CFC) cycle that enforces coherent scene evolution over time, leading to more accurate scene understanding and, consequently, more reliable planning.

BEV perception in autonomous driving. Perception forms the bedrock of autonomous driving, as it must distill actionable information from raw sensor streams; precise and efficient scene understanding is therefore essential. The steady evolution of bird’s-eye-view (BEV) representations [3, 13, 18, 26, 28, 30, 29, 39, 22, 31, 50] has propelled perception forward, while the lower cost of RGB cameras is gradually supplanting LiDAR-based pipelines [4, 10, 34, 46, 47]. LSS [35] is a landmark effort that used depth prediction to lift perspective features into BEV space. BEVFormer [28] introduces spatial and temporal attention within a Transformer, achieving strong camera-only detection. Consequently, dense BEV features are now the de facto scene representation in most autonomous-driving systems  [28], though their richness comes at a high computational cost. To mitigate this burden, sparse-perception methods have begun to emerge [40, 53, 49, 24]. SparseDrive [40] presents a symmetric sparse-perception module that jointly learns detection, tracking, and online mapping to produce a fully sparse scene representation. SparseAD [49] uses a compact query set to encode the driving scene sparsely. SSR [24] goes further by learning a minimal, navigation-guided token set. Our work follows this sparse-representation paradigm and, by enforcing temporal consistency through a self-supervised CFC cycle, attains higher scene-understanding accuracy for planning.

Figure: Figure 2: The overview of our Echo Planning framework. Echo Planning is trained through two complementary loops. The forward loop, indicated by black arrows in the figure, predicts future BEV features through the sparse scene representation model [24] and applies self-supervision against the ground-truth BEV of future frames. The echo loop, shown in red, takes those predicted future features via the Motion-aware Layer Normalization (MLN) module [43, 24] and TokenFuser [38, 24], reconstructs the current BEV, and self-supervises against the ground-truth BEV of the current frame. At no point are extra tasks or annotations introduced. By validating perception in both directions along the Current → Future → Current (CFC) cycle, the model cross-checks its understanding of the surrounding scene and thus produces reliable trajectory plans.
Refer to caption: extracted/6472113/imgs/nips25_fig2.png

## 3 Method

### 3.1 Overview

Given raw sensor inputs (*i.e.*, surrounding $N$-view camera images $I^{i},i=1,...,N$) and high-level navigation command $C_{navi}$ extracted from the dataset, our Echo planning model plans the future trajectory $T$ of the ego Vehicle. The overall framework is illustrated in Fig [2](https://arxiv.org/html/2505.18945v1#S2.F2). EPAD first encodes the image features with a backbone network and then uses bev queries to convert image features into BEV feature space and generate BEV features [28](Sec. [3.2](https://arxiv.org/html/2505.18945v1#S3.SS2)). Second, EPAD employs a sparse Scene Representation module [24] to distill task-critical cues from the ego-centric BEV space and condition them on the high-level navigation command, thereby mimicking the selective attention that human drivers pay to salient scene elements (Sec. [3.2](https://arxiv.org/html/2505.18945v1#S3.SS2)). Finally, at the core of our approach, EPAD introduces a Current → Future → Current (CFC) cycle: it first predicts the ego-vehicle’s future trajectory from the navigation-conditioned scene features, then inversely infers the present ego state from this forecast, and finally reconstructs the current BEV feature map (Sec. [3.3](https://arxiv.org/html/2505.18945v1#S3.SS3)). Unlike prior end-to-end planners that propagate information only in the forward temporal direction, this bidirectional reasoning mechanism leverages temporal consistency within the ego sequence, enabling richer leverage of sequential context and yielding more reliable planning.

### 3.2 Preliminary

BEV features generation.
At the $t$-th time step, the surrounding $N$-view camera images $I^{i},\left(i=1,...,N\right)$, are transformed into a BEV representation using BEVFormer [28]. Specifically, sensor images $I^{i}$ first pass through a image backbone to yield image features $F_{I}^{i},\left(i=1,...,N\right)$. Retaining the BEV representation $B_{t-1}$ from the preceding timestamp $t-1$, a group of learnable BEV queries $Q\in\mathbb{R}^{H\times W\times K}$ is utilized to simultaneously extract temporal context from the previous BEV features $B_{t-1}$ and spatial information from the current multi-camera features $\left[F_{I}^{i}\right]_{t}$ via a cross attention mechanism. This results in the updated BEV representation $B_{t}\in\mathbb{R}^{H\times W\times K}$, where $H,W$ define the spatial resolution of the BEV plane, and $K$ denotes the feature channel dimensionality.

Sparse scene representation module.
In order to efficiently represent the scene feature, some methods are proposed to leverage part of the BEV features through task-specific queries to effectively represent the scene. Given the BEV features and navigation command, the sparse scene representation module, such as SSR [24], adopts the scene token learner to extract scene queries $S\in\mathbb{R}^{N_{s}\times K}$ from dense BEV features $B$, where $N_{s}$ is the number of scene queries. Firstly, the navigation command $C_{navi}$ is encoded into the dense BEV feature [14]. Then, the navigation-aware BEV feature is used through the BEV TokenLearner module [38] and multi-layer self-attention [42] to get the current frame sparse scene representation $S_{t}$.

After obtaining the current-frame scene representation $S_{t}$ that encodes the salient BEV context, we introduce a set of waypoint queries $W_{t}\in\mathbb{R}^{N_{t}\times N_{c}\times K}$. These queries attend to $S_{t}$ to capture the prospective motion of vehicle, where $N_{t}$ is the prediction horizon (in time steps), $N_{c}$ is the number of discrete navigation commands, and $K$ is the feature dimensionality. A lightweight multi-layer perceptron subsequently regresses the waypoint embeddings into planar coordinates, producing the predicted ego trajectory $T_{P}\in\mathbb{R}^{N_{t}\times 2}$ over the next $N_{t}$ time steps. For supervision, the high-level navigation command $C_{navi}$ selects the mode-consistent branch $\hat{T}_{P}$, which is compared with the ground-truth trajectory $T_{GT}$ using an L1 loss:

$$ $\mathcal{L}_{traj}=\left\|\hat{T}_{P}-T_{GT}\right\|_{1}.$ (1) $$

### 3.3 Echo planning

Motivation.
In end-to-end autonomous driving planning, real-time scene context must inform the prediction of the trajectory of the vehicle over the next few seconds, making temporal consistency indispensable. Beyond ensuring that the forecasted future state aligns with the true future trajectory, we also require that the current state, inferred in reverse from this prediction, matches the actual present state. Motivated by this bidirectional temporal constraint, we propose EPAD, a cycle-loop self-correcting framework that leverages self-supervised training across time to substantially enhance planning accuracy without requiring external supervision. As illustrated in Fig [2](https://arxiv.org/html/2505.18945v1#S2.F2), we introduce a recurrent planner that forms a Current → Future → Current BEV cycle loop (CFC cycle), continuously validating trajectory predictions by inversely reconstructing the present BEV state.

Forward pass (Current → Future). The navigation-consistent trajectory prediction $\hat{T}_{P}$ and the current scene embedding $S_{t}$ are first processed by a Motion-aware Layer Normalization (MLN) module [43, 24], producing preliminary prediction queries. A self-attention block [42] then refines these queries into the future scene representation $\hat{S}_{t+1}$. Because the regions of interest in autonomous-driving scenes shift over time, we provide an additional source of supervision. The predicted future scene embedding $\hat{S}_{t+1}$ is fed to TokenFuser [38, 24] to hallucinate a dense BEV feature map $\hat{B}_{t+1}$. We then enforce an L2 penalty between $\hat{B}_{t+1}$ and $B_{t}$ the ground-truth BEV, yielding the future-BEV reconstruction loss $\mathcal{L}_{futbev}$:

$$ $\mathcal{L}_{futbev}=\left\|\hat{B}_{t+1}-B_{t+1}\right\|_{2}.$ (2) $$

Echo pass (Future → Current). Our goal in this stage is to start from the predicted dense BEV at $T+1$, $\hat{B}_{t+1}$, and run the pipeline in reverse to reconstruct the current BEV $\hat{B}_{t}$ for self-supervision. The Echo pass comprises four steps. Firstly, to mimic backward driving, we derive a reversed navigation command ${C}^{\prime}_{navi}$ from the original $C_{navi}$ and feed $\left[\hat{B}_{t+1},{C}^{\prime}_{navi}\right]$ into the same sparse-scene-representation module. Secondly, the scene token learner [24], equipped with an attention module, extracts predicted future scene queries $\hat{S}^{\prime}_{t+1}$ from the input BEV features $\hat{B}_{t+1}$. Thirdly, a new set of waypoint queries ${W}^{\prime}_{t+1}$ attends to $\hat{S}^{\prime}_{t+1}$, generating a reversed trajectory proposal $\hat{T}^{\prime}_{p}$, after which the inverse command ${C}^{\prime}_{navi}$ selects the mode-consistent branch of $\hat{T}^{\prime}_{p}$. Finally, analogous to the forward pass, $\hat{T}^{\prime}_{p}$ and $\hat{S}^{\prime}_{t+1}$ are processed by an MLN block followed by stacked self-attention layers, producing the current-frame scene queries $\hat{S}^{\prime}_{t}$. TokenFuser [38, 24] then reconstructs the dense BEV map $\hat{B}_{t}$ from $\hat{S}^{\prime}_{t}$.
The discrepancy between the reconstructed and ground-truth BEV features derives the current BEV reconstruction loss as:

$$ $\mathcal{L}_{curbev}=\left\|\hat{B}_{t}-B_{t}\right\|_{2}.$ (3) $$

### 3.4 Total loss

In summary, EPAD is trained end-to-end with a composite objective comprising the trajectory loss $\mathcal{L}_{traj}$, the future-BEV reconstruction loss $\mathcal{L}_{futbev}$, and the current-BEV reconstruction loss $\mathcal{L}_{curbev}$, combined as:

$$ $\mathcal{L}_{total}=\mathcal{L}_{traj}+\mathcal{L}_{futbev}+\mathcal{L}_{ curbev}.$ (4) $$

**Table 1: Planning results on the nuScenes benchmark. The first block with ^o means the lidar-based methods. The middle block follows the UniAD protocol (final/max aggregation). The lower block with ^‡ follows VAD protocol (average over all predicted frames). ^⋆: The backbone is ResNet-101 [12], while other methods without ^⋆ adopt ResNet-50 or similar variants. ^§: The result that we re-implement SSR with official weights [24]. The $\downarrow$ indicates that lower is better.**
| Method | Auxiliary Task | L2 (m) $\downarrow$ | Collision Rate (%) $\downarrow$ |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1s | 2s | 3s | Avg. | 1s | 2s | 3s | Avg. |  |  |
| NMP^o [48] | Det & Motion | 0.53 | 1.25 | 2.67 | 1.48 | 0.04 | 0.12 | 0.87 | 0.34 |
| FF^o [15] | FreeSpace | 0.55 | 1.20 | 2.54 | 1.43 | 0.06 | 0.17 | 1.07 | 0.43 |
| EO^o [23] | FreeSpace | 0.67 | 1.36 | 2.78 | 1.60 | 0.04 | 0.09 | 0.88 | 0.33 |
| ST–P3 [16] | Det & Map & Depth | 1.72 | 3.26 | 4.86 | 3.28 | 0.44 | 1.08 | 3.01 | 1.51 |
| UniAD^$\star$ [17] | Det&Track&Map&Motion&Occ | 0.48 | 0.96 | 1.65 | 1.03 | 0.05 | 0.17 | 0.71 | 0.31 |
| OccNeXt^$\star$ [41] | Det & Map & Occ | 1.29 | 2.13 | 2.99 | 2.14 | 0.21 | 0.59 | 1.37 | 0.72 |
| VAD–Base [21] | Det & Map & Motion | 0.54 | 1.15 | 1.98 | 1.22 | 0.04 | 0.39 | 1.17 | 0.53 |
| PARA–Drive [44] | Det&Track&Map&Motion&Occ | 0.40 | 0.77 | 1.31 | 0.83 | 0.07 | 0.25 | 0.60 | 0.30 |
| GenAD [52] | Det & Map & Motion | 0.36 | 0.83 | 1.55 | 0.91 | 0.06 | 0.23 | 1.00 | 0.43 |
| UAD–Tiny [11] | Det | 0.47 | 0.99 | 1.71 | 1.06 | 0.08 | 0.39 | 0.90 | 0.46 |
| UAD^$\star$ [11] | Det | 0.39 | 0.81 | 1.50 | 0.90 | 0.01 | 0.12 | 0.43 | 0.19 |
| SSR^§ [24] | None | 0.25 | 0.64 | 1.33 | 0.74 | 0.16 | 0.21 | 0.51 | 0.29 |
| EPAD (Ours) | None | 0.23 | 0.60 | 1.27 | 0.70 | 0.02 | 0.12 | 0.39 | 0.17 |
| ST–P3^‡ [16] | Det & Map & Depth | 1.33 | 2.11 | 2.90 | 2.11 | 0.23 | 0.62 | 1.27 | 0.71 |
| UniAD^$\star{\ddagger}$ [17] | Det&Track&Map&Motion&Occ | 0.44 | 0.67 | 0.96 | 0.69 | 0.04 | 0.08 | 0.23 | 0.12 |
| VAD–Tiny^‡ [21] | Det & Map & Motion | 0.46 | 0.76 | 1.12 | 0.78 | 0.21 | 0.35 | 0.58 | 0.38 |
| VAD–Base^‡ [21] | Det & Map & Motion | 0.41 | 0.70 | 1.05 | 0.72 | 0.07 | 0.17 | 0.41 | 0.22 |
| BEV–Planner^‡ [27] | None | 0.28 | 0.42 | 0.68 | 0.46 | 0.04 | 0.37 | 1.07 | 0.49 |
| PARA–Drive^‡ [44] | Det&Track&Map&Motion&Occ | 0.25 | 0.46 | 0.74 | 0.48 | 0.14 | 0.23 | 0.39 | 0.25 |
| LAW^‡ [25] | None | 0.26 | 0.57 | 1.01 | 0.61 | 0.14 | 0.21 | 0.54 | 0.30 |
| GenAD^‡ [52] | Det & Map & Motion | 0.28 | 0.49 | 0.78 | 0.52 | 0.08 | 0.14 | 0.34 | 0.19 |
| SparseDrive^‡ [40] | Det&Track&Map&Motion | 0.29 | 0.58 | 0.96 | 0.61 | 0.01 | 0.05 | 0.18 | 0.08 |
| UAD^$\star{\ddagger}$ [11] | Det | 0.28 | 0.41 | 0.65 | 0.45 | 0.01 | 0.03 | 0.14 | 0.06 |
| SSR^§^‡ [24] | None | 0.19 | 0.36 | 0.62 | 0.39 | 0.10 | 0.13 | 0.22 | 0.15 |
| EPAD^‡ (Ours) | None | 0.17 | 0.33 | 0.58 | 0.36 | 0.03 | 0.05 | 0.14 | 0.07 |

**Table 2: Ablation for our CFC cycle. The upper block follows the UniAD protocol (final/max aggregation). The lower block with ^‡ follows VAD protocol (average over all predicted frames).^§: The result that we retrain the SSR [24] on 4 GPUs. The $\downarrow$ indicates that lower is better.**
| Method | CFC cycle | L2 (m) $\downarrow$ | Collision Rate (%) $\downarrow$ |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1s | 2s | 3s | Avg. | 1s | 2s | 3s | Avg. |  |  |
| SSR^§ [24] |  | 0.25 | 0.64 | 1.33 | 0.74 | 0.16 | 0.21 | 0.51 | 0.29 |
| EPAD (Ours) | ✓ | 0.23 | 0.60 | 1.27 | 0.70 | 0.02 | 0.12 | 0.39 | 0.17 |
| SSR^§^‡ [24] |  | 0.19 | 0.36 | 0.62 | 0.39 | 0.10 | 0.13 | 0.22 | 0.15 |
| EPAD^‡ (Ours) | ✓ | 0.17 | 0.33 | 0.58 | 0.36 | 0.03 | 0.05 | 0.14 | 0.07 |

**Table 4: Ablation for the loss weight. The table reports how varying the weights of the proposed loss terms, *i.e.*, $\mathcal{L}_{curbev}$, $\mathcal{L}_{futbev}$, influences the planner’s performance. The block without a gray background follows the UniAD protocol (final/max aggregation), while the block with a gray background follows the VAD protocol (average over all predicted frames). The $\downarrow$ indicates that lower is better.**
| $\mathcal{L}_{curbev}$ | $\mathcal{L}_{futbev}$ | L2 (m) $\downarrow$ | Collision Rate (%) $\downarrow$ |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1s | 2s | 3s | Avg. | 1s | 2s | 3s | Avg. |  |  |
| 0.1 | 0.5 | 0.23 | 0.60 | 1.27 | 0.70 | 0.02 | 0.12 | 0.39 | 0.17 |
| 0.5 | 0.5 | 0.24 | 0.63 | 1.34 | 0.73 | 0.00 | 0.10 | 0.35 | 0.15 |
| 0.1 | 0.8 | 0.24 | 0.63 | 1.31 | 0.72 | 0.04 | 0.14 | 0.45 | 0.21 |
| 0.1 | 0.5 | 0.17 | 0.33 | 0.58 | 0.36 | 0.03 | 0.05 | 0.14 | 0.07 |
| 0.5 | 0.5 | 0.19 | 0.36 | 0.64 | 0.39 | 0.08 | 0.06 | 0.13 | 0.09 |
| 0.1 | 0.8 | 0.18 | 0.35 | 0.61 | 0.38 | 0.00 | 0.05 | 0.21 | 0.09 |

Figure: Figure 3: Visualization of planning trajectories in different scenes. We visualize several driving scenes, overlaying the trajectories predicted by our method alongside those of one of the one-shot planners, *i.e.*, GenAD [52], and the ground-truth trajectories. Map context is rendered directly from the dataset annotations. In each figure, the green icon denotes the ego vehicle, while red icons mark surrounding vehicles and other dynamic objects.
Refer to caption: extracted/6472113/imgs/nips25_fig3new.png

## 4 Experiment

### 4.1 Experimental settings

Dataset.
We conduct extensive experiments on the widely adopted nuScenes dataset [2] to assess the end-to-end planning capability of our EPAD framework. nuScenes comprises 1,000 driving logs, partitioned into 700, 150, and 150 scenes for training, validation, and testing, respectively. Each scene provides 20 seconds of synchronized RGB and LiDAR data captured at 12 Hz, together with 2 Hz key-frame annotations. The ego vehicle is equipped with six surround cameras that cover a full 360° field of view and a 32-beam LiDAR; the dataset supplies dense semantic maps along with 1.4 M three-dimensional bounding boxes spanning 23 object classes.

Evaluation metrics.
Following prior work, we evaluate with two metrics: L2 displacement error (the Euclidean distance between the planned and ground-truth trajectories) and collision rate (the proportion of planned trajectories that collide with any traffic participant). Unless specified otherwise, the model is fed with 2 s of history corresponding to five frames, and planning quality is reported for 1 s, 2 s, and 3 s horizons. Because VAD [21] and UniAD [17] adopt different aggregation rules, we report both results. VAD averages over all predicted frames, whereas UniAD uses the final value (maximum collision). Table [1](https://arxiv.org/html/2505.18945v1#S3.T1) presents the outcomes under both protocols.

Implementation details.
EPAD is trained for 12 epochs on 4 RTX-A6000 GPUs with a batch size of 1 per GPU. The image encoder is a ResNet-50 [12] fed with 640 × 360 images, yielding a 100 × 100 BEV grid and 16 × 256 sparse tokens. The navigation-command set remains 3, matching prior work. For training, optimization uses AdamW [33] with a learning rate of $5\times 10^{-5}$, and all other hyperparameters follow SSR [24]. Unless stated, loss weights for trajectory, future-BEV, and current-BEV are 1.0, 0.5, and 0.1 by default.

### 4.2 Comparison with the State-of-the-art Methods

To assess the effectiveness of Echo Planning, we train and evaluate the model on the widely used nuScenes benchmark. As summarised in Table [1](https://arxiv.org/html/2505.18945v1#S3.T1), Echo Planning achieves the best results among contemporary end-to-end planners on both the L2 error and the collision rate, attaining an average $L2_{MAX}$ error of only 0.70 m.
Compared to strong one-shot planners, our approach delivers consistent benefits. Against UniAD the average $L2_{MAX}$ error drops by 0.33 m (32% relatively) and the average $CR_{MAX}$ falls by 0.14% (45% relatively). Compared with VAD-Base, the gains are even larger, with reductions of 0.52 m in average $L2_{MAX}$ (43% relatively) and 0.36% in average $CR_{MAX}$ (68% relatively). In addition, we compare our approach to the SSR baseline. Due to recent updates in the SSR codebase, our reproduced results on a 4-GPU setup differ from those reported in the original paper, a point also acknowledged by the authors on GitHub, as their modifications affected collision rate performance. For a fair comparison, we benchmark Echo Planning against our reproduced SSR results and observe substantial improvements in both L2 error and collision rate. Echo Planning still lowers the average $L2_{MAX}$ by 0.04 m and the average $CR_{MAX}$ by 0.12%, while also improving the $L2_{AVG}$ and $CR_{AVG}$ metrics by 0.03 m and 0.08% respectively. The consistent improvements across all indicators confirm that the Current → Future → Current cycle provides a reliable mechanism for accurate and safe trajectory planning.

Figure: Figure 4: Visualization of failure cases. We highlight two distinct planning failure cases. The first arises when the navigation cue in the ground-truth annotation is ambiguous, and the second occurs in open spaces where the predicted trajectory deviates during a turn. In each figure, the green icon denotes the ego vehicle, while red icons mark surrounding vehicles and other dynamic objects.
Refer to caption: extracted/6472113/imgs/nips25_fig4.png

### 4.3 Ablation studies and further discussion

Effect of the CFC cycle. In Table [3](https://arxiv.org/html/2505.18945v1#S3.T3) we present an ablation of the Current → Future → Current BEV cycle loop (CFC cycle). Applying the CFC cycle to our one-shot planners reduces the average $L2_{MAX}$ error by 0.04 m and lowers the average $CR_{MAX}$ by 0.12%. At the same time, the average $L2_{AVG}$ error falls by 0.03 m and the average $CR_{AVG}$ drops by 0.08%. These results highlight the significant benefit of introducing bidirectional validation through inverse reconstruction, thereby improving model understanding of dynamic driving scenes. Our Echo Planning framework consequently enables safer trajectory generation and leads to substantial improvements in planning.

Effect of different scene representation numbers. Table [3](https://arxiv.org/html/2505.18945v1#S3.T3) assesses the robustness of the CFC cycle when the scene representation is compressed to different capacities. We compare two settings: 8 and 16 scene tokens, while keeping all other factors fixed. In both cases, the CFC-augmented model secures clear improvements over its non-cycle counterpart. Remarkably, even with only 8 tokens, our method can still learn the surrounding environment of the vehicle very well, confirming that the cycle constraint remains effective even in highly compact representations.

Effect of the loss weight. As shown in Table [4](https://arxiv.org/html/2505.18945v1#S3.T4), we show an ablation study on the weights of our three loss terms, with emphasis on the two components tied to the Current → Future → Current (CFC) BEV cycle. Keeping the trajectory loss $\mathcal{L}_{traj}$ fixed, we systematically vary the forward-pass loss $\mathcal{L}_{futbev}$ and the echo-pass loss $\mathcal{L}_{curbev}$. The results show that planning quality remains consistently high across a broad range of weight settings, underscoring the stability imparted by the CFC cycle and confirming that EPAD reliably strengthens planning performance.

Why the Current → Future → Current BEV cycle loop (CFC cycle) effective? Accurate trajectory planning requires the model to form a faithful understanding of the driving scene. Most recent planners follow a one-shot paradigm: the network observes a single sensor frame, predicts the future trajectory, and is trained either with auxiliary labels such as detection or mapping or with a forward self-supervised loss that compares the predicted future scene to ground truth. This design leaves two critical gaps. First, heavy auxiliary heads greatly increase model complexity while offering no direct feedback on how well the planner actually understands its surroundings. Second, relying only on forward supervision ignores the reverse constraint: because the driving scene evolves unpredictably, a plan generated from the present frame alone may remain unverified and can drift from reality. Echo Planning addresses these issues with a CFC cycle. Without any extra supervision, the framework not only learns from the forward prediction of the future scene but also reconstructs the current BEV from that prediction, thereby checking whether the imagined future is consistent with the observed present. This bidirectional loop provides an intrinsic error detector that continuously validates trajectories and significantly lowers the probability of collisions, yielding more reliable planning.

### 4.4 Qualitative results

Visualization of planning trajectories. Figure [3](https://arxiv.org/html/2505.18945v1#S3.F3) qualitatively compares trajectories planned by EPAD with ground truth (GT) and GenAD [52] across four representative scenarios.
Figure [3](https://arxiv.org/html/2505.18945v1#S3.F3) (a) depicts an intersection requiring a left turn. EPAD generates a smoother and more anticipatory turning arc than the GT trajectory, aligning better with the lane geometry.
Figure [3](https://arxiv.org/html/2505.18945v1#S3.F3) (b) considers driving conditions under rain-induced poor visibility. Despite water puddles and reflections obscuring lane markings, EPAD accurately infers the roadway layout, generating a right-turn trajectory more faithful to the lane than the GT.
Figure [3](https://arxiv.org/html/2505.18945v1#S3.F3) (c) illustrates navigation through a complex multi-way junction. Here, EPAD closely follows the GT trajectory without crossing road boundaries, whereas GenAD drifts into non-drivable areas.
Figure [3](https://arxiv.org/html/2505.18945v1#S3.F3) (d) shows a scenario of low-light night driving. Under dim illumination, EPAD closely matches the GT trajectory, while GenAD exhibits significant deviations. These examples validate that the Current → Future → Current cycle in EPAD intrinsically corrects trajectory predictions, enhancing robustness across diverse conditions.

Failure cases. Figure [4](https://arxiv.org/html/2505.18945v1#S4.F4) visualizes two representative failure modes of EPAD.
The first stems from ambiguous navigation cues in the nuScenes annotation: in Fig [4](https://arxiv.org/html/2505.18945v1#S4.F4) (a), the annotation marks the command as “go straight,” yet the ground-truth trajectory actually includes a rightward lane change. When such inconsistencies arise, the planner can be misled and produce a biased path.
The second limitation appears on very wide roads that lack explicit lane markings. Although the CFC cycle improves scene understanding, an open roadway grants considerable freedom even for human drivers. As a result, EPAD may generate a trajectory that deviates from the ground truth, as in Fig [4](https://arxiv.org/html/2505.18945v1#S4.F4) (b), while still remaining within a plausible driving corridor and preserving the correct turning direction.

## 5 Conclusion

The Echo Planning framework introduces a self-correcting Current → Future → Current BEV cycle loop (CFC cycle) that endows end-to-end planners with an intrinsic notion of temporal consistency. By predicting a future trajectory, inverting it to reconstruct the present BEV, and penalising any mismatch through a cycle loss, the framework detects and suppresses implausible motion without extra labels or auxiliary heads. Extensive nuScenes experiments confirm that this bidirectional feedback reduces displacement error and collision rate beyond state-of-the-art one-shot planners, while incurring no annotation overhead and minimal computation. Echo Planning, therefore, offers a practical, safety-oriented upgrade for modern autonomous-driving stacks, and we expect its cycle-consistency principle to extend naturally to longer horizons, richer modalities, and closed-loop simulation in future work.

Limitations. Echo Planning improves temporal consistency without external supervision, but several limitations inherit from datasets and backbones. First, very long-range dependencies can be imperfectly captured when traffic evolves over many seconds. Second, the planner assumes that navigation commands are unambiguous. Mislabeled or contradictory instructions can degrade performance. Finally, while the cycle loss detects gross inconsistencies, it does not explicitly reason about rare corner cases such as sudden occlusions.

## References

- Bojarski et al. [2016]
Mariusz Bojarski, DavideDel Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, L.D. Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, Xin Zhang, Jake Zhao, and Karol Zieba.
End to end learning for self-driving cars.
*arXiv: Computer Vision and Pattern Recognition,arXiv: Computer Vision and Pattern Recognition*, Apr 2016.
- Caesar et al. [2020]
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom.
nuscenes: A multimodal dataset for autonomous driving.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 11621–11631, 2020.
- Chen et al. [2022]
Shaoyu Chen, Tianheng Cheng, Xinggang Wang, Wenming Meng, Qian Zhang, and Wenyu Liu.
Efficient and robust 2d-to-bev representation learning via geometry-guided kernel transformer.
*arXiv preprint arXiv:2206.04584*, 2022.
- Chen et al. [2023]
Yukang Chen, Jianhui Liu, Xiangyu Zhang, Xiaojuan Qi, and Jiaya Jia.
Voxelnext: Fully sparse voxelnet for 3d object detection and tracking.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 21674–21683, 2023.
- Chen et al. [2024]
Zhili Chen, Maosheng Ye, Shuangjie Xu, Tongyi Cao, and Qifeng Chen.
Ppad: Iterative interactions of prediction and planning for end-to-end autonomous driving.
In *European Conference on Computer Vision*, pages 239–256. Springer, 2024.
- Cheng et al. [2022]
Jie Cheng, Ren Xin, Sheng Wang, and Ming Liu.
Mpnp: Multi-policy neural planner for urban driving.
In *2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*, pages 10549–10554. IEEE, 2022.
- Codevilla et al. [2018]
Felipe Codevilla, Matthias Muller, Antonio Lopez, Vladlen Koltun, and Alexey Dosovitskiy.
End-to-end driving via conditional imitation learning.
In *2018 IEEE International Conference on Robotics and Automation (ICRA)*, May 2018.
doi: 10.1109/icra.2018.8460487.
URL [http://dx.doi.org/10.1109/icra.2018.8460487](http://dx.doi.org/10.1109/icra.2018.8460487).
- Codevilla et al. [2019]
Felipe Codevilla, Eder Santana, Antonio M López, and Adrien Gaidon.
Exploring the limitations of behavior cloning for autonomous driving.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pages 9329–9338, 2019.
- Dauner et al. [2023]
Daniel Dauner, Marcel Hallgarten, Andreas Geiger, and Kashyap Chitta.
Parting with misconceptions about learning-based vehicle motion planning.
In *Conference on Robot Learning*, pages 1268–1281. PMLR, 2023.
- Graham et al. [2018]
Benjamin Graham, Martin Engelcke, and Laurens van der Maaten.
3d semantic segmentation with submanifold sparse convolutional networks.
In *2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition*, Jun 2018.
doi: 10.1109/cvpr.2018.00961.
URL [http://dx.doi.org/10.1109/cvpr.2018.00961](http://dx.doi.org/10.1109/cvpr.2018.00961).
- Guo et al. [2024]
Mingzhe Guo, Zhipeng Zhang, Yuan He, Ke Wang, and Liping Jing.
End-to-end autonomous driving without costly modularization and 3d manual annotation.
*arXiv preprint arXiv:2406.17680*, 2024.
- He et al. [2016]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.
Deep residual learning for image recognition.
In *Proceedings of the IEEE conference on computer vision and pattern recognition*, pages 770–778, 2016.
- Hu et al. [2021a]
Anthony Hu, Zak Murez, Nikhil Mohan, Sofia Dudas, Jeffrey Hawke, Vijay Badrinarayanan, Roberto Cipolla, and Alex Kendall.
Fiery: Future instance prediction in bird’s-eye view from surround monocular cameras.
In *2021 IEEE/CVF International Conference on Computer Vision (ICCV)*, Oct 2021a.
doi: 10.1109/iccv48922.2021.01499.
URL [http://dx.doi.org/10.1109/iccv48922.2021.01499](http://dx.doi.org/10.1109/iccv48922.2021.01499).
- Hu et al. [2018]
Jie Hu, Li Shen, and Gang Sun.
Squeeze-and-excitation networks.
In *2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition*, Jun 2018.
doi: 10.1109/cvpr.2018.00745.
URL [http://dx.doi.org/10.1109/cvpr.2018.00745](http://dx.doi.org/10.1109/cvpr.2018.00745).
- Hu et al. [2021b]
Peiyun Hu, Aaron Huang, John Dolan, David Held, and Deva Ramanan.
Safe local motion planning with self-supervised freespace forecasting.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 12732–12741, 2021b.
- Hu et al. [2022]
Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, and Dacheng Tao.
St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning.
In *European Conference on Computer Vision*, pages 533–549. Springer, 2022.
- Hu et al. [2023]
Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al.
Planning-oriented autonomous driving.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 17853–17862, 2023.
- Huang et al. [2021]
Junjie Huang, Guan Huang, Zheng Zhu, Yun Ye, and Dalong Du.
Bevdet: High-performance multi-camera 3d object detection in bird-eye-view.
*arXiv preprint arXiv:2112.11790*, 2021.
- Jia et al. [2023]
Xiaosong Jia, Penghao Wu, Li Chen, Jiangwei Xie, Conghui He, Junchi Yan, and Hongyang Li.
Think twice before driving: Towards scalable decoders for end-to-end autonomous driving.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 21983–21994, 2023.
- Jia et al. [2024]
Xiaosong Jia, Zhenjie Yang, Qifeng Li, Zhiyuan Zhang, and Junchi Yan.
Bench2drive: Towards multi-ability benchmarking of closed-loop end-to-end autonomous driving.
In *NeurIPS 2024 Datasets and Benchmarks Track*, 2024.
- Jiang et al. [2023]
Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang.
Vad: Vectorized scene representation for efficient autonomous driving.
In *Proceedings of the IEEE/CVF International Conference on Computer Vision*, pages 8340–8350, 2023.
- Ju et al. [2024]
Hao Ju, Shaofei Huang, Si Liu, and Zhedong Zheng.
Video2bev: Transforming drone videos to bevs for video-based geo-localization.
*arXiv preprint arXiv:2411.13610*, 2024.
- Khurana et al. [2022]
Tarasha Khurana, Peiyun Hu, Achal Dave, Jason Ziglar, David Held, and Deva Ramanan.
Differentiable raycasting for self-supervised occupancy forecasting.
In *European Conference on Computer Vision*, pages 353–369. Springer, 2022.
- Li and Cui [2025]
Peidong Li and Dixiao Cui.
Navigation-guided sparse scene representation for end-to-end autonomous driving.
In *The Thirteenth International Conference on Learning Representations*, 2025.
- Li et al. [2024a]
Yingyan Li, Lue Fan, Jiawei He, Yuqi Wang, Yuntao Chen, Zhaoxiang Zhang, and Tieniu Tan.
Enhancing end-to-end autonomous driving with latent world model.
*arXiv preprint arXiv:2406.08481*, 2024a.
- Li et al. [2023]
Yinhao Li, Zheng Ge, Guanyi Yu, Jinrong Yang, Zengran Wang, Yukang Shi, Jianjian Sun, and Zeming Li.
Bevdepth: Acquisition of reliable depth for multi-view 3d object detection.
In *Proceedings of the AAAI conference on artificial intelligence*, volume 37, pages 1477–1485, 2023.
- Li et al. [2024b]
Zhiqi Li, Zhiding Yu, Shiyi Lan, Jiahan Li, Jan Kautz, Tong Lu, and Jose M Alvarez.
Is ego status all you need for open-loop end-to-end autonomous driving?
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 14864–14873, 2024b.
- Li et al. [2025]
Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Qiao Yu, and Jifeng Dai.
Bevformer: Learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers.
*IEEE Transactions on Pattern Analysis and Machine Intelligence*, 47(3):2020–2036, 2025.
doi: 10.1109/TPAMI.2024.3515454.
- Liao et al. [2022]
Bencheng Liao, Shaoyu Chen, Xinggang Wang, Tianheng Cheng, Qian Zhang, Wenyu Liu, and Chang Huang.
Maptr: Structured modeling and learning for online vectorized hd map construction.
*arXiv preprint arXiv:2208.14437*, 2022.
- Liao et al. [2024]
Bencheng Liao, Shaoyu Chen, Bo Jiang, Tianheng Cheng, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang.
Lane graph as path: Continuity-preserving path-wise modeling for online lane graph construction.
In *European Conference on Computer Vision*, pages 334–351. Springer, 2024.
- Liu et al. [2023a]
Zhi Liu, Shaoyu Chen, Xiaojie Guo, Xinggang Wang, Tianheng Cheng, Hongmei Zhu, Qian Zhang, Wenyu Liu, and Yi Zhang.
Vision-based uneven bev representation learning with polar rasterization and surface estimation.
In *Conference on Robot Learning*, pages 437–446. PMLR, 2023a.
- Liu et al. [2023b]
Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela L Rus, and Song Han.
Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation.
In *2023 IEEE international conference on robotics and automation (ICRA)*, pages 2774–2781. IEEE, 2023b.
- Loshchilov and Hutter [2017]
Ilya Loshchilov and Frank Hutter.
Decoupled weight decay regularization.
*arXiv preprint arXiv:1711.05101*, 2017.
- Mao et al. [2021]
Jiageng Mao, Yujing Xue, Minzhe Niu, Haoyue Bai, Jiashi Feng, Xiaodan Liang, Hang Xu, and Chunjing Xu.
Voxel transformer for 3d object detection.
In *2021 IEEE/CVF International Conference on Computer Vision (ICCV)*, Oct 2021.
doi: 10.1109/iccv48922.2021.00315.
URL [http://dx.doi.org/10.1109/iccv48922.2021.00315](http://dx.doi.org/10.1109/iccv48922.2021.00315).
- Philion and Fidler [2020]
Jonah Philion and Sanja Fidler.
*Lift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D*, page 194–210.
Jan 2020.
doi: 10.1007/978-3-030-58568-6_12.
URL [http://dx.doi.org/10.1007/978-3-030-58568-6_12](http://dx.doi.org/10.1007/978-3-030-58568-6_12).
- Pomerleau [1988]
Dean Pomerleau.
Alvinn: An autonomous land vehicle in a neural network.
*Neural Information Processing Systems,Neural Information Processing Systems*, Jan 1988.
doi: 10.1184/r1/6603146.v1.
- Prakash et al. [2021]
Aditya Prakash, Kashyap Chitta, and Andreas Geiger.
Multi-modal fusion transformer for end-to-end autonomous driving.
In *2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, Jun 2021.
doi: 10.1109/cvpr46437.2021.00700.
URL [http://dx.doi.org/10.1109/cvpr46437.2021.00700](http://dx.doi.org/10.1109/cvpr46437.2021.00700).
- Ryoo et al. [2021]
Michael Ryoo, AJ Piergiovanni, Anurag Arnab, Mostafa Dehghani, and Anelia Angelova.
Tokenlearner: Adaptive space-time tokenization for videos.
*Advances in neural information processing systems*, 34:12786–12797, 2021.
- Shao et al. [2023]
Hao Shao, Letian Wang, Ruobing Chen, Steven L Waslander, Hongsheng Li, and Yu Liu.
Reasonnet: End-to-end driving with temporal and global reasoning.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 13723–13733, 2023.
- Sun et al. [2024]
Wenchao Sun, Xuewu Lin, Yining Shi, Chuang Zhang, Haoran Wu, and Sifa Zheng.
Sparsedrive: End-to-end autonomous driving via sparse scene representation.
*arXiv preprint arXiv:2405.19620*, 2024.
- Tong et al. [2023]
Wenwen Tong, Chonghao Sima, Tai Wang, Li Chen, Silei Wu, Hanming Deng, Yi Gu, Lewei Lu, Ping Luo, Dahua Lin, et al.
Scene as occupancy.
In *Proceedings of the IEEE/CVF International Conference on Computer Vision*, pages 8406–8415, 2023.
- Vaswani et al. [2017]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, AidanN. Gomez, Lukasz Kaiser, and Illia Polosukhin.
Attention is all you need.
*Neural Information Processing Systems,Neural Information Processing Systems*, Jun 2017.
- Wang et al. [2023]
Shihao Wang, Yingfei Liu, Tiancai Wang, Ying Li, and Xiangyu Zhang.
Exploring object-centric temporal modeling for efficient multi-view 3d object detection.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pages 3621–3631, 2023.
- Weng et al. [2024]
Xinshuo Weng, Boris Ivanovic, Yan Wang, Yue Wang, and Marco Pavone.
Para-drive: Parallelized architecture for real-time autonomous driving.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 15449–15458, 2024.
- Wu et al. [2022]
Penghao Wu, Xiaosong Jia, Li Chen, Junchi Yan, Hongyang Li, and Yu Qiao.
Trajectory-guided control prediction for end-to-end autonomous driving: A simple yet strong baseline.
*Advances in Neural Information Processing Systems*, 35:6119–6132, 2022.
- Ye et al. [2023]
Tengju Ye, Wei Jing, Chunyong Hu, Shikun Huang, Lingping Gao, Fangzhen Li, Jingke Wang, Ke Guo, Wencong Xiao, Weibo Mao, et al.
Fusionad: Multi-modality fusion for prediction and planning tasks of autonomous driving.
*arXiv preprint arXiv:2308.01006*, 2023.
- Yuan et al. [2024]
Chengran Yuan, Zhanqi Zhang, Jiawei Sun, Shuo Sun, Zefan Huang, Christina Dao Wen Lee, Dongen Li, Yuhang Han, Anthony Wong, Keng Peng Tee, et al.
Drama: An efficient end-to-end motion planner for autonomous driving with mamba.
*arXiv preprint arXiv:2408.03601*, 2024.
- Zeng et al. [2019]
Wenyuan Zeng, Wenjie Luo, Simon Suo, Abbas Sadat, Bin Yang, Sergio Casas, and Raquel Urtasun.
End-to-end interpretable neural motion planner.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 8660–8669, 2019.
- Zhang et al. [2024a]
Diankun Zhang, Guoan Wang, Runwen Zhu, Jianbo Zhao, Xiwu Chen, Siyu Zhang, Jiahao Gong, Qibin Zhou, Wenyuan Zhang, Ningzi Wang, et al.
Sparsead: Sparse query-centric paradigm for efficient end-to-end autonomous driving.
*arXiv preprint arXiv:2404.06892*, 2024a.
- Zhang et al. [2022]
Yunpeng Zhang, Zheng Zhu, Wenzhao Zheng, Junjie Huang, Guan Huang, Jie Zhou, and Jiwen Lu.
Beverse: Unified perception and prediction in birds-eye-view for vision-centric autonomous driving.
*arXiv preprint arXiv:2205.09743*, 2022.
- Zhang et al. [2024b]
Yunpeng Zhang, Deheng Qian, Ding Li, Yifeng Pan, Yong Chen, Zhenbao Liang, Zhiyao Zhang, Shurui Zhang, Hongxu Li, Maolei Fu, et al.
Graphad: Interaction scene graph for end-to-end autonomous driving.
*arXiv preprint arXiv:2403.19098*, 2024b.
- Zheng et al. [2024]
Wenzhao Zheng, Ruiqi Song, Xianda Guo, Chenming Zhang, and Long Chen.
Genad: Generative end-to-end autonomous driving.
In *European Conference on Computer Vision*, pages 87–104. Springer, 2024.
- Zhu et al. [2024]
Rui-Jie Zhu, Ziqing Wang, Leilani Gilpin, and Jason Eshraghian.
Autonomous driving with spiking neural networks.
*Advances in Neural Information Processing Systems*, 37:136782–136804, 2024.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: - How does EchoP performance compare to BridgeAD and MomAD, which also aim to enforce temporal consistency in dense planning?
- How does EchoP perform under the temporal prediction consistency (TPC) metric introduced in MomAD?
- Does EchoP introduce any additional inference latency relative to SSR?
- Could similar consistency be achieved through directly predicting the previous BEV frame  (t-1) or a further future BEV (t+2), without the explicit cycle loss?
soundness: 3
strengths: - Introduces a cycle-consistency self-supervision scheme to enhances temporal coherence with bidirectional consistency in the planning process.
- Demonstrates improvements across open- and closed-loop evaluations, supported by ablations on loss weighting and token size.
- The formulation is conceptually intuitive and integrates easily into existing sparse token planning frameworks such as SSR.
summary: This paper proposes Echo Planning (EchoP), a self-supervised end-to-end autonomous driving framework that enforces temporal consistency between predicted trajectories and scene dynamics through a future–past cycle consistency. The model predicts the future BEV representation and then the current BEV representation, penalizing inconsistencies via a cycle-consistency loss. Experiments on nuScenes and Bench2Drive demonstrate improvements over strong baselines such as UniAD and SSR with gains in open and closed-loop driving metrics.
weaknesses: - The paper omits discussion of recent, related works on temporal consistency in end-to-end planning, notably BridgeAD (CVPR 2025) and MomAD (CVPR 2025). While these methods differ in formulation (historical aggregation, momentum modeling) they share similar motivations.
- The conceptual advance over SSR is modest. It is unclear whether explicit cycle consistency is essential, as no alternative reconstruction formulations were tested. For instance, directly predicting the previous BEV (t−1) or a further future BEV (t+2).
- Open-loop performance gains are minor and do not clearly demonstrate stronger generalization beyond the SSR baseline.
- Computational cost and latency metrics are not reported.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: 1. What makes the CFC cycle superior to forward-only or future-conditioned losses? Has alternatives like future-to-current reconstruction been explored?

2. Is the reverse navigation command (flipping left/right) sufficient for modeling backward dynamics? How does this handle U-turns or complex maneuvers?

3. Why does LiDAR-free, camera-only BEV suffice for bidirectional reconstruction? Is the geometry preservation under inversion guaranteed? Is there any error evaluations on the reconstructed performance of BEV features?

4. The cycle loss is applied in feature space (BEV) explicitly, rather than trajectory space. How does this ensure trajectory plausibility?

5. Why is batch size = 1 per GPU used, which maybe harmful for stable training. Is this due to memory constraints or design choice? What is the training time compared to SSR?

6. Scalability study. Why is ResNet-50 used instead of stronger backbones (e.g., ResNet-101, ViT)? Is the improvement backbone-agnostic? Is it possible for such framework to scale up with larger model size?

7. Why is closed-loop success rate on Bench2Drive still low (26.54%)? What are the main failure modes in Bench2Drive?
soundness: 2
strengths: - Sufficient technical details. The CFC cycle is a creative and intuitive way to enforce bidirectional temporal consistency without extra labels, distinguishing it from prior one-shot planners.

- Strong empirical results. EchoP achieves SOTA performance on both open-loop (nuScenes) and closed-loop (Bench2Drive) benchmarks, with notable collision rate reductions.

- Exhaustive Ablation studies. Comprehensive ablations on loss weights, token numbers, and CFC cycle inclusion demonstrate methodological rigor and effectiveness of module design.

- Qualitative insights: Provided visualizations effectively illustrate how EchoP corrects trajectory drift and handles challenging driving scenes (e.g., low-light, intersections).
summary: This paper introduces Echo Planning (EchoP), a self-correcting framework for end-to-end autonomous driving that enforces temporal consistency via a Current → Future → Current (CFC) cycle. The key idea is that predicted trajectories should not only be consistent with future states but also be able to reconstruct the current scene, thereby acting as an implicit consistency check. EchoP achieves this by predicting future trajectories from BEV features, then reconstructing the current BEV from those predictions with inverse driving command, and penalizing inconsistencies. The method is evaluated on nuScenes and Bench2Drive, showing performance improvements without requiring additional supervision.
weaknesses: - Limited novelty. The CFC cycle is conceptually interesting, but the core modules (BEVFormer, TokenLearner, TokenFuser, MLN) are borrowed from prior works. It seem that the main contribution of EchoP is the additional branch for reversed reconstruction compared to SSR, which is somewhat limited.

- Underexplored theoretical grounding: The bidirectional consistency idea is intuitive but lacks formal justification or comparison with alternative consistency losses (e.g., contrastive, adversarial).

- Missing comparisons with new baselines. Some baselines (e.g., NMP, FF, EO) are outdated. The gain over SSR, the closest baseline, is marginal (e.g., 0.04m L2 improvement). Recent works such as DriveTransformer and GoalFLow with advanced performance should be included for further comparisons and discussions.

- Incomplete ablations. No ablation on reverse navigation command design (e.g., why simply flipping left/right is sufficient). No comparison with alternative cycle designs (e.g., Future → Current → Future). No analysis of failure modes beyond ambiguous nav cues or open roads. Besides, most of ablations are conducted on nuScenes dataset, where the open-loop metrics maybe somewhat unconvinced under closed-loop settings, and the performance gains among different designs are incremental, hindering the credibility of the proposed method.

- Writing and presentation issues. Inconsistent use of “EchoP” vs. “Echo Planning”.

---

confidence: 5
contribution: 2
initial_rating: 4
presentation: 2
questions: See weaknesses.
soundness: 2
strengths: 1. Overall presentation: The paper is well-organized and clearly presented, with informative figures, well-structured tables, and clear writing. The overall presentation quality makes the paper easy to follow and understand.

2. Experiments and visualization: The experimental section is comprehensive. The visualizations are clear and intuitive, effectively demonstrating the qualitative performance.
summary: This paper introduces EchoP, a self-correcting trajectory planning framework for autonomous driving that enforces bidirectional temporal consistency through a Current→Future→Current (CFC) feedback cycle; unlike conventional one-shot planners that predict a future trajectory from a single scene snapshot, EchoP first generates a future trajectory from the current Bird's-Eye-View (BEV) representation and then inversely reconstructs the current BEV state from this predicted future, using the discrepancy between the reconstructed and actual current BEV as a self-supervised signal to penalize implausible trajectories. This CFC cycle, which requires no external supervision or auxiliary tasks, significantly enhances planning accuracy and safety, as demonstrated by state-of-the-art results on the nuScenes and Bench2Drive benchmarks.
weaknesses: 1. **Limited novelty**: This paper is highly similar to SSR [1] and LAW [2], with only minor modifications to the loss calculation for the predicted current BEV feature. The overall contribution appears largely story-driven and incremental, which may not meet the novelty threshold for a top-tier conference such as ICLR.

2. **Unconvincing experiments**: The experiments on the nuScenes dataset have been widely questioned by previous works [3], and the results on the Bench2Drive benchmark are inferior to several recent methods, such as DriveTransformer [4]. Therefore, the experimental evidence does not sufficiently support the paper’s claims.

**References**:

[1] Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Driving, ICLR 2025.

[2] Enhancing End-to-End Autonomous Driving with Latent World Model, ICLR 2025.

[3] Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?, CVPR 2024.

[4] DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving, ICLR 2025.

---

confidence: 5
contribution: 2
initial_rating: 2
presentation: 3
questions: 1. I find the motivation of this paper difficult to understand. It is unclear why predicting future BEV features based on the predicted trajectory, then reconstructing the current BEV features from these predicted future features (conditioned on the inverted predicted trajectory), and finally enforcing consistency with the real current BEV features would lead to improved performance. The underlying reasoning for why this bidirectional prediction and consistency supervision should provide meaningful benefits is not well justified.
2. In line 224, the paper states that the reverse instruction “go straight” remains unchanged. However, this could be ambiguous. During the backward reconstruction of BEV features, the instruction actually refers to moving straight backward, not forward. If the same instruction token is used for both forward and backward processes, it is unclear how the SceneTokenLearner distinguishes between these two different contexts.
soundness: 3
strengths: 1. The proposed strategy EchoP is clearly described and well visualized, making the algorithm easy to understand.
2. The paper conducts quantitative experiments on the nuScenes and Bench2Drive datasets, showing noticeable improvements on Bench2Drive.
summary: This paper proposes an end-to-end planning algorithm that predicts future trajectories to generate corresponding future BEV features. It then reconstructs the current BEV features from these predictions and applies a consistency constraint between the reconstructed BEV and the BEV features encoded from multi-view images, leading to improved planning performance.
weaknesses: 1. In the abstract, Figure 1(d), and Table 1, the paper seems to give the impression that the proposed method is self-supervised and does not rely on auxiliary task supervision (such as detection or mapping) shown in Figure 1(a,b). However, as I understand it, the BEV module still depends on these supervised tasks. This is also mentioned in line 161: “…are transformed into a BEV representation using BEVFormer…”, where BEVFormer indeed relies on such supervision to generate BEV features. Therefore, I think the related statement about "self-supervision" could be somewhat misleading.
2. The proposed approach involves forward and backward BEV feature prediction (resulting in four BEV feature maps in total). Since these predictions inevitably contain errors, it is questionable whether performing supervision on these imperfect BEV features can truly provide a benefit. Although ablation studies are provided, the results on the nuScenes dataset (which is dominated by straight-driving scenarios) are not very convincing.
3. When reconstructing the current BEV features based on predicted future BEV features, ego-motion naturally leads to blank or noisy regions near the boundaries (areas unobservable at the current time). Using these imperfect features to reconstruct the current BEV may cause information loss or distortion. Consequently, aligning these with the BEV features encoded directly from surround-view images may be unreliable, which raises doubts about the effectiveness of such a supervision strategy.