Embodied Cognition Augmented End2End Autonomous Driving
Authors: Ling Niu
Xiaoji Zheng
han wang
Ziyuan Yang
Chen Zheng
Bokui Chen
Jiangtao Gong
Venue: NeurIPS 2025 poster ()
Keywords: End-to-End Autonomous Driving
Cognitive-Enhanced AI Algorithms
Primary area: deep_learning
TL;DR: Our work the first to incorporate human driving cognition to enhance end-to-end autonomous driving models, yielding significant findings.

Abstract: In recent years, vision-based end-to-end autonomous driving has emerged as a new paradigm. However, popular end-to-end approaches typically rely on visual feature extraction networks trained under label supervision. This limited supervision framework restricts the generality and applicability of driving models. In this paper, we propose a novel paradigm termed $E^{3}AD$, which advocates for comparative learning between visual feature extraction networks and the general EEG large model, in order to learn latent human driving cognition for enhancing end-to-end planning. In this work, we collected a cognitive dataset for the mentioned contrastive learning process. Subsequently, we investigated the methods and potential mechanisms for enhancing end-to-end planning with human driving cognition, using popular driving models as baselines on publicly available autonomous driving datasets. Both open-loop and closed-loop tests are conducted for a comprehensive evaluation of planning performance. Experimental results demonstrate that the $E^{3}AD$ paradigm significantly enhances the end-to-end planning performance of baseline models. Ablation studies further validate the contribution of driving cognition and the effectiveness of comparative learning process. To the best of our knowledge, this is the first work to integrate human driving cognition for improving end-to-end autonomous driving planning. It represents an initial attempt to incorporate embodied cognitive data into end-to-end autonomous driving, providing valuable insights for future brain-inspired autonomous driving systems. Our code will be made available at  https://github.com/AIR-DISCOVER/E-cubed-AD.

Embodied Cognition Augmented End2End Autonomous Driving
Ling Niu
Xiaoji Zheng
Tsinghua University
Tsinghua University
n123@mails s. tsinghua. edu.
zhengx j2 2 smails twingguu. edu. en
Han Wang
Chen Zheng
Tsinghua University hwwssullooooogemmmmele com
Tsinghua University
zhengchen@air. wwwinghaa.n........ edu on
Ziyuan Yang Tsinghua University z yuunn66uu
Bokui Chen
Tsinghua University
chenbokut @t edu. cn
Jiangtao Gong
Tsinghua University gong j jinggt gma i com
Abstract
In recent years, vision-based end-to-end autonomous driving has emerged new paradigm. However, popular end-to-end approaches typically rely on visual feature extraction networks trained under label supervision. This limited supervision frame- work restricts the generality and applicability of driving models. In this paper, we propose novel paradigm termed E AD. which advocates for comparative learn- ing between visual feature extraction networks and the general EEG large model, in order to learn latent human driving cognition for enhancing end-to-end planning. In this work, we collected a cognitive dataset for the mentioned contrastive learning process. Subsequently, we investigated the methods and potential mechanisms for enhancing end-to-end planning with human driving cognition, using popular driving models as baselines on publicly available autonomous driving datasets. Both open-loop and closed-loop tests are conducted for comprehensive evalua- tion of planning performance. Experimental results demonstrate that the AD paradigm significantly enhances the end-to-end planning performance of baseline models. Ablation studies further validate the contribution of driving cognition and the effectiiveness of comparative learning process. To the best of our knowledge. this the first work to integrate human driving cognition for improving end-to-end autonomous driving planning. It represents initial attempt incorporate embod- ied cognitive data into end-to-end autonomous driving, providing valuable insights for future brain-inspired autonomous driving systems. Our code will be made available at https: / //gitttubb com/ AIR- DISCOVER/EI E- wwwpw/ed//euud-d-eeeeeoeo cubed- AD.
"Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

INTRODUCTION
Autonomous driving technology is crucial for transferring driving authority from human drivers to sensors and artificial intelligence, promising to enhance efficiency and safety in transportation. In the pursuit of autonomous driving, the emergence of end-to-end autonomous driving has recently garnered increasing attention [10, 12, 26, 31, 16, 14]. However, existing end-to-end autonomous (E2E-AD) driving model frameworks often rely sequential 3D visual representations, such as Bird's Eye View (BEV) features[18, 17J. BEV features contain rich latent information essential for autonomous driving. However, in existing end-to-end approaches, BEV feature extraction networks are typically supervised using annotated data for downstream tasks such as 3D perception, motion prediction, and semantic segmentation[15, 15, 29]. This limited supervision framework restricts the model's s ability to extract visual information.
Models trained only on labeled data are limited, while the human brain uses embodied reasoning to anticipate hazards and adapt to new situations. Recently developed ceneral-purpose EEG models (e.g.. LaBraM [13]) can extract rich cogmitive features directly from EEG signals and achieve excellent performance across various general tasks. By leveraging the cognitive features provided by such universal brain-inspired models, it becomes possible to offer broader supervision to the feature extraction networks of E2E-AD models.
In this study, we first trained the "Driving-Thinking Model," a spatio-temporal feature extraction network. Unlike traditional supervised approaches relying on manually annotated labels, our model was trained on a specifically collected dataset, utilizing paired video data and corresponding EEG segments. The model, enabling it to infer driving-related cognitive information from visual inputs. Subsequently. we explored multiple frameworks to investigate the potential of the driving cognition in enhancing mainstream end-to-end autonomous driving models.
Thinining Model refined through contrastive learning with a large EEG
The F AD paradigm is the first to incorporate human driving cognition to enhance end-to-end autonomous driving models, yielding significant findings. F AD can be directly applied to baseline end-to-end driving models, achieving substantial improvements in driving performance with only tiny increase in computational cost, while reaching the level of state-of-the-ar methods. Moreover, our study investigates the methods and underlying mechanisms by which driving cognition enhances end- to-end planning, making novel contributions to the field of embodied human intelligence augmentation in Al algorithms. At the same time, Our work represents an exploration of more end-to-end styled autonomous driving framework, enabling the model to acquire richer semantic information from raw data through implicit supervision, rather than being limited manually annotated labels.
2 Related Work
2.1 End-to-End Planning
Learning-bar based planning, particularly reinforcement learning and imitation learning [3, 4, 27], has emerged as a promising approach since Pomerleau's pioneering work [22]. The latest trend in this field is the end-to-end training of multiple functional modules [8, 2]. ST-P3 introduces improvements in perception, prediction, and planning modules, integrating auxiliary tasks such as depth estimation and BEV segmentation enhance spatio-temporal features learning. UniAD [10] integrated six subtasks (object detection, tracking, map segmentation, trajectory prediction, occupancy prediction, and planning) into unified network. VAD [12] uses fully vectorized representations of driving scenarios. GenAD formulates autonomous driving as a generative modeling problem, predicting the evolution of the ego vehicle and its surrounding environment based on past scenes, and employs motion and planning heads consistent with VAD. LAW enhances planning by extracting richer spatiotemporal features through a self-supervised paradigm, predicting future scene features based on current features and the ego trajectory. In contrast, our model advocates for contrastive learning between the spatio-temporal feature extraction network and the general EEG model, aiming to acquire potential human driving cognition from visual inputs to further enhance planning.

---

2.2 Cognitive-Enhanced AI Algorithms
Recent research has explored integrating human cognitive data to enhance AI models. In NLP, BrainBERT and CogBERT incorporate brain signals and eye-tracking data, respectively, improving performance [24, 5]. In autonomous driving, world models mimicking human cognitive structures show promise [7], with models like HLTP enhancing trajectory prediction [19]. Liao et al. proposed model integrating cognitive insights for perceived safety and dynamic decision-making [20]. However, emulating complex human driving processes requires high-quality cognitive data, such as EEG, which provides rich information about intricate cognitive processes. Currently, there is a lack of research exploring whether such high-quality cognitive data can effectively enhance complex tasks like autonomous driving. Recently, the LaBraM framework (Large Brain Model) [13] learns universal EEG representations by superviee pre-trainn pre-training on 2,500+ hours of diverse EEG data: it splits signals into channel patches, trains a vector cutttiiua tokenizer [28], and uses masked modellng to predict masked tokens. LaBraM sets new state-of-the-an on downstream BCI tasks like anomaly detection and emotion recognition [13]. Due to the good performance of LaBraM, we use it in our work to extract rich and generalizable EEG features which will provide human cognition insights for end2end autonomous driving models.
3 METHOD
Given the EEG data are not included in traditional end-to-end autonomous driving training datasets [25, 6, 1], a gap exists between the self-collected dataset and typical autonomous driv- ing datasets. To address this challenge, we propose a two-stage training approach as shown in Fig. 1: EEG data are utilized only during the first stage of training, while in the second stage and during inference, only mainstream autonomous driving datasets are employed to ensure fairness.
1. Driving- Thininng Model Training: A visual feature extraction network is trained on a self- collected dataset and supervised using cognitive features from LaBraM [13]. Specifically, inspired by the successful paradigm of CLIP [23], contrastive learning is employed to achieve cross-modal self-supervised learning.
2. Embodied Cognition Augmented End2End Model Training: We freeze the Driving- Thinking model and integrate into popular E2E-AD frameworks. The integrated model is then trained on large-scale driving datasets, followed by open-loop testing and closed- loop simulation. To ensure fairness, only the original data used at this stage, without introducing any additional inputs, maintaining consistency with other baseline models.
In this section, we will first describe how we train the process of integrating it into an end-to-end autonomous driving algorithm.
model, followed by the
3.1 Embodied Cognitive Dataset
To our knowledge, few cognitive-related datasets have been collected in real-world driving scenarios. Thus, we gathered a multi-modal physiological dataset from subjects driving a fixed route in complex traffic. We recorded CAN bus data, EEG, heart rate, skin conductance, and front-facing camera footage. After filtering data and controlling for variables, we selected 20 male drivers (10 experts, 10 novices).
All EEG recordings were preprocessed in EEGLAB² by first re-refereneing signals to the MI and M2 electrodes. We then applied a 0.1-50 Hz band-pass filter to remove drift and high- figh-freeuency noise, followed by a 50 Hz notch filter to suppress power-line interference. Next, we employed Independent Component Analysis (ICA) to remove ocular, cardiac, channel, and muscle artifacts, effectively filtering out various noise sources. To match the input requirements of the Large Brain Model[13), the data were downsampled from 1000 Hz to 200 Hz, and amplitudes (+0.1 mV) were normalized by dividing by 0.1 VV, mapping signals into [-1,1]. Finally, each driver's continuous session was manually split into 14 condition-specife segments and uniformly cut into 2 s clips (including those shorter than as individual samples).
Pnttps://eenee
3

---

Stagel Training
BBG Singals
ein,
Brain Model
E,
or em Single
Excert BEG
Novize BEG
Volu
Vola
Driving Thinking
Camera Front
Video Swin Transformer
Proj
Video>
Vx
Yo
Stage2 Training & Inference
Spatio-temperal Fentures
Driving Thinking Model
Options
Internet with Ego Query
Camera-Front Video
Interact with Planning Features
Video Swin Transformer
Proj Right
Ego Query
Tracking
Mapping
Cross Attention
Planning Head
Lubel-supervieed
Multi-View Video
BEV Encoder BEV Features
Others
Spatio-temppoal Features
Trajectory
The training is divided into two stages. In the first stage, contrastive learning is conducted between the the large EEG model LaBraM[13] on a self-collected dataset. In the second stage training and inference processes, we use the same inputs as other driving models, without introducing EEG data. Instead, the entire Priivng-TThnnkng Model is kept frozen. Subsequently, we design three different frameworks investigate how the driving cognition learned by the Driving-Thinkinn Model enhances end-to-end planning, as well as the associated mechanisms.
Figure I:
Model pretrained spatio-temporal feature extraction network and
We split the dataset into training set validation, and test set. The structured data was shuffled and divided into two parts with ratio of 80: 10:10. The training set comprised 1894 clips (1037 from expert drivers and 857 from novice drivers), the validation set consists of 236 clips (144 expert, and 92 novice), and the test set contained 237 clips (118 from expert drivers and 119 from novice drivers). For details regarding cognitive data collection and our data, comprehensive information can be found in the Appendix.
3.2
Model
Let [(m, e be minibatch of paired video-EEG samples. We define
Video TRdd ho Rd Rdd fa EEG -EGGRR
Iv
where In is implemented as Video Swin Transformer[21), h2 is learnable linear projection head, and fa the frozen Large Brain Model (LaBraM)[13). During training only the video-branch parameters 0, {9v. h.) are updated, while fe remains fixed.
We begin with feature extraction and normalization: video clips are processed by Iv and h,, and EEG segments by fe. then each is Eznoormaiized:
(o)
feles)
[ho (v:)))
[[f((e))l
Next, we compute similarity matrix using learnable log-temperature 3 (so that 7 e0):
$i _ (2) 25),
S [sts 1=
(2)

---

The model is trained with a symmetric InfoNCE loss:
Lu 1
N
e
log
N
e
1
ei
Le
log
N
2j=1
N e*sa
L = 2 (Lg + Le).
Finally, we update only the video-branch parameters 0, by gradient descent, keeping fe frozen:
0. <- 0g Vo, T.
3.3 Embodied Cognition Augmented End2End Model Training
Perception is crucial in autonomous driving for extracting visual features from sensor data. Among methods, BEV has become popular because effectively integrates spatial and temporal visual information. As illustrated in Fig. popular end-to-end frameworks inherit the BEV encoder architecture to produce label-supervised spatio-temporal features. These features are further interacted with vehicle queries that encapsulate the historical motion of the ego vehicle, which are then processed by planning head to generate the final predicted trajectory. It is worth noting that different end-to-end methods exhibit subtle differences their choices of supervised spatio-temporal features. For example, some approaches[15. [18, 10] adopt the entire BEV feature map as the visual representation. In contrast, other methods[ 14, 12] propose extracting sparse features from the BEV feature map, with the aim of enhancing computational efficiency. Despite these divergent design strategies, the overarching framework adopted by mainstream end-to-end methods remains consistent.
Therefore, this study focuses on how human cognition can be integrated to enhance end-to-end planning, well as the underlying mechanisms. We designed the following three frameworks and corresponding hypotheses:
1. Attach to Spatio-temporal features: We hypothesize that the ThinkigT model has learned human drivers' attention mechanisms toward visual features. Leveraging this driving cognition, the model can select potential visual features related to planning from raw visual inputs, thereby enhancing end-to-end planning.
2. Interact with the Ego Query: We hypothesize that the Driving- ThinkigThinkkng model has acquired driving-relates cognitive knowledge about visual features from LaBraM. This knowledge, when interactively embedded into the ego query, can guide the process by which the ego query interacts with visual features to generate planning features.
3. Interact with planing features: We hypothesize that the Driving-Thinking model learns more advanced driving cognition directly related to planning tasks. We believe that such higher-level driving cognition can directly interact with the features, improving the prelimi- nary planning results. Therefore, we directly enable its interaction with the features, which is then fed into the planning head for trajectory decoding.
3.4 Attach to Spatio-temporal features
BEV features contain visual information aggregated across both spatial and temporal dimensions. Therefore, by interacting the cognitive feature V, generated by the Priving-Thinkiin model with B,, we can highlight the potential brain-aware BEV feature Bhrain, Considering that BEV features typically have large spatial dimensions, employ an Attention Gate to generate attention map based on the max-pooled features of B, and Vz, thereby selecting Blrain.
B Brrain
B, O AttnGate DancGaateMaapooolingggg (B,), V.)
(5)
Both Berain and B, share the same spatial dimensions of b x hw X which is not computationally efficient. Therefore, inspired by the success of the BEV TokenLearner, we adopt adaptive spatial attention to obtain brain-aware sparse visual representations S.. The process is as follows:
S. >==p>>(B (B= B Brainn
(6)
S, = Attn(s)
5

---

where the spatial attention function maps B to an attention map for each sparse visual query: Rexhuxe - Rbx,, xww where ni denotes the number of sparse visual queries. The average pooling function p utilized to aggregate spatial features across the x dimensions. Finally, self-attention layer is applied to enhance the representation capability of S,. Subsequently, Se interacts with the ego-query Qego via decoder-only transformer to generate brain-aware planning features. These features are concatenated with the planning features produced by the end-to-end baseline model, and the combined featuress are then fed into the planning head for trajectory decoding. For HD- -DD-----ree planning, a high-level driving command e has been demonstrated to be recessary[12, 10]. Therefore, we adopt the planning head from mainstream end-to approaches, and feed the updated features together with the high-level driving command as input to output the planned trajectory T,, The decoding process is formulated as follows:
F plan Cross Attn Qego: St, St,St)
F plan Cross Qego: V Vi)
(7)
F plan [F plan
Fb pland
T PlonHead(S plan: cmd c)
where, V denotes the visual features learned by the end-to-end baseline model through supervised learning with labels. F plan and Fb plan represent the planning features obtained from Vi and St. respectively. The symbol [t] denotes the concatenation operation.
3.5 Interaction with the Ego Query
As previously discussed, Qego contains only the historical trajectory information of the ego vehicle. According to the second framework and hypothesis described in Section 3.3, the Driving-Thinkinn model learns the driver's cognition of visual features from the general EEG large model. Therefore, model, and
we construct query-key pairs based on Qego and the output V, of the obtain new ego-vehicle query Qego embedding with driving visual cognition through cross-altention. The enriched Qe then interacts with the spatio-temporal features Vž, allowing the embedded driving visual cognition to inform the generation of planning feature based on visual information. Similarly, the planning head decode the planned trajectory:
Vego Cross Qegor Vt, Vi
plan = Cross O'egor VI, Vi)
(8)
T PlanHeed plan: cmd c)
3.6 Interaction with planing features
Recent studies[13] 13] have shown that EEG signals often exhibit changes in specific frequency bands during tasks involving drcision-making attention, and memory. Inspired by these findings, in third framework, we hypothesize that the Priving-Thinkee model is capable of acquiring advanced cognitive abilities related to driving decisions and reasoning. This includes human-like reasoning and judgment for preliminary decisions, allowing the model to identify and correct mistakes already in the initial planning stage, and thereby directly influence the quality of the planning feature. To this end, we adopt multi-layer decoder architecture, which integrates driving-related cognitive information to perform chained reasoning over the initial planning feature piian- Specifically, F Faan contain information about the trajectory sequence, while Vi is obtained from a spatio-temporal features extraction network. Therefore, we introduce learnable positional encodings to align the latent temporal dependencies present in both F plan and Vü. ultimately producing the planning feature (F" (F inferred through reasoning combined with human driving cognition.
F" plan fornnssformeereeoode
k. v, qposs Kpoc)
q F plan: k
V
(9)
T = Plan PlanHead(F", plan: cmd = c)
where, Spos and kpos are learnable positional encodings.

---

EXPERIMENTS
4.1 Implementation Details
In our implementation, we employ the Video Swin Transformer [21] to extract spatio-temporal visual features from 2s video clips fps, and the Large Brain Model [13] extract generic EEG embeddings. Both branches are initialized with pretrained weights, then fine-tuned via CLIP-style contrastive objective [23]. Each branch's output passes through two-layer MLP adapter head projecting features into a shared 200-dimensional space. We train the model on aligned 2s video-EBG segment pairs with batch size of 16 for 120 epochs using Adam (learning rate 2e-5 on both backbones and adapters, Ir ratio 1:1), dropout of 0.01, and weight decay of le-5. Training on the full dataset requires approximately 12 h on single NVIDIA A100 40 GB GPU.
Table 1:
Open-loop planning performance comparison of different driving models. The ego status was not used in the planning module. f indicates the FPS measured on our NVIDIA A100 GPU, while others use the reported FPS. * denote the best performing models.
ST-P3 Metrics
UniAD Metries
Method
Collision(%%
L2imi
Collisiont
J
1s
AvE
Is
3s
Ave
2x
AVE
Is
2s
36
Avg.
ST-P3/9]
1.33
2.11
2.90
2.11
0.25
0.62
1.27
0.71
1.6
VAD-
0.22
(AD-Tiey(13]
0.46
0.78
0.58
0.38
UniAD [0]
0.48
0.96
1.65
1.03
0.05
0.17
0.71
0,31
1.8
GenAD/EL1
0.34
0.36
0.83
0.06
RFV-Plamer(18]
0.28
1.07
0.49
LAW[16]
0.26
0.57
1.01
0.61
0.14
0.21
0.54
0.30
19.5
VAD-Tiny
0:46
0.76
1.12
0.78
0.21
0.35
0.58
0.38
9.81
0.55
VAD Busel 12
0:41
0.70
1.05
0.72
0.01
0.17
0.41
0.22
3.81
VADDVVD--
0.35
0.62
11.96
0.64
0.06
0.13
0.36
0
3.71
0.96
8 AUX UmIAA))
0.48
0.96
1.64
1.03
0.07
0.10
0.52
0.23
1.41
LentDDotficaal checkkpoint) S1]
0.25
0.46
0.76
0.49
0.11
0.21
0.45
0.26
0.33
0.81
1.58
0:91
0,06
0.43
1.19
0.56
6.71
DenAD/Repoducss
0.25
0.50
0.30
B AeDDGG))
0.24
0.44
0.74
0.47
0.10
0.21
0.42
0.24
0.32
0.78
1.52
0.87
0.15
0.35
1.09
0.53
6.6
LAW
0.26
0.57
1.01
0.61
0.14
0.21
0.54
0.30
16.51
E'ALLAWW
0.28
0.42
0.22
Table 2: Open-loop and Closed-loop Results of E2E-AD Methods in Bench2Drive under base training set.
Open-loop Metric
Closed-loop Metrie
Method
Avg.L2((m) +
DS t
SR(%)
AD-MLP UniAD-Tiny
3.64
18.05
0.00
0.80
40.73
13.18
UniAD-Basc
0.73
45.81
16.36
VAD
0.91
42.35
15.00
E' ADr
0.69
50.07
20.12
E' AD(VAD
0.86
47.63
19.54
Table 3: Ablation study the impact of the Briving-Thinking model on driving models is conducted, using the E AD(V AD Base) as the experimental baseline.
Contrastive Learning
Freeze Video Encoder
L2(m) $
Collision( %) |
Expert
Novice
1s
2s
3s
Avg.
Is
2s
3s
Avg.
0.39
0.68
1.02
0.70
0.13
0.21
0.40
0.25
0.40
0.64
1.04
0.69
0.15
0.18
0.42
0.25
0.33
0.59
0.92
0.61
0.05
0.18
0.40
0.21
0.38
0.65
0.99
0.67
0.07
0.18
0.38
0.21
0.35
0.62
0.96
0.64
0.06
0.13
0.36
0.18
7

---

Table 4: Comparison of different Frameworks, using the E AD(V AD - -Bas)) as the baseline.
L2 +
Collision(%))
Framework
1s
2s
38
Avg.
1s
2s
3s
Avg.
Attach to Spatio-temporal features
0.38
0.67
1.02
0.69
0.09
0.17
0.39
0.22
Interact with the Ego Query
0.37
0.66
1.04
0.69
0.06
0.14
0.41
0.20
Interact with the Planning Features
0.35
0.62
0.96
0.64
0.06
0.13
0.36
0.18
4.2 Dataset and Metries
We conducted experiments on the publicly available nuScenes dataset [1], which provides 1,000 urban driving scenes under diverse weather conditions. The dataset includes 700 training. 150 validation, and 150 test scenes, each lasting about 20 with keyframes annotated at Hz. offers multi-sensor inputs, including six cameras with 360" horizontal FOV, LiDAR, radar, and IMUs. Following established end-to-end model testing methodologies [9, 10, 12], we employ 12 displacement errors and collision rates metrics evaluate the quality of the planning process. For closed-loop evaluation, we use Bench2Drriee11 [1], which under CARLA Leaderboard 2.0 for end-to-end autonomous driving. It provides an official training set, where we use the base set (1000 clips) for fair comparison with all the other baselines. We use the official 220 routes for evaluation.
4.3 Main Result
Open-Loop Evaluation Our method achieves outstanding performance on the NuScenes dataset, as shown in Tab. I. In line with recent studies, we recognize that the L2 error primarily reflects model convergence, yet we report it for completeness. According to the experimental results, after applying the ESAD method, the average collision rates of UniAD and VAD-Base decreased by 0.08% (25.8% relatively) and 0.04% (18.2% relatively). respectively, even surpassing the recent state-of-the-ar methods. In addition, the convergence L2 eπrors were reduced by 0.08 m (11.1% relatively) and 0.05 m (6.7% relatively), respectively. Moreover, the E AD method is also applicable to the lightweight VAD-Tiny model, where the average collision rate and the L2 eгror was significantly discreased. It is worth noting that, beyond the aforementioned autoregressive trajectory generation framework, our approach generalizes well to other paradigms, including GenAD, which employs a VAE-basee trajectory generation mechanism, and LAW, which leverages world model theory for self-supervised BEV feature learning. In both cases, our method yields consistent improvements compared the official checkpoints and our reproduced results. Despite these improvements, our method maintains the efficient inference speed of lightweight driving models.
Closed-Loop Evaluation As presented in Tab. 2. In the CARLA simulation environment, our method significantly outperforms vision-based baseline driving models on the challenging Bench2Drive benchmark in terms of route completion rate and driving score. After enhancement with our approach, the completion rates of the UniAD and VAD baselines reduced by 3.76% (23.0% relatively) and 4.54% (30.3% relatively), and the driving scores also increased by 4.26 (10.1 1% relatively) and 5.28 (12.5% relatively). In the supplementary material, we present several driving scenarios where the original models failed but could be completed successfully with the E AD method, along with more detailed statistical results for further reference. These findings demonstrate the comprehensive capability and robust performance of E3 AD in challenging environments.
4.4 Ablation Study and Comparison
In Tab. 3,we conducted ablation studies on key components of the Video Encoder was used solely pretrained visual feature extraction network without contrastive learning with LaBraM, experimental results showed that the additional visual features were redundant and did not yield performance improvements. Furthermore, when the Driving- Thinking model was unfrozen during the stage training phase, the model tended to forget the acquired driving cognition during random training, which similarly did not lead to any improvement. These results comprehensively validate the effectiveness of the proposed contrastive learning paradigm, which the Video Encoder learns from the features of LaBraM.
model. When the
8

---

In addition, we conducted ablation studies on the sources of EEG data. Compared to novice drivers, using EEG from expert drivers significantly reduced the L2 error by bringing the model outputs closer to expert trajectories. Meanwhile, both expert and novice EEG enhanced the driving performance of the model by reducing collision rates. We also observed that, by mixing EEG data from both expert and novice drivers and training the Driving-Thinking model with larger volume of EEG data, the driving performance of the model could be further improved.
In Tab, 4, we compared the three frameworks described in Section 3.3. Detailed experimental data on the design specifications and hyperparameter selection for each framework are provided in the supplementary material. Here, we present only the results of the optimal configurations. The first framework did not yield significant improvements, while both the second and third frameworks enhanced the model's driving ability by reducing collision rates and L2 error. Notably, the third framework achieved the greatest improvement in driving performance. These results demonstrate that the Driving- Thinking model learns high-level planning-related cognition, which can directly enhance the planning performance of driving models.
4.5 Analysis and Discussion
What enhances E2E-AD? Our ablation studies demonstrate that the improvements brought by F AD to end-to-end autonomous driving baseline models do not originate from additional visual feature inputs, nor from the capacity of the Video Swin Transformer framework, but rather from the process of contrastive learning with LaBraM. LaBraM a powerful brain-inspired large model capable of extracting general features from input EEG signals to accomplish complex and diverse downstream tasks. By learning on paired EEG and video data, the Priving-Thinking model can acquire corresponding driving cognition from the brain feature space based on visual inputs. Our experiments confirm that the information learned from the brain-inspired large model is what assists the driving model's 's planning, indicating that it is indeed driving cognition that benefits E2E-AD.
How driving cognition enhances E2E-AD? To investigate the mechanisms by which driving cognition enhances E2E-AD, we first formulated three hypotheses. Specifically, we hypothesized that the driving cognition learned by the Driving-Thinking model falls into three categories: visual feature attention, understanding, and decision reasoning. Each of these categories represents hypothesis at an increasing level of cognitive difficulty and complexity. For each hypothesis, we carefully designed a corresponding framework, enabling the driving cognition associated with each hypothesis to enhance key planning features in an appropriate manner. Our experiments show that the effectiveness of the three frameworks in improving E2E-AD also increases correspondingly. This suggests that, for end-to-end driving tasks, the greatest improvements may result from high-level driving cognition, including knowledge related to human-like decision reasoning.
5 Contributions and Limitations
The E AD paradigm is the first to incorporate human driving cognition to enhance end-to-end autonomous driving models. It can be directly applied to baseline models, achieving substantial improvements in driving performance with small computational cost, while reaching the level of state-of-the-art methods. Moreover, this study making novel contributions to the field of embodied human intelligence augmentation in AI algorithms. However, due to the cost and time of EEG acquisition and processing, the paired EEG-video dataset is relatively small. In addition, the precise mechanisms by which EEG alignment enhances planning have not been fully explored. To address these limitations, we commit to publicly releasing the dataset and data collection procedures soon to facilitate data expansion. Furthermore, our future work will investigate the models' underlying mechanisms.
Acknowledgement
This work was supported by Beijing Natural Science Foundation 1.233033, the Beijing Municipal Science and Technology Project (Nos. 7231 10001032300)) and the China National Natural Science Foundation Youth Fund 62202267.

---

References
[1] Holger Caesar, Varun Bankiti, Alex Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621-11631, 2020.
[2] Sergio Casas, Abbas Sadat, and Raquel Urtasun. Mp3: A unified model to map, perceive, predict and plan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14403-14412, 2021.
[3] Raphael Chekroun, Marin Toromanoff, Sascha Hornauer, and Fabien Moutarde. Gri: General reinforced imitation and its application to rision-based autonomous driving. Robotics, 12(5):127, 2023.
[4] Dian Chen, Vladlen Koltun, and Philipp Krähenbühl. Learning to drive from world on rails. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15590-15599, 2021.
[5] Xiao Ding, Bowen Chen, Li Du, Bing Qin, and Ting Liu. Cogbert: iiggittion-guided pre- trained language models. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3210-3225, 2022.
[6] Scott Ettinger, Shuyang Cheng, Benjamin Caine, Chenxi Liu, Hang Zhao, Sabeek Pradhan, Yuning Chai, Ben Sapp. Charles Qi, Yin Zhou, et Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9710-9719, 2021.
[7] Yanchen Guan, Haicheng Liao, Zhenning Li, Jia Hu, Runze Yuan, Yunjian Li, Guohui Zhang, and Chengzhong Xu. World models for autonomous driving: An initial survey. IEEE Transac- tions on Intelligent Vehicles, 2024.
[8] Peiyun Hu, Aaron Huang, John Dolan, David Held, and Deva Ramanan. Safe local motion plan- ning with self-supervised freespace forecasting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12732 12741, 2021.
[9] Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, and Dacheng Tao. St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning. In European Conference Computer Vision, pages 533-549 Springer, 2022.
[10] Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima. Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriente oriented autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17862, 2023.
[11] Xiaosong Jia, Zhenjie Yang, Qifeng Li, Zhiyuan Zhang, and Junchi Yan. Bench2drive: Towards multi-ability benchmarking of closed-loop end-to-end autonomous driving. arXiv preprint arXiv:2406.0368 2024.
[12] Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene representation for efficient autonomous driving. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 8340-8350, 2023.
[13] Wei-Bang Jiang, Li-Ming Zhao, and Bao-Liang Lu. Large brain model for learning generic representations with tremendous eeg data in bci. arXiv preprint arXiv:2405.1876 2024.
[14] Peidong Li and Dixiao Cui. Navigation- guided sparse scene representation for end-to-end autonomous driving. In The Thirteenth International Conference Learning Representations, 2025.
[15] Yingwei Li, Adams Wei Yu, Tianjian Meng, Ben Caine, Jiquan Ngiam, Daiyi Peng, Junyang Shen, Yifeng Lu, Denny Zhou, Quoe V Le, et al. Deepfusion: Lidar-camera deep fusion for multi-modal 3d object detection. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17182-17191, 2022.
10

---

[16] Yingyan Li, Lue Fan, Jiawei He, Yuqi Wang, Yuntao Chen, Zhaoxiang Zhang, and Tieniu Tan. Enhancing end-to-end autonomous driving with latent world model. arXiv preprint arXiv:2400 08481, 2024.
[17] Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bind's-eye-view representation from multi-camera images via spatiotemporal transformers. In European conference on computer vision, pages 1-18. Springer, 2022.
[18] Zhiqi Li, Zhiding Yu, Shiyi Lan, Jiahan Li, Jan Kautz, Tong Lu, and Jose M Alvarez. Is ego status all you need for open-loop end-to-end autonomous driving? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14864-14873, 2024.
[19] Haicheng Liao, Yongkang Li, Zhemning Li, Chengyue Wang, Zhiyong Cui, Shengbo Eben Li, and Chengzhong Xu. A cognitive-based trajectory prediction approach for autonomous driving. IEEE Transactions on Intelligent Vehicles, 2024.
[20] Haicheng Liao, Zhenning Li, Chengyue Wang, Bonan Wang, Hanlin Kong, Yanchen Guan, Guofa Li, Zhiyong Cui, and Chengzhong Xu. A cognitive-driven trajectory prediction model for autonomous driving mixed autonomy environment. arXiv preprint arXiv:2404.17524 2024.
[21] Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu. Video swin transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3202-3211, 2022.
[22] Dean A Pomerleau. Alvinn: An autonomous land vehicle in a neural network. Advances in neural information processing systems, 1, 1988.
[23] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pages 8748-8763. PmL.R, 2021.
[24] Dan Schwartz, Mariya Toneva, and Leila Wehbe. Inducing brain-relevant bias in natural language processing models. Advances in neural information processing systems, 32, 2019.
[25] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo. Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 1446-24544 2020.
[26] Ardi Tampuu, Tambet Matiisen, Maksym Semikin, Dmytro Fishman, and Naveed Muhammad. survey of end-to-end driving: Architectures and training methods. IEEE Transactions on Neural Networks and Learning Systems, 33(4): 1364 1384, 2020.
[27] Marin Toromanoff, Emilie Wirbel, and Fabien Moutarde. End-to-end model-free reinforce- ment learning for urban driving using implicit affordances. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pages 7153-7162, 2020.
[28] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
[29] Haiyang Wang, Hao Tang, Shaoshuai Shi, Aoxue Li, Zhenguo Li, Bernt Schiele, and Liwei Wang. Unitr: A unified and efficient multi-modal transformer for bird's s-yee-view representation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6792-6802. 2023.
[30] -ian-Tiann Zhai, Ze Feng. Jinhao Du, Yongqiang Mao, Jiang-Jiang Liu, Zichang Tan, Yifu Zhang, Xiaoging Ye, and Jingdong Wang. Rethinking the open-loop evaluation of end-to-end autonomous driving in nuscenes. arXiv preprint arXiv:2305. 10430, 2023.
[31] Wenzhao Zheng, Ruiqi Song, Xianda Guo, Chenming Zhang, and Long Chen. Genad: Gen- erative end-to-end autonomous driving. European Conference on Computer Vision, pages 87-104. Springer, 2024.
11

---

A Cognitive Data Collection
In this study, we recruited 27 participants; due incomplete or corrupted data from some subjects, data from 20 participants were ultimately analyzed (10 novice drivers and 10 expert drivers). All experiments employed the Neuroscan 64-channel Quik-Cap' system and Headbox for EEG acquisi- tion, with cables connecting the amplifier to the acquisition unit, the stimulator, and the headbox; stimulation and synchronization were performed using YOGA stimuli host and E-Prime software. Data were recorded in real time via CURRY 9, and usage of experimental and analysis software was controlled by hardware dongles.
The experiment consisted of three phases: setup, recording, and termination. During setup, par- ticipants donned the electrode cap and applied conductive paste, the vehicle idled, and the in-car industrial PC and Docker environment were started; Dreamview was then launched to synchronize system time. During recording, the external operator configured the COM port the tester software, Curry 9 injected timestamps, and E-Prime triggered cross- cross-modal synchronization via spacebar presses and audio cues (marker 161 beeps, 192 other events), while eye-tracking signals were monitored. At the end of the session, the vehicle parked and the parking brake was engaged, the operator stopped recording with Ctrl+C, verified completeness of the record files, and saved the raw cnt files.
Subsequent data processing involved electrode localization, removal of non-EEG electrodes (EKG, EMG, Trigger, etc.), re-referencing to M1/M2, band-pass filtering at 0.1-50 Hz, bad-electrodes rejection, and artifact correction via ICA; events were then imported and data were epoched according to driving conditions. Using the processed data, we conducted statistical comparisons of fatigue levels, workload (theta/alpha plasticity), and task engagement between expert and novice drivers across driving scenarios, finding that experts exhibited more stable fatigue management and greater EEG adaptability in complex road sections.
Table A.1: EEG System Components and Accessories
Type
Hardware Materials
Type
Hardware Materials
Cable (Amplifier Acquisition),
Headbox
64-channel Quik-Cap, Headbox unit
Cables
Cable 2 (Amplifier -> Stimuli, white), Cable 3 (Amplifier - Headbox)
Experimental software dongle, Analysis software dongle, E-Prime dongle
Amplifier (widc- band main unit), Amplifier power cable
Software Licenses
Master Unit
EEG paste (bulk), paste syringe, abrasive gel, cotton swabs, adhesive tape, shampoo, hair dryer, disposable absorbent wipes
Subject Consumables
Master Unit 2
Stimulus generator unit
Power strip, Expansion dock, Speaker set, Portable power bank, Serial-port interface box, Eye-tracking EEG sync cable
Master Unit
Acquisition unit, Power cable
Other Equipment
B Hardware and Software
B.1
Equipment
Hardware
DJI Action 3 action cameras x6
DJI Action battery charging case x2
Insta360 X3 panoramic camera
12

---

RI S
Figure A.1: Example snapshots of the video modalities: top left Baseline (forward road view); top right Driverl (driver's 's face view): bottom left Driver2 (driver's 's posture view); bottom right Driver3 (driver's feet view).
Insta360 charging case
SD memory cards x6
Mounting brackets (multiple)
power Bluetooth speaker (for audio synchronization)
Software
E-Prime (for EEG synchronization)
Jianying (video editing)
B.2
Modalities
RGB Action Cameras Six in- vehicll viewpoints captured by DJI Action 3 cameras:
Baseline: forward road view (dashcam)
Driverl: driver's 's face
Driver2: driver's feet
Driver3: driver's posture
Passengerl: front passenger posture
Passenger2: rear passenger posture
Recording starts when synchronization is initiated and ends when the vehicle is safely parked.
360 Panoramic HDR 360 exterior view recorded by Insta360 X3, capturing surrounding road environment and vehicle pose. Recording interval matches the RGB cameras. All in-vehicle RGB action cameras (DJI Action 3) record at 1080 resolution and 30 fps with wide- angle lens 155%) and subsequent distortion correction; the panoramic camera (Insta360 X3) captures 360 surround video 5.7 HDR 30 fps ensure strict timestamp synchronization with the RGB cameras.
B.3 Collection Procedure
1. Preparation and Inspection
13

---

1) Verify battery levels and free storage on all cameras; prepare spares.
2) Clean windshield and remove obstacles to minimize glare.
3) Mount and secure all six Action 3 cameras at predetermined positions; tighten brackets.
4) Check each camera's 's field of view against the setup diagram.
For the test drive, power on only the Baseline camera.
6) Configure Insta360 X3 to HDR mode and dual-lens (360%) capture; secure on roof mount and level the camera.
2. Power-On
1) During the practice drive, the external operator starts the Baseline camera. Before the formal trial, the in-vehicle operator gives the start command; the external operator powers on all cameras.
3. Recording Start
In-vehicle operator issues a voice command to initiate recording on all RGB cameras
simultancously.
2) External operator confirms LED indicators and starts the Insta360 X3.
4. Recording Stop
Upon safe stop of the vehicle, the in-vehicle operator issues a voice command to stop all RGB cameras.
2) External operator stops the Insta360 X3 recording on command.
C Ablation Study and Comparison
First and foremost, it should be clarified that, in line with many recent studies[30, 18]. we view L2 error solely as indicator of model convergence, whereas the ultimate goal of autonomous driving is safe, collision-free operation. In this study, we doubled the coefficient of the L2 loss term in the source code of VAD-Base[12) and found that, although the I.2 error decreased by 0.15 m (20.8%, respectively). the collision rate increased by 0.07% (31.8%, respectively), as shown in Table. A.2. Additionally, Ego-MLP[30] performs planning using only the vehicle's 's historical state information without any visual input, the L2 error achieved is comparable to that of VAD, yet both the collision rate and closed-loop evaluation results are terrible. These findings motivate collision rate as the primary performance metric for open-loop evaluation during model design and hyperparameter selection, while relegating L2 error to secondary role convergence, and coefficient settings are kept consistent with the baseline model to ensure a fair comparison. Additionally, due the low computational efficiency of UniOD[ we primarily select VAD as the baseline model in the large-scale experiments discussed in the following sections.
adopt
indicator of model
As shown in Table. A.3, compare the number of blocks, the dropout rate, and the number of cross-attention heads in the "Interact with the Ego Query" framework. Our observations indicate that, even with only one layer, the model achieves stable improvement over the baseline. When the number of cross- coss-attentionn heads is reduced by half, the model converges more easily and exhibits lower L2 error, while the collision rate remains unaffected. Additionally, dropout is key parameter; excessively low dropout rates can lead to overfitting, resulting in degraded driving performance on the test set.
The comparison results hyperparameters for the "Interact with Planning Features" " framework are presented in A.ble.A.4. We observe that employing just one layer of our designed Decoder block can already lead to notable performance improvements. Nevertheless, appropriately increasing the number of layers further enhance the overall performance of the model. It is also crucial to select suitable dropout value, as excessively large or small dropout rates may lead to underfitting or overfitting, respectively, thereby degrading performance. Additionally, moderately reducing the number of attention heads can facilitate model convergence and improve driving performance. However, setting the number of attention heads too low may compromise the model's learning capacity.
For the "Attach to the Spatio-temporal Features" framework, compare the impact of the number of vision feature queries ns which are selected by human visual attention. We observe that as n, increases, the model's L2 error gradually decreases. However, the collision rate does not decrease
14

---

accordingly and even increases when n, is excessively high, as shown in Table. A.5. This phenomenon can be attributed to the fact that this framework only captures visual features attended by the human brain, without learning planning-related driving cognition. These visual features primarily help the model to fit the expert driving trajectories, but their contribution enhancing the model's driving performance is limited. Furthermore, when the visual features become overly redundant, they may even impair the model's performance.
Table. A.6 presents a comparison of detailed metrics between our model and baseline models under closed-loop evaluation. It can be observed that our model substantially reduces the rate of vehicle collisions and red light violations by 11.82% (38.8%, respectively) and 1.37% (37.6%, respectively). The rate of stop infraction is also significantly reduced by 0.46% (16.8%. respectively). In addition, occurrences of collisions with layouts and lane departures have also been mitigated.. These results demonstrate that the F AD paradigm achieves significant improvements in driving performance, particularly in complex driving tasks such as interacting with other vehicles and understanding traffic signals.
Table. A.7 presents the results of cross-dataset validation and generalization analysis. To further evaluate the robustness of our model in unseen geographic domains and to address the reviewer's concerns, we conducted additional experiments on the nuScenes dataset. The dataset was divided into two subsets according to the acquisition locations (Singapore and Boston). Specifically, the training set consists of 12,435 and 15,695 samples for Singapore and Boston, respectively, while the test set contains 2,929 and 3,090 samples. We trained the model using data collected from one location and evaluated it on the other, ensuring that the validation was performed on entirely unseen geographic domains. As shown in the Table. A.7, our model consistently outperforms the baseline (VAI--tnn) on unseen data. Particularly, when training on Singapore and validating on Boston, our approach achieves a 14.1% reduction in baseline L2 error and a 22.7% decrease in collision rate compared to the baseline.
Table A.2: Relationship Between L.2 Error and Collision Rate.
L2(m)
Collision
Method
1s
2s
3s
Avg.
1s
2s
3s
Avg.
VAD-Base| [2]
0.41
0.70
1.05
0.72
0.07
0.17
0.41
0.22
VAD-Base (Doubling L2 loss)
0.31
0.54
0.85
0.57
0.16
0.21
0.51
0.29
Ego-MLP[33]
0.46
0.76
1.12
0.78
0.21
0.35
0.58
0.38
Table A.3: Comparison of Hyperparameters in the "Interact with the Ego Query" Framework. Collision (%) )
Options
L.(m)
Framework
Layers
Dropout
heads
2s
38
Avg.
1s
3s
Avg.
VAD[12]
0.41
0.70
1.05
0.72
0.07
0.17
0.41
0.22
VAD (Reproduced)
0,40
0.70
1.04
0.71
0.10
0.16
0.44
0.23
1
0.1
0.37
0.66
1.04
0.69
0.06
0.14
0.41
0.20
2
0.1
8
0.39
0.68
1.03
0.70
0.09
0.19
0.42
0.23
Ours
4
0.1
0.36
0.62
0.95
0.64
0.08
0.19
0.41
0.23
1
0.05
8
0.39
0.66
1.00
0.68
0.09
0.18
0.37
0.21
1
0.1
4
0.33
0.58
0.92
0.61
0.07
0.14
0.39
0.20
D
Visualization
Similar to other end-to-end methods[18, 10, 12, 16], we also provide visualization results, as shown in Fig. A.2. It demonstrates successful case from the closed-loop simulation experiments: when vehicle parked ahead on the right suddenly starts moving, the baseline model adopts a more aggressive and risky avoidance maneuver, attempting to pass quickly and causing severe collision. In contrast, our model learns to interact with the suddenly moving vehicle by slowing down and waiting until it has completely departed before accelerating to proceed along the route, thus avoiding collision.
15

---

Table A.4: Comparison of Hyperparameters in the "Interact with the Planning Features" Framework.
Options
L2(m)L
Collision(8)) {%)
Framework
Layers
Dropout
heads
3s
Avg.
Avg.
VAD
0.41
0.70
1.05
0.72
0.07
0.17
0.41
0.22
VAD (Reproduced)
0.40
0.70
1.04
0.71
0.10
0.16
0.44
0.23
4
0.1
0.33
0.59
0.93
0.06
0.14
0.20
1
0.1
8
0.39
0.70
1.07
0.72
0.03
0.14
0.44
0.20
2
0.1
8
0.34
0.59
0.91
0.61
0.12
0.20
0.32
0.21
6
0.38
0.64
0.96
0.66
0.09
0.18
0.39
0.22
Ours
4
0.05
8
0.38
0.67
1.04
0.70
0.07
0.15
0.41
0.21
4
0.15
0.34
0.96
0.06
0.17
0.40
0.21
4
0.1
4
0.35
0.62
0.96
0.64
0.06
0.13
0.36
0.18
4
0.1
2
0.33
0.59
0.93
0.62
0.06
0.14
0.41
0.20
Table A.5: Comparison of Hyperparameters in the "Attach the Spatio-temporal Features Frame-
work.
Options
L2(m)l
Collision [%)
Framework
n
1s
2s
Avg.
Is
2s
3s
Avg.
VAD
0.41
0.70
1.05
0.72
0.07
0.17
0.41
0.22
VAD (Reproduced)
0.40
0.70
1.04
0.71
0.10
0.16
0.44
0.23
0.38
0.67
1.04
0.70
0.09
0.17
0.400
0.22
Ours
0.38
0.67
1.02
0.69
0.09
0.17
0.38
0.21
16
0.34
0.60
0.95
0.63
0.12
0.19
0.49
0.27
Table A.6: Comparison E AD and Baseline Models on Closed-Loop Metrics.
Closed-loop Metrics
Method
Layouts Collision(3)
Vehicles Collision(
Running light(?)
Infraction(
OlT-vad((S)
VAD-Base
15.46
0.91
30.46
3.64
2.73
23.64
Ours
15.10)
0.91
18.64(]38848 88.)
2.27(137.668 665)
2.27(116.9)
22.73((33885)
Table A.7: Cross-Datased Validation and Generalization Study
L2
Col
Method
1s
2s
3s
Avg.
1s
2s
3s
Avg.
Bascline (Training Singapore)
0.51
0.84
1.21
0.85
0.29
0.42
0.61
0.44
3 AD (Training on Singapore)
0.44
0.71
1.03
0.73
0.15
0.30
0.56
0.34
Baseline (Training Boston)
0.45
0.77
1.17
0.80
0.22
0.35
0.76
0.44
3 AD (Training Boston)
0.46
0.76
1.13
0.78
0.22
0.42
0.50
0.38
16

---

This case to some extent suggest that AD exhibits behaviors that are closer to human driving style and are characterized by enhanced safety.
Fig. A.3 illustrates the real-world data validation conducted China. We collected real-world data using vehicle-mounted platform equipped with Insta360 panoramic fisheye camera and inertial sensor. Following the nuScenes protocol, the panoramic videos were undistorted and split into six perspective- videos. Using these multi-view videos together with CAN bus data, evaluated our model. The E AD model demonstrated smooth trajectory continuity.
Firae
Frame 2
Frame
VAD
VAD
VAD
Frame1
Frame 2
Frame
EAD
EAD
ETA
Figure A.2: Visualization Comparison of AD (VAD-Base and the Baseline Closed-loop Evaluation.
CALL FRONT
FRONT
CALL FRONT RIGHT
BACK
Figure A.3: Real-world data validation
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our abstract and introduction explicitly list the E AD paradigm, the use of EEG- comparative learning, and the resulting planning performance gains, which align exactly with our experimental results in Sec. 4.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [No]
Justification: We have not yet included an explicit discussion of limitations; we will add dedicated "Limitations" section in the final version.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
18

---

Answer: [No]
Justification: Our paper does not include any novel theoretical results or formal proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We provide full preprocessing and data details in Sec. 3.1; complete model architectures, and loss formulations in Secs. 3.2-3.6; and training hyperparameters (learning rate, batch size, optimizer) Experimental setups, metrics, and evaluation protocols are described in Sec. 4.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
19

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We will release the official link and open-source the code after the public review process.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https:/ /ttsss:////p////ee/eeeeee cc/
pub c guides/ CoodeSuomi ss Pooooollioo cyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / /nnps.... cc/ public iibbiic// quuided des CodeS Susesuubiiie. as i onPolicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer:
[Yes]
Justification: We provide information on data splits, hyperparameters, optimizers, etc., in Sections 3.1 and 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[Yes]
Justification:
We detail the various experiments in Sections 4.2-4.5.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
20

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We detail the various experiments in Sections 4.2-4.5.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: /tteunnus..../// http://pwc/tttcceuuueeeeeeeeeeee
Answer: [Yes]
Justification: All EEG data collection followed IRB-approved protocols and informed consent (Sec. 2.1). We anonymized personal identifiers, stored data securely, and comply with privacy, fairness, and participant-safety guidelines in the NeurIPS Code Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss potential benefits benefits-improweee improved safety, robustness, and generality of autonomous driving
Guidelines:
21

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We have explicitly referenced the datasets Section 3.1.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
22

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, papersw. snnersw:. shithe com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The information explicitly cited in the paper. A detailed README will be provided once the code is open-sourced.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsoureing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [No]
Justification: While we describe participant recruitment and EEG cap setup in Sec. 2.1, we have not yet included the full stimulus instructions or compensation details; these will be provided in the supplementary material.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: All EEG data collection was conducted under -ap-oped protocols with informed consent and risk disclosure, in full compliance with ethical guidelines.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
23

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: We used large language models (LLMs) to assist with tasks such as polishing and refining the writing of the paper
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: //n/urirsspp.//p// cc/Conferencess 2025/ LLM) for what should or should not be described.
24


Reviews:
[{"review_id": "l1JbknaGMJ", "reviewer": "Reviewer_cnpF", "summary": "The authors presents EAD, a novel framework that enhances vision-only driving models by incorporating human EEG-derived cognitive features. In the first stage,  a Video‑Swin backbone is trained on synchronized in-car video and 64‑channel EEG data collected from 27 drivers. Through contrastive learning, the video embeddings are aligned with features extracted by the large EEG model LaBraM, resulting in a vision network that implicitly captures human driving cognition. In the second stage, the frozen Driving‑Thinking model is integrated into end‑to‑end planning modules such as VAD and UniAD using three approaches: attaching to BEV features, enriching ego-queries, and refining planning features. Evaluated on both NuScenes open-loop and Bench2Drive closed-loop benchmarks, EAD demonstrates a reduction in collision rates with no significant impact on inference speed.", "questions": "1. Will you release the paired EEG‑video dataset and pretrained Driving‑Thinking model? If so, under what licence and with what privacy safeguards?\n\n2.  Have you tested EAD on unseen geographic domains (e.g., Waymo Open, CARLA Town10) or with synthetic EEG noise to evaluate robustness?\n\n3. How does performance scale if only 5, 10, 15 subjects are used to train the contrastive stage?", "limitations": "1. The generalizability is yet to be explored. \n2. More diverse geographic domains needs to be validated in the future.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths: \n\n1. First to fuse EEG cognition with end‑to‑end planning; contrastive loss is simple yet effective.\n\n2. Clever design means no additional sensors are required in deployment.\n\n3. The authors test three hypotheses (attention, ego‑query guidance, decision reasoning) and show that the planning‑feature interaction yields the biggest gain.\n\nWeaknesses:\n\n1. A small dataset collected and used by the authors raises concerns about over‑fitting and model bias. A broader demographic coverage analysis would strengthen claims.\n\n2. Brain data are sensitive. A dedicated limitations subsection is needed (data consent, anonymisation, potential misuse).", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for recognizing the novelty of our work, appreciating the creativity of our design, and acknowledging our contributions. We appreciate your concerns and will provide detailed, point-by-point responses to each of them below.\n\n---\n## Response to Weakness 1\n\n>Thank you for your question. We agree that overfitting is a common concern when working with small datasets. To mitigate this, we fine-tune our model through a contrastive learning process with LaBraM, a large-scale EEG foundation model pre-trained on over 2,000 hours of EEG recordings. We believe this design can reduce the risk of overfitting by providing robust and generalizable feature representations.\n\n>In addition, we monitored the contrastive loss of our Driving-Thinking Model during Stage 1 training. We observed that the losses on both the training and evaluation sets converge smoothly and consistently, which further suggests that overfitting is not a major issue under our current experimental setting. We will provide the experimental evidence for this part in the supplementary materials of the revised manuscript.\n\n---\n\n## Response to Weakness 2\n\n>Thank you for raising this important concern. We confirm that the entire data collection process strictly followed legal and ethical guidelines, and was conducted under the supervision and approval of our host institution. All necessary permissions and consents were obtained prior to data acquisition.\n\n>We acknowledge the sensitivity of brain data and fully agree that a dedicated limitations subsection is necessary. In the final version of the paper, we will include a clear statement detailing data consent procedures, anonymization protocols, and a discussion of potential risks and safeguards to prevent misuse of cognitive data.\n\n---\n\n## Responses to Questions\n\n**Question 1:** \n> Thank you for your question. We are committed to releasing our paired EEG-video dataset and the pre-trained Driving-Thinking model to support future research in this area. Currently, we are carefully analyzing, organizing, and documenting the dataset to ensure it is accessible and useful for the community. We plan to make the dataset publicly available within the next year.\n\n> While we have not yet finalized the license or specific privacy safeguards, we are actively evaluating appropriate open-access licenses and data-sharing protocols that balance openness with the protection of participant privacy. Our goal is to make a meaningful and responsible contribution to the field, and we will ensure that any released data adheres to ethical standards, including anonymization and consent requirements.\n\n**Question 2:** \n>Thank you for raising this valuable question. Testing autonomous driving models in unseen geographic domains is indeed crucial for evaluating generalization capability. \n\n>First, we would like to clarify that our model was evaluated on the Bench2Drive (NeurIPS 2024) benchmark. This benchmark not only includes CARLA Town 10, as you mentioned, but also comprises a broader range of complex and novel scenarios. Specifically, the Bench2Drive dataset covers 12 Towns from CARLA v2 (e.g., Town 05, Town 10, Town 12), using 220 routes for evaluation. Each route comes from different 12 towns and 23 weather conditions, providing a richer and more diverse set of unseen scenarios for testing. Moreover, unlike evaluations limited to driving score and success rate in a single town such as CARLA Town 10, Bench2Drive also reports performance across five advanced urban driving skills: Merging, Overtaking, Give Way, Traffic Sign, and Emergency Braking. Our supplementary materials provide the results for all of these evaluation metrics, offering a comprehensive demonstration of the generalization capabilities of **${E^3AD}$** in unseen geographic domains. You can find all the relevant details in Table 6 of the supplementary.\n\n>Second, to further validate the performance of our model in unseen geographic domains and to address your concerns, we conducted additional experiments using the nuScenes dataset. We divided the dataset by acquisition location (Singapore and Boston) into two groups: the training set consists of 12,435 and 15,695 samples for Singapore and Boston, respectively, while the test set contains 2,929 and 3,090 samples. We trained the model using data collected in one location and evaluated it on the other, ensuring that the validation involves a substantial number of truly unseen geographic domains. As shown in the table below, our model demonstrates a clear advantage over the baseline (VAD-tiny) in validation on unseen data. Especially in the case of \"training on Singapore and validating on Boston\", **${E^3AD}$** achieves a 14.1% reduction in baseline L2 error and a 22.7% decrease in collision rate compared to the baseline.\n\n|Scenario|L2(1s)|L2(2s)|L2(3s)|L2(Avg)|Col(1s)|Col(2s)|Col(3s)|Col(Avg)|\n|----------------------------------|------:|------:|------:|--------:|-------:|-------:|-------:|--------:|\n|Baseline(TrainingonSingapore)|0.51|0.84|1.21|0.85|0.29|0.42|0.61|0.44|\n|**${E^3AD}$**(TrainingonSingapore)|0.44|0.71|1.03|0.73|0.15|0.30|0.56|0.34|\n|Baseline(TrainingonBoston)|0.45|0.77|1.17|0.80|0.22|0.35|0.76|0.44|\n|**${E^3AD}$**(TrainingonBoston)|0.46|0.76|1.13|0.78|0.22|0.42|0.50|0.38|\n\n\n**Question 3:** \n>You have raised an interesting point regarding data scaling experiments. In fact, in our work, we have already designed specific approaches for data grouping and incremental data experiments. Specifically, we observed notable distribution differences between data from experts (who have over 10 years of driving experience and work as professional drivers) and novices (all other subjects). We visualized the EEG power spectral density (PSD) topomaps for both expert and novice groups within the frequency domain (using the Python MNE package for visualization). We then split the data into five frequency bands (i.e., delta, theta, alpha, beta, and gamma) and found that expert drivers’ power is concentrated in the occipital lobe (visual areas), whereas novice drivers’ power is concentrated in the frontal lobe (responsible for higher cognitive functions such as decision making). This is an interesting finding, suggesting that novices need to exert more cognitive effort to decide their next action, while experts rely primarily on visual processing and intuitively know what to do next.\n\n>Therefore, we grouped homogeneous data together: the expert group consists of data from 10 experts, and the novice group consists of data from 10 novices. In our scaling experiments, we incrementally expanded the data in three settings: “expert group,” “novice group,” and “expert group + novice group.” We believe this approach is more meaningful and better aligned with the focus of our study than a simple incremental scheme such as “5, 10, 15”. The results of these experiments are presented in Table 3 of our paper.\n\n## Responses to Limitations\n\n**Limitations 1:** \n>We agree that additional generality studies would make the empirical evidence more compelling. In response, we have made the following efforts for your consideration:\n>First, we conducted experiments using GenAD as a new baseline model. It is important to note that we observed a discrepancy between the official checkpoints provided by GenAD and the performance reported in their original paper. Specifically, while the L2 error is slightly lower, the collision rate is substantially higher. We then reproduced their results using the official configuration on eight NVIDIA A100 GPUs and obtained similar findings. Therefore, our experiments reported are based on the official checkpoints. After integrating **${E^3AD}$** into GenAD, the performance of our model shows a significant improvement compared to the reproduced results of the baseline, and it also surpasses the official checkpoint. We would like to clarify that, due to the limited time available for the rebuttal, we reduced the total number of training epochs from 60 to 12. We expect that using the original number of training epochs would yield even more promising results.. These experiments further validate the generalization capability of our method.\n\n| Model| L2 (1s) | L2 (2s) | L2 (3s) | L2 (Avg) | Col (1s) | Col (2s) | Col (3s) | Col (Avg) |\n|--------------------|--------:|--------:|--------:|---------:|---------:|---------:|---------:|----------:|\n| GenAD (official)   |   0.25  |   0.46  |   0.76  |    0.49  |    0.11  |    0.21  |    0.45  |     0.26  |\n| GenAD (reproduce)  |   0.25  |   0.46  |   0.76  |    0.49  |    0.14  |    0.26  |    0.50  |     0.30  |\n| **${E^3AD}$** (GenAD)        |   0.24  |   0.44  |   0.74  |    0.47  |    0.10  |    0.21  |    0.42  |     0.24  |\n\n>Second, due to the limited rebuttal period, we commit to making further efforts to explore other recent end-to-end autonomous driving models (such as LAW) and recent benchmarks (such as NavSim), while continuing to enhance the performance of our method on GenAD. To demonstrate our ongoing commitment to improving the model, we also provide in the table below our recent efforts applying **${E^3AD}$**  to VAD. Through the expansion of EEG data and model optimization, our model’s performance has achieved a higher level. We further commit that these efforts will be fully documented in the main text or supplementary materials of the final version of the manuscript.\n\n| Model         | L2 (1s) | L2 (2s) | L2 (3s) | L2 (Avg) | Col (1s) | Col (2s) | Col (3s) | Col (Avg) |\n|---------------|--------:|--------:|--------:|---------:|---------:|---------:|---------:|----------:|\n| Baseline (VAD) |  0.41   |  0.70   |  1.05   |   0.72   |   0.07   |   0.17   |   0.41   |    0.22   |\n| **${E^3AD}$** (VAD)      |  0.40   |  0.68   |  1.03   |   0.70   |   0.04   |   0.12   |   0.30   |    0.15   |\n**Limitations 2:** \n>Please refer to the response to questions 2. We have already provided explanations and have made efforts to present more compelling results."}, {"review_id": "p8t3TlTZqx", "reviewer": "Reviewer_w1pE", "summary": "This paper introduces E3AD, a novel paradigm that explicitly incorporates human driving cognition into E2E autonomous-driving planning. The core idea is to perform contrastive learning between a visual feature-extractor and a large-scale EEG model so that latent human cognitive signals can be distilled and injected into the driving policy. The authors collect a dedicated cognitive dataset to support this procedure and adopt popular E2E driving models as baselines on publicly available benchmarks. Both open-loop and closed-loop experiments are conducted, followed by ablation studies, to demonstrate that E3AD consistently improves the baseline planners and to quantify the contributions of the driving cognition and the contrastive-learning strategy.", "questions": "Please refer to weakness", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths Contribution\n1. Conceptual novelty. This is the first work that fuses human driving cognition with an E2E driving model, effectively bridging embodied intelligence and conventional deep-learning pipelines.\n2. Methodological soundness. The proposed contrastive-learning–based Driving Thinking Model allows the network to infer cognitively relevant information directly from raw visual input.\n3. Efficiency. The cognitive branch introduces negligible computational overhead yet yields measurable gains in both open- and closed-loop planning metrics.\n\nLimitations Weakness\n1. Lack of qualitative evidence. Beyond numerical improvements, please provide intuitive visualizations (e.g., side-by-side roll-outs of identical scenarios) showing that the cognitive module prevents collisions that the vanilla model fails to avoid.\n2. Generality study. Given that GenAD outperforms the baseline models listed in Table 1, integrating E3AD into GenAD—or, more broadly, into every listed backbone—would make the empirical case far more compelling.\n3. Presentation. Add an explicit “Contributions” paragraph at the end of the Introduction to clarify the paper’s key contributions. Highlight the best-performing values in all result tables in bold rather than with asterisks to enhance readability.", "was_revised": true, "final_justification": "The visualizations clearly addressed my earlier concerns, and the new experiments resolved the remaining issues I had. Consequently, I am changing my recommendation from Borderline Reject to Borderline Accept.", "rebuttal": "Thank you for recognizing the innovation of our method and for acknowledging that our work effectively bridges embodied intelligence and conventional deep-learning pipelines. We also appreciate your positive comments on the methodological soundness and our results. We are grateful for your feedback and will provide detailed, point-by-point responses to each of them below.\n\n---\n\n## Response to Weakness 1\n\n>We agree that visualization results provide an intuitive demonstration. In fact, the supplementary materials already contain the visualization results you requested, which were omitted from the main text due to space limitations. In Figure 2 of the supplementary, we present a visualization comparison between **$E^3AD$(VAD-Base)** and the baseline under closed-loop evaluation. This figure illustrates a successful case from the closed-loop simulation experiments: when a vehicle parked ahead on the right suddenly starts moving, the baseline model responds with a more aggressive and risky avoidance maneuver, attempting to pass quickly and consequently causing a severe collision. In contrast, our model exhibits a more cautious behavior by slowing down and waiting until the moving vehicle has fully cleared the path before accelerating to continue along the route, thereby avoiding a collision. This case suggests, to some extent, that **$E^3AD$**  demonstrates driving behaviors more consistent with human driving style and improved safety characteristics.\n\n>In addition, in response to your valuable suggestions, we commit to providing additional visual results in the revised version—including videos of our real-world data validation and visualization comparisons from open-loop evaluations—to more intuitively demonstrate the performance of our model. For the videos of real-world data validation, we collected real-world data using a vehicle-mounted platform equipped with an Insta360 panoramic fisheye camera and an inertial sensor. The panoramic videos were undistorted and split into six view-angle videos, following the nuScenes protocol. Using these multi-view videos along with CAN bus data, we evaluated both the baseline model trained on nuScenes and our **$E^3AD$** model. The **$E^3AD$** model showed better trajectory continuity and more accurate turning intention recognition. \n\n---\n\n## Response to Weakness 2\n\nThank you for your valuable suggestion. We agree that additional generality studies would make the empirical evidence more compelling. In response, we have made the following efforts for your consideration:\n> First, we conducted experiments using GenAD as a baseline model. It is important to note that we observed a discrepancy between the official checkpoints provided by GenAD and the performance reported in their original paper. Specifically, while the L2 error is slightly lower, the collision rate is substantially higher. We then reproduced their results using the official configuration on eight NVIDIA A100 GPUs and obtained similar findings. Therefore, our experiments reported are based on the official checkpoints. After integrating **${E^3AD}$** into GenAD, the performance of our model shows a significant improvement compared to the reproduced results of the baseline, and it also surpasses the official checkpoint. We would like to clarify that, due to the limited time available for the rebuttal, we reduced the total number of training epochs from 60 to 12. We expect that using the original number of training epochs would yield even more promising results.. These experiments further validate the generalization capability of our method.\n\n| Model             | L2 (1s) | L2 (2s) | L2 (3s) | L2 (Avg) | Col (1s) | Col (2s) | Col (3s) | Col (Avg) |\n|-------------------|--------:|--------:|--------:|---------:|---------:|---------:|---------:|----------:|\n| GenAD (official)  |    0.25 |    0.46 |    0.76 |     0.49 |     0.11 |     0.21 |     0.45 |      0.26 |\n| GenAD (reproduce) |    0.25 |    0.46 |    0.76 |     0.49 |     0.14 |     0.26 |     0.50 |      0.30 |\n| **$E^3AD$** (GenAD)       |    0.24 |    0.44 |    0.74 |     0.47 |     0.10 |     0.21 |     0.42 |      0.24 |\n\n>Second, due to the limited rebuttal period, we commit to making further efforts to explore other recent end-to-end autonomous driving models (such as LAW) and recent benchmarks (such as NavSim), while continuing to enhance the performance of our method on GenAD. To demonstrate our ongoing commitment to improving the model, we also provide in the table below our recent efforts applying **${E^3AD}$**  to VAD. Through the expansion of EEG data and model optimization, our model’s performance has achieved a higher level. We further commit that these efforts will be fully documented in the main text or supplementary materials of the final version of the manuscript.\n\n| Model          | L2 (1s) | L2 (2s) | L2 (3s) | L2 (Avg) | Col (1s) | Col (2s) | Col (3s) | Col (Avg) |\n|---------------:|--------:|--------:|--------:|---------:|---------:|---------:|---------:|----------:|\n| Baseline (VAD) |    0.41 |    0.70 |    1.05 |     0.72 |     0.07 |     0.17 |     0.41 |      0.22 |\n| **$E^3AD$** (VAD)      |    0.40 |    0.68 |    1.03 |     0.70 |     0.04 |     0.12 |     0.30 |      0.15 |\n\n---\n\n## Response to Weakness 3\n\n> Thank you for your valuable suggestions regarding the writing. Following your advice, we have replaced the \"*\" symbols in the result tables within the manuscript with bold font to highlight the best-performing values. Additionally, we have added an explicit \"Contributions\" paragraph at the end of the introduction section, stating “The $E^3AD$ paradigm is the first to incorporate human driving cognition to enhance end-to-end autonomous driving models, yielding significant findings. $E^3AD$ can be directly applied to baseline end-to-end driving models, achieving substantial improvements in driving performance with only a tiny increase in computational cost, while reaching the level of state-of-the-art methods. Moreover, our study investigates the methods and underlying mechanisms by which driving cognition enhances end-to-end planning, making novel contributions to the field of embodied human intelligence augmentation in AI algorithms. At the same time, Our work represents an exploration of a more end-to-end styled autonomous driving framework, enabling the model to acquire richer semantic information from raw data through implicit supervision, rather than being limited to manually annotated labels.” \n\nWe sincerely appreciate your practical recommendations, which have greatly improved the readability of our manuscript."}, {"review_id": "KPsL9KDlHp", "reviewer": "Reviewer_7cU7", "summary": "The paper proposes an end-to-end model performance enhancement method based on contrastive learning. By integrating human driving cognitive information to train the “Driving Thinking Model” as an additional feature extractor, the Planning ability of the end-to-end model is enhanced.", "questions": "1. For Weakness 1, could you provide efficiency experiments to show how much computational overhead is added after applying the proposed method?\n2. What is the basis for using EEG as a representative of human driving cognitive information? Have other alternatives been considered?\n3. According to Figure 1, the proposed method requires modifications to the original model and additional adaptation training. Is there an analysis of these extra costs?\n4. The improvement of the proposed method on VAD-BASE and UniAD is impressive. However, these works are relatively old. Could you demonstrate the improvement effects on more recent end-to-end autonomous driving models? If so, I would consider increasing the score.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. It presents the concept of “Driving-Thinking Model”, which uses contrastive learning to integrate human driving cognition for effective feature extraction.\n2. The Driving-Thinking Model, once trained, serves as an extra feature extractor, augmenting the existing end-to-end model's planning capability.\n3. The authors have collected a dataset comprising EEG and driving information, contributing to future autonomous driving research.\n\nWeaknesses:\n1. The claim that the method enhances the baseline end-to-end autonomous driving model with minimal computational overhead lacks experimental evidence.\n2. Using EEG as a representation of human driving cognition to train the model to integrate human perception into environmental feature extraction is questionable in terms of fully capturing human driving intelligence. (See Question 2)\n3. The baselines used for comparison in the experiments are relatively outdated. (See Question 4)", "was_revised": true, "final_justification": "Most of my concerns have been well addressed. Thus, I am leaning to my current positive ratings.", "rebuttal": "Thank you for recognizing the innovation of our proposed approach, its effectiveness, and the contributions we have made to the field of autonomous driving. We appreciate your concerns and will provide detailed, point-by-point responses to each of them below.\n\n---\n\n## Response to Weakness\n**Weakness 1:**  \n> Thank you for your careful review of our manuscript. However, regarding this weakness, we would like to clarify the following: In the last column of Table 1 in the manuscript, we provide the Frames Per Second (FPS) for each model. FPS is a commonly used metric to characterize the computational efficiency and real-time performance of autonomous driving models. In Table 1, for all end-to-end models incorporated into the **${E^3AD}$** method, we recorded FPS both before and after integration on the same hardware setup (NVIDIA A100 GPU). We observed that after integration with **${E^3AD}$**, the FPS remains relatively stable for all models, whether it is a high-efficiency model like VAD-tiny or a large model like Uniad. Based on this observation, we concluded that the method enhances the baseline end-to-end autonomous driving models with minimal computational overhead.\n\n---\n\n**Weakness 2:**  \nThank you for your question. \n\n>First, it is worth noting that our bold attempt to integrate cognitive features into autonomous driving is grounded in the theoretical foundations of this field. Prior research has shown that specific EEG components—such as the P400, associated with attention engagement [1], and the N500, associated with unpredictability processing [2]—are characteristic markers of a driver’s hazard detection in complex scenarios [3]. We believe these neural signals can play a crucial role in enhancing planning capabilities when integrated into autonomous driving models. To provide more intuition behind our design, we will include additional discussion on these neuroscience findings in the revised version.\n\n>Second, most existing autonomous driving datasets (e.g., nuScenes, Waymo) only provide supervision from driver trajectories and CAN bus data, overlooking drivers’ internal cognitive processes before taking action. While recent approaches use visual-language or large language models to approximate human reasoning, these methods remain indirect and are not suitable for the sub-second decision-making required in real driving. In contrast, EEG signals offer a more direct and real-time measure of drivers' cognitive states, capturing otherwise unobservable thoughts and risk assessments. Therefore, our intention is not to model all aspects of human driving intelligence through EEG, but rather to extract those direct, real-time cognitive features that are most relevant to driving.\n\n**References**  \n**(1)** The cortical development of specialized face processing in infancy  \n**(2)** Predicting outcomes of decisions in the brain  \n**(3)** Annotating Covert Hazardous Driving Scenarios Online: Utilizing Drivers’ Electroencephalography (EEG) Signals  \n\n---\n\n**Weakness 3:**  \n\n>Thank you for your valuable suggestion. We agree that demonstrating the improvement effects on more recent end-to-end autonomous driving models would further strengthen the persuasiveness of our study. To this end, we have made the following efforts for your consideration:\n\n>First, we conducted experiments using GenAD as a baseline model. It is important to note that we observed a discrepancy between the official checkpoints provided by GenAD and the performance reported in their original paper. Specifically, while the L2 error is slightly lower, the collision rate is substantially higher. We then reproduced their results using the official configuration on eight NVIDIA A100 GPUs and obtained similar findings. Therefore, our experiments reported are based on the official checkpoints. After integrating **${E^3AD}$** into GenAD, the performance of our model shows a significant improvement compared to the reproduced results of the baseline, and it also surpasses the official checkpoint. We would like to clarify that, due to the limited time available for the rebuttal, we reduced the total number of training epochs from 60 to 12. We expect that using the original number of training epochs would yield even more promising results.. These experiments further validate the generalization capability of our method.\n\n| Model              | L2 (1s) | L2 (2s) | L2 (3s) | L2 (Avg) | Col (1s) | Col (2s) | Col (3s) | Col (Avg) |\n|--------------------|--------:|--------:|--------:|---------:|---------:|---------:|---------:|----------:|\n| GenAD (official)   |   0.25  |   0.46  |   0.76  |    0.49  |    0.11  |    0.21  |    0.45  |     0.26  |\n| GenAD (reproduce)  |   0.25  |   0.46  |   0.76  |    0.49  |    0.14  |    0.26  |    0.50  |     0.30  |\n| **${E^3AD}$** (GenAD)        |   0.24  |   0.44  |   0.74  |    0.47  |    0.10  |    0.21  |    0.42  |     0.24  |\n\n>Second, due to the limited rebuttal period, we commit to making further efforts to explore other recent end-to-end autonomous driving models (such as LAW) and recent benchmarks (such as NavSim), while continuing to enhance the performance of our method on GenAD. To demonstrate our ongoing commitment to improving the model, we also provide in the table below our recent efforts applying **${E^3AD}$**  to VAD. Through the expansion of EEG data and model optimization, our model’s performance has achieved a higher level. We further commit that these efforts will be fully documented in the main text or supplementary materials of the final version of the manuscript.\n\n| Model         | L2 (1s) | L2 (2s) | L2 (3s) | L2 (Avg) | Col (1s) | Col (2s) | Col (3s) | Col (Avg) |\n|---------------|--------:|--------:|--------:|---------:|---------:|---------:|---------:|----------:|\n| Baseline (VAD) |  0.41   |  0.70   |  1.05   |   0.72   |   0.07   |   0.17   |   0.41   |    0.22   |\n| **${E^3AD}$** (VAD)      |  0.40   |  0.68   |  1.03   |   0.70   |   0.04   |   0.12   |   0.30   |    0.15   |\n\n## Responses to Specific Questions\n\n**Question 1:**  \n >Please refer to Response to weakness 1\n\n**Question 2:**  \n> For the first question, please refer to our Response to Weakness 2. Regarding the second question, we have not yet seen relevant reports addressing the relationship between other physiological signals and driving. However, we believe that future studies could explore this topic using data from functional near-infrared spectroscopy (fNIRS) and magnetoencephalography (MEG).\n\n**Question 3:**  \nThank you for your question and for raising this detailed point.\n>Our proposed Driving–Thinking Model is trained with paired EEG and video data using contrastive learning based on LaBraM. It learns general-purpose representations from LaBraM that can be directly applied to different end-to-end autonomous driving models. For a given set of EEG data, the Driving–Thinking Model only needs to be trained once. Thereafter, it can be seamlessly integrated into any end-to-end model via the **${E^3AD}$** framework and trained according to the original procedure and schedule of that specific model. Regarding the training cost of the Driving–Thinking Model, we provide the following data for your reference: 3 hours of training on a single NVIDIA A100 GPU with 10GB VRAM.\n\n**Question 4:**  \n>Thank you for your valuable feedback. We have made further efforts and attempts in response. Please refer to our Response to Weakness 3 for details.\n---"}, {"review_id": "SFXAsWdrTg", "reviewer": "Reviewer_bynr", "summary": "This paper introduces 'Embodied Cognition Augmented End-to-End Autonomous Driving', a novel paradigm that integrates human cognitive features derived from EEG signals to enhance end-to-end autonomous driving planning. The method employs a two-stage training framework: In stage 1, it employs contrastive learning between a visual feature extraction network (Driving-Thinking Model) and a large EEG model (LaBraM) on a self-collected dataset. The dataset includes EEG signals from expert/novice drivers; Then, in stage 2, they integrate the frozen 'Driving-Thinking Model' into a popular end-to-end driving framework (e.g., VAD, UniAD) without EEG inputs. They evaluate on NuScenes (open-loop) and Bench2Drive (closed-loop), which show significant improvements.", "questions": "1. Could you visualize what the Driving-Thinking Model learns (e.g., saliency maps)? What specific cognitive aspects (e.g., hazard anticipation, path planning) do EEG features capture?\n\n2. How do expert vs. novice cognitive features qualitatively differ?", "limitations": "1. Reproducibility Risk:\nThe paper does not have a commitment to releasing the EEG dataset, despite its centrality to Stage 1. Code alone is insufficient without data.\n\n2. Narrow Driver Demographics:\nData are collected from 20 male drivers (10 experts/10 novices), which limits generalizability across genders, ages, and cultural driving contexts.\n\n3. Real-World Gaps:\nThe closed-loop is in CARLA simulations only. There are no real-world deployments or cross-dataset validation (e.g., Waymo).", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. Novelty: First work to incorporate human cognitive data (EEG) into end-to-end autonomous driving, pioneering a brain-inspired paradigm.\n2. Rigorous Evaluation: Comprehensive testing across open-loop (L2 error, collision rate) and closed-loop (driving score, completion rate) metrics, with comparisons against SOTA baselines (VAD, UniAD).\n\n\nWeakness:\n\n1. Limited Alignment Validation in Stage 1:\n1.1 There are no quantitative evaluations (e.g., similarity metrics) to verify if Stage 1 contrastive learning successfully aligns visual and EEG features.\n1.2 It is unclear how well cognitive features correlate with driving semantics beyond collision-rate improvements. Does it have more insights or theoretical analysis?\n\n2. Theoretical Gaps:\n2.1 The paper lacks a mechanistic analysis of why EEG features improve planning (e.g., hazard anticipation, decision-making patterns). There is no discussion on cognitive feature interpretability (e.g., attention maps linking EEG to critical driving events).\n\n3. Dataset Scalability & Release:\nThe collected EEG dataset is the core contribution of the paper. However, the collected EEG dataset is small (20 drivers, 2.3k clips), with no mention of public release. This hinders reproducibility and community validation. \nI also have scalability concerns. Could the EEG data be easily scaled up? Recently, the autonomous driving community is using larger and larger data to scale the performance. However, from the paper, I could not figure out if this method could still achieve better performance when scaling data. If so, could it solve various corner cases with scaled data?\n\n4. Overreliance on LaBraM:\nThe paper uses a frozen LaBraM model pre-trained on thousands of hours of EEG data. No ablation on LaBraM’s necessity or alternatives for resource-constrained settings.", "was_revised": true, "final_justification": "The paper proposes a novel paradigm that integrates human cognitive features derived from EEG signals to enhance end-to-end autonomous driving planning. Authors have committed to releasing all datasets and collection details. This will provide insight for solving E2E driving. Considering the novelty and thorough ablations, I would like to improve my rate to borderline accept.", "rebuttal": "Thank you for highlighting the innovation of our integration of human cognitive data (EEG) into end-to-end autonomous driving, as well as recognizing the rigor of our experimental design. We appreciate your concerns and address each of them below.\n\n---\n## Response to Weakness\n**Weakness 1:**\n>Regarding point 1.1, since there are no standard metrics for EEG-video alignment, we adopt the widely used contrastive loss from CLIP-related works as a proxy. Our loss is a symmetric cross-entropy over the similarity matrix of normalized EEG and video embeddings in a minibatch. The contrastive loss encourages higher cosine similarity for matching EEG-video pairs than mismatched ones. In our experiments, the evaluation contrastive loss dropped from 2.7 to 2.2, compared to a random baseline of roughly ln(16) ≈ 2.77, indicating stronger alignment despite dataset noise.\n\n>Regarding point 1.2, our improvements extend beyond standard metrics like L2 error and collision rate. Supplementary Table 6 compares detailed driving metrics under the large-scale Bench2Drive benchmark (220 routes, 12 towns, 23 weather conditions in CARLA v2). Our model reduces vehicle collisions and red light violations by 11.82% (38.8%) and 1.37% (37.6%), respectively, and decreases stop infractions by 0.46% (16.8%). Collisions with layouts and lane departures are also mitigated. These results show that **$E^3AD$** significantly enhances driving performance, especially in complex scenarios involving traffic interactions, mainly through a more disciplined driving style and collision avoidance.\n\n---\n\n**Weakness 2:**\n\n>Thank you for pointing this out. In the current version of our work, We have conducted preliminary experiments exploring cognitive feature-enhanced driving; however, we have not yet performed a detailed investigation into the underlying mechanisms. Therefore, we plan to visualize the outputs of our Driving-Thinking Model and explore their relationship with the visual driving scene, in order to better understand the cognitive-planning connection.\n\n>It is worth noting that our bold attempt to integrate cognitive features into autonomous driving is grounded in the theoretical foundations of this field. Prior research has shown that specific EEG components—such as the P400, associated with attention engagement [1], and the N500, associated with unpredictability processing [2]—are characteristic markers of a driver’s hazard detection in complex scenarios [3]. We believe these neural signals can play a crucial role in enhancing planning capabilities when integrated into autonomous driving models. To provide more intuition behind our design, we will include additional discussion on these neuroscience findings in the revised version.\n\n**References**  \n**(1)** The cortical development of specialized face processing in infancy  \n**(2)** Predicting outcomes of decisions in the brain  \n**(3)** Annotating Covert Hazardous Driving Scenarios Online: Utilizing Drivers’ Electroencephalography (EEG) Signals  \n\n---\n\n**Weakness 3:**\n\nThank you for highlighting these important concerns.\n>First, We are committed to releasing the full paired EEG-video dataset, ensuring its usability and clarity, along with detailed usage instructions and ethical safeguards.\n\n>Second, Regarding scalability, we commit to publicly sharing the data collection process and expanding the dataset, with plans to release it within the next year. We acknowledge the challenges of scaling EEG data collection due to the need for specialized equipment. Therefore, we are actively seeking collaborations with industry partners.\n\n>Thirdly, with respect to dataset size, in this work we leverage LaBraM to enhance the generalization capability of our model. Specifically, we freeze LaBraM during training and use contrastive learning in the proposed Driving–Thinking model to learn generalizable features from LaBraM. This allows us to mitigate the limitations arising from the relatively small size of our collected dataset, thanks to LaBraM’s strong generalization and the universality of its features.\n\n---\n\n**Weakness 4:**\n\n>Thank you for your thoughtful review and question. Interestingly, we have explored the alternative you mentioned: in our initial work, a pre-trained ResNet50 and projection layer were used to align visual and cognitive representations, with MSE loss to reconstruct EEG signals. This approach yielded notable improvements over the baseline, reducing average L2 error by 0.105 m (13.9%) and collision rate by 0.03% (13.6%), as shown in the table below. However, we later found that combining LaBraM with contrastive learning offered significantly better generalization—especially for collision rate and closed-loop scenarios—so we adopted this improved method. We commit to providing these results and code in the supplementary materials of the revised version.\n\n| Model | L2 (1s) | L2 (2s) | L2 (3s) | L2 (Avg) | Col (1s) | Col (2s) | Col (3s) | Col (Avg) |\n|------------------------------:|--------:|--------:|--------:|---------:|---------:|---------:|---------:|----------:|\n| Baseline (VAD)|    0.41 |0.70 | 1.05 |0.72 |0.07 |     0.17 |     0.41 |      0.22 |\n|**$E^3AD$** (regression) – all|    0.33 |    0.60 |    0.92 |     0.62 |     0.09 |     0.15 |     0.33 |      0.19 |\n|**$E^3AD$**(regression) – expert    |    0.34 |    0.60 |    0.93 |     0.62 |     0.11 |     0.17 |     0.34 |      0.21 |\n|**$E^3AD$**(regression) – novice    |    0.40 |    0.65 |    0.96 |     0.67 |     0.11 |     0.20 |     0.35 |      0.22 |\n\n> Regarding resource-constrained settings: Our Driving–Thinking Model is trained once using paired EEG and video data with contrastive learning (LaBraM). It can then be integrated into various end-to-end autonomous driving models via the **${E^3AD}$** framework without retraining. As shown in Table 1 of manuscript, we measured FPS before and after **${E^3AD}$** integration on the same hardware (NVIDIA A100 GPU). Results show FPS remained stable for both lightweight models like VAD-tiny and larger models like Uniad. Therefore, our method adds minimal computational overhead and suits resource-limited environments.\n\n---\n\n## Responses to Specific Questions\n\n**Question 1:**  \n>First, as discussed in the Response to Weakness 2, previous research has demonstrated that specific EEG signals are associated with drivers' risk perception and unpredictability processing. These include a driver’s perception of scene hazards and their planning strategies. The experiments in our paper also support that EEG signals primarily encode information related to planning styles in different driving scenarios.\n\n>Second, we agree that visualizing the learned features—particularly in relation to specific cognitive aspects—would provide valuable insights. Recently, we have attempted to perform clustering analysis on the output features of the Driving-Thinking Model in different scenarios. However, due to the limited time available for this rebuttal, we commit to discussing these results in the revised version.\n\n**Question 2:**  \n> Before we designed our model, we visualized the EEG PSD (power spectral density) topomap of expert group and novice group in the frequency model (we use python mne package for visualization). We split five frequency bands (i.e. Delta, theta, alpha, beta and gamma) and find expert drivers' power concentrate on occipital lobe (visual areas) while novice drivers' power concentrate on frontal lobe (higher cognitive functions, e.g. Decision making). That's an interesting finding. It seems that novice needs to pay more energy to think about what I should do next, while expert only needs to see, and he knows how to do next.\n\n---\n\n## Response to Limitations\n\n**Limitation 1:**  \n>Please refer to Response to weakness 3\n\n**Limitation 2:**  \n>For this limitation, we will clearly state it in the Limitation section. In addition, as addressed in our response to Weakness 3, we will make the dataset and the corresponding data collection procedures publicly available. On this basis, we will further expand the dataset to better address this limitation.\n\n**Limitation 3:**  \n>To address this limitation, we took the following steps:\n\n>First, we collected real-world data using a vehicle-mounted platform equipped with an Insta360 panoramic fisheye camera and an inertial sensor. The panoramic videos were undistorted and split into six view-angle videos, following the nuScenes protocol. Using these multi-view videos along with CAN bus data, we evaluated both the baseline model trained on nuScenes and our **$E^3AD$** model. The **$E^3AD$** model showed better trajectory continuity and more accurate turning intention recognition. We will provide these videos as supplementary material.\n\n>Second, to verify cross-dataset performance and generalization to unseen scenarios, we conducted experiments on the nuScenes dataset divided by location: Singapore with 12,435 training and 2,929 test samples, and Boston with 15,695 training and 3,090 test samples. The model was trained on one city’s data and tested on the other, ensuring evaluation on truly unseen geographic domains. As shown in the table below, our model demonstrates a clear advantage over the baseline (VAD-tiny) in validation on unseen data.\n\n| Scenario | L2 (1s) | L2 (2s) | L2 (3s) | L2 (Avg) | Col (1s) | Col (2s) | Col (3s) | Col (Avg) |\n|---------------------------------------------|--------:|--------:|--------:|---------:|---------:|---------:|---------:|----------:|\n| Baseline (Training on Singapore)| 0.51 |    0.84 |    1.21 |     0.85 |     0.29 | 0.42 | 0.61 | 0.44 |\n| **$E^3AD$** (Training on Singapore)  | 0.44 |    0.71 |    1.03 |     0.73 |     0.15 |     0.30 | 0.56 |  0.34 |\n| Baseline (Training on Boston)  | 0.45 | 0.77 |    1.17 |     0.80 |     0.22 |     0.35 |     0.76 |      0.44 |\n| **$E^3AD$** (Training on Boston)| 0.46 | 0.76 |    1.13 |     0.78 |     0.22 |     0.42 |     0.50 |      0.38 |"}]

Author rebuttal:

Meta-review: This paper introduces a framework which learns a “Driving-Thinking” encoder by contrastively aligning in-car video with EEG features from a pretrained brain activity model. The encoder is then frozen and integrated into existing end-to-end planners such as VAD and UniAD, without requiring EEG at inference. Results on nuScenes (open-loop) and Bench2Drive (closed-loop) show consistent improvements in L2 error, collision rate, and driving score, with negligible runtime overhead. The integration is straightforward and reproducible, and the planned release of data and code would further enhance its value.

The AC views the contribution as primarily conceptual and exploratory. The idea of incorporating human cognitive signals into end-to-end driving models is novel and interesting, but the current implementation remains early-stage. The paired EEG–video dataset is relatively small and demographically narrow, raising concerns about generalization and bias. The precise mechanisms by which EEG alignment improves planning are not deeply analyzed, and the evaluation relies heavily on open-loop nuScenes and simulation-only closed-loop tests, which do not yet capture real-world safety-critical aspects.

Overall, despite these limitations, the work is technically sound, clearly presented, and demonstrates a promising new direction for cognition-augmented learning in end-to-end driving. The AC thus recommends acceptance, and the authors are encouraged to calibrate claims, provide a transparent discussion of dataset limitations, and strengthen evaluation and analysis in future versions.
Decision: Accept (poster)