ICML 2025

When Every Millisecond Counts: Real-Time Anomaly Detection via the
Multimodal Asynchronous Hybrid Network
Dong Xiao * 1 2 Guangyao Chen * 1 Peixi Peng 3 Yangru Huang 1 Yifan Zhao 4 Yongxing Dai 5
Yonghong Tian 1 2 3 6
Abstract
Anomaly detection is essential for the safety and
reliability of autonomous driving systems. Cur-
rent methods often focus on detection accuracy
but neglect response time, which is critical in
time-sensitive driving scenarios. In this paper,
we introduce real-time anomaly detection for au-
tonomous driving, prioritizing both minimal re-
sponse time and high accuracy. We propose a
novel multimodal asynchronous hybrid network
that combines event streams from event cameras
with image data from RGB cameras. Our net-
work utilizes the high temporal resolution of event
cameras through an asynchronous Graph Neural
Network and integrates it with spatial features
extracted by a CNN from RGB images. This com-
bination effectively captures both the temporal
dynamics and spatial details of the driving en-
vironment, enabling swift and precise anomaly
detection. Extensive experiments on benchmark
datasets show that our approach outperforms ex-
isting methods in both accuracy and response
time, achieving millisecond-level real-time per-
formance.
The code is available at https:
//github.com/PKU-XD/EventAD.
1. Introduction
Autonomous driving technology has been at the forefront
of research and development in recent years, promising to
*Equal contribution 1National Key Laboratory for Multimedia
Information Processing, School of Computer Science, Peking Uni-
versity 2Department of Software and Microelectronics, Peking
University 3School of Electronic and Computer Engineering,
Peking University 4State Key Laboratory of Virtual Reality Tech-
nology and Systems, SCSE, Beihang University 5Baidu Inc
6Peng Cheng Laboratory. Correspondence to: Guangyao Chen
<gy.chen@pku.edu.cn>, Peixi Peng <pxpeng@pku.edu.cn>,
Yonghong Tian <yhtian@pku.edu.cn>.
Proceedings of the 42 ...