DiffSemanticFusion: Semantic Raster BEV Fusion for Autonomous Driving
  via Online HD Map Diffusion
Authors: name: Zhigang Sun, name: Yiru Wang, name: Anqing Jiang, name: Shuo Wang, name: Yu Gao, name: Yuwen Heng, name: Shouyi Zhang, name: An He, name: Hao Jiang, name: Jinhao Chai, name: Zichong Gu, name: Wang Jijun, name: Shichen Tang, name: Lavdim Halilaj, name: Juergen Luettin, name: Hao Sun

Autonomous driving requires accurate scene understanding, including road
geometry, traffic agents, and their semantic relationships. In online HD map
generation scenarios, raster-based representations are well-suited to vision
models but lack geometric precision, while graph-based representations retain
structural detail but become unstable without precise maps. To harness the
complementary strengths of both, we propose DiffSemanticFusion -- a fusion
framework for multimodal trajectory prediction and planning. Our approach
reasons over a semantic raster-fused BEV space, enhanced by a map diffusion
module that improves both the stability and expressiveness of online HD map
representations. We validate our framework on two downstream tasks: trajectory
prediction and planning-oriented end-to-end autonomous driving. Experiments on
real-world autonomous driving benchmarks, nuScenes and NAVSIM, demonstrate
improved performance over several state-of-the-art methods. For the prediction
task on nuScenes, we integrate DiffSemanticFusion with the online HD map
informed QCNet, achieving a 5.1\% performance improvement. For end-to-end
autonomous driving in NAVSIM, DiffSemanticFusion achieves state-of-the-art
results, with a 15\% performance gain in NavHard scenarios. In addition,
extensive ablation and sensitivity studies show that our map diffusion module
can be seamlessly integrated into other vector-based approaches to enhance
performance. All artifacts are available at
https://github.com/SunZhigang7/DiffSemanticFusion.