V2V-GoT: Vehicle-to-Vehicle Cooperative Autonomous Driving with
  Multimodal Large Language Models and Graph-of-Thoughts
Authors: name: Hsu-kuang Chiu, name: Ryo Hachiuma, name: Chien-Yi Wang, name: Yu-Chiang Frank Wang, name: Min-Hung Chen, name: Stephen F. Smith
Organization: NVIDIA

Current state-of-the-art autonomous vehicles could face safety-critical
situations when their local sensors are occluded by large nearby objects on the
road. Vehicle-to-vehicle (V2V) cooperative autonomous driving has been proposed
as a means of addressing this problem, and one recently introduced framework
for cooperative autonomous driving has further adopted an approach that
incorporates a Multimodal Large Language Model (MLLM) to integrate cooperative
perception and planning processes. However, despite the potential benefit of
applying graph-of-thoughts reasoning to the MLLM, this idea has not been
considered by previous cooperative autonomous driving research. In this paper,
we propose a novel graph-of-thoughts framework specifically designed for
MLLM-based cooperative autonomous driving. Our graph-of-thoughts includes our
proposed novel ideas of occlusion-aware perception and planning-aware
prediction. We curate the V2V-GoT-QA dataset and develop the V2V-GoT model for
training and testing the cooperative driving graph-of-thoughts. Our
experimental results show that our method outperforms other baselines in
cooperative perception, prediction, and planning tasks.