Title: Tri-Agent Driving: Learning to Coordinate Agents via Scenario Complexity Representation for Efficient Autonomous Driving
Primary area: applications to robotics, autonomy, planning
Keywords: Scenario Complexity-Aware, Vision-Language Models, Autonomous Driving

Abstract: We present Tri-Agent Driving (TAD), a novel learning framework that dynamically coordinates multiple autonomous agents by learning lightweight scenario complexity representation directly from sensor inputs. This learned representation serves as a routing signal to enable real-time activation of the optimal agent, balancing computational efficiency and reasoning depth on demand. TAD integrates three complementary agents: a Fast Agent optimized for low-complexity and routine scenarios, a Smart Agent for medium-complexity scenes and a Deep Thinking Agent enhanced with Chain-of-Thought (CoT) reasoning for high-complexity corner cases. The core of TAD lies in the trainable Agent Coordination module, which proactively predicts scenario complexity and triggers agent switching without human intervention. On a challenging hybrid test set spanning diverse traffic conditions, TAD achieves state-of-the-art trajectory prediction, while reducing average inference latency by 26\% (4.2s vs. 5.7s) and GPU memory consumption by 30\% (15.4 GB vs. 22 GB) compared to the strongest VLM-based model. This ``fast when possible, deep when necessary” paradigm establishes a new standard for efficient, robust, and adaptive end-to-end autonomous driving.