Title: LLM-based Multi-Agent Blackboard System for Information Discovery in Data Science
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: multi-agent, data science, data discovery, large language models

Abstract: The rapid advancement of Large Language Models (LLMs) has opened new opportunities in data science, yet their practical deployment is often constrained by the challenge of discovering relevant data within large data lakes. Existing methods struggle with this: single-agent systems are quickly overwhelmed by large, heterogeneous files in the data lakes, while multi-agent systems designed based on a master–slave paradigm depend on a rigid central controller that requires precise knowledge of each sub-agent's capabilities. To address these limitations, we propose a novel multi-agent communication paradigm inspired by the blackboard architecture for traditional AI models. In this framework, a central agent posts requests to a shared blackboard, and autonomous subordinate agents--either responsible for a partition of the data lake or general information retrieval--volunteer to respond based on their capabilities. This design improves scalability and flexibility by eliminating the need for a central coordinator to have prior knowledge of all sub-agents expertise. We evaluate our method on three benchmarks that require explicit data discovery: KramaBench and modified versions of DS-Bench and DA-Code to incorporate data discovery. Experimental results demonstrate that the blackboard architecture substantially outperforms baselines, including RAG and the master–slave multi-agent paradigm, achieving between 13% to 57% relative improvement in end-to-end task success and up to a 9\% relative gain in F1 score for data discovery over the best-performing baselines across both proprietary and open-source LLMs. Our findings establish the blackboard paradigm as a scalable and generalizable communication framework for multi-agent systems.

TL;DR: A new blackboard-based multi-agent system helps LLMs accurately discover relevant data in large data lakes, significantly outperforming existing methods on data discovery and end-to-end task success.