BABE: Biology Arena BEnchmark
Authors: name: Junting Zhou, name: Jin Chen, name: Linfeng Hao, name: Denghui Cao, name: Zheyu Wang, name: Qiguang Chen, name: Chaoyou Fu, name: Jiaze Chen, name: Yuchen Wu, name: Ge Zhang, name: Mingxuan Wang, name: Wenhao Huang, name: Tong Yang
Organization: ByteDance Seed

The rapid evolution of large language models (LLMs) has expanded their capabilities from basic dialogue to advanced scientific reasoning. However, existing benchmarks in biology often fail to assess a critical skill required of researchers: the ability to integrate experimental results with contextual knowledge to derive meaningful conclusions. To address this gap, we introduce BABE(Biology Arena BEnchmark), a comprehensive benchmark designed to evaluate the experimental reasoning capabilities of biological AI systems. BABE is uniquely constructed from peer-reviewed research papers and real-world biological studies, ensuring that tasks reflect the complexity and interdisciplinary nature of actual scientific inquiry. BABE challenges models to perform causal reasoning and cross-scale inference. Our benchmark provides a robust framework for assessing how well AI systems can reason like practicing scientists, offering a more authentic measure of their potential to contribute to biological research.