AIDE: AI-Driven Exploration in the Space of Code
Authors: name: Zhengyao Jiang; name: Dominik Schmidt; name: Dhruv Srikanth; name: Dixing Xu; name: Ian Kaplan; name: Deniss Jacenko; name: Yuxiang Wu
Published: 2025-02-18 18:57:21

Abstract: Machine learning, the foundation of modern artificial intelligence, has
driven innovations that have fundamentally transformed the world. Yet, behind
advancements lies a complex and often tedious process requiring labor and
compute intensive iteration and experimentation. Engineers and scientists
developing machine learning models spend much of their time on trial-and-error
tasks instead of conceptualizing innovative solutions or research hypotheses.
To address this challenge, we introduce AI-Driven Exploration (AIDE), a machine
learning engineering agent powered by large language models (LLMs). AIDE frames
machine learning engineering as a code optimization problem, and formulates
trial-and-error as a tree search in the space of potential solutions. By
strategically reusing and refining promising solutions, AIDE effectively trades
computational resources for enhanced performance, achieving state-of-the-art
results on multiple machine learning engineering benchmarks, including our
Kaggle evaluations, OpenAI MLE-Bench and METRs RE-Bench.