MLGym: A New Framework and Benchmark for Advancing AI Research Agents
Authors: name: Deepak Nathani, name: Lovish Madaan, name: Nicholas Roberts, name: Nikolay Bashlykov, name: Ajay Menon, name: Vincent Moens, name: Amar Budhiraja, name: Despoina Magka, name: Vladislav Vorotilov, name: Gaurav Chaurasia, name: Dieuwke Hupkes, name: Ricardo Silveira Cabral, name: Tatiana Shavrina, name: Jakob Foerster, name: Yoram Bachrach, name: William Yang Wang, name: Roberta Raileanu

We introduce Meta MLGym and MLGym-Bench, a new framework and benchmark for
evaluating and developing LLM agents on AI research tasks. This is the first
Gym environment for machine learning (ML) tasks, enabling research on
reinforcement learning (RL) algorithms for training such agents. MLGym-bench
consists of 13 diverse and open-ended AI research tasks from diverse domains
such as computer vision, natural language processing, reinforcement learning,
and game theory. Solving these tasks requires real-world AI research skills
such as generating new ideas and hypotheses, creating and processing data,
implementing ML methods, training models, running experiments, analyzing the
results, and iterating through this process to improve on a given task. We
evaluate a number of frontier large language models (LLMs) on our benchmarks
such as Claude-3.5-Sonnet, Llama-3.1 405B, GPT-4o, o1-preview, and Gemini-1.5
Pro. Our MLGym framework makes it easy to add new tasks, integrate and evaluate
models or agents, generate synthetic data at scale, as well as develop new
learning algorithms for training agents on AI research tasks. We find that
current frontier models can improve on the given baselines, usually by finding
better hyperparameters, but do not generate novel hypotheses, algorithms,
architectures, or substantial improvements. We open-source our framework and
benchmark to facilitate future research in advancing the AI research
capabilities of LLM agents.