The AI Scientist: Towards Fully Automated Open-Ended Scientific
  Discovery
Authors: name: Chris Lu; name: Cong Lu; name: Robert Tjarko Lange; name: Jakob Foerster; name: Jeff Clune; name: David Ha
Published: 2024-08-12 16:58:11

Abstract: One of the grand challenges of artificial general intelligence is developing
agents capable of conducting scientific research and discovering new knowledge.
While frontier models have already been used as aids to human scientists, e.g.
for brainstorming ideas, writing code, or prediction tasks, they still conduct
only a small part of the scientific process. This paper presents the first
comprehensive framework for fully automatic scientific discovery, enabling
frontier large language models to perform research independently and
communicate their findings. We introduce The AI Scientist, which generates
novel research ideas, writes code, executes experiments, visualizes results,
describes its findings by writing a full scientific paper, and then runs a
simulated review process for evaluation. In principle, this process can be
repeated to iteratively develop ideas in an open-ended fashion, acting like the
human scientific community. We demonstrate its versatility by applying it to
three distinct subfields of machine learning: diffusion modeling,
transformer-based language modeling, and learning dynamics. Each idea is
implemented and developed into a full paper at a cost of less than $15 per
paper. To evaluate the generated papers, we design and validate an automated
reviewer, which we show achieves near-human performance in evaluating paper
scores. The AI Scientist can produce papers that exceed the acceptance
threshold at a top machine learning conference as judged by our automated
reviewer. This approach signifies the beginning of a new era in scientific
discovery in machine learning: bringing the transformative benefits of AI
agents to the entire research process of AI itself, and taking us closer to a
world where endless affordable creativity and innovation can be unleashed on
the world's most challenging problems. Our code is open-sourced at
https://github.com/SakanaAI/AI-Scientist