Curie: Toward Rigorous and Automated Scientific Experimentation with AI
  Agents
Authors: name: Patrick Tser Jern Kon; name: Jiachen Liu; name: Qiuyi Ding; name: Yiming Qiu; name: Zhenning Yang; name: Yibo Huang; name: Jayanth Srinivasa; name: Myungjin Lee; name: Mosharaf Chowdhury; name: Ang Chen
Published: 2025-02-22 03:58:19

Abstract: Scientific experimentation, a cornerstone of human progress, demands rigor in
reliability, methodical control, and interpretability to yield meaningful
results. Despite the growing capabilities of large language models (LLMs) in
automating different aspects of the scientific process, automating rigorous
experimentation remains a significant challenge. To address this gap, we
propose Curie, an AI agent framework designed to embed rigor into the
experimentation process through three key components: an intra-agent rigor
module to enhance reliability, an inter-agent rigor module to maintain
methodical control, and an experiment knowledge module to enhance
interpretability. To evaluate Curie, we design a novel experimental benchmark
composed of 46 questions across four computer science domains, derived from
influential research papers, and widely adopted open-source projects. Compared
to the strongest baseline tested, we achieve a 3.4times improvement in
correctly answering experimental questions.Curie is open-sourced at
https://github.com/Just-Curieous/Curie.