ICML 2025

LLM-SRBench: A New Benchmark for Scientific Equation Discovery with
Large Language Models
Parshin Shojaee∗1 Ngoc-Hieu Nguyen∗2 Kazem Meidani 3 4 Amir Barati Farimani 3
Khoa D Doan 2 Chandan K Reddy 1
Website: https://github.com/deep-symbolic-mathematics/llm-srbench
Abstract
Scientific equation discovery has long been a
cornerstone of scientific progress, enabling the
derivation of laws governing natural phenomena.
Recently, Large Language Models (LLMs) have
gained interest for this task due to their poten-
tial to leverage embedded scientific knowledge
for hypothesis generation. However, it is diffi-
cult to assess the true discovery capabilities of
these methods because existing benchmarks of-
ten use well-known equations. This makes them
vulnerable to memorization by LLMs and results
in inflated performance metrics that do not re-
flect genuine discovery.
In this paper, we in-
troduce LLM-SRBench, a comprehensive bench-
mark with 239 challenging problems across four
scientific domains specifically designed to eval-
uate LLM-based scientific equation discovery
methods while preventing trivial memorization.
Our benchmark comprises two main categories:
LSR-Transform, which transforms common phys-
ical models into less common mathematical repre-
sentations to test reasoning beyond memorized
forms, and LSR-Synth, which introduces syn-
thetic, discovery-driven problems requiring data-
driven reasoning. Through extensive evaluation
of several state-of-the-art methods, using both
open and closed LLMs, we find that the best-
performing system so far achieves only 31.5%
symbolic accuracy. These findings highlight the
challenges of scientific equation discovery, posi-
tioning LLM-SRBench as a valuable resource for
future research.
*Equal contribution 1Virginia Tech 2VinUniversity 3Carnegie
Mellon University 4Capital One. Correspondence to: Parshin
Shojaee <parshinshojaee@vt.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 2...