Title: LeanForPhysics: Comprehensive Reasoning Framework for University-level Physics in Lean4
Primary area: datasets and benchmarks
Keywords: Lean4, Reasoning, AIforScience

Abstract: We present **Lean4PHYS**, a comprehensive reasoning framework for college-level physics problems in Lean4. **Lean4PHYS** includes *LeanPhysBench*, a college-level benchmark for Lean4 formal physics reasoning, which contains 200 hand-crafted and peer-reviewed statements formalized from university textbooks and physics competition problems. To establish a solid foundation for formal reasoning in physics, we also launch *PhysLib*, a community-driven repository that includes fundamental unit systems and theorems essential for formal physics reasoning. Based on the benchmark and Lean4 repository we composed in **Lean4PHYS**, we report baseline results using major expert Math Lean4 provers and state-of-the-art closed-source models, and provide an analysis of their performance. In the experiment, we identify that most expert provers do not outperform general models as they did in the math domain. This indicates a potential overfitting of the math domain rather than learning formal reasoning. We also conduct a comprehensive experiment showing that with *PhysLib* in the context, LLMs' performance on*LeanPhysBench* increases by 11.88\% on average, proving the effectiveness of our repository in assisting LLMs to solve the Lean4 physics problem.  To the best of our knowledge, we are the first study to provide a physics benchmark in Lean4.

TL;DR: This paper presents Lean4PHYS, a reasoning framework for college-level physics problems in Lean4. It includes LeanPhysBench, the first benchmark in the field, and PhysLib, a community-driven repository that sets the foundation for the field.