ICLR 2025

Published as a conference paper at ICLR 2025
LEAN-STAR:
LEARNING TO INTERLEAVE THINKING AND PROVING
Haohan Lin2∗
Zhiqing Sun1
Sean Welleck1
Yiming Yang1
1Language Technologies Institute, Carnegie Mellon University
2Institute for Interdisciplinary Information Sciences, Tsinghua University
{haohanl,zhiqings,swelleck,yiming}@cs.cmu.edu
https://leanstar.github.io/
ABSTRACT
Traditional language model-based theorem proving assumes that by training on a
sufficient amount of formal proof data, a model will learn to prove theorems. Our
key observation is that a wealth of informal information that is not present in formal
proofs can be useful for learning to prove theorems. For instance, humans think
through steps of a proof, but this thought process is not visible in the resulting code.
We present Lean-STaR, a framework for training language models to produce
informal thoughts prior to each step of a proof, thereby boosting the model’s
theorem-proving capabilities. Lean-STaR uses retrospective ground-truth tactics to
generate synthetic thoughts for training the language model. At inference time, the
trained model directly generates the thoughts prior to the prediction of the tactics
in each proof step. Building on the self-taught reasoner framework, we then apply
expert iteration to further fine-tune the model on the correct proofs it samples and
verifies using the Lean solver. Lean-STaR significantly outperforming base models
(43.4% →46.3%, Pass@64). We also analyze the impact of the augmented
thoughts on various aspects of the theorem proving process, providing insights into
their effectiveness.
1
INTRODUCTION
Theorem proving is a fundamental aspect of mathematics, and mathematical reasoning is an important
part of artificial intelligence (Newell & Simon, 1956; Zhou, 2023). Formalized mathematics in
particular provides a challenging testbed for assessing mathematical reasoning capabilities. Since
theorems and proofs in this setting can be represented in the form of checkab...