Mathesis: Towards Formal Theorem Proving from Natural Languages
Authors: name: Yu Xuejun; name: Jianyuan Zhong; name: Zijin Feng; name: Pengyi Zhai; name: Roozbeh Yousefzadeh; name: Wei Chong Ng; name: Haoxiong Liu; name: Ziyi Shou; name: Jing Xiong; name: Yudong Zhou; name: Claudia Beth Ong; name: Austen Jeremy Sugiarto; name: Yaoxi Zhang; name: Wai Ming Tai; name: Huan Cao; name: Dongcai Lu; name: Jiacheng Sun; name: Qiang Xu; name: Shen Xin; name: Zhenguo Li
Published: 2025-06-08 09:04:14

Abstract: Recent advances in large language models show strong promise for formal
reasoning. However, most LLM-based theorem provers have long been constrained
by the need for expert-written formal statements as inputs, limiting their
applicability to real-world problems expressed in natural language. We tackle
this gap with Mathesis, the first end-to-end theorem proving pipeline
processing informal problem statements. It contributes Mathesis-Autoformalizer,
the first autoformalizer using reinforcement learning to enhance the
formalization ability of natural language problems, aided by our novel
LeanScorer framework for nuanced formalization quality assessment. It also
proposes a Mathesis-Prover, which generates formal proofs from the formalized
statements. To evaluate the real-world applicability of end-to-end formal
theorem proving, we introduce Gaokao-Formal, a benchmark of 488 complex
problems from China's national college entrance exam. Our approach is carefully
designed, with a thorough study of each component. Experiments demonstrate
Mathesis's effectiveness, with the autoformalizer outperforming the best
baseline by 22% in pass-rate on Gaokao-Formal. The full system surpasses other
model combinations, achieving 64% accuracy on MiniF2F with pass@32 and a
state-of-the-art 18% on Gaokao-Formal.