Towards Robust Mathematical Reasoning
Authors: name: Thang Luong; name: Dawsen Hwang; name: Hoang H. Nguyen; name: Golnaz Ghiasi; name: Yuri Chervonyi; name: Insuk Seo; name: Junsu Kim; name: Garrett Bingham; name: Jonathan Lee; name: Swaroop Mishra; name: Alex Zhai; name: Clara Huiyi Hu; name: Henryk Michalewski; name: Jimin Kim; name: Jeonghyun Ahn; name: Junhwi Bae; name: Xingyou Song; name: Trieu H. Trinh; name: Quoc V. Le; name: Junehyuk Jung
Published: 2025-11-03 18:53:02

Abstract: Finding the right north-star metrics is highly critical for advancing the
mathematical reasoning capabilities of foundation models, especially given that
existing evaluations are either too easy or only focus on getting correct short
answers. To address these issues, we present IMO-Bench, a suite of advanced
reasoning benchmarks, vetted by a panel of top specialists and that
specifically targets the level of the International Mathematical Olympiad
(IMO), the most prestigious venue for young mathematicians. IMO-AnswerBench
first tests models on 400 diverse Olympiad problems with verifiable short
answers. IMO-Proof Bench is the next-level evaluation for proof-writing
capabilities, which includes both basic and advanced IMO level problems as well
as detailed grading guidelines to facilitate automatic grading. These
benchmarks played a crucial role in our historic achievement of the gold-level
performance at IMO 2025 with Gemini Deep Think (Luong and Lockhart, 2025). Our
model achieved 80.0% on IMO-AnswerBench and 65.7% on the advanced IMO-Proof
Bench, surpassing the best non-Gemini models by large margins of 6.9% and 42.4%
respectively. We also showed that autograders built with Gemini reasoning
correlate well with human evaluations and construct IMO-GradingBench, with 1000
human gradings on proofs, to enable further progress in automatic evaluation of
long-form answers. We hope that IMO-Bench will help the community towards
advancing robust mathematical reasoning and release it at
https://imobench.github.io/.