RoMath: A Mathematical Reasoning Benchmark in Romanian
Authors: name: Adrian Cosma; name: Ana-Maria Bucur; name: Emilian Radoi
Published: 2024-09-17 11:03:46

Abstract: Mathematics has long been conveyed through natural language, primarily for
human understanding. With the rise of mechanized mathematics and proof
assistants, there is a growing need to understand informal mathematical text,
yet most existing benchmarks focus solely on English, overlooking other
languages. This paper introduces RoMath, a Romanian mathematical reasoning
benchmark suite comprising three datasets: RoMath-Baccalaureate,
RoMath-Competitions and RoMath-Synthetic, which cover a range of mathematical
domains and difficulty levels, aiming to improve non-English language models
and promote multilingual AI development. By focusing on Romanian, a
low-resource language with unique linguistic features, RoMath addresses the
limitations of Anglo-centric models and emphasizes the need for dedicated
resources beyond simple automatic translation. We benchmark several open-weight
language models, highlighting the importance of creating resources for
underrepresented languages. We make the code and dataset available.