Alpha Excel Benchmark
Authors: name: David Noever; name: Forrest McKee
Published: 2025-05-07 03:56:26

Abstract: This study presents a novel benchmark for evaluating Large Language Models
(LLMs) using challenges derived from the Financial Modeling World Cup (FMWC)
Excel competitions. We introduce a methodology for converting 113 existing FMWC
challenges into programmatically evaluable JSON formats and use this dataset to
compare the performance of several leading LLMs. Our findings demonstrate
significant variations in performance across different challenge categories,
with models showing specific strengths in pattern recognition tasks but
struggling with complex numerical reasoning. The benchmark provides a
standardized framework for assessing LLM capabilities in realistic
business-oriented tasks rather than abstract academic problems. This research
contributes to the growing field of AI benchmarking by establishing proficiency
among the 1.5 billion people who daily use Microsoft Excel as a meaningful
evaluation metric that bridges the gap between academic AI benchmarks and
practical business applications.