AetherCode: Evaluating LLMs' Ability to Win In Premier Programming
  Competitions
Authors: name: Zihan Wang, name: Jiaze Chen, name: Zhicheng Liu, name: Markus Mak, name: Yidi Du, name: Geonsik Moon, name: Luoqi Xu, name: Aaron Tua, name: Kunshuo Peng, name: Jiayi Lu, name: Mingfei Xia, name: Boqian Zou, name: Chenyang Ran, name: Guang Tian, name: Shoutai Zhu, name: Yeheng Duan, name: Zhenghui Kang, name: Zhenxing Lin, name: Shangshu Li, name: Qiang Luo, name: Qingshen Long, name: Zhiyong Chen, name: Yihan Xiao, name: Yurong Wu, name: Daoguang Zan, name: Yuyi Fu, name: Mingxuan Wang, name: Ming Ding

Competitive programming has emerged as a critical benchmark for evaluating
the reasoning and coding capabilities of Large Language Models (LLMs). Despite
impressive progress on existing benchmarks, we argue that current evaluations
overstate model proficiency, masking a substantial gap between LLMs and elite
human programmers. This gap arises from two key limitations: insufficient
difficulty and scope of benchmark problems, and evaluation bias from
low-quality test cases. To address these shortcomings, we present AetherCode, a
new benchmark that draws problems from premier programming competitions such as
IOI and ICPC, offering broader coverage and higher difficulty. AetherCode
further incorporates comprehensive, expert-validated test suites built through
a hybrid of automated generation and human curation, ensuring rigorous and
reliable assessment. By combining challenging problem design with robust
evaluation, AetherCode provides a more faithful measure of LLM capabilities and
sets a new standard for future research in code reasoning.