Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following
Authors: name: Yun He; name: Wenzhe Li; name: Hejia Zhang; name: Songlin Li; name: Karishma Mandyam; name: Sopan Khosla; name: Yuanhao Xiong; name: Nanshu Wang; name: Selina Peng; name: Beibin Li; name: Shengjie Bi; name: Shishir G. Patil; name: Qi Qi; name: Shengyu Feng; name: Julian Katz-Samuels; name: Richard Yuanzhe Pang; name: Sujan Gonugondla; name: Hunter Lang; name: Yue Yu; name: Yundi Qian; name: Maryam Fazel-Zarandi; name: Licheng Yu; name: Amine Benhalloum; name: Hany Awadalla; name: Manaal Faruqui
Published: 2025-11-13 17:14:01

Abstract: Recent progress in large language models (LLMs) has led to impressive performance on a range of tasks, yet advanced instruction following (IF)-especially for complex, multi-turn, and system-prompted instructions-remains a significant challenge. Rigorous evaluation and effective training for such capabilities are hindered by the lack of high-quality, human-annotated benchmarks and reliable, interpretable reward signals. In this work, we introduce AdvancedIF (we will release this benchmark soon), a comprehensive benchmark featuring over 1,600 prompts and expert-curated rubrics that assess LLMs ability to follow complex, multi-turn, and system-level instructions. We further propose RIFL (Rubric-based Instruction-Following Learning), a novel post-training pipeline that leverages rubric generation, a finetuned rubric verifier, and reward shaping to enable effective reinforcement learning for instruction following. Extensive experiments demonstrate that RIFL substantially improves the instruction-following abilities of LLMs, achieving a 6.7% absolute gain on AdvancedIF and strong results on public benchmarks. Our ablation studies confirm the effectiveness of each component in RIFL. This work establishes rubrics as a powerful tool for both training and evaluating advanced IF in LLMs, paving the way for more capable and reliable AI systems.