Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following
Authors: name: Yun He, name: Wenzhe Li, name: Hejia Zhang, name: Songlin Li, name: Karishma Mandyam, name: Sopan Khosla, name: Yuanhao Xiong, name: Nanshu Wang, name: Selina Peng, name: Beibin Li, name: Shengjie Bi, name: Shishir G. Patil, name: Qi Qi, name: Shengyu Feng, name: Julian Katz-Samuels, name: Richard Yuanzhe Pang, name: Sujan Gonugondla, name: Hunter Lang, name: Yue Yu, name: Yundi Qian, name: Maryam Fazel-Zarandi, name: Licheng Yu, name: Amine Benhalloum, name: Hany Awadalla, name: Manaal Faruqui

Recent progress in large language models (LLMs) has led to impressive performance on a range of tasks, yet advanced instruction following (IF)-especially for complex, multi-turn, and system-prompted instructions-remains a significant challenge. Rigorous evaluation and effective training for such capabilities are hindered by the lack of high-quality, human-annotated benchmarks and reliable, interpretable reward signals. In this work, we introduce AdvancedIF (we will release this benchmark soon), a comprehensive benchmark featuring over 1,600 prompts and expert-curated rubrics that assess LLMs ability to follow complex, multi-turn, and system-level instructions. We further propose RIFL (Rubric-based Instruction-Following Learning), a novel post-training pipeline that leverages rubric generation, a finetuned rubric verifier, and reward shaping to enable effective reinforcement learning for instruction following. Extensive experiments demonstrate that RIFL substantially improves the instruction-following abilities of LLMs, achieving a 6.7% absolute gain on AdvancedIF and strong results on public benchmarks. Our ablation studies confirm the effectiveness of each component in RIFL. This work establishes rubrics as a powerful tool for both training and evaluating advanced IF in LLMs, paving the way for more capable and reliable AI systems.