Sailing AI by the Stars: A Survey of Learning from Rewards in
  Post-Training and Test-Time Scaling of Large Language Models
Authors: name: Xiaobao Wu
Published: 2025-05-05 14:33:49

Abstract: Recent developments in Large Language Models (LLMs) have shifted from
pre-training scaling to post-training and test-time scaling. Across these
developments, a key unified paradigm has arisen: Learning from Rewards, where
reward signals act as the guiding stars to steer LLM behavior. It has
underpinned a wide range of prevalent techniques, such as reinforcement
learning (in RLHF, DPO, and GRPO), reward-guided decoding, and post-hoc
correction. Crucially, this paradigm enables the transition from passive
learning from static data to active learning from dynamic feedback. This endows
LLMs with aligned preferences and deep reasoning capabilities. In this survey,
we present a comprehensive overview of the paradigm of learning from rewards.
We categorize and analyze the strategies under this paradigm across training,
inference, and post-inference stages. We further discuss the benchmarks for
reward models and the primary applications. Finally we highlight the challenges
and future directions. We maintain a paper collection at
https://github.com/bobxwu/learning-from-rewards-llm-papers.