R3: Robust Rubric-Agnostic Reward Models
Authors: name: David Anugraha; name: Zilu Tang; name: Lester James V. Miranda; name: Hanyang Zhao; name: Mohammad Rifqi Farhansyah; name: Garry Kuwanto; name: Derry Wijaya; name: Genta Indra Winata
Published: 2025-05-19 17:29:03

Abstract: Reward models are essential for aligning language model outputs with human
preferences, yet existing approaches often lack both controllability and
interpretability. These models are typically optimized for narrow objectives,
limiting their generalizability to broader downstream tasks. Moreover, their
scalar outputs are difficult to interpret without contextual reasoning. To
address these limitations, we introduce R3, a novel reward modeling framework
that is rubric-agnostic, generalizable across evaluation dimensions, and
provides interpretable, reasoned score assignments. R3 enables more transparent
and flexible evaluation of language models, supporting robust alignment with
diverse human values and use cases. Our models, data, and code are available as
open source at https://github.com/rubricreward/r3