Title: RLAR: An Agentic Reward System for Multi-task Reinforcement Learning on Large Language Models
Primary area: foundation or frontier models, including LLMs
Keywords: reinforcement learning, large language model, agent

Abstract: Large language model alignment via reinforcement learning depends critically on reward function quality. However, generic reward models often underperform on heterogeneous task distributions due to distribution shifts, while training task‑specific reward models is costly and prone to annotation difficulty, catastrophic forgetting, and loss of generalization.
We present RLVR (Reinforcement Learning from Agent Rewards), a unified, agent‑driven framework that dynamically assigns tailored reward functions to individual training queries. RLVR combines two automated LLM‑based stages. First, the tool generation stage where web-agents and code-agents generate rule‑, metric‑, and model‑based reward functions and wrap them as a callable tool. Then, there is a reward tool calling stage where a central decision LLM assign the reward function tools to individual queries.
Across diverse tasks including translation, summarization, question answering, and mathematics, RLVR delivers 5–10% average improvement over a widely‑used generic reward model (Skywork‑Reward‑V2) and matches GPT‑4.1‑as‑judge performance, while generalizing well to untrained benchmarks such as BenchMAX and MT‑Bench. Our code-agent successfully produces executable tools in 94.9% of cases and the web agent integrates 47.6% of retrieved repositories, enabling robust reward construction even in resource‑constrained settings.
By systematically leveraging and extending existing reward sources, RLVR offers a scalable path to high‑quality RL alignment over heterogeneous task domains.