Title: Adapting Rewards to the Agent Using Rational Activation Functions
Primary area: reinforcement learning
Keywords: Reward Shape

Abstract: Reinforcement learning agents vary in representation capacity and credit assignment, yet environment rewards are fixed, leading to miscalibrated gradients, instability, and wasted samples when signals exceed what an agent can effectively internalize. This paper introduces Rational Reward Shaping (RRS), a capacity-aware transformation that converts raw rewards into internal signals aligned with the agent’s learning bandwidth. By combining experience-normalized scaling with a monotone rational activation to reshape curvature and sensitivity while preserving order. The approach can adapt automatically to changing reward regimes and drops into standard actor–critic updates by simply replacing the immediate reward in the target, requiring minimal code changes and no task-specific reward engineering. Integrated into DDPG, TD3, and SAC across six MuJoCo benchmarks, RRS consistently improves average return and stability in both noiseless and perturbed-reward settings, with larger gains under noise. Altogether, RRS offers a simple, broadly applicable way to make reward feedback better calibrated to an agent’s learning dynamics, yielding stronger learning signals without modifying environment design.

TL;DR: Capacity-aware reward shaping that normalizes replay rewards and applies an adaptively tuned rational activation to improve stability.