Before revision:  While the policy update is performed with the inferred, potentially dense reward function, the original sparse reward is used to provide a self-supervisory signal for the reward update by serving as an ordering over the observed trajectories .
After revision:  While the policy update is performed with the inferred, potentially dense reward function, the original sparse reward provides a self-supervisory signal for the reward update by serving as an ordering over the observed trajectories .