Before revision:  The recent remarkable progress of DRL stands on regularization of policy . By constraining the update of policy, DRL allows the policy to improve stably and efficiently. Among them, a popular method, named proximal policy optimization (PPO), has been introduced  . PPO clips density ratio of the latest and baseline policies with a threshold, while its minimization target is unclear.
After revision:  The recent remarkable progress of DRL stands on regularization of policy for stable and efficient learning. A popular method, named proximal policy optimization (PPO), has been introduced . PPO clips density ratio of the latest and baseline policies with a threshold, while its minimization target is unclear.