Are Deep Policy Gradient Algorithms Truly Policy Gradient Algorithms?
Policy Gradient Methods for Reinforcement Learning with Function Approximation