  In this paper, we are interested in optimal control problems with purely
economic costs, which often yield optimal policies having a (nearly) bang-bang
structure. We focus on policy approximations based on Model Predictive Control
(MPC) and the use of the deterministic policy gradient method to optimize the
MPC closed-loop performance in the presence of unmodelled stochasticity or
model error. When the policy has a (nearly) bang-bang structure, we observe
that the policy gradient method can struggle to produce meaningful steps in the
policy parameters. To tackle this issue, we propose a homotopy strategy based
on the interior-point method, providing a relaxation of the policy during the
learning. We investigate a specific well-known battery storage problem, and
show that the proposed method delivers a homogeneous and faster learning than a
classical policy gradient approach.
