Trajectory optimization with learned dynamics models can often suffer from erroneous predictions of out-of-distribution trajectories. We propose to regularize trajectory optimization by means of a denoising autoencoder that is trained on the same trajectories as the dynamics model. We visually demonstrate the effectiveness of the regularization in gradient-based trajectory optimization for open-loop control of an industrial process. We compare with recent model-based reinforcement learning algorithms on a set of popular motor control tasks to demonstrate that the denoising regularization enables state-of-the-art sample-efficiency. We demonstrate the efficacy of the proposed method in regularizing both gradientbased and gradient-free trajectory optimization.
Boney et al. REF proposed to use denoising autoencoders to prevent trajectory optimization from exploiting modelling inaccuracies and demonstrated sample-efficient learning from very few episodes, using gradient-based trajectory optimization.
Regularizing Trajectory Optimization with Denoising Autoencoders
ArXiv
ArXiv
Computer Science Mathematics