Model-free deep reinforcement learning algorithms have been shown to be capable of learning a wide range of robotic skills, but typically require a very large number of samples to achieve good performance. Model-based algorithms, in principle, can provide for much more efficient learning, but have proven difficult to extend to expressive, high-capacity models such as deep neural networks. In this work, we demonstrate that neural network dynamics models can in fact be combined with model predictive control (MPC) to achieve excellent sample complexity in a model-based reinforcement learning algorithm, producing stable and plausible gaits that accomplish various complex locomotion tasks. We further propose using deep neural network dynamics models to initialize a model-free learner, in order to combine the sample efficiency of model-based approaches with the high taskspecific performance of model-free methods. We empirically demonstrate on MuJoCo locomotion tasks that our pure modelbased approach trained on just random action data can follow arbitrary trajectories with excellent sample efficiency, and that our hybrid algorithm can accelerate model-free learning on high-speed benchmark tasks, achieving sample efficiency gains of 3 − 5× on swimmer, cheetah, hopper, and ant agents. Videos can be found at https://sites.google.com/view/mbmf
With this design, which combines model-predictive control and reinforcement learning of a neural network, Fugu belongs to a family of approaches known as model-based reinforcement learning REF .
Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning
2018 IEEE International Conference on Robotics and Automation (ICRA)
2018 IEEE International Conference on Robotics and Automation (ICRA)
Mathematics Engineering Computer Science