Learning a Faster Locomotion Gait for a Quadruped Robot with ModelFree Deep Reinforcement Learning Quadruped robots have great agility, flexibility and stability, which enables them to walk through uneven terrain. Motion control of legged robots is always a difficult problem. Previous approaches mostly either use a predefined gait that results in clumsy and unnatural behavior, or use reinforcement learning approach to generate a gait strategy that needs longtime computation and elegant network design. In this paper, we present an effective approach that uses deep reinforcement learning with prior knowledge to optimize the gait of quadruped robot. By using a specific quadruped robot walking gait as a priori knowledge, this approach adopts the technique of distributed proximal policy optimization to optimize the search for better gait. The proposed approach does not require modelling of complex robots, and has good network convergence speed and learning effect. Simulation results demonstrate that our proposed approach converges faster than other deep reinforcement learning methods without prior knowledge. Besides, our achieved gait has higher speed that is 50% faster than the trot gait without optimization.

Hierarchical Reinforcement Learning for Quadruped Locomotion Legged locomotion is a challenging task for learning algorithms, especially when the task requires a diverse set of primitive behaviors. To solve these problems, we introduce a hierarchical framework that can automatically learn to decompose complex locomotion tasks. A highlevel policy issues commands in the form of a latent vector and also selects for how long the lowlevel policy will execute the latent command. Concurrently, the lowlevel policy uses the latent command and only the robotxe2x80x99s onboard sensors to control the robotxe2x80x99s actuators. Our approach allows the highlevel policy to run at a lower frequency than the lowlevel one. We test our framework on a pathfollowing task for a dynamic quadruped robot and we show that steering behaviors automatically emerge in the latent command space as lowlevel skills are needed for this task. We then show efficient adaptation of the trained policy to new tasks by transfer of the trained lowlevel policy. Finally, we validate the policies on a real quadruped robot. To the best of our knowledge, this is the first application of endtoend hierarchical learning to a real robotic locomotion task.