  Deep reinforcement learning has emerged as a popular and powerful way to
develop locomotion controllers for quadruped robots. Common approaches have
largely focused on learning actions directly in joint space, or learning to
modify and offset foot positions produced by trajectory generators. Both
approaches typically require careful reward shaping and training for millions
of time steps, and with trajectory generators introduce human bias into the
resulting control policies. In this paper, we instead explore learning foot
positions in Cartesian space, which we track with impedance control, for a task
of running as fast as possible subject to environmental disturbances. Compared
with other action spaces, we observe less needed reward shaping, much improved
sample efficiency, the emergence of natural gaits such as galloping and
bounding, and ease of sim-to-sim transfer. Policies can be learned in only a
few million time steps, even for challenging tasks of running over rough
terrain with loads of over 100% of the nominal quadruped mass. Training occurs
in PyBullet, and we perform a sim-to-sim transfer to Gazebo, where our
quadruped is able to run at over 4 m/s without a load, and 3.5 m/s with a 10 kg
load, which is over 83% of the nominal quadruped mass. Video results can be
found at https://youtu.be/roE1vxpEWfw.
