# Model Predictive Control of Quadruped Robot Based on Reinforcement Learning
## 1 Introduction  
The quadrupeds are outstanding in high-speed running, weight-bearing, and terrain passing. For example, cheetahs' sprints have been measured at a maximum of 114 km/h, and they routinely reach velocities of 80-100 km/h while pursuing prey. Goliath frogs can jump up to 5 m high while their body length is only from 17 to 32 cm. Blue sheep are usually found near cliffs, in preparation to run toward rugged slopes to avoid danger. Camels are known as the \"ships of the desert\", they are able to carry hundreds of kilos of cargo for long trips in harsh desert environments. These striking features have inspired researchers' enthusiasm for the bionic quadruped robots. It is hoped that one day such quadruped robots can surpass animals in movement skills and perform tasks in challenging environments. These visions put forward higher requirements for the performance of robot controllers. Half a century has witnessed the development of legged robots, and many excellent quadruped robots have emerged. The issue of controller design has shifted from static position planning [1] to highly dynamic optimization [2; 3]. Among numerous optimization algorithms, Model Predictive Control (MPC) has emerged as the most widely used control algorithm in the robot field. There are three main elements of MPC [4]: the predictive model, the reference trajectory, and the control algorithm. This is now more clearly stated as model-based prediction, receding horizontal optimization, and feedback correction. The vast literature invariably says that the greatest attraction of predictive control is its ability to deal explicitly with control and state quantity constraints, this ability arises from the predictions of future dynamic behavior based on the analytical model of the control object, by adding the constraints to the future inputs, outputs, and state. The receding horizontal optimization ensures that the system can quickly respond to the uncertainty from the internal system or external environment. Since MIT Biomimetic Robotics Lab has open-sourced Cheetah-Software [5], MPC has become a new baseline method for the locomotion control of quadruped robots. [PERSON] proposed the policy-regularized MPC [6] to stabilize different gaits with several heuristic reference policies. [PERSON] presented the representation-free MPC [7] framework that directly represents orientation using the rotation matrix and can stabilize dynamic motions that involve the singularity. [PERSON] proposed a design approach of gait parameters with minimum energy consumption [8]. [PERSON] presented the Proportional Differential MPC (PDMPC) controller [9] that has the ability to compensate for the unmodeled leg mass or payload. Although MPC has made significant progress in the field of legged-robot control, it still faces many challenges. The precise predictive model is difficult to establish; the simplified model will introduce model mismatch, and the fixed parameters controller does not have strong generalization ability. In recent years, with the revival of artificial intelligence, achieving autonomous learning control for robots has become one of the research highlights. [PERSON] [10] learned parameters through policy search based on fixed strategies and realized the action of jumping from the ground to obstacles on Boston Dynamics' little dag. [PERSON] [11] developed a variant of the soft actor-critic algorithm to realize the level walking of a quadruped robot with only 8 Degree-of-Freedom (DOF). [PERSON] proposed a video imitation adaptation network [12] that can imitate the action of animals and adapt it to the robot from a few seconds of video. [PERSON] applied model-free reinforcement learning on ANYmal, realized omnidirectional motion on flat ground, and fall recovery. The trained controller can follow the command speed well in any direction. In order to enable the simulation results to be transferred to the real robots, domain randomization, actuator modeling, and random noise during training were used [13].