# Model Predictive Control of Quadruped Robot Based on Reinforcement Learning
## 2 PDMPC Formulation  
Our quadruped robot is Unitree A1, as shown in Figure 1. It has four elbow-like legs, and each leg has three degrees of freedom, called hip side, hip front and knee respectively according to the order of attachment. The first two joints are directly driven, and the knee joint is driven by a connecting rod. This design concentrates the majority of quality on the body. In order to reduce the computational consumption and the difficulty of optimization solution, the dynamic model of the quadruped robot used as a predictive model is simplified to a single rigid body model with four massless variable-length rods:  
\[\ddot{\mathbf{p}}=\frac{\sum_{i=1}^{k}\mathbf{f}_{i}}{m}+\mathbf{g} \tag{1}\]  
\[\frac{d}{dt}(\mathbf{I}\mathbf{\omega})=\mathbf{I}\dot{\mathbf{\omega}}+\mathbf{\omega}\times\mathbf{I} \mathbf{\omega}=\sum_{i=1}^{k}(\mathbf{r}_{i}\times\mathbf{f}_{i}) \tag{2}\]  
where \(m\) and \(\mathbf{I}\) are the mass and body inertia matrix, \(\mathbf{p}\) and \(\mathbf{\omega}\) are the positions and angular velocity of the body, \(\mathbf{g}\) is the gravitational acceleration vector, \(\mathbf{f}_{i}\) and \(\mathbf{r}_{i}\) are the foot reaction force and the position vector from the foot end to the center of mass (CoM) in the world coordinate system. \(\mathbf{\omega}\times\mathbf{I}\mathbf{\omega}\) is neglected under the assumption that small angular velocity during the robot locomotion. The control framework of MPC is shown in Figure 2. The estimated states of the quadruped robot are 6-DOF pose and corresponding 6-DOF velocity, the inputs are gait pattern, desired speed, and attitude angle, the reference trajectory generator plans a desired path within the prediction horizon based on the user inputs, and the current state, the swing planner schedules the legs' phase and the foot trajectory, the MPC controller outputs the desired ground reaction force \(\mathbf{f}_{i}^{0}\), at last, the low-level joint torque/position controller executes the control commands to drive the robot. \[\mathbf{\tau}_{i}=\mathbf{J}_{i}^{T}\mathbf{R}^{T}\mathbf{\dot{\rho}}_{i}+\mathbf{k}_{p}\Big{(}\bm {q}^{des}-\mathbf{q}\Big{)}+\mathbf{k}_{d}\Big{(}\dot{\mathbf{q}}^{des}-\dot{\mathbf{q}}\Big{)} \tag{3}\]  
where \(\mathbf{\tau}\) is the joint torque vector, \(\mathbf{R}\) is the rotation matrix of body, the leg jacobian matrix \(\mathbf{J}\) maps the force in operation space to the torque in joint space. \(q^{des}\) and \(\dot{\mathbf{q}}^{des}\) are the desired trajectory of joint, \(\mathbf{k}_{p}\) and \(\mathbf{k}_{d}\) are the gain of the PD controller. Due to the above simplification, model mismatch will inevitably occur, especially when the leg mass is large or the dynamic parameter estimation is inaccurate, and the  
Figure 1: Unitree A1 robot. Figure 2: Control framework. conventional MPC is no longer effective to the uncertainty. Therefore, PDMPC is designed to solve this problem. The structure of the PDMPC controller is shown in Figure 3.  
Based on the above-simplified model in Equations (1) and (2), the MPC controller obtains the expected force \(\mathbf{f}^{\prime}\) of the stance leg through linear optimization. As well, our PD compensator is proposed to compensate for the model uncertainty. We divide the compensator into two parts based on force and torque. \[\mathbf{F}_{e}=\mathbf{K}_{f}(\mathbf{p}_{des}-\mathbf{p})+\mathbf{D}_{f}(\dot{\mathbf{p}}_{des}-\dot{ \mathbf{p}}) \tag{4}\]  
\[\mathbf{\tau}_{e}=\mathbf{K}_{\tau}(\mathbf{\Theta}_{des}-\mathbf{\Theta})+\mathbf{D}_{\tau}(\mathbf{ \omega}_{des}-\mathbf{\omega}) \tag{5}\]  
where \(\mathbf{\Theta}\) is the Euler angle, \(\mathbf{F}_{v}\) and \(\mathbf{\tau}_{v}\) are the additional virtual force and torque acting on the body. \(\mathbf{K}\) and \(\mathbf{D}\) are the diagonal gain matrix with corresponding dimensions. \[\mathbf{K}_{f}= \begin{bmatrix}k_{fx}&&\\ &k_{fy}&\\ &&k_{fz}\end{bmatrix} \qquad\mathbf{D}_{f}= \begin{bmatrix}d_{fx}&&\\ &d_{fy}&\\ &&d_{fz}\end{bmatrix}  \tag{6}\]  
\[\mathbf{K}_{\tau}=