# Robust Quadrotor Control through Reinforcement Learning with Disturbance Compensation
## 2 Preliminary Information
### Quadrotor Dynamic Model  
In this paper, we assume that a quadrotor is a rigid and symmetrical body whose center of gravity coincides with its geometric center.  
The vector \(\mathbf{x}=[x,y,z]^{\mathrm{T}}\) denotes the position of the quadrotor in an inertial frame. The translation dynamics of the quadrotor can be expressed as follows:  
\[\dot{\mathbf{x}} =\mathbf{v} \tag{1}\] \[\dot{\mathbf{v}} =m^{-1}(\sum_{i=1}^{4}T_{i}\mathbf{b_{z}}+\mathbf{F}_{ext})-g\dot {\mathbf{i_{z}}},\]where \(m\) and \(g\) are the mass of the quadrotor and the acceleration due to gravity, respectively; \(\mathbf{R}=[\mathbf{b_{x}}\ \mathbf{b_{y}}\ \mathbf{b_{z}}]\in SO(3)\) is the rotation matrix, which is used to transform a coordinate from the body-fixed reference frame to the inertial reference frame; and \(T_{i}\) is the thrust generated from motors and applied on the \(z\)-axis of the body frame \(\mathbf{b_{z}}\). Figure 1 displays the order of motors placement. Finally, the vector \(\mathbf{F}_{ext}\) represents the external disturbance force accounting for all other forces acting on the quadrotor.  
For the rotation dynamics of system, we use a quaternion representation of quadrotor attitude to avoid gimbal lock and ensure better computational efficiency.  
\[\begin{split}\dot{\mathbf{q}}&=0.5\cdot\mathbf{q }\otimes\begin{bmatrix}0\\ \mathbf{\Omega}\end{bmatrix}^{T}\\ \dot{\mathbf{\Omega}}&=J^{-1}(\mu-\mathbf{\Omega}\times J \mathbf{\Omega}),\end{split} \tag{2}\]  
where \(\dot{\mathbf{q}}=[q_{w},q_{x},q_{y},q_{z}]^{T}\) is the normed quaternion attitude vector, \(\otimes\) is the quaternion multiplication. \(\Omega\) is the angular velocity of body-frame, \(\mu\) is the control moment vector and \(J\) is the matrix of vehicle moment of inertia tensor. The rotation transformation between the quaternion \(q\) to the rotation matrix \(\mathbf{R}\) can be expressed as follows:  
\[\mathbf{R}=\begin{bmatrix}1-2(q_{y}^{2}+q_{z}^{2})&2(q_{x}q_{y}-q_{z}q_{w})&2 (q_{x}q_{z}+q_{y}q_{w})\\ 2(q_{x}q_{y}+q_{z}q_{w})&1-2(q_{x}^{2}+q_{x}^{2})&2(q_{y}q_{z}-q_{x}q_{w})\\ 2(q_{x}q_{z}-q_{y}q_{w})&2(q_{y}q_{z}+q_{x}q_{w})&1-2(q_{x}^{2}+q_{y}^{2}) \end{bmatrix} \tag{3}\]  
Each thrust from the propeller axis is assumed to be aligned perfectly with the \(z\)-axis. The force \(T_{i}\) and motor moment \(\mu\) produced at a motor spinning speed of \(\omega_{i}\) can be expressed as follows:  
\[\begin{split} T_{i}&=c_{f}{\omega_{i}}^{2}\\ \mu&=\frac{1}{\sqrt{2}}\begin{bmatrix}-lc_{f}&lc_{f}&lc_{f}&-lc_{f}\\ lc_{f}&-lc_{f}&lc_{f}&-lc_{f}\\ c_{d}&c_{d}&-c_{d}&-c_{d}\end{bmatrix}\begin{bmatrix}\omega_{1}\\ \omega_{2}\\ \omega_{3}\\ \omega_{4}\end{bmatrix}^{2}\end{split} \tag{4}\]  
where \(i=1,2,3,4\). \(\Omega_{i}\) is the speed of motors; \(l\) is the arm length of the quadrotor; and \(c_{f}\), \(c_{d}\) are the coefficients of the generated force and \(z\)-axis moment, respectively.  
The developed dynamic model is based on the following assumptions: (a) the quadrotor structure is rigid, (b) the center of mass of the quadrotor and the rotor thrusts are in the same plane, and (c) blade flapping and aerodynamics can be ignored.