# Motion Capture Research: 3D Human Pose Recovery Based on RGB Video Sequences
## 2 Proposed Method
### 3D Human Pose Estimation Based on a Single Frame Image
#### 2.1.2 End-to-End Network Structure  
Using the weak-perspective camera model, we assume that the global rotation coefficient is \(\mathbf{R}\in\mathbb{R}^{3}\) (indicator of the axial angle), the translation vector is \(\vec{t}\in\mathbb{R}^{2}\), and the scaling coefficient is \(s\in\mathbb{R}\). The SMPL model contains 23 joints, hence we can get rotation matrices by pose parameters \(\vec{\theta}\), using the Rodriguez formula: \(\exp\left(\overline{\omega}_{j}\right)=I+\hat{\overline{\omega}}\sin\left( \left\|\vec{\omega}_{j}\right\|\right)+\hat{\overline{\omega}}^{2}\cos\left( \left\|\vec{\omega}_{j}\right\|\right)\), where \(\vec{\omega}_{j}\) is the skew symmetric matrix of 3-vector, \(\overline{\omega}=\frac{\vec{\omega}}{\left\|\vec{\omega}\right\|}\) and \(I\) are the \(3\times 3\) identity matrix. Therefore, combined with the shape coefficient solved by the SMPL model: \(\vec{\beta}\in\mathbb{R}^{10}\), and the relative 3D rotation coefficient: \(\vec{\theta}\in\mathbb{R}^{3*23}\) (indicator of the axial angle), we can obtain 85 parameters: \(\Theta=\{\vec{\beta},\vec{\theta},\mathbf{R},\vec{t},s\}\) that need to be solved in the 3D pose recovery. On the one hand, the human pose parameters: \(\vec{\theta}\), human shape parameters: \(\vec{\beta}\), and extra global parameters: \(\mathbf{R}\), \(\vec{t}\), s are used to calculate the GAN constraint; On the other hand, we can get the 2D/3D human pose using the output parameters, hence the 2D/3D key-point constraints can be obtained. Similar to existing research [31], the global loss function is calculated as follows:  
\[L=w_{r}L_{reproject}+1_{3d}w_{3d}L_{3d}+w_{a}L_{adversarial}+1_{height}w_{h}L_{ Height} \tag{1}\]  
where \(1_{3d}\), \(1_{height}\) are indicator functions, when the real label exists, the value is 1, otherwise the value is 0. The following are the different components of the loss function, where \(W\) represents the weight of the loss function, \(L_{reproject}\) represents the error value between the key point of the 3D human model projected onto the 2D plane (\(\hat{\mathbf{x}}\)) and the real 2D key point (\(\mathbf{x}\)):  
\[L_{reproject}=\sum_{i}\left\|v_{i}\left(\mathbf{x}_{i}-\hat{\mathbf{x}}_{i}\right) \right\|_{1} \tag{2}\]  
where \(v_{i}\in\{0,1\}\) indicates whether the key point of \(i\) is visible. The projection process of \(\hat{\mathbf{x}}\) is calculated as follows:  
\[\hat{\mathbf{x}}=s\Pi(\mathbf{R}\mathbf{X}(\vec{\beta},\vec{\theta}))+\vec{t} \tag{3}\]  
where \(\Pi\) represents an orthogonal projection, \(\mathbf{X}(\vec{\beta},\vec{\theta})\) represents the set of 3D key points calculated by SMPL model. \(L_{reproject}\) uses the projection constraints of the 2D space, so that the generated 3D human model projection basically coincides with the key points of the 2D space. The solutions of 3D models generated by the projection constraints are not unique, and some of the results may not be the real human body. So in order to avoid the diversity of solutions, the 3D constraint is calculated as follows:  
\[L_{3d}=L_{3d\text{\;joints}}+1_{smpl}L_{3d\text{\;smpl}} \tag{4}\]  
\[L_{3d\text{\;joints}}=\sum_{i}\left\|\mathbf{X}_{i}-\hat{\mathbf{X}}_{i}\right\|_{2}^ {2} \tag{5}\]  
\[L_{3d\text{\;smpl}}=\sum_{i}\left\|\left[\vec{\beta}_{i},\vec{\theta}_{i} \right]-\left[\vec{\beta}_{i},\hat{\vec{\theta}}_{i}\right]\right\|_{2}^{2}. \tag{6}\]  
Figure 2: An illustration of the SMPL model. The real 3D human parameters: \(\vec{\beta},\vec{\theta}\) can be calculated by the Mosh method [35] using real 3D skeleton points. \(1_{smpl}\) is an indicator function, when the SMPL parameter exists, the value is 1, otherwise the value is 0. Without any a priori condition constraints, the generated 3D human model may have some abnormalities, such as limb crossing, abnormal bending, and abnormal limb morphology.