# Towards Single 2D Image-Level Self-Supervision for 3D Human Pose and Shape Estimation
## 3 Method
### Network Architectures
#### 3.2.2 Pre-Training Stage
for\(t=1,\ldots,T_{1}+T_{2}\)do for\(n=1,\ldots,N/N^{\prime}\)do if\(t<=T_{1}\)then  For each data RGB image \(\mathbf{x}\) in the mini-batch \(D_{n}\), output \(\hat{\mathbf{o}}_{c}\), \(\hat{\mathbf{r}}_{c}\), and \(\hat{\mathbf{p}}\);  Calculate gradient \(\
abla L_{\text{jigsaw}}\), \(\
abla L_{\text{rot}}\), \(\
abla L_{\text{inp}}\) and \(\
abla L_{\text{adv}}\) (Equations (2)-(5)) with respect to (the weights of) \(f^{\text{Jigsaw}}\), \(f^{\text{Rot}}\), \(f^{\text{Imp}}\) and \(f^{\text{Disc}}\) on \(D_{n}\), and update \(f^{\text{Jigsaw}}\), \(f^{\text{Rot}}\), \(f^{\text{Imp}}\) and \(f^{\text{Disc}}\);  Calculate gradient \(\
abla L_{\text{RGB}}\) (Equation (6)) with respect to (the weights of) \(f^{\text{Tex}}\) on \(D_{n}\), and update \(f^{\text{Tex}}\);  end if if\(t>T_{1}\)then  For each data RGB image \(\mathbf{x}\) in the mini-batch \(D_{n}\), output \(\hat{\mathbf{o}}_{c}\), \(\hat{\mathbf{r}}_{c}\), \(\hat{\mathbf{p}}\), and generate \(\hat{\mathbf{x}}_{\text{render}}\);  Calculate gradient \(\
abla L\) (Equation (1)) with respect to (the weights of) \(f^{\text{Feat}}\), \(f^{\text{Mesh}}\) and \(f^{\text{Disc}}\) on \(D_{n}\),  and update \(f^{\text{Feat}}\), \(f^{\text{Mesh}}\), and \(f^{\text{Disc}}\);  end if  end for  end for
```  
**Algorithm 1**The summary of our entire training process  
**Part in-painting loss \(L_{\text{inp}}\). ** The part in-painting task could be framed as the image generation task as in [21]. We erased a patch \(\mathbf{p}\) from the center of the RGB images \(\mathbf{x}\) to make an image with a hole \(\mathbf{x}_{\text{hole}}\). The image \(\mathbf{x}_{\text{hole}}\) was inputted to the \(f^{\text{Imp}}(f^{\text{Feat}}(\mathbf{x}_{\text{hole}}))\), and the part in-painter \(f^{\text{Imp}}\) was trained to reconstruct the output \(\hat{\mathbf{p}}=f^{\text{Imp}}(f^{\text{Feat}}(\mathbf{x}_{\text{hole}}))\). We trained the network using both the mean square error (MSE) loss and the GAN-type adversarial loss by the discriminator \(f^{\text{Disc}}\) to reconstruct a realistic patch \(\hat{\mathbf{p}}\). The MSE loss was enforced to make the reconstructed patch \(\hat{\mathbf{p}}\) look similar to the original patch \(\mathbf{p}\), and GAN-type adversarial loss was enforced to make the reconstructed image \(\hat{\mathbf{x}}_{\text{inp}}\) that combined \(\mathbf{x}_{\text{hole}}\) and \(\hat{\mathbf{p}}\) look more realistic. \(f^{\text{Imp}}\) was pre-trained using the following rules:  
\[L_{\text{inp}}(f^{\text{Imp}}|\mathbf{x}_{\text{hole}})=\lambda_{\text{inp}} \|\mathbf{p}-\hat{\mathbf{p}}\|_{2}^{2}+\lambda_{\text{disc}}\|f^{\text{Disc} }(\hat{\mathbf{x}}_{\text{inp}}))-1\|_{2}^{2} \tag{4}\]  
where \(\hat{\mathbf{x}}_{\text{inp}}\) is the in-painted image that fills the hole in \(\mathbf{x}_{\text{hole}}\) with the estimated \(\hat{\mathbf{p}}\), and \(\lambda_{\text{imp}}\) and \(\lambda_{\text{disc}}\) are set at 0.999 and 0.001, respectively. **Adversarial loss \(L_{\text{adv}}\). ** The discriminator \(f^{\text{Disc}}\) was trained to discriminate original images \(\mathbf{x}\) from images \(\mathbf{\hat{x}}_{\text{inp}}\)--that is, the combination of \(\mathbf{x}_{\text{hole}}\) and \(\hat{\mathbf{p}}\)--outputted from the \(f^{\text{Imp}}\) using the LSGAN [23] objective, as follows:  
\[L_{\text{adv}}(f^{\text{Disc}}|\mathbf{x},\hat{\mathbf{x}}_{\text{inp}})=\|f^{ \text{Disc}}(\mathbf{x}))-1\|_{2}^{2}+\|f^{\text{Disc}}(\hat{\mathbf{x}}_{ \text{inp}})-0\|_{2}^{2}. \tag{5}\]RGB rendering loss \(L_{\text{RGB}}\).When projecting 3D meshes into the RGB images, RGB values for each vertex of 3D mesh were required.