# RepNet: A Lightweight Human Pose Regression Network Based on Re-Parameterization
## 3 Method
### Residual Log-Likelihood Estimation
#### 3.3.2 Flow Generation Model  
By training a generator \(G\) and sampling the input data, the depth-generating model transforms the sample \(z\) in simple distribution \(\pi(z)\) into the sample \(x\) in complex distribution \(p_{G}(x)\):  
\[p_{G}(x)=\pi(z)|\det(J_{G^{-1}})| \tag{9}\]  
In the flow-based generation model, \(p_{G}(x)\) can be calculated directly by constructing a reversible transformation function. The key to the method is how to design the target distribution to be learned, and the details of the implementation of the invertible function are beyond the scope of this paper, so we will not repeat them here. For the pose estimation task, a straightforward idea is to first learn a simple Gaussian distribution \(P_{\Theta}(z|I)\) through a regression network, and then transform it to a true distribution \(P_{\Theta,\hat{\sigma}}(x|I)\) of the data with a flow model, as shown in Figure 7a; the relationship between the two is derived from Equation (10):  
\[\log P_{\Theta,\phi}(x|I)=\log P_{\Theta}(z|I)+\log\left|\det\frac{\partial f_{ \phi}^{-1}}{\partial x}\right| \tag{10}\]  
In Equation (10), \(\Theta\) is the parameter of the regression network learning, and \(\phi\) is the learning parameter of the flow model. As long as \(f_{\phi}\) is complex enough, it can fit any distribution. For a given arbitrary \(x\), the likelihood probability can be obtained by calculating \(z\) in reverse. However, the keypoint of this method is the distribution of truth value in the data set, which does not meet the needs of this paper. Based on the above research, we can further assume that the underlying distribution of all data shares the same density function cluster, but has different variance and mean on input \(I\). The normal distribution can then be mapped to a primitive probability density function \(P_{\phi}(\overline{x})\) with a flow model, and the translation coefficient \(\hat{\mu}\) and the scaling coefficient \(\hat{\sigma}\) of the original distribution can be predicted by the regression network to obtain the final true distribution. At this point, the loss in Equation (5) can be re-expressed as:  
\[Lmle=-\log P_{\Theta,\phi}(x|I)\Big{|}_{x=\mu_{g}}=-\log P_{\phi}\left( \overline{\mu}_{g}\right)+\log\hat{\sigma} \tag{11}\]  
where \(\overline{\mu}_{g}=\left(\mu_{g}-\hat{\mu}\right)/\hat{\sigma}\). Thus, the probability distribution based on the flow model allows for end-to-end training evaluation, as shown in Figure 7b. However, from Equation (11), it can be seen that the learning of the regression model depends entirely on the results of the transformation of the flow model, and the performance of the model is reduced. To avoid this problem, the idea of residual design can be followed, i.e., an initial distribution  
Figure 7: RLE design flow. function can be set artificially, and the training is on this basis. The logarithm of distribution \(P_{\phi}(\overline{x})\) can be expressed as:  
\[\log P_{\phi}(\overline{x})=\log Q(\overline{x})+\log G_{\phi}(\overline{x}) \tag{12}\]  
In Equation (12), \(Q(\overline{x})\) is a known simple distribution, which can be set as a Gaussian distribution or a Laplace distribution probability density function, and \(G_{\phi}(\overline{x})\) is a distribution learned from the flow model, which is a residual estimate of the simple distribution to bring \(P_{\phi}(\overline{x})\) closer to the real data distribution. The design flow is shown in Figure 6(c). The residual log-likelihood estimation can be calculated by Equation (11), which is expressed as:  
\[Lrile=-\log P_{\Theta,\phi}(x|I)\big{|}_{x=\mu_{g}}=-\log Q\Big{(}\overline{ \mu}_{g}\Big{)}-\log G_{\phi}\Big{(}\overline{\mu}_{g}\Big{)}+\log\hat{\sigma} \tag{13}\]  
The RepNet network uses Equation (13) to replace the original regression loss to train the model. In the concrete implementation of the RLE module, the lightweight flow model is selected to learn the residual distribution, and the mean \(\hat{\mu}\) and deviation \(\hat{\sigma}\) of the predicted data are taken as the actual results.