# Human Pose Estimation Using MediaPipe Pose and Optimization Method Based on a Humanoid Model
## 4 Experimental Setup and Results
### Pose Estimation with Experiment  
For the performance validation of the proposed pose estimation method, several activities in standing, sitting, and lying poses were filmed with an RGB camera and analyzed by our system. Figure 11 shows three poses captured and analyzed while standing and squatting. Figure 11a,b show the 2D poses estimated by MPP and the reprojected poses attained by uDEAS, and Figure 11c shows the reconstructed 3D poses. Figure 10: Pose estimation results for the six representative poses generated by the humanoid model (solid line: true pose, dotted line: estimated pose, red line: right parts, blue line: left parts). The loss function needs to reflect the pose match, pose stability, pose symmetry, and penalty values for the joint angles of the torso and both shoulders as follows:  
\[L(\mathbf{v})= MJCD(\mathbf{v})+\gamma_{CoM}CoMD(\mathbf{v})+\gamma_{sym}\Big{(} \Big{|}\theta_{hp}^{I}-\theta_{kp}^{r}\Big{|}+\Big{|}\theta_{kn}^{I}-\theta_{kn }^{r}\Big{|}\Big{)}+\gamma_{sqg\_,tr}P_{sn}(\theta_{tr},-10^{\circ})\] \[+\gamma_{cor\_tr}P_{d}(\theta_{tr},-10^{\circ},10^{\circ})+\gamma _{tun\_tr}P_{d}(\psi_{tr},-15^{\circ},15^{\circ})+\gamma_{sqg\_sh}P_{sn}\Big{(} \theta_{sk}^{I},-10^{\circ}\Big{)}\] \[+\gamma_{sqg\_sh\_r}P_{sn}\big{(}\theta_{sk}^{r},-10^{\circ}\big{)} +\gamma_{cor\_sh\_r}P_{sp}\Big{(}\phi_{sk}^{I},20^{\circ}\Big{)}+\gamma_{cor\_ sh\_r}P_{sp}\Big{(}\phi_{sk}^{I},20^{\circ}\Big{)}\]  
where the weights \(\gamma_{CoM}\) and \(\gamma_{cor\_sh\_r}\) denote the effects of each term on the loss function to reflect the adequacy of the reconstructed 3D pose. Because the MPJPE was in the order of \(10^{-3}\), these weights needed to be 0.01. The threshold values of the penalty functions can be selected appropriately for a target pose and activity. Figure 11: Pose estimation results for three poses generated by the humanoid model. (**a**) Original images and MPP results overlaid, (**b**) comparison between the MPP model (solid line) and the fitted humanoid model (dotted line) in 2D plane, and (**c**) the humanoid model reconstructed in 3D plane (red line: right parts, blue line: left parts). Figure 12 shows the trajectories of the estimated joint angles for the poses from Figure 11a-c. It is noteworthy that the sagittal hip and knee joints move from 0 to around 100 degrees, and the sagittal torso angles change from 0 to 60 degrees, which match the actual human joint angles rather well. These angle trajectories also provide information on the current stance, making them helpful for medical or therapeutic purposes. For real-time application of the proposed pose optimization system, the execution time needs to be measured while running uDEAS on NUC. Table 3 lists the mean run time per frame measured while changing the number of restarts and the maximum row length of uDEAS. Interestingly, the optimal row length is found to be six when number of restarts is set at 10 because the loss of the next row length (\(7.22\times 10^{-3}\)) increases significantly. In the same manner, the optimal number of restarts is 6 when the optimal row length is 12. As a combination of these results, the loss and mean run time per frame are \(7.04\times 10^{-3}\) and 0.033 s, respectively, in the case when the number of restarts and the maximum row length are both six. Since this optimization execution time is below 100 ms, i.e., camera-capturing time for each frame at 10 fps, it is likely that real-time pose estimation is possible with the proposed system.