# Human Pose Estimation Using MediaPipe Pose and Optimization Method Based on a Humanoid Model
## 3 Proposed Pose Estimation Algorithm  
### Pose Estimation Process  
Figure 5 shows the overall flow diagram of the proposed pose estimation system. * Step 1. Calibration of link length: Our system checks whether the human subject is a new user or not because the subject's bone length information is basically necessary for the model-based pose estimation. If the present system has no link length data for the current subject, the link length measurement process begins; the subject stands with the arms stretched down, images are captured for at least 10 frames, and the length of each bone link is calculated as the average distance between the coordinates of the end joints of the bone at each frame. Figure 5: Flow diagram of the proposed pose estimation process. * Step 2. Acquire images from an RGB camera with an image grabber module of SBC. Although an Intel RealSense camera is used in the present system, commercial RGB webcams are also available. * Step 3. Execute MPP and obtain 2D pixel coordinates of the 17 landmarks for the captured human body. * Step 4. Execute uDEAS to seek for unknown pose-relevant variables, such as the camera's distance factor and viewing angles, and the intrabody joint angles by reducing the loss function formulated with the L2 norm between the joint coordinates obtained with MPP and those reprojected onto the corresponding 2D plane. * Step 5. Plot the estimated poses in 2D or 3D depending on the application field. * Step 6. If the current image frame is the last one or a termination condition is met, stop the pose estimation process. Otherwise, go to Step 2. For an estimation of arbitrary poses at each frame, the size factor, \(\gamma\), and the three body angle values related to the camera view angle, such as \(\theta_{bd}\), \(\phi_{bd}\), and \(\psi_{bd}\) mentioned in Section 2.3, are added to the list of optimization variables. Therefore, a complete optimization vector for pose estimation consisting of 19 variables is estimated as follows:  
\[\mathbf{V}=\left[\gamma,\,\theta_{bd},\,\phi_{bd},\,\psi_{bd},\,\theta_{tr}, \phi_{tr},\psi_{tr},\theta_{hp}^{l},\theta_{kt}^{l},\theta_{hp}^{r},\theta_{ kn}^{r},\theta_{sh}^{l},\theta_{el}^{l},\theta_{sh}^{r},\theta_{ct}^{l}, \phi_{tp}^{l},\phi_{sh}^{r},\phi_{sh}^{l}\right]^{T} \tag{6}\]  
The loss function to be minimized by uDEAS is designed to reflect three features: mean per joint position error (MPJPE) between the 2D MPP skeleton model and the fitted humanoid model; deviation of the 2D coordinates of the center of mass (CoM) from the center of the supporting feet of the 3D humanoid's pose; and penalty values concerning the joint angles of the humanoid pose for consistency with the natural human pose. The MPJPE is calculated by the mean distance between the true coordinates (for simulation case) or the MPP pixel coordinates (for experimental case), \(\left(x_{p}^{i,j},\,y_{p}^{i,j}\right)\) and the coordinates of the 2D reprojected humanoid model onto the camera-based frame in Figure 4, \(\left(y_{c}^{i,j},\,z_{c}^{i,j}\right)\), which is described as  
\[MPJPE(\mathbf{v})=\frac{\sum\left\|\left(x_{p}^{i,j},\,y_{p}^{i,j}\right)- \,\left(y_{c}^{i,j},\,z_{c}^{i,j}\right)\right\|_{2}}{12},\,\,i=l,r,\,j=sh,el, wr,hp,kn,an \tag{7}\]  
When the two poses overlap exactly, this value decreases to zero. Figure 6 shows three models: the MPP pixel model, the initial humanoid model, and the tuned model, which pelvis centers are moved to the origin of the coordinate frame. uDEAS simultaneously adjusts the model contraction factor \(\gamma\) and the body and joint angles in Equation (7) to minimize the average deviation between the MPP pixel model and the contracted humanoid model in terms of the distance between the joint coordinates in the 2D plane, which is illustrated with red arrows. The humanoid's CoM deviation (CoMD) measured on the floor as the second pose metric is significant for measuring the standing stability of the current pose generated by uDEAS.