Figure 1: A schematic illustration of our modular framework. We tackle the ambiguity in the 2D-to-3D mapping by training a CVAE to generate 3D-pose samples conditioned on the 2D-pose, that are scored and weighted-averaged using joint-ordinal relations, which are regressed together with the 2D-pose. Our upper-bound performance is obtained by using an Oracle. Monocular 3D human-pose estimation from static images is a challenging problem, due to the curse of dimensionality and the ill-posed nature of lifting 2D-to-3D. In this paper, we propose a Deep Conditional Variational Autoencoder based model that synthesizes diverse anatomically plausible 3D-pose samples conditioned on the estimated 2D-pose. We show that CVAE-based 3D-pose sample set is consistent with the 2D-pose and helps tackling the inherent ambiguity in 2D-to-3D lifting. We propose two strategies for obtaining the final 3D pose-(a) depthordering/ordinal relations to score and weight-average the candidate 3D-poses, referred to as OrdinalScore, and (b) with supervision from an Oracle. We report close to stateof-the-art results on two benchmark datasets using OrdinalScore, and state-of-the-art results using the Oracle. We also show that our pipeline yields competitive results without paired image-to-3D annotations. The training and evaluation code is available at https://github.com/ ssfootball04/generative_pose.
Such constraints were also used by Sharma et al. REF to score and refine candidate 3D poses.
Monocular 3D Human Pose Estimation by Generation and Ordinal Ranking
2019 IEEE/CVF International Conference on Computer Vision (ICCV)
2019 IEEE/CVF International Conference on Computer Vision (ICCV)
Computer Science