Single Image 3D Human Pose Estimation from Noisy Observations
E. Simo-Serra

A. Ramisa
G. Alenyà
C. Torras
Institut de Robòtica i Informàtica Industrial, CSIC-UPC
08028 Barcelona, Spain

PROBLEM:

F. Moreno-Noguer

Method Overview

Retrieval of a 3D Human Pose from a single image.

STATE-OF-THE ART LIMITATIONS:
Use of temporal information or background subtraction
 Unable to handle large amounts of 2D noise
CONTRIBUTIONS:
Proposal of an approach to efficiently explore the space of possible 3D
solutions given noisy 2D input
 Coarse to fine approach to constrain the solution space, until a single
solution is obtained

Exploration of Ambiguous Hypotheses

Problem Definition
GIVEN:
 2D noisy detections of the parts asa set of Gaussians ui ~ N(ui , Σu )
2D Gaussians

3D Pose

Error metric based on reprojection and learnt limb lengthsεlr = εl · εrInternal Calibration Matrix AWE WANT TO RETRIEVE:
 The 3D pose of the input image“Off-the-shelf” 2D Body Part Detector [1]ΣLength ErrorΣ√

Reprojection
Error~train-1εl = || lij – lij || σiji,j є Nnv
T -1 ~~
εr =(ui – ui) Σui (ui – ui)i

Sampling solution space using Covariance Matrix Adaptation [2] Ambiguity in orientation facing solved by proposing twohypotheses for forward and backward poses

Experimental Results
HumanEva I Dataset

Best Error

Our
Approach

Err: 72.6 mm

Err: 105.3 mm

Err: 89.5 mm

Err: 24.8 mm

Err: 46.6 mm

Err: 805.0 mm

Err: 55.3 mm

Err: 22.1 mm

Err: 102.4 mm

Err: 284.5 mm

Err: 102.4 mm

OptimalPCABestReconst.

S3, Jog, Frame #280 Err: 17.5 (13.0) Err: 26.3 mm

S1, Walk, Frame #280 Err: 9.8 (5.8)

S2, Walk, Frame #20 Err: 13.7 (5.2)

Ground Truth

Part Detection

TUD Stadmitte

Output modified to provide
local Gaussian estimations
 Provides robust 2D detections
at the cost of location noiseHighError

Low

Projective Linear Deformation Model

Potential solutions (~10,000) are clustered to reduce their
number (~300 clusters)
 Error function is not discriminative enough for a single solution


2D-3D correspondences can be written as
the rank-deficient linear system

Mx = 0

Shape = Linear combination of deformation modes:
Deformation Modes

Modal Weights

x = Qα + x0

Eigenvalues of M

One-Class Support Vector Machine is used to obtain a value
indicative of pose anthropomorphism from distance matrices
Sample #1
Sample #1

The correspondence problem becomes:
Eigenvalues of MQ

Still rank deficient, but much less.

Projective Linear Deformation Modelμα = −(MQ)† Mx0 , where M function of A, ui
α Tα Covariance: Σ =ΣuαuuMean Shape

MQα + Mx0 = 0
Hypotheses DisambiguationMean:( )Error Value (εlr)

6.883

SVM Output

2.8e-04

Reconst. Err. (mm)199.9

Aligned Err. (mm)56.4

Sample #2≈>≈<PoseAlignedSample #2
Pose

AlignedNumeric Results
Walking
S1

S2

S3

Our Approach

99.6 (42.7)

108.3 (42.3)

127.4 (24.0)

2D Input (px)14.1 (7.5)

19.1 (8.1)

26.8 (8.0)[3] (tracking)-107 (15)-[4] (tracking)

89.3

108

113.5

38.2 (21.4)

32.8 (23.1)

40.2 (23.2)[5] (background subtraction)

Jogging

6.885

S1S2S3-7.6e-03

Our Approach

109.2 (41.5)

93.1 (41.1)

115.8 (40.6)

214.9

2D Input (px)18.3 (6.3)

18.1 (6.0)

20.9 (6.1)

167.7[5] (background subtraction)

42.0 (12.9)

34.7 (16.6)

46.6 (28.9)

Robustness to noise is obtained by linear deformation model
Propagation ofsynthetic noisefrom 2D
estimations to3D pose

Our Approach (OA) outperforms using only error
functions to obtain the 3D pose (BRep, BErr, BLen)and is generally close to finding the bestreconstruction sample BRec

References[1] Y. Yang, D. Ramanan. Articulated pose estimation with flexible mixtures-of-parts. In CVPR, 2011.

[2] N. Hansen. The CMA evolution strategy: a comparing review. In Towards a new evolutionary computation, Adv. On estimation ofdistribution alg., pp 75-102. Springer, 2006.
[3] M. Andriluka, S. Roth, B. Schiele. Monocular 3d pose estimation and tracking by detection. In CVPR, 2010.
[4] B. Daubney, X. Xie. Tracking 3d human pose with large root node uncertainty. In CVPR, 2011.
[5] L. Bo, C. Sminchisescu. Twin Gaussian Processes for Structured Prediction. IJCV, 87(1-2): 28-52, 2010.