Superquadrics Revisited:
Learning 3D Shape Parsing beyond Cuboids
Andreas Geiger
Autonomous Vision Group
University of Tübingen / MPI for Intelligent Systems

University of Tübingen
MPI for Intelligent Systems
Autonomous Vision GroupSuperquadrics Revisited:
Learning 3D Shape Parsing beyond Cuboids[Paschalidou, Ulusoy & Geiger, CVPR 2019]3D Representations

VoxelsPointsMeshes

Implicit

Limitations of existing 3D representations:
I Large number of unstructured geometric elements
I Do not convey semantic information (parts, functionality, etc.)

3What is a useful 3D representation?

3D PrimitivesPrimitive-based 3D Representations:
I Few primitives required to represent a 3D object
I Convey semantic information (parts, functionality, etc.)
I Challenges: Variable number of primitives, few annotated datasets
5

3D Shape Abstraction

Goal of this work:
I Learn 3D shape abstraction
I Infer number of primitives
I No supervision at primitive level
I Input: points/volumes/images

6Pentland’s Superquadrics Revisited

I 1 superquadric = 11 parameters ⇒ scene on the left stored in only 1000 bytes!
I However, early ﬁtting-based approaches did not work robustlyPentland: Parts: Structured descriptions of shape. AAAI, 1986.

7

Can we learn to parse 3D shapes?

Learning 3D Shape Parsing
PoseWorld Coordinates

Observations

Scale

Predictions per primitive:
I 11 parameters: 6 pose (R, t) + 3 scale (α) + 2 shape ()I Probability of existence: γ ∈ [0, 1]
Paschalidou, Ulusoy and Geiger: Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids. CVPR, 2019.

9

Network Architecture

Paschalidou, Ulusoy and Geiger: Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids. CVPR, 2019.

10

Loss Function
Overall Loss:
L(P, X) = LP →X (P, X) + LX→P (X, P) + Lγ (P)Composed of:
I LP →X (P, X): Primitive-to-Pointcloud LossI LX→P (X, P): Pointcloud-to-Primitive Loss
I Lγ (P): Existence and Parsimony Loss

Paschalidou, Ulusoy and Geiger: Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids. CVPR, 2019.

11

Primitive-to-Pointcloud Loss

PrimitiveLm
P →X (P, X) =∆mk =PrimitiveK
1 XK∆mk

LP →X (P, X) = Ep(z) minLmP →X (P, X)

m|zm =1k=1i=1,..,NX

kTm (xi ) − ykm k2

MX=γm Lm
P →X (P, X)

m=1
Paschalidou, Ulusoy and Geiger: Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids. CVPR, 2019.

12

Pointcloud-to-Primitive Loss

PrimitivePrimitive
LiX→P (X, P) = min ∆mim|zm =1

LX→P (X, P) = Ep(z) 

LiX→P (X, P)

xi ∈Xm∆mi = min kTm (xi ) − yk k2
k=1,..,KX

MX X=xi ∈X m=1
Paschalidou, Ulusoy and Geiger: Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids. CVPR, 2019.

∆mi γmm−1Y(1 − γm̄ )m̄=113

Existence and Parsimony Loss

Lγ (P) = max 1 −MX!
γm , 0vu M
uX+ βtγmm=1m=1I First term: Enforces at least one primitive to exist
I Second term: Encourages parsimony

Paschalidou, Ulusoy and Geiger: Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids. CVPR, 2019.

14

Tulsiani et al.
Ours

10−1

Tulsiani et al.
Ours

1.5
1.0

10−20.5
10−3Time per iteration (s)Relative variance of gradientComparison to Tulsiani et al./ REINFORCE

0.0
12

3
4
5
6
Number of samples

7Paschalidou, Ulusoy and Geiger: Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids. CVPR, 2019.

815

Results

Paschalidou, Ulusoy and Geiger: Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids. CVPR, 2019.

16

Summary17

Thank you!
http://autonomousvision.github.io