Sensitivity-optimized Rigging
for Example-based Real-time Clothing Synthesis	
(*=joint	  1st	  authors)	

Weiwei	  Xu*	  Nobuyuki	  Umentani*	  Qianwen	  Chao	  Jie	  Mao	  Xiaogang	  Jin	  Xin	  Tong	Motivation• Fast	  clothing	  synthesis	  for	  interacEve	  applicaEon	  [Winning	  Eleven	  2014]	

[Assassins	  Creed]	

Key Assumption: Pose-to-Clothing Mapping	Eght	

our	  interest	loose	Example-based Clothing Synthesis
Input: body pose
(i.e. joint rotations)

Database	  Output: clothing(i.e. vertex positions)CPU Implementation 60FPS	CPU+GPU 200 Characters 20FPS	

Related Works: Example-based Clothing	[de	  Aguiar	  et	  al.	  2010]	

[Peng	  et	  al.	  2010]	

[Wang	  et	  al.	  2013]	

[Kim	  et	  al.	  2008]	

[Wang	  et	  al.	  2010]	

[Feng	  et	  al.	  2010]	

[Kavan	  et	  al.	  2010]	

[Hahn	  et	  al.	  2014]	

State of the Art (quality vs. speed)	
Quality	
Detailed	  wrinkle	

Regression	  +	  RegularizaEon	

MulE-­‐resoluEon	

MulE-­‐character	  	  detailed	  wrinkle	
Our	  approach	Regression	Speed	
Real-­‐Eme	  MulE-­‐character	Our Approach: Rigged Clothing	We	  use	  Linear	  Blend	  Skinning	  for	  clothing	Bone	  weight	
Deformed	  vertex	  posiEon	
IniEal	  vrigid	  ertex	  transformaEon	
posiEon	
Body	  rigging	x = ∑ w b (Rb x 0 + Tb )b

Bone	  rotaEon	
cloth	  rigging	

€Bone	  translaEon	

y = ∑ w b (Rb y 0 + Tb )bChallenges for Rigged Clothing	• Clothing	  move	  diﬀerently	  from	  body	  forming	  wrinkle	  Dividing Clothing Into Parts	
•Global	  joint-­‐clothing	  inﬂuence	•Locality	  of	  the	  clothing	  wrinkle	
raise	  arm	parEEon	

Multi-prediction approach	• Clothing	  deformaEon	  is	  very	  nonlinear	right	  arm	  up	led	  arm	  up	

both	  arms	  up	+ ≠Rigging	Rigging	Rigging	Our Approach: Rigged Clothing	We	  use	  Linear	  Blend	  Skinning	  for	  clothing	Body	  rigging	x = ∑ w b (Rb x 0 + Tb )bGlobal	  inﬂuence	
cloth	  rigging	€Near	  example	

y = ∑ w b (Rb y e + Tb )b

Procedure: Rigging & Blending
Input pose

Output cloth

ExamplesNearestexamplesRigging

Blend

Procedure: Skinning & Blending
Input pose

Output cloth

ExamplesNearestexamplesRigging

Blend

Database: Example Poses & Clothing	Procedure: Rigging & Blending
Input pose

Output cloth

ExamplesNearestexamplesRigging

BlendProcedure: Skinning & Blending
Input pose

Output cloth

ExamplesNearestexamplesRigging

Blend

Procedure: Skinning & Blending
Input pose

Output cloth

ExamplesNearestexamplesRigging

Blend

Contributions
Examples’	  rigging	  model	  	  SensiEvity-­‐opEmized	  rigging	  Find	  nearest	  examples	  to	  	  SensiEvity	  distance	  measure	  Determine	  example	  poses	  MCMC	  approach	  Which is near?

Procedure: Skinning & Blending
Input pose

Output cloth

ExamplesNearest
examplesRiggingBlend1st order Prediction of Clothing Deformation	

derivaEve	d∂=∂SensiEvity	
1st	  order	  esEmaEon	

dSensitivity for Pose change	
3D	  cloth	  shape	  •Good	  representaEon	  around	  example	  J	  •Memory	  intensive	  L	  •Linear	  approximaEon	  L	  Pose	  Skinning Weight Optimization	
Linear	  Blend	  Skinning	y' = ∑ w b (Rb y + Tb )b

opEmize	€argminwbrigged	∂∂

simulated	∂∂Sensitivity for Pose Change	
• OpEmize	  weight	  to	  best	  approximate	  sensiEvity	  3D	  cloth	  shape	  Pose	  Optimized Clothing Rigging Weight

Body	  weight	  Naïve	  clothing	  weight	  OpEmized	  clothing	  weight	  Comparison against Naïve WeightProcedure: Skinning & Blending
Input pose

Output cloth

ExamplesNearestexamplesRigging

BlendSelecting & Blending with Distance Measure• K-­‐Nearest	  neighbor	  interpolaEon	
Pose	  Space	

example	input	
1/Distance	

1/Distance	

Distance	
Distance	
Distance	
example	example	Distance Measure	• Compare	  body	  shape	  or	  joint	  angles	  diﬀerence	
Input	

Distance=	Input	  	  pose	

example	  	  pose	

example	Distance	
Distance	

example	Distance	

example	Drawback of the Typical Approach	• Cannot	  tell	  amount	  of	  inﬂuence	  of	  joint	  to	  cloth	diﬀerent	

similar	90	  degree	  	  Knee	  bending	90	  degree	  	  shoulder	  bending	Same	  distance	Distance Measure: Our approach• Compare	  clothing	  shape	  diﬀerence	Distance=	cloth	  on	  input	  	  pose	cloth	  on	  example	  	  pose	

Distance	
Distance	
Distance	

example	Sensitivity-based Distance	• Diﬀerence	  between	  sensiEvity	  predicEons	  cloth	  shape	  Our	  distance	Our	  distance	
input	
Typical	  distance	

Typical	  distance	

Pose	  Comparison with Naive Approaches

Procedure: Skinning & Blending
Input pose

Output cloth

ExamplesNearestexamplesSkinningBlendIncremental Database Construction
Maximum	  Residual	
Residual	Pose	  Space	New	  example	
Example	  Pose	

1.	  Find	  the	  maximum	  residual	  pose	  (MCMC	  method)	  2.	  Solve	  cloth	  deformaEon	  at	  the	  pose	  3.	  Add	  the	  deformaEon	  to	  database	  4.	  Goto	  1.	  More Results	Manipulating 200 characters using GPUs	

Kinect “try-on”	Comparison with Physics Simulation	

Limitations	• Secondary	  moEon	  • Hysteresis	  • Database	  accuracy	  guarantee	  Acknowledgement	• We	  would	  like	  to	  thank:	  – Anonymous	  Reviewer	  – CMU	  MoCap	  data	  – Ryan	  Schmidt	  and	  Michel	  Tao	  	  • Funded	  by:	  – NSFC	  61272392,	  61322204,	  61272298,	  61328204	  – State	  Key	  Lab	  of	  CAD&CG,	  A1307	  – NaEonal	  High-­‐tech	  R&D	  Program	  2012AA011503	  Thanks for Your Attention!	
• Summary:	
Fast	  example-­‐based	  clothing	  synthesis	  using	  rigging	  approach	  Database Size	Fast Cloth-Body Intersection Resolving	1.	  Blend	  deformed	  example	  2.	  Project	  blended	  deformaEon	  for	  each	  example	  3.	  Blend	  the	  deformaEon	  again	  Result of Intersection Resolution	
Without	  ResoluEon	  	With	  ResoluEon