OptiMo:
Optimization-Guided Motion Editingfor Keyframe Character Animation
Yuki Koyama & Masataka Goto

Optimization Techniques:- Powerful- Used in computer science and engineeringhttps://en.wikipedia.org/wiki/Mathematical_optimization

How can animators utilize optimization techniques
for motion editing eﬀectively?

We propose“Optimization-Guided Motion Editing”

Background: How Do Animators
Create Motions?

How Do Animators Create Motions?

1. Keyframing

2. Curve Editing

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 20187

How Do Animators Create Motions?
1. Keyframing

2. Curve Editing…
frame = 4frame = 14

Time

Define key poses atseveral keyframes

Adjust interpolation curves(how key poses are interpolated
between keyframes)

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 20188

How Do Animators Create Motions?
1. Keyframing

2. Curve Editing…
frame = 4

OurTarget
frame = 14

Define key poses atseveral keyframes

Adjust interpolation curves(how key poses are interpolatedbetween keyframes)

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 20189

Curve Editing is Important for Nuances

Even with the same
keyframing, motions
could be very diﬀerent
in nuances

frame = 4frame = 14

frame = 24

frame = 34

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201810

Difficulties in Curve EditingDifficulties in Curve Editing
Unintuitive (indirect)
eﬀectsNeed to play motions
every time manipulating
handles
Many parameters
High-dimensional
search task
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201812

Difficulties in Curve Editing
Unintuitive (indirect)
eﬀectsNeed to play motions
every time manipulating
handles
Many parameters
High-dimensional
search task
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201813

A Possible (?) Solution:Naïve Optimization

Optimization for Full Automation?

Optimization can fully automate the task,
but results are not always satisfactory (no control…)
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201815

Optimization-Guided Motion Editing
Live Demo!

Design Goals:
What Should be Considered for
Effective Interaction?

Three Design Goals

Original
motion?

OptimalmotionBlackbox

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201818

Three Design Goals

Originalmotion

Optimalmotion1. Transparent

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201819

Three Design Goals

Original
motionOptimalmotion1. Transparent

Feature: Optimization Process Visualization (for Transparency)
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201820

Three Design Goals

Original
motionOptimalmotion1. Transparent

Feature: Cost Visualization (for Transparency)
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201821

Three Design Goals

Original
motionOptimalmotion1. Transparent

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201822

Three Design Goals
2. Controllable

Original
motionOptimalmotion1. Transparent
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201823

Three Design Goals
2. Controllable

Original
motionOptimalmotion

1. Transparent

Feature: Interactive Regularization (for Controllability)
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201824

Three Design Goals
2. Controllable

Original
motionOptimalmotion1. Transparent

Feature: Time-Varying Cost Control (for Controllability)
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201825

Three Design Goals
2. Controllable

Original
motionOptimalmotion1. Transparent
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201826

Three Design Goals
2. Controllable
3. Editable
Original
motion…

Optimalmotion1. Transparent

Manual revision
allowedYuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201827

Three Design Goals
Control handles

2. Controllable
Opt.

3. Editable
Preserved

Original
motionOptimalmotion1. Transparent…

Manual revision
allowedFeature: Preservation of Control Handles (for Editability)
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201828

Three Design Goals
Control handles
Diﬀerent arrangement

2. Controllable
Opt.

3. Editable
Preserved

Original
motionOptimalmotion…

Diﬀerent data structure
Manual revision

1. Transparent

allowedFeature: Preservation of Control Handles (for Editability)
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201829

Three Design Goals
2. Controllable
3. Editable
Originalmotion…

Optimalmotion1. Transparent

Manual revision
allowedYuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201830

Math: How to Formulate
Optimization with Controllability

Optimization with Controllability
Typical minx
optimization⇢ZC(x, t)dt

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201832

Optimization with Controllability
Typical minx
optimization⇢ZControllable minx
optimization⇢ZC(x, t)dtwcost(t)C(x, t)dt + w

Control bytime-varying cost weightregD(x, xinit)

Control byregularization weight

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201833

Optimization with Controllability
Typical minx
optimization⇢Z

Controllable minx
optimization⇢ZC(x, t)dtw

cost

Adjusted in real time(t)C(x, t)dt + w

Control bytime-varying cost weightregD(x, xinit)

Control byregularization weight

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201834

Example Usage Scenarios

Example: Fox Tail•#parameters: 27•

Rig: Forward kinematics (FK)Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201836

Example: Fox Tail

Initial motion- Too robotic…

Let’s apply the (naïve)optimization firstExample: Fox TailNaïve optimization- Too much changed!

Let’s control it byregularization

Example: Fox Tail

Optimization
with regularization- Looks good!!
Inspiration:
How about makingthe swing speedier?

Example: Fox Tail

Optimization
with regularization& time-varying cost

Time-varying
cost weight

Validation: Interview withProfessional Animators

Informal InterviewGoals•

Validation of our approach (design goals, interactions, etc.)

Participants•

2 professional animators (A1 & A2) who are familiar with 3D
character animation authoring

Procedure•

Explain our design goals, explain our system features, andask feedback comments (approx. 1 hour)
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201842

ResultsEditability•

Both A1 and A2 loved editability, which was described“indispensable” (A1)

Transparency•

We observed that both A1 and A2 easily understood the concept ofoptimization by just seeing the visualizations

Controllability•

All features on controllability were strongly appreciated•

The interactivity makes the adjustment “easier” and “lessstressful” (A1)
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201843

Discussions

Related Work: Motion Editing TechniquesTangibles

Sketches

Principles

Optimization[Glauser+16][Choi+16][Kazi+16]

Ours

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201845

Related Work: Design Interface with Optimization
Photo color
adjustment, etc.

Paperairplanes

Graphic
layoutCharactermotions[Koyama+14][Umetani+14][O’Donovan+15]

Ours

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201846

Future Work
Integration to & evaluation in
professional workflow

Applications toother design domains

E.g., Autodesk Maya

E.g., architecture design[Kilian+17]

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201847

Summary

Optimization-Guided Motion Editing
Interactive control
by animators
Animator’scontrol

lessiveng•

Optimization is used forenhancing manual editing•

OptiMo allows animators tointeractively control
optimization

Automatic adjustment
by optimization
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201849

http://koyama.xyz/project/optimo/
Slides & videos available! We also plan to release our source codes!

Yuki Koyama and Masataka Goto. OptiMo: Optimization-Guided Motion
Editing for Keyframe Character Animation. CHI 2018.

OptiMo:
Optimization-Guided Motion Editingfor Keyframe Character Animation
Yuki Koyama & Masataka Goto

Appendix

Related Work

Motion Editing Techniques — Automatic
Optimization withspace-time
constraints

Use oflearned motion
controllers

Difficult to reflect artistic
intention➡ Lack of controllability
Resulting motions do nothave curve-based data
representation➡ Lack of editability[Witkin+88][Merel+17]

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 2018

54

OptiMo’s Features

We Designed 6 Features for OptiMo
1. Preservation of control handles [ T & E ]Transparency
Editability
Controllability

2. Optimization process visualization [ T ]3. Cost visualization [ T ]4. Interactive regularization [ T & C ]5. Per-Joint regularization [ C ]6. Time-varying cost control [ C ]Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 2018

56

Algorithm:
How Does Optimization Work?

Typical Optimization Formulation
minx

⇢Z

Search parametersx

C(x, t)dt
Cost function
C(x, t)Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201858

Parametrization (Search Space)
Use the control handle
parameters (thatanimators otherwise
manipulate manually) asthe search parametersof the optimization

Rig valueTime➡ This ensures editability (and also transparency)
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201859

Cost Function: Our (Current) Choice
C(x, t) =X

j2J2k⌧ j (x, t)k•Is well-established and used in computer graphics androbotics (c.f. [Safonova+04])•Is a summation of necessary torques (rotational forces) oneach joint•

Is useful for avoiding physically unfeasible motions
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 2018

60

Validation: Interview withProfessional Animators

Comments on Editability•

Both A1 and A2 loved editability••

A1: Editability is “indispensable”

This comment especially validates our parametrization:•

A2: It is acceptable for “the [tangent] handles to change,”
but “the keyframes (key points) should not be change” byoptimization

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201862

Comments on Transparency•Both A1 and A2 loved the visualization features••

A1: The process visualization “is very interesting”

We observed that both A1 and A2 easily understood theconcept of optimization by seeing the visualizations

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 2018

63

Comments on Controllability•

All features on controllability were strongly appreciated••A2: “I would be puzzled if I can’t use this”

The interactivity of control features was appreciated•

A1: These features could make adjustment “easier” and“less stressful”

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201864

Comments on Usages in Production••

The current system is not perfect yet, but it could be useful…
•

A2: For “sub-characters’ motion,” or•A1: In “projects with only limited time”

Both A1 and A2 got very excited about the (production-level)
quality of the fox tail animation•

A2: “It is possible to provide it by hand, but it requires much
tweaking to make it (the motion) look natural”
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 2018

65

Other Comments•

It would be nice if motion styles could be handled in thecost function as well as physics•A2: “Each art project has a specific style of motions” and “itwould be nice if it automates [the process of applying suchstyles]”Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201866

Example Usage ScenariosExample: Stick Figure Waving His Arm•#parameters: 78•

Rig: Forward kinematics (FK)Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201868

Example 1: Stick Figure Waving His Arm

InitialNaïve opt.

Controlled opt.

Too robotic…

The spine/elbow bend wrongly…

Looks natural :)Example: Fox Tail•#parameters: 27•

Rig: Forward kinematics (FK)Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201870

Example: Fox Tail

Inspiration:
Let’s make the swing speedierTime-varying cost control

Example: Music Conductor•#parameters: 54•

Rig: Forward kinematics (FK)and inverse kinematics (IK)

The next slide contains audio!
Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201872

Example: Music Conductor

Example: Music Conductor

Example: Music Conductor

Example: Music Conductor

Discussions

Discussion: “One is not Enough”
Providing more options of cost functions as well as ourphysics-based cost function — One is not enough!

Muscle models [Wang+12]

Motion styles [Grochow+04]

Yuki Koyama and Masataka Goto (AIST, Japan) | OptiMo: Optimization-Guided Motion Editing for Keyframe Character Animation | CHI 201878